Senior Software Engineer - Realtime Compute

The Position

We are embarking on a mission to redefine how data processing is done at Twitter. The team is building an integrated platform that unifies data processing with all other parts of the development ecosystem that is needed to develop products. Our work is a vital part of an overall Twitter goal to double development speed by the end of 2022.

We are making the Terabytes of data Twitter collects every day accessible to all applications. Platforms and tooling we’ve built handle millions of events per second and hundreds of thousands of queries per second today, with plans to grow both by an order of magnitude in the next couple of years. Engineering, Data Science, and Product Management teams need that data to be broken down by segments and user attributes that cover as many as 50 dimensions. Our next-generation analytics framework makes that data easier to generate, store and query. The team is pioneering new technologies and was the first to run production services in Google Cloud. Using public cloud technologies has allowed us to focus all of our time on core large-scale problems of Twitter while offloading pager and infrastructure maintenance to Google. 

The team was assembled from leading engineers in streaming and cloud development across Twitter and you’ll have the opportunity to contribute to a key part of Twitter’s future and uplevel your own career. If you have a philosophy of building tools that deal with a broad set of problems, this is the team for you. If you have a steadfast customer focus that will help us to accelerate developer and analytic productivity, this is the team for you.

What You'll Do:

Rather than speak in generalities, here is a sampling of what we’ve been building:

  • Unified Data Platform (UDP) - Unified platform for building, configuring, and productionizing data processing across Twitter’s control plane.

  • Large Data Collider (LDC) - Integrated managed platform for generating and serving aggregated datasets.

  • DataAPI - Metrics-driven query service that enables the reuse of metrics across use cases and backends

Qualifications

  • Proven experience working in a distributed computing environment.
  • 4+ years of experience with Data Platform engineering (Java/Scala or Python preferred), distributed systems, and asynchronous programming. 
  • A strong passion for building tools and systems that enable other developers to build great things.
  • Proven experience building Data Platform Services with large customer bases from early stages.
  • Previous experience with cloud server environments such as Amazon Web Services (AWS) and/or Google Cloud Platform (GCP).
  • An understanding of continuous integration and a strong drive toward automation.
  • Proficient in multi-threading, scaling, concurrency, caching, queuing, RPC, and distributed data storage.
  • Ability to thrive in a dynamic, fast-paced, and collaborative environment.

Company Description

Twitter is what’s happening and what people are talking about right now. For us, life's not about a job, it's about purpose. We believe real change starts with conversation. Here, your voice matters. Come as you are and together we'll do what's right (not what's easy) to serve the public conversation.

Additional Information

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran, genetic information, marital status or any other legally protected status.

San Francisco applicants: Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records

Team

Software Engineering

Location

San Francisco, New York City, Remote US, Toronto, Remote Canada

 

Application

U.S. Equal Employment Opportunity information (Completion is voluntary)
Non U.S. Equal Employment Opportunity information (Completion is voluntary)
Privacy and data