Sr. Software Engineer - Compute for Data and ML

The Position

Who We Are

We are the Compute Platform team, responsible for providing Twitter Engineering with an approachable, productive, and reliable compute platform. Our infrastructure powers most of the services behind Twitter.com, Twitter’s revenue-generating businesses, and a variety of internal infrastructure. Twitter’s 1,000+ engineers use our compute platform to launch and operate their services on a fleet of over 100,000 physical and virtual machines.

The Compute for Data and Machine Learning (ML) team provides Compute services to Data and ML service providers at Twitter. This includes a Kubernetes-based operating environment that spans Twitter datacenters and the public cloud along with a supporting set of systems that provide Data and ML practitioners the Compute capabilities that they need and when/where they need it.

What You'll Do

  • Be an early engineer on a new team that supports and empowers the Data and ML spaces within Twitter.
  • Partner with internal Data and ML service providers to offer new and improved capabilities to practitioners.
  • Be an active participant in our systems design and review processes.
  • Scope out, participate in, and lead cross-functional projects.
  • Optimize Kubernetes and our own systems on clusters in the thousands of nodes.
  • Develop our team through mentorship and knowledge sharing.
  • Participate in the wider Kubernetes/CNCF open source community.

Qualifications

  • 5+ years of experience in Infrastructure, Platform, or Backend development.
  • Strong development skills in at least one programming language, Go preferred.
  • Prior experience with modern infrastructure tooling with Kubernetes as an administrator or operator in a production setting, e.g. container orchestration, cloud providers (AWS, GCP, etc) and infrastructure as code tools.

Preferred Experience

  • Experience designing, developing, operating, and debugging high traffic production systems.
  • Experience building applications against the Kubernetes API.
  • Familiarity with Kubernetes internals (architecture and/or codebase).
  • Prior experience (as a user or operator) with Kubeflow, PyTorch, GCP BigQuery, GCP Dataflow, or GCP Dataproc.
  • Experience with other container-centric orchestration systems (Mesos, Nomad, etc).
  • Prior contributions to projects within the CNCF community.
  • Experience working with physical infrastructure (bare metal in data centers).
  • Familiarity with Linux or similar operating systems.

Company Description

Twitter is what’s happening and what people are talking about right now. For us, life's not about a job, it's about purpose. We believe real change starts with conversation. Here, your voice matters. Come as you are and together we'll do what's right (not what's easy) to serve the public conversation.

Additional Information

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran, genetic information, marital status or any other legally protected status.

San Francisco applicants: Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records

Team

Infrastructure Engineering, Software Engineering

Location

San Francisco, New York City, Remote US, Seattle, Remote Canada

 

Application

U.S. Equal Employment Opportunity information (Completion is voluntary)
Non U.S. Equal Employment Opportunity information (Completion is voluntary)
Privacy and data