Software Engineer - Machine Learning Data and Observability, Cortex
Who we are
Our mission in Cortex is to empower teams at Twitter to effectively practice Machine Learning (ML) within Twitter by providing both a platform and modeling expertise. If you’re interested in the platform side of this and get excited about reusable, powerful tools which increase developer efficiency, read on!
Our software in the Cortex Platform org allows ML practitioners to share features, automate workflows, train models using advanced techniques, and serve models quickly and with confidence in their operation.
Our team specifically is called Machine Learning Data and Observability, and we seek to provide visibility, awareness, and understanding of the ML lifecycle. Our efforts affect every other product in Cortex in an effort to make our users able to understand their data, data pipelines, and models.
What you’ll do
If this sounds like a team you want to be a part of, fantastic! We are looking for experienced engineers who love writing code, understanding our customers, and collaborating with teammates to ship useful software. More specifically, you will:
- Design and own projects and drive their delivery
- Explore problems in partnership with our users to understand new technical domains
- Craft pragmatic solutions using existing tools as well as writing our own
- Help drive larger decisions within Cortex that connect with our core mission
Sample projects we’ve worked on
- Ongoing collaboration with Twitter’s ML Ethics, Transparency, and Accountability team
- Instrumenting data anomalies in production model systems
- Migrating and improving the data pipelines behind core models
Who you are
You prefer to collaborate with users to uncover their most pressing problems and incrementally deliver pragmatic solutions. You love bringing transparency to systems and you've seen how logging, metrics, observability, and visualizations can help organizations run better. You've added these approaches and more to your toolkit. Additionally, you consider yourself an engineer first and foremost with an interest in the fields of ML and Data Science.
On our team we need people who
- 4+ years of professional experience
- Enjoy mapping ambiguous business problems into technical plans and solutions.
- Have implemented efficient distributed data systems with both online and offline processing.
- Enjoy working with our internal customers and having empathy for their problems.
- Embrace a growth mindset and want to improve ourselves, the team, our process, and the products we work on.
Additionally it would be nice if you had
- Scala experience, development experience with Python, Jupyter Notebooks, Spark.
- Experience specifying re-usable, ergonomic libraries.
- Basic understanding of ML modeling lifecycle workflows to help communicate with our customers.
- A proven understanding of data processing and storage technologies. We work Hadoop and Scalding to analyze large datasets.
Engineering Hiring Process
Once your application is received, a recruiter will reach out pending your qualifications are a match for the role.
If your background is a match, you may have 1-2 technical phone interviews or be given the chance to provide a work sample depending on the role.
If the phone interviews go well or your work sample is strong, the final step includes interviews with 5-6 people held onsite in our office.