Software Engineer - Machine Learning Platform - Cortex
Who we Are:
Cortex empowers internal teams to efficiently leverage ML by providing a platform and by unifying, educating, and advancing the state of the art in ML technologies within Twitter.
We win when our customers win by helping our users stay informed, share and discuss what matters; by serving the public conversation. We’re building an AI-first company and every major initiative is increasingly dependent on the successful application of machine learning. Cortex is at the nexus of this evolution.
Our team of ML software engineers is constructing one of the strongest machine learning platforms in the world, based on the latest ML industry practices, deep learning, engineering excellence, powered by Twitter data at scale.
Twitter operates some of the highest traffic Deep Learning systems in the industry. Training algorithms routinely process terabytes of data; prediction services make millions of inferences per second. Twitter's real time nature demands latencies in the double digit millisecond range. We provide frameworks and tools for model-building, distributed training, prediction and evaluation systems that meet such stringent demands. We leverage open source solutions like tensorflow and extend them for our business needs. This is a challenging and exciting area, with a unique opportunity to create advanced technologies in uncharted territories while having a large impact.
Current projects include:
- Deepbird, with Tensorflow at the core https://blog.twitter.com/engineering/en_us/topics/insights/2018/twittertensorflow.html
- Framework for large scale distributed and online training of deep learning models
- Inference engine for serving 10s of millions of prediction requests
- Framework to analyze trained models
- Performance optimization of model training and serving pipelines using low level software and hardware primitives
- Custom extensions to Tensorflow e.g Implement recent research techniques to reduce training times
What You'll Do:
If this sounds like a team you want to be part of, great! We are looking for engineers who love writing code, have a desire to learn new technologies, thrive on teamwork and are committed to serving their customers.
Come join us in building a deeplearning framework for twitter's real time needs
Who You Are:
- You have a passion for machine learning.
- You thrive on working in concert with other smart people, including from distributed offices.
- You communicate fluidly, at the level of your audience, and seek to understand and being understood.
- You have the ability to take on complex problems, learn quickly, iterate, and persist towards a good solution.
- You are adamant about studying customer needs and enabling their success through our products.
- You take pride in polishing and supporting our products.
- You welcome feedback on are constantly looking for ways to improve yourself.
- You thrive on building frameworks and tools for developers.
- You focus deeply on making API’s easy to use without compromising performance.
- You constantly strive to reduce the latencies and runtime cost.
- You can provide thought leadership in ML infrastructure and best practices to the team and company at large.
- Masters or PhD in a Computer Science or Machine Learning related degree; or equivalent work experience in the field.
- 4+ Years experience writing production grade software.
- Familiarity with Tensorflow internals.
- Experience with Machine Learning and/or Deep Learning.
- Experience with low-level programming (C, C++, Vectorization) and related tools (e.g. Valgrind).
- Experience writing and operating large scale distributed systems.
Nice to Have:
- Experience with CUDA (GPU) programming.
- Experience with DL frameworks like Pytorch, Theano.
We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, color, ethnicity, ancestry, national origin, religion, sex, gender, gender identity, gender expression, sexual orientation, age, disability, veteran status, genetic information, marital status or any legally protected status.
After you apply, a recruiter may reach out to you for an introductory call.
If your background is a match for the role, you may phone interview with 1-2 people.
If you continue through the process, you will come onsite 1-2 times to interview with a total of 5-10 people.