Research Intern, Frontier Speculative Decoding (Winter 2027)
Together AI
$58 - $70 / hour
Need a reasonable accommodation to apply or interview? Contact us.
JobMinglr uses automated technology to recommend jobs based on profile information and job preferences. Match Score does not determine eligibility for a position, prevent a user from viewing or applying to a job, or make hiring decisions on behalf of an employer.
Description
About the Role
The Speculative Decoding team at Together AI builds the draft models and algorithms that let large language models generate faster without changing their outputs. Our speculators run in production on frontier open models such as DeepSeek, Kimi and GLM. Our work sits at the intersection of model training, inference algorithms and serving systems, and we co-design all three to lower the cost and latency of generation.
As a research intern, you will dive into open questions in speculative decoding: new drafter architectures and verification schemes, knowledge distillation and training recipes that align draft and target models, and drafters that adapt as traffic shifts. You will train and fine-tune draft models at scale on dedicated GPU clusters, and test your ideas against realistic serving workloads, including long-context and agentic traffic.
Projects aim to answer important research questions in speculative decoding, enabling faster generation on frontier models, drafters that keep pace with changing workloads, and robust, reproducible evaluation under real traffic.
Past Research
Speculative decoding research by Together researchers, interns and academic collaborators includes:
- Speculative Speculative Decoding (ICLR 2026)
- When RL Meets Adaptive Speculative Training: A Unified Training-Serving System (ICML 2026)
- Beat the Long Tail: Distribution-Aware Speculative Decoding for RL Training (MLSys 2026)
Responsibilities
- Research and prototype new efficient speculative decoding algorithms and draft model architectures
- Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
- Implement optimized version for serving engines to show efficacy in real world workloads
- Communicate the plans, progress, and results of projects to the broader team
- Share findings through technical write-ups and blog posts, and help bring successful ideas into production
Requirements
- Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field
- Strong knowledge of Machine Learning and Deep Learning fundamentals
- Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
- Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
Preferred Qualifications
- Prior research experience in speculative decoding, efficient inference, or efficient machine learning
- Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)
- Experience with knowledge distillation, post-training, or reinforcement learning
- Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)
- Contributions to open-source machine learning projects
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Internship Program Details
Our internship program runs 12 to 14 weeks, giving you the opportunity to work alongside industry-leading engineers and researchers across multiple teams. This cohort's internship dates span January 4th to April 9th.
Compensation
We offer competitive compensation, housing stipends, and other competitive benefits. The estimated US hourly rate for this role is $58 - $70 an hour. Our hourly rates are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
For more information about our privacy policy, please visit: https://www.together.ai/privacy
About this role
Together AI's Speculative Decoding team is hiring a research intern to work on algorithms and models that accelerate large language model inference without sacrificing output quality. The role sits at the intersection of model training, inference optimization, and serving systems—you'll prototype new drafter architectures and verification schemes, train and fine-tune draft models on GPU clusters, and validate your work against real production traffic patterns including long-context and agentic workloads. This is hands-on research with immediate real-world application; the team's recent work has been published at top venues like ICLR and ICML, and successful projects move into production on frontier open models.
The internship expects strong fundamentals in machine learning and deep learning, plus practical experience with distributed training frameworks like Megatron-LM or FSDP. You should be comfortable with modern architectures such as Mixture-of-Experts and hybrid attention. Prior research or publications in machine learning are preferred but not required; background in speculative decoding, efficient inference, knowledge distillation, or LLM serving engines would be a plus. You'll be pursuing a Master's or Ph.D. in Computer Science, Electrical Engineering, or a related field.
The 12- to 14-week internship runs January 4 to April 9, 2027, in San Francisco on a hybrid basis. Compensation is $58–$70 per hour depending on experience and location, plus housing stipends and other benefits.
How this employer is doing
solid
- Recently Raised Funding
In the news
Together AI Raises $800 Million at $8.3 Billion Valuation to Make Frontier AI Accessible to All, Business Wire
This role's local market on JobMinglr
Pay for this role
$58 to $70 per hour
That is the employer's posted range, and where you land in it is usually decided in one conversation. Before that conversation, run the numbers: analyze an offer for this role.
Before you apply, worth reading
How JobMinglr reads this job
Every listing here is scored against your profile before you apply: skills overlap, experience level, location and work arrangement, each weighted and explained. You see the score and the reasons, not just a list. How the matching works.