← All roles

Senior Machine Learning Engineer, Voice AI

Together AI· Engineering
Apply on company site
San Francisco, CA$200k - $260kfull-timePosted 6mo ago

About the role

Senior ML Engineer responsible for the model serving layer of Together AI's voice platform, optimizing real-time inference for STT, TTS, and speech-to-speech workloads, and collaborating with partners to bring models to production.

Responsibilities

  • Own the model serving stack that powers Together's voice platform across STT, TTS, and speech-to-speech.
  • Work directly with state-of-the-art accelerators (H100s, H200s, B200s) to optimize voice model inference.
  • Collaborate with model partners (Cartesia, Deepgram, Rime, and others) to bring their models to production on Together's infrastructure.
  • Build quality evaluation frameworks that guide model selection for customers and inform the roadmap.
  • Join a small, early-stage team with outsized impact on a fast-growing product area.
  • Optimize inference performance for voice models (STT, TTS, speech-to-speech) — targeting best-in-class TTFB, throughput, and GPU utilization across our curated model set.
  • Productionize voice models on serverless and dedicated endpoints, including batching strategies, streaming inference, and memory management tailored to audio workloads.
  • Build and maintain a voice model evaluation framework — measuring WER across accents, languages, and noise conditions for STT; naturalness, latency, and pronunciation accuracy for TTS.
  • Enable new model architectures in our serving stack as the field evolves, including audio-native LLMs, codec-based models (SNAC), and speech-to-speech systems.
  • Collaborate with model partners to integrate and optimize their models (Cartesia, Deepgram, Rime, and others) running on Together's infrastructure.
  • Profile and debug performance across the full inference stack — from GPU kernels to framework-level bottlenecks — and ship measurable improvements.
  • Work with the platform engineering side of the team to ensure the serving layer meets the latency and reliability requirements of real-time voice APIs.
  • Contribute to voice model fine-tuning capabilities (STT and TTS) as we enable customers to build differentiated voice experiences on Together.
  • Lay the groundwork for multiple new products down the line.

Qualifications

  • 5+ years of experience in ML engineering, with a focus on model serving, inference optimization, or ML infrastructure.
  • Hands-on experience with LLM serving engines (vLLM, SGLang, TensorRT-LLM, or similar) — comfortable reading and modifying engine internals, not just using APIs.
  • Strong proficiency in Python and PyTorch; experience with GPU profiling and optimization (CUDA, memory management, kernel-level debugging).
  • Track record of shipping ML systems to production with measurable performance improvements.
  • Strong product sense — you think about what developers building voice apps actually need, not just what's technically interesting.
  • Comfort working on a small, early-stage team where you'll wear multiple hats and move fast.
  • Experience with speech and audio ML (ASR, TTS architectures, audio signal processing) is a strong plus but not required.
  • Familiarity with audio codecs and tokenization schemes (SNAC, Encodec, DAC) is a plus.
  • Experience training or fine-tuning speech models is a plus.
  • Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience

Skills

Required:Model serving and inference optimizationLLM serving enginesPythonPyTorchGPU profiling

Benefits

  • competitive compensation
  • startup equity
  • health insurance and other competitive benefits
Last updated Sep 9, 2026. Interested? Apply on the company site.

About Together AI

The AI Acceleration Cloud - Fast Inference, Fine-Tuning & Training

Run and fine-tune generative AI models with simple APIs and scalable GPU clusters. Train & deploy at scale on The AI Acceleration Cloud.

AI native cloudFine-TuningGPU clustersInferenceTrainingai-infrastructure-and-developmentbrex-fastest-growing-2025liquid-ai-recruiting-target
Company profileWebsite

More open roles at Together AI