← All roles

Machine Learning Engineer, Ops

Cantina· Research
Apply on company site
Europe$125k - $165kFull-timePosted 2mo ago

About the role

ML Engineer responsible for production inference stack of text-to-speech (TTS) and speech recognition (ASR) models, end-to-end, including inference engines, Kubernetes infrastructure, and performance benchmarking to ship low-latency, real-time streaming services on GPUs.

Responsibilities

  • Build and optimize streaming inference engines for TTS and ASR models, including multi-stage model pipelines.
  • Optimize latency (time to first chunk, p50/p99) and throughput using continuous batching, CUDA Graphs, torch.compile and mixed precision.
  • Maintain numerical parity with the reference implementation through automated accuracy regression tests, per stage and end to end.
  • Deploy and operate GPU inference services on Kubernetes: autoscaling, containerization and infrastructure as code.
  • Build CI/CD for model artifacts, with model versioning and reproducible releases.
  • Own observability and performance benchmarking: p50/p99 latency, real-time factor (RTF), GPU utilization and cost per request.
  • Partner with research to productionize new models.

Qualifications

  • Hands-on experience deploying, scaling and optimizing ML models in production, beyond consuming model APIs.
  • Working knowledge of the full inference stack, from CUDA kernels to serving frameworks, with depth in some layers; expertise across all of them is not expected.
  • GPU performance optimization: profiling (Nsight Systems, PyTorch Profiler), memory bandwidth, CPU-GPU synchronization.
  • Solid understanding of model serving: continuous batching, KV cache management, request scheduling.
  • Production experience with Kubernetes, CI/CD and cloud infrastructure.
  • Strong Python and PyTorch, and a rigorous approach to performance benchmarking.
  • Experience with speech or audio models (TTS, ASR, voice conversion).
  • Experience with inference frameworks such as vLLM (vLLM-Omni), SGLang, TensorRT-LLM, TensorRT or Triton Inference Server.

Skills

Required:Streaming inference engineeringLatency and throughput optimizationKubernetes and cloud infrastructureCI/CD and model versioningObservability and benchmarkingPython and PyTorchSpeech/Audio models

Benefits

  • Competitive salary and generous company equity
  • Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
  • 42 days of paid time off, including: 15 PTO days, 10 sick days, 15 company holidays, 2 floating holidays
  • Generous parental leave & fertility support
  • 401(k) retirement savings plan
  • Lifestyle spending account – $500/month to use however you’d like
  • Complimentary lunch and snacks for in-office employees
  • One Medical membership, and more!
Last updated Oct 14, 2026. Interested? Apply on the company site.
Cantina logo

About Cantina

Infinite Creativity Unlocked

Chat with friends and AI. Build characters with personality. Create video in seconds — only on Cantina.

AICreativityVideoCharactersChat
Company profileWebsite

More open roles at Cantina