← All roles

ML Model Serving Engineer

Sesame· Software
Apply on company site
San Francisco, New York, Bellevue$175k - $280kFull-timePosted 11mo ago

About the role

Engineer focused on powering and optimizing a high-performance ML model serving stack across multiple models (LLMs, speech, vision) with emphasis on throughput, latency, and reliability.

Responsibilities

  • Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
  • Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
  • Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.
  • Work with the training team to identify opportunities to produce faster models without sacrificing quality.
  • Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
  • Find ways to reduce model initialization times without sacrificing quality.

Qualifications

  • Expert in some differentiable array computing framework, preferably PyTorch.
  • Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
  • Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.
  • Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.
  • Always up to date on the latest techniques for model serving optimization.

Skills

Required:PyTorchHigh-performance model servingSystems programmingPerformance engineering

Benefits

  • 401 (k) max employer match: 3.5% of compensation
  • 100% employer-paid health, vision, and dental benefits for you and your dependents
  • Unlimited PTO and sick time
  • Flexible spending account with employer matching up to $1,650/year (medical FSA)
  • Guardian Employee Assistance Program (EAP)
  • Opportunity to share in the company's success with competitive stock options
Last updated Sep 3, 2026. Interested? Apply on the company site.

About Sesame

Bringing the computer to life

We believe in a future where computers are lifelike. Where they can see, hear, and collaborate with us – as we do with each other. With this vision, we're designing a new kind of computer.

AIcollaborationcomputersliquid-ai-recruiting-targetvoice
Company profileWebsite

More open roles at Sesame