Software Engineer, Inference
San Francisco, CA$230k - $390kFull-timePosted 2w ago
About the role
Sierra’s AI agents depend on foundation models to reason and act in real time. The Inference team builds the systems that make those models fast, reliable, and efficient at scale. As a Software Engineer on Inference, you’ll help define Sierra’s inference architecture across self-hosted models and third-party inference providers, focusing on serving, routing, capacity management, latency, reliability, and cost in a hybrid infrastructure.
Responsibilities
- Partner with frontier labs and providers. At our scale, we rely on frontier labs, and inference providers to supply capacity, training and inference infrastructure.
- Shape Sierra’s inference architecture. Design how inference traffic flows across models, infrastructure, and providers, including new serving and proxy layers as Sierra scales.
- Build for low latency and high reliability. Develop systems for routing, failover, capacity management, and quota that keep inference performant and available across large-scale production workloads.
- Build and operate self-hosted inference. Run models on GPU infrastructure, from building containers and operating inference engines to managing the underlying compute capacity.
- Optimize inference performance. Work with the Applied Research team on techniques such as speculative decoding and serving-engine optimizations that improve latency, throughput, and cost.
- Build across a hybrid inference stack. Work with both Sierra-managed infrastructure and leading inference platforms, making architectural decisions about where and how workloads should run.
- Push the serving stack forward. Work closely with inference providers to tune engines and infrastructure for Sierra’s workloads.
- Support the broader model lifecycle. Contribute to infrastructure that enables post-training while partnering closely with our Models and Agent Runtime teams.
Qualifications
- Deep systems thinking and strong distributed systems fundamentals.
- Experience designing, building, and operating large-scale production systems.
- Strong judgment around tradeoffs involving latency, reliability, capacity, and cost.
- Experience taking ownership of complex infrastructure from architecture through production operation.
- Excitement about applying systems expertise to AI infrastructure and learning quickly as the underlying technology evolves.
Skills
Required:Distributed systemsInfrastructure ownershipLatency optimization
Nice to have:GPU/inference hardware
Benefits
- Flexible (unlimited) paid time off
- Medical, dental, and vision benefits for you and your family
- Life insurance and disability benefits
- Retirement plan dependent on country of employment
- Parental leave
- Fertility and family building benefits through Carrot
- Lunch, snacks and coffee
- Discretionary benefit stipend
- Free alphorn lessons
Last updated Oct 7, 2026. Interested? Apply on the company site.
About Sierra
Better customer experiences. Built on Sierra.
Sierra helps businesses build better, more human customer experiences with AI.
AICustomer ExperienceCustomer SupportVoiceai-agents-and-applicationsorchestrationvoice-ai-stackwing-list-2026
More open roles at Sierra
- Product Counsel5d agoSan Francisco, CA · Legal
- Integrated Marketing Lead5d agoSan Francisco, CA · Marketing & Communications
- Software Engineer Intern, Agent (London 2027)5d agoLondon · Early Career
- Sales Engineering Manager, High Growth1w agoSan Francisco, CA · Sales
- Strategist, Agent Development (MBA Grad 2027)1w agoSan Francisco, CA, New York, NY · Product