Inference Engineer
Hybrid (San Francisco)Full-timePosted 2w ago
About the role
Role focused on owning the cost and performance of the inference stack, optimizing throughput, latency, and reliability for model serving across changing workloads and hardware.
Responsibilities
- Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
- Optimize long-context prefill and decode workloads based on real production traffic.
- Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
- Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
- Build profiling and measurement systems that show where time, memory, and compute are being spent.
Qualifications
- 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
- Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
- Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
- Strong Python skills and proficiency in C++, Rust, or another systems language.
- Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.
Skills
Required:KV-cache managementcontinuous batchingspeculative decodingquantizationprofiling and measurement
Benefits
- Flexible work: In-person collaboration in the Bay Area, a distributed global-first team, and team offsites.
- Adaption Passport: Annual travel stipend to explore a country you've never visited.
- Lunch Stipend: Weekly meal allowance for take-out or grocery delivery.
- Well-Being: Comprehensive medical benefits and generous paid time off.
Last updated Oct 8, 2026. Interested? Apply on the company site.
About Adaption
Everything intelligent adapts. So should AI.
Adaptive DataAdaptive IntelligenceAdaptive InterfacesproductAI that adaptsAI that learnson-the-fly malleable datasetscontinual learning
More open roles at Adaption
- Developer Community Lead (6 Month Contract)2w agoRemote · DevRel
- Platform Engineer, APIs and Security2mo agoHybrid (San Francisco) · Platform
- Agent Systems Engineer2mo agoHybrid (San Francisco) · Platform
- Modelling Resident4mo agoRemote · Modelling
- Senior Research Scientist4mo agoRemote · Modelling