Member of Technical Staff - ML Systems & Inference
San Francisco, CA$150k - $390kFull-timePosted 3mo ago
About the role
Role focused on building end-to-end ML inference systems for production, including batching, scheduling, memory placement, KV cache management, and cross-hardware optimization to improve latency, throughput, and utilization.
Responsibilities
- Build inference systems that execute models end-to-end in production
- Design and optimize batching, scheduling, and concurrency across the inference pipeline
- Manage KV cache and intermediate state movement between accelerators
- Optimize memory placement and system performance across hardware characteristics
- Bring up models on novel hardware and support new model architectures and inference techniques
- Collaborate with compiler, kernel, networking, and distributed systems engineers to optimize the full execution path
- Improve latency, throughput, and efficiency of production inference workloads
- Design execution strategies across batching, scheduling, concurrency, and resource utilization
- Improve KV cache management and memory efficiency under load
- Enable new models and accelerator architectures to run efficiently in production
Qualifications
- Bachelor's degree in a relevant field, or equivalent combination of education, training, and professional experience.
Skills
Required:Software engineering fundamentalsML inference or model serving systemsPerformance reasoning
Nice to have:Inference runtimes (TensorRT-LLM, vLLM, or custom serving systems)Model architectures and attention mechanismsBatching, scheduling, concurrency controlKV cache management and memory placementProfiling and latency/throughput tuningPython and C++
Last updated Oct 11, 2026. Interested? Apply on the company site.
About Gimlet Labs
Gimlet Labs
Gimlet Labs is an applied research lab focused on building high performance AI systems.
Autonomous kernel generation for heterogeneous hardwareCost-aware optimization frameworks for AI workloadsGimlet cloudHeadless hardware architectures for serving AI inferenceHybrid edge/cloud workload partitioning and orchestrationSLA-aware dynamic datacenter scheduling of AI agent workloadsUniversal AI compiler for heterogeneous hardwarebessemer-ai-infrastructure-2026
More open roles at Gimlet Labs
- FP&A Manager, Infrastructure & CorporateyesterdaySan Francisco, CA · G&A
- Materials and Product Lifecycle Program ManageryesterdaySan Francisco, CA · Research and Development
- Project Coordinator, Hardware Partnerships1w agoSan Francisco, CA · Sales and Marketing
- Data Center Architect1w agoSan Francisco, CA · Technology and Infrastructure
- Strategic Hardware Partnerships1w agoSan Francisco, CA · Sales and Marketing