← All roles

Inference Infrastructure Engineer, Serving

Elorian· All Departments
Apply on company site
Hybrid (Palo Alto)$200k - $400kFull-timePosted 3mo ago

About the role

Infrastructure engineer to design, optimize, and scale inference serving systems for large multimodal models, focusing on low latency, high throughput, and reliable deployment at scale.

Responsibilities

  • Build low-latency, high-throughput inference serving systems for our large multimodal models
  • Design and implement techniques that improve latency, throughput, and efficiency, including quantization, batching, speculative decoding, and KV cache management
  • Optimize our codebase and GPU fleet to fully utilize hardware FLOPs, bandwidth, and memory
  • Implement multi-GPU and multi-node model parallelism for serving (tensor or pipeline parallel)
  • Build autoscaling and load balancing for production ML services
  • Establish standards for reliability, observability, and reproducibility across the inference stack
  • Collaborate with researchers to enable high-performance inference for novel architectures

Qualifications

  • 3+ years of experience building low-latency, high-throughput inference serving systems for large models
  • Strong knowledge of inference optimization techniques (quantization, batching, speculative decoding, KV cache management)
  • Hands-on experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang
  • Experience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline parallel)
  • Strong systems programming skills; C++/CUDA a plus alongside Python
  • Experience with autoscaling and load balancing for production ML services
  • A track record of GPU cost optimization at scale

Skills

Required:low-latency inference servinginference optimization techniquesserving frameworksmulti-GPU/multi-node parallelismautoscaling and load balancingsystems programming

Benefits

  • health, dental, and vision benefits
  • unlimited PTO
  • paid parental leave
  • relocation support as needed
Last updated Oct 9, 2026. Interested? Apply on the company site.

About Elorian

Building the foundation of visual reasoning.

Elorian is a research and product lab pushing the next leap forward in reasoning, focused on visual reasoning in AI, grounded in foundational work by key researchers in the field.

visual reasoningAImultimodal modelsfoundation modelsscientific research
Company profileWebsite

More open roles at Elorian