← All roles

Member of Technical Staff - Inference

Prime Intellect· Engineering
Apply on company site
Remote$150k - $300kFull-timePosted 11mo ago

About the role

Hybrid role focused on cloud LLM serving, inference optimization, and RL systems to evaluate and serve models trained with the RL Lab at scale, including multi-tenant LLM serving, GPU-aware scheduling, and integration with RL training stacks.

Responsibilities

  • Multi‑tenant LLM Serving: Build a multi-tenant LLM serving platform that operates across our cloud GPU fleets.
  • GPU‑Aware Scheduling: Design placement and scheduling algorithms for heterogeneous accelerators.
  • Resilience & Failover: Implement multi‑region/zone failover and traffic shifting for resilience and cost control.
  • Autoscaling & Routing: Build autoscaling, routing, and load balancing to meet throughput/latency SLOs.
  • Model Distribution: Optimize model distribution and cold-start times across clusters.
  • Framework Development: Integrate and contribute to LLM inference frameworks such as vLLM, SGLang, TensorRT‑LLM.
  • Parallelism and Configuration Tuning: Optimize configurations for tensor/pipeline/expert parallelism, prefix caching, memory management and other axes for maximum performance.
  • End‑to‑End Performance: Profile kernels, memory bandwidth and transport; apply techniques such as quantization and speculative decoding.
  • Perf Suites: Develop reproducible performance suites (latency, throughput, context length, batch size, precision).
  • RL Integration: Embed and optimize distributed inference within our RL stack.
  • CI/CD: Establish CI/CD with artifact promotion, performance gates, and reproducible builds.
  • Observability: Build metrics, logs, tracing; structured incident response and SLO management.
  • Docs & Collaboration: Document architectures, playbooks, and API contracts; mentor and collaborate cross‑functionally.

Qualifications

  • 3+ years building and running large‑scale ML/LLM services with clear latency/availability SLOs.
  • Hands‑on with at least one of vLLM, SGLang, TensorRT‑LLM.
  • Familiarity with distributed and disaggregated serving infrastructure such as NVIDIA Dynamo.
  • Deep understanding of prefill vs. decode, KV‑cache behavior, batching, sampling, speculative decoding, parallelism strategies.
  • Comfortable debugging CUDA/NCCL, drivers/kernels, containers, service mesh/networking, and storage, owning incidents end‑to‑end.

Skills

Required:Building ML Systems at ScaleInference BackendsDistributed Serving InfraInference InternalsFull‑Stack Debugging

Benefits

  • Cash Compensation Range of $150-300k with significant equity incentives
  • Flexible work arrangement (remote or San Francisco office)
  • Full visa sponsorship and relocation support
  • Professional development budget
  • Regular team off-sites and conference attendance
  • Opportunity to shape decentralized AI and RL at Prime Intellect
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Prime Intellect

Commoditizing Compute & Intelligence

Prime Intellect democratizes AI development at scale. Our platform makes it easy to find global compute resources and train state-of-the-art models through distributed training across clusters. Collectively own the resulting open AI innovations, from language models to scientific breakthroughs.

AI developmentbessemer-ai-infrastructure-2026compute resourcesdistributed trainingmodelsreinforcement-learning-platformsscience
Company profileWebsite

More open roles at Prime Intellect