← All roles

Backend Performance Engineer

LiteLLM· Engineering
Apply on company site
Hybrid (San Francisco, CA, US)$150k - $200kFull-timePosted 7mo ago

About the role

LiteLLM is seeking a Python-based performance engineer to maximize throughput, minimize latency, and ensure production reliability for an open-source LLM gateway that calls 100+ LLM APIs. Focus areas include achieving industry-parity RPS/latency, reducing cache miss/hit overhead, and addressing environment-specific and memory-leak issues for enterprise customers.

Responsibilities

  • Own maximizing throughput, minimizing latency and ensuring platform reliability in production.
  • Reduce e2e overhead latency for cache misses and cache hits to meet industry benchmarks.
  • Ensure overhead latency scales with added components (e.g., Redis, Redis Cluster, DB, Non-Admin Virtual Keys).
  • Ensure overhead latency scales with payload size (e.g., 1MB prompt with streaming < 100ms).
  • Address customer-specific and pipeline-specific latency issues (on support calls, debug environment settings).
  • Address paying customer memory leaks (enterprise clients).
  • Longer term: add coverage over new endpoints - /realtime, /audio/transcriptions/, /audio/speech.

Qualifications

  • Experience: Any (new grads ok)
  • Skills: Python, Rust

Skills

Required:PythonRust
Last updated Sep 23, 2026. Interested? Apply on the company site.
LiteLLM logo

About LiteLLM

Call every LLM API like it's OpenAI [100+ LLMs]

LiteLLM is an open-source LLM Gateway with 18K+ stars on GitHub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe. LiteLLM provides an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format We have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund

aiopsartificial-intelligencedeveloper-toolsgenerative-aiopen-source
Company profileWebsite

More open roles at LiteLLM