← All roles

Senior Backend Engineer

LiteLLM
Apply on company site
San Francisco$200k - $260kFull-timePosted 2mo ago

About the role

Senior Backend Engineer at LiteLLM, a Python SDK and Proxy Server enabling access to 100+ LLM APIs. Focus on building guardrails, logging, and observability tooling to support high-volume, secure, and reliable backend systems.

Responsibilities

  • Build and scale our product, ensuring performance, reliability, and continuous improvement.
  • Ensure all guardrail and policy enforcement calls (e.g., applyguardrail) are properly logged and traceable through our SpendLogs and relevant database tables
  • Build and design CPU-level guardrails to cover common attacks on LLM API's / MCP servers / Agents
  • Identify and fix areas where silent failures occur in guardrail creation, registration, and policy application—ensuring robust error handling and transparency to end users
  • Work with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain accurate, configurable, and usable monitoring and logging for backend systems
  • Enhance observability integrations to work for 1B+ requests/mo., with minimal latency overhead and no memory leaks
  • Collaborate cross-functionally on backend engineering priorities (performance, reliability, security)

Qualifications

  • Bachelor’s or Master’s in Computer Science or related field
  • 1+ years of experience with Python and backend frameworks (e.g. FastAPI, Flask)
  • Understanding of logging best practices, error handling, and secure backend development
  • Exposure to monitoring, logging, or metrics platforms (Datadog, Splunk, Prometheus, OpenTelemetry)
  • Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.)
  • Driven to deliver high-quality backend code with strong guardrails, auditing, and debugging capabilities
  • Eagerness to tackle hard bugs and ensure system transparency for end users

Benefits

  • Competitive salary
  • Health, dental, and vision benefits
Last updated Sep 23, 2026. Interested? Apply on the company site.
LiteLLM logo

About LiteLLM

Call every LLM API like it's OpenAI [100+ LLMs]

LiteLLM is an open-source LLM Gateway with 18K+ stars on GitHub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe. LiteLLM provides an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format We have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund

aiopsartificial-intelligencedeveloper-toolsgenerative-aiopen-source
Company profileWebsite

More open roles at LiteLLM