← All roles

Head of Engineering

LlamaIndex· Engineering
Apply on company site
Hybrid (San Francisco)FullTimePosted 1w ago

About the role

Lead and own engineering execution, technical direction, hiring, and team development for a B2B SaaS AI/ML platform, with hands-on code review, architecture, and production operations across model serving, training, and ML infrastructure.

Responsibilities

  • Lead and develop our engineering team, setting clear priorities, ownership, and accountability.
  • Partner with company leadership to translate customer needs into a focused roadmap and deliver software predictably.
  • Stay directly involved in architecture, code reviews, production debugging, and implementation of critical changes.
  • Set technical direction across application services, APIs, data pipelines, and ML infrastructure, balancing immediate customer needs with long-term maintainability.
  • Work with ML engineers and researchers on model training, fine-tuning, evaluation, and deployment, ensuring model improvements translate into better production outcomes.
  • Guide model serving decisions across latency, throughput, GPU utilization, capacity, reliability, and inference cost.
  • Improve engineering practices for testing, observability, security, incident response, and releases.
  • Hire strong engineers, coach technical leaders and managers, and build a culture of direct communication, customer focus, and responsibility for results.

Qualifications

  • Experience leading multiple engineering teams in B2B SaaS, at a scope comparable to a 20–30-person organization, with responsibility for delivering and operating customer-facing products.
  • Strong, current hands-on engineering skills: you can read and write production code, review architecture, and diagnose complex technical problems.
  • Practical familiarity with model serving and model training, including training or fine-tuning workflows, evaluation, deployment, and production operations.
  • Understanding of the tradeoffs between model quality, latency, throughput, GPU resources, reliability, and cost.
  • Strong background in backend systems, APIs, distributed systems, and cloud infrastructure.
  • A track record of hiring and developing engineers, setting clear expectations, and giving constructive feedback.
  • Good product judgment, including knowing when to move quickly and when to invest in correctness, reliability, and security.
  • Clear written and verbal communication with technical teams, customers, and company leadership.

Skills

Required:Engineering leadershipHands-on engineeringModel serving and trainingBackend systems & APIs
Last updated Oct 14, 2026. Interested? Apply on the company site.
LlamaIndex logo

About LlamaIndex

Redefine document workflows with AI agents

LlamaIndex brings together VLM-powered and an enterprise‑grade framework for building, serving, and deploying agents.

AI agentsDocument processingRAGWorkflowsai-infrastructure-and-developmentwing-list-2026wing-list-2026-early-stage
Company profileWebsite

More open roles at LlamaIndex