Engineering Manager, Model Infrastructure
Hybrid (San Francisco)$260k - $350kFull-timePosted 3mo ago
About the role
Engineering Manager for Model Infrastructure leading the platform powering Harvey's model requests. Responsible for reliability, scalability, cost-efficiency, and building infrastructure to train, evaluate, deploy, and operate models while partnering across AI Research, Product Engineering, Infrastructure, and external providers.
Responsibilities
- Lead and grow a high-performing team of software engineers responsible for Harvey's Model Infrastructure platform.
- Define the technical roadmap for model reliability, scalability, and operational excellence.
- Build highly reliable systems for model provisioning, capacity management, failover, and incident response across multiple AI providers.
- Own Harvey's multi-provider model platform, including provider integrations, SDK upgrades, API migrations, and onboarding new model providers.
- Drive the evolution of our Unified Model Controller (UMC) and Model Selector platform to automatically detect degraded models and intelligently route traffic based on health, latency, quality, compliance, and cost.
- Improve observability through health dashboards, alerting, token usage analytics, cost reporting, and end-to-end model telemetry.
- Partner with Product Engineering to support new model launches, capacity planning, experimentation, and proactive production monitoring.
- Lead initiatives to improve inference efficiency, reduce infrastructure costs, and increase model utilization across providers.
- Build the infrastructure foundation for Harvey's future model training efforts, including data pipelines, model operations, training environments, and AI platform capabilities.
- Partner with executive leadership on long-term AI infrastructure strategy and vendor relationships.
- Recruit, mentor, and develop exceptional engineering talent while fostering a culture of technical excellence and operational ownership.
Qualifications
- 8+ years of software engineering experience, including multiple years managing high-performing engineering teams.
- Experience leading teams responsible for large-scale distributed systems or cloud infrastructure.
- Strong technical background that enables you to guide architectural decisions and mentor senior engineers.
- Experience operating highly available production services with strong reliability and operational excellence.
- Experience building platforms that require scalability, observability, automation, and cost optimization.
- Strong cross-functional leadership skills with the ability to partner effectively across Engineering, Research, Product, and external vendors.
- Excellent communication skills and the ability to influence technical strategy across organizations.
- A passion for building teams and developing engineering talent.
Skills
Required:Software engineering leadershipLarge-scale distributed systems / cloud infrastructureSystem reliability and observabilityCross-functional collaboration
Benefits
- Comprehensive health, dental and vision coverage
- retirement benefits (401k match up to 4%)
- flexible PTO
Last updated Oct 22, 2026. Interested? Apply on the company site.
About Harvey
Professional Class AI
Harvey is a platform built to meet the standards of the world’s leading professional service firms.
AILegalLegal AILegal AnalysisLegal Document AnalysisLegal ResearchLegal Techsetter-30
More open roles at Harvey
- Senior Deal Desk Analyst, APAC4d agoSydney · Sales
- Deal Operations Analyst4d agoHybrid (Chicago) · Sales
- Deal Operations Analyst4d agoHybrid (Dallas) · Sales
- Account Executive, Enterprise4d agoHybrid (Toronto) · Sales
- Head of Mid Market Sales, North America4d agoHybrid (Chicago) · Sales