Staff Software Engineer, AI Platform
Hybrid (New York)$231k - $340kFull-timePosted 3mo ago
About the role
Harvey is building an AI platform foundation for agent-based AI, focusing on model routing, agent architecture, context management, and evaluation. This role supports the AI Platform team to enable Harvey’s products and agents with scalable backend infrastructure and multi-model AI systems.
Responsibilities
- Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
- Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
- Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
- Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
- Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.
Qualifications
- 8+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering and a track record of technical leadership across teams.
- Experience building and shipping multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory systems in AI or distributed systems.
- A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted.
- Strong judgment about abstractions and pragmatic shipping mindset.
- Excitement about agentic AI and infrastructure challenges of making autonomous systems reliable.
- Bias toward full ownership and ability to navigate ambiguity.
- Bonus: experience building evaluation frameworks, working with agent/function-calling architectures, familiarity with legal or high-stakes domains, or time at startups with rapidly changing tech.
Skills
Required:Backend system designMulti-model / multi-provider AI systemsContext management / memory systemsInternal platforms / SDKs
Benefits
- Comprehensive health, dental and vision coverage
- 401k match up to 4%
- flexible PTO
Last updated Aug 26, 2026. Interested? Apply on the company site.
About Harvey
Professional Class AI
Harvey is a platform built to meet the standards of the world’s leading professional service firms.
AILegalLegal AILegal AnalysisLegal Document AnalysisLegal ResearchLegal Techsetter-30
More open roles at Harvey
- Senior Deal Desk Analyst, APAC4d agoSydney · Sales
- Deal Operations Analyst4d agoHybrid (Chicago) · Sales
- Deal Operations Analyst4d agoHybrid (Dallas) · Sales
- Account Executive, Enterprise4d agoHybrid (Toronto) · Sales
- Head of Mid Market Sales, North America4d agoHybrid (Chicago) · Sales