Engineering Manager, Agent Oversight
About the role
Engineering Manager for Agent Oversight leading the team building Scale's platform for monitoring, evaluating, and improving agentic applications across enterprise and government customers. Responsible for technical delivery, roadmap, cross-functional collaboration, and growing the team in a fast-growing, US-based unit across New York and San Francisco.
Responsibilities
- Lead a multi-disciplinary team of software and ML engineers to drive technical delivery across the Scale Generative AI Platform (SGP)
- Own the platform's roadmap across deployment, monitoring, evaluation, and ML-driven improvement of agentic applications
- Work cross-functionally with customers, forward deployed teams, product, and internal engineering teams to translate enterprise and government requirements into platform capabilities
- Build and ship features end-to-end, from system design through debugging and testing
- Drive high-velocity experimentation to validate and improve platform capabilities based on real customer usage
- Establish the technical direction, culture, and processes for a fast-growing team
- Mentor and develop both engineers and ML engineers/scientists, and influence how the team scales technically and organizationally
Qualifications
- 7+ years of engineering experience, including 2+ years directly managing engineers or ML engineers responsible for a production ML/LLM-powered system — not just consuming a third-party ML API within a feature
- Hands-on familiarity with agent architectures — tool use, planning, multi-agent orchestration
- Review ML experiment design or evaluation methodology well enough to ask sharp questions and earn credibility with ML engineers and scientists
- Track record owning the full lifecycle of platform-level infrastructure — from initial design through scaling it across multiple internal or external teams as usage, headcount, and complexity grow
- Experience collaborating with product managers, forward deployed engineering (FDE) teams, and customers to translate real-world requirements into prioritization decisions and shipped platform capabilities
- Track record of building and growing high-performing engineering teams — including hiring, retention, or measurable improvements in team output or velocity
Skills
Benefits
- comprehensive health, dental and vision coverage
- retirement benefits
- learning and development stipend
- generous PTO
- additional benefits such as a commuter stipend
About Scale AI
Data-centric infrastructure to accelerate the development of AI
Scale accelerates the development of AI within organizations of any size to deliver critical business insights and operational efficiency. Its data-centric infrastructure platform leverages RLHF (Reinforced Learning with Human Feedback) to help organizations build the strongest AI models that supercharge their business, with customers across industries including Meta, Microsoft, U.S. Army, DoD’s Defense Innovation Unit, Open AI, General Motors, Toyota Research Institute, Brex, Instacart and Flexport.
More open roles at Scale AI
- Strategic Contracts Lead, Public Sector5d agoWashington, DC · Public Sector
- Web Designer, Brand Web Experience5d agoSan Francisco, CA · Marketing
- Head of Strategic Partnerships, Global Systems Integrators5d agoSan Francisco, CA · Enterprise Sales
- Safety Research Scholar - Human Frontier Labs5d agoHybrid (United States) · Human Frontier Collective
- Machine Learning Research Intern (Summer 2027)5d agoHybrid (San Francisco, CA), Hybrid (New York, NY) · University