← All roles

Product Manager, Evals & Improvement

Meta· Product Strategy
Apply on company site
Hybrid (Menlo Park, CA)$146k - $204kFull-timePosted 2mo ago

About the role

The Agent Transformation Accelerator (ATA) Product team is building Meta’s internal Metamate platform to improve agentic AI across the company. This role focuses on measuring and raising the quality of agentic systems, owning evals that gate model upgrades and major launches, and turning usage signals into a continuous improvement loop in collaboration with engineering, data science, and ML teams.

Responsibilities

  • Own the end-to-end feedback and improvement system: collection, analysis, prioritization, and resolution.
  • Define how we measure agent quality across dimensions (accuracy, helpfulness, reliability, safety).
  • Build product mechanisms that capture implicit and explicit user signal at scale.
  • Partner with ML and data science to turn feedback into model improvements, eval frameworks, and training data.
  • Drive the evals and measurement strategy: what "good" looks like, how we detect regressions, and how we track progress.
  • Create transparency and accountability for quality across the ATA Product team.

Qualifications

  • Strong quantitative skills and experience defining complex metrics
  • Comfort with ambiguous problem spaces where the right metrics do not yet exist
  • 5+ years of relevant industry experience with at least 2 years in Product Management
  • Bachelor's degree (or relevant degree equivalent): STEM subject ideal but not essential (Computer Science, Engineering, Information Systems, Analytics, Mathematics, Physics, Applied Sciences)
  • Experience partnering closely with data science and ML engineering teams
  • Track record of driving product improvements through data and experimentation
  • Direct experience building evals for AI products end to end: task set construction, rubric design, instrumentation, and analysis
  • Experience running eval-driven development cycles, with a clear view of where evals are informative and where they mislead
  • Willingness to get into the weeds of eval data and rubrics, with high standards for eval rigor and an obsession with quality
  • Deep understanding of LLM capabilities and failure modes

Skills

Required:Quantitative metric designData-driven product managementAI/evals developmentCollaboration with ML/data scienceEvaluation strategy & quality
Last updated Oct 23, 2026. Interested? Apply on the company site.

About Meta

Shop the latest AI glasses and wearable technology from Meta.

Shop the latest AI glasses and wearable technology from Meta. Explore Ray-Ban Meta, Oakley Meta, display glasses, and cutting-edge VR headsets.

AI glassesOakley MetaRay-Ban MetaVR headsetsliquid-ai-recruiting-targetwearable technology
Company profileWebsite

More open roles at Meta