Applied Science Manager, Artificial General Intelligence , Frontier AI Assets: Assessments
Amazon· Applied Science
BELLEVUE, WA, BOSTON, MA$183.8k - $248.7kFull-timePosted 1mo ago
About the role
Applied Science Manager leading a team of Applied Scientists to develop quality strategies, auditing frameworks, and automated evaluation systems for data quality in AGI initiatives, focusing on LLMs and multimodal systems, including data collection workflows, audits, and judge architectures.
Responsibilities
- Lead and mentor a team of Applied Scientists to develop comprehensive quality strategies and auditing frameworks for data collection workflows.
- Design auditing strategies with SOPs, quality metrics, and sampling methodologies aligned with the development of Amazon Nova models.
- Oversee expert-level manual audits, meta-audits to evaluate auditor performance, and coaching to uplift quality capabilities across the team.
- Manage development and maintenance of LLM-as-a-Judge systems, including judge architectures, evaluation rubrics, and ML models for automated quality assessment.
- Configure data collection workflows and communicate quality feedback to stakeholders.
- Recruit, hire, and develop team members; conduct performance reviews; set expectations and growth plans; foster scientific excellence and innovation.
- Communicate requirements, product features, and technical designs to senior leadership, cross-functional teams, and customers.
Qualifications
- 3+ years of scientists or machine learning engineers management experience
- Knowledge of ML, NLP, Information Retrieval and Analytics
Skills
Required:People managementML/NLP/IR analytics
Nice to have:Auditing methodologies
Benefits
- Health insurance (medical, dental, vision, prescription)
- Basic Life & AD&D insurance with option for Supplemental life plans
- EAP, Mental Health Support
- Flexible Spending Accounts
- Adoption and Surrogacy Reimbursement coverage
- 401(k) matching
- Paid time off
- Parental leave
Last updated Oct 25, 2026. Interested? Apply on the company site.
More open roles at Amazon
- IT Support Associate II, Ops Tech Solutions (OTS)todayPlainfield, Indiana · Operations IT Support Engineering
- Head of Energy and Water, AWS LegaltodayHybrid (Seattle, WA) · Legal
- Software Development Engineer, Creative Asset Library, CreativeXtodaySeattle · Software Development
- Software Development Engineer, Creative Asset Library, CreativeXtodayHybrid (Seattle, WA) · Software Development
- IT Support Associate II, OTStodayIrving, TX · Operations IT Support Engineering