← All roles

Director, Evaluation and Quality Assurance, Services Special Projects

Apple· Software and Services
Apply on company site
Cupertino, CA$311.1k - $496.9kFull-timePosted 1w ago

About the role

Director to build and lead the evaluation and quality assurance function for a multi-year engineering initiative. Establish end-to-end quality metrics, automation/infrastructure, and an organization to run evaluation and QA from the ground up, partnering with engineering, applied ML, and program management.

Responsibilities

  • Design metrics and automated grading systems (autograders) that scale to large, frequently changing datasets, and determine balance with human evaluation.
  • Wire evaluation results into release processes as explicit quality gates.
  • Catch and prevent quality regressions across the product surface from user-facing correctness to backend transaction integrity.
  • Partner with engineering and applied ML teams to define what 'quality' and 'done' mean before product definitions settle.
  • Navigate ambiguity and build process, tooling, and org structure in a zero-existing-scenario.
  • Make judgment calls on where to invest limited evaluation resources for maximum risk reduction.

Qualifications

  • 12+ years of experience in data science, machine learning evaluation, quality assurance, or test engineering in a technical product environment, including leading multi-level QA/evaluation organizations through managers.
  • Proven track record of building evaluation/QA from the ground up with metrics, tooling, and organizational design.
  • Extensive experience with LLM-based autograders and rubrics, including rubric design, evaluation set development, validation, calibration, and human-model alignment.
  • Deep experience with traditional human evaluation methods, logging, and A/B testing for consumer products at scale.
  • Experience evaluating conversational AI, recommendation/ranking/search systems; ideally agentic or tool-use systems with real-world actions.
  • BS in Computer Science/Engineering or related field, or equivalent experience.
  • Experience with QA for third-party API integrations and logging/instrumentation for AI product quality measurement.
  • Exceptional written and verbal communication skills with a history of presenting quality status, risk, and trade-offs to senior leadership.

Skills

Required:Evaluation strategy and governanceAutograder design and rubric developmentHuman evaluation methodologies

Benefits

  • Comprehensive medical and dental coverage
  • Retirement benefits
  • Discounted products and services
  • Educational expense reimbursement (tuition)
  • Discretionary bonuses or commissions and relocation potential
  • Stock purchase plan eligibility
Last updated Oct 16, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple