← All roles

Evaluation Science Lead

Apple· Globalization Quality and Operations
Apply on company site
Austin, TX, Cupertino, CAfull-timePosted 1w ago

About the role

Lead the evaluation science for multilingual Globalization AI solutions across Apple Services, defining measurement, validation, and quality frameworks for 50+ languages and dozens of markets. Build evaluation science as a discipline at the intersection of language, culture, and AI and turn evaluation into a strategic capability.

Responsibilities

  • Framework & Methodology Design: Define the long-term evaluation science strategy and roadmap for Globalization AI solutions across 50+ languages, including low-resource locales; design statistical methodologies (power analysis, sampling, confidence intervals, significance thresholds); architect scalable evaluation workflows blending human annotation with automated systems; expand criteria beyond linguistic quality to include behavioral and engagement signals.
  • Insights & Continuous Improvement: Translate evaluation data into actionable recommendations and go/no-go evidence for Engineering and leadership; build continuous performance monitoring and drift-detection systems; establish feedback loops to refine evaluation rubrics as models evolve; serve as resident authority on multilingual AI evaluation and uplift best practices across Globalization Apple Services.

Qualifications

  • 5+ years of experience in evaluation science, data science, or ML systems development with end-to-end evaluation systems at scale
  • Experience applying statistical methodology (sampling, significance testing, confidence intervals)
  • Practical understanding of measurement validity principles
  • Hands-on experience measuring annotator agreement, diagnosing divergence, and improving annotation protocols
  • Proficiency with statistical tools and languages (R, Python, SQL) for analysis and reproducibility
  • Experience designing evaluation frameworks adopted and scaled by operational teams
  • Strong written and verbal communication skills for technical and non-technical audiences
  • Direct experience handling sensitive and confidential information with integrity and discretion
  • Ability to be onsite; this role is an in-person, onsite position
  • Availability to work occasional evenings and weekends as needed
  • Up to 10% travel (domestic and international)
  • Master’s, PhD, or comparable experience in Statistics, Computational Linguistics, Computer Science, Psychometrics, Data Science, or related quantitative field
  • Experience evaluating generative AI systems (hallucination detection, safety and cultural alignment, autograders/LLM-as-judge systems, benchmark design, synthetic data evaluation)
  • Experience with A/B testing, causal inference, or experimental design
  • Experience in applied linguistics — translating cultural and linguistic nuances into quantitative evaluation framework

Skills

Required:Evaluation design and strategyStatistical methodologiesAnnotation and calibrationData analysis toolsCommunicationConfidential information handling

Benefits

  • Includes accessibility and diversity commitments; benefits referenced in Apple culture and workplace guidance
  • Onsite work and travel opportunities as needed
Last updated Oct 16, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple