← All roles

Research Scientist / Engineer, Foundation Model Evaluation

Apple· Software and Services
Apply on company site
Cupertino, CA$184.7k - $324.8kFull-timePosted 7mo ago

About the role

Hands-on role designing and executing evaluation systems for frontier foundation models at Apple, with focus on evaluation-to-training feedback, benchmarking, product-aligned evaluation, and methodology development to improve model performance while powering Apple Silicon-enabled experiences.

Responsibilities

  • Benchmark Design & Development: Design and implement evaluation benchmarks, metrics, and test suites that rigorously measure model capabilities across reasoning, knowledge, code, and agentic workflows.
  • Product-Aligned Evaluation: Develop evaluation methods that capture how models behave in real product settings, and validate that evaluation metrics predict user-perceived quality and product outcomes.
  • Evaluation Methodology Research & Tooling: Research and apply state-of-the-art evaluation techniques — including scoring frameworks, model-based judging, and contamination-resistant benchmark design. Build reusable tools, scorer libraries, and analysis frameworks that scale across the team's benchmark portfolio.
  • Experimental Analysis: Design and execute rigorous experiments comparing model capabilities, engage with third-party vendors on benchmarking, and perform detailed gap analysis to guide model development priorities.
  • Cross-Team Collaboration: Work closely with model training, training data, and product teams to ensure evaluation insights inform training strategies, data decisions, and product quality improvements.

Qualifications

  • 3+ years of experience in AI model evaluation, NLP, or a related area (e.g., natural language generation, information retrieval, or conversational AI)
  • Strong fundamentals in machine learning, natural language processing, and statistical analysis
  • Proficiency in Python and experience with ML frameworks (PyTorch, JAX, or equivalent)
  • Demonstrated ability to translate research insights into practical implementations
  • Strong experimental design skills: ability to design rigorous comparisons and draw valid conclusions from results
  • Clear technical communication: ability to distill evaluation results into actionable recommendations for cross-functional partners
  • MS or PhD in Computer Science, Machine Learning, Natural Language Processing or a related technical field. Equivalent practical experience will be considered.

Skills

Required:Python

Benefits

  • Comprehensive medical and dental coverage
  • Retirement benefits
  • Discounted products and free services
  • Educational reimbursement (tuition)
  • Discretionary bonuses or commission payments (role dependent)
  • Relocation assistance (where applicable)
  • Apple stock program eligibility (RSU)
Last updated Jul 3, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple