← All roles

AIML - Sr Machine Learning Engineering Manager, Evaluation

Apple· Machine Learning and AI
Apply on company site
Cupertino, CA$237.6k - $401.7kFull-timePosted 1mo ago

About the role

Senior hands-on ML Engineering Manager leading a small team focused on evaluation, agent optimization, and data generation to close gaps between product development and model quality. Responsible for building scalable evaluation systems, defining end-to-end evaluation flywheels, and translating research into production-quality pipelines for Apple Foundation Models and product teams.

Responsibilities

  • Architects and builds scalable evaluation systems for foundation models and agents, including benchmarks, LLM-based evaluators, simulation environments, trajectory analysis, and regression testing.
  • Establishes an end-to-end evaluation flywheel with Apple Foundation Models and product teams that connects observed failures to diagnosis, targeted refinement, post-training, and measurable quality improvement.
  • Leads, mentors, and grows a small team of machine learning engineers while remaining deeply involved in technical design, experimentation, implementation, and review.
  • Defines the technical strategy and roadmap for automatic prompt, context, tool, rubric, and agent-harness optimization for agentic development and model evaluation.
  • Develops methods that convert evaluation findings into actionable model-improvement signals, including targeted datasets, synthetic trajectories, reward or preference signals, and optimization objectives.
  • Partners across AIML to design and scale synthetic data generation pipelines for evaluation and post-training.
  • Applies and adapts recent research in LLM and agent evaluation, automatic optimization, LLM-as-judge, reward modeling, test-time search, and post-training to production-quality workflows.

Qualifications

  • 8+ years of professional experience in machine learning, applied research, or software engineering, including experience building production ML systems or large-scale experimentation platforms.
  • 3+ years of technical leadership experience, including direct people management of machine learning or software engineers and a demonstrated ability to mentor and grow strong technical talent.
  • Master’s or PhD in Computer Science, Machine Learning, Artificial Intelligence, or a related technical field.
  • Strong hands-on programming and software engineering skills, particularly in Python, with experience building reliable ML pipelines using modern machine learning or deep learning frameworks.
  • Deep experience with large language models or agentic systems, including evaluation of multi-turn behavior, tool use, planning, reasoning, or other action-taking workflows.
  • Experience building automated evaluation methods such as LLM-based judges, rubrics, reward models, simulation-based evaluation, or scalable benchmark infrastructure.
  • Experience with at least one model or agent refinement area such as automatic prompt or context optimization, post-training, preference optimization, reinforcement learning, or agent-harness optimization.
  • Excellent communication and collaboration skills, with demonstrated ability to align research, engineering, and product teams around ambiguous technical problems.

Skills

Required:PythonProduction ML SystemsLLM/Agent EvaluationLeadership & Mentoring

Benefits

  • Comprehensive medical and dental coverage
  • retirement benefits
  • range of discounted products and free services
  • reimbursement for certain educational expenses — including tuition
  • potential discretionary bonuses or commission payments
  • relocation assistance
Last updated Aug 28, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple