← All roles

AI Engineer – Algorithm Evaluation & Agentic Systems

Apple· Hardware
Apply on company site
Sunnyvale, CA$150.4k - $277.6kFull-timePosted 2mo ago

About the role

AI Engineer specializing in algorithm evaluation and agentic systems design for advanced computer vision and video understanding algorithms within Apple’s DAQ team. Focus on benchmarking, evaluation pipelines, failure analysis, and architecting agentic multi-modal workflows that bridge experimentation with production.

Responsibilities

  • Design, build, and scale comprehensive evaluation pipelines for end-to-end system evaluation and granular component testing of image and video understanding models.
  • Conduct deep failure analysis to identify root causes of visual hallucinations and temporal inconsistencies in video.
  • Build, deploy, and evaluate agentic workflows that utilize vision models to autonomously solve multi-step user problems (e.g., video summarization, visual search).
  • Isolate and triage which parts of the agentic workflow (tool selection, memory retrieval, visual reasoning) are succeeding or failing.
  • Lead strategy for curating high-quality, schematized datasets and ground-truth benchmarks for evaluating multi-modal capabilities.
  • Partner with core model training teams to provide data-driven insights and metrics to guide next iterations of model training and fine-tuning.

Qualifications

  • MS and a minimum of 3 years relevant industry experience
  • 3+ years of applied experience in Machine Learning, Computer Vision, or AI System Evaluation
  • Solid ML Foundation: probability, statistics, data distributions, model bias/variance
  • Computer Vision Expertise: CV, Vision-Language Models, ViTs, spatial-temporal modeling, image/video processing
  • Advanced Evaluation Skills: robust metrics/KPIs, evaluation frameworks, benchmark creation, automated regression testing, LLM/VLM-as-a-judge, human-in-the-loop evaluation
  • Agentic Systems: building and evaluating LLM/VLM-powered agents with tool use, multi-step reasoning, planning, memory management
  • Failure Analysis: probing models for edge cases, hallucinations, performance bottlenecks; translating findings into improvements
  • Engineering Excellence: proficiency in Python and PyTorch for running inference, embeddings extraction, scalable evaluation pipelines
  • Demonstrated ability to lead technical evaluation strategies end-to-end, drive testing infrastructure decisions, mentor engineers
  • Strong foundation in statistics: hypothesis testing, confidence intervals, experimental design
  • Knowledge of reinforcement learning, planning, or decision-making systems
  • Experience evaluating multi-modal or multi-agent systems
  • Prior work on AI reliability, safety, or benchmarking

Skills

Required:Evaluation pipeline designFailure analysisAgentic systems developmentDataset curation

Benefits

  • Comprehensive medical and dental coverage
  • Retirement benefits and employee stock purchase plan opportunities
  • Discounted products and free services
  • Educational expense reimbursement including tuition
  • Potential discretionary bonuses or commissions and relocation assistance
Last updated Aug 14, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple