← All roles

Senior Staff Machine Learning Engineer, Data & Eval

Airbnb· AI and ML / Core ML team
Apply on company site
Hybrid (United States)$244k - $305kFull-timePosted 6mo ago

About the role

Senior leadership role to set technical direction and lead end-to-end ML evaluation and data flywheel powering Airbnb Assistance Engineering, focusing on GenAI/LLM evaluation, data infrastructure, and scalable evaluation systems.

Responsibilities

  • Define evaluation strategy and success metrics for GenAI systems, aligning offline evaluation with online business and customer experience outcomes.
  • Build and scale evaluation frameworks (golden sets, synthetic data, automated regressions, rubric-based grading, LLM-as-judge where appropriate) with strong controls for bias, drift, and reliability.
  • Design the data flywheel: instrumentation, feedback collection, data quality checks, labeling strategy, dataset versioning, and governance to support continuous improvement.
  • Lead cross-functional quality initiatives across product, ops, and engineering, driving clarity on what “good” looks like and how teams act on evaluation results.
  • Develop and productionize pipelines for dataset creation, model monitoring, evaluation-at-scale, and continuous testing (pre-deploy and post-deploy).
  • Drive technical decisions and architecture for evaluation and data infrastructure, balancing speed, rigor, cost, and safety.

Qualifications

  • PhD in Computer Science, Mathematics, Statistics, or related technical field (or equivalent practical experience).
  • 10+ years building, testing, and shipping ML/AI systems end-to-end; including 2+ years of experience with GenAI/LLM systems in production.
  • 5+ years leading large, ambiguous technical initiatives as a senior IC, influencing roadmap and engineering/science direction across teams.
  • Deep expertise in evaluation methodology (offline/online alignment, metric design, human-in-the-loop evaluation, A/B testing, power analysis, regression testing).
  • Hands-on experience with GenAI systems, including orchestration, retrieval, tool calling, memory, etc.
  • Experience building data pipelines and quality systems (labeling workflows, dataset curation, versioning, monitoring, and governance).
  • Solid ML fundamentals and best practices (model selection, training/serving, monitoring, reliability, and model lifecycle management).

Skills

Required:Evaluation methodologyGenAI/LLM systemsData pipelines & quality systemsML life cycle & deployment
Last updated Oct 22, 2026. Interested? Apply on the company site.
Airbnb logo

About Airbnb

Book accommodations around the world.

Founded in August of 2008 and based in San Francisco, California, Airbnb is a trusted community marketplace for people to list, discover, and book unique accommodations around the world — online or from a mobile phone. Whether an apartment for a night, a castle for a week, or a villa for a month, Airbnb connects people to unique travel experiences, at any price point, in more than 33,000 cities and 192 countries. And with world-class customer service and a growing community of users, Airbnb is the easiest way for people to monetize their extra space and showcase it to an audience of millions. No global movement springs from individuals. It takes an entire team united behind something big. Together, we work hard, we laugh a lot, we brainstorm nonstop, we use hundreds of Post-Its a week, and we give the best high-fives in town. Headquartered in San Francisco, we have satellite offices in Dublin, London, Barcelona, Paris, Milan, Copenhagen, Berlin, Moscow, São Paolo, Sydney, and Singapore.

marketplacetravel
Company profileWebsite

More open roles at Airbnb