← All roles

Research, Evals

Exa· Research
Apply on company site
San Francisco, California$180k - $350kFull-timePosted 11mo ago

About the role

ML evals engineer to design and build Exa's eval stack for evaluating search in an LLM world. Role involves designing comprehensive evaluation frameworks, building scalable eval pipelines, and shaping the future direction of the research team and company.

Responsibilities

  • Design and implement evaluation frameworks that probe the limits of search
  • Build scalable, reliable eval pipelines that track regressions, drift, and quality signals across billions of documents
  • Create golden datasets, synthetic benchmarks, agentic tasks, and real-world test suites that reflect how developers, agents, and humans actually use Exa
  • Partner closely with ML researchers, data engineers, infra engineers, and product to shape the feedback loops that improve our search models
  • Write a manifesto of what perfect search means

Qualifications

  • Hands-on ML experience (training, finetuning, or evaluating models)
  • Strong engineering fundamentals and can build reliable systems (Python, Rust, distributed pipelines, GPU/cluster jobs)
  • Enjoy diving into data via building eval sets, inspecting edge cases, designing creative measurement strategies

Skills

Required:Machine Learning evaluationEngineeringData analysis

Benefits

  • premium healthcare benefits (medical, dental, vision)
  • fertility benefits
  • 16 weeks of fully paid parental leave for all new parents
  • monthly wellness stipend
Last updated Jul 15, 2026. Interested? Apply on the company site.
Exa logo

About Exa

Web search rebuilt for LLMs

Exa is a search engine built for AIs. We enable developers to filter the internet's content with complex queries in order to bring the world's knowledge into their AI apps.

ai-infrastructure-and-developmentapisartificial-intelligencebrex-fastest-growing-2025searchwing-list-2026wing-list-2026-mid-stage
Company profileWebsite

More open roles at Exa