← All roles

Research Scientist, Clinical AI Evaluation Lead

Meta· Artificial Intelligence
Apply on company site
Burlingame, CA$184k - $257kFull-timePosted 1w ago

About the role

Hands-on Research Scientist role to lead evaluation strategy and benchmarking for Meta Health GenAI, partnering with clinicians, engineers, product, and regulatory teams to connect model-level metrics to clinical meaning and drive clinically validated AI health solutions.

Responsibilities

  • Own the evidence standard for Health GenAI launches: define the readiness bar, produce the evidence package behind each launch decision, and serve as the scientific authority that decision rests on.
  • Define and lead the evaluation strategy for GenAI health products, establishing what “good,” “safe,” and “ready to ship” mean, and the evidence required to claim each.
  • Design and build LLM and multimodal benchmarks for health: clinical reasoning, factuality and hallucination, safety and harm avoidance, appropriate escalation and refusal, calibration and uncertainty, and fairness across populations.
  • Develop clinical confirmation methodology, pairing automated evaluations with clinician-in-the-loop adjudication, inter-rater reliability, and study designs that produce defensible evidence.
  • Partner with product and AI engineering from concept through launch of Meta's Health GenAI features, defining the claim, the evidence, and the guardrails before the feature exists.
  • Partner with clinical research, regulatory, and legal to align evaluation evidence with submission and documentation needs where products are on a regulated path.
  • Define new research directions in health AI evaluation where no established methodology exists, and publish work that shapes how the field measures clinical generative AI.
  • Set technical direction across teams, raise the scientific bar through review and mentorship, and represent Meta's position on Health AI evaluation externally through publication, benchmarks, and standards work.
  • Generates entirely new research directions, works on problems of the highest complexity and ambiguity, and makes technical contributions likely to reshape how the field thinks about the area.

Qualifications

  • 10+ years of experience in AI/ML or a quantitative research field, including 3+ years designing and running evaluations of LLMs or foundation models. Experience building clinical benchmarks, rubrics, and measurement methodology (safety, factuality, calibration, and bias)
  • 3+ years applying measurement science to generative AI: constructing benchmarks and eval sets, defining scoring rubrics, validating human and model-based graders, and quantifying uncertainty in model quality claims
  • Experience applying machine learning or GenAI to healthcare or medical problems (particularly in electronic health records) in direct partnership with clinical domain experts, including at least one system deployed to real users via real-world clinical confirmation study
  • Experience setting technical direction across organizational boundaries, influencing clinical, product, engineering, and regulatory partners
  • Strong publication record at top-tier venues or high-impact peer-reviewed journals
  • MD and/or PhD in Biomedical Informatics, Machine Learning, Computer Science, Statistics, or a related field, or equivalent practical experience
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience

Skills

Required:AI/ML evaluation designClinical benchmarkingGenAI measurement science

Benefits

  • Bonus
  • Equity
  • Benefits
Last updated Oct 23, 2026. Interested? Apply on the company site.

About Meta

Shop the latest AI glasses and wearable technology from Meta.

Shop the latest AI glasses and wearable technology from Meta. Explore Ray-Ban Meta, Oakley Meta, display glasses, and cutting-edge VR headsets.

AI glassesOakley MetaRay-Ban MetaVR headsetsliquid-ai-recruiting-targetwearable technology
Company profileWebsite

More open roles at Meta