← All roles

Member of Technical Staff - Applied AI Lead, Health

Microsoft· Software Engineering
Apply on company site
London£93.5k - £161.8kFull-timePosted 3mo ago

About the role

Applied AI Lead role in Microsoft AI Health team focused on building Copilot Health, leading technical direction for LLM evaluations and orchestration, and driving production-ready health AI products.

Responsibilities

  • Lead, mentor and grow a team of Applied AI Engineers, fostering a collaborative, inclusive and high-performing environment where engineers do the best work of their careers.
  • Stay deeply hands-on. Set the technical bar through code and design reviews, lead by example on the hardest problems, and remain a credible technical authority on evals and LLM systems.
  • Co-own the roadmap. Partner with product leads to qualify and size new opportunities, co-author the product roadmap, and lead the architecture and development of new products and features from 0 to 1.
  • Own delivery. Plan and prioritise the team’s roadmap, balance a strong bias towards shipping and learning with a high-quality bar, and ensure the reliability of what reaches production.
  • Define the evaluation strategy. Design and oversee evaluation systems that test LLM capabilities in the health domain, including internal benchmarking and regression testing that capture model accuracy, safety and utility - and make sure results are interpreted and clearly communicated to stakeholders.
  • Architect LLM orchestration. Guide the design of agentic, multi-step systems that combine prompt / context engineering, tool use and retrieval, and champion best practices for building and deploying them reliably at scale.
  • Run and direct experiments to determine how different prompting and orchestration techniques affect results on internal and industry benchmarks, and turn those findings into product improvements.
  • Invest in tooling. Improve the internal tooling used to implement, run and analyse evaluations, and the data pipelines - dataset sourcing, curation and synthesis - that feed them.

Qualifications

  • Bachelor’s or higher degree in Computer Science or a related technical discipline, AND significant Python programming experience / machine learning research.
  • Very strong proficiency with LLM evaluations - demonstrated experience designing, building and running eval pipelines, curating and synthesising datasets, designing automated analyses, and explaining results to internal stakeholders.
  • Very strong proficiency with LLM orchestration - deep, hands-on experience building with and around LLMs, including prompt / context engineering, tool use, harness engineering, retrieval and agentic, multi-step systems, and building tools to analyse and understand their performance.
  • Proven engineering leadership - 8+ years of software engineering experience, including at least 3 years leading technical teams or projects as a tech lead and/or people manager - mentoring and developing engineers, and guiding a group to deliver high-quality results.
  • 0-to-1 experience with a bias towards shipping and learning while balancing a high-quality bar.
  • Experience collaborating in cross-functional teams, working through ambiguity to deliver high-quality results, and a proven ability to contribute to a positive, inclusive work environment.

Skills

Required:Python programmingLLM evaluationsLLM orchestration
Last updated Sep 3, 2026. Interested? Apply on the company site.

About Microsoft

Cloud ComputingConsumer ElectronicsSoftwareTechnologyliquid-ai-recruiting-target
Company profileWebsite

More open roles at Microsoft