← All roles

Principal Software Engineer, ML & Distributed Systems

Microsoft· Software Engineering
Apply on company site
Redmond, WA$142.8k - $274.8kFull-timePosted 2mo ago

About the role

Principal Software Engineer to design, build, and operationalize machine learning systems and hyperscale distributed services powering Microsoft's AI experiences, end-to-end ownership from model development to serving and reliable operation at scale.

Responsibilities

  • Design, build, and operationalize scalable ML and deep learning models using containers and orchestration platforms (e.g., Kubernetes)
  • Develop and refine LLM prompt and fine-tuning strategies, build evaluation pipelines, and continuously optimize model quality, latency, and cost
  • Architect, implement, and operate multi-tiered distributed services at hyperscale, with high availability, fault tolerance, and low latency
  • Build model serving and inference infrastructure, including caching, batching, GPU capacity management, and A/B experimentation at scale
  • Design scalable APIs, data pipelines, and feature/signal stores that ensure efficient, secure, and reliable data flow between ML systems and product surfaces
  • Drive live-site excellence: instrumentation, monitoring, capacity planning, and incident response for ML-backed services
  • Collaborate with applied scientists, data scientists, backend engineers, and product teams to translate requirements into production ML systems
  • Participate in code reviews and architectural discussions, and mentor engineers across both ML and systems disciplines

Qualifications

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR equivalent experience

Skills

Required:Distributed systems designML model development and deployment
Nice to have:LLM prompt engineering and fine-tuning

Benefits

  • Microsoft benefits information available on careers site (link provided in description)
Last updated Sep 3, 2026. Interested? Apply on the company site.

About Microsoft

Cloud ComputingConsumer ElectronicsSoftwareTechnologyliquid-ai-recruiting-target
Company profileWebsite

More open roles at Microsoft