← All roles

Principal Data Engineer

Microsoft· Software Engineering
Apply on company site
Redmond, WA$165.6k - $296.4kFull-timePosted 3w ago

About the role

Lead architecture, development, and operation of scalable, reliable data platforms with expertise in ETL, data pipelines, data modeling, and distributed systems. Hands-on with Apache Spark, Python, and SQL; drive AI-native engineering practices and mentor teams to ensure quality, security, and scalable solutions.

Responsibilities

  • Lead data-platform architecture: Design and evolve ETL workflows, data pipelines, and data models using Spark, Python, and SQL.
  • Drive hands-on, end-to-end implementation: Build data solutions using Spark (or similar distributed systems), open table formats such as Delta Lake, and Parquet files; develop scalable, secure APIs and data access controls, including RBAC; provision resources through Infrastructure as Code; and deliver production-grade deployments through CI/CD with observability and operational runbooks.
  • Think beyond immediate requirements: Own complex system-design decisions, evaluate alternatives, and account for cross-team dependencies, scalability, resiliency, disaster recovery, and cost.
  • Make evidence-based decisions: Proactively gather facts, validate assumptions, test hypotheses, and use telemetry and investigation to resolve complex problems rather than relying on unverified assumptions.
  • Advance AI-native engineering: Apply AI across design, coding, testing, and troubleshooting; validate AI-generated outputs and help the team adopt effective, responsible practices.
  • Own production outcomes: Establish monitoring and operational readiness, drive incident resolution and root-cause analysis, and implement preventive improvements. Automate safe deployments, targeting zero-touch deployment where possible.
  • Raise engineering quality: Lead design and code reviews, strengthen testing, and embed security, privacy, and compliance into solutions throughout their lifecycle.
  • Collaborate and mentor: Communicate trade-offs clearly, align partner teams on dependencies and ownership, mentor engineers, and share knowledge to improve team capabilities.

Qualifications

  • Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Skills

Required:Data platform architectureApache SparkPythonSQL

Benefits

  • This information is not explicitly listed in the job description.
Last updated Oct 1, 2026. Interested? Apply on the company site.

About Microsoft

Cloud ComputingConsumer ElectronicsSoftwareTechnologyliquid-ai-recruiting-target
Company profileWebsite

More open roles at Microsoft