← All roles

Senior Software Engineer, Model Serving

Databricks· Engineering
Apply on company site
San Francisco, CA$166k - $225kFull-timePosted 11mo ago

About the role

Senior engineer role focused on designing and building high-throughput, low-latency inference systems for Databricks Model Serving, working with CPU and GPU workloads, and collaborating across product, platform, infrastructure, and research teams to deliver a scalable serving platform.

Responsibilities

  • Design and implement core systems and APIs that power Databricks Model Serving, ensuring scalability, reliability, and operational excellence.
  • Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for CPU and GPU serving workloads.
  • Contribute to key components across the serving infrastructure — from model container builds and deployment workflows to runtime systems like routing, caching, observability, and intelligent autoscaling — ensuring smooth and efficient operations at scale.
  • Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
  • Lead technical initiatives that improve latency, availability, and cost-effectiveness across both customer-facing and foundational serving layers.
  • Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.

Qualifications

  • 5+ years of experience building and operating large-scale distributed systems.
  • Experience in model serving, inference systems, or related infrastructure (e.g., routing, scheduling, autoscaling, and observability).
  • Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
  • Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
  • Experience building architecture for large-scale, performance-sensitive CPU/GPU inference systems.
  • Strong communication skills and ability to collaborate across teams in fast-moving environments.
  • Customer-focused mindset with the ability to align implementation details with product goals.
  • Passion for mentoring, growing engineers, and fostering technical excellence.

Skills

Required:Distributed systemsModel serving / inference systemsAlgorithms and data structuresArchitectural leadershipCross-functional collaboration

Benefits

  • Benefits information available via regional page (link provided in job description)
Last updated Oct 15, 2026. Interested? Apply on the company site.

About Databricks

Leading Data and AI Platform for Enterprises

Databricks offers a unified platform for data, analytics and AI. Build better AI with a data-centric approach. Simplify ETL, data warehousing, governance and AI on the Data Intelligence Platform.

AIanalyticsbig datadatadata warehousingenterprise-infrastructuregovernanceliquid-ai-recruiting-target
Company profileWebsite

More open roles at Databricks