← All roles

Distinguished Engineer, Scaled Out Inferencing

NVIDIA· Distinguished Engineer
Apply on company site
Santa Clara, CA, Hybrid (Remote (CA))$320k - $488.8kfull-timePosted 2mo ago

About the role

Lead the development of NVIDIA's global strategy for scaled-out AI inferencing, architect high-throughput, low-latency distributed pipelines and model serving strategies for massive scale and production reliability across enterprise and cloud environments.

Responsibilities

  • Architect distributed pipelines, define and drive the technical implementation of high-throughput, low-latency, distributed inference systems to support massive-scale AI workloads.
  • Hardware-software co-optimization, drive performance tuning at the kernel and driver level, optimizing GPU resource management and hardware acceleration for production-grade model serving.
  • Guide and influence open source projects Dynamo, TensorRT-LLM, and ecosystem projects (vLLM, SGLang, Linux, Kubernetes, Ray) to bring state of the art inferencing on NVIDIA accelerated hardware.
  • Orchestrate model lifecycles, lead the strategy for full-lifecycle model management, including automated deployment, versioning, and intelligent scaling across varied cloud and datacenter environments.
  • Collaborate with customers, infrastructure providers, and partners to ensure NVIDIA’s solutions set the industry standard for performance and availability.
  • From ideation to architecture, design, development, deployment, operations, and full lifecycle management, lead all technical aspects of planning and continuous evolution of a large technical scope.

Qualifications

  • 16+ overall years in technical roles with a recent long-term focus on AI infrastructure and more recent direct experience in large-scale inference orchestration; proven track record building secure, highly available, and durable production distributed systems.
  • 7-10+ years of leadership experience
  • BS/MS or higher or equivalent experience in systems / software engineering, or related engineering fields
  • Deep Technical Expertise: Proficiency in GPU architecture, hardware acceleration, and low-level performance tuning (CUDA, kernels) alongside cloud-native architectures for multi-tenant model serving.
  • Proven success delivering high-impact technically complex solutions that achieve transparency into resource utilization, performance, and operational insights.
  • Technical Leadership: Develop and advance consensus and organizational alignment across technical leadership and senior corporate leadership; synthesize cross-functional needs into architecture and design while guiding internal execution across diverse teams.
  • Communication and Teamwork: Strong collaboration and influence skills, capable of leading engineering engagement, communicating with peers, partners, and working with high performance and accelerated computing customers.

Skills

Required:Distributed systems designGPU architecture and CUDAModel serving/container orchestrationLeadership and cross-functional collaboration

Benefits

  • Competitive salaries
  • Generous benefits package
  • Equity eligibility
Last updated Aug 14, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA