← All roles

Senior Deep Learning Software Engineer, Inference and Model Optimization

NVIDIA· Engineer, Sys SW
Apply on company site
Santa Clara, CA, Hybrid (Remote - US)$184k - $356.5kFull-timePosted 1w ago

About the role

Senior Deep Learning Software Engineer to develop and scale automated inference and deployment solutions for optimization of generative AI models (LLMs and diffusion models) using NVIDIA’s AI software stack, including CUDA kernels, TRT-LLM, and Triton, with a focus on performance, scalability, and user experience.

Responsibilities

  • Train, develop, and deploy state-of-the generative AI models like LLMs and diffusion models using NVIDIA's AI software stack.
  • Leverage and build upon the torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc...) to analyze and extract standardized model graph representation from arbitrary torch models for automated deployment solution.
  • Develop high-performance optimization techniques for inference, such as automated model sharding techniques (e.g. tensor parallelism, sequence parallelism), efficient attention kernels with kv-caching, and more.
  • Collaborate with teams across NVIDIA to use performant kernel implementations within automated deployment solution.
  • Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.
  • Continuously innovate on inference performance to maintain and increase leadership in NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer).
  • Play a pivotal role in architecting and designing a modular and scalable software platform to provide an excellent user experience with broad model support and optimization techniques to increase adoption.

Qualifications

  • Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field.
  • 8+ years of relevant work or research experience in Deep Learning.
  • Excellent software design skills, including debugging, performance analysis, and test design.
  • Strong proficiency in Python, PyTorch, and related ML tools (e.g. HuggingFace).
  • Strong algorithms and programming fundamentals.
  • Good written and verbal communication skills and the ability to work independently and collaboratively in a fast-paced environment.

Skills

Required:PythonPyTorchHuggingFace

Benefits

  • comprehensive benefits package
  • equity
  • base salary and location-based compensation
  • opportunities for growth and professional development
Last updated Oct 16, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA