← All roles

Software Engineer, AI and DL Kernel Libraries

NVIDIA· Architect
Apply on company site
Shanghaifull-timePosted 4mo ago

About the role

AI systems software engineer role focused on developing core AI systems software to accelerate workloads on NVIDIA GPUs, including deep learning primitives, kernel libraries, LLM inference runtimes, and related runtimes and abstractions.

Responsibilities

  • Develop production-quality software that ships as part of NVIDIA's AI software stack, including cuDNN, FlashInfer, and optimized support for large language model inference workloads.
  • Innovate and develop new AI systems technologies for efficient inference, with emphasis on performance, scalability, maintainability, and usability.
  • Design, implement, and optimize kernels for high-impact AI workloads across LLM inference, generative AI, computer vision, autonomous driving, and recommender systems.
  • Design and implement extensible software abstractions for deep learning libraries, LLM serving engines, and runtime systems.
  • Build and improve just-in-time compilation, code generation, and runtime technologies for performance-critical GPU workloads.
  • Analyze workload performance, tune current software, and propose improvements to future software and hardware-software interfaces.
  • Collaborate with engineers across deep learning frameworks, libraries, kernels, compilers, and GPU architecture teams at NVIDIA.
  • Contribute to open-source communities and ecosystem integrations such as FlashInfer, vLLM, and SGLang.

Qualifications

  • Master's degree in Computer Science, Electrical Engineering, or related field, or equivalent experience.
  • 3+ years of relevant industry, research, or systems software development experience in machine learning, deep learning systems, compilers, or GPU software. More experience for senior level.
  • Strong programming skills in C/C++ and Python, with hands-on experience developing high-performance software.
  • Solid experience with CUDA development and GPU programming fundamentals.
  • Strong experience developing or using deep learning frameworks such as PyTorch, JAX, TensorFlow, or ONNX.
  • Good understanding of linear algebra, performance analysis, profiling, and code optimization.
  • Experience designing software abstractions, APIs, or higher-level system architecture for performance-sensitive systems.
  • Familiarity with modern ML and inference system trends, especially around LLMs and generative AI.
  • For senior candidates, strong experience in GPU kernel development and performance optimization, especially using CUDA C/C++, cuTile, Triton, or similar technologies.

Skills

Required:C/C++PythonCUDADeep learning frameworksPerformance optimizationSoftware abstractions/API design
Nice to have:GPU kernel development

Benefits

  • Not specified in the description
Last updated Jun 26, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA