← All roles

Senior System Software Engineer - Local AI

NVIDIA· Engineer, Sys SW
Apply on company site
PuneFull-timePosted 1mo ago

About the role

Senior Systems Software Engineer to lead architecture of efficient on-device AI software for RTX and DGX-class systems, focusing on high-performance local inference with low latency, optimized memory usage, and deployment on resource-constrained platforms.

Responsibilities

  • Partner with NVIDIA's software, research, architecture, and product leadership, as well as external partners such as Microsoft, to align technical requirements and strategic priorities, fostering the AI ecosystem on RTX and DGX PCs.
  • Build and optimize the local AI inference stack for RTX, RTX Pro, and DGX GPUs, with a focus on performance, stability, and scalability across diverse hardware architectures.
  • Architect and develop modern inference runtimes and execution stacks using frameworks such as llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX, supporting LLM, vision-language, TTS, ASR, and diffusion-based AI workloads.
  • Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to maximize performance on current and next-generation GPU architectures. Apply model optimization techniques, including quantization, pruning, sparsity, and distillation, to enable efficient deployment of large models on local and edge devices.
  • Lead system-level debugging, performance tuning, and performance-accuracy trade-off analysis; develop infrastructure for performance and accuracy sweeps; analyse results to identify gaps and drive fixes; and establish engineering guidelines to accelerate bring-up and ensure production readiness of new models and inference backends.
  • Mentor other engineers and review architecture proposals spanning several groups, acting as a force multiplier for technical quality and consistency across the broader LocalAI organization.

Qualifications

  • 5+ Years of experience with Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or related field, or equivalent experience.
  • Excellent C++ programming and debugging skills, with a strong foundation in data structures, algorithms, and machine learning.
  • Proven experience architecting and optimizing AI inference pipelines and applications using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, and TensorRT.
  • Deep understanding of inference backends and runtime internals, including scheduling, memory management, KV-cache behavior, graph execution, quantization, and hardware-aware optimization.
  • Demonstrated ability to establish technical direction and influence outcomes across multiple teams and organizations without relying on formal management authority.
  • Strong analytical and problem-solving skills, with the ability to manage multiple priorities effectively in a fast-paced environment.
  • Excellent written and verbal communication skills, enabling effective collaboration across engineering teams and management.

Skills

Required:C++AI inference pipelinesFrameworksRuntime internalsLeadership

Benefits

  • competitive salaries
  • generous benefits package
  • opportunity to work with industry-leading professionals
  • equal opportunity workplace
  • diversity commitment
Last updated Oct 2, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA