← All roles

Senior System Software Engineer - LocalAI

NVIDIA· Engineer, Sys SW
Apply on company site
Pune, Indiafull-timePosted 2mo ago

About the role

Senior Systems Software Engineer focused on developing efficient on-device AI software for RTX and DGX-class systems, delivering high-performance local inference with low latency and optimized memory on resource-constrained platforms.

Responsibilities

  • - Partner with NVIDIA’s software, research, architecture, and product teams to align technical requirements and strategic priorities, fostering the AI ecosystem on RTX and DGX PCs.
  • - Build and optimize the local AI inference stack for RTX, RTX Pro, and DGX GPUs, focusing on performance, stability, and scalability across diverse hardware architectures.
  • - Design and develop modern inference runtimes and execution stacks using frameworks such as llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX, supporting LLM, vision-language, TTS, ASR, and diffusion-based AI workloads.
  • - Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to maximize performance on current and next-generation GPU architectures; apply quantization, pruning, sparsity, and distillation for efficient deployment on local/edge devices.
  • - Conduct system-level debugging, performance tuning, and performance-accuracy trade-off analysis; develop infrastructure for performance and accuracy sweeps; establish engineering guidelines for production readiness of new models and inference backends.

Qualifications

  • - 5+ Years of experience with Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or related field, or equivalent experience.
  • - Excellent C++ programming and debugging skills with strong foundation in data structures, algorithms, and machine learning.
  • - Proven experience developing and optimizing AI inference pipelines using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, and TensorRT.
  • - Deep understanding of inference backends and runtime internals, including scheduling, memory management, KV-cache behavior, graph execution, quantization, and hardware-aware optimization.
  • - Strong analytical and problem-solving skills; ability to manage multiple priorities in a fast-paced environment.
  • - Excellent written and verbal communication skills for collaboration across engineering teams and management.

Skills

Required:C++ programmingAI inference pipelinesFrameworks: llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, TensorRTInference backends and runtime internalsPerformance tuning and optimizationCommunication

Benefits

  • competitive salaries
  • generous benefits package
  • opportunity to work with leading AI computing technology and talented professionals
Last updated Jul 31, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA