Senior Software Engineer, LLM Inference
NVIDIA· Architect
Tel AvivFull-timePosted 5mo ago
About the role
Senior Software Engineer focused on performance optimization and generative AI for LLM inference, working on advanced architectures and productionizing research within NVIDIA’s hardware ecosystem.
Responsibilities
- Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models
- Profile inference pipelines using NVIDIA's profiling and simulation tools; correlate simulation predictions against real hardware across data center and edge devices
- Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs
- Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment
- Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer
- Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide
Qualifications
- B.Sc., M.Sc., or equivalent experience in Computer Science or Computer Engineering
- 5+ years of hands-on software engineering experience in performance-critical systems
- Solid understanding of deep learning architectures (Transformers, SSMs, MoE, …)
- Experience with systems where hardware constraints matter: GPU programming, memory hierarchy, networking, or distributed computing
- Strong software engineering fundamentals: clean design, extensibility, testability. Good judgment about when complexity is warranted
- Effective communicator who works well across teams and time zones
- Experience optimizing deep learning workloads on NVIDIA GPUs using roofline models, Nsight/PyTorch profilers and end-to-end traces
Skills
Required:In-depth knowledge of deep learning architecturesGPU programmingProfiling and performance optimization
Nice to have:Distributed inference
Last updated Jul 24, 2026. Interested? Apply on the company site.
About NVIDIA
World Leader in AI Computing
NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.
AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
More open roles at NVIDIA
- Senior Physical Verification Engineer3d agoEngineer, ASIC
- Senior System Software Engineer, Display Security3d agoEngineer, Sys SW
- Senior FPGA Prototyping Engineer - Hardware3d agoSanta Clara, CA · Engineer, ASIC
- Senior Product Engineer3d agoHybrid (Thailand) · Engineer, Product Development
- Senior Architect - Data Processing Systems, Analytics and Data Intelligence3d agoHybrid (Remote / Local) · Architect