← All roles

Senior System Software Engineer, Agentic Inference - Dynamo

NVIDIA· Engineer, Sys SW
Apply on company site
Santa Clara, CA, Hybrid (US (Remote))$224k - $356.5k + equityFull-timePosted 2mo ago

About the role

Senior System Software Engineer to work on Dynamo, NVIDIA's GPU-accelerated deep learning inference platform. Focus on open source inference serving, agentic inference workloads, and distributed frontend across multiple inference engines, with emphasis on performance, scalability, and integration with stateful, multi-turn AI tasks.

Responsibilities

  • Develop open source software to serve inference of trained AI models running on GPUs.
  • Contribute to the development of disaggregated serving for Dynamo-supported inference engines (vLLM, SGLang, TRT-LLM) and expand capabilities to support agentic inference workloads (long-horizon reasoning, tool calling, and stateful, multi-turn execution).
  • Innovate in inference-state management for long-running agents (KV- and prefix-cache reuse, transfer across memory/storage hierarchies with NIXL) to reduce repeated prompt processing, improve latency and token throughput, maximize GPU utilization, and lower costs for self-hosted LLMs.
  • Build and evolve Dynamo’s distributed inference frontend across vLLM, SGLang, and TensorRT-LLM, delivering day-0 support for new models, model-specific parameters, API compatibility, and stateful Responses API semantics.
  • Balance objectives to build robust, scalable, high-performance components; collaborate with team leads to prioritize features; load-balance asynchronous requests; optimize throughput under latency constraints; and integrate latest open source technology.

Qualifications

  • Masters or PhD or equivalent experience
  • 10+ years in Computer Science, Computer Engineering, or related field
  • Ability to work in a fast-paced, agile team environment
  • Excellent Rust/Python programming and software design skills, including debugging, performance analysis, and test design
  • Understanding of modern LLM API semantics, including structured outputs, tool calling, reasoning controls, token accounting, context management, and multimodal inputs

Skills

Required:RustPythonDistributed systemsLLM API semantics

Benefits

  • equity
  • benefits
Last updated Jul 31, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA