← All roles

Senior Product Manager –AI Inference Software

AMD· Engineering
Apply on company site
Hybrid (Bay Area)$205.7kFull-timePosted 3mo ago

About the role

Role owning the framework-layer inference product strategy for ROCm, translating signals into roadmap decisions for production AI inference on AMD Instinct hardware. Collaborates with engineering, open-source communities, and customers to enable scalable, high-performance AI inference across single-GPU to rack-scale deployments.

Responsibilities

  • Own the product strategy and roadmap for ROCm’s inference frameworks software capabilities.
  • Define how AMD’s framework-layer software stack for inference enables production workloads from single-GPU through rack-scale deployments, focusing on performance, developer experience, portability, and competitive differentiation.
  • Identify emerging model architectures, serving technologies, and infrastructure shifts requiring changes in the inference frameworks layer and translate into product priorities.
  • Balance customer needs, ecosystem direction, technical opportunities, and engineering investment to determine differentiation in the software stack.
  • Partner with engineering teams across inference engines, serving and orchestration, memory systems, GPU libraries, runtimes, kernels, and communications, emphasizing framework and orchestration layers for large-scale inference.
  • Translate customer and ecosystem needs into prioritized engineering requirements and drive cross-functional execution.
  • Identify inference performance bottlenecks and determine where framework-level software investment has greatest impact.
  • Provide software-informed input to future GPU architecture and platform decisions relevant to inference performance at scale.
  • Drive release readiness across a fast-moving open-source software ecosystem, with attention to performance, compatibility, regression risk, and adoption.
  • Serve as an active AMD presence in the open-source inference community and build relationships with maintainers, developers, researchers, and contributors.
  • Track emerging technologies and ecosystem direction through open-source projects, research, technical communities, and developer feedback.
  • Communicate AMD’s roadmap and technical direction through community engagement, technical writing, and industry events.
  • Represent AMD constructively in technical discussions where ecosystem and company priorities may differ.
  • Work directly with sophisticated AI infrastructure customers to understand deployment requirements, performance constraints, and future needs at the framework and serving layer.
  • Distinguish individual customer requests from broader technical and market signals, translating signals into scalable product decisions.
  • Balance needs of strategic customers with those of the broader open-source ecosystem.
  • Collaborate with cloud providers, model developers, infrastructure companies, and other ecosystem partners on joint integrations and solutions.

Qualifications

  • Deep understanding of production AI inference systems, including inference engines such as vLLM, SGLang, and AMD ATOM, and serving/orchestration systems such as llm-d, plus memory management and distributed inference.
  • Experience driving complex technical initiatives across multiple engineering teams or organizations.
  • Experience working with sophisticated AI infrastructure customers, open-source communities, or ecosystem partners.
  • Experience in AI inference infrastructure, GPU computing, distributed systems, cloud infrastructure, high-performance computing, developer platforms, research, or technical product leadership.
  • Open-source contributions, technical writing, conference talks, or sustained community engagement are strong positive signals.
  • Ability to communicate technical tradeoffs clearly to engineering leaders, customer stakeholders, open-source contributors, and executive audiences.
  • Product management experience in deeply technical products is valuable but not required; candidates from engineering/research with product interest are encouraged.
  • Familiarity with AMD Instinct hardware and ROCm software ecosystem is a plus.

Skills

Required:AI inference systemsOpen-source ecosystem engagementTechnical leadership

Benefits

  • Benefits offered are described in AMD benefits at a glance.
Last updated Oct 15, 2026. Interested? Apply on the company site.
AMD logo

About AMD

together we advance_AI

AMD delivers leadership high-performance and adaptive computing solutions to advance data center AI, AI PCs, intelligent edge devices, gaming, & beyond.

AIData CenterHigh-Performance ComputingGamingEdge Devices
Company profileWebsite

More open roles at AMD