Staff Software Engineer, ML Performance, GPU
Google· Google Cloud / Core ML GPU Performance (Implicit)
Sunnyvale, CA, USA$207k - $300kFull-timePosted 4w ago
About the role
Software engineer role focused on optimizing GPU performance for ML workloads, with responsibilities spanning benchmarking, architecture-level simulations, and collaboration with product teams to scale LLMs and ML models on GPU hardware.
Responsibilities
- Identify and maintain LLM training and serving benchmarks; use them to identify performance opportunities, drive XLA:GPU/Triton performance and guide XLA releases.
- Partner with product teams (e.g., Google DeepMind) to onboard, optimize, and scale LLMs and machine learning models on GPU hardware.
- Conduct architecture-level simulations, performance benchmarking, and roofline analyses using tools like TRT-LLM, vLLM, and SGLang to guide system designs.
- Analyze fleet-wide performance and efficiency metrics to identify bottlenecks and engineer scalable optimizations across Google's infrastructure.
- Research and implement model/data efficiency techniques, tooling, and profiling mechanisms to improve workload performance and training efficiency.
Qualifications
- Bachelor’s degree or equivalent practical experience.
- 8 years of experience in software development.
- 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
- Experience with modern GPU architectures, memory hierarchies, and performance bottlenecks.
- Experience with low-level GPU programming (CUDA, Triton, CUTLASS, etc.) and performance engineering techniques.
- Experience with modern LLMs and their deployment on AI accelerators.
Skills
Required:LLM benchmarkingGPU performance optimizationLow-level GPU programmingSystem architecture and modelingCollaboration with cross-functional teams
Benefits
- Learn more about benefits at Google (link provided in description)
Last updated Oct 23, 2026. Interested? Apply on the company site.
More open roles at Google
- Staff Network Security Engineer, Google Enterprise Network2d agoHybrid (Austin, TX; Sunnyvale, CA)
- Group Product Manager, Google Lens AI Shopping2d agoMountain View, CA, USA · Product Management
- UX Program Manager2d agoHybrid (New York, NY; San Bruno, CA) · UX Programs & Operations
- Senior Staff Software Engineer, Android Automotive OS2d agoHybrid (Kraków, Poland; Munich, Germany)
- 2d ago