← All roles

Senior ML Software Engineer, Data Plane

Amazon· Annapurna Labs (U.S.) Inc.
Apply on company site
Cupertino$193.3k - $261.5kFull-timePosted 1mo ago

About the role

Senior Software Development Engineer responsible for design and implementation of the inference data plane for a custom ML accelerator, focusing on high-performance model execution, memory management, data movement, and integration with ML serving frameworks.

Responsibilities

  • Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
  • Implement and validate LLM architectures end-to-end - from PyTorch model definition through distributed execution on custom hardware.
  • Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
  • Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
  • Profile and optimize inference workloads - identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
  • Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
  • Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.
  • Mentor engineers, drive design reviews, and raise the engineering bar across the team.

Qualifications

  • Bachelor's degree in computer science or equivalent
  • 7+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
  • Strong proficiency in C/C++
  • Strong Linux systems knowledge
  • Experience developing compute kernels for GPUs, DSPs, or custom accelerators
  • Proven track record of owning and delivering complex software features end-to-end

Skills

Required:Compute kernel developmentC/C++Linux
Nice to have:PyTorchvLLM

Benefits

  • health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans)
  • EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts
  • 401(k) matching
  • paid time off
  • parental leave
Last updated Oct 25, 2026. Interested? Apply on the company site.

About Amazon

Amazon.com, Inc. or its affiliates

liquid-ai-recruiting-target
Company profileWebsite

More open roles at Amazon