← All roles

Software Engineer, Model Inference

OpenAI· Scaling
Apply on company site
San Francisco$266k - $500k + equity + bonusFull-timePosted 11mo ago

About the role

Engineer to optimize large AI models for high-volume, low-latency production and research environments, focusing on model inference stack performance, reliability, and efficiency.

Responsibilities

  • Work alongside machine learning researchers, engineers, and product managers to bring latest technologies into production.
  • Work alongside researchers to enable advanced research through engineering.
  • Introduce new techniques, tools, and architecture to improve performance, latency, throughput, and efficiency of the model inference stack.
  • Build tools to gain visibility into bottlenecks and instability, and design/implement solutions for high-priority issues.
  • Optimize code and Azure VM fleet to maximize GPU utilization (FLOPs and GPU RAM).

Qualifications

  • At least 5 years of professional software engineering experience.
  • Understanding of modern ML architectures with emphasis on inference optimization.
  • Familiarity with PyTorch, NVIDIA GPUs and optimization stacks (NCCL, CUDA).
  • Experience with HPC technologies such as InfiniBand, MPI, NVLink.
  • Experience architecting, building, observing, and debugging production distributed systems; experience with performance-critical distributed systems preferred.
  • Experience rebuilding or refactoring production systems at increasing scale.
  • Self-directed, problem-focused, collaborative attitude; eager to help teammates.

Skills

Required:Software engineeringPyTorchCUDA / NCCLDistributed systems

Benefits

  • Medical, dental, and vision insurance for you and your family with employer contributions to Health Savings Accounts
  • Pre-tax Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents, 20 weeks for non-birthing parents) and paid medical/caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt and up to 15 days annually for non-exempt
  • 13+ paid company holidays and additional paid closures, plus paid sick or safe time
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend
  • Daily office meals and meal delivery credits
  • Relocation support for eligible employees
  • Additional fringe benefits (e.g., charitable donation matching, wellness stipends)
  • Details available during hiring process
Last updated Sep 3, 2026. Interested? Apply on the company site.

About OpenAI

ai-infrastructure-and-developmentliquid-ai-recruiting-targetllmsetter-30setter-30-2026-q1speech-to-texttext-to-speechvoice-ai-stack
Company profileWebsite

More open roles at OpenAI