← All roles

Staff Software Engineer, Inference

CoreWeave· Technology
Apply on company site
Hybrid (Sunnyvale, CA), Hybrid (Bellevue, WA)$188k - $275kfull-timePosted 5mo ago

About the role

Staff-level technical leader role on CoreWeave's Inference team, focused on architecture, performance, and reliability across distributed Kubernetes-based inference services, with hands-on leadership in scheduling, batching, memory optimization, and GPU resource management for low-latency, high-throughput AI workloads.

Responsibilities

  • Lead cross-team design initiatives and influence engineering direction across the organization.
  • Optimize inference performance (latency, throughput, GPU utilization) and system reliability at scale.
  • Work deeply in distributed systems and Kubernetes-based infrastructure, focusing on scheduling, batching, and memory optimization.
  • Drive architecture and performance improvements across multiple services and teams.

Qualifications

  • 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms
  • Proven experience leading cross-team technical initiatives impacting multiple services or organizations
  • Strong programming skills in Go, Python, or C++
  • Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
  • Strong understanding of distributed systems, networking, and performance optimization
  • Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements
  • Hands-on experience with inference systems, including batching or micro-batching strategies, caching, and memory optimization
  • Experience improving system performance using metrics-driven approaches (e.g., latency, throughput, utilization)
  • Familiarity with mixed precision (BF16, FP8) and streaming inference workloads

Skills

Required:GoPythonC++Kubernetes

Benefits

  • Medical, dental, and vision insurance - 100% paid for by CoreWeave
  • Company-paid Life Insurance
  • Voluntary supplemental life insurance
  • Short and long-term disability insurance
  • Flexible Spending Account
  • Health Savings Account
  • Tuition Reimbursement
  • Employee Stock Purchase Program (ESPP)
  • Mental Wellness Benefits through Spring Health
  • Family-Forming support provided by Carrot
  • Paid Parental Leave
  • Flexible, full-service childcare support with Kinside
  • 401(k) with a generous employer match
  • Flexible PTO
  • Catered lunch each day in our office and data center locations
  • Casual work environment
  • Work culture focused on innovative disruption
Last updated Aug 12, 2026. Interested? Apply on the company site.

About CoreWeave

The Essential Cloud for AI

CoreWeave is the force multiplier that empowers pioneers with momentum, magnitude, and mastery—enabling them to innovate with confidence. Explore the #1 AI Cloud.

AI CloudCluster ManagementGPU computeManaged ServicesNetworkingPlatformStorageliquid-ai-recruiting-target
Company profileWebsite

More open roles at CoreWeave