← All roles

Senior Software Engineer, Inference

CoreWeave· Technology
Apply on company site
Hybrid (Sunnyvale, CA), Hybrid (Bellevue, WA)$152k - $204k + equity + bonusFull-timePosted 7mo ago

About the role

Senior engineer leading designs and reliability improvements for a Kubernetes-native inference platform, partnering across product, orchestration, and hardware teams to meet strict P99 SLAs at scale.

Responsibilities

  • Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones.
  • Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release.
  • Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact.
  • Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies.
  • Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards.
  • Own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation).

Qualifications

  • ~ 5-8 years industry experience building distributed systems or cloud services.
  • Strong coding in Python or Go (C++ a plus) and deep familiarity with networked systems and performance.
  • Optimize end-to-end ML system performance by developing and tuning CUDA kernels, reducing model latency, maximizing compute and memory bandwidth utilization, and leveraging custom accelerators for high-efficiency workloads
  • Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks (Prometheus, Grafana, OpenTelemetry).
  • Practical knowledge of inference internals: batching, caching, mixed precision (BF16/FP8), streaming token delivery.
  • Proven track record improving tail latency (P95/P99) and service reliability through metrics-driven work.

Skills

Required:PythonGoKubernetesCI/CDObservabilityInference internals
Nice to have:C++

Benefits

  • Discretionary bonus
  • Equity awards
  • Comprehensive benefits program (US-based)
Last updated Aug 12, 2026. Interested? Apply on the company site.

About CoreWeave

The Essential Cloud for AI

CoreWeave is the force multiplier that empowers pioneers with momentum, magnitude, and mastery—enabling them to innovate with confidence. Explore the #1 AI Cloud.

AI CloudCluster ManagementGPU computeManaged ServicesNetworkingPlatformStorageliquid-ai-recruiting-target
Company profileWebsite

More open roles at CoreWeave