Senior Software Engineer II, Inference
CoreWeave· Technology
Sunnyvale, CA, Bellevue, WA$182k - $242k + equity + bonusfull-timePosted 3d ago
About the role
Lead engineering design and reliability improvements for CoreWeave’s Kubernetes-native inference platform, optimizing latency, throughput, and performance at scale.
Responsibilities
- Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones.
- Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release.
- Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact.
- Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies.
- Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards.
- For IC4: own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation).
Qualifications
- IC3: ~3–5 years; IC4: ~5–8 years industry experience building distributed systems or cloud services.
- Strong coding in Python or Go (C++ a plus) and deep familiarity with networked systems and performance.
- Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks (Prometheus, Grafana, OpenTelemetry).
- Practical knowledge of inference internals: batching, caching, mixed precision (BF16/FP8), streaming token delivery.
- Proven track record improving tail latency (P95/P99) and service reliability through metrics-driven work.
- Bachelor's or Master's in Computer Science or related field (or equivalent practical experience).
Skills
Required:Distributed systemsPythonGoNetworked systems and performanceKubernetesCI/CDObservabilityInference internalsTail latency and reliability optimizationComputer Science
Nice to have:C++Inference frameworksGPU computing and interconnectsMulti-team leadership
Benefits
- Medical, dental, and vision insurance - 100% paid for by CoreWeave
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Ability to Participate in Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- 401(k) with a generous employer match
- Flexible PTO
- Catered lunch each day in our office and data center locations
- A casual work environment
- A work culture focused on innovative disruption
Last updated Nov 18, 2026. Interested? Apply on the company site.
About CoreWeave
The Essential Cloud for AI
CoreWeave is the force multiplier that empowers pioneers with momentum, magnitude, and mastery—enabling them to innovate with confidence. Explore the #1 AI Cloud.
AI CloudCluster ManagementGPU computeManaged ServicesNetworkingPlatformStorageliquid-ai-recruiting-target
More open roles at CoreWeave
- Principal Engineer, IaaS3d agoChicago, Illinois · Technology
- Senior Production Engineer3d agoSunnyvale, CA · Technology
- Senior Product Manager - Tooling & Experience3d agoSunnyvale, CA · Technology
- Senior Network Capacity Planner3d agoNew York, NY, Sunnyvale, CA +2 more · Supply Chain & Capacity Operations - G&A
- Senior Supply Chain Compliance Lead3d agoSunnyvale, CA, Bellevue, WA · Supply Chain & Capacity Operations - G&A