← All roles

Senior Software Engineer, Private Cloud Compute Inference Routing

Apple· iCloud Platform Engineering
Apply on company site
Hybrid (Seattle, Washington and Cupertino, California)$142.3k - $263.3kFull-timePosted 3d ago

About the role

Senior software engineer to design, build, and operate inference routing and scheduling for Private Cloud Compute (PCC), enabling scalable, private, low-latency cloud inference for Siri AI and Apple Intelligence. Responsibilities span distributed systems design, network protocols, performance benchmarking, and collaboration with model serving, infrastructure, security, and SRE teams to deliver reliable routing capabilities at internet scale.

Responsibilities

  • Design, build, and operate PCC's inference routing and request scheduling services, including global traffic management, capacity aware request placement, load balancing, and failover.
  • Develop adaptive routing strategies that respond to changing demand, fleet health, and available compute capacity while maintaining latency and availability objectives.
  • Build admission control, prioritization, and overload management mechanisms that keep the service responsive and allocate capacity fairly across competing workloads.
  • Partner with model serving and infrastructure teams to evaluate and develop routing strategies for prefill/decode disaggregation and cache locality optimizations within PCC’s privacy architecture.
  • Design network protocols and implement high-performance server software for inference request dispatch and response streaming.
  • Define performance metrics and benchmark routing strategies under realistic workloads, using queueing delay, request latency, throughput, and fleet utilization to guide improvements.
  • Own projects from architecture through production rollout, partnering with Siri AI, Apple Intelligence, infrastructure, security, and SRE teams to deliver new capabilities and improve reliability.

Qualifications

  • 3+ years of professional experience designing, building and operating large distributed systems in production
  • Server development in Java, C/C++, C#, Go or Swift
  • Knowledge of LLM inference serving, including inference request routing, KV-cache placement and reuse (e.g., prefix-aware routing), prefill/decode disaggregation, and the tradeoffs between latency, throughput and accelerator use
  • Experience with network protocols, application level networking and client-server architectures, including request routing, load balancing or scheduling at scale
  • Experience benchmarking, debugging and tuning the performance of distributed systems, plus knowledge of Unix system internals
  • Experience with cloud deployment and observability technologies such as Kubernetes
  • Bachelor's degree in an engineering related field or equivalent experience
  • Experience building LLM inference serving or routing systems such as llm-d, NVIDIA Dynamo, vLLM, SGLang or similar
  • Familiarity with KV cache aware routing, prefix caching, prefill/decode disaggregation and accelerator capacity placement
  • Experience with TLS, X.509 certificate evaluation or other secure protocols, and awareness of recent developments in standard protocols
  • Understanding of cryptographic protocols, encryption, key management and privacy-preserving system design (e.g., Oblivious HTTP, remote attestation)
  • Experience leading a substantial engineering project from design through production, resolving technical tradeoffs and coordinating across teams.
  • Ability to explain technical tradeoffs, collaborate across engineering teams, and make progress on problems with incomplete requirements.

Skills

Required:Distributed systems design and operationJava/C/C++/C#/Go/Swift server developmentLLM inference serving and routingNetworking and load balancing at scaleBenchmarking and performance tuningCloud deployment and observability (Kubernetes)

Benefits

  • Comprehensive medical and dental coverage
  • Retirement benefits
  • Discounted products and free services
  • Education-related reimbursement (tuition)
  • Discretionary bonuses or commission payments (eligibility varies)
  • Relocation assistance (where applicable)
  • Apple stock purchase plan eligibility
Last updated Oct 23, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple