← All roles

Senior Software Engineer, Private Cloud Compute Inference Routing

Apple· iCloud Platform Engineering
Apply on company site
Cupertino, CA, Seattle, WA$150.4k - $277.6kFull-timePosted 3d ago

About the role

Senior software engineer to build next generation inference routing for Private Cloud Compute (PCC), designing distributed systems to route inference requests across a global fleet of inference servers while balancing latency, availability, and compute efficiency within a privacy-preserving model.

Responsibilities

  • Design, build, and operate PCC's inference routing and request scheduling services, including global traffic management, capacity aware request placement, load balancing, and failover.
  • Develop adaptive routing strategies that respond to changing demand, fleet health, and available compute capacity while maintaining latency and availability objectives.
  • Build admission control, prioritization, and overload management mechanisms that keep the service responsive and allocate capacity fairly across competing workloads.
  • Partner with model serving and infrastructure teams to evaluate and develop routing strategies for prefill/decode disaggregation and cache locality optimizations within PCC’s privacy architecture.
  • Design network protocols and implement high-performance server software for inference request dispatch and response streaming.
  • Define performance metrics and benchmark routing strategies under realistic workloads, using queueing delay, request latency, throughput, and fleet utilization to guide improvements.
  • Own projects from architecture through production rollout, partnering with Siri AI, Apple Intelligence, infrastructure, security, and SRE teams to deliver new capabilities and improve reliability.

Qualifications

  • 3+ years of professional experience designing, building and operating large distributed systems in production
  • Server development in Java, C/C++, C#, Go or Swift
  • Knowledge of LLM inference serving, including inference request routing, KV-cache placement and reuse (e.g., prefix-aware routing), prefill/decode disaggregation, and the tradeoffs between latency, throughput and accelerator use
  • Experience with network protocols, application level networking and client-server architectures, including request routing, load balancing or scheduling at scale
  • Experience benchmarking, debugging and tuning the performance of distributed systems, plus knowledge of Unix system internals
  • Experience with cloud deployment and observability technologies such as Kubernetes
  • Bachelor's degree in an engineering related field or equivalent experience

Skills

Required:Distributed systems design and operationServer developmentLLM inference serving and routingNetwork protocols and client-server architecturesBenchmarking and performance tuning of distributed systemsCloud deployment and observability (Kubernetes)
Nice to have:Security and privacyLeadership of engineering projectsCommunication of tradeoffs

Benefits

  • Comprehensive medical and dental coverage
  • retirement benefits
  • discounted products and free services
  • educational expense reimbursement for advancing career
  • discretionary bonuses or commission payments (possible)
  • relocation assistance (possible)
  • employee stock programs including RSUs and Employee Stock Purchase Plan
Last updated Oct 23, 2026. Interested? Apply on the company site.

About Apple

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.

AirPods Max 2Apple ArcadeApple CardApple Fitness+Apple MusicApple TVApple TV+Apple Watch Series 11
Company profileWebsite

More open roles at Apple