← All roles

Staff Site Reliability Engineer

Hippocratic AI· R&D
Apply on company site
Menlo Park, CAFull-timePosted 4w ago

About the role

Senior SRE responsible for building and operating a GPU management and scheduling platform to efficiently manage a fleet of ~30 models on heterogeneous hardware, with end-to-end ownership from systems engineering to software development, focusing on admission control, autoscaling, and reliable production infrastructure for a healthcare AI platform.

Responsibilities

  • Design and build our GPU management and scheduling platform — the system that decides when, where, and how inference calls run across a fleet of ~30 models on heterogeneous hardware
  • Build the metrics pipeline that collects GPU load and utilization data, and the logic that turns those signals into decisions
  • Implement admission control to protect capacity — deciding when to accept, queue, or shed inference requests so we operate within fleet limits
  • Build autoscaling that adjusts the number of model replicas in response to real-time demand and utilization
  • Develop cloud orchestration systems and operators in Python and Go to manage the model fleet
  • Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure
  • Design and build infrastructure automation and deployment pipelines (Terraform, CI/CD) as first-class software
  • Stand up and maintain monitoring, logging, and alerting that keep the platform reliable and performant
  • Develop and enforce security and compliance policies appropriate to a healthcare AI platform
  • Partner with engineers and research scientists to diagnose and resolve complex infrastructure, deployment, and operational issues
  • Mentor engineers and raise the technical bar across the team

Qualifications

  • 10+ years of professional experience across site reliability / DevOps engineering and software engineering
  • Computer Science Degree Required from a top CS program
  • Strong software engineering fundamentals — you build orchestration and scheduling systems in Python and/or Go, not just configure off-the-shelf tools
  • Experience designing systems that make decisions from operational metrics — collecting signals, interpreting them, and driving control loops such as autoscaling, load shedding, or admission control
  • Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD, or similar)
  • Hands-on production experience with at least one major cloud platform (AWS, GCP, or Azure)
  • Strong knowledge of containerization and orchestration (Docker, Kubernetes)
  • Experience with monitoring and logging stacks (ELK, Grafana, Datadog, or similar)
  • Familiarity with secrets management and security tooling (HashiCorp Vault, AWS KMS, Azure Key Vault)
  • Excellent problem-solving skills and the ability to work both independently and collaboratively
  • Strong communication and interpersonal skills

Skills

Required:PythonGoTerraformCI/CD tooling (GitLab CI/CD or similar)AWS / GCP / AzureDocker / KubernetesMonitoring/logging stacks (ELK, Grafana, Datadog)

Benefits

  • Note: Benefits not explicitly listed in the description; omitted to avoid filler.
Last updated Sep 30, 2026. Interested? Apply on the company site.

About Hippocratic AI

Safest Healthcare Generative AI Agents

Safest Healthcare Generative AI Agents Our AI agents do not diagnose or prescribe. Learn more. Listen to these 4 real de-identified calls from the over 115 million clinical interactions.

HealthcareAIGenerative AIClinical InteractionsPatient Care
Company profileWebsite

More open roles at Hippocratic AI