← All roles

Reliability Engineer, Supercomputing

Thinking Machines Lab· Core Engineering, Product, and Infrastructure
Apply on company site
Hybrid (San Francisco, CA)$350k - $475kfull-timePosted 3mo ago

About the role

Engineer to ensure reliability of GPU supercomputing fleet, bridging hardware, firmware, and OS; diagnose root causes of hardware/software issues across large GPU clusters and coordinate fixes with vendors.

Responsibilities

  • Investigate, reproduce, and remediate issues across large GPU clusters.
  • Own the drivers, kernel surface, and diagnostics that span hardware, firmware, and OS.
  • Automate the monitoring of fleet reliability and analyze error rates to validate fixes or firmware changes.
  • Drive the firmware lifecycle: tracking, qualification, staged rollout, and regression analysis.
  • Engage vendors directly with GPUs, server OEMs, NIC vendors, and storage vendors; manage RMA flows when hardware is replaced.
  • Monitor and improve GPU hardware health signals and translate into actionable reliability improvements.
  • Write clear postmortems and vendor cases that move issues forward.

Qualifications

  • Bachelor’s degree or equivalent experience in computer science, engineering, or similar.
  • Proficiency in at least one backend language (Python or Rust).
  • Experience operating large-scale clusters and container orchestration systems (e.g. Kubernetes or Slurm).
  • Comfort operating across the stack and owning projects end-to-end.
  • Thrive in a highly collaborative environment with cross-functional partners.
  • Bias for action with initiative to work across stacks and teams to ship.

Skills

Required:Backend programmingCluster operations
Nice to have:Linux systems

Benefits

  • generous health, dental, and vision benefits
  • unlimited PTO
  • paid parental leave
  • relocation support as needed
Last updated Jul 8, 2026. Interested? Apply on the company site.
Thinking Machines Lab logo

About Thinking Machines Lab

Connectionism: Research Blog by Thinking Machines Lab

Thinking Machines Lab is an artificial intelligence research and product company. We’re building a future where everyone has access to the knowledge and tools to make AI work for their unique needs and goals.

AIartificial intelligencebessemer-ai-infrastructure-2026deep learningliquid-ai-recruiting-targetmachine learningproductreinforcement-learning-platforms
Company profileWebsite

More open roles at Thinking Machines Lab