← All roles

Senior Software Engineer, GPU Infrastructure (HPC)

Cohere· Modeling
Apply on company site
Hybrid (Canada), Hybrid (United States)$200k - $340k + equityFull-timePosted 8mo ago

About the role

Senior Software Engineer focused on building and scaling ML-optimized HPC infrastructure, including Kubernetes-based GPU/TPU superclusters, across multiple clouds, to support AI model training and research.

Responsibilities

  • Build and scale ML-optimized HPC infrastructure: Deploy and manage Kubernetes-based GPU/TPU superclusters across multiple clouds, ensuring high throughput and low-latency performance for AI workloads.
  • Optimize for AI/ML training: Collaborate with cloud providers to fine-tune infrastructure for cost efficiency, reliability, and performance, leveraging technologies like RDMA, NCCL, and high-speed interconnects.
  • Troubleshoot and resolve complex issues: Proactively identify and resolve infrastructure bottlenecks, performance degradation, and system failures to ensure minimal disruption to AI/ML workflows.
  • Enable researchers with self-service tools: Design intuitive interfaces and workflows that allow researchers to monitor, debug, and optimize their training jobs independently.
  • Drive innovation in ML infrastructure: Work closely with AI researchers to understand emerging needs (e.g., JAX, PyTorch, distributed training) and translate them into robust, scalable infrastructure solutions.
  • Champion best practices: Advocate for observability, automation, and infrastructure-as-code (IaC) across the organization, ensuring systems are maintainable and resilient.
  • Mentorship and collaboration: Share expertise through code reviews, documentation, and cross-team collaboration, fostering a culture of knowledge transfer and engineering excellence.

Qualifications

  • Deep expertise in ML/HPC infrastructure: Experience with GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow), and HPC environments.
  • Kubernetes at scale: Proven ability to deploy, manage, and troubleshoot cloud-native Kubernetes clusters for AI workloads.
  • Strong programming skills: Proficiency in Python (ML tooling) and Go (systems engineering), with a preference for open-source contributions.
  • Low-level systems knowledge: Familiarity with Linux internals, RDMA networking, and performance optimization for ML workloads.
  • Research collaboration experience: A track record of working closely with AI researchers or ML engineers to solve infrastructure challenges.
  • Self-directed problem-solving: Ability to identify bottlenecks, propose solutions, and drive impact in a fast-paced environment.

Skills

Required:GPU/TPU clustersKubernetesPythonGoLinux internals

Benefits

  • A weekly lunch stipend of $75/£75 or equivalent in local currency
  • Full health and dental benefits, including a separate budget for mental health
  • RRSP matching, 401K, Pension Scheme
  • 100% Parental Leave top-up for up to 6 months, for either parent
  • Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, workspace improvement credit
  • Education & learning stipend for conferences, courses, and coaching
  • 6 weeks of paid vacation (30 working days)
  • Budget for traveling to other offices if remote, plus annual company offsite
  • Co-working benefit and $500 home office stipend
Last updated Oct 21, 2026. Interested? Apply on the company site.
Cohere logo

About Cohere

Support automation that's actually intelligent

Cohere resolves 40-70% of support requests using AI, over all channels, within just a few days.

artificial-intelligenceautomationliquid-ai-recruiting-target
Company profileWebsite

More open roles at Cohere