← All roles

Staff Software Engineer, Inference Infrastructure

Cohere· Inference
Apply on company site
Hybrid (San Francisco), Hybrid (New York), Hybrid (Toronto), Hybrid (Montreal)$190k - $275k + equityFull-timePosted 8mo ago

About the role

Staff Software Engineer responsible for building and operating the AI platform that delivers Cohere's large language models through API endpoints, focusing on high-performance, scalable, low-latency, high-throughput inference infrastructure.

Responsibilities

  • Develop, deploy, and operate the AI platform delivering Cohere's large language models through easy to use API endpoints
  • Collaborate with multiple teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments
  • Interface with customers and create customized deployments to meet their specific needs
  • Solve complex technical challenges in evolving production infrastructure and ensure smooth operations and efficient teamwork

Qualifications

  • 5+ years of engineering experience running production infrastructure at a large scale
  • Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
  • Experience with Kubernetes dev and production coding and support
  • Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
  • Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
  • Experience in compute/storage/network resource and cost management
  • Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators) and their impact on latency and throughput of inference
  • Strong understanding or working experience with distributed systems
  • Experience in Golang, C++ or other languages designed for high-performance scalable servers

Skills

Required:KubernetesGPU/accelerator computeDistributed systemsGolang / C++

Benefits

  • Weekly lunch stipend of $75/£75 or equivalent
  • Full health and dental benefits, including mental health budget
  • RRSP matching, 401K, Pension Scheme
  • 100% Parental Leave top-up for up to 6 months
  • Annual enrichment benefits (arts & culture, fitness/wellness, workspace improvement, education & learning stipend)
  • 6 weeks of paid vacation (30 working days)
  • Travel budget to other offices if remote, plus annual company offsite
  • $500 home office stipend
Last updated Oct 21, 2026. Interested? Apply on the company site.
Cohere logo

About Cohere

Support automation that's actually intelligent

Cohere resolves 40-70% of support requests using AI, over all channels, within just a few days.

artificial-intelligenceautomationliquid-ai-recruiting-target
Company profileWebsite

More open roles at Cohere