← All roles

Senior ML Systems Engineer, Frameworks & Tooling

Cohere· Modeling
Apply on company site
Remote$175k - $535k + equityFull-timePosted 10mo ago

About the role

Senior engineer to build, maintain and evolve the training framework for frontier-scale language models, interfacing large-scale training, distributed systems, and HPC infrastructure. Responsible for core components enabling fast, reliable, scalable model training and tooling that connects research ideas to thousands of GPUs.

Responsibilities

  • Build and own the training framework responsible for large-scale LLM training.
  • Design distributed training abstractions (data/tensor/pipeline parallelism, FSDP/ZeRO strategies, memory management, checkpointing).
  • Improve training throughput and stability on multi-node clusters (e.g., GB200/300, AMD, H200/100).
  • Develop and maintain tooling for monitoring, logging, debugging, and developer ergonomics.
  • Collaborate closely with infra teams to ensure our cluster, container environments, and hardware configurations support high-performance training.
  • Investigate and resolve performance bottlenecks across the ML systems stack.
  • Build robust systems that ensure reproducible, debuggable, large-scale runs.

Qualifications

  • Strong engineering experience in large-scale distributed training or HPC systems.
  • Deep familiarity with JAX internals, distributed training libraries, or custom kernels/fused ops.
  • Experience with multi-node cluster orchestration (Slurm, Ray, Kubernetes, or similar).
  • Comfort debugging performance issues across CUDA/NCCL, networking, IO, and data pipelines.
  • Experience working with containerized environments (Docker, Singularity/Apptainer).
  • A track record of building tools that increase developer velocity for ML teams.
  • Excellent judgment around trade-offs: performance vs complexity, research velocity vs maintainability.
  • Strong collaboration skills — you’ll work closely with infra, research, and deployment teams.

Skills

Required:Large-scale distributed trainingJAX internalsMulti-node cluster orchestrationCUDA/NCCL debuggingContainerized environments

Benefits

  • A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch.
  • Full health and dental benefits, including a separate budget for mental health.
  • RRSP matching, 401K, Pension Scheme.
  • 100% Parental Leave top-up for up to 6 months, for either parent.
  • Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit; Education & learning stipend for conferences, courses, and coaching.
  • 6 weeks of paid vacation (30 working days).
  • Budget for traveling to other offices if you are remote, plus an annual company offsite.
  • $500 home office stipend to set up your workspace properly.
Last updated Oct 21, 2026. Interested? Apply on the company site.
Cohere logo

About Cohere

Support automation that's actually intelligent

Cohere resolves 40-70% of support requests using AI, over all channels, within just a few days.

artificial-intelligenceautomationliquid-ai-recruiting-target
Company profileWebsite

More open roles at Cohere