← All roles

Senior Software Engineer - AI Compute, Together Cloud

Together AI· Engineering
Apply on company site
Amsterdamfull-timePosted 8mo ago

About the role

Senior Software Engineer focusing on AI Compute in the Together Cloud org. Build and own major components of a next-generation AI cloud platform including GPU virtualization, in-DC IaaS, and global management for on-demand/reserved Kubernetes/Slurm clusters across data centers and thousands of GPUs.

Responsibilities

  • Build the GPU and network virtualization stack: hypervisor, kernel, and SDN for high-performance, portable, and isolated GPU compute across heterogeneous hardware.
  • Build and maintain our in-DC IaaS layer: services, Kubernetes operators, and libraries provisioning compute, storage, and networks in data centers (VMs, parallel filesystems, VPCs, InfiniBand partitions). Harden bring-up path for Vera Rubin data center with thousands of GPUs.
  • Scale the distributed GPU scheduling and global management plane: control-plane services for on-demand and reserved clusters; automate onboarding of new capacity and per-cluster limit management.
  • Harden the monitoring and automated remediation layer for fault tolerance: automated detection, isolation, and recovery of failed nodes for fault-tolerant pretraining and large-scale inference.
  • Own components end-to-end: write design docs, break work into milestones, ship incrementally, and improve production reliability.
  • Raise the bar: code reviews, design feedback, mentoring junior engineers.
  • Build tooling for other teams: testing frameworks, developer tools, and documentation; contribute to core open-source Together AI platform.

Qualifications

  • 5+ years of professional software development experience, proficient in at least one backend language (Golang preferred).
  • Demonstrated ownership of large-scale projects end-to-end, producing high-performance, well-tested, production-quality code.
  • Experience building and operating high-performance and/or globally distributed microservice architectures across one or more cloud providers (AWS, Azure, GCP).
  • Strong systems knowledge across compute, networking, and storage, including concurrency, memory management, performant I/O, and scale.
  • Excellent communication skills with ability to write clear design docs and collaborate with technical and non-technical teammates.
  • Experience building and operating reliable customer-facing production systems at scale; owning infrastructure automation (Terraform, Ansible), observability (Prometheus, Grafana), and CI/CD (GitHub Actions, ArgoCD).

Skills

Required:Backend programmingDistributed systemsCloud platformsSoftware architecture and designCommunication

Benefits

  • (Not specified in description)
Last updated Sep 30, 2026. Interested? Apply on the company site.

About Together AI

The AI Acceleration Cloud - Fast Inference, Fine-Tuning & Training

Run and fine-tune generative AI models with simple APIs and scalable GPU clusters. Train & deploy at scale on The AI Acceleration Cloud.

AI native cloudFine-TuningGPU clustersInferenceTrainingai-infrastructure-and-developmentbrex-fastest-growing-2025liquid-ai-recruiting-target
Company profileWebsite

More open roles at Together AI