← All roles

Staff Software Engineer - AI Compute, Together Cloud

Together AI· Engineering
Apply on company site
Amsterdamfull-timePosted 4d ago

About the role

Staff-level role focused on AI Compute within the Together Cloud organization to architect and build the next-generation AI cloud platform, including GPU virtualization, in-DC IaaS, and global management planes for on-demand and reserved GPU clusters across multiple data centers.

Responsibilities

  • Own the GPU and network virtualization stack: hypervisor, kernel, and SDN for high-performance, portable, isolated GPU compute and DC networking across heterogeneous hardware.
  • Own the in-DC IaaS layer: architect and roadmap services, Kubernetes operators, and libraries provisioning compute, storage, and networks in data centers (VMs, parallel filesystems, VPCs, InfiniBand partitions); lead build-out for Vera Rubin data center.
  • Design the GPU scheduling and global management plane: distributed control plane for on-demand and reserved clusters across data centers, including per-cluster limits scaling and onboarding automation.
  • Architect monitoring and automated remediation for fault tolerance: automated detection, isolation, and recovery of failed nodes to keep distributed training and inference running.
  • Set technical direction across teams: lead design reviews, resolve cross-cutting architectural disagreements, unblock dependencies, and define standards.
  • Grow the team: mentor engineers and raise hiring bar for Together Cloud.
  • Set the engineering bar: create testing frameworks, tools, and documentation to support robust, user-friendly systems.

Qualifications

  • 7+ years of professional software development experience, expert-level proficiency in at least one backend language (Golang preferred), with high-performance, well-tested production-quality code.
  • Track record of owning the architecture of large distributed systems from blank page to production at scale.
  • Deep experience building and operating globally distributed, high-performance microservice architectures across one or more cloud providers (AWS, Azure, GCP).
  • Expert systems knowledge across compute, networking, and storage, including concurrency, memory management, I/O performance, and global-scale considerations.
  • Demonstrated technical leadership beyond individual contributions: mentoring, design reviews, cross-team alignment.
  • Excellent communication and diplomacy skills, able to write design docs that settle arguments and work with both technical and non-technical stakeholders.
  • Experience building and operating reliable production systems at scale with infrastructure automation (Terraform, Ansible), observability (Prometheus, Grafana), and CI/CD (GitHub Actions, ArgoCD).

Skills

Required:Backend developmentDistributed systems architectureCloud platformsNetworking and GPU virtualization
Nice to have:Kubernetes

Benefits

  • Not specified in the description
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Together AI

The AI Acceleration Cloud - Fast Inference, Fine-Tuning & Training

Run and fine-tune generative AI models with simple APIs and scalable GPU clusters. Train & deploy at scale on The AI Acceleration Cloud.

AI native cloudFine-TuningGPU clustersInferenceTrainingai-infrastructure-and-developmentbrex-fastest-growing-2025liquid-ai-recruiting-target
Company profileWebsite

More open roles at Together AI