← All roles

Senior or Staff Software Engineer, Inference / Compute Infrastructure Engineering

Together AI· Engineering
Apply on company site
Hybrid (Amsterdam)full-timePosted 2mo ago

About the role

Senior/Staff software engineer to build manifest-driven infrastructure for turning racks of GPUs into running inference clusters. Owns provisioning state machines, self-service APIs, and automated lifecycle management for production-grade AI infrastructure, with a product-focused mindset and CI/CD automation.

Responsibilities

  • Build the provisioning state machine: model the full lifecycle of a physical host from discovery to inference bring-up, GPU stack, health validation, and decommission/RMA as explicit states and transitions.
  • Build the self-service API: design declarative APIs and a control plane to allow the inference team to request, scale, and tear down clusters with one API call.
  • Automate self-healing: detect degraded/failed nodes, drain safely, trigger repair or replacement, and reintroduce healthy capacity automatically.
  • Own reliability of the pipeline: ensure idempotency, retries, rollback, and drift detection for a dependable provisioning system.
  • Partner with the inference/ML platform team: encode cluster shapes (topology, interconnect, scheduling constraints) as abstractions in the platform.
  • Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code.

Qualifications

  • Strong software engineering background in Go, Python, Rust, or similar.
  • Experience with durable workflow orchestration tools (Temporal, Cadence, or equivalent).
  • Experience building software control planes or orchestration systems that model state and reconcile it over time.
  • Experience with event-driven systems (message queues, event streams, pub/sub such as Kafka, NATS, SQS).
  • Product mindset with experience building internal platforms/APIs for other engineering teams.

Skills

Required:Go, Python, Rust (or similar)Durable workflow orchestrationControl plane / orchestration systemsEvent-driven architecturesProduct mindset / internal platform API
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Together AI

The AI Acceleration Cloud - Fast Inference, Fine-Tuning & Training

Run and fine-tune generative AI models with simple APIs and scalable GPU clusters. Train & deploy at scale on The AI Acceleration Cloud.

AI native cloudFine-TuningGPU clustersInferenceTrainingai-infrastructure-and-developmentbrex-fastest-growing-2025liquid-ai-recruiting-target
Company profileWebsite

More open roles at Together AI