← All roles

Member of Technical Staff - Training Platform

Prime Intellect· Engineering
Apply on company site
Remote$150k - $300kFull-timePosted 5mo ago

About the role

Prime Intellect is building an open frontier AI platform and is seeking a Member of Technical Staff to help build a hosted training platform and underlying Kubernetes-based training infrastructure for multi-cluster, multi-cloud GPU fleets, enabling LoRA and full fine-tuning runs with a streamlined API and UI.

Responsibilities

  • Design and operate Kubernetes-based training and inference orchestration across multi-cluster, multi-cloud GPU fleets
  • Build and maintain Helm charts that compose trainers, inference servers, environment servers, and supporting services into reproducible "Training stacks"
  • Develop the Python control-plane agents that watch pods, report run state to the platform, and keep clusters in sync
  • Implement scheduling and autoscaling for heterogeneous hardware (H100/H200/B200) using KEDA, LeaderWorkerSet, taints/tolerations, and gang scheduling
  • Run a tight GitOps workflow - every change ships through PRs, Helm values, and CI
  • Build node-local model caches, checkpoint pipelines, and shared storage for fast cold starts
  • Operate the observability stack (Prometheus, Grafana, Loki, DCGM) and make GPU cluster debugging fast
  • Build the developer-facing surfaces for hosted training: job submission, live run monitoring, logs, metrics, model/adapter management, comparisons
  • Develop FastAPI backend services and REST APIs that bridge the platform to running clusters
  • Build real-time monitoring and debugging tools (streaming logs, step-level metrics, failure analysis)
  • Ship product UI in Next.js / React / TypeScript with shadcn, Tailwind, tRPC, and TanStack Query
  • Interface with the RL trainer, inference servers, and environment servers running inside our clusters
  • Productize new training capabilities (new model architectures, RL algorithms, modes)

Qualifications

  • Strong working knowledge of the modern AI stack - open model families, finetuning techniques (LoRA, QLoRA, full FT, RLHF/RLAIF), inference engines (vLLM, SGLang, TensorRT-LLM)
  • Familiarity with GPU hardware tradeoffs (H100 / H200 / B200, NVLink, interconnects, memory hierarchy) and what they mean for training and inference workloads
  • Understanding of distributed training fundamentals (data/tensor/pipeline/expert parallelism, NCCL, multi-node scheduling)
  • Awareness of frontier trends and ability to translate into product decisions
  • Strong Kubernetes operations experience - Helm, CRDs, operators, KEDA, gang scheduling, GPU operator
  • Debugging production clusters (kubectl, pod lifecycle, node issues, networking)
  • Cloud platform experience (GCP preferred - GCS, GKE, Cloud Run, Cloud Tasks)
  • Infrastructure automation (Helm, Terraform, Ansible) and GitOps mindset
  • Observability: Prometheus, Grafana, Loki, OpenTelemetry, DCGM
  • Linux fundamentals: networking, namespaces, performance tuning
  • Strong Python backend development (FastAPI, async, SQLAlchemy)
  • Experience building Python control-plane agents that talk to Kubernetes APIs
  • Modern frontend development (TypeScript, React/Next.js, Tailwind, shadcn) - end-to-end product surfaces
  • REST and tRPC API design
  • Experience building developer tools, dashboards, and live-monitoring UIs

Skills

Required:Kubernetes administrationPython backend development
Nice to have:Frontend development

Benefits

  • Cash compensation $150K–$300K with significant equity
  • Flexible work arrangement (remote or San Francisco office)
  • Full visa sponsorship and relocation support
  • Professional development budget for courses and conferences
  • Regular team off-sites and conference attendance
  • Opportunity to shape the future of decentralized AI development
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Prime Intellect

Commoditizing Compute & Intelligence

Prime Intellect democratizes AI development at scale. Our platform makes it easy to find global compute resources and train state-of-the-art models through distributed training across clusters. Collectively own the resulting open AI innovations, from language models to scientific breakthroughs.

AI developmentbessemer-ai-infrastructure-2026compute resourcesdistributed trainingmodelsreinforcement-learning-platformsscience
Company profileWebsite

More open roles at Prime Intellect