Software Engineer, Distributed Training
Palo Alto, CA$200k - $420kPosted 1mo ago
About the role
Role focused on building and optimizing distributed training engines for River AI, enabling fast, numerically correct, and reliable fine-tuning and reinforcement learning across large GPU clusters.
Responsibilities
- Build and optimize distributed training for large dense and mixture-of-experts models, including low-rank adapter training.
- Improve reinforcement-learning pipelines by coordinating sampling, reward computation, training updates, and weight transfer.
- Optimize GPU memory use, parallelism, and communication to increase training throughput.
- Implement reliable checkpointing, resumption, and worker recovery while preserving consistent training state.
- Validate losses, gradients, and optimizer behavior, and diagnose numerical or distributed execution failures.
- Partner with researchers to implement new algorithms and make them accessible through the River API.
Qualifications
- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical industry experience.
- Hands-on experience building or substantially improving distributed model-training systems.
- Strong proficiency in Python and a modern deep-learning framework, such as PyTorch or JAX.
- Solid understanding of backpropagation, optimizers, mixed-precision training, and GPU memory management.
- Strong debugging skills across concurrent execution, collective communication, and distributed failure recovery.
- A highly collaborative mindset and a bias for action to push boundaries across the stack.
Skills
Required:PythonPyTorch or JAXDistributed model-training systemsBackpropagation, optimizers, mixed-precision, GPU memory managementDebugging concurrent execution and distributed failures
Benefits
- Comprehensive health, dental, and vision insurance; unlimited PTO; relocation assistance as needed.
Last updated Sep 23, 2026. Interested? Apply on the company site.
About River AI
Intelligence that flows with you
River is building a new stack for Personal AI.
LoRA fine-tuningReinforcement learningOpen modelsPay per tokenAPIPersonal AI
More open roles at River AI
- IT Lead2w agoPalo Alto, CA · Staff
- Design Verification Emulation, Hardware3w agoHybrid (Austin, TX or Palo Alto, CA) · Personal AI Hardware
- People Lead4w agoPalo Alto, CA · Staff
- Operations Lead4w agoPalo Alto, CA · Staff
- Software Engineer, GPU Kernels1mo agoPalo Alto, CA · River API