Member of Technical Staff - Distributed Training Engineer
Hybrid (San Francisco)Full-timePosted 11mo ago
About the role
High-ownership training systems role focused on runtime/performance/reliability within Liquid AI's Training Infrastructure team, building distributed systems for large-scale training and optimizing infrastructure from the ground up.
Responsibilities
- Design and build core systems that make large training runs fast and reliable
- Build scalable distributed training infrastructure for GPU clusters
- Implement and tune parallelism/sharding strategies for evolving architectures
- Optimize distributed efficiency (topology-aware collectives, comm/compute overlap, straggler mitigation)
- Build data loading systems that eliminate I/O bottlenecks for multimodal datasets
- Develop checkpointing mechanisms balancing memory constraints with recovery needs
- Create monitoring, profiling, and debugging tools for training stability and performance
Qualifications
- Hands-on experience building distributed training infrastructure (PyTorch Distributed DDP/FSDP, DeepSpeed ZeRO, Megatron-LM TP/PP)
- Experience diagnosing performance bottlenecks and failure modes (profiling, NCCL/collectives issues, hangs, OOMs, stragglers)
- Understanding of hardware accelerators and networking topologies
- Experience optimizing data pipelines for ML workloads
Skills
Required:Distributed training infrastructurePerformance bottlenecks & failure diagnosisHardware accelerators & networkingData pipeline optimization
Benefits
- Greenfield challenges: Build systems from scratch for novel architectures. High ownership from day one.
- Compensation: Competitive base salary with equity in a unicorn-stage company
- Health: We pay 100% of medical, dental, and vision premiums for employees and dependents
- Financial: 401(k) matching up to 4% of base pay
- Time Off: Unlimited PTO plus company-wide Refill Days throughout the year
Last updated Aug 19, 2026. Interested? Apply on the company site.
About Liquid AI
Build efficient general-purpose AI at every scale.
We build efficient general-purpose AI at every scale. Liquid Foundation Models (LFMs) are a new generation of generative AI models that achieve state-of-the-art performance at every scale, while maintaining a smaller memory footprint and more efficient inference.
AIEdge AIFoundation ModelsGenerative AIliquid-ai-recruiting-target
More open roles at Liquid AI
- Member of Technical Staff - Partner Inference Engineer4d agoBoston, San Francisco · GTM
- Developer Relations1w agoHybrid (San Francisco) · Marketing
- Strategic Business Development, Life Sciences1w agoHybrid (Boston) · Business
- Controller1w agoHybrid (San Francisco) · G&A
- Member of Go-to-Market Staff2w agoHybrid (North America l San Francisco, Boston or Remote) · GTM