← All roles

Senior AI R&D Engineer - Performance - Pre-training (f/m/d)

Aleph Alpha· Research
Apply on company site
Hybrid (Heidelberg)Full-timePosted 7mo ago

About the role

Senior Performance Engineer specializing in pre-training efficiency for foundation models, ensuring maximum hardware utilization and training throughput on large GPU clusters (thousands of NVIDIA Blackwell GPUs) at Aleph Alpha.

Responsibilities

  • End-to-End Optimization: Profile training loops using PyTorch Profiler, Nsight Systems and Nsight Compute to identify system- and kernel-level bottlenecks in order to maximize model throughput.
  • Distributed Strategy and Topology: Configure and tune composite parallelism strategies (e.g. TP, DP, HSDP/FSDP, EP), optimizing load balance, minimizing critical-path bottlenecks, and managing communication-to-computation trade-offs for large-scale LLM training.
  • Hardware-Aware Modeling: Partner with AI Researchers to define model architectures for hardware efficiency without compromising convergence.

Qualifications

  • Proficient in Python and the PyTorch library.
  • Strong engineering background in parallel and/or distributed systems with proven track record of excellence.
  • Hands-on experience with modern machine learning techniques (especially large language models and their life cycle).
  • Deeply understand the CUDA programming model.
  • Experience in distributed programming with APIs like NCCL or MPI.
  • Experience analysing profiling traces with tools such as PyTorch Profiler and Nvidia Nsight.
  • Regular on-site collaboration in Heidelberg as a member of the Training Efficiency Team.

Skills

Required:PythonPyTorchCUDANCCL / MPIProfiling tools (PyTorch Profiler, Nsight)Large-scale distributed training

Benefits

  • 30 days of paid vacation
  • Access to Wellhub fitness & wellness offerings
  • Mental health support through nilo.health
  • Substantially subsidized company pension plan
  • Subsidized Germany-wide transportation ticket
  • Budget for additional technical equipment
  • Flexible working hours for better work-life balance and hybrid working model
  • JobRad® Bike Lease
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Aleph Alpha

Sovereign AI Solutions for Enterprises and Governments by Aleph Alpha

Aleph Alpha empowers enterprises and governments with sovereign AI solutions for complex and critical processes. Secure your sovereignty, protect your data, and shape the future of AI-driven knowledge work.

AI TechnologyEnterprisesGovernmentsResearchSovereign AIliquid-ai-recruiting-target
Company profileWebsite

More open roles at Aleph Alpha