← All roles

Senior AI Researcher- Pre-training (f/m/d)

Aleph Alpha· Research
Apply on company site
Hybrid (Heidelberg)Full-timePosted 6mo ago

About the role

Senior AI Researcher focused on pre-training next-generation foundation models, optimizing architecture, training recipes, and scaling on large GPU clusters. Hybrid role based in Heidelberg, Germany.

Responsibilities

  • Recipe & Architecture Optimization: Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.
  • Scaling Strategy & Predictability: Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions.
  • Stability, Diagnostics & Debugging: Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors.
  • System-Model Co-Design: Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.

Qualifications

  • You are proficient in Python and deeply familiar with PyTorch-based training workflows.
  • You have a strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.
  • You have a strong mathematical foundation and are comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics.
  • You deeply understand transformer training dynamics, optimisation, and the behaviour of large distributed training jobs.
  • You can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.
  • Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters).
  • You apply strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.
  • You are able to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems.
  • You collaborate effectively within a research and engineering team and communicate clearly about your work across Pre-training and the broader AAR/AA organization.
  • You are able to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team.

Skills

Required:PythonPyTorchMachine Learning ResearchDistributed TrainingExperiment Design

Benefits

  • 30 days of paid vacation
  • Access to fitness & wellness offerings via Wellhub
  • Mental health support through nilo.health
  • Substantially subsidized company pension plan
  • Subsidized Germany-wide transportation ticket
  • Budget for additional technical equipment
  • Flexible working hours for better work-life balance and hybrid working model
  • JobRad® Bike Lease
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Aleph Alpha

Sovereign AI Solutions for Enterprises and Governments by Aleph Alpha

Aleph Alpha empowers enterprises and governments with sovereign AI solutions for complex and critical processes. Secure your sovereignty, protect your data, and shape the future of AI-driven knowledge work.

AI TechnologyEnterprisesGovernmentsResearchSovereign AIliquid-ai-recruiting-target
Company profileWebsite

More open roles at Aleph Alpha