← All roles

Research Engineer, Synthetic Data & Evals

Fleet
Apply on company site
San Francisco, New YorkFull-timePosted 4mo ago

About the role

Role focused on building reusable systems for synthetic data, evaluation sets, verifiers, and quality measurement. Develop new methods, turn successful approaches into scalable systems, and iterate based on research findings to improve research speed, reproducibility, and domain scalability. On-site in San Francisco or New York.

Responsibilities

  • Build reusable and reliable systems for producing high-quality synthetic data, tasks, and verifiers across multiple domains.
  • Leverage research signals and expert knowledge to define and enforce quality standards for each eval set.
  • Build agentic hill-climbing and feedback loops to continuously improve systems and outputs.
  • Collaborate with environment and research teams, frontier labs, and Fleet’s expert network to translate research needs into system capabilities.
  • Treat systems and outputs as products: own quality and usability end to end, including researcher education, adoption, and continuous iteration.

Qualifications

  • Strong engineering fundamentals and experience building reliable systems. Proficient in Python or another modern backend language; comfortable with containers, distributed systems, and GPUs when needed.
  • Strong ML fundamentals with understanding of how training objectives and optimization shape model behavior.
  • Ability to convert a research prototype into a maintainable system with clear interfaces, reliable execution, and useful debugging tools.
  • Strong judgment about data and system quality; able to define what “good” looks like and diagnose failures.
  • Ability to work effectively with internal research teams, domain experts, and frontier lab partners.
  • High ownership, low ego, and preference for simple, boring systems.

Skills

Required:Engineering fundamentalsPython or backend languageDistributed systemsML fundamentals
Last updated Oct 8, 2026. Interested? Apply on the company site.

About Fleet

Fleet - AI-powered workflows

Fleet creates simulated worlds and real-world challenges to better understand and shape the behavior of artificial intelligences.

AI-powered workflowsbessemer-ai-infrastructure-2026reinforcement-learning-platforms
Company profileWebsite

More open roles at Fleet