← All roles

Senior Software Engineer, AI Evaluation Infrastructure

Training Superintelligence· Frontier Data Operations
Apply on company site
United Statesfull-timePosted 5d ago

About the role

Role focused on building and maintaining the infrastructure used to evaluate coding agents and language models, including data models, APIs, background jobs, and UI components. Requires independent, impact-driven engineering across the stack with emphasis on scalable, reliable systems and LLM evaluation knowledge.

Responsibilities

  • Design and ship features end to end: data model, API, background workers, and the UI when the change needs it.
  • Build and improve how agents and models are evaluated: running trials in sandboxed environments, scoring results, and turning thousands of runs into results people can trust.
  • Improve how benchmark tasks are quality-checked: verifiers, rubric-based and LLM-as-a-judge grading, and deciding whether a task is clear, solvable, and fair to score.
  • Integrate new models, agents, and benchmarks in a way that generalizes, without a special case for each one.
  • Keep the system fast and reliable as usage grows: efficient queries, safe migrations, and concurrency that holds up under load.
  • Review teammates’ pull requests against our engineering guidelines and help keep the codebase consistent.

Qualifications

  • Typically 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
  • Strong Python. Experience with Django (or a similar framework), PostgreSQL, and a task queue such as Celery. What matters most is having run a real system, not knowing a particular framework.
  • Hands-on experience in LLM evaluation, beyond calling a model API. You understand how a benchmark task is built, how an agent trial is scored, and how results are aggregated and compared. Experience with Harbor, Terminal-Bench, SWE-bench, Inspect, or a similar harness is a strong plus.
  • Judgment about task quality. You can spot ambiguous instructions, weak or gameable verifiers, leaked answers, and checks that pass for the wrong reason.
  • Good tradeoffs. You reach for the smallest design that solves the problem, extend what already exists before adding something new, and can explain what you chose not to build.
  • Ownership. You take a fuzzy request, find the relevant code, ask questions only when the answer changes the design, and finish the job, including migrations, tests, and edge cases users will hit.
  • Quick adoption of team conventions. You read the written guidelines and your first pull requests already match them in naming, structure, tests, and error handling.
  • Comfort with a TypeScript and React frontend when a change is visible there. You don’t need to be a frontend specialist.
  • Clear writing. Your pull request descriptions and design notes explain why, not just what.

Skills

Required:PythonDjango or similar frameworkPostgreSQLCelery or equivalentLLM evaluation
Nice to have:Frontend basics (TypeScript/React)

Benefits

  • We are client first: We put our clients at the center of everything we do, because their success is the ultimate measure of our value.
  • We work at Start-Up Speed: We move fast, stay agile and favor action because momentum is the foundation of perfection
  • We are AI forward: We help our clients build the future of AI and implement it in our own roles and workflow to amplify productivity.
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Training Superintelligence

Turing develops large-scale RL environments and data generation systems that train multimodal agents to improve model performance in coding, real-world, economically valuable tasks, and advanced STEM reasoning.

Train AIBuild AIHire AI TalentAI for BFSIEMNAIDatasetsEnterprise Knowledge WorkMultimodal
Company profileWebsite

More open roles at Training Superintelligence