← All roles

Evaluations Team Lead

Fundamental· Engineering
Apply on company site
RemoteFull-timePosted 1w ago

About the role

Lead the evaluation platform for NEXUS, ensuring trustworthy, reproducible benchmarks and datasets; hire and manage a small team while remaining hands-on with code, experiment design, and methodology.

Responsibilities

  • Build a shared evaluation platform for engineering, research, and Applied AI. Continuously add and maintain models and curated datasets so teams can run benchmarks and investigate results independently.
  • Define evaluation standards for metrics, data splits, leakage prevention, calibration, uncertainty, and benchmark contamination.
  • Build reproducible pipelines with versioned inputs and artifacts, and integrate regression checks into research and release workflows.
  • Benchmark NEXUS against competing approaches using fair tuning budgets, data access, compute, and latency measurement protocols.
  • Support Applied AI’s customer POC evaluations with tooling, methodological guidance, and analysis.
  • Measure predictive quality, latency, and cost across deployment configurations, task types, and dataset characteristics.
  • Turn findings into research priorities, release recommendations, and evidence-backed customer improvement plans.
  • Hire and develop the team, set priorities, and stay hands-on with code and experimental design.

Qualifications

  • Experience owning evaluation for tabular ML systems used in production or consequential customer decisions.
  • Strong statistical judgment: choosing metrics and validation schemes, estimating uncertainty, comparing models across datasets, and accounting for repeated experimentation.
  • Practical experience finding leakage in preprocessing, feature construction, joins, temporal dependencies, and related entities across splits.
  • Strong Python and SQL skills, familiarity with scikit-learn and gradient-boosted trees, and experience building reliable ML tooling or platforms used by other teams.
  • Experience designing fair model comparisons, including hyperparameter search, resource budgets, and end-to-end latency measurement.
  • Prior people management experience, including hiring, technical coaching, and performance feedback, while remaining technically involved.
  • Clear written and spoken communication with researchers, engineers, and customer data scientists, including the willingness to challenge claims the evidence does not support.

Skills

Required:Evaluation designPythonSQLData leakage detectionModel evaluationManagementCommunication

Benefits

  • Competitive compensation with salary and equity
  • Comprehensive health coverage for you and your dependents
  • Paid parental leave for all new parents, inclusive of adoptive and surrogate journeys
  • Relocation support for employees moving to join the team in one of our office locations
  • A mission-driven, low-ego culture that values diversity of thought, ownership, and bias toward action
Last updated Oct 20, 2026. Interested? Apply on the company site.

About Fundamental

The Power to Predict

The Power to Predict. Our most powerful Large Tabular Model. Pre-trained on billions of real-world enterprise tables, our model knows which patterns drive which outcomes.

NEXUSOur World’s Most Powerful Large Tabular ModelPre-trained on billions of real-world enterprise tablesEnterprise AIData AnalyticsTime SeriesConfidential ComputingCloud
Company profileWebsite

More open roles at Fundamental