Applied AI Researcher, Agent Systems & Evaluation
Mountain View, CA$193.9k - $352.3kFull-timePosted 2mo ago
About the role
Nuro is hiring an Applied AI Researcher focused on agent systems and evaluation to build a rigorous closed-loop evaluation platform for AI agents operating inside Nuro’s engineering organization, with a strong emphasis on real-world production data, post-training work, and end-to-end evaluation pipelines.
Responsibilities
- Establish the evaluation foundation for the agent fleet (eval data sources, task suites, noise floors, statistical acceptance standards).
- Take a high-volume workflow from unmeasured to automatically hill-climbing end to end as a template for the system.
- Run a post-training experiment on an open-weight vision-language model against driving data to assess pipeline viability.
- Quantify platform value by identifying which workflows improved, by how much, with what confidence.
- Read frontier literature and convert findings into live experiments against workloads.
- Own the evaluation pipeline end to end across three stages: eval data collection, eval loop construction, and automated hill climbing.
- Involve hands-on model work: supervised fine-tuning and RL on open-source vision-language models using proprietary driving data; leverage labeling workforce for targeted data needs.
- Develop experiments and task suites that reflect real production distribution and are resistant to gaming.
Qualifications
- Graduate degree in CS, ML, statistics, or related field, or equivalent research experience.
- Fluent in current literature with ability to judge real results from hype.
- Deep understanding of how LLMs work from pretraining to post-training and inference.
- Firm grasp of the full evaluation pipeline (sourcing data, loop construction, automation of climb).
- Rigorous experimentalist capable of designing fail-safe experiments with variance and power considerations.
- Hands-on post-training experience (SFT and RL), ideally on open-weight models; vision-language model experience is a strong plus.
- Strong Python engineering background; comfortable with production systems and data.
- Direct experience with LLM agent systems—building, evaluating, or analyzing failures.
- Orientation toward impact with measurable results on a weekly basis.
Skills
Required:Evaluation pipeline designPost-training / fine-tuningPythonExperiment design
Benefits
- Base pay range: $193,930 – $352,290 (base pay for the level scoped).
- Annual performance bonus eligibility.
- Equity.
- Competitive benefits package.
Last updated Aug 12, 2026. Interested? Apply on the company site.
About Nuro
Autonomy for all. All roads, all rides.
Driving the future of autonomy with the most scalable AI driver.
AutonomyAI driverDelivery robotsRobotaxiAutonomous vehicles
More open roles at Nuro
- Senior Software Engineer, Autonomy Behavior Testing4d agoMountain View, CA · Systems
- Staff Robotaxi Program Manager4d agoMountain View, CA · Autonomy
- Staff Autonomy Program Manager4d agoMountain View, CA · Autonomy
- Senior Embedded Software Engineer1w agoMountain View, CA · Controls, Pose and ESWE
- Senior Radar Systems Engineer1w agoMountain View, CA · Sensors Engineering