Machine Learning Engineer: Evaluation
Hybrid (San Francisco, CA), Hybrid (New York, NY)Full-timePosted 3mo ago
About the role
Bedrock is deploying autonomous systems on heavy construction equipment. The role focuses on evaluating complex ML systems in real-world production environments, building evaluation pipelines, metrics, and data processing to accelerate operator adoption.
Responsibilities
- Design and maintain eval systems: Build pipelines for measuring system performance – across open loop and closed loop simulation, hardware in the loop systems, and field data from Bedrock Operator equipped machinery.
- Develop metrics: Connect product goals and system behavior by bridging real-world specification to measurable indicators from logged data.
- Classify data sources for training and testing: Implement infrastructure and classifiers to self-annotate data and create datasets for training and evaluation use cases; leverage models to source rich annotations for large datasets.
- Predict system performance: Model metrics and interpret results from various sources, determine deployment readiness and drive deployment decisions.
Qualifications
- Engineers who are currently Senior or Staff level with 5+ years of professional software engineering, data science, or research experience
- 2+ years of professional experience analyzing modern ML or robotics system performance on real-world problems
- Proficiency in Python and a data warehouse query language and comfort with development on infrastructure within parallelized cloud-based frameworks
- Strong statistical analysis skills (e.g. classification, model fit bias determination, hypothesis testing, and uncertainty quantification)
- Experience working with large datasets
Skills
Required:PythonData warehouse query languageStatistical analysis
Last updated Sep 10, 2026. Interested? Apply on the company site.
About Bedrock Robotics
Advanced Autonomy for the Built World
Advanced Autonomy for the Built World
AutonomyBedrock Operatorautonomyheavy machineryindustrial automationindustrial roboticsmachine intelligenceon-site autonomy
More open roles at Bedrock Robotics
- Hydraulics System Engineer3d agoHybrid (San Francisco, CA) · Engineering
- Electrical Systems Integration Engineer3d agoHybrid (San Francisco, CA), Hybrid (Lockhart, TX) · Engineering
- Bedrock Training & Certification Lead3d agoHybrid (San Francisco, CA), Hybrid (Lockhart, TX) · Operations
- Cloud - Staff Site Reliability Engineer3d agoHybrid (New York, NY), Hybrid (San Francisco, CA) · Engineering
- Head of Bedrock Operations Center1w agoHybrid (San Francisco, CA), Hybrid (Lockhart, TX) · Operations