Sr. Staff Machine Learning Systems Engineer
Hims & Hers· ENGINEERING
Remote$240k - $265k + equityFull-timePosted 2mo ago
About the role
Senior leadership role owning end-to-end evaluation and data pipelines for trusted AI in a regulated healthcare environment. Responsible for data ingestion, feature/dataset pipelines, evaluation infrastructure (judges, scorers, regression gates, red-team testing), and cross-team alignment from idea to production to ensure AI models are effective and safe for patients.
Responsibilities
- Own the evaluation as a whole, not just a slice of it
- Set the technical direction for our evaluation systems; metric, judge and scorer design, the statistical methodology behind regression decisions, and the infrastructure that tracks and categorizes failures over time
- Design and scale the data pipelines — ingestion, transformation, dataset versioning, labeling and calibration workflows — that both evaluation and downstream data science work depend on
- Proactively address challenges in scaling and complexity AI evaluation
- Lead projects that span teams and quarters
- Define how we evaluate any new AI service from scratch
- Drive multi-team initiatives like replacing manual, inconsistent review processes with statistically sound, automated gates
- Own our approach to adversarial and red-team evaluation as a risk-reduction program
- Work through complex, cross-team technical disagreements and drive alignment across engineering, product and AI leaders
- Turn hard, ambiguous problems into solutions other teams can build on
- Originate new approaches and methodology that becomes a reusable standard
- Take vague, cross-team pain points from idea to fully-specified, shipped system
- Lead major platform improvements; re-architect core systems and modernize operations
- Grow the people and the network around you
- Build cross-functional relationships across ML engineering, data science, platform engineering, clinical, legal, and product
- Become a go-to voice on evaluation methodology and data pipeline design
- Mentor other engineers and raise the technical and statistical bar
Qualifications
- 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems
- Hands-on depth in evaluation systems: designing and calibrating LLM judges/scorers, regression-testing methodology with proper multiple comparisons, measuring agreement against human labels, adversarial/red-team evaluation approaches
- Hands-on depth in data pipeline engineering: dataset versioning, feature and benchmark pipelines, labeling and calibration workflows, high-throughput ingestion and transformation systems
- History of building standard approaches adopted by others
- Experience leading multi-team projects to completion and resolving technical disagreements
- Track record of mentoring engineers and raising team standards
- Excellent communication and ability to tailor messaging to different audiences
- Strong Python and statistical fluency to design and defend a testing framework
Skills
Required:PythonEvaluation framework designData pipeline engineeringAdversarial/red-team evaluationCross-functional leadership
Benefits
- Competitive salary & equity compensation for full-time roles
- Unlimited PTO, company holidays, and quarterly mental health days
- Comprehensive health benefits including medical, dental & vision, and parental leave
- Employee Stock Purchase Program (ESPP)
- 401k benefits with employer matching contribution
- Offsite team retreats
Last updated Oct 21, 2026. Interested? Apply on the company site.
About Hims & Hers
More open roles at Hims & Hers
- Quality Assurance Investigations Specialist4d agoMesa, Arizona, Gilbert, Arizona · QUALITY
- Environmental Monitoring (EM) Specialist4d agoNew Albany, Ohio · QUALITY
- Quality Control Specialist, Sterile (Nights)4d agoNew Albany, Ohio · QUALITY
- Quality Control Specialist, Sterile (Nights)4d agoNew Albany, Ohio · QUALITY
- Quality Systems Document Control Specialist4d agoGilbert, Arizona, Mesa, Arizona · QUALITY