Research Engineer, Judgment Systems
About the role
Role focused on building AI agents for high-stakes judgment tasks in fraud review and investigations. Sits at the intersection of research and engineering, aiming to improve agent capability, reliability, and decision quality in regulated environments. Fully in-person in San Francisco.
Responsibilities
- Train, fine-tune, and improve models for fraud, scams, abuse, and other high-stakes judgment workflows
- Own research threads focused on improving agent capability, reliability, and decision quality
- Build proprietary benchmarks, datasets, and evals that reflect real customer workflows, regulatory constraints, and real failure modes
- Design and run experiments across post-training, retrieval, tool use, planning, memory, and long-horizon agent behavior
- Study where models break, why they break, and how to make them more robust
- Prototype new training strategies, agent architectures, and evaluation methods, then turn the best ideas into production systems
- Work closely with founders and engineering to translate research advances into deployed product capabilities
- Push the boundary of what AI agents can do in regulated industries
Qualifications
- Experience training, fine-tuning, or evaluating modern ML systems
- Strong programming skills and comfort working in research-heavy codebases
- Familiarity with LLMs, agent systems, post-training, reinforcement learning, retrieval, or adjacent areas
- Ability to design clean experiments and draw reliable conclusions from noisy results
- Strong engineering judgment and a bias toward building
- Interest in fraud, risk, trust and safety, compliance, or other regulated and adversarial domains
Skills
Benefits
- Competitive salary and meaningful equity
- Platinum-level medical, dental, and vision insurance
- Unlimited PTO, sick leave, and parental leave
- Up to $100 per month in reimbursement for personal health and wellness expenses
- 401(k) plan
About Variance
World-Class Tooling for Trust & Safety teams
Build safer platforms with Intrinsic. Get access to the best safety technologies from one unified API. Enable your Trust & Safety team to tell cohesive stories from policy to product, to enforcement. Michael and Karine first met two years ago at Apple’s Fraud Engineering, Algorithmic Risk team, which evolved to become Apple’s Trust & Safety function. We are both engineers and helped solve online abuse problems, including spam, botting, account security, and developer fraud for Apple’s customer base of over a billion users.
More open roles at Variance
- Enterprise Account Executive4mo agoSan Francisco, CA · Operations
- Research Engineer, Evals4mo agoHybrid (San Francisco)
- Product Marketing Lead4mo agoHybrid (San Francisco)
- Senior Software Engineer4mo agoHybrid (San Francisco) · Engineering
- Enterprise Account Executive4mo agoHybrid (New York, NY) · Operations