Senior Research Scientist, Model Evaluation
Hybrid (Toronto), Hybrid (San Francisco), Hybrid (Canada), Hybrid (Seattle), Hybrid (London), Hybrid (United States), Hybrid (New York)CA$295k - CA$535kFull-timePosted 11mo ago
About the role
Role focused on creating next-generation evaluation methods and infrastructure to measure LLM progress, with responsibility for benchmarks, cross-functional collaboration, and scalable evaluation tools.
Responsibilities
- Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
- Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
- Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
- Build scalable and reusable tools for digging into model performance.
Qualifications
- Enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and have developed resources to measure those capabilities.
- Spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
- Obsessive about rigorously measuring AI capabilities and ensuring measurements align with the desired capabilities.
- Strong software engineering skills.
Skills
Required:Evaluation methodologyPrototypingData quality reviewSoftware engineering
Benefits
- A weekly lunch stipend of $75/£75 or equivalent in local currency for lunch.
- Full health and dental benefits, including a separate budget for mental health.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months, for either parent.
- Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit.
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days).
- Budget for traveling to other offices if remote, plus an annual company offsite.
- Remote-friendly with home office stipend and coworking benefits.
Last updated Oct 21, 2026. Interested? Apply on the company site.
About Cohere
Support automation that's actually intelligent
Cohere resolves 40-70% of support requests using AI, over all channels, within just a few days.
artificial-intelligenceautomationliquid-ai-recruiting-target
More open roles at Cohere
- Partner Development Manager - Berlin4d agoRemote · Revenue
- VP of Sales - Public Sector4d agoRemote · Revenue
- Delivery Operations Analyst4d agoRemote · Product Management & Program Management
- Technical Program Manager, AI Delivery for Public Sector & Defense, Germany4d agoRemote · Product Management & Program Management
- Senior/Staff Software Engineer (Back-End), Inference4d agoRemote · Inference