Member of Technical Staff (Applied AI Research)
Artificial Analysis· Technical Staff (incl. Product)
San Francisco (On-site)Full-timePosted 1mo ago
About the role
Applied AI research role focused on designing evaluations, building benchmarks, and publishing analysis to shape how AI capabilities are measured. On-site in San Francisco, full-time.
Responsibilities
- Design Frontier Evaluations: Conceive, build and ship novel evaluation methodologies that advance how AI capabilities are measured, and stay ahead of frontier models
- Build Evaluation Datasets and Infrastructure: Construct datasets, harnesses and scoring systems behind benchmarks, engineered for contamination resistance and repeatability at frontier scale
- Evaluate Every Major Model: Run evaluation suite across frontier releases as they land, and own the integrity of the results the industry quotes
- Publish Influential Analysis: Produce reports, indexes and data visualizations that shape industry understanding of AI progress
- Work with Frontier Labs on Pre-Release Models: Benchmark leading labs’ systems, including pre-release and newly launched models, collaborating with their research teams (commercial team handles client relationships)
- Become AI-Native: Use cutting-edge AI tools to maintain competitive edge in benchmarking
Qualifications
- 3+ years of relevant professional experience across industry or research
- Strong analytical and coding skills
- Clear fit with one of three profiles: AI and Machine Learning, Strategy Consulting, or Technical Product Management
- Proficiency in Python and data analysis
- Genuine interest and knowledge of Frontier AI
Skills
Required:PythonData analysis
Benefits
- Competitive compensation including equity
Last updated Oct 1, 2026. Interested? Apply on the company site.
About Artificial Analysis
Independent analysis of AI
AI ModelAPI ProvidersAnalysisBenchmarkingIndependenceEvaluationIntelligence IndexLeaderboards
More open roles at Artificial Analysis
- Senior AI / ML Engineer1mo agoMelbourne (On-site) · Engineering
- Member of Technical Staff (Commercial) - Strategy Consulting track1mo agoSan Francisco (On-site) · Technical Staff (incl. Product)
- Forward Deployed Engineer - Language Models1mo agoSan Francisco (On-site), Melbourne (On-site) · Platform & Forward Deployed Engineering
- Senior AI / ML Engineer1mo agoSan Francisco (On-site) · Engineering
- Product Manager (Media Generation - Image & Video)1mo agoSan Francisco (On-site) · Technical Staff (incl. Product)