← All roles

Member of Technical Staff - Research

Vals AI· Engineering & Research
Apply on company site
San Francisco$130k - $185kFull-timePosted 3mo ago

About the role

Research-focused role designing and building AI benchmarks to evaluate real-world capabilities of LLMs; lead development of novel benchmarks, collaborate with labs and enterprises, publish findings, and scale benchmarks with infrastructure.

Responsibilities

  • Design and develop novel, high-impact benchmarks that assess challenging real-world capabilities
  • Conduct research to ensure our benchmarks are valid, reliable, and meaningful
  • Collaborate with foundation model labs and enterprises to understand evolving evaluation needs
  • Analyze model performance across benchmarks and articulate results
  • Publish research findings and contribute to the broader evaluation research community
  • Work closely with the infrastructure team to implement your benchmark designs at scale
  • Stay current with the latest developments in LLM capabilities and evaluation methodologies

Qualifications

  • Advanced research experience: Master's degree or PhD in Computer Science, NLP, Machine Learning, or related field. Undergrads with very strong research backgrounds may also be considered.
  • Publication track record: Published papers in reputable venues (NeurIPS, ICML, ACL, EMNLP, etc.) with focus on NLP, ML evaluation, or benchmarking.
  • Research methodology: Strong understanding of experimental design, statistical analysis, and evaluation frameworks.
  • Technical skills: Proficiency in Python for research and experimentation.
  • Communication: Ability to clearly communicate complex research ideas to both technical and non-technical audiences.
  • Collaboration: Experience working in research teams and integrating feedback.
  • Portfolio: Demonstrated track record of impactful research work.
  • Location: We are an in-person team based in San Francisco. We will support your relocation or transportation as needed.

Skills

Required:Advanced researchPublication record in NeurIPS/ICML/ACL/EMNLPExperimental design and statistical analysisPython

Benefits

  • Relocation and transportation support
  • Health/dental insurance coverage
  • Lunch and dinner provided, free snacks/coffee/drinks
  • Unlimited PTO
  • $1,500 housing stipend (within one-mile radius)
  • Collaborate with leading AI labs and work on the front lines of AI development and evaluations
  • Opportunity to publish and present your work
Last updated Oct 18, 2026. Interested? Apply on the company site.

About Vals AI

Private, domain-specific benchmarks in legal, tax, and finance.

financelawsoftwarehealthcarebenchmarksVals IndexSWE-benchTaxEval
Company profileWebsite

More open roles at Vals AI