Linguist - Speech & Conversational AI
About the role
Besimple AI is building datasets, benchmarks, and training environments for speech and conversational AI. The Linguist role focuses on defining and improving the quality of speech and conversations used to train models, translating linguistic expertise into practical data decisions and evaluation workflows.
Responsibilities
- Evaluate speech and conversational quality. Assess pronunciation, intelligibility, prosody, pacing, tone, and turn-taking in human recordings and model outputs. Explain why a response sounds natural, awkward, or inappropriate for the situation.
- Build quality standards and evaluation rubrics. Translate customer requirements and feedback into criteria, examples, and decision rules. Define reviewers’ acceptance, flags, and rejections, including variations across roles and tasks.
- Screen and support speech contributors. Design auditions, evaluate recordings, and provide actionable feedback on delivery. Identify contributors who can sustain natural conversations within scenario requirements.
- Create and review linguistic annotations. Develop guidelines for transcripts, speaker turns, speech delivery, and conversational behavior. Review labels and resolve ambiguities to improve consistency.
- Improve conversational tasks and model evaluation. Collaborate with researchers to refine scenarios, instructions, and turn-level rubrics; identify unclear wording or grading criteria and distinguish model failures from task issues.
- Turn recurring problems into better workflows. Analyze disagreements and rejection patterns; calibrate reviewers; improve guidance in the crowdsourcing platform; partner with engineers and audio specialists to separate language and delivery issues from recording defects.
Qualifications
- A background in linguistics, speech science, phonetics, phonology, or related field, through academic study or substantial applied experience.
- Strong listening and analytical skills; ability to describe features of speech and how they affect meaning, clarity, and naturalness.
- Experience with speech data, linguistic annotation, transcription, language assessment, or conversational analysis.
- Ability to turn nuanced judgments into scalable guidelines for others.
- Clear written communication and comfort giving specific, constructive feedback.
- Careful judgment about ambiguity, language variation, and the difference between personal preference and task requirements.
- Recent graduates with relevant research or projects, as well as linguists with industry experience; evidence in the form of research, annotation guidelines, evaluation studies, and applied linguistic work.
Skills

About Besimple AI
Expert-in-the-loop eval data for AI
Besimple AI helps teams ship AI with confidence by making high-quality, human-in-the-loop data fast, flexible, and scalable. We’re not a generic annotation vendor—we’re your on-demand evaluation and safety data engine. In under a minute, you can spin up your own data-annotation workspace tailored to your workflow, so your team can move from “we think it works” to measurable, repeatable results. Modern AI changes quickly. Models evolve, prompts shift, and new edge cases appear the moment you go live. Besimple is built for that reality. Our platform generates task-specific annotation experiences aligned to your data and goals, so you can capture reliable judgment on everything from factuality and safety to preference and policy compliance. The result is an always-current evaluation loop that keeps pace with your product, rather than a one-off test that goes stale. We specialize in evaluation and expert-grade safety data—not commodity labeling. That means we recruit and train domain experts and SMEs to judge model outputs against high bars, and we design the interfaces and guidelines to make those judgments consistent, auditable, and useful for both offline evals and production monitoring. Your team shouldn’t be waiting on back-and-forth specs and custom tooling. With Besimple, you can paste or stream your data, click to generate a tailored interface, and start collecting ground truth right away. When the task changes, the UI adapts with it. We combine pragmatic product design with a human-in-the-loop operating model. Besimple helps you create clear, actionable guidelines and then enforces them with workflow guardrails, reviewer consensus, and targeted spot-checks—so you can trust the numbers you ship. Because we’re built for iterative work, teams can quickly compare prompts, models, and policies; spot regressions; and push fixes backed by fresh, expert-reviewed data. Great models don’t happen by accident—they’re the product of tight feedback loops, expert judgment, and tooling that bends to your data, not the other way around. If you’re ready to replace slow, one-size-fits-all processes with a purpose-built evaluation and safety engine, Besimple gives you the fastest path from raw data to reliable decisions—so you can iterate boldly and ship with confidence.
More open roles at Besimple AI
- RL Environment Engineer (Full-time, Part-time, or Contract)4d agoHybrid (San Mateo, CA, US) · Engineering
- Community Growth Manager, Contributor Acquisition4w agoHybrid (San Mateo, CA) · Operations
- Strategic Projects Lead — Audio Data3mo agoHybrid (San Mateo, CA) · Operations
- Audio QA Lead - Part Time Contractor4mo agoHybrid (San Mateo, CA) · Operations
