← All roles

Senior Software Engineer, AI Agent & LLM

Otter.ai· Engineering
Apply on company site
Mountain View, CA$185k - $230kfull-timePosted 2mo ago

About the role

Senior AI Agent & LLM Engineer focused on building and improving Otter’s conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform required to operate them at scale. Hands-on role from problem exploration to production, with emphasis on AI quality and end-to-end delivery.

Responsibilities

  • Build AI-agent experiences for Otter AI Chat that help users reason over conversations, retrieve knowledge, and complete complex tasks.
  • Advance Otter’s conversational knowledge engine through better knowledge extraction, annotation, and indexing.
  • Develop robust evaluation datasets, automated graders, regression tests, and release gates for AI quality.
  • Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and product workflows.
  • Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability.
  • Improve task completion, correctness, groundedness, reliability, latency, and cost.
  • Turn production traces, customer feedback, and usage signals into measurable product and model improvements.
  • Collaborate across AI, product, infrastructure, data, security, and application teams to deliver end-to-end capabilities.

Qualifications

  • 5+ years of AI Agent engineering, machine learning engineering, or related experience.
  • Strong backend or distributed-systems engineering skills.
  • Hands-on experience building and shipping AI-agent or LLM-powered products.
  • Strong focus on AI quality and evaluating nondeterministic systems.
  • Ability to use data, traces, logs, and qualitative examples to identify and resolve complex failures.
  • Effective collaboration across services, technical domains, and organizational boundaries.
  • Strong product judgment with rigorous engineering and evaluation practices.
  • High agency, ownership, and bias toward action.
  • Ability to take ambiguous problems from exploration to production launch and continuous improvement.
  • Demonstrated ability to use coding agents effectively while rigorously reviewing and controlling output (a plus).

Skills

Required:AI Agent engineeringBackend engineeringAI quality evaluation
Last updated Jul 29, 2026. Interested? Apply on the company site.

About Otter.ai

Turn conversations into transcripts

Otter is the AI Notetaker that builds your knowledge base and powers your workflow with transcription, automated summaries, AI Chat, and more.

transcriptionAI NotetakersummariesAI Chatmeetingsproductivityintegrations
Company profileWebsite

More open roles at Otter.ai