← All roles

Principal AI Engineer - Context - Agents and Context

Elastic· Enterprise Search - Engineering
Apply on company site
Full-timePosted 4w ago

About the role

Principal AI Engineer responsible for the end-to-end improvement loop of the Context Engine product, focusing on how agents, automations and skills behave in production, implementing telemetry, evaluation, and safe shipping of changes. Hybrid role at the intersection of engineering, data science, and product, with code and designs shipped in public.

Responsibilities

  • Own the production improvement loop for Context Engine: understand extraction automations, retrieval tools and memory behavior, identify failure modes, fix them, and prove the fix.
  • Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, guardrails to change behavior without breaking customers.
  • Design the telemetry needed for data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval; how it lands in Elasticsearch; how it feeds evaluation, dashboards and feedback loops.
  • Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
  • Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write technical proposals shaping the roadmap.

Qualifications

  • 10+ years of software engineering experience, with recent years shipping and operating AI-driven products on real production traffic, ideally with public APIs and evolving data models.
  • Track record of eval-driven product improvement: diagnosing agent or LLM behaviour from traces and user feedback, designing evaluation, shipping fixes and measuring outcomes.
  • Direct experience building agents with state and memory, iterating on prompts, skills and tool behaviour safely in production.
  • Familiarity with MCP, including exposing public MCP servers and tools.
  • Experience designing telemetry for AI systems and using it to inform engineering/product decisions.
  • Experience running product experiments end to end: instrumentation, unattended execution, interpreting results.
  • Experience building products with public APIs and evolving data models, maintaining external contracts as products change.
  • Strong backend engineering skills in Python or TypeScript: APIs, stateful workflows, data pipelines and production services.
  • Comfort collaborating with data scientists, engineers, and product managers; translating between measurement and shipping; clearly communicating trade-offs.
  • Pragmatic, low-ego style suited to a distributed, async-first team.

Skills

Required:Production-grade AI product development

Benefits

  • Competitive pay based on the work you do here and not your previous salary
  • Health coverage for you and your family in many locations
  • Flexible locations and schedules for many roles
  • Generous number of vacation days each year
  • Up to $2000 (or local currency equivalent) matched for financial donations and service
  • Up to 40 hours per year toward volunteer projects
  • Minimum of 16 weeks parental leave
  • Inclusive, equal opportunity employer with diverse perspectives considered in hiring
Last updated Sep 24, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic