← All roles

Principal AI Engineer - Context - Agents and Context

Elastic· Enterprise Search - Engineering
Apply on company site
Hybrid (Ireland)€104.8k - €165.7kfull-timePosted 4w ago

About the role

Principal AI Engineer owning the end-to-end improvement loop for the Context Engine, focused on how agents, automations and skills behave in production, how we observe and evaluate them, and how we ship safe changes. Hybrid role combining engineering, data science, and product.

Responsibilities

  • Own the production improvement loop for Context Engine: understand extraction automations, retrieval tools and memory behavior from offline evaluations, telemetry and customer conversations; identify and fix failure modes; prove the fix.
  • Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged/shadow evaluation, and guardrails to avoid breaking customers.
  • Design telemetry to inform engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval; how it lands in Elasticsearch; how it feeds evaluation, dashboards and feedback loop.
  • Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
  • Raise the bar across the team: review designs/PRs, mentor engineers in eval-driven development, and write technical proposals shaping the roadmap.

Qualifications

  • 10+ years of software engineering experience, with recent focus on shipping and operating AI-driven products on real production traffic, ideally with public APIs and data models that evolved without breaking customers.
  • Track record of eval-driven product improvement: diagnosing agent/LLM behavior from traces and user feedback, designing evaluation, shipping fixes and measuring outcomes.
  • Direct experience building agents with state and memory, iterating on prompts, skills and tool behavior in production.
  • Familiarity with MCP, including exposing public MCP servers and tools.
  • Experience designing telemetry for AI systems and using it to inform engineering/product decisions.
  • Experience running product experiments end to end: instrumentation, unattended execution, interpreting results.
  • Experience building products with public APIs and evolving data models, with judgment to maintain external contracts.
  • Strong backend engineering skills in Python or TypeScript: APIs, stateful workflows, data pipelines and production services.
  • Comfort translating between data scientists, engineers and product managers; clear communication of trade-offs.
  • Pragmatic, low-ego, async-first team fit.

Skills

Required:Software engineering experience with AI-driven productsEval-driven product improvementAgent or memory-enabled systemsTelemetry design for AI systemsBackend developmentCommunication and collaboration

Benefits

  • Competitive pay based on the work you do here and not your previous salary
  • Health coverage for you and your family in many locations
  • Flexible locations and schedules for many roles
  • Generous vacation days
  • Match up to $2000 for charitable donations
  • Up to 40 hours per year for volunteer projects
  • Minimum 16 weeks of parental leave
Last updated Sep 24, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic