← All roles

Principal AI Engineer - Context - Agents and Context

Elastic· Enterprise Search - Engineering
Apply on company site
Hybrid (Greece)€73.3k - €115.9kfull-timePosted 2w ago

About the role

Principal AI Engineer responsible for the end-to-end production improvement loop of the Context Engine, shaping how agents, automations and skills behave in production, with emphasis on evaluation, telemetry, safety, and shipping changes. Hybrid role at the intersection of engineering, data science, and product, collaborating with data scientists, backend engineers, product, and UX to impact customer-built solutions on Elastic Stack.

Responsibilities

  • Own the production improvement loop for Context Engine: understand extraction automations, retrieval tools and memory behavior via offline evaluations, customer conversations, and telemetry; identify and fix failure modes; prove the fix.
  • Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and guardrails to avoid breaking customers.
  • Design the telemetry needed for data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval; how it lands in Elasticsearch and feeds evaluation, dashboards, and feedback loops.
  • Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
  • Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write technical proposals shaping the roadmap.

Qualifications

  • 10+ years of software engineering experience, with recent focus shipping and operating AI-driven products on real production traffic, ideally with public APIs and evolving data models.
  • Track record of eval-driven product improvement: diagnosing agent/LLM behavior from traces and user feedback, designing evaluations, shipping fixes and measuring outcomes.
  • Direct experience building agents with state and memory, iterating prompts, skills, and tool behavior safely in production.
  • Familiarity with MCP, including exposing public MCP servers and tools.
  • Experience designing telemetry for AI systems and using it to inform engineering and product decisions.
  • Experience running product experiments end to end: instrumentation, unattended execution, and interpreting results.
  • Experience building products with public APIs and evolving data models while maintaining external contracts.
  • Strong backend engineering skills in Python or TypeScript: APIs, stateful workflows, data pipelines and production services.
  • Ability to collaborate with data scientists, engineers, and product managers; translate measurement into shipping decisions; communicate trade-offs clearly.
  • A pragmatic, low-ego style suitable for a distributed, async-first team.

Skills

Required:Software engineeringEval-driven product improvementAgent/stateful memoryTelemetry design for AI systemsExperimentation and instrumentationPublic APIs and data model evolutionBackend development
Nice to have:MCP familiarity

Benefits

  • Competitive pay
  • Health coverage for you and your family in many locations
  • Flexible locations and schedules
  • Generous vacation days
  • Up to $2000 donation match
  • Up to 40 hours/year for volunteer projects
  • At least 16 weeks parental leave
Last updated Oct 8, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic