← All roles

Principal AI Engineer - Context - Agents and Context

Elastic· Enterprise Search - Engineering
Apply on company site
Hybrid (Poland)full-timePosted 2w ago

About the role

Principal AI Engineer owning the end-to-end production improvement loop for the Context Engine, focusing on agents, automations and skills, telemetry, evaluation, and safe iteration in production. Hybrid role combining engineering, data science, and product.

Codebase is TypeScript; Ship code and designs in public.

Responsibilities

  • Own the production improvement loop for Context Engine: understand extraction automations, retrieval tools and memory in production, identify and fix failure modes, prove the fix.
  • Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, guardrails for changing behaviour without breaking customers.
  • Design telemetry to support data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval, how it lands in Elasticsearch, and how it feeds evaluation and dashboards.
  • Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
  • Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write technical proposals that shape the roadmap.

Qualifications

  • 10+ years of software engineering experience, with recent years shipping and operating AI-driven products on real production traffic, ideally with public APIs and evolving data models.
  • Track record of eval-driven product improvement: diagnosing agent/LLM behaviour from traces and user feedback, designing evaluation, shipping fixes and measuring outcomes.
  • Direct experience building agents with state and memory, iterating on prompts, skills and tool behaviour in production.
  • Familiarity with MCP, including exposing public MCP servers and tools.
  • Experience designing telemetry for AI systems and using it to inform engineering/product decisions.
  • Experience running product experiments end-to-end: instrumentation, unattended execution, interpreting results.
  • Experience building products with public APIs and evolving data models, maintaining external contracts.
  • Strong backend engineering skills in Python or TypeScript: APIs, stateful workflows, data pipelines, production services.
  • Comfort collaborating with data scientists, engineers, and product managers; translating between measurement and shipping; clear trade-offs.
  • Pragmatic, low-ego, async-first team mindset.

Skills

Required:Software engineeringAgent/LLM behaviour evaluationAgent frameworks and memoryTelemetry design for AI systemsPublic APIs and data model evolution

Benefits

  • Competitive pay based on work done (no reliance on past salary)
  • Health coverage for you and your family in many locations
  • Flexible calendar with locations and schedules
  • Generous vacation days
  • Up to $2000 matched for charitable donations
  • Up to 40 hours per year for volunteer projects
  • Minimum 16 weeks parental leave
Last updated Oct 8, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic