← All roles

Principal AI Engineer - Context - Agents and Context

Elastic· Enterprise Search - Engineering
Apply on company site
Hybrid (Canada)CA$154k - CA$243.6kfull-timePosted 4w ago

About the role

Principal AI Engineer responsible for the end-to-end production improvement loop of the Context Engine, including agent and memory behavior, evaluation, telemetry, and safe rollout of changes. Hybrid role at the intersection of engineering, data science, and product."

Responsibilities

  • Own the production improvement loop for Context Engine: understand extraction automations, retrieval tools and memory behavior, perform offline evaluations, engage with telemetry and customer conversations, identify and fix failure modes, prove fixes.
  • Define safe iteration practices for agents and skills: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and guardrails to prevent breaking customers.
  • Design telemetry for data-informed decisions: determine what to capture from agent traces, tool calls, and knowledge retrieval; integrate with Elasticsearch; feed evaluation dashboards and feedback loops.
  • Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
  • Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and author technical proposals shaping the roadmap.

Qualifications

  • 10+ years of software engineering experience with recent focus shipping and operating AI-driven products on production traffic
  • Track record of eval-driven product improvement: diagnosing agent/LLM behavior from traces and user feedback, designing evaluation, shipping fixes, and measuring outcomes
  • Direct experience building agents with state and memory; iterating on prompts, skills and tool behavior in production
  • Familiarity with MCP, including exposing public MCP servers and tools
  • Experience designing telemetry for AI systems and using it to inform engineering and product decisions
  • Experience running product experiments end to end: instrumentation, unattended execution, interpreting results
  • Experience building products with public APIs and evolving data models while maintaining external contracts
  • Strong backend engineering skills in Python or TypeScript: APIs, stateful workflows, data pipelines, production services
  • Ability to collaborate with data scientists, engineers, and product managers; translate measurement to shipping; clear communication of trade-offs
  • A pragmatic, low-ego style suited to a distributed, async-first team

Skills

Required:Software engineeringEval-driven product developmentAgent and memory modellingTelemetry & instrumentationPublic APIs and data model evolutionCollaboration with cross-functional teams

Benefits

  • Health coverage for you and your family in many locations
  • Flexible locations and schedules for many roles
  • Generous number of vacation days each year
  • Company-matched RRSP with dollar-for-dollar matching up to 6% of eligible earnings
  • Stock program participation
  • Volunteer time and charitable donation matching (up to $2000 or local currency equivalent)
Last updated Sep 24, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic