← All roles

DevOps Engineer - AI Runtime Services

Demandbase· Engineering
Apply on company site
Hybrid (Hyderabad)Full-timePosted 2mo ago

About the role

Role focused on building and operating the shared AI runtime and tools at Demandbase, enabling and observing AI features, with ownership across gateway, reliability, cost control, observability, evals infra, caching, and guardrails.

Responsibilities

  • The LLM gateway. The single front door to every model provider we use — keys, routing, rate limits, failover, provider quotas.
  • Reliability, for real. SLOs, on-call, incident response, and the postmortems for the runtime.
  • Cost control. Per-team and per-model attribution, budgets, quotas.
  • LLM observability. Traces, spans, prompt/response capture, ingest and billing visibility for every AI feature in the company.
  • Evals and experimentation infra. The shared harnesses, datasets, and scoring plumbing product teams use to know whether a prompt or model change actually made things better.
  • Caching and guardrails. Response and semantic caching to cut latency and redundant spend; input/output safety, PII handling, and policy enforcement at the gateway.

Qualifications

  • Strong production infrastructure background: Kubernetes, AWS, Terraform, GitOps.
  • Python, at a level where you're comfortable owning services and tooling in it — not just scripting.
  • Real on-call and incident-response experience.
  • Observability fluency beyond dashboards: instrumented systems, cost and cardinality analysis in metrics/logging backends, alerting.
  • Hands-on familiarity with how LLM systems actually work: built tools with these technologies (agent, RAG pipeline, internal tooling); familiarity with tokens, context windows, prompt/response tracing, and eval suites.

Skills

Required:KubernetesAWSTerraformGitOpsPythonObservabilityLLM systems

Benefits

  • Group Medical
  • Personal Accident
  • Term Life Insurance
  • preventive healthcare including dental, vision, OPD
  • mental health support
  • fitness benefit
  • car lease policy
  • gratuity for long-term financial well-being
Last updated Oct 23, 2026. Interested? Apply on the company site.

About Demandbase

Grow high-quality pipeline faster with the only AI platform built for how B2B really buys

Elevate your B2B go-to-market strategy with Demandbase's ABM platform, delivering smarter insights, agent-led automation, and AI-powered pipeline.

ABM platformAIDataB2B salesMarketing automationGTM
Company profileWebsite

More open roles at Demandbase