← All roles

Member of Technical Staff, Backend/Infrastructure

Judgment Labs· Engineering
Apply on company site
San FranciscoFull-timePosted 3mo ago

About the role

Senior Backend Engineer responsible for ingesting, structuring, evaluating, and serving agent experience data at production scale, with focus on backend and data infrastructure surfaces including telemetry ingestion, OLAP with ClickHouse, evaluation pipelines, RabbitMQ/Temporal workflows, multi-tenant scheduling, and customer-facing APIs.

Responsibilities

  • Design and build backend systems for trace ingestion, trajectory processing, evaluation orchestration, scoring, labeling, rubric generation, and customer-facing analytics.
  • Own the API surface used by the Judgment platform UI, SDKs, JudgmentHub libraries, MCP server, Slack agent, and customer integrations.
  • Build and operate the RabbitMQ / Temporal evaluation pipeline, including retry semantics, failure recovery, state reconciliation, and tenant-level scheduling.
  • Optimize the ClickHouse OLAP layer: schema design, partitioning, skip indexes, full-text-search pruning, query rewrites, deduplication, pagination correctness, and storage growth.
  • Turn raw spans, conversations, tool calls, scorer outputs, and agent-judge results into clean data models customers can use for evals, labeling, context engineering, and RL workflows.
  • Ship features end to end, often across Next.js, backend APIs, queues/workflows, and the data layer.
  • Work directly with customers to understand where their agents fail, what data is useful, and how Judgment should structure that experience for learning.
  • Roll out safely with feature flags, design docs, code reviews, tests, observability, and production debugging.
  • Raise the engineering bar through clear interfaces, maintainable systems, thoughtful reviews, and strong ownership.

Qualifications

  • Strong backend engineering experience building and operating production systems under real load.
  • Excellent fundamentals in distributed systems, API design, data modeling, reliability, and performance.
  • Experience working with high-volume event, trace, log, metric, or telemetry data.
  • Strong intuition for data systems: query patterns, storage layout, indexing, partitioning, latency, correctness, and cost.
  • Comfort owning systems beyond initial launch: debugging production issues, improving observability, scaling bottlenecks, and cleaning up abstractions as the product evolves.
  • Ability to work across backend, data, product, and infrastructure boundaries rather than treating them as separate silos.
  • Product judgment and willingness to ship across the stack when needed.
  • Clear communication. You can write a design doc, review a diff, explain a tradeoff, and unblock others without turning everything into process.

Skills

Required:Backend engineeringDistributed systemsHigh-volume data
Nice to have:SQL/OLAPRabbitMQ/Temporal
Last updated Oct 25, 2026. Interested? Apply on the company site.
Judgment Labs logo

About Judgment Labs

The continuous-improvement stack for agents.

The continuous-improvement stack for agents. Monitor and improve your agent's behavior at scale.

AIJudgmentLabsagentbessemer-ai-infrastructure-2026harness-infrastructureproductiontrust
Company profileWebsite

More open roles at Judgment Labs