← All roles

Software Engineer, Agent Optimization & Evaluation

Reddit· Dev Platform
Apply on company site
Remote$109.8k - $325kFull-timePosted 5d ago

About the role

Software Engineer role focused on leading Reddit's agent optimization and evaluation platform, building systems to measure and improve agents’ impact on engineering velocity, including evaluation environments, model orchestration, and feedback loops across production usage.

Responsibilities

  • Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows.
  • Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
  • Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
  • Design model routing and orchestration strategies that optimize for quality, latency, and cost.
  • Define metrics and experimentation frameworks that measure developer productivity and engineering velocity.
  • Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
  • Partner closely with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows.
  • Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that gives us control over our AI capabilities and economics.

Qualifications

  • 5-12+ years of engineering experience
  • Extensive experience building and shipping large-scale production software systems.
  • Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems.
  • Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively.
  • Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost.
  • Excellent software engineering fundamentals and a passion for building production-quality systems.
  • Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives.
  • Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations.

Skills

Required:Agent optimization & evaluationModel routing and orchestrationEvaluation frameworks & metrics

Benefits

  • Comprehensive Healthcare Benefits and Income Replacement Programs
  • 401k with Employer Match
  • Global Benefit programs that fit your lifestyle, from workspace to professional development to caregiving support
  • Family Planning Support
  • Gender-Affirming Care
  • Mental Health & Coaching Benefits
  • Flexible Vacation & Paid Volunteer Time Off
  • Generous Paid Parental Leave
Last updated Oct 21, 2026. Interested? Apply on the company site.
Reddit logo

About Reddit

The heart of the internet

Reddit is where millions of people gather for conversations about the things they care about, in over 100,000 subreddit communities.

social mediacommunityforumnewsentertainmentdiscussion
Company profileWebsite

More open roles at Reddit