Member of Technical Staff, Agents
San FranciscoFull-timePosted 2mo ago
About the role
Role focused on building product experiences and agents to run scalable, end-to-end agent investigations and improvements within Judgment Labs’ AI agent platform.
Responsibilities
- Judgment Agent: Shape how the Judgment Agent runs large-scale investigations across production traces and merge results into a single answer.
- Verification: Build platform for verifying agent changes including simulated environments, trajectory replay, and monitors for unintended behavior changes.
- Agent investigation interfaces: Design interfaces to understand agent decisions, long traces, tool calls, and reasoning; enable fast debugging of multi-step trajectories.
- Swarm UX: Design dashboards and workflows to monitor parallel investigations and guide investigators.
- The improvement loop: Create workflows to turn production trajectories into datasets, judges, and regression checks to streamline problem-to-fix flow.
- The platform underneath: Develop workspaces, roles, permissions, billing, usage, and limits for multi-environment agent deployments.
- Judgment everywhere agents are built: Provide an SDK and terminal-first experience to summon Judgment as a subagent during development.
Qualifications
- Experience building and scaling end-to-end production systems from data layer to UI.
- Strong technical problem-solving skills in fast-changing environments.
- A builder and tinkerer's mindset with high agency and shipping mindset.
- Hands-on experience building with LLMs or agents, or the drive to acquire it quickly.
- Comfort working directly with customers to understand needs and solve real-world problems.
- Excellent communication skills across technical and non-technical audiences.
- Reinforcement learning, post-training, agents, or machine learning fundamentals.
Skills
Required:End-to-end production system designLLMs / AI agentsCustomer interaction
Last updated Oct 25, 2026. Interested? Apply on the company site.
About Judgment Labs
The continuous-improvement stack for agents.
The continuous-improvement stack for agents. Monitor and improve your agent's behavior at scale.
AIJudgmentLabsagentbessemer-ai-infrastructure-2026harness-infrastructureproductiontrust
More open roles at Judgment Labs
- Member of Technical Staff, Product Engineer/Full Stack3mo agoSan Francisco · Engineering
- Member of Technical Staff, Backend/Infrastructure3mo agoSan Francisco · Engineering
- Member of Technical Staff, Deployment Engineer3mo agoSan Francisco · Engineering