Member of Technical Staff (Evals & Post-Training)
About the role
Role focused on building and evaluating a replayable environment engine for enterprise workflows, bridging research, infrastructure, and production systems to improve model performance and reduce inference costs. Reports to founders/CTO with significant ownership over research direction and production systems.
Responsibilities
- Building an environment factory that converts recorded enterprise data and task definitions into runnable environments
- Designing graders that turn ambiguous business objectives into verifiable rewards
- Developing methods for mining useful tasks, trajectories, and evaluation cases from historical workflows
- Creating eval sets that are representative, reproducible, and resistant to overfitting
- Finding the right combinations of models, tools, context, and policies to maximize performance while reducing inference cost
- Training and evaluating agents that operate over long horizons, incomplete information, and large tool spaces
- Building replay and observability systems that make agent behavior explainable and measurable
- Scaling from individual environments to thousands of concurrent training and evaluation runs
- You’ll work directly with the CTO, deploy into real enterprise workflows, and see your research tested against consequential problems and observable outcomes.
Qualifications
- 1-7 years of experience building production software or machine-learning systems (we're hiring at multiple levels for this role)
- Bonus points for working on reinforcement-learning environments, LLM post-training, evaluation infrastructure, agent harnesses, or closely related systems
- Understanding of how environment design, reward design, context, tooling, and policy behavior interact
- Comfort turning fuzzy business objectives into tasks and signals that can be evaluated reliably
- Ability to diagnose whether a model’s limitations come from the model itself, its context, its tools, its harness, or its training
- Ability to move between research questions and production implementation without treating them as separate jobs
- Strong software skills and ability to build systems that process large, messy datasets at scale
- Commitment to reproducibility, observability, and understanding why a model behaves the way it does
- Ambition to do the best work of your life and build something lasting
Skills
Benefits
- Significant equity and ownership
- Equinox membership
- Free meals, coffee, and snacks
- Health insurance
- Unlimited PTO
About Ambral
AI-powered Account Managers
Ambral is an AI Account Management team. Ambral synthesizes signals from all customer activity and interactions into AI-powered models of every account. It pinpoints who needs attention and why, and autonomously drives or recommends expansions, prevents churn, and ensures no customer slips through the cracks. It's like mission control for your customers. Team: Sam (CEO): 2x founder. Led AI at Everlywell and early PM at Wonder (now valued at $7bn). Lived the pain of losing customer intimacy at scale and is obsessed with solving it. Cornell CS alum. Jack (CTO): Ex-SpaceX Flight Software Special Projects, where he developed telemetry routing and analysis systems for the most complex machines ever created. He’s now doing the same for the complexities of customer relationships. Cornell CS alum. Why: As B2B companies scale, the founder-level customer intimacy that drives early growth becomes impossible to maintain. Customer context that once fit in a founder's head scatters across systems, logs, and communication channels, drowning teams in noise no human can make sense of. Account managers juggling multiple customers become reactive firefighters, bleeding revenue through missed upsells and preventable churn. We've lived this pain and are building Ambral to help teams scale their most valuable asset: customer obsession.
More open roles at Ambral
- Head of Research1mo agoHybrid (New York City, NY / San Francisco, CA) · Research
- Member of Technical Staff (New Grad)2mo agoHybrid (New York City, NY / San Francisco, CA) · Operations