Data Scientist, Agent
Stockholm, LondonFull-timePosted 2mo ago
About the role
Data scientist responsible for measuring and improving Lovable's AI agent through eval systems, experiments, and telemetry-driven fixes to increase success rates and reduce errors.
Responsibilities
- Define and own the metrics for agent quality: success, completion, error rates, and the behaviors that drive them.
- Build the eval systems and experiment framework that decide whether an agent change ships, like an A/B-tested rollout that catches a change increasing errors before it reaches everyone.
- Turn agent traces and telemetry into concrete fixes, working directly with the agent engineering team.
- Build the tooling and agents that produce these evaluations continuously as the agent evolves.
- Set the bar for how we judge agent behavior where there is no answer key to check against.
Qualifications
- Experience or strong interest in LLM evaluation and observability: building evals, scoring outputs, tracing agent behavior, and catching regressions.
- Strong SQL and Python, applied statistics, and experimentation. Comfortable designing A/B tests for agent changes where outcomes are noisy.
- Entrepreneurial. Thrives in ambiguity, and works closely with the engineers building the agent.
Skills
Required:SQLPython
Last updated Sep 9, 2026. Interested? Apply on the company site.
About Lovable
Build something
Build software products, using only a chat interface
ai coderai developerai engineerai full stack engineerai software engineerai web agencyai website builderai-infrastructure-and-development
More open roles at Lovable
- Partner Operations Manager4d agoStockholm · GTM
- Technical Writer1w agoStockholm · EPD
- Data Infrastructure Engineer1w agoStockholm, London · EPD
- Growth Engineer1w agoLondon · EPD
- Marketing Growth Engineer2w agoLondon · EPD