← All roles

AI Product Manager, AI Platform - Quality and Evaluation

Ridgeline· Product Management
Apply on company site
Hybrid (San Ramon, CA; Reno, NV; New York, NY)$136k - $170kPosted 2mo ago

About the role

Product Manager for Ridgeline's AI Platform team focused on quality, observability, and evaluation of AI systems, including LLM evaluation frameworks, model benchmarking, evaluation harnesses, and AI quality platforms.

Responsibilities

  • Define and execute the product strategy and roadmap for Ridgeline's AI Quality Platform
  • Build scalable LLM evaluation frameworks and automated evaluation harnesses that enable continuous model validation
  • Establish standardized offline and online evaluation methodologies, golden datasets, and quality benchmarks across AI products
  • Define the roadmap for AI observability, enabling teams to monitor quality, latency, cost, reliability, and user outcomes in production
  • Partner with engineering and AI researchers to build continuous regression testing and experimentation capabilities for generative AI systems
  • Drive product decisions using AI quality metrics, customer feedback, and quantitative evaluation data
  • Collaborate with engineering teams to deliver platform capabilities that enable trusted, measurable, and continuously improving AI experiences across Ridgeline
  • Translate advances in foundation models, evaluation techniques, and AI tooling into platform capabilities that create meaningful customer value
  • Prioritize product investments using customer insights, product analytics, and measurable business outcomes
  • Communicate product vision, roadmap, and priorities clearly across technical and executive stakeholders
  • Foster a collaborative environment rooted in ownership, continuous learning, experimentation, and transparency

Qualifications

  • 5+ years of product management experience building highly technical platform, infrastructure, or developer-focused products
  • Demonstrated experience shipping production AI products powered by large language models
  • Experience defining products that incorporate LLM evaluation frameworks, AI quality measurement, and continuous model evaluation
  • Deep understanding of offline and online evaluation methodologies, model benchmarking, regression testing, and AI quality metrics
  • Experience with LLM observability platforms such as Datadog LLM Observability, Langfuse, LangSmith, Arize AI, Braintrust, Galileo, or similar AI quality and evaluation solutions
  • Familiarity with prompt engineering, experimentation frameworks, evaluation datasets, and LLM-as-a-judge techniques for generative AI applications
  • Experience partnering effectively with software engineers, ML engineers, AI researchers, and cross-functional stakeholders
  • Strong analytical skills with the ability to translate AI performance signals into product decisions
  • Excellent written and verbal communication skills
  • Willingness to learn about cutting-edge technologies while cultivating expertise in investment management workflows and enterprise AI
  • An aptitude for problem solving
  • Ability to communicate effectively
  • Serious interest in having fun at work

Skills

Required:Product management for AI platformsAI/ML product StrategyLLM evaluation frameworksObservability and monitoringCross-functional collaborationData-driven decision making
Nice to have:Prompt engineering and experimentation

Benefits

  • Unlimited vacation
  • Educational and wellness reimbursements
  • $0 cost employee insurance plans
  • Company Stock Plan (subject to applicable Stock Option Agreement)
Last updated Aug 19, 2026. Interested? Apply on the company site.

About Ridgeline

Revolutionizing Investment Management

Ridgeline: Revolutionizing Investment Management with a single front-to-back system of record, embedded AI, and automation capabilities.

Investment ManagementAIAutomationPortfolio ManagementTradingComplianceData
Company profileWebsite

More open roles at Ridgeline