Senior Engineer, Applied AI
Hybrid (New York, NY)$190.8k - $234.5kFull-timePosted 3mo ago
About the role
Senior Engineer, Applied AI at Ro to build and ship LLM-powered features across patient experience, clinical operations, and internal tooling. Own AI features from prototype through production, collaborating with clinicians and operators to ensure reliability, safety, and measurable impact in healthcare workflows.
Responsibilities
- Design, build, and ship production LLM powered features end-to-end.
- Build the AI application layer, including prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and structured outputs.
- Develop evaluation infrastructure, including regression suites, LLM-as-a-judge evaluations, synthetic datasets, human review workflows, and quality metrics that catch regressions before users do.
- Design observability systems that measure accuracy, latency, cost, hallucination rates, and model behavior in production.
- Build safety and reliability guardrails that ensure AI systems meet quality, compliance, and operational requirements.
- Evaluate and integrate third-party tools, including orchestration frameworks, evaluation platforms, vector databases, and model providers; and make pragmatic build-versus-buy decisions.
- Raise engineering standards across the team through technical leadership, code reviews, mentorship, and thoughtful system design.
- Sit with EHR operators, Patient Advocates, and clinical teams to understand the workflows you're automating.
- Prototype quickly, ship early, measure outcomes, and iterate based on real-world usage.
Qualifications
- 5+ years building production software, with at least 1–2 years building and deploying LLM-powered applications in production (or equivalent depth through substantial side projects or open-source work).
- Strong Python engineer with experience building backend services and integrating modern AI APIs and frameworks.
- Comfortable using AI-assisted development tools throughout the software development lifecycle.
- You've shipped AI features that served real users, and you can clearly articulate how you measured quality, reliability, latency, cost, and business impact.
- You have strong product instincts and thrive in ambiguous environments, translating operational problems into technical solutions without relying on detailed specifications.
Skills
Required:Production software developmentPythonAI APIs and frameworksLLM-powered features
Nice to have:Orchestration frameworks (e.g., LangChain, LangGraph, AgentCore)RAG architecturesVector databasesEvaluation/observability platforms
Last updated Oct 21, 2026. Interested? Apply on the company site.
About Ro
it's not a shortcut. it's healthcare.
Ro is a telehealth company connecting patients with US-licensed professionals entirely online. Get treatment now for weight loss, sexual health, hair loss, fertility, & more.
Weight lossSexual healthHair lossFertilitySkin
More open roles at Ro
- Associate Manager, Offline Marketing4d agoHybrid (NY or Remote) · Growth
- Seasonal, Virtual Medical Assistant / Patient Advocate4d agoHybrid (NY / Remote) · Care Delivery
- Associate Manager, Clinical Claims1w agoHybrid (New York, NY or Remote) · Clinical
- Principal Product Designer1w agoHybrid (New York, NY / Remote) · Design
- Senior Payroll Analyst2w agoHybrid (New York, NY) · Finance