ML Engineering Lead (LLM Ops)
Hybrid (Stockholm), Hybrid (Berlin), Hybrid (London)Full-timePosted 2w ago
About the role
Lead the operational lifecycle of Neko's LLM, GenAI and RAG based systems, building production-grade platforms for clinical ML and GenAI workflows on proprietary sensor and device data within a regulated medical device QMS.
Responsibilities
- Stand up MLflow Tracing observability, prompts, tool calls, retrievals, latency and cost, live across production LLM and agent pipelines.
- Build an evaluation suite combining built-in and custom LLM judges and scorers, with a human-feedback loop via review apps, replacing today's ad hoc review.
- Ship at least one RAG or agentic pipeline to production with prompt and application versioning through the MLflow Prompt Registry and Unity Catalog, enabling safe rollout, A/B testing and rollback.
- Produce a documented cost, latency and GPU capacity framework for choosing serving strategy: third party API versus Databricks External Models versus self-hosted.
- Integrate LLM Ops tightly with the existing MLOps team so it operates as a natural extension of the platform rather than a separate track, protecting reliability on clinical workflows.
Qualifications
- Solid MLOps fundamentals across the full lifecycle, experiment tracking, training and monitoring, demonstrated through independent ownership of complex, production grade work.
- Fluent in Python and core ML concepts, with a track record of shipping end-to-end production ML systems and platformisation initiatives.
- Practical, hands-on experience building LLM or GenAI applications: prompt engineering, RAG, agents or chains, using frameworks such as LangChain, LangGraph, or comparable orchestration tools.
- Working knowledge of PyTorch, distributed systems and ML orchestration.
- Conceptual understanding of LLM-specific MLOps trade-offs: fine-tuning versus prompting versus RAG, vector databases, embedding models, and human-feedback loops for non-deterministic outputs. Hands-on fine-tuning ownership is not required, as execution sits with a separate track.
- Genuine, demonstrable motivation for LLM, GenAI and RAG work specifically, not generic MLOps, and the ability to navigate complex systems spanning the medical domain, regulation, firmware and hardware.
Skills
Required:MLOps lifecyclePythonLLM/GenAI application developmentFrameworks (LangChain, LangGraph)PyTorchLLM trade-offs understanding
Benefits
- Not specified in description
Last updated Oct 9, 2026. Interested? Apply on the company site.
About Neko Health
A better health check. Finally.
Try the Neko Body Scan. Get answers right away, and let us monitor your health - so you don't have to.
HealthHealth CentresMedical
More open roles at Neko Health
- Member Care Lead - New York4d agoHybrid (New York) · Operations & Markets
- Member Care Lead - UK4d agoHybrid (London) · Operations & Markets
- Global Security Lead4d agoHybrid (New York) · Operations & Markets
- Global Health and Safety Lead4d agoHybrid (London) · Operations & Markets
- Medical Recruiter4d agoHybrid (London) · People