Applied AI Engineer
Hybrid (Bengaluru, India)INR 2000k - INR 5000kFull-timePosted 5mo ago
About the role
Infer is building the operating system for insurance agencies and hiring an Applied AI Engineer to own and improve the evaluation harness for voice agents, focusing on end-to-end quality across transcription, LLM reasoning, tool use, TTS, and call outcomes, with a strong emphasis on eval frameworks, CI integration, and self-improvement loops.
Responsibilities
- Building and maintaining the eval framework that scores voice agent quality across transcription, LLM reasoning, tool use, TTS, and full-conversation outcomes
- Design voice agent behavior: system prompts, tool use, conversation flow, error recovery, and guardrails for real-time interactions
- Drive STT and TTS accuracy improvements by comparing providers, tuning configurations, and running rigorous A/B experiments the team can act on
- Drive TTS quality improvements: voice selection, latency vs. fidelity tradeoffs, prosody, edge cases
- Curate and grow evaluation datasets, including hard-case mining from production traffic
- Build benchmarks to run against any new model in days, run a red-team pipeline that probes jailbreaks, hallucinations, and compliance
- Partner with backend engineers to wire eval signals into CI to catch regressions before shipping
- Wire eval signals into CI so regressions block merges, and build self-improvement loops where hard production cases auto-feed the eval set and prompts optimize themselves over time
Qualifications
- ML engineering experience shipping production systems
- Strong Python and a working ML stack (PyTorch, Huggingface, pandas, scikit-learn)
- Hands-on experience designing LLM-based agents: prompting, tool/function calling, multi-turn state, structured outputs
- Hands-on experience building evals or eval frameworks for ML, LLM, or voice systems (LLM-as-judge eval pipelines and understanding failure modes)
- Practical experience with ASR/STT comparing providers, fine-tuning, or running open models like Whisper
- Practical experience with TTS systems (ElevenLabs or open models)
- Comfortable working with audio data: sample rates, codecs, noise, alignment
Skills
Required:eval framework design and implementationLLM-based agent designASR/STTTTS
Last updated Sep 9, 2026. Interested? Apply on the company site.
About Infer
Personalized AI Voice Agents for lead qualification for insurance &…
artificial-intelligencegenerative-aib2bapi
More open roles at Infer
- Engineering Lead5mo agoHybrid (Bengaluru, KA, IN / Bengaluru, Karnataka, IN) · Engineering
- Senior backend Engineer5mo agoHybrid (Bengaluru) · Engineering
- Forward Deployed Engineer5mo agoHybrid (San Francisco, CA) · Engineering