Member of Technical Staff, Evals & Post-Training Product
San Mateo, CA$175k - $220kFull-timePosted 11mo ago
About the role
Role focused on defining and shipping evals and post-training workflows to connect evaluation and fine-tuning into a continuous loop, including internal eval tooling, open-source Eval Protocol SDK, and fine-tuning experiences across SFT/RFT.
Responsibilities
- Build internal eval workflows: Design and scale evaluation tooling used by internal teams to measure model quality, compare model changes, and inform post-training decisions.
- Own fine-tuning product experiences: Build and improve user-facing product workflows for post-training, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
- Work closely with users: Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs, support high-priority engagements, triage issues, and convert bespoke workflows into productized solutions.
Qualifications
- 1 - 7 years of software engineering experience (We are hiring at multiple levels for this role).
- Hands-on experience with LLM evaluations and/or post-training methods: How to design useful evals and use their results to guide model improvement.
- Product Engineering Skills: The ability to work across backend systems and developer-facing product surfaces. Comfortable shipping full-stack features when needed.
- Understanding of the GenAI Lifecycle: You understand the end-to-end workflow—from prompting a base model to curating a dataset, fine-tuning, and productionizing agents—and how these steps interconnect.
- User-Centric Mindset: Willing to talk to users, triage GitHub issues for open-source projects, and build products from scratch to serve emerging needs.
Skills
Required:Software engineeringLLM evaluationsPost-training methodsProduct engineering
Benefits
- Total compensation includes meaningful equity, competitive salary, and comprehensive benefits package.
Last updated Jul 29, 2026. Interested? Apply on the company site.
About Fireworks AI
Fastest Inference for Generative AI
Use state-of-the-art, open-source LLMs and image models at blazing fast speed, or fine-tune and deploy your own at no additional cost with Fireworks AI!
Generative AIImageLLMMultimediaSearchai-infrastructure-and-developmentliquid-ai-recruiting-targetllm
More open roles at Fireworks AI
- Technical Account Manager, Enterprise2mo agoHybrid (London, UK; New York, NY; San Mateo, CA) · Go To Market
- AI Field Engineer, Singapore2mo agoHybrid (Singapore) · Go To Market
- Enterprise Account Executive, EMEA2mo agoHybrid (London, UK) · Go To Market
- Applied Machine Learning Engineer, EMEA2mo agoHybrid (London) · Engineering
- Enterprise Account Executive, Singapore2mo agoSingapore · Go To Market