Software Engineer, LLM Infrastructure
Hybrid (New York, NY), Hybrid (San Mateo, CA)$175k - $220kPosted 11mo ago
About the role
Software Engineer on the AI Infrastructure team building core systems powering Fireworks AI’s generative platform, focusing on reliability, performance, quality, and availability of infrastructure that bridges customers and the proprietary Fireworks inference engine.
Responsibilities
- Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
- Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
- Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
- Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
- Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
- Participate in code reviews, technical discussions, and continuous integration and deployment processes
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
- 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
- Strong programming skills in Python, Go, or a similar language
- Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)
- Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)
Skills
Required:Software engineering in infrastructure or ML systemsPythonGoML infrastructure toolingLLM knowledge
Last updated Jul 29, 2026. Interested? Apply on the company site.
About Fireworks AI
Fastest Inference for Generative AI
Use state-of-the-art, open-source LLMs and image models at blazing fast speed, or fine-tune and deploy your own at no additional cost with Fireworks AI!
Generative AIImageLLMMultimediaSearchai-infrastructure-and-developmentliquid-ai-recruiting-targetllm
More open roles at Fireworks AI
- Technical Account Manager, Enterprise2mo agoHybrid (London, UK; New York, NY; San Mateo, CA) · Go To Market
- AI Field Engineer, Singapore2mo agoHybrid (Singapore) · Go To Market
- Enterprise Account Executive, EMEA2mo agoHybrid (London, UK) · Go To Market
- Applied Machine Learning Engineer, EMEA2mo agoHybrid (London) · Engineering
- Enterprise Account Executive, Singapore2mo agoSingapore · Go To Market