Software Engineer - Hosted Model Infrastructure
Palantir· Dev
Hybrid (Palo Alto, CA)$145k - $200kFull-timePosted 4mo ago
About the role
Software engineering role focusing on hosting and serving ML models in production across diverse environments, with end-to-end ownership of infrastructure, deployment pipelines, and observability for multi-tenant, GPU-constrained setups.
Responsibilities
- Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
- Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
- Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
- Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
- Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
- Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
- Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
- Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems
Qualifications
- 4+ years of professional software engineering experience building and operating production systems
- Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field
- Strong coding skills with proficiency in Java, C++, Python, Rust, or similar languages; familiarity with Python ML ecosystem
- Experience with containers, Kubernetes, and deploying backend services in production environments
- Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and maintaining high quality
Skills
Required:ProgrammingContainers and orchestrationModel serving and inferenceObservability and debugging
Benefits
- Medical, dental, and vision insurance for employees and eligible dependents
- Basic life, AD&D and disability insurance
- Commuter benefits
- Relocation assistance
- Paid time off with set yearly allotment and holidays
- Supportive leave of absence program including military and medical leave
- Paid parental leave and subsidized back-up care
- Fertility and family-building benefits
- Child-related expenses stipend
- Enrollment in Palantir’s 401k plan
Last updated Oct 21, 2026. Interested? Apply on the company site.
More open roles at Palantir
- Physical Security Manager4d agoPalo Alto, CA · Global Security and Investigations
- Compliance Analyst4d agoHybrid (Washington, D.C.) · Information Security
- Compliance Analyst4d agoHybrid (New York, NY) · Information Security
- Software Engineer, Internship - Query Engines4d agoNew York, NY · Dev
- Software Engineer, Internship - Query Engines4d agoLondon · Dev