Software Engineer - Hosted Model Infrastructure
Palantir· Dev
Hybrid (Washington, D.C.)$145k - $200kFull-timePosted 4mo ago
About the role
Software engineering role focused on deploying and maintaining hosted model infrastructure for production AI models across diverse and secure environments, including air-gapped networks and edge deployments.
Responsibilities
- Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
- Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
- Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
- Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
- Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
- Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
- Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
- Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems
Qualifications
- 4+ years of professional software engineering experience building and operating production systems
- Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field
- Strong coding skills with proficiency in Java, C++, Python, Rust, or similar languages; familiarity with the Python ML ecosystem is valuable
- Experience with containers, Kubernetes, and deploying backend services in production environments
- Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality
Skills
Required:JavaPythonKubernetes
Nice to have:C++Rust
Benefits
- Employees (and their eligible dependents) can enroll in medical, dental, and vision insurance as well as voluntary life insurance
- Employees are automatically covered by Palantir’s basic life, AD&D and disability insurance
- Commuter benefits
- Relocation assistance
- Take what you need paid time off, not accrual based
- 2 weeks paid time off built into the end of each year (subject to team and business needs)
- 10 paid holidays throughout the calendar year
- Supportive leave of absence program including time off for military service and medical events
- Paid leave for new parents and subsidized back-up care for all parents
- Fertility and family building benefits including but not limited to adoption, surrogacy, and preservation
- Stipend to help with expenses that come with a new child
- Employees can enroll in Palantir’s 401k plan
Last updated Oct 21, 2026. Interested? Apply on the company site.
More open roles at Palantir
- Physical Security Manager4d agoPalo Alto, CA · Global Security and Investigations
- Compliance Analyst4d agoHybrid (Washington, D.C.) · Information Security
- Compliance Analyst4d agoHybrid (New York, NY) · Information Security
- Software Engineer, Internship - Query Engines4d agoNew York, NY · Dev
- Software Engineer, Internship - Query Engines4d agoLondon · Dev