Member of Technical Staff, ML Inference Engineering
Sanas· Engineering
Palo Alto, CAFull-timePosted 2mo ago
About the role
Hands-on ML inference engineering role focused on building and optimizing on-premise, low-latency inference infrastructure for real-time speech and language models, with emphasis on core infrastructure decisions, performance optimizations, and scalable model serving.
Responsibilities
- Implement custom kernels and low-level optimizations to push GPU compute limits.
- Apply graph optimization, operator fusion, and hardware-specific code generation at the ML compiler level.
- Profile, analyze, and resolve deep system bottlenecks to improve latency, throughput, and memory efficiency.
- Drive model-level execution improvements, including mixed precision and quantization strategies.
- Write and optimize custom inference server backends to handle complex business logic, scripting, and state management.
- Own and scale overarching model serving infrastructure across multi-GPU, multi-node deployments to meet strict on-premise latency budgets.
- Build robust, fault-tolerant runtime services with comprehensive performance benchmarking and monitoring.
Qualifications
- 8+ years of experience writing high-quality, high-performance code, including at least 5 years focused on machine learning systems.
- Experience with one of C/C++/Rust and Python.
- Deep familiarity with modern NVIDIA GPU architectures (e.g., Ada Lovelace, Blackwell), CUDA, and low-level system profiling.
- Hands-on experience with ML compilers and optimization frameworks (e.g., Apache TVM, TorchInductor/Dynamo, TensorRT, Triton).
- Experience building or extending model serving infrastructure, specifically writing custom C++ backends for Triton Inference Server (or similar serving engines).
- Fluency in the AI serving stack, from kernels and quantization up to schedulers, state management, and autoscaling.
- A record of shipping research or systems that other people build on, whether in a lab or in industry.
Skills
Required:C/C++/RustPythonNVIDIA GPU architectures (Ada Lovelace, Blackwell) and CUDAML compilers and optimization frameworksModel serving infrastructure / Triton
Last updated Sep 22, 2026. Interested? Apply on the company site.
About Sanas
If it's Speech, it's Sanas.
Sanas is the real-time speech platform that removes communication barriers in real time, making conversations between people - and conversations with systems - clear, natural, and scalable.
Accent TranslationLanguage TranslationSpeech EnhancementSpeech IntelligenceDeveloper PlatformHealthcareFinancial ServicesRetail
More open roles at Sanas
- Head of Developer Platform & SDK6d agoPalo Alto, CA · Engineering
- Senior Data & BI Developer6d agoBengaluru · General & Administration
- Staff Software Engineer6d agoBengaluru · Engineering
- Senior Solutions Engineer6d agoHybrid (Boston, MA) · Sales Engineering
- Head of GTM And Strategic Finance1w agoPalo Alto, CA · General & Administration