Embedded AI Engineer, On-Device Models
About the role
Embedded AI Engineer focused on writing and optimizing custom kernels and operators for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs to run Deepgram’s speech models on edge/hardware-constrained platforms.
Responsibilities
- Write and optimize custom kernels and operators (C, C++, Rust, and platform assembly or intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs where the vendor's standard operator set is insufficient for Deepgram models.
- Own target-side optimization: collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation to meet latency, memory, power, and thermal budgets.
- Integrate with vendor NPU/DSP toolchains and edge inference runtimes, and extend them with custom operators when the graph doesn't map cleanly.
- Deliver kernels and runtime components as reusable building blocks that Applied ML Engineers can target when adapting models, with clear interfaces and documented constraints.
- Build performance-critical runtime code for embedded environments, including embedded Linux, bare-metal, and RTOS targets.
- Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization, and catch regressions before they ship.
- Partner with silicon and platform vendors on SDK integration and low-level performance tuning for new chipsets and reference platforms.
- Feed hardware constraints back to Applied ML and Research so model designs are easier to land on constrained targets.
Qualifications
- Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices.
- Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
- Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation.
- Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
- A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance.
- Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development.
- Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly.
Skills
Benefits
- -
About Deepgram
Building foundational AI for speech transcription and understanding.
Deepgram is a foundational AI company on a mission to understand human language. We give any developer access to the most advanced speech AI transcription and understanding with just an API call. Our models deliver the fastest, most accurate transcription alongside contextual features like summarization, sentiment analysis, and topic detection. Beyond that, developers can: 🔊 Process live-streaming or pre-recorded audio 🌎 Transcribe in dozens of languages ⚙️ Train custom models for unique use cases 🔑 Access deep NLU with a unified API 💻 Build in any programming language with our SDKs ✅ Deploy on-prem or on DG’s managed cloud 📈 Get scalable GPU infra for training and inference Deepgram is a proud NVIDIA partner and Y Combinator company, and we recently completed a $72M Series B to define the future of AI Speech Understanding, making us the most-funded speech AI company at its stage.
More open roles at Deepgram
- Research Staff, Applied Voice AI4d agoRemote · Research
- Revenue Operations Analyst4d agoRemote · Customer Success
- Account Executive (German Speaking)4d agoSwitzerland · Sales - AE
- Director, GTM Engineering4d agoRemote · Customer Success
- Account Executive (India)4d agoRemote · Sales - AE