← All roles

Software Engineer, Inference Runtime

LM Studio· Engineering
Apply on company site
Hybrid (New York City)$150k - $350kFull-timePosted 2mo ago

About the role

Inference Runtime Software Engineer to push forward LM Studio's on-device and cloud inference stack, integrate new inference engines, bring up open-weight models, and optimize execution across CPU/GPU targets; contribute improvements to open-source projects.

Responsibilities

  • Maintain and push forward our inference stack on-device and in the cloud
  • Bring up new model architectures and multimodal models
  • Improve latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimes
  • Build runtime capabilities for model loading, batching, scheduling, caching, and distributed execution
  • Benchmark and diagnose correctness and performance problems across the inference stack
  • Contribute upstream to open-source projects such as llama.cpp and MLX

Qualifications

  • Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructure
  • Strong programming ability in Python and C++
  • Deep understanding of transformer architectures and the mechanics of model inference
  • Experience profiling CPU or GPU workloads and reasoning about compute, memory, synchronization, and data movement
  • Experience with PyTorch and inference systems such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM
  • Strong debugging instincts across model code, runtime internals, operating systems, and CPU or GPU execution
  • Takes personal responsibility for the correctness and performance of their work

Benefits

  • Competitive salary and equity grants
  • Great medical, vision, dental healthcare plans
  • Catered team lunch / expensed dinners in the office
  • Flexible PTO
  • Flexible WFH
  • Sun-drenched office in SoHo in NYC
Last updated Aug 12, 2026. Interested? Apply on the company site.

About LM Studio

Local AI, on Your Computer.

Run local AI models like gpt-oss, Llama, Gemma, Qwen, and DeepSeek privately on your computer.

local AIlm airun local AI modelsLM Studioon-device AIgpt-ossQwenLlama
Company profileWebsite

More open roles at LM Studio