← All roles

Software Engineer, GPU Inference

Cerebras· Software Engineering
Apply on company site
Sunnyvale, CA, Toronto, CANFull-timePosted 3mo ago

About the role

Software Engineer role focused on productionizing and optimizing Cerebras' GPU serving stack for disaggregated AI inference, working across custom inference APIs, vLLM, AMD ROCm, and rack-scale AMD GPU infrastructure to improve reliability, correctness, and performance.

Responsibilities

  • Productionize the GPU inference stack. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.
  • Own GPU operational readiness. Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet. Build automation that makes driver, firmware, runtime, model, and container compatibility explicit and reproducible.
  • Drive reliability in production. Define service-level indicators and objectives for GPU-backed inference. Improve fault isolation, graceful degradation, automated recovery, incident response, and post-incident remediation across the serving stack.
  • Improve inference performance. Profile and optimize time to first token, request throughput, tokens per second per GPU, tail latency, GPU utilization, memory efficiency, and rack-level capacity under representative production workloads.
  • Optimize model-serving behavior. Tune and improve scheduling, continuous batching, prefix caching, KV-cache management, tensor and expert parallelism, request admission, quantization, graph execution, and distributed communication.
  • Debug across system layers. Diagnose complex failures and performance regressions across application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware, networking, and hardware.
  • Ensure numerical correctness. Build validation and regression infrastructure for model quality, numerical accuracy, precision changes, quantization, determinism, and compatibility across software and hardware releases.
  • Build performance and correctness infrastructure. Develop representative benchmarks, workload replay tools, profiling automation, release qualification, dashboards, and regression gates. Turn one-off investigations into repeatable engineering systems.

Qualifications

  • 5+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
  • Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
  • Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.
  • Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.
  • Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
  • Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.
  • Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services in production.
  • Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
  • Strong communication and technical leadership skills, with a demonstrated ability to drive ambiguous cross-functional projects to completion.
  • Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Skills

Required:C++PythonGPU inference systemsvLLM / model-serving frameworksLinux / containers / KubernetesGPU performance tuningDistributed systems debugging

Benefits

  • (Not explicitly enumerated in the description; included only if present in the provided text)
Last updated Oct 11, 2026. Interested? Apply on the company site.

About Cerebras

Cerebras is the go-to platform for fast and effortless AI training. Learn more at cerebras.ai.

Cerebras is the go-to platform for fast and effortless AI training.

AICerebras Wafer-Scale Enginechipshardwareinferenceliquid-ai-recruiting-targetmultimodalsetter-30
Company profileWebsite

More open roles at Cerebras