← All roles

Senior AI Compute Infrastructure Engineer

Kraken· Engineering
Apply on company site
RemoteFull-timePosted 5mo ago

About the role

Kraken is building a dedicated AI Compute and Infrastructure team to power model training, inference, evaluation, and experimentation across the exchange. This role focuses on GPU/accelerator infrastructure, scheduling, observability, and cost-efficient compute at scale to enable in-house AI workloads with high reliability and performance.

Responsibilities

  • Own and operate GPU and accelerator clusters used for training, inference, evaluation, and experimentation, including drivers, runtimes, kernels, device plugins, node configuration, scheduling primitives, and workload isolation
  • Design infrastructure that enables Kraken teams to run models locally on GPUs to reduce external provider dependency and contain compute costs
  • Build and improve scheduling, orchestration, placement, quota management, and utilization systems across heterogeneous accelerator environments
  • Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using frameworks such as vLLM, Triton Inference Server, TensorRT, or equivalent serving stacks
  • Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch inference, online inference, deployment, and production debugging workflows
  • Build observability for GPU utilization, memory pressure, queue depth, saturation, token throughput, request latency, failed workloads, capacity pressure, and spend
  • Drive reliability, incident response, alerting, runbooks, and post-incident improvements for always-on AI compute infrastructure
  • Evaluate and integrate new hardware, cloud instance families, specialized accelerators, runtimes, schedulers, and serving frameworks as the AI infrastructure landscape evolves
  • Build tooling that makes GPU usage visible, accountable, and easier for internal teams to consume without needing to become infrastructure experts
  • Contribute to long-term architecture decisions that balance performance, cost efficiency, scalability, operational simplicity, and production safety

Qualifications

  • 5+ years of infrastructure engineering experience, with significant time spent on GPU compute, ML infrastructure, distributed systems, high-performance computing, or large-scale production platforms
  • Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments, including scheduling, orchestration, utilization monitoring, and cost optimization
  • Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging
  • Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems
  • Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows
  • Practical understanding of performance tradeoffs across batching, concurrency, memory usage, GPU utilization, model size, latency, throughput, availability, and cost
  • Track record of optimizing compute costs while maintaining clear performance, reliability, and availability expectations
  • Experience building observable systems with useful metrics, logs, traces, dashboards, alerts, and incident workflows
  • Comfortable working in high-stakes, always-on environments where uptime, throughput, correctness, and operational discipline are critical
  • Clear communicator who can translate infrastructure tradeoffs for researchers, product teams, platform engineers, security stakeholders, and engineering leadership

Skills

Required:GPU compute infrastructureLinux systems engineeringML serving frameworksPythonObservability and incident responseCommunication

Benefits

  • Competitive compensation
  • Discretionary bonus
  • Remote-friendly culture
  • Global, diverse team
  • Open, collaborative environment for AI/machine learning work
Last updated Jul 15, 2026. Interested? Apply on the company site.

About Kraken

setter-30setter-30-2026-q1
Company profileWebsite

More open roles at Kraken