← All roles

Inference Infrastructure Architect (Remote)

Telnyx· Product
Apply on company site
Hybrid (Mainland China)Senior / StaffPosted 1mo ago

About the role

Senior/Staff remote role for founding China team focused on turning Telnyx's private AI inference fleet into a high-throughput, cost-efficient, enterprise-ready platform with serverless and dedicated deployments, spanning bare-metal to OpenAI-compatible endpoints.

Responsibilities

  • Operate and expand the fleet efficiently. Maximize useful inference throughput per GPU-dollar while meeting latency and reliability SLOs, and continuously reduce cost per token.
  • Make the product run on it. Serverless inference for the open-weight catalog, and dedicated, tuned model deployments for enterprises.
  • Build and maintain: serverless serving pools (vLLM / SGLang) with continuous batching, prefix caching, low-precision serving, and MoE expert parallelism; the fleet layer (llm-d / NVIDIA Dynamo on Kubernetes Gateway API) with KV-cache-aware routing and multi-LoRA sharing; the platform (Kubernetes on bare metal, GPU Operator, topology-aware scheduling, Kueue, Kata Containers, OpenStack Ironic); weight logistics (Dragonfly, safetensors streaming), auto-scaling; the dedicated tier (per-tenant pools, latency SLOs, private networking); observability and economics (Prometheus/OpenTelemetry, roofline-based capacity planning).
  • Experience we especially value includes operating inference platforms at large scale, contributing to or using vLLM/SGLang/llm-d/NVIDIA Dynamo/Ray/KubeRay, and involvement in CNCF/OpenInfra communities.
  • Nice to have includes fine-tuning/RL infra, real-time voice latency work, and multi-region deployments.

Qualifications

  • You have owned production LLM serving under meaningful traffic and latency constraints and can explain the architecture, bottlenecks, changes, and measurable improvements.
  • Kubernetes on GPU fleets, end-to-end GPU Operator, device plugins, topology-aware placement, gang scheduling, GitOps, and debuggability of pod placement.
  • Deep operational command of vLLM or SGLang: deployment, tuning, upgrades (parallelism, quantization, batching, KV-cache, prefix caching, disaggregation).
  • Performance engineering at the system level: interpreting engine/DCGM metrics, roofline reasoning, scaling with quantified metrics.
  • Proficiency in Python and Go for automation; strong Linux/networking/storage depth; not required to write CUDA but able to identify CUDA-related issues and route upstream.
  • Ability to work with the community in English and Chinese, and to author runbooks and design docs.

Skills

Required:Production LLM serving at scaleKubernetes on GPU fleetsvLLM / SGLangSystem performance engineeringPythonGo

Benefits

  • Open-source first. Upstream contribution and conference travel support.
  • The newest hardware, own the platform from bare metal to customer endpoint.
  • Founding China team with influence on hiring and team patterns.
  • Remote work in mainland China with async-friendly global team; visa sponsorship available for moves to the Netherlands, the United States, Ireland or Saudi Arabia.
Last updated Sep 24, 2026. Interested? Apply on the company site.
Telnyx logo

About Telnyx

Telnyx owns the full stack from carrier network to AI inference, so there's nothing between your agent and the person on the other end. No handoffs. No lag. Just voice, messaging, and AI that keeps up.

Carrier-owned global communications platform for voice AI agents, SIP trunking, programmable voice, SMS/MMS, and AI inference. Single vendor for the full voice-AI stack on a Tier-1 network across 40+ countries.

Cloud communicationsGlobal eSIMMessagingPhone number APIPrivate networkProgrammable SMSSIP trunkingVoIP
Company profileWebsite

More open roles at Telnyx