← All roles

Senior AI and HPC Observability Engineer

NVIDIA· Engineer, Sys SW
Apply on company site
Santa Clara, CA, Seattle, WA$152k - $241.5kFull-timePosted 7mo ago

About the role

Role focused on designing, building, and operating high-throughput observability and telemetry platforms for AI/HPC workloads within NVIDIA's Managed AI Superclusters (MARS) team. Emphasis on distributed systems, production-grade software, and operational excellence to enable researchers and engineers working on advanced AI/ML workloads.

Responsibilities

  • Design and scale observability platforms handling high-volume metrics, logs, and traces across distributed environments
  • Build high-performance backend services for telemetry ingestion, processing, and routing
  • Develop and extend OpenTelemetry collectors, processors, exporters, and instrumentation libraries
  • Build and optimize metrics pipelines using large-scale time-series storage systems
  • Design and operate real-time and batch telemetry pipelines using streaming and distributed data technologies
  • Improve platform reliability, performance, and cost efficiency through tuning, capacity planning, and system optimization
  • Develop monitoring, alerting, and service reliability frameworks to ensure platform health and performance
  • Collaborate with platform engineering, infrastructure, and site reliability teams to deliver production-grade observability solutions

Qualifications

  • Bachelor’s degree in Computer Science, Computer Engineering, or related field or equivalent experience
  • 5+ years of experience building backend or distributed systems in production environments
  • Strong programming skills in Python, Go, or Java, with experience developing production-quality software
  • Hands-on experience with modern observability architectures, including metrics, logs, and traces
  • Solid experience with PromQL and time-series data systems
  • Experience building or operating distributed data pipelines using technologies such as Kafka, Spark, or Flink
  • Experience working with Kubernetes and cloud-native infrastructure
  • Strong understanding of distributed systems, concurrency, and fault-tolerant system design. Strong debugging, performance tuning, and production operations skills

Skills

Required:PythonGoOpenTelemetryPromQLKafka
Nice to have:JavaSpark

Benefits

  • equity and benefits (see NVIDIA benefits)
Last updated Jul 3, 2026. Interested? Apply on the company site.

About NVIDIA

World Leader in AI Computing

NVIDIA invents the GPU and drives advances in AI, HPC, gaming, creative design, autonomous vehicles, and robotics.

AIautonomous vehiclesgaminggpugraphics cardhigh performance computingliquid-ai-recruiting-targetllm
Company profileWebsite

More open roles at NVIDIA