← All roles

Software Engineer - Model Performance Systems

Baseten· EPD
Apply on company site
Hybrid (San Francisco), Hybrid (Toronto), Hybrid (New York), Hybrid (Montreal), Hybrid (Seattle)$165k - $330k + equityFull-timePosted 9mo ago

About the role

Specialized software engineering role at the intersection of high-performance computing and Large Language Model engineering. Owns the roadmap and delivery of a speedometer/diagnostic suite for AI infrastructure, including benchmarking, tooling, and optimization automation.

Responsibilities

  • Benchmarking: Evaluate, run and automate standard LLM quality benchmarks (GSM8K, MMLU) alongside custom performance suites for specific workloads (e.g., long-context window, KV cache reuse, disaggregated serving).
  • DevEx Improvement: Develop and maintain internal GPU-enabled development environments (similar to GitHub Codespaces).
  • Tool Development: Build and contribute to open-source tools such as InferenceMAX and genai-bench to automate model evaluation, benchmarking and analysis.
  • System Profiling: Use profilers like PyTorch Profiler, NVIDIA Nsight Systems and py-spy to collect performance profiles, identify bottlenecks, and debug the compute/networking stack.
  • Monitoring & Observability: Develop real-time dashboards and alerts to monitor system health, model startup times, and runtime performance.
  • Continuous Integration: Automate performance testing via CI/CD pipelines to catch regressions and build release workflow automation for the model runtimes stack.
  • Optimization Automation: Build tools to find the Pareto frontier—identifying the absolute best configuration (latency vs. cost vs. quality) for a given model and workload.

Qualifications

  • A Love for Systems & Hardware: You understand GPU memory subsystems, InfiniBand, and data movement across a cluster.
  • An Automation Mindset: You script repetitive tasks and enjoy stress-testing and fuzzy testing to find breaking points.
  • Mathematical Curiosity: Interest in the math of Transformers, FLOPs, and memory requirements.
  • Technical Toolkit: Familiarity with Python; eagerness to master NVIDIA software stack; C++ familiarity is a plus.

Skills

Required:PythonGPU and HPC conceptsProfiling toolsAutomation and scripting
Nice to have:C++

Benefits

  • Competitive compensation, including meaningful equity
  • (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
  • Flexible PTO policy including company wide Winter Break (offices closed from Christmas Eve to New Year's Day)
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • (U.S. only) Company-facilitated 401(k)
  • Exposure to a variety of ML startups, offering learning and networking opportunities
Last updated Oct 21, 2026. Interested? Apply on the company site.
Baseten logo

About Baseten

Inference is everything

Effortlessly serve optimized open source & custom models on the fastest, most reliable model delivery network.

AICloud InfrastructureDeveloper ToolsInference PlatformMachine LearningModel DeploymentModel Runtimesai-infrastructure-and-development
Company profileWebsite

More open roles at Baseten