← All roles

Model Serving Engineer

Fundamental· Engineering
Apply on company site
RemoteFull-timePosted 4mo ago

About the role

Role focused on turning the NEXUS Large Tabular Model into a reliable, scalable production system. Works across the full inference stack, optimizing Python runtime and distributed serving infrastructure, collaborating with research and infrastructure teams to improve throughput, latency, and resource efficiency.

Responsibilities

  • Optimize Python inference code for performance under real concurrency constraints, including GIL contention, multi-threading, multiprocessing, async execution, and long-running production workloads
  • Work closely with research to understand model internals and support the continuous evolution of the architecture, especially around complex and non-obvious computational behavior under production load
  • Collaborate with research and infrastructure teams to reason about hardware utilization and serving tradeoffs across GPU, CPU, memory, networking, batching, and concurrency
  • Define and evolve the architecture behind our distributed inference and asynchronous execution stack, including orchestration, worker coordination, and end-to-end concurrency patterns
  • Own the Triton serving layer for NEXUS, including how models are packaged, configured, and executed as part of our production inference pipeline
  • Build observability and performance tooling across the serving stack, and use production metrics to drive tuning decisions around latency, throughput, and resource efficiency
  • Solve cross-cutting serving challenges that emerge from deploying the same model across environments with very different scale, isolation, and reliability constraints
  • Evaluate and integrate new inference runtimes, serving strategies, and infrastructure approaches as the model ecosystem evolves

Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent practical experience)
  • 5+ years of experience in model serving, ML infrastructure, or a closely related backend engineering role
  • Deep expertise in Python concurrency, including GIL behavior, multi-threading, thread safety, multiprocessing
  • Experience building asynchronous and message-driven systems
  • High-performance, large-scale distributed systems
  • Ability to read and reason about ML model implementations at a computational level, including compute behavior, batching, memory usage, and inference characteristics
  • Experience profiling and optimizing performance across CPU, memory, I/O, and ideally GPU workloads, and translating findings into architectural improvements

Skills

Required:Python concurrencyAsynchronous programmingDistributed systemsML model understanding

Benefits

  • Competitive compensation with salary and equity
  • Comprehensive health coverage for you and your dependents
  • Paid parental leave for all new parents, inclusive of adoptive and surrogate journeys
  • Relocation support for employees moving to join the team in one of our office locations
  • A mission-driven, low-ego culture that values diversity of thought, ownership, and bias toward action
Last updated Sep 22, 2026. Interested? Apply on the company site.

About Fundamental

The Power to Predict

The Power to Predict. Our most powerful Large Tabular Model. Pre-trained on billions of real-world enterprise tables, our model knows which patterns drive which outcomes.

NEXUSOur World’s Most Powerful Large Tabular ModelPre-trained on billions of real-world enterprise tablesEnterprise AIData AnalyticsTime SeriesConfidential ComputingCloud
Company profileWebsite

More open roles at Fundamental