Research Engineer - Model Evaluation & MLOps
Sciforium· Engineering
San Francisco, CA$155k - $200k + equityFull-timePosted 1mo ago
About the role
Role focused on building tools and infrastructure to evaluate, deploy, and operate multimodal foundation models with automated benchmarking, CI/CD-ready workflows, and robust MLOps practices.
Responsibilities
- Rapidly integrate new internal and open-weight language and multimodal models into GPU evaluation and inference environments.
- Build automated benchmarks for model quality and systems performance, including latency, throughput, and memory usage.
- Create standardized, reproducible comparisons across Sciforium models, external baselines, and runtime configurations.
- Build and maintain experiment tracking, model registry, and versioning for models, datasets, and evaluation configurations.
- Automate the path from research checkpoints to validated deployments through CI/CD and reproducible workflows.
- Monitor model quality and systems performance, and diagnose failures or regressions across model and deployment pipelines.
- Build reusable tools that help researchers launch evaluations, compare experiments, and reproduce results.
- Profile end-to-end model workloads and collaborate with distributed systems, inference, and GPU kernel engineers on deeper performance issues.
Qualifications
- 2+ years of professional ML or software engineering experience, including work on production ML systems, ML platforms, or MLOps infrastructure.
- Strong Python and software engineering skills, with experience building reliable production systems.
- Hands-on experience with PyTorch, TensorFlow, or JAX and a good understanding of modern language or multimodal model architectures.
- Experience with model evaluation or benchmarking and core model lifecycle workflows such as experiment tracking, versioning, deployment, or monitoring.
- Experience running, benchmarking, and debugging models with one or more GPU inference runtimes, such as vLLM, SGLang, TensorRT-LLM, or equivalent, in containerized cloud or on-premises environments.
- Ability to document systems clearly and collaborate across research, infrastructure, and product engineering teams.
- MS or PhD in Computer Science, Computer Engineering, Machine Learning, or a related technical field, or equivalent practical experience.
Skills
Required:PythonMachine Learning SystemsModel Evaluation & BenchmarkingExperiment Tracking & Model RegistryCI/CD for MLGPU Inference & Runtime
Benefits
- Medical, dental, and vision insurance
- 401k plan
- Daily lunch, snacks, and beverages
- Flexible time off
- Competitive salary and equity
Last updated Sep 4, 2026. Interested? Apply on the company site.
About Sciforium
Byte-Native AI Models & Infrastructure
Sciforium is pioneering the future of AI infrastructure and research. We're developing byte-native multimodal foundation models while delivering serverless LLM serving at a fraction of traditional costs
More open roles at Sciforium
- Office Manager2d agoSan Francisco, CA · Operations
- Pre-training Research Engineer1mo agoSan Francisco, CA · Engineering
- GPU Cluster Engineer, Networking2mo agoSan Francisco, CA · Engineering
- GPU Cluster Engineer, Systems & Platform2mo agoSan Francisco, CA · Engineering
- GPU Cluster Engineer, Hardware Operations5mo agoSan Jose, CA, San Francisco, CA · Engineering