Member of Technical Staff, Multimodal Vision
San Jose, CA$180k - $450kfull-timePosted 7mo ago
About the role
Role focused on developing multimodal AI experiences (text, vision, video, audio) across data, modeling, training, serving, and product integration. Involves mid-training/post-training research, RL-based optimization, and building scalable multimodal training infrastructure.
Responsibilities
- Drive mid-training and post-training research for multimodal foundation models — continued/mid-training on large-scale interleaved vision-text and video-audio corpora, supervised fine-tuning, and reinforcement learning to advance image, video, and video-audio understanding.
- Own the RL stack for multimodal post-training: reward modeling, preference optimization and RLVR-style verifiable rewards, policy optimization (PPO / GRPO / DPO and variants), rollout and sampling strategies, and mitigation of reward hacking and capability regression.
- Build multimodal evaluation frameworks and internal benchmarks covering perception, reasoning, instruction following, long-video and video-audio understanding, robustness, and RL-specific regressions; turn eval signal into the next training iteration.
- Build and optimize multimodal training infrastructure — distributed training and RL rollout systems, long-context and multi-stream (vision + audio) data loading, throughput and memory efficiency, checkpointing and experiment tooling — for scalability and production deployment.
Qualifications
- Proven track record of advancing multimodal or vision-language models through innovations in mid-training, post-training, data, or training systems.
- Hands-on experience with post-training methods at scale — SFT, RLHF/RLAIF, RLVR, reward modeling, and preference optimization — ideally applied to multimodal models.
- Strong experience in image understanding, video modeling, or multimodal learning.
- Strong background in data-driven experimentation, evaluation design, and iterative model development.
Skills
Required:Multimodal model developmentPost-training methods at scaleReinforcement learning for multimodal modelsImage understanding and video modeling
Last updated Aug 27, 2026. Interested? Apply on the company site.
About Hark
building the most advanced personal intelligence in the world.
At Hark, we are building the most advanced personal intelligence in the world.
personal intelligenceartificial intelligence
More open roles at Hark
- Warehouse Associate5d agoSan Jose, CA · Supply Chain
- Industrial Designer1w agoHybrid (San Jose, CA) · Design
- Member of Technical Staff, LLM & Production1w agoSan Jose · AI Foundation Models
- Member of Technical Staff, Hark Desktop2w agoHybrid (San Jose) · Product Engineering
- Application Security Engineer2w agoSan Jose, CA · Privacy & Security