← All roles

AI Model Policy Trainer, Image Evaluation - Seattle Onsite

Handshake· HAI Delivery Ops
Apply on company site
Seattle, WA$36/hr - $72/hrFull-timePosted 1mo ago

About the role

AI Image Evaluator role focused on evaluating generated images against prompts, judging adherence, quality, and policy compliance to train and calibrate AI models.

Responsibilities

  • Evaluate generated images against their prompts for adherence, composition, realism, style consistency, and technical defects; maintain accuracy and consistency across repeated evaluations
  • Compare images side by side and select the stronger one with a clear, evidence-based rationale
  • Classify images against customer content policies covering sexual content, violence, hate symbols, real-person likeness, intellectual property, and depictions of minors
  • Select the most defensible classification when an image is genuinely ambiguous, and write concise rationales that cite rubric language and specific visual details
  • Distinguish "I do not like this" from "this fails the prompt" from "this violates policy," and keep those judgments separate while applying customer policy consistently
  • Write and refine prompts that probe where a model's quality or safety behavior breaks down
  • Identify rubric gaps, contradictions, and emerging edge cases, and raise them with project leads and policy teams
  • Participate actively in calibration discussions; challenge interpretations respectfully and update your judgment when stronger reasoning emerges

Qualifications

  • A trained eye from photography, illustration, concept art, art direction, retouching, photo editing, VFX, or visual design with ability to justify image quality judgments
  • Experience using generative image tools (Midjourney, Stable Diffusion, ComfyUI, Flux, DALL-E, Ideogram) and knowledge of failure modes and safety filters
  • Moderated or reviewed visual content at scale with policy taxonomy
  • Strong attention to detail to detect subtle issues (extra fingers, shadows, logos, brand marks)
  • Ability to maintain rubric consistency over long sessions and handle sensitive material with maturity
  • Clear written reasoning to enable auditability of judgments
  • Ability to separate personal taste from customer-required standards

Skills

Required:Image evaluation and critiquePolicy compliance & taxonomyRubric calibration and justification

Benefits

  • Medical coverage including prescription and mental health benefits
  • Dental and vision insurance
  • Healthcare and dependent care flexible spending accounts
  • Pretax commuter benefits
  • 401(k) with employer match
  • Voluntary accident, critical illness, and term life insurance
  • Wellness and pet-related reimbursements
  • Charitable matching and employee discounts
  • PTO accrual and paid holidays as described in policy
Last updated Oct 22, 2026. Interested? Apply on the company site.
Handshake logo

About Handshake

Careers start here

The better career platform for Gen Z changing how, where, and why the next generation of talent builds their career.

educationcareer platformtalent acquisitionstudent engagement
Company profileWebsite

More open roles at Handshake