← All roles

Senior Master Annotator

Skild AI· Operations
Apply on company site
Hybrid (Bengaluru)full-timePosted 3mo ago

About the role

Senior Master Annotator to define, maintain, and continuously improve annotation standards for robotic datasets, act as gold standard authority for labeling tasks, own Inter-Annotator Agreement (IAA) measurement, and train calibrate annotators to ensure high-quality annotations for robotics model training.

Responsibilities

  • Define and maintain gold-standard annotations and benchmark (“gold set”) datasets for new and existing tasks
  • Establish clear acceptance criteria, tolerance thresholds, and task-specific quality standards (e.g., frame-level tolerances)
  • Develop, document, version-control, and continuously refine annotation guidelines and SOPs based on disagreement patterns and quality findings
  • Design and implement Inter-Annotator Agreement (IAA) frameworks (e.g., Cohen’s/Fleiss’ kappa, tolerance-based agreement models) and produce regular quality reports
  • Analyze disagreement trends, identify root causes, and recommend data-driven process improvements
  • Lead onboarding and ongoing training for annotators, including structured calibration sessions
  • Build and maintain edge-case libraries and example banks to improve annotation consistency
  • Provide targeted feedback and retraining to improve annotator performance and alignment with gold standards
  • Conduct precision-based QC audits according to task requirements and validate pilot batches prior to production scaling
  • Review escalated edge cases and partner with QA and Operations teams to optimize sampling strategies and quality processes

Qualifications

  • 4–7+ years of experience in data annotation, quality control, or video labeling environments
  • Strong expertise in temporal video annotation; experience with robotics, computer vision, or autonomous systems datasets preferred
  • Deep understanding of Inter-Annotator Agreement methodologies (e.g., Cohen’s/Fleiss’ kappa), quality metrics, sampling strategies, and gold set creation
  • Experience mentoring, training, or guiding annotators to improve accuracy and consistency
  • Strong analytical skills with proficiency in Excel or Google Sheets (Python preferred but not required)
  • Ability to translate ambiguous SOPs into precise, actionable annotation guidelines
  • Exceptional attention to detail and sound judgment in ambiguous visual tasks
  • Data-driven, process-oriented mindset with strong ownership and accountability
  • Excellent written and verbal communication skills

Skills

Required:Gold-standard annotation creationInter-Annotator Agreement (IAA)Annotation guideline developmentTraining and onboardingTemporal video annotationQuality metrics and sampling strategies

Benefits

  • Competitive compensation package
  • Opportunities to work on cutting-edge robotics AI projects
  • Collaborative, fast-paced environment in Bengaluru, India
Last updated Jul 5, 2026. Interested? Apply on the company site.
Skild AI logo

About Skild AI

Bringing AI into the physical world

Building general purpose robotic intelligence

omni-bodiedAIPhysical AIliquid-ai-recruiting-targetphysical-ai-funding-landscaperobotics
Company profileWebsite

More open roles at Skild AI