Key Skills

Job Description

Product Designers & UX/UI Designer. We are seeking experienced product designers, UX/UI designers, design system specialists, AI-assisted developers, rapid prototypers, and AI-native product builders to create realistic evaluation tasks for advanced AI systems. What You’ll Do Analyze task execution frame by frame when necessary Identify important actions, events, behaviors, and outcomes Apply structured labels, tags, and grades based on project guidelines Mark timestamps or video segments corresponding to key events Determine whether tasks were completed correctly Identify partial completions, failed actions, mistakes, or unusual behavior Document observations and evaluation rationales clearly Apply annotation guidelines consistently across large volumes of videos Participate in reviewer calibration and quality discussions Collaborate asynchronously with project trainers and other contributors Flag unclear or ambiguous cases for additional review Provide feedback on labeling guidelines and annotation workflows Organize and submit completed annotations within project timelines Maintain high standards for precision, consistency, and quality Who Can Apply Relevant backgrounds include: Product Designers, Senior Product Designers, Staff Product Designers, Principal Product Designers, Lead Product Designers, UX Designers, UI Designers, UX/UI Designers, Interaction Designers, Experience Designers, Digital Product Designers, Visual Designers, Interface Designers, Web Designers, Mobile Product Designers, and Application Designers. Design leadership backgrounds may include: Design Leads, Design Managers, Product Design Managers, Heads of Design, Design Directors, UX Directors, Creative Directors, Experience Design Directors, Design Principals, and Design Consultants. Design systems backgrounds may include: Design Systems Designers, Design Systems Leads, Design Systems Engineers, UI Systems Designers, Component Library Designers, Design Technologists, UX Engineers, Design Engineers, Frontend Design Engineers, and Design Infrastructure Specialists. Video and media backgrounds may include: Video Editors, Video Producers, Multimedia Editors, Content Editors, Motion Designers, Motion Graphics Artists, Filmmakers, Videographers, Cinematographers, Post-Production Specialists, and Audiovisual Professionals. Research and usability backgrounds may include: UX Researchers, User Researchers, Product Researchers, Usability Researchers, Human Factors Specialists, Human-Computer Interaction Professionals, HCI Researchers, UX Strategists, Service Designers, Information Architects, and Customer Experience Designers. Annotation and AI data backgrounds may include: Video Annotators, Data Annotators, Data Labelers, AI Data Trainers, AI Evaluators, Computer Vision Annotators, Image Annotators, Multimedia Annotators, Human Feedback Specialists, Annotation Quality Analysts, and Data Quality Reviewers. Requirements Professional or academic background in UX/UI, product design, video editing, photography, multimedia, engineering, data annotation, or a closely related field Strong visual attention to detail Ability to analyze video content carefully Comfortable applying detailed annotation and grading guidelines Ability to work consistently across repetitive review tasks Strong written communication skills Ability to explain annotation decisions clearly Comfortable learning new annotation tools Strong organizational skills Ability to meet deadlines Comfortable collaborating with remote teams Ability to work independently while maintaining high quality Preferred Background Portfolio demonstrating shipped products, applications, interfaces, or design systems Experience at a product-led technology company, design-forward organization, startup, or independent product studio Experience building and shipping products using AI coding assistants Experience owning large-scale design systems or component libraries Experience moving rapidly from concept to functional prototype Strong understanding of both UX principles and technical implementation Experience reviewing or critiquing the work of designers or engineers Experience creating product design standards, training materials, design critiques, or evaluation frameworks A formal credential is not required. A strong portfolio of shipped work, real product ownership, and demonstrated senior-level judgment are highly valued. We are a referral partner of the client.

Core Responsibilities

Review video task executions and annotate key actions, events, outcomes, timestamps, and errors using structured labels and project guidelines. Document evaluation rationales, maintain annotation quality and consistency, collaborate with reviewers and trainers, and submit work on schedule.

Requirements

Applicants need a professional or academic background in UX/UI, product design, video or multimedia work, engineering, data annotation, or a related field, along with strong visual attention, written communication, organization, and the ability to follow detailed guidelines. Shipped product experience, design systems expertise, AI-assisted product development, prototyping, and senior-level design judgment are preferred; formal credentials are not required.