What the work actually looks like

You'll be handed a queue of paired video clips — two outputs generated from the same prompt — and a rubric that breaks the comparison into dimensions like prompt adherence, motion quality, visual artifacts, and physical plausibility. For each pair you score both clips on those dimensions, pick an overall preference, and where prompted write a short justification that points to what you actually saw. The clips are short, but the volume is high and the judgments are fine-grained: two clips can both be flawed in different ways, and the rubric is what decides which flaw matters more.

The hard part is not spotting an obvious defect. It's staying calibrated across hundreds of comparisons — not drifting toward whichever clip is prettier, not letting fatigue collapse your scale into 3s, and not quietly inventing a criterion the rubric never mentioned. Your written justifications are read by researchers checking whether the rubric is being applied as intended, so they need to cite observable evidence ("the subject's left hand passes through the mug at 0:03") rather than aesthetic impressions.

What the screen looks for

Mercor runs an AI-led interview of roughly 15 minutes. Expect it to confirm your degree and background, then push on judgment: how you handle a tie, what you do when the rubric doesn't cover something you noticed, whether you can describe a video defect precisely in words. Writing quality matters more here than video production experience — the listing asks for strong writing and critical thinking, and prefers a couple of years in teaching or research, where explaining a judgment clearly is routine. Availability is a real filter, not a formality.

Logistics

  • Fully remote and asynchronous; you work through the platform's task queue on your own schedule.
  • At least 20 hours per week, over a commitment of about 1.5 weeks.
  • Observed pay band $15–20/hr — as reported for this listing, not a guarantee.
  • You'll need a display and connection good enough to judge motion artifacts in video without stuttering confusing your assessment.