The work

You receive artifacts an AI system produced as if it were a working professional in your field: a technical design doc, a data pipeline cost model in a spreadsheet, a migration readiness deck for an executive audience. Your job is to judge whether the artifact would survive contact with a real engineering org — whether the architecture claims hold up, whether the numbers reconcile, whether a diagram actually describes the system it labels, and whether the slide would embarrass whoever presented it. You then write structured feedback keyed to a rubric: what failed, where, and why it matters.

Expect a mix of scoring dimensions — factual and technical correctness, reasoning rigor, completeness against the prompt, and formatting or aesthetic quality. Presentation quality is not an afterthought here; the listing calls out Slides and PowerPoint proficiency specifically because a large share of the work products are decks, and evaluators are expected to catch misaligned elements, unreadable charts, and inconsistent typography alongside substantive errors.

What the screen looks for

  • Five or more years of real professional work in software engineering, ML/AI, IT operations, or data. Screens probe for specifics: stacks you owned, incidents you ran, models you shipped.
  • Depth that survives follow-up. The AI interviewer will push on one claim from your background rather than sampling broadly, so pick examples you can go three layers deep on.
  • Evaluation judgment — can you separate "wrong" from "stylistically different from how I'd do it," and can you justify a score against a rubric rather than a gut reaction.
  • Written clarity in English, since the deliverable is prose feedback others act on.
  • An advanced degree helps but does not substitute for shipped work.

Logistics

Fully remote, hourly, async. Most contributors log time in self-selected blocks; volume fluctuates by project cohort and is not guaranteed week to week. Onboarding typically involves a calibration set graded against reference answers before paid work begins. Pay bands are as observed on the platform and vary with specialization and project — treat the range as a signal, not a promise.