What the work actually is

You take a piece of brand marketing work you have genuinely done — a share-of-voice read that contradicted the brand tracker, a regional team drifting off the messaging architecture, a distinctive asset quietly disappearing from paid creative — and rebuild it as a task inside a simulated marketing org. That means writing the brief, deciding what the task is really testing, planting the issue an average practitioner would miss, and specifying the surrounding reality: which decks and dashboards exist, what the GA4 property shows, who the personas are, what state the CMS and project tracker are in. Then you write the reference answer and the criteria that distinguish a strong response from one that is plausible, fluent and wrong.

The second half of the job is judging. You read AI agent transcripts and score them against your own rubric — which is where most environment designs fail, because a rubric that sounded rigorous in the abstract turns out to reward any answer that mentions the right vocabulary. Expect to revise your own criteria after seeing what agents actually produce.

What the screen looks for

  • A short multiple-choice knowledge screener on brand marketing specifics, taken before human review
  • Evidence you have owned positioning, brand tracking and brand investment decisions — not commissioned them or presented them onward
  • Fluency with brand tracking study methodology, share-of-voice data and its measurement caveats, and GA4
  • Written reasoning that holds up under follow-up: the value here is in explaining why a decision is right, not asserting it
  • Prior work on AI training environments, RL environments or simulated case studies, and Rubric Academy or Rubric Bootcamp certification, are both strongly preferred rather than required

Logistics

Remote and largely asynchronous, with work delivered as written artifacts — briefs, specs, reference answers, rubrics, scored transcripts. Contributors typically set their own hours against agreed batch deadlines, with occasional synchronous calibration. The observed band for this role is $60–100/hr, varying with depth of experience and the review tier you are assigned; it is what has been reported, not a guarantee.