Evaluation work can feel opaque: you rate outputs, submit, and never see what happens next. Understanding the lineage makes you a better — and more confident — evaluator.
2022: human feedback goes mainstream
OpenAI's InstructGPT work showed that a modest amount of human preference data — people ranking model outputs — made models dramatically more helpful. This is RLHF: reinforcement learning from human feedback. Every preference-rating task descends from it.
2023: automating the easy cases
Anthropic's Constitutional AI showed some feedback could come from models guided by written principles. The labor-market effect: routine rating got automated, and human work concentrated on the cases where models disagree or lack competence — specialized domains.
2024–25: the expert turn
As models improved, crowd-worker feedback became actively harmful for specialized content: non-experts reward confident-sounding wrong answers. Research on evaluation quality pushed labs toward credentialed reviewers, structured rubrics, and written critiques. This era created the expert platforms this site covers.
Now: reasoning and agents
Current models produce long reasoning chains and take actions — browsing, coding, using tools. Grading them means auditing a process, not a paragraph. Research on process supervision — rewarding correct reasoning steps, not just correct answers — is why modern tasks ask you to grade step by step.
Why this matters when you work
- When a task asks for a critique, write the one a journal reviewer would write — that text may train the model directly.
- When a rubric seems pedantic, remember: consistency across thousands of evaluators is the point.
- Your judgment on hard cases is the scarce resource. The easy cases were automated before the task reached you.