Human Data Manager

$40–60/hrrate as listed

Own the delivery of human-generated training and evaluation data for AI lab clients — scoping tasks, staffing expert annotators, and holding quality bars across a project's lifecycle. Observed pay for this role sits at $40–60/hr, structured as contract work with hours that flex with active project load.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.

What the work involves

Human data managers sit between an AI lab's research team and the pool of contributors producing the data. Day to day that means translating a research request into a workable task spec, deciding what counts as an acceptable response, and then running the operation that produces it at volume. You will write and revise annotation guidelines, build calibration sets, staff projects with contributors whose credentials actually match the domain, and audit output when quality drifts.

The judgment-heavy part is quality. You will be reading samples of contributor work, deciding whether a disagreement is a guideline gap or a contributor problem, and reporting honestly to a client who would prefer to hear that throughput is fine. Expect to escalate rubric ambiguity early, track inter-annotator agreement, and make calls about removing contributors from a project.

Scoping and spec-writing with research stakeholders
Recruiting, onboarding, and calibrating expert contributors
QA sampling, agreement analysis, and rubric revision
Throughput and quality reporting; delivery against deadlines

What the platform screens for

micro1 runs an AI-led screening interview before human review. It probes for verifiable operational history: projects you actually ran, headcounts you actually managed, quality metrics you actually used. Vague claims about "managing data teams" tend to collapse under follow-up questions asking for numbers, tooling names, and specific failure cases. Screens also test evaluation judgment directly — you may be given a messy rubric or a pair of conflicting annotations and asked to reason aloud about the right resolution.

Logistics

Fully remote and largely asynchronous, with some overlap expected with US Pacific or Eastern hours for client syncs. Engagements are typically part-time to full-time contract, and load varies sharply with project cycles — candidates who can absorb a surge week without disappearing afterward are preferred.

What they look for

Project Management
Data Operations
Quality Analysis
Human Data

Before you apply

Would you pass this screen?

Drawn from what this listing actually requires. Answered in your browser — nothing is sent to us, and we do not store your answers.

Have you managed the end-to-end operation of an annotation, labeling, or data collection project—including recruiting contributors, writing task specs, and tracking quality metrics—for at least one completed project?
Data management operations experience · required
Have you used inter-annotator agreement, error rate, precision/recall, or similar quantitative quality metrics to measure and monitor data quality on a real project?
Quality metric experience · required
Are you able to work asynchronously and have overlap availability during US Pacific or US Eastern business hours at least three days per week?
Asynchronous remote work · required
Do you have deep expertise (publication, credential, or 5+ years experience) in at least one domain where you could evaluate or improve contributor work—such as medicine, law, engineering, academic research, or a specialized field?
Domain expertise or technical comfort · preferred

0 of 4 answered

Included with this listing

Interview prep — 14 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: Whether the candidate has hands-on operational history running a real project—not just observing one. The numbers (headcount, volume, metric) expose whether they can recall specifics or are generalizing.
A strong answer: Name the project, the domain, the size of the contributor pool, the specific quality metric used (e.g., inter-annotator agreement, F1 on a held-out set, or error rate on audits), and what the target was. Be ready to say how you measured it.
Where people lose it: Vague language like 'managed a team' or 'oversaw quality' without naming the metric, the tool, or the numbers. If pressed, you cannot recall what disagreement rate you aimed for or how you actually measured it.