Legal Domain Expert — AI Training & Evaluation

$60–100/hrrate as listed

A full-time W-2 role through Cincinnatus LLC, embedded with a frontier AI lab's research team to build legal evaluation sets, instruction specs, and golden solutions. Requires a JD, active U.S. bar admission, 5+ years of senior-level practice, and Bay Area residency with occasional on-site days.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.

What the work involves

You are the legal substance behind a frontier model's training pipeline. Day to day, that means reviewing legal knowledge-work tasks and model outputs for the failure modes only a practitioner catches — an argument that cites the right doctrine but misses the controlling standard, a memo that reads persuasively and would still get you called into a partner's office. You will write instruction specs that define what "correct" looks like for a given legal task, produce golden solutions yourself, and design benchmark sets hard enough to expose real weakness rather than confirm progress.

The calibration side matters as much as the drafting. You will work alongside client researchers and experts in adjacent domains to convert tacit legal judgment — the reason a senior associate rejects a first draft — into explicit, teachable criteria that other reviewers can apply consistently. Expect substantial written output and close collaboration inside the client's own tooling, on client-issued accounts and equipment.

What the screen looks for

Mercor's screening is AI-led and follow-up heavy. It is built to distinguish a practising specialist from a credentialed generalist, so expect to be pushed on the specifics of matters you actually owned: the practice area, the posture, the decisions that were yours. Verifiable specifics carry more weight than titles.

Depth over breadth — one genuine specialization (corporate/transactional, litigation, regulatory and compliance, IP, employment and labor, or tax) probed several layers down.
Evaluation judgment — whether you can articulate why an answer fails, not just that it does.
Real seniority — partner, of counsel, counsel, senior associate, senior in-house, or GC, with ownership of matters. Clerkships and internships alone do not count.
AI fluency — hands-on use of LLMs in professional work, and a calibrated sense of where they hallucinate.

Logistics

Full-time W-2 employment with Cincinnatus LLC (employer of record; payroll, benefits, and compliance run through them, not Mercor), with placement at a leading AI lab's extended workforce. 40 hours per week, initial engagement of six months. Work is remote day to day, but you must live in the Bay Area and be able to come on-site when required — on-site days are expected to be infrequent. Observed pay for comparable legal evaluation roles on the platform runs $50–90/hr; the band is not a guarantee and depends on practice area, seniority, and client scoping.

What they look for

Juris Doctor (JD) from an accredited U.S. law school
Active admission to at least one U.S. state bar, in good standing
5+ years post-qualification practice at a firm, in-house department, regulator, or court
Demonstrated specialization in a substantive practice area (corporate, litigation, regulatory, IP, employment, or tax)
Senior-level ownership of matters (senior associate, counsel, of counsel, partner, senior in-house, GC)
Hands-on professional use of large language models and precise written feedback skills

Before you apply

Would you pass this screen?

Drawn from what this listing actually requires. Answered in your browser — nothing is sent to us, and we do not store your answers.

Do you hold a Juris Doctor (JD) from an accredited law school?
Juris Doctor from accredited law school · required
Are you currently admitted to at least one U.S. state bar in good standing?
Active U.S. state bar admission · required
Do you have 5 or more years of substantive legal practice at a law firm, corporate legal department, regulatory body, court, or equivalent reputable institution after qualifying to practice law?
5+ years post-qualification legal practice · required
Do you have genuine specialization and hands-on experience in at least one substantive legal practice area (such as corporate/transactional, litigation/dispute resolution, regulatory/compliance, IP, employment/labor, or tax)?
Specialization in a substantive practice area · required
Have you achieved or do you currently hold a senior position in your practice (such as partner, of counsel, counsel, senior associate, senior in-house counsel, or general counsel) with real ownership of matters?
Senior-level progression · required
Have you used large language models (such as ChatGPT, Claude, or legal-specific AI tools) in your professional legal work?
Hands-on LLM experience in legal work · preferred
Do you currently live in the Bay Area, California?
Bay Area residency · required

0 of 7 answered

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: Whether the candidate can articulate the genuine complexity and judgment calls in their own practice area, not recite doctrine—and whether they think in terms of what models need to learn rather than what lawyers intuitively know.
A strong answer: The answer should describe a specific file with real complications (fact disputes, competing precedents, client constraints, or strategic trade-offs), name the exact legal moves that mattered, and identify which of those moves are not obvious from a rule statement alone.
Where people lose it: Naming a practice area but then describing only the surface-level legal rule ('we reviewed the contract for liability clauses') rather than the judgment embedded in how experienced lawyers actually solve it.