What the work actually is

You sit between a leading AI lab's research team and the pile of engineering tasks used to train and measure its models. Day to day that means three things: QA on engineering knowledge-work tasks and model outputs, authoring instruction specs and golden solutions that pin down what "correct" means, and designing evaluation sets hard enough to expose where the model is still guessing. Most of the judgment is negative judgment — catching code that compiles and passes the happy path but mishandles concurrency, retries, partial failure, or an obvious adversarial input; catching answers that are fluent, well-organized, and would still be sent back in a real code review.

The other half is calibration. You work with client researchers and reviewers in adjacent domains to keep standards consistent, which means converting tacit engineering taste into written criteria someone else can apply the same way tomorrow. Expect to defend a rubric line in writing, and expect to be asked why your golden solution is the golden one.

What the screen looks for

  • A specialization, not a survey. Distributed systems and backend infra, security and cryptography, embedded and firmware, ML systems, developer tooling and platform engineering, or quality and test engineering — the platform probes one of these to depth and follows up.
  • Real seniority with ownership. Senior, Staff, Principal, Architect, or Engineering Lead, with design decisions you actually owned and can defend, backed by 4+ years of shipped production work. Internships and coursework do not count.
  • Evidence. Shipped systems, patents, open source, or published technical work is preferred and gives follow-up questions something to land on.
  • LLM fluency in your own practice. Not opinions about AI — specific instances where you caught a model being confidently wrong in your domain.

Logistics, plainly

This is not remote. It is hybrid in the Bay Area, California, on-site with the client's team multiple days a week when required; if you are not already local you must relocate at your own cost, and no relocation assistance is offered. Employment is W-2 with Cincinnatus LLC, the employer of record, which handles payroll, benefits, and compliance while placing you inside client teams and client-issued tooling and equipment. Commitment is 40 hours per week for an initial six-month engagement. The $65–105/hr band reflects rates observed on this listing and is not a guarantee — placement, specialization, and seniority move it.