Does the AI get the law right? Someone should actually check.
I build the benchmarks that test whether an AI system handles legal work the way a competent attorney would. The scenarios. The correct answers. The scoring criteria that decide whether an output passes.
I do that for the companies building legal AI, who need practitioners to define what right looks like. And I do it as an independent check for law firms and legal departments deciding whether to trust what they're being sold.
Why immigration
Immigration is high-stakes, adversarial and fast-moving. Statute, regulations, agency policy and adjudicator discretion, and it changes monthly. It's exactly the kind of domain that's hard for an AI to get right and hard for a non-lawyer to evaluate.
Almost none have a written policy for it, and most have never had the output checked against a practicing attorney's judgment.
That gap is the work.
What an engagement looks like
For companies building legal AI
Scenario design, gold-standard answers, grading rubrics, and a written assessment of where the system holds and where it fails.
For a firm or legal department
An independent evaluation of a tool you're considering or already using, against real work, with scoring criteria you keep.
For counsel
Consulting and litigation support where the reliability of an AI system is at issue