Hightouch Proved a Hybrid LLM Judge Hits Near-Opus Accuracy at 4x Lower Cost. That's an AI Cost-Routing Audit You Can Sell This Week.

Identity resolution — mapping scattered records back to one real person — has a quiet problem: it's hard to check.

The failure mode Hightouch cared about is "over-merging," where one identity accidentally represents more than one person. Catching that by hand means sampling results and grading them, and a human can only check about 50 identities an hour. Getting a statistically solid error rate needs more than 1,000 samples. That's not a QA process, that's a part-time job.

So Hightouch tried using LLMs as the judge instead — and the result is a service business, not just a benchmark post.

The numbers, exactly as published

On the easier test scenario:

• Claude Opus 5.5 alone: "99.0% accuracy, 2.2 s per identity, $3.86 per 1,000"
• Jev (the cheap, fast model) alone: "95.7% accuracy, 0.15 s, $0.05 per 1,000"
• "Hybrid Jev" — Jev judges everything, then anything in the 30–90% confidence range gets escalated to Opus: "98.8% accuracy, $0.88 per 1,000"

On the harder test scenario, the hybrid setup didn't just stay close to Opus — it won on both axes:

• Claude Opus 5.5 alone: "95.1% accuracy, $5.39 per 1,000"
• Hybrid Jev: "95.3% accuracy, $2.94 per 1,000"

Hightouch's own explanation for why the hybrid held up on the harder set: "Hybrid Jev's better confidence calibration means it elegantly sends more cases to Opus on harder data without any harness or prompting change."

The service business hiding in this story

Every company running a frontier model over every single row of a high-volume grading job is paying for certainty it doesn't need on the easy 90% of cases.

That gap is exactly what a routing layer closes, and it's exactly what you can sell:

• A cost-routing audit that measures what fraction of an existing LLM-grading pipeline is "easy" (cheap model is already confident and correct) versus genuinely ambiguous
• A fixed-scope build of the hybrid judge itself — cheap model first pass, confidence-band escalation to the frontier model second pass
• A monthly calibration retainer, because the thing that made Hybrid Jev work wasn't the cheap model's raw accuracy. Hightouch was blunt about this: "Most of Jev's mistakes are closer to 50% confidence...Luna, on the other hand, seems quite overconfident." A model that knows when it's unsure is worth more than one that's slightly more accurate but overconfident — and that property has to be re-measured as data drifts

Who to sell this to

• Teams running identity resolution, support-ticket QA, content moderation, or data-labeling pipelines that call a frontier model on every item
• Anyone whose "LLM-as-judge" setup was built once and never recalibrated
• Finance or ops stakeholders who'd read a four-x cost line item on a model API bill a lot faster than they'd read a model-quality memo

Bottom line

Hightouch didn't publish "use a cheaper model." They published exact numbers across two difficulty tiers and explained the one design choice — confidence-based escalation — that made the cheap path safe. That's a reproducible audit methodology, not just a research note, and it's sitting there for anyone willing to package it as a service.

Source: https://hightouch.com/blog/llm-judges-identity-resolution