Labs Are Trading Facts for Reasoning in Small Models. The Service: Audit Which of Your AI Tools Are Quietly Running on a Hallucination-Prone Base and Bolt On a Retrieval Layer.

A blog post arguing that AI labs are deliberately trading factual knowledge for reasoning ability just made the Hacker News front page, and it comes loaded with specific numbers that make the tradeoff hard to dismiss.

The core claim: "GLM-5.2 scores 99.2% on AIME 2026 with about 40 billion parameters active per token," and "Qwen3.5 scores 91.3% with 17 billion active," while "DeepSeek V4-Flash runs 13 billion active." For comparison, "GPT-4 was rumored to run around 280 billion active parameters in 2023." Reasoning performance per parameter has gone up dramatically in a short time.

But factual recall hasn't followed the same curve. The post cites SimpleQA — "a benchmark of factual recall with no tools allowed" — where "the current leader is Gemini 2.5 Pro at 53%." And on the small end, the numbers get worse fast: "Artificial Analysis measures Qwen3.5 4B and 9B at hallucination rates of 80 to 82%."

Why this is happening

The author's explanation: "reasoning compresses much better than facts do, because it's a relatively small set of procedures applied over and over." Research on knowledge capacity — "the 'Physics of Language Models' series has the cleanest measurements" — puts factual storage "on the order of two bits of factual knowledge per parameter," making memorized facts expensive relative to reasoning procedures. Labs optimizing for benchmark reasoning scores on a fixed parameter budget are, in effect, choosing to shed facts.

The business opportunity

Most teams that adopted a smaller, cheaper, reasoning-strong model did it for cost or latency — not because they benchmarked the factual-recall tradeoff first. That's the audit: take a client's current model, run it against domain-specific factual questions, and report the hallucination rate before recommending anything.

The fix isn't "use a bigger model." It's architectural: retrofit a retrieval layer — search, internal documentation, structured lookups — so the model answers facts from a traceable source instead of memory. This mirrors the post's own framing of the solution: wrong facts in an external knowledge base become "ordinary data bugs" that can be found and fixed, unlike errors embedded in model weights that require retraining to correct.

Who buys this

Teams that already swapped to a cheaper or faster model and are fielding quality complaints they haven't traced to factual recall specifically, plus any team building on the current wave of small open-weight reasoning models without a retrieval layer already in place.

Bottom line

The tradeoff described in this post isn't a bug labs will quietly fix — it's a deliberate compression strategy that will keep showing up as models get smaller and cheaper. That makes the audit a recurring service, not a one-time fix.

Source: https://w4g1.dev/blog/models-are-getting-dumber-on-purpose