Frontier models can recover up to 65% of facts they can't directly recall - just by thinking longer

A new study from Google Research and the Technion shows that large language models often fail to answer questions not because they lack the facts, but because they cannot access information they already store. The researchers found that frontier models such as GPT-5 and Gemini-3 encode 95-98% of tested facts in their parameters, yet they fail to directly recall 26-34% of those facts without additional computational effort. This finding challenges the common industry assumption that incorrect answers signal missing knowledge, which typically leads engineering teams to scale up models or expand training data.

The study proposes a fact-level profiling framework that tests a single piece of information across multiple conditions, distinguishing between whether a model has encoded a fact and whether it actually knows it. A fact is considered encoded if the model can reproduce it when primed with its original training context, while knowing a fact requires the model to answer questions about it from varied phrasings and directions. The researchers categorize model behavior into five profiles, including direct recall, encoding failure, recall failure, recall with thinking, and inference without encoding. For instance, a model might encode that Oasis played their first gig at the Boardwalk club but fail to answer a direct question about it, only retrieving the fact after generating intermediate thoughts about the band's early history.

Inference-time computation serves as a significant recovery mechanism, successfully retrieving 40-65% of the encoded facts that models initially fail to recall. The researchers compared this to a human tip-of-the-tongue state, where deliberate effort helps jog memory. Their experiments involved evaluating 13 LLMs on over 4 million responses using a benchmark called WikiProfile, which contains 2,150 facts from Wikipedia tested across formats from exact context completion to multiple-choice questions. The paper also found that rare facts are encoded at rates similar to popular ones, but there is a recall gap between long-tail and highly popular facts exceeding 25% for frontier models.

Scaling model size does not automatically fix this recall problem, according to the study. When the researchers scaled the Gemma3 model from 1 billion to 27 billion parameters, encoding failures dropped from 85% to 23%, but recall failures increased, peaking at 40% without thinking. This suggests that scaling primarily solves the storage problem, leaving a larger pool of knowledge trapped in an encoded but inaccessible state. Nitay Calderon, a research scientist at Google, said that companies often try to solve recall failures by training larger models, but if facts are already stored, that expensive approach does not help.

The researchers note that how a user phrases a question directly affects whether a model can unlock a stored answer. Models struggle with reverse questions, such as asking who played their first gig at a specific venue instead of where a band performed, though they can recognize the correct answer in multiple-choice format. The authors argue that these failures should be reframed as recall issues rather than missing knowledge, which points to post-training interventions that improve how models use what they already encode, rather than pre-training changes like scaling or adding data. This distinction has practical implications for developers building reliable applications, suggesting that lower-cost inference-time strategies may address many accuracy problems without resorting to larger systems.

More AI news from TechManNews.