Wins

A memory that beats retrieval when nobody wrote down the answer

Nobody filed the answer. That was the point of the test.

We gave two systems the same stream of facts. The stream was deliberately annoying in the way real information is annoying: facts arrived over time, some were revised, old facts were buried under newer noise, and the current state had to be assembled from the whole mess.

One system was a strong retrieval baseline: a RAG that could reread its notes, retrieve iteratively, and use timestamps to prefer newer facts. The reader was not the weak link.

The other system was the memory. It did not wait for the question and then start searching. It updated its picture of the world while the stream was arriving.

Then both systems got the same two tests. These are not extra features added to the memory; they are two ways to ask whether the system can answer something that was never written down:

  • Derived. The answer is never stated. To get it, you have to chain facts together and work out the missing consequence.
  • What-if. Change one fact and say what follows, even though that scenario never happened in the stream.

This is where the memory stopped acting like a filing system and started acting like someone who had been keeping the map updated all along.

The score is plain accuracy: what fraction of questions did each system answer correctly? Higher is better.

On derived questions, the RAG baseline answered 22% correctly; the memory answered 80%. On what-if questions, the RAG baseline answered 31% correctly; the memory answered 78%.

That is the value: when the answer is not sitting in the notes, a memory that already organized the changing world can answer far more often than a system that has to rebuild that world from retrieval at question time.

Two separate question types where the answer was never stored. Higher accuracy is better; memory scores far higher than a strong retrieval baseline on both.

Where the win stops

This is the part that keeps the result from dressing up as a universal claim.

Hand the retrieval system the true dynamics, and it catches up. Give it enough context to reconstruct the whole changing world on every question, and it catches up. So the result is not "retrieval cannot reason." That would be tidy, flattering, and wrong.

The result is narrower and more useful: under a fixed context budget, rebuilding the map on every question loses to a memory that kept the map current while events were happening.

That is the actual win. Not magic reasoning. Not a secret executive module. Just the unfashionable advantage of doing the bookkeeping before the exam starts.

Results are on a synthetic world, six seeds, with the boundary mapped rather than hidden. Try a version of it in the demo.