Skip to content

Self-Correcting Memory: Evidence-Backed Claim Repair

Self-correcting memory anchors each claim to versioned evidence, so a diff marks stale entries without a model call, and fails where evidence is unversioned.

Self-correcting memory records the evidence behind every fact it stores, along with the version of that evidence, and rechecks those versions before trusting the fact again. The worked implementation is OpenWiki's claims runtime, where the store is a wiki over a codebase and the evidence is a line range in the repository (LangChain, 2026). A claim whose recorded version no longer matches the current source is marked stale, which means "OpenWiki can no longer safely assume that the claim is still true without checking the source again."

When it applies

One precondition decides whether any of this works: the source of truth has to be versioned and addressable per claim. Code in a repository qualifies, because the runtime can compare the version it recorded against the version on disk (LangChain, 2026). Slack threads, meeting notes, and user preferences do not: with nothing to diff, no fact gets marked suspect until a model reads it and forms an opinion.

Whether it pays for itself turns on two more things. Change volume has to stay low relative to store size, since the claimed payoff is that "update cost scales with how much the code changed, not with how many claims the wiki holds" (LangChain, 2026). And repair is lazy: a correction reaches a page only when an update touches it.

How the loop runs

Writing a page also records the claims that page makes and the code supporting each one (LangChain, 2026):

{
  "statement": "Failed tasks are retried three times by default.",
  "evidence": ["repo://src/scheduler.ts#L393-L404"]
}

The runtime stores a version for that evidence. Staleness detection then runs at the start of every update, before the agent acts: it walks the claim set and compares each persisted version against the code on disk, which the post calls "a deterministic check with no model calls." No status flag is kept, because the stored version re-derives freshness on its own, so a claim cannot quietly return to trusted.

The width of that anchor is a second decision. A September 2026 preprint on a system called Assay binds each claim to the Merkle hash of its transitive dependency cone (arXiv:2609.36170v1). That scope re-verifies 7.9% to 81.9% of claims when four modules are edited, against 100% for a repo-wide hash. Binding to the edited module's own hash instead misses 23% to 68% of the invalidations the claim set required. The tests ran on five public repositories. The preprint "does not measure how much better language-model agents perform with Assay than without", so its figures bound re-verification work rather than agent performance.

Repair happens inside work the agent is already doing. "The agent never sweeps the claim set itself." Stale claims surface beside the page content when an update reads that page, and the agent either re-verifies the claim and refreshes its version or rewrites claim and page text together. Whatever it leaves unresolved stays flagged.

Why it works

The detection half asks the model nothing. A version comparison decides which claims are suspect, and the model is invoked only afterwards, with the current source in front of it. That split maps onto the sharpest finding in the self-correction literature: correction succeeds on tasks with reliable external feedback, and "no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction" (Kamoi et al., 2024). Without external feedback, models "struggle to self-correct their responses, and at times, their performance even degrades after self-correction" (Huang et al., 2024). A versioned evidence anchor is how a memory store manufactures that external signal for itself.

Durability is the other half. Uncertainty lives in the stored version rather than in a flag someone can clear, so a claim stays suspect until it is rechecked. Outdated entries sitting beside current ones both reduce answer accuracy and mislead models into harmful outputs "even when current information is available" (Ouyang et al., 2025).

A replay benchmark reports stale claims falling from 3.5% to 0.5% and hallucinated claims from 0.7% to zero across 2,000 claims per arm (LangChain, 2026). The vendor built and ran it, nobody has replicated it, and no variance is reported, so treat the figures as a direction rather than a measurement.

A named deployment has since been published, and it changes that assessment very little. The AI and ML engineering teams at Credit Genie, a financial wellness platform, run OpenWiki across their repositories (LangChain, 2026). Docs live in an openwiki/ folder per repo, and coding agents are told to read it before acting. OpenWiki there "runs nightly, checks for commit changes, and generates documentation updates when it finds meaningful differences, opening a pull request in the repo." No repository size, claim count, or change volume appears anywhere in the account, so the ratio the cost argument turns on is still unmeasured on real code. Nor does it mention claims, evidence anchors, or staleness detection, and it comes from that same vendor. The loop runs somewhere real. That is all it establishes.

When this backfires

  • The anchor is wrong. A claim's link to its evidence is model-produced, and the best model in a 2025 evaluation reached F1 scores of 79.4% and 80.4% on documentation-to-code trace links (Alor et al., 2025). A claim anchored to the wrong lines never fires when the code it truly depends on changes, and fires whenever the wrong lines churn.
  • Mechanical edits flood the queue. Staleness over-approximates by design, so a formatting sweep or a file move marks every claim anchored in the touched ranges. Detection stays free; the per-claim verification the agent then performs does not.
  • Cold pages never heal. Repair reaches only a page an update reads, so a page nobody touches carries its stale claims indefinitely.
  • The repair itself is wrong. A model that misjudges a stale claim against current code rewrites a correct entry into a false one and stamps it with a fresh version, which makes the damage look verified. Nothing checks the corrector, and the flip is documented: without external feedback "the model is more likely to modify a correct answer to an incorrect one than to revise an incorrect answer" (Huang et al., 2024).
  • Auto-merge removes the last check. Merging the agent's doc pull requests without review leaves the failure above with nothing downstream of it. The one public deployment does that by design. Credit Genie "added an action to automatically approve and merge OpenWiki update PRs" to clear merge bottlenecks (LangChain, 2026). Engineers from two other teams there "verified the accuracy of the generated docs" for the repositories they added, which covers a first snapshot and not the updates after it.
  • Regeneration is simpler at small scale. Rebuilding the store from current source has no claim schema, no anchor rot, and no way for a claim to stay stale forever. The claims runtime earns its complexity only once regeneration cost scales with store size while change volume stays low.

Key Takeaways

  • Store the evidence and its version, not just the fact. The version is what lets a memory layer distrust itself on a schedule it does not control.
  • Keep detection deterministic and model-free, and invoke the model only after the diff has already said that something changed.
  • The pattern does not transfer to memory over unversioned sources, where it collapses into the intrinsic self-correction the literature says fails.
  • Budget for two failure modes the mechanism cannot see: an anchor pointing at the wrong code, and a repair pass that overwrites a true claim and marks it fresh.
  • Every public number comes from the vendor, and the one named deployment adds none. Pilot on a single repository before a store depends on this.
Emerging — single recent source Last reviewed: 2026-10-02