Run the new memory prompt back over an old bank

The prompt change only reaches memories written after it. plan/18 decided to
leave the existing ones alone and let eviction age them out at
memory_bank_capacity, on the grounds that re-summarizing would duplicate
whatever was still in the bank because nothing deletes the old rows.

That was wrong about the only option. A memory can be rewritten in place. The
row carries more than its text — whether it is pinned, how often it has been
retrieved, and the node it hangs off, which is what makes a fork inherit the
right memories — and rewriting `text` keeps all of it. Deleting the bank and
rewinding the cursor would lose that, and would trickle memories back at
MAX_MEMORIES_PER_RUN per turn, so an adventure nobody is playing would never
recover.

tools/rewrite_memories.py does it. Without --write it makes no model calls and
only reports the scope; --write rewrites, --embed re-embeds in the run rather
than leaving it to the app's post-turn pass. It reads whichever database the
app reads, so it works against the hosted Postgres as well as a local file.

Two things it needed from the app. `summarize_block` is now the one place a
memory prompt is assembled, and the post-turn pass calls it too — a backfill
that built its own prompt would be writing memories with a prompt that never
shipped, and nothing would report the drift. `source_block` reads a memory's
block back out of the story, which nothing has ever had to do: it reads on the
lineage of the branch the memory was written on, not the branch being played,
because after a fork the same depths hold different actions on each side and a
read through the adventure's path would summarize the wrong story silently.
It also excludes the discarded attempts at a retried turn, and tolerates a
block an action has since been deleted from.

Left alone: a memory with no source range, which is hand-written or migrated by
62 and may be the player's own words; a memory whose actions are gone; and an
adventure whose owner has no API key, because summarization spends the user's
own key by construction and never the shared demo key. --api-key/--model/
--endpoint override that, the last of them aiming a run at claude_shim.py.

The vector is cleared for every rewrite, because the stored one describes
wording that no longer exists. Re-embedding always uses the owner's own
embedding model, never --endpoint: a vector only means anything against the
vectors it is ranked beside.

17 tests, 627 green. The fork case is the one that would fail quietly, so the
test builds a fork whose depths hold different actions on each side.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tqgupw5CZGjSZrUTNUd4fW
This commit is contained in:
Claude
2026-08-31 10:11:26 +00:00
parent 71b24b6229
commit 0633cb624e
5 changed files with 841 additions and 18 deletions
+32 -1
View File
@@ -80,7 +80,7 @@ needed; nothing requires reading a row of anyone's story.
**`plan/18-persona-and-memory-quality.md` is the writeup; it is on
`claude/ai-dnd-memories-summarization-3muo98`, not yet merged.** Two changes, both green
at 610 tests.
at 627 tests, plus the backfill below.
**The protagonist now has a name.** An adventure carries `persona_name`,
`persona_pronouns` and `persona_desc` (migrations 74-76), and the player's stat block
@@ -120,6 +120,37 @@ through the shim. The app talks to an OpenAI-compatible endpoint, and
`worldstate/parse.py` tolerates trailing commas because free models emit them. Point
`memory_ab.py --endpoint` at the real provider to find out.
**A bank written under the old prompt can now be rewritten.** The earlier answer was to
let eviction age those memories out, on the grounds that re-summarizing would duplicate
rows. It does not have to: `backend/tools/rewrite_memories.py` rewrites a memory's text
in place from the same actions it was written from, keeping the pin, the use counts and
the node it hangs off. Run it from `backend/`; without `--write` it makes no model calls
and only reports what would change.
```
python -m tools.rewrite_memories # what would change
python -m tools.rewrite_memories --write --limit 3 # try three of them
python -m tools.rewrite_memories --write --embed # the whole backfill
```
It reads whichever database the app reads (`AIDND_DB_PATH`, or `DATABASE_URL` on the
hosted deploy), so **take a copy first** — the old text is overwritten and kept nowhere.
`--endpoint`/`--model`/`--api-key` point the summarizer somewhere else, `claude_shim.py`
included. Hand-written memories, and memories whose actions have been deleted, are left
alone; so is an adventure whose owner has no API key, because summarization spends the
user's own key and never the demo key.
**Two things it made necessary in the app.** `memorybank.summarize_block` is now the one
place a memory prompt is assembled, so a backfilled memory cannot be written by a prompt
that never shipped. And `memorybank.source_block` reads a memory's block back out of the
story — on the lineage of the branch the memory was written on, not the branch being
played, because after a fork the same depths hold different actions on each side.
**The vector is cleared for every memory it rewrites**, since the stored one describes
the old wording; the app's post-turn pass re-embeds them `MAX_EMBED_BATCH` per turn, or
`--embed` does it in the run. Stop the app before using `--embed`: a running process
caches vectors by memory id and expects to be the only writer.
---
## Pick up here