M9: a campaign you can actually get back

A campaign could already be exported and imported. What could not survive the
trip was everything that explains it: the state events behind the authoritative
document, the prompt each turn was actually given, the passages it was shown,
the summaries that carry long-story continuity, and which take belonged to which
turn. An imported campaign could be read and could no longer say why it was what
it was — and a manual correction, the one state change no narration explains,
was indistinguishable from something the story had established.

The bundle is now `ai-dnd-adventure-v3`, and the version is the design rather
than a side effect. Everything added here could have been another optional key,
the way persona, Save Points, narrative state and imported knowledge each were.
That mechanism stops working at exactly this addition: a v2 file with no prompt
provenance is ambiguous between "written before M9" and "written by M9 from a
campaign that has none", and those are different facts about a campaign. A
version number is how a recovery file states what it was capable of recording.
v1 and v2 still import, and every seam from pre-active-head onward is tested for
the rule that an older file is never reinterpreted under a newer assumption.

Two categories became three. "Chosen travels, derived is recomputed" was enough
until stored prompts had to be decided: they are derived, and they must travel
anyway. The test that separates evidence from cache is not "could this be
recomputed" but "would a recomputation answer the same question" — a rebuilt
search index answers the same question, a rebuilt prompt says what the turn
would be told *now*, which is the opposite of what the inspector is for.

Also here: a real SQLite backup, through the online backup API rather than a
file copy, taken while the application is running and verified before it is
kept; story cards settled as compatibility-only legacy data and taken out of the
narrator's prompt, because they were the untracked path around knowledge
authority that IMPORTED-KNOWLEDGE-DESIGN §73 already forbade; and no schema
change at all, proved against a database M8's own code wrote.

Three defects, found by running the milestone's own tests rather than by reading
them. Deleting a campaign leaked its FTS index rows, and SQLite then handed the
freed ids to the next source imported into any campaign, which failed with an
integrity error that Reindex could not repair — both ends are closed, and a
database already carrying the damage now repairs itself. An imported node with
no state snapshot was being stamped with the campaign's head state, so an Undo
to turn 2 showed what the story knew at turn 20. And the snapshot relink did not
persist at all, because it mutated a dict in place on a column SQLAlchemy tracks
by assignment: it looked correct in memory and wrote the wrong ids to disk.

Carrying per-turn prompts looked like it would halve the length of campaign that
can be restored. Measured — and after compressing them inside the file —
everything M9 added costs 12% of it: the import ceiling moves from about 318
turns to about 279, against a 100-turn certification target. The dominant cost
is not M9's at all. The per-position narrative state document is 74% of a
bundle, and v2 already carried it.

Backend 1,102 passed / 14 skipped / 0 failed. Frontend 145 passed. Lint,
production build and Docker build clean. Verified across two server processes
with two data directories, and in a real browser against a real narrator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qyn3oRd4D6pi72nKBG725B
This commit is contained in:
JesseMarkowitz
2026-09-07 01:55:45 -04:00
co-authored by Claude Opus 5
parent 1ce9972760
commit 44edece67e
46 changed files with 9227 additions and 178 deletions
+83
View File
@@ -608,6 +608,89 @@ stated reason. A misplaced head affects every read in the file; a bookmark
pointing outside the story affects only itself, and rejecting a whole campaign
to protect one bookmark would lose the story to save the pointer.
### 9.3 As implemented in M9 — the version, and the third category
**The format is now `ai-dnd-adventure-v3`, and the bump is the design.** §9.1 and
§9.2 each declined one, correctly: an absent `headDepth` or `checkpoints` key is
unambiguous, because a file either states a position or it does not. That
property fails for what M9 adds. A v2 file with no prompt provenance may have
been written before M9, when no file could carry any, or by M9 from a campaign
whose turns predate the column — different facts about the campaign, and a
reader has to be able to tell them apart. A version number is how a recovery
file states what it was capable of recording, which is exactly the reasoning
§9.1 uses to justify opening a pre-M3 file at its tip. The reader keeps every
version; only the writer moved.
**§9.1's two categories became three.** "Chosen travels, derived is recomputed"
was sufficient until M9 had to decide about stored prompts, which are derived —
a machine assembled them — and must travel anyway:
```text
chosen the story, the head, the takes, the Save Points, the
classifications, the canon travels
evidence the state events and proposals, the per-turn prompt and the
passages it was shown, the model and generation settings that
turn ran under travels
rebuildable knowledge passages, the FTS index, embeddings, the branch
lineage cache rebuilt on import
```
The test separating the last two is not "could this be recomputed" but "would a
recomputation answer the same question". A rebuilt FTS index answers the same
question. A rebuilt prompt does not — it says what the turn *would be told now*,
from today's canon, today's sources and today's state, which is the opposite of
what the inspector is for. Historical evidence is not a cache, so M9 does not
regenerate one on import at any point.
`DATA-MODEL.md` §29 lists what v3 carries. Three implementation facts belong
here rather than there:
1. **The snapshots are encoded, not summarised.** A per-turn prompt contains the
story so far, so one per turn is O(turns²) — measured at 68% of a 9.7 MB file
at 120 turns, against a 20 MB import ceiling. The snapshot therefore travels
as `contextSnapshotZ`, zlib-compressed and base64-encoded through the same
`compression.pack`/`unpack` the database column already uses. The file is
still JSON and every other section of it is still plain text. The readable
`contextSnapshot` key is still accepted and wins when both are present, so a
hand-edited file keeps importing. A residual ceiling remains and is stated in
the M9 report rather than hidden.
2. **One more pointer is translated, and only pointers ever are.** Branch
numbers already were. A restored retrieval record's `source_id` names a row
on the machine that wrote the file, so it is repointed at the source that
landed here, or set to `null` when the file carries no such source. The text
the record holds — the evidence — is never rewritten.
3. **Import stays two-phase inside one transaction.** `plan` refuses everything
a hand-edited file can get wrong before a row exists; `materialize` writes,
and the endpoint commits once and rolls back explicitly otherwise. A
*rebuildable* index failing after that does not roll the campaign back: it is
reported on the response as a warning, shown per source in the Knowledge
panel, and repaired by Reindex. So a caller sees either "the campaign is not
there" or "the campaign is complete", never a third thing.
### 9.4 The database backup, as implemented in M9
A second recovery tool, deliberately not merged with the first. The bundle is a
logical, portable, human-readable copy of **one campaign** and is the supported
way to move a campaign between installations; the backup is a physical copy of
**this machine's whole database** and is what you take before an upgrade.
`backend/app/backup.py` uses SQLite's online backup API rather than a file copy,
because a copy taken while the application runs can read one page before a
transaction and another after it and produce a file that opens, reports a schema
and is quietly missing rows. It writes to a temporary name beside the
destination, runs `PRAGMA quick_check` against the finished file, and only then
renames it into place; it opens the source read-only, never overwrites an
existing backup, and leaves nothing behind on failure.
No path comes from a caller: the destination is derived from the database the
application already has open and the filename from the clock, so the endpoints
accept no body at all (H08).
**There is no restore endpoint, and that is a decision.** Restoring means
replacing the file the running process has open, which is how both copies are
lost at once. The procedure is in `DEVELOPMENT.md` and is a procedure precisely
because each step needs the application stopped.
## 10. Authoritative Narrative State
### 10.1 Do not retain the RPG state protocol as the product model