Let a backup carry a story that went two ways

A bundle had one list and a forked adventure has two stories, so export was
emitting every branch's turns interleaved by index — a mangled story rather
than lost data, and unreachable only because forking has no UI yet.
`ai-dnd-adventure-v2` carries the branches, the depth each one left its
parent at, which attempt at every turn is the story, and what each node
left behind. That last one is not decoration: the after-snapshots are what
a branch switch puts back, and a bundle without them imports a tree nobody
can switch inside.

`app/bundle.py` owns both formats and nothing else knows either. The v1
reader stays — those files are already on people's disks — and it is now
the only place a `variants` array exists anywhere.

The rule the module is built on is that a bundle carries what was chosen
and never what is derived. The head branch, the fork points, the live flags
and the anchors are decisions somebody made. The lineage, the head depth,
the legacy `index` and the variant ordinals are computed from those and are
rebuilt on the way in, because a bundle is a text file anybody can edit and
a derived field shipped beside its source is a chance for the file to
disagree with itself where no read would report it.

`index` is the one that stops being academic here. It agreed with `depth`
until SP5, and this is the first writer that has to fill it for a forked
story, where two branches both hold a node at depth 4. It is allocated one
per turn instead: siblings share it, no two coordinates do.

Everything a hand-edited file can get wrong about the shape of a tree is a
400 raised before the adventure row exists, because a half-applied import
is exactly the failure this phase exists to end — a story that goes quiet.
A file wrong about which attempt is live is corrected rather than refused;
that is an invariant of the database, not of the format.

Measured on the 600-action fixture: 587 kB to 911 kB, and all of the
increase is the outcomes at 489 B a node — the coordinates themselves save
57.5 B a node against the old turn-and-variants shape. Twenty forks add
660 B. 4.3% of the import body cap.

381 tests green, 16 of them new in test_bundle_v2.py. No migration, no
vacuum owed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g

plan/14-phase-story-tree.md

@@ -37,8 +37,10 @@ existed for undo and delete; see the trap note below. Two rows want a footnote:
   stale exactly as before. What *has* changed is that the machinery to fix it now exists
   — an edit could write a sibling and switch to it, which is a retry the player typed —
   so it is a small change whenever it is wanted.
-- **The 1NF violation** is resolved in the database. The `variants` array survives in
-  exactly one place: the v1 export bundle, which SP6 replaces.
+- **The 1NF violation** is resolved. Nothing writes a `variants` array any more, in the
+  database or out of it — SP6 replaced the bundle that was its last producer, and the
+  array survives only in the v1 *reader*, which exists so files already saved still
+  import.
 
 ## Design decisions (settled 2026-08-16)
 
@@ -610,6 +612,88 @@ bundle's linear actions plus `variants` onto one branch with siblings.
 **Verify:** v2 round-trip of a branched adventure is lossless; a v1 bundle still imports;
 a bundle claiming more branches than rows is rejected rather than half-applied.
 
+**Done, 2026-08-18** (branch `sp6-bundle-v2`). **381 tests green**, the 365 SP5 finished
+with plus 16 in the new `test_bundle_v2.py`. `app/bundle.py` owns both formats; the two
+endpoints in `routers/adventures.py` are a delegation and the shared plumbing, and the
+`variants` array now exists nowhere but the v1 *reader*.
+
+**The rule the module is built on: a bundle carries what was *chosen*, never what is
+*derived*.** The head branch, the fork points, the live flags and the anchors are
+decisions somebody made, and they are in the file. `lineage`, the head *depth*, `index`
+and the variant ordinals are computed from those and are rebuilt on the way in. That is
+not tidiness — a bundle is a text file anybody can edit, and every derived field shipped
+beside its source is a chance for the file to disagree with itself in a way no read would
+report. It is also the answer to "is the round trip lossless?": everything omitted is
+reconstructed, and the tests assert the reconstruction rather than the bytes.
+
+Six things worth not rediscovering:
+
+- **`depth` cannot be the legacy `index`, and this is where that stops being academic.**
+  They agreed until SP5, and a bundle is the first writer that has to fill `index` for a
+  *forked* story — where two branches both have a node at depth 4. `index`'s one
+  remaining job is handing the next row a number nothing else holds, which is a fact
+  about the adventure rather than about a path, so the import allocates one per turn in
+  bundle order: siblings share it, the way SP4 leaves them, and no two coordinates do.
+- **Validation happens before the adventure row exists.** Everything a hand-edited file
+  can get wrong about the shape of a tree — a node naming a branch that is not listed, a
+  fork with no depth, a branch forking from one listed after it — is a 400 raised by
+  `plan`, which touches no session. The alternative is an adventure holding a story with
+  a hole in it, and this whole phase exists because a story with a hole in it fails by
+  going quiet.
+- **A branch may only fork from one listed before it.** That is how the export writes
+  them, and requiring it buys acyclicity for the price of a comparison — a cycle in the
+  parent chain would be an import that never returns rather than one that fails.
+- **`{}` and absent are different snapshots.** An empty `state_after` means "this node
+  left an empty scoreboard behind"; a missing one means "nobody knows, leave the live
+  state alone" (`attempts.restore_state`). Trimming empty dicts on the way out would have
+  saved eighteen bytes a row and turned an undo that clears a score into one that leaves
+  it standing. Only `worldDelta`, which is display, is dropped when empty.
+- **A file is allowed to be wrong about which attempt is live, and the import corrects it
+  rather than refusing.** "Exactly one sibling in a group is live" is an invariant of the
+  *database*, not of the format; a coordinate with none is a turn no read can see, so the
+  first attempt is made live. That is a different class from a missing branch, which is
+  structure, and is refused.
+- **`limits.MAX_BRANCHES_PER_ADVENTURE` (1000) has no live counterpart.** Forking is a
+  POST that adds one row and has no cap of its own, so this is the one bundle cap that
+  does not mirror something creation enforces. Worth closing if branch management ever
+  makes forking cheap to repeat.
+
+**The verify line above was slightly wrong, and the code does the honest version.** "More
+branches than rows" fails on an adventure with no actions, which legitimately has one
+branch and no rows. It became two rules instead: a cap on the branch list, and *every
+branch a node names must exist*.
+
+Measured with `tools/measure_bundle.py` on the 600-action `--rich` fixture — 600 turns,
+750 nodes, because 150 of them were retried:
+
+| | bytes | vs v1 |
+|---|---|---|
+| v1 shape | 587,475 | — |
+| **v2** | **911,229** | **1.551×** |
+| v2 without the outcomes | 544,318 | 0.927× |
+
+**The tree is free; the outcomes are what cost.** Coordinates *save* 57.5 B a node
+against v1's turn-and-variants shape, and the entire 1.55× is `state_after` /
+`world_state_after` at 489 B a node — which are there because a bundle without them
+imports a tree nobody can switch inside. Twenty forks add 660 B to the same file, **33 B
+a branch**, so the format is as indifferent to fork count as the reads are. The longest
+adventure production holds exports at 4.3 % of `MAX_IMPORT_BODY_BYTES`.
+
+**Two lines of the SP0 baseline changed, not the one it predicted.** The note in
+`test_story_tree_baseline.py` allowed for the `format` assertion; the `variants` array in
+`test_export_keeps_retry_attempts` is the same fact from the other side — a bundle with
+coordinates has no use for a repeating group. Everything else in that file still passes
+unmodified.
+
+**No migration, no vacuum.** SP6 adds no column and rewrites no row.
+
+**And one trap, paid for once.** `tools/measure_bundle.py` imported `app` before
+`tools.stress_session`, which is what points `AIDND_DB_PATH` at a throwaway file —
+`app.database` reads it at module scope. It fails by *working*: the first run seeded a
+synthetic user and adventure into the local `backend/data.db` and printed perfectly good
+numbers, and only the second run tripped over the unique email. Anything importing that
+harness must import it first, and the file now says so where the imports are.
+
 ### SP7 — Frontend: full tree visualisation
 
 `VariantPager` is removed. A spatial tree view replaces it, plus switch, rename and

plan/STATUS.md

@@ -78,37 +78,41 @@ needed; nothing requires reading a row of anyone's story.
 
 ## Pick up here
 
-**`plan/14-phase-story-tree.md`, SP6 — export/import v2.** SP0–SP5 are done and green
-(**365 tests**); **nothing is deployed yet**. The tree is complete as a storage model: a
-retry writes a sibling node, continuing from a discarded attempt forks a branch, and
-`GET /branches`, `POST /branches/{id}/switch` and `POST /actions/{id}/fork` are the
-endpoints SP7's tree view will be drawn on. What is left is the bundle format (SP6), the
-frontend (SP7) and dropping the legacy columns (SP8).
+**`plan/14-phase-story-tree.md`, SP7 — the frontend.** SP0–SP6 are done and green
+(**381 tests**); **nothing is deployed yet**. The tree is complete everywhere except the
+screen: a retry writes a sibling node, continuing from a discarded attempt forks a branch,
+and a backup carries the whole thing (`ai-dnd-adventure-v2`, with the v1 reader kept so
+existing bundles still import). What is left is the frontend (SP7) and dropping the legacy
+columns (SP8).
 
-**Do SP6 before SP7, and the reason is a live gap.** A forked adventure has no honest v1
-export — the format has one story and there are two — so export currently emits every
-branch's turns interleaved by `index`, which reads as a mangled story. Nobody can reach
-that state through the product yet, because forking has no UI until SP7. That ordering is
-the whole mitigation, so keep it.
+**SP7 is the release gate, and it is unscoped.** `VariantPager` comes out, a spatial tree
+view replaces it, and branch management — switch, rename, delete-with-confirm — is a hard
+dependency rather than a nice-to-have, because nothing auto-prunes and storage otherwise
+grows without limit. `api.js` gains `GET /branches`, `POST /branches/{id}/switch` and
+`POST /actions/{id}/fork`, which SP5 built for exactly this.
+
+**Drive the 600-action `--keep` fixture in a browser by hand.** That is SP7's own verify
+line and the standing open gap in this project: the scroll path has never been driven by
+hand and has already hidden one bug. A vitest + jsdom harness covers the prepend
+arithmetic, but jsdom has no layout, so scroll position still needs eyes.
 
 **The schema is live in code but not on production.** When this ships, the deploy needs
 one `VACUUM FULL actions;` on the direct (non-`-pooler`) endpoint afterwards — SP1's
 migration rewrites every row and SP4's rewrites it three times more, so **two vacuums are
-owed and one run settles both**. SP3's and SP5's changes touch `adventures` only (SP5 adds
-no migration at all) and need none. See the 144 MB lesson at the top of this file.
+owed and one run settles both**. SP3's, SP5's and SP6's changes need none (SP5 and SP6 add
+no migration at all). See the 144 MB lesson at the top of this file.
 
-Three things to carry into SP6:
+Three things to carry into SP7:
 
-- **The `variants` array now exists in exactly one place: the export bundle.** Nothing in
-  the database holds one. `export_adventure` folds each sibling group back into the shape
-  the v1 reader expects, and `_imported_turn` splits one back out into rows. Those two
-  functions are the whole v1 surface, and v2 replaces them.
-- **A v2 bundle needs branches, `live`, and both after-snapshots.** `state_after` /
-  `world_state_after` are what a branch switch restores; a bundle that carried the
-  actions but not the outcomes would import a tree nobody could switch inside.
-  `limits.check_bundle_lists` has to learn about branches too.
-- **Weigh new columns in bytes.** `actions` is already the table that fills the disk.
-  `tests/test_egress.py` has byte ceilings — they will tell you.
+- **The bundle is the one thing here a migration can never reach.** `app/bundle.py` owns
+  both formats and nothing else knows either. Its rule — *carry what was chosen, never
+  what is derived* — is worth borrowing anywhere else state has to leave the database.
+- **`variant_count` and `variant_index` die with the pager.** SP4 left them as a
+  maintained cache of the sibling group's shape because the pager reads both for every
+  row of a page. They are dead the moment the tree view replaces it, and SP8 drops them.
+- **Nothing on the screen has ever seen a second branch.** Forking has no UI, which is
+  why the v1-export gap could be left open through SP5 — SP7 is the subphase that makes
+  a fork reachable, so it is also the one that makes every branch-shaped bug reachable.
 
 And one known cost, not a bug: the two memory marks are a single pair on the adventure,
 so switching branches makes the mark on the branch being left unreadable from the new one
@@ -125,6 +129,44 @@ drive it before rewriting it.
 
 ---
 
+## What happened on 2026-08-18, part three — the tree, SP6
+
+The backup learned the tree. `ai-dnd-adventure-v2` carries branches, the fork point each
+one left its parent at, which attempt at every turn is the story, and what each node left
+behind — that last one because it is what a branch switch puts back, and a bundle that
+imported a tree nobody could switch inside would be a backup of the wrong thing. The v1
+*reader* stays: those files are already on disk. **381 tests green**, 16 of them new in
+`test_bundle_v2.py`. Branch `sp6-bundle-v2`, no migration, no vacuum.
+
+The gap SP5 left is closed — a forked adventure now has an honest export — so the
+ordering constraint that has governed the last two subphases is discharged, and SP7 is
+free.
+
+Three things to carry forward:
+
+- **Carry what was chosen, never what is derived.** The head branch, the fork points, the
+  live flags and the anchors are decisions, and they are in the file. `lineage`, the head
+  depth, the legacy `index` and the variant ordinals are computed from those, so they are
+  rebuilt on import instead. A bundle is a text file anybody can edit, and a derived field
+  shipped beside its source is a chance for the file to contradict itself in a way no read
+  reports. It also turns "is the round trip lossless?" into a testable question: every
+  omitted field is reconstructed, and the tests assert the reconstruction.
+- **Check the shape before creating the row.** A node naming a branch the file does not
+  list is a 400 raised by a pure function, not a half-written adventure. The failure this
+  phase exists to end is a story that goes quiet, and a half-applied import is exactly
+  that.
+- **The tree is free; the outcomes are what cost.** On the 600-action fixture the bundle
+  goes from 587 kB to 911 kB, and *all* of it is `state_after`/`world_state_after` at
+  489 B a node — the coordinates themselves save 57.5 B a node against v1's
+  turn-and-variants shape. Twenty forks add 660 B. 4.3 % of the import body cap at
+  production's longest adventure.
+
+Paid for once, and worth not repeating: a new measuring script imported `app` before
+`tools.stress_session`, which is what redirects `AIDND_DB_PATH` at a throwaway file. It
+failed by *working* — the first run seeded a synthetic user into the local `data.db` and
+printed good numbers; the second tripped over the unique email. **A harness that decides
+where the database lives has to be imported before anything that reads it.**
+
 ## What happened on 2026-08-18, part two — the tree, SP4 and SP5
 
 A retry stopped rewriting a row, and a story learned to go two ways at once.