M9: a campaign you can actually get back

A campaign could already be exported and imported. What could not survive the
trip was everything that explains it: the state events behind the authoritative
document, the prompt each turn was actually given, the passages it was shown,
the summaries that carry long-story continuity, and which take belonged to which
turn. An imported campaign could be read and could no longer say why it was what
it was — and a manual correction, the one state change no narration explains,
was indistinguishable from something the story had established.

The bundle is now `ai-dnd-adventure-v3`, and the version is the design rather
than a side effect. Everything added here could have been another optional key,
the way persona, Save Points, narrative state and imported knowledge each were.
That mechanism stops working at exactly this addition: a v2 file with no prompt
provenance is ambiguous between "written before M9" and "written by M9 from a
campaign that has none", and those are different facts about a campaign. A
version number is how a recovery file states what it was capable of recording.
v1 and v2 still import, and every seam from pre-active-head onward is tested for
the rule that an older file is never reinterpreted under a newer assumption.

Two categories became three. "Chosen travels, derived is recomputed" was enough
until stored prompts had to be decided: they are derived, and they must travel
anyway. The test that separates evidence from cache is not "could this be
recomputed" but "would a recomputation answer the same question" — a rebuilt
search index answers the same question, a rebuilt prompt says what the turn
would be told *now*, which is the opposite of what the inspector is for.

Also here: a real SQLite backup, through the online backup API rather than a
file copy, taken while the application is running and verified before it is
kept; story cards settled as compatibility-only legacy data and taken out of the
narrator's prompt, because they were the untracked path around knowledge
authority that IMPORTED-KNOWLEDGE-DESIGN §73 already forbade; and no schema
change at all, proved against a database M8's own code wrote.

Three defects, found by running the milestone's own tests rather than by reading
them. Deleting a campaign leaked its FTS index rows, and SQLite then handed the
freed ids to the next source imported into any campaign, which failed with an
integrity error that Reindex could not repair — both ends are closed, and a
database already carrying the damage now repairs itself. An imported node with
no state snapshot was being stamped with the campaign's head state, so an Undo
to turn 2 showed what the story knew at turn 20. And the snapshot relink did not
persist at all, because it mutated a dict in place on a column SQLAlchemy tracks
by assignment: it looked correct in memory and wrote the wrong ids to disk.

Carrying per-turn prompts looked like it would halve the length of campaign that
can be restored. Measured — and after compressing them inside the file —
everything M9 added costs 12% of it: the import ceiling moves from about 318
turns to about 279, against a 100-turn certification target. The dominant cost
is not M9's at all. The per-position narrative state document is 74% of a
bundle, and v2 already carried it.

Backend 1,102 passed / 14 skipped / 0 failed. Frontend 145 passed. Lint,
production build and Docker build clean. Verified across two server processes
with two data directories, and in a real browser against a real narrator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qyn3oRd4D6pi72nKBG725B

backend/app/knowledge/fts.py

@@ -137,13 +137,53 @@ def index_line(heading_path: str, text_: str) -> str:
 
 
 def add(db: Session, chunk_id: int, heading_path: str, text_: str) -> None:
-    """Indexes one passage. The caller supplies the chunk's id as the rowid."""
+    """Indexes one passage. The caller supplies the chunk's id as the rowid.
+
+    `OR REPLACE`, and the reason is a defect M9 found rather than a defensive
+    habit. The rowid is a chunk's primary key, so a row already sitting at it is
+    by definition stale: the chunk that owned it does not exist, or is being
+    rewritten by the reindex that called this. Either way the new passage is the
+    truth and the old row is not.
+
+    Without it, an orphaned index row makes an ordinary import fail. SQLite
+    reuses primary keys once the highest row is gone, so the next campaign to
+    import a source is handed rowid 1 again, collides with an orphan, and gets a
+    500 from `INSERT` — and `clear_index` cannot clear the orphan, because it
+    finds index rows *through* the chunks, and there are none. That made Reindex,
+    which is the documented repair, unable to repair this. `REPLACE` closes it
+    from both ends: a leaked row is overwritten the moment the id comes round
+    again, so an existing database repairs itself rather than needing a
+    migration, and Reindex is the repair it is described as.
+
+    The leak itself is closed separately, in `importer.clear_campaign_index`.
+    """
     db.execute(
-        sql(f"INSERT INTO {TABLE} (rowid, text) VALUES (:id, :text)"),
+        sql(f"INSERT OR REPLACE INTO {TABLE} (rowid, text) VALUES (:id, :text)"),
         {"id": chunk_id, "text": index_line(heading_path, text_)},
     )
 
 
+def remove_adventure(db: Session, adventure_id: int) -> int:
+    """Drops every index row belonging to one campaign. Returns how many.
+
+    Scoped through the chunks, which is the only place the campaign is
+    recorded — the index deliberately holds no copy of it
+    (see "The table" above). So this has to run **before** the chunk rows go,
+    which is what `importer.clear_campaign_index` is for.
+    """
+    result = db.execute(
+        sql(
+            f"""
+            DELETE FROM {TABLE} WHERE rowid IN (
+                SELECT id FROM knowledge_chunks WHERE adventure_id = :adventure_id
+            )
+            """
+        ),
+        {"adventure_id": adventure_id},
+    )
+    return result.rowcount or 0
+
+
 def remove_chunks(db: Session, chunk_ids: list[int]) -> None:
     """Drops passages from the index by id.
 

backend/app/knowledge/importer.py

@@ -362,6 +362,28 @@ def clear_index(db: Session, source: models.KnowledgeSource) -> None:
     db.expire(source, ["chunks"])
 
 
+def clear_campaign_index(db: Session, adventure: models.Adventure) -> int:
+    """Removes a whole campaign's lexical index rows. Returns how many.
+
+    Called before a campaign is deleted, and it has to be: the FTS index is a
+    virtual table, so no foreign key reaches it and no `ON DELETE CASCADE`
+    covers it. Deleting a campaign cascades `knowledge_sources` to
+    `knowledge_chunks` and stops there, leaving one index row per passage
+    belonging to a chunk that no longer exists.
+
+    Found in M9. The leak is not cosmetic. SQLite hands out the lowest free
+    primary key, so once the highest chunk is gone the *next* source imported
+    into *any* campaign is given a chunk id that an orphan already occupies, and
+    the import fails with an integrity error — a 500 on an ordinary upload, in a
+    campaign that has nothing to do with the deleted one. `fts.add` now repairs
+    such a collision when it meets one; this stops it happening.
+
+    Vectors and passages need no equivalent, because both are real tables whose
+    foreign keys cascade.
+    """
+    return fts.remove_adventure(db, adventure.id)
+
+
 def delete_source(db: Session, source: models.KnowledgeSource) -> None:
     """Removes a source and everything derived from it.
 

backend/app/knowledge/inject.py

@@ -50,10 +50,17 @@ from .records import Candidate, Result
 
 #: Share of the non-protected budget that retrieved knowledge may spend.
 #:
-#: Story cards already take up to 40% (`CARD_BUDGET_SHARE`), and the history is
-#: what is left. A third is enough for several passages at the chunker's
-#: typical size and leaves the majority of the window to the story itself,
-#: which is the thing the reader came for.
+#: A third is enough for several passages at the chunker's typical size and
+#: leaves the majority of the window to the story itself, which is the thing the
+#: reader came for.
+#:
+#: This share was chosen when story cards could take up to 40% of the same
+#: budget and the history took what was left. M9 removed that injection
+#: (`IMPORTED-KNOWLEDGE-DESIGN.md` §73), so the history now gets that 40% back.
+#: The number here is deliberately unchanged: a third of the budget was chosen
+#: as the right amount of *imported material* to put in front of the narrator,
+#: not as a leftover, and raising it because room appeared would be changing
+#: retrieval behaviour under cover of a portability milestone.
 KNOWLEDGE_SHARE = 0.33
 
 #: What each class may take of the knowledge budget. Canon may take all of it;