# Adventure Storyteller — Data Model **Status:** v1.0 conceptual model aligned to Phase 0B decisions **Purpose:** Define the persistent information the application must represent, independent of the final fork or database implementation. ## 1. Design Goals The data model must support persistent interactive stories, complete authoritative history, non-destructive branching, checkpoints and rollback, genre-independent narrative state, long-term memory, imported local knowledge, prompt/context provenance, future image/video/audio/TTS/STT generation, export/restore, and local-only operation. The same core schema should work for fantasy, science fiction, mystery, horror, historical fiction, westerns, and other narrative genres. ## 2. Core Principles ### 2.1 Application-owned authority The database is the source of truth. The model may propose narration and state changes, but those proposals become authoritative only after application validation. ### 2.2 Non-destructive history Accepted turns are historical records. Going backward should move the active story head or create a branch, not silently delete accepted history. ### 2.3 Historical reconstruction The system must answer both: - What is true now? - What was true at a particular earlier turn on a particular branch? ### 2.4 Genre neutrality Avoid fantasy- or science-fiction-specific core fields. Use generic concepts such as characters, locations, organizations, items, vehicles, facts, relationships, conditions, scenes, and story threads. ## 3. Conceptual Entity Map ```text Campaign ├── Branches │ └── Turns │ ├── Prompt Snapshot │ ├── State Version │ ├── Scene Snapshot │ └── Retrieval Records ├── Checkpoints ├── Narrative Entities ├── Facts ├── Relationships ├── Story Threads ├── Memories ├── Summaries ├── Knowledge Sources │ └── Knowledge Chunks └── Media ├── Media Jobs └── Media Assets ``` The exact SQL schema may differ from this conceptual model. ## 4. Campaign A campaign is the top-level story container. ```yaml campaign: id: uuid title: string created_at: timestamp updated_at: timestamp active_branch_id: uuid active_head_turn_id: optional uuid status: active | archived story_profile: genre: string subgenre: optional string tone: optional string style: optional string point_of_view: optional string tense: optional string ``` The active head is **stored on the campaign, not derived** from its newest turn. This is the concept M3 implemented; the current implementation carries it as a branch reference plus a depth on that branch rather than as a turn id, which is an equivalent coordinate and is what the export format records. What matters conceptually is that the position is a decision the campaign remembers: two campaigns holding identical turns can be being read at different places, and nothing about the turns themselves can tell them apart. Campaigns also store durable narrator rules and model configuration. Potential model roles: - narrator, - state extractor, - summarizer, - embedding model. For v1, all model roles should use local Ollama-compatible models. ## 5. Branch A branch represents one valid continuation of story history. ```yaml branch: id: uuid campaign_id: uuid name: string created_at: timestamp created_from_branch_id: optional uuid fork_turn_id: optional uuid tip_turn_id: optional uuid disposition: active | retained | disposable status: active | archived ``` Rules: - branches may share ancestral turns, - shared history should not be duplicated unnecessarily, - creating a branch must not modify the source branch, - the campaign active head may sit behind the retained branch tip after Undo, - Redo moves the active head forward while the prior continuation remains selected, - a new write below the retained tip creates a new continuation and leaves the old future retained/disposable. `disposition` above is conceptual. As implemented in M3 it is stored as **the fact that produced it** rather than as a word: a branch records the depth a divergent write left it at, and when. No value means active; a value means the story past that depth is retained history no active head is reading. The shallowest departure wins if a branch is left more than once. Two properties of that representation are deliberate and worth carrying in this document: - **Nothing reads it to decide behavior.** Whether Redo is available, what the transcript shows, and which continuation a write belongs to are all decided by the lineage. A stale or hand-edited disposition therefore cannot make the story wrong; it can only mislead a cleanup or recovery feature about what is abandoned. - **It survives export and import.** Every row of an abandoned line is exported either way, so the disposition is the only thing distinguishing it from an active one in a restored campaign. Detailed behavior will be defined separately in `STORY-BRANCH-SEMANTICS.md`; the architecture is recorded in ADR 012. ## 6. Turn A turn is one accepted story interaction: ```text user input -> narrator response -> accepted state transition ``` ```yaml turn: id: uuid campaign_id: uuid branch_id: uuid parent_turn_id: optional uuid created_at: timestamp input: mode: action | dialogue | direction | continue text: string output: narration: string generation: provider: ollama model: string generation_settings: object prompt_snapshot_id: uuid state_before_id: uuid state_after_id: uuid scene_snapshot_id: optional uuid status: pending | accepted | failed | superseded ``` The root turn has no parent. An accepted historical turn should not silently disappear if another continuation is chosen. ## 7. Alternate Take The system may need to distinguish: - a different narrator response to the same user action, - a genuinely different story branch. Conceptual form: ```yaml take: id: uuid turn_request_id: uuid output_text: string model_metadata: object selected: boolean ``` Physical representation remains implementation-specific. The selected AI-DnD base already models alternate takes within its lineage machinery; production should retain that approach if it satisfies the required Retry/select/retention semantics without forcing a separate table. ## 8. Checkpoint ```yaml checkpoint: id: uuid campaign_id: uuid turn_id: uuid name: string notes: optional string created_at: timestamp ``` A checkpoint is a named pointer to a recoverable story position. It should normally remain tied to the turn where it was created. Restoring it moves the campaign active head; it does not delete later retained history. A new branch is created on the first divergent write after restore, not merely because the checkpoint was opened. As implemented in M4, the pointer is a **coordinate rather than a turn id**: `(branch_id, depth)`, which is the same pair §4 records as the campaign's active head and which `head.node_at` resolves. This is the equivalence §4 already draws between a turn reference and a branch-plus-depth, applied to the same position from the other end, and it is not a shortcut — it is the more correct pointer of the two for this data model: - **A coordinate follows a retry; a row id does not.** One coordinate holds every attempt at a turn and exactly one of them is live (§7). A Save Point names the turn, so it must land on whichever take the story currently tells. Pinning the row would leave the pointer on a superseded attempt the reader cannot see. - **The user-facing term is Save Point**; `checkpoint` remains the internal name (`BROWSER-UX-SPEC.md` §23). The row carries the name, an optional note, the coordinate, and its timestamps. It carries **no** copy of the transcript, the state, the prompt, a memory, a summary, or a branch's contents. Everything a restore produces comes from the retained history the coordinate points into. The retry case is what settles the coordinate-versus-turn-id question, and M4 closeout measured it rather than arguing it. A Save Point named a turn whose live row was id 17; retrying that turn made id 17 dead and id 18 live at the same coordinate; the Save Point resolved to id 18 and restored correctly. A row id would have pinned a take the story no longer tells. **A Save Point names a story position, not a particular take of it.** Three further properties of the implemented model, recorded so they are decided rather than incidental: - **Names are not unique**, and nothing requires them to be. No product requirement asks for uniqueness, and two names for one moment is a reasonable thing for a player to want. - **Several Save Points may name the same position.** Same reason. - **Ordinary list presentation is newest-created first.** Story order is not something the list can honestly claim: depths on lines that have parted company are not comparable, so ordering by depth would draw a sequence that no reading of the story passes through. When each was made is a fact about all of them. None of this is genre-specific. A coordinate is a position in a story; what the state at that position *contains* is M5's question, and changing it does not change what a Save Point is. Two consequences worth recording here: - **Restore reuses the campaign's one head-movement mechanism.** It resolves the coordinate and moves the head; nothing is reconstructed and nothing is deleted. The branch half of the head moves only when the coordinate is not on the path being read, which is what makes a Save Point on a departed line restorable at all — and what keeps a Save Point in a shared prefix from dragging the reader off the line they chose. See `TECHNICAL-DESIGN.md` §8.8. - **A checkpoint is durable against everything but its own explicit deletion.** No pass removes one for going stale, sitting behind the head, or naming a line the story left (`STORY-BRANCH-SEMANTICS.md` §19). Deleting a *branch* does not remove one either: the deletion is refused while a checkpoint names any position in the subtree, and the user deletes the checkpoint first (§19.1). Deleting the whole campaign removes them, which is what deleting a campaign means. ## 9. Narrative Entity An entity is a persistent thing or concept in the fictional world. ```yaml entity: id: uuid campaign_id: uuid type: string name: string aliases: [string] description: string status: active | inactive | destroyed | dead | unknown created_turn_id: optional uuid metadata: object ``` Recommended built-in categories: - character, - location, - organization, - item, - vehicle, - creature, - structure, - concept, - other. These are descriptive categories, not separate game systems. ## 10. Character ```yaml character: entity_id: uuid role: optional string description: string current_location_id: optional uuid condition: [string] personality_notes: [string] goals: [string] secrets: [string] ``` Optional visual continuity fields: ```yaml visual_profile: apparent_age: optional string build: optional string hair: optional string eyes: optional string clothing: optional string distinctive_features: [string] continuity_notes: [string] ``` ## 11. Location ```yaml location: entity_id: uuid description: string parent_location_id: optional uuid current_status: optional string visual_profile: architecture: optional string environment: optional string lighting: optional string signature_features: [string] continuity_notes: [string] ``` ## 12. Organization Organizations may represent factions, governments, companies, guilds, military units, religious organizations, or informal groups. ```yaml organization: entity_id: uuid purpose: optional string current_status: optional string ``` Specific characteristics should usually live in facts and relationships. ## 13. Items and Vehicles Items and vehicles remain generic entities. Examples: - silver key, - longsword, - encrypted data crystal, - survey ship, - horse-drawn carriage. Possession and location should normally be represented as relationships or facts: ```text Aldric --possesses--> Silver Key Persephone --docked_at--> Ceres Station ``` ## 14. Fact Facts represent assertions about the story world. ```yaml fact: id: uuid campaign_id: uuid subject_entity_id: optional uuid predicate: string object_entity_id: optional uuid value: optional scalar_or_object authority: string source_type: string source_id: optional uuid created_turn_id: optional uuid invalidated_turn_id: optional uuid status: active | superseded | disputed ``` Examples: - Mara knows Aldric has the silver key. - The Persephone cannot travel faster than light. - Edrin disappeared three weeks before the campaign began. - The eastern bridge collapsed during Turn 47. Minimum authority categories: - campaign_canon, - accepted_story, - current_state, - imported_canon, - reference, - heuristic, - inspiration. Exact context behavior will be defined in `CONTEXT-AND-MEMORY.md`. ## 15. Relationship ```yaml relationship: id: uuid campaign_id: uuid source_entity_id: uuid target_entity_id: uuid type: string status: string description: optional string created_turn_id: optional uuid ended_turn_id: optional uuid ``` Examples: - Aldric -> trusts -> Mara - Mara -> member_of -> Circle of Ash - Silver Key -> belongs_to -> Aldric - Persephone -> docked_at -> Ceres Station Relationship types should remain extensible. ## 16. Story Thread Story threads track unresolved or resolved narrative business. ```yaml story_thread: id: uuid campaign_id: uuid title: string description: string status: open | dormant | resolved | abandoned importance: optional number_or_label opened_turn_id: optional uuid resolved_turn_id: optional uuid ``` These are narrative continuity tools, not RPG quests. ## 16A. Derived Context Tables (M6, as implemented) Two tables and one column carry M6's derived context. All three are derived data: deleting them changes no accepted history, no authoritative state and no head position. ```text summaries id, adventure_id text branch_id, depth the coordinate of the last node covered source_start, source_end the stretch of story summarized trigger "interval" (generated) or "manual" (reader-written) model_name, created_at derived_status one row per (adventure, kind) kind "memory" | "summary" | "embedding" status "ok" (did work) | "idle" (nothing pending) | "failed" detail, failures last_attempt_at, last_success_at memories.authority "accepted_story" | "heuristic" ``` `summaries` mirrors the shape `memories` already had, deliberately: both are derived rows anchored to a coordinate on a path, and both are filtered by the same lineage clause — and both are also the *input* to the next round of derived work, which is why summary generation reads `summaries.current` rather than any campaign-global field. `adventures.story_summary` is retained as the reader's edit surface and the export field, mirroring whichever summary is eligible. It carries no lineage of its own and nothing authoritative reads it. ## 17. State Version The system must reconstruct authoritative state at any retained turn. ```yaml state_version: id: uuid campaign_id: uuid turn_id: optional uuid parent_state_version_id: optional uuid created_at: timestamp state_hash: optional string ``` Possible implementation models: ### A. Full snapshots Simple restore, but duplicates data. ### B. Event sourcing Excellent auditability, but requires replay. ### C. Hybrid Validated events plus periodic/current snapshots. **Selected for v1: Hybrid.** Store validated authoritative events plus efficient state snapshots/cache for normal reads and restore. ## 18. State Change Event If the hybrid/event model is selected: ```yaml state_event: id: uuid campaign_id: uuid turn_id: uuid event_type: string payload: object sequence: integer ``` Potential event types: - entity_created, - set_entity_status, - set_entity_attribute, - fact_added, - fact_invalidated, - relationship_added, - relationship_ended, - thread_opened, - thread_resolved, - current_location_set, - possession_set, - scene_set. Event semantics must be explicit and typed. Prefer unambiguous absolute assignments for mutable values. If incremental operations are ever needed, encode the operation explicitly (for example `increment_value`) rather than relying on one numeric field whose interpretation is implicit. Events must be schema-validated, semantically checked where deterministic rules exist, and accepted by the application before commit. ## 19. State Proposal The model's extracted state proposal must be distinct from accepted state. ```yaml state_proposal: id: uuid turn_id: uuid model: string raw_output: string parsed_payload: object validation_status: accepted | partially_accepted | rejected | repair_required ``` The model must never write directly to authoritative state tables. The production protocol must not depend on AI-DnD-style ambiguous relative deltas; see ADR 010. ## 20. Scene Snapshot A scene snapshot captures the immediate narrative situation. ```yaml scene: id: uuid campaign_id: uuid branch_id: uuid source_turn_start_id: optional uuid source_turn_end_id: optional uuid location_id: optional uuid time_description: optional string mood: optional string environment: optional string participants: [uuid] significant_objects: [uuid] current_actions: [string] visual_notes: [string] continuity_notes: [string] ``` Scene snapshots support: - current context, - narrative continuity, - future image generation, - future multi-turn video/storyboard generation. ## 21. Summary Summaries compress history but never replace authoritative history. ```yaml summary: id: uuid campaign_id: uuid branch_id: uuid type: campaign | arc | rolling | turn_range source_start_turn_id: uuid source_end_turn_id: uuid text: string created_at: timestamp model: optional string ``` Summaries are derived data. Branch changes must invalidate or lineage-filter incompatible summaries. ## 22. Memory ```yaml memory: id: uuid campaign_id: uuid branch_scope: optional uuid source_turn_id: optional uuid type: string text: string authority: string importance: optional number embedding_ref: optional string created_at: timestamp ``` Potential types: - event, - character, - relationship, - location, - promise, - discovery, - conflict, - heuristic. A retrieved memory does not automatically become canon. ## 23. Knowledge Source A knowledge source is a user-imported local file or manually authored campaign document. ```yaml knowledge_source: id: uuid campaign_id: optional uuid title: string source_type: file | manual classification: canon | reference | inspiration original_filename: optional string content_hash: string enabled: boolean imported_at: timestamp metadata: object ``` Initial supported file types should be `.txt` and `.md`. ## 24. Knowledge Chunk ```yaml knowledge_chunk: id: uuid source_id: uuid sequence: integer text: string heading_path: optional string token_count: optional integer embedding_ref: optional string metadata: object ``` Requirements: - preserve provenance, - preserve chunk order, - permit re-indexing, - never execute imported content. ## 24A. Imported Knowledge Tables (M7, as implemented) Three tables, and the boundary between them is the boundary between what the reader gave the campaign and what the machine derived from it. ```text knowledge_sources the file, and the reader's judgements about it id, adventure_id campaign-scoped; no branch coordinate, deliberately title, original_filename the filename is metadata and is never a path classification "canon" | "reference" | "inspiration" enabled out of retrieval without being deleted visibility "normal" | "hidden" (narrator-only) always_include Canon only: in force whatever the scene is content the accepted text, as decoded content_hash SHA-256 of the normalized text; the duplicate test byte_size, media_type parser_version what produced the passages now on disk chunking_version index_state, index_detail "pending" | "ready" | "failed" — the lexical half embed_state, embed_detail "idle" | "pending" | "ok" | "failed" — the semantic half notes, imported_at, updated_at knowledge_chunks derived: a deterministic function of the content id, source_id, adventure_id chunk_index, heading_path text, token_count, content_hash knowledge_embeddings derived: rebuildable, and its own table so that id, chunk_id, adventure_id "rebuild the semantic index" is one DELETE vector packed float32, as `memories.embedding_blob` is model, dimensions what makes a stale vector detectable parser_version, chunking_version, created_at knowledge_fts a SQLite FTS5 virtual table over heading + text, keyed by chunk id. Not describable in SQLAlchemy metadata, so it is attached to `knowledge_chunks` as a DDL hook and travels with it. ``` **Only the first two columns of a source are not derivable**: its content and its classification. Everything else about a source is metadata describing one of those two, and everything in the other two tables is rebuilt from the content by a deterministic chunker. That is what lets the export carry the source alone (§29) and what makes a reindex safe. There is deliberately **no branch coordinate** anywhere here. An imported file is campaign source material and does not become a different file because the story forked (`CONTEXT-AND-MEMORY.md` §39). The rule that abandoned story content must not reach the prompt is met at the *query* instead: the retrieval query is built from the head-capped lineage and the authoritative state at the position being read, never from the uncapped action table. A future knowledge record *derived* from story history would need a coordinate; M7 introduces no such record. ## 25. Retrieval Record Every turn should record which memories or knowledge chunks were supplied to the narrator. ```yaml retrieval_record: id: uuid turn_id: uuid source_kind: memory | knowledge | summary | fact source_id: uuid retrieval_method: lexical | semantic | hybrid | forced score: optional number rank: integer ``` This lets the prompt inspector answer: > Why did the narrator know this? ### As implemented (M6 for memories, M7 for imported knowledge) There is no `retrieval_record` table. The record lives in the turn's own context snapshot, which every turn already stores, and it carries the **rendered text** alongside the identifiers: ```text context_snapshot.knowledge used[] source_id, title, filename, classification, visibility, chunk_id, chunk_index, heading_path, always_include, mode ("lexical" | "semantic" | "hybrid" | "always"), lexical, semantic, cosine, score, tokens, text, rendered, prompt_tokens dropped[] the same, plus why there was no budget for it suppressed[] the same, plus the passage it repeated terms, considered, floor, budget, spent semantic_used, semantic_note, scan_truncated ``` Carrying the text rather than a foreign key is the whole point. A separate table of ids would turn every historical turn's evidence into dangling references the moment a source were deleted, and §49-50 of `IMPORTED-KNOWLEDGE-DESIGN.md` require the opposite: a turn must go on being able to say what it was given. ## 26. Prompt Snapshot ```yaml prompt_snapshot: id: uuid turn_id: uuid created_at: timestamp system_instructions: text campaign_context: text_or_structured state_context: text_or_structured summary_context: text_or_structured memory_context: text_or_structured knowledge_context: text_or_structured recent_history: text_or_structured user_input: text token_accounting: object ``` The physical representation may be compressed or normalized. Reproducibility and inspection are the requirements. ## 27. Media Job Not required for v1 behavior, but the data model should not prevent it. ```yaml media_job: id: uuid campaign_id: uuid scene_id: optional uuid source_turn_start_id: optional uuid source_turn_end_id: optional uuid type: image | video | audio | tts | stt provider: string model: string status: queued | running | completed | failed request_payload: object created_at: timestamp completed_at: optional timestamp ``` ## 28. Media Asset ```yaml media_asset: id: uuid campaign_id: uuid media_job_id: optional uuid scene_id: optional uuid type: image | video | audio | tts | stt file_path: string metadata: object created_at: timestamp ``` Potential metadata includes prompt, seed, model, workflow, dimensions, duration, character references, and source turn range. ## 28A. Media Extension Points (M10, as implemented) M10 implemented the seams the two sections above describe, and the implementation is mostly an account of what it did **not** build. ```text visual_profiles the one thing M10 persists id, adventure_id campaign-scoped; no branch coordinate, deliberately entity_key the M5 narrative-state key: "mara", "the_office" descriptors open map of trait -> value features list of distinctive visible things style_notes free text about how it should be rendered created_at, updated_at UNIQUE (adventure_id, entity_key) ``` No `scenes` table. No `media_jobs` table. No `media_assets` table. **The scene snapshot of §20 already exists**, and has since M5. It is `narrative_state["scene"]` — `summary`, `location`, `present[]`, and the `at: {branch_id, depth}` coordinate that says where it was written. It is produced by the validated `set_scene` event, snapshotted per position in `actions.narrative_state_after`, restored by the head move on every Undo, Redo, Retry and Save Point restore, and carried in the v3 bundle. Building a second scene record beside it would have been a duplicate representation of the same fact with its own lineage rules to get wrong — and the lineage rules are the hard part, which is exactly why the answer is to reuse the one that already works. So the **Scene Packet** (`app/media/packet.py`) is *derived on read* and stored nowhere. Its identity is `c:b:-`, computed from the campaign and the position rather than allocated, so the same position yields the same id in any process and after any restart without a row to keep in step. `media_job` and `media_asset` (§27, §28) remain **unbuilt**. M10 defines their contracts as `typing.Protocol` structural types in `app/media/providers.py` — `MediaProvider`, `SpeechProvider`, `TranscriptionProvider`, and the `MediaRequest` / `MediaResult` / `DraftTranscription` shapes — with an empty registry. A queue with no producer and no consumer would be speculative architecture, and this codebase has already declined that once: M6's `derived_status` carries the note "not a job queue". **Why a visual profile has no branch coordinate.** Every other derived record in the schema carries `(branch_id, depth)` because it describes a *moment*. A profile describes none: a character does not change appearance because the story forked. Making it per-position would have hidden a reader's cast from them the moment they diverged, and would have put a descriptor document into every per-position snapshot — measured at 245 copies of 367 bytes in a 120-turn campaign, to say something that never varies (`tools/m10_media_cost.py`). **A profile is not a fact.** Nothing here is state the story established. Writing one cannot change `narrative_state`, and the guarantee is structural rather than remembered: nothing in `app/media/` imports the code that writes it. The reverse direction is the same rule seen from the other side — a depiction never becomes canon (`MEDIA-EXTENSION-CONTRACT.md` §35, §37). ## 29. Export Package A campaign export should be capable of preserving: - campaign configuration, - branches, - turn graph, - checkpoints, - state/events, - entities, - facts, - relationships, - story threads, - summaries, - memories, - scene snapshots, - knowledge-source metadata, - knowledge chunks/source files if selected, - prompt provenance if selected, - media metadata, - media files if selected. The physical container format remains an implementation choice, but the export must preserve the exact active branch **and active head position**, even when the head is behind a retained tip after Undo. A ZIP containing a database plus manifest remains a strong candidate. As implemented in M3, the export carries the active branch, the active head position on it, and each branch's disposition, alongside the whole retained turn graph. **M4 added the checkpoints**, by the same rule: a position someone chose to name cannot be recomputed from the turns, because nothing about a turn records that it was bookmarked. The head and the checkpoints stay independent on import — a campaign opens where its head says, never at a checkpoint merely because one is in the file. The governing rule for this package is that an export carries what was *chosen* and recomputes what is *derived* — and the active head moved from the second category to the first, because once Undo stops deleting, two campaigns with identical turns can be being read at different positions and no import can tell which. An export written before the field existed is opened at its retained tip, which is the position such a file recorded. **M7 added the imported knowledge library**, by the same rule and no other. The bundle carries each source's content, classification, enabled state, visibility, always-include flag, title, filename, notes, import timestamp and content hash — everything the reader chose, and one derived value whose only purpose is to be checked against what arrived. It carries no passages, no FTS rows and no vectors: those are a deterministic function of the content, and the import rebuilds the passages and the lexical index before it returns, so an imported campaign is searchable immediately with no reindex step. Vectors rebuild separately against whatever embedding model *this* machine has, which is the right answer and the reason exporting them would have been the wrong one. A source that fails to rebuild is recorded as failed rather than refusing the import: by that point the story, its tree, its head and its Save Points are already written, and the index is the cheap half. A malformed *knowledge section* — an unknown classification, missing content — does refuse the import, because a campaign whose imported Canon quietly did not arrive is a campaign whose narrator has stopped being told the rules, with nothing to notice. A bundle written before M7 has no knowledge section and imports with an empty library, which is what such a campaign had. ### M9: the format became a version, and the evidence started travelling **M9 bumped the format to `ai-dnd-adventure-v3`**, and the reason is a rule rather than a preference. Everything M9 added *could* have been an optional key read with `.get`, the way `persona`, `checkpoints`, `narrativeState` and `knowledge` each were. That mechanism stops working at exactly this addition: a v2 file carrying no prompt provenance is **ambiguous** — written before M9, when no file could carry one, or by M9 from a campaign whose turns predate the column? Those are different facts about the campaign and a reader has to be able to tell them apart. It is the same distinction the head rule above draws when it says a pre-M3 file opens at its tip *because that is the position such a file recorded*. A version number is how a recovery file states what it was capable of recording. The reader keeps every older version; only the writer moved. What each version can be trusted to say: ```text v1 a linear story, its turns, and its retries as a repeating group v2 + the tree, the live flags, the after-snapshots, the chosen head, Save Points, the narrative state document, imported knowledge v3 + state events and proposals, historical prompt/context provenance, lineage-anchored summaries, take parentage, memory authority ``` **Three categories, not two.** §31 below distinguishes authoritative from derived, which was sufficient until M9 had to decide about stored prompts. They are derived — a machine assembled them — and they must travel anyway, so the rule the bundle applies has a middle category: ```text chosen what a person decided: the story, the head, the takes, the Save Points, the classifications, the canon. travels evidence what happened, and what the application was told at the time: the state events and proposals, the per-turn prompt and the passages it was shown, the model and generation settings that turn ran under. travels rebuildable a deterministic function of what travels: knowledge passages, the FTS index, embeddings, the branch lineage cache. rebuilt on import ``` The test that separates evidence from rebuildable is **not** "could this be recomputed" but "would a recomputation answer the same question". Rebuilding the FTS index answers the same question it answered before. Rebuilding an old turn's prompt does not — it would say what that turn *would be told now*, from today's canon, today's sources and today's state, which is the opposite of what the context inspector is for. Historical evidence is not a cache. So v3 additionally carries, all restored verbatim: - **`stateEvents` and `stateProposals`.** §17's hybrid keeps the events for audit and the snapshots for restore; v2 carried only the snapshots, so a moved campaign could be read at any position and could no longer say what changed there, who asserted it, or what the value was before. A **manual correction** was the worst case: the one state change no narration explains, and with the events gone nothing distinguished it from something the story established. Both tables travel, because the inspector reads both — the event says what was accepted and the proposal says what the model asked for and what was refused. - **A per-node context snapshot**, which is `SPECIFICATION.md` §6.1's exact prompt/context record. It carries the assembled prompt section by section, the passages retrieved with the text each supplied, which summary was eligible, and the model and generation settings the call ran under — so an old turn can still say what it was told after the source was deleted, the canon edited, the chunker changed and the state moved on. Stored once per turn on the live attempt, so a retried turn is not a multiplier. - **`summaries`**, with the coordinate that decides eligibility. v2 carried only the `storySummary` mirror, which has no lineage of its own, so a restored campaign resumed with no usable long-story continuity — and a summary belonging to an abandoned line stays ineligible after the move for the same reason it was before it: eligibility is the coordinate lying on the active capped lineage, not a stored flag. - **Take parentage**, so attempts under two different takes of one turn stay two pagers rather than merging into one. - **Memory `authority`**, so a heuristic memory is not promoted to accepted story by being moved (F07). - **Per-source `parserVersion`/`chunkingVersion`**, recording what produced the passages a historical retrieval record describes. **Encoding.** A per-turn prompt contains the story so far, so one per turn is O(turns²) in campaign length — measured at 20,797 bytes per turn at turn 20 and 55,291 at turn 120, 68% of a 9.7 MB file. The snapshot therefore travels as `contextSnapshotZ`: the same JSON, zlib-compressed and base64-encoded, using the same pack/unpack the database column already uses. Nothing is dropped or summarised; the file is still JSON, and every other section of it is still plain text. The plain `contextSnapshot` key is still read and takes precedence, so a hand-edited file keeps importing. **Two pointers are translated on import, and nothing else is.** Branch numbers already were. M9 adds the `source_id` inside a restored retrieval record: it names a row on the machine that wrote the file, so left alone it would point the inspector's "open this source" at whatever holds that id here. Where the file's own knowledge section contains the source it is repointed; where it does not — a source deleted before the export — it becomes `null`, and the record keeps its text and filename. The evidence is never rewritten; only the pointer is. ## 30. Deletion vs Archival The system must distinguish: - archive, - detach/disable, - permanent delete. Retry, Undo, Restore, and branch switching must not silently perform permanent deletion. ## 31. Authoritative vs Derived Data ### Authoritative - accepted turns, - branch lineage, - campaign configuration, - accepted facts, - accepted relationships, - accepted state events, - checkpoints. ### Derived - summaries, - embeddings, - semantic indexes, - lexical indexes, - some automatically produced scene descriptions. Derived data should be rebuildable where practical. **"Where practical" does real work in that sentence, and M9 had to split this list to act on it** (§29). Embeddings and the lexical and semantic indexes are deterministic functions of content that travels, so a rebuild answers the same question and they are not exported. A **summary** is not: it took a model call, it describes a stretch of story that may since have been abandoned, and regenerating one on another machine produces different prose about a different reading — so it is derived, not practically rebuildable, and it travels with the coordinate that decides whether it still applies. The same reasoning puts **stored prompt/context snapshots** on the travelling side, and they are not in either list above because they are neither: they are not authoritative — nothing decides anything from them — and calling them derived would invite a rebuild. They are *evidence*: a record of what the application was told at the time, which a regeneration would not reproduce because it would use today's canon, today's sources and today's state. §29 states the three-way rule the export applies. ## 32. Provenance Important information should answer: > Where did this come from? Potential provenance: - campaign setup, - manual user edit, - accepted narrator turn, - state extraction, - imported canon, - imported reference, - imported inspiration, - derived inference. ## 33. Phase 0B Decisions Applied Phase 0B resolved the foundational data-model questions: - AI-DnD story lineage/alternate-take machinery is the production starting point. - The active head is distinct from retained tip history. - Undo/Redo use head movement; destructive deletion is not part of ordinary history operations. - Named checkpoints are durable pointers to recoverable head positions. - State uses a hybrid event + snapshot/cache model. - State proposals use explicit typed operations with unambiguous value semantics. - Memories/summaries must be lineage-filtered or lineage-anchored. - Imported knowledge requires separate source/chunk/index tables rather than overloading Story Cards. - Scene/media records remain optional derived extensions and must carry source lineage. - Export/import must preserve active head position as well as the retained history graph. ## 34. Acceptance Criteria The final v1 data model must support all of these without destructive hacks: - close/restart/resume exact story, - Undo/Redo without deleting accepted turns, - active head behind retained tip, - branch from an earlier turn while retaining the original future, - mark abandoned futures/takes retained/disposable, - create and restore named checkpoints, - know current characters/locations/relationships/story threads, - reconstruct earlier authoritative state, - validate typed state proposals before committing events, - retrieve old events outside the active context window, - identify which imported passages informed a turn, - reconstruct what was sent to Ollama, - export/import an undone campaign without silently redoing it, - run fantasy and science-fiction campaigns without schema changes, - attach future image/video/audio/TTS/STT metadata to scenes or turn ranges without making media authoritative. ## 35. Selected Conceptual Model ```text Retained Turn/Take Lineage | +--> Active branch + movable active head | +--> Validated typed state events | | | +--> state snapshot/cache | +--> lineage-safe summaries/memories | +--> prompt/retrieval provenance | +--> scene snapshot | +--> future optional media records Separate campaign knowledge subsystem +--> sources +--> chunks +--> FTS/local embeddings +--> authority/provenance ``` This combines AI-DnD's retained story lineage and snapshot infrastructure with ai-adventure-style explicit event/commit discipline while preserving the project's own specification as the authority.