interactive-story/backend/app/bundle.py

"""Phase 14, SP6: the export bundle, as a tree.

A bundle is the one place a story leaves the database, and the only part of the
tree no migration can reach. A file downloaded today has to still import into a
build shipped next year. The format is therefore versioned, both versions are
defined here, and nothing else in the app knows either of them.

Version 1 is a flat list of turns, each with an optional `variants` array, which
is the repeating group SP4 unpacked into rows. Nothing writes that shape now.
The reader stays, because bundles already on people's disks still use it, and a
backup that stops importing is not a backup.

Version 2 carries the tree. It holds three things version 1 could not, and each
one is required:

* The branches, because a forked adventure is two stories and a flat list holds
  one. A version 1 export interleaved them by turn number, which read as a
  garbled story rather than as lost data.
* `live`, because a coordinate can hold several attempts at one turn and exactly
  one of them is the story.
* Both after-snapshots, because they are what a branch switch and an undo
  restore. A bundle carrying the actions but not the outcomes would import a
  tree nobody could switch inside.

## The rule about what a bundle carries

A bundle carries what was chosen, never what is derived. The head branch, the
fork points, the live flags, and the anchors are decisions somebody made, so
they are in the file. `lineage` and the head depth are computed from those, and
the import recomputes them:

* `lineage` is a cache of `parent` plus `fork_depth`. Shipping it as well would
  put a second source of truth for one fact into a file anyone can hand-edit,
  and the two could then disagree without any read reporting it.
* The head depth used to be derived the same way — the tip of the head branch,
  a fact about the nodes that arrived with it. M3 moved it to the other side of
  the rule. Undo no longer deletes, so a story can be read at a position behind
  its retained tip, and where the reader stopped is a decision nobody can
  recompute from the rows: the same tree exports identically whether the user
  undid three turns or none. `headDepth` is therefore written, and an import
  that ignored it would silently Redo the story to its newest retained turn,
  which is the Phase 0B export finding this milestone exists to close.

A bundle written before M3 has no `headDepth` key, and there is nothing lost in
that: at the time it was written the head could not sit behind the tip, so the
derived answer *was* the recorded one. Such a file imports by deriving, exactly
as it always did.

Every hand-editable coordinate is therefore checked before a row is written, in
`plan`, rather than repaired afterwards. An import that fails partway leaves an
adventure holding half a tree, and a tree missing a branch is a story that stops
without reporting anything.

## Version 3, and why the version was bumped (M9)

Version 3 adds the evidence that explains a campaign, rather than more of the
campaign itself:

* `stateEvents` and `stateProposals` — the audit half of the hybrid
  (`DATA-MODEL.md` §17). Version 2 carried the per-position snapshots, so an
  imported campaign could be *read* at any position but could not say why the
  state there was what it was, and a manual correction was indistinguishable
  from something the story established.
* a per-node `contextSnapshot` — the exact prompt a turn was given, the
  passages it was shown and their text, and the model and generation settings it
  ran under. This is `SPECIFICATION.md` §6.1's "exact prompt/context snapshot or
  reproducible equivalent", and it is the one thing here that genuinely cannot
  be recreated: an imported source can be deleted, canon can be edited, the
  chunker can change, and the evidence of what an old turn was actually told has
  to survive all of it (`IMPORTED-KNOWLEDGE-DESIGN.md` §49-50).
* `summaries` — the generated and hand-written rolling summaries, each with the
  coordinate that decides whether it is eligible. Version 2 carried only the
  `storySummary` mirror, which has no lineage, so a restored campaign resumed
  with no usable long-story continuity at all.
* a per-node `parent` — which attempts belong to the same turn (SP9). Version 2
  left every imported node parentless, and the coordinate fallback merges the
  attempts under two different takes of the turn above into one pager.
* per-memory `authority`, and per-source `parserVersion`/`chunkingVersion`.

### Why this is a version and not five more optional keys

Everything above could have been added to version 2 the way `persona`,
`checkpoints`, `narrativeState` and `knowledge` were: read with `.get`, absent
meaning the campaign had none. That mechanism is real and this module has used
it four times.

It stops working here, for the reason the `headDepth` rule above exists. A
version 2 file with no `contextSnapshot` anywhere is **ambiguous**: it may have
been written before M9, when no bundle could carry one, or by M9 from a campaign
whose turns predate the column. Those are different facts and a reader has to be
able to tell them apart — the same distinction I07 draws when it says a file
written before the head was carried opens at the tip *because tip was the only
position that format could represent*. A version number is how a recovery file
states what it was capable of recording. So the writer bumps, and the reader
keeps every older version.

### What each version can be trusted to say

    v1   a linear story, its turns, and its retries as a repeating group
    v2   + the tree, the live flags, the after-snapshots, the chosen head,
           Save Points, the narrative state document, imported knowledge
    v3   + state events and proposals, historical prompt/context provenance,
           lineage-anchored summaries, take parentage, memory authority

An older file is never reinterpreted under a newer rule. A v2 file has no state
events because v2 could not carry them, not because the campaign had none, and
the import says so rather than inventing an audit trail.

## Three kinds of data, and the rule for each

M9 states the rule the module has been applying, because the additions above are
the first ones where the second and third categories differ:

1. **Chosen.** What a person decided: the story, the head, the takes, the Save
   Points, the classifications, the canon. It travels.
2. **Historical evidence.** What happened, and what the application was told at
   the time. It travels *even though* something resembling it could be
   regenerated, because a regeneration would describe today's campaign rather
   than the turn it claims to explain. Historical evidence is not a cache.
3. **Rebuildable derived data.** A deterministic function of what travels:
   knowledge passages, the FTS index, embeddings, the branch lineage cache. It
   does not travel, and the import rebuilds it.

The test that separates 2 from 3 is not "could this be recomputed" but "would a
recomputation answer the same question". Rebuilding the FTS index answers the
same question it answered before. Rebuilding an old turn's prompt does not: it
would say what that turn *would be told now*, which is the opposite of what the
inspector is for.

## Why the snapshots are compressed inside the file

A per-turn prompt contains the story so far, so storing one per turn costs
O(turns²) in a campaign's length. That is already true of the database —
`compression.py` records the column as 89% of production storage and compresses
it for exactly this reason — and carrying the snapshots makes it true of the
export too. Measured on this build with a 16,384-token budget: 20,797 bytes of
snapshot per turn at turn 20, 55,291 at turn 120, and 68% of a 9.7 MB file.
Against `limits.MAX_IMPORT_BODY_BYTES` a campaign would stop being importable —
stop being *recoverable* — somewhere around its 170th turn.

So the snapshot travels as `contextSnapshotZ`: the same JSON, zlib-compressed by
`compression.pack` and base64-encoded so the file is still JSON. It is the same
bytes on the way out as on the way in; nothing is dropped, summarised or
reconstructed, and `compression.py`'s existing pack/unpack is the only
implementation. The measured cost falls to about a quarter, which moves the
ceiling out by roughly the square root of that.

Everything else in the file stays plain, readable JSON — the story, the tree,
the head, the Save Points, the state events, the imported source text. The one
field that is encoded is the machine-assembled prompt archive, which nobody
reads by eye in a text editor and which the context inspector shows properly.

`contextSnapshot`, holding the plain object, is still **read** on import and
takes precedence when both keys are present. A file somebody hand-edited to make
a prompt legible has to keep importing, and the plain form is what they will
have written.
"""

import base64
import binascii
import zlib
from datetime import datetime

from fastapi import HTTPException
from sqlalchemy import insert, update
from sqlalchemy.orm import Session, undefer

from . import attempts, compression, memorybank, models, schemas, summaries
from .context import cursors, lineage
from .knowledge import chunking as knowledge_chunking
from .knowledge import classes as knowledge_classes
from .knowledge import importer as knowledge_importer
from .media import profiles as visual_profiles
from .narrative import model as narrative_model

#: What `export` writes. The family name is inherited from the production base
#: and is deliberately unchanged: renaming it would break every reader for no
#: gain, and `PROVENANCE.md` is where the fork is recorded.
FORMAT = "ai-dnd-adventure-v3"

#: Every version the importer still accepts, newest first. A backup that stops
#: importing is not a backup, so this list only ever grows.
TREE_FORMAT = "ai-dnd-adventure-v2"
LEGACY_FORMAT = "ai-dnd-adventure-v1"
READABLE = (FORMAT, TREE_FORMAT, LEGACY_FORMAT)

#: The versions that carry the tree — everything except the flat v1 shape.
TREE_FORMATS = (FORMAT, TREE_FORMAT)

# `actions.type` is VARCHAR(20), and a raw-dict import bypasses the schemas.
TYPE_MAX = 20


# ---------------------------------------------------------------- exporting

def export(db: Session, adventure: models.Adventure) -> dict:
    """Returns the whole adventure as a version 3 bundle.

    The export is not scoped to a path. A backup holds the entire tree, not the
    branch its owner is currently reading. Every deferred per-node column is
    undeferred in the one query, because they are columns nothing else reads in
    bulk and requesting them a row at a time would cost one query per turn — and
    since M9 that includes `context_snapshot`, which is the largest of them.

    **The bundle now carries the context snapshots (M9).** Every version before
    this one left them out, on the reasoning that they explain a generation
    rather than form part of the story and that the viewer reading them reads
    the adventure they came from. That reasoning holds right up to the moment
    the campaign moves to another machine, which is what this milestone is
    about: an imported campaign with no snapshots has no historical prompt
    provenance at all, so `SPECIFICATION.md` §6.1's per-turn record is local to
    the machine that played it. See the module docstring on the three kinds of
    data — this is evidence, not a cache.

    They are stored once per turn on the live attempt (`attempts.py`), so a
    campaign with many retries pays for the prompt once and for the few hundred
    bytes of per-attempt slices per take. The measured cost is in the M9 report.
    """
    branches = (
        db.query(models.Branch)
        .filter(models.Branch.adventure_id == adventure.id)
        .order_by(models.Branch.id)
        .all()
    )
    # Branch ids are local to the file and are positions in this list, because
    # the database ids they hold here are already in use on the importing
    # side.
    local = {branch.id: i for i, branch in enumerate(branches)}
    nodes = (
        db.query(models.Action)
        .filter(models.Action.adventure_id == adventure.id)
        .options(
            undefer(models.Action.state_after),
            undefer(models.Action.world_state_after),
            undefer(models.Action.narrative_state_after),
            undefer(models.Action.context_snapshot),
        )
        .order_by(
            models.Action.branch_id, models.Action.depth, models.Action.id,
        )
        .all()
    )
    knowledge_sources = list(adventure.knowledge_sources)
    return {
        "format": FORMAT,
        "title": adventure.title,
        "memory": adventure.memory,
        "authorsNote": adventure.authors_note,
        "aiInstructions": adventure.ai_instructions,
        "storySummary": adventure.story_summary,
        # Phase 18. A bundle written before personas existed has no key here,
        # and the import below reads it with `.get`, so it lands with an empty
        # persona — which is the same as having none. No FORMAT bump needed.
        "persona": {
            "name": adventure.persona_name,
            "pronouns": adventure.persona_pronouns,
            "desc": adventure.persona_desc,
        },
        "worldState": adventure.world_state,
        # M5. The authoritative narrative state, and the campaign's own rules.
        # Both are decisions rather than derivations — the state is what the
        # campaign established, and canon is what its owner wrote — so both go
        # in the file by the rule at the top of this module. A bundle written
        # before M5 has neither key and imports with an empty state, which is
        # what such a campaign had.
        "narrativeState": adventure.narrative_state,
        "campaignCanon": adventure.campaign_canon,
        "autoSummarize": adventure.auto_summarize,
        "memoryBankEnabled": adventure.memory_bank_enabled,
        # Write a root entry even for an adventure whose branch row was never
        # created. A story with no branch is a pre-tree story, and the tree it
        # belongs to is the root. `_local` places its nodes there.
        "branches": [_exported_branch(b, local) for b in branches] or [_ROOT],
        "headBranch": local.get(adventure.head_branch_id, 0),
        # M3. Where the story is being read, which is not always where it
        # ends. See the head-depth rule at the top of this module.
        "headDepth": adventure.head_depth,
        "memoryCursor": _exported_anchor(adventure, cursors.MEMORY, local),
        "summaryCursor": _exported_anchor(adventure, cursors.SUMMARY, local),
        "memories": [_exported_memory(m, local) for m in adventure.memories],
        # M4. A Save Point is a decision — someone chose this position and gave
        # it a name — so it goes in the file by the rule at the top of this
        # module. Nothing here is derived: the coordinate is the one stored, not
        # one recomputed from the rows, because the whole point of the pointer
        # is that no amount of reading the turns can tell you which one somebody
        # named. A bundle written before M4 has no key here and imports with no
        # Save Points, which is what such a campaign had.
        "checkpoints": [
            _exported_checkpoint(c, local) for c in _checkpoints_of(db, adventure)
        ],
        "storyCards": [
            {"type": c.type, "name": c.name, "keys": c.keys,
             "entry": c.entry, "notes": c.notes}
            for c in adventure.story_cards
        ],
        # M7. The imported knowledge library, carried by the same rule as
        # everything else here: what somebody chose goes in the file, what a
        # machine derives does not.
        #
        # So the source text and the reader's judgements about it travel —
        # content, classification, enabled, visibility, always-include, the
        # title and filename, the hash. Passages, FTS rows and vectors do not:
        # they are a deterministic function of the content, and the import
        # rebuilds them. That keeps a bundle a readable record of a campaign
        # rather than a database dump, and it keeps a campaign exported on one
        # machine importable on another whose embedding model is different.
        #
        # `contentHash` is exported although it is derivable, because it is the
        # identity the reader can check a restored file against — the one place
        # a derived value earns a place in the file is when its purpose is to
        # detect that the thing it describes has changed underneath it. The
        # import verifies it rather than trusting it.
        #
        # A bundle written before M7 has no key here and imports with an empty
        # library, which is what such a campaign had.
        "knowledge": [_exported_source(k) for k in knowledge_sources],
        # M6/M9. The lineage-anchored summaries, which version 2 did not carry.
        # A summary is derived from the story, so the earlier rule left it out —
        # but it is derived by a *model*, at a cost, from a stretch of story that
        # may since have been abandoned, and regenerating one on the importing
        # machine would produce different prose describing a different reading.
        # It is provenance-bearing derived data, and the coordinate on it is
        # what decides eligibility (`CONTEXT-AND-MEMORY.md`; E03), so the row
        # travels and its vectors do not.
        "summaries": [_exported_summary(s, local) for s in adventure.summaries],
        # M10. How the campaign's entities look.
        #
        # **Chosen**, by the rule at the top of this module: a reader wrote
        # these, and nothing in the campaign can recompute them — the state
        # document records what an entity *is*, never what it looks like. A
        # campaign that arrived without them would have lost the descriptions
        # its owner wrote and there would be no way to tell.
        #
        # Campaign-scoped and therefore carrying no coordinate, which is the one
        # thing that makes this section shaped differently from every other list
        # here. A profile does not belong to a position (`models.VisualProfile`),
        # so there is no branch to remap and nothing to check against the tree.
        #
        # **No format bump.** Applying M9's own test — does an absent key create
        # an ambiguity about what an older file could record? — the answer is
        # no. A v3 file with no `visualProfiles` is unambiguous in the way a v2
        # file with no `contextSnapshot` was not: appearance is not something a
        # campaign has by default and then loses in the writing, it is something
        # a reader adds. Absent means the campaign had none, which is exactly
        # what it means for `checkpoints` before M4 and `knowledge` before M7,
        # and both of those were added without a bump for the same reason.
        "visualProfiles": [
            _exported_visual_profile(v) for v in adventure.visual_profiles
        ],
        # M5/M9. The audit half of the hybrid. `DATA-MODEL.md` §17 keeps the
        # events for audit and the snapshots for restore, and version 2 carried
        # only the snapshots — so a moved campaign could be read at any position
        # and could no longer say what changed there, who asserted it, or what
        # the value was before. A manual correction was the worst case: it is
        # the one state change no narration explains, so with the events gone
        # nothing distinguished it from something the story established.
        #
        # Both tables travel, because the inspector reads both: the event says
        # what was accepted, and the proposal beside it says what the model
        # actually asked for and what was refused. Exporting only the accepted
        # half would keep the answers and lose every question.
        "stateProposals": _exported_proposals(db, adventure, local),
        "stateEvents": _exported_events(db, adventure, local),
        "actions": [_exported_node(a, local) for a in nodes],
    }


def _exported_source(source: models.KnowledgeSource) -> dict:
    """One knowledge source, as it goes into the file."""
    return {
        # M9. The id this source had in the campaign that wrote the file. It is
        # not an identity the import reuses — the row gets a new id like
        # everything else — but the historical retrieval records inside the
        # context snapshots name it, and without it a restored inspector could
        # not tell which of five imported sources an old turn was shown. The
        # import maps it forward; see `_relink_snapshot`.
        "sourceId": source.id,
        "title": source.title,
        "originalFilename": source.original_filename,
        "classification": source.classification,
        "enabled": source.enabled,
        "visibility": source.visibility,
        "alwaysInclude": source.always_include,
        "contentHash": source.content_hash,
        "mediaType": source.media_type,
        "notes": source.notes,
        # M9. What produced the passages this source last had. The import
        # rebuilds the passages with *this* build's chunker and records its own
        # versions, so these two say what the source was chunked by when it was
        # exported — which is how a later reader can tell that a historical
        # retrieval record describes passages a newer parser would not produce.
        "parserVersion": source.parser_version,
        "chunkingVersion": source.chunking_version,
        "importedAt": source.imported_at.isoformat() if source.imported_at else None,
        "content": source.content,
    }


def _exported_visual_profile(profile: models.VisualProfile) -> dict:
    """One visual profile, as it goes into the file.

    The entity key travels as itself. It is a key inside the campaign's own
    state document, which travels in the same file, so it needs no translation —
    unlike a branch number or a knowledge source id, both of which name rows
    whose identity is local to a database.
    """
    return {
        "entityKey": profile.entity_key,
        "descriptors": profile.descriptors or {},
        "features": profile.features or [],
        "styleNotes": profile.style_notes or "",
        "createdAt": profile.created_at.isoformat() if profile.created_at else None,
    }


def _exported_summary(summary: models.Summary, local: dict[int, int]) -> dict:
    """One summary, with the coordinate that decides whether it is eligible."""
    return {
        "text": summary.text,
        "branch": _local(summary.branch_id, local),
        "depth": summary.depth,
        "sourceStart": summary.source_start,
        "sourceEnd": summary.source_end,
        "trigger": summary.trigger,
        "model": summary.model_name,
        "createdAt": summary.created_at.isoformat() if summary.created_at else None,
    }


def _exported_proposals(
    db: Session, adventure: models.Adventure, local: dict[int, int],
) -> list[dict]:
    """Every state proposal, in the order they were made.

    `id` is the row's identity in the source campaign, exported for the reason
    `_exported_node` exports one: the events below name it, and the import needs
    something to translate. It is a *file-local* identity — the imported row
    gets a fresh id — and the only guarantee it carries is that references
    inside one file agree with each other.

    `detail` is deferred and compressed, and it is the largest thing here — a
    busy turn's rejections — so it is undeferred once for the whole list rather
    than touched per row.
    """
    rows = (
        db.query(models.StateProposal)
        .filter(models.StateProposal.adventure_id == adventure.id)
        .options(undefer(models.StateProposal.detail))
        .order_by(models.StateProposal.id)
        .all()
    )
    return [
        {
            "id": row.id,
            "action": row.action_id,
            "branch": local.get(row.branch_id) if row.branch_id is not None else None,
            "depth": row.depth,
            "model": row.model_name,
            "source": row.source,
            "status": row.status,
            "rawOutput": row.raw_output,
            "detail": row.detail,
            "createdAt": row.created_at.isoformat() if row.created_at else None,
        }
        for row in rows
    ]


def _exported_events(
    db: Session, adventure: models.Adventure, local: dict[int, int],
) -> list[dict]:
    """Every accepted state event, in the order they were applied.

    `proposal` and `action` name rows by the identity those rows export, and the
    import translates both. An event whose proposal or action is missing from
    the file keeps its coordinate and loses the pointer, which is what a
    `SET NULL` and a `CASCADE` respectively already mean in the schema.
    """
    rows = (
        db.query(models.StateEvent)
        .filter(models.StateEvent.adventure_id == adventure.id)
        .order_by(models.StateEvent.id)
        .all()
    )
    return [
        {
            "proposal": row.proposal_id,
            "action": row.action_id,
            "branch": local.get(row.branch_id) if row.branch_id is not None else None,
            "depth": row.depth,
            "sequence": row.sequence,
            "eventType": row.event_type,
            "payload": row.payload,
            "before": row.before,
            # Who asserted this. `manual_correction` is the value that has to
            # survive: it is what C04 and the audit view use to say the user
            # overruled the story rather than the story establishing it.
            "source": row.source,
            "createdAt": row.created_at.isoformat() if row.created_at else None,
        }
        for row in rows
    ]


_ROOT = {"parent": None, "forkDepth": None}


def _local(branch_id: int | None, local: dict[int, int]) -> int:
    return local.get(branch_id, 0) if branch_id is not None else 0


def _exported_branch(branch: models.Branch, local: dict[int, int]) -> dict:
    parent = (
        local.get(branch.parent_branch_id)
        if branch.parent_branch_id is not None else None
    )
    out = (
        dict(_ROOT) if parent is None
        else {"parent": parent, "forkDepth": branch.fork_depth}
    )
    # A player chose the name, so it goes into the file. That is the same rule
    # that puts the fork points in the file and leaves `lineage` out. An unnamed
    # branch omits the key rather than carry a null, which keeps the file for an
    # unnamed tree byte-identical to the one SP6 wrote.
    if branch.name:
        out["name"] = branch.name
    # M3. A branch the story left behind is a decision too — the depth a
    # divergent write departed at — so it goes in the file by the same rule that
    # puts the fork points in it. A restored backup that could not tell abandoned
    # history from active history would have lost the only thing distinguishing
    # them, since every row of both arrives either way. Omitted when the branch
    # is active, which keeps a file for a never-undone tree as it was.
    if branch.superseded_at is not None:
        out["supersededAt"] = branch.superseded_at.isoformat()
        out["supersededDepth"] = branch.superseded_depth
    return out


def _exported_node(action: models.Action, local: dict[int, int]) -> dict:
    node = {
        "branch": _local(action.branch_id, local),
        "depth": action.depth,
        "live": bool(action.live),
        "type": action.type,
        "text": action.text,
        "createdAt": action.created_at.isoformat() if action.created_at else None,
    }
    # M9. Which turn this node is an attempt at (SP9). A coordinate cannot
    # answer it: attempts under two different takes of the turn above share a
    # branch and a depth, and an attempt forked onto its own line leaves the
    # coordinate its siblings still hold. Version 2 exported neither, so every
    # imported node landed parentless and `attempts.group` fell back to the
    # coordinate — which merges those two groups into one pager reading 5/5
    # where the reader should see 2/2 and 3/3.
    #
    # The value is the parent's own database id, translated on the way in. A
    # root node has none, and so does a row written before SP9.
    if action.parent_id is not None:
        node["parentId"] = action.parent_id
    # M9. The node's own id, which is what `parentId` above and the audit
    # records refer to. Exported rather than derived from list position because
    # a hand-edited file that reorders `actions` would otherwise silently
    # re-parent every take.
    node["id"] = action.id
    if action.reasoning:
        node["reasoning"] = action.reasoning
    # `{}` and an absent key mean different things. `{}` means the node left an
    # empty state behind, and an absent key means the state is unknown and the
    # live state stays as it is. An empty snapshot is therefore written rather
    # than omitted. It costs about eighteen bytes per row, and it decides
    # whether an undo clears a score or leaves it in place.
    if action.state_after is not None:
        node["stateAfter"] = action.state_after
    if action.world_state_after is not None:
        node["worldStateAfter"] = action.world_state_after
    # M5. Without this a restored campaign could be read but not moved around
    # inside: every Undo, Redo and Save Point restore reads the destination
    # node's snapshot, so a bundle carrying the turns and not the snapshots
    # imports a story whose history cannot be walked.
    if action.narrative_state_after is not None:
        node["narrativeStateAfter"] = action.narrative_state_after
    if action.state_changes:
        node["stateChanges"] = action.state_changes
    if action.world_delta:
        node["worldDelta"] = action.world_delta
    # M9. What this turn was actually given: the assembled prompt section by
    # section, the passages retrieved and their rendered text, which summary was
    # eligible, and the model and generation settings the call ran under.
    #
    # This is the one field here that is neither a decision somebody made nor a
    # deterministic function of the rows beside it. It is evidence, and the
    # module docstring says why that is a third category: regenerating a
    # historical prompt would produce what the turn *would* be told now, from
    # today's canon, today's sources and today's state, which is precisely the
    # question the inspector does not ask.
    #
    # Stored once per turn on the live attempt, so a superseded take carries
    # only `attempts.ATTEMPT_KEYS` — its own reply, its own state proposal and
    # its own token accounting — and a retried turn is not a multiplier on the
    # largest thing in the file.
    #
    # Compressed, for the size reason in the module docstring. The plain form is
    # still accepted on the way in.
    if action.context_snapshot is not None:
        node["contextSnapshotZ"] = _packed(action.context_snapshot)
    return node


def _packed(snapshot: dict) -> str:
    """One context snapshot as base64-encoded zlib, for the file."""
    return base64.b64encode(compression.pack(snapshot)).decode("ascii")


def _unpacked(value) -> dict | None:
    """Reads a `contextSnapshotZ` field back, or `None` if it cannot be read.

    A snapshot that will not decode is dropped rather than raising, which is the
    rule `compression.CompressedJSON.process_result_value` already applies to
    the same data in the database: the evidence for one turn is worth less than
    the campaign, and a turn with no snapshot is a state the inspector has
    always been able to render. It is never *partially* decoded — the whole
    value is one compressed document, so it arrives intact or not at all.
    """
    if not isinstance(value, str):
        return None
    try:
        unpacked = compression.unpack(base64.b64decode(value, validate=True))
    except (binascii.Error, zlib.error, UnicodeDecodeError, ValueError):
        return None
    return unpacked if isinstance(unpacked, dict) else None


def _exported_memory(memory: models.Memory, local: dict[int, int]) -> dict:
    return {
        "text": memory.text, "pinned": memory.pinned, "forgotten": memory.forgotten,
        "sourceStart": memory.source_start, "sourceEnd": memory.source_end,
        "useCount": memory.use_count,
        # M9. How much weight the narrator may give this (`CONTEXT-AND-MEMORY.md`
        # §14, F07). `accepted_story` is something the story established;
        # `heuristic` is a reading of it. Version 2 carried neither, so every
        # imported memory landed on the column default — which silently promoted
        # a heuristic memory to accepted story, the one direction F07 forbids.
        "authority": memory.authority,
        # The node this memory is attached to. A hand-written memory summarizes
        # no node, so it has a branch and no depth, and it keeps that shape
        # here.
        "branch": _local(memory.branch_id, local),
        "depth": memory.depth,
    }


def _checkpoints_of(db: Session, adventure: models.Adventure) -> list[models.Checkpoint]:
    """Returns the campaign's Save Points in creation order.

    Read with a query rather than through a relationship, for the reason
    `models.Branch` declares none: a relationship on `Adventure` would be loaded
    by anything that touches an adventure, and the export is the only thing in
    the application that wants every Save Point at once.
    """
    return (
        db.query(models.Checkpoint)
        .filter(models.Checkpoint.adventure_id == adventure.id)
        .order_by(models.Checkpoint.id)
        .all()
    )


def _exported_checkpoint(checkpoint: models.Checkpoint, local: dict[int, int]) -> dict:
    return {
        "name": checkpoint.name,
        "note": checkpoint.note,
        # The branch as a position in this file's list, like every other branch
        # reference in the bundle. The depth is a coordinate along it and needs
        # no translation.
        "branch": _local(checkpoint.branch_id, local),
        "depth": checkpoint.depth,
        "createdAt": checkpoint.created_at.isoformat() if checkpoint.created_at else None,
    }


def _imported_persona(persona) -> dict:
    """Reads a bundle's `persona` block into `Adventure` keyword arguments.

    A raw-dict import bypasses the schemas, so the strings are truncated to the
    widths the columns declare, in the same way the rest of `_import` does. A
    bundle written before Phase 18 has no block at all, and an empty persona is
    the same as having none.
    """
    if not isinstance(persona, dict):
        return {}
    return {
        "persona_name": str(persona.get("name") or "")[:schemas.PERSONA_NAME_MAX],
        "persona_pronouns":
            str(persona.get("pronouns") or "")[:schemas.PERSONA_PRONOUNS_MAX],
        "persona_desc": str(persona.get("desc") or ""),
    }


def _exported_anchor(
    adventure: models.Adventure, cursor: cursors.Cursor, local: dict[int, int]
) -> dict:
    branch_id, depth = cursor.stored(adventure)
    return {
        "branch": local.get(branch_id) if branch_id is not None else None,
        "depth": depth,
    }


# ---------------------------------------------------------------- importing

def check_format(bundle: dict) -> str:
    """Returns the bundle's version, or raises a 400.

    Every version this build has ever written is accepted, newest first, because
    a backup that stops importing is not a backup. A version this build has never
    heard of is refused rather than read as the newest one it knows: a file from
    a later build may use a key this one would misread, and guessing is how an
    import silently loses half a campaign.
    """
    if not isinstance(bundle, dict):
        raise HTTPException(400, "This file does not contain an adventure export.")
    fmt = bundle.get("format")
    if fmt in READABLE:
        return fmt
    if isinstance(fmt, str) and fmt.startswith("ai-dnd-adventure-v"):
        raise HTTPException(
            400,
            f"This file was written in format {fmt}, which this version of the "
            f"application does not understand. The newest format it can read is "
            f"{FORMAT}.",
        )
    raise HTTPException(
        400,
        "Not an adventure export file (expected format "
        + ", ".join(READABLE) + ").",
    )


def plan(bundle: dict, version: str) -> dict:
    """Returns the bundle's tree, checked and normalized, before a row is written.

    The function has no side effects. It opens no session, needs no adventure,
    and creates nothing. It catches everything a hand-edited file can get wrong
    about the shape of a tree, because the alternative is an import that fails
    partway and leaves an adventure holding a story with a gap in it.

    Every version produces the same shape, so `write` never learns that there
    are three formats. A version 1 bundle is a linear story, which is a tree
    with one branch, and its `variants` array is a sibling group written the old
    way; a version 2 bundle is a version 3 bundle with the evidence sections
    absent, and absent means the format could not carry them.
    """
    tree = version in TREE_FORMATS
    branches = _planned_branches(bundle) if tree else [dict(_ROOT)]
    nodes = (
        _planned_nodes(bundle, len(branches)) if tree
        else _planned_v1_nodes(bundle)
    )
    head = _as_index(bundle.get("headBranch"), len(branches), default=0)
    # M9. Which node in the file each exported node id refers to. Built here so
    # that `parentId` and every audit reference can be checked against it before
    # a row is written, and so a reference the file cannot satisfy is a refusal
    # rather than a dangling pointer discovered later.
    by_id = _planned_node_ids(nodes)
    return {
        "branches": branches,
        "nodes": nodes,
        "memories": _planned_memories(bundle, len(branches)),
        # M4. Empty for a version 1 bundle and for any tree bundle written
        # before Save Points existed, which is the same answer: no one had named
        # a position in those campaigns.
        "checkpoints": (
            _planned_checkpoints(bundle, len(branches), nodes) if tree else []
        ),
        "head": head,
        # None means the file does not say, which is every version 1 bundle and
        # every version 2 bundle written before M3. `_point_the_head` derives it
        # then, which is what those files were written expecting.
        "headDepth": _planned_head_depth(bundle, branches, nodes, head),
        # Versions 2 and 3 record where the derived work reached. Version 1
        # counted it, and a count cannot become a node until the nodes exist.
        # See `settle`.
        "anchors": _planned_anchors(bundle, len(branches)) if tree else None,
        "positions": None if tree else {
            "memory": _as_int(bundle.get("memoryCursor"), 0),
            "summary": _as_int(bundle.get("summaryCursor"), 0),
        },
        # M7. Checked here with everything else, before a row is written, so a
        # hand-edited library fails the import rather than half-landing in it.
        "knowledge": _planned_knowledge(bundle),
        # M9. Three sections version 3 introduced. Each is empty for an older
        # file, and that is a statement about the format rather than about the
        # campaign: a version 2 bundle has no state events because version 2
        # could not carry any, and the import does not invent an audit trail to
        # fill the gap.
        "summaries": _planned_summaries(bundle, len(branches)) if tree else [],
        # M10. Empty for every file written before it, which for a
        # campaign-scoped description means "nobody wrote one" rather than
        # "the format could not say".
        "visualProfiles": _planned_visual_profiles(bundle) if tree else [],
        "proposals": _planned_proposals(bundle, len(branches), by_id),
        "events": _planned_events(bundle, len(branches), by_id),
    }


def _planned_node_ids(nodes: list[dict]) -> dict[int, int]:
    """Maps each exported node id to its position in the planned node list.

    A file that names the same node id twice is refused. The ids are what the
    take parentage and the whole audit trail hang off, so two rows claiming one
    identity would silently attach half the evidence to the wrong turn.
    """
    by_id: dict[int, int] = {}
    for position, node in enumerate(nodes):
        node_id = node.get("id")
        if node_id is None:
            continue
        if node_id in by_id:
            raise HTTPException(
                400, f"Two actions in this file both call themselves {node_id}."
            )
        by_id[node_id] = position
    return by_id


def _planned_knowledge(bundle: dict) -> list[dict]:
    """The knowledge sources in a bundle, checked and normalized.

    Every field is validated here rather than at write time, for the same reason
    the tree is: a file anyone can edit must be found wrong before it has
    written anything. A source that fails validation refuses the import — it is
    not silently dropped. A campaign whose imported Canon quietly did not arrive
    is a campaign whose narrator has stopped being told the rules, and the
    reader would have no way to notice.

    The one thing not trusted from the file is the hash. It is recomputed from
    the content that actually arrived, and a mismatch is reported: that is the
    whole reason a derived value is in the file at all.
    """
    entries = bundle.get("knowledge")
    if entries is None:
        return []
    if not isinstance(entries, list):
        raise HTTPException(400, "The knowledge section of this file is not a list.")
    if len(entries) > knowledge_importer.MAX_SOURCES_PER_ADVENTURE:
        raise HTTPException(
            400,
            f"This file contains {len(entries)} knowledge sources — the limit "
            f"is {knowledge_importer.MAX_SOURCES_PER_ADVENTURE}.",
        )
    planned: list[dict] = []
    for i, entry in enumerate(entries):
        if not isinstance(entry, dict):
            raise HTTPException(400, f"Knowledge source {i + 1} is not an object.")
        content = entry.get("content")
        if not isinstance(content, str) or not content.strip():
            raise HTTPException(400, f"Knowledge source {i + 1} carries no content.")
        if len(content.encode("utf-8")) > knowledge_importer.MAX_SOURCE_BYTES:
            raise HTTPException(
                400, f"Knowledge source {i + 1} is larger than the import limit."
            )
        classification = entry.get("classification")
        if not knowledge_classes.is_class(classification):
            raise HTTPException(
                400,
                f"Knowledge source {i + 1} has no valid classification "
                "(expected canon, reference or inspiration).",
            )
        visibility = entry.get("visibility")
        if not knowledge_classes.is_visibility(visibility):
            visibility = knowledge_classes.NORMAL
        filename = knowledge_importer.safe_filename(
            str(entry.get("originalFilename") or "")
        )
        stated = entry.get("contentHash")
        actual = knowledge_chunking.digest(content)
        planned.append({
            "title": str(entry.get("title") or filename or "Imported source")[:200],
            "original_filename": filename,
            "classification": classification,
            "enabled": bool(entry.get("enabled", True)),
            "visibility": visibility,
            "always_include": bool(entry.get("alwaysInclude", False))
            and classification == knowledge_classes.CANON,
            "media_type": (
                str(entry.get("mediaType"))
                if entry.get("mediaType") in ("text/plain", "text/markdown")
                else "text/markdown"
            ),
            "notes": str(entry.get("notes") or ""),
            "imported_at": _as_time(entry.get("importedAt")),
            "content": content,
            "content_hash": actual,
            "hash_mismatch": isinstance(stated, str) and bool(stated) and stated != actual,
            # M9. The id this source had where the file was written, so the
            # retrieval records inside the historical context snapshots can be
            # pointed at the row that arrives here. Not an identity: two files
            # imported side by side will both have named their sources 1, 2, 3.
            "source_id": _int_or_none(entry, "sourceId"),
        })
    return planned


def _planned_summaries(bundle: dict, branches: int) -> list[dict]:
    """The summaries in a bundle, with the coordinates that make them eligible.

    A summary with no usable coordinate is dropped rather than refused, and
    dropped rather than repaired. The reasoning is `_planned_checkpoints`':
    losing one summary costs a paragraph of derived prose, and refusing the
    whole file over it would lose the campaign to save the paragraph. Repairing
    it would be worse than either — a summary placed at a guessed coordinate is
    a summary that may become eligible on a line it does not describe, which is
    the E03 leak arriving by a new route.
    """
    raw = bundle.get("summaries")
    out: list[dict] = []
    for entry in raw if isinstance(raw, list) else []:
        if not isinstance(entry, dict) or not str(entry.get("text") or "").strip():
            continue
        branch = _as_index(entry.get("branch"), branches, default=None)
        depth = entry.get("depth")
        if branch is None or not _is_int(depth):
            continue
        out.append({
            "text": str(entry["text"]),
            "branch": branch,
            "depth": depth,
            "sourceStart": _int_or_none(entry, "sourceStart"),
            "sourceEnd": _int_or_none(entry, "sourceEnd"),
            "trigger": str(entry.get("trigger") or "interval")[:20],
            "model": str(entry.get("model") or "")[:200],
            "createdAt": _as_time(entry.get("createdAt")),
        })
    return out


def _planned_visual_profiles(bundle: dict) -> list[dict]:
    """The visual profiles in a bundle, checked and normalised.

    A malformed profile is **dropped rather than refused**, and it is worth
    saying why this lands on the opposite side of the line from a knowledge
    source, which refuses.

    An imported Canon file that quietly did not arrive is a campaign whose
    narrator has silently stopped being told the rules, with nothing on screen
    to notice. A visual profile that did not arrive costs a description of how
    somebody looks: nothing reads it during play, no prompt changes, no state
    moves, and the reader can see at a glance that it is missing because the
    profile list is the surface it appears on. Refusing a whole campaign to
    protect a description would trade the story for the caption.

    Bounds are reused from `media.profiles` rather than restated, so a file
    cannot carry a profile the API would have refused to create.
    """
    raw = bundle.get("visualProfiles")
    if not isinstance(raw, list):
        return []
    out: list[dict] = []
    seen: set[str] = set()
    for entry in raw:
        if not isinstance(entry, dict):
            continue
        key = entry.get("entityKey")
        if not isinstance(key, str) or not key.strip():
            continue
        key = key.strip()[:visual_profiles.MAX_KEY]
        # One profile per entity is the model's own uniqueness rule; a file
        # naming the same entity twice would violate it on write, so the first
        # is kept and the rest dropped rather than raising a constraint error
        # halfway through the import.
        if key in seen:
            continue
        seen.add(key)
        try:
            out.append({
                "entity_key": key,
                "descriptors": visual_profiles._checked_descriptors(
                    entry.get("descriptors")),
                "features": visual_profiles._checked_features(
                    entry.get("features")),
                "style_notes": visual_profiles._checked_notes(
                    entry.get("styleNotes")),
                "created_at": _as_time(entry.get("createdAt")),
            })
        except visual_profiles.ProfileError:
            continue
    return out


def _planned_proposals(
    bundle: dict, branches: int, by_id: dict[int, int]
) -> list[dict]:
    """The state proposals in a bundle, checked against the tree beside them.

    A proposal names the node whose narration produced it. A proposal naming a
    node this file does not contain is **refused**, not dropped: unlike a Save
    Point, an audit record that quietly did not arrive leaves a campaign whose
    state cannot be explained, and it is the explanation the reader would go
    looking for precisely when something looks wrong. A file disagreeing with
    itself about which turn asserted something is a file whose audit trail is
    not worth restoring at all.

    A proposal with **no** action is a different thing and is accepted: that is
    a manual correction, which has a coordinate and no narration behind it.
    """
    raw = bundle.get("stateProposals")
    if raw is None:
        return []
    if not isinstance(raw, list):
        raise HTTPException(400, "The state history in this file is not a list.")
    out: list[dict] = []
    for i, entry in enumerate(raw):
        if not isinstance(entry, dict):
            raise HTTPException(400, f"State proposal {i + 1} is not an object.")
        out.append({
            "id": _int_or_none(entry, "id"),
            "action": _planned_action_ref(entry.get("action"), by_id,
                                          f"State proposal {i + 1}"),
            "branch": _as_index(entry.get("branch"), branches, default=None),
            "depth": _int_or_none(entry, "depth"),
            "model": str(entry.get("model") or "")[:200],
            "source": str(entry.get("source") or "accepted_story")[:40],
            "status": str(entry.get("status") or "accepted")[:30],
            "rawOutput": str(entry.get("rawOutput") or ""),
            "detail": _dict_or_none(entry, "detail"),
            "createdAt": _as_time(entry.get("createdAt")),
        })
    return out


def _planned_events(
    bundle: dict, branches: int, by_id: dict[int, int]
) -> list[dict]:
    """The accepted state events in a bundle, checked the same way.

    `proposal` is resolved against the proposals in the same file at write time
    rather than here, because a proposal's identity is its own exported id and
    the planner has both lists. An event naming a proposal the file does not
    contain keeps its coordinate and loses the pointer, which is what the
    schema's `ON DELETE SET NULL` already says happens when a proposal goes
    away. An event naming a *node* the file does not contain is refused, for
    the reason above.
    """
    raw = bundle.get("stateEvents")
    if raw is None:
        return []
    if not isinstance(raw, list):
        raise HTTPException(400, "The state events in this file are not a list.")
    out: list[dict] = []
    for i, entry in enumerate(raw):
        if not isinstance(entry, dict):
            raise HTTPException(400, f"State event {i + 1} is not an object.")
        event_type = entry.get("eventType")
        if not isinstance(event_type, str) or not event_type.strip():
            raise HTTPException(400, f"State event {i + 1} has no type.")
        out.append({
            "proposal": _int_or_none(entry, "proposal"),
            "action": _planned_action_ref(entry.get("action"), by_id,
                                          f"State event {i + 1}"),
            "branch": _as_index(entry.get("branch"), branches, default=None),
            "depth": _int_or_none(entry, "depth"),
            "sequence": _as_int(entry.get("sequence"), 0),
            "eventType": event_type[:60],
            "payload": _dict_or_none(entry, "payload"),
            "before": _dict_or_none(entry, "before"),
            "source": str(entry.get("source") or "accepted_story")[:40],
            "createdAt": _as_time(entry.get("createdAt")),
        })
    return out


def _planned_action_ref(value, by_id: dict[int, int], subject: str) -> int | None:
    """Resolves an audit record's node reference to a position in the plan."""
    if value is None:
        return None
    if not _is_int(value) or value not in by_id:
        raise HTTPException(
            400, f"{subject} names action {value!r}, which is not in this file."
        )
    return by_id[value]


def _derived_tip(branches: list[dict], nodes: list[dict], head: int) -> int:
    """Returns where the head branch's story ends, which is where a file that
    does not state a head depth is opened.

    This is the rule `tree.refresh_head` applies and the one every bundle
    written before M3 was exported under: the deepest node that arrived on the
    head branch, or, for a branch that carries none of its own, the fork point
    it inherits its story up to.
    """
    depths = [n["depth"] for n in nodes if n["branch"] == head]
    if depths:
        return max(depths)
    fork_depth = branches[head].get("forkDepth")
    return fork_depth if fork_depth is not None else lineage.NO_DEPTH


def _planned_head_depth(
    bundle: dict, branches: list[dict], nodes: list[dict], head: int
) -> int | None:
    """Returns the active head depth the file states, or None if it states none.

    Checked here rather than repaired later, for the reason the module docstring
    gives: a coordinate that disagrees with the rows is not a value any read can
    be given, and an import that discovers it afterwards has already written
    half a tree.

    Two bounds. A head past the retained tip is a file claiming the story is
    read somewhere it does not reach — the shape a truncated or hand-edited
    export takes, and the one that would silently move the reader forward.
    Below, `NO_DEPTH` is the floor, because an adventure with no story at all
    sits one step in front of its first node.

    A head *behind* the tip is not an error. It is the whole point: it is a
    story the user undid and did not redo, and it must import undone.
    """
    stated = bundle.get("headDepth")
    if stated is None:
        return None
    tip = _derived_tip(branches, nodes, head)
    if not _is_int(stated) or stated < lineage.NO_DEPTH:
        raise HTTPException(
            400, f"The file gives the active head depth as {stated!r}."
        )
    if stated > tip:
        raise HTTPException(
            400,
            f"The file reads its story at depth {stated}, but branch {head} "
            f"ends at {tip}.",
        )
    return stated


def _planned_branches(bundle: dict) -> list[dict]:
    raw = bundle.get("branches")
    entries = [b for b in raw if isinstance(b, dict)] if isinstance(raw, list) else []
    if not entries:
        return [dict(_ROOT)]
    specs: list[dict] = []
    for i, entry in enumerate(entries):
        parent = entry.get("parent")
        name = _planned_branch_name(entry, i)
        left = _planned_supersession(entry)
        if parent is None:
            specs.append(dict(_ROOT, **({"name": name} if name else {}), **left))
            continue
        # A branch may fork only from a branch listed before it. The export
        # writes them that way, because branches are numbered in creation order
        # and a parent always exists first. Requiring it here guarantees the
        # graph is acyclic for the cost of one comparison. A lineage is computed
        # by walking to the parent, so a cycle would be an import that never
        # returns.
        if not _is_int(parent) or not 0 <= parent < i:
            raise HTTPException(
                400,
                f"Branch {i} forks from branch {parent!r}, which is not one of "
                f"the {i} branches listed before it.",
            )
        fork_depth = entry.get("forkDepth")
        if not _is_int(fork_depth):
            raise HTTPException(
                400,
                f"Branch {i} forks from branch {parent} but does not say at "
                f"what depth.",
            )
        specs.append({
            "parent": parent, "forkDepth": fork_depth,
            **({"name": name} if name else {}),
            **left,
        })
    return specs


def _planned_supersession(entry: dict) -> dict:
    """Returns the branch's disposition as the file gives it, or nothing.

    Both keys or neither. A time with no depth cannot say what was displaced and
    a depth with no time is not a record of anything having happened, so a file
    carrying one of them is treated as carrying neither rather than half of a
    fact — no read depends on these columns, and inventing the missing half
    would be the only way to get a wrong answer out of them.
    """
    at = _as_time(entry.get("supersededAt"))
    depth = entry.get("supersededDepth")
    if at is None or not _is_int(depth):
        return {}
    return {"supersededAt": at, "supersededDepth": depth}


def _planned_branch_name(entry: dict, i: int) -> str | None:
    """Returns the name a branch entry carries, or `None` if nobody named it.

    The check runs before the row is created rather than being left to the
    column, for the reason the planner exists. A 400 from a function with no side
    effects is better than a half-written adventure and a database error three
    branches in.
    """
    raw = entry.get("name")
    if raw is None:
        return None
    if not isinstance(raw, str):
        raise HTTPException(400, f"Branch {i} has a name that is not text.")
    name = raw.strip()
    if len(name) > schemas.BRANCH_NAME_MAX:
        raise HTTPException(
            400,
            f"Branch {i}'s name is longer than {schemas.BRANCH_NAME_MAX} "
            f"characters.",
        )
    return name or None


def _planned_nodes(bundle: dict, branches: int) -> list[dict]:
    raw = bundle.get("actions")
    nodes: list[dict] = []
    for entry in raw if isinstance(raw, list) else []:
        if not isinstance(entry, dict) or not str(entry.get("text") or ""):
            continue
        branch = entry.get("branch", 0)
        if not _is_int(branch) or not 0 <= branch < branches:
            raise HTTPException(
                400,
                f"An action names branch {branch!r}, but the file lists "
                f"{branches}.",
            )
        depth = entry.get("depth")
        if not _is_int(depth) or depth < 0:
            raise HTTPException(
                400, f"An action on branch {branch} has no depth to sit at."
            )
        nodes.append({
            "branch": branch,
            "depth": depth,
            "live": bool(entry.get("live", True)),
            "narrativeStateAfter": _as_dict(entry.get("narrativeStateAfter")),
            "stateChanges": _as_dict(entry.get("stateChanges")),
            "type": str(entry.get("type") or "story")[:TYPE_MAX],
            "text": str(entry.get("text") or ""),
            "reasoning": _as_text(entry.get("reasoning")),
            "stateAfter": _as_dict(entry.get("stateAfter")),
            "worldStateAfter": _as_dict(entry.get("worldStateAfter")),
            "worldDelta": _as_dict(entry.get("worldDelta")),
            "createdAt": _as_time(entry.get("createdAt")),
            # M9. The node's identity within this file, and the turn it is an
            # attempt at. Both are absent from every version 2 bundle, and a
            # node with no id simply takes part in nothing that refers to one:
            # `attempts.group` falls back to the coordinate for it, exactly as
            # it does for a pre-SP9 row, which is the rule such a file was
            # written under.
            "id": _int_or_none(entry, "id"),
            "parentId": _int_or_none(entry, "parentId"),
            # M9. Historical evidence, restored verbatim. Nothing here is
            # regenerated and nothing is validated beyond its being an object:
            # it is a record of what an older build assembled, and imposing
            # today's expectations on it would be the retroactive reading the
            # inspector exists to rule out. The one thing the import does touch
            # is the source ids inside it, which name rows on the machine that
            # wrote the file; see `_relink_snapshots`.
            #
            # The plain key wins over the compressed one when a file carries
            # both, because somebody who hand-edited a prompt into readable JSON
            # meant the readable one.
            "contextSnapshot": (
                _as_dict(entry.get("contextSnapshot"))
                or _unpacked(entry.get("contextSnapshotZ"))
            ),
        })
    return nodes


def _planned_v1_nodes(bundle: dict) -> list[dict]:
    """Returns a version 1 bundle's turns as nodes, one node per attempt.

    The `variants` array is the repeating group SP4 unpacked, so reading one
    performs the same split that migration 60 does. Every attempt becomes a row
    at the turn's coordinate, and `variantIndex` selects the live one. The index
    is clamped, because a hand-edited bundle can name an attempt its own list
    does not contain, and a turn with no live node is a turn no read can see.

    The depth is the bundle's `index`. A version 1 bundle has one branch, where
    the two numbers agree.
    """
    raw = bundle.get("actions")
    nodes: list[dict] = []
    for i, entry in enumerate(raw if isinstance(raw, list) else []):
        if not isinstance(entry, dict) or not str(entry.get("text") or ""):
            continue
        depth = _as_int(entry.get("index"), i)
        kind = str(entry.get("type") or "story")[:TYPE_MAX]
        variants = [v for v in (entry.get("variants") or []) if isinstance(v, dict)]
        if not variants:
            variants = [{"text": entry["text"], "reasoning": entry.get("reasoning")}]
        live = min(max(_as_int(entry.get("variantIndex"), 0), 0), len(variants) - 1)
        for n, variant in enumerate(variants):
            text = str(variant.get("text") or "")
            if not text:
                continue
            nodes.append({
                "branch": 0,
                "depth": max(depth, 0),
                "live": n == live,
                "type": kind,
                "text": text,
                "reasoning": _as_text(variant.get("reasoning")),
                "stateAfter": None,
                "worldStateAfter": None,
                "worldDelta": None,
                "createdAt": _as_time(variant.get("createdAt")),
                # A version 1 file names no node and carries no prompt. Both
                # keys are present so that `_write_nodes` reads one shape for
                # every version; `None` here means the format could not say,
                # which for parentage puts the row on `attempts.group`'s
                # coordinate fallback — the rule it was written under.
                "id": None,
                "parentId": None,
                "contextSnapshot": None,
            })
    return nodes


def _planned_memories(bundle: dict, branches: int) -> list[dict]:
    raw = bundle.get("memories")
    out: list[dict] = []
    for entry in raw if isinstance(raw, list) else []:
        if not isinstance(entry, dict) or not str(entry.get("text") or "").strip():
            continue
        out.append({
            "text": str(entry["text"]),
            "pinned": bool(entry.get("pinned", False)),
            "forgotten": bool(entry.get("forgotten", False)),
            "sourceStart": entry.get("sourceStart"),
            "sourceEnd": entry.get("sourceEnd"),
            "useCount": _as_int(entry.get("useCount"), 0),
            # A value that is out of range, rather than absent, means the file
            # disagrees with itself. The root is the safe reading, because a
            # memory on a branch nothing can see never reaches a prompt
            # again.
            "branch": _as_index(entry.get("branch"), branches, default=0),
            "depth": _int_or_none(entry, "depth"),
            # M9. `heuristic` or `accepted_story` (F07). An unrecognised value
            # is read as `heuristic`, which is the safe direction: the failure
            # this guards against is a guess about the story being presented to
            # the narrator as something the story established, and an
            # accepted-story memory demoted to a hedged one costs a little
            # weight in ranking rather than a false fact.
            #
            # A file with no key here is every bundle before version 3, and it
            # gets the column default. That is not a repair — the value simply
            # was not recorded, and re-classifying the text now would be this
            # build's reading of a memory an older one wrote.
            "authority": (
                entry["authority"]
                if entry.get("authority") in (memorybank.ACCEPTED_STORY,
                                              memorybank.HEURISTIC)
                else memorybank.HEURISTIC if "authority" in entry
                else None
            ),
        })
    return out


def _planned_checkpoints(
    bundle: dict, branches: int, nodes: list[dict]
) -> list[dict]:
    """Returns the file's Save Points, checked against the tree it also carries.

    A Save Point whose coordinate names no turn in the file is dropped rather
    than imported, and dropped rather than raising. The two halves of that are
    each deliberate:

    * Dropped, because an imported pointer to a position the imported story does
      not contain is a Save Point that can only ever refuse to restore. It would
      be a row that exists to disappoint.
    * Not a 400, unlike the head depth. The head is a position the campaign is
      read at, so a file that misplaces it opens the story in the wrong place
      and every read is affected. A Save Point is a bookmark, and a bad one
      spoils nothing else in the file — refusing to import a whole campaign
      because one bookmark is wrong would lose the story to save the bookmark.

    A name that is blank once trimmed is dropped for the same reason the create
    endpoint refuses one: an unnamed Save Point is not identifiable in a list.
    """
    # Every coordinate the file writes, not only the ones it marks live.
    # `_write_nodes` makes exactly one attempt at each coordinate live whatever
    # the file says, so a coordinate that exists is a coordinate that will
    # resolve — and reading the flags here would drop a Save Point over a
    # question the writer has already settled.
    written = {(n["branch"], n["depth"]) for n in nodes}
    raw = bundle.get("checkpoints")
    out: list[dict] = []
    for entry in raw if isinstance(raw, list) else []:
        if not isinstance(entry, dict):
            continue
        name = str(entry.get("name") or "").strip()[:schemas.CHECKPOINT_NAME_MAX]
        if not name:
            continue
        depth = entry.get("depth")
        if not _is_int(depth):
            continue
        branch = _as_index(entry.get("branch"), branches, default=None)
        if branch is None or (branch, depth) not in written:
            continue
        out.append({
            "name": name,
            "note": str(entry.get("note") or ""),
            "branch": branch,
            "depth": depth,
            "createdAt": _as_time(entry.get("createdAt")),
        })
    return out


def _planned_anchors(bundle: dict, branches: int) -> dict:
    anchors = {}
    for name in ("memory", "summary"):
        raw = bundle.get(f"{name}Cursor")
        raw = raw if isinstance(raw, dict) else {}
        branch = raw.get("branch")
        anchors[name] = (
            _as_index(branch, branches) if branch is not None else None,
            _as_int(raw.get("depth"), lineage.NO_DEPTH),
        )
    return anchors


# ------------------------------------------------------------------ writing

def write(db: Session, adventure: models.Adventure, story: dict) -> dict:
    """Writes a planned tree onto a newly created adventure.

    The order is fixed, and every step in it needs something the step before it
    produced. Branches come first, because a node needs a branch id. The nodes
    come next, because the head, the anchors, the audit records and the take
    parentage all name a node. The knowledge comes before the snapshots are
    relinked, because the relink needs the ids the sources land on.

    Returns what the caller has to report to the reader: the knowledge sources
    whose rebuildable index could not be built. The authoritative campaign is
    committed either way — see `_write_knowledge` — and the difference between
    "the import failed" and "the import succeeded and the search index did not"
    is one a reader has to be told, not one to leave in a column.
    """
    ids = _write_branches(db, adventure, story["branches"])
    rows = _write_nodes(db, adventure, story["nodes"], ids)
    # The nodes need ids of their own before anything can point at them. One
    # flush covers the parentage, the audit records and the Save Points.
    db.flush()
    _link_take_parents(story["nodes"], rows)
    _write_memories(db, adventure, story["memories"], ids)
    _point_the_head(adventure, story, ids)
    _write_checkpoints(db, adventure, story["checkpoints"], ids)
    _write_anchors(adventure, story, ids)
    _write_summaries(db, adventure, story.get("summaries") or [], ids)
    _write_visual_profiles(db, adventure, story.get("visualProfiles") or [])
    _write_state_history(db, adventure, story, ids, rows)
    sources = _write_knowledge(db, adventure, story.get("knowledge") or [])
    # Last, because it needs both halves: the nodes carrying the snapshots and
    # the knowledge rows the snapshots' retrieval records point at.
    _relink_snapshots(rows, sources)
    return {
        "knowledge_index_failures": [
            {"title": source.title, "detail": source.index_detail}
            for source in sources.values() if source.index_state == "failed"
        ],
    }


def _link_take_parents(specs: list[dict], rows: list[models.Action]) -> None:
    """Points each imported node at the turn it is an attempt at (M9, SP9).

    The file names the parent by the id it had where the file was written, so
    this walks the pair of lists once and translates. A node whose parent is not
    in the file is left with none — that is a root node, a pre-SP9 row, or an
    older bundle that carried no parentage at all, and for all three
    `attempts.group` falls back to the coordinate, which is the rule they were
    written under.

    The parentage is deliberately not validated into a refusal. A wrong parent
    costs a take pager that groups oddly; a refused import costs the campaign.
    """
    by_exported_id = {
        spec["id"]: row for spec, row in zip(specs, rows)
        if spec.get("id") is not None
    }
    for spec, row in zip(specs, rows):
        parent = by_exported_id.get(spec.get("parentId"))
        # A node cannot be its own parent, and `attempts.group` would loop on
        # one that was. A file claiming it is treated as claiming nothing.
        if parent is not None and parent is not row:
            row.parent_id = parent.id


def _write_knowledge(
    db: Session, adventure: models.Adventure, specs: list[dict]
) -> dict[int, models.KnowledgeSource]:
    """Restores the imported library, and rebuilds the index it needs.

    The bundle carries the source and not its passages, so this is where they
    come back: `build_index` runs the same deterministic chunker the original
    import ran, against the same text, and produces the same passages. Lexical
    retrieval therefore works the moment the import finishes, with no reindex
    step and no explanation owed to the reader.

    Vectors do not come back, because they were never in the file. The source
    lands `embed_state = "idle"` with no vectors, and the next turn's post-turn
    pass builds them against whatever embedding model *this* machine has — which
    is the right answer, and the reason exporting the vectors would have been
    the wrong one.

    A source whose passages cannot be built is recorded as `failed` with the
    reason rather than raising. By this point the story, its tree, its head and
    its Save Points are already written, and refusing the whole campaign over a
    rebuildable index would trade the valuable thing for the cheap one. The
    failure is visible on the source and in the knowledge status endpoint, and
    Reindex is the repair.
    """
    landed: dict[int, models.KnowledgeSource] = {}
    for spec in specs:
        source = models.KnowledgeSource(
            adventure_id=adventure.id,
            title=spec["title"],
            original_filename=spec["original_filename"],
            classification=spec["classification"],
            enabled=spec["enabled"],
            visibility=spec["visibility"],
            always_include=spec["always_include"],
            content=spec["content"],
            content_hash=spec["content_hash"],
            byte_size=len(spec["content"].encode("utf-8")),
            media_type=spec["media_type"],
            notes=spec["notes"],
            parser_version=knowledge_chunking.PARSER_VERSION,
            chunking_version=knowledge_chunking.CHUNKING_VERSION,
            index_state="pending",
            embed_state="idle",
        )
        if spec["imported_at"] is not None:
            source.imported_at = spec["imported_at"]
        if spec["hash_mismatch"]:
            # Not a refusal. The content is what it is, and the recomputed hash
            # above is the one stored — but the file said something different,
            # which means it was edited after it was written, and the reader
            # should be able to find that out.
            source.notes = (
                f"{source.notes}\n[import] The content hash in the export file "
                "did not match the content it carried; the stored hash was "
                "recomputed from what arrived."
            ).strip()
        db.add(source)
        db.flush()
        if spec.get("source_id") is not None:
            landed[spec["source_id"]] = source
        try:
            knowledge_importer.build_index(db, source)
        except Exception as exc:  # noqa: BLE001 - recorded, not raised
            source.index_state = "failed"
            source.index_detail = f"{type(exc).__name__}: {exc}"[:2000]
    return landed


def _write_summaries(
    db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
) -> None:
    """Restores the rolling summaries onto the coordinates that hold them.

    Nothing about eligibility is decided here, and that is the point. A summary
    is eligible when its coordinate lies on the active capped lineage, which
    `summaries.current` works out from the tree at read time — so restoring the
    coordinate restores the answer, including the answer "no" for a summary
    belonging to a line this campaign has left (E03).

    `adventures.story_summary` is the reader-facing mirror and is set from the
    bundle's own `storySummary` by `materialize`. `settle_the_mirror` corrects it
    afterwards from whichever summary the restored head is actually entitled to,
    because a mirror that disagrees with the lineage is exactly the M6 defect
    that made summaries rows in the first place.
    """
    for spec in specs:
        summary = models.Summary(
            adventure_id=adventure.id,
            text=spec["text"],
            branch_id=ids[spec["branch"]],
            depth=spec["depth"],
            source_start=spec["sourceStart"],
            source_end=spec["sourceEnd"],
            trigger=spec["trigger"],
            model_name=spec["model"],
        )
        if spec["createdAt"] is not None:
            summary.created_at = spec["createdAt"]
        db.add(summary)


def _write_visual_profiles(
    db: Session, adventure: models.Adventure, specs: list[dict]
) -> None:
    """Restores the campaign's visual profiles.

    No entity check on the way in, unlike `media.profiles.set_profile`. The
    check there catches a typo against the campaign the reader is looking at; on
    an import the state document arrives in the same file, so a profile naming
    an entity the file also carries is correct by construction, and one naming
    an entity that only exists on a branch this campaign has left is still worth
    keeping — the description is about how something looks, and the entity may
    become reachable again.
    """
    for spec in specs:
        profile = models.VisualProfile(
            adventure_id=adventure.id,
            entity_key=spec["entity_key"],
            descriptors=spec["descriptors"],
            features=spec["features"],
            style_notes=spec["style_notes"],
        )
        if spec["created_at"] is not None:
            profile.created_at = spec["created_at"]
        db.add(profile)


def _write_state_history(
    db: Session, adventure: models.Adventure, story: dict, ids: list[int],
    rows: list[models.Action],
) -> None:
    """Restores the audit half of the hybrid: proposals, then the events.

    Proposals first, because an event names one. Both are written as they were
    recorded and neither is replayed: `DATA-MODEL.md` §17 is explicit that the
    events are the audit record and the snapshots are the restore path, so
    nothing here recomputes a state document and nothing here checks that the
    events add up to the snapshot beside them. They are two accounts of the same
    history and the import is restoring both, not reconciling them.

    A rejected proposal is restored exactly like an accepted one. It changed
    nothing then and it changes nothing now — it is the record that explains why
    the state does not say what the narration seems to say, which is the case
    where a reader most needs it.
    """
    proposals: dict[int, models.StateProposal] = {}
    for spec in story.get("proposals") or []:
        proposal = models.StateProposal(
            adventure_id=adventure.id,
            action_id=_action_id(spec["action"], rows),
            branch_id=ids[spec["branch"]] if spec["branch"] is not None else None,
            depth=spec["depth"],
            model_name=spec["model"],
            source=spec["source"],
            status=spec["status"],
            raw_output=spec["rawOutput"],
            detail=spec["detail"],
        )
        if spec["createdAt"] is not None:
            proposal.created_at = spec["createdAt"]
        db.add(proposal)
        if spec["id"] is not None:
            proposals[spec["id"]] = proposal
    # The events name proposals, so the proposals need ids first.
    if proposals:
        db.flush()
    for spec in story.get("events") or []:
        proposal = proposals.get(spec["proposal"])
        event = models.StateEvent(
            adventure_id=adventure.id,
            proposal_id=proposal.id if proposal is not None else None,
            action_id=_action_id(spec["action"], rows),
            branch_id=ids[spec["branch"]] if spec["branch"] is not None else None,
            depth=spec["depth"],
            sequence=spec["sequence"],
            event_type=spec["eventType"],
            payload=spec["payload"],
            before=spec["before"],
            source=spec["source"],
        )
        if spec["createdAt"] is not None:
            event.created_at = spec["createdAt"]
        db.add(event)


def _action_id(position: int | None, rows: list[models.Action]) -> int | None:
    """The database id of the node at `position` in the planned list."""
    return rows[position].id if position is not None else None


def _relink_snapshots(
    rows: list[models.Action], sources: dict[int, models.KnowledgeSource]
) -> None:
    """Points a restored snapshot's retrieval records at the sources that arrived.

    The snapshot itself is evidence and is restored verbatim — the prompt, the
    passages, the text each one supplied, the settings the call ran under. What
    is *not* evidence is `source_id`: it is a pointer at a row on the machine
    that wrote the file, and the imported library holds the same sources under
    different ids. Left alone it would point the "open this source" control in
    the context inspector at whatever happens to hold that id here, which is
    either nothing or somebody else's file.

    So the pointer is translated and the evidence is not. Where the file's own
    knowledge section contains the source, the record names the row it landed
    on; where it does not — a source deleted before the export was taken — the
    record is left saying `null`, and the inspector shows the filename as plain
    text rather than a link. That is the honest answer: this passage came from a
    file that is no longer here, and here is what it said.

    `chunk_id` is deliberately untouched. Passages are rebuilt on import and get
    new ids, so no translation exists; the record's `chunk_index` and
    `heading_path` still say which passage of the source it was, and the
    rendered text is in the record either way.

    ## A new document, never a mutation in place

    `context_snapshot` is a plain `CompressedJSON` column and not a
    `MutableDict`, so SQLAlchemy tracks it by assignment and not by content.
    Editing the dict the attribute already holds and assigning it back changes
    nothing: at flush time the loaded value and the current value are the same
    object, the history reports no net change, and no UPDATE is emitted. The
    import then looks correct in memory and writes the untranslated ids to disk.

    So a modified snapshot is built as a new document and assigned. The copy is
    shallow at every level this touches — a new outer dict, a new `knowledge`
    dict, new lists, and a new dict per record — and shares everything it does
    not change, including the rendered passage text, which is the large part.
    """
    if not sources:
        # Nothing to translate against. Every record keeps whatever it says,
        # which for a campaign with no library is nothing that resolves anyway.
        return
    for row in rows:
        snapshot = row.context_snapshot
        if not isinstance(snapshot, dict):
            continue
        knowledge = snapshot.get("knowledge")
        if not isinstance(knowledge, dict):
            continue
        rebuilt = {}
        touched = False
        for key in ("used", "dropped", "suppressed"):
            records = knowledge.get(key)
            if not isinstance(records, list):
                continue
            replaced = []
            for record in records:
                if not isinstance(record, dict) or "source_id" not in record:
                    replaced.append(record)
                    continue
                landed = sources.get(record["source_id"])
                replaced.append(
                    dict(record, source_id=landed.id if landed is not None else None)
                )
                touched = True
            rebuilt[key] = replaced
        if touched:
            row.context_snapshot = dict(
                snapshot, knowledge=dict(knowledge, **rebuilt)
            )


def _write_branches(
    db: Session, adventure: models.Adventure, specs: list[dict]
) -> list[int]:
    """Writes one row per branch, computing the lineage rather than reading it.

    The rows are inserted through Core and the lineage is written second, for the
    reason `tree.root_branch` gives: the lineage names the row's own id. The
    parent's cached ancestry is capped at this fork, which is the arithmetic
    `tree.fork` performs. A fork made now and a fork made a year ago and then
    exported have to produce the same rows.
    """
    ids: list[int] = []
    lineages: list[list[list]] = []
    for spec in specs:
        parent = spec.get("parent")
        fork_depth = spec.get("forkDepth")
        new_id = db.execute(
            insert(models.Branch).values(
                adventure_id=adventure.id,
                parent_branch_id=ids[parent] if parent is not None else None,
                fork_depth=fork_depth if parent is not None else None,
                lineage=[],
                name=spec.get("name"),
                created_at=models.utcnow(),
                superseded_at=spec.get("supersededAt"),
                superseded_depth=spec.get("supersededDepth"),
            )
        ).inserted_primary_key[0]
        entries = [[new_id, None]]
        if parent is not None:
            entries += [
                [branch_id, fork_depth if cap is None else min(cap, fork_depth)]
                for branch_id, cap in lineages[parent]
            ]
        db.execute(
            update(models.Branch)
            .where(models.Branch.id == new_id)
            .values(lineage=entries)
        )
        ids.append(new_id)
        lineages.append(entries)
    return ids


def _write_nodes(
    db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
) -> list[models.Action]:
    """Writes the nodes, grouped into the turns they are attempts at.

    Returns the rows in the order the specs were given, so that the caller can
    match each one back to the file entry that produced it — the take parentage
    and every audit record need exactly that correspondence.

    One value is decided here rather than read from the file. Exactly one
    attempt in each group is made live, because a file can name none or several,
    and a turn with no live node disappears from the story.
    """
    rows: list[models.Action] = []
    groups: dict[tuple[int, int], list[models.Action]] = {}
    for spec in specs:
        key = (spec["branch"], spec["depth"])
        action = models.Action(
            adventure_id=adventure.id,
            branch_id=ids[spec["branch"]],
            depth=spec["depth"],
            type=spec["type"],
            text=spec["text"],
            reasoning=spec["reasoning"],
            live=spec["live"],
            state_after=spec["stateAfter"],
            world_state_after=spec["worldStateAfter"],
            world_delta=spec["worldDelta"],
            # M9. Written explicitly, including when the file carries none.
            #
            # Leaving it NULL hands the decision to `tree.stamp_outcome`, which
            # runs on every flush and fills a missing snapshot from *the
            # campaign's current state*. On an import that is the state at the
            # exported head, so every position whose snapshot the file did not
            # carry would come back holding the newest position's state — an
            # Undo landing on turn 2 showing what the story knew at turn 20,
            # which is exactly the failure M5's review finding 3 was about,
            # arriving by a different route.
            #
            # The empty document is the honest value: this position established
            # nothing that the file records. It is also what `restore_state`
            # falls back to for a NULL and what migration 88 backfilled for
            # pre-M5 rows, so a legacy bundle lands exactly where it landed
            # before — the fallback wrote `empty()` for those files too, because
            # a campaign with no `narrativeState` has no head state to copy.
            narrative_state_after=(
                spec.get("narrativeStateAfter")
                if spec.get("narrativeStateAfter") is not None
                else narrative_model.empty()
            ),
            state_changes=spec.get("stateChanges"),
            # M9. Restored exactly as it was written. See `_planned_nodes`.
            context_snapshot=spec.get("contextSnapshot"),
        )
        if spec["createdAt"] is not None:
            action.created_at = spec["createdAt"]
        db.add(action)
        rows.append(action)
        groups.setdefault(key, []).append(action)

    for group in groups.values():
        live = next((row for row in group if row.live), group[0])
        for row in group:
            row.live = row is live
    return rows


def _write_memories(
    db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
) -> None:
    for spec in specs:
        memory = models.Memory(
            adventure_id=adventure.id,
            text=spec["text"],
            pinned=spec["pinned"],
            forgotten=spec["forgotten"],
            source_start=spec["sourceStart"],
            source_end=spec["sourceEnd"],
            use_count=spec["useCount"],
            branch_id=ids[spec["branch"]],
            depth=spec["depth"],
        )
        if spec.get("authority") is not None:
            memory.authority = spec["authority"]
        # A version 1 memory has no depth of its own. `source_end` is the index
        # of the last action it summarizes, which on one branch is that node's
        # depth.
        if memory.depth is None and memory.source_end is not None:
            memory.depth = memory.source_end
        # A version 1 memory that summarizes nothing, which means one the player
        # typed, has no depth to derive. Leaving it NULL here would recreate on
        # import the state migration 62 exists to end. `Path._entry_clause`
        # compares `depth <= max_depth`, which a NULL fails, so the memory would
        # disappear from every branch as soon as the imported adventure was
        # forked. The root is the answer the migration gives, for the same
        # reason: 0 is at or before every fork point, so the memory is visible
        # from every path this adventure can grow.
        if memory.depth is None:
            memory.depth = lineage.ROOT_DEPTH
        db.add(memory)


def _write_checkpoints(
    db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
) -> None:
    """Writes the Save Points, and moves nothing.

    Note what this function does not touch. The head is pointed by
    `_point_the_head` from the file's own `headBranch`/`headDepth`, and importing
    a Save Point must not disturb it — a campaign exported at turn 30 with a
    Save Point at turn 12 opens at turn 30. The bundle records where the story
    was being read and, separately, which positions someone named; restoring one
    of them is a thing the user does afterwards, not a thing an import does for
    them.
    """
    for spec in specs:
        checkpoint = models.Checkpoint(
            adventure_id=adventure.id,
            name=spec["name"],
            note=spec["note"],
            branch_id=ids[spec["branch"]],
            depth=spec["depth"],
        )
        if spec["createdAt"] is not None:
            checkpoint.created_at = spec["createdAt"]
        db.add(checkpoint)


def _point_the_head(
    adventure: models.Adventure, story: dict, ids: list[int]
) -> None:
    """Sets which branch the story is played on, and where it is being read.

    Both come from the file now (M3). A bundle that states its head depth is
    opened exactly where its owner left it, undone turns and all, and the turns
    past that point arrive as the retained future they were exported as.

    A file that states none is opened at the tip of its head branch — whatever
    arrived on it, or, for a branch carrying no nodes of its own, the fork point
    that is the last node its story contains. That is the rule
    `tree.refresh_head` applies and the only answer a pre-M3 bundle can be given,
    because at the time it was written the head could not be anywhere else.
    """
    head = story["head"]
    adventure.head_branch_id = ids[head]
    if story["headDepth"] is not None:
        adventure.head_depth = story["headDepth"]
        return
    adventure.head_depth = _derived_tip(story["branches"], story["nodes"], head)


def _write_anchors(
    adventure: models.Adventure, story: dict, ids: list[int]
) -> None:
    """Records how far the memories and the summary have read. Version 2 only.

    A version 1 bundle stores a count instead, and a count cannot be resolved to
    a node until the nodes are in the database. `settle` does that part
    afterwards.
    """
    anchors = story["anchors"]
    if anchors is None:
        return
    for cursor in cursors.ALL:
        branch, depth = anchors[cursor.name]
        cursor.anchor(adventure, ids[branch] if branch is not None else None, depth)


def settle(adventure: models.Adventure, story: dict) -> None:
    """Resolves a version 1 bundle's counts into anchors, once the nodes exist.

    A version 1 file records how far the memories and the summary have read as
    a count, so the anchor is found by counting that far along the story. A
    version 2 file carries the anchor itself, which `_write_anchors` has
    already stored, so there is nothing left to do.

    The caller runs this after the flush, because counting needs the actions to
    be queryable.
    """
    positions = story["positions"]
    if positions is None:
        return
    for cursor in cursors.ALL:
        cursors.anchor_at_position(adventure, cursor, positions[cursor.name])


def materialize(
    db: Session, payload: dict, story: dict, user_id: int | None
) -> tuple[models.Adventure, dict]:
    """Writes a planned bundle into a new adventure owned by `user_id`.

    Call `check_format` and `plan` first, and apply any rate or size limits
    before calling this. The split exists because those checks belong to the
    import endpoint alone: the guest starter writes a file the server ships, so
    it has nothing to rate-limit and no untrusted list to cap.

    Returns the adventure and a report of the derived work that did not
    complete. **The caller commits**, and everything below happens in that one
    transaction: an import that raises anywhere in here leaves no adventure at
    all rather than half of one (L01's rule, applied to the import path).

    Two flushes happen here, because the anchors and the legacy counts describe
    one boundary in two coordinate systems, and aligning them needs the actions
    to be queryable.
    """
    # A raw-dict import bypasses the schemas, so truncate strings bound for
    # VARCHAR columns. Postgres enforces the widths. See `schemas.py`.
    adventure = models.Adventure(
        user_id=user_id,
        title=str(payload.get("title") or "Imported Adventure")[:schemas.NAME_MAX],
        memory=str(payload.get("memory") or ""),
        authors_note=str(payload.get("authorsNote") or ""),
        ai_instructions=str(payload.get("aiInstructions") or ""),
        story_summary=str(payload.get("storySummary") or ""),
        world_state=payload.get("worldState") or {},
        # M5. Normalised on the way in, so a hand-edited or truncated state
        # section costs the section rather than the campaign — the story is the
        # valuable thing, and a malformed document should not refuse an import.
        narrative_state=narrative_model.normalize(payload.get("narrativeState"))
        if isinstance(payload.get("narrativeState"), dict) else None,
        campaign_canon=payload.get("campaignCanon")
        if isinstance(payload.get("campaignCanon"), dict) else None,
        auto_summarize=bool(payload.get("autoSummarize", False)),
        memory_bank_enabled=bool(payload.get("memoryBankEnabled", False)),
        **_imported_persona(payload.get("persona")),
    )
    db.add(adventure)
    db.flush()

    for card in payload.get("storyCards") or []:
        if isinstance(card, dict):
            db.add(models.StoryCard(
                adventure_id=adventure.id,
                type=str(card.get("type") or "")[:schemas.CARD_TYPE_MAX],
                name=str(card.get("name") or "")[:schemas.NAME_MAX],
                keys=str(card.get("keys") or ""),
                entry=str(card.get("entry") or ""),
                notes=str(card.get("notes") or ""),
            ))

    # A bundle exported before M2 may carry "scripts" and "scriptState".
    # Campaign scripting is gone, so both are ignored rather than rejected: the
    # story, its tree, its cards and its memories still import intact, which is
    # what the bundle is for.

    report = write(db, adventure, story)
    db.flush()
    db.expire(adventure, ["actions"])
    settle(adventure, story)
    settle_the_mirror(db, adventure)
    return adventure, report


def settle_the_mirror(db: Session, adventure: models.Adventure) -> None:
    """Points `story_summary` at whichever restored summary the head is entitled to.

    The bundle carries the mirror column as well as the rows, because it always
    has and because a v1 or v2 file carries nothing else. But the column has no
    lineage of its own, and the value that was correct where the file was written
    is only correct here if the head landed in the same place — which it does,
    but relying on that would be relying on a coincidence rather than on the
    rule. M6's finding M6-F1 was precisely a mirror that had drifted from the
    lineage, so the import ends by asking the lineage.

    A campaign whose file carried no summary rows keeps whatever the column
    said: an older bundle's mirror is the only summary it has, and blanking it
    would lose the one thing that file recorded.
    """
    if not adventure.summaries:
        return
    summaries.refresh_mirror(db, adventure)

# ------------------------------------------------------------------ reading
# Small coercions. A raw-dict import bypasses the schemas, so every value from a
# bundle is whatever the JSON held.

def _is_int(value) -> bool:
    """Returns whether `value` is an integer. Python counts `True` as an `int`,
    and a coordinate does not."""
    return isinstance(value, int) and not isinstance(value, bool)


def _as_int(value, default: int) -> int:
    return value if _is_int(value) else default


def _as_index(value, count: int, default: int | None = None) -> int | None:
    """Returns a local branch number, or `default` when the file names a branch
    that is not present.

    An out-of-range value means the file disagrees with itself, and it is not a
    value any read can be given.
    """
    return value if _is_int(value) and 0 <= value < count else default


def _as_text(value) -> str | None:
    return str(value) if value else None


def _as_dict(value) -> dict | None:
    return value if isinstance(value, dict) else None


def _int_or_none(entry: dict, key: str) -> int | None:
    """An optional integer field, or `None` when the file gives something else.

    Distinct from `_as_int`, which substitutes a default: the fields this serves
    are ones where absent and unusable mean the same thing — *the file does not
    say* — and where inventing a number would be inventing a coordinate.
    """
    value = entry.get(key)
    return value if _is_int(value) else None


def _dict_or_none(entry: dict, key: str) -> dict | None:
    """An optional object field, or `None` when the file gives something else."""
    value = entry.get(key)
    return value if isinstance(value, dict) else None


def _as_time(value) -> datetime | None:
    if not isinstance(value, str):
        return None
    try:
        return datetime.fromisoformat(value.replace("Z", "+00:00"))
    except ValueError:
        return None