Let a backup carry a story that went two ways
A bundle had one list and a forked adventure has two stories, so export was emitting every branch's turns interleaved by index — a mangled story rather than lost data, and unreachable only because forking has no UI yet. `ai-dnd-adventure-v2` carries the branches, the depth each one left its parent at, which attempt at every turn is the story, and what each node left behind. That last one is not decoration: the after-snapshots are what a branch switch puts back, and a bundle without them imports a tree nobody can switch inside. `app/bundle.py` owns both formats and nothing else knows either. The v1 reader stays — those files are already on people's disks — and it is now the only place a `variants` array exists anywhere. The rule the module is built on is that a bundle carries what was chosen and never what is derived. The head branch, the fork points, the live flags and the anchors are decisions somebody made. The lineage, the head depth, the legacy `index` and the variant ordinals are computed from those and are rebuilt on the way in, because a bundle is a text file anybody can edit and a derived field shipped beside its source is a chance for the file to disagree with itself where no read would report it. `index` is the one that stops being academic here. It agreed with `depth` until SP5, and this is the first writer that has to fill it for a forked story, where two branches both hold a node at depth 4. It is allocated one per turn instead: siblings share it, no two coordinates do. Everything a hand-edited file can get wrong about the shape of a tree is a 400 raised before the adventure row exists, because a half-applied import is exactly the failure this phase exists to end — a story that goes quiet. A file wrong about which attempt is live is corrected rather than refused; that is an invariant of the database, not of the format. Measured on the 600-action fixture: 587 kB to 911 kB, and all of the increase is the outcomes at 489 B a node — the coordinates themselves save 57.5 B a node against the old turn-and-variants shape. Twenty forks add 660 B. 4.3% of the import body cap. 381 tests green, 16 of them new in test_bundle_v2.py. No migration, no vacuum owed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
This commit is contained in:
committed by
Parth
co-authored by
Claude Opus 5
parent
d051501517
commit
a7bf47a35e
@@ -0,0 +1,609 @@
|
|||||||
|
"""Phase 14, SP6 — the export bundle, as a tree.
|
||||||
|
|
||||||
|
A bundle is the one place a story leaves the database, and the only part of the
|
||||||
|
tree no migration can ever reach: a file downloaded today has to still import
|
||||||
|
into a build shipped next year. So the format is versioned, both versions live
|
||||||
|
here, and nothing else in the app knows either of them.
|
||||||
|
|
||||||
|
**v1** is a flat list of turns, each with an optional `variants` array — the
|
||||||
|
repeating group SP4 unpacked into rows. Nothing writes that shape any more.
|
||||||
|
The *reader* stays, because bundles already on people's disks still have it and
|
||||||
|
a backup that stops importing is not a backup.
|
||||||
|
|
||||||
|
**v2** carries the tree. Three things it holds that v1 could not, each
|
||||||
|
load-bearing:
|
||||||
|
|
||||||
|
* **the branches**, because a forked adventure is two stories and a flat list
|
||||||
|
can hold one — v1 export interleaved them by `index`, which read as a mangled
|
||||||
|
story rather than as lost data;
|
||||||
|
* **`live`**, because a coordinate can hold several attempts at one turn and
|
||||||
|
exactly one of them is the story;
|
||||||
|
* **both after-snapshots**, because they are what a branch switch and an undo
|
||||||
|
put back. A bundle carrying the actions but not the outcomes would import a
|
||||||
|
tree nobody could switch inside.
|
||||||
|
|
||||||
|
## The rule about what a bundle carries
|
||||||
|
|
||||||
|
**What was chosen, never what is derived.** The head *branch*, the fork points,
|
||||||
|
the live flags and the anchors are decisions somebody made; they are in the
|
||||||
|
file. `lineage`, the head *depth*, `index` and the variant ordinals are all
|
||||||
|
computed from those, and they are recomputed on import instead:
|
||||||
|
|
||||||
|
* `lineage` is a cache of `parent` + `fork_depth`. Shipping it too would put a
|
||||||
|
second source of truth for one fact in a file anybody can hand-edit, and the
|
||||||
|
two could then disagree in a way no read would ever report.
|
||||||
|
* the head depth is the tip of the head branch, which is a fact about the nodes
|
||||||
|
that arrived with it.
|
||||||
|
* `index` is the legacy column SP8 drops. Its one remaining job is to hand the
|
||||||
|
next row a number nothing else holds — a fact about the *adventure*, not
|
||||||
|
about a path — so `depth` cannot be it: two branches have a node at depth 4.
|
||||||
|
The import allocates one per turn instead, which keeps `max_action_index`
|
||||||
|
honest and keeps siblings sharing an index the way SP4 leaves them.
|
||||||
|
* the variant ordinals are `attempts.renumber`'s to maintain, and it is the
|
||||||
|
only place allowed to.
|
||||||
|
|
||||||
|
Every hand-editable coordinate is therefore checked before a row is written
|
||||||
|
(`plan`), not fixed up afterwards: an import that fails halfway leaves an
|
||||||
|
adventure holding half a tree, and a tree missing a branch is a story that
|
||||||
|
silently stops rather than one that reports.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy import insert, update
|
||||||
|
from sqlalchemy.orm import Session, undefer
|
||||||
|
|
||||||
|
from . import attempts, models
|
||||||
|
from .context import cursors, lineage
|
||||||
|
|
||||||
|
FORMAT = "ai-dnd-adventure-v2"
|
||||||
|
LEGACY_FORMAT = "ai-dnd-adventure-v1"
|
||||||
|
|
||||||
|
# VARCHAR(20) on `actions.type`; a raw-dict import bypasses the schemas.
|
||||||
|
TYPE_MAX = 20
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- exporting
|
||||||
|
|
||||||
|
def export(db: Session, adventure: models.Adventure) -> dict:
|
||||||
|
"""The whole adventure as a v2 bundle.
|
||||||
|
|
||||||
|
Deliberately un-pathed: a backup wants the entire tree, not the branch its
|
||||||
|
owner happens to be standing on. Both after-snapshots are undeferred in the
|
||||||
|
one query — they are per-node columns nothing else reads in bulk, and asking
|
||||||
|
for them a row at a time would be a query per turn.
|
||||||
|
|
||||||
|
No context snapshots. A bundle has never carried the assembled prompts and
|
||||||
|
still does not: they are ~163 kB a turn, they are an explanation of a
|
||||||
|
generation rather than part of the story, and the Insights viewer they feed
|
||||||
|
is reading the adventure it came from.
|
||||||
|
"""
|
||||||
|
branches = (
|
||||||
|
db.query(models.Branch)
|
||||||
|
.filter(models.Branch.adventure_id == adventure.id)
|
||||||
|
.order_by(models.Branch.id)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
# Branch ids are local to the file — positions in this list — because the
|
||||||
|
# database ids they had here are already taken over there.
|
||||||
|
local = {branch.id: i for i, branch in enumerate(branches)}
|
||||||
|
nodes = (
|
||||||
|
db.query(models.Action)
|
||||||
|
.filter(models.Action.adventure_id == adventure.id)
|
||||||
|
.options(
|
||||||
|
undefer(models.Action.state_after),
|
||||||
|
undefer(models.Action.world_state_after),
|
||||||
|
)
|
||||||
|
.order_by(
|
||||||
|
models.Action.branch_id, models.Action.depth,
|
||||||
|
models.Action.variant_index, models.Action.id,
|
||||||
|
)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"format": FORMAT,
|
||||||
|
"title": adventure.title,
|
||||||
|
"memory": adventure.memory,
|
||||||
|
"authorsNote": adventure.authors_note,
|
||||||
|
"aiInstructions": adventure.ai_instructions,
|
||||||
|
"storySummary": adventure.story_summary,
|
||||||
|
"scriptState": adventure.script_state,
|
||||||
|
"worldState": adventure.world_state,
|
||||||
|
"autoSummarize": adventure.auto_summarize,
|
||||||
|
"memoryBankEnabled": adventure.memory_bank_enabled,
|
||||||
|
# A root entry even for an adventure whose branch row was never created
|
||||||
|
# — a story with no branch is a pre-tree one, and the tree it belongs to
|
||||||
|
# is the root. `_local` puts its nodes there.
|
||||||
|
"branches": [_exported_branch(b, local) for b in branches] or [_ROOT],
|
||||||
|
"headBranch": local.get(adventure.head_branch_id, 0),
|
||||||
|
"memoryCursor": _exported_anchor(adventure, cursors.MEMORY, local),
|
||||||
|
"summaryCursor": _exported_anchor(adventure, cursors.SUMMARY, local),
|
||||||
|
"memories": [_exported_memory(m, local) for m in adventure.memories],
|
||||||
|
"storyCards": [
|
||||||
|
{"type": c.type, "name": c.name, "keys": c.keys,
|
||||||
|
"entry": c.entry, "notes": c.notes}
|
||||||
|
for c in adventure.story_cards
|
||||||
|
],
|
||||||
|
"scripts": [
|
||||||
|
{
|
||||||
|
"position": s.position, "enabled": s.enabled,
|
||||||
|
"name": s.name, "description": s.description,
|
||||||
|
"library": s.library_js, "input": s.input_js,
|
||||||
|
"context": s.context_js, "output": s.output_js,
|
||||||
|
}
|
||||||
|
for s in adventure.scripts
|
||||||
|
],
|
||||||
|
"actions": [_exported_node(a, local) for a in nodes],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_ROOT = {"parent": None, "forkDepth": None}
|
||||||
|
|
||||||
|
|
||||||
|
def _local(branch_id: int | None, local: dict[int, int]) -> int:
|
||||||
|
return local.get(branch_id, 0) if branch_id is not None else 0
|
||||||
|
|
||||||
|
|
||||||
|
def _exported_branch(branch: models.Branch, local: dict[int, int]) -> dict:
|
||||||
|
parent = (
|
||||||
|
local.get(branch.parent_branch_id)
|
||||||
|
if branch.parent_branch_id is not None else None
|
||||||
|
)
|
||||||
|
if parent is None:
|
||||||
|
return dict(_ROOT)
|
||||||
|
return {"parent": parent, "forkDepth": branch.fork_depth}
|
||||||
|
|
||||||
|
|
||||||
|
def _exported_node(action: models.Action, local: dict[int, int]) -> dict:
|
||||||
|
node = {
|
||||||
|
"branch": _local(action.branch_id, local),
|
||||||
|
# A pre-tree row's depth is the number `index` already held.
|
||||||
|
"depth": action.depth if action.depth is not None else action.index,
|
||||||
|
"live": bool(action.live),
|
||||||
|
"type": action.type,
|
||||||
|
"text": action.text,
|
||||||
|
"createdAt": action.created_at.isoformat() if action.created_at else None,
|
||||||
|
}
|
||||||
|
if action.reasoning:
|
||||||
|
node["reasoning"] = action.reasoning
|
||||||
|
# `{}` and absent mean different things — "this node left an empty
|
||||||
|
# scoreboard behind" against "nobody knows, leave the live state alone" —
|
||||||
|
# so an empty snapshot is written out rather than trimmed. It costs about
|
||||||
|
# eighteen bytes a row and it is the difference between an undo that clears
|
||||||
|
# a score and one that leaves it standing.
|
||||||
|
if action.state_after is not None:
|
||||||
|
node["stateAfter"] = action.state_after
|
||||||
|
if action.world_state_after is not None:
|
||||||
|
node["worldStateAfter"] = action.world_state_after
|
||||||
|
if action.world_delta:
|
||||||
|
node["worldDelta"] = action.world_delta
|
||||||
|
return node
|
||||||
|
|
||||||
|
|
||||||
|
def _exported_memory(memory: models.Memory, local: dict[int, int]) -> dict:
|
||||||
|
return {
|
||||||
|
"text": memory.text, "pinned": memory.pinned, "forgotten": memory.forgotten,
|
||||||
|
"sourceStart": memory.source_start, "sourceEnd": memory.source_end,
|
||||||
|
"useCount": memory.use_count,
|
||||||
|
# The node it hangs off. A hand-written memory summarises no node, so it
|
||||||
|
# has a branch and no depth, and keeps that shape here.
|
||||||
|
"branch": _local(memory.branch_id, local),
|
||||||
|
"depth": memory.depth,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _exported_anchor(
|
||||||
|
adventure: models.Adventure, cursor: cursors.Cursor, local: dict[int, int]
|
||||||
|
) -> dict:
|
||||||
|
branch_id, depth = cursor.stored(adventure)
|
||||||
|
return {
|
||||||
|
"branch": local.get(branch_id) if branch_id is not None else None,
|
||||||
|
"depth": depth,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- importing
|
||||||
|
|
||||||
|
def check_format(bundle: dict) -> str:
|
||||||
|
"""The bundle's version, or 400."""
|
||||||
|
fmt = bundle.get("format")
|
||||||
|
if fmt in (FORMAT, LEGACY_FORMAT):
|
||||||
|
return fmt
|
||||||
|
raise HTTPException(
|
||||||
|
400,
|
||||||
|
f"Not an adventure export file (expected format {FORMAT} or {LEGACY_FORMAT}).",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def plan(bundle: dict, version: str) -> dict:
|
||||||
|
"""The bundle's tree, checked and normalised, before a row is written.
|
||||||
|
|
||||||
|
Pure: no session, no adventure, nothing created. Everything a hand-edited
|
||||||
|
file can get wrong about the *shape* of a tree is caught here, because the
|
||||||
|
alternative is an import that fails partway and leaves an adventure holding
|
||||||
|
a story with a hole in it.
|
||||||
|
|
||||||
|
Both versions land in the same shape, so `write` never learns there are two
|
||||||
|
formats: a v1 bundle is a linear story, which is a tree with one branch, and
|
||||||
|
its `variants` array is a sibling group written the old way.
|
||||||
|
"""
|
||||||
|
branches = (
|
||||||
|
_planned_branches(bundle) if version == FORMAT else [dict(_ROOT)]
|
||||||
|
)
|
||||||
|
nodes = (
|
||||||
|
_planned_nodes(bundle, len(branches)) if version == FORMAT
|
||||||
|
else _planned_v1_nodes(bundle)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"branches": branches,
|
||||||
|
"nodes": nodes,
|
||||||
|
"memories": _planned_memories(bundle, len(branches)),
|
||||||
|
"head": _as_index(bundle.get("headBranch"), len(branches), default=0),
|
||||||
|
# v2 knows where the derived work got to; v1 counted it, and a count
|
||||||
|
# cannot be turned into a node until the nodes exist (see `settle`).
|
||||||
|
"anchors": _planned_anchors(bundle, len(branches)) if version == FORMAT else None,
|
||||||
|
"positions": None if version == FORMAT else {
|
||||||
|
"memory": _as_int(bundle.get("memoryCursor"), 0),
|
||||||
|
"summary": _as_int(bundle.get("summaryCursor"), 0),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _planned_branches(bundle: dict) -> list[dict]:
|
||||||
|
raw = bundle.get("branches")
|
||||||
|
entries = [b for b in raw if isinstance(b, dict)] if isinstance(raw, list) else []
|
||||||
|
if not entries:
|
||||||
|
return [dict(_ROOT)]
|
||||||
|
specs: list[dict] = []
|
||||||
|
for i, entry in enumerate(entries):
|
||||||
|
parent = entry.get("parent")
|
||||||
|
if parent is None:
|
||||||
|
specs.append(dict(_ROOT))
|
||||||
|
continue
|
||||||
|
# A branch may only fork from one listed before it. That is how the
|
||||||
|
# export writes them — branches are numbered in creation order and a
|
||||||
|
# parent always exists first — and requiring it here buys acyclicity for
|
||||||
|
# the price of a comparison: a lineage is computed by walking to the
|
||||||
|
# parent, and a cycle would be an import that never returns.
|
||||||
|
if not _is_int(parent) or not 0 <= parent < i:
|
||||||
|
raise HTTPException(
|
||||||
|
400,
|
||||||
|
f"Branch {i} forks from branch {parent!r}, which is not one of "
|
||||||
|
f"the {i} branches listed before it.",
|
||||||
|
)
|
||||||
|
fork_depth = entry.get("forkDepth")
|
||||||
|
if not _is_int(fork_depth):
|
||||||
|
raise HTTPException(
|
||||||
|
400,
|
||||||
|
f"Branch {i} forks from branch {parent} but does not say at "
|
||||||
|
f"what depth.",
|
||||||
|
)
|
||||||
|
specs.append({"parent": parent, "forkDepth": fork_depth})
|
||||||
|
return specs
|
||||||
|
|
||||||
|
|
||||||
|
def _planned_nodes(bundle: dict, branches: int) -> list[dict]:
|
||||||
|
raw = bundle.get("actions")
|
||||||
|
nodes: list[dict] = []
|
||||||
|
for entry in raw if isinstance(raw, list) else []:
|
||||||
|
if not isinstance(entry, dict) or not str(entry.get("text") or ""):
|
||||||
|
continue
|
||||||
|
branch = entry.get("branch", 0)
|
||||||
|
if not _is_int(branch) or not 0 <= branch < branches:
|
||||||
|
raise HTTPException(
|
||||||
|
400,
|
||||||
|
f"An action names branch {branch!r}, but the file lists "
|
||||||
|
f"{branches}.",
|
||||||
|
)
|
||||||
|
depth = entry.get("depth")
|
||||||
|
if not _is_int(depth) or depth < 0:
|
||||||
|
raise HTTPException(
|
||||||
|
400, f"An action on branch {branch} has no depth to sit at."
|
||||||
|
)
|
||||||
|
nodes.append({
|
||||||
|
"branch": branch,
|
||||||
|
"depth": depth,
|
||||||
|
"live": bool(entry.get("live", True)),
|
||||||
|
"type": str(entry.get("type") or "story")[:TYPE_MAX],
|
||||||
|
"text": str(entry.get("text") or ""),
|
||||||
|
"reasoning": _as_text(entry.get("reasoning")),
|
||||||
|
"stateAfter": _as_dict(entry.get("stateAfter")),
|
||||||
|
"worldStateAfter": _as_dict(entry.get("worldStateAfter")),
|
||||||
|
"worldDelta": _as_dict(entry.get("worldDelta")),
|
||||||
|
"createdAt": _as_time(entry.get("createdAt")),
|
||||||
|
})
|
||||||
|
return nodes
|
||||||
|
|
||||||
|
|
||||||
|
def _planned_v1_nodes(bundle: dict) -> list[dict]:
|
||||||
|
"""A v1 bundle's turns as the nodes they describe: one per attempt.
|
||||||
|
|
||||||
|
The `variants` array is the repeating group SP4 unpacked, so reading one is
|
||||||
|
the same split migration 60 does — every attempt becomes a row at the turn's
|
||||||
|
coordinate and `variantIndex` picks which is live. Clamped, because a
|
||||||
|
hand-edited bundle can name an attempt its own list does not have, and a
|
||||||
|
turn with no live node is a turn no read can see.
|
||||||
|
|
||||||
|
The depth is the bundle's `index`: v1 is one branch, where the two agree.
|
||||||
|
"""
|
||||||
|
raw = bundle.get("actions")
|
||||||
|
nodes: list[dict] = []
|
||||||
|
for i, entry in enumerate(raw if isinstance(raw, list) else []):
|
||||||
|
if not isinstance(entry, dict) or not str(entry.get("text") or ""):
|
||||||
|
continue
|
||||||
|
depth = _as_int(entry.get("index"), i)
|
||||||
|
kind = str(entry.get("type") or "story")[:TYPE_MAX]
|
||||||
|
variants = [v for v in (entry.get("variants") or []) if isinstance(v, dict)]
|
||||||
|
if not variants:
|
||||||
|
variants = [{"text": entry["text"], "reasoning": entry.get("reasoning")}]
|
||||||
|
live = min(max(_as_int(entry.get("variantIndex"), 0), 0), len(variants) - 1)
|
||||||
|
for n, variant in enumerate(variants):
|
||||||
|
text = str(variant.get("text") or "")
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
nodes.append({
|
||||||
|
"branch": 0,
|
||||||
|
"depth": max(depth, 0),
|
||||||
|
"live": n == live,
|
||||||
|
"type": kind,
|
||||||
|
"text": text,
|
||||||
|
"reasoning": _as_text(variant.get("reasoning")),
|
||||||
|
"stateAfter": None,
|
||||||
|
"worldStateAfter": None,
|
||||||
|
"worldDelta": None,
|
||||||
|
"createdAt": _as_time(variant.get("createdAt")),
|
||||||
|
})
|
||||||
|
return nodes
|
||||||
|
|
||||||
|
|
||||||
|
def _planned_memories(bundle: dict, branches: int) -> list[dict]:
|
||||||
|
raw = bundle.get("memories")
|
||||||
|
out: list[dict] = []
|
||||||
|
for entry in raw if isinstance(raw, list) else []:
|
||||||
|
if not isinstance(entry, dict) or not str(entry.get("text") or "").strip():
|
||||||
|
continue
|
||||||
|
out.append({
|
||||||
|
"text": str(entry["text"]),
|
||||||
|
"pinned": bool(entry.get("pinned", False)),
|
||||||
|
"forgotten": bool(entry.get("forgotten", False)),
|
||||||
|
"sourceStart": entry.get("sourceStart"),
|
||||||
|
"sourceEnd": entry.get("sourceEnd"),
|
||||||
|
"useCount": _as_int(entry.get("useCount"), 0),
|
||||||
|
# Out of range rather than absent means a file that disagrees with
|
||||||
|
# itself; the root is the safe reading, because a memory on a branch
|
||||||
|
# nothing can see is a memory that never reaches a prompt again.
|
||||||
|
"branch": _as_index(entry.get("branch"), branches, default=0),
|
||||||
|
"depth": entry.get("depth") if _is_int(entry.get("depth")) else None,
|
||||||
|
})
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _planned_anchors(bundle: dict, branches: int) -> dict:
|
||||||
|
anchors = {}
|
||||||
|
for name in ("memory", "summary"):
|
||||||
|
raw = bundle.get(f"{name}Cursor")
|
||||||
|
raw = raw if isinstance(raw, dict) else {}
|
||||||
|
branch = raw.get("branch")
|
||||||
|
anchors[name] = (
|
||||||
|
_as_index(branch, branches) if branch is not None else None,
|
||||||
|
_as_int(raw.get("depth"), lineage.NO_DEPTH),
|
||||||
|
)
|
||||||
|
return anchors
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ writing
|
||||||
|
|
||||||
|
def write(db: Session, adventure: models.Adventure, story: dict) -> None:
|
||||||
|
"""Write a planned tree onto a freshly created adventure.
|
||||||
|
|
||||||
|
Order matters and is not negotiable: branches first, because a node needs an
|
||||||
|
id to hang off; then the nodes, because the head and the anchors name one.
|
||||||
|
"""
|
||||||
|
ids = _write_branches(db, adventure, story["branches"])
|
||||||
|
_write_nodes(db, adventure, story["nodes"], ids)
|
||||||
|
_write_memories(db, adventure, story["memories"], ids)
|
||||||
|
_point_the_head(adventure, story, ids)
|
||||||
|
_write_anchors(adventure, story, ids)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_branches(
|
||||||
|
db: Session, adventure: models.Adventure, specs: list[dict]
|
||||||
|
) -> list[int]:
|
||||||
|
"""One row per branch, lineage computed rather than read.
|
||||||
|
|
||||||
|
Inserted through Core and the lineage written second, for the reason
|
||||||
|
`tree.root_branch` spells out: the lineage names the row's own id. The
|
||||||
|
parent's cached ancestry is capped at this fork, which is the same
|
||||||
|
arithmetic `tree.fork` does — a fork made now and a fork made a year ago and
|
||||||
|
exported must produce the same rows.
|
||||||
|
"""
|
||||||
|
ids: list[int] = []
|
||||||
|
lineages: list[list[list]] = []
|
||||||
|
for spec in specs:
|
||||||
|
parent = spec.get("parent")
|
||||||
|
fork_depth = spec.get("forkDepth")
|
||||||
|
new_id = db.execute(
|
||||||
|
insert(models.Branch).values(
|
||||||
|
adventure_id=adventure.id,
|
||||||
|
parent_branch_id=ids[parent] if parent is not None else None,
|
||||||
|
fork_depth=fork_depth if parent is not None else None,
|
||||||
|
lineage=[],
|
||||||
|
created_at=models.utcnow(),
|
||||||
|
)
|
||||||
|
).inserted_primary_key[0]
|
||||||
|
entries = [[new_id, None]]
|
||||||
|
if parent is not None:
|
||||||
|
entries += [
|
||||||
|
[branch_id, fork_depth if cap is None else min(cap, fork_depth)]
|
||||||
|
for branch_id, cap in lineages[parent]
|
||||||
|
]
|
||||||
|
db.execute(
|
||||||
|
update(models.Branch)
|
||||||
|
.where(models.Branch.id == new_id)
|
||||||
|
.values(lineage=entries)
|
||||||
|
)
|
||||||
|
ids.append(new_id)
|
||||||
|
lineages.append(entries)
|
||||||
|
return ids
|
||||||
|
|
||||||
|
|
||||||
|
def _write_nodes(
|
||||||
|
db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
|
||||||
|
) -> None:
|
||||||
|
"""The nodes, grouped into the turns they are attempts at.
|
||||||
|
|
||||||
|
Two things are allocated here rather than trusted from the file. `index` is
|
||||||
|
handed out one per *turn*, in the order the bundle lists them, so siblings
|
||||||
|
share one and no two coordinates do — which is what `max_action_index` needs
|
||||||
|
to keep issuing numbers nothing holds. And exactly one attempt in each group
|
||||||
|
is made live: a file can name none or several, and a turn with no live node
|
||||||
|
is a turn that vanishes from the story.
|
||||||
|
"""
|
||||||
|
groups: dict[tuple[int, int], list[models.Action]] = {}
|
||||||
|
indices: dict[tuple[int, int], int] = {}
|
||||||
|
for spec in specs:
|
||||||
|
key = (spec["branch"], spec["depth"])
|
||||||
|
if key not in indices:
|
||||||
|
indices[key] = len(indices)
|
||||||
|
action = models.Action(
|
||||||
|
adventure_id=adventure.id,
|
||||||
|
branch_id=ids[spec["branch"]],
|
||||||
|
depth=spec["depth"],
|
||||||
|
index=indices[key],
|
||||||
|
type=spec["type"],
|
||||||
|
text=spec["text"],
|
||||||
|
reasoning=spec["reasoning"],
|
||||||
|
live=spec["live"],
|
||||||
|
state_after=spec["stateAfter"],
|
||||||
|
world_state_after=spec["worldStateAfter"],
|
||||||
|
world_delta=spec["worldDelta"],
|
||||||
|
)
|
||||||
|
if spec["createdAt"] is not None:
|
||||||
|
action.created_at = spec["createdAt"]
|
||||||
|
db.add(action)
|
||||||
|
groups.setdefault(key, []).append(action)
|
||||||
|
|
||||||
|
for rows in groups.values():
|
||||||
|
live = next((row for row in rows if row.live), rows[0])
|
||||||
|
for row in rows:
|
||||||
|
row.live = row is live
|
||||||
|
attempts.renumber(rows)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_memories(
|
||||||
|
db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
|
||||||
|
) -> None:
|
||||||
|
for spec in specs:
|
||||||
|
memory = models.Memory(
|
||||||
|
adventure_id=adventure.id,
|
||||||
|
text=spec["text"],
|
||||||
|
pinned=spec["pinned"],
|
||||||
|
forgotten=spec["forgotten"],
|
||||||
|
source_start=spec["sourceStart"],
|
||||||
|
source_end=spec["sourceEnd"],
|
||||||
|
use_count=spec["useCount"],
|
||||||
|
branch_id=ids[spec["branch"]],
|
||||||
|
depth=spec["depth"],
|
||||||
|
)
|
||||||
|
# A v1 memory has no depth of its own; `source_end` is the index of the
|
||||||
|
# last action it summarises, which on one branch is that node's depth.
|
||||||
|
if memory.depth is None and memory.source_end is not None:
|
||||||
|
memory.depth = memory.source_end
|
||||||
|
db.add(memory)
|
||||||
|
|
||||||
|
|
||||||
|
def _point_the_head(
|
||||||
|
adventure: models.Adventure, story: dict, ids: list[int]
|
||||||
|
) -> None:
|
||||||
|
"""Where the story is being played, and how deep it goes.
|
||||||
|
|
||||||
|
The branch comes from the file and the depth does not: the tip of a branch
|
||||||
|
is whatever arrived on it, and a branch with nothing of its own sits at its
|
||||||
|
fork point — the last node its story contains, borrowed but the tip all the
|
||||||
|
same. The same rule as `tree.refresh_head`, applied before a flush.
|
||||||
|
"""
|
||||||
|
head = story["head"]
|
||||||
|
adventure.head_branch_id = ids[head]
|
||||||
|
depths = [n["depth"] for n in story["nodes"] if n["branch"] == head]
|
||||||
|
if depths:
|
||||||
|
adventure.head_depth = max(depths)
|
||||||
|
return
|
||||||
|
fork_depth = story["branches"][head].get("forkDepth")
|
||||||
|
adventure.head_depth = fork_depth if fork_depth is not None else lineage.NO_DEPTH
|
||||||
|
|
||||||
|
|
||||||
|
def _write_anchors(
|
||||||
|
adventure: models.Adventure, story: dict, ids: list[int]
|
||||||
|
) -> None:
|
||||||
|
"""How far the memories and the summary have read — v2 only.
|
||||||
|
|
||||||
|
A v1 bundle counts instead, and a count cannot be resolved to a node until
|
||||||
|
the nodes are in the database; `settle` does that half afterwards.
|
||||||
|
"""
|
||||||
|
anchors = story["anchors"]
|
||||||
|
if anchors is None:
|
||||||
|
return
|
||||||
|
for cursor in cursors.ALL:
|
||||||
|
branch, depth = anchors[cursor.name]
|
||||||
|
cursor.anchor(adventure, ids[branch] if branch is not None else None, depth)
|
||||||
|
|
||||||
|
|
||||||
|
def settle(db: Session, adventure: models.Adventure, story: dict) -> None:
|
||||||
|
"""Line up the two coordinate systems, once the nodes exist.
|
||||||
|
|
||||||
|
The anchors and the legacy counts describe the same boundary in different
|
||||||
|
words, and each version of the bundle brings one of them. A v1 file brings
|
||||||
|
the count, so the anchor is found by counting that far along the story; a v2
|
||||||
|
file brings the anchor, so the count is read back off it. The legacy columns
|
||||||
|
are still written either way — they are what a rolled-back build reads.
|
||||||
|
|
||||||
|
Called after the flush, because both directions need the actions queryable.
|
||||||
|
"""
|
||||||
|
positions = story["positions"]
|
||||||
|
for cursor in cursors.ALL:
|
||||||
|
if positions is not None:
|
||||||
|
setattr(adventure, f"{cursor.name}_cursor", positions[cursor.name])
|
||||||
|
cursors.anchor_at_position(adventure, cursor, positions[cursor.name])
|
||||||
|
else:
|
||||||
|
setattr(
|
||||||
|
adventure, f"{cursor.name}_cursor",
|
||||||
|
cursors.position_of(adventure, cursor.depth(db, adventure)),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ reading
|
||||||
|
# Small coercions. A raw-dict import bypasses the schemas entirely, so
|
||||||
|
# everything out of a bundle is whatever JSON happened to hold.
|
||||||
|
|
||||||
|
def _is_int(value) -> bool:
|
||||||
|
"""`True` is an `int` in Python, and is not one in a coordinate."""
|
||||||
|
return isinstance(value, int) and not isinstance(value, bool)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_int(value, default: int) -> int:
|
||||||
|
return value if _is_int(value) else default
|
||||||
|
|
||||||
|
|
||||||
|
def _as_index(value, count: int, default: int | None = None) -> int | None:
|
||||||
|
"""A local branch number, or `default` when the file names one that is not
|
||||||
|
there. Out of range is a file disagreeing with itself, not a shape a read
|
||||||
|
can be handed."""
|
||||||
|
return value if _is_int(value) and 0 <= value < count else default
|
||||||
|
|
||||||
|
|
||||||
|
def _as_text(value) -> str | None:
|
||||||
|
return str(value) if value else None
|
||||||
|
|
||||||
|
|
||||||
|
def _as_dict(value) -> dict | None:
|
||||||
|
return value if isinstance(value, dict) else None
|
||||||
|
|
||||||
|
|
||||||
|
def _as_time(value) -> datetime | None:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
@@ -66,6 +66,18 @@ class Cursor:
|
|||||||
|
|
||||||
# ------------------------------------------------------------- writing
|
# ------------------------------------------------------------- writing
|
||||||
|
|
||||||
|
def anchor(
|
||||||
|
self, adventure: models.Adventure, branch_id: int | None, depth: int
|
||||||
|
) -> None:
|
||||||
|
"""Put the anchor at a coordinate given outright.
|
||||||
|
|
||||||
|
The plain setter under `anchor_at`. Only an import has a coordinate
|
||||||
|
without a node to read it off — a v2 bundle carries the anchor itself
|
||||||
|
(`app/bundle.py`), and the node it named lives in another database.
|
||||||
|
"""
|
||||||
|
setattr(adventure, self.branch_field, branch_id)
|
||||||
|
setattr(adventure, self.depth_field, max(depth, NO_DEPTH))
|
||||||
|
|
||||||
def anchor_at(self, adventure: models.Adventure, node: models.Action) -> None:
|
def anchor_at(self, adventure: models.Adventure, node: models.Action) -> None:
|
||||||
"""Mark the work done up to and including `node`.
|
"""Mark the work done up to and including `node`.
|
||||||
|
|
||||||
@@ -73,9 +85,8 @@ class Cursor:
|
|||||||
actions can end before the fork this branch was made at, and the
|
actions can end before the fork this branch was made at, and the
|
||||||
coverage belongs where the ground is.
|
coverage belongs where the ground is.
|
||||||
"""
|
"""
|
||||||
setattr(adventure, self.branch_field, node.branch_id)
|
self.anchor(adventure, node.branch_id, lineage.NO_DEPTH
|
||||||
setattr(adventure, self.depth_field, lineage.NO_DEPTH
|
if node.depth is None else node.depth)
|
||||||
if node.depth is None else node.depth)
|
|
||||||
|
|
||||||
def rewind_to(
|
def rewind_to(
|
||||||
self, adventure: models.Adventure, branch_id: int | None, depth: int
|
self, adventure: models.Adventure, branch_id: int | None, depth: int
|
||||||
|
|||||||
@@ -158,6 +158,11 @@ MAX_SCRIPTS_PER_USER = 200
|
|||||||
MAX_STORY_CARDS_PER_OWNER = 200 # per scenario or adventure
|
MAX_STORY_CARDS_PER_OWNER = 200 # per scenario or adventure
|
||||||
MAX_MEMORIES_PER_ADVENTURE = 1000
|
MAX_MEMORIES_PER_ADVENTURE = 1000
|
||||||
MAX_ACTIONS_PER_ADVENTURE = 5000
|
MAX_ACTIONS_PER_ADVENTURE = 5000
|
||||||
|
# Phase 14, SP6. A branch per divergence somebody built a story on, so a tree
|
||||||
|
# with more of them than a story has turns is a file, not a game. Import-only
|
||||||
|
# for now: forking is a POST that adds one row and has no cap of its own, and
|
||||||
|
# the cap that matters there is `MAX_ACTIONS_PER_ADVENTURE` above it.
|
||||||
|
MAX_BRANCHES_PER_ADVENTURE = 1000
|
||||||
|
|
||||||
|
|
||||||
def check_row_cap(
|
def check_row_cap(
|
||||||
@@ -232,12 +237,13 @@ _BUNDLE_LIST_CAPS = {
|
|||||||
"story_cards": MAX_STORY_CARDS_PER_OWNER,
|
"story_cards": MAX_STORY_CARDS_PER_OWNER,
|
||||||
"memories": MAX_MEMORIES_PER_ADVENTURE,
|
"memories": MAX_MEMORIES_PER_ADVENTURE,
|
||||||
"actions": MAX_ACTIONS_PER_ADVENTURE,
|
"actions": MAX_ACTIONS_PER_ADVENTURE,
|
||||||
|
"branches": MAX_BRANCHES_PER_ADVENTURE,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def check_bundle_lists(**lists) -> None:
|
def check_bundle_lists(**lists) -> None:
|
||||||
"""409 when an import bundle's lists exceed the same caps live creation
|
"""409 when an import bundle's lists exceed the same caps live creation
|
||||||
enforces (kwargs: story_cards=, memories=, actions=)."""
|
enforces (kwargs: story_cards=, memories=, actions=, branches=)."""
|
||||||
if not auth.MULTI_USER:
|
if not auth.MULTI_USER:
|
||||||
return
|
return
|
||||||
for name, value in lists.items():
|
for name, value in lists.items():
|
||||||
|
|||||||
@@ -9,7 +9,8 @@ from sqlalchemy.orm import Session, load_only, undefer
|
|||||||
from sqlalchemy.orm.attributes import set_committed_value
|
from sqlalchemy.orm.attributes import set_committed_value
|
||||||
|
|
||||||
from .. import (
|
from .. import (
|
||||||
attempts, auth, images, limits, memorybank, models, schemas, tree, worldstate,
|
attempts, auth, bundle, images, limits, memorybank, models, schemas, tree,
|
||||||
|
worldstate,
|
||||||
)
|
)
|
||||||
from ..context import build_context, cursors
|
from ..context import build_context, cursors
|
||||||
from ..context import history as context_history
|
from ..context import history as context_history
|
||||||
@@ -1303,185 +1304,55 @@ def undo_turn(
|
|||||||
def export_adventure(
|
def export_adventure(
|
||||||
adventure_id: int, db: Session = Depends(get_db), user: models.User = CurrentUser
|
adventure_id: int, db: Session = Depends(get_db), user: models.User = CurrentUser
|
||||||
):
|
):
|
||||||
"""Full backup: plot components, story cards, scripts (+state), every action."""
|
"""Full backup: plot components, story cards, scripts (+state), the tree.
|
||||||
|
|
||||||
|
The format is `app/bundle.py`'s alone — both versions of it. A backup is the
|
||||||
|
one thing here that outlives the schema, so nothing about its shape is
|
||||||
|
decided at a call site.
|
||||||
|
"""
|
||||||
adv = get_adventure_or_404(adventure_id, db, user)
|
adv = get_adventure_or_404(adventure_id, db, user)
|
||||||
# The one read deliberately left un-pathed: a backup wants the whole
|
return bundle.export(db, adv)
|
||||||
# adventure, not the branch its owner happens to be standing on. `index`
|
|
||||||
# orders it because the v1 bundle is a flat list keyed on index and its
|
|
||||||
# reader has no idea branches exist — which is exactly why SP6 replaces the
|
|
||||||
# format rather than quietly widening this query.
|
|
||||||
#
|
|
||||||
# Attempts at one turn share that index (SP4), so the flat list is built by
|
|
||||||
# folding each group back into the `variants` array the format expects.
|
|
||||||
# That array is the *only* remaining producer of the v1 shape: nothing in
|
|
||||||
# the database holds one any more.
|
|
||||||
#
|
|
||||||
# A *forked* adventure has no honest v1 rendering — the format has one
|
|
||||||
# story and there are two — so this emits every branch's turns interleaved
|
|
||||||
# by index, which reads as a mangled story rather than as lost data. SP6's
|
|
||||||
# v2 bundle is what fixes it, and SP7 is where a player first gets a way to
|
|
||||||
# fork at all, so the order those two ship in is the order that matters.
|
|
||||||
exported_actions = (
|
|
||||||
db.query(models.Action)
|
|
||||||
.filter(models.Action.adventure_id == adv.id)
|
|
||||||
.order_by(models.Action.index, models.Action.variant_index, models.Action.id)
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
turns: list[list[models.Action]] = []
|
|
||||||
for action in exported_actions:
|
|
||||||
if turns and turns[-1][0].index == action.index:
|
|
||||||
turns[-1].append(action)
|
|
||||||
else:
|
|
||||||
turns.append([action])
|
|
||||||
return {
|
|
||||||
"format": "ai-dnd-adventure-v1",
|
|
||||||
"title": adv.title,
|
|
||||||
"memory": adv.memory,
|
|
||||||
"authorsNote": adv.authors_note,
|
|
||||||
"aiInstructions": adv.ai_instructions,
|
|
||||||
"storySummary": adv.story_summary,
|
|
||||||
"scriptState": adv.script_state,
|
|
||||||
"worldState": adv.world_state,
|
|
||||||
"autoSummarize": adv.auto_summarize,
|
|
||||||
"memoryBankEnabled": adv.memory_bank_enabled,
|
|
||||||
# The bundle's coordinate system is a position in the story, and the
|
|
||||||
# cursors are nodes now, so they are counted back into one. A v1 bundle
|
|
||||||
# has to stay readable by builds that never heard of a depth — SP6's v2
|
|
||||||
# format carries the anchors themselves.
|
|
||||||
"memoryCursor": cursors.position_of(adv, cursors.MEMORY.depth(db, adv)),
|
|
||||||
"summaryCursor": cursors.position_of(adv, cursors.SUMMARY.depth(db, adv)),
|
|
||||||
"memories": [
|
|
||||||
{
|
|
||||||
"text": m.text, "pinned": m.pinned, "forgotten": m.forgotten,
|
|
||||||
"sourceStart": m.source_start, "sourceEnd": m.source_end,
|
|
||||||
"useCount": m.use_count,
|
|
||||||
}
|
|
||||||
for m in adv.memories
|
|
||||||
],
|
|
||||||
"storyCards": [
|
|
||||||
{"type": c.type, "name": c.name, "keys": c.keys, "entry": c.entry, "notes": c.notes}
|
|
||||||
for c in adv.story_cards
|
|
||||||
],
|
|
||||||
"scripts": [
|
|
||||||
{
|
|
||||||
"position": s.position, "enabled": s.enabled,
|
|
||||||
"name": s.name, "description": s.description,
|
|
||||||
"library": s.library_js, "input": s.input_js,
|
|
||||||
"context": s.context_js, "output": s.output_js,
|
|
||||||
}
|
|
||||||
for s in adv.scripts
|
|
||||||
],
|
|
||||||
"actions": [_exported_turn(group) for group in turns],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _exported_turn(group: list[models.Action]) -> dict:
|
|
||||||
"""One turn as a v1 bundle entry: the attempt in the story, plus the rest.
|
|
||||||
|
|
||||||
Narration only — a bundle carries no context snapshots, so the per-attempt
|
|
||||||
script/world state a switch would restore isn't there to export either.
|
|
||||||
"""
|
|
||||||
live = next((a for a in group if a.live), group[0])
|
|
||||||
return {
|
|
||||||
"index": live.index, "type": live.type, "text": live.text,
|
|
||||||
"reasoning": live.reasoning,
|
|
||||||
"variants": [
|
|
||||||
{"text": a.text, "reasoning": a.reasoning,
|
|
||||||
"createdAt": a.created_at.isoformat() if a.created_at else None}
|
|
||||||
for a in group
|
|
||||||
] if len(group) > 1 else None,
|
|
||||||
"variantIndex": group.index(live),
|
|
||||||
"createdAt": live.created_at.isoformat(),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _imported_turn(
|
|
||||||
adventure: models.Adventure, entry: dict, index: int
|
|
||||||
) -> list[models.Action]:
|
|
||||||
"""A v1 bundle entry as the nodes it describes: one per attempt.
|
|
||||||
|
|
||||||
A bundle's `variants` array is the repeating group SP4 unpacked, so
|
|
||||||
importing one is the same split the migration does — every attempt gets a
|
|
||||||
row at the turn's coordinate, and `variantIndex` picks which is live.
|
|
||||||
Clamped, because a hand-edited bundle can name an attempt its own list
|
|
||||||
doesn't have, and a turn with no live node is a turn no read can see.
|
|
||||||
"""
|
|
||||||
kind = str(entry.get("type") or "story")[:20] # VARCHAR(20)
|
|
||||||
variants = [v for v in (entry.get("variants") or []) if isinstance(v, dict)]
|
|
||||||
if not variants:
|
|
||||||
variants = [{"text": entry["text"], "reasoning": entry.get("reasoning")}]
|
|
||||||
live = min(max(int(entry.get("variantIndex", 0)), 0), len(variants) - 1)
|
|
||||||
rows = []
|
|
||||||
for i, variant in enumerate(variants):
|
|
||||||
text = str(variant.get("text") or "")
|
|
||||||
reasoning = variant.get("reasoning")
|
|
||||||
rows.append(models.Action(
|
|
||||||
adventure_id=adventure.id,
|
|
||||||
index=index,
|
|
||||||
type=kind,
|
|
||||||
text=text,
|
|
||||||
reasoning=str(reasoning) if reasoning else None,
|
|
||||||
live=(i == live),
|
|
||||||
variant_index=i,
|
|
||||||
variant_count=len(variants) if len(variants) > 1 else 0,
|
|
||||||
))
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/import", response_model=schemas.AdventureOut, status_code=201)
|
@router.post("/import", response_model=schemas.AdventureOut, status_code=201)
|
||||||
def import_adventure(
|
def import_adventure(
|
||||||
request: Request,
|
request: Request,
|
||||||
bundle: dict = Body(...),
|
payload: dict = Body(...),
|
||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
user: models.User = CurrentUser,
|
user: models.User = CurrentUser,
|
||||||
):
|
):
|
||||||
if bundle.get("format") != "ai-dnd-adventure-v1":
|
version = bundle.check_format(payload)
|
||||||
raise HTTPException(400, "Not an adventure export file (expected format ai-dnd-adventure-v1).")
|
|
||||||
limits.rate_limit("import", request, user)
|
limits.rate_limit("import", request, user)
|
||||||
limits.check_row_cap("adventures", db, user)
|
limits.check_row_cap("adventures", db, user)
|
||||||
limits.check_bundle_lists(
|
limits.check_bundle_lists(
|
||||||
story_cards=bundle.get("storyCards"),
|
story_cards=payload.get("storyCards"),
|
||||||
memories=bundle.get("memories"),
|
memories=payload.get("memories"),
|
||||||
actions=bundle.get("actions"),
|
actions=payload.get("actions"),
|
||||||
|
branches=payload.get("branches"),
|
||||||
)
|
)
|
||||||
|
# The tree is checked before the adventure row exists, so a file that
|
||||||
|
# disagrees with itself is a 400 and not a half-imported adventure holding a
|
||||||
|
# story with a hole in it.
|
||||||
|
story = bundle.plan(payload, version)
|
||||||
|
|
||||||
# Raw-dict import bypasses the schemas — clamp strings headed for VARCHAR
|
# Raw-dict import bypasses the schemas — clamp strings headed for VARCHAR
|
||||||
# columns (Postgres enforces the widths; see schemas.py).
|
# columns (Postgres enforces the widths; see schemas.py).
|
||||||
adventure = models.Adventure(
|
adventure = models.Adventure(
|
||||||
user_id=user.id,
|
user_id=user.id,
|
||||||
title=str(bundle.get("title") or "Imported Adventure")[:schemas.NAME_MAX],
|
title=str(payload.get("title") or "Imported Adventure")[:schemas.NAME_MAX],
|
||||||
memory=str(bundle.get("memory") or ""),
|
memory=str(payload.get("memory") or ""),
|
||||||
authors_note=str(bundle.get("authorsNote") or ""),
|
authors_note=str(payload.get("authorsNote") or ""),
|
||||||
ai_instructions=str(bundle.get("aiInstructions") or ""),
|
ai_instructions=str(payload.get("aiInstructions") or ""),
|
||||||
story_summary=str(bundle.get("storySummary") or ""),
|
story_summary=str(payload.get("storySummary") or ""),
|
||||||
script_state=bundle.get("scriptState") or {},
|
script_state=payload.get("scriptState") or {},
|
||||||
world_state=bundle.get("worldState") or {},
|
world_state=payload.get("worldState") or {},
|
||||||
auto_summarize=bool(bundle.get("autoSummarize", False)),
|
auto_summarize=bool(payload.get("autoSummarize", False)),
|
||||||
memory_bank_enabled=bool(bundle.get("memoryBankEnabled", False)),
|
memory_bank_enabled=bool(payload.get("memoryBankEnabled", False)),
|
||||||
memory_cursor=int(bundle.get("memoryCursor", 0)),
|
|
||||||
summary_cursor=int(bundle.get("summaryCursor", 0)),
|
|
||||||
)
|
)
|
||||||
db.add(adventure)
|
db.add(adventure)
|
||||||
db.flush()
|
db.flush()
|
||||||
# A v1 bundle is a linear story, which is a tree with one branch. SP6's v2
|
|
||||||
# format carries the branches themselves.
|
|
||||||
tree.head_branch(db, adventure)
|
|
||||||
|
|
||||||
for m in bundle.get("memories") or []:
|
for card in payload.get("storyCards") or []:
|
||||||
if isinstance(m, dict) and str(m.get("text") or "").strip():
|
|
||||||
memory = models.Memory(
|
|
||||||
adventure_id=adventure.id,
|
|
||||||
text=str(m["text"]),
|
|
||||||
pinned=bool(m.get("pinned", False)),
|
|
||||||
forgotten=bool(m.get("forgotten", False)),
|
|
||||||
source_start=m.get("sourceStart"),
|
|
||||||
source_end=m.get("sourceEnd"),
|
|
||||||
use_count=int(m.get("useCount", 0)),
|
|
||||||
)
|
|
||||||
tree.place_memory(db, adventure, memory)
|
|
||||||
db.add(memory)
|
|
||||||
|
|
||||||
for card in bundle.get("storyCards") or []:
|
|
||||||
if isinstance(card, dict):
|
if isinstance(card, dict):
|
||||||
db.add(models.StoryCard(
|
db.add(models.StoryCard(
|
||||||
adventure_id=adventure.id,
|
adventure_id=adventure.id,
|
||||||
@@ -1492,7 +1363,7 @@ def import_adventure(
|
|||||||
notes=str(card.get("notes") or ""),
|
notes=str(card.get("notes") or ""),
|
||||||
))
|
))
|
||||||
|
|
||||||
for i, s in enumerate(bundle.get("scripts") or []):
|
for i, s in enumerate(payload.get("scripts") or []):
|
||||||
if isinstance(s, dict):
|
if isinstance(s, dict):
|
||||||
db.add(models.AdventureScript(
|
db.add(models.AdventureScript(
|
||||||
adventure_id=adventure.id,
|
adventure_id=adventure.id,
|
||||||
@@ -1506,21 +1377,14 @@ def import_adventure(
|
|||||||
output_js=str(s.get("output") or ""),
|
output_js=str(s.get("output") or ""),
|
||||||
))
|
))
|
||||||
|
|
||||||
for i, a in enumerate(bundle.get("actions") or []):
|
bundle.write(db, adventure, story)
|
||||||
if isinstance(a, dict) and str(a.get("text") or ""):
|
|
||||||
for action in _imported_turn(adventure, a, int(a.get("index", i))):
|
|
||||||
tree.place_action(db, adventure, action)
|
|
||||||
db.add(action)
|
|
||||||
|
|
||||||
# The bundle's cursors are positions in a flat story and the marks are
|
# The anchors and the legacy counts are the same boundary in two coordinate
|
||||||
# nodes, so the translation waits until the actions exist — this is the
|
# systems, and lining them up needs the actions queryable — this is the only
|
||||||
# only moment the two coordinate systems can be lined up against each
|
# moment both exist.
|
||||||
# other. The legacy columns keep the numbers the bundle gave: they are what
|
|
||||||
# a rolled-back build would read.
|
|
||||||
db.flush()
|
db.flush()
|
||||||
db.expire(adventure, ["actions"])
|
db.expire(adventure, ["actions"])
|
||||||
cursors.anchor_at_position(adventure, cursors.MEMORY, adventure.memory_cursor)
|
bundle.settle(db, adventure, story)
|
||||||
cursors.anchor_at_position(adventure, cursors.SUMMARY, adventure.summary_cursor)
|
|
||||||
|
|
||||||
db.commit()
|
db.commit()
|
||||||
db.refresh(adventure)
|
db.refresh(adventure)
|
||||||
|
|||||||
@@ -380,18 +380,25 @@ def test_attempts_module_agrees_with_the_endpoint(client):
|
|||||||
db.close()
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
def test_export_folds_the_group_back_into_one_v1_entry(client):
|
def test_export_carries_every_attempt_as_its_own_node(client):
|
||||||
|
"""SP6 changed the answer here, and the reason is the whole of that subphase.
|
||||||
|
|
||||||
|
A v1 bundle had one entry per turn and folded the group back into a
|
||||||
|
`variants` array, because the format had nowhere else to put a second take.
|
||||||
|
A v2 bundle has coordinates, so an attempt is a node in the file exactly as
|
||||||
|
it is a node in the database, and `live` says which one is the story.
|
||||||
|
"""
|
||||||
ScriptedProvider.replies = ["One.", "Two."]
|
ScriptedProvider.replies = ["One.", "Two."]
|
||||||
_play(client)
|
_play(client)
|
||||||
_retry(client)
|
_retry(client)
|
||||||
|
|
||||||
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
||||||
ai = [a for a in bundle["actions"] if a["type"] == "ai"]
|
ai = [a for a in bundle["actions"] if a["type"] == "ai"]
|
||||||
assert len(ai) == 1, "a v1 bundle carries one entry per turn, not per attempt"
|
assert [(a["text"], a["live"]) for a in ai] == [("One.", False), ("Two.", True)]
|
||||||
assert [v["text"] for v in ai[0]["variants"]] == ["One.", "Two."]
|
assert len({(a["branch"], a["depth"]) for a in ai}) == 1, "one turn, two takes"
|
||||||
assert ai[0]["variantIndex"] == 1
|
assert "variants" not in ai[0], "nothing writes the repeating group any more"
|
||||||
|
|
||||||
# ...and importing it splits it back out into the rows it describes.
|
# ...and importing it puts the group back exactly as it stood.
|
||||||
imported = client.post("/api/adventures/import", json=bundle).json()["id"]
|
imported = client.post("/api/adventures/import", json=bundle).json()["id"]
|
||||||
rows = _rows(imported)
|
rows = _rows(imported)
|
||||||
ai_rows = [a for a in rows if a.type == "ai"]
|
ai_rows = [a for a in rows if a.type == "ai"]
|
||||||
|
|||||||
@@ -0,0 +1,539 @@
|
|||||||
|
"""Phase 14 SP6 — the export bundle carries the tree.
|
||||||
|
|
||||||
|
A bundle is the only part of this phase a migration can never reach: the file
|
||||||
|
is already on somebody's disk. So there are two formats, and the two halves of
|
||||||
|
this file watch different things.
|
||||||
|
|
||||||
|
**v2 has to be lossless for a story that forked**, which v1 could not be — it
|
||||||
|
had one list and there were two stories, so it interleaved them by `index` and
|
||||||
|
read as a mangled story. Losslessness here means the *tree*: every branch, the
|
||||||
|
fork point it left its parent at, which attempt at each turn is the story, and
|
||||||
|
what each node left behind — because that last one is what a branch switch puts
|
||||||
|
back, and a tree nobody can switch inside is not the tree that was exported.
|
||||||
|
|
||||||
|
**v1 has to still import**, because a backup that stops importing is not a
|
||||||
|
backup.
|
||||||
|
|
||||||
|
Underneath both is the rule the module is built on: a bundle carries what was
|
||||||
|
*chosen* and never what is *derived*. The lineage, the head depth, the legacy
|
||||||
|
`index` and the variant ordinals are all rebuilt on the way in, so a
|
||||||
|
hand-edited file cannot disagree with itself — and the tests that matter most
|
||||||
|
here are the ones that hand it a file which does.
|
||||||
|
|
||||||
|
python -m pytest tests/test_bundle_v2.py -v
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||||
|
_tmp.close()
|
||||||
|
os.environ["AIDND_DB_PATH"] = _tmp.name
|
||||||
|
os.environ.pop("AIDND_DATABASE_URL", None)
|
||||||
|
os.environ.pop("DATABASE_URL", None)
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import Depends
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from app import auth, bundle, limits, models
|
||||||
|
from app.context import lineage
|
||||||
|
from app.database import Base, SessionLocal, engine, get_db
|
||||||
|
from app.main import app
|
||||||
|
from app.providers import PromptParts
|
||||||
|
from app.routers import adventures
|
||||||
|
|
||||||
|
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||||
|
|
||||||
|
# Ten gold a turn, so the script scoreboard is a number that says how many turns
|
||||||
|
# the story behind it has — which makes an after-snapshot visible from outside.
|
||||||
|
GOLD_SCRIPT = """
|
||||||
|
const modifier = (text) => {
|
||||||
|
state.gold = (state.gold || 0) + 10;
|
||||||
|
return { text };
|
||||||
|
};
|
||||||
|
modifier(text);
|
||||||
|
"""
|
||||||
|
|
||||||
|
OPENING = "You enter a cave."
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedProvider:
|
||||||
|
replies: list = []
|
||||||
|
calls = 0
|
||||||
|
|
||||||
|
def __init__(self, *a, **k):
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def generate(self, parts: PromptParts, *, temperature, max_tokens):
|
||||||
|
index = min(ScriptedProvider.calls, len(ScriptedProvider.replies) - 1)
|
||||||
|
ScriptedProvider.calls += 1
|
||||||
|
yield ("text", ScriptedProvider.replies[index])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(monkeypatch):
|
||||||
|
Base.metadata.create_all(bind=engine)
|
||||||
|
setup = SessionLocal()
|
||||||
|
user = models.User(is_guest=False, email="bundle@example.com")
|
||||||
|
setup.add(user)
|
||||||
|
setup.flush()
|
||||||
|
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||||
|
scenario = models.Scenario(user_id=user.id, title="S", stat_schema=SCHEMA)
|
||||||
|
setup.add(scenario)
|
||||||
|
setup.flush()
|
||||||
|
adv = models.Adventure(
|
||||||
|
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||||
|
script_state={}, world_state={"player": {"hp": 100}},
|
||||||
|
)
|
||||||
|
setup.add(adv)
|
||||||
|
setup.flush()
|
||||||
|
setup.add(models.Action(adventure_id=adv.id, index=0, type="start", text=OPENING))
|
||||||
|
setup.add(models.AdventureScript(
|
||||||
|
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||||
|
))
|
||||||
|
setup.commit()
|
||||||
|
adv_id, user_id = adv.id, user.id
|
||||||
|
setup.close()
|
||||||
|
|
||||||
|
ScriptedProvider.replies = ["A reply."]
|
||||||
|
ScriptedProvider.calls = 0
|
||||||
|
monkeypatch.setattr(adventures, "OpenAICompatibleProvider", ScriptedProvider)
|
||||||
|
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||||
|
"http://fake", "k", "test-model", False))
|
||||||
|
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||||
|
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||||
|
|
||||||
|
def _current_user(db=Depends(get_db)):
|
||||||
|
return db.get(models.User, user_id)
|
||||||
|
|
||||||
|
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||||
|
c = TestClient(app)
|
||||||
|
c.adv_id = adv_id
|
||||||
|
try:
|
||||||
|
yield c
|
||||||
|
finally:
|
||||||
|
app.dependency_overrides.clear()
|
||||||
|
adventures._active_turns.clear()
|
||||||
|
Base.metadata.drop_all(bind=engine)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ helpers
|
||||||
|
|
||||||
|
def _play(client, adv_id, text="look around", type="do"):
|
||||||
|
r = client.post(f"/api/adventures/{adv_id}/actions", json={"type": type, "text": text})
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
|
||||||
|
|
||||||
|
def _retry(client, adv_id):
|
||||||
|
assert client.post(f"/api/adventures/{adv_id}/retry").status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def _export(client, adv_id) -> dict:
|
||||||
|
r = client.get(f"/api/adventures/{adv_id}/export")
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
return r.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _import(client, payload):
|
||||||
|
return client.post("/api/adventures/import", json=payload)
|
||||||
|
|
||||||
|
|
||||||
|
def _imported(client, payload) -> int:
|
||||||
|
r = _import(client, payload)
|
||||||
|
assert r.status_code == 201, r.text
|
||||||
|
return r.json()["id"]
|
||||||
|
|
||||||
|
|
||||||
|
def _branches(client, adv_id) -> list[dict]:
|
||||||
|
r = client.get(f"/api/adventures/{adv_id}/branches")
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
return r.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _switch(client, adv_id, branch_id):
|
||||||
|
r = client.post(f"/api/adventures/{adv_id}/branches/{branch_id}/switch")
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
|
||||||
|
|
||||||
|
def _texts(client, adv_id) -> list[str]:
|
||||||
|
return [a["text"] for a in client.get(f"/api/adventures/{adv_id}").json()["actions"]]
|
||||||
|
|
||||||
|
|
||||||
|
def _every_branch_story(client, adv_id) -> list[list[str]]:
|
||||||
|
"""What each branch tells, in branch order — the whole tree as text."""
|
||||||
|
stories = []
|
||||||
|
for branch in _branches(client, adv_id):
|
||||||
|
_switch(client, adv_id, branch["id"])
|
||||||
|
stories.append(_texts(client, adv_id))
|
||||||
|
return stories
|
||||||
|
|
||||||
|
|
||||||
|
def _adventure_count() -> int:
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
return db.query(models.Adventure).count()
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _rows(adv_id) -> list[models.Action]:
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
return (
|
||||||
|
db.query(models.Action)
|
||||||
|
.filter(models.Action.adventure_id == adv_id)
|
||||||
|
.order_by(models.Action.branch_id, models.Action.depth,
|
||||||
|
models.Action.variant_index)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _branch_rows(adv_id) -> list[models.Branch]:
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
return (
|
||||||
|
db.query(models.Branch)
|
||||||
|
.filter(models.Branch.adventure_id == adv_id)
|
||||||
|
.order_by(models.Branch.id)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _script_state(adv_id) -> dict:
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
return db.get(models.Adventure, adv_id).script_state
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _forked_story(client) -> int:
|
||||||
|
"""A story that went two ways, and stayed both.
|
||||||
|
|
||||||
|
root: start · do · [attempt two] · do · next turn
|
||||||
|
fork: [ATTEMPT ONE] · do · elsewhere
|
||||||
|
|
||||||
|
The *discarded* attempt is the one that gets promoted, because a fork moves
|
||||||
|
the take you are leaving for and leaves the line you came from untouched.
|
||||||
|
|
||||||
|
Returns the adventure id, with the head on the fork.
|
||||||
|
"""
|
||||||
|
adv_id = client.adv_id
|
||||||
|
ScriptedProvider.replies = ["Attempt one.", "Attempt two.", "Next turn.", "Elsewhere."]
|
||||||
|
_play(client, adv_id)
|
||||||
|
_retry(client, adv_id)
|
||||||
|
_play(client, adv_id, "go deeper")
|
||||||
|
discarded = [a.id for a in _rows(adv_id) if a.type == "ai" and not a.live][0]
|
||||||
|
assert client.post(f"/api/adventures/{adv_id}/actions/{discarded}/fork").status_code == 200
|
||||||
|
_play(client, adv_id, "go sideways")
|
||||||
|
return adv_id
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------- the round trip (v2)
|
||||||
|
|
||||||
|
def test_a_forked_story_survives_the_round_trip(client):
|
||||||
|
"""The headline: both futures come back, and both are still readable.
|
||||||
|
|
||||||
|
This is the thing v1 could not do. The check is not "the same rows" — the
|
||||||
|
ids are new — but "the same stories", read the way a player reads them: by
|
||||||
|
switching to a branch and looking at what it says.
|
||||||
|
"""
|
||||||
|
original = _forked_story(client)
|
||||||
|
before = _every_branch_story(client, original)
|
||||||
|
assert len(before) == 2, "the fixture forked"
|
||||||
|
assert before[0] != before[1], "and the two branches tell different stories"
|
||||||
|
|
||||||
|
copy = _imported(client, _export(client, original))
|
||||||
|
assert copy != original
|
||||||
|
assert _every_branch_story(client, copy) == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_fork_point_comes_back_where_it_was_put(client):
|
||||||
|
"""`fork_depth` is stored, never inferred — including through a file.
|
||||||
|
|
||||||
|
Inferring it from where two branches' nodes first differ would be a guess
|
||||||
|
about how the story was played, and a wrong one the moment an attempt
|
||||||
|
happens to repeat its parent's text.
|
||||||
|
"""
|
||||||
|
original = _forked_story(client)
|
||||||
|
before = [(b["parent_branch_id"] is None, b["fork_depth"]) for b in _branches(client, original)]
|
||||||
|
|
||||||
|
copy = _imported(client, _export(client, original))
|
||||||
|
assert [(b["parent_branch_id"] is None, b["fork_depth"])
|
||||||
|
for b in _branches(client, copy)] == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_head_comes_back_on_the_branch_it_was_left_on(client):
|
||||||
|
original = _forked_story(client)
|
||||||
|
head_before = [b["is_head"] for b in _branches(client, original)]
|
||||||
|
assert head_before == [False, True], "the fixture left the head on the fork"
|
||||||
|
|
||||||
|
copy = _imported(client, _export(client, original))
|
||||||
|
assert [b["is_head"] for b in _branches(client, copy)] == head_before
|
||||||
|
# And the tip it sits at is derived from the nodes that arrived, not read
|
||||||
|
# out of the file — the bundle never says how deep a branch goes.
|
||||||
|
assert _texts(client, copy) == _texts(client, original)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_switch_in_the_copy_restores_what_that_branch_left_behind(client):
|
||||||
|
"""The after-snapshots are why the bundle carries them.
|
||||||
|
|
||||||
|
The gold script adds ten a turn, so the scoreboard is a count of the story
|
||||||
|
behind it. A bundle that carried the actions but not the outcomes would
|
||||||
|
import a tree that reads correctly and switches wrong.
|
||||||
|
"""
|
||||||
|
original = _forked_story(client)
|
||||||
|
# One more turn on the fork, so the two tips are genuinely different
|
||||||
|
# numbers: played turn for turn, the branches earn the same gold and a
|
||||||
|
# switch that restored nothing at all would still look right.
|
||||||
|
ScriptedProvider.replies = ["Further still."]
|
||||||
|
_play(client, original, "press on")
|
||||||
|
|
||||||
|
per_branch = []
|
||||||
|
for branch in _branches(client, original):
|
||||||
|
_switch(client, original, branch["id"])
|
||||||
|
per_branch.append(_script_state(original).get("gold"))
|
||||||
|
assert len(set(per_branch)) == len(per_branch), "the tips are at different totals"
|
||||||
|
|
||||||
|
copy = _imported(client, _export(client, original))
|
||||||
|
restored = []
|
||||||
|
for branch in _branches(client, copy):
|
||||||
|
_switch(client, copy, branch["id"])
|
||||||
|
restored.append(_script_state(copy).get("gold"))
|
||||||
|
assert restored == per_branch
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_memory_comes_back_on_the_node_it_hangs_off(client):
|
||||||
|
"""Derived work is addressed by coordinate, so the coordinate is carried."""
|
||||||
|
original = _forked_story(client)
|
||||||
|
tip = [a for a in _rows(original) if a.live][-1]
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
db.add(models.Memory(
|
||||||
|
adventure_id=original, text="They met a goblin.",
|
||||||
|
source_start=0, source_end=tip.depth,
|
||||||
|
branch_id=tip.branch_id, depth=tip.depth,
|
||||||
|
))
|
||||||
|
db.commit()
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
exported = _export(client, original)
|
||||||
|
assert [(m["branch"], m["depth"]) for m in exported["memories"]] == [(1, tip.depth)]
|
||||||
|
|
||||||
|
copy = _imported(client, exported)
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
memories = db.query(models.Memory).filter(
|
||||||
|
models.Memory.adventure_id == copy).all()
|
||||||
|
branches = [b.id for b in _branch_rows(copy)]
|
||||||
|
assert [(branches.index(m.branch_id), m.depth) for m in memories] == [(1, tip.depth)]
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------- what is not in the file
|
||||||
|
|
||||||
|
def test_the_lineage_is_rebuilt_rather_than_carried(client):
|
||||||
|
"""A cache of `parent` + `fork_depth` is not a second thing to ship.
|
||||||
|
|
||||||
|
The file says where each branch forked; the ancestry that makes the fork
|
||||||
|
readable is computed from that on the way in, capped at the fork exactly as
|
||||||
|
`tree.fork` caps it. Shipping the cache too would put two sources of truth
|
||||||
|
for one fact in a file anybody can hand-edit.
|
||||||
|
"""
|
||||||
|
original = _forked_story(client)
|
||||||
|
exported = _export(client, original)
|
||||||
|
assert all("lineage" not in b for b in exported["branches"])
|
||||||
|
|
||||||
|
root, forked = _branch_rows(_imported(client, exported))
|
||||||
|
assert root.lineage == [[root.id, None]]
|
||||||
|
assert forked.lineage == [[forked.id, None], [root.id, forked.fork_depth]]
|
||||||
|
# Which is the arithmetic the reader depends on: the parent is capped one
|
||||||
|
# depth short of the attempt that was promoted, so the fork cannot see it.
|
||||||
|
assert lineage.entries_of(forked) == [(forked.id, None), (root.id, forked.fork_depth)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_legacy_index_is_reissued_so_two_branches_never_share_one(client):
|
||||||
|
"""`index` is a fact about the adventure, and `depth` is one about a path.
|
||||||
|
|
||||||
|
Two branches have a node at depth 3, so depth cannot be the number
|
||||||
|
`max_action_index` hands out next. The import allocates one per turn
|
||||||
|
instead: siblings share it, the way SP4 leaves them, and no two coordinates
|
||||||
|
do.
|
||||||
|
"""
|
||||||
|
copy = _imported(client, _export(client, _forked_story(client)))
|
||||||
|
rows = _rows(copy)
|
||||||
|
by_index: dict[int, set] = {}
|
||||||
|
for row in rows:
|
||||||
|
by_index.setdefault(row.index, set()).add((row.branch_id, row.depth))
|
||||||
|
assert all(len(coords) == 1 for coords in by_index.values()), \
|
||||||
|
"one index per turn, whatever branch it is on"
|
||||||
|
assert len(by_index) == len({(r.branch_id, r.depth) for r in rows})
|
||||||
|
|
||||||
|
# The case that makes the rule necessary: the fork and the line it left
|
||||||
|
# both hold a turn at depth 2, and they are not the same turn.
|
||||||
|
at_depth_2 = [r for r in rows if r.depth == 2]
|
||||||
|
assert len({r.branch_id for r in at_depth_2}) == 2
|
||||||
|
assert len({r.index for r in at_depth_2}) == 2, "same depth, different turns"
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------- a file that is wrong
|
||||||
|
|
||||||
|
def test_a_node_naming_a_branch_the_file_does_not_list_is_refused(client):
|
||||||
|
"""Refused, not half-applied. A tree missing a branch is a story that
|
||||||
|
silently stops, which is the failure this whole phase exists to end."""
|
||||||
|
payload = _export(client, _forked_story(client))
|
||||||
|
payload["branches"] = payload["branches"][:1]
|
||||||
|
before = _adventure_count()
|
||||||
|
|
||||||
|
r = _import(client, payload)
|
||||||
|
assert r.status_code == 400, r.text
|
||||||
|
assert "branch" in r.json()["detail"].lower()
|
||||||
|
assert _adventure_count() == before, "nothing was created"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_branch_forking_from_one_listed_after_it_is_refused(client):
|
||||||
|
"""The ordering rule buys acyclicity for the price of a comparison — and a
|
||||||
|
cycle would be an import that never returns rather than one that fails."""
|
||||||
|
payload = _export(client, _forked_story(client))
|
||||||
|
payload["branches"] = [{"parent": 1, "forkDepth": 0}, {"parent": None, "forkDepth": None}]
|
||||||
|
before = _adventure_count()
|
||||||
|
|
||||||
|
r = _import(client, payload)
|
||||||
|
assert r.status_code == 400, r.text
|
||||||
|
assert _adventure_count() == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_fork_with_no_depth_is_refused(client):
|
||||||
|
payload = _export(client, _forked_story(client))
|
||||||
|
payload["branches"][1].pop("forkDepth")
|
||||||
|
before = _adventure_count()
|
||||||
|
|
||||||
|
r = _import(client, payload)
|
||||||
|
assert r.status_code == 400, r.text
|
||||||
|
assert "depth" in r.json()["detail"].lower()
|
||||||
|
assert _adventure_count() == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_more_branches_than_the_cap_is_refused(client, monkeypatch):
|
||||||
|
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||||
|
payload = {
|
||||||
|
"format": bundle.FORMAT, "title": "Too many",
|
||||||
|
"branches": [{"parent": None, "forkDepth": None}]
|
||||||
|
* (limits.MAX_BRANCHES_PER_ADVENTURE + 1),
|
||||||
|
"actions": [],
|
||||||
|
}
|
||||||
|
before = _adventure_count()
|
||||||
|
|
||||||
|
r = _import(client, payload)
|
||||||
|
assert r.status_code == 409, r.text
|
||||||
|
assert _adventure_count() == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_turn_the_file_gives_no_live_attempt_still_tells_one(client):
|
||||||
|
"""A coordinate with nothing live is a turn no read can see.
|
||||||
|
|
||||||
|
The file is allowed to be wrong about this — it is a text file — so the
|
||||||
|
import picks the first attempt rather than importing a story with a hole.
|
||||||
|
"""
|
||||||
|
payload = _export(client, _forked_story(client))
|
||||||
|
for node in payload["actions"]:
|
||||||
|
node["live"] = False
|
||||||
|
copy = _imported(client, payload)
|
||||||
|
|
||||||
|
assert _texts(client, copy), "the story is readable"
|
||||||
|
live = [(r.branch_id, r.depth) for r in _rows(copy) if r.live]
|
||||||
|
assert len(live) == len(set(live)), "exactly one live attempt per coordinate"
|
||||||
|
assert len(live) == len({(r.branch_id, r.depth) for r in _rows(copy)})
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- the v1 reader
|
||||||
|
|
||||||
|
def test_a_v1_bundle_still_imports(client):
|
||||||
|
"""The reader stays after the writer goes: those files are already saved."""
|
||||||
|
payload = {
|
||||||
|
"format": bundle.LEGACY_FORMAT,
|
||||||
|
"title": "Old backup",
|
||||||
|
"memoryCursor": 0, "summaryCursor": 0,
|
||||||
|
"actions": [
|
||||||
|
{"index": 0, "type": "start", "text": OPENING},
|
||||||
|
{"index": 1, "type": "do", "text": "> You go north."},
|
||||||
|
{
|
||||||
|
"index": 2, "type": "ai", "text": "Two.",
|
||||||
|
"variants": [{"text": "One."}, {"text": "Two."}],
|
||||||
|
"variantIndex": 1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
copy = _imported(client, payload)
|
||||||
|
|
||||||
|
assert _texts(client, copy) == [OPENING, "> You go north.", "Two."]
|
||||||
|
# One branch, and the `variants` array split back into the sibling group it
|
||||||
|
# always described.
|
||||||
|
assert len(_branches(client, copy)) == 1
|
||||||
|
ai = [r for r in _rows(copy) if r.type == "ai"]
|
||||||
|
assert [(r.text, r.live) for r in ai] == [("One.", False), ("Two.", True)]
|
||||||
|
assert len({(r.branch_id, r.depth) for r in ai}) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_v1_bundle_with_a_cursor_lands_it_on_a_node(client):
|
||||||
|
"""v1 counts covered actions; the tree anchors them. The translation needs
|
||||||
|
the nodes to exist, so it happens after they are written."""
|
||||||
|
payload = {
|
||||||
|
"format": bundle.LEGACY_FORMAT, "title": "Old backup",
|
||||||
|
"memoryCursor": 2, "summaryCursor": 2,
|
||||||
|
"actions": [
|
||||||
|
{"index": 0, "type": "start", "text": OPENING},
|
||||||
|
{"index": 1, "type": "story", "text": "A corridor."},
|
||||||
|
{"index": 2, "type": "story", "text": "A door."},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
copy = _imported(client, payload)
|
||||||
|
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
adventure = db.get(models.Adventure, copy)
|
||||||
|
assert adventure.memory_cursor == 2, "the legacy count is kept as given"
|
||||||
|
assert adventure.memory_cursor_branch_id is not None
|
||||||
|
assert adventure.memory_cursor_depth == 1, "the second story action"
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_v2_bundle_brings_its_anchors_back(client):
|
||||||
|
"""The other direction: v2 carries the anchor and the count is read off it."""
|
||||||
|
original = _forked_story(client)
|
||||||
|
tip = [a for a in _rows(original) if a.live][-1]
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
adventure = db.get(models.Adventure, original)
|
||||||
|
adventure.memory_cursor_branch_id = tip.branch_id
|
||||||
|
adventure.memory_cursor_depth = tip.depth
|
||||||
|
db.commit()
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
exported = _export(client, original)
|
||||||
|
assert exported["memoryCursor"] == {"branch": 1, "depth": tip.depth}
|
||||||
|
|
||||||
|
copy = _imported(client, exported)
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
adventure = db.get(models.Adventure, copy)
|
||||||
|
branches = [b.id for b in _branch_rows(copy)]
|
||||||
|
assert branches.index(adventure.memory_cursor_branch_id) == 1
|
||||||
|
assert adventure.memory_cursor_depth == tip.depth
|
||||||
|
assert adventure.memory_cursor > 0, "the legacy count was read back off it"
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_format_is_refused(client):
|
||||||
|
r = _import(client, {"format": "ai-dnd-adventure-v3", "title": "From the future"})
|
||||||
|
assert r.status_code == 400, r.text
|
||||||
|
assert bundle.FORMAT in r.json()["detail"]
|
||||||
@@ -330,9 +330,12 @@ def test_export_and_import_round_trips_variants(client):
|
|||||||
_play(client)
|
_play(client)
|
||||||
_retry(client)
|
_retry(client)
|
||||||
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
||||||
ai = [a for a in bundle["actions"] if a["type"] == "ai"][0]
|
# SP6: the attempts are nodes in the bundle too, sharing one coordinate,
|
||||||
assert [v["text"] for v in ai["variants"]] == ["One.", "Two."]
|
# and `live` says which of them the story tells. The `variants` array
|
||||||
assert ai["variantIndex"] == 1
|
# survives only in the v1 *reader* — see the hand-edited bundle below.
|
||||||
|
ai = [a for a in bundle["actions"] if a["type"] == "ai"]
|
||||||
|
assert [(a["text"], a["live"]) for a in ai] == [("One.", False), ("Two.", True)]
|
||||||
|
assert len({(a["branch"], a["depth"]) for a in ai}) == 1
|
||||||
|
|
||||||
r = client.post("/api/adventures/import", json=bundle)
|
r = client.post("/api/adventures/import", json=bundle)
|
||||||
assert r.status_code == 201, r.text
|
assert r.status_code == 201, r.text
|
||||||
|
|||||||
@@ -451,9 +451,11 @@ def test_memories_are_created_listed_and_deleted(client):
|
|||||||
def test_export_carries_the_whole_story(client):
|
def test_export_carries_the_whole_story(client):
|
||||||
"""A bundle is a backup: every action, not the window.
|
"""A bundle is a backup: every action, not the window.
|
||||||
|
|
||||||
NOTE for SP6 — the `format` assertion below is the one line in this file
|
SP6 changed the format assertion below, as this note said it would. It also
|
||||||
expected to change, when the bundle becomes `ai-dnd-adventure-v2`. It is
|
changed one more line in this file than the note allowed for — the
|
||||||
correct through SP1-SP5.
|
`variants` array in `test_export_keeps_retry_attempts`, which is the same
|
||||||
|
fact seen from the other side: a bundle that has coordinates has no use for
|
||||||
|
a repeating group. Everything else here still passes unmodified.
|
||||||
"""
|
"""
|
||||||
ScriptedProvider.replies = ["One.", "Two."]
|
ScriptedProvider.replies = ["One.", "Two."]
|
||||||
_play(client, "go north")
|
_play(client, "go north")
|
||||||
@@ -462,7 +464,7 @@ def test_export_carries_the_whole_story(client):
|
|||||||
r = client.get(f"/api/adventures/{client.adv_id}/export")
|
r = client.get(f"/api/adventures/{client.adv_id}/export")
|
||||||
assert r.status_code == 200, r.text
|
assert r.status_code == 200, r.text
|
||||||
bundle = r.json()
|
bundle = r.json()
|
||||||
assert bundle["format"] == "ai-dnd-adventure-v1"
|
assert bundle["format"] == "ai-dnd-adventure-v2"
|
||||||
assert bundle["title"] == "Cave"
|
assert bundle["title"] == "Cave"
|
||||||
assert [a["text"] for a in bundle["actions"]] == [
|
assert [a["text"] for a in bundle["actions"]] == [
|
||||||
OPENING, "> You go north.", "One.", "> You go south.", "Two.",
|
OPENING, "> You go north.", "One.", "> You go south.", "Two.",
|
||||||
@@ -490,8 +492,9 @@ def test_export_keeps_retry_attempts(client):
|
|||||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||||
|
|
||||||
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
||||||
ai = [a for a in bundle["actions"] if a["type"] == "ai"][-1]
|
ai = [a for a in bundle["actions"] if a["type"] == "ai"]
|
||||||
assert [v["text"] for v in ai["variants"]] == ["Attempt one.", "Attempt two."]
|
assert [a["text"] for a in ai] == ["Attempt one.", "Attempt two."]
|
||||||
|
assert [a["live"] for a in ai] == [False, True]
|
||||||
|
|
||||||
|
|
||||||
# -------------------------------------------------------------- world state
|
# -------------------------------------------------------------- world state
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
"""What a v2 bundle costs, measured on a production-sized adventure.
|
||||||
|
|
||||||
|
SP6 gave the bundle coordinates, and coordinates cost bytes: a branch number
|
||||||
|
and a depth on every node, plus the two after-snapshots a switch needs to put
|
||||||
|
back. This puts a number on that, against the v1 shape it replaces, on the same
|
||||||
|
fixture the egress work is measured on.
|
||||||
|
|
||||||
|
python -m tools.measure_bundle --actions 600 --rich
|
||||||
|
python -m tools.measure_bundle --actions 600 --rich --forks 20
|
||||||
|
|
||||||
|
Reuses `tools.stress_session`'s fixture, so read that module's warnings first:
|
||||||
|
a `--rich` run is a correctness fixture, and its byte figures are not
|
||||||
|
comparable to a plain one.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import random
|
||||||
|
import sys
|
||||||
|
|
||||||
|
# `stress_session` FIRST, and it is not a style preference. Importing it is what
|
||||||
|
# points `AIDND_DB_PATH` at a throwaway file, and `app.database` reads that at
|
||||||
|
# module scope — so an `app` import above this line silently runs the whole
|
||||||
|
# fixture against `backend/data.db` instead. It fails by *working*: the first
|
||||||
|
# run seeds a synthetic user and adventure into the local database and reports
|
||||||
|
# perfectly good numbers, and only the second run trips over the unique email.
|
||||||
|
from tools import stress_session as stress # noqa: I001 (see above)
|
||||||
|
|
||||||
|
from app import bundle, models, tree
|
||||||
|
from app.database import SessionLocal
|
||||||
|
|
||||||
|
|
||||||
|
def _v1_shape(v2: dict) -> dict:
|
||||||
|
"""The same story as v1 would have written it, for a like-for-like count.
|
||||||
|
|
||||||
|
One entry per turn, the siblings folded back into a `variants` array, no
|
||||||
|
coordinates and no outcomes — which is exactly what v1 could carry.
|
||||||
|
"""
|
||||||
|
turns: dict[tuple[int, int], list[dict]] = {}
|
||||||
|
order: list[tuple[int, int]] = []
|
||||||
|
for node in v2["actions"]:
|
||||||
|
key = (node["branch"], node["depth"])
|
||||||
|
if key not in turns:
|
||||||
|
turns[key] = []
|
||||||
|
order.append(key)
|
||||||
|
turns[key].append(node)
|
||||||
|
actions = []
|
||||||
|
for i, key in enumerate(order):
|
||||||
|
group = turns[key]
|
||||||
|
live = next((n for n in group if n["live"]), group[0])
|
||||||
|
actions.append({
|
||||||
|
"index": i, "type": live["type"], "text": live["text"],
|
||||||
|
"reasoning": live.get("reasoning"),
|
||||||
|
"variants": [
|
||||||
|
{"text": n["text"], "reasoning": n.get("reasoning"),
|
||||||
|
"createdAt": n["createdAt"]}
|
||||||
|
for n in group
|
||||||
|
] if len(group) > 1 else None,
|
||||||
|
"variantIndex": group.index(live),
|
||||||
|
"createdAt": live["createdAt"],
|
||||||
|
})
|
||||||
|
old = {k: v for k, v in v2.items() if k not in ("branches", "headBranch")}
|
||||||
|
old["format"] = bundle.LEGACY_FORMAT
|
||||||
|
old["actions"] = actions
|
||||||
|
old["memories"] = [
|
||||||
|
{k: v for k, v in m.items() if k not in ("branch", "depth")}
|
||||||
|
for m in v2["memories"]
|
||||||
|
]
|
||||||
|
old["memoryCursor"] = 0
|
||||||
|
old["summaryCursor"] = 0
|
||||||
|
return old
|
||||||
|
|
||||||
|
|
||||||
|
def _bytes(obj) -> int:
|
||||||
|
return len(json.dumps(obj).encode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None) -> int:
|
||||||
|
argv = list(sys.argv[1:] if argv is None else argv)
|
||||||
|
forks = 0
|
||||||
|
if "--forks" in argv:
|
||||||
|
i = argv.index("--forks")
|
||||||
|
forks = int(argv[i + 1])
|
||||||
|
del argv[i:i + 2]
|
||||||
|
args = stress.parse_args(argv)
|
||||||
|
rng = random.Random(args.seed)
|
||||||
|
stress._build_text(args, random.Random(args.seed ^ 0x5F5F))
|
||||||
|
adv_id, _ = stress.build_fixture(args, rng)
|
||||||
|
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
adventure = db.get(models.Adventure, adv_id)
|
||||||
|
if forks:
|
||||||
|
# Twenty divergences off one line, each a little deeper — the shape
|
||||||
|
# SP5 measured the fork cost on. The nodes are chosen up front and
|
||||||
|
# the session is flushed after every fork: `fork` moves a row onto
|
||||||
|
# its new branch, and with `autoflush=False` a query issued before
|
||||||
|
# that move is written still finds the node where it used to be.
|
||||||
|
root = adventure.head_branch_id
|
||||||
|
candidates = [
|
||||||
|
node.id for node in
|
||||||
|
db.query(models.Action)
|
||||||
|
.filter(
|
||||||
|
models.Action.adventure_id == adventure.id,
|
||||||
|
models.Action.branch_id == root,
|
||||||
|
models.Action.depth > 1,
|
||||||
|
)
|
||||||
|
.order_by(models.Action.depth)
|
||||||
|
.all()
|
||||||
|
]
|
||||||
|
step = max(len(candidates) // (forks + 1), 1)
|
||||||
|
made = 0
|
||||||
|
for node_id in candidates[::step][:forks]:
|
||||||
|
tree.fork(db, adventure, db.get(models.Action, node_id))
|
||||||
|
db.flush()
|
||||||
|
made += 1
|
||||||
|
db.commit()
|
||||||
|
print(f"forked {made} times")
|
||||||
|
|
||||||
|
v2 = bundle.export(db, adventure)
|
||||||
|
v1 = _v1_shape(v2)
|
||||||
|
nodes = len(v2["actions"])
|
||||||
|
turns = len({(n["branch"], n["depth"]) for n in v2["actions"]})
|
||||||
|
branches = len(v2["branches"])
|
||||||
|
|
||||||
|
stripped = json.loads(json.dumps(v2))
|
||||||
|
for node in stripped["actions"]:
|
||||||
|
node.pop("stateAfter", None)
|
||||||
|
node.pop("worldStateAfter", None)
|
||||||
|
node.pop("worldDelta", None)
|
||||||
|
|
||||||
|
v2_bytes, v1_bytes, bare = _bytes(v2), _bytes(v1), _bytes(stripped)
|
||||||
|
print(f"{turns} turns · {nodes} nodes · {branches} branches")
|
||||||
|
print(f"v1 shape {v1_bytes:>12,} B")
|
||||||
|
print(f"v2 {v2_bytes:>12,} B "
|
||||||
|
f"{v2_bytes / v1_bytes:.3f}× v1")
|
||||||
|
print(f"v2 w/o outcomes {bare:>12,} B "
|
||||||
|
f"{bare / v1_bytes:.3f}× v1")
|
||||||
|
print(f"the outcomes {v2_bytes - bare:>12,} B "
|
||||||
|
f"{(v2_bytes - bare) / nodes:.1f} B/node")
|
||||||
|
print(f"coordinates {bare - v1_bytes:>12,} B "
|
||||||
|
f"{(bare - v1_bytes) / nodes:+.1f} B/node")
|
||||||
|
print(f"import cap {bundle.__name__}: "
|
||||||
|
f"{v2_bytes / (20 * 1024 * 1024):.1%} of MAX_IMPORT_BODY_BYTES")
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -37,8 +37,10 @@ existed for undo and delete; see the trap note below. Two rows want a footnote:
|
|||||||
stale exactly as before. What *has* changed is that the machinery to fix it now exists
|
stale exactly as before. What *has* changed is that the machinery to fix it now exists
|
||||||
— an edit could write a sibling and switch to it, which is a retry the player typed —
|
— an edit could write a sibling and switch to it, which is a retry the player typed —
|
||||||
so it is a small change whenever it is wanted.
|
so it is a small change whenever it is wanted.
|
||||||
- **The 1NF violation** is resolved in the database. The `variants` array survives in
|
- **The 1NF violation** is resolved. Nothing writes a `variants` array any more, in the
|
||||||
exactly one place: the v1 export bundle, which SP6 replaces.
|
database or out of it — SP6 replaced the bundle that was its last producer, and the
|
||||||
|
array survives only in the v1 *reader*, which exists so files already saved still
|
||||||
|
import.
|
||||||
|
|
||||||
## Design decisions (settled 2026-08-16)
|
## Design decisions (settled 2026-08-16)
|
||||||
|
|
||||||
@@ -610,6 +612,88 @@ bundle's linear actions plus `variants` onto one branch with siblings.
|
|||||||
**Verify:** v2 round-trip of a branched adventure is lossless; a v1 bundle still imports;
|
**Verify:** v2 round-trip of a branched adventure is lossless; a v1 bundle still imports;
|
||||||
a bundle claiming more branches than rows is rejected rather than half-applied.
|
a bundle claiming more branches than rows is rejected rather than half-applied.
|
||||||
|
|
||||||
|
**Done, 2026-08-18** (branch `sp6-bundle-v2`). **381 tests green**, the 365 SP5 finished
|
||||||
|
with plus 16 in the new `test_bundle_v2.py`. `app/bundle.py` owns both formats; the two
|
||||||
|
endpoints in `routers/adventures.py` are a delegation and the shared plumbing, and the
|
||||||
|
`variants` array now exists nowhere but the v1 *reader*.
|
||||||
|
|
||||||
|
**The rule the module is built on: a bundle carries what was *chosen*, never what is
|
||||||
|
*derived*.** The head branch, the fork points, the live flags and the anchors are
|
||||||
|
decisions somebody made, and they are in the file. `lineage`, the head *depth*, `index`
|
||||||
|
and the variant ordinals are computed from those and are rebuilt on the way in. That is
|
||||||
|
not tidiness — a bundle is a text file anybody can edit, and every derived field shipped
|
||||||
|
beside its source is a chance for the file to disagree with itself in a way no read would
|
||||||
|
report. It is also the answer to "is the round trip lossless?": everything omitted is
|
||||||
|
reconstructed, and the tests assert the reconstruction rather than the bytes.
|
||||||
|
|
||||||
|
Six things worth not rediscovering:
|
||||||
|
|
||||||
|
- **`depth` cannot be the legacy `index`, and this is where that stops being academic.**
|
||||||
|
They agreed until SP5, and a bundle is the first writer that has to fill `index` for a
|
||||||
|
*forked* story — where two branches both have a node at depth 4. `index`'s one
|
||||||
|
remaining job is handing the next row a number nothing else holds, which is a fact
|
||||||
|
about the adventure rather than about a path, so the import allocates one per turn in
|
||||||
|
bundle order: siblings share it, the way SP4 leaves them, and no two coordinates do.
|
||||||
|
- **Validation happens before the adventure row exists.** Everything a hand-edited file
|
||||||
|
can get wrong about the shape of a tree — a node naming a branch that is not listed, a
|
||||||
|
fork with no depth, a branch forking from one listed after it — is a 400 raised by
|
||||||
|
`plan`, which touches no session. The alternative is an adventure holding a story with
|
||||||
|
a hole in it, and this whole phase exists because a story with a hole in it fails by
|
||||||
|
going quiet.
|
||||||
|
- **A branch may only fork from one listed before it.** That is how the export writes
|
||||||
|
them, and requiring it buys acyclicity for the price of a comparison — a cycle in the
|
||||||
|
parent chain would be an import that never returns rather than one that fails.
|
||||||
|
- **`{}` and absent are different snapshots.** An empty `state_after` means "this node
|
||||||
|
left an empty scoreboard behind"; a missing one means "nobody knows, leave the live
|
||||||
|
state alone" (`attempts.restore_state`). Trimming empty dicts on the way out would have
|
||||||
|
saved eighteen bytes a row and turned an undo that clears a score into one that leaves
|
||||||
|
it standing. Only `worldDelta`, which is display, is dropped when empty.
|
||||||
|
- **A file is allowed to be wrong about which attempt is live, and the import corrects it
|
||||||
|
rather than refusing.** "Exactly one sibling in a group is live" is an invariant of the
|
||||||
|
*database*, not of the format; a coordinate with none is a turn no read can see, so the
|
||||||
|
first attempt is made live. That is a different class from a missing branch, which is
|
||||||
|
structure, and is refused.
|
||||||
|
- **`limits.MAX_BRANCHES_PER_ADVENTURE` (1000) has no live counterpart.** Forking is a
|
||||||
|
POST that adds one row and has no cap of its own, so this is the one bundle cap that
|
||||||
|
does not mirror something creation enforces. Worth closing if branch management ever
|
||||||
|
makes forking cheap to repeat.
|
||||||
|
|
||||||
|
**The verify line above was slightly wrong, and the code does the honest version.** "More
|
||||||
|
branches than rows" fails on an adventure with no actions, which legitimately has one
|
||||||
|
branch and no rows. It became two rules instead: a cap on the branch list, and *every
|
||||||
|
branch a node names must exist*.
|
||||||
|
|
||||||
|
Measured with `tools/measure_bundle.py` on the 600-action `--rich` fixture — 600 turns,
|
||||||
|
750 nodes, because 150 of them were retried:
|
||||||
|
|
||||||
|
| | bytes | vs v1 |
|
||||||
|
|---|---|---|
|
||||||
|
| v1 shape | 587,475 | — |
|
||||||
|
| **v2** | **911,229** | **1.551×** |
|
||||||
|
| v2 without the outcomes | 544,318 | 0.927× |
|
||||||
|
|
||||||
|
**The tree is free; the outcomes are what cost.** Coordinates *save* 57.5 B a node
|
||||||
|
against v1's turn-and-variants shape, and the entire 1.55× is `state_after` /
|
||||||
|
`world_state_after` at 489 B a node — which are there because a bundle without them
|
||||||
|
imports a tree nobody can switch inside. Twenty forks add 660 B to the same file, **33 B
|
||||||
|
a branch**, so the format is as indifferent to fork count as the reads are. The longest
|
||||||
|
adventure production holds exports at 4.3 % of `MAX_IMPORT_BODY_BYTES`.
|
||||||
|
|
||||||
|
**Two lines of the SP0 baseline changed, not the one it predicted.** The note in
|
||||||
|
`test_story_tree_baseline.py` allowed for the `format` assertion; the `variants` array in
|
||||||
|
`test_export_keeps_retry_attempts` is the same fact from the other side — a bundle with
|
||||||
|
coordinates has no use for a repeating group. Everything else in that file still passes
|
||||||
|
unmodified.
|
||||||
|
|
||||||
|
**No migration, no vacuum.** SP6 adds no column and rewrites no row.
|
||||||
|
|
||||||
|
**And one trap, paid for once.** `tools/measure_bundle.py` imported `app` before
|
||||||
|
`tools.stress_session`, which is what points `AIDND_DB_PATH` at a throwaway file —
|
||||||
|
`app.database` reads it at module scope. It fails by *working*: the first run seeded a
|
||||||
|
synthetic user and adventure into the local `backend/data.db` and printed perfectly good
|
||||||
|
numbers, and only the second run tripped over the unique email. Anything importing that
|
||||||
|
harness must import it first, and the file now says so where the imports are.
|
||||||
|
|
||||||
### SP7 — Frontend: full tree visualisation
|
### SP7 — Frontend: full tree visualisation
|
||||||
|
|
||||||
`VariantPager` is removed. A spatial tree view replaces it, plus switch, rename and
|
`VariantPager` is removed. A spatial tree view replaces it, plus switch, rename and
|
||||||
|
|||||||
+66
-24
@@ -78,37 +78,41 @@ needed; nothing requires reading a row of anyone's story.
|
|||||||
|
|
||||||
## Pick up here
|
## Pick up here
|
||||||
|
|
||||||
**`plan/14-phase-story-tree.md`, SP6 — export/import v2.** SP0–SP5 are done and green
|
**`plan/14-phase-story-tree.md`, SP7 — the frontend.** SP0–SP6 are done and green
|
||||||
(**365 tests**); **nothing is deployed yet**. The tree is complete as a storage model: a
|
(**381 tests**); **nothing is deployed yet**. The tree is complete everywhere except the
|
||||||
retry writes a sibling node, continuing from a discarded attempt forks a branch, and
|
screen: a retry writes a sibling node, continuing from a discarded attempt forks a branch,
|
||||||
`GET /branches`, `POST /branches/{id}/switch` and `POST /actions/{id}/fork` are the
|
and a backup carries the whole thing (`ai-dnd-adventure-v2`, with the v1 reader kept so
|
||||||
endpoints SP7's tree view will be drawn on. What is left is the bundle format (SP6), the
|
existing bundles still import). What is left is the frontend (SP7) and dropping the legacy
|
||||||
frontend (SP7) and dropping the legacy columns (SP8).
|
columns (SP8).
|
||||||
|
|
||||||
**Do SP6 before SP7, and the reason is a live gap.** A forked adventure has no honest v1
|
**SP7 is the release gate, and it is unscoped.** `VariantPager` comes out, a spatial tree
|
||||||
export — the format has one story and there are two — so export currently emits every
|
view replaces it, and branch management — switch, rename, delete-with-confirm — is a hard
|
||||||
branch's turns interleaved by `index`, which reads as a mangled story. Nobody can reach
|
dependency rather than a nice-to-have, because nothing auto-prunes and storage otherwise
|
||||||
that state through the product yet, because forking has no UI until SP7. That ordering is
|
grows without limit. `api.js` gains `GET /branches`, `POST /branches/{id}/switch` and
|
||||||
the whole mitigation, so keep it.
|
`POST /actions/{id}/fork`, which SP5 built for exactly this.
|
||||||
|
|
||||||
|
**Drive the 600-action `--keep` fixture in a browser by hand.** That is SP7's own verify
|
||||||
|
line and the standing open gap in this project: the scroll path has never been driven by
|
||||||
|
hand and has already hidden one bug. A vitest + jsdom harness covers the prepend
|
||||||
|
arithmetic, but jsdom has no layout, so scroll position still needs eyes.
|
||||||
|
|
||||||
**The schema is live in code but not on production.** When this ships, the deploy needs
|
**The schema is live in code but not on production.** When this ships, the deploy needs
|
||||||
one `VACUUM FULL actions;` on the direct (non-`-pooler`) endpoint afterwards — SP1's
|
one `VACUUM FULL actions;` on the direct (non-`-pooler`) endpoint afterwards — SP1's
|
||||||
migration rewrites every row and SP4's rewrites it three times more, so **two vacuums are
|
migration rewrites every row and SP4's rewrites it three times more, so **two vacuums are
|
||||||
owed and one run settles both**. SP3's and SP5's changes touch `adventures` only (SP5 adds
|
owed and one run settles both**. SP3's, SP5's and SP6's changes need none (SP5 and SP6 add
|
||||||
no migration at all) and need none. See the 144 MB lesson at the top of this file.
|
no migration at all). See the 144 MB lesson at the top of this file.
|
||||||
|
|
||||||
Three things to carry into SP6:
|
Three things to carry into SP7:
|
||||||
|
|
||||||
- **The `variants` array now exists in exactly one place: the export bundle.** Nothing in
|
- **The bundle is the one thing here a migration can never reach.** `app/bundle.py` owns
|
||||||
the database holds one. `export_adventure` folds each sibling group back into the shape
|
both formats and nothing else knows either. Its rule — *carry what was chosen, never
|
||||||
the v1 reader expects, and `_imported_turn` splits one back out into rows. Those two
|
what is derived* — is worth borrowing anywhere else state has to leave the database.
|
||||||
functions are the whole v1 surface, and v2 replaces them.
|
- **`variant_count` and `variant_index` die with the pager.** SP4 left them as a
|
||||||
- **A v2 bundle needs branches, `live`, and both after-snapshots.** `state_after` /
|
maintained cache of the sibling group's shape because the pager reads both for every
|
||||||
`world_state_after` are what a branch switch restores; a bundle that carried the
|
row of a page. They are dead the moment the tree view replaces it, and SP8 drops them.
|
||||||
actions but not the outcomes would import a tree nobody could switch inside.
|
- **Nothing on the screen has ever seen a second branch.** Forking has no UI, which is
|
||||||
`limits.check_bundle_lists` has to learn about branches too.
|
why the v1-export gap could be left open through SP5 — SP7 is the subphase that makes
|
||||||
- **Weigh new columns in bytes.** `actions` is already the table that fills the disk.
|
a fork reachable, so it is also the one that makes every branch-shaped bug reachable.
|
||||||
`tests/test_egress.py` has byte ceilings — they will tell you.
|
|
||||||
|
|
||||||
And one known cost, not a bug: the two memory marks are a single pair on the adventure,
|
And one known cost, not a bug: the two memory marks are a single pair on the adventure,
|
||||||
so switching branches makes the mark on the branch being left unreadable from the new one
|
so switching branches makes the mark on the branch being left unreadable from the new one
|
||||||
@@ -125,6 +129,44 @@ drive it before rewriting it.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## What happened on 2026-08-18, part three — the tree, SP6
|
||||||
|
|
||||||
|
The backup learned the tree. `ai-dnd-adventure-v2` carries branches, the fork point each
|
||||||
|
one left its parent at, which attempt at every turn is the story, and what each node left
|
||||||
|
behind — that last one because it is what a branch switch puts back, and a bundle that
|
||||||
|
imported a tree nobody could switch inside would be a backup of the wrong thing. The v1
|
||||||
|
*reader* stays: those files are already on disk. **381 tests green**, 16 of them new in
|
||||||
|
`test_bundle_v2.py`. Branch `sp6-bundle-v2`, no migration, no vacuum.
|
||||||
|
|
||||||
|
The gap SP5 left is closed — a forked adventure now has an honest export — so the
|
||||||
|
ordering constraint that has governed the last two subphases is discharged, and SP7 is
|
||||||
|
free.
|
||||||
|
|
||||||
|
Three things to carry forward:
|
||||||
|
|
||||||
|
- **Carry what was chosen, never what is derived.** The head branch, the fork points, the
|
||||||
|
live flags and the anchors are decisions, and they are in the file. `lineage`, the head
|
||||||
|
depth, the legacy `index` and the variant ordinals are computed from those, so they are
|
||||||
|
rebuilt on import instead. A bundle is a text file anybody can edit, and a derived field
|
||||||
|
shipped beside its source is a chance for the file to contradict itself in a way no read
|
||||||
|
reports. It also turns "is the round trip lossless?" into a testable question: every
|
||||||
|
omitted field is reconstructed, and the tests assert the reconstruction.
|
||||||
|
- **Check the shape before creating the row.** A node naming a branch the file does not
|
||||||
|
list is a 400 raised by a pure function, not a half-written adventure. The failure this
|
||||||
|
phase exists to end is a story that goes quiet, and a half-applied import is exactly
|
||||||
|
that.
|
||||||
|
- **The tree is free; the outcomes are what cost.** On the 600-action fixture the bundle
|
||||||
|
goes from 587 kB to 911 kB, and *all* of it is `state_after`/`world_state_after` at
|
||||||
|
489 B a node — the coordinates themselves save 57.5 B a node against v1's
|
||||||
|
turn-and-variants shape. Twenty forks add 660 B. 4.3 % of the import body cap at
|
||||||
|
production's longest adventure.
|
||||||
|
|
||||||
|
Paid for once, and worth not repeating: a new measuring script imported `app` before
|
||||||
|
`tools.stress_session`, which is what redirects `AIDND_DB_PATH` at a throwaway file. It
|
||||||
|
failed by *working* — the first run seeded a synthetic user into the local `data.db` and
|
||||||
|
printed good numbers; the second tripped over the unique email. **A harness that decides
|
||||||
|
where the database lives has to be imported before anything that reads it.**
|
||||||
|
|
||||||
## What happened on 2026-08-18, part two — the tree, SP4 and SP5
|
## What happened on 2026-08-18, part two — the tree, SP4 and SP5
|
||||||
|
|
||||||
A retry stopped rewriting a row, and a story learned to go two ways at once.
|
A retry stopped rewriting a row, and a story learned to go two ways at once.
|
||||||
|
|||||||
Reference in New Issue
Block a user