Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5: genre-neutral authoritative narrative state, with review corrections

Browse Source 
Replaces AI-DnD's RPG relative-delta world state with the genre-neutral typed
narrative state of ADR 010: explicit, absolute, allowlisted events proposed by
the model, validated by the application, applied to one authoritative document,
and snapshotted per position so restore stays a row read.

This commit includes the corrective pass that followed the independent review
in planning/reports/M5-IMPLEMENTATION-REPORT.md. The invariant it exists to
hold is:

    visible active transcript position == stored head == authoritative state

Narrator editing (D10, STORY-BRANCH-SEMANTICS §§14-15)

  A narrator edit no longer rewrites a row. It returns to the state before the
  turn, takes the reader's exact text as the accepted narration, re-derives the
  state that text implies, and becomes a new active continuation — while the
  original narration keeps its words, its live flag and its whole future as
  retained history. At the tip the correction is another take; with story below
  it, it forks. No new history machinery: this is the existing fork/take/head
  path with the reader's text in place of a generated reply. The §14A refusal
  is therefore gone for narrator turns, and remains only for player input.

Pre-M5 positions

  Migration 88 backfills the empty narrative document onto every action written
  before M5, and a missing snapshot now restores the empty document instead of
  leaving the previous position's state standing. Restoring to an old Save
  Point no longer leaves a later position's entities and facts on screen.

Narrator context

  Replayed history carries prose only; the machine-readable block is no longer
  reconstructed into past turns, where it contradicted the authoritative state
  in the same prompt. A fact withdrawn by a manual correction is now named as
  no longer true, with the reader's reason, rather than silently dropped.

Also

  - state_changes joins the action-list bulk read, removing one query per row.
  - Extraction takes only the application's own protocol payload: an ordinary
    ```json or ```python block in a story survives, and a mangled proposal
    still does not reach the reader.

Planning: ADR 013 records the authoritative document shape; §§14-15/14A, D10,
C04 and BUILD-MILESTONES are updated to describe what exists. Debt is recorded
against M8 (scenario editor UX) and M9 (export of the audit trail).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2
This commit is contained in:
JesseMarkowitz
2026-09-05 07:01:50 -04:00
co-authored by Claude Opus 5
parent 62a997f364
commit b7005e6fdd
57 changed files with 7257 additions and 474 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/routers/adventures/__init__.py
+2
View File
@@ -14,6 +14,7 @@ Read the modules in this order to follow a turn from end to end:
takes retries and the attempts that collect at one coordinate
branches where a story splits
checkpoints Save Points: durable names for positions the head can return to
state the authoritative narrative state, and correcting it by hand
What this package re-exports, and what it deliberately does not:
@@ -33,6 +34,7 @@ from . import ( # noqa: F401
takes,
branches,
checkpoints,
state,
bundle_io,
refresh,
insights,
backend/app/routers/adventures/actions.py
+179 -19
View File
@@ -8,7 +8,8 @@ coordinate through `nodes.delete_turn`.
from fastapi import Depends, HTTPException
from sqlalchemy.orm import Session
from ... import attempts, head, models, schemas, tree
from ... import attempts, head, memorybank, models, narrative, schemas, tree
from ...context import cursors, lineage
from ...database import get_db
from . import turns
@@ -60,19 +61,20 @@ def update_action(
action = db.get(models.Action, action_id)
if action is None or action.adventure_id != adventure_id:
raise HTTPException(404, "Action not found")
# An edit rewrites this row and re-evaluates nothing after it, which is what
# makes it a correction rather than a new continuation. That is safe while
# everything descending from the row is on screen, and unsafe the moment
# something descends from it that is not — an undone future, or a line a
# divergence left behind. The reader cannot see that story, so they cannot
# see what their correction has just contradicted (M3).
#
# Refusing is the whole of the fix, deliberately. Making the edit fork, so
# that the original text and its future stay whole, is
# `STORY-BRANCH-SEMANTICS.md` §14-15 — and §15 requires re-evaluating the
# state the edited prose implies, which is M5's extraction pass. Neither is
# started here. What is closed is the one case where the application could
# produce retained history that silently disagrees with itself.
# A narrator turn the story is currently telling is corrected through the
# §§14-15 path, which forks. A take the story is *not* telling is a
# different thing: it has no continuation of its own — keeping one is what
# forking is for — so correcting its words cannot contradict anything, and
# it stays the plain in-place edit it has always been.
if action.type == "ai" and lineage.path_of(db, adventure).contains(action):
return _edit_narration(db, adventure, action, payload.text)
# A player's own words. Editing one rewrites this row and re-evaluates
# nothing after it, which is what makes it a correction rather than a new
# continuation. That is safe while everything descending from the row is on
# screen, and unsafe the moment something descends from it that is not — an
# undone future, or a line a divergence left behind. The reader cannot see
# that story, so they cannot see what their correction has just contradicted
# (M3, `STORY-BRANCH-SEMANTICS.md` §13).
if head.displaced_history_under(db, adventure, action):
raise HTTPException(
400,
@@ -81,14 +83,172 @@ def update_action(
"the words that story was written from. Redo to bring it back "
"first, or play the turn again to start a new line from here.",
)
# One row holds one text. Nothing mirrors it now, so nothing else has to be
# updated. The edit used to have to be written into the live variant entry
# as well, or paging away and back reverted it.
action.text = payload.text
db.commit()
turns.acquire_turn_lock(adventure_id)
try:
action.text = payload.text
db.commit()
finally:
turns._active_turns.discard(adventure_id)
db.refresh(action)
return action
def _edit_narration(
db: Session, adventure: models.Adventure, action: models.Action, text: str
) -> models.Action:
"""Corrects narrator prose by hand, per `STORY-BRANCH-SEMANTICS.md` §§14-15.
A narrator edit is not a rewrite of a row. It is a continuation written from
the same place the original was written from, using the reader's words
instead of the model's. §15 lists what that has to mean, and each clause
maps to a step below:
1. return to the state immediately before the edited narration — the
preceding node's snapshot, one row read;
2. treat the edited text as the accepted narrator output — it is stored
verbatim, with only the protocol block stripped, and no model is called;
3. re-evaluate the state that output implies — the normal M5 extraction and
validation path, run against that starting state;
4. create a new active continuation — a new node, and the head on it;
5. retain the original narration and its future as disposable history —
nothing on the old line is written to at all.
The M5 review found the previous implementation failing 3-5 together: it
edited the row in place and rewound the campaign's live state to that
position while the head stayed at the tip, so the reader saw a full
transcript over a state document describing an earlier moment, and the
snapshots below the edit still described prose that no longer existed
(Finding 1). Forking is what fixes it, and no new machinery is needed to
fork — this function is the ⑂ path from `takes.py` with the reader's text in
place of a generated one.
Two shapes, chosen by whether anything was written after the turn:
at the tip the attempts of the turn are still leaves, so the
correction joins them as a sibling take and the
original is retained beside it in the pager;
anything below the story after the turn was written as a
continuation of the words that are there now, so it
keeps them: the correction leaves the path just
before the turn and the old line keeps its node, its
future, and its live flag.
The §14A refusal is gone from this path, and this is what replaces it. It
refused an in-place edit under an off-screen future because the edit would
silently change the words that story was written from. Nothing is changed
now — the off-screen future keeps the exact narration it descends from — so
the case that had to be refused is simply handled.
"""
if action.depth is None:
raise HTTPException(400, "That turn is not on the story you are reading.")
turns.acquire_turn_lock(adventure.id)
try:
# §15.2. The reader's words are the narration; a block they pasted in is
# protocol and is stripped before storage, exactly as a model's is.
prose, parsed, raw_block = narrative.extract.split(text)
# §15.1. Not the campaign's current state — the state this turn was
# played from. One row read, not a replay (ADR 012).
before = attempts.preceding(db, adventure, action)
starting_state = (
narrative.model.normalize(before.narrative_state_after)
if before is not None and isinstance(before.narrative_state_after, dict)
else narrative.model.empty()
)
corrected = models.Action(
adventure_id=adventure.id,
type="ai",
text=prose,
# No model was called, so there is no prompt to show for this node.
# In the sibling case the turn's assembled prompt moves to whichever
# attempt is live, which is what the Insights viewer reads; in the
# forked case the original keeps it, because the original is still
# the live node of its own line.
context_snapshot=None,
)
tip = db_tip(db, adventure)
# A turn the head rests on is not a leaf while a retained future
# descends from it, and `db_tip` reads the capped path and cannot see
# that future. Ask the head module as well (M3).
at_the_tip = (
tip is not None
and tip.id == action.id
and not head.behind_tip(db, adventure)
)
if at_the_tip:
# §15.4-5 as a take. The original stays at this coordinate as a
# prior attempt, reachable through the pager, and the correction
# becomes the one the story tells.
attempts.hand_over_the_prompt(action, corrected)
attempts.add_attempt(db, adventure, action, corrected)
db.add(corrected)
# The words at this coordinate changed, so anything derived from
# them no longer describes the story.
memorybank.forget_node(db, adventure, action)
cursors.rewind_all(adventure, action.branch_id, action.depth - 1)
db.flush()
else:
# §15.4-5 as a branch. Nothing on the departed line is written to:
# the original node keeps its text, its live flag and every turn
# that was played after it.
departed = lineage.branch_of(db, adventure)
tree.branch_at(db, adventure, action.depth - 1)
if departed is not None:
head.mark_superseded(departed, action.depth - 1)
tree.place_action(db, adventure, corrected)
db.add(corrected)
db.flush()
# §15.3. The same validation path a generated turn takes, so a hand
# -typed event is no more trusted than a model's: the allowlist, the
# schema, the references and the canon all still apply.
review = narrative.validate.review(
parsed if parsed is not None else {"events": []},
starting_state,
narrative.store.canon_of(adventure),
)
# `record` writes the events and the provenance. Its returned document
# applies them to the campaign's *current* state, which is not what an
# edit derives from, so the document this node leaves behind is computed
# from the turn's own starting point below.
narrative.store.record(
db, adventure,
review=review,
raw_block=raw_block,
parsed=parsed,
action=corrected,
branch_id=corrected.branch_id,
depth=corrected.depth,
source="narrator_edit",
)
new_state = narrative.apply.apply_events(
starting_state, review.accepted,
branch_id=corrected.branch_id, depth=corrected.depth,
source="narrator_edit",
)
corrected.state_changes = {
"accepted": review.accepted,
"rejected": [r.as_dict() for r in review.rejected],
"summary": narrative.apply.diff(starting_state, new_state),
}
# The head is on the corrected node, so the campaign's live state is
# what that node leaves behind, and the node's own snapshot is the same
# document. That equality is the invariant the review found broken:
# visible position == head == authoritative state.
narrative.store.set_current(adventure, new_state)
attempts.snapshot_outcome(adventure, corrected)
adventure.updated_at = models.utcnow()
db.commit()
finally:
turns._active_turns.discard(adventure.id)
db.refresh(corrected)
return corrected
@router.delete("/{adventure_id}/actions/{action_id}", status_code=204)
def delete_action(
adventure_id: int,
backend/app/routers/adventures/paging.py
+7
View File
@@ -28,6 +28,13 @@ ACTION_LIST_COLUMNS = (
models.Action.text,
models.Action.reasoning,
models.Action.world_delta,
# M5: `world_delta`'s counterpart, and listed for exactly the reason stated
# above it. `ActionOut.state_summary` reads it for every row on the page, so
# leaving it out of the bulk read cost one lazy load per action — 51 rows
# bought 53 queries (M5 review, Finding 2). It holds one turn's accepted
# events and its summary lines, the same order of size as `world_delta`, not
# the deferred snapshot.
models.Action.state_changes,
# SP9: the pager's key. If `parent_id` were deferred, every row on the page
# would cost a lazy load, which is the cost `load_only` is here to prevent.
# `branch_id` is listed for the same reason. The pager reads it to tell a
backend/app/routers/adventures/state.py
+173
View File
@@ -0,0 +1,173 @@
"""M5: reading the authoritative narrative state, and correcting it by hand.
Three endpoints, and the split between them is the point:
GET /state what the campaign currently believes
POST /state/corrections the user overruling it (C04)
GET /state/events how it came to believe that (§8's audit)
The browser reads the first and writes the second. It never writes state
directly — `BUILD-MILESTONES.md` M5 is explicit that the browser is a
presentation layer and must not become the owner of state — so a correction goes
through the same validator, the same applier and the same event log as a
narration does. The only difference is the `source` recorded on it, and that
difference is the whole of C04's audit requirement.
The state returned here is always the state at the **active head**, because that
is what `adventure.narrative_state` holds: head movement restores it from the
destination node's snapshot, so an undone story is described by what was true
then rather than by what the campaign later became.
"""
from fastapi import Depends, HTTPException
from sqlalchemy.orm import Session
from ... import head, models, narrative, schemas
from ...database import get_db
from . import turns
from .deps import current_adventure, router
@router.get("/{adventure_id}/state", response_model=schemas.NarrativeStateOut)
def read_state(
db: Session = Depends(get_db),
adventure: models.Adventure = Depends(current_adventure),
):
"""The authoritative state at the position the story is being read at.
Grouped for display, with only the categories that actually hold something —
a heading with no rows under it tells a reader nothing, and the panel should
not have to decide what to hide.
"""
state = narrative.store.current(adventure)
view = narrative.render.for_inspector(state)
return schemas.NarrativeStateOut(
groups=[schemas.StateGroup(**group) for group in view["groups"]],
empty=view["empty"],
# The raw document, for the correction form to name a key with and for a
# test to assert on without parsing prose.
document=state,
)
@router.post(
"/{adventure_id}/state/corrections",
response_model=schemas.NarrativeStateOut,
status_code=201,
)
def correct_state(
adventure_id: int,
payload: schemas.StateCorrection,
db: Session = Depends(get_db),
adventure: models.Adventure = Depends(current_adventure),
):
"""Applies the user's own state events, as an explicit correction.
C04. The user says "Mara never learned where the silver key was found", and
that becomes authoritative for everything that follows — while the transcript
stays exactly as it was written. Correcting the world is not editing the
story, and conflating them would rewrite prose the user did not ask to
change.
The events go through the **same validator** as a narration's. A user is
trusted more than a model, but not with references that do not resolve or
with an event type the application does not implement: a typo should be a
clear refusal, not a corrupt document. What being trusted buys is authority —
the resulting facts carry `manual_correction`, which outranks
`accepted_story` when the two disagree, and which the prompt renders so the
model is told the reader overruled it.
Held under the turn lock, for the reason creating a Save Point is: this reads
the head and writes a snapshot onto the node the head rests on, and a turn in
flight is about to move both.
"""
if not payload.events:
raise HTTPException(400, "A correction needs at least one change.")
turns.acquire_turn_lock(adventure_id)
try:
state = narrative.store.current(adventure)
review = narrative.validate.review(
{"events": [event.model_dump(exclude_none=True) for event in payload.events]},
state,
narrative.store.canon_of(adventure),
)
if not review.accepted:
raise HTTPException(400, _refusal_message(review))
node = head.node_at(db, adventure, adventure.head_depth)
new_state, _proposal = narrative.store.record(
db, adventure,
review=review,
raw_block=payload.note or "",
parsed={"events": [e.model_dump(exclude_none=True) for e in payload.events]},
action=node,
branch_id=node.branch_id if node is not None else adventure.head_branch_id,
depth=node.depth if node is not None else adventure.head_depth,
source="manual_correction",
)
narrative.store.set_current(adventure, new_state)
# The correction belongs to the position it was made at, so a later Undo
# past it drops it and a Redo back brings it again — the same rule every
# other state change follows. Without re-snapshotting the node, the
# correction would survive a head movement that stepped over it.
if node is not None:
node.narrative_state_after = new_state
adventure.updated_at = models.utcnow()
db.commit()
db.refresh(adventure)
finally:
turns._active_turns.discard(adventure_id)
view = narrative.render.for_inspector(narrative.store.current(adventure))
return schemas.NarrativeStateOut(
groups=[schemas.StateGroup(**group) for group in view["groups"]],
empty=view["empty"],
document=narrative.store.current(adventure),
)
def _refusal_message(review) -> str:
"""Why a correction was refused, in the words the user needs.
The first rejection's detail, because a correction is usually one or two
events and a wall of them helps nobody.
"""
if review.rejected:
first = review.rejected[0]
return f"That correction can't be applied — {first.detail or first.reason}."
return "That correction can't be applied."
@router.get("/{adventure_id}/state/events", response_model=list[schemas.StateEventOut])
def read_state_events(
limit: int = 100,
db: Session = Depends(get_db),
adventure: models.Adventure = Depends(current_adventure),
):
"""The accepted state changes, newest first: §8's audit trail.
What changed, which turn caused it, whether the model or the user asserted
it, and what the value was before. Bounded by default — this is an audit
view, and an unbounded read of a long campaign's every event is the query
shape this project keeps a regression test about.
"""
limit = max(1, min(limit, 500))
rows = narrative.store.history(db, adventure, limit=limit)
return [
schemas.StateEventOut(
id=row.id,
action_id=row.action_id,
branch_id=row.branch_id,
depth=row.depth,
turn=(row.depth + 1) if row.depth is not None else None,
sequence=row.sequence,
event_type=row.event_type,
payload=row.payload or {},
before=row.before,
source=row.source,
created_at=row.created_at,
)
for row in rows
]
backend/app/routers/adventures/turns.py
+67 -19
View File
@@ -13,7 +13,8 @@ from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from ... import (
attempts, head, limits, memorybank, models, schemas, tree, worldstate,
attempts, head, limits, memorybank, models, narrative, schemas, tree,
worldstate,
)
from ...context import build_context, cursors
from ...database import get_db
@@ -207,25 +208,46 @@ async def _generate_turn(
yield turn_error(detail)
return
# RPG world state (Phase 12): read the AI's state delta out of the reply,
# apply it through the engine, and strip the block from the displayed text.
# M5: read the typed state proposal out of the reply, validate it, apply
# what survives, and strip the block from the displayed text.
#
# A retry re-runs the same turn, so it is played at that turn's depth. The
# cooldown rules run on a position in the story, and a second attempt at turn
# 12 is still turn 12. This was `retry_of.index`, which held the same number
# until SP4. Depth stays correct once a branch has its own numbering.
# This replaced the Phase 12 relative-delta pipeline. The shape of the turn
# is unchanged — extract, referee, snapshot — because ADR 010 changed the
# protocol, not the lifecycle. What changed is that the referee now works on
# explicit typed events with absolute values, so an accepted proposal cannot
# mean something other than it says.
#
# A retry re-runs the same turn, so it is played at that turn's depth. This
# was `retry_of.index`, which held the same number until SP4. Depth stays
# correct once a branch has its own numbering.
ai_depth = retry_of.depth if retry_of is not None else next_depth(adventure)
stat_schema = adventure.scenario.stat_schema if adventure.scenario else None
if worldstate.has_schema(stat_schema):
text, delta = worldstate.extract_delta(text)
if not text.strip():
yield turn_error("The AI returned only a state update and no story text.")
return
new_world_state, ws_report = worldstate.apply_delta(
adventure.world_state, stat_schema, delta, ai_depth
)
adventure.world_state = new_world_state
snapshot["world_state"] = {"delta": delta, "report": ws_report, "state": new_world_state}
text, parsed, raw_block = narrative.extract.split(text)
if not text.strip():
yield turn_error("The AI returned only a state update and no story text.")
return
review = narrative.validate.review(
parsed if parsed is not None else {"events": []},
narrative.store.current(adventure),
narrative.store.canon_of(adventure),
)
# Held until the action exists, because a proposal record names the node
# whose narration produced it and the node has no id yet. Everything lands
# in the single commit below (L01).
# The coordinate is read off the node after it is placed, not guessed here:
# `tree.place_action` assigns the branch, and a retry inherits the branch of
# the attempt it replaces.
pending_state = {
"review": review,
"parsed": parsed,
"raw_block": raw_block,
"unparseable": parsed is None and bool(raw_block),
}
snapshot["narrative_state"] = {
"accepted": review.accepted,
"rejected": [r.as_dict() for r in review.rejected],
"status": review.status,
}
snapshot["raw_output"] = raw_output
# The cost the endpoint reports for the call, including how much of the
@@ -243,7 +265,6 @@ async def _generate_turn(
context_snapshot=snapshot,
world_delta=world_delta_of(snapshot),
)
attempts.snapshot_outcome(adventure, ai_action)
if retry_of is not None:
attempts.add_attempt(db, adventure, retry_of, ai_action)
db.add(ai_action)
@@ -264,6 +285,33 @@ async def _generate_turn(
else:
tree.place_action(db, adventure, ai_action)
db.add(ai_action)
db.flush()
# The state lands after the node exists and before the one commit, so the
# narration, the head, the accepted events, the provenance and the snapshot
# are one transaction. L01 forbids any window in which a turn looks accepted
# while its state is half-written, and the cheapest guarantee is to have a
# single commit rather than two that could get out of step.
new_state, _proposal = narrative.store.record(
db, adventure,
review=pending_state["review"],
raw_block=pending_state["raw_block"],
parsed=pending_state["parsed"],
action=ai_action,
branch_id=ai_action.branch_id,
depth=ai_action.depth,
model_name=settings.model or "",
source="accepted_story",
)
if pending_state["unparseable"]:
_proposal.status = "unparseable"
before_state = narrative.store.current(adventure)
narrative.store.set_current(adventure, new_state)
ai_action.state_changes = {
"accepted": pending_state["review"].accepted,
"rejected": [r.as_dict() for r in pending_state["review"].rejected],
"summary": narrative.apply.diff(before_state, new_state),
}
attempts.snapshot_outcome(adventure, ai_action)
adventure.updated_at = models.utcnow()
db.commit()
db.refresh(ai_action)