Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5: genre-neutral authoritative narrative state, with review corrections

Browse Source 
Replaces AI-DnD's RPG relative-delta world state with the genre-neutral typed
narrative state of ADR 010: explicit, absolute, allowlisted events proposed by
the model, validated by the application, applied to one authoritative document,
and snapshotted per position so restore stays a row read.

This commit includes the corrective pass that followed the independent review
in planning/reports/M5-IMPLEMENTATION-REPORT.md. The invariant it exists to
hold is:

    visible active transcript position == stored head == authoritative state

Narrator editing (D10, STORY-BRANCH-SEMANTICS §§14-15)

  A narrator edit no longer rewrites a row. It returns to the state before the
  turn, takes the reader's exact text as the accepted narration, re-derives the
  state that text implies, and becomes a new active continuation — while the
  original narration keeps its words, its live flag and its whole future as
  retained history. At the tip the correction is another take; with story below
  it, it forks. No new history machinery: this is the existing fork/take/head
  path with the reader's text in place of a generated reply. The §14A refusal
  is therefore gone for narrator turns, and remains only for player input.

Pre-M5 positions

  Migration 88 backfills the empty narrative document onto every action written
  before M5, and a missing snapshot now restores the empty document instead of
  leaving the previous position's state standing. Restoring to an old Save
  Point no longer leaves a later position's entities and facts on screen.

Narrator context

  Replayed history carries prose only; the machine-readable block is no longer
  reconstructed into past turns, where it contradicted the authoritative state
  in the same prompt. A fact withdrawn by a manual correction is now named as
  no longer true, with the reader's reason, rather than silently dropped.

Also

  - state_changes joins the action-list bulk read, removing one query per row.
  - Extraction takes only the application's own protocol payload: an ordinary
    ```json or ```python block in a story survives, and a mangled proposal
    still does not reach the reader.

Planning: ADR 013 records the authoritative document shape; §§14-15/14A, D10,
C04 and BUILD-MILESTONES are updated to describe what exists. Debt is recorded
against M8 (scenario editor UX) and M9 (export of the audit trail).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2
This commit is contained in:
JesseMarkowitz
2026-09-05 07:01:50 -04:00
co-authored by Claude Opus 5
parent 62a997f364
commit b7005e6fdd
57 changed files with 7257 additions and 474 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/narrative/__init__.py
+23
View File
@@ -0,0 +1,23 @@
"""M5: the authoritative narrative state.
Genre-neutral state (ADR 006), written by explicit typed events with absolute
values (ADR 010), owned by the application rather than the model (ADR 003), and
recovered per story position rather than replayed (ADR 012 and
`TECHNICAL-DESIGN.md` §10.4).
extract.split(reply) prose out, proposal out, block kept for audit
|
validate.review(...) allowlist, schema, references, semantics
|
apply.apply_events(...) accepted events -> a new state document
|
store.commit_proposal(...) events, provenance and snapshot, in one transaction
`model.py` says what a state document is. `render.py` shows it to the model and
to the reader. Nothing outside this package writes authoritative state, and
nothing inside it executes anything a proposal names.
"""
from . import apply, events, extract, model, render, store, validate # noqa: F401
__all__ = ["apply", "events", "extract", "model", "render", "store", "validate"]
backend/app/narrative/apply.py
+272
View File
@@ -0,0 +1,272 @@
"""M5: turning accepted events into a new state document.
Pure and total. Every function here takes a document and returns a new one; none
touches the database, and none can fail on an event `validate.review` accepted —
validation is the only place an event is refused, so this module never has to
decide anything twice.
The dispatch is an explicit `if/elif` chain over `events.SPECS`, not a lookup
table keyed on the payload. The difference matters: a table maps a string a model
supplied to a callable, and the security of that arrangement rests entirely on
the allowlist being correct. A chain of literal comparisons cannot be steered by
a payload at all, whatever the allowlist does.
"""
from __future__ import annotations
import copy
from . import model
def apply_events(
state: dict,
accepted: list[dict],
*,
branch_id: int | None = None,
depth: int | None = None,
source: str = "accepted_story",
) -> dict:
"""Returns `state` with every event in `accepted` applied, in order.
The input document is never mutated: head movement stores snapshots by
reference in places, and a mutation here would edit the past.
`branch_id`/`depth` stamp facts and relationships with where they were
established, which is what makes the audit trail answer "which turn caused
this" without a join. `source` records whether the campaign, the story or
the user established it — C04's provenance, carried on the value itself.
"""
document = model.normalize(state)
for event in accepted:
_apply_one(document, event, branch_id, depth, source)
return document
def _apply_one(state: dict, event: dict, branch_id, depth, source: str) -> None:
kind = event["type"]
if kind == "create_entity":
state["entities"][event["entity"]] = model.new_entity(
type=event.get("entity_type") or "other",
name=event["name"],
description=event.get("description") or "",
aliases=event.get("aliases") or [],
)
elif kind == "set_entity_status":
_entity(state, event["entity"])["status"] = event["status"]
elif kind == "set_entity_attribute":
# Absolute assignment. The whole reason ADR 010 exists.
_entity(state, event["entity"])["attributes"][event["attribute"]] = event["value"]
elif kind == "set_entity_conditions":
_entity(state, event["entity"])["conditions"] = list(event["conditions"])
elif kind == "set_current_location":
_entity(state, event["entity"])["location"] = event["location"]
elif kind == "set_possession":
state["possessions"][event["item"]] = event["owner"]
elif kind == "clear_possession":
state["possessions"].pop(event["item"], None)
elif kind == "add_fact":
state["facts"].append({
"id": event.get("fact_id") or _fact_id(state),
"subject": event.get("subject"),
"predicate": event["predicate"],
"object": event.get("object"),
"value": event.get("value"),
"authority": _authority(source),
"source": source,
"status": "active",
"branch_id": branch_id,
"depth": depth,
})
elif kind == "invalidate_fact":
for fact in state["facts"]:
if fact.get("id") == event["fact_id"]:
# Withdrawn, not removed: C04 needs the record of what the
# campaign used to believe, and a deleted row audits nothing.
fact["status"] = "invalidated"
fact["invalidated_by"] = source
fact["invalidated_at"] = {"branch_id": branch_id, "depth": depth}
if event.get("reason"):
fact["invalidated_reason"] = event["reason"]
elif kind == "add_relationship":
state["relationships"].append({
"id": _relationship_id(state),
"source": event["source"],
"target": event["target"],
"type": event["relationship"],
"description": event.get("description") or "",
"status": "active",
"established_by": source,
"branch_id": branch_id,
"depth": depth,
})
elif kind == "end_relationship":
for relationship in state["relationships"]:
if (
relationship.get("source") == event["source"]
and relationship.get("target") == event["target"]
and relationship.get("type") == event["relationship"]
and relationship.get("status") == "active"
):
relationship["status"] = "ended"
relationship["ended_at"] = {"branch_id": branch_id, "depth": depth}
elif kind == "open_story_thread":
state["threads"][event["thread"]] = {
"title": event["title"],
"description": event.get("description") or "",
"status": "open",
"opened_at": {"branch_id": branch_id, "depth": depth},
}
elif kind == "resolve_story_thread":
thread = state["threads"].get(event["thread"])
if isinstance(thread, dict):
thread["status"] = "resolved"
thread["resolution"] = event.get("resolution") or ""
thread["resolved_at"] = {"branch_id": branch_id, "depth": depth}
elif kind == "set_scene":
scene = dict(state.get("scene") or {})
if "summary" in event:
scene["summary"] = event["summary"]
if "location" in event:
scene["location"] = event["location"]
if "present" in event:
scene["present"] = list(event["present"] or [])
scene["at"] = {"branch_id": branch_id, "depth": depth}
state["scene"] = scene
# No `else`. Every allowed type is handled above, and an unhandled one
# cannot arrive: `validate.review` refuses anything outside the allowlist,
# and the allowlist is this list. A silent fall-through would be the one way
# an event could appear accepted and do nothing.
def _entity(state: dict, key: str) -> dict:
"""The entity record for `key`, created bare if a snapshot lost it.
Validation guarantees the entity exists, so this is a repair path for a
hand-edited or partially imported document rather than a normal branch. A
bare record is better than a KeyError: the story is still readable, and the
inspector shows an entity with nothing known about it, which is true.
"""
entities = state["entities"]
found = entities.get(key)
if not isinstance(found, dict):
found = model.new_entity(name=key)
entities[key] = found
found.setdefault("attributes", {})
found.setdefault("conditions", [])
return found
def _authority(source: str) -> str:
"""Which authority band a source's assertions carry.
A user's correction outranks the story (C04); the story outranks a guess.
`DATA-MODEL.md` §14 orders the bands, and this is the mapping into them.
"""
if source == "manual_correction":
return "manual_correction"
if source == "campaign_canon":
return "campaign_canon"
return "accepted_story"
def _fact_id(state: dict) -> str:
return f"f{len(state['facts']) + 1}"
def _relationship_id(state: dict) -> str:
return f"r{len(state['relationships']) + 1}"
def diff(before: dict, after: dict) -> list[str]:
"""A short human-readable list of what changed between two documents.
Shown under a turn the way the world-state chip used to be, and recorded on
the node for the bulk read. Text rather than structure, because its only
consumer is a person reading "Aldric now holds the silver key".
"""
before = model.normalize(before)
after = model.normalize(after)
lines: list[str] = []
for key, entity in after["entities"].items():
was = before["entities"].get(key)
name = model.entity_name(after, key)
if was is None:
lines.append(f"{name} enters the story")
continue
if was.get("status") != entity.get("status"):
lines.append(f"{name} is now {entity.get('status')}")
if was.get("location") != entity.get("location") and entity.get("location"):
lines.append(f"{name} is at {model.entity_name(after, entity['location'])}")
if sorted(was.get("conditions") or []) != sorted(entity.get("conditions") or []):
now = ", ".join(entity.get("conditions") or []) or "nothing"
lines.append(f"{name}: {now}")
for attribute, value in (entity.get("attributes") or {}).items():
if (was.get("attributes") or {}).get(attribute) != value:
lines.append(f"{name} {attribute} = {value}")
for item, owner in after["possessions"].items():
if before["possessions"].get(item) != owner:
lines.append(
f"{model.entity_name(after, item)} → {model.entity_name(after, owner)}"
)
for item in before["possessions"]:
if item not in after["possessions"]:
lines.append(f"{model.entity_name(after, item)} is held by nobody")
known = {f.get("id") for f in before["facts"]}
for fact in after["facts"]:
if fact.get("id") not in known:
lines.append(f"fact: {_fact_text(after, fact)}")
was_active = {f["id"] for f in model.active_facts(before)}
for fact in before["facts"]:
if fact.get("id") in was_active and fact.get("id") not in {
f["id"] for f in model.active_facts(after)
}:
lines.append(f"withdrawn: {_fact_text(after, fact)}")
known = {r.get("id") for r in before["relationships"]}
for relationship in after["relationships"]:
if relationship.get("id") not in known:
lines.append(
f"{model.entity_name(after, relationship['source'])} "
f"{relationship['type']} "
f"{model.entity_name(after, relationship['target'])}"
)
for key, thread in after["threads"].items():
was = before["threads"].get(key)
if was is None:
lines.append(f"opened: {thread.get('title', key)}")
elif was.get("status") != thread.get("status"):
lines.append(f"{thread.get('status')}: {thread.get('title', key)}")
return lines
def _fact_text(state: dict, fact: dict) -> str:
parts = []
if fact.get("subject"):
parts.append(model.entity_name(state, fact["subject"]))
parts.append(str(fact.get("predicate", "")))
if fact.get("object"):
parts.append(model.entity_name(state, fact["object"]))
if fact.get("value") is not None:
parts.append(str(fact["value"]))
return " ".join(p for p in parts if p)
backend/app/narrative/events.py
+194
View File
@@ -0,0 +1,194 @@
"""M5: the typed event vocabulary, and the allowlist that bounds it.
ADR 010 replaced AI-DnD's relative-delta protocol because the ambiguity was
architectural: a number in a delta field is syntactically legal whether the
model meant "add 50" or "set to 50", and no validator can tell which. Every
event here therefore states its operation in its `type`, and every value it
carries is **absolute**. There is no event whose meaning depends on a prompt
instruction having been followed.
## The allowlist is a security boundary, not a convenience
Model output is untrusted input (`SECURITY-THREAT-MODEL.md`), and this table is
the entire set of things a model may cause to happen. H05's
`{"event_type": "execute_shell", ...}` is refused here — not because "shell" is
recognised and blocked, but because it is not in `SPECS`, and nothing outside
`SPECS` is dispatched. There is no fallback branch, no generic handler and no
name-to-callable lookup that a payload could steer.
Adding an event means adding a spec here and a case in `apply.py`. Nothing else
in the application can widen the vocabulary, which is what keeps
"state extraction" from drifting into "tool execution".
## Shape of a spec
required fields that must be present and non-empty
optional fields that may be present
refs fields naming an entity that must already exist
creates the field naming an entity this event may bring into being
`refs` is what `validate.py` uses for referential integrity, and `creates` is
the deliberate exception: exactly one event type may introduce an entity, so a
typo in any other event surfaces as an unknown reference rather than silently
creating a second, empty Mara.
"""
from __future__ import annotations
# Field types the schema layer enforces. Kept deliberately small: a narrative
# state event carries names, labels and plain values, and nothing here needs a
# nested structure a model could hide something inside.
TEXT = "text"
KEY = "key" # an entity/thread identifier: a slug the campaign chose
VALUE = "value" # a JSON scalar — str, int, float, bool or None
LABELS = "labels" # a list of short strings
#: The whole vocabulary. Nothing outside this mapping is dispatched, ever.
SPECS: dict[str, dict] = {
"create_entity": {
"required": {"entity": KEY, "name": TEXT},
"optional": {"entity_type": TEXT, "description": TEXT, "aliases": LABELS},
"refs": (),
"creates": "entity",
"summary": "brings a person, place, thing or group into the story",
},
"set_entity_status": {
"required": {"entity": KEY, "status": TEXT},
"optional": {},
"refs": ("entity",),
"creates": None,
"summary": "sets whether an entity is active, gone, destroyed …",
},
"set_entity_attribute": {
# The one numeric-capable event, and it is an assignment. ADR 010's
# `set_value`: the operation is in the name, so a value of 50 can only
# mean fifty. An `increment_value` could be added later without
# ambiguity, because it would be a different `type`.
"required": {"entity": KEY, "attribute": TEXT, "value": VALUE},
"optional": {},
"refs": ("entity",),
"creates": None,
"summary": "sets a named value on an entity, absolutely",
},
"set_entity_conditions": {
# Absolute too: the full set replaces the old one. "Add a condition"
# would need the current set to be known by the model, which is exactly
# the assumption that made deltas unreliable.
"required": {"entity": KEY, "conditions": LABELS},
"optional": {},
"refs": ("entity",),
"creates": None,
"summary": "replaces the conditions an entity is under",
},
"set_current_location": {
"required": {"entity": KEY, "location": KEY},
"optional": {},
"refs": ("entity", "location"),
"creates": None,
"summary": "moves an entity to a location",
},
"set_possession": {
"required": {"item": KEY, "owner": KEY},
"optional": {},
"refs": ("item", "owner"),
"creates": None,
"summary": "gives an item to an owner",
},
"clear_possession": {
"required": {"item": KEY},
"optional": {},
"refs": ("item",),
"creates": None,
"summary": "leaves an item held by nobody",
},
"add_fact": {
"required": {"predicate": TEXT},
"optional": {
"subject": KEY, "object": KEY, "value": VALUE, "fact_id": TEXT,
},
# Only the subject is checked as an entity. The *object* of a fact is
# routinely not one — "Mara knows where the key was found" has another
# fact as its object, and C03 needs exactly that — so it is checked
# against entities *and* known facts in `validate._check`. Requiring an
# entity here would make the knowledge distinction C03 asks for
# unrepresentable.
"refs": ("subject",),
"creates": None,
"summary": "asserts something about the world",
},
"invalidate_fact": {
"required": {"fact_id": TEXT},
"optional": {"reason": TEXT},
"refs": (),
"creates": None,
"summary": "withdraws a fact without deleting the record of it",
},
"add_relationship": {
"required": {"source": KEY, "target": KEY, "relationship": TEXT},
"optional": {"description": TEXT},
"refs": ("source", "target"),
"creates": None,
"summary": "ties two entities together",
},
"end_relationship": {
"required": {"source": KEY, "target": KEY, "relationship": TEXT},
"optional": {},
"refs": ("source", "target"),
"creates": None,
"summary": "ends a tie without erasing that it existed",
},
"open_story_thread": {
"required": {"thread": KEY, "title": TEXT},
"optional": {"description": TEXT},
"refs": (),
"creates": None,
"summary": "records narrative business left open",
},
"resolve_story_thread": {
"required": {"thread": KEY},
"optional": {"resolution": TEXT},
"refs": (),
"creates": None,
"summary": "closes narrative business",
},
"set_scene": {
"required": {},
"optional": {"summary": TEXT, "location": KEY, "present": LABELS},
"refs": ("location",),
"creates": None,
"summary": "records the immediate situation",
},
}
#: The allowlist itself, as a set, for the one question that matters most.
ALLOWED = frozenset(SPECS)
def is_allowed(event_type) -> bool:
"""Whether `event_type` names an event this application will ever apply.
A string is required: a dict, a list or None is not a type, and coercing one
with `str()` would turn a malformed payload into a lookup that might
accidentally succeed.
"""
return isinstance(event_type, str) and event_type in ALLOWED
def spec(event_type: str) -> dict | None:
return SPECS.get(event_type)
def vocabulary_for_prompt() -> str:
"""The event list as the narrator prompt describes it.
Generated from `SPECS` rather than written out beside it, so the model can
never be told about an event the application does not implement — the drift
that would produce proposals rejected for reasons nobody could see.
"""
lines = []
for name, definition in SPECS.items():
fields = list(definition["required"]) + [
f"{field}?" for field in definition["optional"]
]
lines.append(f' {name}({", ".join(fields)}) — {definition["summary"]}')
return "\n".join(lines)
backend/app/narrative/extract.py
+259
View File
@@ -0,0 +1,259 @@
"""M5: getting a typed proposal out of a narration, and keeping it out of the prose.
The model writes the story and, after it, one fenced block of typed events. This
module holds the instruction it is given, the parser that survives the ways a
model gets a format wrong, and the separation that keeps machine-readable output
from reaching the reader.
Two properties matter more than elegance here:
* **The prose must never carry the protocol.** A reader should not see a JSON
block under their story, and a stored narration should not contain one either,
because everything downstream — memory, summaries, export, the transcript —
treats stored text as the story. The block is removed before the text is
stored, not before it is displayed.
* **An unreadable block must not be a failed turn.** A narration the user watched
arrive is worth keeping even when the state block after it is garbage. Parsing
returns "no events" rather than raising, the turn commits with the state
unchanged, and the proposal record keeps the raw output so the failure is
visible in the audit rather than only in a log.
"""
from __future__ import annotations
import json
import re
from . import events
# The block the model is asked to append. Built from the vocabulary rather than
# written beside it, so the instruction cannot describe an event the application
# would then reject (`events.vocabulary_for_prompt`).
EMIT_RULE = (
"After your narration, append a fenced code block labelled `state` containing "
"a JSON object with an \"events\" list, recording what your own narration made "
"true. Treat your narration as authoritative: if you wrote that someone moved, "
"took something, learned something, was hurt, or that a new person or place "
"appeared, record it.\n"
"\n"
"Every value is ABSOLUTE — the new state of things, never a change or a "
"difference. Use only these events:\n"
f"{events.vocabulary_for_prompt()}\n"
"\n"
"Identifiers are short lower-case slugs (mara, silver-key, old-abbey) and must "
"match the ones already in the state you were shown. Introduce a person, place "
"or thing with create_entity before referring to it. If the turn established "
"nothing, send an empty events list.\n"
"Example:\n"
'```state\n'
'{"events": [{"type": "set_possession", "item": "silver-key", "owner": "aldric"},'
' {"type": "set_current_location", "entity": "aldric", "location": "old-abbey"}]}\n'
'```'
)
# Placed last, where recency is strongest, the same way the delta protocol did.
EMIT_REMINDER = (
"[Reminder: end your reply with a ```state block listing the events your "
"narration made true, with absolute values. Send an empty events list if "
"nothing changed.]"
)
# Three patterns, and the difference between them is the whole of this module's
# safety. A story is allowed to contain code, and taking a code block out of
# someone's prose is a worse failure than leaving a stray proposal in it.
#
# `state` is the label the application asks for, so a fence carrying it is ours
# whatever is inside it — including a truncated `{oh no` that no JSON parser
# will take. That block must still leave the prose, and must still be recorded,
# because an unparseable proposal is exactly the failure the audit exists to
# make visible.
_STATE_FENCE_RE = re.compile(
r"```state[^\S\n]*\n?(.*?)```", re.DOTALL | re.IGNORECASE
)
# `json` is *not* our label. Models reach for it anyway, so a ```json fence is
# taken only when what it contains is actually a proposal. A character who
# writes `{"name": "Mara"}` into a terminal keeps their code block (M5 review,
# Finding 6).
_JSON_FENCE_RE = re.compile(
r"```json[^\S\n]*\n?(.*?)```", re.DOTALL | re.IGNORECASE
)
# An *unlabelled* fence is ours on the same terms: it has to be a proposal, not
# merely JSON-shaped.
_BARE_FENCE_RE = re.compile(r"```\s*([\[{].*?[\]}])\s*```", re.DOTALL)
# A bare object hugging the end of the text, for a model that forgets the fence.
_TRAILING_RE = re.compile(r"(\{.*\})\s*$", re.DOTALL)
# An opener with no closing fence. A model that runs out of output tokens
# mid-block leaves one of these, and everything after it is protocol rather than
# story — so the story ends where the opener begins.
#
# Our own label ends the story unconditionally. A dangling ```json fence is
# judged on what follows it, because an unterminated code block in a story is
# still the author's (M5 review, Finding 6).
_DANGLING_STATE_RE = re.compile(r"\n?```state\b.*\Z", re.DOTALL | re.IGNORECASE)
_DANGLING_JSON_RE = re.compile(r"\n?```json\b(.*)\Z", re.DOTALL | re.IGNORECASE)
# The reminder, parroted back. Small local models reproduce the bracketed
# instruction they were given, and it arrives as ordinary prose — no fence, so
# nothing above strips it, and the reader is shown a piece of the prompt.
#
# The bracket is *found* broadly and *judged* narrowly. Merely naming the
# protocol is not enough: a story may end on an aside about a state block, and
# deleting that sentence is the worse failure (M5 review, Finding 6). What marks
# the echo is the shape of the instruction itself — the fence token, the word it
# opens with, or the pair of phrases the reminder uses together.
_TRAILING_BRACKET_RE = re.compile(r"\n?\[([^\]]*)\]\s*\Z", re.DOTALL)
def _is_echoed_instruction(inner: str) -> bool:
"""Whether a trailing bracketed segment is the prompt's own reminder."""
low = inner.lower()
if "```state" in low:
return True
if low.lstrip().startswith("reminder:"):
return True
# The reminder names both; prose about the protocol rarely names either the
# way the instruction does, and effectively never both.
return "state block" in low and "events list" in low
def _clean(prose: str) -> str:
"""Removes protocol the block extraction could not, and nothing else.
Found by the M5 realistic-context run (§12), which is the failure class
Phase 0B warned about: under a full prompt the model echoed its own
instruction into the narration, and the reader would have been shown it.
Neither case here is hypothetical — both were observed against a real local
model.
"""
cleaned = prose
bracket = _TRAILING_BRACKET_RE.search(cleaned)
if bracket is not None and _is_echoed_instruction(bracket.group(1)):
cleaned = cleaned[: bracket.start()]
cleaned = _DANGLING_STATE_RE.sub("", cleaned)
dangling = _DANGLING_JSON_RE.search(cleaned)
if dangling is not None and _reads_as_protocol(dangling.group(1)):
cleaned = cleaned[: dangling.start()]
return cleaned.strip()
def _reads_as_protocol(tail: str) -> bool:
"""Whether a truncated fence was on its way to being a proposal."""
if '"events"' in tail:
return True
return any(f'"{name}"' in tail for name in events.SPECS)
def _tolerant_load(blob: str):
"""Parses a block, forgiving what small local models get wrong.
Trailing commas and a leading `+` on a number are both common and both
rejected by strict JSON. Repairing them is not guessing at meaning — the
intended value is unambiguous — which is the line this function stays on the
right side of. Anything it cannot parse returns None, and the caller treats
that as no proposal rather than as an empty one.
"""
cleaned = re.sub(r",(\s*[}\]])", r"\1", blob)
cleaned = re.sub(r"(:\s*)\+(\d)", r"\1\2", cleaned)
try:
parsed = json.loads(cleaned)
except (json.JSONDecodeError, ValueError):
return None
return parsed
def split(text: str) -> tuple[str, dict | None, str]:
"""Separates a reply into `(prose, proposal, raw_block)`.
`proposal` is None when there is no block or it cannot be parsed at all,
which the caller records as a malformed proposal. `raw_block` is what the
model actually wrote, kept for the audit record even — especially — when it
did not parse.
A bare trailing object is only stripped when it parses *and* looks like a
proposal. Prose that happens to end in a brace is left alone, because
removing a sentence from someone's story to satisfy a regex is a worse
failure than leaving a stray brace in it.
"""
matches = list(_STATE_FENCE_RE.finditer(text))
if matches:
match = matches[-1]
raw = match.group(1).strip()
prose = _clean(text[: match.start()] + text[match.end():])
return prose, _tolerant_load(raw), raw
# A `json` or unlabelled fence is ours only when its contents are this
# protocol. That is judged two ways, and it needs both: a block that parses
# into a proposal, or one that plainly reads as protocol even though it does
# not parse. The second half matters — a small model that mangles its own
# JSON must not have the wreckage shown to the reader, which is what the
# realistic-model run caught during the corrective pass.
for pattern in (_JSON_FENCE_RE, _BARE_FENCE_RE):
for match in reversed(list(pattern.finditer(text))):
raw = match.group(1).strip()
parsed = _tolerant_load(raw)
if _looks_like_proposal(parsed) or _reads_as_protocol(raw):
prose = _clean(text[: match.start()] + text[match.end():])
return prose, parsed, raw
match = _TRAILING_RE.search(text)
if match:
raw = match.group(1)
parsed = _tolerant_load(raw)
if _looks_like_proposal(parsed):
return _clean(text[: match.start()]), parsed, raw
# No block at all — but the reply may still carry protocol the model wrote
# as prose, or a fence it never closed.
cleaned = _clean(text)
if cleaned != text.strip():
return cleaned, None, text.strip()[len(cleaned):].strip()
return cleaned, None, ""
def _looks_like_proposal(parsed) -> bool:
"""Whether a bare trailing object is this protocol rather than prose."""
if not isinstance(parsed, dict):
return False
if isinstance(parsed.get("events"), list):
return True
return isinstance(parsed.get("type"), str) and events.is_allowed(parsed["type"])
def render_block(accepted: list[dict]) -> str:
"""Renders accepted events back into the block the model emitted.
Replayed into the prompt for past turns so the model copies the format it is
being asked for. **Accepted** events rather than proposed ones, for the
reason the delta protocol learned the hard way: showing the model a refused
event standing as though it had worked, contradicted by the state in the
same prompt, teaches it to send the event again.
"""
if not accepted:
return ""
return "```state\n" + json.dumps({"events": accepted}, ensure_ascii=False) + "\n```"
def render_rejections(rejected: list[dict]) -> str:
"""The correction note appended after the most recent AI turn.
Only what was lost. A model that is told what it got wrong can fix it next
turn; a model told nothing repeats it.
"""
if not rejected:
return ""
lines = []
for entry in rejected[:6]:
if not isinstance(entry, dict):
continue
detail = entry.get("detail") or entry.get("reason") or ""
if detail:
lines.append(f"- {detail}")
if not lines:
return ""
body = "\n".join(lines)
return (
"[Part of your last state block was not accepted. Correct it in this "
f"turn's block:\n{body}]"
)
backend/app/narrative/model.py
+307
View File
@@ -0,0 +1,307 @@
"""M5: the authoritative narrative state, and what shape it has.
This is the genre-neutral state ADR 006 requires and ADR 010's typed events
write into. It replaces the inherited RPG world state, which assumed stats,
bands, cooldowns and per-turn delta caps — assumptions that are a *game system*,
not a story.
## What a state document is
One JSON document per story position, holding what the campaign currently
believes:
entities the things that exist: who, where, what
possessions which entity holds which item
facts assertions about the world, with an authority
relationships directed ties between entities
threads narrative business that is open or resolved
scene the immediate situation
Nothing here names a genre. A character, a location, an organization, an item
and a vehicle are all `entities` with a `type`, which is a descriptive label the
campaign chooses, not a branch in the code (`DATA-MODEL.md` §9). The same
document holds Aldric in an abbey and the Persephone at Ceres Station, and
`J03` is satisfied because moving between them is data.
## Why a document rather than normalised tables
`DATA-MODEL.md` §17 selects the **hybrid**: validated events for audit, plus a
snapshot for reads and restore. M3 and M4 make that choice load-bearing rather
than an optimisation. Every position in a retained story must be recoverable in
bounded time — `TECHNICAL-DESIGN.md` §10.4 — because Undo, Redo and Save Point
restore all resolve a coordinate and read the state recorded there. Current-value
tables would leave the *future's* values standing when the head moves back, which
`BUILD-MILESTONES.md` M5 forbids in as many words, and rebuilding them would mean
replaying the campaign.
So the authoritative current state is this document, snapshotted per node exactly
as the world state was, and the event log beside it is the audit record rather
than the reconstruction path. The events say *why* the document changed; the
document says what is true now.
Everything in this module is pure. It builds and reads documents; it does not
touch the database, and it does not decide whether a proposal is acceptable —
that is `validate.py`, and applying an accepted event is `apply.py`.
"""
from __future__ import annotations
import copy
# The document version, so a later milestone can migrate a stored snapshot
# without guessing what it was written by. Bump only for a shape change that a
# reader cannot infer.
VERSION = 1
# Entity categories the product suggests. This is a vocabulary, not a
# constraint: `DATA-MODEL.md` §9 calls these "descriptive categories, not
# separate game systems", so an unknown type is accepted and simply described.
# Rejecting one would make the schema genre-specific by the back door.
SUGGESTED_TYPES = (
"character", "location", "organization", "item", "vehicle",
"creature", "structure", "concept", "other",
)
# Entity lifecycle status. `DATA-MODEL.md` §9.
ENTITY_STATUSES = ("active", "inactive", "destroyed", "dead", "unknown")
# Where a fact came from, in descending authority. `DATA-MODEL.md` §14 lists the
# minimum categories; the order here is what a later context builder ranks by.
AUTHORITIES = (
"campaign_canon", # the campaign's own rules — the highest
"manual_correction", # the user said so, explicitly (C04)
"accepted_story", # derived from narration the user accepted
"current_state",
"imported_canon", # M7
"reference", # M7
"heuristic",
"inspiration", # M7
)
FACT_STATUSES = ("active", "superseded", "disputed", "invalidated")
THREAD_STATUSES = ("open", "dormant", "resolved", "abandoned")
RELATIONSHIP_STATUSES = ("active", "ended")
def empty() -> dict:
"""A campaign that has established nothing yet.
Every key is present, so no reader needs a `.get` with a default and no
writer has to decide whether a section exists. An empty document is a real
document, not a missing one.
"""
return {
"version": VERSION,
"entities": {},
"possessions": {},
"facts": [],
"relationships": [],
"threads": {},
"scene": {},
}
def normalize(state) -> dict:
"""Returns `state` as a well-formed document, repairing what it can.
Called on every read of a stored snapshot. A document can arrive from a
hand-edited database, an imported bundle, or a snapshot written by an older
version of this module, and a read must not raise on any of them: the story
is the valuable thing, and a malformed state section should cost the
section, not the campaign.
Repair is deliberately shallow — wrong-typed sections are replaced with
empty ones rather than coerced, because guessing what a malformed section
meant is exactly the kind of invention `§19` of the M5 brief forbids.
"""
if not isinstance(state, dict):
return empty()
out = empty()
out["version"] = state.get("version") if isinstance(state.get("version"), int) else VERSION
for key in ("entities", "possessions", "threads", "scene"):
value = state.get(key)
if isinstance(value, dict):
out[key] = copy.deepcopy(value)
for key in ("facts", "relationships"):
value = state.get(key)
if isinstance(value, list):
out[key] = copy.deepcopy([item for item in value if isinstance(item, dict)])
return out
def is_empty(state) -> bool:
"""Whether a document says nothing about the world.
`version` alone does not count as content, so a freshly created campaign
reads as empty and the prompt builder can leave the section out entirely
rather than showing a heading with nothing under it.
"""
document = normalize(state)
return not any(
document[key] for key in
("entities", "possessions", "facts", "relationships", "threads", "scene")
)
# ------------------------------------------------------------------ entities
def entity(state: dict, key: str) -> dict | None:
"""Returns the entity stored under `key`, or None."""
entities = state.get("entities")
if not isinstance(entities, dict):
return None
found = entities.get(key)
return found if isinstance(found, dict) else None
def entity_name(state: dict, key: str) -> str:
"""The display name for `key`, falling back to the key itself.
A key is a slug the campaign chose, so it is readable enough to show when an
entity was referenced before it was described.
"""
found = entity(state, key)
if found and isinstance(found.get("name"), str) and found["name"].strip():
return found["name"]
return key
def new_entity(
*, type: str = "other", name: str = "", description: str = "",
status: str = "active", aliases: list | None = None,
) -> dict:
return {
"type": type or "other",
"name": name,
"description": description,
"status": status or "active",
"aliases": list(aliases or []),
# Where this entity currently is, as another entity's key. None means
# the campaign has not placed it, which is different from placing it
# nowhere.
"location": None,
# Free-form condition labels: "injured", "depressurised", "asleep".
# Labels rather than numbers, because a number implies a scale and a
# scale implies a game system.
"conditions": [],
# Named values the campaign cares about. Genre-neutral by construction:
# the campaign chooses the names, and every write is an absolute
# assignment (ADR 010).
"attributes": {},
}
def entities_of_type(state: dict, wanted: str) -> dict:
"""Every entity whose `type` matches, keyed as they are stored."""
entities = state.get("entities")
if not isinstance(entities, dict):
return {}
return {
key: value for key, value in entities.items()
if isinstance(value, dict) and value.get("type") == wanted
}
# --------------------------------------------------------------- possessions
def owner_of(state: dict, item_key: str) -> str | None:
"""Which entity holds `item_key`, or None if nobody does.
Possession is stored as one map from item to owner rather than as a list per
owner, because an item has exactly one holder and the map makes that
structural. Two owners for one item is then unrepresentable rather than
merely invalid.
"""
possessions = state.get("possessions")
if not isinstance(possessions, dict):
return None
owner = possessions.get(item_key)
return owner if isinstance(owner, str) else None
def held_by(state: dict, owner_key: str) -> list[str]:
"""Every item `owner_key` currently holds, in stable order."""
possessions = state.get("possessions")
if not isinstance(possessions, dict):
return []
return sorted(
item for item, owner in possessions.items() if owner == owner_key
)
# ------------------------------------------------------------------- facts
def withdrawn_facts(state: dict) -> list[dict]:
"""Facts a correction or retcon took back, newest last.
The prompt needs these as well as the ones that stand. Dropping a withdrawn
fact silently leaves the narration that first asserted it as the only
account in the prompt, and the model reads surviving prose as current truth
(M5 review, Finding 4). Naming the withdrawal is what makes the reader's
correction win.
"""
facts = state.get("facts")
if not isinstance(facts, list):
return []
return [
fact for fact in facts
if isinstance(fact, dict) and fact.get("status") == "invalidated"
]
def active_facts(state: dict) -> list[dict]:
"""Facts that still stand, newest last.
An invalidated fact stays in the document rather than being removed. C04
requires a correction to be auditable, and a fact that vanished would leave
nothing to audit — the record of what the campaign used to believe is the
point.
"""
facts = state.get("facts")
if not isinstance(facts, list):
return []
return [
fact for fact in facts
if isinstance(fact, dict) and fact.get("status", "active") == "active"
]
def facts_about(state: dict, subject_key: str) -> list[dict]:
return [f for f in active_facts(state) if f.get("subject") == subject_key]
def knows(state: dict, subject_key: str, object_key: str) -> bool:
"""Whether an accepted fact says `subject` knows `object`.
C03's question, asked the way the state model can answer it. "The campaign
knows X" is a fact with no subject; "Mara knows X" is a fact whose subject
is Mara. The distinction is structural, so nothing has to infer it.
"""
return any(
fact.get("predicate") == "knows" and fact.get("object") == object_key
for fact in facts_about(state, subject_key)
)
# ----------------------------------------------------------- relationships
def active_relationships(state: dict) -> list[dict]:
relationships = state.get("relationships")
if not isinstance(relationships, list):
return []
return [
r for r in relationships
if isinstance(r, dict) and r.get("status", "active") == "active"
]
# ---------------------------------------------------------------- threads
def open_threads(state: dict) -> dict:
threads = state.get("threads")
if not isinstance(threads, dict):
return {}
return {
key: value for key, value in threads.items()
if isinstance(value, dict) and value.get("status", "open") in ("open", "dormant")
}
backend/app/narrative/render.py
+291
View File
@@ -0,0 +1,291 @@
"""M5: showing the narrative state — to the model, and to the reader.
Two audiences, one document, and they want different things. The model needs the
state compactly, in the vocabulary it must answer in, close to where it
generates. The reader needs it grouped and named, in the words the campaign uses.
Both are read-only views. Neither can change state, and the browser gets its own
data from the API rather than from anything assembled here, because
`BUILD-MILESTONES.md` M5 is explicit that the browser is a presentation layer and
must not become the owner of state.
"""
from __future__ import annotations
from . import model
# How much of a long section reaches the prompt. A campaign accumulates facts
# faster than it accumulates anything else, and the context budget is finite;
# the newest are the ones the current scene is most likely to need. M6 owns
# retrieval-ranked selection, so this is deliberately a simple recency cut and
# is documented as such rather than pretending to be a relevance model.
PROMPT_FACTS = 30
PROMPT_RELATIONSHIPS = 20
PROMPT_THREADS = 12
def for_prompt(state) -> str:
"""The current state as the narrator is shown it.
Empty string when the campaign has established nothing, so a new story's
prompt carries no heading with nothing under it.
"""
document = model.normalize(state)
if model.is_empty(document):
return ""
lines: list[str] = []
scene = document.get("scene") or {}
if scene.get("summary") or scene.get("location"):
where = scene.get("location")
head = "Scene: " + str(scene.get("summary") or "").strip()
if where:
head += f" (at {model.entity_name(document, where)})"
lines.append(head.strip())
entities = document["entities"]
if entities:
lines.append("")
lines.append("Who and what exists:")
for key, entity in entities.items():
lines.append(f" {key}: {_entity_line(document, key, entity)}")
possessions = document["possessions"]
if possessions:
lines.append("")
lines.append("Held:")
for item, owner in sorted(possessions.items()):
lines.append(
f" {model.entity_name(document, item)} — "
f"{model.entity_name(document, owner)}"
)
facts = model.active_facts(document)
if facts:
lines.append("")
lines.append("Established:")
for fact in facts[-PROMPT_FACTS:]:
lines.append(f" {_fact_line(document, fact)}")
# What the campaign has taken back. Placed straight after what stands, so
# the contradiction is resolved in the same breath it could be raised: the
# story above may still narrate the moment, and this says it did not hold
# (C04, M5 review Finding 4).
withdrawn = model.withdrawn_facts(document)
if withdrawn:
lines.append("")
lines.append("No longer true — do not treat these as established:")
for fact in withdrawn[-PROMPT_FACTS:]:
line = f" {_fact_line(document, fact)}"
reason = fact.get("invalidated_reason")
if reason:
line += f" — {reason}"
lines.append(line)
relationships = model.active_relationships(document)
if relationships:
lines.append("")
lines.append("Between them:")
for relationship in relationships[-PROMPT_RELATIONSHIPS:]:
lines.append(
f" {model.entity_name(document, relationship['source'])} "
f"{relationship['type']} "
f"{model.entity_name(document, relationship['target'])}"
)
threads = model.open_threads(document)
if threads:
lines.append("")
lines.append("Still open:")
for key, thread in list(threads.items())[:PROMPT_THREADS]:
lines.append(f" {key}: {thread.get('title', key)}")
return "\n".join(lines).strip()
def _entity_line(document: dict, key: str, entity: dict) -> str:
parts = [entity.get("name") or key]
kind = entity.get("type")
if kind and kind != "other":
parts.append(f"({kind})")
status = entity.get("status")
if status and status != "active":
parts.append(f"[{status}]")
where = entity.get("location")
if where:
parts.append(f"at {model.entity_name(document, where)}")
conditions = entity.get("conditions") or []
if conditions:
parts.append("— " + ", ".join(conditions))
attributes = entity.get("attributes") or {}
if attributes:
parts.append(
"— " + ", ".join(f"{name}={value}" for name, value in sorted(attributes.items()))
)
return " ".join(str(p) for p in parts)
def _fact_line(document: dict, fact: dict) -> str:
parts = []
if fact.get("subject"):
parts.append(model.entity_name(document, fact["subject"]))
parts.append(str(fact.get("predicate", "")))
if fact.get("object"):
parts.append(model.entity_name(document, fact["object"]))
if fact.get("value") is not None:
parts.append(str(fact["value"]))
line = " ".join(str(p) for p in parts if p)
if fact.get("authority") == "manual_correction":
# The reader corrected this. Saying so in the prompt is what stops the
# model re-deriving the thing the correction removed.
line += " [corrected by the player]"
return line
def for_inspector(state) -> dict:
"""The current state grouped for the browser panel.
Only categories that actually hold something are returned, so the panel can
render what it is given without deciding what to hide — a category with no
rows is a heading that tells the reader nothing.
Every entry carries the key as well as the name. The key is what a manual
correction has to name, so the panel can offer a correction without the user
having to guess at an identifier.
"""
document = model.normalize(state)
groups: list[dict] = []
scene = document.get("scene") or {}
if scene.get("summary") or scene.get("location"):
rows = []
if scene.get("summary"):
rows.append({"key": "summary", "label": str(scene["summary"])})
if scene.get("location"):
rows.append({
"key": scene["location"],
"label": model.entity_name(document, scene["location"]),
"detail": "location",
})
groups.append({"title": "Current Scene", "rows": rows})
by_type: dict[str, list] = {}
for key, entity in document["entities"].items():
by_type.setdefault(entity.get("type") or "other", []).append((key, entity))
# Characters and locations first because they are what a reader looks for;
# everything else in whatever categories the campaign actually used, so a
# science-fiction campaign's `vehicle` appears without this code knowing the
# word (J02).
order = ["character", "location"] + sorted(
set(by_type) - {"character", "location"}
)
for kind in order:
members = by_type.get(kind)
if not members:
continue
rows = []
for key, entity in sorted(members):
detail = []
if entity.get("status") and entity["status"] != "active":
detail.append(str(entity["status"]))
if entity.get("location"):
detail.append("at " + model.entity_name(document, entity["location"]))
if entity.get("conditions"):
detail.append(", ".join(entity["conditions"]))
for name, value in sorted((entity.get("attributes") or {}).items()):
detail.append(f"{name}: {value}")
held = model.held_by(document, key)
if held:
detail.append(
"carrying " + ", ".join(model.entity_name(document, i) for i in held)
)
rows.append({
"key": key,
"label": entity.get("name") or key,
"detail": " · ".join(detail),
})
groups.append({"title": _title_for(kind), "rows": rows})
possessions = document["possessions"]
if possessions:
groups.append({"title": "Possessions", "rows": [
{
"key": item,
"label": model.entity_name(document, item),
"detail": "held by " + model.entity_name(document, owner),
}
for item, owner in sorted(possessions.items())
]})
facts = model.active_facts(document)
if facts:
groups.append({"title": "Important Facts", "rows": [
{
"key": fact.get("id") or "",
"label": _fact_line(document, fact),
"detail": _source_label(fact),
}
for fact in facts
]})
relationships = model.active_relationships(document)
if relationships:
groups.append({"title": "Relationships", "rows": [
{
"key": relationship.get("id") or "",
"label": (
f"{model.entity_name(document, relationship['source'])} "
f"{relationship['type']} "
f"{model.entity_name(document, relationship['target'])}"
),
"detail": relationship.get("description") or "",
}
for relationship in relationships
]})
threads = model.open_threads(document)
if threads:
groups.append({"title": "Open Story Threads", "rows": [
{
"key": key,
"label": thread.get("title") or key,
"detail": thread.get("description") or "",
}
for key, thread in sorted(threads.items())
]})
return {"groups": groups, "empty": not groups}
def _title_for(kind: str) -> str:
"""A heading for an entity category the campaign chose.
Pluralised generically rather than from a table, because the categories are
open: `DATA-MODEL.md` §9 suggests nine and permits any, so a lookup would
silently mislabel the tenth.
"""
known = {
"character": "Characters",
"location": "Locations",
"organization": "Organizations",
"item": "Items",
"vehicle": "Vehicles",
"creature": "Creatures",
"structure": "Structures",
"concept": "Concepts",
"other": "Other",
}
if kind in known:
return known[kind]
word = kind.replace("_", " ").strip().title()
return word if word.endswith("s") else word + "s"
def _source_label(fact: dict) -> str:
source = fact.get("authority") or fact.get("source") or ""
return {
"manual_correction": "your correction",
"campaign_canon": "campaign canon",
"accepted_story": "from the story",
}.get(source, str(source).replace("_", " "))
backend/app/narrative/store.py
+195
View File
@@ -0,0 +1,195 @@
"""M5: writing accepted state, atomically with the turn that caused it.
This is the only module in the package that touches the database, and the only
place authoritative narrative state is written.
## The atomicity rule (L01)
Everything a turn establishes goes in one transaction: the narration, the head
movement, the accepted events, the resulting snapshot, and the provenance. This
function *adds* to the caller's session and never commits — the turn engine's
single `db.commit()` remains the one commit point, so a failure anywhere before
it rolls the whole turn back rather than leaving narration accepted with half its
state written.
That ordering is deliberate and load-bearing. `L01` forbids a head position that
implies an accepted reply whose state commit did not complete, and the cheapest
way to guarantee that is to never have two commits to get out of step.
## What is not here
No reconstruction. Nothing in this module reads `state_events` to rebuild a
document — the snapshot on the node is the restore path
(`TECHNICAL-DESIGN.md` §10.4). The events are the audit trail, and an audit
trail that the system depends on for correctness stops being an audit trail and
becomes a replay engine.
"""
from __future__ import annotations
import copy
from sqlalchemy.orm import Session
from .. import models
from . import apply as apply_module
from . import model
def current(adventure: models.Adventure) -> dict:
"""The campaign's authoritative state right now, as a document.
Normalised on the way out, so every caller gets the same shape whatever a
hand-edited row or an older snapshot contains.
"""
return model.normalize(adventure.narrative_state)
def set_current(adventure: models.Adventure, state: dict) -> None:
adventure.narrative_state = model.normalize(state)
def canon_of(adventure: models.Adventure) -> dict:
"""The campaign's own rules, which outrank anything a narration proposes.
Configuration rather than code (C01, J03): the campaign says what it forbids,
and `validate` enforces it without knowing what the rule means.
"""
canon = adventure.campaign_canon
return canon if isinstance(canon, dict) else {}
def record(
db: Session,
adventure: models.Adventure,
*,
review,
raw_block: str = "",
parsed=None,
action: models.Action | None = None,
branch_id: int | None = None,
depth: int | None = None,
model_name: str = "",
source: str = "accepted_story",
) -> tuple[dict, models.StateProposal]:
"""Applies a reviewed proposal and records everything about it.
Returns `(new_state, proposal_row)`. The caller is responsible for putting
the new state where it belongs — on the campaign, and on the node's snapshot
— because only the caller knows whether this is a turn, a retry or a
correction.
Nothing is committed here. See the module docstring.
"""
before = current(adventure)
after = apply_module.apply_events(
before, review.accepted, branch_id=branch_id, depth=depth, source=source
)
proposal = models.StateProposal(
adventure_id=adventure.id,
action_id=action.id if action is not None else None,
branch_id=branch_id,
depth=depth,
model_name=model_name or "",
source=source,
status=review.status,
raw_output=raw_block or "",
detail={
"parsed": parsed,
"accepted": review.accepted,
"rejected": [r.as_dict() for r in review.rejected],
},
)
db.add(proposal)
# The proposal needs an id before its events can point at it, and the
# session does not autoflush. This is a flush, not a commit: still one
# transaction, still all-or-nothing.
db.flush()
for sequence, event in enumerate(review.accepted):
db.add(models.StateEvent(
adventure_id=adventure.id,
proposal_id=proposal.id,
action_id=action.id if action is not None else None,
branch_id=branch_id,
depth=depth,
sequence=sequence,
event_type=event.get("type", ""),
payload=copy.deepcopy(event),
before=_before_value(before, event),
source=source,
))
return after, proposal
def _before_value(state: dict, event: dict) -> dict | None:
"""What the value this event changes was, immediately beforehand.
Recorded per event so §8's "what was the previous value" is answerable
without replaying anything. Only the slice the event touches: a whole
document per event would duplicate the snapshot for no extra answer.
"""
kind = event.get("type")
if kind in ("set_entity_status", "set_entity_attribute",
"set_entity_conditions", "set_current_location"):
entity = model.entity(state, event.get("entity", ""))
if entity is None:
return None
if kind == "set_entity_status":
return {"status": entity.get("status")}
if kind == "set_entity_attribute":
attribute = event.get("attribute")
return {"attribute": attribute,
"value": (entity.get("attributes") or {}).get(attribute)}
if kind == "set_entity_conditions":
return {"conditions": list(entity.get("conditions") or [])}
return {"location": entity.get("location")}
if kind in ("set_possession", "clear_possession"):
return {"owner": model.owner_of(state, event.get("item", ""))}
if kind == "invalidate_fact":
for fact in state.get("facts") or []:
if fact.get("id") == event.get("fact_id"):
return {"status": fact.get("status"), "predicate": fact.get("predicate")}
return None
if kind == "resolve_story_thread":
thread = (state.get("threads") or {}).get(event.get("thread", ""))
return {"status": thread.get("status")} if isinstance(thread, dict) else None
if kind == "end_relationship":
return {"status": "active"}
return None
# ------------------------------------------------------------------ reading
def events_for(
db: Session, adventure: models.Adventure, action_id: int
) -> list[models.StateEvent]:
"""The accepted events one node's narration produced, in order."""
return (
db.query(models.StateEvent)
.filter(
models.StateEvent.adventure_id == adventure.id,
models.StateEvent.action_id == action_id,
)
.order_by(models.StateEvent.sequence, models.StateEvent.id)
.all()
)
def history(
db: Session, adventure: models.Adventure, limit: int = 200
) -> list[models.StateEvent]:
"""The campaign's accepted state events, newest first.
Bounded by default: this is an audit view, and an unbounded read of a long
campaign's every event is the kind of query this project keeps a regression
test about.
"""
return (
db.query(models.StateEvent)
.filter(models.StateEvent.adventure_id == adventure.id)
.order_by(models.StateEvent.id.desc())
.limit(limit)
.all()
)
backend/app/narrative/validate.py
+317
View File
@@ -0,0 +1,317 @@
"""M5: deciding which proposed events the application will accept.
A proposal is untrusted model output. This module is the gate between it and the
authoritative state, and it is layered so that a rejection can say *which* rule
refused and a test can aim at one layer at a time:
1. envelope is this a proposal at all — a dict with a list of events?
2. allowlist is each event type one this application implements? (H05)
3. schema are the required fields present, and the right shape?
4. referential do the entities and threads it names exist?
5. semantic does it contradict campaign canon, or itself?
Layer 2 is the security boundary and runs before any field is read, so a payload
carrying `command` or `path` alongside an unknown type is discarded without those
fields ever being looked at.
## What rejection means
Nothing is partially applied. `review` returns accepted and rejected events
separately and the caller decides; `apply.py` is only ever handed the accepted
list. A proposal with one bad event out of four therefore lands three, which is
`partially_accepted` — the alternative, discarding all four because the model
misspelled one entity, loses story the user watched happen.
What is *never* allowed is a rejected event mutating anything, or a rejection
being silent: every refusal carries a reason, is counted, and is stored on the
proposal record for §8's audit.
## What this module does not do
It does not decide whether the model was *right*. A typed event can be
well-formed, reference real entities, contradict nothing, and still describe
something the narration did not say. That is C06's territory and no validator
can settle it — ADR 010 says so plainly. What validation buys is that a wrong
proposal is wrong in a way a person can see in the audit trail, rather than one
that silently means something other than it appears to.
"""
from __future__ import annotations
from . import events, model
# A rejected event carries one of these, so tests and the debug view can assert
# on the reason rather than on prose.
UNKNOWN_TYPE = "unknown_event_type"
NOT_AN_OBJECT = "not_an_object"
MISSING_FIELD = "missing_field"
BAD_FIELD_TYPE = "bad_field_type"
UNKNOWN_REFERENCE = "unknown_reference"
CANON_CONFLICT = "canon_conflict"
SELF_CONTRADICTION = "self_contradiction"
DUPLICATE_ENTITY = "duplicate_entity"
# How many events one proposal may carry. A narration describes a turn, not a
# migration; a hundred events is a runaway model or a payload trying to be
# something else, and either way the cap bounds the work before it is done.
MAX_EVENTS = 40
# How long a text field may be. Long enough for a description, short enough that
# a proposal cannot smuggle a document into the state.
MAX_TEXT = 2_000
MAX_LABELS = 40
class Rejection:
"""One event that will not be applied, and why."""
__slots__ = ("event", "reason", "detail")
def __init__(self, event, reason: str, detail: str = ""):
self.event = event
self.reason = reason
self.detail = detail
def as_dict(self) -> dict:
return {"event": self.event, "reason": self.reason, "detail": self.detail}
def __repr__(self) -> str: # pragma: no cover - debugging aid
return f"<Rejection {self.reason}: {self.detail}>"
class Review:
"""The verdict on one proposal."""
__slots__ = ("accepted", "rejected")
def __init__(self, accepted: list[dict], rejected: list[Rejection]):
self.accepted = accepted
self.rejected = rejected
@property
def status(self) -> str:
"""`DATA-MODEL.md` §19's validation_status."""
if self.rejected and self.accepted:
return "partially_accepted"
if self.rejected:
return "rejected"
return "accepted"
def as_dict(self) -> dict:
return {
"status": self.status,
"accepted": self.accepted,
"rejected": [r.as_dict() for r in self.rejected],
}
def review(payload, state: dict, canon: dict | None = None) -> Review:
"""Returns which of `payload`'s events may be applied to `state`.
`state` is the document the events would apply to, needed because
referential checks ask what already exists. `canon` carries the campaign's
own rules, which outrank anything a narration proposes (C01).
The state is **not** mutated. Events are checked against a running view that
accounts for entities earlier events in the same proposal create, so a
proposal may introduce Mara and then move her, but nothing is written until
the caller applies the accepted list.
"""
accepted: list[dict] = []
rejected: list[Rejection] = []
proposed = _events_of(payload)
if proposed is None:
return Review([], [Rejection(payload, NOT_AN_OBJECT,
"the proposal is not an object with an event list")])
# Entities this proposal has introduced, so a later event in the same
# proposal may refer to them. Kept separately from `state` so that a
# rejected create cannot make a later reference resolve.
introduced: set[str] = set()
for raw in proposed[:MAX_EVENTS]:
problem = _check(raw, state, introduced, canon)
if problem is not None:
rejected.append(problem)
continue
accepted.append(raw)
spec = events.spec(raw["type"])
if spec and spec["creates"]:
introduced.add(str(raw[spec["creates"]]))
for extra in proposed[MAX_EVENTS:]:
rejected.append(Rejection(extra, BAD_FIELD_TYPE,
f"more than {MAX_EVENTS} events in one proposal"))
return Review(accepted, rejected)
def _events_of(payload) -> list | None:
"""The event list, from either shape a proposal may legitimately take."""
if isinstance(payload, list):
return [e for e in payload]
if not isinstance(payload, dict):
return None
found = payload.get("events")
if found is None:
return []
if not isinstance(found, list):
return None
return found
def _check(raw, state: dict, introduced: set[str], canon: dict | None) -> Rejection | None:
"""Returns why `raw` is unacceptable, or None if it may be applied."""
# ---- layer 1: is it an event-shaped object at all ----
if not isinstance(raw, dict):
return Rejection(raw, NOT_AN_OBJECT, "event is not an object")
# ---- layer 2: the allowlist, before any field is read ----
#
# H05 lands here. `execute_shell` is refused because it is not in the
# vocabulary, and its `command` field is never looked at — there is no
# branch in this application that could reach it.
event_type = raw.get("type", raw.get("event_type"))
if not events.is_allowed(event_type):
return Rejection(raw, UNKNOWN_TYPE, f"{event_type!r} is not a state event")
raw["type"] = event_type
spec = events.spec(event_type)
# ---- layer 3: schema ----
for field, kind in spec["required"].items():
if field not in raw:
return Rejection(raw, MISSING_FIELD, f"{event_type} needs {field!r}")
bad = _bad_shape(raw[field], kind, field)
if bad:
return Rejection(raw, BAD_FIELD_TYPE, bad)
for field, kind in spec["optional"].items():
if field in raw and raw[field] is not None:
bad = _bad_shape(raw[field], kind, field)
if bad:
return Rejection(raw, BAD_FIELD_TYPE, bad)
# ---- layer 4: referential integrity ----
known = set(state.get("entities") or {}) | introduced
for field in spec["refs"]:
named = raw.get(field)
if named is None or field not in raw:
continue # optional reference, absent
if not isinstance(named, str) or named not in known:
return Rejection(raw, UNKNOWN_REFERENCE,
f"{event_type} names {field}={named!r}, which does not exist")
if event_type == "add_fact" and raw.get("object") is not None:
# An object may name an entity or another fact. Checking both keeps the
# reference meaningful — a typo is still caught — without forcing every
# thing a fact can be about to be promoted to an entity first.
known_facts = {f.get("id") for f in (state.get("facts") or [])}
target = raw["object"]
if not isinstance(target, str) or (target not in known and target not in known_facts):
return Rejection(raw, UNKNOWN_REFERENCE,
f"add_fact names object={target!r}, which does not exist")
if event_type == "invalidate_fact":
if not any(f.get("id") == raw["fact_id"] for f in (state.get("facts") or [])):
return Rejection(raw, UNKNOWN_REFERENCE,
f"no fact {raw['fact_id']!r} to invalidate")
if event_type == "resolve_story_thread":
if raw["thread"] not in (state.get("threads") or {}):
return Rejection(raw, UNKNOWN_REFERENCE,
f"no story thread {raw['thread']!r} to resolve")
if spec["creates"]:
key = raw[spec["creates"]]
if key in known:
return Rejection(raw, DUPLICATE_ENTITY,
f"{key!r} already exists; use set_* to change it")
# ---- layer 5: semantics ----
return _semantic(raw, state, canon)
def _bad_shape(value, kind: str, field: str) -> str | None:
"""Returns why `value` is the wrong shape for `kind`, or None."""
if kind in (events.TEXT, events.KEY):
if not isinstance(value, str) or not value.strip():
return f"{field!r} must be a non-empty string"
if len(value) > MAX_TEXT:
return f"{field!r} is longer than {MAX_TEXT} characters"
return None
if kind == events.VALUE:
# A scalar. Explicitly not a dict or a list: a nested payload is how a
# value field becomes somewhere to hide a second protocol.
if not isinstance(value, (str, int, float, bool)) and value is not None:
return f"{field!r} must be a plain value, not a structure"
if isinstance(value, str) and len(value) > MAX_TEXT:
return f"{field!r} is longer than {MAX_TEXT} characters"
return None
if kind == events.LABELS:
if not isinstance(value, list):
return f"{field!r} must be a list"
if len(value) > MAX_LABELS:
return f"{field!r} has more than {MAX_LABELS} entries"
for item in value:
if not isinstance(item, str) or not item.strip():
return f"{field!r} must contain only non-empty strings"
if len(item) > MAX_TEXT:
return f"{field!r} contains an over-long entry"
return None
return f"{field!r} has an unknown field kind" # pragma: no cover
def _semantic(raw: dict, state: dict, canon: dict | None) -> Rejection | None:
"""Deterministic checks the application can actually make.
Deliberately modest. ADR 010 is explicit that typed events do not make a
model correct, and pretending arbitrary fiction can be validated would be
worse than admitting it cannot: it would produce confident rejections of
perfectly good story. So this refuses only what the application *knows* is
wrong — a self-contradiction, or a collision with a rule the campaign wrote
down.
"""
event_type = raw["type"]
# An entity cannot hold itself, and cannot be in itself.
if event_type == "set_possession" and raw["item"] == raw["owner"]:
return Rejection(raw, SELF_CONTRADICTION, "an item cannot possess itself")
if event_type == "set_current_location" and raw["entity"] == raw["location"]:
return Rejection(raw, SELF_CONTRADICTION, "an entity cannot be inside itself")
if event_type in ("add_relationship", "end_relationship") and raw["source"] == raw["target"]:
return Rejection(raw, SELF_CONTRADICTION,
"a relationship needs two different entities")
# C01: campaign canon outranks narration. The rule is generic — a campaign
# declares transitions it forbids, and any event proposing one is refused.
# Nothing here knows what any of those transitions mean; the campaign
# says which it forbids, in data.
conflict = _canon_conflict(raw, state, canon)
if conflict is not None:
return Rejection(raw, CANON_CONFLICT, conflict)
return None
def _canon_conflict(raw: dict, state: dict, canon: dict | None) -> str | None:
"""Whether campaign canon forbids what this event proposes.
Canon is configuration, not code (J03). A campaign writes:
{"forbidden_status_changes": [{"from": "dead", "to": "active"}]}
and a narration that tries to bring a dead character back is refused —
without this module, or any other, containing the word for what that is. A
science-fiction campaign forbidding a different transition uses the same
field and the same code path.
"""
if not isinstance(canon, dict):
return None
if raw["type"] != "set_entity_status":
return None
forbidden = canon.get("forbidden_status_changes")
if not isinstance(forbidden, list):
return None
current = (model.entity(state, raw["entity"]) or {}).get("status")
for rule in forbidden:
if not isinstance(rule, dict):
continue
if rule.get("from") == current and rule.get("to") == raw["status"]:
return (
f"campaign canon does not allow {raw['entity']!r} to go from "
f"{current!r} to {raw['status']!r}"
)
return None