Files
interactive-story/backend/app/narrative/render.py
T
JesseMarkowitzandClaude Opus 5 0c7316f951 Keep the state section and its proposal out of the story
The first complete M01 run with the memory bank on (f8d4010, 101 turns on
a GPU host) reported "complete". It still did not prove M04. The planted
clue was found at turn 100 only because the narrator had pasted the
narrative-state section into its own prose, and the paste was still in
recent history. No memory and no summary carried the clue.

The narrator is a small local model. It wrote protocol into its stored
narration on 42 of 104 turns, starting at depth 2, in four shapes:

- a copy of the state section: `Scene:`, `Who and what exists:`, `Held:`,
  `Established:`, `Still open:`
- that copy above a correct ```state block, which was stripped while the
  copy stayed
- the copy, then a bare `State` heading, then a `> {"events": ...}`
  proposal quoted like a player turn, sometimes with story after it
- the same block cut off by the output-token limit, on 10 turns

Stored text is replayed verbatim as history, so each leak also put a second,
older account of the state into the next prompt. That is what M5 review
Finding 4 removed from history replay, and every leak gave the model another
example to copy.

The extractor now removes:

- a pasted state section, recognised by at least two of the renderer's own
  headings as whole lines. The headings are constants in `render.py`, so the
  renderer and the extractor cannot drift apart. One heading alone, or a
  `Scene:` line of prose, is left.
- an unfenced proposal that starts a line, quoted or not, when it parses and
  is a proposal. With no fence it becomes the turn's proposal. A `State`
  heading directly above goes with it. Candidates are taken outermost first,
  so a finished event line inside an unfinished block is never taken as a
  proposal by itself.
- an unfinished unfenced proposal at the end that reads as protocol.
- whatever is left at the end: a `State` heading, a bare `>`, a parroted
  reminder or continue hint (closed or not), and a ```json fence cut off
  before it names its events. These are cut repeatedly until nothing more
  comes off.

This also fixes an older bug. `_STATE_FENCE_RE` read "a ```state block"
inside a parroted reminder as a fence opening and cut out the middle of the
reminder. The label must now end its line or run straight into the payload.

A reply whose only removal is a pasted state section records no raw block,
so the turn is not marked unparseable for a block it never started.

Every AI turn in four real runs was replayed through the new extractor:
draco M01, the two 26-turn GPU trials, and this M01 run. 339 turns in all.
No turn the old extractor had left clean changed. Every leak of our own
protocol is gone: 42 of 42 in this M01 run, 5 in trial 2, 3 on draco.
Trial 1 still has model-invented headings ("Identifiers established:",
"Set of events made true:") on 10 turns. They paraphrase the instruction and
are not our renderer's text, so they are left, not guessed at.

The long-run harness now records an explicit M04 verdict, which is never a
recovery while the clue is still in recent history. It also counts the AI
turns in the export that still carry protocol.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136VBTMUKWYeU6G9HgbDbND
2026-09-13 21:18:18 -04:00

309 lines
12 KiB
Python

"""M5: showing the narrative state — to the model, and to the reader.
Two audiences, one document, and they want different things. The model needs the
state compactly, in the vocabulary it must answer in, close to where it
generates. The reader needs it grouped and named, in the words the campaign uses.
Both are read-only views. Neither can change state, and the browser gets its own
data from the API rather than from anything assembled here, because
`BUILD-MILESTONES.md` M5 is explicit that the browser is a presentation layer and
must not become the owner of state.
"""
from __future__ import annotations
from . import model
# How much of a long section reaches the prompt. A campaign accumulates facts
# faster than it accumulates anything else, and the context budget is finite;
# the newest are the ones the current scene is most likely to need. M6 owns
# retrieval-ranked selection, so this is deliberately a simple recency cut and
# is documented as such rather than pretending to be a relevance model.
PROMPT_FACTS = 30
PROMPT_RELATIONSHIPS = 20
PROMPT_THREADS = 12
# The headings of `for_prompt`, each a whole line. `extract` recognises a copy of
# this section pasted into a narration by these, so they are named once here and
# the two cannot drift apart. A small local model reproduced the section in its
# prose on 42 of 104 turns in the first M01 run with the memory bank on.
HEADING_SCENE = "Scene:"
HEADING_ENTITIES = "Who and what exists:"
HEADING_HELD = "Held:"
HEADING_FACTS = "Established:"
HEADING_WITHDRAWN = "No longer true — do not treat these as established:"
HEADING_RELATIONSHIPS = "Between them:"
HEADING_THREADS = "Still open:"
#: Every heading except the scene's, which also opens the line it heads.
SECTION_HEADINGS = (
HEADING_ENTITIES, HEADING_HELD, HEADING_FACTS, HEADING_WITHDRAWN,
HEADING_RELATIONSHIPS, HEADING_THREADS,
)
def for_prompt(state) -> str:
"""The current state as the narrator is shown it.
Empty string when the campaign has established nothing, so a new story's
prompt carries no heading with nothing under it.
"""
document = model.normalize(state)
if model.is_empty(document):
return ""
lines: list[str] = []
scene = document.get("scene") or {}
if scene.get("summary") or scene.get("location"):
where = scene.get("location")
head = f"{HEADING_SCENE} " + str(scene.get("summary") or "").strip()
if where:
head += f" (at {model.entity_name(document, where)})"
lines.append(head.strip())
entities = document["entities"]
if entities:
lines.append("")
lines.append(HEADING_ENTITIES)
for key, entity in entities.items():
lines.append(f" {key}: {_entity_line(document, key, entity)}")
possessions = document["possessions"]
if possessions:
lines.append("")
lines.append(HEADING_HELD)
for item, owner in sorted(possessions.items()):
lines.append(
f" {model.entity_name(document, item)} — "
f"{model.entity_name(document, owner)}"
)
facts = model.active_facts(document)
if facts:
lines.append("")
lines.append(HEADING_FACTS)
for fact in facts[-PROMPT_FACTS:]:
lines.append(f" {_fact_line(document, fact)}")
# What the campaign has taken back. Placed straight after what stands, so
# the contradiction is resolved in the same breath it could be raised: the
# story above may still narrate the moment, and this says it did not hold
# (C04, M5 review Finding 4).
withdrawn = model.withdrawn_facts(document)
if withdrawn:
lines.append("")
lines.append(HEADING_WITHDRAWN)
for fact in withdrawn[-PROMPT_FACTS:]:
line = f" {_fact_line(document, fact)}"
reason = fact.get("invalidated_reason")
if reason:
line += f" — {reason}"
lines.append(line)
relationships = model.active_relationships(document)
if relationships:
lines.append("")
lines.append(HEADING_RELATIONSHIPS)
for relationship in relationships[-PROMPT_RELATIONSHIPS:]:
lines.append(
f" {model.entity_name(document, relationship['source'])} "
f"{relationship['type']} "
f"{model.entity_name(document, relationship['target'])}"
)
threads = model.open_threads(document)
if threads:
lines.append("")
lines.append(HEADING_THREADS)
for key, thread in list(threads.items())[:PROMPT_THREADS]:
lines.append(f" {key}: {thread.get('title', key)}")
return "\n".join(lines).strip()
def _entity_line(document: dict, key: str, entity: dict) -> str:
parts = [entity.get("name") or key]
kind = entity.get("type")
if kind and kind != "other":
parts.append(f"({kind})")
status = entity.get("status")
if status and status != "active":
parts.append(f"[{status}]")
where = entity.get("location")
if where:
parts.append(f"at {model.entity_name(document, where)}")
conditions = entity.get("conditions") or []
if conditions:
parts.append("— " + ", ".join(conditions))
attributes = entity.get("attributes") or {}
if attributes:
parts.append(
"— " + ", ".join(f"{name}={value}" for name, value in sorted(attributes.items()))
)
return " ".join(str(p) for p in parts)
def _fact_line(document: dict, fact: dict) -> str:
parts = []
if fact.get("subject"):
parts.append(model.entity_name(document, fact["subject"]))
parts.append(str(fact.get("predicate", "")))
if fact.get("object"):
parts.append(model.entity_name(document, fact["object"]))
if fact.get("value") is not None:
parts.append(str(fact["value"]))
line = " ".join(str(p) for p in parts if p)
if fact.get("authority") == "manual_correction":
# The reader corrected this. Saying so in the prompt is what stops the
# model re-deriving the thing the correction removed.
line += " [corrected by the player]"
return line
def for_inspector(state) -> dict:
"""The current state grouped for the browser panel.
Only categories that actually hold something are returned, so the panel can
render what it is given without deciding what to hide — a category with no
rows is a heading that tells the reader nothing.
Every entry carries the key as well as the name. The key is what a manual
correction has to name, so the panel can offer a correction without the user
having to guess at an identifier.
"""
document = model.normalize(state)
groups: list[dict] = []
scene = document.get("scene") or {}
if scene.get("summary") or scene.get("location"):
rows = []
if scene.get("summary"):
rows.append({"key": "summary", "label": str(scene["summary"])})
if scene.get("location"):
rows.append({
"key": scene["location"],
"label": model.entity_name(document, scene["location"]),
"detail": "location",
})
groups.append({"title": "Current Scene", "rows": rows})
by_type: dict[str, list] = {}
for key, entity in document["entities"].items():
by_type.setdefault(entity.get("type") or "other", []).append((key, entity))
# Characters and locations first because they are what a reader looks for;
# everything else in whatever categories the campaign actually used, so a
# science-fiction campaign's `vehicle` appears without this code knowing the
# word (J02).
order = ["character", "location"] + sorted(
set(by_type) - {"character", "location"}
)
for kind in order:
members = by_type.get(kind)
if not members:
continue
rows = []
for key, entity in sorted(members):
detail = []
if entity.get("status") and entity["status"] != "active":
detail.append(str(entity["status"]))
if entity.get("location"):
detail.append("at " + model.entity_name(document, entity["location"]))
if entity.get("conditions"):
detail.append(", ".join(entity["conditions"]))
for name, value in sorted((entity.get("attributes") or {}).items()):
detail.append(f"{name}: {value}")
held = model.held_by(document, key)
if held:
detail.append(
"carrying " + ", ".join(model.entity_name(document, i) for i in held)
)
rows.append({
"key": key,
"label": entity.get("name") or key,
"detail": " · ".join(detail),
})
groups.append({"title": _title_for(kind), "rows": rows})
possessions = document["possessions"]
if possessions:
groups.append({"title": "Possessions", "rows": [
{
"key": item,
"label": model.entity_name(document, item),
"detail": "held by " + model.entity_name(document, owner),
}
for item, owner in sorted(possessions.items())
]})
facts = model.active_facts(document)
if facts:
groups.append({"title": "Important Facts", "rows": [
{
"key": fact.get("id") or "",
"label": _fact_line(document, fact),
"detail": _source_label(fact),
}
for fact in facts
]})
relationships = model.active_relationships(document)
if relationships:
groups.append({"title": "Relationships", "rows": [
{
"key": relationship.get("id") or "",
"label": (
f"{model.entity_name(document, relationship['source'])} "
f"{relationship['type']} "
f"{model.entity_name(document, relationship['target'])}"
),
"detail": relationship.get("description") or "",
}
for relationship in relationships
]})
threads = model.open_threads(document)
if threads:
groups.append({"title": "Open Story Threads", "rows": [
{
"key": key,
"label": thread.get("title") or key,
"detail": thread.get("description") or "",
}
for key, thread in sorted(threads.items())
]})
return {"groups": groups, "empty": not groups}
def _title_for(kind: str) -> str:
"""A heading for an entity category the campaign chose.
Pluralised generically rather than from a table, because the categories are
open: `DATA-MODEL.md` §9 suggests nine and permits any, so a lookup would
silently mislabel the tenth.
"""
known = {
"character": "Characters",
"location": "Locations",
"organization": "Organizations",
"item": "Items",
"vehicle": "Vehicles",
"creature": "Creatures",
"structure": "Structures",
"concept": "Concepts",
"other": "Other",
}
if kind in known:
return known[kind]
word = kind.replace("_", " ").strip().title()
return word if word.endswith("s") else word + "s"
def _source_label(fact: dict) -> str:
source = fact.get("authority") or fact.get("source") or ""
return {
"manual_correction": "your correction",
"campaign_canon": "campaign canon",
"accepted_story": "from the story",
}.get(source, str(source).replace("_", " "))