interactive-story/plan/18-persona-and-memory-quality.md

The protagonist has no name, and the summarizer is told nothing

Two changes, in order. Phase 1 gives the adventure a persona. Phase 2 uses it, along with the cast, to fix the memories. Phase 1 is worth shipping on its own; Phase 2 depends on it and is much smaller once it lands.

Both phases are built and green (610 backend tests). Phase 1 was driven in a browser (21/21 checks). Phase 2's prompts were verified against the real seeded scenario — see "Phase 2 as built" — but no memory has been generated by a real model yet, because that needs an API key.

Last updated: 2026-08-31.

---

The one sentence version

Memories come back inconsistent and vague because the summarizer is handed six actions of second-person prose and nothing else — no protagonist, no cast, no setting, no instruction about what person to write in — so it cannot say who "you" is or who "she" is, and neither can any memory it writes.

The evidence

This is the entire prompt that writes a memory (memorybank.py:391):

system: You compress interactive-fiction story excerpts into memories. Respond
        with 1-2 plain sentences in past tense stating the concrete facts and
        events (names, places, items, promises, injuries). No preamble, no
        commentary.

user:   Story excerpt:

        <6 actions, joined by blank lines, truncated to 2000 tokens>

        Memory:

Nothing else is passed. Not adventure.memory (the plot essentials), not the NPC names and descriptions the scenario already defines, not the world state, and not the protagonist — because until Phase 1 there is no protagonist to pass.

Two failures follow from that, and they are the two complaints:

Inconsistent framing. The prompt never says what person to write in. The model picks one per call. Across a single bank you get "You entered the crypt", "The player entered the crypt", and "He entered the crypt" describing the same kind of event.

No idea who anyone is. Given You push the door open. She grabs your arm. the only honest memory is "You entered a room and she stopped you." Retrieved forty turns later into a scene with three women in it, that memory is worse than nothing.

The summary inherits both problems, because _update_story_summary builds from the memory bullets.

---

Phase 1 — the persona

What already exists, and what does not

stat_schema already gives the player a stat block, and it is already namespaced beside the NPCs:

world.day
player.hp          <- the player's stats, already sectioned
npc.gwen.trust
flags.alarm_raised
milestones.escaped

NPCs carry name, keys, and desc (schema.py, render_reference). The player section carries none of those. That asymmetry is the whole gap: the player has stats but no identity, so the block renders as You: hp 100/100 and reads to the model as a floating global rather than a character.

The paths do not change

player.hp stays player.hp. It does not become kaelen.hp or user.hp.

The reason is _history_text() in context/builder.py. It replays every past turn's stored delta back into the prompt, and those stored blobs contain literal strings like {"player.hp": -15}. A path that carries the persona's name breaks the moment a player renames their character: every replayed block in history then names a path the schema no longer defines, applied_delta renders it anyway, and the model copies the broken form. A rename would also require migrating every stored world_state, every seed scenario, and the EMIT_RULE example text, for no functional gain.

What changes is the label, using the trick NPCs already use — print the display name and the path together:

before:  You: hp 100/100, mana 30/50.
after:   Kaelen (player): hp 100/100, mana 30/50.

The name is visible to the reader, the path stays copyable by the model.

Where the persona lives

Three plain columns on adventures, alongside memory and authors_note:

persona_name: Mapped[str] = mapped_column(String(80), default="")
persona_pronouns: Mapped[str] = mapped_column(String(40), default="")
persona_desc: Mapped[str] = mapped_column(Text, default="")

Not in stat_schema. Two reasons. It has to work for an adventure with no RPG layer at all, which is most of them, and _initials() in worldstate/schema.py treats every dict inside a section as a stat definition — a persona key dropped into stat_schema.player would be instantiated as a stat, rendered as a stat line in the guide, and handed an initial value.

Adventure-level, not scenario-level. Two people playing the same scenario are different characters. A scenario steers the protagonist through its plot essentials, which it already can. Keeping personas off scenarios also avoids having to decide what "Update from scenario" does to a persona the player has edited: the answer is nothing, because the scenario never had one.

Not per-branch. Branches are alternative futures within one adventure; the protagonist is the same person down all of them.

Empty persona_name means the feature is off and behavior is exactly what it is today. That is the entire backward-compatibility story — no backfill.

Pronouns get their own field

One short string: he/him, she/her, they/them. It exists because Phase 2 will tell the summarizer to write in third person. Without a stated pronoun the model infers one from the name, and once it infers wrong that error is baked into every memory it writes from then on and into the summary built from them. A field is cheaper than a wrong guess repeated forever.

Blank is a valid value. When it is blank nothing is rendered, and the Phase 2 prompt tells the summarizer to use the name rather than a pronoun.

Where it enters the prompt

A new static section in build_context, between ai_instructions and plot_essentials:

Player character:
You are Kaelen (he/him). A half-elf ranger, exiled from the northern holds.

It goes in the system block on purpose. The description is user-only and never changes during a turn, so it sits inside the cached prefix and costs nothing after the first turn. This is why "only the user can change it" is not just a product choice — it is what keeps the section free. Anything the AI could rewrite would have to move below the history with the other live sections, and would re-price the prompt on every change.

It is emitted whether or not the adventure has an RPG layer. It is not gated behind has_ws.

Where it enters the world state

worldstate/render.py, two edits:

• render_state_section takes the display name and prints Kaelen (player): … in place of the hardcoded You: …. Falls back to You: when no persona is set.
• render_reference gains one line tying the name to the path, mirroring the NPC header it already writes: - Protagonist Kaelen — stats are addressed as player.<stat>. Without this the model reads "Kaelen" in the prose and invents kaelen.hp.

The persona description is not repeated in the stat guide. It already has its own section, and the guide is about paths.

The AI cannot edit it

Nothing to build. _resolve() in worldstate/apply.py rejects any path it does not recognize, so a delta of {"persona.name": "Bob"} already falls through to "persona.name is not a tracked value", and that refusal already reaches the model through render_refusals. Worth one test to hold the behavior in place.

Setting it, and changing it

At the start. PlaceholderModal (components.jsx) already exists and already opens before an adventure begins — but only when the scenario contains ${...} tokens. It becomes a "Begin adventure" modal that always opens, with the three persona fields at the top and any placeholder fields below.

Placeholders stay independent of the persona. A scenario using ${Name} will ask for a name twice. That is accepted: no scenario in the repo uses placeholders at all (grepped across seed_data/ and starter_data/ — zero occurrences), so the collision is hypothetical. If it ever shows up in practice, pre-filling the Name field from the persona is a few lines in the modal with no backend change.

Later. AdventureUpdate gains the three fields, and PlotPanel gets a Protagonist block at the top. The debounced-save wiring is already there. WorldStateDrawer shows the name as the character-sheet heading but does not edit it — one edit surface, not two.

Files

File	Change
models.py	three columns on Adventure
migrations.py	74, 75, 76
schemas.py	AdventureCreate, AdventureUpdate, AdventureOut
context/builder.py	persona section after plot_essentials
worldstate/render.py	state-section label, reference line
worldstate/__init__.py	export whatever render.py newly exposes
routers/adventures/crud.py	store persona at creation
bundle.py	export key + import read
frontend/components.jsx	modal gains persona fields
frontend/pages/Scenarios.jsx, Home.jsx	always open the modal
frontend/pages/Play/panels/PlotPanel.jsx	Protagonist block
frontend/pages/Play/drawers/WorldStateDrawer.jsx	heading
frontend/api.js	pass the fields through

Tests

File	What
new test_persona.py	create with persona; update; renders in the system block; works with no stat_schema; empty name is a no-op; AI delta at persona.* refused
test_worldstate.py	state line reads Kaelen (player):; player.hp still resolves
test_prompt_caching.py	persona is in the static prefix, above history
test_bundle_v2.py	round-trips; a bundle without the key still imports

Bundle format

No FORMAT bump. The export gains a persona key and the import reads it with .get(), so a v2 bundle written before this change imports with an empty persona — which is the same as not having one.

---

Phase 2 — what the summarizer is told

Depends on Phase 1 only for the protagonist's name. Everything else it needs is already in the database and simply never passed.

The cast brief

Build one short block and prepend it to both the memory prompt and the summary prompt:

Cast:
- Kaelen (he/him) — the protagonist. A half-elf ranger, exiled from the
  northern holds...
- Gwen — a loyal ranger and Kaelen's ally. Quick with a bow, dry-humoured.

Setting:
<adventure.memory, the plot essentials>

Sources, in order:

1. Protagonist — the Phase 1 persona. Falls back to "the player" when unset, so the change still helps adventures with no persona.
2. NPCs — stat_schema.npcs: name + desc. Already there, never used outside the turn prompt.
3. Story cards — for adventures with no RPG layer, run the existing _match_cards() over the block being summarized and take the matched cards' names and entries. This reuses the trigger code and yields exactly the entities that appear in that block, not the whole world.

Two rules for the brief

Fixed descriptions only. No live stats. It is tempting to include Gwen: trust 40 (wary). Do not. It makes every memory prompt different, which loses prompt caching, and worse, it makes the same event summarized at two different times come out framed differently — which is the problem being fixed.

The summary gets it for free. _update_story_summary builds from the memory bullets, so better memories produce a better summary with no further change. Pass the brief there too, because that function falls back to raw story text when memory-writing has fallen behind.

The prompt changes

MEMORY_SYSTEM_PROMPT gains an explicit framing rule:

• write in third person, never "you"
• refer to the protagonist by name
• name characters rather than using bare pronouns

Expected effect:

before:  You entered a room and she stopped you.
after:   Kaelen bribed Gwen with fifty silver to hold the north door while
         he went down alone.

---

Phase 1 as built

Everything above describes what shipped. Three notes on where it differs from the sketch, and what has not been verified.

Decisions taken during the build

The section reads as one paragraph. render_persona joins the sentence and the description with a space rather than SEPARATOR. A blank line inside a two-sentence section about one character reads as two unrelated notes.

Field gained a maxLength prop. The persona name is VARCHAR(80) and the schema rejects 81 characters with a 422. Without the attribute the player only learns that from a toast after typing, so the cap is now enforced in the input as well. Every other Field is unaffected — the prop is optional.

The blank-adventure button opens the modal too. It used to create the adventure immediately. It now collects a persona first, and passes title: 'Blank Adventure' explicitly, because there is no scenario to take a title from.

test_prompt_caching.py's fixture now sets a persona, so every test in the file that guards the static block runs with one present.

Verified

• Migration 73 → 76 on a real pre-existing SQLite database: columns added, the existing row preserved, persona empty, render_persona returns "".
• 593 backend tests pass, including 26 new ones in test_persona.py and 3 in test_bundle_v2.py.
• npm run build is clean.

Driven in a browser

Chromium via Playwright, against a fresh database with the demo scenarios seeded. No API key needed: Insights assembles the prompt without calling a model, so every check below runs on the real assembled context rather than on a unit-test stub. 21/21 checks passed.

What was confirmed on screen and in the live /context payload:

The modal opens for a scenario with no ${...} placeholders	it is now the only way to name a character, so it can no longer be conditional
The section order is real	narrator, world_state_guide, world_state_rule, ai_instructions, **persona**, plot_essentials, …
The persona is in the system half	You are Kaelen (he/him). A half-elf ranger… present in prompt.system, absent from prompt.story
The live stat line carries the name and the path	Kaelen (player): hp 100/100 (full health), mana 30/50 (brimming)…
The stat guide ties the name to the path	Protagonist Kaelen … player.<stat>
The drawer heading follows the name	rail read KAELEN, not You
A rename propagates	renamed to Aria in the Plot panel → drawer read ARIA after a reload, and the prompt read You are Aria (he/him). / Aria (player):
Clearing every field restores the old behavior exactly	no persona section, stat line back to You:, no Protagonist line in the guide
The blank-adventure button collects a persona and keeps its title	title='Blank Adventure' persona='Wren'
A persona works with no RPG layer at all	a blank adventure's sections were ['narrator', 'persona', 'length_hint'] — the case this feature was added for

Two request failures in the run were the sandbox rather than the app: Google Fonts is blocked by the egress policy, and the analytics beacon is aborted when the page unloads. Neither appears with normal network access.

The driver script is not in the repo. There is no frontend test runner yet (plan/17 stage 5), and one Playwright script is not the place to start one.

A note for whoever runs the tests

tiktoken downloads cl100k_base from openaipublic.blob.core.windows.net on first use, and 182 tests fail with a proxy error where that host is blocked. The encoding is reconstructable offline from the npm package js-tiktoken, whose dist/ranks/cl100k_base.cjs holds the same ranks in a compressed form — decode it, write <base64> <rank> per line sorted by rank, and the result matches the SHA-256 that tiktoken_ext/openai_public.py hardcodes, so the reconstruction is verifiable rather than trusted. Drop it at $TIKTOKEN_CACHE_DIR/<sha1 of the URL>.

---

Phase 2 as built

The cast comes from the story cards, not from stat_schema

This is the one thing the sketch above got wrong, and it made the change much smaller. scenario_text.scenario_card_specs already turns every schema NPC into a story card on the adventure at creation, deduplicated against the hand-written cards by name. So the cards are a single unified cast source that covers schema NPCs, an author's own cards, and an adventure with no RPG layer, through one path instead of three. memorybank never reads stat_schema.

Keyword matching alone was not enough

The sketch said to run _match_cards over the block. Built that way first, and it failed the exact case the change exists for.

The block You push the door open. She grabs your arm. matches no card keyword, so the brief listed the protagonist and nobody else — leaving the summarizer guessing at precisely the moment it was handed a brief to stop guessing. Seed 04 gives Gwen the trigger keys "Gwen, ranger, her", and even that does not save it: the text says "she", not "her".

So the roster is matched cards first, then topped up with the other character cards to MAX_CAST_MEMBERS. Places and items are not topped up — an unmentioned tavern is not who "she" was — but a place that is mentioned still matches normally.

That asymmetry with the turn prompt is deliberate. Including an untriggered card as lore would be wrong: it is not relevant to the next sentence. Including an untriggered character in a roster is right: the question the roster answers is "who could these pronouns be", not "what is on stage".

_match_cards became match_cards

Two callers now run the same rule, so it is public and exported from app.context. One rule, one implementation.

What actually gets sent

Verified against the seeded Bandit Camp scenario, with the real _create_due_memories path and a stub provider:

Cast:
- Kaelen (he/him) — the protagonist. A half-elf ranger, exiled from the
  northern holds for a killing he still won't explain.
- Bandit Camp — A rough camp of bandits in a forest clearing, holding a
  stolen caravan strongbox.
- Gwen — A loyal ranger and the player's ally. Quick with a bow,
  dry-humoured, fiercely protective. …
- Bandit Leader — The scarred leader of the bandit camp, guarding the
  strongbox. …

Setting:
The player and Gwen, a loyal ranger ally, are raiding a bandit camp to
recover a stolen strongbox. …

Story excerpt:

… You are six paces from the strongbox when she hisses a warning. …

Memory:

That last line is the case in miniature: "she" is now resolvable.

With no persona set, the roster still lists the NPCs and the setting, and the system prompt tells the model to call the protagonist "the player". Phase 2 therefore improves adventures that never set a persona at all.

With no persona, no cards and no plot essentials, the user message is byte for byte what it was before this change — Story excerpt: first, no stray blank lines. A test holds that.

Decided, from the sketch's open questions

Existing memories: left alone. A bank written under the old prompt will mix "you" and named framing for a while. Re-summarizing costs one call per 6 actions and duplicates whatever is still in the bank, because nothing deletes the old rows. Eviction ages them out at memory_bank_capacity (80). Revisit only if the mixture visibly hurts.

Retrieval framing mismatch: watched, not fixed. retrieve_memories still embeds the last 4 actions raw, in second person, while new memories are third person and named. Embeddings handle paraphrase well, so this is speculative. If retrieval quality visibly dips, prepending the same brief to the query text is the first thing to try.

Still to check with a real model

Everything above is the prompt, not the output. Nobody has yet run a real provider over it and read the memories that come back. That needs an API key, and it is the only way to know whether the framing rule actually holds across a whole adventure. What to look for:

1. Every new memory names the protagonist and uses no bare pronouns.
2. The framing is the same across memories written many turns apart.
3. The rewritten story summary inherits it.
4. Memories do not get noticeably longer — the brief is context, not content to be repeated back.