Ran the memory prompt end to end against a real model as a controlled
A/B: one story generated through the app's own build_context a turn at
a time, then both prompts run over the same blocks, so the story is held
constant and the prompt is the only variable. Fresh process per call, so
neither arm sees the other and the model is never told what is being
tested. The control is the exact prompt from 9cdcb55.
The reported fault reproduced. Two consecutive memories written from one
story minutes apart came back in two different persons — "You crept low
through the mist" and "The player asked Gwen to". With the cast brief
both named Kaelen. The control also inverted who acted on a move whose
player text was "grab her wrist and pull her down", which is the failure
the brief predicts: with no cast there is nothing to say whose wrist
"her wrist" was.
It also found something the prompt review had not. "1-2 plain sentences"
is not a length, and the same model wrote 34 words for one block and 105
for the next. A 105-word memory is a paragraph, and `memory_top_k`
injects five every turn, so the bank's running cost was set by a number
nobody had ever stated.
MEMORY_MAX_WORDS states it, and the prompt now says which details to
keep when trimming: the ones a later scene could turn on. Re-run over
the identical story, the same two blocks came back at 32 and 58 words,
still named, still third person, still carrying the camp map, the
strongbox behind the second tent, and the strap frayed near through.
Variance is the real gain — 34..105 became 32..58.
Overshooting 50 slightly is expected. Models exceed word budgets, which
is why builder.length_hint already carries a buffer for the same reason.
What this does not show: the run used a Claude model, and the app talks
to an OpenAI-compatible endpoint whose weaker models are why
worldstate/parse.py tolerates trailing commas. The prompt is followable
and the brief supplies the missing information; a weaker model is not
proven to comply as well.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
Claude
07192767d8