M11: what the server will actually read

The release-validation milestone, and the thing it had to settle first was
whether any of the earlier evidence meant what it said. M8 measured a deployment
enforcing a 4,096-token input window while the application budgeted 16,384.
Every request returned 200. What Ollama does with the excess is drop the oldest
tokens, and the oldest tokens here are the system block — the narrator's rules
and the campaign canon. A hundred-turn certification against that server would
have looked perfect and proved nothing, which is why this milestone could not
begin with a hundred turns.

So the application asks now. Ollama's window is a property of how a model was
loaded rather than of the request — sending num_ctx is accepted, ignored, and
worse, reloads the model at the server's own default — so the only honest move
is to find out and then tell the truth about it. /api/ps reports what a resident
model is being served with, /api/show what an unloaded one will load with, both
on the same host inference already uses, through the same endpoint policy and
the same TLS trust store. A verified window is a ceiling on the budget; an
unverified one leaves the budget alone and is recorded as unverified in the
turn's own provenance, so an old turn can be asked afterwards whether it was
built against a checked window. There is no third behaviour, and in particular
no hard-coded 4,096: a number the server did not say would be right on one
machine and wrong on the next.

The proof that this is doing something is a campaign whose canon sits at the
front of the prompt, 120 turns of history, and a 4,096-token window. The canon
is still there afterwards and the oldest history is gone. The same campaign
built the old way produces a prompt more than twice the window — the defect,
reproduced, so the fix is measured against it rather than asserted.

Two defects the validation found on its own, and they are the same defect twice:
something was true and nobody was told. A manual state correction of four
changes with one bad reference applied three, returned 201, and said nothing —
while recording the refusal on the audit row nobody reads. It came to light
because the identity diagnostic's own fixture was refused that way and the whole
run proceeded on a campaign with no scene, which would have read as a model
failure. And the narration-length setting moved no number: brief, medium and
long each became one English sentence, while the numeric hint the model actually
reads was derived from the global reply cap and said the same thing for all
three. Both now say what they did.

The other two post-M8 findings are closed as well. The tab said AI D&D, which no
document had ever claimed it did not; it says Interactive Story now, with the
open campaign first, and the name is the owner's decision rather than a
find-and-replace to something narrower than the engine. After an Undo the reader
could not tell where they had landed; the control row now ends with
"Moment 11 · later story ahead", from the server's own answer, in the word the
transcript already uses, with none of head, branch or depth anywhere near it.

The identity diagnostic exists and the root cause does not. That campaign was
destroyed, so no cause can be established — what M11 owes the finding is
something that can classify the next occurrence, and a diagnostic that makes only
the judgements a program can honestly make: duplicate keys, shared names,
protagonist drift, state and context disagreeing. Whether prose misattributed a
line is left to a person reading it beside its prompt, because a regex cannot
read dialogue and one that pretended to would produce exactly the confident wrong
answer this finding is about. Its detectors are proved to fire against a planted
second Alice.

Two entities may still share a display name. That was checked first, as the
finding asked, and left permitted: a mother and a daughter, or a stranger giving
a false name, are ordinary fiction, and refusing them to guard against a model
mistake would refuse the wrong thing. What was missing was that it happened
silently. It is reported now.

Evidence, not inference: a hundred accepted turns against a real narrator with
genuine process restarts; a real browser against the built SPA; a container with
no network at all; a campaign moved into a data directory that never existed.
Each was discarded and re-run whenever the product changed under it, and the runs
that were thrown away are listed in the report with the reason, along with ten
defects in the harnesses themselves — because a harness that has only ever
agreed with itself is not evidence, and two of M8's five harness defects were
masking real ones.

No dependency was added, removed or upgraded. No acceptance test was retired,
relaxed or reclassified. M11 is implemented and verified; it is not accepted, and
there is no release tag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qyn3oRd4D6pi72nKBG725B
This commit is contained in:
JesseMarkowitz
2026-09-07 14:01:20 -04:00
co-authored by Claude Opus 5
parent 1013c94eb1
commit 144406cd48
57 changed files with 7374 additions and 97 deletions
+4 -1
View File
@@ -15,7 +15,10 @@
src/styles/fonts.css and served from /fonts/. They used to be linked
from fonts.googleapis.com, which made every page load an Internet
request; see frontend/tools/vendor_fonts.py. -->
<title>AI D&amp;D</title>
<!-- The name is set here for the first paint and kept in step with the
open campaign by src/documentTitle.js, which is where it is
defined. See that file for why it is not "Adventure Storyteller". -->
<title>Interactive Story</title>
</head>
<body>
<div id="root"></div>
+33
View File
@@ -0,0 +1,33 @@
/* The browser tab, and the one place the product's name is written.
*
* M11, post-M8 finding A. The tab still read `AI D&D` — inherited from upstream
* and never claimed by M8, which changed the navigation, the inspector and the
* screens but not the shell metadata. It was an uncovered gap rather than a
* false claim, and it is the first thing a reader sees.
*
* The name is deliberately **not** "Adventure Storyteller", which is what the
* planning package calls the project. `SPECIFICATION.md` requires a
* genre-agnostic engine and the same schema holds a silver key in an abbey and
* a data crystal on an orbital station, so a name with *Adventure* in it is
* narrower than the thing it names. `Interactive Story` is the repository
* owner's decision, recorded here so a later change is one edit.
*
* The campaign comes first because that is what the reader is looking for in a
* row of tabs: `Westhaven — Interactive Story`, not the other way round.
*/
import { useEffect } from 'react'
export const PRODUCT_NAME = 'Interactive Story'
export function titleFor(campaign) {
const name = (campaign || '').trim()
return name ? `${name} — ${PRODUCT_NAME}` : PRODUCT_NAME
}
/** Sets the tab title while mounted, and puts it back on the way out. */
export function useDocumentTitle(campaign) {
useEffect(() => {
document.title = titleFor(campaign)
return () => { document.title = PRODUCT_NAME }
}, [campaign])
}
+134
View File
@@ -0,0 +1,134 @@
/* M11: the three post-M8 playtest findings that changed the browser.
*
* They arrived from a real play session against accepted M8 rather than from a
* test, which is why each one is a case no existing test would have caught: a
* tab title nobody had claimed, a position the reader could not see, and a
* setting that moved no number. The tests are grouped here, by finding, so that
* a reviewer can read them against `BUILD-MILESTONES.md`'s write-up of each.
*/
import { screen } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { api } from './api'
import { PRODUCT_NAME, titleFor } from './documentTitle'
import { Composer } from './pages/Play/Composer'
import { CampaignSettingsPanel } from './pages/Play/panels/CampaignSettingsPanel'
import { composeInstructions } from './pages/NewCampaign'
import { mockModelStatus, renderWith } from './test/helpers'
describe('finding A — the product has a name of its own', () => {
it('is not the inherited one', () => {
expect(PRODUCT_NAME).not.toMatch(/D&D|DnD|Dungeons/i)
})
it('puts the campaign first, because that is what a row of tabs is scanned for', () => {
expect(titleFor('Westhaven')).toBe('Westhaven — Interactive Story')
})
it('falls back to the product name outside a campaign', () => {
expect(titleFor('')).toBe(PRODUCT_NAME)
expect(titleFor(undefined)).toBe(PRODUCT_NAME)
expect(titleFor(' ')).toBe(PRODUCT_NAME)
})
})
describe('finding B — the reader can tell where they are (§8A)', () => {
beforeEach(() => { vi.restoreAllMocks(); mockModelStatus(api) })
const props = {
input: '', setInput: vi.fn(), direction: false, setDirection: vi.fn(),
busy: false, canRetry: true, onSend: vi.fn(), onContinue: vi.fn(),
onRetry: vi.fn(), onUndo: vi.fn(), onRedo: vi.fn(), onSavePoint: vi.fn(),
onStop: vi.fn(),
}
it('names the position in the vocabulary the transcript already uses', async () => {
await renderWith(<Composer {...props} moment={12} canUndo canRedo={false} />)
expect(screen.getByTestId('story-position')).toHaveTextContent('Moment 12')
})
it('says when there is story ahead, which is the other half of being lost', async () => {
// The state right after an Undo: one moment further back, and the moment
// stepped over is still there to walk into.
await renderWith(<Composer {...props} moment={11} canUndo canRedo />)
const position = screen.getByTestId('story-position')
expect(position).toHaveTextContent('Moment 11')
expect(position).toHaveTextContent('later story ahead')
})
it('says nothing about story ahead when there is none', async () => {
await renderWith(<Composer {...props} moment={11} canUndo canRedo={false} />)
expect(screen.getByTestId('story-position')).not.toHaveTextContent('ahead')
})
it('uses no implementation vocabulary (§38)', async () => {
await renderWith(<Composer {...props} moment={11} canUndo canRedo />)
const text = screen.getByTestId('story-position').textContent.toLowerCase()
// Whole words. The first version of this test used substring matching and
// failed on "ahead", which contains "head" — the same class of harness
// defect M10 hit when a search for "media" matched "immediately".
for (const word of ['branch', 'fork', 'node', 'head', 'depth']) {
expect(text).not.toMatch(new RegExp(`\\b${word}\\b`))
}
})
it('shows nothing on a campaign that has no story yet', async () => {
await renderWith(<Composer {...props} moment={0} canUndo={false} canRedo={false} />)
expect(screen.queryByTestId('story-position')).toBeNull()
})
})
describe('finding C — narration length is a setting with an effect', () => {
beforeEach(() => { vi.restoreAllMocks() })
const campaign = {
id: 1, title: 'Continuity Test', ai_instructions: '', canon_rules: [],
persona_name: 'Aldric', persona_desc: '', narration_length: 'brief',
}
it('shows the campaign its own current choice', async () => {
await renderWith(
<CampaignSettingsPanel
adventure={campaign} setAdventure={vi.fn()} onError={vi.fn()} moments={3} />,
)
expect(screen.getByLabelText('Narration length')).toHaveValue('brief')
})
it('saves the choice as data, not only as a sentence', async () => {
// The finding: the setup screen turned the choice into one English sentence
// inside the instructions and changed no number, so the storyteller could
// not act on it. What is asserted here is that the field the prompt builder
// reads is the field this control writes.
const update = vi.spyOn(api, 'updateAdventure').mockResolvedValue({
...campaign, narration_length: 'long',
})
await renderWith(
<CampaignSettingsPanel
adventure={campaign} setAdventure={vi.fn()} onError={vi.fn()} moments={3} />,
)
await userEvent.selectOptions(screen.getByLabelText('Narration length'), 'long')
await userEvent.click(screen.getByRole('button', { name: /save/i }))
expect(update).toHaveBeenCalledWith(1, expect.objectContaining({
narration_length: 'long',
}))
})
it('offers having no preference at all', async () => {
await renderWith(
<CampaignSettingsPanel
adventure={{ ...campaign, narration_length: '' }}
setAdventure={vi.fn()} onError={vi.fn()} moments={0} />,
)
expect(screen.getByLabelText('Narration length')).toHaveValue('')
})
it('still writes the sentence, so the reader can read and edit it', () => {
// Both halves are kept deliberately: the sentence is what a person editing
// the instructions sees, and the field is what the builder measures.
const written = composeInstructions({
genre: 'fantasy', tone: 'grim', pov: 'second-present', length: 'brief', style: '',
})
expect(written).toContain('brief')
})
})
+4
View File
@@ -137,6 +137,10 @@ export default function NewCampaign() {
persona_name: protagonist.trim(),
persona_pronouns: pronouns.trim(),
persona_desc: protagonistDesc.trim(),
// M11 (post-M8 finding C): sent as data as well as prose. The sentence
// below still goes into the instructions where the reader can edit it;
// this is what the prompt builder turns into an actual word range.
narration_length: length,
})
const instructions = composeInstructions({ genre, tone, pov, length, style })
if (instructions) {
+19
View File
@@ -46,6 +46,7 @@ export function Composer({
canUndo,
canRedo,
canRetry,
moment,
onSend,
onContinue,
onRetry,
@@ -97,6 +98,24 @@ export function Composer({
<button type="button" onClick={onSavePoint} disabled={busy}>
Save Point
</button>
{/* M11, post-M8 finding B / BROWSER-UX-SPEC §8A. Undo worked and the
reader still could not tell where they had landed: the transcript
simply got shorter, which is not something you notice happening to
a page you were already scrolled into.
So the position is stated, in the vocabulary the transcript already
uses for it ("Read the 12 earlier moments"), and it says whether
there is story ahead — which is the other half of being lost after
an Undo: not knowing whether what you stepped back over still
exists. No implementation words (§38): not head, not branch, not
depth. */}
{moment > 0 && (
<p className="story-position" data-testid="story-position" aria-live="polite">
<span className="moment">Moment {moment}</span>
{canRedo && <span className="ahead"> · later story ahead</span>}
</p>
)}
</div>
<div className={`input-bar ${direction ? 'directing' : ''}`}>
+4
View File
@@ -30,6 +30,7 @@ import { useNavigate, useParams } from 'react-router-dom'
import { api } from '../../api'
import { AutoTextarea, useToast } from '../../components'
import { ConfirmDialog } from '../../Dialog'
import { useDocumentTitle } from '../../documentTitle'
import { classifyError } from '../../errors'
import { ExternalLinkDialog } from '../../ExternalLinkDialog'
import { ModelSetupNotice } from '../../ModelSetupNotice'
@@ -49,6 +50,8 @@ export default function Play() {
const { status: modelStatus, refresh: refreshModelStatus } = useModelStatus()
const [adventure, setAdventure] = useState(null)
// M11 (post-M8 finding A): the tab says which campaign is open.
useDocumentTitle(adventure?.title)
const [actions, setActions] = useState([])
const [input, setInput] = useState('')
const [direction, setDirection] = useState(false)
@@ -609,6 +612,7 @@ export default function Play() {
busy={busy}
canUndo={history.undo}
canRedo={history.redo}
moment={total}
canRetry={lastIsAi}
onSend={send}
onContinue={() => send('continue')}
@@ -28,6 +28,7 @@ export function CampaignSettingsPanel({ adventure, setAdventure, onError, moment
const toast = useToast()
const [title, setTitle] = useState(adventure.title || '')
const [instructions, setInstructions] = useState(adventure.ai_instructions || '')
const [narrationLength, setNarrationLength] = useState(adventure.narration_length || '')
const [canon, setCanon] = useState((adventure.canon_rules || []).join('\n'))
const [persona, setPersona] = useState(adventure.persona_name || '')
const [personaDesc, setPersonaDesc] = useState(adventure.persona_desc || '')
@@ -39,6 +40,7 @@ export function CampaignSettingsPanel({ adventure, setAdventure, onError, moment
useEffect(() => {
setTitle(adventure.title || '')
setInstructions(adventure.ai_instructions || '')
setNarrationLength(adventure.narration_length || '')
setCanon((adventure.canon_rules || []).join('\n'))
setPersona(adventure.persona_name || '')
setPersonaDesc(adventure.persona_desc || '')
@@ -60,6 +62,7 @@ export function CampaignSettingsPanel({ adventure, setAdventure, onError, moment
const updated = await api.updateAdventure(adventure.id, {
title: title.trim() || 'Untitled campaign',
ai_instructions: instructions,
narration_length: narrationLength,
canon_rules: canon.split('\n').map((r) => r.trim()).filter(Boolean),
persona_name: persona.trim(),
persona_desc: personaDesc,
@@ -108,7 +111,29 @@ export function CampaignSettingsPanel({ adventure, setAdventure, onError, moment
value={instructions} onChange={edit(setInstructions)}
/>
<p className="field-hint">
Genre, tone, voice, length — anything you want true of every turn.
Genre, tone, voice — anything you want true of every turn.
</p>
</div>
{/* M11, post-M8 finding C. Length was a sentence in the box above and
nothing else, so it read as a preference the storyteller could not
act on. It is its own control now because the prompt builder turns it
into an actual word range (`context.builder.LENGTH_BANDS`). */}
<div className="field">
<label htmlFor="cs-length">Narration length</label>
<select
id="cs-length"
value={narrationLength}
onChange={edit(setNarrationLength)}
>
<option value="">No preference</option>
<option value="brief">Brief — a moment, not a scene</option>
<option value="medium">Medium — a scene</option>
<option value="long">Long — a scene with room to breathe</option>
</select>
<p className="field-hint">
Roughly 70–180, 150–380 or 320–700 words. A shorter reply limit in
Settings still wins, since that is what the model is given room for.
</p>
</div>
+27 -2
View File
@@ -37,6 +37,9 @@ function StatePanel({ advId, refreshKey, onCorrected, onError }) {
return () => { cancelled = true }
}, [advId, refreshKey, tick])
// M11: refusals from the most recent correction (see `saveCorrection`).
const [refused, setRefused] = useState([])
async function showHistory() {
if (history) { setHistory(null); return }
try {
@@ -61,7 +64,11 @@ function StatePanel({ advId, refreshKey, onCorrected, onError }) {
predicate: text,
...(correcting.key ? { subject: correcting.key } : {}),
}]
await api.correctNarrativeState(advId, events, text)
const answer = await api.correctNarrativeState(advId, events, text)
// M11: a correction can be partly refused, and until M11 that came back
// looking like success. The reader is told which change did not land and
// why, rather than discovering later that the state never changed.
setRefused(answer?.refused || [])
setCorrecting(null)
setTick((t) => t + 1)
// The corrected state is what the next turn is built from, so anything
@@ -76,8 +83,9 @@ function StatePanel({ advId, refreshKey, onCorrected, onError }) {
async function withdraw(factId) {
try {
await api.correctNarrativeState(
const answer = await api.correctNarrativeState(
advId, [{ type: 'invalidate_fact', fact_id: factId }])
setRefused(answer?.refused || [])
setTick((t) => t + 1)
onCorrected()
} catch (err) {
@@ -90,6 +98,23 @@ function StatePanel({ advId, refreshKey, onCorrected, onError }) {
return (
<div className="state-panel">
{refused.length > 0 && (
<p className="state-refused" role="status" data-testid="correction-refused">
{refused.length === 1 ? 'One change was not applied' :
`${refused.length} changes were not applied`}
{': '}
{refused.map((r) => r.detail || r.reason).join('; ')}
</p>
)}
{state.duplicate_names && Object.keys(state.duplicate_names).length > 0 && (
<p className="state-note" data-testid="duplicate-names">
Two or more characters share a name
{': '}
{Object.entries(state.duplicate_names)
.map(([name, keys]) => `${name} (${keys.length})`).join(', ')}
. That is allowed — the storyteller may find it harder to tell them apart.
</p>
)}
{state.empty && (
<p className="state-intro">
Nothing established yet. As the story names people, places and things,
+16
View File
@@ -364,6 +364,22 @@ export default function Settings() {
: testResult.detail}
</p>
)}
{/* M11: the context window the server will actually give this model.
Its own line rather than folded into the message above, because a
missing model and a too-small window are different problems with
different fixes, and a reader can have both at once. */}
{testResult?.window?.warning && (
<p className="test-result warn" role="status" data-testid="window-warning">
{testResult.window.warning}
</p>
)}
{testResult?.window?.verified && !testResult.window.warning && (
<p className="test-result ok" data-testid="window-ok">
Context window checked: {testResult.window.tokens.toLocaleString()} tokens,
which covers the {testResult.window.budget.toLocaleString()}-token story budget.
</p>
)}
</section>
<section className="settings-section">
+19
View File
@@ -201,3 +201,22 @@
margin: 6px 0;
}
/* M11: a correction that was partly refused, and a shared display name. Both
are things the reader must be able to see rather than infer — the first
because a change they made did not land, the second because it is one of
post-M8 finding D's candidate failure modes. */
.state-refused {
margin: 0 0 10px;
padding: 8px 10px;
border-left: 3px solid var(--warning);
background: rgba(224, 154, 95, 0.08);
color: var(--text);
font-size: 0.82rem;
}
.state-note {
margin: 0 0 10px;
color: var(--text-dim);
font-size: 0.8rem;
}
+3
View File
@@ -189,6 +189,9 @@
.test-result { margin-top: 12px; font-size: 0.83rem; }
.test-result.ok { color: var(--accent); }
.test-result.bad { color: var(--danger); }
/* M11: a window that could not be checked, or is smaller than the budget.
Not an error — the story still plays — so it is not --danger. */
.test-result.warn { color: var(--warning); }
.advanced-block {
margin-top: 14px;
+13 -1
View File
@@ -289,7 +289,19 @@
background: linear-gradient(to top, var(--bg) 62%, transparent);
}
.story-controls { display: flex; gap: 6px; margin-bottom: 8px; flex-wrap: wrap; }
.story-controls { display: flex; gap: 6px; margin-bottom: 8px; flex-wrap: wrap; align-items: center; }
/* M11: where the reader is in the story (post-M8 finding B). Pushed to the end
of the control row so it reads as a status rather than another button, and
quiet enough that it is not competing with the prose. */
.story-position {
margin: 0 0 0 auto;
font-size: 0.78rem;
color: var(--text-dim);
white-space: nowrap;
}
.story-position .moment { color: var(--text); }
.story-position .ahead { color: var(--accent); }
.story-controls button {
padding: 5px 13px;
font-size: 0.78rem;
+5
View File
@@ -12,6 +12,11 @@
--accent-dim: #96773a;
--accent-glow: rgba(212, 169, 78, 0.25);
--danger: #d06565;
/* M11: a caution that is not a failure — a context window that could not be
checked, or is smaller than the story budget. Distinct in hue from both
--accent and --danger, and measured at 7.85:1 against --bg-panel
(backend/tools/contrast_audit.py), so it clears WCAG AA for body text. */
--warning: #e09a5f;
--player: #9fc7d1;
/* Chart hues (analytics dashboard). Deeper and more saturated than --accent
and --player, which are tuned for text and borders and turn muddy once