M11: what the server will actually read

The release-validation milestone, and the thing it had to settle first was
whether any of the earlier evidence meant what it said. M8 measured a deployment
enforcing a 4,096-token input window while the application budgeted 16,384.
Every request returned 200. What Ollama does with the excess is drop the oldest
tokens, and the oldest tokens here are the system block — the narrator's rules
and the campaign canon. A hundred-turn certification against that server would
have looked perfect and proved nothing, which is why this milestone could not
begin with a hundred turns.

So the application asks now. Ollama's window is a property of how a model was
loaded rather than of the request — sending num_ctx is accepted, ignored, and
worse, reloads the model at the server's own default — so the only honest move
is to find out and then tell the truth about it. /api/ps reports what a resident
model is being served with, /api/show what an unloaded one will load with, both
on the same host inference already uses, through the same endpoint policy and
the same TLS trust store. A verified window is a ceiling on the budget; an
unverified one leaves the budget alone and is recorded as unverified in the
turn's own provenance, so an old turn can be asked afterwards whether it was
built against a checked window. There is no third behaviour, and in particular
no hard-coded 4,096: a number the server did not say would be right on one
machine and wrong on the next.

The proof that this is doing something is a campaign whose canon sits at the
front of the prompt, 120 turns of history, and a 4,096-token window. The canon
is still there afterwards and the oldest history is gone. The same campaign
built the old way produces a prompt more than twice the window — the defect,
reproduced, so the fix is measured against it rather than asserted.

Two defects the validation found on its own, and they are the same defect twice:
something was true and nobody was told. A manual state correction of four
changes with one bad reference applied three, returned 201, and said nothing —
while recording the refusal on the audit row nobody reads. It came to light
because the identity diagnostic's own fixture was refused that way and the whole
run proceeded on a campaign with no scene, which would have read as a model
failure. And the narration-length setting moved no number: brief, medium and
long each became one English sentence, while the numeric hint the model actually
reads was derived from the global reply cap and said the same thing for all
three. Both now say what they did.

The other two post-M8 findings are closed as well. The tab said AI D&D, which no
document had ever claimed it did not; it says Interactive Story now, with the
open campaign first, and the name is the owner's decision rather than a
find-and-replace to something narrower than the engine. After an Undo the reader
could not tell where they had landed; the control row now ends with
"Moment 11 · later story ahead", from the server's own answer, in the word the
transcript already uses, with none of head, branch or depth anywhere near it.

The identity diagnostic exists and the root cause does not. That campaign was
destroyed, so no cause can be established — what M11 owes the finding is
something that can classify the next occurrence, and a diagnostic that makes only
the judgements a program can honestly make: duplicate keys, shared names,
protagonist drift, state and context disagreeing. Whether prose misattributed a
line is left to a person reading it beside its prompt, because a regex cannot
read dialogue and one that pretended to would produce exactly the confident wrong
answer this finding is about. Its detectors are proved to fire against a planted
second Alice.

Two entities may still share a display name. That was checked first, as the
finding asked, and left permitted: a mother and a daughter, or a stranger giving
a false name, are ordinary fiction, and refusing them to guard against a model
mistake would refuse the wrong thing. What was missing was that it happened
silently. It is reported now.

Evidence, not inference: a hundred accepted turns against a real narrator with
genuine process restarts; a real browser against the built SPA; a container with
no network at all; a campaign moved into a data directory that never existed.
Each was discarded and re-run whenever the product changed under it, and the runs
that were thrown away are listed in the report with the reason, along with ten
defects in the harnesses themselves — because a harness that has only ever
agreed with itself is not evidence, and two of M8's five harness defects were
masking real ones.

No dependency was added, removed or upgraded. No acceptance test was retired,
relaxed or reclassified. M11 is implemented and verified; it is not accepted, and
there is no release tag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qyn3oRd4D6pi72nKBG725B
This commit is contained in:
JesseMarkowitz
2026-09-07 14:01:20 -04:00
co-authored by Claude Opus 5
parent 1013c94eb1
commit 144406cd48
57 changed files with 7374 additions and 97 deletions
+50 -3
View File
@@ -985,7 +985,7 @@ accepted at closeout on 2026-09-06. The independent review returned
the closeout completed: the build-evidence classification in the report's §P and
finding 14's resolution.
`planning/reports/M8-IMPLEMENTATION-REPORT.md` records what was built, what was
`planning/archive/milestone-reports/M8-IMPLEMENTATION-REPORT.md` records what was built, what was
measured and every finding, including the seven product defects verification
found, the five harness defects, and the evidence runs that were discarded.
@@ -1122,7 +1122,7 @@ A campaign can be safely exported, imported into a clean data directory, and reo
Implemented on `m9-recovery` from the signed M8 commit `1ce9972`, measured
before and after against the same fixture, and verified in a real browser
against a real narrator. `planning/reports/M9-IMPLEMENTATION-REPORT.md` is the
against a real narrator. `planning/archive/milestone-reports/M9-IMPLEMENTATION-REPORT.md` is the
implementer's account, written for a reviewer.
**What it delivered, beyond the scope list above:**
@@ -1348,7 +1348,7 @@ Future media providers can be added through defined local interfaces without red
## Status: COMPLETE — 2026-09-07, pending independent review
Implemented on `m10-media-hooks` from the signed M9 commit `44edece`.
`planning/reports/M10-IMPLEMENTATION-REPORT.md` is the implementer's account,
`planning/archive/milestone-reports/M10-IMPLEMENTATION-REPORT.md` is the implementer's account,
written for a reviewer.
**The finding that shaped the milestone: the scene snapshot already existed.**
@@ -1449,6 +1449,53 @@ Release gate in `V1-ACCEPTANCE-TESTS.md`:
The build meets the v1 black-box acceptance contract and can be packaged as the first production release.
## Status: IMPLEMENTED AND VERIFIED — 2026-09-07, awaiting independent review/acceptance
Implemented on `m11-release-validation` from the signed M10 commit `1013c94`.
`planning/reports/M11-IMPLEMENTATION-REPORT.md` is the evidence package, written
for a release reviewer. **M11 is not marked accepted here**; that is the
reviewer's to record, and no release tag exists.
**The release blocker it was given, and how it was closed.** M8 measured the
reference deployment enforcing a **4,096**-token input window while the
application budgeted **16,384** — every request returning 200, and `llama.cpp`
dropping the *oldest* tokens, which in this design are the narrator's rules and
the campaign canon. A 100-turn certification against that server would have
looked perfect and proved nothing.
M11's invariant: *the application must not silently budget more narrator input
than the runtime will accept.* It now asks the server — `/api/ps` for a loaded
model, `/api/show` for one that is not — under the same endpoint policy and TLS
trust as inference, and caps the prompt to what it finds, or records the window
as unverified in the turn's own provenance. Not a hard-coded 4,096, which would
cripple a correctly configured deployment; not a guess from the model's name.
Measured on the reference server: the plain model reports 4,096 and the budget
caps to it; the `num_ctx`-baked model reports 16,384 and the full budget stands.
**The four post-M8 playtest findings, disposed of:**
| | Disposition |
| --- | --- |
| **A.** The tab read `AI D&D` | **Fixed.** `Interactive Story`, with the open campaign first — a name chosen by the repository owner, and deliberately not "Adventure Storyteller", which is narrower than a genre-agnostic engine. One module owns it. |
| **B.** No orientation after Undo | **Fixed.** `Moment 11 · later story ahead`, from the server's own answer, in the transcript's existing vocabulary, with no implementation words. `BROWSER-UX-SPEC.md` §8A records the implementation. |
| **C.** Narration length had no effect | **Fixed.** The choice is data (`adventures.narration_length`), and the prompt builder turns it into a real word band. The generation budget is deliberately untouched: capping it would truncate prose, and the state block is emitted last. |
| **D.** Character identity confusion | **Diagnostic built; root cause remains unestablished, as it must.** The campaign was destroyed. `tools/m11_identity.py` runs the finding's own scenario, makes only the judgements a program can make honestly, preserves everything on a signal, and proves its detectors fire. The structural fact the finding asked M11 to check first — two entities may share a display name silently — is now **reported** rather than refused, because two people called Alice is ordinary fiction. |
**Two product defects found by the release validation itself**, both the same
family — something true that nobody was told:
1. **A partly refused manual state correction reported success.** Four changes,
one refused, HTTP 201, nothing said. Found because the identity diagnostic's
own fixture was refused that way and ran on a degraded campaign without
noticing. The refusal was already on the audit record; the reader was not
told. Now returned as `refused`, and shown in the State panel.
2. **The narration-length setting** above, which is finding C.
**Also corrected:** M9's residual risk 6 was narrower than recorded — the snap
Firefox refuses a WebDriver file path under `/tmp`, not all paths. Staging under
`$HOME` makes browser file import work, so knowledge import is now proved
end-to-end in a real browser rather than in two labelled halves.
---
## 4. Milestone Dependency Summary