4 Commits

Include renames

Author	SHA1	Message	Date
JesseMarkowitzandClaude Opus 5	d63804f22e	v1.1: harden context window and narrator protocol boundary ... WP-A1 and WP-A2, implemented in sequence, plus the corrective work the owner asked for at review. Reported in planning/reports/v1.1/V1.1-WP-A1-A2-REPORT.md (corrective addendum §R). Planning package v4.2. WP-A1: context-window safety reserve - The prompt leaves max(256, ceil(5% of the effective window)) tokens free beside the reply. That is 256 at 4,096 and 820 at 16,384. The value is fixed, not a setting, and not calibrated per model. - M6's 64-token margin is gone. Separators and the chat hint are priced exactly; tokenizer drift is the reserve's job. - Protected context that cannot fit raises ContextOverflow before the model is called. - Streams set stream_options.include_usage. Measured on Ollama 0.33, a stream sent no usage without it. - Each sent turn records fits, exceeded, truncation_suspected or unknown. The status is returned on the done event, logged when bad, and shown in the context inspector. The turn is always kept. - Accounting is per-attempt data (attempts.ATTEMPT_KEYS). - Corrective: a cold model is loaded before its turn is built. When the window is unverified but the server answered, contextwindow.ensure_window makes one bounded POST /api/generate naming only the model. It sends no prompt, generates nothing and writes nothing. It then probes again, and the turn is built to that answer. If the load fails, or the window is still unknown, the turn falls back to the old behaviour. - Real host, 4,096 window: - v1 cold turn: sent 13,875, the server read 2,050. - Same turn after the correction: the window was verified, 3,082 sent, 3,097 read, fits, 499 tokens left beside the reply. - Verified turns elsewhere left 275-2,297 tokens against v1's 23-42. WP-A2: protocol echo and genre-neutral state prompting - The vocabulary is shown as the JSON object the model sends, not as name(field, ...). This costs 121 tokens. - The example uses character-1, item-1 and location-1. - The extractor removes shapes anchored to application-owned text: - a vocabulary call line; - an echoed length hint; - the renderer's scene line left last; - an empty fence opener. - Corrective R5: the echoed continue hint is recognised by its own sentence ("Output only story text"). A Hard-limit-opened bracket is removed only directly above an echo already cut from the same reply. - Replay of all 518 real v1 replies: 9 changed, 0 flagged, and no story prose removed. That is unchanged by R5. - Replay of 64 v1.1 replies: 3 changed, 0 flagged. The depth-16 instruction tail is removed. - Identity diagnostic after the correction: - 0 identity signals; - 0 prompt example identifiers proposed; - 0/10 stored turns with protocol or instruction shapes. - 50-turn run: 51 accepted, 0 of 54 stored turns carry protocol. - SPECS, render.py and validate.py are identical to v1.0.0. Compatibility: a real v1.0.0 database reads identically on v1.0.0 and v1.1, field for field, with schema and user_version 94 unchanged. Undo, redo, Save Point restore, export and import all work on it. There is no schema, migration or bundle-format change. Verification: the backend suite passes 1,534 with 17 skipped and 0 failed. The frontend passes 165/165, and lint and the build are clean. The offline container and the browser regression were re-run on this tree (see §R.3). One test was re-calibrated, not weakened: test_history_block_trim's prefix test had assumed which turn holds the floor at a 2,048 budget. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VvegagkhuCZoFPdv4M1egY	2026-09-14 16:35:05 -04:00
JesseMarkowitzandClaude Opus 5	96c1bf5ded	Measure M04 by where the planted turn is, and catch a section of the model's own ... The M04 re-run on 0c7316f ran 101 turns with no failures, and the verdict still came out `precondition_not_met`. That verdict was wrong. The planted player turn (depth 1) was 65 actions outside the history window, whose floor was 66. The sentinel's text was in recent history for two other reasons. - On 5 turns the narrator wrote a section of its own, `## Established:` over indented facts, with the planted clue copied into it from the state section. The extractor passed it: it was a single heading, and the `## ` meant it did not match. The harness leak count passed it the same way, and read 1 where 5 turns leaked. - On 4 turns the narrator used the sentinel as a name inside ordinary sentences ("the SILVER-KEY-CRYPT-OLD-ABBEY, flickers with latent power"). That is story text and cannot be stripped. So the sentinel's text in history can never be the precondition. M04's written pass is "Fact/event remains recoverable without entire transcript in prompt" (V1-ACCEPTANCE-TESTS.md). The owner agreed on 2026-09-13 that the precondition is positional, and that recovery through authoritative state counts; memory is not required. Extractor: - A state-section heading is recognised with any markdown the model wrapped it in (`## Established:`, `**Held:**`, `> Held:`). - One heading with an indented entry under it now qualifies as protocol. Before, a block needed two headings, or one heading and the scene line. A heading followed by unindented prose is still story. The earlier guard test "Held:" with an indented line is now protocol, and the case was rewritten unindented. Harness: - It records `planted_depth` when the clue is planted, and carries it across --resume. No endpoint reports an action's depth, but a fresh campaign's path is the opening, the planted turn and its reply, so the depth is `total_actions - 2`. That was checked against the database in three runs. - `_recall` reports `planted_depth`, `history_floor_depth` and `planted_turn_in_history_window`. The verdict is `precondition_not_met` only when the planted turn is still in the window, and `precondition_unknown` when its depth was never recorded. `clue_in_recent_history_window` stays as a fact about the prompt. - The protocol-leak count matches a state heading with an indented entry under it, in any markdown. Every AI turn in five real runs was replayed through the new extractor: draco M01, the two 26-turn GPU trials, the f8d4010 M01 run and the M04 re-run. 443 turns in all. No turn the old extractor had left clean changed, and the M04 re-run lost 7 more leaks. The sentinel-as-a-name turns are story and remain. Trial 1's model-invented headings remain, as before. The M04 re-run's own evidence, reclassified under the new precondition from its database and its recall-turn snapshot, reads `recovered_through_state_only`. The original recall.json is kept unchanged beside `recall-reclassified.json`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136VBTMUKWYeU6G9HgbDbND	2026-09-13 22:10:33 -04:00
JesseMarkowitzandClaude Opus 5	0c7316f951	Keep the state section and its proposal out of the story ... The first complete M01 run with the memory bank on (f8d4010, 101 turns on a GPU host) reported "complete". It still did not prove M04. The planted clue was found at turn 100 only because the narrator had pasted the narrative-state section into its own prose, and the paste was still in recent history. No memory and no summary carried the clue. The narrator is a small local model. It wrote protocol into its stored narration on 42 of 104 turns, starting at depth 2, in four shapes: - a copy of the state section: `Scene:`, `Who and what exists:`, `Held:`, `Established:`, `Still open:` - that copy above a correct ```state block, which was stripped while the copy stayed - the copy, then a bare `State` heading, then a `> {"events": ...}` proposal quoted like a player turn, sometimes with story after it - the same block cut off by the output-token limit, on 10 turns Stored text is replayed verbatim as history, so each leak also put a second, older account of the state into the next prompt. That is what M5 review Finding 4 removed from history replay, and every leak gave the model another example to copy. The extractor now removes: - a pasted state section, recognised by at least two of the renderer's own headings as whole lines. The headings are constants in `render.py`, so the renderer and the extractor cannot drift apart. One heading alone, or a `Scene:` line of prose, is left. - an unfenced proposal that starts a line, quoted or not, when it parses and is a proposal. With no fence it becomes the turn's proposal. A `State` heading directly above goes with it. Candidates are taken outermost first, so a finished event line inside an unfinished block is never taken as a proposal by itself. - an unfinished unfenced proposal at the end that reads as protocol. - whatever is left at the end: a `State` heading, a bare `>`, a parroted reminder or continue hint (closed or not), and a ```json fence cut off before it names its events. These are cut repeatedly until nothing more comes off. This also fixes an older bug. `_STATE_FENCE_RE` read "a ```state block" inside a parroted reminder as a fence opening and cut out the middle of the reminder. The label must now end its line or run straight into the payload. A reply whose only removal is a pasted state section records no raw block, so the turn is not marked unparseable for a block it never started. Every AI turn in four real runs was replayed through the new extractor: draco M01, the two 26-turn GPU trials, and this M01 run. 339 turns in all. No turn the old extractor had left clean changed. Every leak of our own protocol is gone: 42 of 42 in this M01 run, 5 in trial 2, 3 on draco. Trial 1 still has model-invented headings ("Identifiers established:", "Set of events made true:") on 10 turns. They paraphrase the instruction and are not our renderer's text, so they are left, not guessed at. The long-run harness now records an explicit M04 verdict, which is never a recovery while the clue is still in recent history. It also counts the AI turns in the export that still carry protocol. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136VBTMUKWYeU6G9HgbDbND	2026-09-13 21:18:18 -04:00
JesseMarkowitzandClaude Opus 5	b7005e6fdd	M5: genre-neutral authoritative narrative state, with review corrections ... Replaces AI-DnD's RPG relative-delta world state with the genre-neutral typed narrative state of ADR 010: explicit, absolute, allowlisted events proposed by the model, validated by the application, applied to one authoritative document, and snapshotted per position so restore stays a row read. This commit includes the corrective pass that followed the independent review in planning/reports/M5-IMPLEMENTATION-REPORT.md. The invariant it exists to hold is: visible active transcript position == stored head == authoritative state Narrator editing (D10, STORY-BRANCH-SEMANTICS §§14-15) A narrator edit no longer rewrites a row. It returns to the state before the turn, takes the reader's exact text as the accepted narration, re-derives the state that text implies, and becomes a new active continuation — while the original narration keeps its words, its live flag and its whole future as retained history. At the tip the correction is another take; with story below it, it forks. No new history machinery: this is the existing fork/take/head path with the reader's text in place of a generated reply. The §14A refusal is therefore gone for narrator turns, and remains only for player input. Pre-M5 positions Migration 88 backfills the empty narrative document onto every action written before M5, and a missing snapshot now restores the empty document instead of leaving the previous position's state standing. Restoring to an old Save Point no longer leaves a later position's entities and facts on screen. Narrator context Replayed history carries prose only; the machine-readable block is no longer reconstructed into past turns, where it contradicted the authoritative state in the same prompt. A fact withdrawn by a manual correction is now named as no longer true, with the reader's reason, rather than silently dropped. Also - state_changes joins the action-list bulk read, removing one query per row. - Extraction takes only the application's own protocol payload: an ordinary ```json or ```python block in a story survives, and a mangled proposal still does not reach the reader. Planning: ADR 013 records the authoritative document shape; §§14-15/14A, D10, C04 and BUILD-MILESTONES are updated to describe what exists. Debt is recorded against M8 (scenario editor UX) and M9 (export of the audit trail). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2	2026-09-05 07:01:50 -04:00