2 Commits

Include renames

Author	SHA1	Message	Date
JesseMarkowitzandClaude Opus 5	d63804f22e	v1.1: harden context window and narrator protocol boundary ... WP-A1 and WP-A2, implemented in sequence, plus the corrective work the owner asked for at review. Reported in planning/reports/v1.1/V1.1-WP-A1-A2-REPORT.md (corrective addendum §R). Planning package v4.2. WP-A1: context-window safety reserve - The prompt leaves max(256, ceil(5% of the effective window)) tokens free beside the reply. That is 256 at 4,096 and 820 at 16,384. The value is fixed, not a setting, and not calibrated per model. - M6's 64-token margin is gone. Separators and the chat hint are priced exactly; tokenizer drift is the reserve's job. - Protected context that cannot fit raises ContextOverflow before the model is called. - Streams set stream_options.include_usage. Measured on Ollama 0.33, a stream sent no usage without it. - Each sent turn records fits, exceeded, truncation_suspected or unknown. The status is returned on the done event, logged when bad, and shown in the context inspector. The turn is always kept. - Accounting is per-attempt data (attempts.ATTEMPT_KEYS). - Corrective: a cold model is loaded before its turn is built. When the window is unverified but the server answered, contextwindow.ensure_window makes one bounded POST /api/generate naming only the model. It sends no prompt, generates nothing and writes nothing. It then probes again, and the turn is built to that answer. If the load fails, or the window is still unknown, the turn falls back to the old behaviour. - Real host, 4,096 window: - v1 cold turn: sent 13,875, the server read 2,050. - Same turn after the correction: the window was verified, 3,082 sent, 3,097 read, fits, 499 tokens left beside the reply. - Verified turns elsewhere left 275-2,297 tokens against v1's 23-42. WP-A2: protocol echo and genre-neutral state prompting - The vocabulary is shown as the JSON object the model sends, not as name(field, ...). This costs 121 tokens. - The example uses character-1, item-1 and location-1. - The extractor removes shapes anchored to application-owned text: - a vocabulary call line; - an echoed length hint; - the renderer's scene line left last; - an empty fence opener. - Corrective R5: the echoed continue hint is recognised by its own sentence ("Output only story text"). A Hard-limit-opened bracket is removed only directly above an echo already cut from the same reply. - Replay of all 518 real v1 replies: 9 changed, 0 flagged, and no story prose removed. That is unchanged by R5. - Replay of 64 v1.1 replies: 3 changed, 0 flagged. The depth-16 instruction tail is removed. - Identity diagnostic after the correction: - 0 identity signals; - 0 prompt example identifiers proposed; - 0/10 stored turns with protocol or instruction shapes. - 50-turn run: 51 accepted, 0 of 54 stored turns carry protocol. - SPECS, render.py and validate.py are identical to v1.0.0. Compatibility: a real v1.0.0 database reads identically on v1.0.0 and v1.1, field for field, with schema and user_version 94 unchanged. Undo, redo, Save Point restore, export and import all work on it. There is no schema, migration or bundle-format change. Verification: the backend suite passes 1,534 with 17 skipped and 0 failed. The frontend passes 165/165, and lint and the build are clean. The offline container and the browser regression were re-run on this tree (see §R.3). One test was re-calibrated, not weakened: test_history_block_trim's prefix test had assumed which turn holds the floor at a 2,048 budget. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VvegagkhuCZoFPdv4M1egY	2026-09-14 16:35:05 -04:00
JesseMarkowitzandClaude Opus 5	ef25b0a876	Stop re-reading the whole prompt every turn, and let a lost run carry on ... M01, the hundred-turn campaign, is the one REQUIRED test still outstanding. Everything here is about it finishing, and being worth believing when it does. No requirement changed, no acceptance test was retired or relaxed, and M11 §P.1's "no performance requirement" still stands: what changed is the cost of a turn, not what a turn contains. An inference server caches a prompt by its prefix. The history window gave up its oldest action every turn, which changed the prompt near the front and threw that cache away, so nearly the whole prompt was reprocessed every turn however little had actually changed. The window now snaps the oldest depth to a block and holds it, stepping every few turns. Measured on real builder output at an 8,192-token budget: 124.0s per turn against 362.4s. The cost is history depth, bounded by TRIM_FRACTION at a quarter of the window, which is the dial between recent history and speed. A run that dies no longer starts again from turn one. m11_long_run checkpoints resume.json after the prologue, after every scheduled step and after every turn, and --resume reattaches to the same campaign. A finished run deletes it, so the file's presence means an unfinished run and starting fresh over one is refused. The model timeout is an option rather than a hard-coded 600s, a turn that overruns is a failed turn instead of an unhandled exception that ends the run with no summary, and a run that has stopped producing turns writes its evidence and stops. Two checks could not fail. M04's planted clue went into an add_fact "detail" key that the event does not define, so it was dropped and fact_still_in_state could never be true; it is now in "value" and proved at turn one, which stops a run measuring nothing for hours. m11_browser degraded silently without a narrator into two failures that read exactly like a product regression, and now requires one, with --no-narrator as an explicit opt-out that marks the run partial. Window discovery speaks Ollama's native API, so against vLLM or llama.cpp's own server the window goes unverified and the budget uncapped -- M11's own failure mode reached by another route. context_window_override lets the operator state what they launched the server with, and is used only where discovery left a hole: a verified window always wins, so a declaration can lower an unknown ceiling into existence and never raise a known one. "verified" still means the server answered, so window_verified in a turn's provenance keeps the meaning M11's report counts on. planning/README.md said the M11 tree was staged rather than committed, in two places; it was committed and signed. Planning package v3.8. Backend 1,376 passed, 17 skipped, 0 failed; frontend 161; lint and build clean. Every M11 harness re-run on this tree: browser 38/0/0, offline 23/0, identity clean, contrast unchanged, recovery 14/0 on a small bundle. M01 itself has not been run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E9LiyxBxnMTXV2wRjdyDGB	2026-09-10 06:13:55 -04:00