Stop re-reading the whole prompt every turn, and let a lost run carry on
M01, the hundred-turn campaign, is the one REQUIRED test still outstanding. Everything here is about it finishing, and being worth believing when it does. No requirement changed, no acceptance test was retired or relaxed, and M11 §P.1's "no performance requirement" still stands: what changed is the cost of a turn, not what a turn contains. An inference server caches a prompt by its prefix. The history window gave up its oldest action every turn, which changed the prompt near the front and threw that cache away, so nearly the whole prompt was reprocessed every turn however little had actually changed. The window now snaps the oldest depth to a block and holds it, stepping every few turns. Measured on real builder output at an 8,192-token budget: 124.0s per turn against 362.4s. The cost is history depth, bounded by TRIM_FRACTION at a quarter of the window, which is the dial between recent history and speed. A run that dies no longer starts again from turn one. m11_long_run checkpoints resume.json after the prologue, after every scheduled step and after every turn, and --resume reattaches to the same campaign. A finished run deletes it, so the file's presence means an unfinished run and starting fresh over one is refused. The model timeout is an option rather than a hard-coded 600s, a turn that overruns is a failed turn instead of an unhandled exception that ends the run with no summary, and a run that has stopped producing turns writes its evidence and stops. Two checks could not fail. M04's planted clue went into an add_fact "detail" key that the event does not define, so it was dropped and fact_still_in_state could never be true; it is now in "value" and proved at turn one, which stops a run measuring nothing for hours. m11_browser degraded silently without a narrator into two failures that read exactly like a product regression, and now requires one, with --no-narrator as an explicit opt-out that marks the run partial. Window discovery speaks Ollama's native API, so against vLLM or llama.cpp's own server the window goes unverified and the budget uncapped -- M11's own failure mode reached by another route. context_window_override lets the operator state what they launched the server with, and is used only where discovery left a hole: a verified window always wins, so a declaration can lower an unknown ceiling into existence and never raise a known one. "verified" still means the server answered, so window_verified in a turn's provenance keeps the meaning M11's report counts on. planning/README.md said the M11 tree was staged rather than committed, in two places; it was committed and signed. Planning package v3.8. Backend 1,376 passed, 17 skipped, 0 failed; frontend 161; lint and build clean. Every M11 harness re-run on this tree: browser 38/0/0, offline 23/0, identity clean, contrast unchanged, recovery 14/0 on a small bundle. M01 itself has not been run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E9LiyxBxnMTXV2wRjdyDGB
This commit is contained in:
co-authored by
Claude Opus 5
parent
fedb7144d0
commit
ef25b0a876
@@ -56,6 +56,31 @@ It does not hard-code 4,096, which would cripple a correctly configured
|
||||
deployment; it does not raise the budget, which is the operator's decision; it
|
||||
does not fall back to a cloud probe, a bundled table of model sizes, or a guess
|
||||
from the model's name. An unknown window is reported as unknown.
|
||||
|
||||
## The server that cannot be asked
|
||||
|
||||
Discovery above is Ollama's native API. Nothing restricts `endpoint_url` to
|
||||
Ollama — any allowed address serving an OpenAI-compatible `/v1` is accepted —
|
||||
and on vLLM, llama.cpp's own server, or anything else, `/api/ps` and `/api/show`
|
||||
are simply not there. Discovery then fails exactly as designed and the window is
|
||||
reported unknown, which is honest but leaves the invariant at the top of this
|
||||
file unenforced: the budget stands at whatever is configured, and if that server
|
||||
enforces a smaller window it drops the oldest tokens again.
|
||||
|
||||
`context_window_override` is the operator's answer to that. It is a number the
|
||||
operator states because they know how the server was launched, and it is used
|
||||
**only when the server could not be asked**:
|
||||
|
||||
verified window -> always wins; a declaration cannot raise it
|
||||
no verified window -> the declaration becomes the ceiling, source DECLARED
|
||||
neither -> unknown, exactly as before
|
||||
|
||||
This does not weaken what `verified` claims. `verified` still means the server
|
||||
itself answered, so `window_verified` in a turn's provenance keeps the meaning
|
||||
the M11 report gives it, and a declared window is identifiable as a declaration
|
||||
wherever it appears. What the declaration buys is enforcement: the prompt is
|
||||
capped, so the failure mode is a shorter prompt rather than a silently truncated
|
||||
one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -86,8 +111,12 @@ NEGATIVE_TTL = 60.0
|
||||
#: Sources, in the order of how much they prove.
|
||||
LOADED = "loaded" # /api/ps: what the runtime is enforcing now
|
||||
PARAMETERS = "parameters" # /api/show: what the model will load with
|
||||
DECLARED = "declared" # the operator said so; the server could not be asked
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
#: Sources that mean *the server answered*, as opposed to somebody asserting.
|
||||
FROM_SERVER = (LOADED, PARAMETERS)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Window:
|
||||
@@ -106,6 +135,18 @@ class Window:
|
||||
|
||||
@property
|
||||
def verified(self) -> bool:
|
||||
"""The **server** answered. An operator's declaration is not this.
|
||||
|
||||
Kept narrow on purpose. `window_verified` travels in every turn's stored
|
||||
provenance and the M11 report counts on it meaning one thing: that the
|
||||
runtime was asked and replied. A declaration is a person's claim about a
|
||||
server, which is worth acting on and is not the same evidence.
|
||||
"""
|
||||
return self.tokens is not None and self.source in FROM_SERVER
|
||||
|
||||
@property
|
||||
def enforceable(self) -> bool:
|
||||
"""There is a number to cap the prompt to, whoever supplied it."""
|
||||
return self.tokens is not None
|
||||
|
||||
|
||||
@@ -128,9 +169,10 @@ def native_base(endpoint_url: str) -> str:
|
||||
def effective_budget(configured: int, window: Window | int | None) -> int:
|
||||
"""The budget the prompt may actually use.
|
||||
|
||||
The whole enforcement, in one line: a verified window is a ceiling. The
|
||||
configured budget still wins when it is *smaller*, because a reader who has
|
||||
deliberately asked for a shorter prompt should get one.
|
||||
The whole enforcement, in one line: a known window is a ceiling — whether
|
||||
the server reported it or the operator declared it. The configured budget
|
||||
still wins when it is *smaller*, because a reader who has deliberately asked
|
||||
for a shorter prompt should get one.
|
||||
"""
|
||||
tokens = window.tokens if isinstance(window, Window) else window
|
||||
if tokens is None or tokens <= 0:
|
||||
@@ -143,17 +185,55 @@ def cache_clear() -> None:
|
||||
_cache.clear()
|
||||
|
||||
|
||||
async def probe(endpoint_url: str, model: str, *, use_cache: bool = True) -> Window:
|
||||
"""Asks the server what window `model` gets. Never raises.
|
||||
async def probe(endpoint_url: str, model: str, *,
|
||||
declared: int | None = None, use_cache: bool = True) -> Window:
|
||||
"""What window `model` gets, asked of the server and only then declared.
|
||||
|
||||
Returns `UNVERIFIED` for every failure — refused endpoint, unreachable
|
||||
server, TLS failure, a non-Ollama endpoint, an unparseable answer. The caller
|
||||
cannot act differently on those and the reader is told the same thing either
|
||||
way: the window could not be checked.
|
||||
Returns `UNVERIFIED` for every discovery failure — refused endpoint,
|
||||
unreachable server, TLS failure, a server with no Ollama-native API, an
|
||||
unparseable answer — unless `declared` supplies a number to fall back on.
|
||||
The caller cannot act differently on those failures and the reader is told
|
||||
the same thing either way: the window could not be checked.
|
||||
|
||||
`declared` is `Settings.context_window_override`. It never overrides a
|
||||
verified answer, so an operator cannot talk the application into a bigger
|
||||
prompt than the runtime will read; it only fills a gap discovery left.
|
||||
"""
|
||||
if not endpoint_url or not model:
|
||||
return Window(None, UNKNOWN, detail="no endpoint or model configured")
|
||||
return _declared_or(declared,
|
||||
Window(None, UNKNOWN,
|
||||
detail="no endpoint or model configured"))
|
||||
|
||||
discovered = await _discover(endpoint_url, model, use_cache=use_cache)
|
||||
return _declared_or(declared, discovered)
|
||||
|
||||
|
||||
def _declared_or(declared: int | None, discovered: Window) -> Window:
|
||||
"""The operator's number, but only where the server left a hole.
|
||||
|
||||
A verified window always wins. That ordering is the whole safety property:
|
||||
a declaration can lower an unknown ceiling into existence, never raise a
|
||||
known one.
|
||||
"""
|
||||
if discovered.verified:
|
||||
return discovered
|
||||
if not declared or declared <= 0:
|
||||
return discovered
|
||||
return Window(
|
||||
declared, DECLARED, discovered.model_max,
|
||||
f"{declared:,} tokens, declared in settings — the server was not able "
|
||||
f"to say ({discovered.detail})",
|
||||
)
|
||||
|
||||
|
||||
async def _discover(endpoint_url: str, model: str, *,
|
||||
use_cache: bool = True) -> Window:
|
||||
"""The server's own answer, cached. Knows nothing about declarations.
|
||||
|
||||
The cache holds only what was discovered, so changing the declared override
|
||||
takes effect on the next turn without having to clear anything: the
|
||||
declaration is layered on afterwards, in `_declared_or`.
|
||||
"""
|
||||
key = (endpoint_url, model)
|
||||
now = time.monotonic()
|
||||
if use_cache:
|
||||
|
||||
Reference in New Issue
Block a user