Measure M04 by where the planted turn is, and catch a section of the model's own
The M04 re-run on0c7316fran 101 turns with no failures, and the verdict still came out `precondition_not_met`. That verdict was wrong. The planted player turn (depth 1) was 65 actions outside the history window, whose floor was 66. The sentinel's text was in recent history for two other reasons. - On 5 turns the narrator wrote a section of its own, `## Established:` over indented facts, with the planted clue copied into it from the state section. The extractor passed it: it was a single heading, and the `## ` meant it did not match. The harness leak count passed it the same way, and read 1 where 5 turns leaked. - On 4 turns the narrator used the sentinel as a name inside ordinary sentences ("the SILVER-KEY-CRYPT-OLD-ABBEY, flickers with latent power"). That is story text and cannot be stripped. So the sentinel's text in history can never be the precondition. M04's written pass is "Fact/event remains recoverable without entire transcript in prompt" (V1-ACCEPTANCE-TESTS.md). The owner agreed on 2026-09-13 that the precondition is positional, and that recovery through authoritative state counts; memory is not required. Extractor: - A state-section heading is recognised with any markdown the model wrapped it in (`## Established:`, `**Held:**`, `> Held:`). - One heading with an indented entry under it now qualifies as protocol. Before, a block needed two headings, or one heading and the scene line. A heading followed by unindented prose is still story. The earlier guard test "Held:" with an indented line is now protocol, and the case was rewritten unindented. Harness: - It records `planted_depth` when the clue is planted, and carries it across --resume. No endpoint reports an action's depth, but a fresh campaign's path is the opening, the planted turn and its reply, so the depth is `total_actions - 2`. That was checked against the database in three runs. - `_recall` reports `planted_depth`, `history_floor_depth` and `planted_turn_in_history_window`. The verdict is `precondition_not_met` only when the planted turn is still in the window, and `precondition_unknown` when its depth was never recorded. `clue_in_recent_history_window` stays as a fact about the prompt. - The protocol-leak count matches a state heading with an indented entry under it, in any markdown. Every AI turn in five real runs was replayed through the new extractor: draco M01, the two 26-turn GPU trials, thef8d4010M01 run and the M04 re-run. 443 turns in all. No turn the old extractor had left clean changed, and the M04 re-run lost 7 more leaks. The sentinel-as-a-name turns are story and remain. Trial 1's model-invented headings remain, as before. The M04 re-run's own evidence, reclassified under the new precondition from its database and its recall-turn snapshot, reads `recovered_through_state_only`. The original recall.json is kept unchanged beside `recall-reclassified.json`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136VBTMUKWYeU6G9HgbDbND
This commit is contained in:
co-authored by
Claude Opus 5
parent
0c7316f951
commit
96c1bf5ded
@@ -226,21 +226,36 @@ def _strip_dangling_object(text: str) -> str:
|
||||
return text
|
||||
|
||||
|
||||
# The markdown a model wraps a heading in: `## Established:`, `**Held:**`,
|
||||
# `> Held:`.
|
||||
_HEADING_DECORATION_RE = re.compile(r"^[\s#>*_]+|[\s*_]+$")
|
||||
|
||||
|
||||
def _section_heading(line: str) -> str | None:
|
||||
"""The state-section heading this line is, markdown aside, or None."""
|
||||
bare = _HEADING_DECORATION_RE.sub("", line)
|
||||
return bare if bare in render.SECTION_HEADINGS else None
|
||||
|
||||
|
||||
def _strip_echoed_state(text: str) -> str:
|
||||
"""Removes a copy of the narrative-state section pasted into the prose.
|
||||
|
||||
Judged by the section's own headings (`render.SECTION_HEADINGS`), as whole
|
||||
lines. A block qualifies only when it carries two of them, or one and the
|
||||
scene line directly above it. A story may contain a line reading "Held:",
|
||||
and one heading on its own is left there. The block runs over the headings,
|
||||
their indented entries and the blank lines between them, and stops at the
|
||||
first line of ordinary prose.
|
||||
Judged by the section's own headings (`render.SECTION_HEADINGS`) as whole
|
||||
lines, with any markdown the model wrapped them in taken off. A block
|
||||
qualifies when it carries two headings, or one and the scene line directly
|
||||
above it, or one heading with an indented entry under it. That last case
|
||||
is the model writing a section of its own: the M04 re-run found
|
||||
`## Established:` over two indented facts on 5 turns, one of them copying
|
||||
the planted clue out of the state section. A lone "Held:" with prose after
|
||||
it is still somebody's story. The block runs over the headings, their
|
||||
indented entries and the blank lines between them, and stops at the first
|
||||
line of ordinary prose.
|
||||
"""
|
||||
lines = text.split("\n")
|
||||
drop = [False] * len(lines)
|
||||
index = 0
|
||||
while index < len(lines):
|
||||
if lines[index].strip() not in render.SECTION_HEADINGS:
|
||||
if _section_heading(lines[index]) is None:
|
||||
index += 1
|
||||
continue
|
||||
start = index
|
||||
@@ -254,20 +269,23 @@ def _strip_echoed_state(text: str) -> str:
|
||||
if scene:
|
||||
start = above
|
||||
headings: set[str] = set()
|
||||
entries = 0
|
||||
end = index
|
||||
cursor = index
|
||||
while cursor < len(lines):
|
||||
line = lines[cursor]
|
||||
stripped = line.strip()
|
||||
if stripped in render.SECTION_HEADINGS:
|
||||
headings.add(stripped)
|
||||
heading = _section_heading(line)
|
||||
if heading is not None:
|
||||
headings.add(heading)
|
||||
end = cursor
|
||||
elif stripped and line[:1] in (" ", "\t"):
|
||||
entries += 1
|
||||
end = cursor
|
||||
elif stripped:
|
||||
break
|
||||
cursor += 1
|
||||
if len(headings) + (1 if scene else 0) >= 2:
|
||||
if len(headings) + (1 if scene else 0) >= 2 or (headings and entries):
|
||||
for position in range(start, end + 1):
|
||||
drop[position] = True
|
||||
index = end + 1
|
||||
|
||||
Reference in New Issue
Block a user