Replace clustering with a catalogue, and hand-write the references to judge it against

The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1

tools/story-to-pack/probe/split.py

@@ -7,7 +7,15 @@ involved -- this is the segmentation step we get for free.
 import re, json, sys, pathlib
 
 ROMAN = re.compile(r'^\s*[IVXLC]+\.\s+')
-CAPS = re.compile(r'^[ \t]*([A-Z0-9][A-Z0-9 ,.;:\'\"\-’‘“”!?&()À-ÖØ-Þ]{3,70})[ \t]*$')
+CAPS = re.compile(r'^[ \t]*([A-Z0-9"“‘][A-Z0-9 ,.;:\'\"\-—’‘“”!?&()À-ÖØ-Þ]{3,70})[ \t]*$')
+# A title may be quoted ("LITTLE SPECK IN GARNERED FRUIT"), and a contents entry may carry a subtitle
+# after a dash that the heading does not (“THE GUILTY PARTY”—AN EAST SIDE TRAGEDY). v1 handled
+# neither and silently merged three stories into the story before them. The title is what comes
+# before the dash, without quotes.
+QUOTES = '"“”‘’\''
+
+def title_of(line):
+    return CAPS.match(line).group(1).split('—')[0].strip().strip(QUOTES).strip()
 
 def strip_boilerplate(text):
     i = text.find('*** START')
@@ -25,7 +33,8 @@ def split_volume(path):
     # Two volumes number their contents ("  I. THE VOICE OF THE CITY") while the
     # body heading is the bare title, so strip the numeral before matching.
     lines = [ROMAN.sub('', l) for l in lines]
-    caps_idx = [(n, CAPS.match(l).group(1).strip()) for n, l in enumerate(lines) if CAPS.match(l)]
+    caps_idx = [(n, title_of(l)) for n, l in enumerate(lines) if CAPS.match(l)]
+    caps_idx = [(n, t) for n, t in caps_idx if len(t) >= 4]
     if not caps_idx:
         return volume, []