Replace clustering with a catalogue, and hand-write the references to judge it against

The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1

tools/story-to-pack/SELECTION.md

@@ -1,10 +1,18 @@
 # story-to-pack — selecting situations
 
 > **Decision (2026-09-15): selection is gate G2 followed by a structured human review.** `probe/review.py`
-> writes a review package (`REVIEW.md`, `review.json`, `candidates.json`); the reviewer fills in
-> `review.json`; `probe/apply_review.py` checks it and writes `situations.json` for the next stage. See
-> the README. The record below is how that decision was reached: four gate designs and two
-> model-judged gates, each tested against blind labels.
+> writes `REVIEW.html`, a page where the reviewer keeps, drops or merges each group, unticks scenes and
+> names the situation; the page saves a review file, and `probe/apply_review.py` checks it and writes
+> `situations.json` for the next stage. See the README.
+>
+> The first version asked the reviewer to read a long Markdown file and edit JSON by hand. The first
+> real attempt stopped before a single entry: what to decide was unclear, there was too much to read at
+> once, hand-edited JSON broke easily, and the wording was hard to write. The page answers each of those.
+> The record below is how the G2 decision was reached: four gate designs and two model-judged gates,
+> each tested against blind labels.
+>
+> **Superseded in part (2026-09-15), see "Predicaments" at the end:** reading the page, the reviewer
+> found the groups were topics, not situations.
 
 The step after clustering: from the clusters of summarised scenes, choose the ones that are genuine
 **recurring situations** — the raw material of a recurring event — and reject the rest. Companion to
@@ -742,3 +750,142 @@ terms of roles and a concrete want** and rejects answers without both; scoring t
 the named situation (for example, requiring the named situation to be closer to the member summaries
 than to the corpus average in embedding space); and G2 AND G4 evaluated as a pre-registered gate on a
 fresh partition, since every labelled cluster now informs this observation.
+
+---
+
+## Predicaments (recorded 2026-09-15, before any model call)
+
+### What the first real review found
+
+The designer opened `REVIEW.html` for run k60-s24 and read four groups closely:
+
+- **Group 1 is artists.** It holds at least three different situations: an artist in a new place using
+  the art to get on, being paid for the work against what the work is worth, and plain money trouble.
+- **Group 2 is police and court officers.** Inside it: being coerced, with the police watching or
+  arriving; and a crime that is stopped or found out.
+- **Group 6 is flirting and infatuation.**
+- **Extra group 1 is room and board**: hotels, dinners and the like.
+
+None of these is "one person wanting something specific from another". Checked across all 20 groups,
+most are held together by who is in the scenes or where they happen: *artist* in 44% of group 1's
+scenes, *policeman* in 54% of group 3's, *uncle* and *lawyer* in group 4, *marriage* and *wedding* in
+group 8, *restaurant* and *customers* in group 12, *hotel* in 41% of extra group 1.
+
+**Why every earlier test missed it.** A topic group is as cohesive and as spread across stories as a
+situation group, so cohesion, story spread, word dominance and the model's own judgement all passed it.
+Summaries keep jobs and places, and the embeddings group on them. It took a person reading the groups.
+
+### Decisions (designer, 2026-09-15)
+
+1. **The unit is a predicament**, not a two-person transaction: what the main person is up against and
+   what they want out of it, as The Ladder's existing events are (the rent bouncing, the mail cart).
+   The sub-themes the designer found inside groups 1 and 2 are this kind of thing.
+2. **Re-describe every scene as a predicament, then regroup**, rather than splitting topic groups by hand.
+
+### Method, fixed before any call
+
+- `probe/redescribe.py` gives the model each scene's own prose (not its summary) and the prompt in that
+  script: one sentence, at most 20 words, no names, jobs, places, businesses or story-specific objects,
+  saying what the person is up against and what they want. No example of the output vocabulary.
+- Names are then removed deterministically (`strip_names.py`), the sentences embedded (`embed.py`), and
+  the embeddings clustered at k = 60.
+- **Pilot first:** the first 40 scenes, with both `qwen2.5:3b-instruct` and `qwen3:14b`, read by the
+  designer side by side before any full run. The full run uses the model the designer prefers.
+
+### How it is judged
+
+- **Topic share** (`probe/topic_share.py`): for each candidate group, the share of its scenes whose
+  original summary contains the group's most common job, relationship or place word; median over groups
+  and seeds 11–13 at k = 60. The baseline on the summary embeddings is measured before any call and
+  recorded below. **Pass: the median falls to half the baseline or less**, with at least 30 candidate
+  groups remaining at k = 60 (enough material for a two-stage pack).
+- **Leakage:** how many predicament sentences still contain a job, relationship or place word. Reported,
+  not gated.
+- **The real test is the designer reading the regrouped top groups** in the review page: most should
+  read as one predicament. Numbers alone passed topic groups before.
+
+### Baseline (measured 2026-09-15, before any model call)
+
+A first measurement counted words for a person with no role at all ("man", "woman", "girl"), and scored
+one group 90% topic-bound for saying "man". Those words (`GENERIC_PEOPLE` in `probe/topic_words.py`)
+were taken out of the measure, and the baseline re-measured, still before any predicament was generated.
+
+`python3 topic_share.py summary_embeddings.json` (`probe/topic-share-baseline.log`):
+
+| Seed | Candidate groups | Median top topic-word share |
+| --- | --- | --- |
+| 11 | 40 | 16% |
+| 12 | 41 | 20% |
+| 13 | 42 | 22% |
+| **Median** | **41** | **20%** |
+
+Most topic-bound at seed 11: *wife* 47% of 15, *dinner* 47% of 15, *policeman* 40% of 15 and 38% of 16,
+*brother* 38% of 8, *artist* 36% of 11. **Pass mark for the predicament groups: median 10% or less, with
+at least 30 candidate groups.**
+
+### Pilot (2026-09-15)
+
+First 40 scenes, one request at a time on the GPU host (3B 39 s, 14B 111 s; no GPU faults; logs in
+`probe/gpu-logs/*-2026-09-15-1803*`). Read side by side in `probe/pilot-compare.html` (`pilot_page.py`).
+
+| | qwen2.5:3b-instruct | qwen3:14b |
+| --- | --- | --- |
+| Has a job, relationship or place word | 15 of 40 | 13 of 40 (mostly *wife*, *husband*) |
+| Mean words (limit 20) | 14 | 25; 34 of 40 over |
+| Most common opening | "The Kid seeks…" ×3 | "A man struggles…" ×6 |
+
+3B mostly retells the plot, keeping names, objects and places. 14B writes something close to a
+predicament but ignores the length limit and repeats a few openings. **Designer's read: scenes 1–10,
+14B dramatically better in every one; stopped there. The full run uses qwen3:14b.**
+
+Found while reading: `split.py` skips a title written in quotation marks, so three stories were
+merged into the story before them. "Little Speck in Garnered Fruit" is inside "Dougherty's
+Eye-Opener", "The Guilty Party" inside "A Harlem Tragedy", and "What You Want" inside "The Duel". The
+corpus has 97 stories, not 94. Separately, `strip_names.py` treats *York*, *Manhattan* and *Broadway*
+as personal names, which gives "New someone" in 29 of 838 summaries.
+
+### Corrections before the full run (2026-09-15, designer's go-ahead)
+
+The v1 data and every G1–G4 record stay as they were, in `probe/`. The corrected corpus is built in
+`probe/v2/`, running the same scripts from that directory.
+
+- **Split.** `split.py` accepts a quoted title and matches a contents entry by the part before a dash
+  (“THE GUILTY PARTY”—AN EAST SIDE TRAGEDY). Result: **97 stories, 839 scenes**. Exactly the three
+  missing stories were added, nothing was lost, and only their three host stories changed.
+- **Summaries carried over.** 829 of the 839 scenes have text identical to a v1 scene and keep that
+  v1 summary verbatim (`carry_summaries.py`). The other 10 are summarised with the v1 model and prompt.
+- **Name stripper.** New York places (New York, Manhattan, Broadway, Harlem, the Bowery, and a few
+  others) become "the city" before names are looked for, and days and holidays are never names. Months
+  stay ambiguous ("May" is a person in scene 5) and are left alone.
+- **The baseline is re-measured on v2** (the v2 summaries, re-embedded), because both fixes change
+  what is embedded. It is recorded below before any predicament is read.
+- **No trimming to 20 words.** Tried on the 14B pilot: cutting at the last clause break within 20
+  words removed what the person is up against in most cut sentences ("…with a promise of future money"
+  without "but his dishonesty sparks a violent confrontation"), and 10 of 40 had no clean break.
+  The full 14B sentence is embedded.
+
+### Result (2026-09-15)
+
+Full run: 839 scenes re-described by `qwen3:14b`, one request at a time, 37 minutes on the GPU host
+(peak 270 W, mean 136 W, 65 °C, no faults; logs in `probe/gpu-logs/*-2026-09-15-1831*`). Sentences
+average 25 words. 219 of 839 still contain a job, relationship or place word, most often a relationship
+(*wife*, *husband*) — reported, not gated.
+
+`topic_share.py`, k = 60, seeds 11–13, both measured on the v2 corpus:
+
+| | Candidate groups | Median top topic-word share |
+| --- | --- | --- |
+| v2 summaries (baseline, `topic-share-baseline-v2.log`) | 45 | **20%** |
+| Predicaments (`topic-share-predicaments.log`) | 42 | **13%** |
+
+**The pre-registered pass mark is not met.** It asked for half the baseline or less (10%); the measure
+fell by about a third, from 20% to 13%. The group-count mark is met (42 ≥ 30). The most topic-bound
+groups are also much weaker than the baseline's: the worst predicament group is *room* at 40% of 5
+scenes, against the baseline's *bride* at 62% of 8 and *policeman* at 60% of 20.
+
+Not a pass and not a wash, so the decision goes to the reading test that this whole step exists for:
+`groups_page.py` builds `v2/groups-predicaments-k60-s24.html`, which lists each group's predicament
+sentences with each scene's original summary beneath, most cohesive group first, and asks the designer
+to mark each group **one predicament**, **two or three mixed**, or **not a predicament**. The openings
+repeat ("A man must" 45 times, "A man struggles" 37), so the risk to watch for is a group held together
+by that phrasing rather than by a predicament.