Replace clustering with a catalogue, and hand-write the references to judge it against

The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
This commit is contained in:
JesseMarkowitz
2026-09-20 17:19:42 -04:00
co-authored by Claude Opus 5
parent fa3769d0fe
commit e5617b86ba
135 changed files with 82027 additions and 1778 deletions
@@ -0,0 +1,15 @@
[
{"chunk":135,"story":10,"summary":"At a crowded family breakfast, a poor widow living with her sister's family gets a letter and reads it so slowly that the others nearly burst with impatience.","predicament":"Someone given news that might change their life cannot take it in, while the people around them itch to know."},
{"chunk":136,"story":10,"summary":"The widow fears the letter is a hoax; her brother-in-law thinks so too, until her sister reads it and rejoices.","predicament":"Someone who has never had luck is told of a windfall and is afraid to believe it."},
{"chunk":137,"story":10,"summary":"The widow has inherited two thousand pounds from a former employer; the brother-in-law grumbles that nobody ever left him anything, and spreads the news through the town.","predicament":"A man sees a relative come into money and resents that luck never comes his way."},
{"chunk":138,"story":10,"summary":"Local men suddenly befriend the brother-in-law; his wife points out that two old dealers are courting the heiress for her money.","predicament":"A man flattered by new friends is told they are really after a relative's money."},
{"chunk":139,"story":10,"summary":"The brother-in-law watches the two suitors flatter the widow and tries to undercut them by announcing her age, nearly giving away his wife's.","predicament":"A man trying to expose flattering suitors trips over his own family's sensibilities."},
{"chunk":140,"story":10,"summary":"The suitors outbid each other with compliments; the wife's temper frays, and the visitors are shown out with the door shut hard.","predicament":"Two rivals courting the same person keep topping each other's compliments while the family grows fed up."},
{"chunk":141,"story":10,"summary":"The family, wanting to keep the money, fail to warn the widow; the slow dealer, losing ground to his glib rival, hatches a plan and enlists a small-time dealer.","predicament":"A slow-talking man losing a contest to a glib rival looks for a trick instead of better words."},
{"chunk":142,"story":10,"summary":"The dealer bribes an accomplice, who holds out for a bigger fee, to tell the rival in confidence that the widow has lost her fortune; the rival swallows it.","predicament":"A man hires someone to spread a false rumour that will drive off his rival, and the hireling drives a hard bargain."},
{"chunk":143,"story":10,"summary":"With the rival gone, the dealer courts the widow alone, preaching the virtues of poverty, and loses seven games of draughts to her on purpose.","predicament":"A man courting for money pretends not to care about it and flatters the one he courts by letting her win."},
{"chunk":144,"story":10,"summary":"He blows her kisses and fusses over a bumped head, invites himself back, and walks home sure the money is his.","predicament":"A schemer who thinks his deal is done walks away gloating too soon."},
{"chunk":145,"story":10,"summary":"He spreads the rumour of her ruin through the town, then proposes publicly 'now that she is poor', to look noble.","predicament":"A man makes a public show of selfless devotion, sure that it will cost him nothing."},
{"chunk":146,"story":10,"summary":"She accepts; the family, touched, apologise for misjudging him, then mention a letter that arrived.","predicament":"A schemer whose plan has worked is thanked for his goodness while the ground starts to shift under him."},
{"chunk":147,"story":10,"summary":"The lawyers' letter says a later will was found and she has nothing; the dealer, now engaged to a poor woman, is left alone with her to court.","predicament":"A man who faked a disaster to win a prize learns the disaster is real, and he is bound to his promise."}
]