Replace clustering with a catalogue, and hand-write the references to judge it against

The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
This commit is contained in:
JesseMarkowitz
2026-09-20 17:19:42 -04:00
co-authored by Claude Opus 5
parent fa3769d0fe
commit e5617b86ba
135 changed files with 82027 additions and 1778 deletions
@@ -0,0 +1,32 @@
[
{"chunk":58,"story":2,"summary":"A society wife who has just published a daring novel dreads the arrival of press clippings now that her uncle, a bishop, is in town.","predicament":"Someone who has done something bold in public dreads the moment a disapproving elder relative hears of it."},
{"chunk":59,"story":2,"summary":"She complains that her husband is embarrassingly proud of the book and will read the reviews aloud to visitors; she fears a scandal.","predicament":"Someone is mortified that their spouse's naive enthusiasm will expose them in front of the people they fear."},
{"chunk":60,"story":2,"summary":"She insists her provocative title is honest, not commercial, and fears she has dragged her husband's name through the mire.","predicament":"Someone who meant to provoke now worries about what it will cost the family whose name they used."},
{"chunk":61,"story":2,"summary":"She says using a pen-name would be cowardice; her cousin, a hard-up hack writer, says an attack from the press would make her rich.","predicament":"Someone proud of their courage is told bluntly that the risk they fear is exactly what would bring success."},
{"chunk":62,"story":2,"summary":"She claims to speak for unhappy women; her divorced cousin warns that women with a past avoid stirring up mud. The bishop's arrival terrifies her.","predicament":"Someone claims to speak for others who have suffered, and is warned by one of them that it may not be welcome."},
{"chunk":63,"story":2,"summary":"The bishop arrives; he favours this niece over her cousin, whose cheap books outsell his own and who refused his advice during her divorce.","predicament":"Someone in authority rewards the relative who flatters them and resents the one who went her own way."},
{"chunk":64,"story":2,"summary":"Left alone with the bishop, the novelist feigns interest in his book to deflect talk of her own, and gets its plot wrong.","predicament":"Someone tries to steer an elder away from a dangerous topic by flattering them, and is caught not knowing their work."},
{"chunk":65,"story":2,"summary":"The bishop says his devout book was praised only by a friend who owed him favours, and that its best hope would be a public denunciation.","predicament":"Someone admits the only thing that would help their failing work is a public scandal they cannot decently arrange."},
{"chunk":66,"story":2,"summary":"The bishop explains that a charge of immorality would sell thousands, needed to fund a church window he has at heart.","predicament":"Someone needs money for a cause and wishes they could be attacked in public to get it."},
{"chunk":67,"story":2,"summary":"The niece blurts out her own fear of not being read; the bishop turns to her novel and asks her to change its title.","predicament":"Someone betrays their own worry and is immediately asked by an elder to tone down their work."},
{"chunk":68,"story":2,"summary":"The bishop assumes the title was forced on her or chosen innocently, and warns it will offend the readers she should value.","predicament":"Someone is lectured by an elder who assumes they could not have meant to give offence."},
{"chunk":69,"story":2,"summary":"She means to defend her choice but, under the bishop's certainty, begins apologising instead; her husband walks in.","predicament":"Someone who meant to stand by their choice finds themselves backing down before an elder's authority."},
{"chunk":70,"story":2,"summary":"The husband bursts in waving the first review, delighted that the bishop is there to hear it.","predicament":"Someone's eager spouse forces a private worry into the open before exactly the wrong witness."},
{"chunk":71,"story":2,"summary":"The narrator explains that the novelist suffers from a husband who admires everything she does without understanding it.","predicament":"Someone is frustrated by a spouse whose unconditional approval makes nothing they do matter."},
{"chunk":72,"story":2,"summary":"She wrote the book partly to provoke her husband's disapproval, but he adores it and presses it on everyone.","predicament":"Someone tries to provoke a reaction from a partner and gets only praise, which defeats the purpose."},
{"chunk":73,"story":2,"summary":"The husband reads the review aloud in front of the bishop; it begins by denouncing the filth of modern fiction.","predicament":"Someone must sit through a public verdict on their work in front of the person they most wanted to keep it from."},
{"chunk":74,"story":2,"summary":"The review calls her novel sweetly harmless; weeks later her cousin finds her crushed.","predicament":"Someone who wanted to shock is dismissed as harmless, which wounds more than condemnation would."},
{"chunk":75,"story":2,"summary":"She reveals that every review has called her daring book a pretty, inoffensive story.","predicament":"Someone who staked their pride on being taken seriously finds everyone treats them as tame."},
{"chunk":76,"story":2,"summary":"The cousin says the bishop blessed the book to bury it, and suggests the bishop could make it sell by denouncing it.","predicament":"Someone learns that a rival's approval was a way of smothering them, and is advised to provoke an attack instead."},
{"chunk":77,"story":2,"summary":"The cousin explains that nobody reads books any more, only puffs, and that only a denunciation from pulpit or press will sell one.","predicament":"Someone is told that merit counts for nothing and that the way to succeed is to engineer a scandal."},
{"chunk":78,"story":2,"summary":"Months later, a famous novelist travelling with her mocks the bishop who publicly attacked her book; she defends her uncle's sense of duty.","predicament":"Someone who has quietly benefited from a public attack defends the attacker's motives to an outsider."},
{"chunk":79,"story":2,"summary":"Her book has sold a hundred and fifty thousand; strangers recognise and praise her courage on the train.","predicament":"Someone enjoys a success whose real origin they cannot admit to the people praising them."},
{"chunk":80,"story":2,"summary":"The bishop's devoted friend tells her the bishop believes the anonymous gift of the window rewarded his courage in denouncing her book.","predicament":"Someone hears a rival take credit for an outcome they secretly paid for."},
{"chunk":81,"story":2,"summary":"The friend praises her forgiveness and hints that others think the bishop's attack was deserved.","predicament":"Someone must smile through backhanded praise from people who think they were rightly punished."},
{"chunk":82,"story":2,"summary":"In the cathedral she enjoys the attention of the congregation and the press, which she owes to the bishop's denunciation.","predicament":"Someone savours public attention that exists only because of an arrangement nobody else knows about."},
{"chunk":83,"story":2,"summary":"She plans a more outspoken second novel; the bishop begins his address celebrating the window's mysterious donor.","predicament":"Someone emboldened by success prepares to go further, while waiting to hear how their secret part is described."},
{"chunk":84,"story":2,"summary":"The bishop hints that the donor was a woman and that a book played its part, and eyes turn to her.","predicament":"Someone fears a public speaker is about to reveal the secret deal they struck."},
{"chunk":85,"story":2,"summary":"Under everyone's gaze she blushes and panics while the bishop expounds on the power of novels for good or ill.","predicament":"Someone sits exposed to public attention, waiting to learn whether they are about to be named."},
{"chunk":86,"story":2,"summary":"She nearly walks out, then learns the bishop is crediting his own book with inspiring the gift.","predicament":"Someone expecting to be exposed instead hears another person take all the credit for what they did."},
{"chunk":87,"story":2,"summary":"A fellow writer guesses she paid for the window herself; she half denies it, indignant that the bishop claims credit.","predicament":"Someone who secretly bought a favour cannot claim credit for it without admitting the bargain."}
]