Replace clustering with a catalogue, and hand-write the references to judge it against

The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
This commit is contained in:
JesseMarkowitz
2026-09-20 17:19:42 -04:00
co-authored by Claude Opus 5
parent fa3769d0fe
commit e5617b86ba
135 changed files with 82027 additions and 1778 deletions
@@ -0,0 +1,27 @@
[
{"chunk":120,"story":4,"summary":"A clever, condescending husband notices his simple wife glowing and teases her into admitting she has done something he will not like.","predicament":"Someone senses their spouse is hiding something and teases it out of them."},
{"chunk":121,"story":4,"summary":"The wife, who holds only inherited opinions, says she expects his disapproval and has allowed for it, then announces 'a baby'.","predicament":"Someone who has always given way to a spouse announces a wish they know will be opposed."},
{"chunk":122,"story":4,"summary":"Baffled, the husband learns she wants the child to be theirs and has long grieved at being childless.","predicament":"Someone discovers that their spouse has carried a deep disappointment they never noticed."},
{"chunk":123,"story":4,"summary":"The wife confesses her loneliness and that she chose nursery wallpaper years ago; the husband feels he has failed her.","predicament":"Someone realises their neglect has left a partner lonely for years."},
{"chunk":124,"story":4,"summary":"She asks to adopt an orphan from a hospital, promising it will never bother him.","predicament":"Someone asks a reluctant partner for the one thing they want, promising it will cost the partner nothing."},
{"chunk":125,"story":4,"summary":"The husband agrees without resistance, accepting future ridicule as penance for having underestimated his wife.","predicament":"Someone agrees to a partner's wish they would normally mock, as a way of making amends."},
{"chunk":126,"story":4,"summary":"The husband reflects that he has been faithful but emotionally absent, keeping his wife out of his inner life.","predicament":"Someone who has kept the letter of a commitment realises they have neglected its spirit."},
{"chunk":127,"story":4,"summary":"At the hospital the wife is transfigured by the child; the husband feels invisible to her.","predicament":"Someone watches their partner's devotion shift wholly to someone new and feels shut out."},
{"chunk":128,"story":4,"summary":"The husband allows a legal adoption but refuses to let the child be renamed, and begins to regret the whole business.","predicament":"Someone who has given way on the big question digs in on a small one and starts to regret giving way."},
{"chunk":129,"story":4,"summary":"The child is quiet and healthy; the wife's character is not changed but magnified by motherhood.","predicament":"Someone finds a change they dreaded is milder than feared, but it makes a partner more of what they already were."},
{"chunk":130,"story":4,"summary":"The wife grows assertive and treats the husband as the child's father; he accepts it because she is happy.","predicament":"Someone is drawn into a role they never chose, and puts up with it because it makes a partner happy."},
{"chunk":131,"story":4,"summary":"The child does not bring the couple together; the wife's personality expands until the husband is squeezed into a corner of his own home.","predicament":"Someone gives ground at home again and again until they have almost no room left."},
{"chunk":132,"story":4,"summary":"The girl grows healthy and good, and the wife, feeling unequal to her, asks the husband to take charge of her education.","predicament":"Someone who feels out of their depth hands a responsibility to a partner they consider more capable."},
{"chunk":133,"story":4,"summary":"The wife humbly admits she is too stupid to know what is best for the child and asks the husband to decide.","predicament":"Someone admits they are not up to a task and asks another to take it on."},
{"chunk":134,"story":4,"summary":"The girl hoards facts without understanding and grows proud and superior; the wife hopes she will be a companion for the husband.","predicament":"Someone takes on a protégé who turns out to be clever but cold and conceited."},
{"chunk":135,"story":4,"summary":"The girl sets about reforming the whole household, driving off cooks and lecturing the husband until he dines out.","predicament":"Someone at home is bossed and corrected by a young person who has made themselves everyone's guardian."},
{"chunk":136,"story":4,"summary":"The husband's attempts to check her only hurt his wife; the girl, pretty but charmless, fails in society despite great expense.","predicament":"Someone cannot rein in a difficult young person without wounding the partner who adores them."},
{"chunk":137,"story":4,"summary":"The girl takes her failure out on her mother, then turns to charity and makes the husband her evening project.","predicament":"Someone watches a young person take out their disappointment on a devoted parent, then turn it on them."},
{"chunk":138,"story":4,"summary":"The wife asks him to bear the girl's attentions; then a young man starts calling, and the parents hardly dare hope he comes for her.","predicament":"Someone longing to be free of a burden sees a chance of it and dares not say so."},
{"chunk":139,"story":4,"summary":"An effusively polite young man courts the girl; the parents pretend not to notice while she shows no emotion at all.","predicament":"Someone hoping for a match that would relieve them must pretend indifference while it hangs in the balance."},
{"chunk":140,"story":4,"summary":"He proposes, and the girl demands a month to think it over, to her parents' agony.","predicament":"Someone longs for a decision that is entirely in another's hands, and that person insists on delaying it."},
{"chunk":141,"story":4,"summary":"The girl accepts and at once begins to dominate her fiancé; the parents, pitying him, wonder whether to warn him.","predicament":"Someone must decide whether to warn an outsider about a relative's faults, at the risk of losing a longed-for outcome."},
{"chunk":142,"story":4,"summary":"The wife tells the fiancé that the girl is always difficult; he says they never understood her and grows cold, which relieves the husband.","predicament":"Someone who speaks an honest warning is rebuffed as disloyal, and finds the result a relief."},
{"chunk":143,"story":4,"summary":"The girl turns hysterical before the wedding and threatens to break it off; the husband frets over every possible mishap until the groom arrives late.","predicament":"Someone waits through a nerve-racking final stretch, fearing any accident could undo the outcome they need."},
{"chunk":144,"story":4,"summary":"The couple leave; alone together at last, the husband and wife draw close and go out to dinner.","predicament":"Someone who has endured a long strain alongside a partner finds it has finally brought them together."}
]