Replace clustering with a catalogue, and hand-write the references to judge it against
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
This commit is contained in:
co-authored by
Claude Opus 5
parent
fa3769d0fe
commit
e5617b86ba
@@ -0,0 +1,27 @@
|
||||
[
|
||||
{"chunk":212,"story":8,"summary":"A mother devoted to her perfect son senses from his walk that he is coming home to tell her something.","predicament":"Someone whose whole life centres on one person senses that person is in trouble and waits to be told."},
|
||||
{"chunk":213,"story":8,"summary":"In her exquisite house the son slumps by the fire, defeated, and she feels suddenly a stranger to him.","predicament":"Someone sees a person they know intimately in an unfamiliar state and feels shut out."},
|
||||
{"chunk":214,"story":8,"summary":"The son says the girl he loves has refused him on principle, because he owns a scandalous newspaper.","predicament":"Someone is turned down by the person they love because of the source of their income."},
|
||||
{"chunk":215,"story":8,"summary":"The girl wants him to give up the paper or run it honestly at a loss; he mocks her fashionable ethics.","predicament":"Someone is asked to give up their livelihood as the price of being loved."},
|
||||
{"chunk":216,"story":8,"summary":"Mother and son agree one cannot change oneself; the girl cites the mother's own example, and the mother notes the girl's family has never called on her.","predicament":"Someone is held up as an example in an argument they would rather not be drawn into."},
|
||||
{"chunk":217,"story":8,"summary":"The girl admires the mother's goodness; the mother, angry, is told her son is not asking her to make the girl love him.","predicament":"Someone resents a person who has hurt their child, and is asked not to show it."},
|
||||
{"chunk":218,"story":8,"summary":"The son asks his mother to visit the girl and show her that the paper has never touched her life.","predicament":"Someone is asked by the person they love most to plead a case they privately doubt."},
|
||||
{"chunk":219,"story":8,"summary":"The mother calls at the girl's plain, comfortless home and reads failure in the furniture.","predicament":"Someone goes to plead on another's behalf in a place where they feel they have no footing."},
|
||||
{"chunk":220,"story":8,"summary":"The girl is immovable; she says it is the mother's own beautiful example that confirms her refusal.","predicament":"Someone tries to persuade a person of firm principle, and their own example is used against them."},
|
||||
{"chunk":221,"story":8,"summary":"The mother argues that the girl is sacrificing her son to an idea; the girl says she is sacrificing herself.","predicament":"Someone argues with a young person who insists on giving up happiness for a principle."},
|
||||
{"chunk":222,"story":8,"summary":"The mother pleads that life is made of compromises and love outweighs theories; the girl says that is exactly why she cannot.","predicament":"Someone urges a younger person to compromise, and is told their plea proves the danger."},
|
||||
{"chunk":223,"story":8,"summary":"Six months later the son has buried his grief in the paper, and the mother seeks refuge in a gallery of paintings.","predicament":"Someone who cannot help a suffering child looks for solace of their own."},
|
||||
{"chunk":224,"story":8,"summary":"The unspoken subject has driven mother and son apart; in the gallery she meets the girl.","predicament":"Someone cut off from a loved one by an unspoken hurt comes face to face with its cause."},
|
||||
{"chunk":225,"story":8,"summary":"The girl looks worn and unhappy; the mother offers her a lift, is refused, and turns away.","predicament":"Someone reaches out to a former adversary and is coldly rebuffed."},
|
||||
{"chunk":226,"story":8,"summary":"The girl confesses she followed the mother there, has been unhappy, and now believes the mother was right; she asks whether the son still cares.","predicament":"Someone who once refused an offer comes back to say they have changed their mind, and asks if it still stands."},
|
||||
{"chunk":227,"story":8,"summary":"The mother says she too has suffered, and now admits the girl was right and she was wrong.","predicament":"Someone who argued for a compromise now confesses they knew all along it was wrong."},
|
||||
{"chunk":228,"story":8,"summary":"The girl believes love alone could change the son; the mother says she once said the same, before marrying his father.","predicament":"Someone about to make a hopeful sacrifice is told by an older person that they made the same one and regret it."},
|
||||
{"chunk":229,"story":8,"summary":"The mother says she speaks because the girl is as alive as she once was, and she cannot watch her be walled up.","predicament":"Someone breaks a lifetime's silence to save a younger person from their own fate."},
|
||||
{"chunk":230,"story":8,"summary":"She married without knowing what the paper was, enjoyed its money, then learned it traded in exposing private scandal.","predicament":"Someone discovers that the comfort they have enjoyed is paid for by something shameful."},
|
||||
{"chunk":231,"story":8,"summary":"The girl tries to stop her from going on.","predicament":"Someone watching another confess something painful wants to spare them."},
|
||||
{"chunk":232,"story":8,"summary":"Her husband promised to sell the paper in time, but her delicate son's life depended on the money, and she sank deeper with each compromise.","predicament":"Someone who meant to refuse tainted money comes to depend on it to save a person they love."},
|
||||
{"chunk":233,"story":8,"summary":"She tried to purify the money through charity and to hide the paper from her son behind beauty; on her husband's death she hoped the son would sell it.","predicament":"Someone who compromised long ago pins their hope of release on the next generation."},
|
||||
{"chunk":234,"story":8,"summary":"At twenty-one the son refused to sell, loving the power the paper gave; she saw he was what her concessions had made him.","predicament":"Someone finds that the person they raised has embraced the very thing they wanted to escape."},
|
||||
{"chunk":235,"story":8,"summary":"She gave up the struggle and learned to look only at the side of her son that was hers.","predicament":"Someone makes peace with a loved one's flaw by refusing to look at it."},
|
||||
{"chunk":236,"story":8,"summary":"She warns that the son's love would never change him and that the girl would slowly die as she did; she asks what to tell him, and leaves.","predicament":"Someone lays bare their own ruin to warn another, then leaves them to make the choice alone."}
|
||||
]
|
||||
Reference in New Issue
Block a user