JesseMarkowitzandClaude Opus 5
e5617b86ba
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
56 lines
2.6 KiB
JSON
56 lines
2.6 KiB
JSON
[
|
|
{
|
|
"chunk": 148,
|
|
"story": 11,
|
|
"summary": "In an inn, men trade ghost stories; a sceptical old sailor mocks sailors' visions, then admits one uncanny experience on land.",
|
|
"predicament": "A sceptic among believers is pressed to admit whether he has ever seen anything he could not explain."
|
|
},
|
|
{
|
|
"chunk": 149,
|
|
"story": 11,
|
|
"summary": "As a young man back from abroad, the narrator found his relatives away, took lodgings at a dull inn, and set out on a long country walk.",
|
|
"predicament": "A young man who arrived to find no one expecting him has to fill empty days in a strange place."
|
|
},
|
|
{
|
|
"chunk": 150,
|
|
"story": 11,
|
|
"summary": "He loses his way in the marshes as fog and night come down, and at last finds a road and hears footsteps.",
|
|
"predicament": "A traveller lost in the dark and the cold must find shelter before night closes in."
|
|
},
|
|
{
|
|
"chunk": 151,
|
|
"story": 11,
|
|
"summary": "A countryman sets him on the road but warns him away from a lit house nearby, said to hide something monstrous; tired, he goes to it anyway.",
|
|
"predicament": "A worn-out traveller must choose between a long walk on and a nearby shelter he has been warned against."
|
|
},
|
|
{
|
|
"chunk": 152,
|
|
"story": 11,
|
|
"summary": "A grim old woman and a huge old man answer; after a furtive exchange the man insists he stay for dinner, and he feels trapped.",
|
|
"predicament": "A stranger taken in by odd hosts who seem keen to keep him wonders whether he has walked into a trap."
|
|
},
|
|
{
|
|
"chunk": 153,
|
|
"story": 11,
|
|
"summary": "The host says his son will dine with them in the dark because of his eyes; he puts out the candles, and a strange voice speaks.",
|
|
"predicament": "A guest must sit down in the dark with someone he cannot see, on hosts' terms he does not understand."
|
|
},
|
|
{
|
|
"chunk": 154,
|
|
"story": 11,
|
|
"summary": "The host leaves him alone with the unseen son, and the guest thinks he sees wild eyes shining at him across the table.",
|
|
"predicament": "A guest left alone with a stranger he cannot see grows afraid, and must decide whether to trust his hosts."
|
|
},
|
|
{
|
|
"chunk": 155,
|
|
"story": 11,
|
|
"summary": "The screen falls, and firelight shows the son's ruined face; the father lights the candles and explains his son was disfigured in a fire.",
|
|
"predicament": "A frightened man sees what he feared, and learns it is someone who was hurt and ashamed to be seen."
|
|
},
|
|
{
|
|
"chunk": 156,
|
|
"story": 11,
|
|
"summary": "The guest apologises warmly; they celebrate the son's birthday until midnight, and the father toasts the children his son saved.",
|
|
"predicament": "A man who shrank from someone must make amends, and finds they deserve his respect."
|
|
}
|
|
] |