JesseMarkowitzandClaude Opus 5
e5617b86ba
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
47 lines
11 KiB
JSON
47 lines
11 KiB
JSON
[
|
|
{"chunk":0,"story":0,"summary":"A writer on assignment casts about for someone who can tell him what the city means, and starts with the woman he courts.","predicament":"A man needs an answer only other people can give him, and does not know whom to ask first."},
|
|
{"chunk":1,"story":0,"summary":"A man presses the woman he courts for an answer she turns aside with jokes; he insists, and she only smiles.","predicament":"Someone presses another for an answer they keep deflecting, and must decide whether to keep pushing."},
|
|
{"chunk":2,"story":0,"summary":"A man questions a bartender about the city's meaning; the bartender is pulled away by a neighbour's errand and talks about his family instead.","predicament":"Someone seeks an answer from a busy person whose own small duties keep crowding out the question."},
|
|
{"chunk":3,"story":0,"summary":"A man asks a policeman what the city says; the policeman says he only takes orders, then talks about his new wife.","predicament":"Someone asks an official a large question and gets only the official's private life in reply."},
|
|
{"chunk":4,"story":0,"summary":"A man asks a poet for a grand utterance summing up the whole city, laying out at length what the answer should contain.","predicament":"Someone asks an expert for a sweeping answer, and specifies it so tightly that no answer can fit."},
|
|
{"chunk":5,"story":0,"summary":"A poet ignores the question to talk about a girl, and a news-seller wants only his money; the questioner leaves with nothing.","predicament":"Everyone someone asks is absorbed in their own affairs, leaving the question unanswered and the asker alone."},
|
|
{"chunk":6,"story":0,"summary":"A man returns to the woman he courts and sits silent beside her; she notes he has not spoken, and that is his answer.","predicament":"Someone comes back to the person who matters, and finds silence says what all the asking could not."},
|
|
{"chunk":7,"story":1,"summary":"A low-paid clerk and his wife live an ordinary flat-bound life, with neither poverty, love nor war to mark it.","predicament":"Someone's life is comfortable, small and unmarked, and they want something to happen in it."},
|
|
{"chunk":8,"story":1,"summary":"A housewife's days are made of bargains, window-sills and small superiorities over a neighbour, in a city where large things happen suddenly.","predicament":"Someone fills empty days with small rivalries while chance waits to overturn everything without warning."},
|
|
{"chunk":9,"story":1,"summary":"A clerk tries to make conversation with his wife about his workplace, fails, and goes out for a cigar instead.","predicament":"Someone tries to make talk with the person nearest them, gets nothing back, and steps outside."},
|
|
{"chunk":10,"story":1,"summary":"A customer finds himself penniless and offers to pay tomorrow; the shopkeeper attacks him and the two fight into the street.","predicament":"Someone asks for a moment's trust over a trifle and is answered with violence."},
|
|
{"chunk":11,"story":1,"summary":"A fleeing man strikes the officer who intervenes, runs, and is picked up by a stranger's car that carries him to a mansion.","predicament":"Someone in flight from a trivial quarrel is swept into a stranger's plans without being asked."},
|
|
{"chunk":12,"story":1,"summary":"A wealthy young woman, mistaking a stranger for a champion, asks him to confront a man who has insulted her.","predicament":"Someone is mistaken for the person another needs, and is asked to fight a stranger's battle."},
|
|
{"chunk":13,"story":1,"summary":"A man takes a woman's flower and confronts her enemy on her behalf; the young man puts him out the door effortlessly.","predicament":"Someone accepts a favour, acts as a champion, and is removed without effort by the one they challenged."},
|
|
{"chunk":14,"story":1,"summary":"The quarrel proves a lovers' game between a couple; the outsider walks home and resumes the dull talk he left.","predicament":"Someone discovers the drama they were pulled into was never theirs, and returns to an unchanged life."},
|
|
{"chunk":15,"story":2,"summary":"A young shop assistant behind a counter is beautiful and shrewd, schooled by the other girls in how to handle men who buy.","predicament":"A young person with looks and little money learns to guard themselves among strangers who want their attention."},
|
|
{"chunk":16,"story":2,"summary":"A rich young man wanders into a store where the counter girls flirt with cheap admirers, and the shop assistant meets his eye.","predicament":"Someone of high standing strays into a place where their advantages mean nothing, and hesitates."},
|
|
{"chunk":17,"story":2,"summary":"A rich man realises he stands among ordinary suitors, resolves to win the shop assistant anyway, and leaves his card on the counter.","predicament":"Someone wants a person far outside their circle, and has no proper way to approach them."},
|
|
{"chunk":18,"story":2,"summary":"A rich man asks permission to call; the shop assistant refuses her crowded home and names a street corner instead.","predicament":"Someone accepts a suitor's approach but cannot let them see how they live."},
|
|
{"chunk":19,"story":2,"summary":"A workmate teases the shop assistant about her rich admirer, while he worries that her easy consent means she is common.","predicament":"Someone's suitor takes their ease of manner for cheapness, and neither can read the other's world."},
|
|
{"chunk":20,"story":2,"summary":"A rich man proposes marriage; the shop assistant disbelieves his station and takes his declarations for the usual flattery.","predicament":"Someone offers everything they have and is not believed by the person they offer it to."},
|
|
{"chunk":21,"story":2,"summary":"A suitor paints a life of foreign travel and palaces; the shop assistant recognises the picture and stands up.","predicament":"Someone describes the future they are offering, and the other hears something entirely different in it."},
|
|
{"chunk":22,"story":2,"summary":"The shop assistant ends it and tells a workmate that the rich man only wanted a cheap wedding trip.","predicament":"Someone rejects the best offer they will ever get, having misread it completely."},
|
|
{"chunk":23,"story":3,"summary":"A man belongs to a clannish set who live in hotel lobbies and keep their home lives out of sight.","predicament":"Someone's standing among their own set depends on keeping their private life invisible."},
|
|
{"chunk":24,"story":3,"summary":"A man treats his wife as a fixture at home, dining out nightly with his set and barely hearing her talk.","predicament":"Someone has stopped noticing the person they live with, and does not know what they are missing."},
|
|
{"chunk":25,"story":3,"summary":"A wife asks her husband to take her out after three years indoors; he agrees, and she comes down transformed.","predicament":"Someone finally asks for what they have gone without for years, and is answered."},
|
|
{"chunk":26,"story":3,"summary":"A husband, surprised by his wife's appearance, changes his plan and resolves to show her to his friends rather than hide her.","predicament":"Someone realises what they kept hidden is worth showing, and reverses course to display it."},
|
|
{"chunk":27,"story":3,"summary":"A man's friends stare as he passes with his wife; at dinner she blooms, and shame reaches him for the years she spent shut in.","predicament":"Someone sees what their neglect cost another, and feels it in front of witnesses."},
|
|
{"chunk":28,"story":3,"summary":"An influential friend joins their table and the wife charms him and a crowd of men, while her husband sits without words.","predicament":"Someone watches the person they took for granted admired by everyone but themselves."},
|
|
{"chunk":29,"story":3,"summary":"The husband takes his wife's cloak before his friend can, walks her home, and refuses to return to his set.","predicament":"Someone, made jealous of what they neglected, chooses to keep it rather than hand it round."},
|
|
{"chunk":30,"story":4,"summary":"Three weeks married, a bride idly asks her husband for out-of-season fruit, and he leaves at once to get it.","predicament":"Someone newly devoted takes a passing whim as a command and sets out to satisfy it."},
|
|
{"chunk":31,"story":4,"summary":"A husband tries a street vendor and a friend's establishment; both say the fruit is out of season and offer a substitute.","predicament":"Someone hunting one particular thing is repeatedly offered a substitute they refuse to take."},
|
|
{"chunk":32,"story":4,"summary":"Finding every shop shut, a husband goes to a police captain and offers to lead a raid on a gambling house.","predicament":"Someone, blocked at every ordinary door, offers an official information to get inside a closed place."},
|
|
{"chunk":33,"story":4,"summary":"A man guides a dozen officers to a gambling room, the door is broken in, and most of the players escape.","predicament":"Someone sets a large action in motion for a reason nobody around them suspects."},
|
|
{"chunk":34,"story":4,"summary":"The gambling house's owner attacks the guide; after beating him, the guide hunts the supper room and questions a servant about fruit.","predicament":"Someone fights through real danger for a trifle that only matters to one person."},
|
|
{"chunk":35,"story":4,"summary":"One piece of fruit survives the supper table; the man pockets it and has a druggist check the rib he hurt falling downstairs.","predicament":"Someone gets the thing they went out for, at a cost nobody will ever weigh."},
|
|
{"chunk":36,"story":4,"summary":"The husband lays the fruit in his wife's hand; she says she would rather have had the other kind after all.","predicament":"Someone delivers what was asked and learns the asker has already changed their mind."},
|
|
{"chunk":37,"story":5,"summary":"Spring is announced in the city by newspaper items, but the true sign is when one person goes seeking another.","predicament":"Someone waits for a sign of change and finds it only when feeling moves them."},
|
|
{"chunk":38,"story":5,"summary":"Three idlers on a bench plot how to get the one dollar a married man's wife earned by washing.","predicament":"Idle companions plan how to take the only money earned by someone who works for it."},
|
|
{"chunk":39,"story":5,"summary":"The husband boasts he can choke the money out of his wife, then goes home to a woman who despises him.","predicament":"Someone boasts of a cruelty to companions, then faces alone the person they must get it from."},
|
|
{"chunk":40,"story":5,"summary":"The husband claims a new position and asks for the money; his wife calls him a liar and refuses.","predicament":"Someone invents a respectable reason for needing money and is seen through at once."},
|
|
{"chunk":41,"story":5,"summary":"The husband tries self-pity; his wife mocks it, and he decides to try the tenderness he abandoned years ago.","predicament":"Someone runs out of excuses and considers using affection they no longer feel."},
|
|
{"chunk":42,"story":5,"summary":"The husband puts his arm around his wife and calls her old endearments; she breaks down, holds him, and hurries out.","predicament":"Someone performs an affection they do not mean, and the other believes it entirely."},
|
|
{"chunk":43,"story":5,"summary":"The wife returns having spent the dollar on a tonic for her husband, and sits on his lap; his friends still wait.","predicament":"Someone's pretence is repaid with a kindness that costs them exactly what they were after."}
|
|
]
|