JesseMarkowitzandClaude Opus 5
e5617b86ba
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
43 lines
9.4 KiB
JSON
43 lines
9.4 KiB
JSON
[
|
|
{"chunk":563,"story":62,"summary":"From his high seat a cab driver sees every passenger as mere goods in transit, to be despised at the reckoning.","predicament":"Someone who carries others for pay sees them only as fares."},
|
|
{"chunk":564,"story":62,"summary":"A wedding is celebrated noisily in a tenement, and a cab driver stands by his fine hansom, plainly drunk.","predicament":"Someone at a celebration has drunk too much to know what they are doing."},
|
|
{"chunk":565,"story":62,"summary":"A young woman steps into his cab and says to drive anywhere; he takes her through the park, singing.","predicament":"Someone accepts a passenger without noticing who they are."},
|
|
{"chunk":566,"story":62,"summary":"He stops at a garden restaurant; she is swept inside, orders one beer with her coins, and watches entranced.","predicament":"Someone with a few coins is set down among luxury and simply enjoys it."},
|
|
{"chunk":567,"story":62,"summary":"Hours later he demands four dollars before going further; she laughs that she has only a few pennies.","predicament":"Someone demands payment from a passenger who never had the money."},
|
|
{"chunk":568,"story":62,"summary":"He drags her to the police station to complain, sobers, and introduces her to the sergeant as the bride he married tonight.","predicament":"Someone about to accuse a stranger recognises the person they married that evening."},
|
|
{"chunk":569,"story":63,"summary":"The narrator dreams of the day of judgment, where an angel policeman asks if he belongs with a prosperous-looking group.","predicament":"Someone at a final reckoning is asked which group of the accused they belong to."},
|
|
{"chunk":570,"story":63,"summary":"A shop girl on six dollars a week tells her friend she has accepted a dinner invitation from a known spender.","predicament":"Someone living on almost nothing accepts an invitation that may cost them more than money."},
|
|
{"chunk":571,"story":63,"summary":"She spends the fifty cents meant for three meals on an imitation lace collar, and goes up to her furnished room.","predicament":"Someone spends their food money on looking fit for an occasion."},
|
|
{"chunk":572,"story":63,"summary":"Her room holds a few treasures and portraits; her weekly budget for rent and food leaves nothing for clothes.","predicament":"Someone's wage exactly covers survival and nothing else."},
|
|
{"chunk":573,"story":63,"summary":"Her pleasures are counted by summers; the man calling for her is a well-dressed connoisseur of hungry shop girls.","predicament":"Someone starved of pleasure is about to be preyed on by a person who knows it."},
|
|
{"chunk":574,"story":63,"summary":"Dressed and happy, she turns for her handkerchief, meets a hero's portrait's stern eyes, and sends word she cannot go.","predicament":"Someone about to accept a dangerous invitation turns it down at the last moment."},
|
|
{"chunk":575,"story":63,"summary":"She cries, changes back into her old gown, and dines on crackers and jam, offering some to the portrait.","predicament":"Someone who refused a temptation consoles themselves alone with what little they have."},
|
|
{"chunk":576,"story":63,"summary":"She scolds the portrait; the story stops, and the dream's prosperous group proves to be employers who paid girls six dollars a week.","predicament":"Someone holds out tonight, knowing they may not the next time loneliness wins."},
|
|
{"chunk":577,"story":64,"summary":"A ragged man on a park bench believes himself a prince in disguise, far richer than the owners of the mansions opposite.","predicament":"Someone with nothing imagines that they have chosen to have nothing."},
|
|
{"chunk":578,"story":64,"summary":"He tells himself he has renounced wealth to study humanity and delights in bestowing gifts on the deserving.","predicament":"Someone powerless sustains themselves with the belief that they help others."},
|
|
{"chunk":579,"story":64,"summary":"A young man in evening clothes watches the clock tower in distress; the ragged prince approaches and offers his help.","predicament":"Someone anxiously waiting is offered help by a stranger who plainly has none to give."},
|
|
{"chunk":580,"story":64,"summary":"The prince denounces clocks as tyrants; the young man keeps glancing from the clock to a lit house opposite.","predicament":"Someone counting down to a verdict is lectured by a stranger about time."},
|
|
{"chunk":581,"story":64,"summary":"At ten to nine the young man nearly leaves, then explains he wronged his fiancee and awaits her decision.","predicament":"Someone who wronged the person they love waits for a sign of forgiveness."},
|
|
{"chunk":582,"story":64,"summary":"She promised a white scarf in the window at half past eight if she forgave him; the hour has passed and he despairs.","predicament":"Someone believes the deadline for forgiveness has passed without a sign."},
|
|
{"chunk":583,"story":64,"summary":"The prince promises a fortune on his wedding day and falls asleep; the young man covers him kindly, then the clock strikes.","predicament":"Someone despairing is kind to a poorer stranger just before their luck turns."},
|
|
{"chunk":584,"story":64,"summary":"The scarf appears: the clock was half an hour fast; next morning police find the sleeping vagrant clutching a fifty-dollar bill.","predicament":"Someone despairs over a deadline that was never real, and a kindness is quietly repaid."},
|
|
{"chunk":585,"story":65,"summary":"A sightseeing coach sets out, and on its highest back seat sit a young man from a small town and his bride.","predicament":"Two people newly married see the great city together for the first time."},
|
|
{"chunk":586,"story":65,"summary":"The guide bellows mistaken sights at passengers who gape at whatever he names, the bride among them.","predicament":"People new to a place believe whatever the guide tells them."},
|
|
{"chunk":587,"story":65,"summary":"The bride's face shows she thinks her husband right and the world excellent; he is twenty-four and rising.","predicament":"Two people newly married are completely content with each other and the world."},
|
|
{"chunk":588,"story":65,"summary":"In front of them sits a girl in a flowered hat beside a sharp-eyed young man who belongs to hard streets.","predicament":"Two couples share a seat; one of them has something to hide."},
|
|
{"chunk":589,"story":65,"summary":"The two women exchange a look and a whisper; the girl's companion slips over the side just as officers stop the coach.","predicament":"Two strangers understand each other at a glance, and one acts to help the other."},
|
|
{"chunk":590,"story":65,"summary":"The officers take the bridegroom for a wanted burglar; he goes down amused, expecting to explain.","predicament":"Someone is mistaken for a wanted person and expects the mistake to clear itself."},
|
|
{"chunk":591,"story":65,"summary":"His bride tells him to go quietly as the burglar, so he fights the police in fury until her uncle clears him.","predicament":"Someone is betrayed in front of the police by the person they trust most."},
|
|
{"chunk":592,"story":65,"summary":"She explains that the other pair had married that morning, and she could not refuse another bride her happiness.","predicament":"Someone sacrifices their partner's peace to protect a stranger in the same happy state."},
|
|
{"chunk":593,"story":66,"summary":"A broker arrives with his stenographer and plunges into work; she lingers, then asks whether he ordered a replacement.","predicament":"Someone expected to be noticed today is treated as part of the furniture."},
|
|
{"chunk":594,"story":66,"summary":"The clerk says the broker ordered one; she says she will work until someone comes, and the day's frenzy begins.","predicament":"Someone learns they are being replaced and quietly carries on working."},
|
|
{"chunk":595,"story":66,"summary":"An applicant arrives; the broker denies ordering any replacement and says the place is hers as long as she likes.","predicament":"Someone's employer forgets a decision made the day before."},
|
|
{"chunk":596,"story":66,"summary":"In a lull the scent of lilac reminds him of her, and he rushes to her desk to propose at once.","predicament":"Someone overwhelmed by work suddenly remembers the one thing that matters to them."},
|
|
{"chunk":597,"story":66,"summary":"He proposes in a hurry, papers in hand; she weeps, then reminds him they were married the evening before.","predicament":"Someone is so consumed by work they forget the most important thing already happened."},
|
|
{"chunk":598,"story":67,"summary":"A patrolman on his beat finds a man in a dark doorway who says he is keeping an appointment made twenty years ago.","predicament":"Someone keeps a promise to meet an old friend after twenty years apart."},
|
|
{"chunk":599,"story":67,"summary":"He tells of parting from his best friend twenty years ago, each to make a fortune, and shows a diamond watch.","predicament":"Someone who prospered far away returns trusting a friend to keep their old word."},
|
|
{"chunk":600,"story":67,"summary":"The patrolman moves on; twenty minutes later a tall man hurries across the street and greets him as his old friend.","predicament":"Someone waiting in the rain is greeted by the person they have waited for."},
|
|
{"chunk":601,"story":67,"summary":"They walk arm in arm until, under a bright light, he sees the stranger is not his friend at all.","predicament":"Someone realises the person they trusted on sight is not who they claimed."},
|
|
{"chunk":602,"story":67,"summary":"The stranger arrests him and hands him a note: his friend was the patrolman, and could not make the arrest himself.","predicament":"Someone must choose between their friend and their duty, and sends another to act."}
|
|
]
|