JesseMarkowitzandClaude Opus 5
e5617b86ba
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
246 lines
15 KiB
JSON
246 lines
15 KiB
JSON
{
|
|
"model": "qwen3:14b",
|
|
"prompt": "You describe the predicament of the main person in a passage of fiction, in ONE sentence.\n\nA predicament is what that person is up against and what they want out of it: the kind of trouble or opportunity that could happen to anyone, in any line of work, in any town.\n\nRules:\n- Exactly one sentence, at most 20 words.\n- Leave out everything that ties it to this story: no names, no jobs or professions, no places, no businesses, and no objects that only belong to this story.\n- Say what the person is up against, and what they want.\n- Do not describe the writing, the mood, or who is telling the story.\n- Output only the sentence.",
|
|
"summaries": [
|
|
{
|
|
"chunk": 0,
|
|
"story": 0,
|
|
"summary": "A man struggles to understand the overwhelming complexity of a city's collective voice, seeking clarity in a world filled with individual sounds but no unified message.",
|
|
"model_output": "A man struggles to understand the overwhelming complexity of a city's collective voice, seeking clarity in a world filled with individual sounds but no unified message."
|
|
},
|
|
{
|
|
"chunk": 1,
|
|
"story": 0,
|
|
"summary": "The person is trying to understand the city's hidden message, hoping to find a unified voice that reveals its true nature.",
|
|
"model_output": "The person is trying to understand the city's hidden message, hoping to find a unified voice that reveals its true nature."
|
|
},
|
|
{
|
|
"chunk": 2,
|
|
"story": 0,
|
|
"summary": "A man determined to uncover the unique voice of a city faces the challenge of distinguishing its true character from the distractions and distractions of daily life.",
|
|
"model_output": "A man determined to uncover the unique voice of a city faces the challenge of distinguishing its true character from the distractions and distractions of daily life."
|
|
},
|
|
{
|
|
"chunk": 3,
|
|
"story": 0,
|
|
"summary": "A man tries to uncover the city's hidden voice through a cop, only to find that the cop's personal life and indifference to the city's soul leave him with no real answers.",
|
|
"model_output": "A man tries to uncover the city's hidden voice through a cop, only to find that the cop's personal life and indifference to the city's soul leave him with no real answers."
|
|
},
|
|
{
|
|
"chunk": 4,
|
|
"story": 0,
|
|
"summary": "A man seeks to capture the essence of a city's soul through a unique blend of its sounds, hoping to create a single, powerful vocalization that represents its spirit.",
|
|
"model_output": "A man seeks to capture the essence of a city's soul through a unique blend of its sounds, hoping to create a single, powerful vocalization that represents its spirit."
|
|
},
|
|
{
|
|
"chunk": 5,
|
|
"story": 0,
|
|
"summary": "A man seeks understanding from a city that refuses to speak, desperate to uncover a hidden truth that eludes everyone else.",
|
|
"model_output": "A man seeks understanding from a city that refuses to speak, desperate to uncover a hidden truth that eludes everyone else."
|
|
},
|
|
{
|
|
"chunk": 6,
|
|
"story": 0,
|
|
"summary": "Aurelia and the narrator sit in silence, caught between the quiet intimacy of the moment and the unspoken weight of the city's presence.",
|
|
"model_output": "Aurelia and the narrator sit in silence, caught between the quiet intimacy of the moment and the unspoken weight of the city's presence."
|
|
},
|
|
{
|
|
"chunk": 7,
|
|
"story": 1,
|
|
"summary": "John Hopkins must navigate the harsh realities of poverty, love, and war all at once in the crowded, unforgiving city.",
|
|
"model_output": "John Hopkins must navigate the harsh realities of poverty, love, and war all at once in the crowded, unforgiving city."
|
|
},
|
|
{
|
|
"chunk": 8,
|
|
"story": 1,
|
|
"summary": "Mrs. Hopkins must navigate the chaotic unpredictability of city life while clinging to the comfort of her familiar routines.",
|
|
"model_output": "Mrs. Hopkins must navigate the chaotic unpredictability of city life while clinging to the comfort of her familiar routines."
|
|
},
|
|
{
|
|
"chunk": 9,
|
|
"story": 1,
|
|
"summary": "John Hopkins longs for connection but is trapped in a life of monotony and shallow conversation.",
|
|
"model_output": "John Hopkins longs for connection but is trapped in a life of monotony and shallow conversation."
|
|
},
|
|
{
|
|
"chunk": 10,
|
|
"story": 1,
|
|
"summary": "A man tries to pay for a purchase with a promise of future money, but his dishonesty sparks a violent confrontation with a shopkeeper who sees his actions as proof of humanity's corruption.",
|
|
"model_output": "A man tries to pay for a purchase with a promise of future money, but his dishonesty sparks a violent confrontation with a shopkeeper who sees his actions as proof of humanity's corruption."
|
|
},
|
|
{
|
|
"chunk": 11,
|
|
"story": 1,
|
|
"summary": "A man flees from police pursuit, seeking refuge in an unknown vehicle that offers him a chance to escape, but must decide whether to trust his mysterious rescuer.",
|
|
"model_output": "A man flees from police pursuit, seeking refuge in an unknown vehicle that offers him a chance to escape, but must decide whether to trust his mysterious rescuer."
|
|
},
|
|
{
|
|
"chunk": 12,
|
|
"story": 1,
|
|
"summary": "A man finds himself unexpectedly drawn into a personal conflict he did not ask for, yet must navigate with courage and discretion.",
|
|
"model_output": "A man finds himself unexpectedly drawn into a personal conflict he did not ask for, yet must navigate with courage and discretion."
|
|
},
|
|
{
|
|
"chunk": 13,
|
|
"story": 1,
|
|
"summary": "John Hopkins must confront a man who has wronged a woman he now feels compelled to protect, risking his own safety to do so.",
|
|
"model_output": "John Hopkins must confront a man who has wronged a woman he now feels compelled to protect, risking his own safety to do so."
|
|
},
|
|
{
|
|
"chunk": 14,
|
|
"story": 1,
|
|
"summary": "A man struggles to reconcile his growing affection for a woman with the rigid expectations of his social circle.",
|
|
"model_output": "A man struggles to reconcile his growing affection for a woman with the rigid expectations of his social circle."
|
|
},
|
|
{
|
|
"chunk": 15,
|
|
"story": 2,
|
|
"summary": "Masie, trapped in a monotonous job she despises, longs for a life that offers more than just the dull routine of selling gloves.",
|
|
"model_output": "Masie, trapped in a monotonous job she despises, longs for a life that offers more than just the dull routine of selling gloves."
|
|
},
|
|
{
|
|
"chunk": 16,
|
|
"story": 2,
|
|
"summary": "A man finds himself trapped in an awkward encounter with a shopgirl whose icy beauty challenges his composure, forcing him to navigate the delicate line between professional decorum and unexpected attraction.",
|
|
"model_output": "A man finds himself trapped in an awkward encounter with a shopgirl whose icy beauty challenges his composure, forcing him to navigate the delicate line between professional decorum and unexpected attraction."
|
|
},
|
|
{
|
|
"chunk": 17,
|
|
"story": 2,
|
|
"summary": "A man struggling with his pride and social conventions finds himself awkwardly drawn to a shopgirl, determined to win her favor despite his usual confidence.",
|
|
"model_output": "A man struggling with his pride and social conventions finds himself awkwardly drawn to a shopgirl, determined to win her favor despite his usual confidence."
|
|
},
|
|
{
|
|
"chunk": 18,
|
|
"story": 2,
|
|
"summary": "A young woman balances between curiosity and caution as she agrees to meet a man she barely knows, hoping to explore new possibilities without disappointing her family or losing her sense of self.",
|
|
"model_output": "A young woman balances between curiosity and caution as she agrees to meet a man she barely knows, hoping to explore new possibilities without disappointing her family or losing her sense of self."
|
|
},
|
|
{
|
|
"chunk": 19,
|
|
"story": 2,
|
|
"summary": "A man torn between his growing feelings for a woman he barely knows and his fear of losing control of his life struggles to navigate the uncertain path of love.",
|
|
"model_output": "A man torn between his growing feelings for a woman he barely knows and his fear of losing control of his life struggles to navigate the uncertain path of love."
|
|
},
|
|
{
|
|
"chunk": 20,
|
|
"story": 2,
|
|
"summary": "A man insists a woman marry him, claiming his wealth and status make them equal, but she sees through his words and mocks his desperate, insincere pursuit.",
|
|
"model_output": "A man insists a woman marry him, claiming his wealth and status make them equal, but she sees through his words and mocks his desperate, insincere pursuit."
|
|
},
|
|
{
|
|
"chunk": 21,
|
|
"story": 2,
|
|
"summary": "A man tries to convince a hesitant woman to leave her mundane life and embark on an extravagant, uncertain journey with him.",
|
|
"model_output": "A man tries to convince a hesitant woman to leave her mundane life and embark on an extravagant, uncertain journey with him."
|
|
},
|
|
{
|
|
"chunk": 22,
|
|
"story": 2,
|
|
"summary": "A woman hides her growing feelings for a man while pretending to be uninterested, hoping to keep their connection without giving in to the possibility of love.",
|
|
"model_output": "A woman hides her growing feelings for a man while pretending to be uninterested, hoping to keep their connection without giving in to the possibility of love."
|
|
},
|
|
{
|
|
"chunk": 23,
|
|
"story": 3,
|
|
"summary": "Big Jim Dougherty must navigate the rigid codes of his underground world while seeking a way to break free and claim a life beyond the shadows.",
|
|
"model_output": "Big Jim Dougherty must navigate the rigid codes of his underground world while seeking a way to break free and claim a life beyond the shadows."
|
|
},
|
|
{
|
|
"chunk": 24,
|
|
"story": 3,
|
|
"summary": "A man struggles to balance his cultural identity and personal life while navigating the expectations of his community and the quiet routine of his domestic existence.",
|
|
"model_output": "A man struggles to balance his cultural identity and personal life while navigating the expectations of his community and the quiet routine of his domestic existence."
|
|
},
|
|
{
|
|
"chunk": 25,
|
|
"story": 3,
|
|
"summary": "A man accustomed to routine is confronted with an unexpected request for companionship, forcing him to reconsider the balance between habit and connection.",
|
|
"model_output": "A man accustomed to routine is confronted with an unexpected request for companionship, forcing him to reconsider the balance between habit and connection."
|
|
},
|
|
{
|
|
"chunk": 26,
|
|
"story": 3,
|
|
"summary": "A man struggling to reconcile his desire to display his wife publicly with his fear of judgment finds himself torn between pride and secrecy.",
|
|
"model_output": "A man struggling to reconcile his desire to display his wife publicly with his fear of judgment finds himself torn between pride and secrecy."
|
|
},
|
|
{
|
|
"chunk": 27,
|
|
"story": 3,
|
|
"summary": "A man struggles to reconcile his pride with the growing awareness of his wife's hidden talents and the shame of having underestimated her.",
|
|
"model_output": "A man struggles to reconcile his pride with the growing awareness of his wife's hidden talents and the shame of having underestimated her."
|
|
},
|
|
{
|
|
"chunk": 28,
|
|
"story": 3,
|
|
"summary": "A man struggles to reconcile his quiet life with the sudden, overwhelming attention his wife receives, threatening to expose the hidden depths of their marriage.",
|
|
"model_output": "A man struggles to reconcile his quiet life with the sudden, overwhelming attention his wife receives, threatening to expose the hidden depths of their marriage."
|
|
},
|
|
{
|
|
"chunk": 29,
|
|
"story": 3,
|
|
"summary": "A man struggles to maintain his pride and dignity after a public embarrassment, while trying to preserve his relationship and quiet his wife's growing happiness.",
|
|
"model_output": "A man struggles to maintain his pride and dignity after a public embarrassment, while trying to preserve his relationship and quiet his wife's growing happiness."
|
|
},
|
|
{
|
|
"chunk": 30,
|
|
"story": 3,
|
|
"summary": "A woman trapped in a love that bends her will longs for a simple peach, while her powerful but obedient husband struggles to fulfill her wish in a world that resists giving way.",
|
|
"model_output": "A woman trapped in a love that bends her will longs for a simple peach, while her powerful but obedient husband struggles to fulfill her wish in a world that resists giving way."
|
|
},
|
|
{
|
|
"chunk": 31,
|
|
"story": 3,
|
|
"summary": "The Kid needs to find a peach for a woman who insists on it, but every vendor says it's too soon and offers only oranges instead.",
|
|
"model_output": "The Kid needs to find a peach for a woman who insists on it, but every vendor says it's too soon and offers only oranges instead."
|
|
},
|
|
{
|
|
"chunk": 32,
|
|
"story": 3,
|
|
"summary": "The Kid needs to convince the captain to raid a gambling den controlled by an old enemy, but he must do so quickly before the enemy discovers his betrayal.",
|
|
"model_output": "The Kid needs to convince the captain to raid a gambling den controlled by an old enemy, but he must do so quickly before the enemy discovers his betrayal."
|
|
},
|
|
{
|
|
"chunk": 33,
|
|
"story": 3,
|
|
"summary": "A man must prove his worth in a dangerous confrontation to claim a prize he's risking everything for.",
|
|
"model_output": "A man must prove his worth in a dangerous confrontation to claim a prize he's risking everything for."
|
|
},
|
|
{
|
|
"chunk": 34,
|
|
"story": 3,
|
|
"summary": "A man desperate to find a peach before it's too late races against time and rivals to secure his prize before it vanishes entirely.",
|
|
"model_output": "A man desperate to find a peach before it's too late races against time and rivals to secure his prize before it vanishes entirely."
|
|
},
|
|
{
|
|
"chunk": 35,
|
|
"story": 3,
|
|
"summary": "He risks danger to fetch a trivial gift for his bride, only to find her preference has changed.",
|
|
"model_output": "He risks danger to fetch a trivial gift for his bride, only to find her preference has changed."
|
|
},
|
|
{
|
|
"chunk": 36,
|
|
"story": 4,
|
|
"summary": "A man trapped in the monotony of city life longs for the genuine signs of spring, but finds only artificial reports replacing nature's quiet signals.",
|
|
"model_output": "A man trapped in the monotony of city life longs for the genuine signs of spring, but finds only artificial reports replacing nature's quiet signals."
|
|
},
|
|
{
|
|
"chunk": 37,
|
|
"story": 4,
|
|
"summary": "Three men plot to steal a dollar from Mr. Peters's wife, who unknowingly possesses it, forcing him to confront his friends' greed and protect her from their violent intentions.",
|
|
"model_output": "Three men plot to steal a dollar from Mr. Peters's wife, who unknowingly possesses it, forcing him to confront his friends' greed and protect her from their violent intentions."
|
|
},
|
|
{
|
|
"chunk": 38,
|
|
"story": 4,
|
|
"summary": "Mr. Peters must steal a case note from his wife's hidden stash to buy a dollar for beer, but he refuses to harm her despite his desperation.",
|
|
"model_output": "Mr. Peters must steal a case note from his wife's hidden stash to buy a dollar for beer, but he refuses to harm her despite his desperation."
|
|
},
|
|
{
|
|
"chunk": 39,
|
|
"story": 4,
|
|
"summary": "A man trying to manipulate a woman with false promises faces her sharp rejection and must find another way to achieve his goal.",
|
|
"model_output": "A man trying to manipulate a woman with false promises faces her sharp rejection and must find another way to achieve his goal."
|
|
}
|
|
]
|
|
} |