TheLadder/tools/story-to-pack/probe/keefe/README.md

# Keefe reference set: the Jack Keefe stories (Ring Lardner, 1914-1919)

A hand-made comparison baseline for the story-to-pack pipeline, a sibling of the Jacobs set in
`../jacobs/` and the Bierce set in `../bierce/`. Claude wrote every summary, predicament, group and
option by reading the scenes. **No local or hosted inference was used.** Compare the local-model
runs against it. It is not ground truth, just one careful reading.

The reading was split across **three readers**, each taking a run of stories (0-7, 8-15, 16-22) and
writing every scene's record plus a notes file of the recurring situations they saw. The
**grouping was done in one pass** by a fourth reader over all 912 records and the three notes files,
going back to the scene text where a record left it unsure. So no reader's grouping was
reconciled with another's: the groups are one judgement, made across the whole corpus at once.

| File | What it is |
| --- | --- |
| `corpus/jack-keefe-stories.html` | Standard Ebooks single-page text (public domain) |
| `../se_build.py` | → `stories.json` (23 stories, split on the `se:short-story` markup) and `chunks.json` (912 scenes, same 320-word paragraph rule as `../chunk.py`) |
| `claude/batches/sNN.json` | per story: `{chunk, story, summary, predicament}`, the same schema as `../jacobs/claude/batches/` |
| `claude/batches/NOTES-*.md` | the three readers' notes on recurring situations and protagonist standing, the input to grouping |
| `claude/groups_src.py` | the groups, options and per-story protagonist standing |
| `claude/build.py` | checks everything, then writes `claude/records.json` and `claude/groups.json`. It is identical to Jacobs' copy, since its paths are relative to the file. |
| `claude/page.py` | → `claude/REFERENCE.html`, the readable page (Jacobs' copy with the title and heading changed) |

Rebuild, from `../`: `python3 se_build.py keefe "Jack Keefe stories (Ring Lardner, 1914-1919)" && cd keefe/claude && python3 build.py && python3 page.py`.

## What is in it

- **912 predicaments**, one sentence each: median 21 words, range 15 to 25. They follow the
  `redescribe.py` rules, so there are no names, clubs, cities or story-only objects. The
  `topic_words` leak check flags 378 of 912, far more than any earlier set, and almost all of it is
  role words: wife 125, boss 78, newcomer 69, rival 44, father 28, husband 26, admirer 20. These are
  kept on purpose: the design's cast is roles, and this corpus is about exactly those roles. Place
  words are rare (dinner 4, stage 4, house 3, street 2, shop 1).
- **47 groups**, each a situation that recurs in **three or more stories**. **All 47 pass the
  catalogue's recurrence rule** (at least 5 scenes, at least 3 stories, no story over 50%). Three
  span 3 stories, 8 span 4, 7 span 5, 13 span 6, 7 span 7, 2 span 8, 5 span 9, one spans 13 and
  one 14. Each scene sits in at most one group: **650 are assigned and 262 are not**. Most of the
  262 fit nothing that recurs, and some were held out by the 50% cap (see Known defects). All 23
  stories feed at least one group.
- **Story 6 is capped like any other.** *The Busher Abroad* is 146 scenes, almost all pranks and tall
  tales. It supplies 64 assigned scenes to 20 groups, and 82 stay out. Its largest shares are 50% of
  the stinginess group (G33), 44% of the spotlight group (G31), 42% of *no word from home* (G40), and
  35-36% of the tall-tale, new-skill and swindle groups (G21, G29, G32). Many of its unassigned
  scenes are more of the same prank on the same man, such as the Bight bug, the "Thomas Hawks" and
  the equator rope. Adding them would push story 6 toward half of G21, and they add nothing the
  group does not already have. The rest are sightseeing, seasickness and banquets with no
  predicament in them.
- **Options**: 3 to 5 per group, 205 in all (28 groups have 4, 18 have 5, 1 has 3). 157 come from
  what someone in a scene did or refused to do (`scene:N`), 7 are archetypes and 41 fill gaps.
  **20 scene options cite a scene outside their own group**, and `groups.json` marks each one
  `in_group: false`. 13 of the 20 are within three scenes of a member, where the character acts.
- **An unreliable narrator.** Keefe tells his friend he won concessions, saw through every joke
  and only held back out of restraint. The summaries record what really happened, and the options
  do the same: a `scene:N` option is what he did, not what he claimed. Every group also carries at
  least one move a wiser person would make: own the mistake, check the story, turn the drink down,
  ask what the others are paid. Where nobody in the book makes that move it is marked `filled`,
  which is why this set has more fills than Bierce's (41 against 17).
- **Moves**: spine paths with a direction only (`+`, `-`, `risk`, `set`), and no magnitudes.
  `standing` moves 83 times, the two skills 71 times and `money` 38 times. Relationship roles are
  generic and **the same in both halves of the book**. `boss` is the manager or owner in baseball
  and the captain or sergeant in the army, and `colleagues` are teammates or comrades. The others are
  `spouse`, `friend` (the one back home), `family`, `inlaws`, `admirer`, `jokers`, `victim` and
  `rival`. Flags record what the player did: `held_out`, `threatened_to_leave`, `walked_out`,
  `fear_as_duty`, `saved_face`, `strung_along`, `looked_away`, `gamed_the_rules` and others.
- **Skills** proposed for this corpus: **`savvy`**, for seeing through a joke, a con or a flatterer
  and knowing your own worth, and **`discipline`**, for doing what the work demands (training,
  sobriety, turning up) whatever you feel like. These are the two things Keefe lacks in every
  story. `nerve` from Jacobs and Bierce was considered and dropped. His danger is almost never
  physical, and where it is (G15-G17) the question is honesty about fear, not the fear itself.
- **Stage** comes from protagonist standing, judged **per story, not per scene**, in `STANDING` in
  `groups_src.py`, starting from the readers' notes. A rookie on trial, a pitcher sent to the
  minors, a draftee private, a corporal of seven men and a wounded veteran out of work rank low.
  A regular big-league pitcher ranks middle. **No story ranks high.** Keefe never rises above a
  working pitcher, and the owners and generals above him are never the protagonist. The groups come
  out 25 middle and 22 low. A pack built from this corpus alone would have no top stage. It would
  need the ladder's upper rungs from another source, or would have to treat "regular starter" as the
  top.

## Groups against the existing catalogue

The 82-entry catalogue is in `../catalogue/v2/catalogue_v2_src.py`. The mapping below follows its
rule: a group joins an entry when the player faces the same decision from the same side, and it is
new when the decision differs. It was judged by reading, not by `classify.py`. **28 groups land on 25
existing entries, and 19 are new situation types.** Keefe is not yet in `SOURCES`. Adding him is a
separate change to that file.

| Group | Scenes / stories | Halves | Catalogue |
| --- | --- | --- | --- |
| G01 Asking for more money | 25 / 7 | baseball | **new** (`pay_talks`) |
| G02 Threatening to leave for a rival employer | 36 / 8 | baseball | **new** (`threaten_to_leave`) |
| G03 Sent down, or left on the bench | 20 / 9 | both | **new** (`sent_down`) |
| G04 A boss who manages by mockery | 20 / 9 | baseball | `power_over_you`, sharpened |
| G05 Whose fault was it? | 26 / 14 | both | **new** (`blame_elsewhere`) |
| G06 A step up goes to your head | 14 / 6 | both | **new** (`raised_a_step`) |
| G07 Called to account for a lapse | 14 / 7 | both | `called_to_account`, sharpened |
| G08 Sure you know better than the boss | 14 / 9 | both | `orders_without_reasons`, widened |
| G09 The boss takes a hand in your private life | 16 / 7 | baseball | `bow_for_your_living` (its own example) |
| G10 Counting on who you used to be | 8 / 4 | both | **new** |
| G11 The big occasion, and you are not ready | 15 / 6 | both (thin) | **new** (`big_chance`) |
| G12 A personal score in the middle of the work | 10 / 5 | baseball | `revenge`, partly |
| G13 A drink against the rules | 12 / 6 | both | **new** (nearest `habit`) |
| G14 Looking for a way out of what everyone must do | 8 / 5 | both | **new** (`way_out`) |
| G15 Fear passed off as something nobler | 16 / 9 | both | `courage_doubted`, sharpened |
| G16 Talking brave, and being taken at your word | 9 / 8 | both | `courage_doubted`, sharpened |
| G17 Sure you are going to your death | 8 / 6 | both (thin) | `condemned`, partly |
| G18 Waiting, with nothing to do | 12 / 7 | both | `on_watch` |
| G19 Hardship you think is beneath you | 10 / 6 | both (thin) | **new** (nearest `far_from_home`) |
| G20 New, and sure you are already the best | 13 / 6 | both | `newcomer`, sharpened |
| G21 A tall tale of danger from the old hands | 23 / 7 | both | `warning`, widened; sharpens `newcomer` |
| G22 A flattering prospect that may be a joke | 15 / 5 | both | `sure_thing`, widened |
| G23 Saving face after being made a fool of | 23 / 13 | both | **new** (`saving_face`) |
| G24 Made a public laughing-stock | 12 / 7 | both | **new** |
| G25 A cruel joke on someone who cannot take it | 8 / 3 | army | `safe_while_others_suffer`, partly |
| G26 Needled by a colleague | 7 / 6 | both | **new** (the peer version of `power_over_you`) |
| G27 Sure a colleague is up to something | 10 / 4 | both | `not_who_they_seem`, from the mistaken side |
| G28 Boasting to people who know better | 12 / 6 | both | `pretence`, sharpened |
| G29 A new skill that looks easy | 14 / 5 | both | **new** (nearest `outmatched`) |
| G30 You swore you were above it | 10 / 4 | both | `own_words_back` |
| G31 In the spotlight, sure you are the hit | 9 / 4 | both (thin) | **new** |
| G32 A bargain that is a swindle | 17 / 4 | both (thin) | `sure_thing` |
| G33 Expected to pay, and you hate paying | 10 / 5 | baseball | `grease`, from the payer's side |
| G34 Leaning on the friend back home | 10 / 7 | both | **new** |
| G35 Whose money is it at home? | 16 / 6 | baseball | `peace_at_home` |
| G36 A spouse who earns more | 14 / 4 | baseball | **new** (`spouse_earns_more`) |
| G37 A quarrel at home: who gives way? | 11 / 6 | both (thin) | `peace_at_home` |
| G38 Coming home to find them gone | 10 / 4 | both (thin) | `home_unseen`, partly |
| G39 Suspecting your spouse | 7 / 3 | both (thin) | `what_they_hid`, partly |
| G40 Far away, and no word from home | 12 / 5 | both | `far_from_home` |
| G41 Work pulls against a promise to your family | 10 / 6 | both | `duty_against_love`, a lighter register |
| G42 Relatives under one roof | 11 / 4 | baseball | `unwelcome_in_home`, from both sides |
| G43 Your spouse snubs people you owe | 5 / 3 | both (thin) | `station`, partly |
| G44 A new parent sure they know best | 11 / 6 | both (thin) | **new** (`new_parent`) |
| G45 Sure they are smitten with you | 22 / 9 | both | **new** (nearest `mistook_them`) |
| G46 Stringing along an admirer while you are not free | 26 / 6 | both | `promised_elsewhere`, from the bound side |
| G47 Caught with an admirer | 9 / 5 | both | `caught_out` |

"Halves" counts stories 0-7 and 18-22 as baseball and 9-17 as the army. Story 8 is the
draft summer, spent at home while he is still a ballplayer, and it is counted with baseball here. "Both
(thin)" means one half supplies only one story.

**The readers' ten flagged workplace situations, and where they went:**

- **Pay talks where you ask big and settle small** (G01) and **threatening to leave for a rival
  employer** (G02) are new. They are the corpus's spine: G02 is the largest group, with 36 scenes
  over 8 stories. The catalogue has nothing on bargaining over your own pay or on leverage you
  may not have.
- **Being sent down** (G03) is new. The nearest entry, `overlooked`, is about good work going
  unnoticed. Here the decision is how to take a demotion: report and climb back, refuse, or
  pretend it is a compliment.
- **A boss who manages you by mockery** (G04) sharpens `power_over_you`. The contempt is a
  management style, and the player's choice is whether to read it as praise.
- **Blaming everyone but yourself** (G05) is new, and it spans the most stories of any group (14).
- **A small promotion going to your head** (G06) is new.
- **Fear passed off as duty** (G15) sharpens `courage_doubted`. It adds the move the entry lacked:
  hiding fear behind a nobler motive rather than proving courage. G16, *talking brave and being
  taken at your word*, sharpens the same entry from the boaster's side.
- **Being called to account for a lapse** (G07) sharpens `called_to_account`. Here the lapse is real,
  and the judge is often fair. In the army half one captain lets him off because good corporals
  are scarce (506), and another asks for time to investigate before judging (566).
- **The gullible newcomer pranked by old hands** splits four ways. G20, the arrogant newcomer,
  sharpens `newcomer`. G21, the false warning, widens `warning`: the player faces the same decision,
  whether to believe it, but here the danger is invented. G22, the flattering hoax, widens
  `sure_thing`: the lure is recognition rather than money, and the price is ridicule. G23 and G24
  are new, and they are about the aftermath: what story you tell, and how you take being the joke.
- **A spouse who earns more** (G36) is new.

The other new workplace types are G10 *counting on who you used to be*, G11 *the big occasion
and you are not ready*, G13 *a drink against the rules*, G14 *a way out of what everyone must do*,
G26 *needled by a colleague* and G29 *a new skill that looks easy*. G30 gives `own_words_back`,
marked `recurs=False` in v1, 10 scenes over 4 stories. It recurs here.

## Do situations transfer between baseball and the army?

This is a direct test of the project's premise, because one narrator moves between two unlike
institutions and the same kind of reader grouped both. **Mostly yes.**

- **27 of 47 groups draw at least two stories from each half**, and 10 more reach both halves
  with a single story on one side. Only 9 are baseball-only and 1 army-only.
- **The situations that transfer are about standing inside a hierarchy.** Examples: being sent down
  (a pitcher to the minors, a corporal back to private), a step up going to your head, being called
  to account, sure you know better than the boss, blaming everyone else, the arrogant newcomer
  (the rookie at spring training and the draftee at camp), saving face, a new skill that looks easy.
  So do the social and home situations: admirers, no word from home, the friend back home.
- **The situations that do not transfer are about a labour market.** Pay talks (G01), threatening to
  leave for a rival (G02), the boss who meddles in your private life to protect his investment (G09)
  and a personal score settled on the field (G12) have no army instances. That is structural: a
  draftee cannot name his price, and he cannot quit for a rival army. The nearest army moves are
  G14 (looking for an exemption) and G15 (hiding fear behind duty). They are the soldier's version
  of "I could always walk": he looks for a way out rather than a better offer.
- **One situation transfers with its role moved.** In baseball, mockery comes from above (G04:
  Gleason, Callahan, Comiskey, McGraw). In the army the officers he faces are mostly fair-minded (506,
  566, 671), and the mockery comes from peers: the jokers of G21, G22 and G24, 29 army scenes between them.
  The situation "someone who knows more makes a fool of you" survives, but the counterpart changes
  from `boss` to `jokers`. A pack generator that keyed relationships to institution-specific roles
  would miss this. Generic roles, `boss` and `colleagues` in both halves, keep it visible.

The limit of the test: both halves share one narrator, one voice and one set of flaws. What transfers
may partly be Keefe rather than the situation. A cleaner test would need two authors writing the two
institutions.

## How this corpus differs

- **It is the largest set so far by scenes**: 912 from 23 stories, against O. Henry's 839 and
  Bierce's 299. The
  stories are long letter sequences, so a situation often runs over many consecutive scenes, like
  the pay standoff of story 18, the admirer of story 11 and the spy hunt of story 15. The 50% cap
  is what keeps those arcs from becoming one-story groups.
- **It is comic, and the comedy is dramatic irony.** Nearly every predicament is about the gap
  between what happened and what Keefe believes. That makes it rich in decisions about
  self-knowledge, such as whether to believe the praise, the warning or the admirer, and poor in
  decisions about danger. `money` still moves (38 times, against Bierce's 9), because pay and
  household money are real stakes here.
- **It is the richest corpus yet in workplace situations.** 13 of the 19 new types are about work and the people you
  work with, where earlier corpora recurred on courtship, secrets, fear and grief.
- **Scenes straddle letters.** Many chunks join the end of one letter to the start of the next. The
  reader's predicament follows the half that carries more weight, so a few scenes sit in a group
  on the strength of half their text.

## Known defects

- **Three scenes are only a dateline or a sign-off**: 184 (4 words), 264 (9) and 274 (10). A short
  closing paragraph starts a new scene and is then flushed alone when the next paragraph is long.
  They are kept so that `chunks.json` stays the corpus as split, and none is in a group.
- **One-story arcs stay unassigned even when they are strong situations.** Examples: the
  go-between who reads the reply as love for himself (story 21, scenes 841-858), courtship on the
  rebound (story 1) and the night rescue of Simple Simon (story 17, 726-737). None recurs in three
  stories.
- **The cap holds out scenes that fit.** Beyond story 6, the exemption board in story 8 (421, 430,
  432-436) fits G14 but would push story 8 over half of it. The rest of the spy hunt in story 15
  (656-661, 664-666, 674) fits G27 for the same reason.
- No batch record was found to be wrong. The readers' notes cite example chunk ids that sometimes
  differ from the final groups, because the notes were input, not a grouping.