story-to-pack: add the corpora and the catalogue v2/v3

Fourteen corpora split and classified: aesop, bierce, chekhov, holmes,
keefe, lawson, lorimer, maupassant, nobody, plaintales, poe, torchy,
wallingford and winesburg, each with its splitter and the hand-written
groups and pages; catalogue v2 and v3; and the shared splitters
gutenberg_chunks.py, se_split.py and se_build.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rwKTmug58sEsJ72AuWsEi

tools/story-to-pack/probe/plaintales/README.md

@@ -0,0 +1,186 @@
+# Plain Tales reference set: *Plain Tales from the Hills* (1888)
+
+A hand-made comparison baseline for the story-to-pack pipeline, a sibling of the Jacobs set in
+`../jacobs/` and the Bierce set in `../bierce/`. Claude read every scene and wrote each summary,
+predicament, group and option. **No local inference was used.** Compare the local-model runs against it.
+Do not treat it as ground truth: it is one careful reading.
+
+| File | What it is |
+| --- | --- |
+| `corpus/plain-tales-from-the-hills.txt` | Project Gutenberg #1858, Rudyard Kipling, *Plain Tales from the Hills* |
+| `split_gutenberg.py` | → `stories.json` (35 stories, split on the body's upper-case headings and checked against the contents) and `chunks.json` (287 scenes, same 320-word paragraph rule as `../chunk.py`) |
+| `claude/batches/sNN.json` | per story: `{chunk, story, summary, predicament}`, the same schema as `../jacobs/claude/batches/` |
+| `claude/groups_src.py` | the groups, options and per-story protagonist standing |
+| `claude/build.py` | checks everything, then writes `claude/records.json` and `claude/groups.json` (unchanged from Jacobs: it resolves paths relative to itself) |
+| `claude/page.py` | → `claude/REFERENCE.html`, the readable page |
+
+Rebuild: `python3 split_gutenberg.py && cd claude && python3 build.py && python3 page.py`.
+
+## What is in it
+
+- **287 predicaments**: one sentence each (median 21 words, min 16, max 24), written to the
+  `redescribe.py` rules: no names, jobs, ranks, places or story-only objects, and no race,
+  nationality, caste or religion. The `topic_words` leak check flags 85 of 287. 82 are role or kin
+  words (wife, rival, husband, newcomer, suitor, guest, patron, host, thief), kept on purpose because
+  the design's cast is roles. The other 3 are "judge" used as a verb and "model" and "patient" used
+  as adjectives. Rank is kept only as relation: *someone above you*, *a superior*, *a junior*,
+  *a subordinate*, *the one at the top*, *a deputy*.
+- **29 groups**, each a situation that recurs in **three or more stories**, has at least five scenes,
+  and draws no more than half its members from one story (8 span 3 stories, 14 span 4, 5 span 5,
+  2 span 6). **All 29 pass the catalogue's recurrence rule.** Each scene sits in at most one group:
+  **184 are assigned and 103 fit nothing that recurs**. 34 of the 35 stories feed at least one group.
+  *The Story of Muhammad Din* feeds none.
+- **Options**: 3 to 5 per group, 135 in all (19 groups have 5, 10 have 4). 108 come from what
+  someone in a scene did or refused to do (`scene:N`), 10 are archetypes and 17 fill gaps. **30 scene
+  options cite a scene outside their own group.** That is looser than DESIGN.md's "drawn from other
+  scenes in the cluster", and `groups.json` marks each one `in_group: false`. As in Bierce, most are
+  the scene just before or after a member, where the character acts.
+- **Moves**: spine paths with a direction only (`+`, `-`, `risk`, `set`). There are no magnitudes,
+  as the design requires. Relationships are moved 113 times, `standing` 66, flags 49, the two skills
+  39 and `money` only 17. The relationship roles are mostly institutional: `superior`, `the_top`,
+  `patron`, `power_broker`, `company`, `colleagues`, `old_hands`, `mentor`, `protege` and `gatekeeper`,
+  besides `beloved`, `spouse`, `family` and `rival`. Flags record what the player did:
+  `placed_by_favour`, `favour_owed`, `overworked`, `secret_marriage`, `kind_lie`, `cover_story`,
+  `vendetta`, `kept_silent`, `double_life` and others.
+- **Skills** proposed for this corpus: `tact`, for handling people, reading the unwritten rules and
+  keeping a thing quiet, and `cunning`, reused from Jacobs, for the trick, the lever and the secret
+  put to use. Nerve and wits hardly arise. This book's dangers are social.
+- **Stage** comes from protagonist standing, judged **per story, not per scene**, in `STANDING` in
+  `groups_src.py`. Subalterns, first-post juniors, a signaller, an orphan, an addict and a loafer rank
+  low. Officials of some years' standing, their wives and the narrator rank middle. A colonel's
+  household, the viceroy's circle, the supreme council and a senior department's indispensable expert
+  rank high. The groups come out 15 low, 12 middle and 2 high. Two stories are hard calls, and both
+  are noted in the file. *Watches of the Night* is judged by the colonel's household, which bears its
+  consequences. *The Rout of the White Hussars* is judged middle, because its scenes alternate between
+  the colonel and the subaltern who defies him.
+
+## Against the catalogue
+
+The catalogue has 82 entries (`../catalogue/v2/catalogue_v2_src.py`). The mapping rule is the
+catalogue's own: a group joins an entry when it is the same predicament seen from the same side.
+
+**Fifteen groups match an existing entry. Four of them sharpen it:**
+
+- **G01 A word from someone powerful → `patron`.** The v1 entry is about being *seen* with the great.
+  Kipling makes it about **who gets the post**. The place goes to whoever has someone influential
+  speaking for them: a shiftless nephew's job, an appointment won through a society woman, an
+  accountant foisted on a bank by a politician and about to be replaced by a director's nominee.
+- **G06 Good work that counts for nothing → `overlooked`** (the v2 widening "good work that goes
+  unrecognised"). Kipling adds the edge: work that **shows up your seniors** is resented, not merely
+  unseen.
+- **G09 The rules nobody tells you** and **G10 Sent out young, with nobody watching → `newcomer`.**
+  Both are the newcomer's predicament, split in two. G09 is the clash with unwritten rules: the
+  know-all junior, the lecturing accountant, the ragged recruit. G10 is the **young man ruined by his
+  first posting**, going wrong far from home while nobody who sees it says a word. That is how Kipling
+  sharpens the entry, so it is not treated as a new one.
+- **G20 Swallowing it from someone above → `power_over_you`.** It is sharpened by *Miss Youghal's
+  Sais*, where a man in disguise must take insults from people he outranks.
+
+The other eleven map directly: G12 → `family_against_your_path` (the arranged-match side of it),
+G13 → `rival`, G15 → `wrongly_judged`, G17 → `revenge`, G23 → `promised_elsewhere`, G25 → `too_late`,
+G26 → `sign`, G27 → `dread`, G28 → `home_unseen` and G29 → `self_betrayal`. G17 is a new register for
+`revenge`: cold, comic and bloodless. The weapon is a hoax, a wager or the bureaucracy itself, and
+never a blow.
+
+**Fourteen are new situation types.** Each names its nearest neighbour, so a reviewer can merge it
+if it is judged a near-duplicate:
+
+- **Career politics:** G02 **the power behind the title** (the real decider has no title: a
+  colonel's wife who breaks marriages, a society woman who can have a man gazetted on leave, a
+  private secretary who runs the viceroy; neighbour `patron`). G03 **one chance to be heard at the
+  top** (reaching the one person who can decide; neighbour `patron`). G04 **a change imposed by
+  someone who does not know the ground** (a new colonel sells the drum-horse, a council drafts a
+  tenancy rule nobody asked for; neighbour `orders_without_reasons`). G05 **knowing what goes on
+  below** (knowledge nobody above has asked for, used to settle a matter; neighbour `despised`).
+- **Work and strain:** G07 **work nobody sees** (neighbour `dead_end`). G08 **the strain gives way**,
+  the breakdown that makes it visible (neighbour `courage_doubted` for the jockey only).
+- **Marriage and prospects:** G11 **the marriage or the career** (a match that would sink your
+  prospects, or prospects that forbid the match; neighbour `duty_against_love`, but the pull here is
+  ambition, not duty).
+- **Reputation:** G14 **everyone will hear of it** (containing the talk in a place where everyone
+  knows everyone; neighbours `caught_out`, `cover_up_spiral`). G16 **a lie to spare them** (neighbour
+  `their_secret`). G18 **it fell into your hands** (a misdelivered secret, an overheard story, a
+  quirk of the ground, used for advantage; neighbour `seen_too_much`, but the decision is opportunity,
+  not danger).
+- **Onlookers and outsiders:** G19 **saving them from themselves**, which is Kipling's "playing
+  Providence" (neighbour `losing_them`, but the meddler acts behind the other's back). G21 **seeing
+  it, and saying nothing** (neighbour `safe_while_others_suffer`). G22 **belonging to neither side**
+  (neighbours `station`, `outside_looking_in`). G24 **devoted to someone who does not care**
+  (neighbour `used`).
+
+**The brief's eight targets**, and where each landed:
+
+| Target | Found as | Strength |
+| --- | --- | --- |
+| a post decided by patronage | G01, G03 | strong: 5 stories |
+| a young man ruined by his first posting | G10 (and *In the Pride of His Youth*, whose scenes run through G06, G07, G08, G10, G11 and G25) | strong |
+| gossip that decides a career | G14, G15 | partial: gossip decides *reputation*, and only *The Bronckhorst Divorce-Case* ties it to a career outright |
+| a superior's wife who wields his power | G02 | thin: only *Watches of the Night* is literally that, so G02 generalises to unofficial power |
+| a clever junior who outwits the system | G03, G17, G18 | strong, but split by method: forcing a hearing, a hoax, a found secret |
+| overwork nobody sees | G07, G08 | strong: 6 stories |
+| a career sacrificed for a marriage, or the reverse | G11 | strong: 4 stories, both directions |
+| a newcomer who does not know the unwritten rules | G09 | strong |
+
+## How this corpus differs from Jacobs and Bierce
+
+It has the most stories of any reference set, 35, but they are short: 5 to 13 scenes each, 287 in all.
+So the bar was kept at three stories per group, and 29 groups cleared it.
+
+**It is one small institutional world.** Where Bierce recurs on fear and duty, Kipling recurs on
+posts, patronage, reputation and marriage prospects. The moves show it: `relationships` and
+`standing` dominate, `money` is light again (17 moves), and the proposed skills are social
+(`tact`, `cunning`) rather than physical. The predicaments transplant readily. "Someone set over you
+is imposing a change on something they do not understand" is as true of a freight yard as of a
+cavalry regiment.
+
+**A recurring cast inflates recurrence.** Mrs Hauksbee appears in four stories, and Mrs Reiver
+and Strickland each appear in two and are mentioned in a third. A group drawn largely from one recurring character is partly story identity
+under another name. The groups were checked for this. G02 and G19 each have two Hauksbee stories,
+but G02 has two other stories and G19 three that have nothing to do with her. G05 has two Strickland stories and two
+without him. A tool should still track characters who cross stories, as well as the stories themselves.
+
+**The narrator's essays open many stories.** Kipling often begins with a general reflection: the
+sheltered boy (14), the creed that nothing matters (16), the rottenness of racing (148), the
+earnestness that governments reward (197), every man thinking his work the pivot (262). Some become
+group members, because they name the situation the story then dramatises. Others stay unassigned.
+They are the scenes a summariser is most likely to turn into topics.
+
+**Unassigned (103)** is mostly plot mechanics with no second instance: Golightly's ride through the
+rain (118-121), the opium den's inventory (239-249), the regimental field day (209), the love-charm's
+history. It also includes three verse-only epigraph scenes (126, 147, 225) and the whole of
+*The Story of Muhammad Din*, a quiet sketch of a friendship with a servant's child that recurs nowhere.
+
+## How the colonial material was handled
+
+Kipling writes British India with the racial and imperial assumptions of 1888. Several stories turn
+on them: a convert returning to her people (*Lispeth*), a mixed-descent signaller whose courage the
+narrator credits to "white blood" (*His Chance in Life*), an "impossible" match with a girl of mixed
+descent (*Kidnapped*), a love affair across the colour line (*Beyond the Pale*), and men who have
+"gone native" (*To Be Held for Reference*, *The Gate of a Hundred Sorrows*). The rules applied:
+
+- **Predicaments name no race, nationality, caste or religion,** and never use them as a role word.
+  They carry the human situation. The signaller is "a lowly junior who takes command of a crisis
+  himself". The impossible match is "someone his whole circle considers impossible". The tenants
+  in *Tod's Amendment* are "the people affected".
+- **Groups are formed on that human situation,** so these stories sit beside stories with no
+  colonial content. *Beyond the Pale* shares G23 with a young man in love with a married woman.
+  *Kidnapped* shares G11 with *Miss Youghal's Sais* and *His Chance in Life*.
+- **Summaries describe the plot plainly and do not adopt the narrator's framing.** Where the
+  narration itself carries the prejudice, the summary says so ("the narrator describes, with the
+  prejudice of his day...", scene 70). Epigraphs labelled as proverbs of one faith or people were
+  read as part of the first scene and did not drive any predicament.
+- **Two groups are the most shaped by the source, and a reviewer should look at them first.** G05
+  "knowing what goes on below" is Kipling's recurring figure of the Englishman who knows the natives,
+  here Strickland. G22 "belonging to neither side" is his anxiety about crossing over. Both are kept
+  because the abstract situation is real and portable: the manager who knows the shop floor, the
+  person who has left their class. But they are where the 1888 framing is closest to the surface.
+
+## Known defects
+
+- Scenes 126, 147 and 225 are verse epigraphs only. Scene 183 is a 55-word transition. All four were
+  summarised anyway, and none of them is in a group.
+- G23 and G27 each draw exactly half their members from one story (*On the Strength of a Likeness*
+  and *The House of Suddhoo*), which is on the recurrence rule's limit.
+- The standing of *Watches of the Night* and *The Rout of the White Hussars* is a judgement call (see
+  `STANDING`). Moving *Watches* to middle changes the stage of G02 and G15, and moving it to low
+  changes G02 and G14. Moving *White Hussars* changes G14, and G04 as well if it is moved up.