story-to-pack: add the corpora and the catalogue v2/v3

Fourteen corpora split and classified: aesop, bierce, chekhov, holmes,
keefe, lawson, lorimer, maupassant, nobody, plaintales, poe, torchy,
wallingford and winesburg, each with its splitter and the hand-written
groups and pages; catalogue v2 and v3; and the shared splitters
gutenberg_chunks.py, se_split.py and se_build.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rwKTmug58sEsJ72AuWsEi

tools/story-to-pack/probe/lawson/README.md

@@ -0,0 +1,141 @@
+# Lawson reference set: *While the Billy Boils* (1896)
+
+A hand-made comparison baseline for the story-to-pack pipeline, a sibling of the Jacobs set in
+`../jacobs/` and the Bierce set in `../bierce/`. Claude read every scene and wrote each summary,
+predicament, group and option. **No inference of any kind was used.** Compare the local-model runs
+against it. Do not treat it as ground truth: it is one careful reading.
+
+| File | What it is |
+| --- | --- |
+| `corpus/while-the-billy-boils.html` | Standard Ebooks single-page text (public domain) |
+| `stories.json`, `chunks.json` | 52 stories and 308 scenes, cut by `../se_build.py` (the `se:short-story` markup, then the same 320-word paragraph rule as `../chunk.py`). 81,507 words; scenes run from 62 to 543 words, median 282 |
+| `claude/batches/sNN.json` | per story: `{chunk, story, summary, predicament}`, the same schema as `../jacobs/claude/batches/` |
+| `claude/groups_src.py` | the groups, options and per-story protagonist standing |
+| `claude/build.py` | checks everything, then writes `claude/records.json` and `claude/groups.json` (unchanged from Jacobs: it resolves paths relative to itself) |
+| `claude/page.py` | → `claude/REFERENCE.html`, the readable page (Jacobs' page with the title and counts changed) |
+
+Rebuild: `python3 ../se_build.py lawson "While the Billy Boils (Henry Lawson, 1896)"` from `probe/`,
+then `cd lawson/claude && python3 build.py && python3 page.py`.
+
+## What is in it
+
+- **308 predicaments**: one sentence each (median 22 words, min 17, max 24), written to the
+  `redescribe.py` rules: no names, jobs, trades, places or story-only objects. Shearer, drover,
+  swagman, selector and squatter all become generic roles (a worker, a traveller, a small holder, a
+  boss, a powerful neighbour). The `topic_words` leak check flags 67 of 308. 66 are role or kin
+  words (wife, boss, mother, father, neighbour, newcomer, son, brother, sweetheart, visitor, lodger,
+  tenant, host), kept on purpose because the design's cast is roles. The other one is "makes room
+  for" in scene 0, an idiom, not a place. Ten place words (camp, table, house, street, store) were
+  caught by the check and rewritten.
+- **31 groups**, each a situation that recurs in **three or more stories** (8 span 3, 14 span 4, 5
+  span 5, 4 span 6). Every group has at least 5 scenes, and no story supplies more than half of any
+  group's members, so **all 31 pass the catalogue's recurrence rule** (at least 5 scenes, at least 3
+  stories, no story over 50%). Eight groups sit exactly at the 50% limit (G03, G04, G14, G16, G17,
+  G21, G22, G26), which is the thinnest margin in the set. Each scene sits in at
+  most one group: **210 are assigned and 98 fit nothing that recurs**. All 52 stories feed at least
+  one group.
+- **Options**: 3 to 5 per group, 142 in all (18 groups have 5, 13 have 4). 118 come from what
+  someone in a scene did or refused to do (`scene:N`), 12 are archetypes and 12 fill gaps. **14
+  scene options cite a scene outside their own group.** That is looser than DESIGN.md's "drawn from
+  other scenes in the cluster", and `groups.json` marks each one `in_group: false`. As in Bierce,
+  most are the scene just before or after a member, where the character acts (scene 22 hobbles the
+  pursuit that scene 23 escapes; scene 30 is the rescue scene 29 sets up).
+- **Moves**: spine paths with a direction only (`+`, `-`, `risk`, `set`). There are no magnitudes,
+  as the design requires. `money` is moved 63 times and `standing` 51, the two skills 43 times
+  between them, and relationships 123 times. Relationship roles are generic and mostly working ones:
+  `family`, `mate` (one friend), `mates` (the crew or your side), `boss`, `creditor`, `stranger`,
+  `authority`, `sweetheart`, `crew`, `butt`, `child`, `rival`. Flags record what the player did:
+  `did_a_flit`, `worked_sick`, `pilfered`, `informer`, `on_the_spree`, `worked_a_mark`,
+  `went_away`, `moved_on`, `fought_for_mate`.
+- **Skills** proposed for this corpus: `grit`, for enduring hardship and holding on when it would be
+  easier to give up, and `nous`, for the practical wit of getting by, getting round and getting fed.
+  Jacobs' `cunning` and Bierce's `nerve` do not fit: Lawson's people are rarely afraid and seldom
+  scheming, but they are always tired and usually short of a meal.
+- **Stage** comes from protagonist standing, judged **per story, not per scene**, in `STANDING` in
+  `groups_src.py`. 46 of the 52 stories rank low: swagmen, shearers, selectors, fossickers, the
+  unemployed and the city poor. Five rank middle (a man back from prospering abroad, the narrator
+  travelling on his own account, a settled householder, and two men long in the city). Only the
+  squatter of *Baldy Thompson* ranks high. **So every one of the 31 groups comes out stage `low`.**
+  That is the book, not a flaw in the grouping, but it means this corpus alone cannot fill the upper
+  stages of a pack. Stage for Lawson would have to come from something other than standing, for
+  example the size of the stake, or be supplied by a second corpus.
+
+## Against the catalogue (v2, 82 entries)
+
+**Ten groups match an existing entry closely:** G03 a mate always one move ahead of you
+(`outmatched`, with `used` for the mate who lets you carry the risk), G05 someone worse off than you
+(`plea_for_help`, the asked side, though here nobody has to ask), G17 keeping up appearances while
+broke (`pretence`), G20 talking a kind host out of a free feed (`used`, the user's side), G21
+vouching for a stranger (`not_who_they_seem`), G24 caught out at home (`caught_out`), G27 coming
+back to find it changed (`changed`), G28 the pull of the place you left (`homecoming`), G29 doing
+right by the dead (`death_as_burden`) and G30 too late (`too_late`).
+
+**Eight overlap an entry in part but put a different decision to the player:**
+
+- G01 an old mate down on his luck leans on you: near `plea_for_help` and `unwelcome_in_home`, but the lever is the claim of old mateship, not need or nuisance.
+- G02 standing by a mate nobody else will: near `against_the_verdict`, but the pull is loyalty to a mate who let himself down (a drunk, a bully, a dog), and the price is paid to the crowd.
+- G08 the one who holds your living: near `bow_for_your_living`, but the boss asks for nothing personal. He just will not wait, will not raise pay, and can replace you.
+- G12 going away for work, leaving people behind: near `far_from_home`, but the decision is whether to go at all.
+- G16 up against a powerful man with the rules on his side: near `at_their_mercy`, but it is your holding and your living at stake, not your life.
+- G18 looked down on for being poor: near `power_over_you` and `despised`, but the snob has no power over you, so the only stake is pride.
+- G19 money in your pocket and the drink on offer: near `habit` and `self_betrayal`, but the drink is a shared, social pay-day choice, not a private compulsion.
+- G31 passed from door to door: near `grease`, but no bribe would help. Everyone says it is not their business.
+
+**Thirteen are new situation types**, and most of them are about work:
+
+- **Work and its lack:** G09 no work to be had; G13 time to walk away; G11 a venture that keeps failing (the sunk-cost choice); G10 taking from those who will not miss it (pilfering and skiving, with the informer's flag).
+- **Debt:** G14 owing more than you can pay, with the creditor holding your roof; G15 slipping away from what you owe (the midnight flit).
+- **Mates and crews:** G04 keeping an old mateship alive at a distance; G06 the one the crew picks on (joining or stopping the joke on a workmate); G07 whose side are you on (workers against masters).
+- **Home:** G22 holding the place together alone (the partner left behind while the other is away earning); G23 a young one eager to do grown-up work; G25 trifling with someone who takes it seriously; G26 a quarrel neither side will end.
+
+**The brief's targets, one by one.** Mateship against self-interest: G01, G02, G05, G04. A boss or
+squatter who holds your living: G08, G16. Work that doesn't come: G09. A wife left alone on the
+selection: G22. Drink and pay-day: G19. Walking away from a job or a place: G13, with G12 for
+leaving to find work. Loyalty to a mate who lets you down: G02 and G03. **A newcomer tested by
+hardened hands was not found as its own recurring situation.** Lawson's newcomers (the man who
+forgot, a sick new chum, the boys at the railway camp) are sheltered or humoured, not tested.
+The hazing in this book falls on an insider the crew has already made its butt, which is G06.
+
+## How this corpus differs
+
+**It is the largest reference set so far**, 52 stories and 308 scenes, twice Bierce's story count.
+But many pieces are sketches, not stories. *Hungerford*, *In a Dry Season*, *In a Wet Season*,
+*Across the Straits*, *Coming Across* and *"Dossing Out" and "Camping"* are travel or social
+sketches with no plot, and their landscape and asides fit nothing. But the long single stories
+leave even more unassigned. *His Father's Mate* leaves 8 of 13 scenes: an abandoned goldfield, a
+father's memories of a brother and a son taken by the police. *Going Blind*, *The Drover's Wife* and
+*The Bush Undertaker* leave 6 each. Their middles are one-story material with no second instance.
+The recurring situations come mostly from the yarns (Mitchell, Steelman, the shearers' huts) and
+from the opening and closing turns of the full stories.
+
+**One world, one class.** Every earlier corpus ranged across classes; this one stays almost wholly
+with people who own nothing. So `money` does real work in the moves, which it barely did in Bierce,
+and nearly every group turns on a small sum: a shilling for beer, a week's board, the price of a
+steerage ticket. The recurring pressures are material ones, like the season's work, the creditor
+and the boss. Few are of the heart. That makes it the closest corpus yet to the Frontier setting,
+and its predicaments transplant well. "Your living depends on someone who will not wait" and
+"you could slip away before the one you owe can stop you" read as naturally in an office or a
+freight yard as in a shearing shed.
+
+**Recurring characters cross story boundaries.** Mitchell narrates, stars in or is named in about
+a dozen pieces, and Steelman and Smith in three. Macquarie, Stiffner, Arvie Aspinall and his
+mother, and Jimmy Nowlett each appear in three or four. A group spanning three stories is therefore sometimes one
+character's habit seen three times. G03, the sharp mate, draws on two Steelman stories, and G20, the
+con, on two. The grouping counts stories, not characters, as the recurrence rule does, but a tool
+that grouped by character would find some of these groups for the wrong reason.
+
+**Comic and bleak halves share situations.** G29 doing right by the dead holds the union's
+half-drunk funeral for a stranger, the child's burial at *Golden Gully*, a shepherd tipsily burying
+his old mate, and a street boy's condolence visit. G15 the flit holds the farce of the nail in the
+publican's boot and a widow's move by night past a policeman who looks the other way. A tool that
+grouped by mood would split both.
+
+## Known defects
+
+- Scene 120 repeats a whole paragraph of Mitchell's speech, and scene 190 repeats the narrator's
+  paragraph about his city rig. Both repeats are in the text as split. They are left alone, so that
+  `chunks.json` stays the corpus as split.
+- Scene 0's predicament contains "room" as an idiom ("makes room for a past"), which the
+  `topic_words` check counts as a place word.
+- Standing is almost uniform (see Stage above), so the stage field in `groups.json` carries no
+  information for this corpus.