Jesse.Markowitz/TheLadder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

story-to-pack: add the corpora and the catalogue v2/v3

Browse Source 
Fourteen corpora split and classified: aesop, bierce, chekhov, holmes,
keefe, lawson, lorimer, maupassant, nobody, plaintales, poe, torchy,
wallingford and winesburg, each with its splitter and the hand-written
groups and pages; catalogue v2 and v3; and the shared splitters
gutenberg_chunks.py, se_split.py and se_build.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rwKTmug58sEsJ72AuWsEi
This commit is contained in:
JesseMarkowitz
2026-10-07 06:05:51 -04:00
co-authored by Claude Opus 5.5
parent e5617b86ba
commit 3a4758172d
566 changed files with 303901 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/story-to-pack/probe/nobody/README.md
+175
View File
@@ -0,0 +1,175 @@
# Nobody reference set: *The Diary of a Nobody* (George and Weedon Grossmith, 1892)
A hand-made comparison baseline for the story-to-pack pipeline, a sibling of the Jacobs set in
`../jacobs/` and the Keefe set in `../keefe/`. Claude wrote every summary, predicament, group and
option by reading all 157 scenes in full. **No local or hosted inference was used.** Compare the
local-model runs against it. It is not ground truth, just one careful reading, by one reader, in
one pass.
| File | What it is |
| --- | --- |
| `corpus/diary-of-a-nobody.txt` | Project Gutenberg #1026 (public domain) |
| `split_gutenberg.py` | → `stories.json` (23 chapters, split on `CHAPTER <roman>`) and `chunks.json` (157 scenes, same 320-word paragraph rule as `../chunk.py`). Each chapter's opening synopsis is dropped from the text and kept as `synopsis`. |
| `claude/batches/sNN.json` | per chapter: `{chunk, story, summary, predicament}`, the same schema as `../jacobs/claude/batches/` |
| `claude/groups_src.py` | the groups, options and per-chapter protagonist standing |
| `claude/build.py` | checks everything, then writes `claude/records.json` and `claude/groups.json`. Identical to Jacobs' copy. |
| `claude/page.py` | → `claude/REFERENCE.html`, the readable page (Jacobs' copy with the title, heading, source line and unassigned note changed) |
Rebuild, from this directory: `python3 split_gutenberg.py && cd claude && python3 build.py && python3 page.py`.
## What is in it
- **157 predicaments**, one sentence each: median 22 words, range 17 to 24. They follow the
`redescribe.py` rules, so there are no names, jobs, places or story-only objects. The
`topic_words` leak check flags 117 of 157, and almost all of it is role words: son 60, father 32,
wife 26, employer 25, host 17, husband 15, guest 14. These are kept on purpose: the design's cast is
roles, and this book is about exactly these. Event and place words are few (party 8, house 6,
dinner 4). Two hits are false: `station` is social station, and `flat` is "falls flat".
- **19 groups**, and **all 19 pass the recurrence rule** below. Each scene sits in at most one group:
**142 are assigned and 15 are not**. All 23 chapters feed at least one group; chapter 17 (the
son taken into the father's firm) feeds only G16.
- **Options**: 3 to 5 per group, 93 in all (17 groups have 5, 2 have 4). 74 come from what someone
in a scene did (`scene:N`), 2 are archetypes and 17 fill gaps. **3 scene options cite a scene
outside their own group**, and `groups.json` marks each one `in_group: false`.
- **An unreliable narrator.** Pooter records every slight as the other man's disgrace and every
climb-down as magnanimity. The summaries say plainly what happened (he apologised for being
insulted; the joke fell flat; the tradesman snubbed him and he bought him champagne), and a
`scene:N` option is what he did, not what he says it amounted to. Every group also carries one move
a man with more sense of proportion would make, marked `filled` where nobody in the book makes it.
- **Moves**: spine paths with a direction only (`+`, `-`, `risk`, `set`), and no magnitudes.
`standing` moves 42 times, `money` 24, the two skills 40 (`composure` 23, `tact` 17). Relationship
roles are generic: `child` 22, `spouse` 15, `friends` 13, `employer` 10, and `juniors`, `tradesmen`,
`servants`, `guests`, `host`, `betters`, `acquaintance` and `family` a few times each. Flags record
what the player did: `put_foot_down`, `hid_the_bill`, `faked_it`, `proud_of_plainness`,
`vouched_for_son`, `spoke_against_match`, `lost_temper`, `cutting_letter`, `content`, `doubts`.
- **Skills** proposed for this corpus: **`composure`**, for keeping your temper and your dignity when
slighted and not needing the last word, and **`tact`**, for saying the right thing to the people above
and below you and knowing when to say nothing. Those are the two things Pooter lacks in nearly
every chapter. Keefe's `savvy` was considered and dropped: Pooter is rarely conned, he is
embarrassed. `discipline` was dropped too, because his lapses are of temper, not effort.
- **Stage** comes from protagonist standing, judged **per chapter, not per scene**, in `STANDING` in
`groups_src.py`. Chapters 1-13 are low: a clerk of twenty years below the senior clerks, cheeked by
the newest junior. Chapter 14 opens with his promotion to senior clerk and a £100 rise, so chapters
14-23 are middle. **No chapter ranks high**: the principal is never the protagonist. The groups come
out 11 low and 8 middle.
## The recurrence rule, and how it was applied
The book is one continuous diary, so its "stories" are 23 chapters of one life, and one incident
often runs across a chapter break: the Mansion House ball and its aftermath (chapters 4-5), the
Irving imitator (11), the collapsed shares (15-16), the lost client (21), the séances (22). A group
therefore counts as recurring only if it meets **all** of:
1. at least 5 scenes;
2. at least **3 chapters that are pairwise not consecutive**. Chapters 2, 5 and 18 qualify; 4, 5 and
6 do not, and neither do 14, 15 plus one more unless that one is at least two chapters from both;
3. no chapter supplies more than half the members.
`groups_src.py` was checked against this with a script that takes the largest set of pairwise
non-consecutive chapters in each group. Every group passes, with between 3 and 6 such chapters.
Two groups sit exactly at the 50% cap: G17 (4 of its 8 scenes from chapter 21) and G19 (4 of 8 from
chapter 11).
Two things the rule forced:
- **Some groups had to be merged to recur.** Being stood up by old friends (chapters 2, 14, 15) and
being reproached by an old friend for never asking after him (14, 18, 23) each fail alone, since 14
and 15 are consecutive and the second has only 3 scenes. Both are a falling-out with an old friend
where someone must make the first move, so they are one group, G04. Tradesmen and servants are
likewise one group, G03, because the decision is the same (how much deference to insist on).
- **Arcs are capped, and the rest stays out.** Of the eight lost-client scenes in chapter 21, four go
to G17, two to G14 and one to G11; 133, the father drafting his letter, is held out by G17's cap.
Of the nine séance scenes in chapter 22 (138-147, less the laundress in 142), three go to G13 and
one each to G05 and G06; 138, 141, 145 and 146 stay unassigned.
## Groups against the existing catalogue
The 82-entry catalogue is in `../catalogue/v2/catalogue_v2_src.py`. The mapping below follows its
rule: a group joins an entry when the player faces the same decision from the same side, and it is
new when the decision differs. It was judged by reading, not by `classify.py`. Keefe's proposed new
types are not yet in the catalogue; where a group matches one of them, the table says so. **7 groups
land on 6 existing entries, and 12 are new situation types.** This corpus is not yet in `SOURCES`.
| Group | Scenes / chapters | Catalogue |
| --- | --- | --- |
| G01 Cheeked by younger colleagues | 5 / 4 | **new** (`cheeked_by_juniors`); `power_over_you` turned upside down, and the junior-to-senior version of Keefe's G26 *needled by a colleague* |
| G02 A reward at work, and what it does at home | 7 / 3 | **new** (`rewarded_at_last`); nearest Keefe's G06 `raised_a_step`, but here nothing goes to his head |
| G03 Tradesmen and servants who show you no deference | 13 / 9 | **new** (`no_deference`); partly `mistook_them` (the ironmonger at the ball) |
| G04 A falling-out with an old friend | 11 / 6 | **new** (`falling_out`); nearest Keefe's G34 *leaning on the friend back home* |
| G05 Old friends who make free with your home | 8 / 6 | **new** (`taken_for_granted`); nearest `unwelcome_in_home` |
| G06 A joke nobody laughs at | 6 / 5 | **new** (`joke_falls_flat`); nearest Keefe's G31 *in the spotlight, sure you are the hit* |
| G07 A mishap at the very moment of dignity | 6 / 6 | **new** to the catalogue; matches Keefe's G24 *made a public laughing-stock*, except that the cause is your own clumsiness |
| G08 Keeping up appearances on a small income | 5 / 5 | `pretence` |
| G09 Playing the host beyond your means | 7 / 4 | **new** (`playing_host`); nearest `pretence` |
| G10 Out of your depth among your betters | 8 / 5 | **new** (`above_your_station`); nearest `station` and `outside_looking_in` |
| G11 Told your steady life is mediocre | 7 / 4 | **new** (`steady_life_scorned`); nearest `dead_end`, but he still believes in his life |
| G12 A tiff at home | 8 / 6 | `peace_at_home` |
| G13 Nobody at home takes you seriously | 7 / 4 | `home_unseen` |
| G14 A grown son who looks down on your life | 10 / 7 | `family_against_your_path`, from the parent's side ("a child who rejects your way") |
| G15 A grown son's unwise match | 6 / 4 | `family_against_your_path`, its own example ("a parent who despises your match") |
| G16 Placing a son who throws his places away | 7 / 4 | **new** (`placing_a_child`); nearest `patron`, from the patron's side |
| G17 Left to answer for your son | 8 / 5 | **new** (`answer_for_them`); partly `go_between` (the letter to the lost client) |
| G18 Your son's fast money | 5 / 4 | `sure_thing`, with your own son as the tipster |
| G19 Your son's friends take over your home | 8 / 4 | `unwelcome_in_home` |
**The eight situations the brief asked about, and where they went:**
- **Dignity slighted by juniors at work** is G01, and it is new. It is small (5 scenes) because
Pooter's office appears only in glimpses: the clerks' cheek, the thrown paper ball, "Hornpipe", the
leaking pen blamed on him.
- **A small rise or honour at the office, and what it does at home** is G02, new. The book's
version is double-edged: the Mansion House invitation is deflated by learning his employer arranged
it, and the £100 rise is outshone the same evening by his son's £200 windfall (91, which sits in G18).
- **Keeping up appearances on a clerk's income** splits in two: the cheap sham (G08, `pretence`) and
the real spending that runs short (G09, new).
- **A grown child whose schemes embarrass you** is the book's largest theme and splits five ways:
scorn (G14), the match (G15), the lost posts (G16), answering for him (G17) and his money (G18).
G19, his friends in the house, is a sixth. G14 and G15 share `family_against_your_path`, which covers
both directions of the parent-child quarrel.
- **Being snubbed at a social event above your station** is G10, new. `station` in the catalogue is
about courting across class, and `pretence` about faking a rank. Pooter fakes nothing: he goes as
himself and is out of his depth.
- **Feuds with tradesmen or servants over small respect** is G03, new, and the widest group (9
chapters).
- **Old friends who take you for granted** splits into G05 (liberties with your home, new) and G04
(who makes the first move after a quarrel, new).
- **A joke that nobody laughs at** is G06, new.
**What did not appear.** `power_over_you` is not in this corpus. The one superior, Mr Perkupp, is
unfailingly kind, and every slight comes from below or beside Pooter (juniors, tradesmen, friends, his
son). That is the sharpest difference from the Corporate Ladder pack's world: seen from the dinner
table, the office's danger is not the boss but the people who do not defer to you.
## How this corpus differs
- **It is the first clerk-in-an-office corpus told from inside the home.** Only 5 of 19 groups are
set at work (G01, G02, G16, G17 and part of G11), and even those are mostly decided at home: the rise
is celebrated there and the lost client is answered from there. The Corporate Ladder pack would
gain home-side events that change `standing` and `relationships.employer` without the player being
at the office.
- **It is comic, and the comedy is dignity.** Most predicaments turn on how much respect to insist
on, so `standing` moves more than anything else (42) and the two skills are social. `money` moves 24
times, because a clerk's purse sets what he can pretend to (G08, G09, G18).
- **A long novel-like diary still recurs.** DESIGN.md calls one long novel "the hard case". This one
passes because a diary repeats: the same friends, the same tradesmen and the same son come back
every few chapters.
- **Scenes straddle diary entries.** Many chunks join the end of one day to the start of the next.
The predicament follows the half that carries more weight. Scene 19 is the one deliberate exception:
its first half is the pun that offends his friends, its second the junior's thrown paper ball, and
the predicament follows the second half so that G01 can use it; the joke group keeps scene 18.
## Known defects
- **The wife's fashionable friend recurs but is not a group.** Mrs James's influence on Carrie (dress,
parasols, table decorations, manicure, Spiritualism) appears in chapters 3, 5, 7, 18 and 22, but in
every chapter except 22 it is the lighter half of a scene whose predicament is about something else.
Grouping it would mean rewriting those predicaments around a background thread.
- **Unassigned scenes (15)**: 0 (settling in), 20 (the red bath), 45 (the neighbours' mockery), 46
(the antimacassar), 79 and 83 (the anonymous card, one chapter), 80 (a stranger kisses his wife), 104
(the son's revenge on his rival), 117 (the hotel invitation), 127 (the celebrity sneers at the host;
it would put chapter 20 at half of G11), 133 (held out by G17's cap), and 138, 141, 145 and 146 (the
séances).
- **Two groups sit exactly at the 50% cap** (G17, G19). They pass the rule as written; a stricter cap
would need each widened by one scene from another chapter.
- No batch record was found to be wrong on a second read. Summaries use the book's names, as
Keefe's do. Predicaments use none.