Files
TheLadder/tools/story-to-pack/probe/winesburg/README.md
T
JesseMarkowitzandClaude Opus 5.5 3a4758172d story-to-pack: add the corpora and the catalogue v2/v3
Fourteen corpora split and classified: aesop, bierce, chekhov, holmes,
keefe, lawson, lorimer, maupassant, nobody, plaintales, poe, torchy,
wallingford and winesburg, each with its splitter and the hand-written
groups and pages; catalogue v2 and v3; and the shared splitters
gutenberg_chunks.py, se_split.py and se_build.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rwKTmug58sEsJ72AuWsEi
2026-10-07 06:05:51 -04:00

151 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Winesburg reference set: *Winesburg, Ohio* (1919)
A hand-made comparison baseline for the story-to-pack pipeline, a sibling of the Jacobs set in
`../jacobs/` and the Bierce set in `../bierce/`. Claude read every scene and wrote each summary,
predicament, group and option. **No local inference was used.** Compare the local-model runs against it.
Do not treat it as ground truth: it is one careful reading.
| File | What it is |
| --- | --- |
| `corpus/winesburg-ohio.html` | Standard Ebooks single-page text of Sherwood Anderson's *Winesburg, Ohio* (public domain) |
| `../se_build.py` | → `stories.json` (24 stories, split on the `se:short-story` markup) and `chunks.json` (284 scenes, same 320-word paragraph rule as `../chunk.py`) |
| `claude/batches/sNN.json` | per story: `{chunk, story, summary, predicament}`, the same schema as `../jacobs/claude/batches/` |
| `claude/groups_src.py` | the groups, options and per-story protagonist standing |
| `claude/build.py` | checks everything, then writes `claude/records.json` and `claude/groups.json` (unchanged from Jacobs: it resolves paths relative to itself) |
| `claude/page.py` | → `claude/REFERENCE.html`, the readable page |
Rebuild: `python3 ../se_build.py . "Winesburg, Ohio (Sherwood Anderson, 1919)" && cd claude && python3 build.py && python3 page.py`
(run from this directory). The split was made once and is not redone here.
## What is in it
- **284 predicaments**: one sentence each (median 21 words, min 16, max 24), written to the
`redescribe.py` rules: no names, jobs, places or story-only objects. The `topic_words` leak check
flags 48 of 284. 47 are role or kin words (son, mother, wife, father, newcomer, suitor, admirer,
rival), kept on purpose because the design's cast is roles. The other is "court" used as a verb.
A second pass took "town" and "city" out of 26 predicaments, and "room" or "house" out of 3 more,
where they tied a situation to the setting. Each should now read as well in an office or a
freight yard ("everyone around them", "somewhere bigger").
- **19 groups**, each a situation that recurs in **three or more stories** (3 span 4, 5 span 5,
2 span 6, 5 span 7, 2 span 8, 2 span 9). Every group has at least 6 scenes, and no story supplies more than
half of any group's members (the largest share is 43%). **All 19 pass the catalogue's recurrence rule**
(≥ 5 scenes, ≥ 3 stories, no story over 50%). Each scene sits in at most one group: **163 are
assigned and 121 fit nothing that recurs**. All 24 stories feed at least one group.
- **Options**: 3 to 5 per group, 93 in all (17 groups have 5, 2 have 4). 81 come from what
someone in a scene did or refused to do (`scene:N`), 6 are archetypes and 6 fill gaps. **27
scene options cite a scene outside their own group.** That is looser than DESIGN.md's "drawn from
other scenes in the cluster", and `groups.json` marks each one `in_group: false`. 17 of the 27
are the scene just before or after a member, where the character acts on the predicament the
member scene sets up.
- **Moves**: spine paths with a direction only (`+`, `-`, `risk`, `set`). There are no magnitudes,
as the design requires. Relationship paths are moved 78 times, `standing` 38, flags 26, the two
skills 17 each and `money` only 12. Relationship roles are generic (`family`, `child`, `spouse`,
`beloved`, `confidant`, `elder`, `community`, `absent`, `rival`, `peers`, `close`, `friend`).
Flags record what the player did (`left_home`, `confessed`, `kept_silent`, `fled_closeness`,
`secret_savings`, `false_front`, `broke_out`).
- **Skills** proposed for this corpus: `voice`, for daring to say what you mean, and `insight`, for
reading what people feel under what they say. Neither Jacobs' `cunning` nor Bierce's `nerve` fits:
almost nobody here schemes or faces danger, and nearly every predicament turns on speaking or
understanding. A pack from this corpus would test whether the numeric search can make two
communication skills carry what money carries elsewhere.
- **Stage** comes from protagonist standing, judged **per story, not per scene**, in `STANDING` in
`groups_src.py`. A day labourer, a shop clerk, a demoted operator, a neglected child, a recluse,
a failing shopkeeper's son, a farm hand and a stable boy rank low. The young reporter, his mother,
the town's odd professionals and a teacher rank middle. The valley's great landowner, his daughter
and grandson, and the best-paid pastor rank high. The groups come out 11 middle, 5 low and 3 high.
## Against the existing catalogue (82 entries, v2)
**Eight groups match an existing entry.** Seven of them widen or sharpen it rather than duplicate it:
- **G09 the thing you cannot get out → `unspoken`**, widened. The entry is shy lovers. Here it is
also a mother and son who sit together every evening and say nothing real, a boy who cannot tell
his friends what his painting means, a young man who calls someone out to confront him and freezes,
and a mother dying paralysed while trying to speak of her son's future. The entry's definition
("someone you want is waiting for you to say it") could drop "want": the listener need not be a lover.
- **G11 on the edge of everyone else's life → `outside_looking_in`**, sharpened. The entry is a lit
window seen from outside. Winesburg puts the outsider in the middle of the crowd, at the county
fair, among arguing men or among talking artists, and still outside it.
- **G12 waiting for someone who may never come → `waiting`**, widened: waiting on someone who has
already forgotten you, and the moment of saying aloud that it is not coming.
- **G13 alone so long your habits have become your company → `isolation`**, widened. The entry is
lost or posted somewhere empty. Here the solitude is chosen, in the middle of town, and the danger
is that invented company and small rituals replace people.
- **G16 a desire at war with who you mean to be → `self_betrayal`**, sharpened: a respectable person
returning to a temptation while telling himself it is a test.
- **G17 someone else wants the one you want → `rival`**, unchanged.
- **G18 acting bigger than you feel → `pretence`**, widened from pretending to a station to young
bravado: boasting of future greatness, talking worldly among peers, playing the solid citizen.
- **G19 unseen at home → `home_unseen`**, widened from spouses to parent and child: a son whose mother
favours his cruel brother, a child whose father is busy arguing against God.
**Eleven groups are new situation types** (the brief's seven targets are marked ★):
- ★ **G01 whether to leave for somewhere bigger.** `newcomer` is arriving and `homecoming` is word
from home. Neither is the decision to go. This is the book's spine: it recurs in 8 stories and ends
the book.
- ★ **G02 telling it all to the one who will listen.** A lonely person finally tells a life story
to a single listener, usually the young reporter. It is not `owning_up` (no wrong is confessed to
the wronged) and not `grief` (the entry is a loss nobody will hear; here someone does).
- **G03 an elder presses their hard lesson on you.** Seen from the young person's side: take it,
humour it or refuse it. It recurs in 8 stories and is the other half of G02.
- ★ **G04 one idea has taken over your life.** This is Anderson's "grotesque" from the prologue: a
man who seized one truth and became warped by it.
- ★ **G05 your child's chances are the life you never had.** Closest is `family_against_your_path`,
but there the parent opposes the child's life. Here the parent is living through it.
- ★ **G06 the one everyone watches.** Being the town's odd one. `despised` is the nearest entry, but
its decision is whether to help when given a chance. This one is how to live under the stares.
- ★ **G07 what they say about you is not who you are.** A public reputation against a private life.
It sits between `wrongly_judged` and `pretence` and matches neither: the story people tell may be
admiring as well as damning, and the player did not invent it.
- ★ **G08 reaching out, then pulling back.** A moment of connection that fails. It differs from
`unspoken`: the person does reach out, and then recoils, turns harsh or drives the other away.
- **G10 understood without a word**: the rare moment of connection that holds, the success case of G08.
- **G14 the life you settled into**: a marriage or life taken because it was at hand, felt years later
as a harness. Nearest are `dead_end` (thankless work, the temptation to stop wanting) and
`home_unseen`. The decision here is to break out, stay, or keep a secret way out.
- **G15 the urge to break out, now**: a sudden impulse to run out into the night or the storm. It
recurs in 7 stories, one scene each.
## How this corpus differs from Jacobs and Bierce
**It is the design's target corpus: one town, 24 stories, many lives.** The same streets, the hotel,
the newspaper office and the fairground recur in every story, and the young reporter passes through
most of them. That shows in recurrence: 9 of 19 groups span 7 or more stories, against a maximum of
6 in Bierce. But it also means that "which story" and "which situation" are harder to separate than
in a mixed collection: scene text shares names, places and a narrator voice across all 24 stories. A
tool that clusters on scene prose will pull the town together, not the situations.
**The register is inward and the stakes are small.** Nobody is in danger and almost nobody schemes.
The predicaments are about saying, hearing, leaving and being seen. That is why `money` moves only 12
times and relationships carry the moves. It is also why the brief's targets all exist here, several
of them in 7 to 9 stories.
**The listener problem.** A great many scenes are one person talking while the reporter listens. In
each such scene the decision belongs to the talker, not the listener. G02 and G03 split this by side:
G02 is the talker's predicament and G03 the listener's. Scenes where the reporter only witnesses
(the Kings coming to threaten a suitor, the pastor bursting in with a bleeding fist) are unassigned.
**The prologue.** *The Book of the Grotesque* (story 0, 4 scenes) grouped as expected for a framing
piece: only its statement of the theory, scene 3, fits a recurring situation (G04, one idea has taken
over your life). Its other three scenes are an old writer's reverie and stay unassigned. The prologue
names the book's theme and supplies one example of it, not a situation of its own.
**Godliness is one tale in four parts.** Stories 6, 7 and 8 (*Godliness* I–II, *Surrender*, *Terror*)
are one family's story. Every group still spans three or more stories if those three count as one;
G05 falls from 5 stories to 4 and G04, G19 from 7 and 6 to 6 and 5. The long historical narration of
the tale (scenes 47–50, 57) and the grandfather's two woodland rites (73–77, 93–98) are one-story
material and stay unassigned.
**Why 121 scenes are unassigned.** Besides the above, most are scene-setting narration (the spring
woods, the empty fairground at night, the snowbound town), or single-story episodes with no second
instance. Examples are the schoolmaster's false accusation and flight in *Hands*, the Kings'
visit, the wife pushed naked into the parlour in *Respectability*, and the travelling salesman
driven off at gunpoint in *"Queer"*.
## Known defects
- The Standard Ebooks part headings stay inline in scene text: scene 47 begins "A Tale in Four Parts",
scene 78 "Part III" and scene 91 "Part IV". They are not removed, so that `chunks.json` stays the
corpus as split.