# story-to-pack

Research toward a tool that builds a The Ladder content pack from a corpus of stories.

**Independent of the game.** Nothing in `src/`, `content/` or `test/` imports anything here, and
nothing here is part of `npm test` or `npm run build`. It lives on the `story-to-pack` branch so it is
versioned alongside the pack format it targets, without touching `main`.

- **What was decided and why:** [`DESIGN.md`](DESIGN.md)
- **How situations are selected, and every test of it:** [`SELECTION.md`](SELECTION.md)
- **The 2026-09-10 scripts, recovered from a transcript:** [`recovered/`](recovered/README.md)

## Status

Selection is settled as **gate G2 followed by a structured human review** (see below). Every automated
alternative tried — three more geometric gates and two model-judged gates — was worse on fresh,
blind-labelled data. Nothing downstream of selection (paths and directions, numbers, pack output) is
built yet.

## Layout

```
probe/
  corpus/                O. Henry's Four Million cycle, Project Gutenberg (public domain)
  split.py, chunk.py     volumes -> stories.json -> chunks.json (838 scenes)
  ollama.py              the only module that talks to an inference host (host from STP_OLLAMA)
  summarise.py           scene -> one sentence, via the model          -> summaries.json
  strip_names.py         names removed deterministically               -> summaries_clean.json
  embed.py               texts -> vectors                              -> embeddings.json, summary_embeddings.json
  measure.py, signals.py, gate2.py, gate3.py, judge.py …   the selection experiments
  labels-*.json, sheet-*.txt, candidates-*.json, *-map.json   blind labels and what they labelled
  review.py              G2 on a partition -> a review package for a human
  review_format.py       the review form: schema, checks, conversion
  apply_review.py        filled review.json -> situations.json
  test_review.py         tests for the review form
  review/<run>/          review packages: REVIEW.md, review.json, candidates.json, situations.json
```

Not committed (see `.gitignore`): the inference host's power and kernel logs, and caches that the
scripts rebuild on first use (`sim.pkl`, `null*.pkl`, `coassoc.json`). The embeddings are committed
(about 16 MB) because recreating them needs the inference host.

## Running

Everything runs from `probe/` with Python 3 and no third-party packages.

Anything that calls a model reads the host from the environment and never from a file:

```sh
export STP_OLLAMA=http://<inference-host>:11434
```

## Reviewing situations

```sh
cd probe
python3 review.py                      # k = 60, seed 24  ->  review/k60-s24/
```

That writes three files into the run directory:

- **`REVIEW.md`** — what the program found, and exactly what the reviewer is asked to decide for
  each candidate, with every scene listed. Read this.
- **`review.json`** — the form. The only file the reviewer edits.
- **`candidates.json`** — what the program found, for the checker. Do not edit.

When the form is filled in:

```sh
python3 apply_review.py review/k60-s24
```

It reports every problem at once and writes nothing until the form is complete and consistent; then
it writes **`situations.json`**, the machine-readable input for the next stage: each situation's
wording, roles and stakes, the scenes that show it (with story and summary), and which candidates it
came from.

```sh
python3 -m unittest test_review        # the review form's checks
```