Files
TheLadder/tools/story-to-pack/probe/review_page.html
T
JesseMarkowitzandClaude Opus 5 e5617b86ba Replace clustering with a catalogue, and hand-write the references to judge it against
The pipeline's embed-and-cluster step is dead, and this commit holds both the
evidence for that and the step proposed to replace it.

Predicaments. Scenes are re-described as "what the person is up against", with
no names, jobs or places, then embedded and clustered (redescribe.py,
topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading
both side by side. Two defects the pilot exposed are fixed: split.py missed
titles in quotes and a contents subtitle after a dash, so three stories had been
merged into their neighbours, and strip_names.py read New York place names as
people. The corrected corpus is probe/v2 (97 stories, 839 scenes);
carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from
20% to 13% at k=60, short of the pre-registered 10%.

Hand references. Three corpora were read scene by scene and written up by hand,
under the same prompt rules the local models get, as a baseline to judge them
against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's
Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of
the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a
readable page. No inference was used for any of them.

Catalogue. probe/catalogue maps every hand group in the three references onto 36
situation entries, with an answer key per corpus and one recurrence rule applied
to all three. classify.py assigns a scene one entry or none, leave-one-corpus-
out; score.py checks it against the key, with a self-test on random labels.

Why clustering is out: hand-written predicaments, embedded and clustered exactly
as the model's were, agree with the hand grouping at ARI 0.05 — no better than
the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even
shortlist: the hand label is the nearest entry 13% of the time and in the top 8
half the time.

The classification runs are not here. The dev and test runs are pre-registered
in probe/catalogue/README.md with the bar set beforehand, and are blocked on the
inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene
partial output in out/ is not a result.

Review page. The situation review is now a browser page rather than JSON edited
by hand (review_page.py, review_page_logic.cjs with Node tests, format schema
v2). It has never been rendered in a real browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
2026-09-20 17:19:42 -04:00

443 lines
22 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Situation review · __RUN_ID__</title>
<style>
:root {
--bg: #f7f6f2; --card: #ffffff; --ink: #1d1d1b; --muted: #66655f; --line: #dedcd4;
--accent: #2f5d8a; --accent-ink: #ffffff; --good: #2e7d4f; --warn: #9a5a00; --bad: #b3261e;
--chip: #e9eef5; --struck: #9a9890;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #161614; --card: #211f1c; --ink: #ecebe6; --muted: #a6a49c; --line: #3a3934;
--accent: #86b4e2; --accent-ink: #0d1b28; --good: #74cf99; --warn: #f0b64d; --bad: #f28b82;
--chip: #2a3440; --struck: #77756e;
}
}
* { box-sizing: border-box; }
body { margin: 0; background: var(--bg); color: var(--ink);
font: 16px/1.55 system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; }
header.top { position: sticky; top: 0; z-index: 2; background: var(--card);
border-bottom: 1px solid var(--line); padding: 10px 16px; }
.wrap { max-width: 900px; margin: 0 auto; padding: 16px; }
.row { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; }
.spacer { flex: 1; }
h1 { font-size: 1.05rem; margin: 0; }
h2 { font-size: 1.35rem; margin: 0; }
.muted { color: var(--muted); }
.pills { display: flex; flex-wrap: wrap; gap: 4px; margin-top: 8px; align-items: center; }
.pill { border: 1px solid var(--line); background: var(--card); color: var(--ink); border-radius: 999px;
padding: 1px 10px; font: inherit; font-size: .85rem; cursor: pointer; min-width: 2.2em; }
.pill.current { outline: 2px solid var(--accent); outline-offset: 1px; }
.pill.done { background: var(--good); border-color: var(--good); color: var(--card); }
.pill.problem { background: var(--warn); border-color: var(--warn); color: var(--card); }
.pill.skipped { color: var(--muted); }
.card { background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: 16px 18px; margin: 14px 0; }
.btn { font: inherit; border: 1px solid var(--line); background: var(--card); color: var(--ink);
border-radius: 8px; padding: 7px 14px; cursor: pointer; }
.btn.primary { background: var(--accent); border-color: var(--accent); color: var(--accent-ink); }
.btn.on { background: var(--accent); border-color: var(--accent); color: var(--accent-ink); }
.btn:disabled { opacity: .45; cursor: not-allowed; }
.q { font-weight: 650; margin: 0 0 10px; }
.tip { font-size: .92rem; color: var(--muted); margin: 8px 0 0; }
.status { font-weight: 600; }
.status.problem { color: var(--bad); }
.status.done { color: var(--good); }
ul.scenes { list-style: none; margin: 0; padding: 0; border-top: 1px solid var(--line); }
ul.scenes li { display: flex; gap: 12px; padding: 8px 2px; border-bottom: 1px solid var(--line); }
ul.scenes li.out .text { text-decoration: line-through; color: var(--struck); }
ul.scenes input { width: 20px; height: 20px; margin-top: 3px; flex: none; cursor: pointer; }
.story { display: block; font-size: .8rem; color: var(--muted); }
.shape { margin: 10px 0 0; font-weight: 600; }
.shape.ok { color: var(--good); }
.shape.bad { color: var(--bad); }
.name { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; font-size: 1.05rem; }
.name input { font: inherit; padding: 6px 8px; border: 1px solid var(--line); border-radius: 6px;
background: var(--bg); color: var(--ink); width: 15em; max-width: 100%; }
.chips { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 10px; align-items: center; }
.chip { font: inherit; border: none; background: var(--chip); color: var(--ink); border-radius: 6px;
padding: 2px 9px; cursor: pointer; font-size: .9rem; }
.preview { margin: 12px 0 0; font-size: 1.05rem; }
select { font: inherit; padding: 6px; border-radius: 6px; border: 1px solid var(--line);
background: var(--bg); color: var(--ink); max-width: 100%; }
.example { border-left: 3px solid var(--line); padding: 2px 14px; margin: 12px 0; }
.example ul { margin: 4px 0; }
table.summary { width: 100%; border-collapse: collapse; }
table.summary td { border-bottom: 1px solid var(--line); padding: 6px 4px; vertical-align: top; }
pre { background: var(--bg); border: 1px solid var(--line); border-radius: 6px; padding: 8px 10px; overflow-x: auto; }
.saved { font-size: .85rem; color: var(--muted); }
kbd { border: 1px solid var(--line); border-radius: 4px; padding: 0 5px; font-size: .85rem; }
</style>
</head>
<body>
<header class="top">
<div class="row" id="topbar"></div>
<div id="pills"></div>
</header>
<main class="wrap" id="app"></main>
<noscript><p class="wrap">This review page needs JavaScript turned on.</p></noscript>
<script>
/*__LOGIC__*/
</script>
<script>
const DOC = /*__DATA__*/null;
(() => {
const L = ReviewLogic;
const run = DOC.run;
const order = L.orderedIds(DOC);
const KEY = `stp-review:${run.id}`;
let saveNote = '';
let lastRoleField = null;
let state = load() || { entries: L.initialEntries(DOC), view: 'intro', current: 0 };
// ---- saving -------------------------------------------------------------------------
function load() {
try {
const raw = localStorage.getItem(KEY);
if (!raw) return null;
const saved = JSON.parse(raw);
const restored = L.fromReviewFile(DOC, saved.file);
if (restored.error) return null;
saveNote = 'Picked up where you left off.';
const view = ['intro', 'candidate', 'finish'].includes(saved.view) ? saved.view : 'candidate';
const current = Math.min(Math.max(0, Number(saved.current) || 0), order.length - 1);
return { entries: restored.entries, view, current };
} catch (_) {
return null;
}
}
function persist() {
try {
const file = L.toReviewFile(DOC, state.entries, new Date().toISOString());
localStorage.setItem(KEY, JSON.stringify({ file, view: state.view, current: state.current }));
saveNote = 'Progress saved in this browser.';
} catch (_) {
saveNote = 'This browser is not keeping progress: use “Save review file” before closing.';
}
}
function update(change, { top = false } = {}) {
change();
persist();
render();
if (top) window.scrollTo(0, 0);
}
function download() {
const file = L.toReviewFile(DOC, state.entries, new Date().toISOString());
const blob = new Blob([JSON.stringify(file, null, 1)], { type: 'application/json' });
const link = document.createElement('a');
link.href = URL.createObjectURL(blob);
link.download = `review-${run.id}.json`;
document.body.appendChild(link);
link.click();
link.remove();
setTimeout(() => URL.revokeObjectURL(link.href), 2000);
saveNote = `Saved review-${run.id}.json.`;
renderTop();
}
function upload(event) {
const chosen = event.target.files && event.target.files[0];
if (!chosen) return;
const reader = new FileReader();
reader.onload = () => {
let data;
try { data = JSON.parse(reader.result); } catch (_) { alert('That file is not a review file.'); return; }
const restored = L.fromReviewFile(DOC, data);
if (restored.error) { alert(restored.error); return; }
if (!confirm('Replace what is on this page with the review in that file?')) return;
update(() => { state.entries = restored.entries; state.view = 'finish'; });
};
reader.readAsText(chosen);
}
// ---- building the page ----------------------------------------------------------------
function el(tag, attrs, ...children) {
const node = document.createElement(tag);
for (const [key, value] of Object.entries(attrs || {})) {
if (value === null || value === undefined || value === false) continue;
if (key === 'class') node.className = value;
else if (key.startsWith('on')) node.addEventListener(key.slice(2), value);
else if (key === 'value') node.value = value;
else if (key === 'checked') node.checked = !!value;
else node.setAttribute(key, value === true ? '' : value);
}
for (const child of children.flat()) {
if (child === null || child === undefined || child === false) continue;
node.append(child instanceof Node ? child : document.createTextNode(String(child)));
}
return node;
}
const entryOf = (id) => state.entries.find((e) => e.id === id);
const go = (index) => update(() => { state.view = 'candidate'; state.current = index; }, { top: true });
function render() {
renderTop();
const app = document.getElementById('app');
if (state.view === 'intro') app.replaceChildren(intro());
else if (state.view === 'finish') app.replaceChildren(finish());
else app.replaceChildren(candidateView(order[state.current]));
}
function renderTop() {
const p = L.progress(DOC, state.entries);
document.getElementById('topbar').replaceChildren(
el('h1', {}, `Situation review · ${run.id}`),
el('span', { class: 'muted' },
`${p.decided} of ${p.total} groups decided${p.problems ? ` · ${p.problems} need attention` : ''}`),
el('span', { class: 'spacer' }),
el('span', { class: 'saved' }, saveNote),
el('button', { class: 'btn', onclick: () => update(() => { state.view = 'intro'; }, { top: true }) }, 'How this works'),
el('button', { class: 'btn primary', onclick: () => update(() => { state.view = 'finish'; }, { top: true }) }, 'Finish & save'),
);
const pill = (id) => {
const index = order.indexOf(id);
const s = L.status(DOC, state.entries, id);
const current = state.view === 'candidate' && state.current === index;
return el('button', {
class: `pill ${s.state}${current ? ' current' : ''}`,
title: `${L.displayName(DOC, id)}: ${s.message}`,
onclick: () => go(index),
}, L.shortName(DOC, id));
};
const accepted = order.filter((id) => DOC.candidates[id].kind === 'accepted');
const extras = order.filter((id) => DOC.candidates[id].kind === 'borderline');
document.getElementById('pills').replaceChildren(el('div', { class: 'pills' },
el('span', { class: 'muted' }, 'Groups:'),
accepted.map(pill),
extras.length ? el('span', { class: 'muted' }, ' Optional extras:') : null,
extras.map(pill),
));
}
function example(verdict, lines, note) {
return el('div', { class: 'example' },
el('strong', {}, verdict),
el('ul', {}, lines.map((line) => el('li', {}, line))),
el('div', { class: 'muted' }, note));
}
function intro() {
const p = L.progress(DOC, state.entries);
return el('div', {},
el('div', { class: 'card' },
el('h2', {}, 'What you are doing'),
el('p', {},
`The program read ${run.scenes} scenes from ${run.stories} stories, summarised each in a sentence, ` +
`and grouped scenes that seem alike. ${p.total} groups might each be one recurring situation: ` +
'something that could come up again and again in the game, with different people. ' +
'Your job is to check each group.'),
el('p', {}, 'Each group asks you three things, one at a time:'),
el('ol', {},
el('li', {}, el('strong', {}, 'Is it one situation? '), 'Keep it, drop it, or mark it the same as a group you already kept.'),
el('li', {}, el('strong', {}, 'Which scenes don’t fit? '), 'Untick them. Every scene stays in unless you untick it.'),
el('li', {}, el('strong', {}, 'What is it? '), 'Fill in one short sentence: “A ___ wants ___ from ___.”')),
el('p', {},
'Expect to drop roughly a quarter of the groups. Allow about a minute each. ' +
'Everything saves in this browser as you go, so you can stop and come back. ' +
'At the end, “Finish & save” saves a review file for the next step.'),
),
el('div', { class: 'card' },
el('h2', {}, 'What counts as a situation'),
el('p', {}, 'One person wanting something specific from another person — the kind of moment that could come round again with different people in it.'),
example('Keep', [
'A lodger begs the landlady for one more week.',
'A tenant hides on the stairs on rent day.',
'A landlady tells a family they are out on Monday.',
], 'Name it: “A lodger wants more time to pay from a landlady.”'),
example('Drop: a shared place, not a situation', [
'Two diners argue about politics.',
'A waiter carries six plates at once.',
'A cook burns the soup.',
], 'They all happen in a restaurant, but nobody wants the same kind of thing from anybody.'),
example('Drop: too vague to name', [
'Someone is tense with someone.',
'A conflict over a personal matter.',
'Power shifts between two people.',
], 'If the best you can say is “conflict” or “people want things”, drop it.'),
example('Same as another group', [
'Group 2 is suitors pressing women to marry them.',
'Group 7 is a wealthy man proposing and being turned down.',
], 'Keep Group 2. On Group 7, choose “Same situation as… Group 2”. Its scenes join Group 2, and it needs no name of its own.'),
el('p', { class: 'tip' }, 'These examples are made up, to show the idea.'),
),
el('div', { class: 'row' },
el('button', { class: 'btn primary', onclick: () => update(() => { state.view = 'candidate'; }, { top: true }) },
p.decided ? 'Carry on reviewing' : 'Start with Group 1'),
el('span', { class: 'muted' }, 'Shortcuts while reviewing: ', el('kbd', {}, 'K'), ' keep · ', el('kbd', {}, 'D'), ' drop · ',
el('kbd', {}, '→'), ' next · ', el('kbd', {}, '←'), ' back')),
);
}
function roleField(entry, key, placeholder) {
return el('input', {
value: entry[key], placeholder, maxlength: DOC.rules.maxFieldChars, 'aria-label': placeholder,
onfocus: () => { if (key !== 'wants') lastRoleField = key; },
oninput: (event) => {
entry[key] = event.target.value;
persist();
renderTop();
const preview = document.getElementById('preview');
if (preview) preview.textContent = L.sentence(entry);
},
});
}
function fillRole(entry, role) {
const target = lastRoleField && !entry[lastRoleField].trim()
? lastRoleField
: (!entry.actor.trim() ? 'actor' : (!entry.counterpart.trim() ? 'counterpart' : (lastRoleField || 'actor')));
update(() => {
entry[target] = target === 'counterpart' ? `a ${role}` : role;
lastRoleField = target === 'actor' ? 'counterpart' : 'actor';
});
}
function candidateView(id) {
const c = DOC.candidates[id];
const e = entryOf(id);
const index = order.indexOf(id);
const s = L.status(DOC, state.entries, id);
const set = (patch) => update(() => Object.assign(e, patch));
const inGroup = e.decision === 'keep' || e.decision === 'same';
const targets = L.keptTargets(DOC, state.entries, id);
const excluded = new Set(e.excluded);
const shape = L.shapeCheck(L.groupMembers(DOC, state.entries, e.decision === 'same' && e.same_as ? e.same_as : id), DOC.rules);
const decisions = el('div', { class: 'row' },
el('button', { class: `btn${e.decision === 'keep' ? ' on' : ''}`, onclick: () => set({ decision: 'keep', same_as: null }) },
'Yes, keep it'),
c.kind === 'accepted'
? el('button', { class: `btn${e.decision === 'drop' ? ' on' : ''}`, onclick: () => set({ decision: 'drop', same_as: null }) }, 'No, drop it')
: el('button', { class: `btn${e.decision === 'skip' ? ' on' : ''}`, onclick: () => set({ decision: 'skip', same_as: null }) }, 'Leave it out'),
el('span', { class: 'muted' }, 'or'),
el('select', {
disabled: targets.length === 0 && e.decision !== 'same',
onchange: (event) => set(event.target.value
? { decision: 'same', same_as: event.target.value }
: { decision: c.kind === 'accepted' ? null : 'skip', same_as: null }),
},
el('option', { value: '' }, targets.length ? 'Same situation as…' : 'Same situation as… (keep a group first)'),
targets.map((t) => el('option', { value: t, selected: e.decision === 'same' && e.same_as === t },
`${L.displayName(DOC, t)}: ${L.sentence(entryOf(t))}`))),
);
const scenes = el('ul', { class: 'scenes' }, c.members.map((m) => {
const out = excluded.has(m.member);
return el('li', { class: out ? 'out' : '' },
el('input', {
type: 'checkbox', checked: !out, disabled: !inGroup, 'aria-label': 'This scene fits',
onchange: (event) => set({
excluded: event.target.checked ? e.excluded.filter((x) => x !== m.member) : [...e.excluded, m.member],
}),
}),
el('div', { class: 'text' }, m.summary, el('span', { class: 'story' }, m.title)));
}));
let third = null;
if (e.decision === 'keep') {
third = el('div', { class: 'card' },
el('p', { class: 'q' }, '3. What is it?'),
el('div', { class: 'name' },
'A', roleField(e, 'actor', 'who wants something'),
'wants', roleField(e, 'wants', 'what they want'),
'from', roleField(e, 'counterpart', 'whom'), '.'),
c.role_hints && c.role_hints.length
? el('div', { class: 'chips' }, el('span', { class: 'muted' }, 'People in these scenes (click to use):'),
c.role_hints.map((role) => el('button', { class: 'chip', onclick: () => fillRole(e, role) }, role)))
: null,
el('p', { class: 'preview', id: 'preview' }, L.sentence(e)),
el('p', { class: 'tip' },
'Use roles, not names: “a lodger”, “her husband”, “a wealthy suitor”. ' +
'If you can’t say what they want, that usually means the group should be dropped.'));
} else if (e.decision === 'same' && e.same_as) {
third = el('div', { class: 'card' },
el('p', {}, `Merged into ${L.displayName(DOC, e.same_as)}: “${L.sentence(entryOf(e.same_as))}”`),
el('p', { class: 'tip' }, 'Its ticked scenes join that group, so it needs no name of its own.'));
}
return el('div', {},
el('div', { class: 'row' },
el('h2', {}, L.displayName(DOC, id)),
el('span', { class: 'muted' }, `${c.members.length} scenes from ${c.stats.stories} stories`),
el('span', { class: 'spacer' }),
el('span', { class: `status ${s.state}` }, s.message)),
c.kind === 'borderline'
? el('p', { class: 'tip' }, 'Optional extra. The program nearly left this group out, so leave it out unless it clearly shows one situation.')
: null,
el('div', { class: 'card' },
el('p', { class: 'q' }, '1. Do most of these scenes show one situation: one person wanting something specific from another?'),
decisions,
el('p', { class: 'tip' }, 'Read the scenes below first. A shared place, job, mood or word is not enough.')),
el('div', { class: 'card' },
el('p', { class: 'q' }, inGroup ? '2. Untick any scene that doesn’t show it.' : 'The scenes'),
scenes,
inGroup ? el('p', { class: `shape ${shape.ok ? 'ok' : 'bad'}` }, shape.text) : null),
third,
el('div', { class: 'row' },
el('button', { class: 'btn', disabled: index === 0, onclick: () => go(index - 1) }, '← Back'),
el('span', { class: 'spacer' }),
index < order.length - 1
? el('button', { class: 'btn primary', onclick: () => go(index + 1) }, 'Next →')
: el('button', { class: 'btn primary', onclick: () => update(() => { state.view = 'finish'; }, { top: true }) }, 'Finish & save')),
);
}
function finish() {
const p = L.progress(DOC, state.entries);
const ready = p.decided === p.total && p.problems === 0;
const rows = order.map((id) => {
const s = L.status(DOC, state.entries, id);
return el('tr', {},
el('td', {}, el('button', { class: `pill ${s.state}`, onclick: () => go(order.indexOf(id)) }, L.displayName(DOC, id))),
el('td', {}, s.message));
});
return el('div', {},
el('div', { class: 'card' },
el('h2', {}, ready ? 'Ready to save' : 'Not finished yet'),
el('p', {}, ready
? 'Every group is decided and nothing needs attention.'
: `${p.total - p.decided} groups not decided${p.problems ? `, ${p.problems} need attention` : ''}. ` +
'You can still save now: the file keeps your progress, and the next step lists what is missing.'),
el('div', { class: 'row' },
el('button', { class: 'btn primary', onclick: download }, 'Save review file'),
el('label', { class: 'btn' }, 'Load a review file',
el('input', { type: 'file', accept: '.json,application/json', style: 'display:none', onchange: upload }))),
el('p', { class: 'tip' },
`The browser saves review-${run.id}.json, usually into Downloads. Then, in the story-to-pack probe folder, run:`),
el('pre', {}, `python3 apply_review.py review/${run.id}`),
el('p', { class: 'tip' }, 'It finds the file in Downloads by itself, checks it, and writes situations.json.')),
el('div', { class: 'card' }, el('table', { class: 'summary' }, rows)),
);
}
document.addEventListener('keydown', (event) => {
if (state.view !== 'candidate' || event.ctrlKey || event.metaKey || event.altKey) return;
// Never while typing a name or choosing from the list.
const tag = (event.target.tagName || '').toLowerCase();
if (['input', 'select', 'textarea'].includes(tag)) return;
const id = order[state.current];
const e = entryOf(id);
const kind = DOC.candidates[id].kind;
if (event.key === 'k' || event.key === 'K') update(() => { e.decision = 'keep'; e.same_as = null; });
else if (event.key === 'd' || event.key === 'D') update(() => { e.decision = kind === 'accepted' ? 'drop' : 'skip'; e.same_as = null; });
else if (event.key === 'ArrowRight' && state.current < order.length - 1) go(state.current + 1);
else if (event.key === 'ArrowLeft' && state.current > 0) go(state.current - 1);
});
render();
})();
</script>
</body>
</html>