JesseMarkowitzandClaude Opus 5
e5617b86ba
The pipeline's embed-and-cluster step is dead, and this commit holds both the evidence for that and the step proposed to replace it. Predicaments. Scenes are re-described as "what the person is up against", with no names, jobs or places, then embedded and clustered (redescribe.py, topic_words.py, topic_share.py). The pilot chose qwen3:14b over 3b by reading both side by side. Two defects the pilot exposed are fixed: split.py missed titles in quotes and a contents subtitle after a dash, so three stories had been merged into their neighbours, and strip_names.py read New York place names as people. The corrected corpus is probe/v2 (97 stories, 839 scenes); carry_summaries.py reuses the 829 unchanged v1 summaries. Topic share fell from 20% to 13% at k=60, short of the pre-registered 10%. Hand references. Three corpora were read scene by scene and written up by hand, under the same prompt rules the local models get, as a baseline to judge them against: O. Henry (probe/v2/claude, 839 scenes, 20 situations), Wharton's Descent of Man (probe/wharton, 262 scenes, 16 groups) and Jacobs's The Lady of the Barge (probe/jacobs, 157 scenes, 19 groups). Each has its own README and a readable page. No inference was used for any of them. Catalogue. probe/catalogue maps every hand group in the three references onto 36 situation entries, with an answer key per corpus and one recurrence rule applied to all three. classify.py assigns a scene one entry or none, leave-one-corpus- out; score.py checks it against the key, with a self-test on random labels. Why clustering is out: hand-written predicaments, embedded and clustered exactly as the model's were, agree with the hand grouping at ARI 0.05 — no better than the 14B text's 0.07. Better rewriting cannot rescue it. Embeddings cannot even shortlist: the hand label is the nearest entry 13% of the time and in the top 8 half the time. The classification runs are not here. The dev and test runs are pre-registered in probe/catalogue/README.md with the bar set beforehand, and are blocked on the inference host, whose GPU has fallen off the PCIe bus three times. The 30-scene partial output in out/ is not a result. Review page. The situation review is now a browser page rather than JSON edited by hand (review_page.py, review_page_logic.cjs with Node tests, format schema v2). It has never been rendered in a real browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014BygvsUXV9eU6oHkTCkKZ1
50 lines
3.3 KiB
Python
50 lines
3.3 KiB
Python
#!/usr/bin/env python3
|
|
"""Render groups.json + records.json as one self-contained page: REFERENCE.html."""
|
|
import html, json, pathlib
|
|
here = pathlib.Path(__file__).parent
|
|
d = json.loads((here / 'groups.json').read_text())
|
|
recs = {r['chunk']: r for r in json.loads((here / 'records.json').read_text())}
|
|
stories = json.loads((here.parent / 'stories.json').read_text())
|
|
e = html.escape
|
|
SRC = {'archetype': 'archetype', 'filled': 'filled gap'}
|
|
def src(o):
|
|
if o['source'].startswith('scene:'):
|
|
n = o['source'].split(':')[1]
|
|
return f'from scene {n}' + ('' if o['in_group'] else ' <span class="warn">(outside this group)</span>')
|
|
return SRC[o['source']]
|
|
def moves(m): return ' '.join(f'<code>{e(k)} {e(v)}</code>' for k, v in m.items())
|
|
def scene(c):
|
|
r = recs[c]
|
|
return (f'<details><summary><b>{c}</b> · <i>{e(stories[r["story"]]["title"])}</i> — {e(r["predicament"])}</summary>'
|
|
f'<p class="sum">{e(r["summary"])}</p></details>')
|
|
out = []
|
|
for g in d['groups']:
|
|
out.append(f'''<section><h2>{g["id"]} · {e(g["name"])}</h2>
|
|
<p class="pred">{e(g["predicament"])}</p>
|
|
<p class="meta">{e(g["sentence"])} · {len(g["members"])} scenes from {len(g["stories"])} stories
|
|
({e(", ".join(g["story_titles"]))}) · stage <b>{g["stage"]}</b></p>
|
|
<h3>Options</h3><ol>{"".join(f"<li>{e(o['text'])} <span class='src'>{src(o)}</span><br>{moves(o['moves'])}</li>" for o in g["options"])}</ol>
|
|
<h3>Scenes</h3>{"".join(scene(m["chunk"]) for m in g["members"])}</section>''')
|
|
un = ''.join(scene(c) for c in d['unassigned'])
|
|
page = f'''<!doctype html><html lang="en"><head><meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width,initial-scale=1"><title>Jacobs Reference Groups</title>
|
|
<style>
|
|
:root{{--bg:#fbfaf7;--fg:#1d1d1b;--mute:#6b6760;--line:#e2ded6;--acc:#2f5d8a;--warn:#a4481c}}
|
|
@media (prefers-color-scheme:dark){{:root{{--bg:#191917;--fg:#e8e6e1;--mute:#a09c94;--line:#34332f;--acc:#8ab4dc;--warn:#e6915f}}}}
|
|
body{{background:var(--bg);color:var(--fg);font:16px/1.5 Georgia,serif;max-width:52rem;margin:0 auto;padding:1rem 16px 4rem}}
|
|
h1{{font-size:1.5rem}} h2{{font-size:1.15rem;margin:0 0 .3rem}} h3{{font-size:.8rem;text-transform:uppercase;letter-spacing:.06em;color:var(--mute);margin:.9rem 0 .3rem}}
|
|
section{{border-top:1px solid var(--line);padding:1.2rem 0}} .pred{{font-size:1.05rem;margin:.2rem 0}}
|
|
.meta,.src,.sum{{color:var(--mute);font-size:.88rem}} .warn{{color:var(--warn)}}
|
|
code{{font:12px ui-monospace,monospace;border:1px solid var(--line);border-radius:3px;padding:0 .3em;margin-right:.2em}}
|
|
li{{margin:.35rem 0}} details{{margin:.2rem 0}} summary{{cursor:pointer}} .sum{{margin:.2rem 0 .5rem 1.2rem}}
|
|
</style></head><body>
|
|
<h1>The Lady of the Barge: reference groups</h1>
|
|
<p class="meta">W. W. Jacobs, 1902 (Standard Ebooks text). 12 stories, 157 scenes cut by the same rule as <code>chunk.py</code>.
|
|
Summaries, predicaments, groups and options written by Claude by reading every scene: no local inference.
|
|
A comparison baseline, not a pack. Option moves give a direction only, never a number.</p>
|
|
{"".join(out)}
|
|
<section><h2>Unassigned ({len(d["unassigned"])})</h2><p class="meta">Scenes that fit no situation recurring in two or more stories.</p>{un}</section>
|
|
</body></html>'''
|
|
(here / 'REFERENCE.html').write_text(page, encoding='utf-8')
|
|
print('wrote', here / 'REFERENCE.html', len(page), 'bytes')
|