Jesse.Markowitz/TheLadder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e5617b86ba64310cc6478d8c3e4a4c938f430b2a
TheLadder/tools/story-to-pack/probe/validate3-score.log
T
JesseMarkowitzandClaude Opus 5 fa3769d0fe Add story-to-pack research and a structured situation review 
Research toward building a content pack from a story corpus, kept on its own
branch and independent of the game. Records the selection experiments against
blind labels, and settles selection as gate G2 followed by a human review:
review.py writes REVIEW.md and a review.json form, apply_review.py checks the
filled form and writes situations.json for the next stage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C6UDQ9o6L6Ey173U7XVou6
2026-09-15 06:59:35 -04:00

104 lines
7.8 KiB
Plaintext
Raw Blame History

V0: 25 sampled, 5 labelled >=1
old accepted 2 label>=1 1 label2 0 P1+ 0.50
G2 accepted 8 label>=1 4 label2 0 P1+ 0.50
G3 accepted 11 label>=1 4 label2 0 P1+ 0.36
V1: 20 sampled, 5 labelled >=1
old accepted 1 label>=1 0 label2 0 P1+ 0.00
G2 accepted 1 label>=1 1 label2 1 P1+ 1.00
G3 accepted 4 label>=1 2 label2 0 P1+ 0.50
V2: 20 sampled, 12 labelled >=1
old accepted 0 label>=1 0 label2 0 P1+ 0.00
G2 accepted 4 label>=1 4 label2 1 P1+ 1.00
G3 accepted 2 label>=1 2 label2 1 P1+ 1.00
<=9 members in V0+V1: 27 clusters, 4 labelled >=1
old accepted 1 label>=1 0 P1+ 0.00
G2 accepted 3 label>=1 1 P1+ 0.33
G3 accepted 11 label>=1 3 P1+ 0.27
V3: 10 grown, stopped by gate 3, by cap 7, label>=1 6, P1+ 0.60
V0-V2 pooled: G3 17 accepted, 8 >=1, P1+ 0.47; G2 13, 9, 0.69
criterion 1 (P1+ >= 0.80 and >= 3 accepted on V1 and V2): fail
criterion 2 (<=9 members: >= 2 accepted, P1+ >= 0.67, more than G2): fail
criterion 3 (V3 mostly gate-stopped, P1+ >= 0.80): fail
criterion 4 (V0-V2: at least G2's label>=1 count at no lower precision): fail
OVERALL: fail
per cluster:
Y001 V1 size 23 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.530 stop -
Y002 V0 size 9 label 0 old - G2 Y G3 Y PCT 1.000 MMIN 0.631 stop -
Y003 V3 size 25 label 1 old - G2 Y G3 Y PCT 1.000 MMIN 0.554 stop gate struggling artists and money
Y004 V2 size 28 label 1 old - G2 - G3 - PCT 0.091 MMIN 0.350 stop - landlord or housekeeper against tenant: rooms and eviction
Y005 V0 size 8 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.528 stop -
Y006 V3 size 45 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.586 stop cap schemes, cons and theft over money
Y007 V3 size 45 label 0 old Y G2 Y G3 Y PCT 1.000 MMIN 0.574 stop cap
Y008 V1 size 5 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.669 stop -
Y009 V1 size 8 label 0 old - G2 - G3 - PCT 0.948 MMIN 0.525 stop -
Y010 V2 size 18 label 1 old - G2 Y G3 Y PCT 1.000 MMIN 0.549 stop - struggling artists and money
Y011 V1 size 15 label 0 old Y G2 - G3 - PCT 1.000 MMIN 0.596 stop -
Y012 V0 size 9 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.572 stop -
Y013 V1 size 7 label 1 old - G2 - G3 Y PCT 1.000 MMIN 0.597 stop - struggling artists and money
Y014 V2 size 13 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.547 stop -
Y015 V0 size 12 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.544 stop -
Y016 V3 size 45 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.555 stop cap a proposal or engagement, accepted or refused
Y017 V3 size 45 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.562 stop cap an invitation to dine
Y018 V2 size 19 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.522 stop -
Y019 V0 size 10 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.568 stop -
Y020 V1 size 11 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.518 stop -
Y021 V3 size 45 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.561 stop cap schemes, cons and theft over money
Y022 V2 size 24 label 1 old - G2 Y G3 - PCT 1.000 MMIN 0.576 stop - schemes, cons and theft over money
Y023 V2 size 8 label 1 old - G2 - G3 - PCT 0.898 MMIN 0.470 stop - working girls: jobs, wages and standing
Y024 V1 size 12 label 2 old - G2 Y G3 - PCT 1.000 MMIN 0.547 stop - someone coerces someone with a threat
Y025 V0 size 5 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.582 stop -
Y026 V1 size 6 label 0 old - G2 - G3 - PCT 0.999 MMIN 0.567 stop -
Y027 V1 size 6 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.568 stop -
Y028 V1 size 7 label 1 old - G2 - G3 - PCT 0.999 MMIN 0.548 stop - actresses and the stage
Y029 V0 size 8 label 0 old - G2 Y G3 Y PCT 1.000 MMIN 0.632 stop -
Y030 V1 size 7 label 1 old - G2 - G3 Y PCT 1.000 MMIN 0.561 stop - who holds sway in a dining room or hotel
Y031 V0 size 6 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.602 stop -
Y032 V0 size 13 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.570 stop - a run-in with the police
Y033 V0 size 5 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.631 stop -
Y034 V0 size 5 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.597 stop -
Y035 V1 size 11 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.524 stop -
Y036 V1 size 6 label 0 old - G2 - G3 - PCT 0.999 MMIN 0.556 stop -
Y037 V0 size 10 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.553 stop -
Y038 V2 size 22 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.529 stop - someone coerces someone with a threat
Y039 V0 size 8 label 0 old - G2 - G3 - PCT 0.999 MMIN 0.522 stop -
Y040 V0 size 6 label 1 old - G2 Y G3 Y PCT 1.000 MMIN 0.631 stop - who holds sway in a dining room or hotel
Y041 V0 size 7 label 0 old - G2 - G3 - PCT 0.969 MMIN 0.522 stop -
Y042 V3 size 33 label 0 old Y G2 Y G3 Y PCT 1.000 MMIN 0.530 stop gate
Y043 V2 size 31 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.538 stop - a run-in with the police
Y044 V0 size 5 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.646 stop -
Y045 V0 size 5 label 0 old Y G2 - G3 Y PCT 1.000 MMIN 0.577 stop -
Y046 V1 size 5 label 0 old - G2 - G3 - PCT 0.790 MMIN 0.508 stop -
Y047 V0 size 18 label 2 old - G2 - G3 - PCT 1.000 MMIN 0.532 stop - working girls: jobs, wages and standing
Y048 V1 size 12 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.536 stop -
Y049 V2 size 22 label 2 old - G2 Y G3 Y PCT 1.000 MMIN 0.610 stop - a spouse schemes against or confronts a spouse over infidelity
Y050 V2 size 8 label 0 old - G2 - G3 - PCT 0.989 MMIN 0.509 stop -
Y051 V2 size 24 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.521 stop -
Y052 V1 size 23 label 2 old - G2 - G3 - PCT 1.000 MMIN 0.591 stop - an invitation to dine
Y053 V2 size 45 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.467 stop - a brawl or showdown
Y054 V1 size 10 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.570 stop -
Y055 V2 size 6 label 0 old - G2 - G3 - PCT 0.959 MMIN 0.519 stop -
Y056 V2 size 15 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.527 stop -
Y057 V0 size 8 label 0 old - G2 - G3 - PCT 0.927 MMIN 0.514 stop -
Y058 V2 size 5 label 0 old - G2 - G3 - PCT 0.936 MMIN 0.518 stop -
Y059 V0 size 15 label 0 old - G2 Y G3 - PCT 1.000 MMIN 0.547 stop -
Y060 V0 size 5 label 0 old - G2 - G3 - PCT 0.991 MMIN 0.539 stop -
Y061 V2 size 37 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.546 stop -
Y062 V3 size 45 label 1 old Y G2 Y G3 Y PCT 1.000 MMIN 0.583 stop cap a spouse schemes against or confronts a spouse over infidelity
Y063 V3 size 8 label 0 old - G2 - G3 Y PCT 1.000 MMIN 0.592 stop gate
Y064 V2 size 60 label 1 old - G2 Y G3 - PCT 1.000 MMIN 0.532 stop - a proposal or engagement, accepted or refused
Y065 V1 size 11 label 0 old - G2 - G3 - PCT 1.000 MMIN 0.552 stop -
Y066 V0 size 17 label 0 old - G2 Y G3 Y PCT 1.000 MMIN 0.579 stop -
Y067 V2 size 24 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.544 stop - illness, loss and a missing relative
Y068 V1 size 6 label 0 old - G2 - G3 - PCT 0.996 MMIN 0.559 stop -
Y069 V0 size 12 label 1 old - G2 Y G3 Y PCT 1.000 MMIN 0.566 stop - trying to impress or win someone's favour
Y070 V0 size 5 label 0 old - G2 - G3 - PCT 0.977 MMIN 0.543 stop -
Y071 V2 size 21 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.502 stop - schemes, cons and theft over money
Y072 V3 size 45 label 0 old Y G2 Y G3 Y PCT 1.000 MMIN 0.577 stop cap
Y073 V2 size 36 label 1 old - G2 - G3 - PCT 1.000 MMIN 0.527 stop - writers and editors
Y074 V0 size 11 label 1 old - G2 Y G3 Y PCT 1.000 MMIN 0.578 stop - working girls: jobs, wages and standing
Y075 V1 size 5 label 0 old - G2 - G3 - PCT 0.999 MMIN 0.564 stop -
Reference in New Issue View Git Blame Copy Permalink