Jesse.Markowitz/TheLadder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e5617b86ba64310cc6478d8c3e4a4c938f430b2a
TheLadder/tools/story-to-pack/probe/validate-score.log
T
JesseMarkowitzandClaude Opus 5 fa3769d0fe Add story-to-pack research and a structured situation review 
Research toward building a content pack from a story corpus, kept on its own
branch and independent of the game. Records the selection experiments against
blind labels, and settles selection as gate G2 followed by a human review:
review.py writes REVIEW.md and a review.json form, apply_review.py checks the
filled form and writes situations.json for the next stage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C6UDQ9o6L6Ey173U7XVou6
2026-09-15 06:59:35 -04:00

88 lines
5.8 KiB
Plaintext
Raw Blame History

V1: 30 sampled, 13 labelled >=1 (base rate 0.43)
old gate accepted 3 label>=1 2 label2 0 P1+ 0.67
G2 accepted 10 label>=1 8 label2 2 P1+ 0.80
V2: 30 sampled, 16 labelled >=1 (base rate 0.53)
old gate accepted 1 label>=1 1 label2 0 P1+ 1.00
G2 accepted 4 label>=1 3 label2 0 P1+ 0.75
V3: 13 grown clusters, stopped by gate 2, by cap 11, label>=1 10, P1+ 0.77
criterion 1 (P1+ >= 0.80 and >= 3 accepted on V1 and V2): fail
criterion 2 (G2 accepts more small clusters than old gate on V1): PASS
criterion 3 (V3 mostly gate-stopped, P1+ >= 0.80): fail
OVERALL: fail
per cluster:
X001 V3 size 45 label 1 old Y G2 Y stop cap schemes, cons and theft over money
X002 V1 size 10 label 1 old - G2 Y stop - struggling artists and money
X003 V2 size 39 label 0 old - G2 Y stop -
X004 V1 size 29 label 1 old - G2 Y stop - schemes, cons and theft over money
X005 V1 size 17 label 1 old Y G2 Y stop - who holds sway in a dining room or hotel
X006 V2 size 15 label 1 old - G2 - stop - a meal given as patronage
X007 V2 size 9 label 0 old - G2 - stop -
X008 V1 size 11 label 2 old - G2 Y stop - a reunion or first meeting
X009 V3 size 45 label 1 old Y G2 Y stop cap a proposal or engagement, accepted or refused
X010 V1 size 11 label 0 old - G2 - stop -
X011 V2 size 43 label 1 old - G2 - stop - drinking in the saloon
X012 V2 size 10 label 0 old - G2 - stop -
X013 V1 size 7 label 1 old - G2 - stop - what to do with money, and whose advice to take
X014 V2 size 10 label 0 old - G2 - stop -
X015 V2 size 38 label 0 old - G2 - stop -
X016 V1 size 18 label 1 old - G2 Y stop - who holds sway in a dining room or hotel
X017 V1 size 9 label 0 old - G2 - stop -
X018 V1 size 7 label 1 old - G2 - stop - shop and service work and standing
X019 V3 size 45 label 1 old Y G2 Y stop cap a public quarrel with a shopkeeper, driver or official
X020 V2 size 24 label 0 old - G2 - stop -
X021 V1 size 21 label 1 old Y G2 Y stop - courtship
X022 V2 size 35 label 1 old - G2 - stop - working girls: jobs, wages and standing
X023 V3 size 45 label 1 old Y G2 Y stop cap who holds sway in a dining room or hotel
X024 V1 size 7 label 0 old - G2 - stop -
X025 V2 size 19 label 0 old - G2 - stop -
X026 V1 size 6 label 0 old - G2 - stop -
X027 V1 size 7 label 0 old - G2 - stop -
X028 V1 size 6 label 0 old - G2 - stop -
X029 V1 size 5 label 0 old - G2 - stop -
X030 V2 size 18 label 1 old - G2 - stop - a brawl or showdown
X031 V1 size 12 label 1 old - G2 - stop - actresses and the stage
X032 V1 size 11 label 0 old - G2 - stop -
X033 V3 size 45 label 0 old Y G2 Y stop cap
X034 V3 size 36 label 1 old - G2 Y stop gate struggling artists and money
X035 V3 size 45 label 0 old Y G2 Y stop cap
X036 V2 size 47 label 1 old - G2 - stop - illness, loss and a missing relative
X037 V1 size 5 label 0 old - G2 - stop -
X038 V1 size 8 label 0 old - G2 - stop -
X039 V2 size 11 label 1 old - G2 - stop - working girls: jobs, wages and standing
X040 V1 size 21 label 0 old - G2 Y stop -
X041 V1 size 17 label 1 old - G2 Y stop - money and marriage: wealthy suitors, inheritances
X042 V1 size 5 label 2 old - G2 - stop - a public quarrel with a shopkeeper, driver or official
X043 V2 size 38 label 1 old - G2 - stop - who holds sway in a dining room or hotel
X044 V2 size 18 label 2 old - G2 - stop - landlord or housekeeper against tenant: rooms and eviction
X045 V2 size 25 label 1 old - G2 - stop - struggling artists and money
X046 V2 size 6 label 0 old - G2 - stop -
X047 V2 size 17 label 1 old - G2 - stop - family pressures over work and marriage
X048 V2 size 20 label 1 old - G2 - stop - money owed and repayment
X049 V2 size 25 label 0 old - G2 - stop -
X050 V3 size 45 label 1 old Y G2 Y stop cap schemes, cons and theft over money
X051 V1 size 17 label 2 old - G2 Y stop - a spouse schemes against or confronts a spouse over infidelity
X052 V2 size 27 label 1 old - G2 Y stop - schemes, cons and theft over money
X053 V2 size 16 label 0 old - G2 - stop -
X054 V1 size 5 label 0 old - G2 - stop -
X055 V3 size 21 label 2 old Y G2 Y stop gate a spouse schemes against or confronts a spouse over infidelity
X056 V3 size 45 label 1 old Y G2 Y stop cap trying to impress or win someone's favour
X057 V3 size 45 label 1 old Y G2 Y stop cap schemes, cons and theft over money
X058 V1 size 16 label 0 old - G2 - stop -
X059 V2 size 19 label 1 old - G2 - stop - a proposal or engagement, accepted or refused
X060 V1 size 18 label 0 old - G2 Y stop -
X061 V2 size 41 label 0 old - G2 - stop -
X062 V2 size 10 label 0 old - G2 - stop -
X063 V1 size 6 label 0 old - G2 - stop -
X064 V3 size 45 label 1 old Y G2 Y stop cap a spouse schemes against or confronts a spouse over infidelity
X065 V2 size 40 label 1 old - G2 - stop - a heated disagreement over whose view is right
X066 V1 size 10 label 0 old - G2 - stop -
X067 V2 size 23 label 0 old - G2 - stop -
X068 V1 size 6 label 2 old - G2 - stop - a spouse schemes against or confronts a spouse over infidelity
X069 V2 size 12 label 1 old - G2 Y stop - trying to impress or win someone's favour
X070 V2 size 32 label 1 old Y G2 Y stop - a spouse schemes against or confronts a spouse over infidelity
X071 V1 size 16 label 0 old Y G2 - stop -
X072 V2 size 17 label 0 old - G2 - stop -
X073 V3 size 45 label 0 old Y G2 Y stop cap
Reference in New Issue View Git Blame Copy Permalink