480414efe082a4bfe0600a19fe23961f6bddd925
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
480414efe0 |
M7: a first-class imported knowledge library
A campaign can import local .txt and .md files as Canon, Reference or Inspiration, and the class is load-bearing rather than a label: it decides the words a passage is framed with in the prompt, the weight it carries when passages are ranked, and which budget it competes in when the context is tight. This is a separate subsystem, which is the Phase 0B decision (IMPORTED-KNOWLEDGE-DESIGN.md §73). Story Cards do not carry classification, provenance, content identity, chunking, an index or a lifecycle, and they were not promoted into something that does. Nothing here reads or writes one. The subsystem, in backend/app/knowledge/: classes the three classes, their weights, and the prompt framing chunking deterministic, heading-aware, 60-800 tokens, no overlap fts SQLite FTS5 with porter stemming; scoped and bounded in SQL importer validate, hash, store, chunk, index — in one transaction embeddings local Ollama vectors through the shared provider retrieval query construction, hybrid merge, rerank inject the budgeted cut and the rendered prompt sections Relevance admission is a separate stage from ranking, and that separation is the milestone's most expensive lesson. An independent review found the first implementation deciding relevance with a floor expressed as a share of the best candidate — which the best clears by construction — so a passage was admitted on every turn regardless of the scene. A query about tide tables and container tonnage retrieved all five sources of a fantasy campaign, narrator-only hidden Canon among them. So the pipeline is now: candidate generation -> admission -> ranking -> class weighting -> budget Admission reads raw, candidate-set-independent signals: the cosine the model returned, and how many distinct meaningful query terms a passage contains. Ranking reads normalized ones, because bm25 has no fixed range and cosine's zero is not zero. Normalization decides order among things that matched; it can never decide whether anything matched. Authority is applied after admission, so a class orders what matched and never rescues what did not. Retrieval may therefore return nothing, and on a scene unrelated to the library it does. The other decisions that each replaced an obvious wrong one: - The class multiplies relevance rather than adding to it. An additive bonus satisfies "Canon outranks Reference" and makes "do not include irrelevant Canon" impossible, because a large enough constant wins on its own. - The semantic floor is measured, not guessed: 113 production-path pairs against nomic-embed-text put targeted matches at 0.55-0.85 and off-topic pairs at 0.36-0.56, and 0.58 sits between them. Because it is a property of that model and not of cosine similarity, it is keyed to the model rather than applied to whatever is configured: an embedding model with no measured calibration in this build does not borrow the number. Semantic admission is skipped, the campaign retrieves lexically, and the reason is stated in the knowledge status and in the turn's provenance. Degrading to lexical keeps the library usable; lending the threshold to an unmeasured model is how the admitted-everything defect would return. - One lexical term is not evidence. Two distinct meaningful terms, or one that is neither a standing campaign entity nor a negligible share of the query. The stop list grew from 42 words to 261, all function words — no subject matter, because a stop list that removes subject matter stops finding "The Silver Key". - Lexical retrieval is a production path, not a fallback. It finds the proper nouns and invented terms a setting bible is made of, and the library is fully usable with no embedding model configured. Safety is structural rather than filtered. Imported text reaches the prompt whole, inside a section that says what it is, under a rule stating the authority order in words and refusing every instruction inside it. No endpoint accepts a filesystem path, so H08 has no mechanism to escape from. Nothing renders imported content as HTML, so a script tag is five visible characters and a remote image is never fetched. Import, chunking, indexing, retrieval and a turn open no socket at all; only embeddings do, through the endpoint allowlist the memory bank already uses. Provenance is the rendered text, not a foreign key: deleting a source cannot turn a historical turn's evidence into dangling ids. Schema: knowledge_sources, knowledge_chunks, knowledge_embeddings, and an FTS5 virtual table attached to knowledge_chunks as a DDL hook so it is created and dropped with the table it indexes. Migration 92. A pre-M7 database opens unchanged and needs no sources to play. Bundle: the source content and the reader's judgements about it travel; the passages, index rows and vectors are rebuilt on import, so a restored campaign is searchable immediately without a reindex step. One runtime dependency: python-multipart, Starlette's multipart parser. It is what makes the upload surface possible, and the upload surface is why no pathname is ever accepted. The test doubles were the reason the defect shipped, so they were corrected too. The retrieval stub scored unrelated text at 0.06-0.20 where the real model scores it at 0.43-0.44, and its docstring said it had deliberately removed the constant component that "would put a similarity floor under every pair" — which is exactly the property real models have. The stub now has that floor, one test fails if it is ever removed, and another reproduces the superseded rule and asserts it is still fooled by the same fixture. Run against the pre-corrective implementation, the new suite fails 13 of 18. Tests: 939 passed, 14 skipped (836/7 at M6). 110 new across seven files, one of which mocks nothing between itself and Ollama and re-measures the similarity separation on every run. 43/43 checks in a real Firefox, reproduced. Docker build clean. Four other defects found by review or by the browser run were fixed here rather than carried: an unreachable relevance constant that appeared to enforce something and did not; acceptance tests using the wrong fixture files, so G07's trap was never exercised; a bidirectional override surviving into displayed filenames; and, from the implementation pass, the Insights panel showing M5's two state sections as raw keys and the source inspector refetching on every keystroke. M7 was independently reviewed, which returned PASS WITH CORRECTIVE WORK REQUIRED. Both blocking findings are closed, and closeout resolved the embedding-model calibration boundary the corrective pass had left as debt. planning/reports/M7-IMPLEMENTATION-REPORT.md carries the review, the corrective closeout and the closeout verification in sequence, none overwriting another. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HdaXiFbscatQaLS7dJk6b |
||
|
|
b7005e6fdd |
M5: genre-neutral authoritative narrative state, with review corrections
Replaces AI-DnD's RPG relative-delta world state with the genre-neutral typed
narrative state of ADR 010: explicit, absolute, allowlisted events proposed by
the model, validated by the application, applied to one authoritative document,
and snapshotted per position so restore stays a row read.
This commit includes the corrective pass that followed the independent review
in planning/reports/M5-IMPLEMENTATION-REPORT.md. The invariant it exists to
hold is:
visible active transcript position == stored head == authoritative state
Narrator editing (D10, STORY-BRANCH-SEMANTICS §§14-15)
A narrator edit no longer rewrites a row. It returns to the state before the
turn, takes the reader's exact text as the accepted narration, re-derives the
state that text implies, and becomes a new active continuation — while the
original narration keeps its words, its live flag and its whole future as
retained history. At the tip the correction is another take; with story below
it, it forks. No new history machinery: this is the existing fork/take/head
path with the reader's text in place of a generated reply. The §14A refusal
is therefore gone for narrator turns, and remains only for player input.
Pre-M5 positions
Migration 88 backfills the empty narrative document onto every action written
before M5, and a missing snapshot now restores the empty document instead of
leaving the previous position's state standing. Restoring to an old Save
Point no longer leaves a later position's entities and facts on screen.
Narrator context
Replayed history carries prose only; the machine-readable block is no longer
reconstructed into past turns, where it contradicted the authoritative state
in the same prompt. A fact withdrawn by a manual correction is now named as
no longer true, with the reader's reason, rather than silently dropped.
Also
- state_changes joins the action-list bulk read, removing one query per row.
- Extraction takes only the application's own protocol payload: an ordinary
```json or ```python block in a story survives, and a mangled proposal
still does not reach the reader.
Planning: ADR 013 records the authoritative document shape; §§14-15/14A, D10,
C04 and BUILD-MILESTONES are updated to describe what exists. Debt is recorded
against M8 (scenario editor UX) and M9 (export of the audit trail).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2
|
||
|
|
e08d49c3eb |
M4: add durable named Save Points
A Save Point is a name for a story position, and restoring one is head movement. That is the whole architecture, and it is what ADR 012 and BUILD-MILESTONES' note on M4 asked for: M3 made the head a stored (branch, depth) and made arriving at one a row lookup plus a state restore, so a Save Point needs no restore machinery of its own. What the user gets: - Name the moment they are reading, keep playing, restart the app, and come back to it. Restoring moves the story back and deletes nothing: the later turns stay, Redo still walks forward into them, and writing something different is what starts a new line while the old one is kept. - Rename, delete, and a list, in a Save Points panel beside the branch panel, with a Save Point button next to Undo and Redo. Both confirmations say what is *not* destroyed, because that is the part the screen cannot show. - Save Points survive export and import. What was deliberately not built: - No second restore path. `head.move_to_node` is the only new movement: its depth half is M3's `head.move_to` unchanged, and its branch half is the single assignment `switch_branch` already makes. No head field is written in the checkpoint router, nothing reconstructs state, nothing prunes a memory, nothing copies or deletes a turn, and restore never forks — the first write below the restored head does, through `fork_if_behind_head`. - No automatic cleanup. A Save Point behind the head, or naming a line the story left, is doing its job (STORY-BRANCH-SEMANTICS §19). The one removal is a cascade: deleting a branch takes its Save Points, as it takes its memories, because the story they named went with it. - No new ADR. ADR 012 already decides the architecture, and a table is not a decision. The one call the planning package did not already make: restore moves the branch half of the head only when the coordinate is off the path being read. Doing it unconditionally would quietly hand back an abandoned continuation whenever a Save Point in a shared prefix was restored; never doing it would make a Save Point on a departed line unrestorable, which contradicts §19. TECHNICAL-DESIGN §8.8 records it. Schema: a `checkpoints` table holding a name, an optional note and a (branch, depth) coordinate — no copy of any story. `create_all` builds it as it did `memories` and `branches`; migration 80 adds the index. No backfill, because nobody had named a position before M4. The coordinate is deliberately not an action id: one coordinate holds every attempt at a turn and exactly one is live, so a coordinate follows a retry where a row id would pin a take the story no longer tells. Tests: 680 pass (638 before). 42 new in tests/test_save_points.py covering D11-D14, I04, L03, E-series lineage and memory isolation after restore and divergence, the edge cases, and an M3-database migration. One pre-existing fixture in test_tree_migration.py needed `checkpoints` added to its drop list — SQLite refuses to drop a table another table references. Not verified: the browser. No session has had a usable one, so the Save Point panel's DOM behaviour is unobserved — as M3's Redo control still is. The twenty-step sequence was driven over HTTP against a live server with a real process restart instead, and all seventeen checks pass. M4 is implemented, not accepted: no review has been written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2 |
||
|
|
7f082b61d8 |
M3: complete non-destructive history and active-head export
The head-cursor model landed in
|
||
|
|
8c65ae99de |
M2: cut the hosted product away from the local one
94 files, +1,395 -6,578. Three files are new; twenty-four are gone. The milestone is subtraction, and what is left is the single-user local storyteller the specification describes. Removed in full: campaign scripting and its QuickJS sandbox; multi-user accounts, guest sessions, login, registration and the shared demo key; the visitor-analytics tables, dashboard and page beacon; the access log of sign-ins, addresses and devices; per-IP and per-user rate limiting and quotas; Render deployment config; Postgres and psycopg; cloud inference providers, the API-key field and the key encryption that existed to store it; session-cookie signing. None of it was hidden behind a flag — the routes are gone and answer 404. Two things were kept that the brief allowed keeping. The `users` table and its foreign keys stay as an internal ownership detail, because rewriting them out means a migration across most of the schema to delete a column that costs nothing; nothing creates a second user and no request carries an identity. Five inert tables and four inert columns stay for the same reason, so an M1 campaign database opens unchanged. The one addition is app/endpoints.py, which decides where a story may be sent. Loopback, RFC1918, link-local, unique-local and CGNAT — an explicit allowlist of networks, not a guess at what `ipaddress` means by "private", which calls the documentation ranges private and IPv6 loopback reserved. Every address a hostname resolves to must be in it, so a split answer does not squeak through, and the rule runs both when the endpoint is saved and before every outbound request, because a name that resolved to the LAN this morning can resolve elsewhere this afternoon. Known cloud hosts are named in the refusal so the error says why rather than looking like broken DNS. TLS is never traded against it: M1's shared trust context is intact on all four clients and there is no way to skip verification. The hardcoded 120-second model timeout is now a setting. That was not theoretical — on this GPU-less four-core host a cold load of qwen2.5:3b-instruct took 648.9 seconds to produce the first turn, while turns 2 to 5 of the same campaign took 3.6 to 13.1. Connect stays short at 10s so a wrong address still fails fast; the read timeout defaults to 300s and is bounded at 3600, because "wait longer" must stay a number. Two defects found while testing and fixed here. An unknown /api path fell through the SPA catch-all and came back as HTML with status 200, so a client asking for JSON parsed a web page instead of learning the route was gone. And AIDND_CORS_ORIGINS accepted "*", which on an unauthenticated loopback API would hand every page on the Internet a write handle on the campaign database; it now refuses to start. Verified rather than assumed. Offline, on a network with no route out and no DNS: five turns, retry with both takes retained, restart with an identical transcript digest, a failed model call leaving the accepted AI-turn count untouched, and a capture with zero non-loopback unicast packets. Against a real second machine on the LAN over HTTPS with a private CA: four turns, restart, and a capture showing 289 packets to the approved host, 344 loopback, zero anywhere else, zero DNS queries. Cloud and public endpoints refused with their reasons; no API key settable; every removed route 404. 604 backend tests pass, down from 648 by the fifteen retired with the subsystems they tested and up by the twenty-nine added for the endpoint policy and the removed surface. The scripting tests were not deleted: eight files used a JavaScript counter as instrumentation for the state snapshot and rollback machinery, which M2 does not touch, so the counter moved to the world-state engine and those tests still assert what they always did. Frontend lint and build are clean; the image builds, and its wheel-building stage is gone with quickjs. No M3 work. Undo is still destructive and there is still no Redo. |
||
|
|
9ee052c51e |
Give the adventure a persona, so the protagonist has a name
The player had stats but no identity. `stat_schema.player` carried hp and
mana beside `npc.gwen.trust`, but where an NPC has a name and a
description the player had neither, so the block rendered as
`You: hp 100/100` and nothing in the prompt said who "you" was.
Three columns on `adventures`: name, pronouns, description. All
user-only, all optional, and an empty name means the app behaves exactly
as it did before — no backfill, no special case for an adventure that
predates the migration.
They are adventure columns rather than part of `stat_schema` for two
reasons. An adventure with no RPG layer still has a protagonist, and
that is the case this was added for. And `worldstate.schema._initials`
treats every dict inside a stat section as a stat definition, so a
persona placed there would be instantiated, rendered in the guide, and
handed an `initial` value as though it were one.
The paths do not change. `player.hp` stays `player.hp`; only the label
moves, to `Kaelen (player): hp 100/100`, the same way NPC lines already
print a display name beside the id. A path carrying the persona's name
would break the moment a player renamed their character, because
`_history_text` replays every past turn's stored delta into the prompt
and those blobs hold literal `player.hp` strings.
The section sits in the system block. Only the user can edit it, so it
never changes mid-story and stays inside the cached prefix. That is what
makes it free, and it is why the AI must not be able to move it — a
delta aimed at `persona.*` is already refused by `_resolve`, and there
is now a test holding that in place.
The modal that used to appear only for scenarios with `${Placeholder}`
tokens now always opens, and is where the character is named. Persona
and placeholders stay independent: a scenario asking for `${Name}` is
asking its own question. No scenario in the repo uses placeholders at
all, so the overlap is hypothetical.
Phase 2, which feeds the persona and the cast to the summarizer, is
written up in plan/18 and not started. That is where the memory-quality
problem actually gets fixed; this change is what gives it a name to use.
Not yet driven in a browser — plan/18 lists what to check by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
|
||
|
|
949f58a054 |
Sweep the last three uses of the columns SP8 dropped
`seed_demo.py` still passed `index=0` to `models.Action`, which raises `TypeError` now that the attribute is gone. Every call site inside `app/` was updated when the column went, but the seed script sits outside the package and was missed. `bundle.settle` wrote `memory_cursor` and `summary_cursor`, which are no longer mapped, so the assignments only set transient Python attributes. The version 2 branch existed solely to make those assignments, and it ran two queries per cursor to do it, so it goes. `settle` now returns early when the bundle carries anchors. The `db` parameter is unused after that. Also move the comment about deleted memories next to the `db.delete(branch)` it describes. Splitting the router package left it after the `finally`, where it read as attached to nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YFQY6WgaE3JaX3dXkxLynV |
||
|
|
f1bebe18d0 |
Drop the eight legacy columns SP8 left behind
Migrations 66 to 73 drop `actions.index`, `variants`, `variant_index`, `variant_count`, `state_before`, and `world_state_before`, plus `adventures.memory_cursor` and `summary_cursor`. `index` is a keyword in SQLite, so migration 71 quotes it. Nothing outside the migrations read these. `models.py`, `schemas.py`, and `ACTION_LIST_COLUMNS` lose the same eight fields, `Adventure.actions` orders by `id`, and `attempts.renumber`, `context.history.max_action_index`, and `nodes.next_index` are deleted. Two changes keep the migration replayable on a `create_all` database: - `_split_variants_into_siblings` wrote through the live ORM table, so it stopped compiling once migration 66 removed five of its columns. It now writes through `_ACTIONS_AT_60`, a frozen `Table` with its own `MetaData`. - Five data passes read columns these migrations drop. Each now calls `_has_columns` and returns early when the columns are absent. `bootstrap` takes a `through` version so a migration test can stop at the schema it asserts on. 555 tests pass, up from 549. Eight of the new cases assert each column is gone after a real schema-45 database migrates all the way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0198qDK3gmgSo7EtQ4GTPqqK |
||
|
|
6cbf6d996f |
Give every new guest an adventure that is already played
An empty account gives a visitor nothing to read, and the daily demo turns are limited, so learning what the app does cost one of them. `app/starter.py` now copies a shipped export bundle into each new guest at the point the row is created. The bundle is two exchanges of the Pokemon demo, which ends on a knockout and shows an applied change, a refused one, and a milestone. The guest row is committed before the copy is attempted, so a failure there still leaves them with an account, and the copy runs inside a savepoint. The row building that `POST /adventures/import` did inline moved into `bundle.materialize`, which both callers use. The rate and size checks stayed in the endpoint: the starter writes a file the server ships, so it has no untrusted list to cap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF |
||
|
|
e7d75c3b05 |
Rewrite Python comments in Google developer documentation style (#12)
* Rewrite comments in Google developer documentation style Rewrite the comments and docstrings across the backend core modules so they read plainly. The previous prose was accurate but dense and figurative, which made it slow to skim. Applies the Google developer documentation style guide: short sentences, active voice, present tense, American spelling, and no metaphors, idioms, or rhetorical asides. Replaces em-dash chains with separate sentences. |
||
|
|
4af6e17406 |
Answer the review, and keep the opening node's bank
Nine findings from a review of the phase-14 stack. The one about a retry withdrawing a memory is not a bug — a memory anchored to a node describes that node, and it goes when the node goes. The root is the exception, and it is the only one: migration 62 parked every memory written before memories had coordinates on depth 0, so withdrawing the opening node would retire a whole bank nobody attached there. A memory with no source range covers no story and now stays; a summary that genuinely ends there is still withdrawn. The rest are repairs. * The adventure list quoted whichever attempt was written last rather than the one the story tells, so switching back left the index disagreeing with the page. * A v1 import gave a typed memory no depth, rebuilding the NULL the migration exists to remove — invisible until the imported adventure forked. * The action cap counted a v1 file's turns, and a turn expands into a row per saved attempt, so a file inside the cap could write a multiple of it. * Forking a live node on a borrowed ancestor promoted a sibling on a branch the caller never named. It is a branch switch, and now says so. * Switching attempts left the state, status and memory panels reading the previous take: the story does not change length, so nothing keyed on its length noticed. Same class as the branch-switch bug this phase already fixed. * A retry after switching back numbered the new attempt into the middle of the group instead of the end. * The cursor backfill numbered every action in the table once per adventure; correlated to the adventure being updated, it is an index lookup instead. * Renaming a branch answered own_actions=0. And one behaviour change recorded rather than repaired: script-visible history and actionCount no longer count blank-text rows. That is the right shape and there is no reading compatible with both, so plan/14 says so. 409 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g |
||
|
|
cf3d52171e |
Let a branch be named, and thrown away
SP7 needs three branch operations and SP5 built one. Switching exists; naming and deleting had no column and no route between them. A name is stored because a player chose it. An unnamed branch keeps NULL rather than a generated "branch 4" — a generated label is derived, and it would go stale the moment a branch before it is deleted and the ordinals shift underneath. The client draws those from the fork depth, which nothing can shift. The v2 bundle carries the name for the same reason it carries the fork points and leaves `lineage` out: it is a decision, not something computed from one. Delete is what stands between a tree and unbounded growth, since nothing prunes one on its own. It refuses two branches: the root, which holds the turns every other branch borrows, and the one being read — including any branch the head was forked from, which is the same mistake in disguise and the one that would cascade the head away and leave head_branch_id pointing at nothing. The nodes, memories and descendants go through the foreign keys that already cascade. A cursor standing on a deleted branch is cleared. On Postgres a stale branch id would simply never resolve; SQLite hands the freed id to the next fork, and then the anchor resolves onto a branch it has never seen and calls a stretch of story already summarized. 396 tests, 15 new. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g |
||
|
|
a7bf47a35e |
Let a backup carry a story that went two ways
A bundle had one list and a forked adventure has two stories, so export was emitting every branch's turns interleaved by index — a mangled story rather than lost data, and unreachable only because forking has no UI yet. `ai-dnd-adventure-v2` carries the branches, the depth each one left its parent at, which attempt at every turn is the story, and what each node left behind. That last one is not decoration: the after-snapshots are what a branch switch puts back, and a bundle without them imports a tree nobody can switch inside. `app/bundle.py` owns both formats and nothing else knows either. The v1 reader stays — those files are already on people's disks — and it is now the only place a `variants` array exists anywhere. The rule the module is built on is that a bundle carries what was chosen and never what is derived. The head branch, the fork points, the live flags and the anchors are decisions somebody made. The lineage, the head depth, the legacy `index` and the variant ordinals are computed from those and are rebuilt on the way in, because a bundle is a text file anybody can edit and a derived field shipped beside its source is a chance for the file to disagree with itself where no read would report it. `index` is the one that stops being academic here. It agreed with `depth` until SP5, and this is the first writer that has to fill it for a forked story, where two branches both hold a node at depth 4. It is allocated one per turn instead: siblings share it, no two coordinates do. Everything a hand-edited file can get wrong about the shape of a tree is a 400 raised before the adventure row exists, because a half-applied import is exactly the failure this phase exists to end — a story that goes quiet. A file wrong about which attempt is live is corrected rather than refused; that is an invariant of the database, not of the format. Measured on the 600-action fixture: 587 kB to 911 kB, and all of the increase is the outcomes at 489 B a node — the coordinates themselves save 57.5 B a node against the old turn-and-variants shape. Twenty forks add 660 B. 4.3% of the import body cap. 381 tests green, 16 of them new in test_bundle_v2.py. No migration, no vacuum owed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g |