A memory whose block ends on the newest action is the one memory a player
can reach: retry and take-switching both refuse anything else. Each retry
of that turn withdrew the memory and wrote it again, and a block closes
every six actions while a normal turn writes two, so that was one turn in
three.
SETTLE_SLACK asks for one action past a block before the block is
summarized. The block is still MEMORY_INTERVAL actions; only the moment
moves. This is not the pre-SP4 holdback returning: that one was about a
retry rewriting text in place, which sibling attempts and forget_node
settled, and correctness still rests on the withdrawal rather than on the
slack. Undo and delete can carry a summarized node back to the tip, so the
withdrawal path stays reachable, just rarely.
The slack buys nothing back. The block that just closed is still in the
history window in full, so a memory of it says what the model can already
read.
Tests: the settling suite asserts the new rule and that a retry at the tip
finds nothing to withdraw; the rewrite suite builds thirteen actions so
both of its blocks settle; the one path test that ended on a block boundary
sets the slack to zero, because it is about which actions a block is read
from rather than about when a block forms. 632 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NcrxCJjqgDvAkamKeLWdn
The report opened with the connection string it was about to work on, taken
straight from AIDND_DATABASE_URL. On the hosted deploy that string carries the
Neon password, so the first line of every run put a live credential into the
console — and from there into scrollback, a screenshot, or a pasted bug report.
Nothing in the output said it was there to notice.
It now prints scheme, user, host and database name. The query string goes whole:
sslmode is the only part worth reading, and some drivers accept a password
there as well.
631 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tqgupw5CZGjSZrUTNUd4fW
The report prints an arrow between the old and new word counts, and it prints
the memories themselves, which are model-written prose. A Windows console
defaults to cp1252 and cannot encode either, so the run dies on
UnicodeEncodeError partway through — after some memories have been rewritten
and committed, which is the worst place to stop. STATUS already carries the
same warning for tools/dbmeter.py.
stdout is reconfigured to UTF-8 with errors="replace" instead, so the report
degrades a character at a time rather than failing.
The usage notes were also written for a POSIX shell, and this project is
developed in PowerShell, where `VAR=x command` is not a thing. Both docs now
give the PowerShell form first, via Read-Host so the Neon URL and the secret
stay out of the history file, and name the venv's Python.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tqgupw5CZGjSZrUTNUd4fW
The hosted database is not the local one. It holds other people's stories, and
`summary_provider` builds from the adventure owner's Settings, so an unfiltered
`--write` against it would spend other people's money rewriting memories they
never asked about. `--adventure` could already hold a run down, but only if you
knew the ids.
`--email` names accounts instead, and every line of the report now says who owns
the adventure, so a dry run answers "whose keys would this spend" before
anything is written. Guests have no email and stay reachable only by id, which
is the right amount of friction for touching a stranger's bank.
The other half is written down rather than built: stored API keys are encrypted
with AIDND_SECRET_KEY, so a run from a checkout against the Neon database needs
the same value the web service holds. With a different one `decrypt_secret`
returns "" instead of failing, and every adventure is reported as having no key
— a run that looks like it worked and did nothing. The Dockerfile copies
backend/app alone, so tools/ is not on the box either way; the recipe is a
checkout pointed at AIDND_DATABASE_URL.
629 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tqgupw5CZGjSZrUTNUd4fW
The prompt change only reaches memories written after it. plan/18 decided to
leave the existing ones alone and let eviction age them out at
memory_bank_capacity, on the grounds that re-summarizing would duplicate
whatever was still in the bank because nothing deletes the old rows.
That was wrong about the only option. A memory can be rewritten in place. The
row carries more than its text — whether it is pinned, how often it has been
retrieved, and the node it hangs off, which is what makes a fork inherit the
right memories — and rewriting `text` keeps all of it. Deleting the bank and
rewinding the cursor would lose that, and would trickle memories back at
MAX_MEMORIES_PER_RUN per turn, so an adventure nobody is playing would never
recover.
tools/rewrite_memories.py does it. Without --write it makes no model calls and
only reports the scope; --write rewrites, --embed re-embeds in the run rather
than leaving it to the app's post-turn pass. It reads whichever database the
app reads, so it works against the hosted Postgres as well as a local file.
Two things it needed from the app. `summarize_block` is now the one place a
memory prompt is assembled, and the post-turn pass calls it too — a backfill
that built its own prompt would be writing memories with a prompt that never
shipped, and nothing would report the drift. `source_block` reads a memory's
block back out of the story, which nothing has ever had to do: it reads on the
lineage of the branch the memory was written on, not the branch being played,
because after a fork the same depths hold different actions on each side and a
read through the adventure's path would summarize the wrong story silently.
It also excludes the discarded attempts at a retried turn, and tolerates a
block an action has since been deleted from.
Left alone: a memory with no source range, which is hand-written or migrated by
62 and may be the player's own words; a memory whose actions are gone; and an
adventure whose owner has no API key, because summarization spends the user's
own key by construction and never the shared demo key. --api-key/--model/
--endpoint override that, the last of them aiming a run at claude_shim.py.
The vector is cleared for every rewrite, because the stored one describes
wording that no longer exists. Re-embedding always uses the owner's own
embedding model, never --endpoint: a vector only means anything against the
vectors it is ranked beside.
17 tests, 627 green. The fork case is the one that would fail quietly, so the
test builds a fork whose depths hold different actions on each side.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tqgupw5CZGjSZrUTNUd4fW
The checked-in harness was run end to end through the shim on a newly
generated story, which is both a check that it works and an independent
replication.
The finding the change is for held: across two runs, none of the four
control memories names the protagonist and all four treatment memories
do. The length finding held too — controls at 34, 89, 105 and 107 words,
a four-fold spread with no budget stated anywhere, against 54 and 55
under MEMORY_MAX_WORDS.
One claim did not replicate, and the writeups now say so. Run 1 produced
two control memories in two different persons, which is the reported
complaint exactly, and plan/18 presented that as reproduced. Run 2's
controls were both "the player", consistently. Drifting between second
and third person is therefore something a model sometimes does, observed
once, not something it does every time. The naming gap is the durable
result and the writeups now lead with that instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
The run that justified MEMORY_MAX_WORDS lived in a scratch directory and
would have been gone with the container. The numbers in plan/18 were
therefore assertions nobody could check.
plan/18-appendix-memory-ab-run.md now carries the whole transcript: both
memories, both summaries, and the thirteen-action story they were
written from.
backend/tools/memory_ab.py reproduces it. It replaces the throwaway
script the first run used, and differs in two ways that matter. It goes
through OpenAICompatibleProvider rather than calling a model directly,
so a run exercises the provider, the streaming path and complete()
instead of a stub. And it reads the control prompt out of git at the
commit given to --before, so the thing being compared against cannot
drift from what actually shipped.
There was already a claude_shim.py serving an OpenAI-compatible endpoint
backed by the CLI, which is exactly what the throwaway script had
reinvented. memory_ab.py points at it by default, so a run spends a
Claude subscription rather than API credit, and --endpoint aims it at
the provider the deployed app really uses — which is the one question
this whole exercise could not answer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
Ran the memory prompt end to end against a real model as a controlled
A/B: one story generated through the app's own build_context a turn at
a time, then both prompts run over the same blocks, so the story is held
constant and the prompt is the only variable. Fresh process per call, so
neither arm sees the other and the model is never told what is being
tested. The control is the exact prompt from 9cdcb55.
The reported fault reproduced. Two consecutive memories written from one
story minutes apart came back in two different persons — "You crept low
through the mist" and "The player asked Gwen to". With the cast brief
both named Kaelen. The control also inverted who acted on a move whose
player text was "grab her wrist and pull her down", which is the failure
the brief predicts: with no cast there is nothing to say whose wrist
"her wrist" was.
It also found something the prompt review had not. "1-2 plain sentences"
is not a length, and the same model wrote 34 words for one block and 105
for the next. A 105-word memory is a paragraph, and `memory_top_k`
injects five every turn, so the bank's running cost was set by a number
nobody had ever stated.
MEMORY_MAX_WORDS states it, and the prompt now says which details to
keep when trimming: the ones a later scene could turn on. Re-run over
the identical story, the same two blocks came back at 32 and 58 words,
still named, still third person, still carrying the camp map, the
strongbox behind the second tent, and the strap frayed near through.
Variance is the real gain — 34..105 became 32..58.
Overshooting 50 slightly is expected. Models exceed word budgets, which
is why builder.length_hint already carries a buffer for the same reason.
What this does not show: the run used a Claude model, and the app talks
to an OpenAI-compatible endpoint whose weaker models are why
worldstate/parse.py tolerates trailing commas. The prompt is followable
and the brief supplies the missing information; a weaker model is not
proven to comply as well.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
`_create_due_memories` sent six actions of second-person prose and
nothing else — no protagonist, no cast, no setting, and no instruction
about what person to write in. So for `You push the door open. She grabs
your arm.` the only honest memory was "You entered a room and she
stopped you", which names nobody when it is retrieved forty turns later.
The framing wandered too: with no rule, the model picked a person per
call, and one bank ended up holding "You entered the crypt", "The player
entered the crypt" and "He entered the crypt" for the same kind of
event.
Both prompts now carry a cast brief and a framing rule: third person,
the protagonist by name, other characters named rather than left as bare
pronouns. The rule states its reason, because a memory really is read in
isolation much later and a model told why complies far more
consistently than one handed a bare instruction.
The cast comes from the story cards, not from `stat_schema`. Every
schema NPC is already turned into a card at adventure creation,
deduplicated against the hand-written ones by name, so the cards cover
schema NPCs, an author's own cards, and an adventure with no RPG layer
at all through one path instead of three.
Keyword matching alone was not enough, and finding that out changed the
design. Built that way first, the brief for "She grabs your arm" listed
the protagonist and nobody else: the block that most needs a cast is
exactly the one written in bare pronouns, and Gwen's trigger keys
include "her" but the text says "she". So matched cards come first and
the remaining slots are filled with the other character cards. Places
and items are not topped up — an unmentioned tavern is not who "she"
was — though a place that is mentioned still matches normally. The
asymmetry with the turn prompt is deliberate: an untriggered card is
wrong as lore and right in a roster, because the roster answers "who
could these pronouns be" rather than "what is on stage".
Fixed descriptions only, never live values. `Gwen: trust 40 (wary)` in
the brief would make the same event summarized at two different times
come out framed differently, which is the fault this removes.
An adventure with no persona still gets the cast and the setting, and
the model is told to write "the player". One with nothing to say sends
byte for byte the prompt it sent before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
Chromium against a fresh database with the demo scenarios seeded. No API
key needed: Insights assembles the prompt without calling a model, so the
checks run on the real assembled context rather than a stub.
21/21. The two request failures in the run were the sandbox — Google
Fonts is blocked by the egress policy, and the analytics beacon is
aborted on unload — not the app.
The case worth naming: a blank adventure with no RPG layer produced
sections `['narrator', 'persona', 'length_hint']`. That is what the
persona was added for, and it is the one thing no unit test in this repo
would have caught if the section had been gated behind `has_ws`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
The player had stats but no identity. `stat_schema.player` carried hp and
mana beside `npc.gwen.trust`, but where an NPC has a name and a
description the player had neither, so the block rendered as
`You: hp 100/100` and nothing in the prompt said who "you" was.
Three columns on `adventures`: name, pronouns, description. All
user-only, all optional, and an empty name means the app behaves exactly
as it did before — no backfill, no special case for an adventure that
predates the migration.
They are adventure columns rather than part of `stat_schema` for two
reasons. An adventure with no RPG layer still has a protagonist, and
that is the case this was added for. And `worldstate.schema._initials`
treats every dict inside a stat section as a stat definition, so a
persona placed there would be instantiated, rendered in the guide, and
handed an `initial` value as though it were one.
The paths do not change. `player.hp` stays `player.hp`; only the label
moves, to `Kaelen (player): hp 100/100`, the same way NPC lines already
print a display name beside the id. A path carrying the persona's name
would break the moment a player renamed their character, because
`_history_text` replays every past turn's stored delta into the prompt
and those blobs hold literal `player.hp` strings.
The section sits in the system block. Only the user can edit it, so it
never changes mid-story and stays inside the cached prefix. That is what
makes it free, and it is why the AI must not be able to move it — a
delta aimed at `persona.*` is already refused by `_resolve`, and there
is now a test holding that in place.
The modal that used to appear only for scenarios with `${Placeholder}`
tokens now always opens, and is where the character is named. Persona
and placeholders stay independent: a scenario asking for `${Name}` is
asking its own question. No scenario in the repo uses placeholders at
all, so the overlap is hypothetical.
Phase 2, which feeds the persona and the cast to the summarizer, is
written up in plan/18 and not started. That is where the memory-quality
problem actually gets fixed; this change is what gives it a name to use.
Not yet driven in a browser — plan/18 lists what to check by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPyQN926gkZTAYfgugcaok
Migrations 66 to 73 drop `actions.index`, `variants`, `variant_index`,
`variant_count`, `state_before`, and `world_state_before`, plus
`adventures.memory_cursor` and `summary_cursor`. `index` is a keyword in
SQLite, so migration 71 quotes it.
Nothing outside the migrations read these. `models.py`, `schemas.py`, and
`ACTION_LIST_COLUMNS` lose the same eight fields, `Adventure.actions` orders
by `id`, and `attempts.renumber`, `context.history.max_action_index`, and
`nodes.next_index` are deleted.
Two changes keep the migration replayable on a `create_all` database:
- `_split_variants_into_siblings` wrote through the live ORM table, so it
stopped compiling once migration 66 removed five of its columns. It now
writes through `_ACTIONS_AT_60`, a frozen `Table` with its own `MetaData`.
- Five data passes read columns these migrations drop. Each now calls
`_has_columns` and returns early when the columns are absent.
`bootstrap` takes a `through` version so a migration test can stop at the
schema it asserts on.
555 tests pass, up from 549. Eight of the new cases assert each column is
gone after a real schema-45 database migrates all the way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198qDK3gmgSo7EtQ4GTPqqK
`frontend/src/pages/Play.jsx` was 2280 lines holding 27 components. It is now
`frontend/src/pages/Play/`: `index.jsx` with the page component, five panels,
two drawers, the reports, the take pager, the refresh dialog, and the shared
formatting helpers.
Every line moved verbatim. A coverage check confirms every non-blank line of
the original appears exactly once, in order, across the twelve files, and a
name-resolution check confirms every identifier each file references is defined
or imported there, with no unused imports.
The line split stranded a comment at six of the boundaries. A leading comment
sits above the section it describes, so each cut left one at the end of the file
before it. All six moved to the section they describe, rewritten in the house
style.
`usePlaySession.js` is not here. The page component still owns all of the
session state. Moving eighteen `useState` calls and seven `useEffect` calls is a
rewrite rather than a move, and no frontend test would catch a mistake in it
today, so it waits for Stage 5.
Four stand-in providers under `backend/tools/` now define `last_usage = None`.
The turn engine reads that attribute after every call, and the fixtures never
defined it, so `tools.tree_fixture` crashed. That break predates this branch.
Verified by 549 passing tests, a clean `npm run lint` and `npm run build`, and
by driving the Play screen: the story view, all five panels, both drawers
including the world-state edit form, the branch map, the refresh dialog, and the
take pager stepping onto a take that lives on another branch. No console errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Dix4oGV3njgWRdu7P9t6r
`backend/app/routers/adventures.py` held 2353 lines and 35 endpoints. It is now
a package of 14 modules, the largest 443 lines.
The split moves text rather than rewriting it. An AST comparison against the
old file confirms all 86 definitions are identical, and the OpenAPI schema
still lists the same 35 operations.
Names a test replaces now live in `turns.py` only, and other modules reach them
as `turns.<name>`. Rebinding a re-exported alias changes the alias and leaves
every caller reading the original, so the package root does not re-export them.
A patch aimed at the old target raises `AttributeError` instead of passing while
doing nothing. Tests and the fixtures in `backend/tools/` say
`adventures.turns.<name>`.
The same rule keeps the turn lock working. One module owns `_active_turns`, so
one lock guards one set.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Dix4oGV3njgWRdu7P9t6r
Phase 17 splits the four files that hold most of the code, finishes the
schema migration SP8 left half done, and stops the published guide from
drifting away from its Markdown source. `plan/17-refactor.md` carries the
plan and the progress table, and `plan/STATUS.md` points at it.
Stage 0 is hygiene only. Both abandoned worktrees are gone, which freed
about 104 MB. Removing `sp7-tree-ui` needed one extra step: a Vite dev
server had been running out of it since 2026-08-18, holding
`frontend/.vite` open and owning port 5173, and serving a tree 54 commits
behind `main`. The three stale `.db` files are deleted; `data.db` is not.
`AIDND_TRUSTED_PROXY_HOPS` is now documented. It was read at `limits.py:55`
and named in no `.env.example`, README, or blueprint. It sets how many
proxy hops the rate limiter trusts in `X-Forwarded-For`, so a deployment
that adds a hop without setting it gets the bucket-rotation bypass back.
The 19 squash-landed branches are still there. `git branch -D` is blocked
by the permission classifier; the verified command is in the plan file.
549 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Dix4oGV3njgWRdu7P9t6r
`previous_titles` stops a rename stranding the row it left behind, but the rows
already stranded still had to be deleted by hand on every deployment. The
seeder now removes them on the next boot, which retires the stale "Road to the
Champion" demo without a database console.
Only rows with a NULL owner and `is_public` are considered, and a player's own
scenario is neither, so nothing anybody created is reachable. An adventure
started from a deleted demo survives: `adventures.scenario_id` is `ON DELETE
SET NULL`, and the adventure holds its own copies of the cards and scripts, so
it loses only the inherited cover art.
Two cases skip the sweep, because neither is an instruction to remove live
content: a seed file that fails to parse claims no title, and an empty seed
directory reads as a packaging failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF
`backend/tools/claude_shim.py` serves an OpenAI-compatible endpoint backed by
the local `claude` command line tool, so a demo can be played against a real
model with no API key. Each request spawns one `claude --print` process, which
suits the turn engine: the app assembles the whole prompt every turn and
expects a stateless endpoint.
Playing the Pokemon demo through it made all five of the world-state changes in
`plan/16` work, and the refusal loop ran end to end for the first time: turn 3
clamped to nothing, turn 4's assembled prompt carried the correction verbatim,
and the model's next delta was right. Two failures previously blamed on the
code were the demo model. One is a schema fault and is still open: Milo's three
Pokemon share one `active_hp` stat, so a switch leaves the newcomer at 0 HP.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF
The model sent a total rather than a change for almost every number:
`npc.ivysaur.hp: 96`, `npc.milo.active_hp: 88`, `player.potions: 2`. Because
every `hp` starts at its maximum, each one clamped back to where it started, and
the potion count rose when the player spent one.
`EMIT_RULE` does say "deltas (not new totals)", but the scenario contradicted it
at closer range. `milo.active_hp`'s description said "Reset this to the
newcomer's full HP", which asks for an absolute and is injected every turn. The
bullets said "drop the HP, and raise it when healed", naming a direction but no
sign. Five HP descriptions said only whose HP it was. The one line that said
"not a delta" covered `player.active_pokemon`, so naming the exception made the
rule look optional. `pokemon_fainted` is the control: its description says "add
1 each time", and it is the only number that behaved.
Every stat description now states the sign, and the lead-in gives a worked
example. `milo.active_hp.max_delta_per_turn` goes 65 to 98, because a switch
moves that stat a full bar from 0 and the old cap made the reset unreachable in
one turn.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF
`plan/STATUS.md` pointed at SP9 as the next step and called it unmerged and
undeployed. SP9 and SP10 are both on `main`. They landed as squash merges, so
`git branch --no-merged` still lists their branches. The doc now points at SP8
and says to check `actions.parent_id` and commit `c0cd6fa` instead of the
branch list.
`plan/15-pokemon-demo-handover.md` named a cause for each of two bugs and both
causes were wrong. It carries a banner and inline corrections rather than a
rewrite, because the reasoning trap it fell into is worth keeping: the visible
evidence was "the number did not move", which reads as the model never trying.
`plan/16-world-state-refusals.md` records what the engine was actually doing,
the five changes, and a browser checklist. None of the UI work has been driven
in a browser.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF
The section was written before the commit and said "not committed, not
deployed", which stopped being true one commit ago. It now names 041f9e2 and
the deploy.
The item that outlives the commit gets its own paragraph:
`AIDND_ANALYTICS_EMAILS` is `sync: false`, so nothing in the repo can set it
and the dashboard stays invisible to everybody until it is filled in by hand
in the Render dashboard. Worth saying plainly that collection runs regardless
— the counters fill either way, so setting it late costs no data, which is the
thing that decides whether this is urgent.
The gate paragraph below it loses its closing sentence, which said the same
thing in the same words.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
A hosted demo raises a question a local app never does: is anyone using it,
and do they reach the part that matters? `/analytics` answers it — visitors,
pages, referrers, countries, devices, which shared scenarios get played, turns
and demo-key spend, API and turn errors, and a funnel from visited to played a
turn to signed up.
Not a third-party script, for reasons specific to this one. The CSP allows
`script-src 'self'`, so a tracker means loosening it; adblockers eat the
popular ones, which silently biases exactly the technical audience this
project gets shown to; and none of them can see the measurement that actually
matters here, which is a turn, not a pageview.
**A visit is a write and never a read.** After the 189x egress fix it would be
perverse to add a feature that reads rows per request, so counts accumulate in
a process-local dict and flush every 60s as UPSERTs. Storage is a generic
`(day, metric, label) -> hits` counter, so measuring something new later costs
a constant rather than a migration, plus one row per visitor per day for the
funnel flags. Every dashboard query is a GROUP BY returning tens of rows
however much traffic sits behind it; a month reads back in a few kilobytes.
The buffer's cost is that a hard restart can lose up to a minute — the flusher
also runs on shutdown, and a tier that sleeps when idle sleeps on an empty
buffer anyway.
**The counters are anonymous; the access log beside them is not, on purpose.**
A visitor is `HMAC(secret, "visitor:<user id>")` truncated to 32 chars —
one-way, so `analytics_daily` and `analytics_visitor_days` cannot be joined
back to `users`, and keyed, so no client can compute one. Story content never
reaches that module, and the only content it ever names is a seeded public
scenario's title; a player's own titles are theirs. `accesslog.py` is the
identifying half and is a separate module writing a separate table so that
separation is a property of the code rather than a convention: `access_events`
records sessions, sign-ins, registrations and failed attempts with address,
email and device, read on a second tab of the same page behind the same gate.
Both halves are gated on `AIDND_ANALYTICS_EMAILS`, not `POWER_USERS`. An
unmetered tester is not automatically someone who should see the traffic. The
route 404s and the nav link is absent for everyone else, the same treatment
AI Chat gets; unset in a hosted deploy means nobody sees it, including me.
Three things came out of building it that a test would not have suggested.
**A failed turn is an HTTP 200 with a bad ending.** The status-code middleware
cannot see one, so a demo whose model had started refusing every request would
look perfectly healthy from outside. All five SSE error paths in
`_generate_turn` now go through a `turn_error()` helper that counts on the way
out. Error buckets elsewhere are labelled by the matched route template rather
than the requested path — one bucket per endpoint instead of one per adventure
id, and, the reason it isn't merely tidier, an unmatched path is entirely
attacker-chosen, so labelling by it would let anyone mint rows.
**The funnel counts people, not clicks.** A player who starts six adventures
is one person who started an adventure. That is the whole reason the
per-visitor-day table exists; its flags only ever turn on, and `is_new` is
settled by the first write of a visitor's first day.
**The tests run on SQLite and production is Neon.** A flush that raises is
caught and logged, so a dialect mistake in the UPSERTs would have stayed
invisible until the dashboard quietly never filled.
`test_the_upserts_compile_for_postgres` compiles both statements against the
Postgres dialect without connecting to one.
Two things this leans on elsewhere. `limits._client_ip` is now public
`client_ip`: the access log needs the same answer, and two functions both
deciding which hop is the caller's is how one of them ends up trusting a
header it shouldn't. And the cleanup sweeper now starts if *either* job has
work — a deployment can keep every guest forever and still want its
visitor-day rows aged out.
No migration. Both tables are new and `bootstrap()` calls `create_all` on
existing databases too, the route `branches` took in Phase 14, so
`LATEST_VERSION` is still 64.
497 tests green, frontend lint and build clean, driven by hand against a
synthetic 90-day fixture at 1568px. The narrow-screen layout follows the
existing 720px block but is unverified: `resize_window` is ignored on a
maximized Chrome and `frame-ancestors 'none'` rules out checking it in a sized
iframe. Also repaired here: a rename in test_ratelimit_hardening.py had run
through the test names themselves, leaving `testclient_ip_*` — still collected
by pytest, which is why it passed unnoticed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
The README, the project page and the engineering guide all describe a linear
story. The tree shipped two days ago. Every published surface is a phase
behind, and the guide is not merely behind — it is wrong in a way that costs a
reader time.
Its 2.2 was "Two coordinate systems, and the bug class they create", and it
explained the codebase through position_of_index, note_action_removed and
settled_story_actions. All three were deleted in SP3. 2.3 explained retry
through Action.variants and state_before. Somebody reading either would go
looking for machinery that is not there, which is worse than a gap.
So 2.2 is now "The story is a tree", written at the depth 1.2 and 1.3 are
written at: the seven bugs that turned out to be one bug, the lineage clause
and the two properties that make fork count free, why takes group by parent_id
rather than by coordinate, cursors becoming anchors, and a closing list of what
the design is honest about. 2.3 is rewritten around state_after and takes, and
1.1 and 1.5 follow, because the pipeline no longer snapshots before the call
and the memory bank no longer holds an action back.
The numbers were simply old: 151 tests where there are 440, 37 migrations where
there are 64, twelve phases where there are fourteen. They appear in four
places across the README, the project page's stat tiles and the guide's results
table. The measured branch cost — 103 B, and 1.007x the page load of the same
story flat — is added beside the egress and turn-cost figures it belongs with,
since it is the number that answers "what does branching cost me".
Three screenshots, on a new tools/shots_fixture.py: the Bandit Camp demo driven
through eight written turns with written deltas, three discarded takes forked
onto branches of their own, one off a branch so the map has to nest. Same
reason tree_fixture.py is committed — the shots have to be reproducible and the
frontend still has no test runner. play-world-state.jpg is reshot because it
predates the entire tree UI; the map and the branches panel are new.
Note for next time: docs/guide.html is hand-written, not generated from the
Markdown, so every guide edit is two edits in two vocabularies. Both files were
checked for tag balance and both pages rendered locally before this landed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
The Branches panel says which lines exist. It cannot say where they parted
or how much story each one is, because those are the two numbers `GET
/branches` already answers and a list has nowhere to put them. A ⌗ See the
tree button opens a map: one horizontal lane per branch, running from the
moment it left its parent to the moment it ends, joined to the parent by an
elbow at the fork. The horizontal axis is the story's own clock, so two
lanes at the same x are at the same moment and a short branch reads as
short.
This is a branch map, not the per-node map SP7 refused, and that is the
whole reason it was cheap. Lanes are bounded by branch count, not node
count, so the 600-node windowing problem never arrives. It reads the single
request the rail already made and nothing else.
`branches.js` holds the tree maths and `BranchMap.jsx` the drawing. Play.jsx
gives up its private copies of branchLabel and orderBranches: the panel and
the map now label and order a branch through the same functions, so a branch
cannot be called two things by the two views. The three operations stay in
BranchPanel and are passed down, and `run` answers whether it worked so
neither view clears a half-typed name on a refusal.
Three things came out of driving it, none of which a test could have seen.
`clientWidth` counts the canvas padding the ResizeObserver leaves out, so
the first paint drew an svg 24px wider than its box — and the observer's
initial observation never arrived here, so dropping the seed left the map
never drawing at all. Both are needed and the seed subtracts the padding.
Only the name was being clipped, not the meta line under it, so a late fork
ran its text off the right edge; both are clipped now, and a lane starting
in the right third hangs its labels back over the fork, where its own band
guarantees nothing to collide with. And the delete rule lived in the server
and in the map but not in the list, which offered Delete on a branch the
head was forked from and answered with a toast from the server's 400.
`headLineage` is the client's copy of that rule and both views use it; the
server stays the authority.
`tools/tree_fixture.py` is the counterpart to `tools/branch_fixture.py` —
four branches at three fork depths, one forked off a fork. With no frontend
test runner it is the whole of the map's coverage, and it exists to be
looked at. Nothing in the backend changed; the 440 tests pass unmoved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
The handover said SP9 still needed a person clicking it. It has had one, and
the two bugs that came out are the point: both were in the gap between the
suite and the screen, and the frontend still has no test runner to close it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
The 11 MB the vacuum gave back against a mid-hundreds guess, and the reason:
the heap is 1.7 MB and the rest is TOAST, so a migration touching small
columns bloats the heap and reuses the toast pointers. The rule keeps its
cost estimate rather than its size, and SP8 is the other shape -- it drops
toasted columns, and DROP COLUMN frees nothing until a VACUUM FULL.
And the handover moves to SP9: what a hand-driven SP7 turned out to be wrong
about, the three findings from fixing it, and the fact that the fix itself has
not been driven by hand either.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
SP7 shipped a tree nobody could use. Driving it by hand found three things,
and the schema is what the third one needs.
A take forked onto its own branch leaves the coordinate its siblings are still
at. `attempts.group` filtered on (branch_id, depth), so that take read as the
only one of its turn -- 1/1 where the player is owed 1/3, with the other takes
unreachable from the line they were taken on. Nesting has the same shape from
the other side: takes under C1 and takes under C2 share a depth, and only the
parent says a pager under C2 reads 2/2 rather than counting C1's three too.
So `actions.parent_id`, and `group` keys on it. The alternative -- pointing a
branch's fork at a node instead of a depth, so a promoted take never moves --
was rejected: `lineage` exists so a read is an OR-clause per branch rather
than a walk up parent pointers, and moving the fork point changes path
resolution itself, which drags in cursors, memory depths and both bundle
formats. This column is read to group takes and for nothing else. One indexed
lookup, never a walk, and no read of the story changes.
Two writers had to learn it. `add_attempt` places a sibling by hand rather
than through `place_action`, and copies the parent, because a take belongs to
the turn it is a take *of*. `place_action` resolves it from the path for a
genuinely new node, which is the honest default.
And one reader had to be kept out of it. `delete_turn` meant "every attempt at
this coordinate"; the group spans branches now, so undoing a turn would have
deleted a take that another branch is telling. `attempts.on_branch` scopes it
back.
409 tests, unchanged from main and green -- a linear story has one take per
parent, so none of this is reachable until a second one exists. The migration
adds one column, backfills the linear case, and leaves NULL where it cannot
honestly place a row; `group` falls back to the coordinate there, which is the
rule those rows were written under. One `VACUUM FULL actions;` owed after
deploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
STATUS still described SP7 as the newest thing and counted 396 tests. The
stack is open as PR #6 and the review pass is answered, so the pick-up
paragraph now says both, and part five records what the review found.
The part worth writing down is the finding that was turned down rather than
the eight that were fixed: a memory anchored to a node is meant to go when the
node goes, the root is the one exception, and the reason the exception is
narrow — protecting depth 0 outright would rebuild the dangling rows
forget_node exists to prevent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
Nine findings from a review of the phase-14 stack. The one about a retry
withdrawing a memory is not a bug — a memory anchored to a node describes that
node, and it goes when the node goes. The root is the exception, and it is the
only one: migration 62 parked every memory written before memories had
coordinates on depth 0, so withdrawing the opening node would retire a whole
bank nobody attached there. A memory with no source range covers no story and
now stays; a summary that genuinely ends there is still withdrawn.
The rest are repairs.
* The adventure list quoted whichever attempt was written last rather than the
one the story tells, so switching back left the index disagreeing with the
page.
* A v1 import gave a typed memory no depth, rebuilding the NULL the migration
exists to remove — invisible until the imported adventure forked.
* The action cap counted a v1 file's turns, and a turn expands into a row per
saved attempt, so a file inside the cap could write a multiple of it.
* Forking a live node on a borrowed ancestor promoted a sibling on a branch the
caller never named. It is a branch switch, and now says so.
* Switching attempts left the state, status and memory panels reading the
previous take: the story does not change length, so nothing keyed on its
length noticed. Same class as the branch-switch bug this phase already fixed.
* A retry after switching back numbered the new attempt into the middle of the
group instead of the end.
* The cursor backfill numbered every action in the table once per adventure;
correlated to the adventure being updated, it is an index lookup instead.
* Renaming a branch answered own_actions=0.
And one behaviour change recorded rather than repaired: script-visible history
and actionCount no longer count blank-text rows. That is the right shape and
there is no reading compatible with both, so plan/14 says so.
409 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
A hand-written memory used to carry a NULL depth, described in the model as
"belongs to the adventure rather than to a path". That sounds harmless and
is not: a NULL is a coordinate no fork can cap, so a note typed on one line
followed the reader onto branches whose story it never described. It takes
the head now — the story you were reading when you wrote it — and obeys
exactly the rule a summarised memory obeys.
The unanchored escape clause in lineage.Path.clause existed for that single
case and is deleted rather than left unused. Its docstring argued that a
capped depth would drop a typed memory the moment its branch stopped being
the newest entry; anchoring answers the same worry better, because the
memory is not exempt from the path, it is on one.
The drawer now shows the path being read and nothing else, filtered by the
clause retrieval itself uses, so the bank you can see is the bank the model
can see. Nothing is stranded: a memory lives on a branch, switching to that
branch shows it, and deleting the branch deletes it. Pinning decides order,
the path decides existence.
Migration 62 lands existing NULL-depth memories at depth 0 of their branch
rather than at the tip. 0 is at or before every fork point, so every memory
stays visible from exactly the paths it is visible from today — nobody's
bank loses a row on deploy. The tip is the tidier-sounding choice and would
have emptied them out of every branch forked earlier than they were typed.
This supersedes the on_path flag and the "another branch" badge from
earlier today; anchoring makes them redundant, and they are removed.
Four tests changed because they asserted the old contract, not because
they broke. The one worth reading is the pair replacing
test_a_hand_written_memory_is_not_lost_at_the_first_fork: typed on shared
trunk it still survives a fork, and typed on ground the fork never
travelled it no longer follows you.
402 tests. Verified on tools/branch_fixture.py: each branch's drawer holds
its own memory and not the other's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
Retrieval has been path-scoped since SP3: a memory on a branch this story
never travelled is never sent to the AI. The drawer listed every memory
alike, so on a fork you read "Fell down the cellar stairs; badly hurt" and
reasonably concluded the model knew it. It does not. That is worse than
either hiding the row or retrieving it — it is the screen claiming
something the engine contradicts.
Hiding them is not the answer either; that was the original comment's
point, and it stands. A memory nobody can list is a memory nobody can
delete, in a phase whose rule is that nothing is removed automatically.
So the whole bank still lists, and the rows off the current path are set
back, dashed, and labelled "another branch". MemoryOut.on_path carries it,
computed from the predicate retrieval itself uses rather than a second
spelling of the same idea — two spellings drift, and the failure mode here
is a badge that says the opposite of what the model gets. Pinning does not
override it: the path clause runs before pinning is considered.
The relationship is asymmetric and there is now a test that says so. A fork
borrows its ancestors, so a memory written on the parent is on the fork's
path too; the reverse never is. Worth pinning before somebody makes it
symmetric on the grounds that it looks wrong.
399 tests, three new, egress ceilings intact — the flag costs one id-only
query. Verified in a browser on tools/branch_fixture.py.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
A branch switch does not change the length of the story. It changes which
story it is. Four panels keyed on actions.length and so could not tell the
difference: the Branches panel drew one branch while the reader was already
on a second, Insights showed the prompt built for the path just left, the
script-state drawer kept the other line's numbers, and the Memory Bank did
not notice a deleted branch taking its memories with it.
Only the world-state drawer was right, and only because it happened to
carry stateKey already. They all key on the pair now, and deleting a branch
bumps it too — that is the one operation that changes what is stored
without a turn being played and without the story on the current path
moving by a single action.
tools/branch_fixture.py is the thing that could show it. The stress
fixture's world state is empty, so it cannot answer whether a switch puts
the scoreboard back, and its story is one branch. This builds a small
bootable adventure with a stat schema, a gold script, two takes on one turn
that differ by 35 hit points, a fork, and a memory on each side — with both
branches the same length on purpose, because equal length is precisely the
case a length-based key cannot see.
Verified in a browser with the drawer open: hp 60 to 95 and back, the bar
redrawn, the story swapped to the other take, Insights carrying the scratch
and not the beating. The Memory Bank deliberately does not change on a
switch: the drawer is adventure-wide so a memory is always findable to
delete, and retrieval is the path-scoped half.
396 tests, build clean, no new lint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
The pager could only step between attempts, and stepping has nothing to say
about the thing the tree exists for: taking a path the story moved past and
keeping both. Every attempt has been its own node since SP4, so a chip is a
node now, and "take this path" forks — or simply switches, when the turn is
still the tip and its attempts are leaves nobody has built on.
Beside it, a Branches panel: every line the story has taken, with where each
left its parent and where it ends, and switch, rename and delete-with-confirm.
It sits with Plot/Memory/Scripts/Insights rather than inventing a new place to
put a rail. An unnamed branch is drawn from its fork depth, never from its
position in the list — a position shifts the moment a branch above it goes.
A spatial per-node map was considered and deliberately not built. At the size
this has to be verified against it is a second windowing problem, and it can be
added later without a new endpoint, since the rail and a map read the same
GET /branches. VariantOut grows an id because a fork is addressed by the node
being taken, not by an ordinal in a group that renumbers.
Driven by hand against the 602-action fixture, which found one bug that no test
could: the panel refreshed on actions.length, and a fork swaps a 60-action
window for another 60-action window, so it went on drawing a one-branch tree
while the story was already on the second. It keys off the counter adoptWindow
bumps now.
The scroll path was driven at the same time — three prepends of ~16,200 px, the
same node holding viewport top 792 to 787, never thrown to the end. That closes
the standing gap in this project. Console clean.
396 tests, build clean, no new lint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
A bundle had one list and a forked adventure has two stories, so export was
emitting every branch's turns interleaved by index — a mangled story rather
than lost data, and unreachable only because forking has no UI yet.
`ai-dnd-adventure-v2` carries the branches, the depth each one left its
parent at, which attempt at every turn is the story, and what each node
left behind. That last one is not decoration: the after-snapshots are what
a branch switch puts back, and a bundle without them imports a tree nobody
can switch inside.
`app/bundle.py` owns both formats and nothing else knows either. The v1
reader stays — those files are already on people's disks — and it is now
the only place a `variants` array exists anywhere.
The rule the module is built on is that a bundle carries what was chosen
and never what is derived. The head branch, the fork points, the live flags
and the anchors are decisions somebody made. The lineage, the head depth,
the legacy `index` and the variant ordinals are computed from those and are
rebuilt on the way in, because a bundle is a text file anybody can edit and
a derived field shipped beside its source is a chance for the file to
disagree with itself where no read would report it.
`index` is the one that stops being academic here. It agreed with `depth`
until SP5, and this is the first writer that has to fill it for a forked
story, where two branches both hold a node at depth 4. It is allocated one
per turn instead: siblings share it, no two coordinates do.
Everything a hand-edited file can get wrong about the shape of a tree is a
400 raised before the adventure row exists, because a half-applied import
is exactly the failure this phase exists to end — a story that goes quiet.
A file wrong about which attempt is live is corrected rather than refused;
that is an invariant of the database, not of the format.
Measured on the 600-action fixture: 587 kB to 911 kB, and all of the
increase is the outcomes at 489 B a node — the coordinates themselves save
57.5 B a node against the old turn-and-variants shape. Twenty forks add
660 B. 4.3% of the import body cap.
381 tests green, 16 of them new in test_bundle_v2.py. No migration, no
vacuum owed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
SP4 and SP5 are written down: what shipped, what it measured, and the
handful of things worth not rediscovering. The bug table at the top is
scored now that six of its seven rows are gone — and the seventh, the
one-turn holdback, is gone for a different reason than the one written
there, which is the correction that matters most. Editing a summarised
action is still unfixed and now says so; no subphase is scheduled for it.
The Open section closes. `retry_of.index` was the last item and SP4
answered it by reusing the retried node's depth.
Two vacuums are owed, SP1's and SP4's, and neither has deployed. One run
after the SP4 deploy settles both. SP8 grows two more columns to drop and
a caveat about which ones are not dead yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015H5qiyiR7gtFQaoDphHZ3g
The memory bank and the story summary each kept a cursor: how many story
actions they had already covered. A count is a position in a list, and this
list moves — delete an action in front of the mark and every later one slides
down a slot, so the mark now covers one it has never read. All the cursor
bookkeeping existed to patch that up.
Both marks are now (branch_id, depth): the node up to and including which the
work is done. A depth is a coordinate along a path, not an offset into a list,
so nothing in front of it can move it. That deletes rather than rewrites
`position_of_index`, `note_action_removed`, `_rewind_cursors_to_index`,
`prune_dangling_memories` and the every-pass clamp in `run_post_turn`.
A memory hangs off the node its block ends on, so a fork inherits its
ancestors' memories without copying any, and retrieval selects through the
branch clause over the *whole* lineage — recall is long-range by definition and
cannot be windowed. Measured: 1,807 B on a story forked twenty times against
1,823 B on a flat one of the same length.
Migrations 53-56 translate the old counts into nodes. They rewrite `adventures`
and not `actions`, so this one needs no VACUUM FULL.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Trust the statement count, not the stopwatch: this machine's suite timings
drift about 20% between runs, so the branch-read regression is recorded as
201 SELECTs -> 2 rather than as seconds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
The identity map holds weak references. A branch row nobody keeps a strong
reference to is collected between two nodes, so resolving the head inside the
placement loop read it back from the database for every node in the flush: 201
SELECTs on `branches` to write 200 actions, and 36 s -> 45 s on the same 297
tests. Nothing about any result changed, which is why only a stopwatch found
it, and why there is now a test counting the reads.
Also: the two reads left un-pathed on purpose say so where they live —
`max_action_index` allocates the legacy `index` and must stay adventure-wide
or two branches issue the same number, and export is a flat v1 bundle whose
reader has no idea branches exist. And `Adventure.actions` keeps its `index`
ordering, because ordering the collection by depth would not make it a story:
it is every branch's actions, and a path is a selection out of it.
318 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Phase 14 SP1. The tree goes into the schema and nothing reads it yet: a
`branches` table, `branch_id`/`depth` on actions and memories, a head pointer
on adventures, migrations 46-52, and a server-side backfill that re-reads every
existing adventure as a tree with one branch. `depth` holds the number `index`
already held, gaps included, so no story changes — a linear story *is* a tree
with one branch, which is what makes the SP0 baseline passing unmodified the
pass condition rather than a hope.
The writer had to come with it. No migration will ever visit a row written
after it ran, so columns backfilled today and populated next subphase would
leave a hole exactly the width of one deploy, and from SP2 on a row without a
branch is a row no read can see. `app/tree.py` owns that: one module, because a
node written without a branch fails by disappearing rather than by raising.
Three things the schema itself insisted on:
- `adventures.head_branch_id` is a plain integer, not a foreign key. Pointing
both ways makes the two tables a cycle create_all cannot order, and its
escape hatch needs an ALTER SQLite does not have. It is a cache, and a head
naming a branch that is gone recovers onto the root.
- `lineage` is NOT NULL, so the backfill inserts `'[]'` and fills it in a
second pass guarded on `json_array_length(lineage) = 0` — not `= '[]'`,
because Postgres `json` has no equality operator.
- SQLite will not drop a column a foreign key names, which is how two existing
tests broke: they simulated an old database by rewinding the stamp while
leaving the new columns in place. Every ADD COLUMN migration is now
idempotent, and `tests/test_tree_migration.py` builds a genuine schema 45 by
rebuilding three tables from frozen DDL so the real ALTERs run.
297 tests green, 14 of them new. `branches` costs 0.1 kB of a 733.5 kB turn;
page load and index are byte-identical to the recorded figures.
The deploy that ships this needs one `VACUUM FULL actions;` on the direct
endpoint afterwards — it rewrites every row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
The stress fixture is sized from production and exists to weigh bytes, so it
leaves every column it does not weigh at its default. Checked against a freshly
built one, that is exactly the set phase 14 has to migrate: state_before and
world_state_before NULL on all 600 rows, no scenario and so no RPG layer, no
adventure scripts, both cursors 0, and a hundred retry histories whose two
attempts carry byte-identical text with variant_index pinned at 0.
That last one matters most. "Which attempt is live?" is the question SP4's
migration answers when it decides which sibling node becomes the head of a
turn, and against that fixture the question had no observable answer -- a
migration that picked wrong would have looked correct.
--rich fills in those columns and no others. A real RPG scenario read from the
seed data rather than invented, with a world state played forward so hp
declines and flags flip; monotonic per-action state snapshots, so a rollback
that does not happen reads as a wrong number instead of as nothing; a gold
script, story cards, non-zero cursors, pinned and forgotten memories; retry
attempts with distinct texts, counts of two and three, and a live attempt that
is often not the last written. Plus a second adventure, because a branch clause
that forgot its adventure still looks right on a database holding one.
The invariant SP4 reads -- text mirrors variants[variant_index] -- is asserted
at build time rather than assumed.
The plain fixture is untouched and re-measured unchanged at 1.8 kB, so the
egress ceilings stay comparable. All six shapes run clean on --rich, including
the turn path with the RPG layer and script pipeline now live. 283 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Phase 14 replaces the action list with a tree, which touches the memory bank,
the context builder, undo, retry and the UI at once. Before any of that moves,
write down what "unaffected" means and prove it holds today.
tests/test_story_tree_baseline.py drives the product over HTTP the way a player
does and asserts only on API responses, never on how anything is stored: the
turn engine, retry and variant switching, undo, paging all the way back to the
start without gaps or repeats, edit/delete, memories, export/import round-trip,
world state. 283 green with it added, from 259.
It must pass unmodified through the schema change, the branch clause and the
move of memories onto nodes. A linear story is a tree with one branch, so those
subphases have no licence to change behaviour, and this is what says so.
The plan itself records four decisions that were open: structural-first with no
runtime flag, legacy columns kept one release, full tree visualisation, and a
v2 bundle with a v1 reader. Plus four traps found by reading rather than
running -- history._from_memory and the scripting pipeline both slice
adventure.actions, which under a tree is every branch rather than the path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
The harness already built a production-shaped 600-action adventure and then
threw it away with the temp file. The one open gap in plan/13 is that nothing
has ever driven the scroll in a browser, and part of why is that there was
never a long adventure to drive it with.
--keep PATH writes the fixture somewhere durable and makes the app able to
serve it. Two edits are needed for that, both of which cost an hour to
rediscover:
- create_all() builds the current schema but leaves the version stamp at its
default, and bootstrap() reads a populated-but-unstamped database as
ancient — it replays every migration against a schema that already has the
columns, and fails on the first.
- the fixture's user is a registered one, but local mode looks for the row
with email IS NULL and is_guest false, so without clearing the email the
app opens on an empty library.
--keep is read before argparse exists, because where the database lives has to
be settled before app.database is imported. That is the same constraint the
AIDND_STRESS_DATABASE_URL block already lives under. SQLite only; combining it
with a Postgres target is rejected rather than half-honoured.
Verified end to end: the fixture boots with no manual step, action_count 600,
a 60-action first payload, and before_id walks back nine more pages to the
start. Nothing about the default path changed; 259 tests pass.
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The post-vacuum sizes had never been read. They were 144.2 MB - above the
99.6 MB the compression work started from, not the ~53 MB it projected. The
earlier VACUUM FULL ran before migrations 42-45, which then rewrote every row
and refilled the table with dead space that only another VACUUM FULL returns.
Ran it: 144.2 -> 65.0 MB, 79.2 MB reclaimed in 5.5s, health green after.
actions is now 52.1 MB holding 50.4 MB of live bytes. The projection was right;
nothing had reclaimed what the compression freed.
Also records where those bytes are. context_snapshot is 94% of the table at
104 kB a row, and the per-action state columns a tree would sit beside are
rounding errors - which is the number phase 14 wants before it adds more.
And a trap: n_live_tup on the TOAST relation implied half the real bloat. It
is a leftover ANALYZE estimate, stale in the flattering direction right after
a migration. Size from sum(octet_length(col)) instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
STATUS was written before the deploy and still asked for a VACUUM that has
since been run. Replaces that with what was actually verified against the
running service, and with the two things a next session would otherwise have
to rediscover.
The post-vacuum sizes were never measured, so the projection stays a
projection; the query to settle it is in the file. And the scroll gap is
promoted from a footnote to the one real open item, because re-reading that
path after shipping found a bug in it -- which is the argument that reading it
again is not the fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Records what the six changes measured, and replaces the pick-up item -- which
was step 6 -- with plan/14, since there is nothing left in 13.
Two things a future session needs and cannot infer from the code. The VACUUM:
migration 43 compresses context_snapshot but Postgres does not return the disk
by itself, so until `VACUUM FULL actions` runs the storage win exists only on
paper and the table is temporarily larger, not smaller. And the gap: nothing
exercises the scroll behaviour in a browser, because the frontend has no test
runner, and prepend-and-restore-scroll is the part most likely to feel wrong
even when it is correct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
The previous commit recorded the service as suspended. It is not. Render
appended a suffix to the `ai-dnd` service name in render.yaml, and plain
ai-dnd.onrender.com is a different, suspended service whose 503 page reads
"suspended by its owner" -- indistinguishable from this deploy being down
unless you notice the host is wrong.
The real host answers {"ok":true} on /api/health in 0.6s, mints a guest,
returns an empty adventure list for that guest and serves the starter
scenarios. The authoritative link is the one docs/index.html points at, not
the service name in the blueprint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
Everything measured so far ran on SQLite against a synthetic fixture, so two
claims were still on trust: that migration 38 spells BYTEA correctly for a real
server, and that the byte figures survive psycopg's encodings.
Both hold. Production reads schema_version 41 with embedding_blob bytea and
embedded boolean present, the backfill is complete at 134/134, and the packed
vectors are 5.04x smaller than the JSON on real data -- 30,971 to 6,144 bytes a
memory, as predicted. stress_session now takes AIDND_STRESS_DATABASE_URL, and
against a throwaway Neon database every shape lands within 0.5% of the SQLite
run: the warm turn is 121.1 kB against 122.3, with memories down to 1.7 kB of
it.
The harness writes, so it refuses any target whose name does not say stress or
scratch -- pointed at the production database it stops rather than seeding it
with a fake user and 200 fake turns. It also empties a Postgres target before
building, which a fresh SQLite temp file never needed.
Two corrections fall out, both recorded in plan/13. The page-load model has the
wrong shape: real actions are half the fixture's weight but real stories run to
607 actions, not 200, so the worst real page load is 589.5 kB. And the decision
to leave context_snapshot in the database costed egress but never storage --
it is 88.9 MB of a 99.6 MB database against a 512 MB free tier, which is the
ceiling this deploy will hit first.
Measured with counts and octet_length sums only. No user content was read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
plan/STATUS.md: what the last session changed, what to pick up next, and the
handful of things that were learned the hard way and would otherwise have to
be rediscovered. Linked from the overview, which now also lists plans 13 and
14 alongside the earlier phases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
Retrieval walked adventure.memories, so every turn loaded every row of the
bank with its vector attached -- 3.1 MB, 96% of everything a turn read. It
now asks SQL which memories are in play (an id and a flag per row), ranks
against vectors held in process, and fetches text only for the five it picks.
Two more callers were doing the same thing and the production SQL could not
see them: _evict_over_capacity walked the bank to count it, and _embed_pending
walked it to find the rows with no vector. Both are counts and filters the
database can do without sending anything back.
one turn 3,258.7 kB -> 723.4 kB cold, 122.3 kB warm
run_post_turn 3,139.1 kB -> 0.7 kB
Insights 3,223.7 kB -> 117.9 kB
Memories drawer ~3.1 MB -> 23.6 kB
A played turn is turn plus post-turn work: 6.4 MB down to 123 kB.
The cache needs no invalidation callbacks, which is what makes it safe. A
vector can only change through set_vector, which drops that one entry;
anything that removes a memory from play leaves the catalogue query, and
entries missing from the catalogue are dropped on the next read. So eviction,
deletion and pruning have nothing to remember to call.
memories.embedded joins the blob, for the same reason actions.variant_count
sits beside actions.variants: with the vector deferred, every "is this
embedded?" check would otherwise be a 6 KB lazy load, once per row.
Capacity drops 200 -> 80, on retrieval quality as much as cost -- ranking two
hundred memories to pick five buries the five. Eviction was measured at scale
first: trimming 100 to 80 costs 0.8 kB and reads no vectors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
Both egress blowouts this project has had were one query fetching a column
nobody read, and a statement count would have shown nothing wrong in either.
So the meter counts bytes, at the DBAPI cursor -- everything that crosses
that line crossed the wire.
tools/stress_session.py drives a production-shaped adventure through the real
routes with only the network faked. It reproduces both figures measured
directly on production: 426.7 kB for a 200-action page load against 423 KB,
and 3,258.7 kB for one turn against 3,153 kB.
The memory bank is on by default, which is the whole point -- the previous
harness ran without an embedding model, so retrieval returned early and the
heaviest read in a turn never happened. --no-embeddings reproduces that
deliberately, and the gap is 29x.
It also turned up two callers the production SQL could not see: run_post_turn
walks the whole bank again every turn, and Insights pays for it a third time.
A played turn costs ~6.4 MB, not 3.2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
