Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Keep the fixture, so there is something to scroll (#5)

Browse Source 
The harness already built a production-shaped 600-action adventure and then
threw it away with the temp file. The one open gap in plan/13 is that nothing
has ever driven the scroll in a browser, and part of why is that there was
never a long adventure to drive it with.

--keep PATH writes the fixture somewhere durable and makes the app able to
serve it. Two edits are needed for that, both of which cost an hour to
rediscover:

  - create_all() builds the current schema but leaves the version stamp at its
    default, and bootstrap() reads a populated-but-unstamped database as
    ancient — it replays every migration against a schema that already has the
    columns, and fails on the first.
  - the fixture's user is a registered one, but local mode looks for the row
    with email IS NULL and is_guest false, so without clearing the email the
    app opens on an empty library.

--keep is read before argparse exists, because where the database lives has to
be settled before app.database is imported. That is the same constraint the
AIDND_STRESS_DATABASE_URL block already lives under. SQLite only; combining it
with a Postgres target is rejected rather than half-honoured.

Verified end to end: the fixture boots with no manual step, action_count 600,
a 60-action first payload, and before_id walks back nine more pages to the
start. Nothing about the default path changed; 259 tests pass.


Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Parth
2026-08-17 16:50:41 +05:30
committed by GitHub
co-authored by Claude Opus 5
parent 78d73b94ea
commit 0f1e05c808
2 changed files with 152 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/tools/stress_session.py
+86
View File
@@ -64,6 +64,24 @@ meaningless.
import os
import sys
import tempfile
from pathlib import Path
def _early_keep(argv: list[str]) -> str:
"""--keep, read before argparse exists.
Where the database lives has to be decided before app.database is
imported, and that import is three lines below. argparse still declares
the flag, so --help documents it and a typo is still an error."""
for i, arg in enumerate(argv):
if arg == "--keep" and i + 1 < len(argv):
return argv[i + 1]
if arg.startswith("--keep="):
return arg.split("=", 1)[1]
return ""
_keep = _early_keep(sys.argv[1:])
# Must precede the app import: database.py reads these at module scope.
#
@@ -78,6 +96,11 @@ import tempfile
# must say it is disposable.
_stress_url = os.environ.get("AIDND_STRESS_DATABASE_URL", "").strip()
if _stress_url:
if _keep:
sys.exit(
"--keep writes a SQLite file for the app to serve; it cannot be\n"
"combined with AIDND_STRESS_DATABASE_URL."
)
_dbname = _stress_url.rsplit("/", 1)[-1].split("?")[0]
if not any(mark in _dbname.lower() for mark in ("stress", "scratch")):
sys.exit(
@@ -87,6 +110,17 @@ if _stress_url:
)
os.environ["AIDND_DATABASE_URL"] = _stress_url
os.environ.pop("DATABASE_URL", None)
elif _keep:
# A fixture to boot the app against rather than a temp file the report
# discards. Rebuilt from empty every run: build_fixture() assumes an empty
# database on the SQLite path, and a second run would otherwise stack a
# second adventure beside the first.
_keep_path = Path(_keep).resolve()
_keep_path.parent.mkdir(parents=True, exist_ok=True)
_keep_path.unlink(missing_ok=True)
os.environ["AIDND_DB_PATH"] = str(_keep_path)
os.environ.pop("AIDND_DATABASE_URL", None)
os.environ.pop("DATABASE_URL", None)
else:
_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
_tmp.close()
@@ -100,6 +134,7 @@ import random
from fastapi import Depends
from fastapi.testclient import TestClient
from sqlalchemy import text
from app import auth, limits, memorybank, models, security
from app.database import Base, SessionLocal, engine, get_db
@@ -341,6 +376,44 @@ def _check(response) -> None:
sys.exit(f"shape failed: {response.status_code} {response.text[:400]}")
# ------------------------------------------------------------------- --keep
def make_bootable() -> None:
"""Two edits that turn a measurement fixture into a database the app will
actually serve. Both exist because build_fixture() builds a database for
the meter, not for a browser."""
from app.migrations import LATEST_VERSION
with engine.begin() as conn:
# create_all() builds the current schema but leaves the stamp at its
# default, and bootstrap() reads a stamped-but-not-fresh database as
# ancient — it would replay all of the migrations against a schema
# that already has every column, and fail on the first one.
conn.execute(text(f"PRAGMA user_version = {LATEST_VERSION}"))
# In local mode (AIDND_MULTI_USER unset) get_current_user() looks for
# the row with email IS NULL and is_guest false. The fixture's user is
# a registered one, so without this nothing owns the adventure and the
# app opens on an empty library.
conn.execute(text("UPDATE users SET email = NULL, is_guest = 0"))
def print_keep_notes(path: str, actions: int) -> None:
port = 8010
print()
print(f"fixture kept: {path}")
print(f" {actions} actions, bootable in local mode. To scroll it:")
print()
print(f" cd backend && AIDND_DB_PATH={path} \\")
print(f" .venv/Scripts/python.exe -m uvicorn app.main:app --port {port}")
print(f" cd frontend && AIDND_API_PORT={port} npm run dev")
print()
# 8000 is the vite proxy's default and another local app squats it, which
# shadows this API with its own SPA catch-all and looks like an empty
# database rather than a proxy problem.
print(f" Port {port} rather than 8000 on purpose; AIDND_API_PORT points vite at it.")
# ----------------------------------------------------------------------- main
@@ -380,6 +453,13 @@ def parse_args(argv=None):
p.add_argument("--no-embeddings", action="store_true",
help="unset the embedding model — reproduces the round-two "
"blind spot, where the bank's cost is invisible")
# Read at import time by _early_keep as well — the database location has
# to be settled before app.database loads. Declared here so it appears in
# --help and an unknown spelling is still rejected.
p.add_argument("--keep", metavar="PATH", default="",
help="write the fixture to PATH and leave it bootable, so "
"the app can serve it in a browser (default: a temp "
"file, discarded). SQLite only")
p.add_argument("--seed", type=int, default=7)
p.add_argument("--statements", type=int, default=5,
help="heaviest statements to print per shape (default: 5)")
@@ -424,6 +504,12 @@ def main(argv=None) -> int:
app.dependency_overrides.clear()
adventures._active_turns.clear()
# After the shapes, not before: make_bootable() writes, and the meter is
# still attached until the report above is rendered.
if args.keep:
make_bootable()
print_keep_notes(os.environ["AIDND_DB_PATH"], args.actions)
return 0
plan/STATUS.md
+66 -3
View File
@@ -95,6 +95,46 @@ Two things from the egress work are worth carrying into it:
lesson of the 144 MB above — a rewrite doubles the table and only a `VACUUM FULL` gives
it back. Phase 14's migration rewrites every row.
**There is a 600-action adventure to test against now** — `--keep`, below. The tree's
frontend work lands on the same scroll path that has still never been driven by hand, so
drive it before rewriting it.
---
## What happened on 2026-08-17, part three
No behaviour change. A way to get a long adventure in front of a browser, because the
one open gap needed a subject and there wasn't one.
**`tools.stress_session --keep PATH`.** The harness already built a production-shaped
600-action adventure and then threw it away with the temp file; `--keep` writes it
somewhere durable and makes the app able to serve it. Two edits are needed for that, and
both are the kind of thing that costs an hour to rediscover:
- **`create_all()` does not stamp the schema version.** `bootstrap()` reads a
populated-but-unstamped database as ancient and replays every migration against a
schema that already has the columns. `--keep` stamps `PRAGMA user_version` to
`LATEST_VERSION`.
- **The fixture's user is a registered one.** In local mode `get_current_user()` looks
for the row with `email IS NULL` and `is_guest` false, so without clearing the email
the app opens on an empty library and nothing owns the 600 actions.
`--keep` is read before argparse exists (`_early_keep`), because where the database
lives has to be settled before `app.database` is imported — the same constraint the
`AIDND_STRESS_DATABASE_URL` block at the top of the module already lives under. SQLite
only; combining it with a Postgres target is rejected rather than half-honoured.
Verified: the fixture boots with no manual step, `action_count` 600, the first payload
carries 60 actions, and `before_id` walks back through 9 more pages to the start — 600
seen, `has_more` false at the end. 259 tests pass.
**Snapshots can be shrunk for this.** `--snapshot-bytes 2000` keeps the file at ~2.5 MB
instead of ~140 MB. `context_snapshot` is deferred and never reaches the browser, so it
changes nothing about what scrolling exercises — but do not shrink it when *measuring*,
where it is most of the point.
**Port 8010, not 8000.** Covered below, and now printed by `--keep` itself.
---
## What happened on 2026-08-16
@@ -259,7 +299,10 @@ with `pg_total_relation_size`, and do not mix them up.
All six of its open items landed on 2026-08-17, and are live. What is left is not from
that plan:
- **Nothing has ever exercised the scroll in a browser.** This is the one real gap, and
- **Nothing has ever exercised the scroll in a browser** — still true, but there is now
something to exercise it *on*: `--keep` builds a 600-action adventure the app will
serve (see 2026-08-17 part three). The subject is no longer the excuse; only the
looking is left. This is the one real gap, and
it has already cost something: re-reading that path after shipping turned up a bug
where loading earlier turns scrolled *past* them to the end of the story, worst on
the short-window case the button exists for (fixed, PR #2). One bug found by reading
@@ -268,7 +311,9 @@ that plan:
scroll-position arithmetic still needs eyes.
- **`ACTION_PAGE = 60` is a guess.** It should be a page or two of reading. If loading
older turns feels like it interrupts, that is the number to move
(`routers/adventures.py`).
(`routers/adventures.py`). One data point: at 600 actions it takes the window plus
**nine** more pages to reach the start, which is a lot of button presses for anyone
going back to the beginning.
- ~~Post-vacuum sizes unmeasured~~ — measured 2026-08-17, and the vacuum that mattered
was run then too. 65.0 MB. See the top of this file.
- **Anyone who switched embedding models has a stale bank.** The bug is fixed, but
@@ -295,8 +340,26 @@ AIDND_STRESS_DATABASE_URL=postgresql://…/stress_scratch \
.venv/Scripts/python.exe -m tools.stress_session
```
**A long adventure to scroll**, instead of a temp file the report discards. The
snapshots are shrunk because they never reach the browser — 2.5 MB rather than 140 MB —
and `--shapes list` skips the measurement work the fixture does not need:
```
cd backend
.venv/Scripts/python.exe -m tools.stress_session \
--keep ./scroll_fixture.db --snapshot-bytes 2000 --shapes list
AIDND_DB_PATH=$PWD/scroll_fixture.db \
.venv/Scripts/python.exe -m uvicorn app.main:app --port 8010
cd ../frontend && AIDND_API_PORT=8010 npm run dev # → localhost:5173
```
Everything in it is synthetic and no real adventure is read. `*.db` is gitignored, so
the fixture never lands in a commit.
On Windows the report's box-drawing characters crash the default cp1252 console;
prefix with `PYTHONIOENCODING=utf-8`.
Port 8000 is shared with the job-pipeline app, which will squat it and silently shadow
the AI-DnD API — free it before running the backend, or move the vite proxy.
the AI-DnD API — free it before running the backend, or move the vite proxy with
`AIDND_API_PORT`, which is what the `--keep` recipe above does.