Keep the local Claude test rig, and record what it found

`backend/tools/claude_shim.py` serves an OpenAI-compatible endpoint backed by
the local `claude` command line tool, so a demo can be played against a real
model with no API key. Each request spawns one `claude --print` process, which
suits the turn engine: the app assembles the whole prompt every turn and
expects a stateless endpoint.

Playing the Pokemon demo through it made all five of the world-state changes in
`plan/16` work, and the refusal loop ran end to end for the first time: turn 3
clamped to nothing, turn 4's assembled prompt carried the correction verbatim,
and the model's next delta was right. Two failures previously blamed on the
code were the demo model. One is a schema fault and is still open: Milo's three
Pokemon share one `active_hp` stat, so a switch leaves the newcomer at 0 HP.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PacdRuPXSkQQy4ZYdH32hF
This commit is contained in:
parththakkar106
2026-08-28 18:42:24 +05:30
co-authored by Claude Opus 5
parent 6cbf6d996f
commit ef4da7fda2
5 changed files with 470 additions and 5 deletions
+68 -3
View File
@@ -86,8 +86,12 @@ nothing still reads `variant_count`/`variant_index` before dropping them, and no
is the migration shape that rewrites toasted values, so it owes one `VACUUM FULL actions;`
on the direct (non-`-pooler`) endpoint afterwards.
Ahead of that, `plan/16-world-state-refusals.md` records a set of world-state fixes that
are merged but **never driven in a browser**. See that file for what to test.
Ahead of that, `plan/16-world-state-refusals.md` is closed. Its fixes were driven in a
browser on 2026-08-28, first on the demo model and then on Claude Sonnet through the
local shim, and all five changes work. One item in it is still open and is the next
world-state job: **Milo's three Pokemon share one `active_hp` stat**, so a switch leaves
the newcomer at 0 HP and every later hit is refused. Give them their own NPC entries,
which also removes the `initial == max` shape from that demo.
**SP9 and SP10 are both on `main`, despite what earlier notes here said.** The branches
`sp7b-take-pager` and `sp10-memory-bank-eviction` still exist and still read as unmerged
@@ -206,6 +210,47 @@ an automated version; see SP7's entry in `plan/14`.
---
## What happened on 2026-08-28 — world state against a real model
**The refusal loop ran end to end for the first time.** `plan/16` shipped a correction
that is injected into the next turn's prompt when the engine clamps or rejects a change,
and nothing had ever observed it working. Turn 3 of the Pokemon demo clamped to nothing,
turn 4's assembled prompt carried the note verbatim, and the model's next delta was
correct. The full record, with the four turns and their chips, is at the end of
`plan/16-world-state-refusals.md`.
**Two failures blamed on the code were the model.** `graveler_defeated` never firing and
`world.turn` never moving both survived four playtests on the free demo model, with the
instructions present and correct in the assembled prompt. Both worked on the first try
against Sonnet. When an instruction is provably in the prompt, the next thing to change
is the model, not the wording.
**There is now a permanent way to test against a real model.**
`backend/tools/claude_shim.py` serves an OpenAI-compatible endpoint backed by the local
`claude` command line tool, so a demo can be played on a real model with no API key.
About $0.04 a turn against the subscription. The README documents it under **Connect a
model**.
**Renaming a seed scenario used to strand its old row.** `seed.py` matches a seed file to
its scenario by title, so changing a title inserted a second public scenario and left the
first orphaned and public forever. This is the same failure this file already records for
"Road to the Champion". Seed files now carry `previous_titles`, and the rename lands on
the existing row. Verified: the Pokemon demo kept its id.
**Every new guest is given a pre-played adventure.** `app/starter.py` copies a shipped
export bundle into each new guest account. The point is the first screen: real turns with
their world-state chips, including a refused change and a milestone, before spending any
of the daily demo turns. The row building inside `POST /adventures/import` moved to
`bundle.materialize` so both callers share one writer.
**An SVG data URI is not usable as scenario art.** `app/images.py` accepts raster formats
only, deliberately, because SVG can carry script and the bytes are served from the app's
own origin. A rejected value is stored but yields an empty `image_url`, which fails
quietly. The Pokeball is a 3.2 kB PNG, drawn by `backend/tools/make_pokeball.py` with
`zlib` alone.
---
## What happened on 2026-08-21 — visit analytics
The hosted demo can now answer whether anyone is using it. `/analytics` is a dashboard —
@@ -899,7 +944,7 @@ the SQLite dev parity this codebase protects on purpose).
```
cd backend
.venv/Scripts/python.exe -m pytest tests/ # 365 tests (~100s)
.venv/Scripts/python.exe -m pytest tests/ # 539 tests (~180s)
.venv/Scripts/python.exe -m tools.stress_session # egress report (SQLite)
# Same harness against a real Postgres. The target must be a THROWAWAY database
@@ -945,6 +990,26 @@ holds the egress ceilings.
On Windows the report's box-drawing characters crash the default cp1252 console;
prefix with `PYTHONIOENCODING=utf-8`.
**A real model, without an API key.** `tools/claude_shim.py` serves an OpenAI-compatible
endpoint backed by the local `claude` command line tool. Point Settings at
`http://127.0.0.1:8787/v1`, put any string in the API key field, and pick `sonnet`. Set
the reasoning budget to `0` or `-1`: a positive budget sends `reasoning.max_tokens`,
which Claude 5 models reject with a 400. It costs about $0.04 a turn against the
subscription, and it does not serve embeddings.
```
cd backend
.venv/Scripts/python.exe tools/claude_shim.py # 127.0.0.1:8787
```
**To exercise the guest path**, which is off in single-user mode:
```
cd backend
AIDND_MULTI_USER=1 AIDND_SECRET_KEY=throwaway AIDND_DB_PATH=$PWD/guest_scratch.db .venv/Scripts/python.exe -m uvicorn app.main:app --port 8001
curl -c jar.txt http://127.0.0.1:8001/api/auth/me # mints a guest and its starter
```
Port 8000 is shared with the job-pipeline app, which will squat it and silently shadow
the AI-DnD API — free it before running the backend, or move the vite proxy with
`AIDND_API_PORT`, which is what the `--keep` recipe above does.