⚔ AI D&D Repo →

Engineering guide

How this thing works, and why it works that way

An AI Dungeon-style storytelling engine. The chat loop is the boring part — the interesting parts are the token-budget allocator, the world-state referee, and the memory system that decides what the model is allowed to remember.

Written to be read end to end. Every section states the decision, the reasoning behind it, and what it cost. · Project page · Source

Contents
  1. Part 0 — Orientation
  2. Part 1 — The AI layer
  3. 1.1 The turn pipeline
  4. 1.2 Context assembly is a budget problem
  5. 1.3 World state: the AI proposes, Python referees
  6. 1.4 Output length, by measurement
  7. 1.5 The memory bank
  8. 1.6 Streaming
  9. 1.7 The scripting sandbox
  10. 1.8 Why there is no agent framework
  11. Part 2 — Data and correctness
  12. 2.2 Two coordinate systems
  13. 2.3 Undo and retry that rewind
  14. 2.5 The 189× egress fix
  15. Part 3 — Production concerns
  16. Part 4 — The web plumbing
  17. Part 5 — Results and limitations

Part 0

Orientation

What the thing is, in the fewest words that are still true.

0.1What it is

An AI Dungeon clone. You write a scenario, then play an open-ended text adventure where a language model narrates the world. You type “I open the door”, the model writes what happens next, and it remembers what came before.

Three things make it more than a chat wrapper:

  1. A context engine. The model has a limited input window. The app decides, every single turn, which pieces of the story get to be in the prompt and which get dropped.
  2. A world-state engine. The scenario declares stats — hp, trust, day. The model proposes changes each turn; a Python engine decides what actually sticks.
  3. A scripting sandbox. Real AI Dungeon JavaScript scripts import and run, inside an embedded QuickJS interpreter.

Runs locally against Ollama for free, or hosted against any OpenAI-compatible endpoint.

0.2The stack, and what each part is doing

PieceWhat it actually does here
FastAPIThe HTTP server. Every URL like /api/adventures/3/actions maps to a Python function. Also does the SSE streaming.
SQLAlchemyLets you write Python classes instead of SQL. Adventure, Action, Memory are Python classes; SQLAlchemy turns them into tables and turns attribute access into SELECTs.
SQLite / PostgresThe database. SQLite is one file on disk (local). Postgres is a server (hosted, on Neon). Same code talks to both.
ReactThe UI. Describes what the screen should look like for a given state; when the state changes it re-renders.
ViteThe frontend build tool and dev server. Bundles React into plain JS the browser can load.
httpxThe Python HTTP client used to call the model endpoint.
tiktokenCounts tokens, so the budgeting is real arithmetic and not a guess.
QuickJSA small embeddable JavaScript engine, used as a sandbox for user scripts.

The whole thing is one process in production: FastAPI serves the API and the built React files from the same port.

The shape of one request

you tap "Do"
  → POST /api/adventures/3/actions
       {type: "do", text: "open the door"}
  → check ownership, rate limit, turn lock
  → assemble the prompt      ← the interesting part
  → POST to the model endpoint, stream=true
  → tokens come back one at a time
  → each is forwarded on as a Server-Sent Event
  → React appends it to the screen as it arrives
  → stream ends: parse the state block, referee
       it, save the action

Part 1

The AI layer

Where most of the design effort went. Everything here is a decision someone could reasonably disagree with.

1.1The turn pipeline

Everything that happens between “player pressed a button” and “text is on screen”. Source: backend/app/routers/adventures.py.

onInput — user JS may rewrite or block the input
store the player action
retrieve memories — embed recent story, cosine-rank the bank
snapshot script + world state — so undo and retry can roll back
build_context() — the budget allocator
onModelContext — user JS may rewrite the whole prompt
snapshot the exact prompt — for the Insights panel
provider.generate() — streamed, token by token
onOutput
extract + referee the state block — then strip it from the prose
save the action
background: summarize + embed — fire-and-forget
model or prompt work user script hook persistence

Two design choices are visible in that list before any of the details.

The prompt is snapshotted, not reconstructed. Every AI action stores the exact text that was sent to the model. That’s what powers the Insights panel — open any turn and see each context component, its token cost, and why it was included. It’s also what makes prompt bugs findable. The cost is storage, about 74 KB per turn, which turns into a real performance problem later (see 2.5).

Snapshots happen before the model call, not after. state_before and world_state_before are stapled onto the action before the hooks and the delta run. That’s the entire mechanism behind undo and retry actually rewinding rather than just deleting text.

1.2Context assembly is a budget problem

Source: backend/app/context/builder.py.

The problem

The model can only read so much. Say the budget is 8,000 tokens. A 200-turn adventure has far more story than that. Something has to be dropped, and what gets dropped decides whether the story stays coherent.

The naive version, and why it breaks

Send the last N turns. That fails in two directions: N turns of short exchanges wastes the window, and N turns of long ones overflows it. Worse, “the last N turns” throws away the things that matter most — the premise, the character sheet, the fact that you promised the innkeeper you’d return.

What this app does

Split the prompt into fixed sections and elastic ones.

CONTEXT TOKEN BUDGET RESERVED CARDS ≤ 40% HISTORY, NEWEST FIRST available = budget − reserved narrator · stat guide · world state · emit rule · ai instructions plot essentials · story summary · retrieved memories ↑ these are always included, whatever they cost
Fixed sections are reserved first and never dropped. What’s left is the elastic budget: triggered story cards may take up to 40% of it, and story history spends the remainder filling backwards from the newest turn.

The algorithm is three lines of arithmetic:

reserved  = every fixed section + note + hint + reminder
available = max(256, token_budget - reserved)

cards   ≤ available * 0.4
history = available - cards_used, newest first

The details that are actually decisions

Cards are capped at 40% of the elastic budget. Story cards are triggered by keyword match, so a scene mentioning six named things could pull in six lore entries and leave no room for the story itself. The cap makes the failure mode “some lore is missing” instead of “the model has no idea what just happened”. Cards that don’t fit are still reported to Insights with included: false, so the UI can show the lore that got squeezed out.

History fills newest-first and stops. Oldest turns fall out. That’s the right direction because the old material isn’t actually lost — it’s been summarized into memories and the running summary, which live in the fixed section.

If even the single newest turn is over budget, it gets hard-truncated rather than dropped. A prompt with no story at all produces nonsense; a prompt with the tail end of the last turn produces something.

The author’s note is injected three actions from the end, not at the top. Instructions placed near the end of a prompt have more influence on what comes next than instructions at the top — recency. The author’s note is a steering control (“keep it tense”), so it goes where steering works.

The world-state reminder goes dead last. The full emit rule lives up in the system block, hundreds of tokens away from where the model starts writing. A one-line reminder occupies the final slot. Same recency logic, applied to the thing most likely to be forgotten.

The subtle one

Past AI turns get their state block re-attached. The block is stripped from the text before storage, so a replayed history would show the model twenty of its own past turns that contain no state block — teaching it, by imitation, to stop emitting one. So the history builder reconstructs the block from the stored delta and re-appends it. The model sees its own pattern and keeps following it.

The performance trap hiding in this

Building the context needs the newest ~6,000 tokens of story. The obvious implementation reads adventure.actions — which loads every row of the adventure — then throws 90% of it away. At turn 200 that was 839 KB of database reads to use maybe 70 KB, growing every turn.

context/history.py fixes it by serving three shapes directly from SQL: a tail, a slice, and a count. window_covering() fetches the newest 32 actions, measures their real token count, and if that’s short of the budget it projects how many more it needs from the average length just measured, rather than blindly doubling:

average   = tokens / len(actions)
projected = int(budget / average * 1.15) + 8

Each round fetches only what it doesn’t already hold, so no row is read twice. The same turn costs 129 KB instead of 839 KB, and stops growing at around turn 50 — the cost is bounded by the context budget instead of by the length of the story.

There’s a second rule in that module worth naming: if the actions are already loaded in memory, slice them instead of querying. The scripting pipeline hands the whole history to user scripts, because AI Dungeon’s API requires it, so on a scripted adventure the rows are already there — issuing a query beside them would mean paying twice.

1.3World state: the AI proposes, Python referees

Source: backend/app/worldstate/engine.py.

The question

You want an RPG layer — hit points, trust, quest progress. Who owns the numbers?

Option A — a deterministic dice engine
The engine rolls and applies damage, the model narrates the result. This is what a real RPG does. It loses here because the action space is unbounded: the player can type anything, and mapping arbitrary natural language onto a fixed rules system is a harder problem than the one being solved.
Option B — the model owns the numbers
Track hp in the prose and trust it. Fails immediately. Models are bad at arithmetic, worse at holding a number across twenty turns, and completely unable to obey their own frequency rules — tell one “only change this every 5 turns” and it changes it every turn.
Option C — chosen: propose and dispose
The model narrates and appends a JSON delta of what changed. Python validates and clamps it before anything is stored. The model owns intent; the engine owns arithmetic.
narration: "The blade catches your shoulder. Gwen shouts and drags you back."

```state
{"player.hp": -15, "npc.gwen.trust": 5, "milestones.escaped": true}
```

The engine then applies, in order:

RuleWhat it stops
Path must exist in the schemaHallucinated stats
Value must be the right type"a lot" instead of -15
CooldownChanging a stat more often than the scenario allows
Counters can’t decreaseThe in-game day going backwards
max_delta_per_turnLosing 90 hp to a stubbed toe
Clamp to min/maxNegative hp, trust above 100
Milestones sticky, true onlyUn-completing a quest
Flags are two-way booleansDeliberately unrestricted — that’s what flags are for

Everything rejected is reported, not silently swallowed. The Insights panel shows applied, clamped and rejected paths per turn, and the chip under each narration shows what actually changed.

The reliability mechanism: word bands

A stat can carry bands:

"hp": { "min": 0, "max": 100, "initial": 100,
        "bands": [[0,20,"very weak"],[20,40,"hurt"],[40,60,"minor damage"],
                  [60,90,"healthy"],[90,100,"full health"]] }

Two things use them. The live state block shows the current band label — hp 55/100 (minor damage) — so the model reads a word, not just a number. And the stat guide shows the whole ladder once per turn, so the model can see the full scale it’s reasoning across.

The point: models reason well over semantics and badly over arithmetic. “He’s badly hurt, so a solid hit should take him to very weak” is a judgement a model can make. “55 minus 22 is 33” is one it will get wrong often enough to matter.

The failure philosophy

Nothing in the world-state engine raises. A malformed delta returns {} and the turn continues. The parser is deliberately tolerant — it strips trailing commas and leading + signs on numbers, both of which weaker free models emit and strict JSON rejects. It accepts a state, json or unlabelled fence, and falls back to a bare JSON object at the end of the text, but only if it parses into something that looks like a delta, so prose ending in } is never eaten.

This matters because the public demo runs on free-tier models. A stricter parser would mean a good model works and a free one doesn’t.

One call, not two

The model narrates and emits the delta in a single request. The alternative — narrate, then a second call to extract structured state — is more reliable per call and costs twice the latency and twice the rate-limit budget. On the free tier (20 requests/minute) that would halve the playable turn rate. The tolerant parser plus the terminal reminder was the cheaper way to buy the same reliability.

1.4Output length, by measurement

The problem

max_output_tokens is a hard wall the endpoint enforces mid-sentence. Hit it and whatever is being written gets cut off. Since the state block is emitted last, the state block is what gets lost. The turn narrates fine and silently records nothing.

First attempt, and the measurement

Tell the model its budget: “keep this turn under about N words”.

174 → 246
average words per turn once the “budget” hint was added — every run longer than every unhinted run (n=5)
170
average after rephrasing the same number as a hard ceiling

Phrased as a budget, the number reads as a target to fill. The hint pushed turns toward the very wall it existed to protect.

The fix

[Hard limit: this turn must not exceed 412 words. Write only as much as the
moment needs — a typical turn is much shorter. Finish the narration and append
the state block well inside the limit.]

And the arithmetic around it

words = int((max_output_tokens - 50) * 0.75 * 0.90)

1.5The memory bank

Source: backend/app/memorybank.py.

The problem

Story history falls out of the context window as the adventure grows. Turn 4 said you promised the innkeeper you’d return. At turn 90 that’s long gone from the prompt — but if you walk back into the inn, it should come back.

Three layers

LayerCadencePurpose
Memoryevery 6 actions, from 12One or two past-tense sentences of concrete fact.
Story summaryevery 15 actionsA single ≤250-word overview, rewritten by folding in the new memories.
Retrievalevery turnEmbed the last 4 actions (≤600 tokens), cosine-rank the bank, inject the top 5.

Retrieval is what answers the innkeeper problem: the promise is a memory, the memory has a vector, walking into the inn produces a query vector near it, and it comes back into the prompt.

The decisions inside it

Only settled actions get summarized. The newest action is always held back one turn. Only the last action can be retried — so if a memory summarized the newest action and the player then retried it, that memory would describe narration that no longer exists, and because its cursor has already advanced it would never be regenerated. Holding one action back costs a turn of latency and makes that state unreachable.

Cursors only advance on success. Every AI call here is best-effort. If summarization fails, the function returns and the cursor is unchanged, so the same block is retried on a later turn. There’s no retry loop, no dead-letter queue, no backoff — the cadence is the retry mechanism.

Summarization is fire-and-forget, in a background task with its own DB session. The player’s turn is already on screen; making them wait would add a second or two of latency every sixth turn for no visible benefit. The task holds a strong reference to itself — the event loop only keeps weak ones, so a fire-and-forget task can otherwise be garbage-collected mid-run — and a per-adventure guard stops two from overlapping.

Pinned memories count toward top_k. Pinned ones are always injected; unpinned fill up to top_k − len(pinned). Without that, 6 pinned memories plus top_k=5 injects 11 and blows the budget the whole context engine exists to respect.

A dimension mismatch scores 0.0, it doesn’t crash. If the user changes their embedding model, old 768-dim vectors get compared against a new 1536-dim query. zip() would happily truncate and score garbage, silently. An explicit length check returns 0.0 instead.

Eviction is LRU-ish, and evicted memories are kept. Over capacity (default 200), the least-used unpinned memories are marked forgotten rather than deleted — so the UI can still show them and you can un-forget one.

Money trap

Background calls never spend the shared demo key. The summarization and embedding providers are built directly from the user’s own settings, never from the demo config, and their call sites are skipped when the turn is running on the demo key. Summarization is unmetered background spend; the demo key is server-funded. Both facts together would be a bill.

1.6Streaming

The model produces tokens one at a time. Waiting for the whole reply before showing anything makes a 20-second generation feel broken.

Server-Sent Events is the mechanism: an HTTP response that stays open and pushes data: {...} lines as they become available. It’s one-directional (server → browser), which is exactly the shape of this problem — WebSockets would be a bidirectional connection for a unidirectional need.

model endpoint  --SSE-->  FastAPI  --SSE-->  browser  -->  React state  -->  screen

FastAPI reads the provider’s stream and for each chunk yields data: {"type":"chunk","text":"…"}. The frontend reads the response body with a ReadableStream reader, buffers on \n\n boundaries, and dispatches each parsed event. Event types: player, reasoning (thinking-model traces, which stream into a separate collapsible panel with their own token budget), chunk, stopped, error, done.

Two production details that only show up when hosted:

The empty-reply case is diagnosed, not reported as “empty”. If a reasoning model streams thinking but no story text, it spent its whole budget thinking — the error says so and names the three settings that fix it.

1.7The scripting sandbox

Real AI Dungeon scripts are JavaScript files defining modifier(text) and calling it as the last line, with globals like state, history, storyCards. To be compatible, this app runs the same contract in an embedded QuickJS interpreter.

The safety properties are mostly structural:

PropertyHow
No filesystem, network or process accessQuickJS has none by default — nothing was removed, nothing was added
Memory cap16 MB per run
CPU cap2 seconds per run
No shared state between runsA fresh context per hook execution
A broken script can’t break a turnEvery failure returns as .error with text, state and cards unchanged; the pipeline logs it and continues

Data crosses the boundary as JSON — Python serializes {state, text, history, storyCards, info} in and the results out. There is no object bridge to exploit.

One deliberate bug-compatibility: addStoryCard returns the new card’s index, so the first card returns 0, which is falsy, so if (!addStoryCard(…)) misfires. That’s upstream AI Dungeon’s behaviour. It’s documented in the code and left alone, because matching real scripts is the entire point of the feature.

1.8Why there is no agent framework

Graph-based agent frameworks (LangGraph and similar) earn their complexity with branching, cyclic, multi-step control flow — a graph of nodes where the path depends on what the model decides, with loops, retries, tool calls, and persisted state between steps.

This turn pipeline is a fixed linear sequence with exactly one model call. There is no routing decision, no tool selection, no loop. Every turn takes the same path. Adding a graph framework would mean carrying its state abstraction, its serialization model and its debugging surface to express a straight line.

There’s also a specific reason a framework’s context handling wouldn’t fit here: the budgeting logic is the product. Buffer-window and summary-memory abstractions are opinionated about how to fit history into a window. This app shows the user every context component, its token cost, and the trigger word that pulled it in — which means the assembly has to be explicit and inspectable.

When it would be the right call: if the design went toward the two-call version — narrate, then a separate structured-extraction step, with a retry branch when extraction fails and a tool-calling path for dice — that is a graph, and hand-rolling it would get ugly fast.

Part 2

Data and correctness

The bugs in this section are the kind that don’t crash. They just quietly produce the wrong answer, which is why each one has a test.

2.1The domain model

User
 ├─ Scenario   (the template)      ── stat_schema, prompt, memory, author's note
 │    └─ StoryCard, Script
 └─ Adventure  (the playthrough)   ── world_state, script_state, cursors
      ├─ Action  (one story entry) ── text, context_snapshot, variants, state_before
      ├─ StoryCard  (its own copy)
      ├─ Memory     (text, embedding, source_start/end, use_count)
      └─ AdventureScript

The one decision that shapes everything: template vs instance. A scenario declares what stats exist; an adventure holds what they are right now. Creating an adventure copies the scenario’s story cards, scripts and plot fields into it, so editing a scenario later never mutates a game in progress. There’s an explicit opt-in “Update from scenario” flow for when you do want that, which diffs the two and shows what would change.

Same reasoning as instantiating a class: shared definition, independent state.

2.2Two coordinate systems, and the bug class they create

The subtlest thing in the codebase.

There are two ways to identify an action:

The memory cursors are positions. Memory.source_start and source_end are Action.index values.

Why it’s nasty

The two spaces are identical until the first deletion, and diverge forever after. Mixing them means summarization silently skips or duplicates blocks — no crash, no error, just a memory describing the wrong turns.

Three things hold it together:

  1. position_of_index() is the explicit translation between the spaces, and every crossing goes through it.
  2. note_action_removed() is called before a delete: if the removed action sat before a cursor, the cursor decrements, so an unsummarized action can’t slide into the “already covered” range and be skipped forever.
  3. One definition of “story action”, written twice — once in SQL and once in Python — with a comment on both saying to keep them in step. The SQL version folds newlines and tabs into spaces before trim(), because SQLite’s and Postgres’ single-argument trim() only strips spaces while Python’s .strip() also drops newlines. An action of nothing but a newline would otherwise count as story text in one and not the other, and every cursor after it would be off by one.

2.3Undo and retry that actually rewind

Most implementations of undo delete the last message. That’s wrong here, because a turn mutates three things: the text, the scripting scoreboard, and the RPG stats.

The mechanism: every action carries state_before and world_state_before — deep copies taken before the turn’s hooks ran. Undo restores from them. Retry rolls back to them, then regenerates.

Retry keeps every attempt. Instead of deleting and replacing, the row survives and each attempt is appended to Action.variants; variant_index names the live one. The UI shows ‹ 2/3 › and you can page back to a discarded take. A variant stores only what differs between attempts — the narration, its reasoning trace, and the state it produced — never the assembled prompt, which is identical across attempts of the same turn and is by far the biggest thing in the snapshot.

Three details that are easy to get wrong:

2.4The turn lock

One turn at a time per adventure. Double-clicking “Continue” must not run two generations.

The subtlety: the check has to happen in the request phase, not when the SSE generator first runs. A streaming response doesn’t start iterating its generator until the response begins, so a check inside the generator lets two rapid requests both pass before either claims the slot. And because sync FastAPI endpoints run in a threadpool, the test-and-set needs a real lock.

def acquire_turn_lock(adventure_id):          # in the request handler
    with _active_turns_guard:
        if adventure_id in _active_turns:
            raise HTTPException(409, "A turn is already generating…")
        _active_turns.add(adventure_id)

async def with_turn_lock(adventure_id, gen):  # wraps the SSE generator
    try:
        async for event in gen: yield event
    finally:
        _active_turns.discard(adventure_id)

In-memory, so it’s a single-process guarantee. That’s honest for the deployment this targets — one Render web service. Two processes would need the lock in the database.

2.5The 189× egress fix

38.5 MB → 0.20 MB
database egress for one adventure load
~74 KB
per-turn prompt snapshot — 94% of the database

The bug: Action.context_snapshot holds the entire assembled prompt for a turn. Every adventure load pulled that column for every action, to read two small fields out of it — the world-state delta for the “what changed” chip, and the applied report. SQLAlchemy loads all columns by default.

The fix, in three parts:

  1. Move the two small things that are needed for every action into their own column.
  2. Mark the heavy columns deferred — snapshot, variants, reasoning — so they’re only fetched when explicitly asked for.
  3. Backfill the new column with dialect-specific server-side SQL, so the old data is extracted inside the database and never crosses the wire.

The part that makes it stick: tests/test_egress.py hooks into SQLAlchemy’s before_cursor_execute event, captures every statement the ORM sends, and fails if a bulk load ever names those columns again. The regression is caught by asserting on the SQL, not on a timing.

One more detail from that test’s design: the count query is written as a real SELECT count(…) rather than query.count(), because SQLAlchemy’s .count() wraps the entity select in a subquery whose SQL names every column — including the deferred ones. No bytes come back either way, but the database still reads them, and a guard that greps SQL can’t tell the two apart.

There’s a companion denormalization for the same reason: variants is deferred, so variant_count exists as its own column to answer “how many attempts?” without fetching them. One function is the only thing allowed to write variants, precisely so the two can’t drift and the pager can’t lie.

2.6Migrations, hand-rolled

No Alembic. An append-only list of (version, SQL) pairs, with the current version stored in SQLite’s PRAGMA user_version or a one-row table on Postgres. 37 versions so far.

Why this and not Alembic: for a single-file SQLite app someone may have been running for months, the entire requirement is “add a column, don’t lose their data”. Alembic’s autogenerate, branching and down-migrations are machinery for a team with a staging environment. This is 250 lines and you can read all of it.

The constraint it creates is written at the top of the file: change models.py so fresh databases are current, and append a pair here so existing ones upgrade. Migrations 2–23 predate Postgres support and use SQLite-only syntax — harmless, because every Postgres database starts fresh and never replays them, but anything added since must run on both dialects.

One migration worth reading (repairing duplicate action indexes) uses UPDATE … FROM with a window function rather than a correlated subquery, because SQLite may evaluate a correlated subquery against partially-updated rows and produce duplicates again while “repairing” them.

Part 3

Production concerns

What changes when the app stops being yours and starts being a URL strangers can open.

3.1Two modes, one codebase

AIDND_MULTI_USER switches the whole app between two personalities:

Local (default)Hosted
UsersOne auto-created local userGuest on first visit, optional account
AuthNone — no cookies, no login UISigned session cookie
Rate limitsOffOn
Row capsOffOn
API docsOnOff
ProviderWhatever Settings points atUser’s key, or the shared demo key

The reasoning: someone running this on their own laptop should never be throttled by their own app, never see a login screen, and should get the interactive API docs. A hosted deployment needs all four to be the opposite. Rather than two builds, the differences are gated at each site.

Guests upgrade in place. A visitor gets a guest User row on first load. Registering sets email and password_hash on that same row — so every adventure they played as a guest survives with no re-parenting and no migration step. Three kinds of row share the users table: local, guest, and registered.

3.2The shared demo key

The demo lets people play with no signup and no API key, on a key the server pays for. That is a spending surface, so it’s the most defended code in the project.

One function makes the BYOK-vs-demo decision, and on the demo branch it pins two things:

Plus a daily per-user turn cap (default 20), checked before the player’s input is stored so a capped player doesn’t get their message saved with no reply, and counted only after a successful turn.

A real bug, recorded in a comment

There’s a defensive check that raises if a demo config somehow carries a non-whitelisted model. It tests using_demo, not api_key == DEMO_API_KEY. Keying on the key value looks stricter but is wrong — the demo key is an ordinary OpenRouter key, so a user can legitimately paste that same key into their own settings as BYOK, and then every resolution raised, 500ing even GET /auth/me and taking the whole SPA down. using_demo is what actually means “the server is paying”.

3.3Secrets

Everything derives from one server-side secret.

ThingMechanism
Passwordshashlib.scrypt, N=2¹⁴, r=8, p=1, per-password salt, constant-time compare. Stdlib, so no extra dependency.
Sessionsv1.<user_id>.<HMAC-SHA256>, no expiry — long-lived guest sessions are the point.
Stored LLM API keysFernet encryption at rest, key derived from the secret, enc: prefix so legacy plaintext rows are recognisable and migratable.

The secret auto-generates into a file next to the database for local installs (zero config), but multi-user mode refuses to start without the env var — with an error that explains why and gives the command to generate one. Hosted filesystems are ephemeral; a regenerated secret on every deploy would silently log out every user and orphan their stored API keys.

A rotated secret makes stored keys undecryptable. Decryption treats that as “unset” rather than raising, so the user just re-enters their key instead of hitting a 500.

3.4Abuse guards

GuardLimit
Turn generation10 / min
Auth attempts (per IP)10 / 5 min
Guest creation (per IP)30 / 5 min
Script test runs30 / min
Connection test10 / min
Adventures / scenarios / scripts per user100 / 200 / 200
Actions per adventure5,000
Request body2 MB (20 MB on import)

Rate limits are keyed per user when one is known — accounts survive IP changes — and per IP otherwise, in fixed windows held in memory, with a pruning pass so the per-IP dict can’t grow without bound. Import endpoints check bundle list lengths against the same caps live creation enforces, otherwise the cap is trivially bypassed by uploading a file.

Security headers on every response: nosniff, X-Frame-Options: DENY, Referrer-Policy: same-origin, and a CSP allowing exactly what the SPA uses.

3.5Deployment

One Docker web service on Render, serving the SPA and the API same-origin, with Postgres on Neon.

The Postgres decision was forced: Render’s free tier has no persistent disk, so a SQLite file wouldn’t survive a deploy. The database lives off-box.

Two things worth knowing about the free tier:

CI runs the backend tests, the frontend lint and build, and a Docker image build on every push.

Part 4

The web plumbing, briefly

The parts that are just how the web works, not decisions.

Frontend and backend are two programs. In development they’re two servers — Vite on 5173 serving React, FastAPI on 8000 serving the API — and Vite proxies /api to FastAPI so the browser thinks it’s all one origin, which avoids CORS entirely. In production there’s one server: FastAPI serves the built React files as static assets from the same port.

SPA routing. React Router handles URLs like /play/3 in the browser without a round trip. But if you reload that URL, the browser asks the server for /play/3, which isn’t a file. So the static-file handler catches the 404 and returns index.html, letting React take over and read the URL itself. API routes are matched before the static mount, so they’re unaffected.

Sessions. A cookie is a small value the browser stores and automatically attaches to every request to that site. Here it holds v1.<user_id>.<signature>. The server doesn’t store sessions anywhere — it re-verifies the signature on each request, which is why there’s no session table.

The 401 retry. If the cookie is missing or stale, any API call returns 401. The frontend catches that once, calls /api/auth/me — which mints a fresh guest session — and retries the original request. So a returning visitor with an expired cookie never sees an error.

React, in one paragraph. A component is a function that returns a description of some UI. useState holds a value; changing it re-renders the component. The streaming turn is the clearest example: each SSE chunk appends to a state string, React re-renders, and the text appears to type itself.

Part 5

Results and limitations

What was measured, and what this design knowingly does not do.

5.1Measured results

Database egress per adventure load38.5 MB → 0.20 MB (~189×)
Prompt snapshot size~74 KB/turn, 94% of the DB
Turn read cost at turn 200839 KB → 129 KB, flat after ~turn 50
Length-hint phrasing174 → 246 words as a budget; 170 as a ceiling (n=5)
Backend tests151, LLM mocked, real QuickJS engine
Schema versions37
Sandbox limits16 MB, 2 s CPU, fresh context per run
Context defaultsauthor’s note at depth 3; cards ≤ 40% of elastic budget
Memory cadencememory / 6 turns, summary / 15 turns, top-5 retrieval

Two of the tests encode a performance property rather than a behaviour: test_egress.py asserts on the SQL the ORM emits, and test_history_window.py asserts that the read cost stops growing with story length.

5.2Known limitations

Deliberate trades for a single-user-first app that also happens to be hosted, listed so nobody has to discover them the hard way.

5.3Cleanup backlog

docs/self-review.md carries an open list of non-bugs — reuse, simplification and efficiency items — kept deliberately separate from the correctness list, which is empty. The largest ones: