Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
62a997f364 | ||
|
|
279a871a77 | ||
|
|
e08d49c3eb | ||
|
|
3c8e91f644 | ||
|
|
d27ee34901 | ||
|
|
c8755c21c2 | ||
|
|
7f082b61d8 | ||
|
|
903fa7a74f | ||
|
|
2fdd2547f0 | ||
|
|
8652fe7cd8 | ||
|
|
8c65ae99de |
@@ -46,6 +46,9 @@ ollama pull qwen2.5:3b-instruct
|
||||
ollama pull nomic-embed-text # only if you want the memory bank
|
||||
```
|
||||
|
||||
There is no account to create and nothing to log in to. The application is
|
||||
single-user: whoever can reach it on loopback is its owner.
|
||||
|
||||
## Running
|
||||
|
||||
**Development** — backend on `:8000`, Vite dev server on `:5173`:
|
||||
@@ -88,21 +91,52 @@ decision that this project's threat model does not cover
|
||||
|
||||
## Pointing the storyteller at Ollama
|
||||
|
||||
The endpoint, model and (unused) API key are **runtime settings stored in the
|
||||
database**, not environment variables. Set them on the app's Settings page, or
|
||||
with one request:
|
||||
The endpoint, the model and the generation parameters are **runtime settings
|
||||
stored in the database**, not environment variables. There is no API key field:
|
||||
M2 removed it along with the cloud providers, and Ollama does not use one. Set
|
||||
them on the app's Settings page, or with one request:
|
||||
|
||||
```bash
|
||||
curl -X PUT http://127.0.0.1:8000/api/settings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"endpoint_url":"http://127.0.0.1:11434/v1","model":"qwen2.5:3b-instruct",
|
||||
"api_mode":"chat","api_key":"","max_output_tokens":200,
|
||||
"api_mode":"chat","max_output_tokens":200,
|
||||
"context_token_budget":4096}'
|
||||
```
|
||||
|
||||
`POST /api/settings/test` (the **Test connection** button) returns
|
||||
`{"ok": true, "models": [...]}` and is the fastest way to tell a wrong endpoint
|
||||
from a missing model.
|
||||
from a missing model. When it fails it says which kind of failure it was, and
|
||||
they need different things done about them:
|
||||
|
||||
| `kind` | What it means |
|
||||
| --- | --- |
|
||||
| `rejected` | The endpoint is outside the policy below. Not a network problem. |
|
||||
| `unreachable` | Nothing answered. Ollama is not running there, or the port is wrong. |
|
||||
| `tls` | The certificate did not verify — install the CA (see below). |
|
||||
| `timeout` | It accepted the connection and then said nothing. |
|
||||
| `http` | It answered with an error status; the body is included. |
|
||||
|
||||
A successful test also warns when the endpoint is reachable but has no model by
|
||||
the configured name, which is the commonest way for a correct endpoint to still
|
||||
fail every turn.
|
||||
|
||||
### Which endpoints are allowed
|
||||
|
||||
`backend/app/endpoints.py` decides, and it is deliberately narrow: **loopback,
|
||||
your own LAN, or nothing.** The allowed networks are `127.0.0.0/8`, the three
|
||||
RFC1918 ranges, link-local, IPv6 loopback and unique-local, and `100.64.0.0/10`
|
||||
(carrier-grade NAT, which is what a mesh VPN such as Tailscale hands out).
|
||||
|
||||
Every address the endpoint's hostname resolves to must be in one of them. A
|
||||
public address is refused, a name resolving to both a private and a public
|
||||
address is refused, and known cloud inference hosts are refused by name so the
|
||||
error says why rather than looking like a DNS fault.
|
||||
|
||||
The rule is applied when you save the endpoint *and* again before every
|
||||
outbound request, so a database edited by hand or a hostname that starts
|
||||
resolving somewhere new cannot turn a local install into an exfiltration path.
|
||||
There is no setting to relax it.
|
||||
|
||||
### Same host (the default)
|
||||
|
||||
@@ -137,9 +171,8 @@ Use an IP address or a name your own network resolves. Then:
|
||||
- the inference machine needs the models installed, not the storyteller;
|
||||
- no Internet is involved in either direction.
|
||||
|
||||
`app/netguard.py` refuses private addresses only in hosted multi-user mode
|
||||
(`AIDND_MULTI_USER=1`), which local installs never turn on, so a LAN endpoint
|
||||
is accepted as configured.
|
||||
A LAN endpoint is accepted because it is on one of the allowed networks above.
|
||||
Nothing else about it is special.
|
||||
|
||||
#### If that endpoint is HTTPS with your own CA
|
||||
|
||||
@@ -176,7 +209,7 @@ visible from within.
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/python -m pytest tests/ -q # 648 tests
|
||||
cd backend && .venv/bin/python -m pytest tests/ -q # 698 tests
|
||||
cd frontend && npm run lint && npm run build
|
||||
```
|
||||
|
||||
@@ -192,6 +225,25 @@ suite as complete evidence.
|
||||
is lost from the union, or if a new HTTP client is added without the shared
|
||||
verification context.
|
||||
|
||||
M4 added `test_save_points.py`, which fails if restoring a Save Point starts
|
||||
deleting history, stops going through the active head, forks on its own, lets a
|
||||
Save Point on one campaign be restored through another, or lets deleting a branch
|
||||
take a Save Point with it. It also fails if listing Save Points goes back to one
|
||||
query per Save Point, or starts fetching narration to render the list.
|
||||
|
||||
`test_process_restart.py` is the durability guard: it starts the application as a
|
||||
real subprocess, kills it, and starts a second one against the same database. A
|
||||
Save Point that survived only because a Python object was still alive would pass
|
||||
an in-process test and fail a user's restart.
|
||||
|
||||
M2 added two more. `test_endpoint_policy.py` fails if the set of reachable
|
||||
addresses widens, or if either place the rule is applied stops applying it —
|
||||
it resolves hostnames through a stub, so it tests the policy rather than
|
||||
whatever DNS the machine has. `test_local_only_surface.py` fails if a removed
|
||||
subsystem comes back as a route, if an API key becomes settable again, if the
|
||||
model timeout stops being configurable or becomes unbounded, or if a supported
|
||||
start path stops binding loopback.
|
||||
|
||||
## What was made offline-safe, and how to check
|
||||
|
||||
Two runtime downloads were removed in Milestone M1. Both were invisible on a
|
||||
@@ -226,21 +278,24 @@ docker exec app python -c "import socket; socket.create_connection(('1.1.1.1',44
|
||||
# -> OSError: Network is unreachable, and story turns still work
|
||||
```
|
||||
|
||||
`planning/reports/M1-BASELINE-REPORT.md` records the run this procedure is
|
||||
`planning/archive/milestone-reports/M1-BASELINE-REPORT.md` records the run this procedure is
|
||||
taken from, including the packet captures.
|
||||
|
||||
## Things inherited from upstream that M1 deliberately did not touch
|
||||
## Things still inherited from upstream
|
||||
|
||||
These are M2's scope (`planning/BUILD-MILESTONES.md`), listed here so nobody
|
||||
reports them as new:
|
||||
M2 removed the hosted, cloud, account, analytics, Postgres/Render and QuickJS
|
||||
scripting surfaces outright — `PROVENANCE.md` lists exactly what went. What is
|
||||
left of upstream that a newcomer might report as a defect:
|
||||
|
||||
- hosted/multi-user/account/demo-key code, analytics tables, Postgres and
|
||||
Render deployment paths, and the OpenRouter default endpoint constant all
|
||||
still exist in the tree. None of them is reachable from a default local run,
|
||||
and none requires a cloud service.
|
||||
- `docs/*.html` is upstream's GitHub Pages project site and still links Google
|
||||
Fonts. It is not served by the application and is not part of any build.
|
||||
- `.github/workflows/ci.yml` is upstream's GitHub Actions pipeline. This
|
||||
- **Inert legacy tables and columns.** Five tables and four columns M2 emptied
|
||||
of meaning are still in the schema, unmapped, so an M1-era campaign database
|
||||
opens unchanged. Nothing reads or writes them. A cleanup migration waits for
|
||||
the schema to settle after M5 (`planning/BUILD-MILESTONES.md`).
|
||||
- **Dual-dialect migration code.** `backend/app/migrations.py` still carries
|
||||
SQLite/Postgres branches from upstream, although Postgres support itself is
|
||||
gone and SQLite is the only store. Same cleanup, same milestone.
|
||||
- **`.github/workflows/ci.yml`** is upstream's GitHub Actions pipeline. This
|
||||
repository lives on a self-hosted Gitea; the workflow is kept for provenance
|
||||
and is not what runs the tests here.
|
||||
- QuickJS campaign scripting is still present and still tested.
|
||||
- **No frontend tests.** `npm run lint && npm run build` is the whole frontend
|
||||
check. A test runner is M8's job.
|
||||
|
||||
@@ -7,20 +7,15 @@ RUN npm ci
|
||||
COPY frontend/ ./
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2 — build Python wheels (quickjs compiles from source if no wheel
|
||||
# matches, so keep the toolchain out of the final image)
|
||||
FROM python:3.12-slim AS python-build
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gcc make \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY backend/requirements.txt /tmp/requirements.txt
|
||||
RUN pip wheel --no-cache-dir -r /tmp/requirements.txt -w /wheels
|
||||
|
||||
# Stage 3 — runtime
|
||||
# Stage 2 — runtime. Every remaining dependency ships a wheel, so there is no
|
||||
# compile step and no toolchain to keep out of the image. The wheel-building
|
||||
# stage that used to sit here existed for quickjs, which compiled from source
|
||||
# and which M2 removed with campaign scripting.
|
||||
FROM python:3.12-slim
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=python-build /wheels /wheels
|
||||
RUN pip install --no-cache-dir /wheels/* && rm -rf /wheels
|
||||
COPY backend/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install --no-cache-dir -r /tmp/requirements.txt && rm /tmp/requirements.txt
|
||||
|
||||
# Layout mirrors the repo: main.py finds the SPA at ../../frontend/dist
|
||||
# relative to backend/app/main.py.
|
||||
@@ -38,14 +33,11 @@ EXPOSE 8000
|
||||
# invitation to put the storyteller on the LAN, which is single-user and
|
||||
# unauthenticated in local mode.
|
||||
WORKDIR /app/backend
|
||||
# --proxy-headers lets uvicorn fix up the request scheme (https) behind the
|
||||
# platform's edge. We deliberately do NOT pass --forwarded-allow-ips "*": that
|
||||
# made uvicorn trust the LEFTMOST X-Forwarded-For value, which the client fully
|
||||
# controls, so anyone could rotate the header to dodge the per-IP rate limits.
|
||||
# The client IP used for rate limiting is derived in limits._client_ip from the
|
||||
# hop the edge appends (rightmost), which a client cannot spoof past; tune with
|
||||
# AIDND_TRUSTED_PROXY_HOPS if the platform adds more proxy hops.
|
||||
# Single worker on purpose: the turn lock, rate limiter, and debug log are
|
||||
# in-process state.
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", \
|
||||
"--proxy-headers"]
|
||||
# Single worker on purpose: the turn lock and the debug log are in-process
|
||||
# state.
|
||||
#
|
||||
# No --proxy-headers. That existed for a hosted deployment behind a platform
|
||||
# edge, along with the per-IP rate limiting that read X-Forwarded-For. Neither
|
||||
# survives M2, and trusting a forwarded header on a loopback-published port
|
||||
# would be a way to lie to the app rather than a feature.
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
@@ -26,9 +26,11 @@ import is a merge of the pinned commit with `--allow-unrelated-histories`, so:
|
||||
|
||||
- `git log d72f7c1bda0f34fccd84afb7a25c34eb01c901de` shows the real upstream
|
||||
history, not a squashed snapshot;
|
||||
- upstream paths are unchanged (`backend/`, `frontend/`, `docs/`, …), so a
|
||||
later upstream commit can still be fetched and cherry-picked against
|
||||
matching files;
|
||||
- upstream code paths are unchanged (`backend/`, `frontend/`, …), so a later
|
||||
upstream commit can still be fetched and cherry-picked against matching
|
||||
files. Upstream's own documentation trees, `plan/` and `docs/`, were removed
|
||||
on 2026-09-03: they described the hosted, scripted, multi-user product this
|
||||
fork is not. They remain in this repository's history and in upstream;
|
||||
- the planning package that predates the fork keeps its own history on the
|
||||
other parent of the merge.
|
||||
|
||||
@@ -78,10 +80,33 @@ text ships beside them as `OFL-cinzel.txt`, `OFL-crimsonpro.txt` and
|
||||
Regenerate with `python3 frontend/tools/vendor_fonts.py`, which also rewrites
|
||||
`frontend/src/styles/fonts.css`.
|
||||
|
||||
## What this fork changed in Milestone M2
|
||||
|
||||
M2 is subtractive. It reduced the inherited application to the intended
|
||||
single-user, local-first trust boundary. **Nothing was added that upstream did
|
||||
not have, except the endpoint policy and the tests that hold these removals in
|
||||
place.**
|
||||
|
||||
Removed in full: campaign scripting and the QuickJS sandbox; multi-user
|
||||
accounts, guest sessions, login, registration and the shared demo key; the
|
||||
visitor analytics tables, dashboard and beacon; the access log; per-IP and
|
||||
per-user rate limiting and quotas; Render deployment config; Postgres/Neon
|
||||
support; cloud inference providers and the API-key field; session-cookie
|
||||
signing and API-key encryption at rest.
|
||||
|
||||
Added: `backend/app/endpoints.py`, which decides what an inference endpoint may
|
||||
be, and a configurable model timeout.
|
||||
|
||||
Three database tables (`scripts`, `adventure_scripts`, `analytics_daily`,
|
||||
`analytics_visitor_days`, `access_log`) and four columns (`adventures.script_state`,
|
||||
`settings.api_key`, `users.demo_turns_used`, `users.demo_turns_date`) are left
|
||||
in place, unmapped or inert, so that an existing M1 campaign database opens
|
||||
unchanged. They are not product functionality and nothing reads or writes them.
|
||||
|
||||
## What this fork changed in Milestone M1
|
||||
|
||||
Nothing was removed from upstream. The changes are the offline/locality
|
||||
hardening M1 called for; see `planning/reports/M1-BASELINE-REPORT.md` for the
|
||||
hardening M1 called for; see `planning/archive/milestone-reports/M1-BASELINE-REPORT.md` for the
|
||||
evidence.
|
||||
|
||||
- `backend/app/context/encoding.py` (new) and `backend/app/context/builder.py` —
|
||||
|
||||
@@ -1,39 +1,38 @@
|
||||
# AI D&D
|
||||
# Adventure Storyteller
|
||||
|
||||
[](https://github.com/parththakkar106/AI-DnD/actions/workflows/ci.yml)
|
||||
[](LICENSE)
|
||||
|
||||
An AI Dungeon-style interactive storytelling app that runs entirely on your own machine, with
|
||||
your own AI model. Create scenarios, play open-ended adventures where an LLM narrates the
|
||||
world, and extend the engine with **JavaScript scripts compatible with real AI Dungeon
|
||||
scripting**.
|
||||
An interactive storytelling app that runs entirely on your own machine, with your own model.
|
||||
Create scenarios and play open-ended adventures where a local LLM narrates the world, keeps
|
||||
track of what is true, and remembers what happened.
|
||||
|
||||
> ### ▶️ Try it live: **[parththakkar106.github.io/AI-DnD](https://parththakkar106.github.io/AI-DnD/)**
|
||||
> The project page loads instantly and launches the hosted demo in one tap. Play a scenario as
|
||||
> a guest: no sign-up and no API key needed. The demo runs on a free tier that sleeps, so the
|
||||
> first load after it's been idle takes about 30 to 60 seconds to wake up.
|
||||
This is the **Adventure Storyteller** fork of [AI-DnD](https://github.com/parththakkar106/AI-DnD).
|
||||
It is deliberately narrower than its upstream: single-user, local-only, and pointed at a model
|
||||
you run yourself. The hosted deployment, the accounts and sessions, the cloud provider support,
|
||||
the Postgres path, and the JavaScript scripting engine have all been removed rather than
|
||||
disabled. What is left is a storyteller you can run offline.
|
||||
|
||||
> **Local-only, by design.** The app talks to one place — an Ollama-compatible endpoint on this
|
||||
> machine or on a machine you control on your own network — and it refuses to be pointed at a
|
||||
> public address. There is no telemetry, no account, no cloud inference, and nothing is fetched
|
||||
> at runtime from the Internet.
|
||||
>
|
||||
> For the internals, read the **[design notes](https://parththakkar106.github.io/AI-DnD/guide.html)**.
|
||||
> They walk through the context budgeting, the world-state referee, and the memory bank, and
|
||||
> state the reasoning behind each one ([Markdown version](docs/GUIDE.md)).
|
||||
> For the internals, read [`planning/TECHNICAL-DESIGN.md`](planning/TECHNICAL-DESIGN.md) and
|
||||
> [`planning/CONTEXT-AND-MEMORY.md`](planning/CONTEXT-AND-MEMORY.md), which cover the context
|
||||
> budgeting, the state model and the memory bank as this fork builds them.
|
||||
|
||||
Built with FastAPI and SQLAlchemy on the backend and React (Vite) on the frontend, running on
|
||||
SQLite locally and Postgres in the cloud. It works with **any OpenAI-compatible endpoint**:
|
||||
Ollama and LM Studio locally, or OpenRouter, OpenAI, Groq, or vLLM in the cloud. Endpoint, key,
|
||||
and model are all runtime settings, and OpenRouter's free-tier models make the whole experience
|
||||
cost nothing.
|
||||
Built with FastAPI and SQLAlchemy on the backend and React (Vite) on the frontend, storing
|
||||
everything in one SQLite file.
|
||||
|
||||

|
||||
|
||||
*The play screen. The left rail shows live world state. The AI proposes changes each turn, and
|
||||
a Python engine decides what actually sticks. The chip under the narration reports what
|
||||
changed. The `‹ 2/2 ›` under a turn steps between the takes it has. Writing below a take that
|
||||
isn't the live one starts a new branch.*
|
||||
On the play screen, the left rail carries live world state. The AI proposes changes each turn
|
||||
and a Python engine decides what actually sticks; the chip under the narration reports what
|
||||
changed. The `‹ 2/2 ›` under a turn steps between the takes it has, and writing below a take
|
||||
that isn't the live one starts a new branch.
|
||||
|
||||
## Features
|
||||
|
||||
- **The full play loop.** Do / Say / Story / Continue actions, streamed AI responses (SSE),
|
||||
retry, undo, and edit. Reasoning models are supported: "thinking" streams into a collapsible
|
||||
retry, undo, redo, and edit. Reasoning models are supported: "thinking" streams into a collapsible
|
||||
💭 panel with its own token budget.
|
||||
- **A branching story tree.** The story is a tree, not a list. Any turn can hold more than one
|
||||
**take**, and `‹ 2/4 ›` steps between them. Stepping is free: the story below simply empties,
|
||||
@@ -47,7 +46,8 @@ isn't the live one starts a new branch.*
|
||||
cast; the adventure carries their live values. The AI proposes deltas, and a Python engine
|
||||
referees them: it clamps values to range, enforces per-turn caps and cooldowns, keeps counters
|
||||
monotonic and milestones sticky, then strips the machine-readable block out of the prose
|
||||
(`backend/app/worldstate/engine.py`). Word-labeled bands (`40–60: minor damage`) make the
|
||||
(`backend/app/worldstate/`: `apply.py` clamps, `parse.py` reads the block back).
|
||||
Word-labeled bands (`40–60: minor damage`) make the
|
||||
model reliable at it. No dice and no scripting are required.
|
||||
- **AI Dungeon-compatible context engine.** Memory, author's note, and story cards (world
|
||||
info) are triggered by keywords in recent story text, then assembled under a token budget
|
||||
@@ -55,41 +55,54 @@ isn't the live one starts a new branch.*
|
||||
- **Insights: total prompt transparency.** Every turn stores the exact prompt sent to the
|
||||
model. Open 🔍 on any AI action to see each context component, its token cost, and why it was
|
||||
included.
|
||||
- **JavaScript scripting, AI Dungeon-compatible.** `onInput` / `onModelContext` / `onOutput`
|
||||
modifiers share `state` and a `worldEntries` API, and run in an embedded quickjs sandbox
|
||||
(`backend/app/scripting/`). Real AI Dungeon scripts import and run as is. An in-app CodeMirror
|
||||
editor is included.
|
||||
- **Auto-summarization and Memory Bank.** The modern AI Dungeon memory system: AI-generated
|
||||
memories every few actions, a running story summary, and embedding-based retrieval that
|
||||
pulls old-but-relevant facts back into context, with similarity scores visible in Insights
|
||||
(`backend/app/memorybank.py`).
|
||||
- **Undo and retry that actually roll back state.** Undo and retry roll back the world state
|
||||
and script state to a per-node snapshot, not just the text, and prune the memories that
|
||||
covered the removed turns. Nothing a retry replaces is discarded: the old attempt stays as
|
||||
another take of that turn, one keystroke and one click from becoming a branch of its own.
|
||||
- **Import and export.** AI Dungeon-compatible formats for scripts and scenarios; JSON for
|
||||
everything else. An adventure exports as `ai-dnd-adventure-v2`, which carries the whole tree:
|
||||
every branch, every take, and the fork points, since those were chosen rather than computed.
|
||||
Files saved in the old single-line format still import.
|
||||
- **Optional accounts for hosted deployments.** By default the app is single-user with zero
|
||||
auth friction. Set `AIDND_MULTI_USER=1` and visitors play instantly as guests (signed
|
||||
session cookie), can register (email and password) at any point to keep their data, and each
|
||||
user gets isolated data plus their own encrypted-at-rest API key. A server-funded **shared
|
||||
demo key** with a daily turn cap lets people try it without bringing a key
|
||||
(`backend/app/auth.py`). Each new guest is also given a copy of a short pre-played
|
||||
adventure, so the first screen shows real turns and their world-state changes without
|
||||
spending a demo turn (`backend/app/starter.py`).
|
||||
- **Undo, Redo, and retry that roll back state and delete nothing.** Undo moves where the story
|
||||
is being read; it removes no accepted turn, so Redo can walk forward into the turns it stepped
|
||||
over. Both restore the world state from a per-node snapshot rather than just the text, and a
|
||||
memory derived from a turn now behind the head stops being retrieved without being deleted or
|
||||
re-embedded. Writing a new turn below a moved-back head is the moment the story forks: the
|
||||
displaced future stays on the line it was written for, and ordinary Redo stops offering it.
|
||||
Nothing a retry replaces is discarded either — the old attempt stays as another take of that
|
||||
turn, one keystroke and one click from becoming a branch of its own.
|
||||
- **Save Points.** Name a moment — "Before entering the abbey" — keep playing,
|
||||
restart the app, and come back to it. Restoring one moves the story back to
|
||||
that moment and deletes nothing: the turns you wrote after it stay, Redo still
|
||||
walks forward into them, and writing something different from the Save Point
|
||||
is what starts a new line while the old one is kept. A Save Point is a name for
|
||||
a position and holds no copy of the story, so restoring it is the same
|
||||
movement Undo makes (`backend/app/routers/adventures/checkpoints.py`,
|
||||
`backend/app/head.py`). They last until *you* delete them: deleting one deletes
|
||||
no story, and deleting a branch a Save Point is kept on is refused until you
|
||||
remove the Save Point yourself, so nothing takes a named moment away behind
|
||||
your back.
|
||||
- **Import and export.** AI Dungeon-compatible scenario format; JSON for everything else. An adventure exports as `ai-dnd-adventure-v2`, which carries the whole tree:
|
||||
every branch, every take, the fork points, which branches the story has left behind, the Save
|
||||
Points and the position it is being read at — all of them chosen rather than computed, which is
|
||||
the rule for what a bundle carries. A campaign opens where its head says, never at a Save Point
|
||||
merely because it has one. A campaign exported after two Undos imports still undone, with its
|
||||
retained future intact, instead of silently reopening at its newest turn. Files that predate
|
||||
the head position, and files saved in the old single-line format, still import.
|
||||
- **Single user, no accounts.** There is no sign-up, no login, no session and no API key
|
||||
anywhere in the product. The storyteller API binds to loopback and is unauthenticated by
|
||||
design, because the only person who can reach it is the person running it. A new install
|
||||
starts with a short pre-played adventure, so the first screen shows real turns and their
|
||||
world-state changes rather than an empty page (`backend/app/starter.py`).
|
||||
- **A refusal you can rely on.** The inference endpoint is checked against an address
|
||||
allowlist when you save it and again before every request, so a public endpoint is refused
|
||||
even if the setting is edited in the database directly. TLS verification is never traded
|
||||
against reachability: a privately issued certificate is verified against your machine's own
|
||||
trust store, and there is no bypass switch.
|
||||
|
||||
## Screenshots
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
|  |  |
|
||||
| **Insights**: the exact prompt for the next turn, broken into components with token counts and the trigger word that pulled each story card in. | **Authoring**: stats with ranges, per-turn caps, cooldowns, and word-labeled bands; NPCs the AI addresses by id. |
|
||||
|  |  |
|
||||
| **Scripting**: the three AI Dungeon hooks with shared persistent `state`, run in a quickjs sandbox. | **Home**: continue a story in progress or start from a scenario. |
|
||||
|  |  |
|
||||
| **The tree**: one lane per line, from the moment it left its parent to the moment it ends. The horizontal axis is the story's own clock, so a short branch reads as short. | **Branches**: every line the story has taken, and the three things you can do to one. A line the one you're reading was forked from can't be deleted, and says so. |
|
||||
None yet. The inherited screenshots showed upstream's UI — a Scripts tab, Log in and Sign up,
|
||||
a guest banner, scripting demo scenarios — none of which this fork has since M2, so they were
|
||||
removed rather than left standing as a picture of a product that no longer exists. The M4
|
||||
closeout drove the real application in a real browser, so the screens exist and work; taking
|
||||
presentable screenshots of them is a job for the UI pass in M8.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -121,58 +134,67 @@ Open http://localhost:5173.
|
||||
|
||||
## Connect a model
|
||||
|
||||
Open **Settings** in the app and point it at any OpenAI-compatible endpoint:
|
||||
Ollama is the inference backend v1 supports. Open **Settings** in the app and point it at one:
|
||||
|
||||
| Provider | Endpoint URL | Notes |
|
||||
| Where Ollama runs | Endpoint URL | Notes |
|
||||
|---|---|---|
|
||||
| Ollama (local) | `http://localhost:11434/v1` | free, private; also serves embedding models for the Memory Bank (e.g. `nomic-embed-text`) |
|
||||
| LM Studio (local) | `http://localhost:1234/v1` | free, private |
|
||||
| OpenRouter | `https://openrouter.ai/api/v1` | `:free` models cost nothing (no embeddings on the free tier) |
|
||||
| OpenAI / Groq / vLLM / … | provider's `/v1` URL | anything speaking `/v1/chat/completions` |
|
||||
| Claude Code CLI (local) | `http://127.0.0.1:8787/v1` | your Claude subscription instead of an API key; see [Playing against Claude locally](#playing-against-claude-locally) |
|
||||
| The same machine | `http://localhost:11434/v1` | the default, and the simplest thing that works |
|
||||
| A machine on your own network | `http://<host>:11434/v1` or `https://<host>/v1` | explicitly configured; see below |
|
||||
|
||||
Model name, API key, generation parameters, and (optionally) summary and embedding models for
|
||||
the Memory Bank are all configured there too. No config files and no rebuild are needed.
|
||||
Model name, generation parameters, and (optionally) summary and embedding models for the
|
||||
Memory Bank are configured there too. No config files and no rebuild are needed. There is no
|
||||
API key field, because there is nothing to authenticate to.
|
||||
|
||||
### Playing against Claude locally
|
||||
The adapter underneath speaks the OpenAI-compatible protocol, because that is what Ollama
|
||||
serves. That is an implementation detail, not a promise of support for arbitrary local
|
||||
servers that happen to speak the same protocol. Public and cloud inference endpoints are
|
||||
prohibited outright — see `planning/DECISIONS/002-ollama-only-v1.md` and
|
||||
`planning/DECISIONS/011-local-inference-endpoint-policy.md`.
|
||||
|
||||
`backend/tools/claude_shim.py` serves an OpenAI-compatible endpoint backed by the
|
||||
`claude` command line tool, so you can play the demos against a real model without an
|
||||
API key. Each request spawns one `claude --print` process, which suits the turn engine:
|
||||
the app assembles the whole prompt every turn and expects a stateless endpoint.
|
||||
### What the endpoint policy allows
|
||||
|
||||
The address is checked when you save it and again before every request. Only loopback and
|
||||
private-network addresses are accepted; every public address is refused, by address rather than
|
||||
by hostname, so a name that resolves outward is refused too. A well-known cloud inference host
|
||||
is named in the error message only so the refusal says *why*.
|
||||
|
||||
Running the model on a second machine you control is supported and expected — that machine
|
||||
does the inference while the storyteller itself stays bound to loopback on yours. If that
|
||||
machine serves HTTPS with a certificate from a CA you installed, it works: certificates are
|
||||
verified against your operating system's trust store as well as the bundled one. Verification
|
||||
itself is never relaxed, and there is no option to turn it off.
|
||||
|
||||
### Playing against a local shim (development only)
|
||||
|
||||
`backend/tools/claude_shim.py` serves an OpenAI-compatible endpoint on `127.0.0.1:8787`
|
||||
backed by a command-line tool, which is useful for testing the turn engine against a stronger
|
||||
model. Each request spawns one process, which suits the engine: the app assembles the whole
|
||||
prompt every turn and expects a stateless endpoint.
|
||||
|
||||
```sh
|
||||
cd backend
|
||||
.venv/Scripts/python.exe tools/claude_shim.py # listens on 127.0.0.1:8787
|
||||
.venv/bin/python tools/claude_shim.py # listens on 127.0.0.1:8787
|
||||
```
|
||||
|
||||
In Settings, choose the OpenAI-compatible provider, set the base URL to
|
||||
`http://127.0.0.1:8787/v1`, put any non-empty string in the API key field, and pick
|
||||
`sonnet`. The shim ignores the key and authenticates as you, through the CLI. Set the
|
||||
reasoning budget to `0` or `-1`: a positive budget sends a `reasoning.max_tokens` field
|
||||
that Claude 5 models reject.
|
||||
Set the base URL to `http://127.0.0.1:8787/v1` and pick a model the tool offers. Set the
|
||||
reasoning budget to `0` or `-1`: a positive budget sends a `reasoning.max_tokens` field that
|
||||
some models reject. Embeddings are not served — leave the embedding model blank, or point the
|
||||
Memory Bank at an endpoint that serves one.
|
||||
|
||||
Embeddings are not served. Leave the embedding model blank, or point the Memory Bank at
|
||||
a real endpoint.
|
||||
|
||||
Run it against a local backend only. The endpoint has no authentication, and anything
|
||||
reaching it spends your Claude quota. `app/netguard.py` blocks localhost endpoints when
|
||||
`AIDND_MULTI_USER` is set, so a deployed instance cannot be pointed at it.
|
||||
The shim has no authentication and spends whatever quota backs it, so run it on loopback and
|
||||
leave it there.
|
||||
|
||||
## How a turn works
|
||||
|
||||
```
|
||||
player input
|
||||
→ onInput script modifier
|
||||
→ assemble context: [narrator prompt] + [world state + stat guide] + [AI instructions]
|
||||
+ [plot essentials] + [story summary] + [retrieved memories]
|
||||
+ [triggered story cards] + [history along this branch, token-budgeted]
|
||||
+ [author's note] + [player action]
|
||||
→ onModelContext script modifier
|
||||
→ snapshot context (Insights)
|
||||
→ provider adapter → AI (streamed)
|
||||
→ extract + referee the world-state delta block, strip it from the prose
|
||||
→ onOutput script modifier
|
||||
→ store & render
|
||||
```
|
||||
|
||||
@@ -180,18 +202,18 @@ player input
|
||||
|
||||
```
|
||||
frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI
|
||||
├─ routers/ auth, scenarios, adventures, story cards, scripts, chat, settings, analytics, debug
|
||||
├─ models.py SQLAlchemy: User, Scenario, Adventure, Branch, Action, StoryCard, Script, Settings, Memory
|
||||
├─ migrations.py hand-rolled, versioned via PRAGMA user_version (64 and counting)
|
||||
├─ auth.py guest/registered users, sessions, shared demo key
|
||||
├─ security.py password hashing, cookie signing, API-key encryption
|
||||
├─ routers/ scenarios, adventures, story cards, chat, settings, debug
|
||||
├─ models.py SQLAlchemy: Scenario, Adventure, Branch, Action, StoryCard, Settings, Memory
|
||||
├─ migrations.py hand-rolled, versioned via PRAGMA user_version (79 and counting)
|
||||
├─ endpoints.py the inference-endpoint address policy
|
||||
├─ tlstrust.py one TLS context: the OS trust store unioned with certifi's
|
||||
├─ tree.py forking, promotion, and where a node is placed
|
||||
├─ head.py the active head: where the story is read, and what moving it costs
|
||||
├─ checkpoints Save Points: durable names for positions, in routers/adventures/
|
||||
├─ attempts.py the takes of one turn, grouped by parent
|
||||
├─ context/ prompt assembly under a token budget + lineage/history windowing
|
||||
├─ worldstate/ the stat engine: clamps, cooldowns, bands, milestones
|
||||
├─ scripting/ quickjs sandbox + AI Dungeon API surface
|
||||
├─ memorybank.py auto-summarization + embedding retrieval
|
||||
├─ analytics.py buffered visit counters + the owner's dashboard query
|
||||
├─ bundle.py the export/import formats, v2 (tree) and a v1 reader
|
||||
├─ providers/ OpenAI-compatible adapter, streaming
|
||||
└─ data.db SQLite (path overridable via AIDND_DB_PATH)
|
||||
@@ -202,9 +224,10 @@ development, Vite proxies `/api` to FastAPI.
|
||||
|
||||
## Tests
|
||||
|
||||
549 backend tests: unit tests plus full HTTP integration through the real quickjs scripting
|
||||
engine, with the LLM provider mocked. CI runs them on every push, alongside the frontend
|
||||
lint/build and a Docker image build.
|
||||
698 backend tests: unit tests plus full HTTP integration through the real turn engine, with
|
||||
the model provider mocked. They run with no route to the Internet, which is a requirement
|
||||
rather than a convenience — an offline claim proved on a machine that has been online once
|
||||
proves nothing.
|
||||
|
||||
```sh
|
||||
cd backend && pip install -r requirements.txt -r requirements-dev.txt
|
||||
@@ -233,59 +256,20 @@ most interesting engineering in the repo.
|
||||
the number of SQL clauses is bounded by the context window rather than by the number of
|
||||
forks.
|
||||
|
||||
## Visit analytics
|
||||
|
||||
The hosted demo keeps its own analytics: an owner-only dashboard at `/analytics` shows
|
||||
traffic, which shared scenarios get played, turns and demo-key spend, errors, and a funnel
|
||||
from *visited* to *played a turn* to *signed up*. It is visible only to the emails listed in
|
||||
`AIDND_ANALYTICS_EMAILS`, and the route returns 404 for everyone else.
|
||||
|
||||
This is built into the app rather than added with a third-party script, for reasons specific
|
||||
to this project: the CSP allows only `script-src 'self'`, ad blockers block the popular
|
||||
trackers, and none of those trackers can see the measurement that matters here, a turn. Counts
|
||||
are aggregated in memory and flushed as UPSERTs, so a visit is a write and never a read, and
|
||||
every dashboard query is a `GROUP BY` that returns tens of rows regardless of traffic volume.
|
||||
That matters: see the egress note above for what reading rows per request costs on this stack.
|
||||
|
||||
## Deploy (Render)
|
||||
|
||||
The repo ships a [`render.yaml`](render.yaml) blueprint: one Docker web service that serves
|
||||
the SPA and API same-origin, backed by external [Neon](https://neon.tech) Postgres. The free
|
||||
Render tier has no persistent disk, so the database lives off-box.
|
||||
|
||||
1. Create a **Neon** project and copy its pooled connection string.
|
||||
2. In Render, choose **New → Blueprint** and point it at this repo. Render reads
|
||||
`render.yaml`.
|
||||
3. Fill in the secrets it prompts for (`sync: false` vars): `AIDND_DATABASE_URL` (the Neon
|
||||
string); `AIDND_DEMO_API_KEY` and `AIDND_DEMO_MODELS` to offer a no-signup demo; and
|
||||
`AIDND_ANALYTICS_EMAILS` (your own account's email) to see the Visitors dashboard.
|
||||
`AIDND_SECRET_KEY` is generated automatically and stays stable across deploys.
|
||||
4. Deploy. Pushes to `main` auto-deploy after this. The health check is `/api/health`.
|
||||
|
||||
On the free tier the service sleeps after about 15 minutes idle, and the first request after
|
||||
that takes about 30 to 60 seconds to wake it. Point any keep-warm pinger at `/api/health`,
|
||||
which deliberately doesn't touch the database: waking the database around the clock costs far
|
||||
more than the cold start saves.
|
||||
|
||||
If you put another proxy or CDN in front of Render, set `AIDND_TRUSTED_PROXY_HOPS` to the
|
||||
number of proxies in the chain. It defaults to 1. The rate limiter reads the client IP that
|
||||
many entries from the right of `X-Forwarded-For`, because the trusted edge appends the real
|
||||
one last. Leave it at 1 behind two proxies and the limiter reads an entry the caller
|
||||
supplied, so anyone can rotate the header for a fresh rate-limit bucket per request.
|
||||
|
||||
## Repo notes
|
||||
|
||||
- `plan/` holds the phased implementation plan this project was built from, kept as a build
|
||||
log. All fourteen phases are complete. The later files (11, 12, 14) also serve as design
|
||||
notes for the state-revert, world-state, and story-tree work.
|
||||
[`plan/STATUS.md`](plan/STATUS.md) is the running thread: what shipped, what was measured,
|
||||
and what is owed next.
|
||||
- [`docs/GUIDE.md`](docs/GUIDE.md) holds design notes: how each subsystem works and why it was
|
||||
built that way, with the measurements behind the decisions. It is also rendered as a
|
||||
[reading page](https://parththakkar106.github.io/AI-DnD/guide.html).
|
||||
- `backend/.env.example` lists the few environment variables the backend reads.
|
||||
- [`docs/self-review.md`](docs/self-review.md) records a full-codebase self-review pass and
|
||||
what came out of it. All correctness findings are resolved.
|
||||
- `planning/` is this fork's own package: the product specification, the architecture
|
||||
decisions, the milestone plan, the acceptance contract, and a review report for every
|
||||
milestone shipped. Start at [`planning/README.md`](planning/README.md).
|
||||
- [`planning/archive/`](planning/archive/README.md) holds the Phase 0 research that chose this
|
||||
base and the completed milestone reports. It is history, not instruction.
|
||||
- [`DEVELOPMENT.md`](DEVELOPMENT.md) is how to set the project up, point it at a model, and run
|
||||
the tests. [`PROVENANCE.md`](PROVENANCE.md) records what came from upstream and what changed.
|
||||
- `backend/.env.example` lists the two environment variables the backend reads. Everything
|
||||
about the model is a runtime setting on the Settings page instead.
|
||||
- Upstream's own `plan/` build log and `docs/` project site were removed in the 2026-09-03
|
||||
documentation pass: they described AI-DnD's hosted, scripted, multi-user product. Both are
|
||||
still in Git history, and in upstream.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -1,110 +1,24 @@
|
||||
# Environment variables read by the backend.
|
||||
#
|
||||
# NOTE: the app reads real environment variables — it does NOT auto-load this
|
||||
# file. Set them in your shell, in docker-compose.yml, or in your host's
|
||||
# dashboard. This file is documentation (and a template for deploy configs).
|
||||
# file. Set them in your shell or in docker-compose.yml. This file is
|
||||
# documentation.
|
||||
#
|
||||
# There are two, and neither is required. Everything about the model — the
|
||||
# endpoint, the model names, the timeout, the context budget — is a runtime
|
||||
# setting stored in the database and edited on the Settings page, because it is
|
||||
# a preference rather than a deployment detail.
|
||||
|
||||
# Absolute path for the SQLite database file. Parent directory is created if
|
||||
# missing. Default when unset: backend/data.db
|
||||
# Docker compose sets this to /data/data.db (a named volume).
|
||||
# Absolute path for the SQLite database file. The parent directory is created
|
||||
# if missing. Default when unset: backend/data.db
|
||||
# docker-compose.yml sets this to /data/data.db (a named volume).
|
||||
AIDND_DB_PATH=
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 9 — production hardening
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Switch from SQLite to a server database (hosted deploys use Neon Postgres).
|
||||
# Any SQLAlchemy URL; postgres:// and postgresql:// schemes are rewritten to
|
||||
# the psycopg3 driver automatically. The platform-conventional DATABASE_URL
|
||||
# is honored too (AIDND_DATABASE_URL wins if both are set). Unset = SQLite.
|
||||
AIDND_DATABASE_URL=
|
||||
|
||||
# Comma-separated list of allowed CORS origins. Only needed when the frontend
|
||||
# is served from a different origin than the API; the production build is
|
||||
# served same-origin by FastAPI, so hosted deploys can leave this unset.
|
||||
# served same-origin by FastAPI, so a normal run can leave this unset.
|
||||
# Default: http://localhost:5173,http://127.0.0.1:5173 (the Vite dev server).
|
||||
#
|
||||
# A wildcard is rejected. The storyteller API is unauthenticated by design and
|
||||
# bound to loopback; letting any origin call it would undo that.
|
||||
AIDND_CORS_ORIGINS=
|
||||
|
||||
# How many proxy hops the rate limiter trusts in `X-Forwarded-For`. It reads
|
||||
# the entry that many places from the right, because the trusted edge appends
|
||||
# the real client IP last. Set this to the number of proxies in front of the
|
||||
# app. Default: 1, which is correct for a single edge such as Render.
|
||||
#
|
||||
# Get it wrong in either direction and the rate limits weaken. Too low reads an
|
||||
# entry the caller supplied, so anyone can rotate the header for a fresh
|
||||
# rate-limit bucket per request and walk past the auth and guest limits. Too
|
||||
# high reads past the real client. Only multi-user mode rate-limits at all, so
|
||||
# local installs can ignore this.
|
||||
AIDND_TRUSTED_PROXY_HOPS=
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 8 — optional accounts & multi-user (all optional; defaults keep the
|
||||
# app in frictionless single-user "local mode")
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# "1"/"true" turns on multi-user mode: guest sessions via signed cookies,
|
||||
# register/login UI, per-user data. Leave unset for local installs.
|
||||
AIDND_MULTI_USER=
|
||||
|
||||
# Secret for signing session cookies and encrypting stored API keys at rest.
|
||||
# If unset in local mode, one is auto-generated into `secret.key` next to the
|
||||
# database (fine for local/docker-volume runs). REQUIRED when
|
||||
# AIDND_MULTI_USER is on — the app refuses to start without it, because a
|
||||
# regenerated secret on an ephemeral hosted filesystem would log out every
|
||||
# user on each deploy. Generate one:
|
||||
# python -c "import secrets; print(secrets.token_urlsafe(48))"
|
||||
AIDND_SECRET_KEY=
|
||||
|
||||
# Session cookie Secure flag (HTTPS-only). Defaults to on when
|
||||
# AIDND_MULTI_USER is on, off otherwise — set 0/1 only to override (e.g. 0
|
||||
# when testing multi-user mode over plain http on a LAN address).
|
||||
AIDND_COOKIE_SECURE=
|
||||
|
||||
# --- Shared demo key (BYOK fallback; only active when AIDND_MULTI_USER=1) ---
|
||||
# Users with no API key of their own get this server-funded endpoint with a
|
||||
# model whitelist and a per-day turn cap. Unset = no demo, users must bring
|
||||
# their own key. Memory bank/auto-summarization are disabled on demo turns.
|
||||
AIDND_DEMO_API_KEY=
|
||||
# Default endpoint if unset: https://openrouter.ai/api/v1
|
||||
AIDND_DEMO_ENDPOINT_URL=
|
||||
# Comma-separated model whitelist. Default: google/gemma-4-26b-a4b-it:free
|
||||
AIDND_DEMO_MODELS=
|
||||
# Successful AI turns per user per day on the demo key. Default: 20
|
||||
AIDND_DEMO_TURNS_PER_DAY=
|
||||
# Comma-separated emails of "power users" (trusted testers) who bypass the daily
|
||||
# demo cap entirely — unmetered turns on the shared demo key — and get the AI Chat
|
||||
# page (a plain scratchpad for talking to a model, hidden from everyone else).
|
||||
# Registered accounts only (guests have no email). Matched case-insensitively.
|
||||
# Local (single-user) installs are always treated as power users.
|
||||
AIDND_POWER_USERS=
|
||||
|
||||
# --- Visit analytics ---
|
||||
# Comma-separated emails allowed to see the Visitors dashboard (/analytics) and
|
||||
# its nav link. Deliberately separate from AIDND_POWER_USERS: a trusted tester
|
||||
# gets unmetered turns, which is no reason to hand them the traffic numbers.
|
||||
# Unset = nobody sees it in a hosted deploy. Local installs always can, and are
|
||||
# the only mode where the viewer's own visits are still counted (excluding them
|
||||
# would leave the page permanently empty on the machine it's developed on).
|
||||
# Collection itself is always on; only the dashboard is gated.
|
||||
AIDND_ANALYTICS_EMAILS=
|
||||
# Days to keep the one-row-per-visitor-per-day table that makes the funnel
|
||||
# count people rather than clicks. The daily counters are aggregate and kept
|
||||
# forever. Default: 400. Set 0 to keep visitor-days forever.
|
||||
AIDND_ANALYTICS_RETENTION_DAYS=
|
||||
|
||||
# --- Guest retention (only active when AIDND_MULTI_USER=1) ---
|
||||
# Every first visit mints a guest account, so a public demo collects one row
|
||||
# per visitor. A guest with no activity for this many days is deleted along
|
||||
# with its scenarios, adventures and actions. Registered accounts are never
|
||||
# touched. Default: 5. Set 0 to keep guests forever.
|
||||
AIDND_GUEST_RETENTION_DAYS=
|
||||
# How often a running process re-checks. The sweep also runs once at startup,
|
||||
# which is what actually fires on hosts that sleep. Default: 6
|
||||
AIDND_CLEANUP_INTERVAL_HOURS=
|
||||
|
||||
# The AI endpoint/API key/model are NOT env vars — they are configured at
|
||||
# runtime in the app's Settings page and stored (encrypted) in the database.
|
||||
#
|
||||
# Rate limits, request size limits, and per-user row caps are hardcoded with
|
||||
# generous values (see backend/app/limits.py) and active only in multi-user
|
||||
# mode — local installs are never throttled.
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
"""The access log: who arrived, when, and from where.
|
||||
|
||||
The deliberate opposite of analytics.py. That module counts and stores nothing
|
||||
that points at a person; this one records addresses, email addresses and
|
||||
devices, because an access log that cannot identify the access is not an access
|
||||
log. The two live in separate modules and separate tables on purpose, so that
|
||||
the anonymity of the counters is a property of the code rather than a convention
|
||||
someone has to remember.
|
||||
|
||||
Owner-only, and never shown to the people it records.
|
||||
|
||||
Four kinds of row:
|
||||
|
||||
- `session` A browser that has a session made a request. For a guest, this
|
||||
is their first visit.
|
||||
- `login` An existing account signed in.
|
||||
- `register` A guest upgraded to an account.
|
||||
- `login_failed` A password attempt that did not match, with the address tried.
|
||||
|
||||
Session rows are the only ones that need thinning. `/auth/me` runs on every page
|
||||
load, and one row per load would be noise rather than a log. A row is written
|
||||
when the day or the address changes for that user. That is the granularity a log
|
||||
is read at, such as seen on the 3rd from 1.2.3.4, and it still records someone
|
||||
moving networks during a day.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import threading
|
||||
|
||||
from sqlalchemy import desc, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import analytics, models
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SESSION = "session"
|
||||
LOGIN = "login"
|
||||
REGISTER = "register"
|
||||
LOGIN_FAILED = "login_failed"
|
||||
|
||||
MAX_UA = 200
|
||||
|
||||
# user id -> (day, ip) of the last session row written for them. Process-local
|
||||
# like the rate limiter's windows, and for the same reason: this is a single
|
||||
# process, and the worst case after a restart is one redundant row per user.
|
||||
_last_session: dict[int, tuple[str, str]] = {}
|
||||
_guard = threading.Lock()
|
||||
_MAX_TRACKED = 10_000
|
||||
|
||||
|
||||
def _client_ip(request) -> str:
|
||||
# This import is deferred. `limits` imports `auth`, which the routers that
|
||||
# call this function import, so a module-level import here would create a
|
||||
# cycle. The spoof resistance lives in `limits` and must not be
|
||||
# reimplemented. A second, looser answer to which address belongs to the
|
||||
# client is how one of them ends up trusting a header it should not.
|
||||
from . import limits
|
||||
|
||||
return limits.client_ip(request)
|
||||
|
||||
|
||||
def describe(user: models.User) -> str:
|
||||
"""Returns how a user is named in the log.
|
||||
|
||||
A guest has no email, and their id is the only handle anyone has for them.
|
||||
The third case is a local install's implicit single user, who also has no
|
||||
email but is the operator rather than a visitor. Naming that user "Guest #1"
|
||||
would be wrong in the one row they are certain to read.
|
||||
"""
|
||||
if user.email:
|
||||
return user.email
|
||||
return f"Guest #{user.id}" if user.is_guest else f"Local user #{user.id}"
|
||||
|
||||
|
||||
def _country(request) -> str:
|
||||
"""Returns the edge's country header, or "" when there is none.
|
||||
|
||||
The blank differs from the counters' "(unknown)" label. A table column reads
|
||||
better as a dash than as a word, and an empty string is the correct value for
|
||||
a country that is not known.
|
||||
"""
|
||||
country = analytics.country_of(request.headers)
|
||||
return "" if country == analytics.UNKNOWN else country
|
||||
|
||||
|
||||
def record(
|
||||
db: Session,
|
||||
kind: str,
|
||||
request,
|
||||
*,
|
||||
user: models.User | None = None,
|
||||
who: str | None = None,
|
||||
) -> None:
|
||||
"""Writes one row.
|
||||
|
||||
This function never raises. The log observes sign-in rather than guarding it,
|
||||
and a logging failure must not lock anyone out.
|
||||
"""
|
||||
try:
|
||||
event = models.AccessEvent(
|
||||
kind=kind,
|
||||
user_id=user.id if user is not None else None,
|
||||
who=(who if who is not None else describe(user) if user else "")[:320],
|
||||
is_guest=bool(user.is_guest) if user is not None else False,
|
||||
ip=_client_ip(request)[:45],
|
||||
country=_country(request),
|
||||
device=analytics.device_of(request.headers.get("user-agent", "")),
|
||||
user_agent=(request.headers.get("user-agent") or "")[:MAX_UA],
|
||||
)
|
||||
db.add(event)
|
||||
db.commit()
|
||||
except Exception: # pragma: no cover - defensive
|
||||
db.rollback()
|
||||
logger.exception("Access log write failed; continuing.")
|
||||
|
||||
|
||||
def note_session(db: Session, user: models.User, request) -> None:
|
||||
"""Records that a session made a request, at most one row per day per address."""
|
||||
try:
|
||||
today = analytics._today()
|
||||
ip = _client_ip(request)
|
||||
with _guard:
|
||||
if _last_session.get(user.id) == (today, ip):
|
||||
return
|
||||
_last_session[user.id] = (today, ip)
|
||||
if len(_last_session) > _MAX_TRACKED:
|
||||
# Nothing here needs to persist. Clearing the map costs at most
|
||||
# one extra row per active user.
|
||||
_last_session.clear()
|
||||
_last_session[user.id] = (today, ip)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Access log session check failed; continuing.")
|
||||
return
|
||||
record(db, SESSION, request, user=user)
|
||||
|
||||
|
||||
def recent(
|
||||
db: Session,
|
||||
*,
|
||||
limit: int = 50,
|
||||
before_id: int | None = None,
|
||||
kind: str | None = None,
|
||||
query: str | None = None,
|
||||
) -> dict:
|
||||
"""Returns a page of the log, newest first.
|
||||
|
||||
The page is anchored on a row id rather than an offset, as the story pager
|
||||
is. Rows keep arriving while the log is read, and an offset would shift the
|
||||
page under whoever is reading it.
|
||||
"""
|
||||
statement = select(models.AccessEvent).order_by(desc(models.AccessEvent.id))
|
||||
if before_id is not None:
|
||||
statement = statement.where(models.AccessEvent.id < before_id)
|
||||
if kind:
|
||||
statement = statement.where(models.AccessEvent.kind == kind)
|
||||
if query:
|
||||
like = f"%{query.strip()}%"
|
||||
statement = statement.where(or_(
|
||||
models.AccessEvent.who.ilike(like),
|
||||
models.AccessEvent.ip.ilike(like),
|
||||
models.AccessEvent.country.ilike(like),
|
||||
))
|
||||
# Requesting one extra row reports whether more rows exist, without a
|
||||
# second COUNT over the whole table.
|
||||
rows = list(db.scalars(statement.limit(limit + 1)))
|
||||
has_more = len(rows) > limit
|
||||
return {"events": rows[:limit], "has_more": has_more}
|
||||
@@ -1,583 +0,0 @@
|
||||
"""Visit analytics for the hosted demo.
|
||||
|
||||
This is a small self-hosted counter that answers whether anyone visited and
|
||||
whether they played. It is built into the app rather than added with a
|
||||
third-party script, because the CSP in `main.py` allows scripts from 'self'
|
||||
only, ad blockers block the popular trackers, and none of those trackers can see
|
||||
what is worth knowing here: turns taken, demo-key spend, and which seeded
|
||||
scenario people pick.
|
||||
|
||||
Three rules shape the design:
|
||||
|
||||
1. It stores nothing personal. It records no IP addresses, no user agents, no
|
||||
user ids, and no title of anything a player wrote. A visitor appears only as
|
||||
an HMAC of their user id, which is one-way and salted with the app's secret
|
||||
key, so these tables cannot be joined back to an account even by someone
|
||||
holding the database. Story content never reaches this module. What one
|
||||
specific person did is unanswerable by design, and only totals are
|
||||
available.
|
||||
2. Egress is the budget. Neon bills for bytes leaving the database, and this
|
||||
project has already paid for forgetting that once. Counts are therefore
|
||||
aggregated in memory and flushed as UPSERTs, so a visit is a write and never
|
||||
a read, and every dashboard query is a GROUP BY that returns tens of rows
|
||||
rather than per-visit rows. A month of traffic costs a few kilobytes to read
|
||||
back.
|
||||
3. The numbers come from the server, not from the browser. The client reports
|
||||
one thing, which is the page that was viewed. Everything with meaning, such
|
||||
as a turn happening or an account being created, is recorded by the code that
|
||||
performs it, where a stranger cannot fake it and an extension cannot block
|
||||
it.
|
||||
|
||||
Storage is two tables, both bounded. `analytics_daily` holds one counter row per
|
||||
day, metric, and label, which is a few dozen rows a day.
|
||||
`analytics_visitor_days` holds one row per visitor per day carrying the funnel
|
||||
flags, which is what makes the funnel count people rather than clicks. It is the
|
||||
only table that grows with traffic, and cleanup ages it out.
|
||||
"""
|
||||
|
||||
import hmac
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from datetime import timedelta
|
||||
from hashlib import sha256
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from sqlalchemy import case, func, or_, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import models, security
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------- Metrics ----------
|
||||
# `metric` is the family, and `label` is the bucket within it. One generic
|
||||
# counter table is better than a column per measurement, because adding a new
|
||||
# question later costs nothing rather than a migration.
|
||||
|
||||
M_PAGE = "pageview"
|
||||
M_EVENT = "event"
|
||||
M_REFERRER = "referrer"
|
||||
M_DEVICE = "device"
|
||||
M_COUNTRY = "country"
|
||||
M_SCENARIO = "scenario" # Which seeded or public scenario was played.
|
||||
M_ERROR = "error" # "<status> <route>" for a 4xx or 5xx on /api.
|
||||
|
||||
EV_SCENARIO_OPEN = "scenario_opened"
|
||||
EV_ADVENTURE = "adventure_created"
|
||||
EV_IMPORT = "adventure_imported"
|
||||
EV_TURN = "turn"
|
||||
EV_DEMO_TURN = "demo_turn" # A turn billed to the shared demo key.
|
||||
EV_TURN_ERROR = "turn_error"
|
||||
EV_SIGNUP = "signup"
|
||||
EV_LOGIN = "login"
|
||||
|
||||
# Events that are also funnel steps. Recording one sets a flag on the visitor's
|
||||
# row for the day, so the funnel counts distinct visitor-days rather than repeat
|
||||
# clicks. This name-to-column map is the whole definition of the funnel, and the
|
||||
# dashboard reads it back in this order.
|
||||
FUNNEL_FLAGS = {
|
||||
EV_SCENARIO_OPEN: "opened",
|
||||
EV_ADVENTURE: "created",
|
||||
EV_TURN: "played",
|
||||
EV_SIGNUP: "signed_up",
|
||||
}
|
||||
|
||||
OTHER = "(other)"
|
||||
NONE_LABEL = "(direct)"
|
||||
UNKNOWN = "(unknown)"
|
||||
|
||||
# ---------- Bounds ----------
|
||||
# These bounds exist so that a hostile visitor can add rows to these tables no
|
||||
# faster than an honest one. The only label a client can influence is the
|
||||
# referrer, and together these caps mean the worst it can do is fill one day's
|
||||
# referrer list and then be folded into "(other)".
|
||||
|
||||
MAX_LABEL_LEN = 80
|
||||
MAX_LABELS_PER_METRIC = 200 # Distinct labels per metric per day, then OTHER.
|
||||
MAX_PENDING = 4000 # Buffered entries before an inline flush.
|
||||
FLUSH_INTERVAL_SECONDS = 60
|
||||
|
||||
# How long the per-visitor-day rows are kept. The daily counters are small and
|
||||
# are kept indefinitely. These rows are the ones that scale with traffic. A
|
||||
# visitor whose last visit ages out counts as new again, which is an acceptable
|
||||
# trade at this horizon and keeps the table from being a permanent record of
|
||||
# anyone.
|
||||
RETENTION_DAYS = int(os.environ.get("AIDND_ANALYTICS_RETENTION_DAYS", "400") or 400)
|
||||
|
||||
_HOST_OK = re.compile(r"^[a-z0-9.-]+$")
|
||||
_COUNTRY_OK = re.compile(r"^[A-Z]{2}$")
|
||||
_NUMERIC_SEGMENT = re.compile(r"^\d+$")
|
||||
|
||||
# SPA routes, in the form the dashboard shows them. Any other path a client
|
||||
# reports becomes OTHER, so the page list cannot be filled with junk and cannot
|
||||
# record which adventure someone is reading.
|
||||
KNOWN_ROUTES = {
|
||||
"/", "/adventures", "/scenarios", "/scenarios/:id", "/play/:id",
|
||||
"/scripts", "/scripts/:id", "/settings", "/chat", "/analytics",
|
||||
}
|
||||
|
||||
# ---------- In-process buffer ----------
|
||||
# The deployment is a single process, which is the same assumption `limits.py`
|
||||
# makes, so a plain dict under a lock is the whole design. Losing up to a minute
|
||||
# of counts to a hard restart is acceptable for traffic numbers, and the flusher
|
||||
# also runs on shutdown. On Render's free tier the service is idle when it
|
||||
# sleeps, so the buffer it sleeps on is empty.
|
||||
|
||||
_counts: dict[tuple[str, str, str], int] = {}
|
||||
_visits: dict[tuple[str, str], set[str]] = {} # (day, visitor) -> flags.
|
||||
_labels_seen: dict[tuple[str, str], set[str]] = {} # (day, metric) -> labels.
|
||||
_guard = threading.Lock()
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return models.utcnow().date().isoformat()
|
||||
|
||||
|
||||
def record(metric: str, label: str = "", *, n: int = 1) -> None:
|
||||
"""Adds `n` to one counter.
|
||||
|
||||
This function never raises. Analytics must not fail a request that it is
|
||||
only observing.
|
||||
"""
|
||||
try:
|
||||
day = _today()
|
||||
label = (label or "").strip()[:MAX_LABEL_LEN]
|
||||
with _guard:
|
||||
seen = _labels_seen.setdefault((day, metric), set())
|
||||
if label not in seen:
|
||||
if len(seen) >= MAX_LABELS_PER_METRIC:
|
||||
label = OTHER
|
||||
else:
|
||||
seen.add(label)
|
||||
key = (day, metric, label)
|
||||
_counts[key] = _counts.get(key, 0) + n
|
||||
pending = len(_counts) + len(_visits)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Analytics counter failed; continuing.")
|
||||
return
|
||||
if pending >= MAX_PENDING:
|
||||
flush()
|
||||
|
||||
|
||||
def visitor_id(user: models.User) -> str:
|
||||
"""Returns a stable, one-way handle for one visitor.
|
||||
|
||||
The handle is an HMAC of the user id under the app's secret key. It is
|
||||
stable, so a returning visitor can be distinguished from a new one. It is
|
||||
one-way, so nothing in the analytics tables points back at an account. It is
|
||||
keyed, so a client cannot compute one and claim to be someone else. One
|
||||
consequence follows: rotating `AIDND_SECRET_KEY` makes every returning
|
||||
visitor look new.
|
||||
"""
|
||||
digest = hmac.new(security.SECRET_KEY, f"visitor:{user.id}".encode(), sha256)
|
||||
return digest.hexdigest()[:32]
|
||||
|
||||
|
||||
def record_visit(user: models.User | None, *, flag: str | None = None) -> None:
|
||||
"""Records that this visitor was here today, and optionally sets one funnel
|
||||
flag.
|
||||
|
||||
Without a user the call does nothing. A page loaded before a session exists
|
||||
still counts as a pageview, but not as a person.
|
||||
"""
|
||||
if user is None:
|
||||
return
|
||||
try:
|
||||
with _guard:
|
||||
flags = _visits.setdefault((_today(), visitor_id(user)), set())
|
||||
if flag:
|
||||
flags.add(flag)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Analytics visit failed; continuing.")
|
||||
|
||||
|
||||
def record_event(name: str, user: models.User | None = None) -> None:
|
||||
"""Records one event, and credits the visitor's day if it is a funnel step.
|
||||
|
||||
This is the whole interface the call sites use.
|
||||
"""
|
||||
record(M_EVENT, name)
|
||||
record_visit(user, flag=FUNNEL_FLAGS.get(name))
|
||||
|
||||
|
||||
# ---------- Normalizing what the browser reports ----------
|
||||
|
||||
def normalize_route(path: str) -> str:
|
||||
"""Reduces a client-reported path to one of `KNOWN_ROUTES`.
|
||||
|
||||
Numeric segments become ":id". That bounds the label count, and it keeps
|
||||
which adventure someone opened out of the statistics.
|
||||
"""
|
||||
path = (path or "/").split("?")[0].split("#")[0]
|
||||
if not path.startswith("/"):
|
||||
path = "/" + path
|
||||
if len(path) > 1:
|
||||
path = path.rstrip("/")
|
||||
parts = [":id" if _NUMERIC_SEGMENT.match(p) else p for p in path.split("/")]
|
||||
route = "/".join(parts) or "/"
|
||||
return route if route in KNOWN_ROUTES else OTHER
|
||||
|
||||
|
||||
def normalize_referrer(referrer: str, own_host: str = "") -> str:
|
||||
"""Returns the sending site as a bare host.
|
||||
|
||||
This app's own host means an internal navigation, which is not a referral.
|
||||
In that case the function returns "", which tells the caller to skip it.
|
||||
"""
|
||||
if not referrer:
|
||||
return NONE_LABEL
|
||||
host = (urlsplit(referrer).hostname or "").lower().lstrip(".")
|
||||
if not host or not _HOST_OK.match(host) or len(host) > MAX_LABEL_LEN:
|
||||
return OTHER
|
||||
if host == (own_host or "").lower() or host in ("localhost", "127.0.0.1"):
|
||||
return ""
|
||||
return host[4:] if host.startswith("www.") else host
|
||||
|
||||
|
||||
def api_route_label(scope: dict, status: int) -> str:
|
||||
"""Returns an error bucket such as "500 /api/adventures/{adventure_id}".
|
||||
|
||||
The label uses the route template, never the request path. That keeps one
|
||||
bucket per endpoint rather than one per adventure id. It also bounds the
|
||||
table: an unmatched path is chosen entirely by the caller, so labeling by it
|
||||
would let anyone create rows by requesting arbitrary paths.
|
||||
"""
|
||||
template = getattr(scope.get("route"), "path", None)
|
||||
return f"{status} {template}" if template else f"{status} (unmatched)"
|
||||
|
||||
|
||||
def device_of(user_agent: str) -> str:
|
||||
"""Returns "mobile", "tablet", or "desktop", and nothing more specific.
|
||||
|
||||
The user-agent string itself is never stored, because it is a fingerprint
|
||||
and the useful answer is one word.
|
||||
"""
|
||||
ua = (user_agent or "").lower()
|
||||
if not ua:
|
||||
return UNKNOWN
|
||||
if any(bot in ua for bot in ("bot", "crawler", "spider", "headless", "preview")):
|
||||
return "bot"
|
||||
if "ipad" in ua or "tablet" in ua or ("android" in ua and "mobile" not in ua):
|
||||
return "tablet"
|
||||
if any(m in ua for m in ("mobi", "iphone", "ipod", "android", "phone")):
|
||||
return "mobile"
|
||||
return "desktop"
|
||||
|
||||
|
||||
# Geo headers an edge network may add. Render fronts services with a CDN that
|
||||
# can set `cf-ipcountry`, and the others cost nothing to check. A value is
|
||||
# trusted only if it looks like an ISO code, because a client can send any
|
||||
# header, so the worst case is a wrong country rather than an unbounded label.
|
||||
_GEO_HEADERS = ("cf-ipcountry", "x-vercel-ip-country", "x-geo-country", "x-country-code")
|
||||
|
||||
|
||||
def country_of(headers) -> str:
|
||||
for name in _GEO_HEADERS:
|
||||
value = (headers.get(name) or "").strip().upper()
|
||||
if _COUNTRY_OK.match(value) and value != "XX":
|
||||
return value
|
||||
return UNKNOWN
|
||||
|
||||
|
||||
# ---------- Flushing ----------
|
||||
|
||||
def _insert(db: Session):
|
||||
return sqlite_insert if db.get_bind().dialect.name == "sqlite" else pg_insert
|
||||
|
||||
|
||||
def _drain() -> tuple[dict, dict]:
|
||||
with _guard:
|
||||
counts, visits = _counts.copy(), _visits.copy()
|
||||
_counts.clear()
|
||||
_visits.clear()
|
||||
# The label sets bound cardinality within one day, so drop the
|
||||
# previous day's rather than grow a map that never shrinks.
|
||||
today = _today()
|
||||
for key in [k for k in _labels_seen if k[0] != today]:
|
||||
del _labels_seen[key]
|
||||
return counts, visits
|
||||
|
||||
|
||||
def _restore(counts: dict, visits: dict) -> None:
|
||||
"""Returns a failed flush's work to the buffer, so the next flush retries it."""
|
||||
with _guard:
|
||||
for key, n in counts.items():
|
||||
_counts[key] = _counts.get(key, 0) + n
|
||||
for key, flags in visits.items():
|
||||
_visits.setdefault(key, set()).update(flags)
|
||||
|
||||
|
||||
def flush(db: Session | None = None) -> None:
|
||||
"""Writes the buffer out. This is safe to call from anywhere and never raises."""
|
||||
counts, visits = _drain()
|
||||
if not counts and not visits:
|
||||
return
|
||||
own_session = db is None
|
||||
if own_session:
|
||||
from .database import SessionLocal
|
||||
db = SessionLocal()
|
||||
try:
|
||||
_write_counts(db, counts)
|
||||
_write_visits(db, visits)
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
_restore(counts, visits)
|
||||
logger.exception("Analytics flush failed; counts held for the next one.")
|
||||
finally:
|
||||
if own_session:
|
||||
db.close()
|
||||
|
||||
|
||||
def _write_counts(db: Session, counts: dict) -> None:
|
||||
if not counts:
|
||||
return
|
||||
table = models.AnalyticsDaily.__table__
|
||||
rows = [
|
||||
{"day": day, "metric": metric, "label": label, "hits": hits}
|
||||
for (day, metric, label), hits in counts.items()
|
||||
]
|
||||
stmt = _insert(db)(table).values(rows)
|
||||
db.execute(stmt.on_conflict_do_update(
|
||||
index_elements=["day", "metric", "label"],
|
||||
set_={"hits": table.c.hits + stmt.excluded.hits},
|
||||
))
|
||||
|
||||
|
||||
def _write_visits(db: Session, visits: dict) -> None:
|
||||
if not visits:
|
||||
return
|
||||
table = models.AnalyticsVisitorDay.__table__
|
||||
ids = {visitor for _, visitor in visits}
|
||||
# One indexed lookup decides new against returning for the whole batch. It
|
||||
# is the only read this module makes outside the dashboard, and it returns
|
||||
# short hashes for the visitors active right now, so the batch bounds it.
|
||||
known = set(db.scalars(
|
||||
select(models.AnalyticsVisitorDay.visitor)
|
||||
.where(models.AnalyticsVisitorDay.visitor.in_(ids))
|
||||
.distinct()
|
||||
))
|
||||
rows = [
|
||||
{
|
||||
"day": day,
|
||||
"visitor": visitor,
|
||||
"is_new": visitor not in known,
|
||||
**{column: column in flags for column in FUNNEL_FLAGS.values()},
|
||||
}
|
||||
for (day, visitor), flags in visits.items()
|
||||
]
|
||||
stmt = _insert(db)(table).values(rows)
|
||||
db.execute(stmt.on_conflict_do_update(
|
||||
index_elements=["day", "visitor"],
|
||||
# Flags only turn on, and `is_new` is absent on purpose. The first
|
||||
# write of a visitor's first day is what decided it.
|
||||
set_={
|
||||
column: or_(table.c[column], stmt.excluded[column])
|
||||
for column in FUNNEL_FLAGS.values()
|
||||
},
|
||||
))
|
||||
|
||||
|
||||
def purge_old_visitor_days(db: Session) -> int:
|
||||
"""Deletes visitor-day rows past the retention horizon.
|
||||
|
||||
The cleanup sweeper calls this. The daily counters are never purged, because
|
||||
they are aggregates, they are small, and this project keeps its history.
|
||||
"""
|
||||
if RETENTION_DAYS <= 0:
|
||||
return 0
|
||||
cutoff = (models.utcnow().date() - timedelta(days=RETENTION_DAYS)).isoformat()
|
||||
removed = db.query(models.AnalyticsVisitorDay).filter(
|
||||
models.AnalyticsVisitorDay.day < cutoff
|
||||
).delete(synchronize_session=False)
|
||||
db.commit()
|
||||
return removed or 0
|
||||
|
||||
|
||||
# ---------- Reading it back ----------
|
||||
# Every query below is an aggregate. The database does the counting and returns
|
||||
# tens of rows, however much traffic is behind them. No query here can return a
|
||||
# row that belongs to one visitor.
|
||||
|
||||
TOP_N = 12
|
||||
|
||||
|
||||
def _top(rows: list[dict], limit: int = TOP_N) -> list[dict]:
|
||||
return rows[:limit]
|
||||
|
||||
|
||||
def summary(db: Session, days: int = 30) -> dict:
|
||||
"""Returns everything the dashboard shows for the last `days` days, including
|
||||
today.
|
||||
|
||||
The function flushes first, so the numbers include the last minute.
|
||||
"""
|
||||
flush(db)
|
||||
today = models.utcnow().date()
|
||||
since = (today - timedelta(days=days - 1)).isoformat()
|
||||
daily = models.AnalyticsDaily
|
||||
visitor = models.AnalyticsVisitorDay
|
||||
|
||||
# 1. Every counter in the window, reduced to (metric, label) totals. The
|
||||
# page, referrer, country, device, scenario, and error tables all come
|
||||
# from this one pass rather than from a query each.
|
||||
by_metric: dict[str, list[dict]] = {}
|
||||
for metric, label, hits in db.execute(
|
||||
select(daily.metric, daily.label, func.sum(daily.hits))
|
||||
.where(daily.day >= since)
|
||||
.group_by(daily.metric, daily.label)
|
||||
):
|
||||
by_metric.setdefault(metric, []).append({"label": label, "hits": int(hits)})
|
||||
for rows in by_metric.values():
|
||||
rows.sort(key=lambda row: -row["hits"])
|
||||
events = {row["label"]: row["hits"] for row in by_metric.get(M_EVENT, [])}
|
||||
|
||||
# 2. The two per-day series the dashboard draws.
|
||||
pageviews_by_day = {
|
||||
day: int(hits)
|
||||
for day, hits in db.execute(
|
||||
select(daily.day, func.sum(daily.hits))
|
||||
.where(daily.day >= since, daily.metric == M_PAGE)
|
||||
.group_by(daily.day)
|
||||
)
|
||||
}
|
||||
turns_by_day = {
|
||||
day: int(hits)
|
||||
for day, hits in db.execute(
|
||||
select(daily.day, func.sum(daily.hits))
|
||||
.where(daily.day >= since, daily.metric == M_EVENT, daily.label == EV_TURN)
|
||||
.group_by(daily.day)
|
||||
)
|
||||
}
|
||||
|
||||
# 3. People, per day. There is one row per visitor per day, so COUNT(*) is
|
||||
# already the day's unique visitors and no DISTINCT is needed.
|
||||
visitors_by_day: dict[str, dict] = {}
|
||||
for day, total, fresh in db.execute(
|
||||
select(
|
||||
visitor.day,
|
||||
func.count(),
|
||||
func.sum(case((visitor.is_new, 1), else_=0)),
|
||||
)
|
||||
.where(visitor.day >= since)
|
||||
.group_by(visitor.day)
|
||||
):
|
||||
visitors_by_day[day] = {"visitors": int(total), "new": int(fresh or 0)}
|
||||
|
||||
# 4. The funnel over the whole window, counting each person once.
|
||||
# COUNT(DISTINCT CASE WHEN flag THEN visitor END) ignores the NULLs the
|
||||
# CASE leaves for everyone who did not reach that step.
|
||||
unique, unique_new, *reached = db.execute(
|
||||
select(
|
||||
func.count(func.distinct(visitor.visitor)),
|
||||
func.count(func.distinct(case((visitor.is_new, visitor.visitor)))),
|
||||
*[
|
||||
func.count(func.distinct(case((visitor.__table__.c[column], visitor.visitor))))
|
||||
for column in FUNNEL_FLAGS.values()
|
||||
],
|
||||
).where(visitor.day >= since)
|
||||
).one()
|
||||
|
||||
series = []
|
||||
for offset in range(days):
|
||||
day = (today - timedelta(days=days - 1 - offset)).isoformat()
|
||||
counted = visitors_by_day.get(day, {})
|
||||
series.append({
|
||||
"day": day,
|
||||
"visitors": counted.get("visitors", 0),
|
||||
"new": counted.get("new", 0),
|
||||
"pageviews": pageviews_by_day.get(day, 0),
|
||||
"turns": turns_by_day.get(day, 0),
|
||||
})
|
||||
|
||||
visits = sum(row["visitors"] for row in series)
|
||||
pageviews = sum(pageviews_by_day.values())
|
||||
turns = events.get(EV_TURN, 0)
|
||||
errors = by_metric.get(M_ERROR, [])
|
||||
return {
|
||||
"days": days,
|
||||
"since": since,
|
||||
"until": today.isoformat(),
|
||||
"generated_at": models.utcnow().isoformat(),
|
||||
"totals": {
|
||||
# `visitors` counts each person once for the window. `visits`
|
||||
# counts them once per day they returned, which is the closest
|
||||
# measure to "sessions" that does not track sessions.
|
||||
"visitors": int(unique),
|
||||
"new_visitors": int(unique_new),
|
||||
"visits": visits,
|
||||
"pageviews": pageviews,
|
||||
"turns": turns,
|
||||
"demo_turns": events.get(EV_DEMO_TURN, 0),
|
||||
"adventures": events.get(EV_ADVENTURE, 0),
|
||||
"signups": events.get(EV_SIGNUP, 0),
|
||||
"logins": events.get(EV_LOGIN, 0),
|
||||
"turn_errors": events.get(EV_TURN_ERROR, 0),
|
||||
"errors": sum(row["hits"] for row in errors),
|
||||
"turns_per_visit": round(turns / visits, 1) if visits else 0,
|
||||
"pages_per_visit": round(pageviews / visits, 1) if visits else 0,
|
||||
},
|
||||
"series": series,
|
||||
# Step 0 is everyone who arrived, so the drop-off between it and
|
||||
# "Opened a scenario" appears as a step like any other.
|
||||
"funnel": [{"step": "Visited", "count": int(unique)}] + [
|
||||
{"step": step, "count": int(count)}
|
||||
for step, count in zip(
|
||||
["Opened a scenario", "Started an adventure", "Played a turn", "Signed up"],
|
||||
reached,
|
||||
)
|
||||
],
|
||||
"pages": _top(by_metric.get(M_PAGE, [])),
|
||||
"referrers": _top(by_metric.get(M_REFERRER, [])),
|
||||
"countries": _top(by_metric.get(M_COUNTRY, [])),
|
||||
"devices": by_metric.get(M_DEVICE, []),
|
||||
"scenarios": _top(by_metric.get(M_SCENARIO, [])),
|
||||
"errors": _top(errors),
|
||||
"events": by_metric.get(M_EVENT, []),
|
||||
}
|
||||
|
||||
|
||||
# ---------- Background flusher ----------
|
||||
# This matches the start and stop pair in `cleanup`, so the lifespan in
|
||||
# `main.py` reads the same way for both. The interval bounds how much a hard
|
||||
# restart can lose.
|
||||
|
||||
async def _flush_loop() -> None:
|
||||
import asyncio
|
||||
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
while True:
|
||||
await asyncio.sleep(FLUSH_INTERVAL_SECONDS)
|
||||
# This is blocking database work, so keep it off the event loop, which
|
||||
# is also serving SSE turn streams.
|
||||
await run_in_threadpool(flush)
|
||||
|
||||
|
||||
def start_flusher():
|
||||
import asyncio
|
||||
|
||||
return asyncio.create_task(_flush_loop())
|
||||
|
||||
|
||||
async def stop_flusher(task) -> None:
|
||||
"""Cancels the loop and writes out whatever it was holding.
|
||||
|
||||
A deploy is the one restart that is both frequent and predictable, so it
|
||||
should not be what loses a minute of counts.
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
await run_in_threadpool(flush)
|
||||
@@ -41,11 +41,11 @@ from .context import lineage
|
||||
|
||||
# The slices of a context snapshot that belong to one attempt rather than to the
|
||||
# turn. They are the world-state delta the attempt proposed and what the engine
|
||||
# did with it, the script report, the model's literal reply, and the endpoint's
|
||||
# did with it, the model's literal reply, and the endpoint's
|
||||
# token accounting. Each attempt is its own API call, and a retry is the call
|
||||
# most likely to read the prompt back out of cache. Everything else in a snapshot
|
||||
# is the prompt, which is assembled once per turn.
|
||||
ATTEMPT_KEYS = ("world_state", "script", "raw_output", "usage")
|
||||
ATTEMPT_KEYS = ("world_state", "raw_output", "usage")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ reading
|
||||
@@ -152,7 +152,7 @@ def preceding(
|
||||
# ------------------------------------------------------------------ writing
|
||||
|
||||
def restore_state(adventure: models.Adventure, node: models.Action | None) -> None:
|
||||
"""Restores the script state and world state that `node` left behind.
|
||||
"""Restores the world state that `node` left behind.
|
||||
|
||||
A NULL snapshot means leave the live state as it is, never reset it. Rows
|
||||
written before SP4 that the migration could not derive an outcome for carry
|
||||
@@ -161,17 +161,16 @@ def restore_state(adventure: models.Adventure, node: models.Action | None) -> No
|
||||
"""
|
||||
if node is None:
|
||||
return
|
||||
if isinstance(node.state_after, dict):
|
||||
adventure.script_state = copy.deepcopy(node.state_after)
|
||||
if isinstance(node.world_state_after, dict):
|
||||
adventure.world_state = copy.deepcopy(node.world_state_after)
|
||||
|
||||
|
||||
def snapshot_outcome(adventure: models.Adventure, node: models.Action) -> None:
|
||||
"""Records on `node` the state of the adventure now that the node has played."""
|
||||
state = adventure.script_state if isinstance(adventure.script_state, dict) else {}
|
||||
world = adventure.world_state if isinstance(adventure.world_state, dict) else {}
|
||||
node.state_after = copy.deepcopy(state)
|
||||
# `state_after` held the scripting engine's shared state, which M2 removed.
|
||||
# The column stays for schema compatibility and is written empty.
|
||||
node.state_after = {}
|
||||
node.world_state_after = copy.deepcopy(world)
|
||||
|
||||
|
||||
|
||||
@@ -1,216 +1,44 @@
|
||||
"""Phase 8: user resolution, sessions, and the shared demo key.
|
||||
"""Resolving the one local user. **There is no authentication in this product.**
|
||||
|
||||
The `AIDND_MULTI_USER` environment variable selects one of two modes:
|
||||
The module keeps its name so the dependency every router already depends on
|
||||
keeps working, but nothing here authenticates anybody. The Adventure
|
||||
Storyteller is a single-user application that binds to loopback: whoever can
|
||||
reach the API is the person who started it, and there is nobody else to tell
|
||||
them apart from.
|
||||
|
||||
* Local mode, the default. Every request resolves to one automatically created
|
||||
local user. There are no cookies and no login UI, so a clone or a
|
||||
docker-compose run behaves like the single-user app from before Phase 8.
|
||||
* Multi-user mode, used for hosted deployments. Requests carry a signed session
|
||||
cookie. `GET /api/auth/me` creates a guest user on the first visit, and
|
||||
registering upgrades that guest in place so their data survives. A request
|
||||
without a valid session gets a 401, and the frontend re-establishes the
|
||||
session through `/me`.
|
||||
Upstream had two modes. `AIDND_MULTI_USER` selected a hosted deployment with
|
||||
signed session cookies, guest accounts, registration, login, a shared demo API
|
||||
key with a per-day cap, "power users", and an owner allowlist for the analytics
|
||||
dashboard. M2 removed all of it: this product has no hosted mode to protect, and
|
||||
every one of those surfaces was a way for the application to be reached by
|
||||
someone other than its owner.
|
||||
|
||||
The shared demo key, which is the fallback when a user brings no key of their
|
||||
own, is also configured here. A user whose settings hold no API key is routed to
|
||||
a server-funded endpoint with a model allowlist and a per-day turn cap.
|
||||
What is left is the local path that upstream already had. Every request
|
||||
resolves to one automatically created user row.
|
||||
|
||||
The `users` table and the `user_id` foreign keys on scenarios, adventures and
|
||||
settings stay. They are an **internal ownership detail**, not an account
|
||||
system: nothing creates a second user, nothing logs in, and no request carries
|
||||
an identity. They remain because rewriting them out would mean a migration
|
||||
across most of the schema to delete a column that costs nothing and keeps every
|
||||
existing M1 database readable.
|
||||
"""
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from datetime import timezone
|
||||
|
||||
from fastapi import Depends, HTTPException, Request
|
||||
from fastapi import Depends, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import models, security
|
||||
from . import models
|
||||
from .database import get_db
|
||||
|
||||
|
||||
def _env_flag(name: str) -> bool:
|
||||
return os.environ.get(name, "").strip().lower() in ("1", "true", "yes", "on")
|
||||
|
||||
|
||||
MULTI_USER = _env_flag("AIDND_MULTI_USER")
|
||||
|
||||
SESSION_COOKIE = "aidnd_session"
|
||||
# Secure cookies are on by default in multi-user mode, because a hosted
|
||||
# deployment serves HTTPS and browsers also accept Secure on http://localhost.
|
||||
# `AIDND_COOKIE_SECURE` overrides the default with 0 or 1. Use 0 when testing
|
||||
# multi-user mode over plain HTTP on a LAN address.
|
||||
_cookie_secure_env = os.environ.get("AIDND_COOKIE_SECURE", "").strip().lower()
|
||||
COOKIE_SECURE = (
|
||||
_cookie_secure_env in ("1", "true", "yes", "on")
|
||||
if _cookie_secure_env
|
||||
else MULTI_USER
|
||||
)
|
||||
COOKIE_MAX_AGE = 60 * 60 * 24 * 365
|
||||
|
||||
# ---------- Shared demo key (BYOK fallback) ----------
|
||||
|
||||
DEMO_API_KEY = os.environ.get("AIDND_DEMO_API_KEY", "").strip()
|
||||
DEMO_ENDPOINT_URL = (
|
||||
os.environ.get("AIDND_DEMO_ENDPOINT_URL", "").strip()
|
||||
or "https://openrouter.ai/api/v1"
|
||||
)
|
||||
DEMO_MODELS = [
|
||||
m.strip()
|
||||
for m in os.environ.get("AIDND_DEMO_MODELS", "").split(",")
|
||||
if m.strip()
|
||||
] or ["google/gemma-4-26b-a4b-it:free"]
|
||||
DEMO_TURNS_PER_DAY = int(os.environ.get("AIDND_DEMO_TURNS_PER_DAY", "20") or 20)
|
||||
|
||||
# Trusted testers, listed by email, who bypass the daily demo cap and take
|
||||
# unmetered turns on the shared demo key. The list is comma-separated, and the
|
||||
# match ignores case.
|
||||
POWER_USERS = {
|
||||
e.strip().lower()
|
||||
for e in os.environ.get("AIDND_POWER_USERS", "").split(",")
|
||||
if e.strip()
|
||||
}
|
||||
|
||||
# Who can see the visit analytics. This is a separate list from `POWER_USERS` on
|
||||
# purpose. A trusted tester gets unmetered turns and the AI Chat page, which is
|
||||
# not a reason to give them the site's traffic numbers. An empty list, which is
|
||||
# the default, means nobody sees the dashboard in a hosted deployment.
|
||||
ANALYTICS_EMAILS = {
|
||||
e.strip().lower()
|
||||
for e in os.environ.get("AIDND_ANALYTICS_EMAILS", "").split(",")
|
||||
if e.strip()
|
||||
}
|
||||
|
||||
DEMO_CAP_MESSAGE = (
|
||||
f"You've used all {DEMO_TURNS_PER_DAY} free demo turns for today. "
|
||||
"Add your own API key in Settings to keep playing (it resets tomorrow)."
|
||||
)
|
||||
|
||||
|
||||
def demo_enabled() -> bool:
|
||||
# The demo key is a hosted-deployment feature. A local install talks to
|
||||
# whatever endpoint Settings points at, even with no API key, such as
|
||||
# Ollama.
|
||||
return MULTI_USER and bool(DEMO_API_KEY)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProviderConfig:
|
||||
"""What the turn engine connects with, after the decision between a
|
||||
user-supplied key and the demo key.
|
||||
|
||||
Build one of these with `resolve_provider_config()`.
|
||||
"""
|
||||
|
||||
endpoint_url: str
|
||||
api_key: str
|
||||
model: str
|
||||
using_demo: bool
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# A second guard around server-funded turns. `resolve_provider_config()`
|
||||
# already pins the model, and this makes the pin a property of the config
|
||||
# object too, so a later caller cannot construct an unpinned one. This
|
||||
# raise is unreachable by design. Reaching it means a new code path
|
||||
# bypassed the pinning, which is worth failing on rather than billing
|
||||
# for.
|
||||
#
|
||||
# The test is `using_demo`, not `api_key == DEMO_API_KEY`. Keying on the
|
||||
# key value looks stricter and is wrong. The demo key is an ordinary
|
||||
# OpenRouter key, so a user can legitimately paste that same key into
|
||||
# their own Settings. Every resolution then raised, which returned a 500
|
||||
# even from `GET /auth/me` and took the whole SPA down. `using_demo` is
|
||||
# what means the server is paying, and only the demo branch below sets
|
||||
# it.
|
||||
if self.using_demo and self.model not in DEMO_MODELS:
|
||||
raise ValueError(
|
||||
f"Refusing to use the shared demo key with non-whitelisted model {self.model!r}"
|
||||
)
|
||||
|
||||
|
||||
def resolve_provider_config(
|
||||
settings: models.Settings, *, model_override: str | None = None
|
||||
) -> ProviderConfig:
|
||||
"""Returns the user's own key when they have one, and the shared demo key
|
||||
otherwise.
|
||||
|
||||
The demo branch is the security-relevant one, and it is the only place the
|
||||
allowlist rule lives. Every caller has to come through this function rather
|
||||
than build a `ProviderConfig` itself. On the demo key:
|
||||
|
||||
* The model is pinned to `DEMO_MODELS`, so a caller-supplied override from
|
||||
the AI Chat page, or a hand-edited Settings row, cannot point a
|
||||
server-funded key at a paid model. An unrecognized model falls back to
|
||||
`DEMO_MODELS[0]`.
|
||||
* The endpoint is pinned to `DEMO_ENDPOINT_URL`, so the key cannot be
|
||||
redirected to a URL the user controls and captured there.
|
||||
|
||||
`model_override` is a per-request preference and never a grant. It is used
|
||||
verbatim with the user's own key, and on the demo key only when the model is
|
||||
on the allowlist.
|
||||
"""
|
||||
key = settings.api_key_plain
|
||||
requested = (model_override or "").strip() or settings.model
|
||||
if key or not demo_enabled():
|
||||
return ProviderConfig(settings.endpoint_url, key, requested, False)
|
||||
model = requested if requested in DEMO_MODELS else DEMO_MODELS[0]
|
||||
return ProviderConfig(DEMO_ENDPOINT_URL, DEMO_API_KEY, model, True)
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return models.utcnow().date().isoformat()
|
||||
|
||||
|
||||
def is_power_user(user: models.User) -> bool:
|
||||
"""Returns whether this user is a trusted tester.
|
||||
|
||||
A trusted tester gets unmetered demo turns, plus tooling that is not part of
|
||||
the game, such as the AI Chat scratchpad. A local install is always trusted,
|
||||
because it runs on the operator's own machine with their own API key. The
|
||||
provider debug log is local-only for the same reason.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
return True
|
||||
return bool(user.email) and user.email.lower() in POWER_USERS
|
||||
|
||||
|
||||
def is_owner(user: models.User) -> bool:
|
||||
"""Returns whether this user may see the visit analytics.
|
||||
|
||||
A local install always may, because it runs on the operator's own machine and
|
||||
shows their own visits. The provider debug log follows the same reasoning. A
|
||||
hosted deployment checks `AIDND_ANALYTICS_EMAILS`.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
return True
|
||||
return bool(user.email) and user.email.lower() in ANALYTICS_EMAILS
|
||||
|
||||
|
||||
def demo_turns_left(user: models.User) -> int:
|
||||
# A power user is never capped, so report the full cap and let the banner
|
||||
# read "N of N" rather than count down.
|
||||
if is_power_user(user):
|
||||
return DEMO_TURNS_PER_DAY
|
||||
used = user.demo_turns_used if user.demo_turns_date == _today() else 0
|
||||
return max(0, DEMO_TURNS_PER_DAY - used)
|
||||
|
||||
|
||||
def count_demo_turn(user: models.User) -> None:
|
||||
"""Records one demo turn. The caller's commit stores it."""
|
||||
if is_power_user(user):
|
||||
return # A power user's turns do not count against the cap.
|
||||
today = _today()
|
||||
if user.demo_turns_date != today:
|
||||
user.demo_turns_date = today
|
||||
user.demo_turns_used = 0
|
||||
user.demo_turns_used += 1
|
||||
|
||||
|
||||
# ---------- User resolution ----------
|
||||
|
||||
def local_user(db: Session) -> models.User:
|
||||
"""Returns the single implicit user used in local mode.
|
||||
"""Returns the single implicit user, creating it on first use.
|
||||
|
||||
A migration gives this user ownership of data written before Phase 8. On a
|
||||
fresh database the user is created on first use.
|
||||
A migration gives this user ownership of data written before per-user rows
|
||||
existed, so an older database resolves to the row that already owns its
|
||||
campaigns rather than to a fresh empty one.
|
||||
"""
|
||||
user = (
|
||||
db.query(models.User)
|
||||
@@ -237,27 +65,14 @@ def _touch(user: models.User, db: Session) -> None:
|
||||
db.commit()
|
||||
|
||||
|
||||
def resolve_session_user(request: Request, db: Session) -> models.User | None:
|
||||
token = request.cookies.get(SESSION_COOKIE)
|
||||
if not token:
|
||||
return None
|
||||
user_id = security.verify_session(token)
|
||||
if user_id is None:
|
||||
return None
|
||||
return db.get(models.User, user_id)
|
||||
def get_current_user(
|
||||
request: Request, db: Session = Depends(get_db)
|
||||
) -> models.User:
|
||||
"""The dependency every router uses. It always succeeds.
|
||||
|
||||
|
||||
def get_current_user(request: Request, db: Session = Depends(get_db)) -> models.User:
|
||||
"""The dependency every router uses to resolve the current user.
|
||||
|
||||
In multi-user mode a 401 means the frontend has to establish a session again
|
||||
through `GET /api/auth/me`.
|
||||
`request` is unused and kept so the signature stays a FastAPI dependency
|
||||
the routers can depend on unchanged.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
user = local_user(db)
|
||||
else:
|
||||
user = resolve_session_user(request, db)
|
||||
if user is None:
|
||||
raise HTTPException(401, "No session. Call GET /api/auth/me first.")
|
||||
user = local_user(db)
|
||||
_touch(user, db)
|
||||
return user
|
||||
|
||||
@@ -32,8 +32,19 @@ the import recomputes them:
|
||||
* `lineage` is a cache of `parent` plus `fork_depth`. Shipping it as well would
|
||||
put a second source of truth for one fact into a file anyone can hand-edit,
|
||||
and the two could then disagree without any read reporting it.
|
||||
* The head depth is the tip of the head branch, which is a fact about the nodes
|
||||
that arrived with it.
|
||||
* The head depth used to be derived the same way — the tip of the head branch,
|
||||
a fact about the nodes that arrived with it. M3 moved it to the other side of
|
||||
the rule. Undo no longer deletes, so a story can be read at a position behind
|
||||
its retained tip, and where the reader stopped is a decision nobody can
|
||||
recompute from the rows: the same tree exports identically whether the user
|
||||
undid three turns or none. `headDepth` is therefore written, and an import
|
||||
that ignored it would silently Redo the story to its newest retained turn,
|
||||
which is the Phase 0B export finding this milestone exists to close.
|
||||
|
||||
A bundle written before M3 has no `headDepth` key, and there is nothing lost in
|
||||
that: at the time it was written the head could not sit behind the tip, so the
|
||||
derived answer *was* the recorded one. Such a file imports by deriving, exactly
|
||||
as it always did.
|
||||
|
||||
Every hand-editable coordinate is therefore checked before a row is written, in
|
||||
`plan`, rather than repaired afterwards. An import that fails partway leaves an
|
||||
@@ -109,7 +120,6 @@ def export(db: Session, adventure: models.Adventure) -> dict:
|
||||
"pronouns": adventure.persona_pronouns,
|
||||
"desc": adventure.persona_desc,
|
||||
},
|
||||
"scriptState": adventure.script_state,
|
||||
"worldState": adventure.world_state,
|
||||
"autoSummarize": adventure.auto_summarize,
|
||||
"memoryBankEnabled": adventure.memory_bank_enabled,
|
||||
@@ -118,23 +128,27 @@ def export(db: Session, adventure: models.Adventure) -> dict:
|
||||
# belongs to is the root. `_local` places its nodes there.
|
||||
"branches": [_exported_branch(b, local) for b in branches] or [_ROOT],
|
||||
"headBranch": local.get(adventure.head_branch_id, 0),
|
||||
# M3. Where the story is being read, which is not always where it
|
||||
# ends. See the head-depth rule at the top of this module.
|
||||
"headDepth": adventure.head_depth,
|
||||
"memoryCursor": _exported_anchor(adventure, cursors.MEMORY, local),
|
||||
"summaryCursor": _exported_anchor(adventure, cursors.SUMMARY, local),
|
||||
"memories": [_exported_memory(m, local) for m in adventure.memories],
|
||||
# M4. A Save Point is a decision — someone chose this position and gave
|
||||
# it a name — so it goes in the file by the rule at the top of this
|
||||
# module. Nothing here is derived: the coordinate is the one stored, not
|
||||
# one recomputed from the rows, because the whole point of the pointer
|
||||
# is that no amount of reading the turns can tell you which one somebody
|
||||
# named. A bundle written before M4 has no key here and imports with no
|
||||
# Save Points, which is what such a campaign had.
|
||||
"checkpoints": [
|
||||
_exported_checkpoint(c, local) for c in _checkpoints_of(db, adventure)
|
||||
],
|
||||
"storyCards": [
|
||||
{"type": c.type, "name": c.name, "keys": c.keys,
|
||||
"entry": c.entry, "notes": c.notes}
|
||||
for c in adventure.story_cards
|
||||
],
|
||||
"scripts": [
|
||||
{
|
||||
"position": s.position, "enabled": s.enabled,
|
||||
"name": s.name, "description": s.description,
|
||||
"library": s.library_js, "input": s.input_js,
|
||||
"context": s.context_js, "output": s.output_js,
|
||||
}
|
||||
for s in adventure.scripts
|
||||
],
|
||||
"actions": [_exported_node(a, local) for a in nodes],
|
||||
}
|
||||
|
||||
@@ -161,6 +175,15 @@ def _exported_branch(branch: models.Branch, local: dict[int, int]) -> dict:
|
||||
# unnamed tree byte-identical to the one SP6 wrote.
|
||||
if branch.name:
|
||||
out["name"] = branch.name
|
||||
# M3. A branch the story left behind is a decision too — the depth a
|
||||
# divergent write departed at — so it goes in the file by the same rule that
|
||||
# puts the fork points in it. A restored backup that could not tell abandoned
|
||||
# history from active history would have lost the only thing distinguishing
|
||||
# them, since every row of both arrives either way. Omitted when the branch
|
||||
# is active, which keeps a file for a never-undone tree as it was.
|
||||
if branch.superseded_at is not None:
|
||||
out["supersededAt"] = branch.superseded_at.isoformat()
|
||||
out["supersededDepth"] = branch.superseded_depth
|
||||
return out
|
||||
|
||||
|
||||
@@ -202,6 +225,35 @@ def _exported_memory(memory: models.Memory, local: dict[int, int]) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _checkpoints_of(db: Session, adventure: models.Adventure) -> list[models.Checkpoint]:
|
||||
"""Returns the campaign's Save Points in creation order.
|
||||
|
||||
Read with a query rather than through a relationship, for the reason
|
||||
`models.Branch` declares none: a relationship on `Adventure` would be loaded
|
||||
by anything that touches an adventure, and the export is the only thing in
|
||||
the application that wants every Save Point at once.
|
||||
"""
|
||||
return (
|
||||
db.query(models.Checkpoint)
|
||||
.filter(models.Checkpoint.adventure_id == adventure.id)
|
||||
.order_by(models.Checkpoint.id)
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def _exported_checkpoint(checkpoint: models.Checkpoint, local: dict[int, int]) -> dict:
|
||||
return {
|
||||
"name": checkpoint.name,
|
||||
"note": checkpoint.note,
|
||||
# The branch as a position in this file's list, like every other branch
|
||||
# reference in the bundle. The depth is a coordinate along it and needs
|
||||
# no translation.
|
||||
"branch": _local(checkpoint.branch_id, local),
|
||||
"depth": checkpoint.depth,
|
||||
"createdAt": checkpoint.created_at.isoformat() if checkpoint.created_at else None,
|
||||
}
|
||||
|
||||
|
||||
def _imported_persona(persona) -> dict:
|
||||
"""Reads a bundle's `persona` block into `Adventure` keyword arguments.
|
||||
|
||||
@@ -262,11 +314,23 @@ def plan(bundle: dict, version: str) -> dict:
|
||||
_planned_nodes(bundle, len(branches)) if version == FORMAT
|
||||
else _planned_v1_nodes(bundle)
|
||||
)
|
||||
head = _as_index(bundle.get("headBranch"), len(branches), default=0)
|
||||
return {
|
||||
"branches": branches,
|
||||
"nodes": nodes,
|
||||
"memories": _planned_memories(bundle, len(branches)),
|
||||
"head": _as_index(bundle.get("headBranch"), len(branches), default=0),
|
||||
# M4. Empty for a version 1 bundle and for any version 2 bundle written
|
||||
# before Save Points existed, which is the same answer: no one had named
|
||||
# a position in those campaigns.
|
||||
"checkpoints": (
|
||||
_planned_checkpoints(bundle, len(branches), nodes)
|
||||
if version == FORMAT else []
|
||||
),
|
||||
"head": head,
|
||||
# None means the file does not say, which is every version 1 bundle and
|
||||
# every version 2 bundle written before M3. `_point_the_head` derives it
|
||||
# then, which is what those files were written expecting.
|
||||
"headDepth": _planned_head_depth(bundle, branches, nodes, head),
|
||||
# Version 2 records where the derived work reached. Version 1 counted
|
||||
# it, and a count cannot become a node until the nodes exist. See
|
||||
# `settle`.
|
||||
@@ -278,6 +342,58 @@ def plan(bundle: dict, version: str) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _derived_tip(branches: list[dict], nodes: list[dict], head: int) -> int:
|
||||
"""Returns where the head branch's story ends, which is where a file that
|
||||
does not state a head depth is opened.
|
||||
|
||||
This is the rule `tree.refresh_head` applies and the one every bundle
|
||||
written before M3 was exported under: the deepest node that arrived on the
|
||||
head branch, or, for a branch that carries none of its own, the fork point
|
||||
it inherits its story up to.
|
||||
"""
|
||||
depths = [n["depth"] for n in nodes if n["branch"] == head]
|
||||
if depths:
|
||||
return max(depths)
|
||||
fork_depth = branches[head].get("forkDepth")
|
||||
return fork_depth if fork_depth is not None else lineage.NO_DEPTH
|
||||
|
||||
|
||||
def _planned_head_depth(
|
||||
bundle: dict, branches: list[dict], nodes: list[dict], head: int
|
||||
) -> int | None:
|
||||
"""Returns the active head depth the file states, or None if it states none.
|
||||
|
||||
Checked here rather than repaired later, for the reason the module docstring
|
||||
gives: a coordinate that disagrees with the rows is not a value any read can
|
||||
be given, and an import that discovers it afterwards has already written
|
||||
half a tree.
|
||||
|
||||
Two bounds. A head past the retained tip is a file claiming the story is
|
||||
read somewhere it does not reach — the shape a truncated or hand-edited
|
||||
export takes, and the one that would silently move the reader forward.
|
||||
Below, `NO_DEPTH` is the floor, because an adventure with no story at all
|
||||
sits one step in front of its first node.
|
||||
|
||||
A head *behind* the tip is not an error. It is the whole point: it is a
|
||||
story the user undid and did not redo, and it must import undone.
|
||||
"""
|
||||
stated = bundle.get("headDepth")
|
||||
if stated is None:
|
||||
return None
|
||||
tip = _derived_tip(branches, nodes, head)
|
||||
if not _is_int(stated) or stated < lineage.NO_DEPTH:
|
||||
raise HTTPException(
|
||||
400, f"The file gives the active head depth as {stated!r}."
|
||||
)
|
||||
if stated > tip:
|
||||
raise HTTPException(
|
||||
400,
|
||||
f"The file reads its story at depth {stated}, but branch {head} "
|
||||
f"ends at {tip}.",
|
||||
)
|
||||
return stated
|
||||
|
||||
|
||||
def _planned_branches(bundle: dict) -> list[dict]:
|
||||
raw = bundle.get("branches")
|
||||
entries = [b for b in raw if isinstance(b, dict)] if isinstance(raw, list) else []
|
||||
@@ -287,8 +403,9 @@ def _planned_branches(bundle: dict) -> list[dict]:
|
||||
for i, entry in enumerate(entries):
|
||||
parent = entry.get("parent")
|
||||
name = _planned_branch_name(entry, i)
|
||||
left = _planned_supersession(entry)
|
||||
if parent is None:
|
||||
specs.append(dict(_ROOT, **({"name": name} if name else {})))
|
||||
specs.append(dict(_ROOT, **({"name": name} if name else {}), **left))
|
||||
continue
|
||||
# A branch may fork only from a branch listed before it. The export
|
||||
# writes them that way, because branches are numbered in creation order
|
||||
@@ -312,10 +429,27 @@ def _planned_branches(bundle: dict) -> list[dict]:
|
||||
specs.append({
|
||||
"parent": parent, "forkDepth": fork_depth,
|
||||
**({"name": name} if name else {}),
|
||||
**left,
|
||||
})
|
||||
return specs
|
||||
|
||||
|
||||
def _planned_supersession(entry: dict) -> dict:
|
||||
"""Returns the branch's disposition as the file gives it, or nothing.
|
||||
|
||||
Both keys or neither. A time with no depth cannot say what was displaced and
|
||||
a depth with no time is not a record of anything having happened, so a file
|
||||
carrying one of them is treated as carrying neither rather than half of a
|
||||
fact — no read depends on these columns, and inventing the missing half
|
||||
would be the only way to get a wrong answer out of them.
|
||||
"""
|
||||
at = _as_time(entry.get("supersededAt"))
|
||||
depth = entry.get("supersededDepth")
|
||||
if at is None or not _is_int(depth):
|
||||
return {}
|
||||
return {"supersededAt": at, "supersededDepth": depth}
|
||||
|
||||
|
||||
def _planned_branch_name(entry: dict, i: int) -> str | None:
|
||||
"""Returns the name a branch entry carries, or `None` if nobody named it.
|
||||
|
||||
@@ -437,6 +571,57 @@ def _planned_memories(bundle: dict, branches: int) -> list[dict]:
|
||||
return out
|
||||
|
||||
|
||||
def _planned_checkpoints(
|
||||
bundle: dict, branches: int, nodes: list[dict]
|
||||
) -> list[dict]:
|
||||
"""Returns the file's Save Points, checked against the tree it also carries.
|
||||
|
||||
A Save Point whose coordinate names no turn in the file is dropped rather
|
||||
than imported, and dropped rather than raising. The two halves of that are
|
||||
each deliberate:
|
||||
|
||||
* Dropped, because an imported pointer to a position the imported story does
|
||||
not contain is a Save Point that can only ever refuse to restore. It would
|
||||
be a row that exists to disappoint.
|
||||
* Not a 400, unlike the head depth. The head is a position the campaign is
|
||||
read at, so a file that misplaces it opens the story in the wrong place
|
||||
and every read is affected. A Save Point is a bookmark, and a bad one
|
||||
spoils nothing else in the file — refusing to import a whole campaign
|
||||
because one bookmark is wrong would lose the story to save the bookmark.
|
||||
|
||||
A name that is blank once trimmed is dropped for the same reason the create
|
||||
endpoint refuses one: an unnamed Save Point is not identifiable in a list.
|
||||
"""
|
||||
# Every coordinate the file writes, not only the ones it marks live.
|
||||
# `_write_nodes` makes exactly one attempt at each coordinate live whatever
|
||||
# the file says, so a coordinate that exists is a coordinate that will
|
||||
# resolve — and reading the flags here would drop a Save Point over a
|
||||
# question the writer has already settled.
|
||||
written = {(n["branch"], n["depth"]) for n in nodes}
|
||||
raw = bundle.get("checkpoints")
|
||||
out: list[dict] = []
|
||||
for entry in raw if isinstance(raw, list) else []:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
name = str(entry.get("name") or "").strip()[:schemas.CHECKPOINT_NAME_MAX]
|
||||
if not name:
|
||||
continue
|
||||
depth = entry.get("depth")
|
||||
if not _is_int(depth):
|
||||
continue
|
||||
branch = _as_index(entry.get("branch"), branches, default=None)
|
||||
if branch is None or (branch, depth) not in written:
|
||||
continue
|
||||
out.append({
|
||||
"name": name,
|
||||
"note": str(entry.get("note") or ""),
|
||||
"branch": branch,
|
||||
"depth": depth,
|
||||
"createdAt": _as_time(entry.get("createdAt")),
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def _planned_anchors(bundle: dict, branches: int) -> dict:
|
||||
anchors = {}
|
||||
for name in ("memory", "summary"):
|
||||
@@ -462,6 +647,7 @@ def write(db: Session, adventure: models.Adventure, story: dict) -> None:
|
||||
_write_nodes(db, adventure, story["nodes"], ids)
|
||||
_write_memories(db, adventure, story["memories"], ids)
|
||||
_point_the_head(adventure, story, ids)
|
||||
_write_checkpoints(db, adventure, story["checkpoints"], ids)
|
||||
_write_anchors(adventure, story, ids)
|
||||
|
||||
|
||||
@@ -489,6 +675,8 @@ def _write_branches(
|
||||
lineage=[],
|
||||
name=spec.get("name"),
|
||||
created_at=models.utcnow(),
|
||||
superseded_at=spec.get("supersededAt"),
|
||||
superseded_depth=spec.get("supersededDepth"),
|
||||
)
|
||||
).inserted_primary_key[0]
|
||||
entries = [[new_id, None]]
|
||||
@@ -575,25 +763,53 @@ def _write_memories(
|
||||
db.add(memory)
|
||||
|
||||
|
||||
def _write_checkpoints(
|
||||
db: Session, adventure: models.Adventure, specs: list[dict], ids: list[int]
|
||||
) -> None:
|
||||
"""Writes the Save Points, and moves nothing.
|
||||
|
||||
Note what this function does not touch. The head is pointed by
|
||||
`_point_the_head` from the file's own `headBranch`/`headDepth`, and importing
|
||||
a Save Point must not disturb it — a campaign exported at turn 30 with a
|
||||
Save Point at turn 12 opens at turn 30. The bundle records where the story
|
||||
was being read and, separately, which positions someone named; restoring one
|
||||
of them is a thing the user does afterwards, not a thing an import does for
|
||||
them.
|
||||
"""
|
||||
for spec in specs:
|
||||
checkpoint = models.Checkpoint(
|
||||
adventure_id=adventure.id,
|
||||
name=spec["name"],
|
||||
note=spec["note"],
|
||||
branch_id=ids[spec["branch"]],
|
||||
depth=spec["depth"],
|
||||
)
|
||||
if spec["createdAt"] is not None:
|
||||
checkpoint.created_at = spec["createdAt"]
|
||||
db.add(checkpoint)
|
||||
|
||||
|
||||
def _point_the_head(
|
||||
adventure: models.Adventure, story: dict, ids: list[int]
|
||||
) -> None:
|
||||
"""Sets which branch the story is played on, and how deep it goes.
|
||||
"""Sets which branch the story is played on, and where it is being read.
|
||||
|
||||
The branch comes from the file and the depth does not. The tip of a branch is
|
||||
whatever arrived on it, and a branch with no nodes of its own sits at its
|
||||
fork point, which is the last node its story contains. That node is borrowed
|
||||
but it is still the tip. This is the rule `tree.refresh_head` applies, run
|
||||
here before a flush.
|
||||
Both come from the file now (M3). A bundle that states its head depth is
|
||||
opened exactly where its owner left it, undone turns and all, and the turns
|
||||
past that point arrive as the retained future they were exported as.
|
||||
|
||||
A file that states none is opened at the tip of its head branch — whatever
|
||||
arrived on it, or, for a branch carrying no nodes of its own, the fork point
|
||||
that is the last node its story contains. That is the rule
|
||||
`tree.refresh_head` applies and the only answer a pre-M3 bundle can be given,
|
||||
because at the time it was written the head could not be anywhere else.
|
||||
"""
|
||||
head = story["head"]
|
||||
adventure.head_branch_id = ids[head]
|
||||
depths = [n["depth"] for n in story["nodes"] if n["branch"] == head]
|
||||
if depths:
|
||||
adventure.head_depth = max(depths)
|
||||
if story["headDepth"] is not None:
|
||||
adventure.head_depth = story["headDepth"]
|
||||
return
|
||||
fork_depth = story["branches"][head].get("forkDepth")
|
||||
adventure.head_depth = fork_depth if fork_depth is not None else lineage.NO_DEPTH
|
||||
adventure.head_depth = _derived_tip(story["branches"], story["nodes"], head)
|
||||
|
||||
|
||||
def _write_anchors(
|
||||
@@ -654,7 +870,6 @@ def materialize(
|
||||
authors_note=str(payload.get("authorsNote") or ""),
|
||||
ai_instructions=str(payload.get("aiInstructions") or ""),
|
||||
story_summary=str(payload.get("storySummary") or ""),
|
||||
script_state=payload.get("scriptState") or {},
|
||||
world_state=payload.get("worldState") or {},
|
||||
auto_summarize=bool(payload.get("autoSummarize", False)),
|
||||
memory_bank_enabled=bool(payload.get("memoryBankEnabled", False)),
|
||||
@@ -674,19 +889,10 @@ def materialize(
|
||||
notes=str(card.get("notes") or ""),
|
||||
))
|
||||
|
||||
for i, item in enumerate(payload.get("scripts") or []):
|
||||
if isinstance(item, dict):
|
||||
db.add(models.AdventureScript(
|
||||
adventure_id=adventure.id,
|
||||
position=int(item.get("position", i)),
|
||||
enabled=bool(item.get("enabled", True)),
|
||||
name=str(item.get("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=str(item.get("description") or ""),
|
||||
library_js=str(item.get("library") or ""),
|
||||
input_js=str(item.get("input") or ""),
|
||||
context_js=str(item.get("context") or ""),
|
||||
output_js=str(item.get("output") or ""),
|
||||
))
|
||||
# A bundle exported before M2 may carry "scripts" and "scriptState".
|
||||
# Campaign scripting is gone, so both are ignored rather than rejected: the
|
||||
# story, its tree, its cards and its memories still import intact, which is
|
||||
# what the bundle is for.
|
||||
|
||||
write(db, adventure, story)
|
||||
db.flush()
|
||||
|
||||
@@ -1,178 +0,0 @@
|
||||
"""Retention policy for throwaway guest accounts.
|
||||
|
||||
In multi-user mode every first visit creates a `users` row through
|
||||
`GET /api/auth/me`, so a public demo accumulates one account per visitor. Most
|
||||
of those visitors never return, and each one leaves behind whatever scenarios,
|
||||
adventures, actions, and memories they generated. This module deletes guests
|
||||
that have been inactive for `AIDND_GUEST_RETENTION_DAYS`, which defaults to 5,
|
||||
along with everything they made.
|
||||
|
||||
Why this is safe to run unattended:
|
||||
|
||||
- Only rows with `is_guest` AND `email IS NULL` are ever touched, and both
|
||||
clauses are checked rather than either alone. Registering upgrades the row
|
||||
in place (is_guest -> False), so a guest who signs up keeps everything;
|
||||
local mode's implicit single user is also is_guest=False.
|
||||
- Idle time is `COALESCE(last_seen_at, created_at)`. `auth._touch` writes
|
||||
`last_seen_at` at most once an hour, and a guest created by `/auth/me` has
|
||||
NULL there until its second request, so `created_at` is the correct floor for
|
||||
a new visitor. Without the coalesce, those rows look arbitrarily old.
|
||||
- Nothing a guest owns is reachable by anyone else. `is_public` is an
|
||||
output-only field, as `schemas.ScenarioBase` shows, so the only shared
|
||||
scenarios are the seeded ones, which have a NULL `user_id` and are outside
|
||||
this filter. Deleting a guest cannot remove content from another user.
|
||||
|
||||
The sweep uses one Core DELETE rather than an ORM cascade. `db.delete(user)`
|
||||
would SELECT every adventure, action, memory, and story card into Python only to
|
||||
delete them, which on Neon is the egress pattern that has already cost this
|
||||
project once. Every foreign key from `users` downward is ON DELETE CASCADE, from
|
||||
users to scenarios, adventures, scripts, and settings, and from those to actions,
|
||||
memories, and cards, so the database deletes the whole graph in one statement and
|
||||
returns a row count.
|
||||
|
||||
The scan gets no index. The sweep runs a few times a day against a table holding
|
||||
at most a few thousand rows, which does not justify a migration and the schema
|
||||
surface it adds.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete, func
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from . import analytics, auth, models
|
||||
from .database import SessionLocal
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _int_env(name: str, default: int) -> int:
|
||||
try:
|
||||
return int(os.environ.get(name, "").strip() or default)
|
||||
except ValueError:
|
||||
logger.warning("%s is not an integer; using %d.", name, default)
|
||||
return default
|
||||
|
||||
|
||||
# Days of inactivity before a guest account is deleted. A value of 0 or less
|
||||
# disables the policy, for a deployment that keeps everything.
|
||||
RETENTION_DAYS = _int_env("AIDND_GUEST_RETENTION_DAYS", 5)
|
||||
|
||||
# How often a long-lived process re-checks. Hours, not minutes: nothing here is
|
||||
# time-critical, and on Render's free tier the service sleeps and cold-starts
|
||||
# often enough that the startup sweep does most of the work by itself.
|
||||
SWEEP_INTERVAL_SECONDS = _int_env("AIDND_CLEANUP_INTERVAL_HOURS", 6) * 3600
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
"""Guests only exist in multi-user mode, so local runs skip the sweep
|
||||
rather than pointing a DELETE at a database that has nothing to collect."""
|
||||
return auth.MULTI_USER and RETENTION_DAYS > 0
|
||||
|
||||
|
||||
def anything_to_sweep() -> bool:
|
||||
"""Whether the periodic task is worth starting at all. The two jobs it runs
|
||||
are independent: a deployment can keep every guest forever and still want
|
||||
its analytics rows aged out, and vice versa."""
|
||||
return enabled() or analytics.RETENTION_DAYS > 0
|
||||
|
||||
|
||||
def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int:
|
||||
"""Delete guests idle for RETENTION_DAYS or more. Returns the row count.
|
||||
|
||||
The caller owns error handling; `sweep` is the safe wrapper.
|
||||
"""
|
||||
if RETENTION_DAYS <= 0:
|
||||
return 0
|
||||
# Stored timestamps are UTC without a timezone on both backends. SQLite
|
||||
# drops the timezone, and the Postgres columns are TIMESTAMP WITHOUT TIME
|
||||
# ZONE with the session pinned to UTC in `database.py`. Match that, so the
|
||||
# comparison does not depend on how a dialect renders a value that carries a
|
||||
# timezone.
|
||||
reference = now or models.utcnow()
|
||||
cutoff = reference.replace(tzinfo=None) - timedelta(days=RETENTION_DAYS)
|
||||
|
||||
stmt = (
|
||||
delete(models.User)
|
||||
.where(
|
||||
models.User.is_guest.is_(True),
|
||||
models.User.email.is_(None),
|
||||
func.coalesce(models.User.last_seen_at, models.User.created_at) < cutoff,
|
||||
)
|
||||
# Without this option, the "auto" strategy cannot evaluate coalesce in
|
||||
# Python and falls back to fetching every matching primary key first.
|
||||
# That is a second round trip for no benefit, because this session holds
|
||||
# no User objects to synchronize.
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
removed = db.execute(stmt).rowcount or 0
|
||||
db.commit()
|
||||
return removed
|
||||
|
||||
|
||||
def sweep() -> int:
|
||||
"""One pass, with its own session. Never raises: a failed cleanup must not
|
||||
be able to take the app down (same rule as seeding). Returns the guest
|
||||
count, which is the number worth logging about."""
|
||||
if not anything_to_sweep():
|
||||
return 0
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Ages out the per-visitor analytics rows, on its own terms: it is not
|
||||
# about guests, and it must still happen on a deployment that has
|
||||
# chosen to keep every account it ever minted.
|
||||
aged = analytics.purge_old_visitor_days(db)
|
||||
if aged:
|
||||
logger.info("Aged out %d analytics visitor-day row(s).", aged)
|
||||
removed = delete_stale_guests(db) if enabled() else 0
|
||||
if removed:
|
||||
logger.info(
|
||||
"Cleaned up %d guest account(s) idle for %d+ days.",
|
||||
removed,
|
||||
RETENTION_DAYS,
|
||||
)
|
||||
return removed
|
||||
except Exception:
|
||||
db.rollback()
|
||||
logger.exception("Guest cleanup failed; continuing without it.")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
async def _sweep_loop() -> None:
|
||||
while True:
|
||||
# Blocking DB work: keep it off the event loop, which is also serving
|
||||
# SSE turn streams.
|
||||
await run_in_threadpool(sweep)
|
||||
await asyncio.sleep(SWEEP_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
def start_sweeper() -> asyncio.Task | None:
|
||||
"""Kick off the periodic sweep; None when there is nothing to sweep."""
|
||||
if not enabled():
|
||||
logger.info("Guest cleanup disabled (multi_user=%s, retention_days=%d).",
|
||||
auth.MULTI_USER, RETENTION_DAYS)
|
||||
else:
|
||||
logger.info(
|
||||
"Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).",
|
||||
RETENTION_DAYS,
|
||||
SWEEP_INTERVAL_SECONDS // 3600,
|
||||
)
|
||||
if not anything_to_sweep():
|
||||
return None
|
||||
return asyncio.create_task(_sweep_loop())
|
||||
|
||||
|
||||
async def stop_sweeper(task: asyncio.Task | None) -> None:
|
||||
if task is None:
|
||||
return
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
@@ -78,7 +78,16 @@ class Path:
|
||||
"""One story, expressed as a SQL clause and as a Python predicate.
|
||||
|
||||
The object holds the lineage entries newest first, plus the depth of the
|
||||
tip. The tip is used only to estimate how much story each entry covers.
|
||||
head. Every entry is read as capped at the head, which is what makes the
|
||||
active head a position the whole application honours (M3).
|
||||
|
||||
Before M3 the head was always the deepest node, so the cap never bit and the
|
||||
tip was used only to estimate how much story each entry covers. Undo now
|
||||
moves the head backward without deleting anything, so a path can have live
|
||||
nodes past its head, and those nodes are not part of the story being told.
|
||||
Capping here is what hides them, and it hides them from every read at once:
|
||||
the transcript, the context builder, `attempts.preceding`, and memory
|
||||
retrieval all funnel through `path_of`.
|
||||
"""
|
||||
|
||||
def __init__(self, entries: list[tuple[int, int | None]], tip: int | None = None):
|
||||
@@ -91,6 +100,40 @@ class Path:
|
||||
def __len__(self) -> int:
|
||||
return len(self.entries)
|
||||
|
||||
# ------------------------------------------------------------- the head
|
||||
|
||||
def _cap(self, max_depth: int | None) -> int | None:
|
||||
"""Returns `max_depth` limited by the head, which no read may pass.
|
||||
|
||||
Three cases, and the third is the one M3 added:
|
||||
|
||||
* No head recorded (`tip is None`). The caller asked for the lineage
|
||||
without a position, so the entry's own cap stands. `tree` builds such
|
||||
a path when it resolves the node in front of a depth.
|
||||
* An uncapped entry, which means "this branch through to its tip". The
|
||||
head is the cap.
|
||||
* A capped entry, which is an ancestor capped at the fork depth. The
|
||||
head still wins when it sits behind that fork, because undoing below
|
||||
a fork point is undoing into the shared prefix. Taking the smaller of
|
||||
the two is what lets Undo walk back past a fork instead of stopping
|
||||
there — safe now that it deletes nothing.
|
||||
"""
|
||||
if self.tip is None:
|
||||
return max_depth
|
||||
if max_depth is None:
|
||||
return self.tip
|
||||
return min(max_depth, self.tip)
|
||||
|
||||
def uncapped(self) -> "Path":
|
||||
"""Returns the same lineage read through to its retained tip.
|
||||
|
||||
This is the retained history, head or no head: what Redo can still walk
|
||||
forward into, and what a write below the head has to fork away from.
|
||||
Only those two callers should use it. Every read of *the story* wants
|
||||
the capped path.
|
||||
"""
|
||||
return Path(self.entries, None)
|
||||
|
||||
# ---------------------------------------------------------------- SQL
|
||||
|
||||
def clause(
|
||||
@@ -125,7 +168,9 @@ class Path:
|
||||
entries = self.entries if count is None else self.entries[:count]
|
||||
if not entries:
|
||||
return false()
|
||||
on_path = or_(*[self._entry_clause(model, b, d) for b, d in entries])
|
||||
on_path = or_(
|
||||
*[self._entry_clause(model, b, self._cap(d)) for b, d in entries]
|
||||
)
|
||||
if model is models.Action:
|
||||
return and_(on_path, models.Action.live.is_(True))
|
||||
return on_path
|
||||
@@ -155,9 +200,10 @@ class Path:
|
||||
for branch_id, max_depth in self.entries:
|
||||
if node.branch_id != branch_id:
|
||||
continue
|
||||
if max_depth is None:
|
||||
cap = self._cap(max_depth)
|
||||
if cap is None:
|
||||
return True
|
||||
if node.depth is not None and node.depth <= max_depth:
|
||||
if node.depth is not None and node.depth <= cap:
|
||||
return True
|
||||
return False
|
||||
|
||||
@@ -187,7 +233,7 @@ class Path:
|
||||
return total
|
||||
covered = 0
|
||||
for i, (_, max_depth) in enumerate(self.entries):
|
||||
top = self.tip if max_depth is None else max_depth
|
||||
top = self._cap(max_depth)
|
||||
below = self.entries[i + 1][1] if i + 1 < total else NO_DEPTH
|
||||
if top is None or below is None:
|
||||
# Either no tip was recorded, or a hand-written row is missing a
|
||||
@@ -211,7 +257,8 @@ class Path:
|
||||
story has forked.
|
||||
"""
|
||||
for i, (_, max_depth) in enumerate(self.entries):
|
||||
if max_depth is not None and max_depth <= depth:
|
||||
cap = self._cap(max_depth)
|
||||
if cap is not None and cap <= depth:
|
||||
return i
|
||||
return len(self.entries)
|
||||
|
||||
@@ -241,7 +288,8 @@ class Path:
|
||||
return depth
|
||||
for entry_branch, max_depth in self.entries:
|
||||
if entry_branch == branch_id:
|
||||
return depth if max_depth is None else min(depth, max_depth)
|
||||
cap = self._cap(max_depth)
|
||||
return depth if cap is None else min(depth, cap)
|
||||
return NO_DEPTH
|
||||
|
||||
|
||||
|
||||
@@ -4,10 +4,17 @@ from pathlib import Path
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import DeclarativeBase, sessionmaker
|
||||
|
||||
# AIDND_DB_PATH lets deployments (Docker volume, hosted disk) relocate the
|
||||
# SQLite database; default stays backend/data.db for local runs. The parent
|
||||
# directory also hosts the auto-generated secret.key (see security.py), so
|
||||
# DB_PATH stays defined even when Postgres is in use.
|
||||
# The one database. It is SQLite, on this machine, in a file.
|
||||
#
|
||||
# Upstream could also point at a server database — `AIDND_DATABASE_URL` or the
|
||||
# platform-conventional `DATABASE_URL`, normalised onto psycopg3, with
|
||||
# pre-ping for a serverless Postgres that suspends when idle. That existed for
|
||||
# a hosted deployment. M2 removed it along with the deployment: a local
|
||||
# single-user storyteller has one reader, and a network database would be one
|
||||
# more thing that has to be running, and one more place the story lives.
|
||||
#
|
||||
# `AIDND_DB_PATH` stays. It is how the Docker image puts the database on a
|
||||
# volume, and how a test points at a throwaway file.
|
||||
_env_db_path = os.environ.get("AIDND_DB_PATH")
|
||||
DB_PATH = (
|
||||
Path(_env_db_path).resolve()
|
||||
@@ -15,48 +22,20 @@ DB_PATH = (
|
||||
else Path(__file__).resolve().parent.parent / "data.db"
|
||||
)
|
||||
|
||||
# `AIDND_DATABASE_URL`, or the conventional `DATABASE_URL`, switches the app to
|
||||
# a server database. Any SQLAlchemy URL works, and hosted deploys use Postgres,
|
||||
# which Phase 9 settled on Neon for. If neither variable is set, the app uses
|
||||
# SQLite.
|
||||
DATABASE_URL = (
|
||||
os.environ.get("AIDND_DATABASE_URL", "").strip()
|
||||
or os.environ.get("DATABASE_URL", "").strip()
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
engine = create_engine(
|
||||
f"sqlite:///{DB_PATH}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
|
||||
|
||||
def _normalize_url(url: str) -> str:
|
||||
"""Map the postgres:// / postgresql:// schemes hosts hand out to the
|
||||
psycopg3 driver installed in requirements.txt."""
|
||||
for prefix in ("postgres://", "postgresql://"):
|
||||
if url.startswith(prefix):
|
||||
return "postgresql+psycopg://" + url[len(prefix):]
|
||||
return url
|
||||
|
||||
|
||||
if DATABASE_URL:
|
||||
engine = create_engine(
|
||||
_normalize_url(DATABASE_URL),
|
||||
# Serverless Postgres (Neon) suspends idle databases; pre-ping
|
||||
# replaces silently-dead pooled connections instead of erroring.
|
||||
pool_pre_ping=True,
|
||||
# Store/read naive UTC like SQLite does, regardless of server default.
|
||||
connect_args={"options": "-c timezone=UTC"},
|
||||
)
|
||||
else:
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
engine = create_engine(
|
||||
f"sqlite:///{DB_PATH}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
|
||||
@event.listens_for(engine, "connect")
|
||||
def _enable_sqlite_foreign_keys(dbapi_connection, _record):
|
||||
# SQLite ships with foreign keys OFF per connection; without this every
|
||||
# ondelete=CASCADE/SET NULL in models.py is silently ignored.
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
@event.listens_for(engine, "connect")
|
||||
def _enable_sqlite_foreign_keys(dbapi_connection, _record):
|
||||
# SQLite ships with foreign keys OFF per connection; without this every
|
||||
# ondelete=CASCADE/SET NULL in models.py is silently ignored.
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
|
||||
|
||||
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Which inference endpoints this product is willing to talk to.
|
||||
|
||||
The Adventure Storyteller sends the player's prose, the assembled context, the
|
||||
retrieved memories, and the embedding inputs to whatever address the model
|
||||
endpoint names. That makes the endpoint the single most consequential setting
|
||||
in the application: point it somewhere else and the whole campaign goes there.
|
||||
|
||||
The v1 rule (`planning/DECISIONS/002-ollama-only-v1.md`, ADR 004) is that
|
||||
inference runs on user-controlled local infrastructure. Two deployments are
|
||||
supported and no third is:
|
||||
|
||||
* **same-host** — Ollama on loopback, the default;
|
||||
* **explicitly configured trusted LAN** — Ollama on another machine the user
|
||||
controls, named by them, reached over HTTP or over HTTPS with a certificate
|
||||
their machine trusts.
|
||||
|
||||
Everything on the public Internet is refused. Not discouraged in the UI, not
|
||||
absent from a dropdown — refused, here, on the way out, so that a hand-edited
|
||||
database row or a hostname that starts resolving somewhere new cannot quietly
|
||||
turn a local install into an exfiltration path.
|
||||
|
||||
## How the line is drawn
|
||||
|
||||
By **address**, not by name, and against an explicit allowlist of networks:
|
||||
loopback, the three RFC1918 ranges, link-local, IPv6 unique-local, and
|
||||
carrier-grade NAT — the last of which is what a mesh VPN such as Tailscale
|
||||
hands out and is as user-controlled as a LAN.
|
||||
|
||||
Every address the host resolves to must be in one of them. One address outside
|
||||
is enough to refuse the endpoint, so a name resolving to both a private and a
|
||||
public address does not squeak through.
|
||||
|
||||
The networks are spelled out rather than inferred from `ipaddress`'s own
|
||||
classifications, which do not mean what this rule needs: `is_private` is true
|
||||
of the documentation ranges and of `0.0.0.0/8`, and `is_reserved` is true of
|
||||
IPv6 loopback — so a rule written around it refuses `http://[::1]:11434/v1`,
|
||||
which is an ordinary same-host Ollama. Naming the networks keeps the policy
|
||||
readable and makes anything unnamed refused by default.
|
||||
|
||||
Checking addresses rather than hostnames is what makes the rule hard to talk
|
||||
around. A cloud provider cannot be reached by spelling its name differently,
|
||||
and `localhost.` or a DNS entry pointing at a public host is judged on where it
|
||||
actually goes.
|
||||
|
||||
## What this is not
|
||||
|
||||
It is not a general network-policy framework, and there is nothing to configure.
|
||||
There is one predicate, and it is applied in two places: when the endpoint is
|
||||
saved, so the user gets a clear error immediately, and again before every
|
||||
outbound request, because a name that resolved to `192.168.1.50` this morning
|
||||
can resolve to something else this afternoon.
|
||||
|
||||
TLS is a separate matter and is never traded against this one. See
|
||||
`tlstrust.py`: certificates are verified in full, and no endpoint — however
|
||||
private its address — may skip that.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
#: The networks an inference endpoint may live on. Anything else is refused.
|
||||
ALLOWED_NETWORKS = tuple(
|
||||
ipaddress.ip_network(cidr)
|
||||
for cidr in (
|
||||
"127.0.0.0/8", # this machine
|
||||
"10.0.0.0/8", # RFC1918
|
||||
"172.16.0.0/12", # RFC1918
|
||||
"192.168.0.0/16", # RFC1918
|
||||
"169.254.0.0/16", # link-local
|
||||
"100.64.0.0/10", # carrier-grade NAT, which mesh VPNs use
|
||||
"::1/128", # this machine, v6
|
||||
"fc00::/7", # unique-local, v6
|
||||
"fe80::/10", # link-local, v6
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _is_local(ip) -> bool:
|
||||
return any(ip in net for net in ALLOWED_NETWORKS)
|
||||
|
||||
#: Hosts that are only ever a cloud inference service. The address rule below
|
||||
#: already refuses every one of them, because they all resolve to public
|
||||
#: addresses; this list exists solely so the error says *why* rather than
|
||||
#: leaving the user to wonder whether their DNS is broken.
|
||||
CLOUD_HOSTS = (
|
||||
"openrouter.ai",
|
||||
"api.openai.com",
|
||||
"api.anthropic.com",
|
||||
"api.groq.com",
|
||||
"api.mistral.ai",
|
||||
"api.together.xyz",
|
||||
"api.deepseek.com",
|
||||
"generativelanguage.googleapis.com",
|
||||
"api.cohere.ai",
|
||||
"api.perplexity.ai",
|
||||
)
|
||||
|
||||
_CLOUD_REASON = (
|
||||
"this build talks to Ollama on your own machine or on your own network, "
|
||||
"and has no cloud provider support"
|
||||
)
|
||||
|
||||
|
||||
def _cloud_host(host: str) -> bool:
|
||||
host = host.lower().rstrip(".")
|
||||
return any(host == h or host.endswith("." + h) for h in CLOUD_HOSTS)
|
||||
|
||||
|
||||
def rejection_reason(url: str) -> str | None:
|
||||
"""Why this endpoint may not be used, or None if it may.
|
||||
|
||||
The string is shown to the user, so it says what to do rather than what
|
||||
went wrong internally.
|
||||
"""
|
||||
parsed = urlparse((url or "").strip())
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return "the endpoint URL must start with http:// or https://"
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return "the endpoint URL has no host"
|
||||
if _cloud_host(host):
|
||||
return f"{host} is a cloud inference service — {_CLOUD_REASON}"
|
||||
|
||||
port = parsed.port or (443 if parsed.scheme == "https" else 80)
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return (
|
||||
f"the host {host!r} could not be resolved — check the address, and "
|
||||
"that the machine running Ollama is reachable from here"
|
||||
)
|
||||
|
||||
for info in infos:
|
||||
try:
|
||||
ip = ipaddress.ip_address(info[4][0])
|
||||
except ValueError:
|
||||
return "the endpoint host resolved to an address that could not be read"
|
||||
if _is_local(ip):
|
||||
continue
|
||||
if ip.is_global:
|
||||
return (
|
||||
f"{host} resolves to {ip}, which is a public Internet address — "
|
||||
f"{_CLOUD_REASON}. Use Ollama on this machine "
|
||||
"(http://127.0.0.1:11434/v1) or on a machine on your own network"
|
||||
)
|
||||
return (
|
||||
f"{host} resolves to {ip}, which is not on this machine and not on "
|
||||
"your own network. Use http://127.0.0.1:11434/v1, or the address of "
|
||||
"a machine on your network"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def check(url: str) -> None:
|
||||
"""Raises `EndpointRejected` if this endpoint is outside the policy."""
|
||||
reason = rejection_reason(url)
|
||||
if reason is not None:
|
||||
raise EndpointRejected(reason)
|
||||
|
||||
|
||||
class EndpointRejected(Exception):
|
||||
"""The configured endpoint is not one this product will send a story to."""
|
||||
|
||||
|
||||
def is_loopback(url: str) -> bool:
|
||||
"""Whether this endpoint is on this machine. Used for reporting, not for
|
||||
gating: a trusted-LAN endpoint is equally allowed."""
|
||||
host = urlparse((url or "").strip()).hostname
|
||||
if not host:
|
||||
return False
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, None, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return False
|
||||
try:
|
||||
return all(ipaddress.ip_address(i[4][0]).is_loopback for i in infos)
|
||||
except ValueError:
|
||||
return False
|
||||
@@ -0,0 +1,405 @@
|
||||
"""M3: where the story is being read, and what moving that point costs.
|
||||
|
||||
Undo used to delete. It removed the trailing nodes, let `tree.refresh_head`
|
||||
recompute the tip from what survived, and the story was wherever the rows ended.
|
||||
That made the head a derived value and made Redo impossible, because the turns it
|
||||
would have moved forward into were gone.
|
||||
|
||||
The head is now a stored position that can sit behind the retained tip. Nothing
|
||||
is deleted, so three things that used to be the same question are now three
|
||||
different ones:
|
||||
|
||||
* **the active head** — `adventure.head_branch_id` and `adventure.head_depth`,
|
||||
the end of the story being told. Every read of the story stops here, because
|
||||
`lineage.Path` caps every entry at it.
|
||||
* **the retained tip** — the deepest live node still on the lineage. Redo walks
|
||||
toward it. It is read through `Path.uncapped()`, and only this module and the
|
||||
divergence check may look at it.
|
||||
* **the opening** — the shallowest node on the story, which is the floor Undo
|
||||
may not pass.
|
||||
|
||||
Everything that moves the head or asks a question about it lives here, so the
|
||||
turn engine, Retry, Add-take, Undo, Redo and Edit share one set of rules rather
|
||||
than four similar ones. The Phase 0B spike put the fork check in the write path
|
||||
and left Retry and Add-take on the old one, which is exactly the divergence this
|
||||
module exists to prevent.
|
||||
|
||||
The state that belongs to a position is not recomputed. Every node carries the
|
||||
world state it left behind (`attempts.snapshot_outcome`), so moving the head is a
|
||||
row lookup plus `attempts.restore_state`, at any distance, in either direction.
|
||||
"""
|
||||
|
||||
from sqlalchemy.orm import Session, undefer
|
||||
|
||||
from . import attempts, models, tree
|
||||
from .context import lineage
|
||||
|
||||
# The kinds of node a player writes. An undo or a redo steps over a whole turn,
|
||||
# which is one of these followed by the reply to it, so both ends need to agree
|
||||
# on what "a player's half of a turn" is.
|
||||
PLAYER_TYPES = ("do", "say", "story", "continue")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ reading
|
||||
|
||||
def opening_depth(db: Session, adventure: models.Adventure) -> int | None:
|
||||
"""Returns the depth of the first node of the story, or None if there is none.
|
||||
|
||||
This is Undo's floor. The Phase 0B spike moved the head to -1 and rendered an
|
||||
empty transcript, because its guard tested for a node of type `start` and an
|
||||
adventure opened with a player-written `story` action has none. Asking the
|
||||
path for its shallowest node needs no such special case: whatever the opening
|
||||
is called, it is the node with the smallest depth, and the story keeps it.
|
||||
|
||||
The read is uncapped. The opening does not move when the head does, and
|
||||
capping would make the floor depend on where the head already is.
|
||||
"""
|
||||
return (
|
||||
db.query(models.Action.depth)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).uncapped().clause(models.Action),
|
||||
)
|
||||
.order_by(models.Action.depth.asc(), models.Action.id.asc())
|
||||
.limit(1)
|
||||
.scalar()
|
||||
)
|
||||
|
||||
|
||||
def retained_tip(db: Session, adventure: models.Adventure) -> int | None:
|
||||
"""Returns the depth of the deepest live node still retained on this lineage.
|
||||
|
||||
This is what the head would be if the story had never been undone, and it is
|
||||
what Redo can reach. It is not the head, and no read of the story may use it.
|
||||
"""
|
||||
return (
|
||||
db.query(models.Action.depth)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).uncapped().clause(models.Action),
|
||||
)
|
||||
.order_by(models.Action.depth.desc(), models.Action.id.desc())
|
||||
.limit(1)
|
||||
.scalar()
|
||||
)
|
||||
|
||||
|
||||
def behind_tip(db: Session, adventure: models.Adventure) -> bool:
|
||||
"""Returns whether retained story sits past the head.
|
||||
|
||||
This one predicate answers every "does this write need to fork?" question in
|
||||
the application. It is true exactly when the user has undone and not redone,
|
||||
which is the only situation in which writing can displace an accepted future.
|
||||
|
||||
It is also what makes "is this turn the tip?" answerable again. Retry,
|
||||
Add-take and `stand_on` each decide between amending a turn in place and
|
||||
giving it a branch, and each used to ask `last_action`, which reads the
|
||||
*capped* path and therefore reports the node at the head as the newest one.
|
||||
Under a moved-back head that answer is wrong in the dangerous direction: it
|
||||
says a turn with an accepted future is a leaf, and amending it in place would
|
||||
leave that future descending from a take that is no longer live.
|
||||
"""
|
||||
tip = retained_tip(db, adventure)
|
||||
return tip is not None and tip > adventure.head_depth
|
||||
|
||||
|
||||
def node_at(
|
||||
db: Session, adventure: models.Adventure, depth: int
|
||||
) -> models.Action | None:
|
||||
"""Returns the live node at `depth` on the retained lineage, outcome loaded.
|
||||
|
||||
The read is uncapped on purpose: Redo asks for a node it is about to move the
|
||||
head onto, which is by definition past the head at the time of asking. The
|
||||
outcome columns are undeferred because the only reason to fetch this row is
|
||||
to restore the state it left behind.
|
||||
"""
|
||||
return (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).uncapped().clause(models.Action),
|
||||
models.Action.depth == depth,
|
||||
)
|
||||
.options(
|
||||
undefer(models.Action.state_after),
|
||||
undefer(models.Action.world_state_after),
|
||||
)
|
||||
.order_by(models.Action.id)
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def redo_target(db: Session, adventure: models.Adventure) -> int | None:
|
||||
"""Returns the depth the head moves to on Redo, or None if there is nowhere.
|
||||
|
||||
Redo steps over a whole turn, the same unit Undo steps back over, so a
|
||||
player's action and the reply to it move together. Landing between them would
|
||||
show the story an input with no answer and would leave the next Undo undoing
|
||||
half a turn.
|
||||
|
||||
The walk is along the retained lineage, which is what makes Redo follow the
|
||||
continuation that was active rather than choosing among branches. After a
|
||||
divergence the new branch *is* the lineage, and the displaced future is no
|
||||
longer on it, so this returns None without having to know that a divergence
|
||||
happened. That is `STORY-BRANCH-SEMANTICS.md` §8 falling out of the lineage
|
||||
rather than being enforced by a flag.
|
||||
"""
|
||||
ahead = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).uncapped().clause(models.Action),
|
||||
models.Action.depth > adventure.head_depth,
|
||||
)
|
||||
.order_by(models.Action.depth.asc(), models.Action.id.asc())
|
||||
.limit(2)
|
||||
.all()
|
||||
)
|
||||
if not ahead:
|
||||
return None
|
||||
first = ahead[0]
|
||||
if (
|
||||
first.type in PLAYER_TYPES
|
||||
and len(ahead) > 1
|
||||
and ahead[1].type == "ai"
|
||||
and ahead[1].depth == (first.depth or 0) + 1
|
||||
):
|
||||
return ahead[1].depth
|
||||
return first.depth
|
||||
|
||||
|
||||
def can_redo(db: Session, adventure: models.Adventure) -> bool:
|
||||
"""Returns whether an ordinary Redo is available from where the head is."""
|
||||
return redo_target(db, adventure) is not None
|
||||
|
||||
|
||||
def undo_target(
|
||||
db: Session, adventure: models.Adventure
|
||||
) -> tuple[int, models.Action] | None:
|
||||
"""Returns where Undo moves the head, and the first node it steps back over.
|
||||
|
||||
None means there is nothing to undo, which is either an empty story or a head
|
||||
already resting on the opening. The caller turns that into a 400; this
|
||||
function does not raise, so that the same question can be asked without
|
||||
committing to undoing.
|
||||
|
||||
A turn is the player's node plus the reply to it, and both move together for
|
||||
the reason given in `redo_target`. The player half is only claimed when it is
|
||||
directly in front of the reply, so a bare `continue`, which writes no player
|
||||
node, steps back over the reply alone.
|
||||
"""
|
||||
newest = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).clause(models.Action),
|
||||
)
|
||||
.order_by(models.Action.depth.desc(), models.Action.id.desc())
|
||||
.limit(2)
|
||||
.all()
|
||||
)
|
||||
if not newest:
|
||||
return None
|
||||
last = newest[0]
|
||||
first_stepped = last
|
||||
before = newest[1] if len(newest) > 1 else None
|
||||
if (
|
||||
last.type == "ai"
|
||||
and before is not None
|
||||
and before.type in PLAYER_TYPES
|
||||
and before.depth == (last.depth or 0) - 1
|
||||
):
|
||||
first_stepped = before
|
||||
floor = opening_depth(db, adventure)
|
||||
if first_stepped.depth is None or floor is None:
|
||||
return None
|
||||
if first_stepped.depth <= floor:
|
||||
# Stepping back over this turn would hide the opening of the campaign,
|
||||
# which is the pre-campaign state `STORY-BRANCH-SEMANTICS.md` §4 stops
|
||||
# at. The floor is the opening node itself rather than depth -1, so an
|
||||
# adventure that opens on a player-written `story` action stops in the
|
||||
# same place as one that opens on a `start` node.
|
||||
return None
|
||||
return first_stepped.depth - 1, first_stepped
|
||||
|
||||
|
||||
def can_undo(db: Session, adventure: models.Adventure) -> bool:
|
||||
"""Returns whether an ordinary Undo is available from where the head is."""
|
||||
return undo_target(db, adventure) is not None
|
||||
|
||||
|
||||
def displaced_history_under(
|
||||
db: Session, adventure: models.Adventure, node: models.Action
|
||||
) -> bool:
|
||||
"""Returns whether story the reader cannot see descends from `node`.
|
||||
|
||||
This is the question an in-place edit has to ask. Editing rewrites one row
|
||||
and re-evaluates nothing, which is what makes it a correction rather than a
|
||||
new continuation. That is harmless while everything descending from the row
|
||||
is on screen: the reader can see what their correction has to stay
|
||||
consistent with. It stops being harmless the moment a continuation descends
|
||||
from the row and is *not* on screen, because the edit then silently changes
|
||||
the words an invisible stretch of story was written from. That is the one
|
||||
way M3's retained history can be made to contradict itself.
|
||||
|
||||
Refusing is deliberately the whole of the fix. Making such an edit fork, so
|
||||
the original text and its future stay whole, is
|
||||
`STORY-BRANCH-SEMANTICS.md` §14-15 — and §15 requires re-evaluating the
|
||||
state the edited prose implies, which is M5's extraction pass. Neither is
|
||||
started here.
|
||||
|
||||
The question is asked as one shape rather than two, because the two ways a
|
||||
descendant becomes invisible turn out to be the same fact. An undone future
|
||||
sits past the head on this very lineage; a displaced line sits past a fork
|
||||
on a branch the story left. In both cases there is a live node, deeper than
|
||||
this one, that descends from it and is not on the path being read — and the
|
||||
departed branch is usually an *ancestor* of the branch now being read, which
|
||||
is why "branches other than the active one" is the wrong set to look at.
|
||||
|
||||
Only the deepest live node on each descending branch is examined. Whether a
|
||||
node is on the read path is monotone in depth: a branch is on the path with
|
||||
a cap, and a node is visible when its depth is at or under that cap. So if
|
||||
the deepest one is visible, every shallower one is too, and if it is not,
|
||||
the answer is already yes.
|
||||
|
||||
A node that is not live has no descendants of its own — a take the story
|
||||
moved past keeps a continuation only by being forked, and that fork is a
|
||||
branch this loop asks about anyway — so editing one is always safe.
|
||||
"""
|
||||
if not node.live or node.depth is None:
|
||||
return False
|
||||
read = lineage.path_of(db, adventure)
|
||||
branches = (
|
||||
db.query(models.Branch)
|
||||
.filter(models.Branch.adventure_id == adventure.id)
|
||||
.all()
|
||||
)
|
||||
for branch in branches:
|
||||
# Uncapped: the question is what this branch's story descends from, not
|
||||
# how much of it the reader is currently being shown.
|
||||
if not lineage.Path(lineage.entries_of(branch)).contains(node):
|
||||
continue
|
||||
deepest = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
models.Action.branch_id == branch.id,
|
||||
models.Action.live.is_(True),
|
||||
models.Action.depth > node.depth,
|
||||
)
|
||||
.order_by(models.Action.depth.desc(), models.Action.id.desc())
|
||||
.first()
|
||||
)
|
||||
if deepest is not None and not read.contains(deepest):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ writing
|
||||
|
||||
def move_to(db: Session, adventure: models.Adventure, depth: int) -> None:
|
||||
"""Moves the active head to `depth` and restores the state recorded there.
|
||||
|
||||
This is the whole of Undo and Redo. Nothing is deleted, nothing is
|
||||
recomputed, and the direction of travel does not matter: the node at the
|
||||
destination carries the world state it left behind, so arriving from in front
|
||||
of it and arriving from behind it restore the same value.
|
||||
|
||||
A destination with no node — the head resting one step in front of the
|
||||
opening — leaves the live state alone, which is `attempts.restore_state`'s
|
||||
rule for a missing snapshot and the reason it is not this function's job to
|
||||
invent an empty one.
|
||||
"""
|
||||
adventure.head_depth = depth
|
||||
attempts.restore_state(adventure, node_at(db, adventure, depth))
|
||||
|
||||
|
||||
def move_to_node(db: Session, adventure: models.Adventure, node: models.Action) -> bool:
|
||||
"""Moves the head onto `node`, changing line only if it is not on this one.
|
||||
|
||||
M4 restores a Save Point through this, and it adds no restoring of its own:
|
||||
the depth half is `move_to` unchanged, so the state, the transcript, the
|
||||
assembled context and memory eligibility all arrive exactly as they do for
|
||||
Undo and Redo. Returns whether the line had to change as well as the depth,
|
||||
which is the one thing about a restore a caller cannot work out afterwards.
|
||||
|
||||
The head is two values, and the two halves move for different reasons. A
|
||||
Save Point almost always names a position on the story being read — its own
|
||||
line, or the shared prefix that line inherits — and then only the depth
|
||||
moves. Leaving the branch alone is what makes the restored position keep the
|
||||
continuation it has: after a divergence, restoring to the shared prefix must
|
||||
put the reader back on the *new* line, where Redo walks into the turns they
|
||||
are still writing, not into the future they left. Reaching for the Save
|
||||
Point's own branch there would quietly hand back the abandoned story.
|
||||
|
||||
The other case is real and has to work. A Save Point survives divergence
|
||||
(`STORY-BRANCH-SEMANTICS.md` §19), so one can name a position on a line the
|
||||
story has since left, and no amount of depth movement reaches a branch this
|
||||
path does not contain. The line then moves as well — one assignment, the
|
||||
same one `switch_branch` makes — and the depth still moves through
|
||||
`move_to`. Nothing is created: a restore never forks, whichever case it
|
||||
takes. The first write below the restored head does, through
|
||||
`fork_if_behind_head`, like every other write.
|
||||
"""
|
||||
switched = not lineage.path_of(db, adventure).uncapped().contains(node)
|
||||
if switched:
|
||||
adventure.head_branch_id = node.branch_id
|
||||
move_to(db, adventure, node.depth)
|
||||
return switched
|
||||
|
||||
|
||||
def fork_if_behind_head(db: Session, adventure: models.Adventure) -> bool:
|
||||
"""Gives the story a new branch when a write would displace a retained future.
|
||||
|
||||
Returns whether a branch was created, which is what a caller reports as a
|
||||
divergence.
|
||||
|
||||
Called before every write that continues the story, and it does nothing on
|
||||
the ordinary path where the head is already at the tip. That is the property
|
||||
worth keeping: a story that is never undone forks exactly as often as it did
|
||||
before M3, so the branch table does not fill up with one branch per turn.
|
||||
|
||||
Undo alone must not fork. Moving the head is not a decision to abandon
|
||||
anything — the user may be reading, or about to Redo. Only the first write
|
||||
below the head states which continuation they mean, which is
|
||||
`STORY-BRANCH-SEMANTICS.md` §8 and §20 and what makes Redo survive an Undo.
|
||||
|
||||
`tree.branch_at` leaves the departed branch exactly as it is: its nodes stay
|
||||
live, at their depths, on their branch. The new branch inherits the story up
|
||||
to the head and owns everything written from here, so the displaced future
|
||||
remains reachable through the branch it was written on.
|
||||
"""
|
||||
if not behind_tip(db, adventure):
|
||||
return False
|
||||
departed = lineage.branch_of(db, adventure)
|
||||
at_depth = adventure.head_depth
|
||||
tree.branch_at(db, adventure, at_depth)
|
||||
if departed is not None:
|
||||
mark_superseded(departed, at_depth)
|
||||
return True
|
||||
|
||||
|
||||
def mark_superseded(branch: models.Branch, depth: int) -> None:
|
||||
"""Records that this branch's story past `depth` was displaced.
|
||||
|
||||
`DATA-MODEL.md` §5 gives a branch a disposition of active, retained or
|
||||
disposable. This is that disposition, stored as the fact that produced it
|
||||
rather than as a word: the depth the story left at, and when. A branch with
|
||||
no `superseded_at` is active; one with a value has retained history past that
|
||||
depth which no active head is reading.
|
||||
|
||||
Nothing in the application reads these columns to make a decision, and that
|
||||
is deliberate. Redo is decided by the lineage, not by a flag, so a stale or
|
||||
hand-edited value here cannot make the story wrong. They exist so that the
|
||||
cleanup and discarded-history features `STORY-BRANCH-SEMANTICS.md` §28 and
|
||||
§29 leave to a later version have something to select on, and so that a
|
||||
divergence is observable in a test.
|
||||
|
||||
The shallowest departure wins. A branch left at depth 9 and later left again
|
||||
at depth 4 has retained history from 4 onward, and recording the later, deeper
|
||||
value would understate what was displaced.
|
||||
"""
|
||||
if branch.superseded_depth is None or depth < branch.superseded_depth:
|
||||
branch.superseded_depth = depth
|
||||
if branch.superseded_at is None:
|
||||
branch.superseded_at = models.utcnow()
|
||||
@@ -1,183 +1,35 @@
|
||||
"""Phase 9: abuse guards for hosted, multi-user deployments.
|
||||
"""Resource bounds on what a single request or a single story may cost.
|
||||
|
||||
Rate limits and row caps do nothing in local mode, because a single local player
|
||||
should never be throttled by their own app. The values are hardcoded on purpose.
|
||||
They are generous enough that a legitimate player never notices them, and tight
|
||||
enough that a hostile visitor cannot exhaust the demo key, saturate the CPU, or
|
||||
fill the database.
|
||||
Upstream carried three things here, and only one of them belongs in a local
|
||||
single-user product. Per-IP and per-user **rate limiting**, the login-attempt
|
||||
throttle, and the per-user **quotas** were hosted-service policy: they existed
|
||||
to stop a hostile visitor exhausting a shared demo key or filling a shared
|
||||
database. M2 removed all of it. There are no visitors, and throttling the one
|
||||
person who started the application would be a bug rather than a guard.
|
||||
|
||||
What is left is defensive programming, and it applies whatever the deployment:
|
||||
|
||||
* a ceiling on the **request body**, so a malformed or hostile payload cannot
|
||||
be read into memory before anything looks at it;
|
||||
* ceilings on how large **one adventure** may grow, in actions, memories,
|
||||
story cards and branches. These bound storage and the cost of the queries
|
||||
that walk them. They are per-story, not per-user: nothing here counts how
|
||||
many campaigns a person may have.
|
||||
|
||||
An import is checked against the same per-adventure ceilings that live creation
|
||||
uses, so a bundle cannot carry a story past a limit that play could not reach.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from collections import defaultdict, deque
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import auth, models
|
||||
from . import models
|
||||
|
||||
# ---------- Rate limiting ----------
|
||||
# Fixed windows per scope and caller. The windows live in memory, which is
|
||||
# enough for the single-process deployment this app targets. The worst case
|
||||
# after a restart is a brief extra allowance.
|
||||
# ---------- Per-story row caps ----------
|
||||
|
||||
# Maps a scope to (max requests, window seconds).
|
||||
RATE_LIMITS: dict[str, tuple[int, int]] = {
|
||||
"turn": (10, 60), # AI turn generation. The demo key also has a daily cap.
|
||||
"chat": (30, 60), # The AI Chat scratchpad, for power users.
|
||||
"script-test": (30, 60), # Sandboxed, but each run costs up to 2s of CPU.
|
||||
"connection-test": (10, 60), # Outbound HTTP to a user-supplied URL.
|
||||
"import": (30, 60), # Large writes.
|
||||
"auth": (10, 300), # Register and login attempts, per IP.
|
||||
"guest": (30, 300), # New guest users, per IP. Each one is a database row.
|
||||
# Pageview beacons. The limit is generous, because a real reader clicking
|
||||
# around a SPA sends a handful a minute, and it is low enough that nobody
|
||||
# can inflate the traffic numbers faster than by reloading the page.
|
||||
"analytics": (120, 60),
|
||||
}
|
||||
|
||||
_windows: dict[tuple[str, str], deque] = defaultdict(deque)
|
||||
_windows_guard = threading.Lock()
|
||||
|
||||
|
||||
# How many proxy hops sit between the app and the real client. On Render, and on
|
||||
# most platforms, that is one, because the platform's edge appends the connecting
|
||||
# IP to the right of `X-Forwarded-For`. A client can prepend any value on the
|
||||
# left, but it cannot push a value past the edge's own append, so the trustworthy
|
||||
# client IP is the entry that many places from the right rather than uvicorn's
|
||||
# leftmost choice. Trusting the leftmost entry let anyone rotate
|
||||
# `X-Forwarded-For` to get a fresh rate-limit bucket per request and bypass the
|
||||
# auth and guest limits. If the deployment adds more hops, set
|
||||
# `AIDND_TRUSTED_PROXY_HOPS`.
|
||||
TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1))
|
||||
|
||||
|
||||
def client_ip(request: Request) -> str:
|
||||
"""Returns the real client IP, resisting a spoofed `X-Forwarded-For`.
|
||||
|
||||
The function reads the hop the trusted edge appended, which is the rightmost
|
||||
entry minus any extra trusted hops. If no forwarded header is present, which
|
||||
happens locally, in development, and on a direct connection, it falls back to
|
||||
the socket peer.
|
||||
|
||||
The function is public because the access log needs the same answer. Two
|
||||
functions that each decide which address belongs to the caller is how one of
|
||||
them ends up trusting a header it should not.
|
||||
"""
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
if forwarded:
|
||||
parts = [p.strip() for p in forwarded.split(",") if p.strip()]
|
||||
if parts:
|
||||
return parts[-min(TRUSTED_PROXY_HOPS, len(parts))]
|
||||
return request.client.host if request.client else "unknown"
|
||||
|
||||
|
||||
def rate_limit(scope: str, request: Request, user: models.User | None = None) -> None:
|
||||
"""Raises a 429 when the caller exceeds the scope's window.
|
||||
|
||||
The window is keyed per user when a user is known, because an account
|
||||
survives an IP change, and per IP otherwise.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
limit, window_seconds = RATE_LIMITS[scope]
|
||||
key = (scope, f"u{user.id}" if user else f"ip{client_ip(request)}")
|
||||
now = time.time()
|
||||
with _windows_guard:
|
||||
window = _windows[key]
|
||||
while window and window[0] < now - window_seconds:
|
||||
window.popleft()
|
||||
if len(window) >= limit:
|
||||
raise HTTPException(
|
||||
429, "You're doing that too fast — wait a minute and try again."
|
||||
)
|
||||
window.append(now)
|
||||
if len(_windows) > 10_000:
|
||||
_prune(now)
|
||||
|
||||
|
||||
# ---------- Per-account login throttle ----------
|
||||
# This is defense in depth next to the per-IP `auth` limit. A botnet dilutes
|
||||
# that limit, because many real source IPs each get their own bucket, so it
|
||||
# cannot by itself stop a distributed guessing run against one account. This cap
|
||||
# keys on the target email rather than on the caller, so guessing one account's
|
||||
# password stays expensive however many addresses the guesses come from.
|
||||
#
|
||||
# Only failures count, and a correct password clears the record. The window
|
||||
# slides over a short period rather than locking the account, so a user who
|
||||
# mistypes a few times recovers within minutes. The trade-off is that an
|
||||
# attacker can keep a known account throttled, which is an inconvenience and is
|
||||
# preferable to letting the account be brute-forced.
|
||||
LOGIN_FAIL_LIMIT = 8 # Failed attempts per account.
|
||||
LOGIN_FAIL_WINDOW = 900 # The window in seconds, which is 15 minutes.
|
||||
|
||||
_login_fails: dict[str, deque] = defaultdict(deque)
|
||||
_login_guard = threading.Lock()
|
||||
|
||||
|
||||
def check_login_allowed(email: str) -> None:
|
||||
"""Raises a 429 when an account has too many recent failed logins.
|
||||
|
||||
Call this before verifying the password, so that a guess never reaches the
|
||||
hash.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
now = time.time()
|
||||
with _login_guard:
|
||||
window = _login_fails[email]
|
||||
while window and window[0] < now - LOGIN_FAIL_WINDOW:
|
||||
window.popleft()
|
||||
if len(window) >= LOGIN_FAIL_LIMIT:
|
||||
raise HTTPException(
|
||||
429,
|
||||
"Too many failed sign-in attempts for this account — "
|
||||
"wait a few minutes and try again.",
|
||||
)
|
||||
|
||||
|
||||
def note_login_failure(email: str) -> None:
|
||||
"""Records one failed attempt against `email`."""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
now = time.time()
|
||||
with _login_guard:
|
||||
_login_fails[email].append(now)
|
||||
if len(_login_fails) > 10_000: # Bound the map against a flood of unique emails.
|
||||
stale = [
|
||||
key for key, window in _login_fails.items()
|
||||
if not window or window[-1] < now - LOGIN_FAIL_WINDOW
|
||||
]
|
||||
for key in stale:
|
||||
del _login_fails[key]
|
||||
|
||||
|
||||
def note_login_success(email: str) -> None:
|
||||
"""Clears the account's failure record after a correct password."""
|
||||
with _login_guard:
|
||||
_login_fails.pop(email, None)
|
||||
|
||||
|
||||
def _prune(now: float) -> None:
|
||||
"""Drops callers whose whole window has expired, so the per-IP dict stays bounded.
|
||||
|
||||
Call this with the guard held.
|
||||
"""
|
||||
longest = max(seconds for _, seconds in RATE_LIMITS.values())
|
||||
stale = [key for key, window in _windows.items()
|
||||
if not window or window[-1] < now - longest]
|
||||
for key in stale:
|
||||
del _windows[key]
|
||||
|
||||
|
||||
# ---------- Per-user row caps ----------
|
||||
|
||||
MAX_ADVENTURES_PER_USER = 100
|
||||
MAX_SCENARIOS_PER_USER = 200
|
||||
MAX_SCRIPTS_PER_USER = 200
|
||||
MAX_STORY_CARDS_PER_OWNER = 200 # Per scenario or per adventure.
|
||||
MAX_MEMORIES_PER_ADVENTURE = 1000
|
||||
MAX_ACTIONS_PER_ADVENTURE = 5000
|
||||
@@ -200,28 +52,14 @@ def check_row_cap(
|
||||
) -> None:
|
||||
"""Raises a 409 when creating one more row of `kind` would exceed its cap.
|
||||
|
||||
The caller has already checked ownership of the scenario or adventure passed
|
||||
in.
|
||||
Only per-story kinds are capped. `adventures` and `scenarios` were per-user
|
||||
quotas and are no longer checked; the callers still pass them, and they are
|
||||
accepted and ignored so that adding a cap back is a change here rather than
|
||||
at every call site.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
if kind in ("adventures", "scenarios"):
|
||||
return
|
||||
if kind == "adventures":
|
||||
count = _count(db, models.Adventure, models.Adventure.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_ADVENTURES_PER_USER, "adventures",
|
||||
"delete one you no longer play to make room",
|
||||
)
|
||||
elif kind == "scenarios":
|
||||
count = _count(db, models.Scenario, models.Scenario.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_SCENARIOS_PER_USER, "scenarios", "delete one to make room"
|
||||
)
|
||||
elif kind == "scripts":
|
||||
count = _count(db, models.Script, models.Script.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_SCRIPTS_PER_USER, "scripts", "delete one to make room"
|
||||
)
|
||||
elif kind == "story_cards":
|
||||
if kind == "story_cards":
|
||||
owner_filter = (
|
||||
models.StoryCard.scenario_id == scenario_id
|
||||
if scenario_id is not None
|
||||
@@ -273,8 +111,6 @@ def check_bundle_lists(**lists) -> None:
|
||||
The keyword arguments are `story_cards`, `memories`, `actions`, and
|
||||
`branches`.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
for name, value in lists.items():
|
||||
cap = _BUNDLE_LIST_CAPS[name]
|
||||
if isinstance(value, list) and len(value) > cap:
|
||||
@@ -286,8 +122,8 @@ def check_bundle_lists(**lists) -> None:
|
||||
|
||||
# ---------- Request body size ----------
|
||||
# The limit is generous enough for the largest legitimate payload, which is an
|
||||
# adventure export holding thousands of actions. It applies in every mode, and no
|
||||
# honest request approaches it.
|
||||
# adventure export holding thousands of actions. No honest request approaches
|
||||
# it.
|
||||
|
||||
MAX_BODY_BYTES = 2 * 1024 * 1024
|
||||
MAX_IMPORT_BODY_BYTES = 20 * 1024 * 1024
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import mimetypes
|
||||
import os
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI
|
||||
@@ -8,15 +7,10 @@ from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
from . import analytics, cleanup
|
||||
from .auth import MULTI_USER
|
||||
from .database import engine
|
||||
from .limits import BodySizeLimitMiddleware
|
||||
from .migrations import bootstrap
|
||||
from .routers import (
|
||||
adventures, analytics as analytics_router, auth, chat, debug, scenarios, scripts,
|
||||
settings, story_cards,
|
||||
)
|
||||
from .routers import adventures, chat, debug, scenarios, settings, story_cards
|
||||
from .seed import seed_public_scenarios
|
||||
|
||||
bootstrap(engine)
|
||||
@@ -24,37 +18,32 @@ seed_public_scenarios(engine)
|
||||
|
||||
# Production serves the SPA same-origin, so CORS only matters for the Vite dev
|
||||
# server; AIDND_CORS_ORIGINS overrides for any other cross-origin setup.
|
||||
#
|
||||
# A wildcard is refused rather than honoured. This API is unauthenticated by
|
||||
# design and bound to loopback, so its only protection from a page the user
|
||||
# happens to have open in another tab is the same-origin policy. `*` would hand
|
||||
# every site on the Internet a write handle on the local campaign database. If the
|
||||
# value is wrong the app refuses to start, because a permissive CORS policy that
|
||||
# nobody notices is worse than one that fails loudly.
|
||||
CORS_ORIGINS = [
|
||||
o.strip()
|
||||
for o in os.environ.get("AIDND_CORS_ORIGINS", "").split(",")
|
||||
if o.strip()
|
||||
] or ["http://localhost:5173", "http://127.0.0.1:5173"]
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
# Sweeps once on boot, then on an interval. Booting is the reliable
|
||||
# trigger on Render's free tier, where the service sleeps after ~15
|
||||
# minutes and a long-running timer rarely gets to fire.
|
||||
sweeper = cleanup.start_sweeper()
|
||||
# Visit counters are buffered in memory and written in batches; this is
|
||||
# what turns them into rows, and stop_flusher writes out the last batch so
|
||||
# a deploy doesn't drop it.
|
||||
flusher = analytics.start_flusher()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await cleanup.stop_sweeper(sweeper)
|
||||
await analytics.stop_flusher(flusher)
|
||||
if any(o == "*" or o.strip() == "*" for o in CORS_ORIGINS):
|
||||
raise RuntimeError(
|
||||
"AIDND_CORS_ORIGINS must not contain '*'. The storyteller API is "
|
||||
"unauthenticated and loopback-bound; a wildcard origin would let any "
|
||||
"web page read and rewrite every campaign. List the exact origins "
|
||||
"instead."
|
||||
)
|
||||
|
||||
|
||||
# The interactive API docs stay local-only: in multi-user mode they just hand
|
||||
# strangers a map of the API surface.
|
||||
app = FastAPI(
|
||||
title="AI D&D",
|
||||
docs_url=None if MULTI_USER else "/docs",
|
||||
title="Adventure Storyteller",
|
||||
docs_url="/docs",
|
||||
redoc_url=None,
|
||||
openapi_url=None if MULTI_USER else "/openapi.json",
|
||||
lifespan=lifespan,
|
||||
openapi_url="/openapi.json",
|
||||
)
|
||||
|
||||
app.add_middleware(
|
||||
@@ -117,50 +106,14 @@ class SecurityHeadersMiddleware:
|
||||
await self.app(scope, receive, send_with_headers)
|
||||
|
||||
|
||||
class ApiErrorMiddleware:
|
||||
"""Counts failed API responses for the analytics dashboard.
|
||||
|
||||
This is middleware rather than an exception handler, because it observes
|
||||
what the client received. A 429 from a rate limiter, a 404 from routing, and
|
||||
a 500 from a handler that never returned all reach it the same way. It is
|
||||
pure ASGI for the same reason as the headers above: an SSE turn must not be
|
||||
buffered on its way out. It watches `/api` only, because a 404 on the SPA
|
||||
mount is a page load rather than a fault.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
self.app = app
|
||||
|
||||
async def __call__(self, scope, receive, send):
|
||||
if scope["type"] != "http" or not scope.get("path", "").startswith("/api"):
|
||||
return await self.app(scope, receive, send)
|
||||
|
||||
async def send_counting(message):
|
||||
if message["type"] == "http.response.start" and message["status"] >= 400:
|
||||
# The router has already put the matched route on the scope by
|
||||
# the time a response starts, so the label can name the
|
||||
# endpoint rather than the caller's path.
|
||||
analytics.record(
|
||||
analytics.M_ERROR,
|
||||
analytics.api_route_label(scope, message["status"]),
|
||||
)
|
||||
await send(message)
|
||||
|
||||
await self.app(scope, receive, send_counting)
|
||||
|
||||
|
||||
app.add_middleware(ApiErrorMiddleware)
|
||||
app.add_middleware(SecurityHeadersMiddleware)
|
||||
|
||||
app.include_router(auth.router)
|
||||
app.include_router(scenarios.router)
|
||||
app.include_router(adventures.router)
|
||||
app.include_router(story_cards.router)
|
||||
app.include_router(scripts.router)
|
||||
app.include_router(settings.router)
|
||||
app.include_router(chat.router)
|
||||
app.include_router(debug.router)
|
||||
app.include_router(analytics_router.router)
|
||||
|
||||
|
||||
@app.get("/api/health")
|
||||
@@ -171,7 +124,8 @@ def health():
|
||||
# In production, serve the built frontend (frontend/dist) as static files.
|
||||
class SPAStaticFiles(StaticFiles):
|
||||
"""Serve index.html for unknown paths so client-side routes (/play/3)
|
||||
survive a page reload. API routes are matched before this mount."""
|
||||
survive a page reload. API routes are matched before this mount, and an
|
||||
unmatched one 404s rather than falling through to the page."""
|
||||
|
||||
async def get_response(self, path, scope):
|
||||
try:
|
||||
@@ -179,11 +133,20 @@ class SPAStaticFiles(StaticFiles):
|
||||
except StarletteHTTPException as exc:
|
||||
if exc.status_code != 404:
|
||||
raise
|
||||
return await super().get_response("index.html", scope)
|
||||
return await self._fallback(path, scope)
|
||||
if response.status_code == 404:
|
||||
return await super().get_response("index.html", scope)
|
||||
return await self._fallback(path, scope)
|
||||
return response
|
||||
|
||||
async def _fallback(self, path, scope):
|
||||
# The mount is a catch-all, so an /api path no router claims — a typo,
|
||||
# or an endpoint this build removed — used to come back as the SPA's
|
||||
# HTML with status 200, and a client asking for JSON parsed a web page
|
||||
# instead of seeing that the route is not there.
|
||||
if path == "api" or path.startswith("api/"):
|
||||
raise StarletteHTTPException(status_code=404)
|
||||
return await super().get_response("index.html", scope)
|
||||
|
||||
|
||||
# Python's mimetypes table has no entry for woff2 on a slim Debian image, so
|
||||
# StaticFiles served the self-hosted fonts as application/octet-stream. Browsers
|
||||
|
||||
@@ -142,26 +142,22 @@ _running: set[int] = set()
|
||||
_tasks: set[asyncio.Task] = set()
|
||||
|
||||
|
||||
# Both factories below use the user's own key by construction. They read the
|
||||
# endpoint and key from `Settings` and never from `auth.DEMO_*`, so
|
||||
# summarization and embedding cannot spend the shared demo key. Their call sites
|
||||
# are also skipped when `using_demo` is true.
|
||||
#
|
||||
# Do not change these to accept a `ProviderConfig`. `summary_model` and
|
||||
# `embedding_model` are free-form user input and are not on the demo allowlist.
|
||||
# Both factories read the endpoint and the model names straight off `Settings`.
|
||||
# They used to also read an API key, which is gone: Ollama does not use one and
|
||||
# M2 removed cloud providers. `summary_model` and `embedding_model` fall back to
|
||||
# the narrator model when the user has not named a separate one.
|
||||
def summary_provider(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
return OpenAICompatibleProvider(
|
||||
settings.endpoint_url,
|
||||
settings.api_key_plain,
|
||||
settings.summary_model or settings.model,
|
||||
settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
|
||||
|
||||
def embedding_provider(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
return OpenAICompatibleProvider(
|
||||
settings.endpoint_url, settings.api_key_plain, settings.embedding_model
|
||||
settings.endpoint_url, settings.embedding_model
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -301,7 +301,8 @@ MIGRATIONS: list[tuple[int, str | dict[str, str]]] = [
|
||||
(63, "ALTER TABLE actions ADD COLUMN parent_id INTEGER REFERENCES actions(id) ON DELETE SET NULL"),
|
||||
(64, "CREATE INDEX IF NOT EXISTS ix_actions_parent ON actions (parent_id)"),
|
||||
# Phase 17: `Settings.stream` was dead state. Nothing ever read it, and every
|
||||
# turn streams. This is item S1 in `docs/self-review.md`. The table holds one
|
||||
# turn streams; upstream's self-review log flagged it as dead state. The
|
||||
# table holds one
|
||||
# row per user, so the rewrite is small and needs no VACUUM FULL.
|
||||
(65, "ALTER TABLE settings DROP COLUMN stream"),
|
||||
# Phase 17, SP8: drop the eight columns the story tree replaced. Each one was
|
||||
@@ -338,6 +339,28 @@ MIGRATIONS: list[tuple[int, str | dict[str, str]]] = [
|
||||
(74, "ALTER TABLE adventures ADD COLUMN persona_name VARCHAR(80) NOT NULL DEFAULT ''"),
|
||||
(75, "ALTER TABLE adventures ADD COLUMN persona_pronouns VARCHAR(40) NOT NULL DEFAULT ''"),
|
||||
(76, "ALTER TABLE adventures ADD COLUMN persona_desc TEXT NOT NULL DEFAULT ''"),
|
||||
# M2: how long to wait for the model. Upstream hardcoded 120s in the HTTP
|
||||
# client, which a cold model load on a CPU-only machine can exceed. The
|
||||
# default matches `providers.openai_compatible.DEFAULT_READ_TIMEOUT`.
|
||||
(77, "ALTER TABLE settings ADD COLUMN model_timeout_seconds INTEGER NOT NULL DEFAULT 300"),
|
||||
|
||||
# M3. A branch left behind by a divergent write records where the story left
|
||||
# it. NULL means active, which is what every existing branch is: before M3
|
||||
# the head could not sit behind the tip, so no branch had been superseded.
|
||||
# No backfill.
|
||||
(78, "ALTER TABLE branches ADD COLUMN superseded_at TIMESTAMP"),
|
||||
(79, "ALTER TABLE branches ADD COLUMN superseded_depth INTEGER"),
|
||||
# M4: Save Points. `create_all` creates the `checkpoints` table itself, on
|
||||
# existing databases as well as fresh ones, exactly as it did for
|
||||
# `memories` at version 2 and `branches` at version 46. What it does not
|
||||
# create is the index every list and every cascade reads, so that is what
|
||||
# this version is.
|
||||
#
|
||||
# No backfill. A Save Point records a decision someone made, and nobody has
|
||||
# made one yet: an M3 database has no position a user chose to name, and
|
||||
# inventing one would be inventing the decision.
|
||||
(80, "CREATE INDEX IF NOT EXISTS ix_checkpoints_adventure "
|
||||
"ON checkpoints (adventure_id)"),
|
||||
]
|
||||
|
||||
LATEST_VERSION = max((v for v, _ in MIGRATIONS), default=1)
|
||||
@@ -1106,23 +1129,3 @@ def bootstrap(engine: Engine, through: int = LATEST_VERSION) -> None:
|
||||
_backfill_parents(conn)
|
||||
current = version
|
||||
_set_version(conn, current)
|
||||
_encrypt_plaintext_api_keys(conn)
|
||||
|
||||
|
||||
def _encrypt_plaintext_api_keys(conn) -> None:
|
||||
"""Encrypts API keys saved before encryption at rest existed (Phase 8).
|
||||
|
||||
Those keys are stored in plain text, so this pass wraps them in Fernet. Plain
|
||||
SQL cannot do it. The pass runs on every start, and it matches no rows once
|
||||
every row carries the `enc:` prefix.
|
||||
"""
|
||||
from . import security # Deferred: security derives its key from DB_PATH setup.
|
||||
|
||||
rows = conn.execute(text(
|
||||
"SELECT id, api_key FROM settings WHERE api_key != '' AND api_key NOT LIKE 'enc:%'"
|
||||
)).all()
|
||||
for row_id, plain in rows:
|
||||
conn.execute(
|
||||
text("UPDATE settings SET api_key = :key WHERE id = :id"),
|
||||
{"key": security.encrypt_secret(plain), "id": row_id},
|
||||
)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
|
||||
String, Table, Text, UniqueConstraint, event,
|
||||
JSON, Boolean, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
|
||||
String, Text, event,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, Session, mapped_column, relationship
|
||||
|
||||
@@ -37,19 +37,12 @@ class User(Base):
|
||||
is_guest: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
last_seen_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
|
||||
# Shared demo key usage (resets when the UTC date changes).
|
||||
# Was the shared demo key's per-day tally. M2 removed the demo key; these
|
||||
# columns stay so existing databases open unchanged and are never written.
|
||||
demo_turns_used: Mapped[int] = mapped_column(Integer, default=0)
|
||||
demo_turns_date: Mapped[str] = mapped_column(String(10), default="")
|
||||
|
||||
|
||||
scenario_scripts = Table(
|
||||
"scenario_scripts",
|
||||
Base.metadata,
|
||||
Column("scenario_id", ForeignKey("scenarios.id", ondelete="CASCADE"), primary_key=True),
|
||||
Column("script_id", ForeignKey("scripts.id", ondelete="CASCADE"), primary_key=True),
|
||||
)
|
||||
|
||||
|
||||
class Scenario(Base):
|
||||
__tablename__ = "scenarios"
|
||||
|
||||
@@ -89,7 +82,6 @@ class Scenario(Base):
|
||||
back_populates="scenario", cascade="all, delete-orphan"
|
||||
)
|
||||
adventures: Mapped[list["Adventure"]] = relationship(back_populates="scenario")
|
||||
scripts: Mapped[list["Script"]] = relationship(secondary=scenario_scripts)
|
||||
|
||||
|
||||
class Adventure(Base):
|
||||
@@ -121,6 +113,9 @@ class Adventure(Base):
|
||||
persona_name: Mapped[str] = mapped_column(String(80), default="")
|
||||
persona_pronouns: Mapped[str] = mapped_column(String(40), default="")
|
||||
persona_desc: Mapped[str] = mapped_column(Text, default="")
|
||||
# Was the campaign scripting engine's shared `state` object. M2 removed
|
||||
# scripting; the column stays so existing databases open unchanged, and it
|
||||
# is never written with anything but an empty dict.
|
||||
script_state: Mapped[dict] = mapped_column(JSON, default=dict)
|
||||
# Phase 12: live RPG world state (world/player/npc stats + milestones),
|
||||
# instantiated from the scenario's stat_schema. Empty when there's no RPG layer.
|
||||
@@ -177,11 +172,6 @@ class Adventure(Base):
|
||||
cascade="all, delete-orphan",
|
||||
order_by="Action.id",
|
||||
)
|
||||
scripts: Mapped[list["AdventureScript"]] = relationship(
|
||||
back_populates="adventure",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="AdventureScript.position",
|
||||
)
|
||||
memories: Mapped[list["Memory"]] = relationship(
|
||||
back_populates="adventure",
|
||||
cascade="all, delete-orphan",
|
||||
@@ -238,6 +228,78 @@ class Branch(Base):
|
||||
# branch by its fork depth, which deleting a branch does not change.
|
||||
name: Mapped[str | None] = mapped_column(String(80), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
# M3: the disposition `DATA-MODEL.md` §5 gives a branch, stored as the fact
|
||||
# that produced it. NULL means active. A value means a divergent write left
|
||||
# this branch at `superseded_depth`, so its nodes past that depth are
|
||||
# retained history that no active head is reading.
|
||||
#
|
||||
# Nothing reads these to decide behaviour. Redo follows the lineage, so a
|
||||
# wrong value here cannot make the story wrong; they exist for the cleanup
|
||||
# and discarded-history features that `STORY-BRANCH-SEMANTICS.md` §28-29
|
||||
# leave to a later version. See `head.mark_superseded`.
|
||||
superseded_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
|
||||
superseded_depth: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
|
||||
class Checkpoint(Base):
|
||||
"""M4: a Save Point — a durable named pointer to a story position.
|
||||
|
||||
"Save Point" is what the user reads; `checkpoint` is what the code calls it
|
||||
(`BROWSER-UX-SPEC.md` §23).
|
||||
|
||||
The row holds a name and a coordinate, and no story. `DATA-MODEL.md` §8
|
||||
describes the pointer as naming a turn; the coordinate here is
|
||||
`(branch_id, depth)`, which is what M3 made the head and what
|
||||
`head.node_at` resolves. Restoring one is therefore head movement with a
|
||||
bounds check rather than a restore system of its own — see ADR 012 and
|
||||
`head.move_to_node`.
|
||||
|
||||
A coordinate rather than an action id, deliberately. One coordinate can
|
||||
hold several attempts at a turn and exactly one of them is live, so a
|
||||
retry replaces the row a Save Point would have pinned. "Turn 42 of this
|
||||
line" survives a retry; "action 918" would point at a take the story no
|
||||
longer tells.
|
||||
|
||||
`branch_id` is the branch the node itself sits on, not the branch that was
|
||||
being read when the Save Point was made. Those differ whenever the head is
|
||||
resting in a shared prefix, and the node's own branch is the one that still
|
||||
names the position after the reader has moved elsewhere.
|
||||
|
||||
**Nothing removes a Save Point but the user.** They are not cleaned up for
|
||||
going stale, for being behind the head, or for pointing into a future the
|
||||
story has left (`STORY-BRANCH-SEMANTICS.md` §19).
|
||||
|
||||
That includes deleting a branch. `branch_id` carries `ON DELETE CASCADE` as
|
||||
referential integrity — a Save Point must never point at a branch that is
|
||||
gone — but the branch endpoint refuses to delete a branch any Save Point
|
||||
names, so the cascade does not fire through the application
|
||||
(`routers/adventures/branches.py`, `STORY-BRANCH-SEMANTICS.md` §19.1). The
|
||||
user deletes the Save Point first, which deletes no story, and then the
|
||||
branch. Deleting the whole campaign does cascade, and should: that is what
|
||||
the user asked for.
|
||||
"""
|
||||
|
||||
__tablename__ = "checkpoints"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
adventure_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("adventures.id", ondelete="CASCADE")
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(120), default="")
|
||||
# `DATA-MODEL.md` §8's optional notes, and `BROWSER-UX-SPEC.md` §24's
|
||||
# optional second field. Empty is the ordinary case.
|
||||
note: Mapped[str] = mapped_column(Text, default="")
|
||||
branch_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("branches.id", ondelete="CASCADE")
|
||||
)
|
||||
depth: Mapped[int] = mapped_column(Integer)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
# Bumped by a rename, which is the only edit a Save Point allows. The
|
||||
# coordinate is never rewritten: `STORY-BRANCH-SEMANTICS.md` §24 keeps a
|
||||
# Save Point's meaning auditable by making "move it" delete-and-recreate.
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime, default=utcnow, onupdate=utcnow
|
||||
)
|
||||
|
||||
|
||||
class Memory(Base):
|
||||
@@ -419,9 +481,10 @@ class Action(Base):
|
||||
# and for re-attaching the emit block when replaying history to the model.
|
||||
# Mirrors the active variant, same as text/reasoning/context_snapshot.
|
||||
world_delta: Mapped[dict | None] = mapped_column(JSON, nullable=True)
|
||||
# Phase 14, SP4: the shared script state and the RPG world state as they
|
||||
# stood after this node was played. These columns record the node's outcome
|
||||
# rather than its starting position.
|
||||
# Phase 14, SP4: the RPG world state as it stood after this node was
|
||||
# played. These columns record the node's outcome rather than its starting
|
||||
# position. (`state_after` held the scripting engine's state, which M2
|
||||
# removed; it is now always written empty.)
|
||||
#
|
||||
# Two operations need this outcome, and neither can use a snapshot taken
|
||||
# before the turn. Switching between siblings must restore the state that
|
||||
@@ -508,52 +571,6 @@ class Action(Base):
|
||||
return out
|
||||
|
||||
|
||||
class Script(Base):
|
||||
__tablename__ = "scripts"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), nullable=True
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(200), default="Untitled Script")
|
||||
description: Mapped[str] = mapped_column(Text, default="")
|
||||
library_js: Mapped[str] = mapped_column(Text, default="")
|
||||
input_js: Mapped[str] = mapped_column(Text, default="")
|
||||
context_js: Mapped[str] = mapped_column(Text, default="")
|
||||
output_js: Mapped[str] = mapped_column(Text, default="")
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, onupdate=utcnow)
|
||||
|
||||
|
||||
class AdventureScript(Base):
|
||||
"""A script copied into an adventure at creation, so library edits don't
|
||||
change running adventures unless the player explicitly re-syncs it from
|
||||
`source_script_id`. `state` lives on Adventure.script_state (one shared
|
||||
state per adventure, as in AI Dungeon)."""
|
||||
|
||||
__tablename__ = "adventure_scripts"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
adventure_id: Mapped[int] = mapped_column(ForeignKey("adventures.id", ondelete="CASCADE"))
|
||||
# The library Script that this copy was made from, which lets the player
|
||||
# re-sync it on demand. The value is NULL for legacy copies that predate
|
||||
# this column, and for demo-derived copies whose source the player does not
|
||||
# own. Those copies fall back to matching by name, or cannot be synced.
|
||||
source_script_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("scripts.id", ondelete="SET NULL"), nullable=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
name: Mapped[str] = mapped_column(String(200), default="Untitled Script")
|
||||
description: Mapped[str] = mapped_column(Text, default="")
|
||||
library_js: Mapped[str] = mapped_column(Text, default="")
|
||||
input_js: Mapped[str] = mapped_column(Text, default="")
|
||||
context_js: Mapped[str] = mapped_column(Text, default="")
|
||||
output_js: Mapped[str] = mapped_column(Text, default="")
|
||||
|
||||
adventure: Mapped[Adventure] = relationship(back_populates="scripts")
|
||||
|
||||
|
||||
class Settings(Base):
|
||||
__tablename__ = "settings"
|
||||
|
||||
@@ -564,8 +581,10 @@ class Settings(Base):
|
||||
ForeignKey("users.id", ondelete="CASCADE"), nullable=True, unique=True
|
||||
)
|
||||
endpoint_url: Mapped[str] = mapped_column(String(500), default="http://localhost:11434/v1")
|
||||
# Encrypted at rest with Fernet, which produces a value that starts with
|
||||
# "enc:". See security.py. To read the key, use `api_key_plain`.
|
||||
# Was a cloud provider's API key, encrypted at rest. Ollama does not use
|
||||
# one and M2 removed cloud providers, so nothing reads or writes this now.
|
||||
# The column stays so existing databases open unchanged; an old value is
|
||||
# left where it is rather than migrated or decrypted.
|
||||
api_key: Mapped[str] = mapped_column(String(500), default="")
|
||||
model: Mapped[str] = mapped_column(String(200), default="")
|
||||
api_mode: Mapped[str] = mapped_column(String(20), default="chat") # chat|completion
|
||||
@@ -573,12 +592,17 @@ class Settings(Base):
|
||||
# 800 leaves room for a full scene; 400 tended to truncate mid-paragraph
|
||||
# and left reasoning models with nothing after their thinking.
|
||||
max_output_tokens: Mapped[int] = mapped_column(Integer, default=800)
|
||||
# Separate thinking budget for reasoning models (OpenRouter-style
|
||||
# `reasoning: {max_tokens}`); 0 = param not sent, -1 = reasoning explicitly
|
||||
# off (`reasoning: {effort: none}`). Added on top of
|
||||
# max_output_tokens so story output keeps its full budget.
|
||||
# Was an OpenRouter-style thinking budget. Ollama's OpenAI-compatible
|
||||
# endpoint ignores the field, so M2 stopped sending it and removed it from
|
||||
# the Settings API and UI. The column stays so existing databases open
|
||||
# unchanged and is never read.
|
||||
reasoning_max_tokens: Mapped[int] = mapped_column(Integer, default=0)
|
||||
context_token_budget: Mapped[int] = mapped_column(Integer, default=16384)
|
||||
# How long to wait for the model, in seconds, before giving up on a turn.
|
||||
# A cold load of a mid-sized model on a CPU-only machine can take minutes,
|
||||
# while the same turn takes seconds once the model is resident. See
|
||||
# `providers.openai_compatible.DEFAULT_READ_TIMEOUT`.
|
||||
model_timeout_seconds: Mapped[int] = mapped_column(Integer, default=300)
|
||||
narrator_prompt: Mapped[str] = mapped_column(
|
||||
Text,
|
||||
default=(
|
||||
@@ -599,120 +623,18 @@ class Settings(Base):
|
||||
memory_bank_capacity: Mapped[int] = mapped_column(Integer, default=80)
|
||||
memory_top_k: Mapped[int] = mapped_column(Integer, default=5)
|
||||
|
||||
@property
|
||||
def has_api_key(self) -> bool:
|
||||
return bool(self.api_key)
|
||||
|
||||
@property
|
||||
def api_key_plain(self) -> str:
|
||||
from . import security # local import: models is imported before security
|
||||
|
||||
return security.decrypt_secret(self.api_key)
|
||||
|
||||
|
||||
# ---------- Visit analytics (see analytics.py) ----------
|
||||
# Two intentionally simple tables. Neither can hold text that a player wrote,
|
||||
# and neither can be joined back to a `users` row, because the visitor column
|
||||
# holds an HMAC and has no foreign key. When guest cleanup deletes an account,
|
||||
# the history that account contributed remains intact and anonymous.
|
||||
|
||||
|
||||
class AnalyticsDaily(Base):
|
||||
"""One counter: how many times `label` happened within `metric` on `day`.
|
||||
|
||||
The table stores a generic triple of metric, label, and hits rather than one
|
||||
column per statistic. Measuring something new therefore costs a constant
|
||||
rather than a migration. The only writer is an UPSERT that runs from a
|
||||
buffer. See `analytics.flush`.
|
||||
"""
|
||||
|
||||
__tablename__ = "analytics_daily"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
day: Mapped[str] = mapped_column(String(10), index=True) # YYYY-MM-DD, UTC
|
||||
metric: Mapped[str] = mapped_column(String(32))
|
||||
label: Mapped[str] = mapped_column(String(80), default="")
|
||||
hits: Mapped[int] = mapped_column(Integer, default=0)
|
||||
|
||||
# The upsert target: one row per bucket per day, created or incremented.
|
||||
__table_args__ = (
|
||||
UniqueConstraint("day", "metric", "label", name="uq_analytics_daily_bucket"),
|
||||
)
|
||||
|
||||
|
||||
class AnalyticsVisitorDay(Base):
|
||||
"""One visitor, one day, and which funnel steps they reached on it.
|
||||
|
||||
This table exists so that the funnel counts people rather than clicks. A
|
||||
player who starts six adventures counts as one person who started an
|
||||
adventure. `is_new` is set when the visitor has no earlier row, which is why
|
||||
the visitor column also has an index of its own.
|
||||
"""
|
||||
|
||||
__tablename__ = "analytics_visitor_days"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
day: Mapped[str] = mapped_column(String(10))
|
||||
# HMAC of the user id under the app secret; not reversible, not a key.
|
||||
visitor: Mapped[str] = mapped_column(String(32))
|
||||
is_new: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
opened: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
created: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
played: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
signed_up: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint("day", "visitor", name="uq_analytics_visitor_day"),
|
||||
Index("ix_analytics_visitor", "visitor"),
|
||||
)
|
||||
|
||||
|
||||
class AccessEvent(Base):
|
||||
"""One sign-in, registration, failed attempt, or session first-seen.
|
||||
|
||||
This table is the counterpart to the two above, and it is kept separate from
|
||||
them on purpose. It identifies people by design, recording address, email,
|
||||
and device. Keeping it in its own table and its own module means that the
|
||||
structure enforces the anonymity of the counters rather than a convention.
|
||||
|
||||
`user_id` is a plain integer with no foreign key. An access log that
|
||||
disappeared when the account did would not serve its purpose, and guest
|
||||
cleanup deletes accounts on a schedule. `who` and `is_guest` are snapshots
|
||||
for the same reason, so a row still reads correctly after the account is
|
||||
gone.
|
||||
"""
|
||||
|
||||
__tablename__ = "access_events"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, index=True)
|
||||
# session | login | register | login_failed
|
||||
kind: Mapped[str] = mapped_column(String(16))
|
||||
user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
# The email for a registered account, or a label such as "Guest #12"
|
||||
# otherwise. For a failed sign-in, this holds the address that was tried,
|
||||
# which is the reason the row exists.
|
||||
who: Mapped[str] = mapped_column(String(320), default="")
|
||||
is_guest: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
ip: Mapped[str] = mapped_column(String(45), default="") # 45 = max IPv6
|
||||
country: Mapped[str] = mapped_column(String(16), default="")
|
||||
device: Mapped[str] = mapped_column(String(16), default="")
|
||||
user_agent: Mapped[str] = mapped_column(String(200), default="")
|
||||
|
||||
|
||||
# Phase 14: the fallback under `tree.place_action`.
|
||||
# The hosted visitor dashboard's two counter tables and the access log that
|
||||
# recorded sign-ins, addresses and devices used to be mapped here. M2 removed
|
||||
# the hosted deployment they served.
|
||||
#
|
||||
# Since SP2, reads filter on `branch_id` and `depth`. A node written without
|
||||
# them is invisible to every page, every context build, and every memory pass.
|
||||
# The failure is silent, because nothing raises an error. Every current writer
|
||||
# places its nodes explicitly, but relying on that would also mean relying on
|
||||
# every fixture, script, and test written from now on. The session therefore
|
||||
# enforces the rule as rows travel to the database.
|
||||
#
|
||||
# This listener is registered here rather than in tree.py so that importing the
|
||||
# models is enough to enable it. The invariant belongs to the rows, not to the
|
||||
# module that usually writes them. The import sits inside the callback because
|
||||
# tree.py imports this module.
|
||||
# The tables are left in the database rather than dropped: they are inert,
|
||||
# nothing reads or writes them, and a destructive migration would risk an
|
||||
# existing campaign database for tidiness alone. They are not product
|
||||
# functionality.
|
||||
|
||||
|
||||
@event.listens_for(Session, "before_flush")
|
||||
def _place_new_nodes_on_the_tree(session, flush_context, instances):
|
||||
from . import tree
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
"""SSRF guard for the one place the server makes an outbound request to a
|
||||
user-supplied address: the BYOK `endpoint_url` (connection test + turns/chat).
|
||||
|
||||
Without this guard, a hosted user could point `endpoint_url` at an internal
|
||||
service or at the cloud metadata endpoint, 169.254.169.254, and have the server
|
||||
fetch it. The connection test even returns part of the response. The guard
|
||||
therefore refuses any URL that resolves to a non-public address.
|
||||
|
||||
The guard does nothing in local mode. A local install talking to
|
||||
http://localhost:11434, which is Ollama, is the intended case. The guard applies
|
||||
only to a hosted, multi-user deployment, where the endpoint comes from an
|
||||
untrusted visitor.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from . import auth
|
||||
|
||||
|
||||
def endpoint_block_reason(url: str) -> str | None:
|
||||
"""A human-readable reason this URL must NOT be fetched server-side, or None
|
||||
if it's allowed. Resolves the host and rejects it if any resulting address
|
||||
is non-public (private, loopback, link-local/metadata, reserved, …).
|
||||
|
||||
Checking at request time (not just on save) is deliberate: it resists a DNS
|
||||
record that flips to a private IP after the value was stored.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return None
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return "the endpoint URL must start with http:// or https://"
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return "the endpoint URL has no host"
|
||||
port = parsed.port or (443 if parsed.scheme == "https" else 80)
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return "the endpoint host could not be resolved"
|
||||
for info in infos:
|
||||
try:
|
||||
ip = ipaddress.ip_address(info[4][0])
|
||||
except ValueError:
|
||||
return "the endpoint host resolved to an unrecognized address"
|
||||
# is_global is the strict allowlist: private/loopback/link-local/CGNAT
|
||||
# all report False, so this one check covers the metadata IP too.
|
||||
if not ip.is_global or ip.is_multicast or ip.is_reserved:
|
||||
return "the endpoint URL resolves to a non-public address"
|
||||
return None
|
||||
@@ -3,32 +3,27 @@ from typing import AsyncIterator
|
||||
|
||||
import httpx
|
||||
|
||||
from .. import debuglog, netguard, tlstrust
|
||||
from .. import debuglog, endpoints, tlstrust
|
||||
from .base import PromptParts, Provider, ProviderError
|
||||
|
||||
# Appended after the story text in chat mode, so a chat-tuned model continues
|
||||
# the prose rather than replying conversationally.
|
||||
CHAT_CONTINUE_HINT = "\n\n[Continue the story directly. Output only story text.]"
|
||||
|
||||
# OpenRouter serves one model from whichever upstream is available, and every
|
||||
# upstream holds its own prompt cache, so a request routed somewhere new starts
|
||||
# with a cold cache however stable the prompt is. Naming a preferred upstream
|
||||
# makes routing deterministic, which is what allows a cache hit at all.
|
||||
#
|
||||
# `allow_fallbacks` stays at its default of true on purpose, because this is a
|
||||
# preference rather than a restriction. If the named upstream is down, the
|
||||
# request still goes elsewhere and only misses the cache, which is the behavior
|
||||
# without this setting.
|
||||
#
|
||||
# This is a list rather than a value derived from the model slug. The vendor half
|
||||
# of a slug is usually the provider slug, such as "deepseek/..." mapping to
|
||||
# "deepseek", which was verified against /api/v1/providers, but not reliably.
|
||||
# Google's models are served by "google-ai-studio" and "google-vertex", and there
|
||||
# is no "google". Look a vendor up on the model's Providers tab before adding it
|
||||
# here. A slug that does not exist is a routing preference that, at best, does
|
||||
# nothing.
|
||||
_OPENROUTER_HOST = "openrouter.ai"
|
||||
_PREFERRED_UPSTREAM = {"deepseek": "deepseek"}
|
||||
# A machine that is not listening refuses in milliseconds, so a slow connect
|
||||
# means the wrong address rather than a busy model.
|
||||
CONNECT_TIMEOUT = 10.0
|
||||
|
||||
# How long to wait for generation when Settings names no value. Upstream
|
||||
# hardcoded 120s, and M1 measured a *cold* load of a 3B model on a GPU-less
|
||||
# four-core host exceeding it three times while the same turn took 6-9 seconds
|
||||
# once the model was resident. 300s covers a cold start on modest hardware and
|
||||
# is still a number: a wedged endpoint fails rather than hanging forever.
|
||||
DEFAULT_READ_TIMEOUT = 300.0
|
||||
|
||||
# Embeddings are short and never cold-load a large model.
|
||||
EMBED_READ_TIMEOUT = 60.0
|
||||
|
||||
|
||||
|
||||
# Completion endpoints have no roles, so a chat has to be flattened into one
|
||||
@@ -44,74 +39,47 @@ def flatten_messages(messages: list[dict]) -> str:
|
||||
|
||||
|
||||
class OpenAICompatibleProvider(Provider):
|
||||
"""Adapter for any /v1-style endpoint.
|
||||
"""Adapter for Ollama's OpenAI-compatible `/v1` API.
|
||||
|
||||
This covers Ollama, LM Studio, OpenAI, OpenRouter, vLLM, and Groq, among
|
||||
others.
|
||||
The protocol is OpenAI's, which is what the module is named for; the
|
||||
product speaks it to Ollama and to nothing else. `endpoints.py` decides
|
||||
which addresses may be reached, and every request re-checks — the shape of
|
||||
the wire format is not the same thing as permission to use it.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
endpoint_url: str,
|
||||
api_key: str,
|
||||
model: str,
|
||||
api_mode: str = "chat",
|
||||
reasoning_max_tokens: int = 0,
|
||||
read_timeout: float | None = None,
|
||||
):
|
||||
self.base_url = endpoint_url.rstrip("/")
|
||||
self.api_key = api_key
|
||||
self.model = model
|
||||
self.api_mode = api_mode # Either "chat" or "completion".
|
||||
# The thinking budget for reasoning models, on top of `max_tokens`. A
|
||||
# value of 0 means the `reasoning` parameter is not sent, because an
|
||||
# endpoint that does not know the field may reject it. A negative value
|
||||
# asks the endpoint to turn reasoning off.
|
||||
self.reasoning_max_tokens = reasoning_max_tokens
|
||||
# How long to wait for the model, in seconds. Cold-loading a model on a
|
||||
# CPU-only machine can take minutes, and a fixed short timeout reports
|
||||
# that as a failure. See `DEFAULT_READ_TIMEOUT`.
|
||||
self.read_timeout = read_timeout or DEFAULT_READ_TIMEOUT
|
||||
# The token accounting from the last call, when the endpoint reported
|
||||
# any. It holds the prompt and completion counts, plus, on OpenRouter,
|
||||
# `prompt_tokens_details.cached_tokens`, which is the number of prompt
|
||||
# tokens read from cache rather than billed in full. Every request method
|
||||
# writes it, so a caller reads it after the call it made. One provider is
|
||||
# built per request.
|
||||
# any. Every request method writes it, so a caller reads it after the
|
||||
# call it made. One provider is built per request.
|
||||
self.last_usage: dict | None = None
|
||||
|
||||
def _headers(self) -> dict:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self.api_key:
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
return headers
|
||||
# No Authorization header: Ollama does not use one, and this build has
|
||||
# no cloud provider to carry a key for.
|
||||
return {"Content-Type": "application/json"}
|
||||
|
||||
def _apply_reasoning_budget(self, body: dict) -> None:
|
||||
"""Gives reasoning models their own thinking budget, in the OpenRouter style.
|
||||
def _timeout(self, seconds: float | None = None) -> httpx.Timeout:
|
||||
"""Short to connect, patient to read.
|
||||
|
||||
The method raises `max_tokens`, so the output keeps its full budget.
|
||||
|
||||
A negative budget does the opposite. It sends `effort: "none"` to turn
|
||||
reasoning off on a model that reasons by default, such as DeepSeek V4
|
||||
Flash. That differs from `exclude: true`, which still reasons and still
|
||||
bills for it while hiding the trace. Zero still means send nothing, so an
|
||||
endpoint that rejects unknown fields, such as Ollama, keeps working.
|
||||
A machine that is not listening says so in milliseconds, so a slow
|
||||
connect is a wrong address rather than a busy model and should fail
|
||||
fast. Generation is the opposite: the first token can be minutes away
|
||||
while a model loads.
|
||||
"""
|
||||
if self.api_mode != "chat":
|
||||
return
|
||||
if self.reasoning_max_tokens < 0:
|
||||
body["reasoning"] = {"effort": "none"}
|
||||
elif self.reasoning_max_tokens > 0:
|
||||
body["reasoning"] = {"max_tokens": self.reasoning_max_tokens}
|
||||
body["max_tokens"] += self.reasoning_max_tokens
|
||||
|
||||
def _apply_provider_routing(self, body: dict) -> None:
|
||||
"""Prefers one upstream on OpenRouter, so the prompt cache stays warm.
|
||||
|
||||
The method does nothing anywhere else. `provider` is an OpenRouter
|
||||
extension, and Ollama and similar servers reject fields they do not know.
|
||||
The `reasoning` parameter above is written around the same constraint.
|
||||
"""
|
||||
if _OPENROUTER_HOST not in self.base_url:
|
||||
return
|
||||
upstream = _PREFERRED_UPSTREAM.get(self.model.split("/", 1)[0].lower())
|
||||
if upstream:
|
||||
body["provider"] = {"order": [upstream]}
|
||||
return httpx.Timeout(seconds or self.read_timeout, connect=CONNECT_TIMEOUT)
|
||||
|
||||
def _record_usage(self, payload: dict) -> None:
|
||||
"""Records the endpoint's own token accounting, if it reported any.
|
||||
@@ -147,8 +115,6 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": True,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
return url, body
|
||||
|
||||
@staticmethod
|
||||
@@ -227,8 +193,6 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": True,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
async for event in self._stream(url, body):
|
||||
yield event
|
||||
|
||||
@@ -238,17 +202,17 @@ class OpenAICompatibleProvider(Provider):
|
||||
The method POSTs a streaming request, yields `("text", chunk)` and
|
||||
`("reasoning", chunk)` pairs, and logs the exchange.
|
||||
"""
|
||||
# SSRF guard for hosted mode. A user-supplied `endpoint_url` must not
|
||||
# point at an internal or metadata address. This does nothing for a
|
||||
# local install.
|
||||
reason = netguard.endpoint_block_reason(url)
|
||||
# Re-checked on every request, not only when the endpoint was saved: a
|
||||
# hostname that resolved to a LAN address yesterday can resolve
|
||||
# somewhere else today, and a database row can be edited by hand.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
received: list[str] = []
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(120, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
async with client.stream("POST", url, json=body, headers=self._headers()) as resp:
|
||||
if resp.status_code != 200:
|
||||
@@ -351,13 +315,16 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
|
||||
# Same check as `_stream`: every outbound request re-tests the
|
||||
# endpoint, so no path reaches an address the policy refuses.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(120, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
resp = await client.post(url, json=body, headers=self._headers())
|
||||
except httpx.HTTPError as exc:
|
||||
@@ -383,10 +350,15 @@ class OpenAICompatibleProvider(Provider):
|
||||
raise ProviderError("No embedding model configured — set one in Settings.")
|
||||
url = f"{self.base_url}/embeddings"
|
||||
body = {"model": self.model, "input": texts}
|
||||
# Same check as `_stream`: every outbound request re-tests the
|
||||
# endpoint, so no path reaches an address the policy refuses.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(60, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(EMBED_READ_TIMEOUT), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
resp = await client.post(url, json=body, headers=self._headers())
|
||||
except httpx.HTTPError as exc:
|
||||
|
||||
@@ -13,6 +13,7 @@ Read the modules in this order to follow a turn from end to end:
|
||||
turns playing a turn, and the lock that allows only one at a time
|
||||
takes retries and the attempts that collect at one coordinate
|
||||
branches where a story splits
|
||||
checkpoints Save Points: durable names for positions the head can return to
|
||||
|
||||
What this package re-exports, and what it deliberately does not:
|
||||
|
||||
@@ -31,8 +32,8 @@ from . import ( # noqa: F401
|
||||
turns,
|
||||
takes,
|
||||
branches,
|
||||
checkpoints,
|
||||
bundle_io,
|
||||
scripts,
|
||||
refresh,
|
||||
insights,
|
||||
memories,
|
||||
@@ -41,7 +42,7 @@ from . import ( # noqa: F401
|
||||
from ... import limits # noqa: F401 `adventures.limits` is patched by tests.
|
||||
from .crud import SNIPPET_MAX, _snippet
|
||||
from .paging import ACTION_PAGE
|
||||
from .takes import retry_action, undo_turn
|
||||
from .takes import redo_turn, retry_action, undo_turn
|
||||
from .turns import world_delta_of
|
||||
|
||||
__all__ = [
|
||||
@@ -49,6 +50,7 @@ __all__ = [
|
||||
"SNIPPET_MAX",
|
||||
"_snippet",
|
||||
"limits",
|
||||
"redo_turn",
|
||||
"retry_action",
|
||||
"router",
|
||||
"undo_turn",
|
||||
|
||||
@@ -8,7 +8,7 @@ coordinate through `nodes.delete_turn`.
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import attempts, models, schemas, tree
|
||||
from ... import attempts, head, models, schemas, tree
|
||||
from ...database import get_db
|
||||
|
||||
from . import turns
|
||||
@@ -41,6 +41,11 @@ def list_actions(
|
||||
],
|
||||
total=total,
|
||||
has_more=has_more,
|
||||
# Every page carries them, not just the newest window: the client reads
|
||||
# the flags off whichever page arrived last, and scrolling up must not
|
||||
# be able to grey out a Redo that is still available (M3).
|
||||
can_undo=head.can_undo(db, adventure),
|
||||
can_redo=head.can_redo(db, adventure),
|
||||
)
|
||||
|
||||
|
||||
@@ -55,6 +60,27 @@ def update_action(
|
||||
action = db.get(models.Action, action_id)
|
||||
if action is None or action.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Action not found")
|
||||
# An edit rewrites this row and re-evaluates nothing after it, which is what
|
||||
# makes it a correction rather than a new continuation. That is safe while
|
||||
# everything descending from the row is on screen, and unsafe the moment
|
||||
# something descends from it that is not — an undone future, or a line a
|
||||
# divergence left behind. The reader cannot see that story, so they cannot
|
||||
# see what their correction has just contradicted (M3).
|
||||
#
|
||||
# Refusing is the whole of the fix, deliberately. Making the edit fork, so
|
||||
# that the original text and its future stay whole, is
|
||||
# `STORY-BRANCH-SEMANTICS.md` §14-15 — and §15 requires re-evaluating the
|
||||
# state the edited prose implies, which is M5's extraction pass. Neither is
|
||||
# started here. What is closed is the one case where the application could
|
||||
# produce retained history that silently disagrees with itself.
|
||||
if head.displaced_history_under(db, adventure, action):
|
||||
raise HTTPException(
|
||||
400,
|
||||
"This turn has a later story that is not on screen — undone, or "
|
||||
"left behind by a new continuation. Editing it here would change "
|
||||
"the words that story was written from. Redo to bring it back "
|
||||
"first, or play the turn again to start a new line from here.",
|
||||
)
|
||||
# One row holds one text. Nothing mirrors it now, so nothing else has to be
|
||||
# updated. The edit used to have to be written into the live variant entry
|
||||
# as well, or paging away and back reverted it.
|
||||
@@ -81,6 +107,7 @@ def delete_action(
|
||||
# This works like undo. The turn is deleted with all of its attempts,
|
||||
# and whatever it produced is withdrawn. The marks are depths, and a
|
||||
# depth does not move when an action before it is deleted.
|
||||
was_at = adventure.head_depth
|
||||
delete_turn(db, adventure, action)
|
||||
db.flush()
|
||||
db.expire(adventure, ["actions"])
|
||||
@@ -88,6 +115,13 @@ def delete_action(
|
||||
# middle leaves a gap in the depths, which is intended. See
|
||||
# `_backfill_tree`.
|
||||
tree.refresh_head(db, adventure)
|
||||
# `refresh_head` recomputes the tip, which since M3 is not the head. A
|
||||
# story sitting behind its retained tip must not be dragged forward to
|
||||
# the tip by an unrelated delete — that would silently Redo it. Keep the
|
||||
# head where the reader left it, unless the delete took the ground out
|
||||
# from under it, in which case the new tip is as far as it can stay.
|
||||
if was_at < adventure.head_depth:
|
||||
adventure.head_depth = was_at
|
||||
# The script state and the world state belong to the adventure, not to
|
||||
# the node, so deleting the node does not take back what it did to
|
||||
# them. Put them back to what the story now ends with, which is the
|
||||
|
||||
@@ -56,6 +56,19 @@ def list_branches(
|
||||
.group_by(models.Action.branch_id)
|
||||
.all()
|
||||
}
|
||||
# M4 closeout: how many Save Points name a position on each line. Deleting a
|
||||
# branch deletes them along with its story, and the panel has to be able to
|
||||
# say so before the button is pressed (review §R B-2). One grouped query for
|
||||
# the whole tree, like the one above it — never one per branch.
|
||||
save_points = {
|
||||
branch_id: count
|
||||
for branch_id, count in db.query(
|
||||
models.Checkpoint.branch_id, func.count(models.Checkpoint.id)
|
||||
)
|
||||
.filter(models.Checkpoint.adventure_id == adventure.id)
|
||||
.group_by(models.Checkpoint.branch_id)
|
||||
.all()
|
||||
}
|
||||
out = []
|
||||
for branch in branches:
|
||||
count, tip = owned.get(branch.id, (0, None))
|
||||
@@ -70,6 +83,7 @@ def list_branches(
|
||||
branch.fork_depth if branch.fork_depth is not None else tree.NO_DEPTH
|
||||
),
|
||||
own_actions=count,
|
||||
save_points=save_points.get(branch.id, 0),
|
||||
is_head=(branch.id == adventure.head_branch_id),
|
||||
name=branch.name,
|
||||
created_at=branch.created_at,
|
||||
@@ -151,15 +165,31 @@ def delete_branch(
|
||||
heavily retried adventure from growing without bound. That is why it ships
|
||||
with the view that first lets anyone create a fork rather than after it.
|
||||
|
||||
Two kinds of branch cannot be deleted. The root cannot, because it holds the
|
||||
turns every other branch borrows, so deleting it deletes the whole story. The
|
||||
branch currently being read cannot, and neither can any branch it was forked
|
||||
from, because the cascade would remove the head under the player and leave
|
||||
`head_branch_id` dangling. Switch branches first.
|
||||
Three kinds of branch cannot be deleted. The root cannot, because it holds
|
||||
the turns every other branch borrows, so deleting it deletes the whole story.
|
||||
The branch currently being read cannot, and neither can any branch it was
|
||||
forked from, because the cascade would remove the head under the player and
|
||||
leave `head_branch_id` dangling. Switch branches first.
|
||||
|
||||
The third is M4's: **a branch a Save Point names cannot be deleted while that
|
||||
Save Point exists.** `STORY-BRANCH-SEMANTICS.md` §19 says a named checkpoint
|
||||
remains until explicitly deleted, and §28 says a future cleanup feature must
|
||||
retain paths referenced by checkpoints. A cascade that removed Save Points
|
||||
along with a branch would break both, and would break them silently: the
|
||||
story the user asked to delete is the visible thing, and the named moments
|
||||
would go without ever being named in the request. So the deletion is refused,
|
||||
the Save Points are listed, and the user decides — delete the Save Point
|
||||
first, then the branch. Deleting a Save Point still deletes no story (§25),
|
||||
so the recovery costs nothing but a click.
|
||||
|
||||
The check covers the whole doomed subtree, not just this branch, because
|
||||
deleting a branch takes everything forked from it.
|
||||
|
||||
Nodes and memories are deleted by `ON DELETE CASCADE`, and descendants by the
|
||||
cascade on `branches.parent_branch_id`, so the delete is a single statement
|
||||
however deep the subtree is.
|
||||
however deep the subtree is. `checkpoints.branch_id` also carries a cascade,
|
||||
as referential integrity — a Save Point must never point at a branch that is
|
||||
gone — but the guard above means it does not fire through this endpoint.
|
||||
"""
|
||||
branch = get_branch_or_404(adventure, branch_id, db)
|
||||
if branch.parent_branch_id is None:
|
||||
@@ -178,6 +208,11 @@ def delete_branch(
|
||||
400, "You are reading this branch, or one forked from it. Switch to "
|
||||
"another branch first.",
|
||||
)
|
||||
# Refused before the lock is taken: this is a decision about the request, not
|
||||
# a race with a turn.
|
||||
protecting = _save_points_protecting(db, adventure, branch)
|
||||
if protecting:
|
||||
raise HTTPException(409, _protected_message(protecting))
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
# Collect the subtree before the delete, because afterwards there is no
|
||||
@@ -199,6 +234,55 @@ def delete_branch(
|
||||
finally:
|
||||
turns._active_turns.discard(adventure_id)
|
||||
|
||||
# How many Save Point names to spell out before the message starts summarising.
|
||||
# Enough to be actionable, few enough to stay a sentence.
|
||||
NAMED_IN_REFUSAL = 3
|
||||
|
||||
|
||||
def _save_points_protecting(
|
||||
db: Session, adventure: models.Adventure, branch: models.Branch
|
||||
) -> list[models.Checkpoint]:
|
||||
"""Returns the Save Points that deleting `branch` would destroy.
|
||||
|
||||
The whole subtree, because deleting a branch takes everything forked from
|
||||
it, and a check that looked only at this branch would let a Save Point on a
|
||||
child be deleted without a word.
|
||||
"""
|
||||
doomed = _branch_subtree(db, adventure, branch)
|
||||
return (
|
||||
db.query(models.Checkpoint)
|
||||
.filter(
|
||||
models.Checkpoint.adventure_id == adventure.id,
|
||||
models.Checkpoint.branch_id.in_(doomed),
|
||||
)
|
||||
.order_by(models.Checkpoint.created_at, models.Checkpoint.id)
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def _protected_message(protecting: list[models.Checkpoint]) -> str:
|
||||
"""Says which Save Points stand in the way, and what to do about it.
|
||||
|
||||
Named rather than counted, because "2 Save Points" leaves the user hunting
|
||||
for which ones. A long list is truncated so the message stays readable; the
|
||||
Save Points panel shows the rest.
|
||||
"""
|
||||
names = [f"“{c.name}”" for c in protecting[:NAMED_IN_REFUSAL]]
|
||||
listed = ", ".join(names)
|
||||
extra = len(protecting) - len(names)
|
||||
if extra > 0:
|
||||
listed += f" and {extra} more"
|
||||
subject = "a Save Point" if len(protecting) == 1 else "Save Points"
|
||||
return (
|
||||
f"This branch, or a branch forked from it, is where {subject} "
|
||||
f"{listed} {'is' if len(protecting) == 1 else 'are'} saved. Delete "
|
||||
f"{'that Save Point' if len(protecting) == 1 else 'those Save Points'} "
|
||||
f"first if you no longer need "
|
||||
f"{'it' if len(protecting) == 1 else 'them'}, then delete the branch. "
|
||||
f"Deleting a Save Point does not delete any story."
|
||||
)
|
||||
|
||||
|
||||
def _branch_subtree(
|
||||
db: Session, adventure: models.Adventure, root: models.Branch
|
||||
) -> set[int]:
|
||||
|
||||
@@ -7,7 +7,7 @@ only check ownership and hand the work over.
|
||||
from fastapi import Body, Depends, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import analytics, bundle, limits, models, schemas
|
||||
from ... import bundle, head, limits, models, schemas
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
@@ -34,7 +34,6 @@ def import_adventure(
|
||||
user: models.User = CurrentUser,
|
||||
):
|
||||
version = bundle.check_format(payload)
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("adventures", db, user)
|
||||
limits.check_bundle_lists(
|
||||
story_cards=payload.get("storyCards"),
|
||||
@@ -63,8 +62,14 @@ def import_adventure(
|
||||
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
# A campaign exported while undone imports undone (M3), so the history
|
||||
# controls have to be right on the response that opens it — otherwise the
|
||||
# first thing the reader sees about a story with a retained future is a
|
||||
# greyed-out Redo.
|
||||
out = schemas.AdventureOut.model_validate(adventure)
|
||||
out.can_undo = head.can_undo(db, adventure)
|
||||
out.can_redo = head.can_redo(db, adventure)
|
||||
# This is not a funnel step. A returning player imports a bundle, so it
|
||||
# says nothing about how far a first-time visitor got. It is counted anyway,
|
||||
# because it is the clearest evidence that anyone uses the export format.
|
||||
analytics.record_event(analytics.EV_IMPORT, user)
|
||||
return adventure
|
||||
return out
|
||||
|
||||
@@ -0,0 +1,351 @@
|
||||
"""M4: Save Points — create, list, rename, delete, and restore.
|
||||
|
||||
A Save Point is a durable named pointer to a story position and nothing else.
|
||||
It stores a coordinate, never a copy of any story, and restoring one moves the
|
||||
active head to that coordinate. That is the whole design, and it is what
|
||||
`BUILD-MILESTONES.md`'s note on M4 and ADR 012 ask for: M3 made the head a
|
||||
stored `(branch, depth)` and made arriving at one a row lookup plus a state
|
||||
restore, so a Save Point needs no restore machinery of its own.
|
||||
|
||||
What is deliberately absent from this module, because a second copy of any of it
|
||||
would be the failure M4 is warned about:
|
||||
|
||||
* no head fields are assigned here — `head.move_to_node` moves the head, and
|
||||
`head.move_to` under it restores the state, exactly as Undo and Redo do;
|
||||
* nothing reconstructs state, prunes a memory, copies a turn, or deletes one;
|
||||
* nothing forks. Restore is not a decision to abandon anything, so it creates no
|
||||
branch. The first write below the restored head forks, through the same
|
||||
`fork_if_behind_head` every other write goes through, and the displaced future
|
||||
stays retained (`STORY-BRANCH-SEMANTICS.md` §20).
|
||||
|
||||
The user-facing word is "Save Point" and the internal one is `checkpoint`
|
||||
(`BROWSER-UX-SPEC.md` §23). Error strings here are read by a player, so they say
|
||||
Save Point.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy import and_, or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import head, models, schemas
|
||||
from ...context import lineage
|
||||
from ...database import get_db
|
||||
|
||||
from . import turns
|
||||
from .deps import current_adventure, router
|
||||
from .paging import current_window
|
||||
|
||||
|
||||
def _node_at(
|
||||
db: Session, adventure: models.Adventure, branch_id: int, depth: int
|
||||
) -> models.Action | None:
|
||||
"""Returns the live turn a Save Point's coordinate names, or None.
|
||||
|
||||
The lookup is by coordinate and is not scoped to any path. That is the
|
||||
point of it: a Save Point outlives the reader moving away, so the question
|
||||
it has to answer is "is this position still in this campaign's retained
|
||||
history", not "is it on the story being read now". Whether it is on the
|
||||
current path is a separate question, and `head.move_to_node` is what acts on
|
||||
the answer.
|
||||
|
||||
`live` is what makes the coordinate follow a retry. One coordinate can hold
|
||||
several attempts at a turn, and a Save Point names the turn rather than the
|
||||
attempt, so it lands on whichever take the story currently tells.
|
||||
"""
|
||||
return (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
models.Action.branch_id == branch_id,
|
||||
models.Action.depth == depth,
|
||||
models.Action.live.is_(True),
|
||||
)
|
||||
.order_by(models.Action.id)
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _Coordinate:
|
||||
"""The shape `lineage.Path` reads, without loading a story row.
|
||||
|
||||
`Path.contains` asks three things of a node: its branch, its depth, and
|
||||
whether it is live. A coordinate already known to resolve has all three, so
|
||||
the membership question can be put to the coordinate itself. That keeps the
|
||||
single implementation of "is this on the path being read" in `lineage`,
|
||||
where M3 put it, while costing no query and no prose.
|
||||
"""
|
||||
|
||||
branch_id: int
|
||||
depth: int
|
||||
live: bool = True
|
||||
|
||||
|
||||
def _live_coordinates(
|
||||
db: Session, adventure: models.Adventure, checkpoints: list[models.Checkpoint]
|
||||
) -> set[tuple[int, int]]:
|
||||
"""Returns which of these Save Points' coordinates still name a live turn.
|
||||
|
||||
One query for the whole list, selecting two integer columns.
|
||||
|
||||
This replaces a resolution per Save Point (M4 review §R B-1), which cost one
|
||||
query each and loaded whole `Action` entities — narration included — to
|
||||
answer a question that is only ever "does a row exist here". `paging.py`
|
||||
states the rule this now follows: a bulk read names the columns it needs, so
|
||||
a new column costs nothing until someone adds it to the list.
|
||||
|
||||
The clause is an OR of exact `(branch, depth)` pairs rather than
|
||||
`branch IN (…) AND depth IN (…)`, which would match the cross product and
|
||||
report a Save Point as resolved because *some other* Save Point's depth
|
||||
exists on *this* one's branch.
|
||||
"""
|
||||
coordinates = {(c.branch_id, c.depth) for c in checkpoints}
|
||||
if not coordinates:
|
||||
return set()
|
||||
rows = (
|
||||
db.query(models.Action.branch_id, models.Action.depth)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
models.Action.live.is_(True),
|
||||
or_(*[
|
||||
and_(models.Action.branch_id == branch, models.Action.depth == depth)
|
||||
for branch, depth in coordinates
|
||||
]),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
return {(branch, depth) for branch, depth in rows}
|
||||
|
||||
|
||||
def _render_all(
|
||||
db: Session, adventure: models.Adventure, checkpoints: list[models.Checkpoint]
|
||||
) -> list[schemas.CheckpointOut]:
|
||||
"""Reads Save Points out with the three facts the panel needs about them.
|
||||
|
||||
Bounded work whatever the length of the list: one query for the coordinates
|
||||
and one lineage for the campaign, both computed before the loop. Rendering
|
||||
one Save Point and rendering fifty differ in Python, not in round trips.
|
||||
"""
|
||||
live = _live_coordinates(db, adventure, checkpoints)
|
||||
# The path is a property of the campaign, not of any Save Point, so it is
|
||||
# read once. Reading it per row was the other half of the N+1.
|
||||
path = lineage.path_of(db, adventure).uncapped()
|
||||
out = []
|
||||
for checkpoint in checkpoints:
|
||||
coordinate = (checkpoint.branch_id, checkpoint.depth)
|
||||
resolved = coordinate in live
|
||||
rendered = schemas.CheckpointOut.model_validate(checkpoint)
|
||||
# The same `depth + 1` the branch list counts with, so a moment number
|
||||
# means the same thing in both places.
|
||||
rendered.turn = checkpoint.depth + 1
|
||||
rendered.resolved = resolved
|
||||
rendered.on_path = resolved and path.contains(
|
||||
_Coordinate(checkpoint.branch_id, checkpoint.depth)
|
||||
)
|
||||
out.append(rendered)
|
||||
return out
|
||||
|
||||
|
||||
def _rendered(
|
||||
db: Session, adventure: models.Adventure, checkpoint: models.Checkpoint
|
||||
) -> schemas.CheckpointOut:
|
||||
"""Reads one Save Point out, through the same path the list uses."""
|
||||
return _render_all(db, adventure, [checkpoint])[0]
|
||||
|
||||
|
||||
def _get_or_404(
|
||||
db: Session, adventure: models.Adventure, checkpoint_id: int
|
||||
) -> models.Checkpoint:
|
||||
"""Resolves a Save Point id, refusing one that belongs to another campaign.
|
||||
|
||||
The ownership check is the reason this is a function rather than a `db.get`
|
||||
at each call site. A Save Point names a position in one campaign's history,
|
||||
and a coordinate from another campaign would name a different story's turn —
|
||||
or, worse, resolve against this one by arithmetic coincidence. So the id is
|
||||
matched against this adventure, and a Save Point belonging to another is a
|
||||
404 rather than a restore of the wrong story.
|
||||
"""
|
||||
checkpoint = db.get(models.Checkpoint, checkpoint_id)
|
||||
if checkpoint is None or checkpoint.adventure_id != adventure.id:
|
||||
raise HTTPException(404, "Save Point not found")
|
||||
return checkpoint
|
||||
|
||||
|
||||
def _clean_name(raw: str) -> str:
|
||||
"""Returns the trimmed name, refusing one that is blank once trimmed."""
|
||||
name = (raw or "").strip()
|
||||
if not name:
|
||||
raise HTTPException(400, "A Save Point needs a name.")
|
||||
return name
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/checkpoints", response_model=list[schemas.CheckpointOut])
|
||||
def list_checkpoints(
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Returns the campaign's Save Points, newest first.
|
||||
|
||||
Newest first rather than in story order, because story order is not
|
||||
something this list can honestly claim. Depths are positions along a path,
|
||||
and two Save Points on lines that parted company are not comparable by depth
|
||||
at all — ordering by it would draw a sequence that no reading of the story
|
||||
passes through. When they were made is a fact about all of them.
|
||||
"""
|
||||
rows = (
|
||||
db.query(models.Checkpoint)
|
||||
.filter(models.Checkpoint.adventure_id == adventure.id)
|
||||
.order_by(models.Checkpoint.created_at.desc(), models.Checkpoint.id.desc())
|
||||
.all()
|
||||
)
|
||||
return _render_all(db, adventure, rows)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{adventure_id}/checkpoints",
|
||||
response_model=schemas.CheckpointOut,
|
||||
status_code=201,
|
||||
)
|
||||
def create_checkpoint(
|
||||
adventure_id: int,
|
||||
payload: schemas.CheckpointCreate,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Names the position the story is currently being read at.
|
||||
|
||||
The active head, not the retained tip. Creating a Save Point after two Undos
|
||||
saves the undone position, because that is where the reader is and the
|
||||
position they are looking at is the one they mean. The distinction only
|
||||
exists at all because M3 stopped Undo from deleting.
|
||||
|
||||
The node at the head is resolved before the row is written, and its own
|
||||
branch is what gets stored — which is not always the branch being read. A
|
||||
head resting in a shared prefix sits on an ancestor's node, and the
|
||||
ancestor is the branch that still names that position after the reader has
|
||||
forked away from it.
|
||||
|
||||
**Held under the campaign's turn lock** (M4 closeout, review §S C-5). "Save
|
||||
where I am" has to name one committed position, and the head is exactly what
|
||||
a turn in flight is about to move. Without the lock this endpoint could read
|
||||
`head_depth` while a turn was mid-commit and store a coordinate for a
|
||||
position the story had already left — a Save Point silently naming the wrong
|
||||
moment, which no later operation could detect. It is the same lock Undo,
|
||||
Redo and Restore take, for the same reason, and not a new mechanism.
|
||||
|
||||
Rename and Delete deliberately do **not** take it: neither reads nor moves a
|
||||
story position, so there is nothing for a turn in flight to race them over.
|
||||
"""
|
||||
name = _clean_name(payload.name)
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
node = head.node_at(db, adventure, adventure.head_depth)
|
||||
if node is None:
|
||||
raise HTTPException(400, "There is no turn here to save yet.")
|
||||
checkpoint = models.Checkpoint(
|
||||
adventure_id=adventure.id,
|
||||
name=name,
|
||||
note=payload.note or "",
|
||||
branch_id=node.branch_id,
|
||||
depth=node.depth,
|
||||
)
|
||||
db.add(checkpoint)
|
||||
db.commit()
|
||||
db.refresh(checkpoint)
|
||||
return _rendered(db, adventure, checkpoint)
|
||||
finally:
|
||||
turns._active_turns.discard(adventure_id)
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/{adventure_id}/checkpoints/{checkpoint_id}",
|
||||
response_model=schemas.CheckpointOut,
|
||||
)
|
||||
def rename_checkpoint(
|
||||
checkpoint_id: int,
|
||||
payload: schemas.CheckpointRename,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Changes a Save Point's label. Nothing else about it moves.
|
||||
|
||||
Not the coordinate, not the head, not a row of story. A Save Point that has
|
||||
been renamed restores to exactly the position it did before, which is
|
||||
`STORY-BRANCH-SEMANTICS.md` §23.
|
||||
"""
|
||||
checkpoint = _get_or_404(db, adventure, checkpoint_id)
|
||||
if payload.name is not None:
|
||||
checkpoint.name = _clean_name(payload.name)
|
||||
if payload.note is not None:
|
||||
checkpoint.note = payload.note
|
||||
db.commit()
|
||||
db.refresh(checkpoint)
|
||||
return _rendered(db, adventure, checkpoint)
|
||||
|
||||
|
||||
@router.delete("/{adventure_id}/checkpoints/{checkpoint_id}", status_code=204)
|
||||
def delete_checkpoint(
|
||||
checkpoint_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Removes the named pointer, and only the pointer.
|
||||
|
||||
The turn it named stays, its branch stays, the future past it stays, and the
|
||||
head does not move. This endpoint deletes one row of the `checkpoints`
|
||||
table. `STORY-BRANCH-SEMANTICS.md` §25.
|
||||
"""
|
||||
checkpoint = _get_or_404(db, adventure, checkpoint_id)
|
||||
db.delete(checkpoint)
|
||||
db.commit()
|
||||
return None
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{adventure_id}/checkpoints/{checkpoint_id}/restore",
|
||||
response_model=schemas.ActionPage,
|
||||
)
|
||||
def restore_checkpoint(
|
||||
adventure_id: int,
|
||||
checkpoint_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Returns the story to a Save Point, deleting nothing.
|
||||
|
||||
Four steps, and the last one is not this module's code: resolve the
|
||||
coordinate, refuse it if it no longer names a live turn, hand it to
|
||||
`head.move_to_node`, and answer with the window the head now caps. The
|
||||
transcript, the world state, the assembled context and which memories can be
|
||||
retrieved all move together, because all four already read through the one
|
||||
path object the head caps — the same reason Undo needed no memory pruning.
|
||||
|
||||
The turns past the restored position are retained, exactly as they are after
|
||||
an Undo, and ordinary Redo can still walk forward into them until the user
|
||||
writes something different. Restore does not fork; the first write below the
|
||||
head does.
|
||||
|
||||
A coordinate that no longer resolves is refused rather than approximated.
|
||||
Moving the head to the nearest surviving turn would be the one outcome worse
|
||||
than doing nothing: a Save Point that silently means somewhere else.
|
||||
"""
|
||||
checkpoint = _get_or_404(db, adventure, checkpoint_id)
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
node = _node_at(db, adventure, checkpoint.branch_id, checkpoint.depth)
|
||||
if node is None:
|
||||
raise HTTPException(
|
||||
409,
|
||||
"That Save Point's position is no longer part of this story.",
|
||||
)
|
||||
head.move_to_node(db, adventure, node)
|
||||
adventure.updated_at = models.utcnow()
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
# A window, not the whole story, for the reason Undo gives: the client
|
||||
# replaces its transcript with this, and the transcript is a window.
|
||||
return current_window(db, adventure)
|
||||
finally:
|
||||
turns._active_turns.discard(adventure_id)
|
||||
@@ -9,7 +9,9 @@ from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm.attributes import set_committed_value
|
||||
|
||||
from ... import analytics, attempts, images, limits, memorybank, models, schemas, tree, worldstate
|
||||
from ... import (
|
||||
attempts, head, images, limits, memorybank, models, schemas, tree, worldstate,
|
||||
)
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
@@ -195,20 +197,6 @@ def create_adventure(
|
||||
if scenario:
|
||||
for ref, spec in scenario_card_specs(scenario, values).items():
|
||||
db.add(models.StoryCard(adventure_id=adventure.id, source_ref=ref, **spec))
|
||||
for position, script in enumerate(scenario.scripts):
|
||||
db.add(
|
||||
models.AdventureScript(
|
||||
adventure_id=adventure.id,
|
||||
source_script_id=script.id,
|
||||
position=position,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
library_js=script.library_js,
|
||||
input_js=script.input_js,
|
||||
context_js=script.context_js,
|
||||
output_js=script.output_js,
|
||||
)
|
||||
)
|
||||
if scenario.prompt.strip():
|
||||
opening = models.Action(
|
||||
adventure_id=adventure.id,
|
||||
@@ -223,12 +211,6 @@ def create_adventure(
|
||||
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
analytics.record_event(analytics.EV_ADVENTURE, user)
|
||||
# Track which shared scenarios players pick. This is the only content this
|
||||
# module records, and it records only public scenarios. A player's own
|
||||
# scenario titles stay private.
|
||||
if scenario is not None and scenario.is_public:
|
||||
analytics.record(analytics.M_SCENARIO, scenario.title)
|
||||
return adventure
|
||||
|
||||
|
||||
@@ -259,23 +241,14 @@ def get_adventure(
|
||||
set_committed_value(adventure, "actions", actions)
|
||||
out = schemas.AdventureOut.model_validate(adventure)
|
||||
out.action_count = total
|
||||
# M3. Opening a story has to render its history controls correctly, and a
|
||||
# campaign whose head sits behind the retained tip — undone and then closed —
|
||||
# must come back with Redo available.
|
||||
out.can_undo = head.can_undo(db, adventure)
|
||||
out.can_redo = head.can_redo(db, adventure)
|
||||
return out
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/script-state")
|
||||
def get_script_state(
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Returns the scripting `state` object.
|
||||
|
||||
The object holds every variable that scripts read and write through
|
||||
`state.x`, persisted after each hook. It stays `{}` until a script sets a
|
||||
variable.
|
||||
"""
|
||||
state = adventure.script_state if isinstance(adventure.script_state, dict) else {}
|
||||
return {"state": state}
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/world-state")
|
||||
def get_world_state(
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
|
||||
@@ -7,7 +7,7 @@ returns the prompt a turn was actually generated from. Neither writes anything.
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import auth, memorybank, models
|
||||
from ... import memorybank, models
|
||||
from ...context import build_context
|
||||
from ...database import get_db
|
||||
from ..settings import get_settings
|
||||
@@ -23,14 +23,7 @@ async def dry_run_context(
|
||||
):
|
||||
"""Returns what the app would send to the AI if the player continued now."""
|
||||
settings = get_settings(db, user)
|
||||
if auth.resolve_provider_config(settings).using_demo:
|
||||
memories = (
|
||||
{"used": [], "error": "Memory bank is unavailable on the shared demo key."}
|
||||
if adventure.memory_bank_enabled
|
||||
else None
|
||||
)
|
||||
else:
|
||||
memories = await memorybank.retrieve_memories(adventure, settings, update_stats=False)
|
||||
memories = await memorybank.retrieve_memories(adventure, settings, update_stats=False)
|
||||
_, _, report = build_context(adventure, settings, memories)
|
||||
return report
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ module in the package can import them.
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy.orm import Session, undefer
|
||||
|
||||
from ... import attempts, memorybank, models, tree
|
||||
from ... import attempts, head, memorybank, models, tree
|
||||
from ...context import cursors
|
||||
from ...context import lineage
|
||||
|
||||
@@ -125,6 +125,10 @@ def stand_on(
|
||||
newest is not None
|
||||
and newest.branch_id == action.branch_id
|
||||
and newest.depth == action.depth
|
||||
# A turn the head rests on is still not a leaf while a retained future
|
||||
# descends from it. `last_action` reads the capped path and cannot see
|
||||
# that future, so switching in place here would strand it (M3).
|
||||
and not head.behind_tip(db, adventure)
|
||||
)
|
||||
if at_the_tip:
|
||||
# The story at this coordinate is about to change, so withdraw whatever
|
||||
|
||||
@@ -8,7 +8,7 @@ columns and apply the same numbering, so both live here.
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session, load_only
|
||||
|
||||
from ... import models, schemas
|
||||
from ... import head, models, schemas
|
||||
from ...context import lineage
|
||||
|
||||
|
||||
@@ -165,4 +165,6 @@ def current_window(db: Session, adventure: models.Adventure) -> schemas.ActionPa
|
||||
],
|
||||
total=total,
|
||||
has_more=has_more,
|
||||
can_undo=head.can_undo(db, adventure),
|
||||
can_redo=head.can_redo(db, adventure),
|
||||
)
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
"""The per-adventure copies of library scripts.
|
||||
|
||||
An adventure snapshots a library `Script` when it starts, so editing the library
|
||||
does not change a story in progress. These endpoints report whether a snapshot
|
||||
has fallen behind its library original, and copy the original over on request.
|
||||
"""
|
||||
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import models, schemas
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
|
||||
|
||||
# Fields that are copied from a library Script into its adventure-script
|
||||
# snapshot, and compared to decide whether a copy is out of date.
|
||||
SYNC_FIELDS = ("name", "description", "library_js", "input_js", "context_js", "output_js")
|
||||
|
||||
|
||||
def resolve_library_script(
|
||||
adv_script: models.AdventureScript, db: Session, user: models.User
|
||||
) -> models.Script | None:
|
||||
"""Returns the library Script an adventure script can re-sync from.
|
||||
|
||||
The result is the script this copy was made from. For a legacy copy with no
|
||||
link, it is one of the player's own scripts with the same name. Only the
|
||||
player's own scripts are considered, so a copy derived from a demo scenario
|
||||
has nothing to sync to.
|
||||
"""
|
||||
if adv_script.source_script_id is not None:
|
||||
script = db.get(models.Script, adv_script.source_script_id)
|
||||
if script is not None and script.user_id == user.id:
|
||||
return script
|
||||
return (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.user_id == user.id, models.Script.name == adv_script.name)
|
||||
.order_by(models.Script.updated_at.desc())
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def _mark_out_of_date(
|
||||
adv_script: models.AdventureScript, db: Session, user: models.User
|
||||
) -> models.AdventureScript:
|
||||
"""Attaches a transient `out_of_date` flag, which `AdventureScriptOut` reads.
|
||||
|
||||
The flag is `True` or `False` when a syncable library version exists, and
|
||||
`None` when none exists.
|
||||
"""
|
||||
library = resolve_library_script(adv_script, db, user)
|
||||
adv_script.out_of_date = (
|
||||
None if library is None
|
||||
else any(getattr(adv_script, f) != getattr(library, f) for f in SYNC_FIELDS)
|
||||
)
|
||||
return adv_script
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/scripts", response_model=list[schemas.AdventureScriptOut])
|
||||
def list_adventure_scripts(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
return [_mark_out_of_date(s, db, user) for s in adventure.scripts]
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{adventure_id}/scripts/{adv_script_id}/sync",
|
||||
response_model=schemas.AdventureScriptOut,
|
||||
)
|
||||
def sync_adventure_script(
|
||||
adventure_id: int,
|
||||
adv_script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Overwrites this copy's code with the latest from its library script.
|
||||
|
||||
`enabled`, `position`, and the adventure's shared `script_state` are kept.
|
||||
"""
|
||||
script = db.get(models.AdventureScript, adv_script_id)
|
||||
if script is None or script.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
library = resolve_library_script(script, db, user)
|
||||
if library is None:
|
||||
raise HTTPException(404, "No library script to sync from")
|
||||
for field in SYNC_FIELDS:
|
||||
setattr(script, field, getattr(library, field))
|
||||
# Store the link, so that a name-matched legacy copy syncs by id next
|
||||
# time.
|
||||
script.source_script_id = library.id
|
||||
db.commit()
|
||||
db.refresh(script)
|
||||
return _mark_out_of_date(script, db, user)
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/{adventure_id}/scripts/{adv_script_id}", response_model=schemas.AdventureScriptOut
|
||||
)
|
||||
def update_adventure_script(
|
||||
adventure_id: int,
|
||||
adv_script_id: int,
|
||||
payload: schemas.AdventureScriptUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
script = db.get(models.AdventureScript, adv_script_id)
|
||||
if script is None or script.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
for field, value in payload.model_dump(exclude_unset=True).items():
|
||||
setattr(script, field, value)
|
||||
db.commit()
|
||||
return script
|
||||
@@ -11,17 +11,16 @@ from fastapi import Depends, HTTPException, Request
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import attempts, limits, memorybank, models, schemas, tree
|
||||
from ... import attempts, head, limits, memorybank, models, schemas, tree
|
||||
from ...context import cursors
|
||||
from ...context import lineage
|
||||
from ...database import get_db
|
||||
from ...scripting import ScriptPipeline
|
||||
from ...sse import SSE_HEADERS
|
||||
|
||||
from . import turns
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
from .nodes import delete_turn, last_action, stand_on
|
||||
from .paging import action_window, annotate_takes, current_window
|
||||
from .nodes import last_action, stand_on
|
||||
from .paging import current_window
|
||||
|
||||
|
||||
@router.post("/{adventure_id}/retry")
|
||||
@@ -34,26 +33,44 @@ def retry_action(
|
||||
):
|
||||
"""Regenerates the last AI action and keeps the discarded attempt.
|
||||
|
||||
The attempt on screen stays as it was written. The shared script state and
|
||||
world state roll back to what the node before it left behind, and the new
|
||||
attempt is stored as a sibling at the same coordinate. No text the AI wrote
|
||||
is rewritten or deleted.
|
||||
The attempt on screen stays as it was written. The world state rolls back to
|
||||
what the node before it left behind, and the new attempt is stored as a
|
||||
sibling at the same coordinate. No text the AI wrote is rewritten or deleted.
|
||||
|
||||
M3 added the one case that cannot be a sibling. Retrying the turn the head
|
||||
rests on while a retained future still descends from it would leave that
|
||||
future hanging off a take that is no longer live — the story after it was
|
||||
written to continue the old text. So a retry from behind the tip takes a
|
||||
branch instead, exactly as `add_take` does for a turn the story has moved
|
||||
past. It is the same operation reached from a different button.
|
||||
"""
|
||||
limits.rate_limit("turn", request, user)
|
||||
turns.check_demo_cap(db, user)
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
last_ai = None
|
||||
try:
|
||||
newest = last_action(adventure, db)
|
||||
if newest is not None and newest.type == "ai":
|
||||
last_ai = newest
|
||||
# Read this before anything moves, and note it is *not*
|
||||
# `fork_if_behind_head`: this fork leaves the path just in front of
|
||||
# the turn being retried rather than at the head, so the new take
|
||||
# lands at the same depth under the same parent.
|
||||
diverging = head.behind_tip(db, adventure)
|
||||
if diverging:
|
||||
departed = lineage.branch_of(db, adventure)
|
||||
tree.branch_at(db, adventure, (newest.depth or 0) - 1)
|
||||
if departed is not None:
|
||||
head.mark_superseded(departed, (newest.depth or 0) - 1)
|
||||
else:
|
||||
# Only a sibling attempt names the node it replaces. A branched
|
||||
# take is a fresh node at the same coordinate, so `generate_turn`
|
||||
# places it through the tree rather than through `add_attempt`.
|
||||
last_ai = newest
|
||||
# Roll the state back to before this AI turn's hooks ran, so that
|
||||
# regenerating starts from a clean state rather than applying output
|
||||
# mutations on top of the attempt being replaced. If the preceding
|
||||
# node has no snapshot, which happens for a pre-SP4 row that the
|
||||
# migration could not derive one for, this call does nothing and
|
||||
# leaves the state as it is.
|
||||
attempts.roll_back_before(db, adventure, last_ai)
|
||||
attempts.roll_back_before(db, adventure, newest)
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
except BaseException:
|
||||
@@ -62,9 +79,7 @@ def retry_action(
|
||||
return StreamingResponse(
|
||||
turns.with_turn_lock(
|
||||
adventure_id,
|
||||
turns.generate_turn(
|
||||
adventure, db, ScriptPipeline(adventure, db), user, retry_of=last_ai
|
||||
),
|
||||
turns.generate_turn(adventure, db, user, retry_of=last_ai),
|
||||
),
|
||||
media_type="text/event-stream",
|
||||
headers=SSE_HEADERS,
|
||||
@@ -142,6 +157,17 @@ def select_variant(
|
||||
"Only the latest message can be switched — the story has already "
|
||||
"continued from this one.",
|
||||
)
|
||||
if head.behind_tip(db, adventure):
|
||||
# The head is behind the retained tip, so this turn reads as the newest
|
||||
# one but still has an accepted future descending from it. Switching the
|
||||
# live take in place would leave that future continuing text the story
|
||||
# no longer tells. Forking is the operation that does this safely, and
|
||||
# `/fork` is where it lives.
|
||||
raise HTTPException(
|
||||
400,
|
||||
"This turn has a later story that was undone but kept. Redo first, "
|
||||
"or use another take to start a new line from here.",
|
||||
)
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
chosen = rows[payload.index]
|
||||
@@ -254,9 +280,7 @@ def add_take(
|
||||
turn, so the new attempt is written at the same depth under the same parent,
|
||||
and the line it leaves is unchanged. No node below is copied.
|
||||
"""
|
||||
limits.rate_limit("turn", request, user)
|
||||
limits.check_row_cap("actions", db, user, adventure=adventure)
|
||||
turns.check_demo_cap(db, user)
|
||||
action = db.get(models.Action, action_id)
|
||||
if action is None or action.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Action not found")
|
||||
@@ -270,7 +294,15 @@ def add_take(
|
||||
retry_of = None
|
||||
try:
|
||||
newest = last_action(adventure, db)
|
||||
at_the_tip = newest is not None and newest.id == action.id
|
||||
# `last_action` reads the capped path, so under a moved-back head it
|
||||
# reports the node at the head as the newest one. A turn with a retained
|
||||
# future is not a leaf, whatever the capped read says, so ask the head
|
||||
# module rather than trusting the depth comparison alone (M3).
|
||||
at_the_tip = (
|
||||
newest is not None
|
||||
and newest.id == action.id
|
||||
and not head.behind_tip(db, adventure)
|
||||
)
|
||||
if at_the_tip and action.type == "ai":
|
||||
# Nothing was played after it, so its attempts are still leaves and
|
||||
# a branch would serve no purpose. This is the `retry` path.
|
||||
@@ -281,7 +313,10 @@ def add_take(
|
||||
# text that is there now. The new attempt leaves the path just
|
||||
# before the turn, so that story keeps the attempt it was written
|
||||
# for.
|
||||
departed = lineage.branch_of(db, adventure)
|
||||
tree.branch_at(db, adventure, action.depth - 1)
|
||||
if departed is not None:
|
||||
head.mark_superseded(departed, action.depth - 1)
|
||||
attempts.roll_back_before(db, adventure, action)
|
||||
adventure.updated_at = models.utcnow()
|
||||
db.commit()
|
||||
@@ -292,9 +327,7 @@ def add_take(
|
||||
if action.type == "ai":
|
||||
# There is no player action to write. The action this turn answers is
|
||||
# already on the path, borrowed from the line being left.
|
||||
stream = turns.generate_turn(
|
||||
adventure, db, ScriptPipeline(adventure, db), user, retry_of=retry_of
|
||||
)
|
||||
stream = turns.generate_turn(adventure, db, user, retry_of=retry_of)
|
||||
else:
|
||||
stream = turns.run_player_turn(
|
||||
adventure,
|
||||
@@ -318,71 +351,78 @@ def undo_turn(
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Deletes the last turn: the trailing AI action and its player action, if any.
|
||||
"""Moves the story back one turn. Deletes nothing (M3).
|
||||
|
||||
The endpoint also rolls the shared `script_state` back to before that turn
|
||||
ran, and it prunes any memory that summarized the removed actions. The turn
|
||||
lock prevents an undo while a turn is still generating.
|
||||
This endpoint used to remove the trailing AI action and the player action in
|
||||
front of it, prune the memories that covered them, and let the tip fall back
|
||||
to whatever survived. Undoing was therefore the one operation in the
|
||||
application that destroyed accepted story, and it was why there was no Redo:
|
||||
the turns to move forward into no longer existed.
|
||||
|
||||
Now it moves `adventure.head_depth`. The rows stay exactly where they are,
|
||||
still live, still on their branch, and `lineage.Path` stops every read at the
|
||||
head instead. The transcript, the assembled context, `attempts.preceding` and
|
||||
memory retrieval all narrow together, because all four already funnelled
|
||||
through the same path object.
|
||||
|
||||
The memory bank needs no pruning for the same reason. A memory carries the
|
||||
coordinate of the node its block ends on, so a memory derived from a turn
|
||||
that is now past the head falls outside the capped clause and stops being
|
||||
retrievable — and becomes eligible again on Redo, without having been deleted
|
||||
and re-embedded. That is `STORY-BRANCH-SEMANTICS.md` §33 for free.
|
||||
|
||||
The state comes back from the node the story now ends on, which recorded what
|
||||
it left behind when it played. See `head.move_to`.
|
||||
"""
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
# Only the last turn is removed, so fetch the two actions it can
|
||||
# consist of rather than the whole story.
|
||||
newest = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adventure.id,
|
||||
lineage.path_of(db, adventure).clause(models.Action),
|
||||
)
|
||||
.order_by(models.Action.depth.desc(), models.Action.id.desc())
|
||||
.limit(2)
|
||||
.all()
|
||||
)
|
||||
if not newest or newest[0].type == "start":
|
||||
target = head.undo_target(db, adventure)
|
||||
if target is None:
|
||||
raise HTTPException(400, "Nothing to undo")
|
||||
last = newest[0]
|
||||
before_that = newest[1] if len(newest) > 1 else None
|
||||
# Undo only what this branch owns. Everything before the fork is
|
||||
# borrowed from an ancestor and is part of that ancestor's story too, so
|
||||
# an undo here must never delete a turn out of another branch. The test
|
||||
# reads the row's own branch rather than the fork depth, because the
|
||||
# branch is what decides the case.
|
||||
if last.branch_id != adventure.head_branch_id:
|
||||
raise HTTPException(
|
||||
400, "Nothing to undo on this branch — the turns before it "
|
||||
"belong to the branch it was forked from.",
|
||||
)
|
||||
first_removed = last
|
||||
if (last.type == "ai" and before_that is not None
|
||||
and before_that.type in ("do", "say", "story")
|
||||
and before_that.branch_id == adventure.head_branch_id):
|
||||
first_removed = before_that
|
||||
# The state the story returns to once the turn is gone, which is what
|
||||
# the node before the earliest removed one left behind. Read it before
|
||||
# the deletes, while those rows are still in the story.
|
||||
restore_to = attempts.preceding(db, adventure, first_removed)
|
||||
delete_turn(db, adventure, last)
|
||||
if first_removed is not last:
|
||||
delete_turn(db, adventure, first_removed)
|
||||
attempts.restore_state(adventure, restore_to)
|
||||
db.flush() # Apply the deletes before anything reads the story back.
|
||||
db.expire(adventure, ["actions"])
|
||||
# The tip moves back with the deleted rows.
|
||||
tree.refresh_head(db, adventure)
|
||||
depth, _first_stepped = target
|
||||
head.move_to(db, adventure, depth)
|
||||
adventure.updated_at = models.utcnow()
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
# Return the newest window rather than the whole story. The client
|
||||
# replaces its transcript with this response, and the transcript is a
|
||||
# window. Returning everything would defeat the paging on the action a
|
||||
# player is most likely to repeat several times in a row.
|
||||
actions, total, has_more = action_window(db, adventure)
|
||||
return schemas.ActionPage(
|
||||
actions=[
|
||||
schemas.ActionOut.model_validate(a)
|
||||
for a in annotate_takes(db, adventure.id, actions)
|
||||
],
|
||||
total=total,
|
||||
has_more=has_more,
|
||||
)
|
||||
return current_window(db, adventure)
|
||||
finally:
|
||||
turns._active_turns.discard(adventure_id)
|
||||
|
||||
|
||||
@router.post("/{adventure_id}/redo", response_model=schemas.ActionPage)
|
||||
def redo_turn(
|
||||
adventure_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Moves the story forward again into the continuation Undo stepped out of.
|
||||
|
||||
Redo exists because Undo stopped deleting. It walks the head forward over one
|
||||
whole turn along the retained lineage, and restores the state that turn left
|
||||
behind.
|
||||
|
||||
It follows the lineage rather than choosing among branches, which is what
|
||||
makes it invalidate itself correctly. Writing below a moved-back head forks,
|
||||
and from the new branch the displaced future is no longer on the lineage at
|
||||
all — so there is nothing ahead to walk into and this returns 400 without any
|
||||
flag having to be set or cleared. `STORY-BRANCH-SEMANTICS.md` §8.
|
||||
|
||||
400 is also what a head already at the tip gets, which is the ordinary case
|
||||
for a story that has never been undone.
|
||||
"""
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
depth = head.redo_target(db, adventure)
|
||||
if depth is None:
|
||||
raise HTTPException(400, "Nothing to redo")
|
||||
head.move_to(db, adventure, depth)
|
||||
adventure.updated_at = models.utcnow()
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
return current_window(db, adventure)
|
||||
finally:
|
||||
turns._active_turns.discard(adventure_id)
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
Everything a test needs to intercept lives here, and other modules reach it as
|
||||
`turns.<name>` rather than importing it by value. That matters twice. The turn
|
||||
lock guards one set only while one module owns it. And a test that replaces
|
||||
`OpenAICompatibleProvider`, `generate_turn`, or `check_demo_cap` patches this
|
||||
module, which every caller reads through.
|
||||
`OpenAICompatibleProvider` or `generate_turn` patches this module, which every
|
||||
caller reads through.
|
||||
"""
|
||||
import threading
|
||||
|
||||
@@ -13,12 +13,11 @@ from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import (
|
||||
analytics, attempts, auth, limits, memorybank, models, schemas, tree, worldstate,
|
||||
attempts, head, limits, memorybank, models, schemas, tree, worldstate,
|
||||
)
|
||||
from ...context import build_context, cursors
|
||||
from ...database import get_db
|
||||
from ...providers import OpenAICompatibleProvider, PromptParts, ProviderError
|
||||
from ...scripting import ScriptPipeline
|
||||
from ...sse import SSE_HEADERS, sse, turn_error
|
||||
from ..settings import get_settings
|
||||
|
||||
@@ -115,14 +114,11 @@ def action_json(action: models.Action, db: Session | None = None) -> dict:
|
||||
async def generate_turn(
|
||||
adventure: models.Adventure,
|
||||
db: Session,
|
||||
pipeline: ScriptPipeline,
|
||||
user: models.User,
|
||||
retry_of: models.Action | None = None,
|
||||
):
|
||||
"""Streams the AI continuation as SSE, then stores the result.
|
||||
|
||||
The continuation passes through the `context` and `output` script hooks.
|
||||
|
||||
If `retry_of` is set, the result is stored as a sibling of that AI action, at
|
||||
the same turn and the same coordinate, and the discarded attempt stays where
|
||||
it was written. Before calling, the caller must roll the adventure back to
|
||||
@@ -132,15 +128,15 @@ async def generate_turn(
|
||||
"""
|
||||
saved = False
|
||||
try:
|
||||
async for event in _generate_turn(adventure, db, pipeline, user, retry_of):
|
||||
async for event in _generate_turn(adventure, db, user, retry_of):
|
||||
if event is _SAVED:
|
||||
saved = True
|
||||
continue
|
||||
yield event
|
||||
finally:
|
||||
if retry_of is not None and not saved:
|
||||
# The turn failed with a provider error, an empty reply, a script
|
||||
# stop, or a disconnected client. No sibling was written, so the
|
||||
# The turn failed with a provider error, an empty reply, or a
|
||||
# disconnected client. No sibling was written, so the
|
||||
# attempt on screen is still the live one. Restore the state it
|
||||
# produced.
|
||||
attempts.restore_state(adventure, retry_of)
|
||||
@@ -155,55 +151,27 @@ _SAVED = object()
|
||||
async def _generate_turn(
|
||||
adventure: models.Adventure,
|
||||
db: Session,
|
||||
pipeline: ScriptPipeline,
|
||||
user: models.User,
|
||||
retry_of: models.Action | None = None,
|
||||
):
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
# On a retry, the attempt being replaced is still the live node of its turn,
|
||||
# because it stays live until a replacement exists. Filter it out of the
|
||||
# context. Otherwise the model reads the attempt it is replacing as
|
||||
# established story and writes a sequel to it.
|
||||
replacing_id = retry_of.id if retry_of is not None else None
|
||||
if cfg.using_demo:
|
||||
# The server-funded key makes no embedding or summarization calls, so
|
||||
# memory retrieval is skipped. If the bank is on, return a note.
|
||||
memories = (
|
||||
{"used": [], "error": "Memory bank is unavailable on the shared demo key — add your own API key in Settings."}
|
||||
if adventure.memory_bank_enabled
|
||||
else None
|
||||
)
|
||||
else:
|
||||
memories = await memorybank.retrieve_memories(
|
||||
adventure, settings, update_stats=True, exclude_action_id=replacing_id
|
||||
)
|
||||
memories = await memorybank.retrieve_memories(
|
||||
adventure, settings, update_stats=True, exclude_action_id=replacing_id
|
||||
)
|
||||
system_text, story_text, snapshot = build_context(
|
||||
adventure, settings, memories, exclude_action_id=replacing_id
|
||||
)
|
||||
|
||||
# onModelContext: scripts read, and can rewrite, the whole assembled
|
||||
# context.
|
||||
combined = f"{system_text}\n\n{story_text}" if system_text else story_text
|
||||
modified, stop = pipeline.run("context", combined)
|
||||
if stop:
|
||||
yield sse({"type": "stopped", "script": pipeline.report()})
|
||||
return
|
||||
context_changed = modified != combined
|
||||
parts = (
|
||||
PromptParts(system="", story=modified)
|
||||
if context_changed
|
||||
else PromptParts(system=system_text, story=story_text)
|
||||
)
|
||||
snapshot["script"] = pipeline.report() | {
|
||||
"context_changed": context_changed,
|
||||
"context_before": combined if context_changed else None,
|
||||
"context_after": modified if context_changed else None,
|
||||
}
|
||||
parts = PromptParts(system=system_text, story=story_text)
|
||||
|
||||
provider = OpenAICompatibleProvider(
|
||||
cfg.endpoint_url, cfg.api_key, cfg.model, settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.endpoint_url, settings.model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
chunks: list[str] = []
|
||||
reasoning_chunks: list[str] = []
|
||||
@@ -239,13 +207,6 @@ async def _generate_turn(
|
||||
yield turn_error(detail)
|
||||
return
|
||||
|
||||
# onOutput
|
||||
text, _ = pipeline.run("output", text)
|
||||
if not text.strip():
|
||||
yield turn_error("A script's output modifier returned empty text.")
|
||||
return
|
||||
snapshot["script"] = snapshot["script"] | pipeline.report()
|
||||
|
||||
# RPG world state (Phase 12): read the AI's state delta out of the reply,
|
||||
# apply it through the engine, and strip the block from the displayed text.
|
||||
#
|
||||
@@ -304,39 +265,15 @@ async def _generate_turn(
|
||||
tree.place_action(db, adventure, ai_action)
|
||||
db.add(ai_action)
|
||||
adventure.updated_at = models.utcnow()
|
||||
if cfg.using_demo:
|
||||
# Successful demo turns count against the daily cap, which the endpoint
|
||||
# checks before the turn starts. A failed provider call above returns
|
||||
# before this line.
|
||||
auth.count_demo_turn(user)
|
||||
db.commit()
|
||||
# Count the turn here, after every path on which it could still have failed,
|
||||
# so the number means "stories advanced" rather than "requests attempted".
|
||||
# The demo tally counts those same turns as spend on the server-funded key.
|
||||
analytics.record_event(analytics.EV_TURN, user)
|
||||
if cfg.using_demo:
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_DEMO_TURN)
|
||||
db.refresh(ai_action)
|
||||
yield _SAVED
|
||||
yield sse({"type": "done", "action": action_json(ai_action, db), "script": pipeline.report()})
|
||||
yield sse({"type": "done", "action": action_json(ai_action, db)})
|
||||
# Phase 6: schedule summarization and embedding without waiting for them.
|
||||
# The task opens its own database session. It is skipped on the demo key,
|
||||
# because background AI calls are unmetered spend on the server-funded
|
||||
# key.
|
||||
if not cfg.using_demo:
|
||||
memorybank.schedule_post_turn(adventure)
|
||||
# The task opens its own database session.
|
||||
memorybank.schedule_post_turn(adventure)
|
||||
|
||||
|
||||
def check_demo_cap(db: Session, user: models.User) -> None:
|
||||
"""Checks the demo cap before a turn starts.
|
||||
|
||||
Checking first avoids storing a capped player's input and then leaving it
|
||||
without a reply.
|
||||
"""
|
||||
settings = get_settings(db, user)
|
||||
if auth.resolve_provider_config(settings).using_demo and auth.demo_turns_left(user) <= 0:
|
||||
raise HTTPException(429, auth.DEMO_CAP_MESSAGE)
|
||||
|
||||
|
||||
async def run_player_turn(
|
||||
adventure: models.Adventure,
|
||||
@@ -353,29 +290,20 @@ async def run_player_turn(
|
||||
formatted, and a plain edit puts that same text in the box and writes it back
|
||||
verbatim. Formatting it a second time produces `> You > You ...`.
|
||||
"""
|
||||
pipeline = ScriptPipeline(adventure, db)
|
||||
|
||||
# An empty do, say, or story action behaves as a continue.
|
||||
if payload.type != "continue" and payload.text.strip():
|
||||
# onInput reads the formatted text, as in AI Dungeon: "> You ...".
|
||||
formatted = (
|
||||
payload.text.strip() if preformatted
|
||||
else format_player_input(payload.type, payload.text)
|
||||
)
|
||||
modified, stop = pipeline.run("input", formatted)
|
||||
if not modified.strip():
|
||||
yield turn_error("A script's input modifier returned empty text.",
|
||||
script=pipeline.report())
|
||||
return
|
||||
player_action = models.Action(
|
||||
adventure_id=adventure.id,
|
||||
depth=next_depth(adventure),
|
||||
type=payload.type,
|
||||
text=modified,
|
||||
text=formatted,
|
||||
)
|
||||
# The state after the input hook has run. The node leaves this state
|
||||
# behind. The AI turn after it starts here, and a retry of that turn
|
||||
# rolls back to here.
|
||||
# The state this node leaves behind. The AI turn after it starts here,
|
||||
# and a retry of that turn rolls back to here.
|
||||
attempts.snapshot_outcome(adventure, player_action)
|
||||
tree.place_action(db, adventure, player_action)
|
||||
db.add(player_action)
|
||||
@@ -387,12 +315,8 @@ async def run_player_turn(
|
||||
# that was just saved.
|
||||
db.expire(adventure, ["actions"])
|
||||
yield sse({"type": "player", "action": action_json(player_action, db)})
|
||||
if stop:
|
||||
# If onInput returns `{ stop: true }`, skip the AI call.
|
||||
yield sse({"type": "stopped", "script": pipeline.report()})
|
||||
return
|
||||
|
||||
async for event in generate_turn(adventure, db, pipeline, user):
|
||||
async for event in generate_turn(adventure, db, user):
|
||||
yield event
|
||||
|
||||
|
||||
@@ -405,12 +329,18 @@ def create_action(
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
limits.rate_limit("turn", request, user)
|
||||
limits.check_row_cap("actions", db, user, adventure=adventure)
|
||||
check_demo_cap(db, user)
|
||||
acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
_move_to_after(db, adventure, payload.after_id)
|
||||
# The first write below a moved-back head is where a divergence happens
|
||||
# (M3). Undo alone does not fork — the user may be reading, or about to
|
||||
# Redo — so this is the moment the story states which continuation it
|
||||
# means. The displaced future keeps its rows on the branch being left.
|
||||
# A head already at the tip, which is every ordinary turn, forks nothing.
|
||||
if head.fork_if_behind_head(db, adventure):
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
except BaseException:
|
||||
_active_turns.discard(adventure_id)
|
||||
raise
|
||||
|
||||
@@ -1,140 +0,0 @@
|
||||
"""Visit analytics: one endpoint the browser writes to, one the owner reads.
|
||||
|
||||
The split matters. `/collect` is public and accepts one fact, which page was
|
||||
viewed, because anything a stranger can POST is a number a stranger can invent.
|
||||
Everything the dashboard relies on, meaning turns, adventures, sign-ups, demo
|
||||
spend, and errors, is recorded on the server by the code that performs it, so
|
||||
those counts are as trustworthy as the app itself.
|
||||
|
||||
The two reading endpoints are owner-only and 404 for everyone else, the same
|
||||
way the AI Chat router does: a feature nobody else can use is better off not
|
||||
appearing to exist. `/summary` serves the anonymous counters (analytics.py,
|
||||
which stores nothing that points at a person) and `/access` serves the access
|
||||
log (accesslog.py, which identifies people on purpose).
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import accesslog, analytics, auth, limits, models
|
||||
from ..database import get_db
|
||||
|
||||
router = APIRouter(prefix="/api/analytics", tags=["analytics"])
|
||||
|
||||
|
||||
def owner(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
) -> models.User:
|
||||
"""Gates the reading half. It returns 404 rather than 403. See the module
|
||||
docstring."""
|
||||
if not auth.is_owner(user):
|
||||
raise HTTPException(404, "Not found")
|
||||
return user
|
||||
|
||||
|
||||
Owner = Depends(owner)
|
||||
|
||||
|
||||
class Pageview(BaseModel):
|
||||
"""What the SPA reports on a page load or a route change.
|
||||
|
||||
`first` marks a real page load rather than a client-side navigation. The
|
||||
facts that describe a visit rather than a view, which are where it came from,
|
||||
on what kind of device, and from which country, are recorded only on a page
|
||||
load, so a visitor who clicks through five pages is still one referral.
|
||||
"""
|
||||
|
||||
path: str = Field("", max_length=300)
|
||||
referrer: str = Field("", max_length=500)
|
||||
first: bool = False
|
||||
|
||||
|
||||
@router.post("/collect", status_code=204)
|
||||
def collect(
|
||||
payload: Pageview,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
) -> Response:
|
||||
"""Record one pageview. Always 204, even when nothing was counted: the
|
||||
browser has no business knowing whether it was."""
|
||||
limits.rate_limit("analytics", request)
|
||||
# Resolved by hand rather than through get_current_user: a pageview that
|
||||
# arrives before /auth/me has minted a session should still be counted as a
|
||||
# view, not turned into a 401 the SPA has to handle.
|
||||
user = (
|
||||
auth.resolve_session_user(request, db)
|
||||
if auth.MULTI_USER
|
||||
else auth.local_user(db)
|
||||
)
|
||||
# The operator's own clicks are not traffic. This applies only in
|
||||
# multi-user mode. Locally every user is the owner, and excluding them would
|
||||
# leave the dashboard empty on the machine the app is developed on.
|
||||
if auth.MULTI_USER and user is not None and auth.is_owner(user):
|
||||
return Response(status_code=204)
|
||||
|
||||
analytics.record(analytics.M_PAGE, analytics.normalize_route(payload.path))
|
||||
analytics.record_visit(user)
|
||||
if payload.first:
|
||||
referrer = analytics.normalize_referrer(
|
||||
payload.referrer, request.url.hostname or ""
|
||||
)
|
||||
if referrer: # "" means same-origin, which is not a referral
|
||||
analytics.record(analytics.M_REFERRER, referrer)
|
||||
analytics.record(
|
||||
analytics.M_DEVICE,
|
||||
analytics.device_of(request.headers.get("user-agent", "")),
|
||||
)
|
||||
analytics.record(analytics.M_COUNTRY, analytics.country_of(request.headers))
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.get("/summary")
|
||||
def summary(
|
||||
days: int = Query(30, ge=1, le=365),
|
||||
db: Session = Depends(get_db),
|
||||
_user: models.User = Owner,
|
||||
) -> dict:
|
||||
"""Returns the whole dashboard in one aggregate response.
|
||||
|
||||
The response is a few kilobytes however much traffic is behind it.
|
||||
"""
|
||||
return analytics.summary(db, days)
|
||||
|
||||
|
||||
@router.get("/access")
|
||||
def access_log(
|
||||
limit: int = Query(50, ge=1, le=200),
|
||||
before_id: int | None = Query(None),
|
||||
kind: str | None = Query(None),
|
||||
q: str | None = Query(None, max_length=120),
|
||||
db: Session = Depends(get_db),
|
||||
_user: models.User = Owner,
|
||||
) -> dict:
|
||||
"""A page of the access log, newest first.
|
||||
|
||||
Unlike `/summary`, this returns rows about people, which is what it is for.
|
||||
It is therefore behind the same owner gate, it is paged rather than returned
|
||||
in full, and the people it describes have no endpoint that reaches it.
|
||||
"""
|
||||
page = accesslog.recent(
|
||||
db, limit=limit, before_id=before_id, kind=kind, query=q
|
||||
)
|
||||
return {
|
||||
"events": [
|
||||
{
|
||||
"id": event.id,
|
||||
"at": event.at.isoformat(),
|
||||
"kind": event.kind,
|
||||
"who": event.who,
|
||||
"is_guest": event.is_guest,
|
||||
"ip": event.ip,
|
||||
"country": event.country,
|
||||
"device": event.device,
|
||||
"user_agent": event.user_agent,
|
||||
}
|
||||
for event in page["events"]
|
||||
],
|
||||
"has_more": page["has_more"],
|
||||
}
|
||||
@@ -1,158 +0,0 @@
|
||||
import re
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import (accesslog, analytics, auth, cleanup, limits, models, schemas,
|
||||
security, starter)
|
||||
from ..database import get_db
|
||||
from .settings import get_settings
|
||||
|
||||
router = APIRouter(prefix="/api/auth", tags=["auth"])
|
||||
|
||||
EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
|
||||
|
||||
|
||||
def _set_session_cookie(response: Response, user_id: int) -> None:
|
||||
response.set_cookie(
|
||||
auth.SESSION_COOKIE,
|
||||
security.sign_session(user_id),
|
||||
max_age=auth.COOKIE_MAX_AGE,
|
||||
httponly=True,
|
||||
samesite="lax",
|
||||
secure=auth.COOKIE_SECURE,
|
||||
path="/",
|
||||
)
|
||||
|
||||
|
||||
def me_payload(user: models.User, db: Session) -> dict:
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
return {
|
||||
"multi_user": auth.MULTI_USER,
|
||||
"id": user.id,
|
||||
"email": user.email,
|
||||
"is_guest": user.is_guest,
|
||||
# Trusted testers: unmetered demo turns, plus the AI Chat scratchpad.
|
||||
"power_user": auth.is_power_user(user),
|
||||
# Separate allowlist: shows the visit-analytics page and its nav link.
|
||||
"analytics": auth.is_owner(user),
|
||||
# How long an idle guest is kept before cleanup deletes it (None when
|
||||
# the policy is off). Served rather than hardcoded in the UI so the
|
||||
# number a guest is shown is the number actually enforced.
|
||||
"guest_retention_days": cleanup.RETENTION_DAYS if cleanup.enabled() else None,
|
||||
"demo": {
|
||||
"enabled": auth.demo_enabled(),
|
||||
"using_demo": cfg.using_demo,
|
||||
"model": cfg.model if cfg.using_demo else None,
|
||||
"turns_per_day": auth.DEMO_TURNS_PER_DAY,
|
||||
"turns_left": auth.demo_turns_left(user) if auth.demo_enabled() else None,
|
||||
"models": auth.DEMO_MODELS if auth.demo_enabled() else [],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
def me(request: Request, response: Response, db: Session = Depends(get_db)):
|
||||
"""Returns the current user.
|
||||
|
||||
In multi-user mode this also establishes the session. If the cookie is
|
||||
missing or invalid, the endpoint creates a guest user and sets a cookie. The
|
||||
frontend calls it on load and after any 401.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
user = auth.local_user(db)
|
||||
else:
|
||||
user = auth.resolve_session_user(request, db)
|
||||
if user is None:
|
||||
# Each new guest is a database row, so cap how fast one IP can
|
||||
# create them.
|
||||
limits.rate_limit("guest", request)
|
||||
user = models.User(is_guest=True)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
# The guest is committed first, so a failure while copying the
|
||||
# starter adventure still leaves them with an account.
|
||||
starter.give(db, user)
|
||||
db.commit()
|
||||
_set_session_cookie(response, user.id)
|
||||
# This endpoint is the SPA's bootstrap call, so it is where a session first
|
||||
# shows itself; accesslog thins the rows down to one per day per address.
|
||||
accesslog.note_session(db, user, request)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/register")
|
||||
def register(
|
||||
payload: schemas.AuthCredentials,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Upgrades the current guest in place.
|
||||
|
||||
The `user_id` does not change, so every adventure, scenario, script, and
|
||||
setting they created as a guest is kept.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
limits.rate_limit("auth", request)
|
||||
email = payload.email.strip().lower()
|
||||
if not EMAIL_RE.match(email):
|
||||
raise HTTPException(422, "Enter a valid email address.")
|
||||
if len(payload.password) < 8:
|
||||
raise HTTPException(422, "Password must be at least 8 characters.")
|
||||
if not user.is_guest:
|
||||
raise HTTPException(400, "This session is already registered.")
|
||||
if db.query(models.User).filter(models.User.email == email).first():
|
||||
raise HTTPException(409, "An account with this email already exists — log in instead.")
|
||||
user.email = email
|
||||
user.password_hash = security.hash_password(payload.password)
|
||||
user.is_guest = False
|
||||
db.commit()
|
||||
analytics.record_event(analytics.EV_SIGNUP, user)
|
||||
accesslog.record(db, accesslog.REGISTER, request, user=user)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/login")
|
||||
def login(
|
||||
payload: schemas.AuthCredentials,
|
||||
request: Request,
|
||||
response: Response,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Point this browser's session at an existing account. Any current guest
|
||||
session is simply abandoned (its data stays under the guest user)."""
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
limits.rate_limit("auth", request)
|
||||
email = payload.email.strip().lower()
|
||||
# Per-account throttle: stops distributed guessing against one email even
|
||||
# when the per-IP limit above is diluted across many source addresses.
|
||||
limits.check_login_allowed(email)
|
||||
user = db.query(models.User).filter(models.User.email == email).first()
|
||||
if (
|
||||
user is None
|
||||
or not user.password_hash
|
||||
or not security.verify_password(payload.password, user.password_hash)
|
||||
):
|
||||
limits.note_login_failure(email)
|
||||
# Logged with the address that was tried, not the account that owns it:
|
||||
# a guessing run against an address that has no account is exactly the
|
||||
# thing worth being able to see.
|
||||
accesslog.record(db, accesslog.LOGIN_FAILED, request, who=email)
|
||||
raise HTTPException(401, "Incorrect email or password.")
|
||||
limits.note_login_success(email)
|
||||
_set_session_cookie(response, user.id)
|
||||
analytics.record_event(analytics.EV_LOGIN, user)
|
||||
accesslog.record(db, accesslog.LOGIN, request, user=user)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/logout")
|
||||
def logout(response: Response):
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
response.delete_cookie(auth.SESSION_COOKIE, path="/")
|
||||
return {"ok": True}
|
||||
@@ -1,22 +1,23 @@
|
||||
"""AI Chat: a plain scratchpad for talking to a model directly.
|
||||
"""AI Chat: a plain scratchpad for talking to the configured model directly.
|
||||
|
||||
Power users reach it, which means the `AIDND_POWER_USERS` email allowlist. It is
|
||||
deliberately thin. It adds no story context, no scripts, and no world state, and
|
||||
it persists nothing. The conversation lives in the browser and is posted in full
|
||||
on each turn. It exists for testing models, prompts, and endpoints without
|
||||
starting an adventure.
|
||||
Deliberately thin. It adds no story context and no world state, and it persists
|
||||
nothing. The conversation lives in the browser and is posted in full on each
|
||||
turn. It exists for checking a model, a prompt, or an endpoint without starting
|
||||
an adventure — which is exactly the kind of thing a local single-user install
|
||||
wants a page for.
|
||||
|
||||
Model choice is free when the user brought their own API key. On the shared demo
|
||||
key the model stays pinned to the `AIDND_DEMO_MODELS` allowlist, exactly as it is
|
||||
for turns. The server funds that key, so this page must not let it reach paid
|
||||
models.
|
||||
Upstream gated this behind a "power user" email allowlist and pinned the model
|
||||
when a shared demo key was in play. M2 removed both: there is one local user,
|
||||
who owns the endpoint, and there is no server-funded key to protect. The model
|
||||
this page talks to is the one in Settings, or one the user names per request —
|
||||
either way it is their own Ollama.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import auth, limits, models, schemas
|
||||
from .. import auth, models, schemas
|
||||
from ..database import get_db
|
||||
from ..providers import OpenAICompatibleProvider, ProviderError
|
||||
from ..sse import SSE_HEADERS, sse
|
||||
@@ -25,80 +26,41 @@ from .settings import get_settings, list_endpoint_models
|
||||
router = APIRouter(prefix="/api/chat", tags=["chat"])
|
||||
|
||||
|
||||
def power_user(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
) -> models.User:
|
||||
"""Gate for the whole router. 404 rather than 403 so the feature simply
|
||||
doesn't appear to exist for everyone else."""
|
||||
if not auth.is_power_user(user):
|
||||
raise HTTPException(404, "Not found")
|
||||
return user
|
||||
|
||||
|
||||
PowerUser = Depends(power_user)
|
||||
|
||||
|
||||
def _resolve_model(
|
||||
settings: models.Settings, requested: str | None
|
||||
) -> tuple[auth.ProviderConfig, str | None]:
|
||||
"""Returns the provider config for this chat, plus a note when the requested
|
||||
model was not used.
|
||||
|
||||
The pinning rule lives in `resolve_provider_config`. This function only
|
||||
reports the substitution that call made, so one place decides what the demo
|
||||
key may talk to.
|
||||
"""
|
||||
cfg = auth.resolve_provider_config(settings, model_override=requested)
|
||||
wanted = (requested or "").strip()
|
||||
if wanted and wanted != cfg.model:
|
||||
return cfg, (
|
||||
f"'{wanted}' isn't available on the shared demo key — using "
|
||||
f"{cfg.model}. Add your own API key in Settings to use any model."
|
||||
)
|
||||
return cfg, None
|
||||
|
||||
|
||||
@router.get("/config")
|
||||
async def chat_config(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Returns what this page can talk to.
|
||||
|
||||
The response holds the resolved endpoint and model, whether model choice is
|
||||
pinned to the demo allowlist, and the endpoint's model listing. The listing
|
||||
is best effort, and an unreachable endpoint returns an empty list.
|
||||
The model listing is best effort: an unreachable endpoint returns an empty
|
||||
list and the reason, rather than failing the page.
|
||||
"""
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
listing = await list_endpoint_models(cfg)
|
||||
listing = await list_endpoint_models(settings.endpoint_url)
|
||||
return {
|
||||
"endpoint_url": cfg.endpoint_url,
|
||||
"model": cfg.model,
|
||||
"using_demo": cfg.using_demo,
|
||||
"endpoint_url": settings.endpoint_url,
|
||||
"model": settings.model,
|
||||
"api_mode": settings.api_mode,
|
||||
"temperature": settings.temperature,
|
||||
"max_tokens": settings.max_output_tokens,
|
||||
# On the demo key the whitelist IS the list of choices; otherwise it's
|
||||
# whatever the endpoint advertises (suggestions, not a restriction).
|
||||
"models": auth.DEMO_MODELS if cfg.using_demo else listing.get("models", []),
|
||||
# Suggestions from the endpoint, not a restriction.
|
||||
"models": listing.get("models", []),
|
||||
"models_error": None if listing.get("ok") else listing.get("detail"),
|
||||
}
|
||||
|
||||
|
||||
async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload: schemas.ChatRequest,
|
||||
note: str | None, db: Session, user: models.User):
|
||||
async def run_chat(
|
||||
settings: models.Settings, model: str, payload: schemas.ChatRequest
|
||||
):
|
||||
"""Streams the reply as SSE, using the turn stream's event shape.
|
||||
|
||||
The generator emits `reasoning` and `chunk` events while generating and then
|
||||
a `done` event, so the frontend reuses the same code.
|
||||
"""
|
||||
if note:
|
||||
yield sse({"type": "note", "detail": note})
|
||||
provider = OpenAICompatibleProvider(
|
||||
cfg.endpoint_url, cfg.api_key, cfg.model, settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.endpoint_url, model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
messages = [m.model_dump() for m in payload.messages]
|
||||
chunks: list[str] = []
|
||||
@@ -106,7 +68,11 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
try:
|
||||
async for kind, chunk in provider.chat(
|
||||
messages,
|
||||
temperature=payload.temperature if payload.temperature is not None else settings.temperature,
|
||||
temperature=(
|
||||
payload.temperature
|
||||
if payload.temperature is not None
|
||||
else settings.temperature
|
||||
),
|
||||
max_tokens=payload.max_tokens or settings.max_output_tokens,
|
||||
):
|
||||
if kind == "reasoning":
|
||||
@@ -123,33 +89,26 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
if not text:
|
||||
detail = (
|
||||
"The model used its entire token budget on reasoning and returned no "
|
||||
"reply — raise max tokens, cap the reasoning budget in Settings, or "
|
||||
"use a non-reasoning model."
|
||||
"reply — raise max tokens or use a non-reasoning model."
|
||||
if reasoning_chunks
|
||||
else "The AI returned an empty response."
|
||||
)
|
||||
yield sse({"type": "error", "detail": detail})
|
||||
return
|
||||
|
||||
if cfg.using_demo:
|
||||
# Unmetered for power users (count_demo_turn is a no-op for them), but
|
||||
# keep the call so the accounting stays right if the gate ever widens.
|
||||
auth.count_demo_turn(user)
|
||||
db.commit()
|
||||
yield sse({
|
||||
"type": "done",
|
||||
"text": text,
|
||||
"reasoning": "".join(reasoning_chunks).strip() or None,
|
||||
"model": cfg.model,
|
||||
"model": model,
|
||||
})
|
||||
|
||||
|
||||
@router.post("/stream")
|
||||
def chat_stream(
|
||||
payload: schemas.ChatRequest,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
total = sum(len(m.content) for m in payload.messages)
|
||||
if total > schemas.CHAT_TOTAL_MAX:
|
||||
@@ -157,13 +116,12 @@ def chat_stream(
|
||||
413, f"This conversation is too long to send ({total:,} characters) — "
|
||||
"clear it or start a new one."
|
||||
)
|
||||
limits.rate_limit("chat", request, user)
|
||||
settings = get_settings(db, user)
|
||||
cfg, note = _resolve_model(settings, payload.model)
|
||||
if not cfg.model:
|
||||
model = (payload.model or "").strip() or settings.model
|
||||
if not model:
|
||||
raise HTTPException(400, "No model configured — set one in Settings or pick one here.")
|
||||
return StreamingResponse(
|
||||
run_chat(cfg, settings, payload, note, db, user),
|
||||
run_chat(settings, model, payload),
|
||||
media_type="text/event-stream",
|
||||
headers=SSE_HEADERS,
|
||||
)
|
||||
|
||||
@@ -1,19 +1,16 @@
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi import APIRouter
|
||||
|
||||
from .. import auth, debuglog
|
||||
from .. import debuglog
|
||||
|
||||
router = APIRouter(prefix="/api/debug", tags=["debug"])
|
||||
|
||||
|
||||
@router.get("/requests")
|
||||
def recent_requests():
|
||||
"""Most-recent-first log of provider requests/responses (no API keys).
|
||||
"""Most-recent-first log of provider requests and responses.
|
||||
|
||||
The log is a single process-wide ring buffer with no per-user attribution,
|
||||
so in multi-user mode, which is how a hosted deployment runs, it would expose
|
||||
other players' prompts. It is disabled there and available on a local
|
||||
install.
|
||||
A single process-wide ring buffer. It holds the prompts this install sent
|
||||
to its own Ollama, which is exactly what the person running it needs to
|
||||
diagnose a turn, and there is nobody else it could expose them to.
|
||||
"""
|
||||
if auth.MULTI_USER:
|
||||
raise HTTPException(403, "The debug log is only available on local installs.")
|
||||
return debuglog.recent()
|
||||
|
||||
@@ -3,7 +3,7 @@ from fastapi.responses import Response
|
||||
from sqlalchemy import or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import analytics, auth, images, limits, models, schemas
|
||||
from .. import auth, images, limits, models, schemas
|
||||
from ..database import get_db
|
||||
|
||||
router = APIRouter(prefix="/api/scenarios", tags=["scenarios"])
|
||||
@@ -54,12 +54,6 @@ def get_scenario(
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
scenario = get_scenario_or_404(scenario_id, db, user)
|
||||
# A funnel step, recorded for shared scenarios only. Opening one is the
|
||||
# first sign that a visitor is interested, and someone editing their own
|
||||
# scenario is already past this point. Their titles are theirs rather than a
|
||||
# statistic.
|
||||
if scenario.is_public:
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, user)
|
||||
return scenario
|
||||
|
||||
|
||||
@@ -96,18 +90,8 @@ def update_scenario(
|
||||
):
|
||||
scenario = get_scenario_or_404(scenario_id, db, user, edit=True)
|
||||
data = payload.model_dump(exclude_unset=True)
|
||||
script_ids = data.pop("script_ids", None)
|
||||
for field, value in data.items():
|
||||
setattr(scenario, field, value)
|
||||
if script_ids is not None:
|
||||
scripts = (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.id.in_(script_ids), models.Script.user_id == user.id)
|
||||
.all()
|
||||
)
|
||||
if len(scripts) != len(set(script_ids)):
|
||||
raise HTTPException(404, "One or more scripts not found")
|
||||
scenario.scripts = sorted(scripts, key=lambda s: script_ids.index(s.id))
|
||||
db.commit()
|
||||
return scenario
|
||||
|
||||
@@ -148,13 +132,6 @@ def export_scenario(
|
||||
{"type": c.type, "name": c.name, "keys": c.keys, "entry": c.entry, "notes": c.notes}
|
||||
for c in s.story_cards
|
||||
],
|
||||
"scripts": [
|
||||
{
|
||||
"name": sc.name, "description": sc.description, "library": sc.library_js,
|
||||
"input": sc.input_js, "context": sc.context_js, "output": sc.output_js,
|
||||
}
|
||||
for sc in s.scripts
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@@ -185,7 +162,6 @@ def import_scenario(
|
||||
):
|
||||
"""Accepts our export format and AI Dungeon scenario exports best-effort;
|
||||
reports any keys it didn't understand."""
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("scenarios", db, user)
|
||||
fields: dict = {}
|
||||
unmapped: list[str] = []
|
||||
@@ -250,21 +226,6 @@ def import_scenario(
|
||||
)
|
||||
)
|
||||
|
||||
for item in bundle.get("scripts") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
script = models.Script(
|
||||
user_id=user.id,
|
||||
name=str(item.get("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=str(item.get("description") or ""),
|
||||
library_js=str(item.get("library") or item.get("sharedLibrary") or ""),
|
||||
input_js=str(item.get("input") or item.get("onInput") or ""),
|
||||
context_js=str(item.get("context") or item.get("onModelContext") or ""),
|
||||
output_js=str(item.get("output") or item.get("onOutput") or ""),
|
||||
)
|
||||
db.add(script)
|
||||
db.flush()
|
||||
scenario.scripts.append(script)
|
||||
|
||||
db.commit()
|
||||
out = schemas.ScenarioOut.model_validate(scenario).model_dump(mode="json")
|
||||
|
||||
@@ -1,159 +0,0 @@
|
||||
from fastapi import APIRouter, Body, Depends, HTTPException, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import auth, limits, models, schemas
|
||||
from ..database import get_db
|
||||
from ..scripting import run_hook
|
||||
|
||||
router = APIRouter(prefix="/api/scripts", tags=["scripts"])
|
||||
|
||||
HOOK_FIELDS = {"input": "input_js", "context": "context_js", "output": "output_js"}
|
||||
|
||||
|
||||
def get_script_or_404(script_id: int, db: Session, user: models.User) -> models.Script:
|
||||
script = db.get(models.Script, script_id)
|
||||
if script is None or script.user_id != user.id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
return script
|
||||
|
||||
|
||||
@router.get("", response_model=list[schemas.ScriptOut])
|
||||
def list_scripts(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
return (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.user_id == user.id)
|
||||
.order_by(models.Script.updated_at.desc())
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
@router.post("", response_model=schemas.ScriptOut, status_code=201)
|
||||
def create_script(
|
||||
payload: schemas.ScriptCreate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
limits.check_row_cap("scripts", db, user)
|
||||
script = models.Script(**payload.model_dump(), user_id=user.id)
|
||||
db.add(script)
|
||||
db.commit()
|
||||
return script
|
||||
|
||||
|
||||
@router.get("/{script_id}", response_model=schemas.ScriptOut)
|
||||
def get_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
return get_script_or_404(script_id, db, user)
|
||||
|
||||
|
||||
@router.patch("/{script_id}", response_model=schemas.ScriptOut)
|
||||
def update_script(
|
||||
script_id: int,
|
||||
payload: schemas.ScriptUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
for field, value in payload.model_dump(exclude_unset=True).items():
|
||||
setattr(script, field, value)
|
||||
db.commit()
|
||||
return script
|
||||
|
||||
|
||||
@router.delete("/{script_id}", status_code=204)
|
||||
def delete_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
db.delete(get_script_or_404(script_id, db, user))
|
||||
db.commit()
|
||||
|
||||
|
||||
@router.post("/{script_id}/test")
|
||||
def test_script(
|
||||
script_id: int,
|
||||
payload: schemas.ScriptTestRequest,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Runs one hook against sample text, making no AI call and storing nothing."""
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
limits.rate_limit("script-test", request, user)
|
||||
result = run_hook(
|
||||
script.library_js,
|
||||
getattr(script, HOOK_FIELDS[payload.hook]),
|
||||
payload.text,
|
||||
payload.state,
|
||||
history=[],
|
||||
story_cards=[],
|
||||
info={"actionCount": 0, "characterNames": [], "memoryLength": 0, "maxChars": 0},
|
||||
)
|
||||
return {
|
||||
"text": result.text,
|
||||
"stop": result.stop,
|
||||
"state": result.state,
|
||||
"storyCards": result.story_cards,
|
||||
"logs": result.logs,
|
||||
"error": result.error,
|
||||
}
|
||||
|
||||
|
||||
# ---------- Import / Export ----------
|
||||
|
||||
@router.get("/{script_id}/export")
|
||||
def export_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""JSON bundle matching how AI Dungeon scripts circulate."""
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
return {
|
||||
"name": script.name,
|
||||
"description": script.description,
|
||||
"library": script.library_js,
|
||||
"input": script.input_js,
|
||||
"context": script.context_js,
|
||||
"output": script.output_js,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/import", response_model=schemas.ScriptOut, status_code=201)
|
||||
def import_script(
|
||||
request: Request,
|
||||
bundle: dict = Body(...),
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Accepts our export bundle; tolerates *_js key names too."""
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("scripts", db, user)
|
||||
def pick(*keys: str) -> str:
|
||||
for key in keys:
|
||||
value = bundle.get(key)
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
return ""
|
||||
|
||||
script = models.Script(
|
||||
user_id=user.id,
|
||||
# A raw-dict import bypasses the schemas, so truncate to the VARCHAR
|
||||
# width.
|
||||
name=(pick("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=pick("description"),
|
||||
library_js=pick("library", "library_js", "sharedLibrary"),
|
||||
input_js=pick("input", "input_js", "onInput"),
|
||||
context_js=pick("context", "context_js", "onModelContext"),
|
||||
output_js=pick("output", "output_js", "onOutput"),
|
||||
)
|
||||
db.add(script)
|
||||
db.commit()
|
||||
return script
|
||||
@@ -1,20 +1,33 @@
|
||||
"""The model settings, and the connection test that tells you why they don't work.
|
||||
|
||||
There is one settings row, belonging to the one local user. It describes an
|
||||
Ollama: where it is, which model to narrate with, which to embed with, and how
|
||||
long to wait for it.
|
||||
|
||||
Upstream let this row name any OpenAI-compatible endpoint and carry an
|
||||
encrypted API key for it. M2 narrowed both: `endpoints.py` decides which
|
||||
addresses may be named, and there is no key field, because Ollama does not use
|
||||
one and this build has no cloud provider to carry a key for.
|
||||
"""
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from .. import auth, limits, models, netguard, schemas, security, tlstrust
|
||||
from .. import auth, endpoints, models, schemas, tlstrust
|
||||
from ..database import get_db
|
||||
from ..providers.openai_compatible import CONNECT_TIMEOUT
|
||||
|
||||
router = APIRouter(prefix="/api/settings", tags=["settings"])
|
||||
|
||||
#: The connection test is a listing, not a generation, so it never waits on a
|
||||
#: model load and does not need the turn engine's patience.
|
||||
TEST_TIMEOUT = 15.0
|
||||
|
||||
|
||||
def get_settings(db: Session, user: models.User) -> models.Settings:
|
||||
"""Returns the user's settings row, creating it on first access.
|
||||
|
||||
Phase 8 made settings per user rather than global. They cover the endpoint,
|
||||
the key, the models, and the memory configuration.
|
||||
"""
|
||||
"""Returns the settings row, creating it on first access."""
|
||||
settings = (
|
||||
db.query(models.Settings).filter(models.Settings.user_id == user.id).first()
|
||||
)
|
||||
@@ -34,16 +47,24 @@ def read_settings(
|
||||
|
||||
|
||||
@router.put("", response_model=schemas.SettingsOut)
|
||||
def update_settings(
|
||||
async def update_settings(
|
||||
payload: schemas.SettingsUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
settings = get_settings(db, user)
|
||||
fields = payload.model_dump(exclude_unset=True)
|
||||
# Write-only API key: absent = unchanged, "" = cleared, else encrypted.
|
||||
if "api_key" in fields:
|
||||
fields["api_key"] = security.encrypt_secret(fields["api_key"].strip())
|
||||
|
||||
if "endpoint_url" in fields:
|
||||
# Refused here so the user finds out while they are looking at the
|
||||
# field, rather than on their next turn. The provider re-checks before
|
||||
# every request regardless; this is the friendly half of the same rule.
|
||||
reason = await run_in_threadpool(
|
||||
endpoints.rejection_reason, fields["endpoint_url"]
|
||||
)
|
||||
if reason is not None:
|
||||
raise HTTPException(400, f"That endpoint can't be used — {reason}.")
|
||||
|
||||
embedding_model_changed = (
|
||||
"embedding_model" in fields
|
||||
and fields["embedding_model"] != settings.embedding_model
|
||||
@@ -53,8 +74,6 @@ def update_settings(
|
||||
if embedding_model_changed:
|
||||
# Vectors from the old model have a different dimensionality/space;
|
||||
# clear them so the post-turn task re-embeds with the new model.
|
||||
# This covers only this user's adventures, because settings are per
|
||||
# user now.
|
||||
#
|
||||
# Both columns, and the flag. This is the one place that clears vectors
|
||||
# in bulk rather than through memorybank.set_vector, and when the
|
||||
@@ -80,28 +99,61 @@ def update_settings(
|
||||
return settings
|
||||
|
||||
|
||||
async def list_endpoint_models(cfg: auth.ProviderConfig) -> dict:
|
||||
"""Fetches the endpoint's /models listing.
|
||||
async def list_endpoint_models(endpoint_url: str) -> dict:
|
||||
"""Fetches the endpoint's `/models` listing, and doubles as the connection test.
|
||||
|
||||
The call also serves as a connectivity check, so a failure returns
|
||||
`{"ok": False, "detail": ...}` rather than raising.
|
||||
Returns `{"ok": False, "detail": ...}` rather than raising, because every
|
||||
caller wants to show the reason rather than fail the page.
|
||||
|
||||
The failure cases are told apart on purpose. "Ollama isn't running", "that
|
||||
address isn't allowed", "the certificate doesn't verify" and "it answered,
|
||||
but with an error" need four different things done about them, and a single
|
||||
"connection failed" leaves the user guessing which they have.
|
||||
"""
|
||||
# SSRF guard. Never probe a non-public address the user supplied.
|
||||
reason = await run_in_threadpool(netguard.endpoint_block_reason, cfg.endpoint_url)
|
||||
if reason:
|
||||
return {"ok": False, "detail": f"Can't reach that endpoint — {reason}."}
|
||||
url = cfg.endpoint_url.rstrip("/") + "/models"
|
||||
headers = {}
|
||||
if cfg.api_key:
|
||||
headers["Authorization"] = f"Bearer {cfg.api_key}"
|
||||
reason = await run_in_threadpool(endpoints.rejection_reason, endpoint_url)
|
||||
if reason is not None:
|
||||
return {
|
||||
"ok": False, "kind": "rejected",
|
||||
"detail": f"That endpoint can't be used — {reason}.",
|
||||
}
|
||||
|
||||
url = endpoint_url.rstrip("/") + "/models"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10, verify=tlstrust.ssl_context()) as client:
|
||||
resp = await client.get(url, headers=headers)
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(TEST_TIMEOUT, connect=CONNECT_TIMEOUT),
|
||||
verify=tlstrust.ssl_context(),
|
||||
) as client:
|
||||
resp = await client.get(url)
|
||||
except httpx.ConnectError as exc:
|
||||
# A TLS failure arrives as a ConnectError too, and it needs a different
|
||||
# answer from "nothing is listening": install the CA, don't start Ollama.
|
||||
if "CERTIFICATE_VERIFY" in str(exc).upper() or "SSL" in str(exc).upper():
|
||||
return {
|
||||
"ok": False, "kind": "tls",
|
||||
"detail": (
|
||||
"The endpoint's TLS certificate could not be verified. If it "
|
||||
"uses a private or self-signed CA, install that CA on this "
|
||||
"machine so the system trusts it. Certificate checking is "
|
||||
"not optional."
|
||||
),
|
||||
}
|
||||
return {
|
||||
"ok": False, "kind": "unreachable",
|
||||
"detail": f"Could not connect to {endpoint_url} — is Ollama running there?",
|
||||
}
|
||||
except httpx.TimeoutException:
|
||||
return {
|
||||
"ok": False, "kind": "timeout",
|
||||
"detail": f"{endpoint_url} did not answer within {TEST_TIMEOUT:.0f}s.",
|
||||
}
|
||||
except httpx.HTTPError as exc:
|
||||
return {"ok": False, "detail": f"Connection failed: {exc}"}
|
||||
return {"ok": False, "kind": "error", "detail": f"Connection failed: {exc}"}
|
||||
|
||||
if resp.status_code != 200:
|
||||
return {"ok": False, "detail": f"HTTP {resp.status_code}: {resp.text[:300]}"}
|
||||
return {
|
||||
"ok": False, "kind": "http",
|
||||
"detail": f"HTTP {resp.status_code}: {resp.text[:300]}",
|
||||
}
|
||||
|
||||
models_available: list[str] = []
|
||||
try:
|
||||
@@ -115,15 +167,19 @@ async def list_endpoint_models(cfg: auth.ProviderConfig) -> dict:
|
||||
|
||||
@router.post("/test")
|
||||
async def test_connection(
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Runs a cheap connectivity check against whatever the turn engine would use.
|
||||
|
||||
That includes the shared demo endpoint, when the user has no key of their
|
||||
own.
|
||||
"""
|
||||
limits.rate_limit("connection-test", request, user)
|
||||
"""Checks the endpoint the turn engine would use, and lists its models."""
|
||||
settings = get_settings(db, user)
|
||||
return await list_endpoint_models(auth.resolve_provider_config(settings))
|
||||
result = await list_endpoint_models(settings.endpoint_url)
|
||||
if result.get("ok") and settings.model and settings.model not in result["models"]:
|
||||
# Reachable, but pointed at a model that is not installed there — the
|
||||
# commonest way for a correct endpoint to still fail every turn.
|
||||
return result | {
|
||||
"warning": (
|
||||
f"{settings.endpoint_url} is reachable, but has no model named "
|
||||
f"'{settings.model}'. Pull it there, or pick one from the list."
|
||||
)
|
||||
}
|
||||
return result
|
||||
|
||||
@@ -159,7 +159,6 @@ def import_story_cards(
|
||||
raise HTTPException(422, 'Expected a "cards" array of story cards.')
|
||||
cards_in = [c for c in cards_in if isinstance(c, dict)]
|
||||
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_bundle_lists(story_cards=cards_in)
|
||||
existing = len(owner.story_cards)
|
||||
if auth.MULTI_USER and existing + len(cards_in) > limits.MAX_STORY_CARDS_PER_OWNER:
|
||||
|
||||
@@ -14,7 +14,6 @@ NAME_MAX = 200 # Titles and names. VARCHAR(200).
|
||||
TAGS_MAX = 500 # VARCHAR(500).
|
||||
CARD_TYPE_MAX = 100 # VARCHAR(100).
|
||||
PROSE_MAX = 50_000 # Memory, author's note, prompts, entries, and notes.
|
||||
SCRIPT_MAX = 200_000 # One JavaScript source file.
|
||||
ACTION_MAX = 20_000 # One player action.
|
||||
MEMORY_TEXT_MAX = 5_000
|
||||
# A scenario cover image, stored inline as a base64 data URI. A 400x300 WebP at
|
||||
@@ -26,17 +25,21 @@ ICON_MAX = 16 # One emoji or glyph. VARCHAR(16).
|
||||
BRANCH_NAME_MAX = 80 # What a player called one line of the story. VARCHAR(80).
|
||||
PERSONA_NAME_MAX = 80 # The protagonist's name. VARCHAR(80).
|
||||
PERSONA_PRONOUNS_MAX = 40 # "they/them" and the like. VARCHAR(40).
|
||||
# M4: what a player called a Save Point. VARCHAR(120). Wider than a branch name
|
||||
# because these are sentences rather than labels — "Before entering the abbey"
|
||||
# is the example the specification uses throughout.
|
||||
CHECKPOINT_NAME_MAX = 120
|
||||
|
||||
Name = Annotated[str, Field(max_length=NAME_MAX)]
|
||||
Tags = Annotated[str, Field(max_length=TAGS_MAX)]
|
||||
CardType = Annotated[str, Field(max_length=CARD_TYPE_MAX)]
|
||||
Prose = Annotated[str, Field(max_length=PROSE_MAX)]
|
||||
ScriptSource = Annotated[str, Field(max_length=SCRIPT_MAX)]
|
||||
ActionText = Annotated[str, Field(max_length=ACTION_MAX)]
|
||||
Image = Annotated[str, Field(max_length=IMAGE_MAX)]
|
||||
Icon = Annotated[str, Field(max_length=ICON_MAX)]
|
||||
PersonaName = Annotated[str, Field(max_length=PERSONA_NAME_MAX)]
|
||||
PersonaPronouns = Annotated[str, Field(max_length=PERSONA_PRONOUNS_MAX)]
|
||||
CheckpointName = Annotated[str, Field(max_length=CHECKPOINT_NAME_MAX)]
|
||||
|
||||
|
||||
class ORMModel(BaseModel):
|
||||
@@ -106,7 +109,6 @@ class ScenarioUpdate(BaseModel):
|
||||
image: Image | None = None
|
||||
icon: Icon | None = None
|
||||
stat_schema: dict | None = None
|
||||
script_ids: list[int] | None = None
|
||||
|
||||
|
||||
class ScenarioOut(ORMModel, ScenarioBase):
|
||||
@@ -115,7 +117,6 @@ class ScenarioOut(ORMModel, ScenarioBase):
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
story_cards: list[StoryCardOut] = []
|
||||
scripts: list["ScriptOut"] = []
|
||||
|
||||
|
||||
class ScenarioListItem(ORMModel):
|
||||
@@ -258,6 +259,10 @@ class BranchOut(ORMModel):
|
||||
fork_depth: int | None = None
|
||||
depth: int
|
||||
own_actions: int = 0
|
||||
# M4: how many Save Points name a position on this line. Deleting the branch
|
||||
# deletes them with its story, so the panel warns with a number rather than
|
||||
# a vague caution. Zero for a line nobody has bookmarked, which is most.
|
||||
save_points: int = 0
|
||||
is_head: bool = False
|
||||
# NULL for a branch nobody has named. The client labels those from the fork
|
||||
# depth rather than the server inventing a name. See the column comment.
|
||||
@@ -271,6 +276,67 @@ class BranchRename(BaseModel):
|
||||
name: Annotated[str, Field(max_length=BRANCH_NAME_MAX)] | None = None
|
||||
|
||||
|
||||
# ---------- Save Points (M4) ----------
|
||||
#
|
||||
# "Save Point" is the user-facing term and `checkpoint` is the internal one
|
||||
# (`BROWSER-UX-SPEC.md` §23). The wire format uses the internal name, as the
|
||||
# rest of this module does.
|
||||
|
||||
|
||||
class CheckpointOut(ORMModel):
|
||||
"""One Save Point: a name and the position it names.
|
||||
|
||||
The position is reported three ways because the panel needs three different
|
||||
things from it. `turn` is what a reader counts — the same `depth + 1` the
|
||||
branch list shows. `depth` and `branch_id` are the coordinate itself.
|
||||
`on_path` says whether the position lies on the story being read, which is
|
||||
how the panel can tell a Save Point on this line from one naming a line the
|
||||
story has left; restoring either works, but they are not the same offer.
|
||||
|
||||
`resolved` is false when the coordinate no longer names a live turn, which
|
||||
an action deleted out of the middle of a story can do. Restore refuses such
|
||||
a Save Point rather than moving the head somewhere approximate, so the list
|
||||
says so before the button is pressed.
|
||||
"""
|
||||
|
||||
id: int
|
||||
adventure_id: int
|
||||
name: str
|
||||
note: str = ""
|
||||
branch_id: int
|
||||
depth: int
|
||||
turn: int = 0
|
||||
on_path: bool = True
|
||||
resolved: bool = True
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class CheckpointCreate(BaseModel):
|
||||
"""A Save Point at wherever the story is being read.
|
||||
|
||||
The position is not a field. A Save Point is made at the campaign's active
|
||||
head, which the server already knows, and accepting a coordinate from the
|
||||
client would be the second way to name a position — the thing this milestone
|
||||
exists not to build.
|
||||
"""
|
||||
|
||||
name: CheckpointName
|
||||
note: Prose = ""
|
||||
|
||||
|
||||
class CheckpointRename(BaseModel):
|
||||
"""A new label, and nothing else.
|
||||
|
||||
There is deliberately no coordinate here. `STORY-BRANCH-SEMANTICS.md` §24
|
||||
keeps a Save Point's meaning auditable by refusing to move one: rename it,
|
||||
or delete it and make another where you are.
|
||||
"""
|
||||
|
||||
name: CheckpointName | None = None
|
||||
note: Prose | None = None
|
||||
|
||||
|
||||
class ActionUpdate(BaseModel):
|
||||
text: ActionText
|
||||
|
||||
@@ -321,6 +387,12 @@ class AdventureOut(ORMModel):
|
||||
# story's length, which is how the client knows more actions exist above.
|
||||
actions: list[ActionOut] = []
|
||||
action_count: int = 0
|
||||
# M3. Whether the history controls have anywhere to go from where the story
|
||||
# is. The client cannot work either out for itself: `can_undo` needs the
|
||||
# campaign opening, which may be off the top of the loaded window, and
|
||||
# `can_redo` needs the retained future, which the client is never sent.
|
||||
can_undo: bool = False
|
||||
can_redo: bool = False
|
||||
|
||||
|
||||
class ActionPage(BaseModel):
|
||||
@@ -331,6 +403,10 @@ class ActionPage(BaseModel):
|
||||
# Whether anything older than this slice exists. The server computes it, so
|
||||
# the client never has to do arithmetic on positions to find the end.
|
||||
has_more: bool = False
|
||||
# The same two flags `AdventureOut` carries, so that the response to Undo,
|
||||
# Redo or a turn updates the controls without a second request.
|
||||
can_undo: bool = False
|
||||
can_redo: bool = False
|
||||
|
||||
|
||||
# ---------- Memory bank (Phase 6) ----------
|
||||
@@ -374,67 +450,6 @@ class AdventureListItem(ORMModel):
|
||||
icon: str = ""
|
||||
|
||||
|
||||
# ---------- Scripts ----------
|
||||
|
||||
class ScriptBase(BaseModel):
|
||||
name: Name = "Untitled Script"
|
||||
description: Prose = ""
|
||||
library_js: ScriptSource = ""
|
||||
input_js: ScriptSource = ""
|
||||
context_js: ScriptSource = ""
|
||||
output_js: ScriptSource = ""
|
||||
|
||||
|
||||
class ScriptCreate(ScriptBase):
|
||||
pass
|
||||
|
||||
|
||||
class ScriptUpdate(BaseModel):
|
||||
name: Name | None = None
|
||||
description: Prose | None = None
|
||||
library_js: ScriptSource | None = None
|
||||
input_js: ScriptSource | None = None
|
||||
context_js: ScriptSource | None = None
|
||||
output_js: ScriptSource | None = None
|
||||
|
||||
|
||||
class ScriptOut(ORMModel, ScriptBase):
|
||||
id: int
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class ScriptTestRequest(BaseModel):
|
||||
hook: Literal["input", "context", "output"]
|
||||
text: Prose = ""
|
||||
state: dict = {}
|
||||
|
||||
|
||||
class AdventureScriptOut(ORMModel):
|
||||
id: int
|
||||
adventure_id: int
|
||||
position: int
|
||||
enabled: bool
|
||||
name: str
|
||||
description: str
|
||||
library_js: str
|
||||
input_js: str
|
||||
context_js: str
|
||||
output_js: str
|
||||
# The router sets this field, which is not stored. It is `True` when a
|
||||
# syncable library version exists whose code differs from this copy, and
|
||||
# `None` when there is nothing to sync from.
|
||||
out_of_date: bool | None = None
|
||||
|
||||
|
||||
class AdventureScriptUpdate(BaseModel):
|
||||
enabled: bool | None = None
|
||||
library_js: ScriptSource | None = None
|
||||
input_js: ScriptSource | None = None
|
||||
context_js: ScriptSource | None = None
|
||||
output_js: ScriptSource | None = None
|
||||
|
||||
|
||||
# ---------- Auth (Phase 8) ----------
|
||||
|
||||
class AuthCredentials(BaseModel):
|
||||
@@ -448,15 +463,12 @@ class AuthCredentials(BaseModel):
|
||||
|
||||
class SettingsOut(ORMModel):
|
||||
endpoint_url: str
|
||||
# The key itself is never returned. It is encrypted at rest and
|
||||
# write-only.
|
||||
has_api_key: bool
|
||||
model: str
|
||||
api_mode: str
|
||||
temperature: float
|
||||
max_output_tokens: int
|
||||
reasoning_max_tokens: int
|
||||
context_token_budget: int
|
||||
model_timeout_seconds: int
|
||||
narrator_prompt: str
|
||||
summary_model: str
|
||||
embedding_model: str
|
||||
@@ -491,18 +503,15 @@ class ChatRequest(BaseModel):
|
||||
|
||||
class SettingsUpdate(BaseModel):
|
||||
endpoint_url: Annotated[str, Field(max_length=500)] | None = None # VARCHAR(500).
|
||||
# Encryption expands the stored value by about four thirds into the same
|
||||
# VARCHAR(500), so 256 plaintext characters is the largest safe input. The
|
||||
# stored form is "enc:" plus Fernet plus base64.
|
||||
api_key: Annotated[str, Field(max_length=256)] | None = None
|
||||
model: Name | None = None
|
||||
api_mode: Annotated[str, Field(max_length=20)] | None = None
|
||||
temperature: Annotated[float, Field(ge=0, le=5)] | None = None
|
||||
max_output_tokens: Annotated[int, Field(ge=1, le=100_000)] | None = None
|
||||
# A value of -1 turns reasoning off explicitly, which sends
|
||||
# `reasoning: {effort: none}`. A value of 0 sends nothing.
|
||||
reasoning_max_tokens: Annotated[int, Field(ge=-1, le=100_000)] | None = None
|
||||
context_token_budget: Annotated[int, Field(ge=256, le=200_000)] | None = None
|
||||
# Seconds to wait for the model. The floor is high enough that a normal
|
||||
# turn cannot trip it; the ceiling exists so that "wait longer" stays a
|
||||
# number rather than becoming "wait forever".
|
||||
model_timeout_seconds: Annotated[int, Field(ge=30, le=3600)] | None = None
|
||||
narrator_prompt: Prose | None = None
|
||||
summary_model: Name | None = None
|
||||
embedding_model: Name | None = None
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
from .engine import HookResult, run_hook
|
||||
from .pipeline import ScriptPipeline
|
||||
|
||||
__all__ = ["HookResult", "ScriptPipeline", "run_hook"]
|
||||
@@ -1,146 +0,0 @@
|
||||
"""AI Dungeon-compatible script execution in an embedded QuickJS sandbox.
|
||||
|
||||
Each hook run is fully isolated (fresh Context), capped at 16 MB memory and
|
||||
2 seconds CPU, with no filesystem/network/process access (QuickJS has none by
|
||||
default). Scripts follow the AI Dungeon contract: define a `modifier(text)`
|
||||
and call it as the last line; its return value `{ text, stop }` is the result.
|
||||
"""
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import quickjs
|
||||
|
||||
MEMORY_LIMIT = 16 * 1024 * 1024
|
||||
TIME_LIMIT_SECONDS = 2
|
||||
HISTORY_WINDOW = 100 # recent actions exposed as `history`
|
||||
|
||||
# Globals per the official docs: text, state, history, storyCards, info,
|
||||
# log/console.log, story card functions, plus legacy worldInfo aliases.
|
||||
PRELUDE = """
|
||||
"use strict";
|
||||
var __logs = [];
|
||||
var state = __DATA__.state;
|
||||
var text = __DATA__.text;
|
||||
var history = __DATA__.history;
|
||||
var storyCards = __DATA__.storyCards;
|
||||
var info = __DATA__.info;
|
||||
|
||||
function log(msg) {
|
||||
__logs.push(typeof msg === "string" ? msg : JSON.stringify(msg));
|
||||
}
|
||||
var console = { log: log };
|
||||
|
||||
// Returns the new card's index, or false if a card with those keys exists —
|
||||
// matching real AI Dungeon. Note index 0 is falsy; that quirk is upstream's.
|
||||
function addStoryCard(keys, entry, type) {
|
||||
for (var i = 0; i < storyCards.length; i++) {
|
||||
if (storyCards[i].keys === keys) return false;
|
||||
}
|
||||
storyCards.push({ id: null, keys: keys || "", entry: entry || "", type: type || "" });
|
||||
return storyCards.length - 1;
|
||||
}
|
||||
function updateStoryCard(index, keys, entry, type) {
|
||||
var card = storyCards[index];
|
||||
if (!card) throw new Error("Story card not found");
|
||||
card.keys = keys;
|
||||
card.entry = entry;
|
||||
card.type = type;
|
||||
}
|
||||
function removeStoryCard(index) {
|
||||
if (!storyCards[index]) throw new Error("Story card not found");
|
||||
storyCards.splice(index, 1);
|
||||
}
|
||||
|
||||
// Legacy aliases used by older AI Dungeon scripts.
|
||||
var worldInfo = storyCards;
|
||||
var worldEntries = storyCards;
|
||||
function addWorldEntry(keys, entry) { return addStoryCard(keys, entry, ""); }
|
||||
function updateWorldEntry(index, keys, entry) {
|
||||
var card = storyCards[index];
|
||||
if (!card) throw new Error("World entry not found");
|
||||
card.keys = keys;
|
||||
card.entry = entry;
|
||||
}
|
||||
function removeWorldEntry(index) { return removeStoryCard(index); }
|
||||
"""
|
||||
|
||||
COLLECT = """
|
||||
JSON.stringify({
|
||||
result: (typeof __result === "undefined" || __result === null) ? null : __result,
|
||||
state: state,
|
||||
storyCards: storyCards,
|
||||
logs: __logs
|
||||
})
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class HookResult:
|
||||
text: str
|
||||
stop: bool = False
|
||||
state: dict = field(default_factory=dict)
|
||||
story_cards: list = field(default_factory=list)
|
||||
logs: list = field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
def run_hook(
|
||||
library_js: str,
|
||||
hook_js: str,
|
||||
text: str,
|
||||
state: dict,
|
||||
history: list[dict],
|
||||
story_cards: list[dict],
|
||||
info: dict,
|
||||
) -> HookResult:
|
||||
"""Run one modifier hook. This function never raises. Failures return as
|
||||
`.error` with text, state, and cards unchanged, so a bad script cannot
|
||||
break a turn."""
|
||||
unchanged = HookResult(text=text, state=state, story_cards=story_cards)
|
||||
source = f"{library_js}\n;\n{hook_js}" if library_js.strip() else hook_js
|
||||
if not source.strip():
|
||||
return unchanged
|
||||
|
||||
data = {
|
||||
"state": state,
|
||||
"text": text,
|
||||
"history": history[-HISTORY_WINDOW:],
|
||||
"storyCards": story_cards,
|
||||
"info": info,
|
||||
}
|
||||
try:
|
||||
ctx = quickjs.Context()
|
||||
ctx.set_memory_limit(MEMORY_LIMIT)
|
||||
ctx.set_time_limit(TIME_LIMIT_SECONDS)
|
||||
ctx.eval(f"var __DATA__ = {json.dumps(data)};")
|
||||
ctx.eval(PRELUDE)
|
||||
ctx.eval(f"var __SRC__ = {json.dumps(source)};")
|
||||
# Indirect eval keeps the script in global scope, so `modifier(text)` as the
|
||||
# script's final expression statement becomes the completion value.
|
||||
ctx.eval("var __result = (0, eval)(__SRC__);")
|
||||
collected = json.loads(ctx.eval(COLLECT))
|
||||
except quickjs.JSException as exc:
|
||||
unchanged.error = f"Script error: {exc}"
|
||||
return unchanged
|
||||
except Exception as exc: # memory limit, invalid JSON state, engine faults
|
||||
unchanged.error = f"Script execution failed: {exc}"
|
||||
return unchanged
|
||||
|
||||
result = collected.get("result")
|
||||
new_text, stop = text, False
|
||||
if isinstance(result, dict):
|
||||
if isinstance(result.get("text"), str):
|
||||
new_text = result["text"]
|
||||
stop = bool(result.get("stop"))
|
||||
elif isinstance(result, str):
|
||||
new_text = result
|
||||
|
||||
new_state = collected.get("state")
|
||||
return HookResult(
|
||||
text=new_text,
|
||||
stop=stop,
|
||||
state=new_state if isinstance(new_state, dict) else {},
|
||||
story_cards=collected.get("storyCards") or [],
|
||||
logs=collected.get("logs") or [],
|
||||
)
|
||||
@@ -1,112 +0,0 @@
|
||||
"""Runs an adventure's enabled scripts through a turn's hook points, applying
|
||||
state and story-card mutations back to the database after each hook."""
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import models
|
||||
from ..context import history as context_history
|
||||
from .engine import run_hook
|
||||
|
||||
MAX_STORY_CARDS = 5000 # AI Dungeon's per-adventure sanity cap
|
||||
|
||||
|
||||
class ScriptPipeline:
|
||||
def __init__(self, adventure: models.Adventure, db: Session):
|
||||
self.adventure = adventure
|
||||
self.db = db
|
||||
self.logs: list[str] = []
|
||||
self.errors: list[str] = []
|
||||
|
||||
@property
|
||||
def message(self) -> str | None:
|
||||
state = self.adventure.script_state
|
||||
msg = state.get("message") if isinstance(state, dict) else None
|
||||
return msg if isinstance(msg, str) and msg.strip() else None
|
||||
|
||||
def _history(self) -> list[dict]:
|
||||
# Read the path rather than `adventure.actions`. That collection holds
|
||||
# every branch's actions, and this is the documented history API a user
|
||||
# script reads. Giving a script the siblings of the turn it is running on
|
||||
# would be the same bug as building a prompt from them, and visible to
|
||||
# the user.
|
||||
#
|
||||
# `story_actions` also drops rows with blank text, which
|
||||
# `adventure.actions` kept, so this array is shorter than it was for an
|
||||
# adventure that has any such rows. `info.actionCount` counts the same
|
||||
# way. That is intended. A row with no text is this app's bookkeeping, it
|
||||
# has no counterpart in the AI Dungeon history a ported script was
|
||||
# written against, and the prompt has never included one. A script keyed
|
||||
# on every N actions lands on different turns than it did before phase
|
||||
# 14, and no reading of this is compatible with both.
|
||||
return [
|
||||
{"text": a.text, "rawText": a.text, "type": a.type}
|
||||
for a in context_history.story_actions(self.adventure)
|
||||
]
|
||||
|
||||
def _cards(self) -> list[dict]:
|
||||
return [
|
||||
{"id": c.id, "keys": c.keys, "entry": c.entry, "type": c.type}
|
||||
for c in self.adventure.story_cards
|
||||
]
|
||||
|
||||
def _info(self) -> dict:
|
||||
return {
|
||||
"actionCount": context_history.count(self.adventure),
|
||||
"characterNames": [],
|
||||
"memoryLength": len(self.adventure.memory),
|
||||
"maxChars": 0,
|
||||
}
|
||||
|
||||
def _apply_cards(self, returned: list) -> None:
|
||||
existing = {c.id: c for c in self.adventure.story_cards}
|
||||
seen_ids = set()
|
||||
added = 0
|
||||
for item in returned:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
card_id = item.get("id")
|
||||
keys = str(item.get("keys") or "")
|
||||
entry = str(item.get("entry") or "")
|
||||
card_type = str(item.get("type") or "")
|
||||
if card_id in existing:
|
||||
seen_ids.add(card_id)
|
||||
card = existing[card_id]
|
||||
card.keys, card.entry, card.type = keys, entry, card_type
|
||||
elif len(existing) + added < MAX_STORY_CARDS:
|
||||
self.db.add(
|
||||
models.StoryCard(
|
||||
adventure_id=self.adventure.id,
|
||||
keys=keys, entry=entry, type=card_type,
|
||||
)
|
||||
)
|
||||
added += 1
|
||||
for card_id, card in existing.items():
|
||||
if card_id not in seen_ids:
|
||||
self.db.delete(card)
|
||||
|
||||
def run(self, hook: str, text: str) -> tuple[str, bool]:
|
||||
"""Chain `hook` across all enabled scripts. Returns (text, stop)."""
|
||||
state = self.adventure.script_state if isinstance(self.adventure.script_state, dict) else {}
|
||||
for script in self.adventure.scripts:
|
||||
hook_js = getattr(script, f"{hook}_js")
|
||||
if not script.enabled or not hook_js.strip():
|
||||
continue
|
||||
result = run_hook(
|
||||
script.library_js, hook_js, text, state,
|
||||
self._history(), self._cards(), self._info(),
|
||||
)
|
||||
if result.error:
|
||||
self.errors.append(f"{script.name} ({hook}): {result.error}")
|
||||
continue # a broken script never breaks the turn
|
||||
self.logs.extend(f"[{script.name}/{hook}] {line}" for line in result.logs)
|
||||
self._apply_cards(result.story_cards)
|
||||
state = result.state
|
||||
self.adventure.script_state = state
|
||||
self.db.commit()
|
||||
text = result.text
|
||||
if result.stop:
|
||||
return text, True
|
||||
return text, False
|
||||
|
||||
def report(self) -> dict:
|
||||
return {"logs": self.logs, "errors": self.errors, "message": self.message}
|
||||
@@ -1,131 +0,0 @@
|
||||
"""Phase 8: secrets and crypto primitives for optional accounts.
|
||||
|
||||
Everything derives from one server-side secret:
|
||||
|
||||
* Session cookies are HMAC-signed with it.
|
||||
* Stored LLM API keys are Fernet-encrypted with a key derived from it.
|
||||
|
||||
The secret comes from `AIDND_SECRET_KEY`, or it is generated once into
|
||||
`secret.key` next to the database, so a local install and a Docker volume work
|
||||
with no configuration. Losing that file logs everyone out and makes the stored
|
||||
API keys unreadable, and users then re-enter them. A multi-user deployment has
|
||||
to set the environment variable, because a hosted filesystem is ephemeral and a
|
||||
`secret.key` regenerated on every deploy would log out every user each time.
|
||||
|
||||
Passwords use `hashlib.scrypt`, which is in the standard library and backed by
|
||||
OpenSSL, so this needs no separate hashing dependency.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
|
||||
from cryptography.fernet import Fernet, InvalidToken
|
||||
|
||||
from .database import DB_PATH
|
||||
|
||||
_SECRET_FILE = DB_PATH.parent / "secret.key"
|
||||
|
||||
|
||||
def _load_secret() -> bytes:
|
||||
env = os.environ.get("AIDND_SECRET_KEY", "").strip()
|
||||
if env:
|
||||
return env.encode()
|
||||
# Same flag parse as auth.MULTI_USER (auth imports this module, so it
|
||||
# can't be imported from there).
|
||||
if os.environ.get("AIDND_MULTI_USER", "").strip().lower() in ("1", "true", "yes", "on"):
|
||||
raise RuntimeError(
|
||||
"AIDND_SECRET_KEY must be set when AIDND_MULTI_USER is on: an "
|
||||
"auto-generated secret.key on an ephemeral hosted filesystem would "
|
||||
"rotate on every deploy, logging out every user and orphaning "
|
||||
"their stored API keys. Generate one with: "
|
||||
"python -c \"import secrets; print(secrets.token_urlsafe(48))\""
|
||||
)
|
||||
if _SECRET_FILE.exists():
|
||||
return _SECRET_FILE.read_bytes().strip()
|
||||
secret = secrets.token_urlsafe(48).encode()
|
||||
_SECRET_FILE.write_bytes(secret)
|
||||
return secret
|
||||
|
||||
|
||||
SECRET_KEY = _load_secret()
|
||||
_fernet = Fernet(base64.urlsafe_b64encode(hashlib.sha256(SECRET_KEY).digest()))
|
||||
|
||||
|
||||
# ---------- Password hashing (scrypt) ----------
|
||||
|
||||
_SCRYPT_N, _SCRYPT_R, _SCRYPT_P = 2**14, 8, 1
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
salt = secrets.token_bytes(16)
|
||||
key = hashlib.scrypt(
|
||||
password.encode(), salt=salt, n=_SCRYPT_N, r=_SCRYPT_R, p=_SCRYPT_P
|
||||
)
|
||||
return f"scrypt${_SCRYPT_N}${_SCRYPT_R}${_SCRYPT_P}${salt.hex()}${key.hex()}"
|
||||
|
||||
|
||||
def verify_password(password: str, stored: str) -> bool:
|
||||
try:
|
||||
scheme, n, r, p, salt_hex, key_hex = stored.split("$")
|
||||
if scheme != "scrypt":
|
||||
return False
|
||||
key = hashlib.scrypt(
|
||||
password.encode(), salt=bytes.fromhex(salt_hex),
|
||||
n=int(n), r=int(r), p=int(p),
|
||||
)
|
||||
return hmac.compare_digest(key, bytes.fromhex(key_hex))
|
||||
except (ValueError, AttributeError):
|
||||
return False
|
||||
|
||||
|
||||
# ---------- Session tokens ----------
|
||||
# The token is "v1.<user_id>.<hmac>". It does not expire, because a long-lived
|
||||
# guest session is what this is for.
|
||||
|
||||
def sign_session(user_id: int) -> str:
|
||||
payload = f"v1.{user_id}"
|
||||
sig = hmac.new(SECRET_KEY, payload.encode(), hashlib.sha256).hexdigest()
|
||||
return f"{payload}.{sig}"
|
||||
|
||||
|
||||
def verify_session(token: str) -> int | None:
|
||||
try:
|
||||
version, user_id, sig = token.split(".")
|
||||
if version != "v1":
|
||||
return None
|
||||
payload = f"{version}.{user_id}"
|
||||
expected = hmac.new(SECRET_KEY, payload.encode(), hashlib.sha256).hexdigest()
|
||||
if not hmac.compare_digest(sig, expected):
|
||||
return None
|
||||
return int(user_id)
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
# ---------- API-key encryption at rest ----------
|
||||
# Stored values carry an "enc:" prefix so plaintext keys from pre-Phase-8
|
||||
# databases can be recognized and migrated.
|
||||
|
||||
ENC_PREFIX = "enc:"
|
||||
|
||||
|
||||
def encrypt_secret(plain: str) -> str:
|
||||
if not plain:
|
||||
return ""
|
||||
return ENC_PREFIX + _fernet.encrypt(plain.encode()).decode()
|
||||
|
||||
|
||||
def decrypt_secret(stored: str) -> str:
|
||||
"""Returns the plaintext key. Tolerates legacy plaintext values (returned
|
||||
as-is) and undecryptable tokens (secret rotated → treated as unset)."""
|
||||
if not stored:
|
||||
return ""
|
||||
if not stored.startswith(ENC_PREFIX):
|
||||
return stored
|
||||
try:
|
||||
return _fernet.decrypt(stored[len(ENC_PREFIX):].encode()).decode()
|
||||
except (InvalidToken, ValueError):
|
||||
return ""
|
||||
@@ -4,8 +4,7 @@ Every JSON file in ``seed_data/`` describes one demo scenario in the same
|
||||
model-native shape the export endpoint produces. Seeded scenarios have a NULL
|
||||
owner and ``is_public=True``, so every visitor (including guests) sees them and
|
||||
can start an adventure from them, while nobody can edit them. Starting an
|
||||
adventure copies the scenario's story cards and scripts into the adventure, so
|
||||
the seeded scripts run for guests too.
|
||||
adventure copies the scenario's story cards into the adventure.
|
||||
|
||||
Seed files are the source of truth for demo content: a scenario is inserted if
|
||||
missing, reconciled in place when a seed file's content changes, and deleted
|
||||
@@ -13,7 +12,7 @@ when no file claims its title any more, so an edit ships on the next deploy.
|
||||
Rename a seed by changing its `title` and listing the old one under
|
||||
`previous_titles`, which moves the rename onto the existing row. When a seed already matches, nothing is written, so
|
||||
this stays cheap to run on every boot. An adventure already started from a demo
|
||||
keeps its own copied cards and scripts and is unchanged. Only a new adventure
|
||||
keeps its own copied cards and is unchanged. Only a new adventure
|
||||
picks up the updated content.
|
||||
"""
|
||||
|
||||
@@ -35,7 +34,6 @@ SEED_DIR = Path(__file__).resolve().parent / "seed_data"
|
||||
_SCALARS = ("title", "description", "prompt", "memory", "authors_note", "ai_instructions",
|
||||
"tags", "image", "icon")
|
||||
_CARD_FIELDS = ("type", "name", "keys", "entry", "notes")
|
||||
_SCRIPT_FIELDS = ("name", "library_js", "input_js", "context_js", "output_js")
|
||||
|
||||
|
||||
def seed_public_scenarios(engine: Engine) -> None:
|
||||
@@ -98,7 +96,7 @@ def _sweep_unclaimed(db, claimed: set[str]) -> int:
|
||||
nothing anybody created can be reached from here.
|
||||
|
||||
An adventure started from a deleted demo survives. `adventures.scenario_id`
|
||||
is `ON DELETE SET NULL`, so the story, its cards, and its scripts are its
|
||||
is `ON DELETE SET NULL`, so the story and its cards are its
|
||||
own copies and stay; the adventure loses the cover art it inherited.
|
||||
|
||||
The caller skips this when a seed file failed to parse. A file that cannot
|
||||
@@ -117,11 +115,6 @@ def _sweep_unclaimed(db, claimed: set[str]) -> int:
|
||||
)
|
||||
for scenario in stale:
|
||||
logger.info("Removing seeded scenario %r; no seed file claims it.", scenario.title)
|
||||
# The scripts are joined through a secondary table, so nothing cascades
|
||||
# to them. They have a NULL owner and no other reader.
|
||||
for script in list(scenario.scripts):
|
||||
db.delete(script)
|
||||
scenario.scripts = []
|
||||
db.delete(scenario)
|
||||
return len(stale)
|
||||
|
||||
@@ -130,10 +123,6 @@ def _card_tuple(source, get) -> tuple:
|
||||
return tuple(get(source, f) for f in _CARD_FIELDS)
|
||||
|
||||
|
||||
def _script_tuple(source, get) -> tuple:
|
||||
return tuple(get(source, f) for f in _SCRIPT_FIELDS)
|
||||
|
||||
|
||||
def find_seeded(db, title: str) -> models.Scenario | None:
|
||||
"""Returns the seeded scenario with this exact title, if there is one."""
|
||||
return (
|
||||
@@ -178,14 +167,7 @@ def _matches(scenario: models.Scenario, data: dict) -> bool:
|
||||
_card_tuple(c, lambda o, f: o.get(f, ""))
|
||||
for c in (data.get("story_cards") or []) if isinstance(c, dict)
|
||||
)
|
||||
if have_cards != want_cards:
|
||||
return False
|
||||
have_scripts = sorted(_script_tuple(s, lambda o, f: getattr(o, f)) for s in scenario.scripts)
|
||||
want_scripts = sorted(
|
||||
_script_tuple(s, lambda o, f: (o.get(f, "") or ("Script" if f == "name" else "")))
|
||||
for s in (data.get("scripts") or []) if isinstance(s, dict)
|
||||
)
|
||||
return have_scripts == want_scripts
|
||||
return have_cards == want_cards
|
||||
|
||||
|
||||
def _insert_scenario(db, data: dict) -> None:
|
||||
@@ -203,9 +185,6 @@ def _update_scenario(db, scenario: models.Scenario, data: dict) -> None:
|
||||
# adventure foreign keys that point at it, intact.
|
||||
for card in list(scenario.story_cards):
|
||||
db.delete(card)
|
||||
for script in list(scenario.scripts):
|
||||
db.delete(script)
|
||||
scenario.scripts = []
|
||||
db.flush()
|
||||
_populate_children(db, scenario, data)
|
||||
|
||||
@@ -231,19 +210,3 @@ def _populate_children(db, scenario: models.Scenario, data: dict) -> None:
|
||||
notes=card.get("notes", ""),
|
||||
)
|
||||
)
|
||||
|
||||
for item in data.get("scripts") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
script = models.Script(
|
||||
user_id=None,
|
||||
name=item.get("name", "Script"),
|
||||
description=item.get("description", ""),
|
||||
library_js=item.get("library_js", ""),
|
||||
input_js=item.get("input_js", ""),
|
||||
context_js=item.get("context_js", ""),
|
||||
output_js=item.get("output_js", ""),
|
||||
)
|
||||
db.add(script)
|
||||
db.flush()
|
||||
scenario.scripts.append(script)
|
||||
|
||||
@@ -6,11 +6,9 @@ frames, so the format lives here rather than in either one.
|
||||
"""
|
||||
import json
|
||||
|
||||
from . import analytics
|
||||
|
||||
# `no-cache` stops an intermediary from caching the stream. `X-Accel-Buffering`
|
||||
# makes nginx-style reverse proxies, which hosted deploys use, flush each event
|
||||
# immediately rather than buffer it.
|
||||
# makes an nginx-style reverse proxy flush each event immediately rather than
|
||||
# buffer it, which matters if anyone puts one in front of the app.
|
||||
SSE_HEADERS = {"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}
|
||||
|
||||
|
||||
@@ -20,11 +18,9 @@ def sse(obj: dict) -> str:
|
||||
|
||||
|
||||
def turn_error(detail: str, **extra) -> str:
|
||||
"""Returns an SSE error for a turn that could not be produced, and counts it.
|
||||
"""Returns an SSE error for a turn that could not be produced.
|
||||
|
||||
A failed turn is still an HTTP 200 response, so the middleware's status-code
|
||||
tally cannot see it. This metric exists so that a demo whose model refuses
|
||||
every request does not report as healthy.
|
||||
A failed turn is still an HTTP 200 response, because the error is reported
|
||||
inside the stream the client is already reading.
|
||||
"""
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_TURN_ERROR)
|
||||
return sse({"type": "error", "detail": detail, **extra})
|
||||
|
||||
@@ -19,10 +19,8 @@ annotated-doc==0.0.5
|
||||
annotated-types==0.8.0
|
||||
anyio==4.14.2
|
||||
certifi==2026.7.22
|
||||
cffi==2.1.1
|
||||
charset-normalizer==3.5.1
|
||||
click==8.5.0
|
||||
cryptography==50.0.1
|
||||
fastapi==0.141.1
|
||||
greenlet==3.5.5
|
||||
h11==0.16.0
|
||||
@@ -33,16 +31,12 @@ idna==3.19
|
||||
iniconfig==2.3.0
|
||||
packaging==26.3
|
||||
pluggy==1.6.0
|
||||
psycopg==3.3.5
|
||||
psycopg-binary==3.3.5
|
||||
pycparser==3.0
|
||||
pydantic==2.13.5
|
||||
pydantic_core==2.46.5
|
||||
Pygments==2.21.0
|
||||
pytest==9.1.1
|
||||
python-dotenv==1.2.3
|
||||
PyYAML==6.0.3
|
||||
quickjs==1.19.4
|
||||
regex==2026.9.3
|
||||
requests==2.34.2
|
||||
SQLAlchemy==2.0.52
|
||||
|
||||
@@ -8,6 +8,3 @@ httpx>=0.27
|
||||
# code imports it by name.
|
||||
certifi
|
||||
tiktoken>=0.7
|
||||
quickjs>=1.19
|
||||
cryptography>=42
|
||||
psycopg[binary]>=3.2
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""The storyteller, run as a real OS process for `test_process_restart.py`.
|
||||
|
||||
Not a test module, and named so pytest does not collect it: it is the program
|
||||
the test starts, twice, against one database file.
|
||||
|
||||
The model is replaced with a deterministic fake before the app is imported, so
|
||||
the process needs no Ollama, no network and no configuration. Everything else —
|
||||
the engine, the migrations, the routers, the session lifecycle — is the real
|
||||
application, which is the whole point of spawning a process at all.
|
||||
|
||||
python _restart_server.py <db_path> <port>
|
||||
"""
|
||||
import itertools
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE.parent)) # backend/, so `app` imports
|
||||
sys.path.insert(0, str(HERE)) # tests/, so `fakes` imports
|
||||
|
||||
db_path, port = sys.argv[1], int(sys.argv[2])
|
||||
os.environ["AIDND_DB_PATH"] = db_path
|
||||
# A developer's shell may point these at Postgres, and `app.database` prefers
|
||||
# either over the SQLite path. The suite's conftest clears them for the same
|
||||
# reason; a spawned process does not inherit that, so clear them here too.
|
||||
os.environ.pop("AIDND_DATABASE_URL", None)
|
||||
os.environ.pop("DATABASE_URL", None)
|
||||
|
||||
from fakes import GOLD_PER_TURN, gold_reply # noqa: E402
|
||||
|
||||
_turn = itertools.count(1)
|
||||
|
||||
|
||||
class DeterministicProvider:
|
||||
"""Banks one turn's worth of gold per reply, numbered so text is checkable.
|
||||
|
||||
The same instrumentation `test_head_cursor.py` and `test_save_points.py`
|
||||
use, for the same reason: it makes "the state at this position" a number the
|
||||
test can assert rather than a paragraph it has to interpret. M5 replaces the
|
||||
machinery underneath; what this measures is where the story is being read.
|
||||
"""
|
||||
|
||||
last_usage = None
|
||||
|
||||
def __init__(self, *a, **k):
|
||||
pass
|
||||
|
||||
async def generate(self, parts, *, temperature, max_tokens):
|
||||
yield ("text", gold_reply(f"Beat {next(_turn)}."))
|
||||
|
||||
|
||||
from app.routers.adventures import turns # noqa: E402
|
||||
|
||||
turns.OpenAICompatibleProvider = DeterministicProvider
|
||||
|
||||
from app.main import app # noqa: E402
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
|
||||
# Loopback only, as every supported start path does.
|
||||
uvicorn.run(app, host="127.0.0.1", port=port, log_level="warning")
|
||||
@@ -39,3 +39,44 @@ class ScriptedProvider:
|
||||
if isinstance(reply, Exception):
|
||||
raise reply
|
||||
yield ("text", reply)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Deterministic per-turn state instrumentation
|
||||
# ---------------------------------------------------------------------------
|
||||
# Several tests need a value that changes by a fixed amount on every turn, so
|
||||
# that a rollback failure is arithmetic rather than a judgement call: if a take
|
||||
# stacks instead of replacing, the total is off by exactly one turn's worth.
|
||||
#
|
||||
# That instrumentation used to be a JavaScript `output` hook doing
|
||||
# `state.gold += 10` in the QuickJS sandbox. M2 removed campaign scripting, and
|
||||
# the tests below are not about scripting — they are about the state snapshot,
|
||||
# rollback, and branch-isolation machinery in `attempts.py` and `tree.py`,
|
||||
# which is unchanged.
|
||||
#
|
||||
# The counter therefore moved to the world-state engine, which is a real
|
||||
# remaining product path: the model emits a ```state delta block, the referee
|
||||
# applies it, and the result lands in `adventure.world_state`. The fake
|
||||
# provider decides what the model "emits", so it is exactly as deterministic as
|
||||
# the script was, and it exercises production code rather than a test hook.
|
||||
|
||||
#: A schema with a plain unbounded counter. No `max_delta_per_turn` and no
|
||||
#: `cooldown`, so every +10 is applied in full, every turn.
|
||||
GOLD_SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
GOLD_PER_TURN = 10
|
||||
|
||||
|
||||
def gold_reply(text: str, amount: int = GOLD_PER_TURN) -> str:
|
||||
"""A model reply that narrates `text` and banks `amount` gold."""
|
||||
return f'{text}\n```state\n{{"player.gold": {amount}}}\n```'
|
||||
|
||||
|
||||
def gold_replies(prefix: str = "Take", count: int = 40) -> list[str]:
|
||||
"""`count` numbered replies, each banking one turn's worth of gold."""
|
||||
return [gold_reply(f"{prefix} {n}.") for n in range(1, count + 1)]
|
||||
|
||||
@@ -1,208 +0,0 @@
|
||||
"""The access log: app/accesslog.py and GET /api/analytics/access.
|
||||
|
||||
This is the half of the analytics work that identifies people on purpose,
|
||||
so these tests pin the details that would quietly make it wrong. The
|
||||
address recorded must be the hardened one, not a header a client chose.
|
||||
Session rows must be thinned instead of written on every page load. And a
|
||||
row must outlive the account it describes, because guest cleanup deletes
|
||||
accounts on a schedule, and a log that deletes itself is not a log.
|
||||
|
||||
python -m pytest tests/test_accesslog.py -v
|
||||
"""
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import accesslog, auth, limits, models, security
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
|
||||
EDGE = "198.51.100.77" # what the trusted proxy appended
|
||||
SPOOF = "10.0.0.1" # what a client put in front of it
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_state():
|
||||
accesslog._last_session.clear()
|
||||
yield
|
||||
accesslog._last_session.clear()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
owner = models.User(is_guest=False, email="owner@example.com")
|
||||
member = models.User(
|
||||
is_guest=False, email="player@example.com",
|
||||
password_hash=security.hash_password("hunter2long"),
|
||||
)
|
||||
setup.add_all([owner, member])
|
||||
setup.commit()
|
||||
ids = {"owner": owner.id, "member": member.id}
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_login_allowed", lambda *a, **k: None)
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
|
||||
|
||||
# /auth/me resolves its own session, so the cookie flow below is the real
|
||||
# one. Every other endpoint goes through get_current_user, and `act_as`
|
||||
# decides who that is.
|
||||
acting = {"id": ids["owner"]}
|
||||
|
||||
def _current(db=Depends(get_db)):
|
||||
return db.get(models.User, acting["id"])
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current
|
||||
try:
|
||||
test_client = TestClient(app)
|
||||
test_client.ids = ids
|
||||
test_client.act_as = lambda user_id: acting.update(id=user_id)
|
||||
yield test_client
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def visit(client, ip=EDGE, ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64)"):
|
||||
return client.get(
|
||||
"/api/auth/me",
|
||||
headers={"x-forwarded-for": f"{SPOOF}, {ip}", "user-agent": ua},
|
||||
)
|
||||
|
||||
|
||||
def rows(kind=None):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
query = db.query(models.AccessEvent).order_by(models.AccessEvent.id)
|
||||
if kind:
|
||||
query = query.filter_by(kind=kind)
|
||||
return query.all()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def read_log(client, **params):
|
||||
return client.get("/api/analytics/access", params=params)
|
||||
|
||||
|
||||
# ---------- Writing ----------
|
||||
|
||||
def test_a_new_session_is_logged(client):
|
||||
visit(client)
|
||||
logged = rows()
|
||||
assert len(logged) == 1
|
||||
entry = logged[0]
|
||||
assert entry.kind == accesslog.SESSION
|
||||
assert entry.is_guest and entry.who.startswith("Guest #")
|
||||
assert entry.device == "desktop"
|
||||
|
||||
|
||||
def test_the_address_is_the_hardened_one_not_the_clients(client):
|
||||
visit(client)
|
||||
# The client prepended its own value. Only the hop the edge appended counts.
|
||||
# Recording the leftmost value would make every row forgeable, which is
|
||||
# worse for a log than having no log at all.
|
||||
assert rows()[0].ip == EDGE
|
||||
|
||||
|
||||
def test_session_rows_are_thinned_to_one_per_day_per_address(client):
|
||||
for _ in range(4):
|
||||
visit(client)
|
||||
assert len(rows(accesslog.SESSION)) == 1
|
||||
|
||||
|
||||
def test_a_changed_address_writes_a_new_row(client):
|
||||
visit(client)
|
||||
visit(client, ip="203.0.113.9")
|
||||
logged = rows(accesslog.SESSION)
|
||||
assert [entry.ip for entry in logged] == [EDGE, "203.0.113.9"]
|
||||
# Same session throughout, so both rows name the same visitor.
|
||||
assert logged[0].who == logged[1].who
|
||||
|
||||
|
||||
def test_sign_in_and_failure_are_both_logged(client):
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "wrong"},
|
||||
headers={"x-forwarded-for": EDGE})
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
|
||||
headers={"x-forwarded-for": EDGE})
|
||||
kinds = [entry.kind for entry in rows()]
|
||||
assert accesslog.LOGIN_FAILED in kinds and accesslog.LOGIN in kinds
|
||||
|
||||
failure = rows(accesslog.LOGIN_FAILED)[0]
|
||||
# This records the address that was tried, not the account it belongs to.
|
||||
# A failed attempt against an address with no matching account is
|
||||
# exactly what this row exists to capture.
|
||||
assert failure.who == "player@example.com"
|
||||
assert failure.user_id is None
|
||||
assert rows(accesslog.LOGIN)[0].user_id == client.ids["member"]
|
||||
|
||||
|
||||
def test_registering_is_logged_against_the_upgraded_account(client):
|
||||
visit(client) # creates the guest whose session then registers
|
||||
client.act_as(rows()[0].user_id)
|
||||
client.post("/api/auth/register", json={"email": "new@example.com", "password": "hunter2long"})
|
||||
entry = rows(accesslog.REGISTER)[0]
|
||||
assert entry.who == "new@example.com" and not entry.is_guest
|
||||
|
||||
|
||||
def test_a_row_outlives_the_account_it_describes(client):
|
||||
visit(client)
|
||||
entry = rows()[0]
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.delete(db.get(models.User, entry.user_id))
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
# There is no foreign key, and `who` is a snapshot. Guest cleanup deletes
|
||||
# accounts on a schedule, and a log that vanishes along with them is not
|
||||
# a log.
|
||||
survivor = rows()[0]
|
||||
assert survivor.who == entry.who and survivor.ip == EDGE
|
||||
|
||||
|
||||
def test_a_long_user_agent_is_truncated(client):
|
||||
visit(client, ua="Mozilla/" + "x" * 500)
|
||||
assert len(rows()[0].user_agent) == accesslog.MAX_UA
|
||||
|
||||
|
||||
def test_a_logging_failure_does_not_break_the_request(client, monkeypatch):
|
||||
monkeypatch.setattr(accesslog, "_client_ip", lambda request: 1 / 0)
|
||||
# The log observes sign-in. A logging failure must not block the request.
|
||||
assert visit(client).status_code == 200
|
||||
|
||||
|
||||
# ---------- Reading ----------
|
||||
|
||||
def test_the_log_is_invisible_to_everyone_but_the_owner(client):
|
||||
visit(client)
|
||||
assert read_log(client).status_code == 200
|
||||
client.act_as(client.ids["member"])
|
||||
assert read_log(client).status_code == 404
|
||||
|
||||
|
||||
def test_the_log_reads_newest_first_and_pages_backwards(client):
|
||||
for index in range(5):
|
||||
visit(client, ip=f"203.0.113.{index}")
|
||||
first = read_log(client, limit=2).json()
|
||||
assert [event["ip"] for event in first["events"]] == ["203.0.113.4", "203.0.113.3"]
|
||||
assert first["has_more"]
|
||||
|
||||
older = read_log(client, limit=2, before_id=first["events"][-1]["id"]).json()
|
||||
assert [event["ip"] for event in older["events"]] == ["203.0.113.2", "203.0.113.1"]
|
||||
|
||||
|
||||
def test_the_log_filters_by_kind_and_searches(client):
|
||||
visit(client)
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
|
||||
headers={"x-forwarded-for": "203.0.113.44"})
|
||||
|
||||
assert len(read_log(client, kind="login").json()["events"]) == 1
|
||||
by_email = read_log(client, q="player@example.com").json()["events"]
|
||||
assert len(by_email) == 1 and by_email[0]["kind"] == "login"
|
||||
by_ip = read_log(client, q="203.0.113.44").json()["events"]
|
||||
assert len(by_ip) == 1
|
||||
assert read_log(client, q="nobody@example.com").json()["events"] == []
|
||||
@@ -48,7 +48,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -1,385 +0,0 @@
|
||||
"""Visit analytics: app/analytics.py and the two endpoints in front of it.
|
||||
|
||||
This file tests three things, and the rest is arithmetic. The counters must
|
||||
survive the buffer/UPSERT round trip: a flush adds to what is already
|
||||
stored instead of replacing it, or every number would show only the last
|
||||
minute. The funnel counts people rather than clicks, which is the only
|
||||
reason the visitor-day table exists. The gate holds: a stranger cannot read
|
||||
the dashboard, and cannot inflate what it reports beyond hitting the page.
|
||||
|
||||
python -m pytest tests/test_analytics.py -v
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import analytics, auth, limits, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_buffer():
|
||||
"""The buffer is process-wide, so a test that leaves counts in it would
|
||||
show up inside the next one's flush."""
|
||||
analytics._counts.clear()
|
||||
analytics._visits.clear()
|
||||
analytics._labels_seen.clear()
|
||||
yield
|
||||
analytics._counts.clear()
|
||||
analytics._visits.clear()
|
||||
analytics._labels_seen.clear()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db():
|
||||
Base.metadata.create_all(bind=engine)
|
||||
session = SessionLocal()
|
||||
try:
|
||||
yield session
|
||||
finally:
|
||||
session.close()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def counter(db, metric, label):
|
||||
row = (
|
||||
db.query(models.AnalyticsDaily)
|
||||
.filter_by(metric=metric, label=label)
|
||||
.one_or_none()
|
||||
)
|
||||
return row.hits if row else 0
|
||||
|
||||
|
||||
def make_user(db, email=None):
|
||||
user = models.User(is_guest=email is None, email=email)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
return user
|
||||
|
||||
|
||||
# ---------- The buffer and its flush ----------
|
||||
|
||||
def test_counts_accumulate_across_flushes(db):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.flush(db)
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.flush(db)
|
||||
# The second flush has to find the existing row and add to it. Replacing it
|
||||
# would leave every counter showing only the newest minute of traffic.
|
||||
assert counter(db, analytics.M_PAGE, "/") == 3
|
||||
|
||||
|
||||
def test_flush_is_a_no_op_when_nothing_happened(db):
|
||||
analytics.flush(db)
|
||||
assert db.query(models.AnalyticsDaily).count() == 0
|
||||
|
||||
|
||||
def test_a_failed_flush_keeps_the_counts(db, monkeypatch):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
monkeypatch.setattr(analytics, "_write_counts", lambda *a: 1 / 0)
|
||||
analytics.flush(db) # must not raise
|
||||
monkeypatch.undo()
|
||||
analytics.flush(db)
|
||||
assert counter(db, analytics.M_PAGE, "/") == 1
|
||||
|
||||
|
||||
def test_label_cardinality_is_capped(db):
|
||||
for i in range(analytics.MAX_LABELS_PER_METRIC + 25):
|
||||
analytics.record(analytics.M_REFERRER, f"host{i}.example")
|
||||
analytics.flush(db)
|
||||
labels = db.query(models.AnalyticsDaily).filter_by(metric=analytics.M_REFERRER).count()
|
||||
# Everything past the cap is folded into one bucket, so a referrer flood
|
||||
# cannot create unlimited rows.
|
||||
assert labels == analytics.MAX_LABELS_PER_METRIC + 1
|
||||
assert counter(db, analytics.M_REFERRER, analytics.OTHER) == 25
|
||||
|
||||
|
||||
# ---------- Visitors ----------
|
||||
|
||||
def test_visitor_id_is_stable_and_keyed(db, monkeypatch):
|
||||
user = make_user(db)
|
||||
handle = analytics.visitor_id(user)
|
||||
assert handle == analytics.visitor_id(user) # a returning visitor
|
||||
assert handle != analytics.visitor_id(make_user(db)) # is still one visitor
|
||||
assert len(handle) == 32 and int(handle, 16) >= 0 # opaque hex, not an id
|
||||
# Keyed on the app secret, not a bare hash of the user id. Otherwise
|
||||
# anyone holding this table could rebuild the mapping by hashing
|
||||
# sequential ids.
|
||||
monkeypatch.setattr(analytics.security, "SECRET_KEY", b"a-different-secret")
|
||||
assert analytics.visitor_id(user) != handle
|
||||
|
||||
|
||||
def test_a_repeat_visitor_is_new_only_once(db):
|
||||
user = make_user(db)
|
||||
analytics.record_visit(user)
|
||||
analytics.flush(db)
|
||||
rows = db.query(models.AnalyticsVisitorDay).all()
|
||||
assert len(rows) == 1 and rows[0].is_new
|
||||
|
||||
# Same visitor, a later day: seen before, so not new. The row is not
|
||||
# merged into the first day's row either.
|
||||
tomorrow = (models.utcnow().date() + timedelta(days=1)).isoformat()
|
||||
analytics._visits[(tomorrow, analytics.visitor_id(user))] = set()
|
||||
analytics.flush(db)
|
||||
rows = db.query(models.AnalyticsVisitorDay).order_by(models.AnalyticsVisitorDay.day).all()
|
||||
assert [row.is_new for row in rows] == [True, False]
|
||||
|
||||
|
||||
def test_one_row_per_visitor_per_day_however_much_they_do(db):
|
||||
user = make_user(db)
|
||||
for _ in range(5):
|
||||
analytics.record_event(analytics.EV_ADVENTURE, user)
|
||||
analytics.flush(db)
|
||||
assert db.query(models.AnalyticsVisitorDay).count() == 1
|
||||
assert counter(db, analytics.M_EVENT, analytics.EV_ADVENTURE) == 5
|
||||
|
||||
|
||||
def test_funnel_flags_only_ever_turn_on(db):
|
||||
user = make_user(db)
|
||||
analytics.record_event(analytics.EV_TURN, user)
|
||||
analytics.flush(db)
|
||||
# A later visit that reaches no funnel step must not clear the earlier one.
|
||||
analytics.record_visit(user)
|
||||
analytics.flush(db)
|
||||
row = db.query(models.AnalyticsVisitorDay).one()
|
||||
assert row.played and not row.created
|
||||
|
||||
|
||||
def test_purge_drops_only_rows_past_the_horizon(db):
|
||||
old = (models.utcnow().date() - timedelta(days=analytics.RETENTION_DAYS + 1)).isoformat()
|
||||
db.add(models.AnalyticsVisitorDay(day=old, visitor="a" * 32))
|
||||
db.add(models.AnalyticsVisitorDay(day=analytics._today(), visitor="b" * 32))
|
||||
db.commit()
|
||||
assert analytics.purge_old_visitor_days(db) == 1
|
||||
assert [r.visitor for r in db.query(models.AnalyticsVisitorDay)] == ["b" * 32]
|
||||
|
||||
|
||||
# ---------- Normalizing what a browser claims ----------
|
||||
|
||||
@pytest.mark.parametrize("path, expected", [
|
||||
("/", "/"),
|
||||
("/adventures", "/adventures"),
|
||||
("/adventures/", "/adventures"),
|
||||
("/play/12?x=1", "/play/:id"),
|
||||
("/scenarios/9#top", "/scenarios/:id"),
|
||||
("/wp-admin", "(other)"),
|
||||
("/play/../../etc", "(other)"),
|
||||
("", "/"),
|
||||
])
|
||||
def test_route_normalization(path, expected):
|
||||
assert analytics.normalize_route(path) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("referrer, expected", [
|
||||
("", "(direct)"),
|
||||
("https://news.ycombinator.com/item?id=1", "news.ycombinator.com"),
|
||||
("https://www.google.com/", "google.com"),
|
||||
("https://ai-dnd.example/scenarios", ""), # our own host: not a referral
|
||||
("javascript:alert(1)", "(other)"),
|
||||
("https://" + "x" * 200 + ".com", "(other)"),
|
||||
])
|
||||
def test_referrer_normalization(referrer, expected):
|
||||
assert analytics.normalize_referrer(referrer, "ai-dnd.example") == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ua, expected", [
|
||||
("Mozilla/5.0 (iPhone; CPU iPhone OS 17_0) AppleWebKit", "mobile"),
|
||||
("Mozilla/5.0 (iPad; CPU OS 17_0) AppleWebKit", "tablet"),
|
||||
("Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "desktop"),
|
||||
("Googlebot/2.1", "bot"),
|
||||
("", "(unknown)"),
|
||||
])
|
||||
def test_device_detection(ua, expected):
|
||||
assert analytics.device_of(ua) == expected
|
||||
|
||||
|
||||
def test_only_iso_looking_country_headers_are_trusted():
|
||||
assert analytics.country_of({"cf-ipcountry": "de"}) == "DE"
|
||||
assert analytics.country_of({"cf-ipcountry": "Norway"}) == analytics.UNKNOWN
|
||||
assert analytics.country_of({"cf-ipcountry": "XX"}) == analytics.UNKNOWN
|
||||
assert analytics.country_of({}) == analytics.UNKNOWN
|
||||
|
||||
|
||||
def test_error_labels_use_the_route_not_the_path():
|
||||
class Route:
|
||||
path = "/api/adventures/{adventure_id}"
|
||||
|
||||
assert analytics.api_route_label({"route": Route()}, 500) == "500 /api/adventures/{adventure_id}"
|
||||
# An unmatched path is entirely attacker-chosen, so it never becomes a label.
|
||||
assert analytics.api_route_label({}, 404) == "404 (unmatched)"
|
||||
|
||||
|
||||
# ---------- The summary ----------
|
||||
|
||||
def test_summary_counts_people_once_per_step(db):
|
||||
one, two = make_user(db), make_user(db)
|
||||
for _ in range(3):
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, one)
|
||||
analytics.record_event(analytics.EV_TURN, one)
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, two)
|
||||
|
||||
result = analytics.summary(db, days=7)
|
||||
steps = {row["step"]: row["count"] for row in result["funnel"]}
|
||||
assert steps["Visited"] == 2
|
||||
assert steps["Opened a scenario"] == 2
|
||||
assert steps["Played a turn"] == 1 # not 3, because one person made three turns
|
||||
assert steps["Signed up"] == 0
|
||||
# Raw event totals still count every occurrence.
|
||||
assert result["totals"]["turns"] == 3
|
||||
assert result["totals"]["visitors"] == 2
|
||||
|
||||
|
||||
def test_summary_series_covers_every_day_including_empty_ones(db):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
result = analytics.summary(db, days=7)
|
||||
assert len(result["series"]) == 7
|
||||
assert result["series"][-1]["day"] == models.utcnow().date().isoformat()
|
||||
assert result["series"][-1]["pageviews"] == 1
|
||||
assert result["series"][0]["pageviews"] == 0
|
||||
|
||||
|
||||
def test_summary_flushes_before_reading(db):
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_TURN)
|
||||
# Never flushed by hand: the dashboard must not be up to a minute stale.
|
||||
assert analytics.summary(db, days=1)["totals"]["turns"] == 1
|
||||
|
||||
|
||||
def test_summary_reports_pages_referrers_and_errors(db):
|
||||
analytics.record(analytics.M_PAGE, "/play/:id", n=4)
|
||||
analytics.record(analytics.M_REFERRER, "news.ycombinator.com", n=2)
|
||||
analytics.record(analytics.M_ERROR, "500 /api/adventures/{adventure_id}")
|
||||
result = analytics.summary(db, days=30)
|
||||
assert result["pages"][0] == {"label": "/play/:id", "hits": 4}
|
||||
assert result["referrers"][0]["label"] == "news.ycombinator.com"
|
||||
assert result["totals"]["errors"] == 1
|
||||
|
||||
|
||||
# ---------- The endpoints ----------
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
visitor = models.User(is_guest=True)
|
||||
owner = models.User(is_guest=False, email="owner@example.com")
|
||||
setup.add_all([visitor, owner])
|
||||
setup.commit()
|
||||
ids = {"visitor": visitor.id, "owner": owner.id}
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
# Multi-user is what makes the gate mean anything: local mode trusts
|
||||
# whoever is at the keyboard, because it is the operator's own machine.
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
|
||||
|
||||
current = {"id": ids["visitor"]}
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, current["id"])
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
monkeypatch.setattr(
|
||||
auth, "resolve_session_user", lambda request, db: db.get(models.User, current["id"])
|
||||
)
|
||||
try:
|
||||
client = TestClient(app)
|
||||
client.ids, client.current = ids, current
|
||||
yield client
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def read_summary(client, days=30):
|
||||
return client.get(f"/api/analytics/summary?days={days}")
|
||||
|
||||
|
||||
def test_dashboard_is_invisible_to_everyone_but_the_owner(client):
|
||||
assert read_summary(client).status_code == 404
|
||||
client.current["id"] = client.ids["owner"]
|
||||
assert read_summary(client).status_code == 200
|
||||
|
||||
|
||||
def test_collect_records_a_pageview_and_the_visit(client):
|
||||
resp = client.post("/api/analytics/collect", json={"path": "/play/7", "first": True,
|
||||
"referrer": "https://news.ycombinator.com/"})
|
||||
assert resp.status_code == 204
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert body["pages"][0] == {"label": "/play/:id", "hits": 1}
|
||||
assert body["referrers"][0]["label"] == "news.ycombinator.com"
|
||||
assert body["totals"]["visitors"] == 1
|
||||
|
||||
|
||||
def test_referrer_and_device_are_recorded_once_per_visit_not_per_view(client):
|
||||
for path in ("/", "/scenarios", "/adventures"):
|
||||
client.post("/api/analytics/collect", json={"path": path, "first": path == "/"})
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert body["totals"]["pageviews"] == 3
|
||||
# Three views, one visit: the referral and the device are facts about the
|
||||
# visit, so counting them per view would multiply every one of them.
|
||||
assert sum(row["hits"] for row in body["devices"]) == 1
|
||||
|
||||
|
||||
def test_the_owners_own_visits_are_not_traffic(client):
|
||||
client.current["id"] = client.ids["owner"]
|
||||
client.post("/api/analytics/collect", json={"path": "/", "first": True})
|
||||
assert read_summary(client).json()["totals"]["pageviews"] == 0
|
||||
|
||||
|
||||
def test_a_client_cannot_invent_pages_or_events(client):
|
||||
client.post("/api/analytics/collect", json={"path": "/../../admin", "first": True})
|
||||
# There is no field for it, so a made-up event is not even expressible.
|
||||
client.post("/api/analytics/collect", json={"path": "/", "event": "signup"})
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert {row["label"] for row in body["pages"]} == {"(other)", "/"}
|
||||
assert body["totals"]["signups"] == 0
|
||||
|
||||
|
||||
def test_api_errors_are_counted_by_route(client):
|
||||
client.get("/api/adventures/999999")
|
||||
client.current["id"] = client.ids["owner"]
|
||||
errors = read_summary(client).json()["errors"]
|
||||
assert errors and errors[0]["label"].startswith("404 /api/adventures/")
|
||||
|
||||
|
||||
# ---------- The dialect the tests never run on ----------
|
||||
|
||||
def test_the_upserts_compile_for_postgres():
|
||||
"""Prod runs on Neon, but these tests run on SQLite, and a failed flush
|
||||
is caught and logged instead of raised. A dialect mistake would
|
||||
therefore stay invisible until the dashboard quietly stayed empty. This
|
||||
test compiles both statements against Postgres without connecting to
|
||||
one.
|
||||
"""
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.dialects import postgresql
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
session = sessionmaker(bind=create_engine("postgresql+psycopg://u:p@localhost/db"))()
|
||||
compiled = []
|
||||
|
||||
def capture(statement, *args, **kwargs):
|
||||
compiled.append(str(statement.compile(dialect=postgresql.dialect())))
|
||||
|
||||
session.execute = capture
|
||||
session.scalars = lambda *a, **k: []
|
||||
|
||||
analytics._write_counts(session, {("2026-01-01", "pageview", "/"): 2})
|
||||
analytics._write_visits(session, {("2026-01-01", "f" * 32): {"played"}})
|
||||
|
||||
counts, visits = compiled
|
||||
assert "ON CONFLICT (day, metric, label) DO UPDATE" in counts
|
||||
assert "analytics_daily.hits + excluded.hits" in counts
|
||||
assert "ON CONFLICT (day, visitor) DO UPDATE" in visits
|
||||
assert "analytics_visitor_days.played OR excluded.played" in visits
|
||||
# is_new is settled by the first write of a visitor's first day and must
|
||||
# not be in the update clause at all.
|
||||
assert "is_new" not in visits.split("DO UPDATE")[1]
|
||||
@@ -19,17 +19,10 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -45,14 +38,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -61,9 +51,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -213,18 +200,40 @@ def test_the_assembled_prompt_is_stored_once_per_turn(client):
|
||||
assert len(moved) == 1 and moved != live_holder, "the prompt follows the story"
|
||||
|
||||
|
||||
# ------------------------------------------------------- removing the turn
|
||||
# ------------------------------------------------- stepping behind the turn
|
||||
|
||||
def test_undo_takes_every_attempt_with_it(client):
|
||||
def test_undo_hides_every_attempt_and_keeps_them_all(client):
|
||||
"""M3 rewrote this test. Undo used to delete the turn, and the assertion was
|
||||
that it took the whole sibling group with it rather than leaving orphaned
|
||||
attempts at a coordinate the story no longer reached.
|
||||
|
||||
The group still moves as one, but it moves out of the story rather than out
|
||||
of the database: one Undo steps behind the turn, so none of its three
|
||||
attempts is in what the story tells, and all three are still on disk for the
|
||||
Redo that walks back into them. The old assertion is kept as the second half
|
||||
— what the story reads — and the row count is the new first half.
|
||||
"""
|
||||
ScriptedProvider.replies = ["One.", "Two.", "Three."]
|
||||
_play(client)
|
||||
_retry(client)
|
||||
_retry(client)
|
||||
assert len([a for a in _rows(client.adv_id) if a.type == "ai"]) == 3
|
||||
before = _rows(client.adv_id)
|
||||
assert len([a for a in before if a.type == "ai"]) == 3
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
assert [a.type for a in _rows(client.adv_id)] == ["start"]
|
||||
# What the story tells: the opening, and none of the turn's attempts.
|
||||
assert [a["type"] for a in r.json()["actions"]] == ["start"]
|
||||
# What it holds: every row that was there before, attempts included.
|
||||
after = _rows(client.adv_id)
|
||||
assert len(after) == len(before)
|
||||
assert {a.id for a in after} == {a.id for a in before}
|
||||
assert len([a for a in after if a.type == "ai"]) == 3
|
||||
# And the group is reachable again, whole, with the same take live.
|
||||
live_before = [a.id for a in before if a.type == "ai" and a.live]
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/redo")
|
||||
assert r.status_code == 200, r.text
|
||||
assert [a.id for a in _rows(client.adv_id) if a.type == "ai" and a.live] == live_before
|
||||
|
||||
|
||||
def test_deleting_a_retried_turn_deletes_its_attempts(client):
|
||||
|
||||
@@ -30,7 +30,6 @@ from app import auth, limits, models, tree
|
||||
from app.context import history, lineage
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.scripting import ScriptPipeline
|
||||
from tools import dbmeter
|
||||
|
||||
|
||||
@@ -224,24 +223,11 @@ def test_an_already_loaded_collection_is_cut_down_to_the_path(forked):
|
||||
assert labels(history.tail(adventure, 3)) == ["B5", "C6", "C7"]
|
||||
|
||||
|
||||
def test_user_scripts_are_handed_the_path(forked):
|
||||
"""The same risk one layer up, in code visible to users:
|
||||
`pipeline._history()` is the documented scripting history API."""
|
||||
db, adventure, _ = forked
|
||||
list(adventure.actions) # the pipeline's caller has usually loaded these
|
||||
pipeline = ScriptPipeline(adventure, db)
|
||||
assert [h["text"] for h in pipeline._history()] == [
|
||||
"A0", "A1", "A2", "A3", "B4", "B5", "C6", "C7"
|
||||
]
|
||||
assert pipeline._info()["actionCount"] == 8
|
||||
|
||||
|
||||
# ------------------------------------------------------------ over the wire
|
||||
|
||||
@pytest.fixture()
|
||||
def client(forked, monkeypatch):
|
||||
db, adventure, ids = forked
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(session=Depends(get_db)):
|
||||
|
||||
@@ -22,7 +22,7 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# `hp` moves freely. `mana` has a cooldown of 2 turns, so an incorrect
|
||||
# advance shows up as a change the referee should have rejected.
|
||||
@@ -30,16 +30,13 @@ SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"mana": {"min": 0, "max": 50, "initial": 50, "cooldown": 2},
|
||||
# The per-turn counter these tests measure rollbacks with. Unbounded and
|
||||
# uncapped on purpose, so every turn's +10 lands in full. See
|
||||
# `fakes.gold_reply`.
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -55,14 +52,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100, "mana": 50}},
|
||||
world_state={"player": {"hp": 100, "mana": 50, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -71,9 +65,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -121,7 +112,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -322,11 +313,13 @@ def test_forking_a_live_node_on_another_branch_is_refused(client):
|
||||
|
||||
# -------------------------------------------------------------- the state
|
||||
|
||||
def test_switching_restores_the_script_and_world_state(client):
|
||||
def test_switching_restores_the_state_a_branch_left_behind(client):
|
||||
# Two stats move: hp differs per attempt, and gold counts turns. Between
|
||||
# them, a switch that restored the wrong snapshot is visible either way.
|
||||
ScriptedProvider.replies = [
|
||||
"A scratch.\n```state\n{\"player.hp\": -5}\n```",
|
||||
"A beating.\n```state\n{\"player.hp\": -40}\n```",
|
||||
"Onward.",
|
||||
'A scratch.\n```state\n{"player.hp": -5, "player.gold": 10}\n```',
|
||||
'A beating.\n```state\n{"player.hp": -40, "player.gold": 10}\n```',
|
||||
gold_reply("Onward."),
|
||||
]
|
||||
_play(client)
|
||||
_retry(client)
|
||||
@@ -336,9 +329,9 @@ def test_switching_restores_the_script_and_world_state(client):
|
||||
|
||||
discarded = [a.id for a in _rows(client.adv_id) if a.type == "ai" and not a.live][0]
|
||||
_fork(client, discarded)
|
||||
script_state, world_state = _state(client.adv_id)
|
||||
player, world_state = _state(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 95, "the attempt this branch tells"
|
||||
assert script_state == {"gold": 10}, "one turn of gold, not three"
|
||||
assert player["gold"] == 10, "one turn of gold, not three"
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/branches/{parent}/switch")
|
||||
assert _state(client.adv_id) == on_parent
|
||||
@@ -439,26 +432,41 @@ def test_a_memory_on_the_line_left_behind_is_out_of_range_on_the_fork(client):
|
||||
|
||||
# ------------------------------------------------------------------- undo
|
||||
|
||||
def test_undo_stops_at_the_fork(client):
|
||||
"""Undoing a turn on a fork must never reach into the branch it forked
|
||||
from. Those turns belong to that branch's story too."""
|
||||
def test_undo_walks_off_a_fork_into_the_story_it_inherits(client):
|
||||
"""M3 rewrote this test, and reversed half of it.
|
||||
|
||||
Undo used to refuse at a fork point, and it had to: it deleted the turns it
|
||||
stepped over, and the turns before the fork belong to the parent branch's
|
||||
story as well. Refusing was the only way to stop one branch's Undo from
|
||||
removing rows another branch was reading.
|
||||
|
||||
Nothing is deleted now, so there is nothing to protect the parent from. A
|
||||
forked branch inherits the story up to its fork, that inherited story is
|
||||
part of what this branch tells, and Undo walks back through it like any
|
||||
other retained history. The floor is the campaign opening, not the fork.
|
||||
"""
|
||||
discarded = _divergent_story(client)
|
||||
_fork(client, discarded)
|
||||
rows_before = len(_rows(client.adv_id))
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
# The promoted attempt is removed, and the player action before it
|
||||
# stays, because that action belongs to the parent and the parent
|
||||
# still has it.
|
||||
assert len(_rows(client.adv_id)) == rows_before - 1
|
||||
assert _texts(client) == ["You enter a cave.", "> You look around."]
|
||||
# One Undo steps over a whole turn, so it takes the player's action with the
|
||||
# reply to it — and that player action is the parent's row, sitting in front
|
||||
# of the fork. Stepping behind it is a read moving backwards, not a branch
|
||||
# reaching into another branch's rows: nothing moved either way.
|
||||
assert len(_rows(client.adv_id)) == rows_before
|
||||
assert _texts(client) == ["You enter a cave."]
|
||||
|
||||
# Nothing is left of this branch's own turns, so undo must refuse
|
||||
# instead of removing the parent's turns.
|
||||
# The opening is the floor, and it is the parent's node too.
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 400
|
||||
assert "forked from" in r.json()["detail"]
|
||||
assert "Nothing to undo" in r.json()["detail"]
|
||||
|
||||
# Redo walks back out to where the fork was left, taking the turn whole.
|
||||
assert client.post(f"/api/adventures/{client.adv_id}/redo").status_code == 200
|
||||
assert _texts(client) == ["You enter a cave.", "> You look around.", "Attempt one."]
|
||||
assert len(_rows(client.adv_id)) == rows_before
|
||||
|
||||
|
||||
# ----------------------------------------------------------- the tree view
|
||||
|
||||
@@ -55,9 +55,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two.", "Next turn."]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -33,19 +33,12 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn, so the stored gold total tells how many turns the
|
||||
# story behind it played. This makes an after-snapshot visible from outside.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
OPENING = "You enter a cave."
|
||||
|
||||
@@ -63,14 +56,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text=OPENING))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -78,9 +68,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = ["A reply."]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -186,7 +173,7 @@ def _branch_rows(adv_id) -> list[models.Branch]:
|
||||
def _script_state(adv_id) -> dict:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return db.get(models.Adventure, adv_id).script_state
|
||||
return (db.get(models.Adventure, adv_id).world_state or {}).get("player", {})
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -272,7 +259,7 @@ def test_a_switch_in_the_copy_restores_what_that_branch_left_behind(client):
|
||||
# Play one more turn on the fork, so the two tips end up at
|
||||
# genuinely different totals. Turn for turn, both branches earn the
|
||||
# same gold, so a switch that restored nothing would still look right.
|
||||
ScriptedProvider.replies = ["Further still."]
|
||||
ScriptedProvider.replies = [gold_reply("Further still.")]
|
||||
_play(client, original, "press on")
|
||||
|
||||
per_branch = []
|
||||
@@ -398,7 +385,6 @@ def test_a_fork_with_no_depth_is_refused(client):
|
||||
|
||||
|
||||
def test_more_branches_than_the_cap_is_refused(client, monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
payload = {
|
||||
"format": bundle.FORMAT, "title": "Too many",
|
||||
"branches": [{"parent": None, "forkDepth": None}]
|
||||
@@ -558,7 +544,6 @@ def test_the_action_cap_counts_the_rows_a_v1_file_expands_into(client, monkeypat
|
||||
multiple of it. The body-size limit does not help here: the text is
|
||||
tiny, and the row count is the actual cost.
|
||||
"""
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(limits, "MAX_ACTIONS_PER_ADVENTURE", 6)
|
||||
monkeypatch.setattr(limits, "_BUNDLE_LIST_CAPS",
|
||||
{**limits._BUNDLE_LIST_CAPS, "actions": 6})
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
"""HTTP tests for the AI Chat scratchpad (power users only).
|
||||
"""HTTP tests for the AI Chat scratchpad.
|
||||
|
||||
Covers the access gate, the streamed reply, and the demo-key model pinning.
|
||||
This pinning must not let a public visitor reach paid models through this
|
||||
page.
|
||||
Most of this file used to be about the shared demo key: an access gate on a
|
||||
"power user" email allowlist, and a pinning rule that stopped a public visitor
|
||||
reaching paid models on a server-funded key. M2 removed the hosted deployment
|
||||
those defended, so the rules they tested no longer exist to be tested. See
|
||||
`planning/archive/milestone-reports/M2-*` for the accounting.
|
||||
|
||||
What remains is what the page still does: stream a reply from the configured
|
||||
model, honour a system prompt and a per-request model override, and refuse a
|
||||
conversation too large to send.
|
||||
|
||||
python -m pytest tests/test_chat.py -v
|
||||
"""
|
||||
@@ -10,25 +16,27 @@ import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, limits, models
|
||||
from app import auth, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import chat
|
||||
|
||||
|
||||
class FakeProvider:
|
||||
"""Records what it was constructed with, then streams a fixed reply. Stands
|
||||
in for the real egress point, so asserting on last_key/last_model is
|
||||
asserting on exactly what would have gone over the wire."""
|
||||
"""Records what it was constructed with, then streams a fixed reply.
|
||||
|
||||
It stands in for the real egress point, so asserting on `last_endpoint` and
|
||||
`last_model` is asserting on exactly what would have gone over the wire.
|
||||
There is no `last_key` any more: the provider takes no API key, because
|
||||
Ollama does not use one.
|
||||
"""
|
||||
last_usage = None
|
||||
last_model = None
|
||||
last_key = None
|
||||
last_endpoint = None
|
||||
last_messages = None
|
||||
|
||||
def __init__(self, endpoint_url, api_key, model, api_mode="chat", reasoning_max_tokens=0):
|
||||
def __init__(self, endpoint_url, model, api_mode="chat", read_timeout=None):
|
||||
FakeProvider.last_model = model
|
||||
FakeProvider.last_key = api_key
|
||||
FakeProvider.last_endpoint = endpoint_url
|
||||
|
||||
async def chat(self, messages, *, temperature, max_tokens):
|
||||
@@ -41,68 +49,35 @@ class FakeProvider:
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False, email="power@example.com")
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(chat, "OpenAICompatibleProvider", FakeProvider)
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
# Multi-user mode is what makes the power-user gate meaningful, because
|
||||
# local mode trusts everyone. The allowlist is set per test.
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "POWER_USERS", {"power@example.com"})
|
||||
# These tests deliberately do not stub resolve_provider_config. The
|
||||
# point is to exercise the real BYOK-vs-demo decision, since that
|
||||
# decision is what keeps the shared key off paid models. Each test
|
||||
# picks a mode with _byok/_demo below.
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield TestClient(app)
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def _send(client, **extra):
|
||||
return client.post("/api/chat/stream", json={"messages": [{"role": "user", "content": "hi"}], **extra})
|
||||
def _send(client, **body):
|
||||
payload = {"messages": [{"role": "user", "content": "hi"}]}
|
||||
payload.update(body)
|
||||
return client.post("/api/chat/stream", json=payload)
|
||||
|
||||
|
||||
def _byok(monkeypatch):
|
||||
"""The user brought their own key: no demo key in play, any model allowed."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: False)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.api_key = "sk-my-own-key" # legacy-plaintext path: used as-is
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _demo(monkeypatch, whitelist=("free/allowed",)):
|
||||
"""The user has no key, so turns run on the server-funded demo key."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", list(whitelist))
|
||||
|
||||
|
||||
def test_non_power_user_gets_404(client, monkeypatch):
|
||||
monkeypatch.setattr(auth, "POWER_USERS", set())
|
||||
assert _send(client).status_code == 404
|
||||
assert client.get("/api/chat/config").status_code == 404
|
||||
|
||||
|
||||
def test_power_user_streams_a_reply(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_the_page_streams_a_reply(client):
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert '"type": "reasoning"' in resp.text
|
||||
@@ -111,136 +86,39 @@ def test_power_user_streams_a_reply(client, monkeypatch):
|
||||
assert FakeProvider.last_messages == [{"role": "user", "content": "hi"}]
|
||||
|
||||
|
||||
def test_system_prompt_and_model_override_are_honoured(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_it_uses_the_configured_endpoint_and_model(client):
|
||||
_send(client)
|
||||
assert FakeProvider.last_model == "test-model"
|
||||
# The default from `models.Settings`, and the only kind of address the
|
||||
# endpoint policy allows without configuration.
|
||||
assert FakeProvider.last_endpoint == "http://localhost:11434/v1"
|
||||
|
||||
|
||||
def test_system_prompt_and_model_override_are_honoured(client):
|
||||
resp = client.post("/api/chat/stream", json={
|
||||
"messages": [
|
||||
{"role": "system", "content": "Be terse."},
|
||||
{"role": "user", "content": "hi"},
|
||||
],
|
||||
"model": "some/other-model",
|
||||
"model": "some-other-model",
|
||||
})
|
||||
assert resp.status_code == 200, resp.text
|
||||
# BYOK: any model the user names is passed straight through, on their key.
|
||||
assert FakeProvider.last_model == "some/other-model"
|
||||
assert FakeProvider.last_key == "sk-my-own-key"
|
||||
assert FakeProvider.last_model == "some-other-model"
|
||||
assert FakeProvider.last_messages[0] == {"role": "system", "content": "Be terse."}
|
||||
|
||||
|
||||
def test_demo_key_pins_model_to_whitelist(client, monkeypatch):
|
||||
_demo(monkeypatch)
|
||||
resp = _send(client, model="expensive/paid-model")
|
||||
assert resp.status_code == 200, resp.text
|
||||
# Refused visibly: the whitelisted model runs instead, with a note. The
|
||||
# paid slug must never reach the wire alongside the server-funded key.
|
||||
assert FakeProvider.last_model == "free/allowed"
|
||||
assert FakeProvider.last_key == "demo-key"
|
||||
assert '"type": "note"' in resp.text
|
||||
|
||||
# A whitelisted model is still selectable on the demo key.
|
||||
_demo(monkeypatch, ["free/allowed", "free/second"])
|
||||
resp = _send(client, model="free/second")
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "free/second"
|
||||
|
||||
|
||||
def test_demo_key_ignores_an_off_whitelist_settings_model(client, monkeypatch):
|
||||
"""The override is not the only untrusted input. `Settings.model` is
|
||||
also user-set, and it must be pinned the same way when there is no
|
||||
BYOK key."""
|
||||
_demo(monkeypatch)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().model = "expensive/paid-model"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "free/allowed"
|
||||
|
||||
|
||||
def test_demo_key_endpoint_cannot_be_redirected(client, monkeypatch):
|
||||
"""A user-controlled `endpoint_url` would leak the key itself, which is
|
||||
worse than spending it. The demo branch pins the URL too."""
|
||||
_demo(monkeypatch)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().endpoint_url = "http://attacker.example/v1"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
assert _send(client).status_code == 200
|
||||
assert FakeProvider.last_endpoint == "http://demo"
|
||||
assert FakeProvider.last_key == "demo-key"
|
||||
|
||||
|
||||
def test_provider_config_refuses_server_funded_paid_model(monkeypatch):
|
||||
"""The structural backstop: a hand-built config (a future code path that
|
||||
forgets to go through resolve_provider_config) cannot run a
|
||||
server-funded turn on an off-whitelist model."""
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
with pytest.raises(ValueError):
|
||||
auth.ProviderConfig("http://demo", "demo-key", "expensive/paid-model", True)
|
||||
auth.ProviderConfig("http://demo", "demo-key", "free/allowed", True) # whitelisted: fine
|
||||
# The user's own key with any model stays fine.
|
||||
auth.ProviderConfig("http://any", "sk-mine", "expensive/paid-model", False)
|
||||
|
||||
|
||||
def test_byok_user_may_reuse_the_demo_keys_value(client, monkeypatch):
|
||||
"""Regression: the demo key is just an OpenRouter key, so a user can paste
|
||||
that same value into their own Settings. That is still BYOK, because
|
||||
the user is paying, and it must not trip the guard. It used to raise
|
||||
on every resolution, which returned a 500 from `GET /auth/me` and
|
||||
broke the entire SPA (no nav, no chat)."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "shared-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
def test_a_request_with_no_model_anywhere_is_refused(client):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.api_key = "shared-key" # same value, but supplied by the user
|
||||
settings.model = "expensive/paid-model" # their spend, their choice
|
||||
settings.model = ""
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
assert client.get("/api/auth/me").status_code == 200
|
||||
assert client.get("/api/chat/config").status_code == 200
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "expensive/paid-model"
|
||||
assert FakeProvider.last_key == "shared-key"
|
||||
assert _send(client).status_code == 400
|
||||
|
||||
|
||||
def test_resolve_provider_config_is_the_single_choke_point(monkeypatch):
|
||||
"""Turns, AI Chat, and the connection test all resolve through this one
|
||||
function, so pinning it here pins every caller. No DB or HTTP needed."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
|
||||
# No key of their own: both endpoint and model are pinned, regardless
|
||||
# of what they set.
|
||||
no_key = models.Settings(endpoint_url="http://mine/v1", api_key="", model="expensive/paid")
|
||||
assert auth.resolve_provider_config(no_key) == auth.ProviderConfig(
|
||||
"http://demo", "demo-key", "free/allowed", True)
|
||||
assert auth.resolve_provider_config(
|
||||
no_key, model_override="expensive/paid").model == "free/allowed"
|
||||
assert auth.resolve_provider_config(
|
||||
no_key, model_override="free/allowed").model == "free/allowed"
|
||||
|
||||
# Their own key: their endpoint, their key, their choice of model.
|
||||
byok = models.Settings(endpoint_url="http://mine/v1", api_key="sk-mine", model="expensive/paid")
|
||||
assert auth.resolve_provider_config(byok) == auth.ProviderConfig(
|
||||
"http://mine/v1", "sk-mine", "expensive/paid", False)
|
||||
|
||||
|
||||
def test_oversized_conversation_is_refused(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_oversized_conversation_is_refused(client):
|
||||
huge = "x" * 90_000
|
||||
resp = client.post("/api/chat/stream", json={
|
||||
"messages": [{"role": "user", "content": huge} for _ in range(5)],
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Deleting a turn puts the shared state back.
|
||||
|
||||
`script_state` and `world_state` belong to the adventure, not to the node
|
||||
`world_state` belongs to the adventure, not to the node
|
||||
that changed them. Undo, retry, a take and a branch switch all restore them;
|
||||
the delete endpoint did not. Deleting an AI turn removed the text and left
|
||||
everything the turn did to the numbers standing.
|
||||
@@ -23,7 +23,7 @@ from app import auth, limits, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# `mana` carries a cooldown, so a clock that was not rolled back shows up as
|
||||
# a refusal rather than as a number that is merely off.
|
||||
@@ -31,20 +31,19 @@ SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"mana": {"min": 0, "max": 50, "initial": 50, "cooldown": 2},
|
||||
# The per-turn counter these tests measure rollbacks with. Unbounded and
|
||||
# uncapped on purpose, so every turn's +10 lands in full. See
|
||||
# `fakes.gold_reply`.
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
# Ten gold a turn. A total that only ever climbs makes a missing rollback
|
||||
# obvious: it is off by exactly one turn's worth.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
DRAIN = 'Drained.\n```state\n{"player.mana": -10}\n```'
|
||||
# The same turn, also banking the per-turn counter the rollback tests measure.
|
||||
DRAIN_AND_GOLD = 'Drained.\n```state\n{"player.mana": -10, "player.gold": 10}\n```'
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -60,24 +59,18 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Tower", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100, "mana": 50}},
|
||||
world_state={"player": {"hp": 100, "mana": 50, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You begin."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = [DRAIN]
|
||||
ScriptedProvider.replies = [DRAIN_AND_GOLD]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -116,7 +109,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -168,17 +161,17 @@ def test_the_next_turn_is_not_refused_for_a_deleted_turn_s_cooldown(client):
|
||||
assert [c for c in _last_changes(client.adv_id) if c["kind"] == "rejected"] == []
|
||||
|
||||
|
||||
def test_deleting_the_ai_turn_rewinds_the_script_state(client):
|
||||
"""The same restore, on the other half of the shared state. Without it a
|
||||
replayed turn stacks its script run on top of the deleted one's."""
|
||||
def test_deleting_the_ai_turn_rewinds_the_counter(client):
|
||||
"""The same restore, seen through a value that only ever climbs. Without it
|
||||
a replayed turn stacks its effects on top of the deleted one's."""
|
||||
_play(client)
|
||||
assert _state(client.adv_id)[0] == {"gold": 10}
|
||||
assert _state(client.adv_id)[0]["gold"] == 10
|
||||
|
||||
_delete(client, _ai_rows(client.adv_id)[-1].id)
|
||||
assert _state(client.adv_id)[0] == {}
|
||||
assert _state(client.adv_id)[0]["gold"] == 0
|
||||
|
||||
_continue(client)
|
||||
assert _state(client.adv_id)[0] == {"gold": 10}, "one turn of gold, not two"
|
||||
assert _state(client.adv_id)[0]["gold"] == 10, "one turn of gold, not two"
|
||||
|
||||
|
||||
# ------------------------------------------------- deleting further back
|
||||
@@ -190,7 +183,7 @@ def test_deleting_a_turn_the_story_moved_past_leaves_the_tip_alone(client):
|
||||
_play(client)
|
||||
_play(client, "press on")
|
||||
before = _state(client.adv_id)
|
||||
assert before[0] == {"gold": 20}
|
||||
assert before[0]["gold"] == 20
|
||||
|
||||
first_ai = _ai_rows(client.adv_id)[0]
|
||||
assert _delete(client, first_ai.id).status_code == 204
|
||||
|
||||
@@ -99,7 +99,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -57,7 +57,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Which inference endpoints the product will and will not talk to.
|
||||
|
||||
The endpoint is where the whole campaign goes: prompts, narration, retrieved
|
||||
memories, embedding inputs. `app/endpoints.py` is the rule that keeps that on
|
||||
the user's own infrastructure, and these tests are what stops the rule quietly
|
||||
widening.
|
||||
|
||||
Nothing here touches the network. Hostnames are resolved through a stub, so the
|
||||
tests are about the *policy*, run identically offline, and cannot be made to
|
||||
pass or fail by whatever DNS the machine happens to have.
|
||||
|
||||
python -m pytest tests/test_endpoint_policy.py -v
|
||||
"""
|
||||
import ipaddress
|
||||
import socket
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, endpoints, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.providers import OpenAICompatibleProvider, ProviderError
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def resolves(monkeypatch):
|
||||
"""Points every hostname at addresses the test names."""
|
||||
table: dict[str, list[str]] = {}
|
||||
|
||||
def fake_getaddrinfo(host, port, *a, **k):
|
||||
# An address literal resolves to itself, as the real resolver does.
|
||||
try:
|
||||
ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
if host not in table:
|
||||
raise socket.gaierror(-2, "Name or service not known")
|
||||
answers = table[host]
|
||||
else:
|
||||
answers = [host]
|
||||
return [
|
||||
(
|
||||
socket.AF_INET6 if ":" in ip else socket.AF_INET,
|
||||
socket.SOCK_STREAM, 6, "", (ip, port or 0),
|
||||
)
|
||||
for ip in answers
|
||||
]
|
||||
|
||||
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
|
||||
return table
|
||||
|
||||
|
||||
# --- allowed: the user's own machine, and the user's own network ----------
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"http://127.0.0.1:11434/v1",
|
||||
"http://[::1]:11434/v1",
|
||||
"http://192.168.1.50:11434/v1", # RFC1918
|
||||
"http://10.0.0.7:11434/v1", # RFC1918
|
||||
"http://172.16.4.4:11434/v1", # RFC1918
|
||||
"https://192.168.1.50:8443/v1", # TLS on the LAN
|
||||
"http://100.100.5.6:11434/v1", # CGNAT, which is what a mesh VPN hands out
|
||||
"http://[fd00::1]:11434/v1", # IPv6 unique-local
|
||||
])
|
||||
def test_local_and_lan_addresses_are_allowed(url, resolves):
|
||||
assert endpoints.rejection_reason(url) is None
|
||||
|
||||
|
||||
def test_a_hostname_resolving_to_the_lan_is_allowed(resolves):
|
||||
resolves["ollama.home.arpa"] = ["192.168.1.50"]
|
||||
assert endpoints.rejection_reason("https://ollama.home.arpa:8443/v1") is None
|
||||
|
||||
|
||||
# --- refused: anywhere else ----------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"http://8.8.8.8:11434/v1",
|
||||
"http://1.1.1.1:11434/v1",
|
||||
"http://[2001:4860:4860::8888]:11434/v1",
|
||||
])
|
||||
def test_public_addresses_are_refused(url, resolves):
|
||||
reason = endpoints.rejection_reason(url)
|
||||
assert reason is not None
|
||||
assert "public Internet address" in reason
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"https://198.51.100.9/v1", # TEST-NET-2, a documentation range
|
||||
"http://0.0.0.0:11434/v1", # "this host", not an address to dial
|
||||
])
|
||||
def test_addresses_that_are_neither_local_nor_public_are_refused(url, resolves):
|
||||
"""Deny by default. `ipaddress` calls both of these "private", which is why
|
||||
the policy names the networks it allows instead of asking that question."""
|
||||
assert endpoints.rejection_reason(url) is not None
|
||||
|
||||
|
||||
def test_a_hostname_resolving_to_the_public_internet_is_refused(resolves):
|
||||
resolves["ollama.example.com"] = ["93.184.216.34"]
|
||||
reason = endpoints.rejection_reason("https://ollama.example.com/v1")
|
||||
assert reason is not None and "public Internet address" in reason
|
||||
|
||||
|
||||
def test_a_split_horizon_answer_is_refused(resolves):
|
||||
"""One public address among the answers is enough. A name that resolves to
|
||||
both a LAN address and a public one must not be usable: which one the
|
||||
request actually reaches is not ours to decide."""
|
||||
resolves["sneaky.example"] = ["192.168.1.50", "93.184.216.34"]
|
||||
assert endpoints.rejection_reason("http://sneaky.example:11434/v1") is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", [
|
||||
"openrouter.ai", "api.openai.com", "api.anthropic.com", "api.groq.com",
|
||||
])
|
||||
def test_known_cloud_providers_are_named_in_the_refusal(host, resolves):
|
||||
"""They would be refused by address anyway — every one resolves publicly.
|
||||
Naming them turns 'your DNS might be broken' into 'this build has no cloud
|
||||
provider support'."""
|
||||
resolves[host] = ["192.168.1.50"] # even if DNS said otherwise
|
||||
reason = endpoints.rejection_reason(f"https://{host}/v1")
|
||||
assert reason is not None
|
||||
assert "cloud inference service" in reason
|
||||
|
||||
|
||||
def test_a_subdomain_of_a_cloud_provider_is_refused(resolves):
|
||||
resolves["eu.api.openai.com"] = ["192.168.1.50"]
|
||||
assert endpoints.rejection_reason("https://eu.api.openai.com/v1") is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"ftp://127.0.0.1/v1",
|
||||
"file:///etc/passwd",
|
||||
"127.0.0.1:11434", # no scheme
|
||||
"",
|
||||
"http://",
|
||||
])
|
||||
def test_things_that_are_not_usable_urls_are_refused(url, resolves):
|
||||
assert endpoints.rejection_reason(url) is not None
|
||||
|
||||
|
||||
def test_an_unresolvable_host_is_refused_with_advice(resolves):
|
||||
reason = endpoints.rejection_reason("http://nope.invalid:11434/v1")
|
||||
assert reason is not None and "could not be resolved" in reason
|
||||
|
||||
|
||||
# --- the rule is applied, not merely available ---------------------------
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def test_saving_a_public_endpoint_is_refused(client, resolves):
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
r = client.put("/api/settings", json={"endpoint_url": "https://openrouter.ai/api/v1"})
|
||||
assert r.status_code == 400, r.text
|
||||
assert "can't be used" in r.json()["detail"]
|
||||
# And it was not written.
|
||||
assert client.get("/api/settings").json()["endpoint_url"] != "https://openrouter.ai/api/v1"
|
||||
|
||||
|
||||
def test_saving_a_lan_endpoint_is_allowed(client, resolves):
|
||||
r = client.put("/api/settings", json={"endpoint_url": "http://192.168.1.50:11434/v1"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.get("/api/settings").json()["endpoint_url"] == "http://192.168.1.50:11434/v1"
|
||||
|
||||
|
||||
def test_the_connection_test_reports_a_refused_endpoint_as_such(client, resolves):
|
||||
"""A row edited by hand, or a name that has started resolving elsewhere,
|
||||
must not simply look 'unreachable'."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().endpoint_url = "https://openrouter.ai/api/v1"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
body = client.post("/api/settings/test").json()
|
||||
assert body["ok"] is False
|
||||
assert body["kind"] == "rejected"
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_the_provider_refuses_before_it_sends_anything(resolves):
|
||||
"""The check that actually matters. Whatever is stored, no request leaves
|
||||
for an address outside the policy — so a database edited behind the app's
|
||||
back cannot turn into an exfiltration path."""
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "m")
|
||||
with pytest.raises(ProviderError) as exc:
|
||||
async for _ in provider.generate(
|
||||
_parts(), temperature=0.8, max_tokens=10
|
||||
):
|
||||
pass
|
||||
assert "can't be used" in str(exc.value)
|
||||
|
||||
|
||||
def _parts():
|
||||
from app.providers.base import PromptParts
|
||||
return PromptParts(system="s", story="t")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anyio_backend():
|
||||
return "asyncio"
|
||||
@@ -1,205 +0,0 @@
|
||||
"""Guest retention policy: app/cleanup.py.
|
||||
|
||||
These tests cover the two things that matter. Idle guests and their whole
|
||||
data graph must actually be deleted, and nothing else must ever be
|
||||
deleted.
|
||||
|
||||
python -m pytest tests/test_guest_cleanup.py -v
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from app import cleanup, models
|
||||
from app.database import Base
|
||||
from app.migrations import bootstrap
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db(tmp_path):
|
||||
engine = create_engine(f"sqlite:///{tmp_path/'t.db'}", connect_args={"check_same_thread": False})
|
||||
|
||||
@event.listens_for(engine, "connect")
|
||||
def _fk(dbapi_connection, _record):
|
||||
# The whole policy relies on ON DELETE CASCADE. SQLite ignores every
|
||||
# one of them unless this is set, the same as database.py does.
|
||||
cur = dbapi_connection.cursor()
|
||||
cur.execute("PRAGMA foreign_keys=ON")
|
||||
cur.close()
|
||||
|
||||
bootstrap(engine)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
session = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)()
|
||||
yield session
|
||||
session.close()
|
||||
|
||||
|
||||
NOW = models.utcnow().replace(tzinfo=None)
|
||||
|
||||
|
||||
def make_user(db, *, days_idle=None, days_old=0, guest=True, email=None):
|
||||
"""A user last seen `days_idle` ago (None = never seen, only created)."""
|
||||
user = models.User(
|
||||
is_guest=guest,
|
||||
email=email,
|
||||
password_hash=None if email is None else "x",
|
||||
created_at=NOW - timedelta(days=days_old),
|
||||
last_seen_at=None if days_idle is None else NOW - timedelta(days=days_idle),
|
||||
)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
return user
|
||||
|
||||
|
||||
def sweep(db):
|
||||
return cleanup.delete_stale_guests(db, now=NOW)
|
||||
|
||||
|
||||
def alive(db, user_id):
|
||||
# A count, not db.get: the sweep deletes with synchronize_session=False, so
|
||||
# the session's identity map still holds the object and db.get would answer
|
||||
# from memory without ever asking the database.
|
||||
return db.query(models.User).filter(models.User.id == user_id).count() == 1
|
||||
|
||||
|
||||
# ---------- what goes ----------
|
||||
|
||||
def test_deletes_guest_idle_past_the_window(db):
|
||||
user = make_user(db, days_idle=6)
|
||||
assert sweep(db) == 1
|
||||
assert not alive(db, user.id)
|
||||
|
||||
|
||||
def test_keeps_guest_inside_the_window(db):
|
||||
user = make_user(db, days_idle=4)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_boundary_is_not_yet_stale(db):
|
||||
# Exactly 5 days survives. The comparison is strict.
|
||||
user = make_user(db, days_idle=cleanup.RETENTION_DAYS)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_never_seen_guest_falls_back_to_created_at(db):
|
||||
"""last_seen_at is NULL until a guest's second request (auth._touch runs
|
||||
hourly), so a coalesce-less query would delete brand-new visitors."""
|
||||
fresh = make_user(db, days_idle=None, days_old=0)
|
||||
stale = make_user(db, days_idle=None, days_old=9)
|
||||
assert sweep(db) == 1
|
||||
assert alive(db, fresh.id)
|
||||
assert not alive(db, stale.id)
|
||||
|
||||
|
||||
def test_recent_visit_beats_an_old_created_at(db):
|
||||
# A long-standing guest who came back yesterday stays.
|
||||
user = make_user(db, days_idle=1, days_old=90)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
# ---------- what must never go ----------
|
||||
|
||||
def test_spares_registered_users(db):
|
||||
"""Registering upgrades the guest row in place. An idle account here is
|
||||
a real user with real data, which is what signing up is meant to
|
||||
protect."""
|
||||
user = make_user(db, days_idle=400, guest=False, email="a@b.com")
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_spares_the_local_mode_user(db):
|
||||
# email NULL but is_guest False: local mode's implicit owner of everything.
|
||||
user = make_user(db, days_idle=400, guest=False)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_spares_a_guest_flagged_row_that_has_an_email(db):
|
||||
# This row should not exist, but both clauses are checked so it cannot be collected.
|
||||
user = make_user(db, days_idle=400, guest=True, email="odd@b.com")
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_leaves_seeded_public_scenarios_alone(db):
|
||||
"""Seeded demo content has user_id NULL, so it is outside the filter."""
|
||||
seeded = models.Scenario(user_id=None, is_public=True, title="Demo")
|
||||
db.add(seeded)
|
||||
make_user(db, days_idle=30)
|
||||
db.commit()
|
||||
assert sweep(db) == 1
|
||||
assert db.query(models.Scenario).filter(models.Scenario.id == seeded.id).count() == 1
|
||||
|
||||
|
||||
def test_disabled_when_retention_is_zero(db, monkeypatch):
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 0)
|
||||
user = make_user(db, days_idle=999)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_enabled_requires_multi_user(monkeypatch):
|
||||
from app import auth
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
assert cleanup.enabled() is False
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 5)
|
||||
assert cleanup.enabled() is True
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 0)
|
||||
assert cleanup.enabled() is False
|
||||
|
||||
|
||||
# ---------- the cascade ----------
|
||||
|
||||
def test_deletes_the_whole_data_graph(db):
|
||||
"""One DELETE must remove the adventure, its actions and memories, the
|
||||
story cards, and the settings row. Nothing is loaded into Python, so if
|
||||
the FK cascade does not reach a table, its rows are silently orphaned,
|
||||
or the statement fails, instead of being removed."""
|
||||
user = make_user(db, days_idle=30)
|
||||
scenario = models.Scenario(user_id=user.id, title="S")
|
||||
db.add(scenario)
|
||||
db.commit()
|
||||
adventure = models.Adventure(user_id=user.id, scenario_id=scenario.id, title="A")
|
||||
db.add(adventure)
|
||||
db.commit()
|
||||
db.add_all([
|
||||
models.Action(adventure_id=adventure.id, type="ai", text="t"),
|
||||
models.Memory(adventure_id=adventure.id, text="m", source_start=0, source_end=0),
|
||||
models.StoryCard(adventure_id=adventure.id, name="c"),
|
||||
models.Settings(user_id=user.id),
|
||||
])
|
||||
db.commit()
|
||||
|
||||
assert sweep(db) == 1
|
||||
|
||||
for model in (models.Scenario, models.Adventure, models.Action,
|
||||
models.Memory, models.StoryCard, models.Settings):
|
||||
assert db.query(model).count() == 0, f"{model.__name__} rows survived"
|
||||
|
||||
|
||||
def test_one_users_cleanup_does_not_touch_another(db):
|
||||
keeper = make_user(db, days_idle=1)
|
||||
keep_adv = models.Adventure(user_id=keeper.id, title="mine")
|
||||
goner = make_user(db, days_idle=30)
|
||||
db.add_all([keep_adv, models.Adventure(user_id=goner.id, title="theirs")])
|
||||
db.commit()
|
||||
|
||||
assert sweep(db) == 1
|
||||
remaining = db.query(models.Adventure).all()
|
||||
assert [a.title for a in remaining] == ["mine"]
|
||||
|
||||
|
||||
def test_sweep_swallows_errors(monkeypatch):
|
||||
"""A broken cleanup must not take the app down (same rule as seeding)."""
|
||||
monkeypatch.setattr(cleanup.auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(cleanup, "delete_stale_guests",
|
||||
lambda *a, **k: (_ for _ in ()).throw(RuntimeError("boom")))
|
||||
assert cleanup.sweep() == 0
|
||||
@@ -0,0 +1,873 @@
|
||||
"""M3: the active head, and what moving it costs.
|
||||
|
||||
This file is the acceptance contract for the milestone that stopped Undo from
|
||||
deleting. Its subject is one invariant and the behaviour that follows from it:
|
||||
|
||||
Undo deletes zero accepted turns.
|
||||
|
||||
Everything else here is a consequence. Redo exists because the turns are still
|
||||
there. Divergence retires a future rather than removing it. Memory and summary
|
||||
coverage narrow and widen again as the head moves, without anything being
|
||||
re-embedded. Export carries where the reader stopped, because that is now a
|
||||
decision rather than a fact about the newest row.
|
||||
|
||||
The tests are named for the acceptance items they discharge — D01-D10, E01-E04,
|
||||
I01-I03, I07, L01-L02 in `planning/V1-ACCEPTANCE-TESTS.md` — so a reader can go
|
||||
from a failing test to the requirement it belongs to without a map.
|
||||
|
||||
The world state is instrumentation here, not the subject. Each scripted reply
|
||||
banks ten gold, which makes "the state at this position" a number a test can
|
||||
assert instead of a paragraph it has to interpret. M5 replaces that machinery
|
||||
with genre-neutral narrative state; these tests then need the *instrumentation*
|
||||
moved, not the assertions removed, because what they measure is where the story
|
||||
is being read.
|
||||
|
||||
python -m pytest tests/test_head_cursor.py -v
|
||||
"""
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import limits, models
|
||||
from app.providers import ProviderError
|
||||
from app.context import cursors, lineage
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app import auth, tree
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import GOLD_PER_TURN, GOLD_SCHEMA, ScriptedProvider, gold_replies
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False, email="head@example.com")
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||
scenario = models.Scenario(user_id=user.id, title="S", stat_schema=GOLD_SCHEMA)
|
||||
setup.add(scenario)
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Tavern", scenario_id=scenario.id,
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter the tavern."))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = gold_replies()
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
c.adv_id = adv_id
|
||||
try:
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
adventures.turns._active_turns.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ helpers
|
||||
|
||||
def _play(client, text="look around", type="do", after_id=None):
|
||||
payload = {"type": type, "text": text}
|
||||
if after_id is not None:
|
||||
payload["after_id"] = after_id
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/actions", json=payload)
|
||||
assert r.status_code == 200, r.text
|
||||
return r
|
||||
|
||||
|
||||
def _turns(client, count):
|
||||
for n in range(count):
|
||||
_play(client, f"turn {n}")
|
||||
|
||||
|
||||
def _undo(client):
|
||||
return client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
|
||||
|
||||
def _redo(client):
|
||||
return client.post(f"/api/adventures/{client.adv_id}/redo")
|
||||
|
||||
|
||||
def _adventure(client) -> dict:
|
||||
r = client.get(f"/api/adventures/{client.adv_id}")
|
||||
assert r.status_code == 200, r.text
|
||||
return r.json()
|
||||
|
||||
|
||||
def _texts(client) -> list[str]:
|
||||
return [a["text"] for a in _adventure(client)["actions"]]
|
||||
|
||||
|
||||
def _rows(adv_id) -> list[models.Action]:
|
||||
"""Every action row, story or not, live or not, head or no head."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return (
|
||||
db.query(models.Action)
|
||||
.filter(models.Action.adventure_id == adv_id)
|
||||
.order_by(models.Action.branch_id, models.Action.depth, models.Action.id)
|
||||
.all()
|
||||
)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _gold(adv_id) -> int:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return (adv.world_state or {}).get("player", {}).get("gold", 0)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _head(adv_id) -> tuple[int, int]:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.head_branch_id, adv.head_depth
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ------------------------------------------------------- D01, D02, D03: undo
|
||||
|
||||
def test_d01_one_undo_returns_the_transcript_and_the_state(client):
|
||||
_turns(client, 2)
|
||||
assert _gold(client.adv_id) == 2 * GOLD_PER_TURN
|
||||
told = _texts(client)
|
||||
|
||||
page = _undo(client)
|
||||
assert page.status_code == 200, page.text
|
||||
|
||||
# Both halves of the turn step back together: a player's action and the
|
||||
# reply to it are one accepted story step.
|
||||
assert [a["text"] for a in page.json()["actions"]] == told[:-2]
|
||||
assert _gold(client.adv_id) == 1 * GOLD_PER_TURN
|
||||
|
||||
|
||||
def test_d02_five_consecutive_undos_each_land_where_they_should(client):
|
||||
_turns(client, 7)
|
||||
assert _gold(client.adv_id) == 7 * GOLD_PER_TURN
|
||||
|
||||
for step in range(1, 6):
|
||||
assert _undo(client).status_code == 200
|
||||
assert _gold(client.adv_id) == (7 - step) * GOLD_PER_TURN
|
||||
# Two rows per turn, plus the opening.
|
||||
assert len(_texts(client)) == 1 + 2 * (7 - step)
|
||||
|
||||
|
||||
def test_d03_undo_walks_back_to_the_campaign_opening_and_stops(client):
|
||||
_turns(client, 7)
|
||||
|
||||
for _ in range(7):
|
||||
assert _undo(client).status_code == 200
|
||||
|
||||
assert _texts(client) == ["You enter the tavern."]
|
||||
assert _gold(client.adv_id) == 0
|
||||
# The opening is the floor. There is no pre-campaign position to reach.
|
||||
refused = _undo(client)
|
||||
assert refused.status_code == 400
|
||||
assert "Nothing to undo" in refused.json()["detail"]
|
||||
assert _adventure(client)["can_undo"] is False
|
||||
|
||||
|
||||
def test_the_m3_invariant_undo_deletes_zero_accepted_turns(client):
|
||||
"""The row count is the milestone. Everything else in this file is a
|
||||
consequence of it, so it is asserted on its own, over the whole retained
|
||||
tree rather than over the story being told."""
|
||||
_turns(client, 5)
|
||||
before = [a.id for a in _rows(client.adv_id)]
|
||||
|
||||
for _ in range(5):
|
||||
assert _undo(client).status_code == 200
|
||||
|
||||
after = [a.id for a in _rows(client.adv_id)]
|
||||
assert after == before
|
||||
assert len(_texts(client)) == 1 # and yet the story is back at its opening
|
||||
|
||||
|
||||
# -------------------------------------------------------------- D04, L02: redo
|
||||
|
||||
def test_d04_redo_restores_the_continuation_and_its_state(client):
|
||||
_turns(client, 4)
|
||||
whole = _texts(client)
|
||||
|
||||
_undo(client)
|
||||
_undo(client)
|
||||
assert _gold(client.adv_id) == 2 * GOLD_PER_TURN
|
||||
|
||||
assert _redo(client).status_code == 200
|
||||
assert _gold(client.adv_id) == 3 * GOLD_PER_TURN
|
||||
assert _redo(client).status_code == 200
|
||||
assert _gold(client.adv_id) == 4 * GOLD_PER_TURN
|
||||
assert _texts(client) == whole
|
||||
# Nowhere further forward to go, and the control says so.
|
||||
assert _adventure(client)["can_redo"] is False
|
||||
assert _redo(client).status_code == 400
|
||||
|
||||
|
||||
def test_l02_state_matches_the_position_in_both_directions(client):
|
||||
"""Each position's state is the snapshot the node left behind, so arriving
|
||||
from in front of it and arriving from behind it must agree."""
|
||||
_turns(client, 5)
|
||||
going_back = []
|
||||
for _ in range(5):
|
||||
_undo(client)
|
||||
going_back.append(_gold(client.adv_id))
|
||||
|
||||
coming_forward = []
|
||||
for _ in range(5):
|
||||
_redo(client)
|
||||
coming_forward.append(_gold(client.adv_id))
|
||||
|
||||
assert going_back == [40, 30, 20, 10, 0]
|
||||
assert coming_forward == [10, 20, 30, 40, 50]
|
||||
|
||||
|
||||
# ------------------------------------------------- D05, E01, E04: divergence
|
||||
|
||||
def test_d05_a_new_turn_below_the_head_retires_redo_and_keeps_the_future(client):
|
||||
_turns(client, 4)
|
||||
abandoned = {a.id for a in _rows(client.adv_id)}
|
||||
|
||||
_undo(client)
|
||||
_undo(client)
|
||||
assert _adventure(client)["can_redo"] is True
|
||||
|
||||
ScriptedProvider.replies = ["A different road.\n```state\n{\"player.gold\": 1}\n```"]
|
||||
_play(client, "go the other way")
|
||||
|
||||
# Ordinary Redo cannot walk into the old future any more...
|
||||
assert _adventure(client)["can_redo"] is False
|
||||
assert _redo(client).status_code == 400
|
||||
# ...and not one row of it was deleted to achieve that.
|
||||
kept = {a.id for a in _rows(client.adv_id)}
|
||||
assert abandoned <= kept
|
||||
# The story now tells the new continuation.
|
||||
assert _texts(client)[-1].startswith("A different road.")
|
||||
|
||||
|
||||
def test_the_departed_branch_records_where_the_story_left_it(client):
|
||||
"""`DATA-MODEL.md` §5 gives a branch a disposition. It is stored as the fact
|
||||
that produced it — the depth the story left at — and nothing reads it to
|
||||
decide behaviour, so this test is what makes a divergence observable."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
left_at = _head(client.adv_id)[1]
|
||||
|
||||
_play(client, "elsewhere")
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
superseded = (
|
||||
db.query(models.Branch)
|
||||
.filter(
|
||||
models.Branch.adventure_id == client.adv_id,
|
||||
models.Branch.superseded_at.isnot(None),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
assert len(superseded) == 1
|
||||
assert superseded[0].superseded_depth == left_at
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_e01_e04_a_fact_from_the_abandoned_future_is_not_current(client):
|
||||
"""E01 and E04 are the same mechanism measured twice: the state that is
|
||||
current is the one belonging to the position the story is read at, so a
|
||||
number only the abandoned future ever reached cannot survive a divergence."""
|
||||
ScriptedProvider.replies = [
|
||||
"You find a purse.\n```state\n{\"player.gold\": 10}\n```",
|
||||
"You find the hoard.\n```state\n{\"player.gold\": 500}\n```",
|
||||
]
|
||||
_play(client, "search")
|
||||
_play(client, "keep searching")
|
||||
assert _gold(client.adv_id) == 510
|
||||
|
||||
_undo(client)
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
ScriptedProvider.replies = ["You leave empty-handed.\n```state\n{\"player.gold\": 1}\n```"]
|
||||
_play(client, "go home")
|
||||
|
||||
assert _gold(client.adv_id) == 11, "the hoard belonged to a story this one is not"
|
||||
|
||||
|
||||
# ------------------------------------------------------ E02, E03: derived work
|
||||
|
||||
def _attach_memory(adv_id, text, at_depth):
|
||||
"""Writes a memory ending on the live node at `at_depth`, the way the
|
||||
summarizer would, and anchors the memory cursor there."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adventure = db.get(models.Adventure, adv_id)
|
||||
node = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adv_id,
|
||||
models.Action.depth == at_depth,
|
||||
models.Action.live.is_(True),
|
||||
)
|
||||
.first()
|
||||
)
|
||||
memory = models.Memory(
|
||||
adventure_id=adv_id, text=text, source_start=0, source_end=node.depth,
|
||||
)
|
||||
tree.attach_memory(memory, node)
|
||||
db.add(memory)
|
||||
cursors.MEMORY.anchor_at(adventure, node)
|
||||
cursors.SUMMARY.anchor_at(adventure, node)
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _retrievable(adv_id) -> set[str]:
|
||||
"""The memories the story can reach, through the clause `memorybank`
|
||||
retrieves with."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adventure = db.get(models.Adventure, adv_id)
|
||||
rows = (
|
||||
db.query(models.Memory)
|
||||
.filter(
|
||||
models.Memory.adventure_id == adv_id,
|
||||
lineage.path_of(db, adventure).clause(models.Memory),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
return {m.text for m in rows}
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_e02_an_abandoned_memory_is_unreachable_and_still_on_disk(client):
|
||||
_turns(client, 3)
|
||||
tip_depth = _head(client.adv_id)[1]
|
||||
_attach_memory(client.adv_id, "Mara reveals she is a spy.", tip_depth)
|
||||
assert _retrievable(client.adv_id) == {"Mara reveals she is a spy."}
|
||||
|
||||
_undo(client)
|
||||
# Negative control: undone, the revelation is not retrievable...
|
||||
assert _retrievable(client.adv_id) == set()
|
||||
_play(client, "talk about the weather")
|
||||
# ...and after diverging it stays unreachable, on a line the story left.
|
||||
assert _retrievable(client.adv_id) == set()
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert db.query(models.Memory).count() == 1, "unreachable, not deleted"
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_e02_positive_control_the_memory_returns_on_the_line_it_belongs_to(client):
|
||||
"""The negative control alone would pass if memories were simply broken.
|
||||
Redo puts the head back on the lineage the memory was written for, and it
|
||||
must be retrievable again — without having been re-embedded."""
|
||||
_turns(client, 3)
|
||||
_attach_memory(client.adv_id, "Mara reveals she is a spy.", _head(client.adv_id)[1])
|
||||
|
||||
_undo(client)
|
||||
assert _retrievable(client.adv_id) == set()
|
||||
_redo(client)
|
||||
assert _retrievable(client.adv_id) == {"Mara reveals she is a spy."}
|
||||
|
||||
|
||||
def test_e03_a_summary_anchor_cannot_claim_coverage_past_the_head(client):
|
||||
"""E03 without redesigning the summarizer. The anchor says how far the
|
||||
derived work has read; resolved against a head-capped path it can never
|
||||
report a stretch in the abandoned future as already covered, so that content
|
||||
is re-derived for the new line rather than carried into it."""
|
||||
_turns(client, 3)
|
||||
tip_depth = _head(client.adv_id)[1]
|
||||
_attach_memory(client.adv_id, "Everything up to the reveal.", tip_depth)
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adventure = db.get(models.Adventure, client.adv_id)
|
||||
assert cursors.SUMMARY.depth(db, adventure) == tip_depth
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
_undo(client)
|
||||
_play(client, "a different question")
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adventure = db.get(models.Adventure, client.adv_id)
|
||||
covered = cursors.SUMMARY.depth(db, adventure)
|
||||
assert covered < tip_depth, "coverage from the abandoned line was claimed"
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ------------------------------------------------ D06-D08: retry and takes
|
||||
|
||||
def test_d06_d08_retry_keeps_the_earlier_take_and_reuses_the_parent_state(client):
|
||||
ScriptedProvider.replies = gold_replies("Take")
|
||||
_play(client, "knock")
|
||||
first = [a for a in _rows(client.adv_id) if a.type == "ai"][0]
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
ai_rows = [a for a in _rows(client.adv_id) if a.type == "ai"]
|
||||
assert len(ai_rows) == 2, "the earlier take is retained"
|
||||
assert first.id in {a.id for a in ai_rows}
|
||||
# Both takes sit at the same coordinate, which is what makes them takes
|
||||
# rather than turns, and the state is one turn's worth either way.
|
||||
assert {a.depth for a in ai_rows} == {first.depth}
|
||||
assert _gold(client.adv_id) == GOLD_PER_TURN
|
||||
|
||||
|
||||
def test_d07_a_retry_from_behind_the_tip_branches_instead_of_amending(client):
|
||||
"""A turn with an accepted future is not a leaf, whatever the capped read
|
||||
says. Retrying it has to leave that future on the line it was written for,
|
||||
which is a branch — the same operation `add_take` performs for a turn the
|
||||
story has already moved past."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
before = {a.id for a in _rows(client.adv_id)}
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
after = {a.id for a in _rows(client.adv_id)}
|
||||
assert before <= after, "the retained future kept every row"
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert db.query(models.Branch).filter_by(adventure_id=client.adv_id).count() == 2
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_switching_a_take_is_refused_while_a_kept_future_hangs_off_it(client):
|
||||
"""The one operation that cannot be made safe by branching, because it
|
||||
changes which take is live in place. It reports rather than guesses."""
|
||||
ScriptedProvider.replies = gold_replies()
|
||||
_play(client, "knock")
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
_play(client, "go in")
|
||||
_undo(client)
|
||||
|
||||
newest = [a for a in _rows(client.adv_id) if a.type == "ai" and a.live][0]
|
||||
r = client.post(
|
||||
f"/api/adventures/{client.adv_id}/actions/{newest.id}/variant", json={"index": 0}
|
||||
)
|
||||
assert r.status_code == 400
|
||||
assert "undone but kept" in r.json()["detail"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------- D09, D10: editing
|
||||
|
||||
def test_d09_d10_replaying_a_turn_forks_and_keeps_the_old_line(client):
|
||||
"""D09 and D10 through the operation the product actually offers. Editing
|
||||
an earlier turn and replaying it is one endpoint — `takes` — and it is
|
||||
already the "return to the parent state, then continue differently" shape
|
||||
the semantics ask for. The plain text edit beside it is a correction to
|
||||
prose that creates no continuation, and M3 does not change it.
|
||||
"""
|
||||
ScriptedProvider.replies = [
|
||||
"You accuse her.\n```state\n{\"player.gold\": 10}\n```",
|
||||
"She draws a knife.\n```state\n{\"player.gold\": 20}\n```",
|
||||
]
|
||||
_play(client, "I accuse Mara of stealing the key.")
|
||||
_play(client, "wait")
|
||||
accusation = [a for a in _rows(client.adv_id) if a.type == "do"][0]
|
||||
old_future = {a.id for a in _rows(client.adv_id)}
|
||||
|
||||
ScriptedProvider.replies = ["She shakes her head.\n```state\n{\"player.gold\": 1}\n```"]
|
||||
r = client.post(
|
||||
f"/api/adventures/{client.adv_id}/actions/{accusation.id}/takes",
|
||||
json={"text": "I quietly ask Mara whether she has seen the key."},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
# The old future is retained whole...
|
||||
assert old_future <= {a.id for a in _rows(client.adv_id)}
|
||||
# ...the edited input is what the story now tells...
|
||||
told = _texts(client)
|
||||
assert any("quietly ask Mara" in t for t in told)
|
||||
assert not any("accuse her" in t for t in told)
|
||||
# ...and no state from the abandoned line leaked into it.
|
||||
assert _gold(client.adv_id) == 1
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ L01
|
||||
|
||||
def test_l01_a_failed_turn_accepts_no_narration_and_strands_no_state(client):
|
||||
"""A provider that fails must not leave the story half-advanced.
|
||||
|
||||
The head does move, by exactly one node, and that is deliberate rather than
|
||||
a gap in the guarantee: A05 keeps the player's submitted text so it can be
|
||||
tried again, and the head sits on it. What L01 forbids is an *accepted*
|
||||
narration with half-written state behind it, a head that has moved past a
|
||||
turn that did not happen, or earlier story becoming unreachable — so those
|
||||
are what this asserts.
|
||||
"""
|
||||
_turns(client, 2)
|
||||
branch, depth = _head(client.adv_id)
|
||||
told = _texts(client)
|
||||
banked = _gold(client.adv_id)
|
||||
ai_before = [a.id for a in _rows(client.adv_id) if a.type == "ai"]
|
||||
|
||||
ScriptedProvider.replies = [ProviderError("the model refused")]
|
||||
client.post(f"/api/adventures/{client.adv_id}/actions",
|
||||
json={"type": "do", "text": "the turn that fails"})
|
||||
|
||||
# One step, onto the retained player action, on the same branch.
|
||||
assert _head(client.adv_id) == (branch, depth + 1)
|
||||
# No narration was accepted, and no state moved with a turn that did not
|
||||
# finish.
|
||||
assert [a.id for a in _rows(client.adv_id) if a.type == "ai"] == ai_before
|
||||
assert _gold(client.adv_id) == banked
|
||||
# Every earlier turn is still readable, in order, unchanged.
|
||||
assert _texts(client)[:len(told)] == told
|
||||
|
||||
# And the story still moves: one Undo steps back over the stranded input
|
||||
# and the position is a complete turn again.
|
||||
ScriptedProvider.replies = gold_replies()
|
||||
assert _undo(client).status_code == 200
|
||||
assert _texts(client) == told
|
||||
assert _gold(client.adv_id) == banked
|
||||
|
||||
|
||||
# ------------------------------------------------- I01, I02, I03, I07: bundles
|
||||
|
||||
def _export(client) -> dict:
|
||||
r = client.get(f"/api/adventures/{client.adv_id}/export")
|
||||
assert r.status_code == 200, r.text
|
||||
return r.json()
|
||||
|
||||
|
||||
def _import(client, bundle) -> dict:
|
||||
"""Imports a bundle and returns the new adventure.
|
||||
|
||||
"A fresh data directory" is an adventure this file has never touched: the
|
||||
import allocates its own branch rows and its own nodes, and resolves the
|
||||
file's local branch numbers against them, which is the whole of what the
|
||||
round trip has to get right. Sharing a database with the original does not
|
||||
weaken that — the two adventures share no row.
|
||||
"""
|
||||
r = client.post("/api/adventures/import", json=bundle)
|
||||
assert r.status_code == 201, r.text
|
||||
return r.json()
|
||||
|
||||
|
||||
def _story_of(adv_id) -> list[str]:
|
||||
"""The story an imported adventure tells, read through its own head."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adventure = db.get(models.Adventure, adv_id)
|
||||
rows = (
|
||||
db.query(models.Action)
|
||||
.filter(
|
||||
models.Action.adventure_id == adv_id,
|
||||
lineage.path_of(db, adventure).clause(models.Action),
|
||||
)
|
||||
.order_by(models.Action.depth, models.Action.id)
|
||||
.all()
|
||||
)
|
||||
return [a.text for a in rows]
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_i01_i02_a_campaign_round_trips(client):
|
||||
_turns(client, 3)
|
||||
told = _texts(client)
|
||||
|
||||
imported = _import(client, _export(client))
|
||||
|
||||
assert _story_of(imported["id"]) == told
|
||||
assert _gold(imported["id"]) == 3 * GOLD_PER_TURN
|
||||
|
||||
|
||||
def test_i03_the_bundle_carries_the_history_the_story_no_longer_tells(client):
|
||||
"""A backup that keeps only the active line is not a backup: the retained
|
||||
future is what Redo and every later recovery feature read."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
_play(client, "another way")
|
||||
active = _texts(client)
|
||||
|
||||
bundle = _export(client)
|
||||
assert len(bundle["branches"]) == 2
|
||||
imported = _import(client, bundle)
|
||||
|
||||
assert _story_of(imported["id"]) == active
|
||||
# Every row of the abandoned line arrived too, on its own branch.
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert (
|
||||
db.query(models.Action).filter_by(adventure_id=imported["id"]).count()
|
||||
== len(_rows(client.adv_id))
|
||||
)
|
||||
assert db.query(models.Branch).filter_by(adventure_id=imported["id"]).count() == 2
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_i07_an_undone_head_survives_export_and_import(client):
|
||||
"""The Phase 0B finding this milestone exists to close. Before M3 the head
|
||||
depth was derived on import from the newest retained row, so a campaign
|
||||
exported after two Undos came back silently redone to its tip."""
|
||||
_turns(client, 5)
|
||||
whole = _texts(client)
|
||||
_undo(client)
|
||||
_undo(client)
|
||||
undone = _texts(client)
|
||||
assert len(undone) == len(whole) - 4
|
||||
|
||||
bundle = _export(client)
|
||||
assert bundle["headDepth"] == _head(client.adv_id)[1]
|
||||
|
||||
imported = _import(client, bundle)
|
||||
|
||||
# It opens exactly where it was left...
|
||||
assert _story_of(imported["id"]) == undone
|
||||
assert imported["can_redo"] is True
|
||||
# ...the later turns arrived as retained history...
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert (
|
||||
db.query(models.Action).filter_by(adventure_id=imported["id"]).count()
|
||||
== len(_rows(client.adv_id))
|
||||
)
|
||||
finally:
|
||||
db.close()
|
||||
# ...and Redo still walks forward into them, twice, to the same story.
|
||||
for _ in range(2):
|
||||
r = client.post(f"/api/adventures/{imported['id']}/redo")
|
||||
assert r.status_code == 200, r.text
|
||||
assert _story_of(imported["id"]) == whole
|
||||
|
||||
|
||||
def test_a_bundle_written_before_m3_opens_at_its_tip(client):
|
||||
"""Backward compatibility. A file with no `headDepth` is one written when
|
||||
the head could not be anywhere but the tip, so deriving it is not a
|
||||
fallback — it is the position that file recorded."""
|
||||
_turns(client, 3)
|
||||
told = _texts(client)
|
||||
bundle = _export(client)
|
||||
del bundle["headDepth"]
|
||||
|
||||
imported = _import(client, bundle)
|
||||
|
||||
assert _story_of(imported["id"]) == told
|
||||
|
||||
|
||||
def test_a_bundle_that_reads_past_its_own_story_is_refused(client):
|
||||
"""Checked in `plan`, before a row is written, for the reason the module
|
||||
docstring gives: an import that discovers a bad coordinate afterwards has
|
||||
already written half a tree."""
|
||||
_turns(client, 2)
|
||||
bundle = _export(client)
|
||||
bundle["headDepth"] = 999
|
||||
|
||||
r = client.post("/api/adventures/import", json=bundle)
|
||||
assert r.status_code == 400
|
||||
assert "ends at" in r.json()["detail"]
|
||||
|
||||
|
||||
def test_the_bundle_carries_which_branches_the_story_left(client):
|
||||
"""Every row of an abandoned line arrives on an import either way, so the
|
||||
disposition is the only thing that distinguishes it from an active one.
|
||||
Losing it on a round trip would leave a restored backup unable to tell them
|
||||
apart — which is what the later cleanup and recovery features select on."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
_play(client, "another way")
|
||||
|
||||
imported = _import(client, _export(client))
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
left = (
|
||||
db.query(models.Branch)
|
||||
.filter(
|
||||
models.Branch.adventure_id == imported["id"],
|
||||
models.Branch.superseded_at.isnot(None),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
origin = (
|
||||
db.query(models.Branch)
|
||||
.filter(
|
||||
models.Branch.adventure_id == client.adv_id,
|
||||
models.Branch.superseded_at.isnot(None),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
assert len(left) == len(origin) == 1
|
||||
assert left[0].superseded_depth == origin[0].superseded_depth
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_a_bundle_with_half_a_disposition_imports_as_active(client):
|
||||
"""Neither column is read to decide anything, so a file carrying one of them
|
||||
is taken as carrying neither rather than having its missing half invented."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
_play(client, "another way")
|
||||
bundle = _export(client)
|
||||
for branch in bundle["branches"]:
|
||||
branch.pop("supersededDepth", None)
|
||||
|
||||
imported = _import(client, bundle)
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert (
|
||||
db.query(models.Branch)
|
||||
.filter(
|
||||
models.Branch.adventure_id == imported["id"],
|
||||
models.Branch.superseded_at.isnot(None),
|
||||
)
|
||||
.count()
|
||||
== 0
|
||||
)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ------------------------------------- the narrator-edit guard (M3 closeout)
|
||||
|
||||
def _edit(client, action_id, text):
|
||||
return client.patch(
|
||||
f"/api/adventures/{client.adv_id}/actions/{action_id}", json={"text": text}
|
||||
)
|
||||
|
||||
|
||||
def _live_ai_at(adv_id, depth):
|
||||
return [
|
||||
a for a in _rows(adv_id)
|
||||
if a.type == "ai" and a.live and a.depth == depth
|
||||
][0]
|
||||
|
||||
|
||||
def test_editing_a_turn_with_nothing_after_it_is_still_allowed(client):
|
||||
"""The guard has to stay out of the way of the operation it protects. A
|
||||
correction to the newest turn contradicts nothing, and that is the case the
|
||||
edit control exists for."""
|
||||
_turns(client, 2)
|
||||
newest = [a for a in _rows(client.adv_id) if a.type == "ai"][-1]
|
||||
|
||||
r = _edit(client, newest.id, "Corrected.")
|
||||
|
||||
assert r.status_code == 200, r.text
|
||||
assert _texts(client)[-1] == "Corrected."
|
||||
|
||||
|
||||
def test_editing_a_turn_on_the_visible_story_is_still_allowed(client):
|
||||
"""Mid-story is not by itself unsafe: everything descending from the turn is
|
||||
on screen, so the reader can see what their correction has to agree with.
|
||||
Making that case fork is `STORY-BRANCH-SEMANTICS.md` §14-15 and belongs to
|
||||
M5 with the state re-evaluation it needs."""
|
||||
_turns(client, 3)
|
||||
older = [a for a in _rows(client.adv_id) if a.type == "ai"][0]
|
||||
|
||||
r = _edit(client, older.id, "Mara wears a green cloak.")
|
||||
|
||||
assert r.status_code == 200, r.text
|
||||
assert "Mara wears a green cloak." in _texts(client)
|
||||
|
||||
|
||||
def test_editing_a_turn_with_an_undone_future_is_refused(client):
|
||||
"""The first unsafe case: the story past the head is not on screen, so an
|
||||
edit here would silently change the words it was written from."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
at_head = [a for a in _rows(client.adv_id) if a.type == "ai" and a.live]
|
||||
target = sorted(at_head, key=lambda a: a.depth)[-2]
|
||||
before = target.text
|
||||
|
||||
r = _edit(client, target.id, "Something else entirely.")
|
||||
|
||||
assert r.status_code == 400
|
||||
assert "not on screen" in r.json()["detail"]
|
||||
# Refused, not partially applied.
|
||||
assert _rows(client.adv_id)[0].adventure_id == client.adv_id
|
||||
assert [a.text for a in _rows(client.adv_id) if a.id == target.id] == [before]
|
||||
|
||||
|
||||
def test_editing_a_turn_a_divergence_left_behind_is_refused(client):
|
||||
"""The second unsafe case, and the one a head check alone would miss: after
|
||||
a divergence the head is back at a tip, but a displaced line still runs on
|
||||
past the shared turn."""
|
||||
_turns(client, 3)
|
||||
shared = [a for a in _rows(client.adv_id) if a.type == "ai"][0]
|
||||
_undo(client)
|
||||
_undo(client)
|
||||
_play(client, "a different road")
|
||||
assert _adventure(client)["can_redo"] is False, "the head is at a tip again"
|
||||
|
||||
r = _edit(client, shared.id, "Rewritten under both lines.")
|
||||
|
||||
assert r.status_code == 400
|
||||
assert "left behind by a new continuation" in r.json()["detail"]
|
||||
|
||||
|
||||
def test_a_turn_the_displaced_line_does_not_descend_from_is_still_editable(client):
|
||||
"""The guard must be narrow. A branch that forked *before* a turn does not
|
||||
descend from it, so a correction there contradicts nothing on that line."""
|
||||
_turns(client, 2)
|
||||
_undo(client)
|
||||
_play(client, "a different road") # forks below turn 1
|
||||
_turns(client, 2) # and continues past the fork
|
||||
newest = [a for a in _rows(client.adv_id) if a.type == "ai" and a.live]
|
||||
target = sorted(newest, key=lambda a: a.depth)[-1]
|
||||
|
||||
r = _edit(client, target.id, "Corrected on the live line.")
|
||||
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
|
||||
def test_a_take_that_is_not_live_stays_editable(client):
|
||||
"""A take the story is not telling has no continuation of its own — keeping
|
||||
one is what forking is for — so correcting its text cannot contradict
|
||||
anything."""
|
||||
ScriptedProvider.replies = gold_replies()
|
||||
_play(client, "knock")
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
discarded = [a for a in _rows(client.adv_id) if a.type == "ai" and not a.live][0]
|
||||
|
||||
r = _edit(client, discarded.id, "The take nobody chose, corrected.")
|
||||
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
|
||||
def test_the_guard_lifts_when_the_story_is_brought_back(client):
|
||||
"""Refusal is a redirection, not a dead end: the error names Redo, so Redo
|
||||
has to make the edit possible again."""
|
||||
_turns(client, 3)
|
||||
_undo(client)
|
||||
live = sorted(
|
||||
[a for a in _rows(client.adv_id) if a.type == "ai" and a.live],
|
||||
key=lambda a: a.depth,
|
||||
)
|
||||
target = live[-2]
|
||||
assert _edit(client, target.id, "x").status_code == 400
|
||||
|
||||
_redo(client)
|
||||
|
||||
r = _edit(client, target.id, "Corrected once the story was whole again.")
|
||||
assert r.status_code == 200, r.text
|
||||
@@ -0,0 +1,269 @@
|
||||
"""What M2 removed stays removed, and what it must not break stays working.
|
||||
|
||||
A subtractive milestone needs tests that fail if the surface grows back. These
|
||||
are cheap, blunt, and deliberately not clever: they assert against the running
|
||||
app's route table, the shipped configuration files, and the settings API.
|
||||
|
||||
python -m pytest tests/test_local_only_surface.py -v
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.providers.openai_compatible import (
|
||||
CONNECT_TIMEOUT, DEFAULT_READ_TIMEOUT, OpenAICompatibleProvider,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _paths() -> set[str]:
|
||||
"""Every path the app serves, including those inside included routers."""
|
||||
found = set()
|
||||
|
||||
def walk(routes):
|
||||
for r in routes:
|
||||
path = getattr(r, "path", None)
|
||||
if path:
|
||||
found.add(path)
|
||||
walk(getattr(r, "routes", []) or [])
|
||||
|
||||
walk(app.routes)
|
||||
return found
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client():
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
# --- the removed surfaces ------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("prefix", ["/api/auth", "/api/analytics", "/api/scripts"])
|
||||
def test_no_route_serves_a_removed_subsystem(prefix):
|
||||
"""Accounts, the visitor dashboard, and campaign scripting are gone as
|
||||
routes, not merely hidden behind a flag."""
|
||||
assert not [p for p in _paths() if p.startswith(prefix)], prefix
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", [
|
||||
"/api/auth/me", "/api/auth/login", "/api/auth/register", "/api/auth/logout",
|
||||
"/api/analytics/summary", "/api/analytics/collect", "/api/analytics/access",
|
||||
"/api/scripts", "/api/adventures/1/scripts", "/api/adventures/1/script-state",
|
||||
])
|
||||
def test_a_removed_endpoint_answers_404(client, path):
|
||||
assert client.get(path).status_code == 404, path
|
||||
|
||||
|
||||
def test_the_application_has_no_scripting_engine():
|
||||
with pytest.raises(ImportError):
|
||||
__import__("app.scripting")
|
||||
|
||||
|
||||
def test_no_module_imports_quickjs():
|
||||
"""The dependency is gone from requirements; this catches an import that
|
||||
would put it back."""
|
||||
for py in (REPO / "backend" / "app").rglob("*.py"):
|
||||
assert "import quickjs" not in py.read_text(), py
|
||||
|
||||
|
||||
def test_requirements_carry_no_hosted_dependencies():
|
||||
text = (REPO / "backend" / "requirements.txt").read_text()
|
||||
for gone in ("quickjs", "psycopg", "cryptography"):
|
||||
assert gone not in text, gone
|
||||
|
||||
|
||||
def test_no_render_deployment_config():
|
||||
assert not (REPO / "render.yaml").exists()
|
||||
|
||||
|
||||
# --- no cloud provider, no key ------------------------------------------
|
||||
|
||||
def test_settings_expose_no_api_key_field(client):
|
||||
body = client.get("/api/settings").json()
|
||||
assert "api_key" not in body
|
||||
assert "has_api_key" not in body
|
||||
|
||||
|
||||
def test_an_api_key_cannot_be_set_through_the_api(client):
|
||||
"""Pydantic ignores unknown fields, so this asserts the value does not
|
||||
land rather than that the request is refused."""
|
||||
client.put("/api/settings", json={"api_key": "sk-should-not-stick"})
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert db.query(models.Settings).first().api_key == ""
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_the_provider_sends_no_authorization_header():
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
assert "Authorization" not in provider._headers()
|
||||
|
||||
|
||||
# --- the model timeout ---------------------------------------------------
|
||||
|
||||
def test_the_default_timeout_is_generous_but_finite():
|
||||
"""M1 measured a cold model load exceeding the inherited hardcoded 120s on
|
||||
a CPU-only host. It must be longer than that, and it must be a number."""
|
||||
assert DEFAULT_READ_TIMEOUT > 120
|
||||
assert DEFAULT_READ_TIMEOUT <= 3600
|
||||
|
||||
|
||||
def test_connect_stays_short_while_reading_stays_patient():
|
||||
"""A wrong address should fail in seconds; a loading model should not."""
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
timeout = provider._timeout()
|
||||
assert timeout.connect == CONNECT_TIMEOUT <= 30
|
||||
assert timeout.read == DEFAULT_READ_TIMEOUT
|
||||
|
||||
|
||||
def test_the_timeout_is_configurable(client):
|
||||
r = client.put("/api/settings", json={"model_timeout_seconds": 900})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.get("/api/settings").json()["model_timeout_seconds"] == 900
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [0, 29, 3601, -1])
|
||||
def test_an_unusable_timeout_is_refused(client, value):
|
||||
"""Not zero, not negative, and not "wait forever" spelled as a big number."""
|
||||
assert client.put(
|
||||
"/api/settings", json={"model_timeout_seconds": value}
|
||||
).status_code == 422
|
||||
|
||||
|
||||
def test_the_provider_honours_the_configured_timeout():
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m", read_timeout=45)
|
||||
assert provider._timeout().read == 45
|
||||
|
||||
|
||||
# --- the storyteller stays on loopback -----------------------------------
|
||||
|
||||
def test_the_native_start_scripts_bind_loopback():
|
||||
for script in ("start.sh", "start.ps1"):
|
||||
text = (REPO / script).read_text(errors="ignore")
|
||||
assert "--host 127.0.0.1" in text, script
|
||||
assert "--host 0.0.0.0" not in text, script
|
||||
|
||||
|
||||
def test_compose_publishes_to_loopback_only():
|
||||
"""The container listens on 0.0.0.0 because a published port cannot reach
|
||||
anything else. What must stay loopback is the *host* side of the mapping."""
|
||||
text = (REPO / "docker-compose.yml").read_text()
|
||||
published = re.findall(r'^\s*-\s*"([^"]+)"', text, re.M)
|
||||
assert published, "no published ports found — has the file moved?"
|
||||
for mapping in published:
|
||||
assert mapping.startswith("127.0.0.1:"), mapping
|
||||
|
||||
|
||||
# --- every provider factory must actually build ---------------------------
|
||||
|
||||
def test_every_provider_factory_builds_from_a_real_settings_row(client):
|
||||
"""M2 shipped with a defect this test would have caught.
|
||||
|
||||
`Settings.api_key_plain` was removed with the API key, but `memorybank`'s
|
||||
two provider factories still read it. Nothing failed at import, and no test
|
||||
noticed, because every memory test stubs those factories out — so the break
|
||||
only appeared at runtime, in a background task, as a swallowed
|
||||
`AttributeError` that silently stopped summaries and embeddings.
|
||||
|
||||
Constructing each factory from a real row is the cheapest thing that would
|
||||
have caught it, and it catches the same shape of mistake next time a
|
||||
Settings column moves.
|
||||
"""
|
||||
from app import memorybank
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.embedding_model = "nomic-embed-text:latest"
|
||||
settings.summary_model = ""
|
||||
db.commit()
|
||||
|
||||
summary = memorybank.summary_provider(settings)
|
||||
assert summary.model == settings.model # falls back to the narrator
|
||||
assert summary.base_url == settings.endpoint_url.rstrip("/")
|
||||
|
||||
embed = memorybank.embedding_provider(settings)
|
||||
assert embed.model == "nomic-embed-text:latest"
|
||||
|
||||
turn = OpenAICompatibleProvider(
|
||||
settings.endpoint_url, settings.model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
assert turn.read_timeout == settings.model_timeout_seconds
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_the_configured_timeout_reaches_every_generating_client(client, monkeypatch):
|
||||
"""The second defect this review caught. The setting was stored, validated
|
||||
and exposed, and then not passed to the provider — so the turn engine kept
|
||||
using the module default and "configurable" was a claim rather than a fact.
|
||||
|
||||
Each generating path is driven for real and the constructed provider is
|
||||
recorded. Embeddings are deliberately excluded: they are short, never
|
||||
cold-load a large model, and keep their own shorter constant.
|
||||
"""
|
||||
from app import memorybank
|
||||
from app.routers import chat as chat_router
|
||||
from app.routers.adventures import turns as turns_router
|
||||
|
||||
seen = []
|
||||
|
||||
class Recorder:
|
||||
last_usage = None
|
||||
|
||||
def __init__(self, endpoint_url, model, api_mode="chat", read_timeout=None):
|
||||
seen.append(read_timeout)
|
||||
|
||||
async def generate(self, *a, **k):
|
||||
yield ("text", "narration")
|
||||
|
||||
async def chat(self, *a, **k):
|
||||
yield ("text", "reply")
|
||||
|
||||
monkeypatch.setattr(turns_router, "OpenAICompatibleProvider", Recorder)
|
||||
monkeypatch.setattr(chat_router, "OpenAICompatibleProvider", Recorder)
|
||||
monkeypatch.setattr(memorybank, "OpenAICompatibleProvider", Recorder)
|
||||
|
||||
assert client.put("/api/settings", json={"model_timeout_seconds": 777}).status_code == 200
|
||||
|
||||
adv = client.post("/api/adventures", json={"title": "T"}).json()["id"]
|
||||
client.post(f"/api/adventures/{adv}/actions", json={"type": "story", "text": "hello"})
|
||||
assert seen and seen[-1] == 777, f"turn engine used {seen[-1]!r}"
|
||||
|
||||
client.post("/api/chat/stream", json={"messages": [{"role": "user", "content": "hi"}]})
|
||||
assert seen[-1] == 777, f"chat used {seen[-1]!r}"
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
memorybank.summary_provider(db.query(models.Settings).first())
|
||||
finally:
|
||||
db.close()
|
||||
assert seen[-1] == 777, f"summarizer used {seen[-1]!r}"
|
||||
@@ -128,7 +128,7 @@ def fill_bank(db, adventure, *, count=2):
|
||||
def options(**overrides):
|
||||
args = dict(write=False, adventure=None, email=None, limit=None,
|
||||
include_forgotten=False, embed=False, endpoint=None,
|
||||
model=None, api_key=None)
|
||||
model=None)
|
||||
args.update(overrides)
|
||||
return argparse.Namespace(**args)
|
||||
|
||||
@@ -248,23 +248,6 @@ def test_the_rewrite_prompt_is_the_one_the_app_sends(db):
|
||||
|
||||
# --------------------------------------------------------------------- the tool
|
||||
|
||||
def test_the_database_line_carries_no_password():
|
||||
"""The report names the database it is about to rewrite. That line ends up
|
||||
in a console, a screenshot or a pasted bug report."""
|
||||
shown = rewrite_memories.safe_dsn(
|
||||
"postgresql://parth:hunter2@ep-cool-frost.us-east-1.aws.neon.tech/aidnd"
|
||||
"?sslmode=require")
|
||||
assert "hunter2" not in shown
|
||||
assert "sslmode" not in shown # a password can be passed there too
|
||||
assert shown == ("postgresql://parth@ep-cool-frost.us-east-1.aws.neon.tech"
|
||||
"/aidnd")
|
||||
|
||||
|
||||
def test_an_unparseable_database_url_shows_nothing_at_all():
|
||||
assert rewrite_memories.safe_dsn("not-a-url") == "(configured)"
|
||||
|
||||
|
||||
|
||||
def test_without_write_nothing_changes(db, monkeypatch):
|
||||
adventure = make_adventure(db)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
@@ -316,9 +299,12 @@ def test_a_hand_written_memory_is_left_alone(db, monkeypatch):
|
||||
assert typed.text == "Kaelen owes a debt."
|
||||
|
||||
|
||||
def test_an_owner_with_no_api_key_is_skipped(db, monkeypatch):
|
||||
"""Summarization spends the user's own key and never the shared demo key."""
|
||||
adventure = make_adventure(db, key="")
|
||||
def test_an_adventure_with_no_model_configured_is_skipped(db, monkeypatch):
|
||||
"""Nothing to summarize with, so the memory is reported rather than
|
||||
silently left as it was. This used to be about the owner's API key; after
|
||||
M2 the thing that can be missing is the model name."""
|
||||
adventure = make_adventure(db)
|
||||
_blank_model(db, adventure)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
stub = StubSummarizer()
|
||||
monkeypatch.setattr(memorybank, "summary_provider", lambda s: stub)
|
||||
@@ -328,10 +314,10 @@ def test_an_owner_with_no_api_key_is_skipped(db, monkeypatch):
|
||||
assert first.text == "You entered the crypt 1."
|
||||
|
||||
|
||||
def test_an_api_key_on_the_command_line_covers_that_owner(db, monkeypatch):
|
||||
"""The way to summarize for an owner who has no key of their own, and the
|
||||
way to point a run at the local Claude shim instead of a paid endpoint."""
|
||||
adventure = make_adventure(db, key="")
|
||||
def test_a_model_on_the_command_line_covers_that_adventure(db, monkeypatch):
|
||||
"""How to run the rewrite against a model other than the configured one."""
|
||||
adventure = make_adventure(db)
|
||||
_blank_model(db, adventure)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
built: list[tuple] = []
|
||||
|
||||
@@ -340,11 +326,22 @@ def test_an_api_key_on_the_command_line_covers_that_owner(db, monkeypatch):
|
||||
return StubSummarizer()
|
||||
|
||||
monkeypatch.setattr("app.providers.OpenAICompatibleProvider", build)
|
||||
assert run_tool(options(write=True, api_key="sk-cli", model="sonnet",
|
||||
assert run_tool(options(write=True, model="other-model",
|
||||
endpoint="http://127.0.0.1:8787/v1")) == 0
|
||||
db.expire_all()
|
||||
assert first.text == "Kaelen entered the crypt 1."
|
||||
assert built[0][:3] == ("http://127.0.0.1:8787/v1", "sk-cli", "sonnet")
|
||||
assert built[0][:2] == ("http://127.0.0.1:8787/v1", "other-model")
|
||||
|
||||
|
||||
def _blank_model(db, adventure) -> None:
|
||||
settings = (
|
||||
db.query(models.Settings)
|
||||
.filter(models.Settings.user_id == adventure.user_id)
|
||||
.first()
|
||||
)
|
||||
settings.model = ""
|
||||
settings.summary_model = ""
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_limit_stops_early(db, monkeypatch):
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
"""Tests for the SSRF guard on the user-supplied BYOK endpoint_url.
|
||||
|
||||
python -m pytest tests/test_netguard.py -v
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from app import auth, netguard
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def hosted(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
|
||||
|
||||
def _resolves_to(monkeypatch, ip: str):
|
||||
"""Pin `getaddrinfo` so the test exercises the address decision, not real DNS."""
|
||||
monkeypatch.setattr(
|
||||
netguard.socket, "getaddrinfo",
|
||||
lambda *a, **k: [(2, 1, 6, "", (ip, 443))],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ip", [
|
||||
"127.0.0.1", # loopback
|
||||
"169.254.169.254", # cloud metadata (link-local)
|
||||
"10.0.0.5", # RFC1918
|
||||
"192.168.1.1", # RFC1918
|
||||
"172.16.0.9", # RFC1918
|
||||
"0.0.0.0", # unspecified
|
||||
"100.64.0.1", # carrier-grade NAT
|
||||
"::1", # IPv6 loopback
|
||||
"fd00::1", # IPv6 unique-local
|
||||
])
|
||||
def test_blocks_non_public_addresses(hosted, monkeypatch, ip):
|
||||
_resolves_to(monkeypatch, ip)
|
||||
assert netguard.endpoint_block_reason("https://evil.example.com/v1") is not None
|
||||
|
||||
|
||||
def test_allows_public_address(hosted, monkeypatch):
|
||||
_resolves_to(monkeypatch, "104.18.0.1") # a public IP
|
||||
assert netguard.endpoint_block_reason("https://openrouter.ai/api/v1") is None
|
||||
|
||||
|
||||
def test_rejects_non_http_scheme(hosted):
|
||||
assert netguard.endpoint_block_reason("file:///etc/passwd") is not None
|
||||
assert netguard.endpoint_block_reason("gopher://x/") is not None
|
||||
|
||||
|
||||
def test_unresolvable_host_is_blocked(hosted, monkeypatch):
|
||||
def boom(*a, **k):
|
||||
raise netguard.socket.gaierror("no such host")
|
||||
monkeypatch.setattr(netguard.socket, "getaddrinfo", boom)
|
||||
assert netguard.endpoint_block_reason("https://nope.invalid/v1") is not None
|
||||
|
||||
|
||||
def test_noop_in_local_mode(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
# Local installs must reach localhost (Ollama). The guard never blocks local mode.
|
||||
assert netguard.endpoint_block_reason("http://localhost:11434/v1") is None
|
||||
assert netguard.endpoint_block_reason("http://127.0.0.1:11434/v1") is None
|
||||
@@ -275,7 +275,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -0,0 +1,325 @@
|
||||
"""D11 and L03 across a genuine OS process boundary.
|
||||
|
||||
The rest of the suite runs the app in-process through `TestClient`, which is the
|
||||
right tool for almost everything and the wrong one for exactly one claim:
|
||||
*durability*. A Save Point that survived only because a Python object was still
|
||||
alive would pass a same-process test and fail a user's restart. `TestClient`
|
||||
cannot tell those apart, so the M4 review recorded the shipped D11/L03 tests as
|
||||
weaker than the acceptance items they were named for (§R B-3).
|
||||
|
||||
This module closes that. It starts the real application as a **subprocess**,
|
||||
plays a story over HTTP, kills the process, starts a **second** process against
|
||||
the same database file, and only then asks whether the Save Point is still
|
||||
there. Everything crossing the boundary crosses it as bytes on disk.
|
||||
|
||||
Deterministic and local: the spawned server replaces the model with a scripted
|
||||
provider (`_restart_server.py`), so there is no Ollama, no network and no
|
||||
sleep-and-hope — readiness is probed, not waited for.
|
||||
|
||||
python -m pytest tests/test_process_restart.py -v
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from fakes import GOLD_PER_TURN
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
SERVER = HERE / "_restart_server.py"
|
||||
|
||||
# How long a spawned server may take to answer before the test gives up. The
|
||||
# process imports the app and runs migrations on a fresh file, which is well
|
||||
# under a second on this project; the ceiling is for a loaded machine.
|
||||
STARTUP_TIMEOUT = 60.0
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
"""Returns a port nothing is listening on.
|
||||
|
||||
Bind, read, release. There is a race between releasing and the server
|
||||
claiming it, which is why the caller probes for readiness rather than
|
||||
assuming success — a lost race shows up as a startup timeout, not as a
|
||||
silent pass.
|
||||
"""
|
||||
with socket.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
|
||||
class Server:
|
||||
"""One storyteller process, and the HTTP calls the test makes against it."""
|
||||
|
||||
def __init__(self, db_path: str, port: int):
|
||||
self.port = port
|
||||
self.proc = subprocess.Popen(
|
||||
[sys.executable, str(SERVER), db_path, str(port)],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
# Never inherit the parent's database redirection; the child is told
|
||||
# which file to open on its command line.
|
||||
env={**os.environ, "AIDND_DB_PATH": db_path},
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ lifecycle
|
||||
|
||||
def wait_until_ready(self) -> None:
|
||||
deadline = time.monotonic() + STARTUP_TIMEOUT
|
||||
while time.monotonic() < deadline:
|
||||
if self.proc.poll() is not None:
|
||||
raise AssertionError(
|
||||
f"server exited early ({self.proc.returncode}):\n{self._output()}"
|
||||
)
|
||||
try:
|
||||
self.call("GET", "/settings")
|
||||
return
|
||||
except (urllib.error.URLError, ConnectionError, OSError):
|
||||
time.sleep(0.05)
|
||||
raise AssertionError(f"server never became ready:\n{self._output()}")
|
||||
|
||||
def stop(self) -> None:
|
||||
"""Ends the process, and does not return until it is actually gone."""
|
||||
if self.proc.poll() is None:
|
||||
self.proc.terminate()
|
||||
try:
|
||||
self.proc.wait(timeout=15)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.proc.kill()
|
||||
self.proc.wait(timeout=15)
|
||||
if self.proc.stdout is not None:
|
||||
self.proc.stdout.close()
|
||||
|
||||
def _output(self) -> str:
|
||||
if self.proc.stdout is None:
|
||||
return "(no output captured)"
|
||||
try:
|
||||
return self.proc.stdout.read().decode(errors="replace")[-2000:]
|
||||
except Exception:
|
||||
return "(output unreadable)"
|
||||
|
||||
def is_listening(self) -> bool:
|
||||
try:
|
||||
self.call("GET", "/settings")
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
# ---------------------------------------------------------------- HTTP
|
||||
|
||||
def call(self, method: str, path: str, payload=None, expect: int | None = None):
|
||||
data = json.dumps(payload).encode() if payload is not None else None
|
||||
request = urllib.request.Request(
|
||||
f"http://127.0.0.1:{self.port}/api{path}",
|
||||
data=data,
|
||||
method=method,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=60) as response:
|
||||
body, status = response.read(), response.status
|
||||
except urllib.error.HTTPError as exc: # a real answer, not a failure
|
||||
body, status = exc.read(), exc.code
|
||||
if expect is not None and status != expect:
|
||||
raise AssertionError(f"{method} {path} -> {status}: {body[:400]!r}")
|
||||
return json.loads(body) if body and status != 204 else None
|
||||
|
||||
def play(self, adventure_id: int, text: str) -> None:
|
||||
"""Plays one turn through the streaming endpoint, to completion."""
|
||||
request = urllib.request.Request(
|
||||
f"http://127.0.0.1:{self.port}/api/adventures/{adventure_id}/actions",
|
||||
data=json.dumps({"type": "do", "text": text}).encode(),
|
||||
method="POST",
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
with urllib.request.urlopen(request, timeout=120) as response:
|
||||
response.read()
|
||||
|
||||
# ------------------------------------------------------------- reading
|
||||
|
||||
def transcript(self, adventure_id: int) -> list[str]:
|
||||
page = self.call("GET", f"/adventures/{adventure_id}", expect=200)
|
||||
return [a["text"] for a in page["actions"]]
|
||||
|
||||
def gold(self, adventure_id: int) -> int:
|
||||
state = self.call("GET", f"/adventures/{adventure_id}/world-state", expect=200)
|
||||
return state["state"]["player"]["gold"]
|
||||
|
||||
def total_rows(self, adventure_id: int) -> int:
|
||||
"""Every row of the whole tree, head or no head.
|
||||
|
||||
`action_count` is scoped to the path being read, so it falls when the
|
||||
head moves back — which is the feature, not a deletion. The export
|
||||
carries the entire tree whatever the head is doing, so it is what
|
||||
"nothing was deleted" has to be measured against.
|
||||
"""
|
||||
bundle = self.call("GET", f"/adventures/{adventure_id}/export", expect=200)
|
||||
return len(bundle["actions"])
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def workspace():
|
||||
"""A database file, and whichever servers a test starts against it."""
|
||||
directory = tempfile.mkdtemp(prefix="m4-restart-")
|
||||
db_path = os.path.join(directory, "campaign.db")
|
||||
started: list[Server] = []
|
||||
|
||||
def start() -> Server:
|
||||
server = Server(db_path, _free_port())
|
||||
started.append(server)
|
||||
server.wait_until_ready()
|
||||
return server
|
||||
|
||||
try:
|
||||
yield start
|
||||
finally:
|
||||
# Every child dies even if the test failed part way through, and each
|
||||
# stop() waits, so a later test cannot inherit a live listener.
|
||||
for server in started:
|
||||
server.stop()
|
||||
|
||||
|
||||
def _campaign_with_a_save_point(server: Server):
|
||||
"""Three turns, a Save Point on the third, then four more turns.
|
||||
|
||||
Returns everything the second process has to be able to reproduce.
|
||||
"""
|
||||
scenario = server.call("POST", "/scenarios", {
|
||||
"title": "Abbey",
|
||||
"stat_schema": {"player": {"gold": {"initial": 0, "min": 0, "max": 9999}}},
|
||||
}, expect=201)
|
||||
adventure = server.call("POST", "/adventures", {
|
||||
"title": "The abbey", "scenario_id": scenario["id"],
|
||||
}, expect=201)
|
||||
adventure_id = adventure["id"]
|
||||
|
||||
for n in range(3):
|
||||
server.play(adventure_id, f"turn {n}")
|
||||
at_save = {
|
||||
"transcript": server.transcript(adventure_id),
|
||||
"gold": server.gold(adventure_id),
|
||||
}
|
||||
save_point = server.call("POST", f"/adventures/{adventure_id}/checkpoints",
|
||||
{"name": "Before entering the abbey"}, expect=201)
|
||||
|
||||
for n in range(4):
|
||||
server.play(adventure_id, f"later {n}")
|
||||
at_tip = {
|
||||
"transcript": server.transcript(adventure_id),
|
||||
"gold": server.gold(adventure_id),
|
||||
"rows": server.total_rows(adventure_id),
|
||||
}
|
||||
return adventure_id, save_point, at_save, at_tip
|
||||
|
||||
|
||||
def test_d11_l03_a_save_point_survives_a_real_process_restart(workspace):
|
||||
"""D11 and L03 together, across a boundary a same-process test cannot cross.
|
||||
|
||||
The first process writes the campaign and exits. The second process is a
|
||||
different interpreter with an empty session, an empty identity map and no
|
||||
memory of anything — everything it knows, it reads off the disk.
|
||||
"""
|
||||
first = workspace()
|
||||
adventure_id, save_point, at_save, at_tip = _campaign_with_a_save_point(first)
|
||||
|
||||
assert at_tip["gold"] == at_save["gold"] + 4 * GOLD_PER_TURN
|
||||
assert len(at_tip["transcript"]) == len(at_save["transcript"]) + 8
|
||||
|
||||
# --- the boundary -----------------------------------------------------
|
||||
first.stop()
|
||||
assert first.proc.poll() is not None, "the first server did not actually exit"
|
||||
assert not first.is_listening(), "the first server is still answering"
|
||||
|
||||
second = workspace()
|
||||
assert second.proc.pid != first.proc.pid
|
||||
|
||||
# --- D11: the Save Point is still there -------------------------------
|
||||
listed = second.call("GET", f"/adventures/{adventure_id}/checkpoints", expect=200)
|
||||
assert [c["id"] for c in listed] == [save_point["id"]]
|
||||
assert listed[0]["name"] == "Before entering the abbey"
|
||||
assert (listed[0]["branch_id"], listed[0]["depth"]) == (
|
||||
save_point["branch_id"], save_point["depth"]
|
||||
)
|
||||
assert listed[0]["resolved"] is True
|
||||
# The story came back too, at the position the first process left it.
|
||||
assert second.transcript(adventure_id) == at_tip["transcript"]
|
||||
|
||||
rows_before_restore = second.total_rows(adventure_id)
|
||||
assert rows_before_restore == at_tip["rows"]
|
||||
|
||||
# --- L03: restoring reconstructs the historical position --------------
|
||||
page = second.call(
|
||||
"POST", f"/adventures/{adventure_id}/checkpoints/{save_point['id']}/restore",
|
||||
expect=200,
|
||||
)
|
||||
assert [a["text"] for a in page["actions"]] == at_save["transcript"]
|
||||
assert second.transcript(adventure_id) == at_save["transcript"]
|
||||
assert second.gold(adventure_id) == at_save["gold"]
|
||||
|
||||
# --- restore deleted nothing, and Redo is still available -------------
|
||||
assert second.total_rows(adventure_id) == rows_before_restore
|
||||
assert page["can_redo"] is True
|
||||
assert second.call("GET", f"/adventures/{adventure_id}", expect=200)["can_redo"] is True
|
||||
|
||||
|
||||
def test_the_retained_continuation_is_reachable_after_a_restart(workspace):
|
||||
"""The later history is not merely present in the database after a restart —
|
||||
it is still the continuation the story tells, walkable by ordinary Redo."""
|
||||
first = workspace()
|
||||
adventure_id, save_point, at_save, at_tip = _campaign_with_a_save_point(first)
|
||||
first.stop()
|
||||
|
||||
second = workspace()
|
||||
second.call("POST",
|
||||
f"/adventures/{adventure_id}/checkpoints/{save_point['id']}/restore",
|
||||
expect=200)
|
||||
assert second.transcript(adventure_id) == at_save["transcript"]
|
||||
|
||||
steps = 0
|
||||
while second.call("GET", f"/adventures/{adventure_id}", expect=200)["can_redo"]:
|
||||
second.call("POST", f"/adventures/{adventure_id}/redo", expect=200)
|
||||
steps += 1
|
||||
assert steps <= 10, "Redo never stopped"
|
||||
|
||||
assert steps == 4
|
||||
assert second.transcript(adventure_id) == at_tip["transcript"]
|
||||
assert second.gold(adventure_id) == at_tip["gold"]
|
||||
assert second.total_rows(adventure_id) == at_tip["rows"]
|
||||
|
||||
|
||||
def test_a_divergent_write_after_a_restart_keeps_the_displaced_future(workspace):
|
||||
"""D13's second half, across the boundary: the fork still happens on the
|
||||
write rather than on the restore, and the displaced turns keep their rows."""
|
||||
first = workspace()
|
||||
adventure_id, save_point, at_save, at_tip = _campaign_with_a_save_point(first)
|
||||
first.stop()
|
||||
|
||||
second = workspace()
|
||||
second.call("POST",
|
||||
f"/adventures/{adventure_id}/checkpoints/{save_point['id']}/restore",
|
||||
expect=200)
|
||||
branches_before = second.call("GET", f"/adventures/{adventure_id}/branches",
|
||||
expect=200)
|
||||
assert len(branches_before) == 1, "restore must not fork"
|
||||
|
||||
second.play(adventure_id, "go around the back")
|
||||
|
||||
branches_after = second.call("GET", f"/adventures/{adventure_id}/branches",
|
||||
expect=200)
|
||||
assert len(branches_after) == 2, "the write should have forked"
|
||||
# Nothing was deleted to achieve it: the tree grew by the new turn only.
|
||||
assert second.total_rows(adventure_id) == at_tip["rows"] + 2
|
||||
# Ordinary Redo no longer offers the displaced future.
|
||||
assert second.call("GET", f"/adventures/{adventure_id}", expect=200)["can_redo"] is False
|
||||
# And the Save Point still names the position it always did.
|
||||
listed = second.call("GET", f"/adventures/{adventure_id}/checkpoints", expect=200)
|
||||
assert (listed[0]["branch_id"], listed[0]["depth"]) == (
|
||||
save_point["branch_id"], save_point["depth"]
|
||||
)
|
||||
@@ -17,10 +17,10 @@ Three things must hold, and each is easy to undo by accident:
|
||||
* Moving a section out of the system block does not drop it from the
|
||||
token budget. It is still in the prompt.
|
||||
|
||||
This file also covers two request-level concerns: preferring one
|
||||
OpenRouter upstream, since each upstream holds its own cache, and reading
|
||||
back the usage the endpoint reports, so the hit rate is measurable rather
|
||||
than assumed.
|
||||
This file also covers one request-level concern: reading back the usage
|
||||
the endpoint reports, so the hit rate is measurable rather than assumed.
|
||||
It covered OpenRouter upstream pinning too, until M2 removed cloud
|
||||
provider support.
|
||||
|
||||
python -m pytest tests/test_prompt_caching.py -v
|
||||
"""
|
||||
@@ -40,50 +40,11 @@ SCHEMA = {
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------- preferring one upstream
|
||||
|
||||
def _routed(endpoint, model):
|
||||
provider = OpenAICompatibleProvider(endpoint, "k", model, "chat", 0)
|
||||
body = {"max_tokens": 100}
|
||||
provider._apply_provider_routing(body)
|
||||
return body
|
||||
|
||||
|
||||
def test_openrouter_deepseek_pins_the_upstream():
|
||||
"""Each upstream has its own cache, so routing has to be deterministic."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert body["provider"] == {"order": ["deepseek"]}
|
||||
|
||||
|
||||
def test_fallbacks_stay_on():
|
||||
"""A preference, not a restriction: if deepseek is down the turn still runs
|
||||
somewhere else and merely misses the cache."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert "allow_fallbacks" not in body["provider"]
|
||||
|
||||
|
||||
def test_non_openrouter_endpoints_get_no_provider_field():
|
||||
"""Ollama and other providers reject fields they do not know. This is
|
||||
the same problem the `reasoning` param works around."""
|
||||
body = _routed("http://localhost:11434/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert "provider" not in body
|
||||
|
||||
|
||||
def test_unknown_vendors_are_left_alone():
|
||||
"""The vendor half of a slug is not reliably a provider slug: Google's
|
||||
models are served by "google-ai-studio", and there is no "google". A guess
|
||||
would be a routing preference naming an upstream that does not exist."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "google/gemma-4-26b-a4b-it:free")
|
||||
assert "provider" not in body
|
||||
|
||||
|
||||
# ------------------------------------------------------ reading usage back
|
||||
|
||||
def test_usage_is_recorded_from_a_final_chunk():
|
||||
"""In a stream, the usage block arrives in a final chunk that carries
|
||||
no choices, which is why it is read separately from the text
|
||||
extraction."""
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "k", "m")
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
assert provider.last_usage is None
|
||||
provider._record_usage({"choices": [{"delta": {"content": "hi"}}]})
|
||||
assert provider.last_usage is None
|
||||
@@ -95,7 +56,7 @@ def test_usage_is_recorded_from_a_final_chunk():
|
||||
|
||||
|
||||
def test_a_later_chunk_without_usage_does_not_erase_it():
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "k", "m")
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
provider._record_usage({"usage": {"prompt_tokens": 5}})
|
||||
provider._record_usage({"choices": [{"delta": {"content": "x"}}]})
|
||||
provider._record_usage({"usage": {}})
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
"""Regression tests for the X-Forwarded-For rate-limit bypass and the
|
||||
per-account login throttle added to close it.
|
||||
|
||||
Background: uvicorn's `--forwarded-allow-ips "*"` trusted the leftmost
|
||||
`X-Forwarded-For` entry. The client controls that entry, so rotating the
|
||||
header issued a fresh rate-limit bucket on every request. `client_ip` now
|
||||
reads the hop the trusted edge appends, which is the rightmost one. Login
|
||||
also has an email-keyed throttle that no IP trick can weaken.
|
||||
|
||||
python -m pytest tests/test_ratelimit_hardening.py -v
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from app import auth, limits
|
||||
|
||||
|
||||
class _Req:
|
||||
"""Minimal stand-in for starlette's Request: a header lookup and a peer."""
|
||||
|
||||
def __init__(self, xff: str | None, peer: str | None = "10.0.0.1"):
|
||||
self.headers = {} if xff is None else {"x-forwarded-for": xff}
|
||||
self.client = None if peer is None else type("C", (), {"host": peer})()
|
||||
|
||||
|
||||
# ---------- client_ip: the spoof-resistant hop ----------
|
||||
|
||||
def test_client_ip_takes_appended_rightmost_hop(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
|
||||
# An attacker prepends a fake IP. The edge appends the real one on the right.
|
||||
req = _Req("203.0.113.9, 198.51.100.77")
|
||||
assert limits.client_ip(req) == "198.51.100.77"
|
||||
|
||||
|
||||
def test_client_ip_ignores_spoofed_leftmost(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
|
||||
# The keyed IP stays the real hop regardless of what the client adds on
|
||||
# the left, so rotating that value no longer creates a new bucket.
|
||||
a = limits.client_ip(_Req("1.1.1.1, 198.51.100.77"))
|
||||
b = limits.client_ip(_Req("2.2.2.2, 198.51.100.77"))
|
||||
c = limits.client_ip(_Req("evil, junk, 198.51.100.77"))
|
||||
assert a == b == c == "198.51.100.77"
|
||||
|
||||
|
||||
def test_client_ip_honours_extra_trusted_hops(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 2)
|
||||
# Two trusted hops: real client is second from the right.
|
||||
req = _Req("9.9.9.9, 203.0.113.5, 198.51.100.77")
|
||||
assert limits.client_ip(req) == "203.0.113.5"
|
||||
|
||||
|
||||
def test_client_ip_falls_back_to_socket_peer():
|
||||
assert limits.client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4"
|
||||
assert limits.client_ip(_Req(None, peer=None)) == "unknown"
|
||||
|
||||
|
||||
# ---------- per-account login throttle ----------
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _multi_user(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
# Isolate the module-level failure map for each test.
|
||||
from collections import defaultdict, deque
|
||||
monkeypatch.setattr(limits, "_login_fails", defaultdict(deque))
|
||||
|
||||
|
||||
def test_login_throttle_blocks_after_limit():
|
||||
email = "victim@example.com"
|
||||
# Each attempt up to the limit is allowed and recorded as a failure.
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.check_login_allowed(email) # does not raise
|
||||
limits.note_login_failure(email)
|
||||
# One more failure exceeds the limit.
|
||||
with pytest.raises(limits.HTTPException) as exc:
|
||||
limits.check_login_allowed(email)
|
||||
assert exc.value.status_code == 429
|
||||
|
||||
|
||||
def test_login_throttle_is_per_account():
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.note_login_failure("a@example.com")
|
||||
with pytest.raises(limits.HTTPException):
|
||||
limits.check_login_allowed("a@example.com")
|
||||
# A different account is unaffected because the throttle keys on email, not IP.
|
||||
limits.check_login_allowed("b@example.com") # must not raise
|
||||
|
||||
|
||||
def test_successful_login_clears_the_streak():
|
||||
email = "typo@example.com"
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.note_login_failure(email)
|
||||
limits.note_login_success(email)
|
||||
limits.check_login_allowed(email) # failure streak cleared, must not raise
|
||||
|
||||
|
||||
def test_throttle_is_noop_in_local_mode(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT * 3):
|
||||
limits.note_login_failure("solo@example.com")
|
||||
limits.check_login_allowed("solo@example.com") # never throttled locally
|
||||
@@ -1,45 +0,0 @@
|
||||
"""What the provider puts in the `reasoning` request field for each budget
|
||||
setting: a positive budget asks for thinking, 0 stays silent, -1 turns it off.
|
||||
|
||||
python -m pytest tests/test_reasoning_param.py -v
|
||||
"""
|
||||
from app.providers.openai_compatible import OpenAICompatibleProvider
|
||||
|
||||
|
||||
def _body(reasoning_max_tokens, api_mode="chat", max_tokens=1000):
|
||||
provider = OpenAICompatibleProvider(
|
||||
"https://openrouter.ai/api/v1", "k", "deepseek/deepseek-v4-flash-0731",
|
||||
api_mode, reasoning_max_tokens,
|
||||
)
|
||||
body = {"max_tokens": max_tokens}
|
||||
provider._apply_reasoning_budget(body)
|
||||
return body
|
||||
|
||||
|
||||
def test_zero_sends_nothing():
|
||||
"""Ollama and other providers reject unknown fields. Sending 0 must not add a `reasoning` field."""
|
||||
assert "reasoning" not in _body(0)
|
||||
|
||||
|
||||
def test_positive_budget_adds_thinking_tokens():
|
||||
body = _body(500)
|
||||
assert body["reasoning"] == {"max_tokens": 500}
|
||||
# the story output keeps its own full budget on top of the thinking budget
|
||||
assert body["max_tokens"] == 1500
|
||||
|
||||
|
||||
def test_negative_turns_reasoning_off():
|
||||
body = _body(-1)
|
||||
assert body["reasoning"] == {"effort": "none"}
|
||||
# "off" must not inflate the output budget
|
||||
assert body["max_tokens"] == 1000
|
||||
|
||||
|
||||
def test_off_is_not_merely_excluded():
|
||||
"""`exclude: true` still generates and bills for reasoning tokens. The off setting must omit the field entirely instead of relying on `exclude`."""
|
||||
assert _body(-1)["reasoning"].get("exclude") is None
|
||||
|
||||
|
||||
def test_completion_mode_never_sends_reasoning():
|
||||
for budget in (-1, 0, 500):
|
||||
assert "reasoning" not in _body(budget, api_mode="completion")
|
||||
@@ -14,18 +14,11 @@ from app.main import app
|
||||
from app.providers import ProviderError
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Each turn spends 10 gold, so a double-applied or un-rolled-back attempt shows.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -41,14 +34,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -57,9 +47,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -80,7 +67,7 @@ def _adv(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -177,15 +164,15 @@ def test_three_attempts_all_kept_in_order(client):
|
||||
|
||||
def test_switching_back_restores_that_attempt_state(client):
|
||||
ScriptedProvider.replies = [
|
||||
"You take a scratch.\n```state\n{\"player.hp\": -5}\n```",
|
||||
"You take a beating.\n```state\n{\"player.hp\": -40}\n```",
|
||||
'You take a scratch.\n```state\n{"player.hp": -5, "player.gold": 10}\n```',
|
||||
'You take a beating.\n```state\n{"player.hp": -40, "player.gold": 10}\n```',
|
||||
]
|
||||
_play(client)
|
||||
assert _adv(client.adv_id)[1]["player"]["hp"] == 95
|
||||
_retry(client)
|
||||
script_state, world_state = _adv(client.adv_id)
|
||||
player, world_state = _adv(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 60
|
||||
assert script_state == {"gold": 10} # rolled back, not stacked to 20
|
||||
assert player["gold"] == 10 # rolled back, not stacked to 20
|
||||
|
||||
last = _actions(client)[-1]
|
||||
r = client.post(
|
||||
@@ -194,9 +181,9 @@ def test_switching_back_restores_that_attempt_state(client):
|
||||
assert r.json()["text"].startswith("You take a scratch")
|
||||
assert r.json()["take_index"] == 0
|
||||
# The stats follow the narration back.
|
||||
script_state, world_state = _adv(client.adv_id)
|
||||
player, world_state = _adv(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 95
|
||||
assert script_state == {"gold": 10}
|
||||
assert player["gold"] == 10
|
||||
|
||||
# And forward again.
|
||||
client.post(f"/api/adventures/{client.adv_id}/actions/{last['id']}/variant",
|
||||
@@ -251,26 +238,26 @@ def test_switching_to_a_missing_index_is_rejected(client):
|
||||
def test_failed_retry_leaves_the_previous_attempt_in_charge(client):
|
||||
"""A provider error mid-retry must undo the rollback, or the stats on
|
||||
screen would silently disagree with the text still shown."""
|
||||
ScriptedProvider.replies = ["Attempt one.", ProviderError("upstream is down")]
|
||||
ScriptedProvider.replies = [gold_reply("Attempt one."), ProviderError("upstream is down")]
|
||||
_play(client)
|
||||
assert _adv(client.adv_id)[0] == {"gold": 10}
|
||||
assert _adv(client.adv_id)[0]["gold"] == 10
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
|
||||
actions = _actions(client)
|
||||
assert actions[-1]["text"] == "Attempt one." # text never lost
|
||||
assert _adv(client.adv_id)[0] == {"gold": 10} # and the state still matches it
|
||||
assert _adv(client.adv_id)[0]["gold"] == 10 # and the state still matches it
|
||||
|
||||
|
||||
def test_undo_removes_the_action_and_its_history(client):
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
_retry(client)
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
# Undo returns the newest window now, not the whole story.
|
||||
assert [a["type"] for a in r.json()["actions"]] == ["start"]
|
||||
assert _adv(client.adv_id)[0] == {}
|
||||
assert _adv(client.adv_id)[0]["gold"] == 0
|
||||
|
||||
|
||||
def test_editing_the_text_updates_the_live_variant(client):
|
||||
|
||||
@@ -125,7 +125,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -34,7 +34,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
"""Tests for undo and retry rolling back the shared `script_state`
|
||||
(plan/11-state-revert-and-retry-fix.md).
|
||||
"""Tests for undo and retry rolling back the shared `world_state`.
|
||||
|
||||
The state being rolled back was the scripting engine's `script_state` until
|
||||
M2 removed campaign scripting. The machinery under test — `attempts.restore_state`,
|
||||
`roll_back_before`, and the per-node outcome snapshot — is unchanged; only the
|
||||
column it moves has. `world_state`/`world_state_after` is now the shared state
|
||||
an adventure carries, so that is what these tests exercise.
|
||||
|
||||
Phase 14 SP4 reversed the snapshots. An action used to carry the state as
|
||||
it stood before it ran, and rolling back read the snapshot off the action
|
||||
@@ -36,11 +41,11 @@ def db():
|
||||
adventures.turns._active_turns.clear()
|
||||
|
||||
|
||||
def _make_adventure(db, script_state):
|
||||
def _make_adventure(db, world_state):
|
||||
user = models.User(is_guest=False)
|
||||
db.add(user)
|
||||
db.flush()
|
||||
adv = models.Adventure(user_id=user.id, title="T", script_state=script_state)
|
||||
adv = models.Adventure(user_id=user.id, title="T", world_state=world_state)
|
||||
db.add(adv)
|
||||
db.flush()
|
||||
return user, adv
|
||||
@@ -49,13 +54,46 @@ def _make_adventure(db, script_state):
|
||||
def _add(db, adv, index, type_, text="x", state_after=None):
|
||||
a = models.Action(
|
||||
adventure_id=adv.id, type=type_, text=text,
|
||||
state_after=state_after,
|
||||
world_state_after=state_after,
|
||||
)
|
||||
db.add(a)
|
||||
db.flush()
|
||||
return a
|
||||
|
||||
|
||||
def _row_count(db, adv):
|
||||
"""Every action row the adventure holds, live or not, head or no head."""
|
||||
return db.query(models.Action).filter_by(adventure_id=adv.id).count()
|
||||
|
||||
|
||||
def _all_types(db, adv):
|
||||
"""The retained story in depth order, which is not the same as the story
|
||||
being told once the head has moved back behind the tip."""
|
||||
rows = (
|
||||
db.query(models.Action)
|
||||
.filter_by(adventure_id=adv.id)
|
||||
.order_by(models.Action.depth, models.Action.id)
|
||||
.all()
|
||||
)
|
||||
return [a.type for a in rows]
|
||||
|
||||
|
||||
def _retrievable(db, adv):
|
||||
"""The memories the story can currently reach, read through the same clause
|
||||
`memorybank` retrieves with — which is capped at the active head."""
|
||||
from app.context import lineage
|
||||
|
||||
rows = (
|
||||
db.query(models.Memory)
|
||||
.filter(
|
||||
models.Memory.adventure_id == adv.id,
|
||||
lineage.path_of(db, adv).clause(models.Memory),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
return {m.text for m in rows}
|
||||
|
||||
|
||||
def _forget_snapshots(db, adv):
|
||||
"""Blank every outcome, the way a row written before SP4 looks.
|
||||
|
||||
@@ -72,19 +110,34 @@ def _forget_snapshots(db, adv):
|
||||
# ---------------------------------------------------------------- undo
|
||||
|
||||
def test_undo_reverts_state_to_before_the_turn(db):
|
||||
# A turn moved script_state from {gold:0} to {gold:10}. The node in
|
||||
# A turn moved world_state from {gold:0} to {gold:10}. The node in
|
||||
# front of the turn records where it started. The current state is
|
||||
# the mutated one.
|
||||
#
|
||||
# M3 rewrote what the second half of this test asserts. Undo used to delete
|
||||
# the turn, so the story was short afterwards because the rows were gone.
|
||||
# It now moves the head, so the story is short because it is being read
|
||||
# from somewhere earlier — and the rows are all still there. The state
|
||||
# assertion is unchanged, because `attempts.restore_state` is unchanged:
|
||||
# the state still comes off the node the story now ends on.
|
||||
user, adv = _make_adventure(db, {"gold": 10})
|
||||
_add(db, adv, 0, "start", state_after={"gold": 0})
|
||||
_add(db, adv, 1, "do", state_after={"gold": 0})
|
||||
_add(db, adv, 2, "ai", state_after={"gold": 10})
|
||||
db.commit()
|
||||
before = _row_count(db, adv)
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
page = adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 0}
|
||||
assert [a.type for a in adv.actions] == ["start"]
|
||||
assert adv.world_state == {"gold": 0}
|
||||
# What the story now tells.
|
||||
assert [a.type for a in page.actions] == ["start"]
|
||||
assert adv.head_depth == 0
|
||||
# What it still holds. Zero accepted turns deleted, which is the M3
|
||||
# invariant this file is the closest test to.
|
||||
assert _row_count(db, adv) == before
|
||||
assert _all_types(db, adv) == ["start", "do", "ai"]
|
||||
assert page.can_redo is True
|
||||
|
||||
|
||||
def test_undo_of_bare_continue_uses_the_node_in_front(db):
|
||||
@@ -95,10 +148,30 @@ def test_undo_of_bare_continue_uses_the_node_in_front(db):
|
||||
_add(db, adv, 1, "ai", state_after={"gold": 5})
|
||||
db.commit()
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
page = adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 0}
|
||||
assert [a.type for a in adv.actions] == ["start"]
|
||||
assert adv.world_state == {"gold": 0}
|
||||
assert [a.type for a in page.actions] == ["start"]
|
||||
assert _all_types(db, adv) == ["start", "ai"]
|
||||
|
||||
|
||||
def test_redo_puts_back_the_state_the_turn_left_behind(db):
|
||||
"""The other half of the same mechanism: undo and redo restore the same
|
||||
snapshot from opposite directions, because it belongs to the node rather
|
||||
than to the direction of travel."""
|
||||
user, adv = _make_adventure(db, {"gold": 10})
|
||||
_add(db, adv, 0, "start", state_after={"gold": 0})
|
||||
_add(db, adv, 1, "do", state_after={"gold": 0})
|
||||
_add(db, adv, 2, "ai", state_after={"gold": 10})
|
||||
db.commit()
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
page = adventures.redo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.world_state == {"gold": 10}
|
||||
assert [a.type for a in page.actions] == ["start", "do", "ai"]
|
||||
assert adv.head_depth == 2
|
||||
assert page.can_redo is False
|
||||
|
||||
|
||||
def test_undo_leaves_state_untouched_when_snapshot_missing(db):
|
||||
@@ -112,7 +185,7 @@ def test_undo_leaves_state_untouched_when_snapshot_missing(db):
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 10}
|
||||
assert adv.world_state == {"gold": 10}
|
||||
|
||||
|
||||
def test_undo_raises_when_nothing_to_undo(db):
|
||||
@@ -141,20 +214,45 @@ def test_undo_blocked_by_active_turn_lock(db):
|
||||
adventures.turns._active_turns.discard(adv.id)
|
||||
|
||||
|
||||
def test_undo_prunes_memory_covering_removed_actions(db):
|
||||
def test_undo_stops_retrieving_a_memory_without_deleting_it(db):
|
||||
"""M3 rewrote this test. Undo used to prune the memories covering the turns
|
||||
it deleted, because those turns were gone and a summary of them described
|
||||
story the adventure no longer had.
|
||||
|
||||
Nothing is deleted now, and nothing needs pruning either. A memory carries
|
||||
the coordinate of the node its block ends on, so one derived from a turn
|
||||
that is now past the head falls outside the head-capped path clause and
|
||||
stops being retrievable — and becomes eligible again on Redo, without having
|
||||
been deleted and re-embedded. That is `STORY-BRANCH-SEMANTICS.md` §33
|
||||
holding as a consequence of the head rather than as its own mechanism.
|
||||
"""
|
||||
user, adv = _make_adventure(db, {})
|
||||
for i in range(4):
|
||||
_add(db, adv, i, "ai" if i % 2 else "do", state_after={})
|
||||
# A memory summarizing actions up to index 3, which undo will delete.
|
||||
covering = models.Memory(adventure_id=adv.id, text="m", source_start=0, source_end=3)
|
||||
keep = models.Memory(adventure_id=adv.id, text="k", source_start=0, source_end=1)
|
||||
db.commit()
|
||||
# A memory ending on the turn undo will step behind, and one ending before
|
||||
# it. The coordinate is what the clause reads; `source_*` only says which
|
||||
# stretch the summarizer covered.
|
||||
covering = models.Memory(
|
||||
adventure_id=adv.id, text="m", source_start=0, source_end=3,
|
||||
branch_id=adv.head_branch_id, depth=3,
|
||||
)
|
||||
keep = models.Memory(
|
||||
adventure_id=adv.id, text="k", source_start=0, source_end=1,
|
||||
branch_id=adv.head_branch_id, depth=1,
|
||||
)
|
||||
db.add_all([covering, keep])
|
||||
db.commit()
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv) # removes indexes 2 & 3
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv) # head moves to depth 1
|
||||
|
||||
texts = {m.text for m in adv.memories}
|
||||
assert texts == {"k"}
|
||||
assert _retrievable(db, adv) == {"k"}
|
||||
# Still on disk, still embedded, still attached to the adventure.
|
||||
assert {m.text for m in adv.memories} == {"k", "m"}
|
||||
|
||||
adventures.redo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert _retrievable(db, adv) == {"k", "m"}
|
||||
|
||||
|
||||
# -------------------------------------------------------- withdrawing a node
|
||||
@@ -188,24 +286,24 @@ def test_snapshot_outcome_is_an_independent_deep_copy(db):
|
||||
_, adv = _make_adventure(db, {"nested": {"n": 1}})
|
||||
node = models.Action(adventure_id=adv.id, type="ai", text="x")
|
||||
attempts.snapshot_outcome(adv, node)
|
||||
adv.script_state["nested"]["n"] = 99
|
||||
assert node.state_after == {"nested": {"n": 1}} # unaffected by later mutation
|
||||
adv.world_state["nested"]["n"] = 99
|
||||
assert node.world_state_after == {"nested": {"n": 1}} # unaffected by later mutation
|
||||
|
||||
|
||||
def test_snapshot_outcome_handles_non_dict(db):
|
||||
_, adv = _make_adventure(db, {})
|
||||
adv.script_state = None
|
||||
adv.world_state = None
|
||||
node = models.Action(adventure_id=adv.id, type="ai", text="x")
|
||||
attempts.snapshot_outcome(adv, node)
|
||||
assert node.state_after == {}
|
||||
assert node.world_state_after == {}
|
||||
|
||||
|
||||
def test_restore_state_ignores_a_node_with_no_outcome(db):
|
||||
_, adv = _make_adventure(db, {"gold": 7})
|
||||
attempts.restore_state(adv, models.Action(adventure_id=adv.id, type="ai"))
|
||||
assert adv.script_state == {"gold": 7}
|
||||
assert adv.world_state == {"gold": 7}
|
||||
attempts.restore_state(adv, None)
|
||||
assert adv.script_state == {"gold": 7}
|
||||
assert adv.world_state == {"gold": 7}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- retry
|
||||
@@ -220,8 +318,6 @@ def test_retry_restores_the_state_the_turn_started_from(db, monkeypatch):
|
||||
_add(db, adv, 2, "ai", state_after={"gold": 20})
|
||||
db.commit()
|
||||
|
||||
monkeypatch.setattr(adventures.limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(adventures.turns, "check_demo_cap", lambda *a, **k: None)
|
||||
|
||||
async def _noop(*a, **k):
|
||||
if False:
|
||||
@@ -230,12 +326,12 @@ def test_retry_restores_the_state_the_turn_started_from(db, monkeypatch):
|
||||
|
||||
adventures.retry_action(adv.id, request=None, db=db, user=user, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 10}
|
||||
assert adv.world_state == {"gold": 10}
|
||||
# Nothing is written until a replacement actually arrives: the attempt on
|
||||
# screen is left exactly as it was, and stays the live one.
|
||||
assert [a.type for a in adv.actions] == ["start", "do", "ai"]
|
||||
last = adv.actions[-1]
|
||||
assert last.live is True
|
||||
assert len(attempts.group(db, last)) == 1 # no sibling was filed
|
||||
assert last.state_after == {"gold": 20} # its own outcome, untouched
|
||||
assert last.world_state_after == {"gold": 20} # its own outcome, untouched
|
||||
adventures.turns._active_turns.discard(adv.id)
|
||||
|
||||
@@ -15,7 +15,7 @@ This file must pass unmodified through SP1 (schema), SP2 (branch clause),
|
||||
and SP3 (memories on nodes). If a change here looks necessary in one of
|
||||
those subphases, the change is wrong, not the test. SP4 is the first
|
||||
subphase allowed to move it, and only for the variant-count semantics
|
||||
called out in plan/14.
|
||||
the story-tree migration changed.
|
||||
"""
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
@@ -26,20 +26,13 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# A world-state schema, so the RPG layer is exercised rather than skipped.
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn. A turn that double-applies its hooks, or one that fails to
|
||||
# roll back on retry, shows up here as a wrong total rather than as nothing.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
OPENING = "You enter a cave."
|
||||
|
||||
@@ -59,7 +52,7 @@ def _make_world(monkeypatch, *, seeded_actions: int = 0):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
@@ -70,20 +63,14 @@ def _make_world(monkeypatch, *, seeded_actions: int = 0):
|
||||
type="ai" if i % 2 else "do",
|
||||
text=f"Seeded turn {i}.",
|
||||
))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = ["A reply."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["A reply."]]
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -145,7 +132,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -172,7 +159,7 @@ def test_short_adventure_returns_everything(client):
|
||||
# --------------------------------------------------------------- the turn
|
||||
|
||||
def test_a_turn_appends_the_player_action_then_the_ai_action(client):
|
||||
ScriptedProvider.replies = ["The dark presses in."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["The dark presses in."]]
|
||||
_play(client, "light a torch")
|
||||
actions = _actions(client)
|
||||
assert [a["type"] for a in actions] == ["start", "do", "ai"]
|
||||
@@ -181,7 +168,7 @@ def test_a_turn_appends_the_player_action_then_the_ai_action(client):
|
||||
|
||||
|
||||
def test_say_and_story_and_continue_all_work(client):
|
||||
ScriptedProvider.replies = ["One.", "Two.", "Three."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two.", "Three."]]
|
||||
_play(client, "hello", type="say")
|
||||
_play(client, "The wind rises.", type="story")
|
||||
_play(client, "", type="continue")
|
||||
@@ -190,7 +177,7 @@ def test_say_and_story_and_continue_all_work(client):
|
||||
|
||||
|
||||
def test_the_story_so_far_is_replayed_into_the_prompt(client):
|
||||
ScriptedProvider.replies = ["First.", "Second."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["First.", "Second."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
story = ScriptedProvider.prompts[-1][1]
|
||||
@@ -201,7 +188,7 @@ def test_the_story_so_far_is_replayed_into_the_prompt(client):
|
||||
|
||||
def test_scripts_run_once_per_turn(client):
|
||||
"""The gold script adds ten a turn. Two turns is twenty — not forty."""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
_play(client)
|
||||
script_state, _ = _state(client.adv_id)
|
||||
@@ -211,7 +198,7 @@ def test_scripts_run_once_per_turn(client):
|
||||
# ------------------------------------------------------------------ retry
|
||||
|
||||
def test_retry_replaces_the_text_and_keeps_the_attempt(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
assert _texts(client)[-1] == "Attempt one."
|
||||
|
||||
@@ -233,7 +220,7 @@ def test_retry_replaces_the_text_and_keeps_the_attempt(client):
|
||||
def test_retry_does_not_stack_script_effects(client):
|
||||
"""The discarded attempt's ten gold is rolled back, so one turn plus one
|
||||
retry is still ten, not twenty."""
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
script_state, _ = _state(client.adv_id)
|
||||
@@ -241,7 +228,7 @@ def test_retry_does_not_stack_script_effects(client):
|
||||
|
||||
|
||||
def test_switching_back_to_an_earlier_attempt_restores_it(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
@@ -259,7 +246,7 @@ def test_switching_back_to_an_earlier_attempt_restores_it(client):
|
||||
def test_only_the_newest_turn_can_be_switched(client):
|
||||
"""An older turn's alternatives stay readable but not selectable. The
|
||||
story after it continues from what is live."""
|
||||
ScriptedProvider.replies = ["One.", "Again.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Again.", "Two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
older_id = _actions(client)[-1]["id"]
|
||||
@@ -274,7 +261,7 @@ def test_only_the_newest_turn_can_be_switched(client):
|
||||
# ------------------------------------------------------------------- undo
|
||||
|
||||
def test_undo_removes_the_whole_turn_and_rolls_state_back(client):
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
assert len(_actions(client)) == 5
|
||||
@@ -365,7 +352,7 @@ def test_paging_past_the_start_reports_the_end(long_client):
|
||||
# ------------------------------------------------------- editing and deleting
|
||||
|
||||
def test_editing_an_action_sticks(client):
|
||||
ScriptedProvider.replies = ["Original."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Original."]]
|
||||
_play(client)
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
r = client.patch(f"/api/adventures/{client.adv_id}/actions/{action_id}",
|
||||
@@ -380,7 +367,7 @@ def test_editing_an_action_sticks(client):
|
||||
def test_editing_a_retried_action_survives_a_reload(client):
|
||||
"""The edit has to reach the live attempt too, or paging away and back
|
||||
reverts it."""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
@@ -390,7 +377,7 @@ def test_editing_a_retried_action_survives_a_reload(client):
|
||||
|
||||
|
||||
def test_deleting_an_action_removes_it(client):
|
||||
ScriptedProvider.replies = ["One."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One."]]
|
||||
_play(client)
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
r = client.delete(f"/api/adventures/{client.adv_id}/actions/{action_id}")
|
||||
@@ -431,7 +418,7 @@ def test_export_carries_the_whole_story(client):
|
||||
same fact: a bundle that stores coordinates has no use for a repeating
|
||||
group. Everything else here still passes unmodified.
|
||||
"""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
|
||||
@@ -446,7 +433,7 @@ def test_export_carries_the_whole_story(client):
|
||||
|
||||
|
||||
def test_export_round_trips_through_import(client):
|
||||
ScriptedProvider.replies = ["One."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One."]]
|
||||
_play(client, "go north")
|
||||
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
||||
|
||||
@@ -461,7 +448,7 @@ def test_export_round_trips_through_import(client):
|
||||
|
||||
|
||||
def test_export_keeps_retry_attempts(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
|
||||
@@ -474,7 +461,7 @@ def test_export_keeps_retry_attempts(client):
|
||||
# -------------------------------------------------------------- world state
|
||||
|
||||
def test_world_state_is_readable_and_survives_a_turn(client):
|
||||
ScriptedProvider.replies = ["Nothing changes."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Nothing changes."]]
|
||||
_play(client)
|
||||
r = client.get(f"/api/adventures/{client.adv_id}/world-state")
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
@@ -57,9 +57,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -58,12 +58,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(
|
||||
auth,
|
||||
"resolve_provider_config",
|
||||
lambda s: auth.ProviderConfig("http://fake", "k", "test-model", False),
|
||||
)
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -24,20 +24,14 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn, every turn. A number that only ever increases makes a
|
||||
# rollback failure obvious: if a take stacks instead of replacing, the gold
|
||||
# total is off by exactly one turn's worth.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
# Ten gold a turn, every turn, banked through the world-state engine. A number
|
||||
# that only ever increases makes a rollback failure obvious: if a take stacks
|
||||
# instead of replacing, the gold total is off by exactly one turn's worth. See
|
||||
# `fakes.gold_reply` for why this is not a script any more.
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -53,24 +47,18 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Vault", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You begin."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.replies = gold_replies("Take", 39)
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -107,7 +95,7 @@ def _take(client, action_id, text=""):
|
||||
def _gold(adv_id) -> int:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return (db.get(models.Adventure, adv_id).script_state or {}).get("gold", 0)
|
||||
return (db.get(models.Adventure, adv_id).world_state or {}).get("player", {}).get("gold", 0)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@@ -31,8 +31,7 @@ from app.main import app
|
||||
# The three tables as they stood at schema 45, frozen. This is a snapshot
|
||||
# of a past schema. It must not be updated to track `models.py`, because
|
||||
# the whole point is that it lacks what SP1 adds. This DDL uses SQLite
|
||||
# syntax only. The migration's Postgres half is exercised against a real
|
||||
# server at deploy time (see plan/14).
|
||||
# syntax only.
|
||||
PRE_TREE_DDL = (
|
||||
"""
|
||||
CREATE TABLE adventures (
|
||||
@@ -120,7 +119,12 @@ def pre_tree():
|
||||
# Dropping the tables is the only way to remove the columns.
|
||||
# SQLite refuses to drop a column that a foreign key references,
|
||||
# and that is exactly the case for `branch_id`.
|
||||
for table in ("actions", "memories", "branches", "adventures"):
|
||||
#
|
||||
# `checkpoints` (M4) is dropped first and for a different reason: it
|
||||
# references both `branches` and `adventures`, and SQLite refuses to
|
||||
# drop a table another table still points at. Any future table that
|
||||
# references these four has to be added to the front of this list.
|
||||
for table in ("checkpoints", "actions", "memories", "branches", "adventures"):
|
||||
conn.execute(text(f"DROP TABLE IF EXISTS {table}"))
|
||||
for ddl in PRE_TREE_DDL:
|
||||
conn.execute(text(ddl))
|
||||
@@ -397,7 +401,6 @@ def client(monkeypatch):
|
||||
user_id, scenario_id = user.id, scenario.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -594,7 +597,7 @@ def pre_split():
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
with engine.begin() as conn:
|
||||
for table in ("actions", "memories", "branches", "adventures"):
|
||||
for table in ("checkpoints", "actions", "memories", "branches", "adventures"):
|
||||
conn.execute(text(f"DROP TABLE IF EXISTS {table}"))
|
||||
for ddl in PRE_TREE_DDL:
|
||||
conn.execute(text(ddl))
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
"""End-to-end HTTP tests for undo and retry state revert. These tests drive
|
||||
real turns through the actual routes and scripting engine, with only the LLM
|
||||
provider mocked.
|
||||
real turns through the actual routes and the world-state engine, with only the
|
||||
LLM provider mocked.
|
||||
|
||||
A script's output hook adds 10 gold each turn. These tests confirm that the
|
||||
adventure's stored gold total stays correct across play, undo, and retry.
|
||||
The model banks 10 gold each turn through its ```state delta block. These tests
|
||||
confirm that the adventure's stored gold total stays correct across play, undo,
|
||||
and retry. It was a JavaScript output hook that added the gold until M2 removed
|
||||
campaign scripting; the state machinery under test is the same either way.
|
||||
|
||||
python -m pytest tests/test_turn_flow_integration.py -v
|
||||
"""
|
||||
@@ -16,15 +18,8 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
AI_REPLY = "The torch flickers as you press onward."
|
||||
@@ -38,24 +33,21 @@ def client(monkeypatch):
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||
adv = models.Adventure(user_id=user.id, title="Cave", script_state={})
|
||||
scenario = models.Scenario(user_id=user.id, title="S", stat_schema=GOLD_SCHEMA)
|
||||
setup.add(scenario)
|
||||
setup.flush()
|
||||
adv = models.Adventure(user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
world_state={"player": {"hp": 100, "gold": 0}})
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold",
|
||||
output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
# Force a real, non-demo turn that uses the fake provider.
|
||||
ScriptedProvider.replies = [AI_REPLY]
|
||||
ScriptedProvider.replies = [gold_reply(AI_REPLY)]
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -73,10 +65,13 @@ def client(monkeypatch):
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def _state(adv_id):
|
||||
def _gold(adv_id) -> int:
|
||||
"""The banked total. Reading one stat rather than the whole section keeps
|
||||
these assertions about the rollback, not about the schema's shape."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return db.get(models.Adventure, adv_id).script_state
|
||||
player = (db.get(models.Adventure, adv_id).world_state or {}).get("player", {})
|
||||
return player.get("gold", 0)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -88,38 +83,38 @@ def _play(client, type_="do", text="look around"):
|
||||
|
||||
|
||||
def test_play_then_undo_reverts_gold(client):
|
||||
assert _state(client.adv_id) == {}
|
||||
assert _gold(client.adv_id) == 0
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
assert _state(client.adv_id) == {} # gold reverted to zero
|
||||
assert _gold(client.adv_id) == 0 # gold reverted to zero
|
||||
|
||||
|
||||
def test_two_turns_then_undo_reverts_only_last(client):
|
||||
_play(client)
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 20}
|
||||
assert _gold(client.adv_id) == 20
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert _state(client.adv_id) == {"gold": 10} # back to after turn 1, not 0
|
||||
assert _gold(client.adv_id) == 10 # back to after turn 1, not 0
|
||||
|
||||
|
||||
def test_retry_does_not_double_apply_gold(client):
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
# Before the fix, this produced 20 because the output hook ran twice.
|
||||
# Before the fix, this produced 20 because the turn's effects ran twice.
|
||||
# Now it stays 10.
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert r.status_code == 200, r.text
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
|
||||
def test_retry_then_undo_still_clean(client):
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert _state(client.adv_id) == {}
|
||||
assert _gold(client.adv_id) == 0
|
||||
|
||||
@@ -63,9 +63,6 @@ def client(monkeypatch):
|
||||
|
||||
ScriptedProvider.replies = [AI_REPLY]
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -4,10 +4,9 @@ Reviewing a prompt tells you what it asks for. It does not tell you what a model
|
||||
does with it. This runs the real pipeline twice over the *same* blocks of the
|
||||
*same* story, changing only the prompt, and prints the memories side by side.
|
||||
|
||||
What it showed the first time it was run, and why `MEMORY_MAX_WORDS` exists, is
|
||||
written up in `plan/18-persona-and-memory-quality.md`. Two consecutive memories
|
||||
from one story came back in two different persons, and the same model wrote 34
|
||||
words for one block and 105 for the next.
|
||||
Why `MEMORY_MAX_WORDS` exists is what it showed the first time it was run: two
|
||||
consecutive memories from one story came back in two different persons, and the
|
||||
same model wrote 34 words for one block and 105 for the next.
|
||||
|
||||
Three things make it a fair test rather than a demonstration:
|
||||
|
||||
|
||||
@@ -56,66 +56,22 @@ Usage, from `backend/`:
|
||||
python -m tools.rewrite_memories --write --limit 3 # try three of them
|
||||
python -m tools.rewrite_memories --write --embed # the whole backfill
|
||||
|
||||
Without `--write` it makes no model calls, spends nothing, and only reports the
|
||||
scope. Every run reads the database the app reads: `AIDND_DB_PATH`, or
|
||||
`DATABASE_URL` for a hosted Postgres. Take a copy of it first — the old text is
|
||||
overwritten and is not kept anywhere.
|
||||
Without `--write` it makes no model calls and only reports the scope. Every run
|
||||
reads the database the app reads, at `AIDND_DB_PATH`. Take a copy of it first —
|
||||
the old text is overwritten and is not kept anywhere.
|
||||
|
||||
**On the hosted deploy, name whose adventures you mean.** That database holds
|
||||
other people's stories, and each adventure is summarized with *its owner's* key,
|
||||
so an unfiltered `--write` spends other people's money on memories they did not
|
||||
ask to have rewritten. `--email` restricts the run to the accounts you name and
|
||||
`--adventure` to single adventures; a dry run costs nothing and lists both, with
|
||||
the owner of each. Guests have no email and can only be reached by id.
|
||||
|
||||
Two environment variables reach that database from a checkout. In PowerShell,
|
||||
which is where this project is developed, they are set first and then persist
|
||||
for the rest of the session:
|
||||
|
||||
$env:AIDND_DATABASE_URL = Read-Host 'Neon URL'
|
||||
$env:AIDND_SECRET_KEY = Read-Host 'Secret key'
|
||||
.venv\Scripts\python.exe -m tools.rewrite_memories --email you@example.com
|
||||
|
||||
`Read-Host` keeps both values out of the PowerShell history file. On a POSIX
|
||||
shell the same thing is one line:
|
||||
|
||||
AIDND_DATABASE_URL=... AIDND_SECRET_KEY=... \
|
||||
python -m tools.rewrite_memories --email you@example.com
|
||||
|
||||
`AIDND_SECRET_KEY` is not optional there. Stored API keys are encrypted with it,
|
||||
and with the wrong one `decrypt_secret` returns "" and every adventure is skipped
|
||||
as having no key (see `security.py`). The deployed image does not carry this
|
||||
directory — the Dockerfile copies `backend/app` alone — so run it from a
|
||||
checkout against the hosted database rather than from a shell on the box.
|
||||
`--adventure` restricts the run to single adventures, and a dry run lists what
|
||||
would change. The deployed image does not carry this directory — the Dockerfile
|
||||
copies `backend/app` alone — so run it from a checkout.
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from sqlalchemy import func, inspect as sa_inspect, select
|
||||
|
||||
|
||||
def safe_dsn(url: str) -> str:
|
||||
"""A connection string with the credentials taken out.
|
||||
|
||||
The report says which database it is about to rewrite, which is worth
|
||||
printing. The password in a Neon URL is not: this output goes to a console,
|
||||
a screenshot, or a pasted bug report, and the operator has no way to know
|
||||
the line carried a credential until it is somewhere else.
|
||||
"""
|
||||
parsed = urlsplit(url)
|
||||
if not parsed.hostname:
|
||||
return "(configured)"
|
||||
who = f"{parsed.username}@" if parsed.username else ""
|
||||
port = f":{parsed.port}" if parsed.port else ""
|
||||
# The query string is dropped whole. `sslmode` is the only part anyone
|
||||
# wants to see, and some drivers accept a password there too.
|
||||
return f"{parsed.scheme}://{who}{parsed.hostname}{port}{parsed.path}"
|
||||
|
||||
|
||||
def words(text: str) -> int:
|
||||
return len(text.split())
|
||||
|
||||
@@ -127,16 +83,16 @@ def one_line(text: str, width: int = 96) -> str:
|
||||
|
||||
async def main(args) -> int:
|
||||
from app import memorybank, models
|
||||
from app.database import DB_PATH, DATABASE_URL, SessionLocal
|
||||
from app.database import DB_PATH, SessionLocal
|
||||
from app.providers import OpenAICompatibleProvider, ProviderError
|
||||
|
||||
db = SessionLocal()
|
||||
print(f"database: {safe_dsn(DATABASE_URL) if DATABASE_URL else DB_PATH}")
|
||||
print(f"database: {DB_PATH}")
|
||||
if not sa_inspect(db.get_bind()).has_table(models.Adventure.__tablename__):
|
||||
# A mistyped path creates an empty SQLite file rather than failing, so
|
||||
# say what is wrong instead of raising "no such table: adventures".
|
||||
print("There are no tables here. Point AIDND_DB_PATH, or DATABASE_URL "
|
||||
"for a hosted deploy, at the database the app uses.")
|
||||
print("There are no tables here. Point AIDND_DB_PATH at the database "
|
||||
"the app uses.")
|
||||
return 2
|
||||
|
||||
adventures = db.query(models.Adventure).order_by(models.Adventure.id)
|
||||
@@ -177,14 +133,13 @@ async def main(args) -> int:
|
||||
|
||||
def provider_for(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
"""The adventure owner's own summarizer, unless the run overrides it."""
|
||||
if not (args.endpoint or args.model or args.api_key):
|
||||
if not (args.endpoint or args.model):
|
||||
return memorybank.summary_provider(settings)
|
||||
return OpenAICompatibleProvider(
|
||||
args.endpoint or settings.endpoint_url,
|
||||
args.api_key or settings.api_key_plain,
|
||||
args.model or settings.summary_model or settings.model,
|
||||
settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
|
||||
totals = {"rewritten": 0, "would rewrite": 0, "no source": 0,
|
||||
@@ -205,11 +160,11 @@ async def main(args) -> int:
|
||||
continue
|
||||
|
||||
settings = settings_for(adventure.user_id)
|
||||
# An adventure whose owner has no key is reported rather than skipped
|
||||
# silently: it is the one reason a memory this tool can rewrite is left
|
||||
# alone, and the operator can fix it with --api-key.
|
||||
# An adventure with no model configured is reported rather than
|
||||
# skipped silently: it is the one reason a memory this tool can rewrite
|
||||
# is left alone, and the operator can fix it with --model.
|
||||
usable = settings is not None and bool(
|
||||
args.api_key or args.endpoint or settings.api_key_plain
|
||||
args.model or settings.summary_model or settings.model
|
||||
)
|
||||
owner = db.get(models.User, adventure.user_id)
|
||||
who = (owner.email if owner and owner.email
|
||||
@@ -333,7 +288,6 @@ if __name__ == "__main__":
|
||||
help="re-embed here. Stop the app first; see the docstring.")
|
||||
parser.add_argument("--endpoint", help="override the owner's endpoint URL.")
|
||||
parser.add_argument("--model", help="override the owner's summary model.")
|
||||
parser.add_argument("--api-key", help="override the owner's API key.")
|
||||
args = parser.parse_args()
|
||||
|
||||
# A Windows console defaults to cp1252, which cannot encode the arrow this
|
||||
|
||||
|
Before Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 103 KiB |
|
Before Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 157 KiB |
|
Before Width: | Height: | Size: 94 KiB |
|
Before Width: | Height: | Size: 84 KiB |
|
Before Width: | Height: | Size: 59 KiB |