Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 JesseMarkowitzandClaude Opus 5
|
2fdd2547f0 | ||
|
JesseMarkowitzandClaude Opus 5
|
8652fe7cd8 | ||
|
JesseMarkowitz
|
8c65ae99de |
+45
-5
@@ -46,6 +46,9 @@ ollama pull qwen2.5:3b-instruct
|
||||
ollama pull nomic-embed-text # only if you want the memory bank
|
||||
```
|
||||
|
||||
There is no account to create and nothing to log in to. The application is
|
||||
single-user: whoever can reach it on loopback is its owner.
|
||||
|
||||
## Running
|
||||
|
||||
**Development** — backend on `:8000`, Vite dev server on `:5173`:
|
||||
@@ -102,7 +105,37 @@ curl -X PUT http://127.0.0.1:8000/api/settings \
|
||||
|
||||
`POST /api/settings/test` (the **Test connection** button) returns
|
||||
`{"ok": true, "models": [...]}` and is the fastest way to tell a wrong endpoint
|
||||
from a missing model.
|
||||
from a missing model. When it fails it says which kind of failure it was, and
|
||||
they need different things done about them:
|
||||
|
||||
| `kind` | What it means |
|
||||
| --- | --- |
|
||||
| `rejected` | The endpoint is outside the policy below. Not a network problem. |
|
||||
| `unreachable` | Nothing answered. Ollama is not running there, or the port is wrong. |
|
||||
| `tls` | The certificate did not verify — install the CA (see below). |
|
||||
| `timeout` | It accepted the connection and then said nothing. |
|
||||
| `http` | It answered with an error status; the body is included. |
|
||||
|
||||
A successful test also warns when the endpoint is reachable but has no model by
|
||||
the configured name, which is the commonest way for a correct endpoint to still
|
||||
fail every turn.
|
||||
|
||||
### Which endpoints are allowed
|
||||
|
||||
`backend/app/endpoints.py` decides, and it is deliberately narrow: **loopback,
|
||||
your own LAN, or nothing.** The allowed networks are `127.0.0.0/8`, the three
|
||||
RFC1918 ranges, link-local, IPv6 loopback and unique-local, and `100.64.0.0/10`
|
||||
(carrier-grade NAT, which is what a mesh VPN such as Tailscale hands out).
|
||||
|
||||
Every address the endpoint's hostname resolves to must be in one of them. A
|
||||
public address is refused, a name resolving to both a private and a public
|
||||
address is refused, and known cloud inference hosts are refused by name so the
|
||||
error says why rather than looking like a DNS fault.
|
||||
|
||||
The rule is applied when you save the endpoint *and* again before every
|
||||
outbound request, so a database edited by hand or a hostname that starts
|
||||
resolving somewhere new cannot turn a local install into an exfiltration path.
|
||||
There is no setting to relax it.
|
||||
|
||||
### Same host (the default)
|
||||
|
||||
@@ -137,9 +170,8 @@ Use an IP address or a name your own network resolves. Then:
|
||||
- the inference machine needs the models installed, not the storyteller;
|
||||
- no Internet is involved in either direction.
|
||||
|
||||
`app/netguard.py` refuses private addresses only in hosted multi-user mode
|
||||
(`AIDND_MULTI_USER=1`), which local installs never turn on, so a LAN endpoint
|
||||
is accepted as configured.
|
||||
A LAN endpoint is accepted because it is on one of the allowed networks above.
|
||||
Nothing else about it is special.
|
||||
|
||||
#### If that endpoint is HTTPS with your own CA
|
||||
|
||||
@@ -176,7 +208,7 @@ visible from within.
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/python -m pytest tests/ -q # 648 tests
|
||||
cd backend && .venv/bin/python -m pytest tests/ -q # 604 tests
|
||||
cd frontend && npm run lint && npm run build
|
||||
```
|
||||
|
||||
@@ -192,6 +224,14 @@ suite as complete evidence.
|
||||
is lost from the union, or if a new HTTP client is added without the shared
|
||||
verification context.
|
||||
|
||||
M2 added two more. `test_endpoint_policy.py` fails if the set of reachable
|
||||
addresses widens, or if either place the rule is applied stops applying it —
|
||||
it resolves hostnames through a stub, so it tests the policy rather than
|
||||
whatever DNS the machine has. `test_local_only_surface.py` fails if a removed
|
||||
subsystem comes back as a route, if an API key becomes settable again, if the
|
||||
model timeout stops being configurable or becomes unbounded, or if a supported
|
||||
start path stops binding loopback.
|
||||
|
||||
## What was made offline-safe, and how to check
|
||||
|
||||
Two runtime downloads were removed in Milestone M1. Both were invisible on a
|
||||
|
||||
+14
-22
@@ -7,20 +7,15 @@ RUN npm ci
|
||||
COPY frontend/ ./
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2 — build Python wheels (quickjs compiles from source if no wheel
|
||||
# matches, so keep the toolchain out of the final image)
|
||||
FROM python:3.12-slim AS python-build
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gcc make \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY backend/requirements.txt /tmp/requirements.txt
|
||||
RUN pip wheel --no-cache-dir -r /tmp/requirements.txt -w /wheels
|
||||
|
||||
# Stage 3 — runtime
|
||||
# Stage 2 — runtime. Every remaining dependency ships a wheel, so there is no
|
||||
# compile step and no toolchain to keep out of the image. The wheel-building
|
||||
# stage that used to sit here existed for quickjs, which compiled from source
|
||||
# and which M2 removed with campaign scripting.
|
||||
FROM python:3.12-slim
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=python-build /wheels /wheels
|
||||
RUN pip install --no-cache-dir /wheels/* && rm -rf /wheels
|
||||
COPY backend/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install --no-cache-dir -r /tmp/requirements.txt && rm /tmp/requirements.txt
|
||||
|
||||
# Layout mirrors the repo: main.py finds the SPA at ../../frontend/dist
|
||||
# relative to backend/app/main.py.
|
||||
@@ -38,14 +33,11 @@ EXPOSE 8000
|
||||
# invitation to put the storyteller on the LAN, which is single-user and
|
||||
# unauthenticated in local mode.
|
||||
WORKDIR /app/backend
|
||||
# --proxy-headers lets uvicorn fix up the request scheme (https) behind the
|
||||
# platform's edge. We deliberately do NOT pass --forwarded-allow-ips "*": that
|
||||
# made uvicorn trust the LEFTMOST X-Forwarded-For value, which the client fully
|
||||
# controls, so anyone could rotate the header to dodge the per-IP rate limits.
|
||||
# The client IP used for rate limiting is derived in limits._client_ip from the
|
||||
# hop the edge appends (rightmost), which a client cannot spoof past; tune with
|
||||
# AIDND_TRUSTED_PROXY_HOPS if the platform adds more proxy hops.
|
||||
# Single worker on purpose: the turn lock, rate limiter, and debug log are
|
||||
# in-process state.
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", \
|
||||
"--proxy-headers"]
|
||||
# Single worker on purpose: the turn lock and the debug log are in-process
|
||||
# state.
|
||||
#
|
||||
# No --proxy-headers. That existed for a hosted deployment behind a platform
|
||||
# edge, along with the per-IP rate limiting that read X-Forwarded-For. Neither
|
||||
# survives M2, and trusting a forwarded header on a loopback-published port
|
||||
# would be a way to lie to the app rather than a feature.
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
@@ -78,6 +78,29 @@ text ships beside them as `OFL-cinzel.txt`, `OFL-crimsonpro.txt` and
|
||||
Regenerate with `python3 frontend/tools/vendor_fonts.py`, which also rewrites
|
||||
`frontend/src/styles/fonts.css`.
|
||||
|
||||
## What this fork changed in Milestone M2
|
||||
|
||||
M2 is subtractive. It reduced the inherited application to the intended
|
||||
single-user, local-first trust boundary. **Nothing was added that upstream did
|
||||
not have, except the endpoint policy and the tests that hold these removals in
|
||||
place.**
|
||||
|
||||
Removed in full: campaign scripting and the QuickJS sandbox; multi-user
|
||||
accounts, guest sessions, login, registration and the shared demo key; the
|
||||
visitor analytics tables, dashboard and beacon; the access log; per-IP and
|
||||
per-user rate limiting and quotas; Render deployment config; Postgres/Neon
|
||||
support; cloud inference providers and the API-key field; session-cookie
|
||||
signing and API-key encryption at rest.
|
||||
|
||||
Added: `backend/app/endpoints.py`, which decides what an inference endpoint may
|
||||
be, and a configurable model timeout.
|
||||
|
||||
Three database tables (`scripts`, `adventure_scripts`, `analytics_daily`,
|
||||
`analytics_visitor_days`, `access_log`) and four columns (`adventures.script_state`,
|
||||
`settings.api_key`, `users.demo_turns_used`, `users.demo_turns_date`) are left
|
||||
in place, unmapped or inert, so that an existing M1 campaign database opens
|
||||
unchanged. They are not product functionality and nothing reads or writes them.
|
||||
|
||||
## What this fork changed in Milestone M1
|
||||
|
||||
Nothing was removed from upstream. The changes are the offline/locality
|
||||
|
||||
+14
-100
@@ -1,110 +1,24 @@
|
||||
# Environment variables read by the backend.
|
||||
#
|
||||
# NOTE: the app reads real environment variables — it does NOT auto-load this
|
||||
# file. Set them in your shell, in docker-compose.yml, or in your host's
|
||||
# dashboard. This file is documentation (and a template for deploy configs).
|
||||
# file. Set them in your shell or in docker-compose.yml. This file is
|
||||
# documentation.
|
||||
#
|
||||
# There are two, and neither is required. Everything about the model — the
|
||||
# endpoint, the model names, the timeout, the context budget — is a runtime
|
||||
# setting stored in the database and edited on the Settings page, because it is
|
||||
# a preference rather than a deployment detail.
|
||||
|
||||
# Absolute path for the SQLite database file. Parent directory is created if
|
||||
# missing. Default when unset: backend/data.db
|
||||
# Docker compose sets this to /data/data.db (a named volume).
|
||||
# Absolute path for the SQLite database file. The parent directory is created
|
||||
# if missing. Default when unset: backend/data.db
|
||||
# docker-compose.yml sets this to /data/data.db (a named volume).
|
||||
AIDND_DB_PATH=
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 9 — production hardening
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Switch from SQLite to a server database (hosted deploys use Neon Postgres).
|
||||
# Any SQLAlchemy URL; postgres:// and postgresql:// schemes are rewritten to
|
||||
# the psycopg3 driver automatically. The platform-conventional DATABASE_URL
|
||||
# is honored too (AIDND_DATABASE_URL wins if both are set). Unset = SQLite.
|
||||
AIDND_DATABASE_URL=
|
||||
|
||||
# Comma-separated list of allowed CORS origins. Only needed when the frontend
|
||||
# is served from a different origin than the API; the production build is
|
||||
# served same-origin by FastAPI, so hosted deploys can leave this unset.
|
||||
# served same-origin by FastAPI, so a normal run can leave this unset.
|
||||
# Default: http://localhost:5173,http://127.0.0.1:5173 (the Vite dev server).
|
||||
#
|
||||
# A wildcard is rejected. The storyteller API is unauthenticated by design and
|
||||
# bound to loopback; letting any origin call it would undo that.
|
||||
AIDND_CORS_ORIGINS=
|
||||
|
||||
# How many proxy hops the rate limiter trusts in `X-Forwarded-For`. It reads
|
||||
# the entry that many places from the right, because the trusted edge appends
|
||||
# the real client IP last. Set this to the number of proxies in front of the
|
||||
# app. Default: 1, which is correct for a single edge such as Render.
|
||||
#
|
||||
# Get it wrong in either direction and the rate limits weaken. Too low reads an
|
||||
# entry the caller supplied, so anyone can rotate the header for a fresh
|
||||
# rate-limit bucket per request and walk past the auth and guest limits. Too
|
||||
# high reads past the real client. Only multi-user mode rate-limits at all, so
|
||||
# local installs can ignore this.
|
||||
AIDND_TRUSTED_PROXY_HOPS=
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 8 — optional accounts & multi-user (all optional; defaults keep the
|
||||
# app in frictionless single-user "local mode")
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# "1"/"true" turns on multi-user mode: guest sessions via signed cookies,
|
||||
# register/login UI, per-user data. Leave unset for local installs.
|
||||
AIDND_MULTI_USER=
|
||||
|
||||
# Secret for signing session cookies and encrypting stored API keys at rest.
|
||||
# If unset in local mode, one is auto-generated into `secret.key` next to the
|
||||
# database (fine for local/docker-volume runs). REQUIRED when
|
||||
# AIDND_MULTI_USER is on — the app refuses to start without it, because a
|
||||
# regenerated secret on an ephemeral hosted filesystem would log out every
|
||||
# user on each deploy. Generate one:
|
||||
# python -c "import secrets; print(secrets.token_urlsafe(48))"
|
||||
AIDND_SECRET_KEY=
|
||||
|
||||
# Session cookie Secure flag (HTTPS-only). Defaults to on when
|
||||
# AIDND_MULTI_USER is on, off otherwise — set 0/1 only to override (e.g. 0
|
||||
# when testing multi-user mode over plain http on a LAN address).
|
||||
AIDND_COOKIE_SECURE=
|
||||
|
||||
# --- Shared demo key (BYOK fallback; only active when AIDND_MULTI_USER=1) ---
|
||||
# Users with no API key of their own get this server-funded endpoint with a
|
||||
# model whitelist and a per-day turn cap. Unset = no demo, users must bring
|
||||
# their own key. Memory bank/auto-summarization are disabled on demo turns.
|
||||
AIDND_DEMO_API_KEY=
|
||||
# Default endpoint if unset: https://openrouter.ai/api/v1
|
||||
AIDND_DEMO_ENDPOINT_URL=
|
||||
# Comma-separated model whitelist. Default: google/gemma-4-26b-a4b-it:free
|
||||
AIDND_DEMO_MODELS=
|
||||
# Successful AI turns per user per day on the demo key. Default: 20
|
||||
AIDND_DEMO_TURNS_PER_DAY=
|
||||
# Comma-separated emails of "power users" (trusted testers) who bypass the daily
|
||||
# demo cap entirely — unmetered turns on the shared demo key — and get the AI Chat
|
||||
# page (a plain scratchpad for talking to a model, hidden from everyone else).
|
||||
# Registered accounts only (guests have no email). Matched case-insensitively.
|
||||
# Local (single-user) installs are always treated as power users.
|
||||
AIDND_POWER_USERS=
|
||||
|
||||
# --- Visit analytics ---
|
||||
# Comma-separated emails allowed to see the Visitors dashboard (/analytics) and
|
||||
# its nav link. Deliberately separate from AIDND_POWER_USERS: a trusted tester
|
||||
# gets unmetered turns, which is no reason to hand them the traffic numbers.
|
||||
# Unset = nobody sees it in a hosted deploy. Local installs always can, and are
|
||||
# the only mode where the viewer's own visits are still counted (excluding them
|
||||
# would leave the page permanently empty on the machine it's developed on).
|
||||
# Collection itself is always on; only the dashboard is gated.
|
||||
AIDND_ANALYTICS_EMAILS=
|
||||
# Days to keep the one-row-per-visitor-per-day table that makes the funnel
|
||||
# count people rather than clicks. The daily counters are aggregate and kept
|
||||
# forever. Default: 400. Set 0 to keep visitor-days forever.
|
||||
AIDND_ANALYTICS_RETENTION_DAYS=
|
||||
|
||||
# --- Guest retention (only active when AIDND_MULTI_USER=1) ---
|
||||
# Every first visit mints a guest account, so a public demo collects one row
|
||||
# per visitor. A guest with no activity for this many days is deleted along
|
||||
# with its scenarios, adventures and actions. Registered accounts are never
|
||||
# touched. Default: 5. Set 0 to keep guests forever.
|
||||
AIDND_GUEST_RETENTION_DAYS=
|
||||
# How often a running process re-checks. The sweep also runs once at startup,
|
||||
# which is what actually fires on hosts that sleep. Default: 6
|
||||
AIDND_CLEANUP_INTERVAL_HOURS=
|
||||
|
||||
# The AI endpoint/API key/model are NOT env vars — they are configured at
|
||||
# runtime in the app's Settings page and stored (encrypted) in the database.
|
||||
#
|
||||
# Rate limits, request size limits, and per-user row caps are hardcoded with
|
||||
# generous values (see backend/app/limits.py) and active only in multi-user
|
||||
# mode — local installs are never throttled.
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
"""The access log: who arrived, when, and from where.
|
||||
|
||||
The deliberate opposite of analytics.py. That module counts and stores nothing
|
||||
that points at a person; this one records addresses, email addresses and
|
||||
devices, because an access log that cannot identify the access is not an access
|
||||
log. The two live in separate modules and separate tables on purpose, so that
|
||||
the anonymity of the counters is a property of the code rather than a convention
|
||||
someone has to remember.
|
||||
|
||||
Owner-only, and never shown to the people it records.
|
||||
|
||||
Four kinds of row:
|
||||
|
||||
- `session` A browser that has a session made a request. For a guest, this
|
||||
is their first visit.
|
||||
- `login` An existing account signed in.
|
||||
- `register` A guest upgraded to an account.
|
||||
- `login_failed` A password attempt that did not match, with the address tried.
|
||||
|
||||
Session rows are the only ones that need thinning. `/auth/me` runs on every page
|
||||
load, and one row per load would be noise rather than a log. A row is written
|
||||
when the day or the address changes for that user. That is the granularity a log
|
||||
is read at, such as seen on the 3rd from 1.2.3.4, and it still records someone
|
||||
moving networks during a day.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import threading
|
||||
|
||||
from sqlalchemy import desc, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import analytics, models
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SESSION = "session"
|
||||
LOGIN = "login"
|
||||
REGISTER = "register"
|
||||
LOGIN_FAILED = "login_failed"
|
||||
|
||||
MAX_UA = 200
|
||||
|
||||
# user id -> (day, ip) of the last session row written for them. Process-local
|
||||
# like the rate limiter's windows, and for the same reason: this is a single
|
||||
# process, and the worst case after a restart is one redundant row per user.
|
||||
_last_session: dict[int, tuple[str, str]] = {}
|
||||
_guard = threading.Lock()
|
||||
_MAX_TRACKED = 10_000
|
||||
|
||||
|
||||
def _client_ip(request) -> str:
|
||||
# This import is deferred. `limits` imports `auth`, which the routers that
|
||||
# call this function import, so a module-level import here would create a
|
||||
# cycle. The spoof resistance lives in `limits` and must not be
|
||||
# reimplemented. A second, looser answer to which address belongs to the
|
||||
# client is how one of them ends up trusting a header it should not.
|
||||
from . import limits
|
||||
|
||||
return limits.client_ip(request)
|
||||
|
||||
|
||||
def describe(user: models.User) -> str:
|
||||
"""Returns how a user is named in the log.
|
||||
|
||||
A guest has no email, and their id is the only handle anyone has for them.
|
||||
The third case is a local install's implicit single user, who also has no
|
||||
email but is the operator rather than a visitor. Naming that user "Guest #1"
|
||||
would be wrong in the one row they are certain to read.
|
||||
"""
|
||||
if user.email:
|
||||
return user.email
|
||||
return f"Guest #{user.id}" if user.is_guest else f"Local user #{user.id}"
|
||||
|
||||
|
||||
def _country(request) -> str:
|
||||
"""Returns the edge's country header, or "" when there is none.
|
||||
|
||||
The blank differs from the counters' "(unknown)" label. A table column reads
|
||||
better as a dash than as a word, and an empty string is the correct value for
|
||||
a country that is not known.
|
||||
"""
|
||||
country = analytics.country_of(request.headers)
|
||||
return "" if country == analytics.UNKNOWN else country
|
||||
|
||||
|
||||
def record(
|
||||
db: Session,
|
||||
kind: str,
|
||||
request,
|
||||
*,
|
||||
user: models.User | None = None,
|
||||
who: str | None = None,
|
||||
) -> None:
|
||||
"""Writes one row.
|
||||
|
||||
This function never raises. The log observes sign-in rather than guarding it,
|
||||
and a logging failure must not lock anyone out.
|
||||
"""
|
||||
try:
|
||||
event = models.AccessEvent(
|
||||
kind=kind,
|
||||
user_id=user.id if user is not None else None,
|
||||
who=(who if who is not None else describe(user) if user else "")[:320],
|
||||
is_guest=bool(user.is_guest) if user is not None else False,
|
||||
ip=_client_ip(request)[:45],
|
||||
country=_country(request),
|
||||
device=analytics.device_of(request.headers.get("user-agent", "")),
|
||||
user_agent=(request.headers.get("user-agent") or "")[:MAX_UA],
|
||||
)
|
||||
db.add(event)
|
||||
db.commit()
|
||||
except Exception: # pragma: no cover - defensive
|
||||
db.rollback()
|
||||
logger.exception("Access log write failed; continuing.")
|
||||
|
||||
|
||||
def note_session(db: Session, user: models.User, request) -> None:
|
||||
"""Records that a session made a request, at most one row per day per address."""
|
||||
try:
|
||||
today = analytics._today()
|
||||
ip = _client_ip(request)
|
||||
with _guard:
|
||||
if _last_session.get(user.id) == (today, ip):
|
||||
return
|
||||
_last_session[user.id] = (today, ip)
|
||||
if len(_last_session) > _MAX_TRACKED:
|
||||
# Nothing here needs to persist. Clearing the map costs at most
|
||||
# one extra row per active user.
|
||||
_last_session.clear()
|
||||
_last_session[user.id] = (today, ip)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Access log session check failed; continuing.")
|
||||
return
|
||||
record(db, SESSION, request, user=user)
|
||||
|
||||
|
||||
def recent(
|
||||
db: Session,
|
||||
*,
|
||||
limit: int = 50,
|
||||
before_id: int | None = None,
|
||||
kind: str | None = None,
|
||||
query: str | None = None,
|
||||
) -> dict:
|
||||
"""Returns a page of the log, newest first.
|
||||
|
||||
The page is anchored on a row id rather than an offset, as the story pager
|
||||
is. Rows keep arriving while the log is read, and an offset would shift the
|
||||
page under whoever is reading it.
|
||||
"""
|
||||
statement = select(models.AccessEvent).order_by(desc(models.AccessEvent.id))
|
||||
if before_id is not None:
|
||||
statement = statement.where(models.AccessEvent.id < before_id)
|
||||
if kind:
|
||||
statement = statement.where(models.AccessEvent.kind == kind)
|
||||
if query:
|
||||
like = f"%{query.strip()}%"
|
||||
statement = statement.where(or_(
|
||||
models.AccessEvent.who.ilike(like),
|
||||
models.AccessEvent.ip.ilike(like),
|
||||
models.AccessEvent.country.ilike(like),
|
||||
))
|
||||
# Requesting one extra row reports whether more rows exist, without a
|
||||
# second COUNT over the whole table.
|
||||
rows = list(db.scalars(statement.limit(limit + 1)))
|
||||
has_more = len(rows) > limit
|
||||
return {"events": rows[:limit], "has_more": has_more}
|
||||
@@ -1,583 +0,0 @@
|
||||
"""Visit analytics for the hosted demo.
|
||||
|
||||
This is a small self-hosted counter that answers whether anyone visited and
|
||||
whether they played. It is built into the app rather than added with a
|
||||
third-party script, because the CSP in `main.py` allows scripts from 'self'
|
||||
only, ad blockers block the popular trackers, and none of those trackers can see
|
||||
what is worth knowing here: turns taken, demo-key spend, and which seeded
|
||||
scenario people pick.
|
||||
|
||||
Three rules shape the design:
|
||||
|
||||
1. It stores nothing personal. It records no IP addresses, no user agents, no
|
||||
user ids, and no title of anything a player wrote. A visitor appears only as
|
||||
an HMAC of their user id, which is one-way and salted with the app's secret
|
||||
key, so these tables cannot be joined back to an account even by someone
|
||||
holding the database. Story content never reaches this module. What one
|
||||
specific person did is unanswerable by design, and only totals are
|
||||
available.
|
||||
2. Egress is the budget. Neon bills for bytes leaving the database, and this
|
||||
project has already paid for forgetting that once. Counts are therefore
|
||||
aggregated in memory and flushed as UPSERTs, so a visit is a write and never
|
||||
a read, and every dashboard query is a GROUP BY that returns tens of rows
|
||||
rather than per-visit rows. A month of traffic costs a few kilobytes to read
|
||||
back.
|
||||
3. The numbers come from the server, not from the browser. The client reports
|
||||
one thing, which is the page that was viewed. Everything with meaning, such
|
||||
as a turn happening or an account being created, is recorded by the code that
|
||||
performs it, where a stranger cannot fake it and an extension cannot block
|
||||
it.
|
||||
|
||||
Storage is two tables, both bounded. `analytics_daily` holds one counter row per
|
||||
day, metric, and label, which is a few dozen rows a day.
|
||||
`analytics_visitor_days` holds one row per visitor per day carrying the funnel
|
||||
flags, which is what makes the funnel count people rather than clicks. It is the
|
||||
only table that grows with traffic, and cleanup ages it out.
|
||||
"""
|
||||
|
||||
import hmac
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from datetime import timedelta
|
||||
from hashlib import sha256
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from sqlalchemy import case, func, or_, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import models, security
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------- Metrics ----------
|
||||
# `metric` is the family, and `label` is the bucket within it. One generic
|
||||
# counter table is better than a column per measurement, because adding a new
|
||||
# question later costs nothing rather than a migration.
|
||||
|
||||
M_PAGE = "pageview"
|
||||
M_EVENT = "event"
|
||||
M_REFERRER = "referrer"
|
||||
M_DEVICE = "device"
|
||||
M_COUNTRY = "country"
|
||||
M_SCENARIO = "scenario" # Which seeded or public scenario was played.
|
||||
M_ERROR = "error" # "<status> <route>" for a 4xx or 5xx on /api.
|
||||
|
||||
EV_SCENARIO_OPEN = "scenario_opened"
|
||||
EV_ADVENTURE = "adventure_created"
|
||||
EV_IMPORT = "adventure_imported"
|
||||
EV_TURN = "turn"
|
||||
EV_DEMO_TURN = "demo_turn" # A turn billed to the shared demo key.
|
||||
EV_TURN_ERROR = "turn_error"
|
||||
EV_SIGNUP = "signup"
|
||||
EV_LOGIN = "login"
|
||||
|
||||
# Events that are also funnel steps. Recording one sets a flag on the visitor's
|
||||
# row for the day, so the funnel counts distinct visitor-days rather than repeat
|
||||
# clicks. This name-to-column map is the whole definition of the funnel, and the
|
||||
# dashboard reads it back in this order.
|
||||
FUNNEL_FLAGS = {
|
||||
EV_SCENARIO_OPEN: "opened",
|
||||
EV_ADVENTURE: "created",
|
||||
EV_TURN: "played",
|
||||
EV_SIGNUP: "signed_up",
|
||||
}
|
||||
|
||||
OTHER = "(other)"
|
||||
NONE_LABEL = "(direct)"
|
||||
UNKNOWN = "(unknown)"
|
||||
|
||||
# ---------- Bounds ----------
|
||||
# These bounds exist so that a hostile visitor can add rows to these tables no
|
||||
# faster than an honest one. The only label a client can influence is the
|
||||
# referrer, and together these caps mean the worst it can do is fill one day's
|
||||
# referrer list and then be folded into "(other)".
|
||||
|
||||
MAX_LABEL_LEN = 80
|
||||
MAX_LABELS_PER_METRIC = 200 # Distinct labels per metric per day, then OTHER.
|
||||
MAX_PENDING = 4000 # Buffered entries before an inline flush.
|
||||
FLUSH_INTERVAL_SECONDS = 60
|
||||
|
||||
# How long the per-visitor-day rows are kept. The daily counters are small and
|
||||
# are kept indefinitely. These rows are the ones that scale with traffic. A
|
||||
# visitor whose last visit ages out counts as new again, which is an acceptable
|
||||
# trade at this horizon and keeps the table from being a permanent record of
|
||||
# anyone.
|
||||
RETENTION_DAYS = int(os.environ.get("AIDND_ANALYTICS_RETENTION_DAYS", "400") or 400)
|
||||
|
||||
_HOST_OK = re.compile(r"^[a-z0-9.-]+$")
|
||||
_COUNTRY_OK = re.compile(r"^[A-Z]{2}$")
|
||||
_NUMERIC_SEGMENT = re.compile(r"^\d+$")
|
||||
|
||||
# SPA routes, in the form the dashboard shows them. Any other path a client
|
||||
# reports becomes OTHER, so the page list cannot be filled with junk and cannot
|
||||
# record which adventure someone is reading.
|
||||
KNOWN_ROUTES = {
|
||||
"/", "/adventures", "/scenarios", "/scenarios/:id", "/play/:id",
|
||||
"/scripts", "/scripts/:id", "/settings", "/chat", "/analytics",
|
||||
}
|
||||
|
||||
# ---------- In-process buffer ----------
|
||||
# The deployment is a single process, which is the same assumption `limits.py`
|
||||
# makes, so a plain dict under a lock is the whole design. Losing up to a minute
|
||||
# of counts to a hard restart is acceptable for traffic numbers, and the flusher
|
||||
# also runs on shutdown. On Render's free tier the service is idle when it
|
||||
# sleeps, so the buffer it sleeps on is empty.
|
||||
|
||||
_counts: dict[tuple[str, str, str], int] = {}
|
||||
_visits: dict[tuple[str, str], set[str]] = {} # (day, visitor) -> flags.
|
||||
_labels_seen: dict[tuple[str, str], set[str]] = {} # (day, metric) -> labels.
|
||||
_guard = threading.Lock()
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return models.utcnow().date().isoformat()
|
||||
|
||||
|
||||
def record(metric: str, label: str = "", *, n: int = 1) -> None:
|
||||
"""Adds `n` to one counter.
|
||||
|
||||
This function never raises. Analytics must not fail a request that it is
|
||||
only observing.
|
||||
"""
|
||||
try:
|
||||
day = _today()
|
||||
label = (label or "").strip()[:MAX_LABEL_LEN]
|
||||
with _guard:
|
||||
seen = _labels_seen.setdefault((day, metric), set())
|
||||
if label not in seen:
|
||||
if len(seen) >= MAX_LABELS_PER_METRIC:
|
||||
label = OTHER
|
||||
else:
|
||||
seen.add(label)
|
||||
key = (day, metric, label)
|
||||
_counts[key] = _counts.get(key, 0) + n
|
||||
pending = len(_counts) + len(_visits)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Analytics counter failed; continuing.")
|
||||
return
|
||||
if pending >= MAX_PENDING:
|
||||
flush()
|
||||
|
||||
|
||||
def visitor_id(user: models.User) -> str:
|
||||
"""Returns a stable, one-way handle for one visitor.
|
||||
|
||||
The handle is an HMAC of the user id under the app's secret key. It is
|
||||
stable, so a returning visitor can be distinguished from a new one. It is
|
||||
one-way, so nothing in the analytics tables points back at an account. It is
|
||||
keyed, so a client cannot compute one and claim to be someone else. One
|
||||
consequence follows: rotating `AIDND_SECRET_KEY` makes every returning
|
||||
visitor look new.
|
||||
"""
|
||||
digest = hmac.new(security.SECRET_KEY, f"visitor:{user.id}".encode(), sha256)
|
||||
return digest.hexdigest()[:32]
|
||||
|
||||
|
||||
def record_visit(user: models.User | None, *, flag: str | None = None) -> None:
|
||||
"""Records that this visitor was here today, and optionally sets one funnel
|
||||
flag.
|
||||
|
||||
Without a user the call does nothing. A page loaded before a session exists
|
||||
still counts as a pageview, but not as a person.
|
||||
"""
|
||||
if user is None:
|
||||
return
|
||||
try:
|
||||
with _guard:
|
||||
flags = _visits.setdefault((_today(), visitor_id(user)), set())
|
||||
if flag:
|
||||
flags.add(flag)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
logger.exception("Analytics visit failed; continuing.")
|
||||
|
||||
|
||||
def record_event(name: str, user: models.User | None = None) -> None:
|
||||
"""Records one event, and credits the visitor's day if it is a funnel step.
|
||||
|
||||
This is the whole interface the call sites use.
|
||||
"""
|
||||
record(M_EVENT, name)
|
||||
record_visit(user, flag=FUNNEL_FLAGS.get(name))
|
||||
|
||||
|
||||
# ---------- Normalizing what the browser reports ----------
|
||||
|
||||
def normalize_route(path: str) -> str:
|
||||
"""Reduces a client-reported path to one of `KNOWN_ROUTES`.
|
||||
|
||||
Numeric segments become ":id". That bounds the label count, and it keeps
|
||||
which adventure someone opened out of the statistics.
|
||||
"""
|
||||
path = (path or "/").split("?")[0].split("#")[0]
|
||||
if not path.startswith("/"):
|
||||
path = "/" + path
|
||||
if len(path) > 1:
|
||||
path = path.rstrip("/")
|
||||
parts = [":id" if _NUMERIC_SEGMENT.match(p) else p for p in path.split("/")]
|
||||
route = "/".join(parts) or "/"
|
||||
return route if route in KNOWN_ROUTES else OTHER
|
||||
|
||||
|
||||
def normalize_referrer(referrer: str, own_host: str = "") -> str:
|
||||
"""Returns the sending site as a bare host.
|
||||
|
||||
This app's own host means an internal navigation, which is not a referral.
|
||||
In that case the function returns "", which tells the caller to skip it.
|
||||
"""
|
||||
if not referrer:
|
||||
return NONE_LABEL
|
||||
host = (urlsplit(referrer).hostname or "").lower().lstrip(".")
|
||||
if not host or not _HOST_OK.match(host) or len(host) > MAX_LABEL_LEN:
|
||||
return OTHER
|
||||
if host == (own_host or "").lower() or host in ("localhost", "127.0.0.1"):
|
||||
return ""
|
||||
return host[4:] if host.startswith("www.") else host
|
||||
|
||||
|
||||
def api_route_label(scope: dict, status: int) -> str:
|
||||
"""Returns an error bucket such as "500 /api/adventures/{adventure_id}".
|
||||
|
||||
The label uses the route template, never the request path. That keeps one
|
||||
bucket per endpoint rather than one per adventure id. It also bounds the
|
||||
table: an unmatched path is chosen entirely by the caller, so labeling by it
|
||||
would let anyone create rows by requesting arbitrary paths.
|
||||
"""
|
||||
template = getattr(scope.get("route"), "path", None)
|
||||
return f"{status} {template}" if template else f"{status} (unmatched)"
|
||||
|
||||
|
||||
def device_of(user_agent: str) -> str:
|
||||
"""Returns "mobile", "tablet", or "desktop", and nothing more specific.
|
||||
|
||||
The user-agent string itself is never stored, because it is a fingerprint
|
||||
and the useful answer is one word.
|
||||
"""
|
||||
ua = (user_agent or "").lower()
|
||||
if not ua:
|
||||
return UNKNOWN
|
||||
if any(bot in ua for bot in ("bot", "crawler", "spider", "headless", "preview")):
|
||||
return "bot"
|
||||
if "ipad" in ua or "tablet" in ua or ("android" in ua and "mobile" not in ua):
|
||||
return "tablet"
|
||||
if any(m in ua for m in ("mobi", "iphone", "ipod", "android", "phone")):
|
||||
return "mobile"
|
||||
return "desktop"
|
||||
|
||||
|
||||
# Geo headers an edge network may add. Render fronts services with a CDN that
|
||||
# can set `cf-ipcountry`, and the others cost nothing to check. A value is
|
||||
# trusted only if it looks like an ISO code, because a client can send any
|
||||
# header, so the worst case is a wrong country rather than an unbounded label.
|
||||
_GEO_HEADERS = ("cf-ipcountry", "x-vercel-ip-country", "x-geo-country", "x-country-code")
|
||||
|
||||
|
||||
def country_of(headers) -> str:
|
||||
for name in _GEO_HEADERS:
|
||||
value = (headers.get(name) or "").strip().upper()
|
||||
if _COUNTRY_OK.match(value) and value != "XX":
|
||||
return value
|
||||
return UNKNOWN
|
||||
|
||||
|
||||
# ---------- Flushing ----------
|
||||
|
||||
def _insert(db: Session):
|
||||
return sqlite_insert if db.get_bind().dialect.name == "sqlite" else pg_insert
|
||||
|
||||
|
||||
def _drain() -> tuple[dict, dict]:
|
||||
with _guard:
|
||||
counts, visits = _counts.copy(), _visits.copy()
|
||||
_counts.clear()
|
||||
_visits.clear()
|
||||
# The label sets bound cardinality within one day, so drop the
|
||||
# previous day's rather than grow a map that never shrinks.
|
||||
today = _today()
|
||||
for key in [k for k in _labels_seen if k[0] != today]:
|
||||
del _labels_seen[key]
|
||||
return counts, visits
|
||||
|
||||
|
||||
def _restore(counts: dict, visits: dict) -> None:
|
||||
"""Returns a failed flush's work to the buffer, so the next flush retries it."""
|
||||
with _guard:
|
||||
for key, n in counts.items():
|
||||
_counts[key] = _counts.get(key, 0) + n
|
||||
for key, flags in visits.items():
|
||||
_visits.setdefault(key, set()).update(flags)
|
||||
|
||||
|
||||
def flush(db: Session | None = None) -> None:
|
||||
"""Writes the buffer out. This is safe to call from anywhere and never raises."""
|
||||
counts, visits = _drain()
|
||||
if not counts and not visits:
|
||||
return
|
||||
own_session = db is None
|
||||
if own_session:
|
||||
from .database import SessionLocal
|
||||
db = SessionLocal()
|
||||
try:
|
||||
_write_counts(db, counts)
|
||||
_write_visits(db, visits)
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
_restore(counts, visits)
|
||||
logger.exception("Analytics flush failed; counts held for the next one.")
|
||||
finally:
|
||||
if own_session:
|
||||
db.close()
|
||||
|
||||
|
||||
def _write_counts(db: Session, counts: dict) -> None:
|
||||
if not counts:
|
||||
return
|
||||
table = models.AnalyticsDaily.__table__
|
||||
rows = [
|
||||
{"day": day, "metric": metric, "label": label, "hits": hits}
|
||||
for (day, metric, label), hits in counts.items()
|
||||
]
|
||||
stmt = _insert(db)(table).values(rows)
|
||||
db.execute(stmt.on_conflict_do_update(
|
||||
index_elements=["day", "metric", "label"],
|
||||
set_={"hits": table.c.hits + stmt.excluded.hits},
|
||||
))
|
||||
|
||||
|
||||
def _write_visits(db: Session, visits: dict) -> None:
|
||||
if not visits:
|
||||
return
|
||||
table = models.AnalyticsVisitorDay.__table__
|
||||
ids = {visitor for _, visitor in visits}
|
||||
# One indexed lookup decides new against returning for the whole batch. It
|
||||
# is the only read this module makes outside the dashboard, and it returns
|
||||
# short hashes for the visitors active right now, so the batch bounds it.
|
||||
known = set(db.scalars(
|
||||
select(models.AnalyticsVisitorDay.visitor)
|
||||
.where(models.AnalyticsVisitorDay.visitor.in_(ids))
|
||||
.distinct()
|
||||
))
|
||||
rows = [
|
||||
{
|
||||
"day": day,
|
||||
"visitor": visitor,
|
||||
"is_new": visitor not in known,
|
||||
**{column: column in flags for column in FUNNEL_FLAGS.values()},
|
||||
}
|
||||
for (day, visitor), flags in visits.items()
|
||||
]
|
||||
stmt = _insert(db)(table).values(rows)
|
||||
db.execute(stmt.on_conflict_do_update(
|
||||
index_elements=["day", "visitor"],
|
||||
# Flags only turn on, and `is_new` is absent on purpose. The first
|
||||
# write of a visitor's first day is what decided it.
|
||||
set_={
|
||||
column: or_(table.c[column], stmt.excluded[column])
|
||||
for column in FUNNEL_FLAGS.values()
|
||||
},
|
||||
))
|
||||
|
||||
|
||||
def purge_old_visitor_days(db: Session) -> int:
|
||||
"""Deletes visitor-day rows past the retention horizon.
|
||||
|
||||
The cleanup sweeper calls this. The daily counters are never purged, because
|
||||
they are aggregates, they are small, and this project keeps its history.
|
||||
"""
|
||||
if RETENTION_DAYS <= 0:
|
||||
return 0
|
||||
cutoff = (models.utcnow().date() - timedelta(days=RETENTION_DAYS)).isoformat()
|
||||
removed = db.query(models.AnalyticsVisitorDay).filter(
|
||||
models.AnalyticsVisitorDay.day < cutoff
|
||||
).delete(synchronize_session=False)
|
||||
db.commit()
|
||||
return removed or 0
|
||||
|
||||
|
||||
# ---------- Reading it back ----------
|
||||
# Every query below is an aggregate. The database does the counting and returns
|
||||
# tens of rows, however much traffic is behind them. No query here can return a
|
||||
# row that belongs to one visitor.
|
||||
|
||||
TOP_N = 12
|
||||
|
||||
|
||||
def _top(rows: list[dict], limit: int = TOP_N) -> list[dict]:
|
||||
return rows[:limit]
|
||||
|
||||
|
||||
def summary(db: Session, days: int = 30) -> dict:
|
||||
"""Returns everything the dashboard shows for the last `days` days, including
|
||||
today.
|
||||
|
||||
The function flushes first, so the numbers include the last minute.
|
||||
"""
|
||||
flush(db)
|
||||
today = models.utcnow().date()
|
||||
since = (today - timedelta(days=days - 1)).isoformat()
|
||||
daily = models.AnalyticsDaily
|
||||
visitor = models.AnalyticsVisitorDay
|
||||
|
||||
# 1. Every counter in the window, reduced to (metric, label) totals. The
|
||||
# page, referrer, country, device, scenario, and error tables all come
|
||||
# from this one pass rather than from a query each.
|
||||
by_metric: dict[str, list[dict]] = {}
|
||||
for metric, label, hits in db.execute(
|
||||
select(daily.metric, daily.label, func.sum(daily.hits))
|
||||
.where(daily.day >= since)
|
||||
.group_by(daily.metric, daily.label)
|
||||
):
|
||||
by_metric.setdefault(metric, []).append({"label": label, "hits": int(hits)})
|
||||
for rows in by_metric.values():
|
||||
rows.sort(key=lambda row: -row["hits"])
|
||||
events = {row["label"]: row["hits"] for row in by_metric.get(M_EVENT, [])}
|
||||
|
||||
# 2. The two per-day series the dashboard draws.
|
||||
pageviews_by_day = {
|
||||
day: int(hits)
|
||||
for day, hits in db.execute(
|
||||
select(daily.day, func.sum(daily.hits))
|
||||
.where(daily.day >= since, daily.metric == M_PAGE)
|
||||
.group_by(daily.day)
|
||||
)
|
||||
}
|
||||
turns_by_day = {
|
||||
day: int(hits)
|
||||
for day, hits in db.execute(
|
||||
select(daily.day, func.sum(daily.hits))
|
||||
.where(daily.day >= since, daily.metric == M_EVENT, daily.label == EV_TURN)
|
||||
.group_by(daily.day)
|
||||
)
|
||||
}
|
||||
|
||||
# 3. People, per day. There is one row per visitor per day, so COUNT(*) is
|
||||
# already the day's unique visitors and no DISTINCT is needed.
|
||||
visitors_by_day: dict[str, dict] = {}
|
||||
for day, total, fresh in db.execute(
|
||||
select(
|
||||
visitor.day,
|
||||
func.count(),
|
||||
func.sum(case((visitor.is_new, 1), else_=0)),
|
||||
)
|
||||
.where(visitor.day >= since)
|
||||
.group_by(visitor.day)
|
||||
):
|
||||
visitors_by_day[day] = {"visitors": int(total), "new": int(fresh or 0)}
|
||||
|
||||
# 4. The funnel over the whole window, counting each person once.
|
||||
# COUNT(DISTINCT CASE WHEN flag THEN visitor END) ignores the NULLs the
|
||||
# CASE leaves for everyone who did not reach that step.
|
||||
unique, unique_new, *reached = db.execute(
|
||||
select(
|
||||
func.count(func.distinct(visitor.visitor)),
|
||||
func.count(func.distinct(case((visitor.is_new, visitor.visitor)))),
|
||||
*[
|
||||
func.count(func.distinct(case((visitor.__table__.c[column], visitor.visitor))))
|
||||
for column in FUNNEL_FLAGS.values()
|
||||
],
|
||||
).where(visitor.day >= since)
|
||||
).one()
|
||||
|
||||
series = []
|
||||
for offset in range(days):
|
||||
day = (today - timedelta(days=days - 1 - offset)).isoformat()
|
||||
counted = visitors_by_day.get(day, {})
|
||||
series.append({
|
||||
"day": day,
|
||||
"visitors": counted.get("visitors", 0),
|
||||
"new": counted.get("new", 0),
|
||||
"pageviews": pageviews_by_day.get(day, 0),
|
||||
"turns": turns_by_day.get(day, 0),
|
||||
})
|
||||
|
||||
visits = sum(row["visitors"] for row in series)
|
||||
pageviews = sum(pageviews_by_day.values())
|
||||
turns = events.get(EV_TURN, 0)
|
||||
errors = by_metric.get(M_ERROR, [])
|
||||
return {
|
||||
"days": days,
|
||||
"since": since,
|
||||
"until": today.isoformat(),
|
||||
"generated_at": models.utcnow().isoformat(),
|
||||
"totals": {
|
||||
# `visitors` counts each person once for the window. `visits`
|
||||
# counts them once per day they returned, which is the closest
|
||||
# measure to "sessions" that does not track sessions.
|
||||
"visitors": int(unique),
|
||||
"new_visitors": int(unique_new),
|
||||
"visits": visits,
|
||||
"pageviews": pageviews,
|
||||
"turns": turns,
|
||||
"demo_turns": events.get(EV_DEMO_TURN, 0),
|
||||
"adventures": events.get(EV_ADVENTURE, 0),
|
||||
"signups": events.get(EV_SIGNUP, 0),
|
||||
"logins": events.get(EV_LOGIN, 0),
|
||||
"turn_errors": events.get(EV_TURN_ERROR, 0),
|
||||
"errors": sum(row["hits"] for row in errors),
|
||||
"turns_per_visit": round(turns / visits, 1) if visits else 0,
|
||||
"pages_per_visit": round(pageviews / visits, 1) if visits else 0,
|
||||
},
|
||||
"series": series,
|
||||
# Step 0 is everyone who arrived, so the drop-off between it and
|
||||
# "Opened a scenario" appears as a step like any other.
|
||||
"funnel": [{"step": "Visited", "count": int(unique)}] + [
|
||||
{"step": step, "count": int(count)}
|
||||
for step, count in zip(
|
||||
["Opened a scenario", "Started an adventure", "Played a turn", "Signed up"],
|
||||
reached,
|
||||
)
|
||||
],
|
||||
"pages": _top(by_metric.get(M_PAGE, [])),
|
||||
"referrers": _top(by_metric.get(M_REFERRER, [])),
|
||||
"countries": _top(by_metric.get(M_COUNTRY, [])),
|
||||
"devices": by_metric.get(M_DEVICE, []),
|
||||
"scenarios": _top(by_metric.get(M_SCENARIO, [])),
|
||||
"errors": _top(errors),
|
||||
"events": by_metric.get(M_EVENT, []),
|
||||
}
|
||||
|
||||
|
||||
# ---------- Background flusher ----------
|
||||
# This matches the start and stop pair in `cleanup`, so the lifespan in
|
||||
# `main.py` reads the same way for both. The interval bounds how much a hard
|
||||
# restart can lose.
|
||||
|
||||
async def _flush_loop() -> None:
|
||||
import asyncio
|
||||
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
while True:
|
||||
await asyncio.sleep(FLUSH_INTERVAL_SECONDS)
|
||||
# This is blocking database work, so keep it off the event loop, which
|
||||
# is also serving SSE turn streams.
|
||||
await run_in_threadpool(flush)
|
||||
|
||||
|
||||
def start_flusher():
|
||||
import asyncio
|
||||
|
||||
return asyncio.create_task(_flush_loop())
|
||||
|
||||
|
||||
async def stop_flusher(task) -> None:
|
||||
"""Cancels the loop and writes out whatever it was holding.
|
||||
|
||||
A deploy is the one restart that is both frequent and predictable, so it
|
||||
should not be what loses a minute of counts.
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
await run_in_threadpool(flush)
|
||||
@@ -41,11 +41,11 @@ from .context import lineage
|
||||
|
||||
# The slices of a context snapshot that belong to one attempt rather than to the
|
||||
# turn. They are the world-state delta the attempt proposed and what the engine
|
||||
# did with it, the script report, the model's literal reply, and the endpoint's
|
||||
# did with it, the model's literal reply, and the endpoint's
|
||||
# token accounting. Each attempt is its own API call, and a retry is the call
|
||||
# most likely to read the prompt back out of cache. Everything else in a snapshot
|
||||
# is the prompt, which is assembled once per turn.
|
||||
ATTEMPT_KEYS = ("world_state", "script", "raw_output", "usage")
|
||||
ATTEMPT_KEYS = ("world_state", "raw_output", "usage")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ reading
|
||||
@@ -152,7 +152,7 @@ def preceding(
|
||||
# ------------------------------------------------------------------ writing
|
||||
|
||||
def restore_state(adventure: models.Adventure, node: models.Action | None) -> None:
|
||||
"""Restores the script state and world state that `node` left behind.
|
||||
"""Restores the world state that `node` left behind.
|
||||
|
||||
A NULL snapshot means leave the live state as it is, never reset it. Rows
|
||||
written before SP4 that the migration could not derive an outcome for carry
|
||||
@@ -161,17 +161,16 @@ def restore_state(adventure: models.Adventure, node: models.Action | None) -> No
|
||||
"""
|
||||
if node is None:
|
||||
return
|
||||
if isinstance(node.state_after, dict):
|
||||
adventure.script_state = copy.deepcopy(node.state_after)
|
||||
if isinstance(node.world_state_after, dict):
|
||||
adventure.world_state = copy.deepcopy(node.world_state_after)
|
||||
|
||||
|
||||
def snapshot_outcome(adventure: models.Adventure, node: models.Action) -> None:
|
||||
"""Records on `node` the state of the adventure now that the node has played."""
|
||||
state = adventure.script_state if isinstance(adventure.script_state, dict) else {}
|
||||
world = adventure.world_state if isinstance(adventure.world_state, dict) else {}
|
||||
node.state_after = copy.deepcopy(state)
|
||||
# `state_after` held the scripting engine's shared state, which M2 removed.
|
||||
# The column stays for schema compatibility and is written empty.
|
||||
node.state_after = {}
|
||||
node.world_state_after = copy.deepcopy(world)
|
||||
|
||||
|
||||
|
||||
+34
-219
@@ -1,216 +1,44 @@
|
||||
"""Phase 8: user resolution, sessions, and the shared demo key.
|
||||
"""Resolving the one local user. **There is no authentication in this product.**
|
||||
|
||||
The `AIDND_MULTI_USER` environment variable selects one of two modes:
|
||||
The module keeps its name so the dependency every router already depends on
|
||||
keeps working, but nothing here authenticates anybody. The Adventure
|
||||
Storyteller is a single-user application that binds to loopback: whoever can
|
||||
reach the API is the person who started it, and there is nobody else to tell
|
||||
them apart from.
|
||||
|
||||
* Local mode, the default. Every request resolves to one automatically created
|
||||
local user. There are no cookies and no login UI, so a clone or a
|
||||
docker-compose run behaves like the single-user app from before Phase 8.
|
||||
* Multi-user mode, used for hosted deployments. Requests carry a signed session
|
||||
cookie. `GET /api/auth/me` creates a guest user on the first visit, and
|
||||
registering upgrades that guest in place so their data survives. A request
|
||||
without a valid session gets a 401, and the frontend re-establishes the
|
||||
session through `/me`.
|
||||
Upstream had two modes. `AIDND_MULTI_USER` selected a hosted deployment with
|
||||
signed session cookies, guest accounts, registration, login, a shared demo API
|
||||
key with a per-day cap, "power users", and an owner allowlist for the analytics
|
||||
dashboard. M2 removed all of it: this product has no hosted mode to protect, and
|
||||
every one of those surfaces was a way for the application to be reached by
|
||||
someone other than its owner.
|
||||
|
||||
The shared demo key, which is the fallback when a user brings no key of their
|
||||
own, is also configured here. A user whose settings hold no API key is routed to
|
||||
a server-funded endpoint with a model allowlist and a per-day turn cap.
|
||||
What is left is the local path that upstream already had. Every request
|
||||
resolves to one automatically created user row.
|
||||
|
||||
The `users` table and the `user_id` foreign keys on scenarios, adventures and
|
||||
settings stay. They are an **internal ownership detail**, not an account
|
||||
system: nothing creates a second user, nothing logs in, and no request carries
|
||||
an identity. They remain because rewriting them out would mean a migration
|
||||
across most of the schema to delete a column that costs nothing and keeps every
|
||||
existing M1 database readable.
|
||||
"""
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from datetime import timezone
|
||||
|
||||
from fastapi import Depends, HTTPException, Request
|
||||
from fastapi import Depends, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import models, security
|
||||
from . import models
|
||||
from .database import get_db
|
||||
|
||||
|
||||
def _env_flag(name: str) -> bool:
|
||||
return os.environ.get(name, "").strip().lower() in ("1", "true", "yes", "on")
|
||||
|
||||
|
||||
MULTI_USER = _env_flag("AIDND_MULTI_USER")
|
||||
|
||||
SESSION_COOKIE = "aidnd_session"
|
||||
# Secure cookies are on by default in multi-user mode, because a hosted
|
||||
# deployment serves HTTPS and browsers also accept Secure on http://localhost.
|
||||
# `AIDND_COOKIE_SECURE` overrides the default with 0 or 1. Use 0 when testing
|
||||
# multi-user mode over plain HTTP on a LAN address.
|
||||
_cookie_secure_env = os.environ.get("AIDND_COOKIE_SECURE", "").strip().lower()
|
||||
COOKIE_SECURE = (
|
||||
_cookie_secure_env in ("1", "true", "yes", "on")
|
||||
if _cookie_secure_env
|
||||
else MULTI_USER
|
||||
)
|
||||
COOKIE_MAX_AGE = 60 * 60 * 24 * 365
|
||||
|
||||
# ---------- Shared demo key (BYOK fallback) ----------
|
||||
|
||||
DEMO_API_KEY = os.environ.get("AIDND_DEMO_API_KEY", "").strip()
|
||||
DEMO_ENDPOINT_URL = (
|
||||
os.environ.get("AIDND_DEMO_ENDPOINT_URL", "").strip()
|
||||
or "https://openrouter.ai/api/v1"
|
||||
)
|
||||
DEMO_MODELS = [
|
||||
m.strip()
|
||||
for m in os.environ.get("AIDND_DEMO_MODELS", "").split(",")
|
||||
if m.strip()
|
||||
] or ["google/gemma-4-26b-a4b-it:free"]
|
||||
DEMO_TURNS_PER_DAY = int(os.environ.get("AIDND_DEMO_TURNS_PER_DAY", "20") or 20)
|
||||
|
||||
# Trusted testers, listed by email, who bypass the daily demo cap and take
|
||||
# unmetered turns on the shared demo key. The list is comma-separated, and the
|
||||
# match ignores case.
|
||||
POWER_USERS = {
|
||||
e.strip().lower()
|
||||
for e in os.environ.get("AIDND_POWER_USERS", "").split(",")
|
||||
if e.strip()
|
||||
}
|
||||
|
||||
# Who can see the visit analytics. This is a separate list from `POWER_USERS` on
|
||||
# purpose. A trusted tester gets unmetered turns and the AI Chat page, which is
|
||||
# not a reason to give them the site's traffic numbers. An empty list, which is
|
||||
# the default, means nobody sees the dashboard in a hosted deployment.
|
||||
ANALYTICS_EMAILS = {
|
||||
e.strip().lower()
|
||||
for e in os.environ.get("AIDND_ANALYTICS_EMAILS", "").split(",")
|
||||
if e.strip()
|
||||
}
|
||||
|
||||
DEMO_CAP_MESSAGE = (
|
||||
f"You've used all {DEMO_TURNS_PER_DAY} free demo turns for today. "
|
||||
"Add your own API key in Settings to keep playing (it resets tomorrow)."
|
||||
)
|
||||
|
||||
|
||||
def demo_enabled() -> bool:
|
||||
# The demo key is a hosted-deployment feature. A local install talks to
|
||||
# whatever endpoint Settings points at, even with no API key, such as
|
||||
# Ollama.
|
||||
return MULTI_USER and bool(DEMO_API_KEY)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProviderConfig:
|
||||
"""What the turn engine connects with, after the decision between a
|
||||
user-supplied key and the demo key.
|
||||
|
||||
Build one of these with `resolve_provider_config()`.
|
||||
"""
|
||||
|
||||
endpoint_url: str
|
||||
api_key: str
|
||||
model: str
|
||||
using_demo: bool
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# A second guard around server-funded turns. `resolve_provider_config()`
|
||||
# already pins the model, and this makes the pin a property of the config
|
||||
# object too, so a later caller cannot construct an unpinned one. This
|
||||
# raise is unreachable by design. Reaching it means a new code path
|
||||
# bypassed the pinning, which is worth failing on rather than billing
|
||||
# for.
|
||||
#
|
||||
# The test is `using_demo`, not `api_key == DEMO_API_KEY`. Keying on the
|
||||
# key value looks stricter and is wrong. The demo key is an ordinary
|
||||
# OpenRouter key, so a user can legitimately paste that same key into
|
||||
# their own Settings. Every resolution then raised, which returned a 500
|
||||
# even from `GET /auth/me` and took the whole SPA down. `using_demo` is
|
||||
# what means the server is paying, and only the demo branch below sets
|
||||
# it.
|
||||
if self.using_demo and self.model not in DEMO_MODELS:
|
||||
raise ValueError(
|
||||
f"Refusing to use the shared demo key with non-whitelisted model {self.model!r}"
|
||||
)
|
||||
|
||||
|
||||
def resolve_provider_config(
|
||||
settings: models.Settings, *, model_override: str | None = None
|
||||
) -> ProviderConfig:
|
||||
"""Returns the user's own key when they have one, and the shared demo key
|
||||
otherwise.
|
||||
|
||||
The demo branch is the security-relevant one, and it is the only place the
|
||||
allowlist rule lives. Every caller has to come through this function rather
|
||||
than build a `ProviderConfig` itself. On the demo key:
|
||||
|
||||
* The model is pinned to `DEMO_MODELS`, so a caller-supplied override from
|
||||
the AI Chat page, or a hand-edited Settings row, cannot point a
|
||||
server-funded key at a paid model. An unrecognized model falls back to
|
||||
`DEMO_MODELS[0]`.
|
||||
* The endpoint is pinned to `DEMO_ENDPOINT_URL`, so the key cannot be
|
||||
redirected to a URL the user controls and captured there.
|
||||
|
||||
`model_override` is a per-request preference and never a grant. It is used
|
||||
verbatim with the user's own key, and on the demo key only when the model is
|
||||
on the allowlist.
|
||||
"""
|
||||
key = settings.api_key_plain
|
||||
requested = (model_override or "").strip() or settings.model
|
||||
if key or not demo_enabled():
|
||||
return ProviderConfig(settings.endpoint_url, key, requested, False)
|
||||
model = requested if requested in DEMO_MODELS else DEMO_MODELS[0]
|
||||
return ProviderConfig(DEMO_ENDPOINT_URL, DEMO_API_KEY, model, True)
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return models.utcnow().date().isoformat()
|
||||
|
||||
|
||||
def is_power_user(user: models.User) -> bool:
|
||||
"""Returns whether this user is a trusted tester.
|
||||
|
||||
A trusted tester gets unmetered demo turns, plus tooling that is not part of
|
||||
the game, such as the AI Chat scratchpad. A local install is always trusted,
|
||||
because it runs on the operator's own machine with their own API key. The
|
||||
provider debug log is local-only for the same reason.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
return True
|
||||
return bool(user.email) and user.email.lower() in POWER_USERS
|
||||
|
||||
|
||||
def is_owner(user: models.User) -> bool:
|
||||
"""Returns whether this user may see the visit analytics.
|
||||
|
||||
A local install always may, because it runs on the operator's own machine and
|
||||
shows their own visits. The provider debug log follows the same reasoning. A
|
||||
hosted deployment checks `AIDND_ANALYTICS_EMAILS`.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
return True
|
||||
return bool(user.email) and user.email.lower() in ANALYTICS_EMAILS
|
||||
|
||||
|
||||
def demo_turns_left(user: models.User) -> int:
|
||||
# A power user is never capped, so report the full cap and let the banner
|
||||
# read "N of N" rather than count down.
|
||||
if is_power_user(user):
|
||||
return DEMO_TURNS_PER_DAY
|
||||
used = user.demo_turns_used if user.demo_turns_date == _today() else 0
|
||||
return max(0, DEMO_TURNS_PER_DAY - used)
|
||||
|
||||
|
||||
def count_demo_turn(user: models.User) -> None:
|
||||
"""Records one demo turn. The caller's commit stores it."""
|
||||
if is_power_user(user):
|
||||
return # A power user's turns do not count against the cap.
|
||||
today = _today()
|
||||
if user.demo_turns_date != today:
|
||||
user.demo_turns_date = today
|
||||
user.demo_turns_used = 0
|
||||
user.demo_turns_used += 1
|
||||
|
||||
|
||||
# ---------- User resolution ----------
|
||||
|
||||
def local_user(db: Session) -> models.User:
|
||||
"""Returns the single implicit user used in local mode.
|
||||
"""Returns the single implicit user, creating it on first use.
|
||||
|
||||
A migration gives this user ownership of data written before Phase 8. On a
|
||||
fresh database the user is created on first use.
|
||||
A migration gives this user ownership of data written before per-user rows
|
||||
existed, so an older database resolves to the row that already owns its
|
||||
campaigns rather than to a fresh empty one.
|
||||
"""
|
||||
user = (
|
||||
db.query(models.User)
|
||||
@@ -237,27 +65,14 @@ def _touch(user: models.User, db: Session) -> None:
|
||||
db.commit()
|
||||
|
||||
|
||||
def resolve_session_user(request: Request, db: Session) -> models.User | None:
|
||||
token = request.cookies.get(SESSION_COOKIE)
|
||||
if not token:
|
||||
return None
|
||||
user_id = security.verify_session(token)
|
||||
if user_id is None:
|
||||
return None
|
||||
return db.get(models.User, user_id)
|
||||
def get_current_user(
|
||||
request: Request, db: Session = Depends(get_db)
|
||||
) -> models.User:
|
||||
"""The dependency every router uses. It always succeeds.
|
||||
|
||||
|
||||
def get_current_user(request: Request, db: Session = Depends(get_db)) -> models.User:
|
||||
"""The dependency every router uses to resolve the current user.
|
||||
|
||||
In multi-user mode a 401 means the frontend has to establish a session again
|
||||
through `GET /api/auth/me`.
|
||||
`request` is unused and kept so the signature stays a FastAPI dependency
|
||||
the routers can depend on unchanged.
|
||||
"""
|
||||
if not MULTI_USER:
|
||||
user = local_user(db)
|
||||
else:
|
||||
user = resolve_session_user(request, db)
|
||||
if user is None:
|
||||
raise HTTPException(401, "No session. Call GET /api/auth/me first.")
|
||||
user = local_user(db)
|
||||
_touch(user, db)
|
||||
return user
|
||||
|
||||
+4
-24
@@ -109,7 +109,6 @@ def export(db: Session, adventure: models.Adventure) -> dict:
|
||||
"pronouns": adventure.persona_pronouns,
|
||||
"desc": adventure.persona_desc,
|
||||
},
|
||||
"scriptState": adventure.script_state,
|
||||
"worldState": adventure.world_state,
|
||||
"autoSummarize": adventure.auto_summarize,
|
||||
"memoryBankEnabled": adventure.memory_bank_enabled,
|
||||
@@ -126,15 +125,6 @@ def export(db: Session, adventure: models.Adventure) -> dict:
|
||||
"entry": c.entry, "notes": c.notes}
|
||||
for c in adventure.story_cards
|
||||
],
|
||||
"scripts": [
|
||||
{
|
||||
"position": s.position, "enabled": s.enabled,
|
||||
"name": s.name, "description": s.description,
|
||||
"library": s.library_js, "input": s.input_js,
|
||||
"context": s.context_js, "output": s.output_js,
|
||||
}
|
||||
for s in adventure.scripts
|
||||
],
|
||||
"actions": [_exported_node(a, local) for a in nodes],
|
||||
}
|
||||
|
||||
@@ -654,7 +644,6 @@ def materialize(
|
||||
authors_note=str(payload.get("authorsNote") or ""),
|
||||
ai_instructions=str(payload.get("aiInstructions") or ""),
|
||||
story_summary=str(payload.get("storySummary") or ""),
|
||||
script_state=payload.get("scriptState") or {},
|
||||
world_state=payload.get("worldState") or {},
|
||||
auto_summarize=bool(payload.get("autoSummarize", False)),
|
||||
memory_bank_enabled=bool(payload.get("memoryBankEnabled", False)),
|
||||
@@ -674,19 +663,10 @@ def materialize(
|
||||
notes=str(card.get("notes") or ""),
|
||||
))
|
||||
|
||||
for i, item in enumerate(payload.get("scripts") or []):
|
||||
if isinstance(item, dict):
|
||||
db.add(models.AdventureScript(
|
||||
adventure_id=adventure.id,
|
||||
position=int(item.get("position", i)),
|
||||
enabled=bool(item.get("enabled", True)),
|
||||
name=str(item.get("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=str(item.get("description") or ""),
|
||||
library_js=str(item.get("library") or ""),
|
||||
input_js=str(item.get("input") or ""),
|
||||
context_js=str(item.get("context") or ""),
|
||||
output_js=str(item.get("output") or ""),
|
||||
))
|
||||
# A bundle exported before M2 may carry "scripts" and "scriptState".
|
||||
# Campaign scripting is gone, so both are ignored rather than rejected: the
|
||||
# story, its tree, its cards and its memories still import intact, which is
|
||||
# what the bundle is for.
|
||||
|
||||
write(db, adventure, story)
|
||||
db.flush()
|
||||
|
||||
@@ -1,178 +0,0 @@
|
||||
"""Retention policy for throwaway guest accounts.
|
||||
|
||||
In multi-user mode every first visit creates a `users` row through
|
||||
`GET /api/auth/me`, so a public demo accumulates one account per visitor. Most
|
||||
of those visitors never return, and each one leaves behind whatever scenarios,
|
||||
adventures, actions, and memories they generated. This module deletes guests
|
||||
that have been inactive for `AIDND_GUEST_RETENTION_DAYS`, which defaults to 5,
|
||||
along with everything they made.
|
||||
|
||||
Why this is safe to run unattended:
|
||||
|
||||
- Only rows with `is_guest` AND `email IS NULL` are ever touched, and both
|
||||
clauses are checked rather than either alone. Registering upgrades the row
|
||||
in place (is_guest -> False), so a guest who signs up keeps everything;
|
||||
local mode's implicit single user is also is_guest=False.
|
||||
- Idle time is `COALESCE(last_seen_at, created_at)`. `auth._touch` writes
|
||||
`last_seen_at` at most once an hour, and a guest created by `/auth/me` has
|
||||
NULL there until its second request, so `created_at` is the correct floor for
|
||||
a new visitor. Without the coalesce, those rows look arbitrarily old.
|
||||
- Nothing a guest owns is reachable by anyone else. `is_public` is an
|
||||
output-only field, as `schemas.ScenarioBase` shows, so the only shared
|
||||
scenarios are the seeded ones, which have a NULL `user_id` and are outside
|
||||
this filter. Deleting a guest cannot remove content from another user.
|
||||
|
||||
The sweep uses one Core DELETE rather than an ORM cascade. `db.delete(user)`
|
||||
would SELECT every adventure, action, memory, and story card into Python only to
|
||||
delete them, which on Neon is the egress pattern that has already cost this
|
||||
project once. Every foreign key from `users` downward is ON DELETE CASCADE, from
|
||||
users to scenarios, adventures, scripts, and settings, and from those to actions,
|
||||
memories, and cards, so the database deletes the whole graph in one statement and
|
||||
returns a row count.
|
||||
|
||||
The scan gets no index. The sweep runs a few times a day against a table holding
|
||||
at most a few thousand rows, which does not justify a migration and the schema
|
||||
surface it adds.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete, func
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from . import analytics, auth, models
|
||||
from .database import SessionLocal
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _int_env(name: str, default: int) -> int:
|
||||
try:
|
||||
return int(os.environ.get(name, "").strip() or default)
|
||||
except ValueError:
|
||||
logger.warning("%s is not an integer; using %d.", name, default)
|
||||
return default
|
||||
|
||||
|
||||
# Days of inactivity before a guest account is deleted. A value of 0 or less
|
||||
# disables the policy, for a deployment that keeps everything.
|
||||
RETENTION_DAYS = _int_env("AIDND_GUEST_RETENTION_DAYS", 5)
|
||||
|
||||
# How often a long-lived process re-checks. Hours, not minutes: nothing here is
|
||||
# time-critical, and on Render's free tier the service sleeps and cold-starts
|
||||
# often enough that the startup sweep does most of the work by itself.
|
||||
SWEEP_INTERVAL_SECONDS = _int_env("AIDND_CLEANUP_INTERVAL_HOURS", 6) * 3600
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
"""Guests only exist in multi-user mode, so local runs skip the sweep
|
||||
rather than pointing a DELETE at a database that has nothing to collect."""
|
||||
return auth.MULTI_USER and RETENTION_DAYS > 0
|
||||
|
||||
|
||||
def anything_to_sweep() -> bool:
|
||||
"""Whether the periodic task is worth starting at all. The two jobs it runs
|
||||
are independent: a deployment can keep every guest forever and still want
|
||||
its analytics rows aged out, and vice versa."""
|
||||
return enabled() or analytics.RETENTION_DAYS > 0
|
||||
|
||||
|
||||
def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int:
|
||||
"""Delete guests idle for RETENTION_DAYS or more. Returns the row count.
|
||||
|
||||
The caller owns error handling; `sweep` is the safe wrapper.
|
||||
"""
|
||||
if RETENTION_DAYS <= 0:
|
||||
return 0
|
||||
# Stored timestamps are UTC without a timezone on both backends. SQLite
|
||||
# drops the timezone, and the Postgres columns are TIMESTAMP WITHOUT TIME
|
||||
# ZONE with the session pinned to UTC in `database.py`. Match that, so the
|
||||
# comparison does not depend on how a dialect renders a value that carries a
|
||||
# timezone.
|
||||
reference = now or models.utcnow()
|
||||
cutoff = reference.replace(tzinfo=None) - timedelta(days=RETENTION_DAYS)
|
||||
|
||||
stmt = (
|
||||
delete(models.User)
|
||||
.where(
|
||||
models.User.is_guest.is_(True),
|
||||
models.User.email.is_(None),
|
||||
func.coalesce(models.User.last_seen_at, models.User.created_at) < cutoff,
|
||||
)
|
||||
# Without this option, the "auto" strategy cannot evaluate coalesce in
|
||||
# Python and falls back to fetching every matching primary key first.
|
||||
# That is a second round trip for no benefit, because this session holds
|
||||
# no User objects to synchronize.
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
removed = db.execute(stmt).rowcount or 0
|
||||
db.commit()
|
||||
return removed
|
||||
|
||||
|
||||
def sweep() -> int:
|
||||
"""One pass, with its own session. Never raises: a failed cleanup must not
|
||||
be able to take the app down (same rule as seeding). Returns the guest
|
||||
count, which is the number worth logging about."""
|
||||
if not anything_to_sweep():
|
||||
return 0
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Ages out the per-visitor analytics rows, on its own terms: it is not
|
||||
# about guests, and it must still happen on a deployment that has
|
||||
# chosen to keep every account it ever minted.
|
||||
aged = analytics.purge_old_visitor_days(db)
|
||||
if aged:
|
||||
logger.info("Aged out %d analytics visitor-day row(s).", aged)
|
||||
removed = delete_stale_guests(db) if enabled() else 0
|
||||
if removed:
|
||||
logger.info(
|
||||
"Cleaned up %d guest account(s) idle for %d+ days.",
|
||||
removed,
|
||||
RETENTION_DAYS,
|
||||
)
|
||||
return removed
|
||||
except Exception:
|
||||
db.rollback()
|
||||
logger.exception("Guest cleanup failed; continuing without it.")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
async def _sweep_loop() -> None:
|
||||
while True:
|
||||
# Blocking DB work: keep it off the event loop, which is also serving
|
||||
# SSE turn streams.
|
||||
await run_in_threadpool(sweep)
|
||||
await asyncio.sleep(SWEEP_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
def start_sweeper() -> asyncio.Task | None:
|
||||
"""Kick off the periodic sweep; None when there is nothing to sweep."""
|
||||
if not enabled():
|
||||
logger.info("Guest cleanup disabled (multi_user=%s, retention_days=%d).",
|
||||
auth.MULTI_USER, RETENTION_DAYS)
|
||||
else:
|
||||
logger.info(
|
||||
"Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).",
|
||||
RETENTION_DAYS,
|
||||
SWEEP_INTERVAL_SECONDS // 3600,
|
||||
)
|
||||
if not anything_to_sweep():
|
||||
return None
|
||||
return asyncio.create_task(_sweep_loop())
|
||||
|
||||
|
||||
async def stop_sweeper(task: asyncio.Task | None) -> None:
|
||||
if task is None:
|
||||
return
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
+22
-43
@@ -4,10 +4,17 @@ from pathlib import Path
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import DeclarativeBase, sessionmaker
|
||||
|
||||
# AIDND_DB_PATH lets deployments (Docker volume, hosted disk) relocate the
|
||||
# SQLite database; default stays backend/data.db for local runs. The parent
|
||||
# directory also hosts the auto-generated secret.key (see security.py), so
|
||||
# DB_PATH stays defined even when Postgres is in use.
|
||||
# The one database. It is SQLite, on this machine, in a file.
|
||||
#
|
||||
# Upstream could also point at a server database — `AIDND_DATABASE_URL` or the
|
||||
# platform-conventional `DATABASE_URL`, normalised onto psycopg3, with
|
||||
# pre-ping for a serverless Postgres that suspends when idle. That existed for
|
||||
# a hosted deployment. M2 removed it along with the deployment: a local
|
||||
# single-user storyteller has one reader, and a network database would be one
|
||||
# more thing that has to be running, and one more place the story lives.
|
||||
#
|
||||
# `AIDND_DB_PATH` stays. It is how the Docker image puts the database on a
|
||||
# volume, and how a test points at a throwaway file.
|
||||
_env_db_path = os.environ.get("AIDND_DB_PATH")
|
||||
DB_PATH = (
|
||||
Path(_env_db_path).resolve()
|
||||
@@ -15,48 +22,20 @@ DB_PATH = (
|
||||
else Path(__file__).resolve().parent.parent / "data.db"
|
||||
)
|
||||
|
||||
# `AIDND_DATABASE_URL`, or the conventional `DATABASE_URL`, switches the app to
|
||||
# a server database. Any SQLAlchemy URL works, and hosted deploys use Postgres,
|
||||
# which Phase 9 settled on Neon for. If neither variable is set, the app uses
|
||||
# SQLite.
|
||||
DATABASE_URL = (
|
||||
os.environ.get("AIDND_DATABASE_URL", "").strip()
|
||||
or os.environ.get("DATABASE_URL", "").strip()
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
engine = create_engine(
|
||||
f"sqlite:///{DB_PATH}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
|
||||
|
||||
def _normalize_url(url: str) -> str:
|
||||
"""Map the postgres:// / postgresql:// schemes hosts hand out to the
|
||||
psycopg3 driver installed in requirements.txt."""
|
||||
for prefix in ("postgres://", "postgresql://"):
|
||||
if url.startswith(prefix):
|
||||
return "postgresql+psycopg://" + url[len(prefix):]
|
||||
return url
|
||||
|
||||
|
||||
if DATABASE_URL:
|
||||
engine = create_engine(
|
||||
_normalize_url(DATABASE_URL),
|
||||
# Serverless Postgres (Neon) suspends idle databases; pre-ping
|
||||
# replaces silently-dead pooled connections instead of erroring.
|
||||
pool_pre_ping=True,
|
||||
# Store/read naive UTC like SQLite does, regardless of server default.
|
||||
connect_args={"options": "-c timezone=UTC"},
|
||||
)
|
||||
else:
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
engine = create_engine(
|
||||
f"sqlite:///{DB_PATH}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
|
||||
@event.listens_for(engine, "connect")
|
||||
def _enable_sqlite_foreign_keys(dbapi_connection, _record):
|
||||
# SQLite ships with foreign keys OFF per connection; without this every
|
||||
# ondelete=CASCADE/SET NULL in models.py is silently ignored.
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
@event.listens_for(engine, "connect")
|
||||
def _enable_sqlite_foreign_keys(dbapi_connection, _record):
|
||||
# SQLite ships with foreign keys OFF per connection; without this every
|
||||
# ondelete=CASCADE/SET NULL in models.py is silently ignored.
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
|
||||
|
||||
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Which inference endpoints this product is willing to talk to.
|
||||
|
||||
The Adventure Storyteller sends the player's prose, the assembled context, the
|
||||
retrieved memories, and the embedding inputs to whatever address the model
|
||||
endpoint names. That makes the endpoint the single most consequential setting
|
||||
in the application: point it somewhere else and the whole campaign goes there.
|
||||
|
||||
The v1 rule (`planning/DECISIONS/002-ollama-only-v1.md`, ADR 004) is that
|
||||
inference runs on user-controlled local infrastructure. Two deployments are
|
||||
supported and no third is:
|
||||
|
||||
* **same-host** — Ollama on loopback, the default;
|
||||
* **explicitly configured trusted LAN** — Ollama on another machine the user
|
||||
controls, named by them, reached over HTTP or over HTTPS with a certificate
|
||||
their machine trusts.
|
||||
|
||||
Everything on the public Internet is refused. Not discouraged in the UI, not
|
||||
absent from a dropdown — refused, here, on the way out, so that a hand-edited
|
||||
database row or a hostname that starts resolving somewhere new cannot quietly
|
||||
turn a local install into an exfiltration path.
|
||||
|
||||
## How the line is drawn
|
||||
|
||||
By **address**, not by name, and against an explicit allowlist of networks:
|
||||
loopback, the three RFC1918 ranges, link-local, IPv6 unique-local, and
|
||||
carrier-grade NAT — the last of which is what a mesh VPN such as Tailscale
|
||||
hands out and is as user-controlled as a LAN.
|
||||
|
||||
Every address the host resolves to must be in one of them. One address outside
|
||||
is enough to refuse the endpoint, so a name resolving to both a private and a
|
||||
public address does not squeak through.
|
||||
|
||||
The networks are spelled out rather than inferred from `ipaddress`'s own
|
||||
classifications, which do not mean what this rule needs: `is_private` is true
|
||||
of the documentation ranges and of `0.0.0.0/8`, and `is_reserved` is true of
|
||||
IPv6 loopback — so a rule written around it refuses `http://[::1]:11434/v1`,
|
||||
which is an ordinary same-host Ollama. Naming the networks keeps the policy
|
||||
readable and makes anything unnamed refused by default.
|
||||
|
||||
Checking addresses rather than hostnames is what makes the rule hard to talk
|
||||
around. A cloud provider cannot be reached by spelling its name differently,
|
||||
and `localhost.` or a DNS entry pointing at a public host is judged on where it
|
||||
actually goes.
|
||||
|
||||
## What this is not
|
||||
|
||||
It is not a general network-policy framework, and there is nothing to configure.
|
||||
There is one predicate, and it is applied in two places: when the endpoint is
|
||||
saved, so the user gets a clear error immediately, and again before every
|
||||
outbound request, because a name that resolved to `192.168.1.50` this morning
|
||||
can resolve to something else this afternoon.
|
||||
|
||||
TLS is a separate matter and is never traded against this one. See
|
||||
`tlstrust.py`: certificates are verified in full, and no endpoint — however
|
||||
private its address — may skip that.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
#: The networks an inference endpoint may live on. Anything else is refused.
|
||||
ALLOWED_NETWORKS = tuple(
|
||||
ipaddress.ip_network(cidr)
|
||||
for cidr in (
|
||||
"127.0.0.0/8", # this machine
|
||||
"10.0.0.0/8", # RFC1918
|
||||
"172.16.0.0/12", # RFC1918
|
||||
"192.168.0.0/16", # RFC1918
|
||||
"169.254.0.0/16", # link-local
|
||||
"100.64.0.0/10", # carrier-grade NAT, which mesh VPNs use
|
||||
"::1/128", # this machine, v6
|
||||
"fc00::/7", # unique-local, v6
|
||||
"fe80::/10", # link-local, v6
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _is_local(ip) -> bool:
|
||||
return any(ip in net for net in ALLOWED_NETWORKS)
|
||||
|
||||
#: Hosts that are only ever a cloud inference service. The address rule below
|
||||
#: already refuses every one of them, because they all resolve to public
|
||||
#: addresses; this list exists solely so the error says *why* rather than
|
||||
#: leaving the user to wonder whether their DNS is broken.
|
||||
CLOUD_HOSTS = (
|
||||
"openrouter.ai",
|
||||
"api.openai.com",
|
||||
"api.anthropic.com",
|
||||
"api.groq.com",
|
||||
"api.mistral.ai",
|
||||
"api.together.xyz",
|
||||
"api.deepseek.com",
|
||||
"generativelanguage.googleapis.com",
|
||||
"api.cohere.ai",
|
||||
"api.perplexity.ai",
|
||||
)
|
||||
|
||||
_CLOUD_REASON = (
|
||||
"this build talks to Ollama on your own machine or on your own network, "
|
||||
"and has no cloud provider support"
|
||||
)
|
||||
|
||||
|
||||
def _cloud_host(host: str) -> bool:
|
||||
host = host.lower().rstrip(".")
|
||||
return any(host == h or host.endswith("." + h) for h in CLOUD_HOSTS)
|
||||
|
||||
|
||||
def rejection_reason(url: str) -> str | None:
|
||||
"""Why this endpoint may not be used, or None if it may.
|
||||
|
||||
The string is shown to the user, so it says what to do rather than what
|
||||
went wrong internally.
|
||||
"""
|
||||
parsed = urlparse((url or "").strip())
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return "the endpoint URL must start with http:// or https://"
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return "the endpoint URL has no host"
|
||||
if _cloud_host(host):
|
||||
return f"{host} is a cloud inference service — {_CLOUD_REASON}"
|
||||
|
||||
port = parsed.port or (443 if parsed.scheme == "https" else 80)
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return (
|
||||
f"the host {host!r} could not be resolved — check the address, and "
|
||||
"that the machine running Ollama is reachable from here"
|
||||
)
|
||||
|
||||
for info in infos:
|
||||
try:
|
||||
ip = ipaddress.ip_address(info[4][0])
|
||||
except ValueError:
|
||||
return "the endpoint host resolved to an address that could not be read"
|
||||
if _is_local(ip):
|
||||
continue
|
||||
if ip.is_global:
|
||||
return (
|
||||
f"{host} resolves to {ip}, which is a public Internet address — "
|
||||
f"{_CLOUD_REASON}. Use Ollama on this machine "
|
||||
"(http://127.0.0.1:11434/v1) or on a machine on your own network"
|
||||
)
|
||||
return (
|
||||
f"{host} resolves to {ip}, which is not on this machine and not on "
|
||||
"your own network. Use http://127.0.0.1:11434/v1, or the address of "
|
||||
"a machine on your network"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def check(url: str) -> None:
|
||||
"""Raises `EndpointRejected` if this endpoint is outside the policy."""
|
||||
reason = rejection_reason(url)
|
||||
if reason is not None:
|
||||
raise EndpointRejected(reason)
|
||||
|
||||
|
||||
class EndpointRejected(Exception):
|
||||
"""The configured endpoint is not one this product will send a story to."""
|
||||
|
||||
|
||||
def is_loopback(url: str) -> bool:
|
||||
"""Whether this endpoint is on this machine. Used for reporting, not for
|
||||
gating: a trusted-LAN endpoint is equally allowed."""
|
||||
host = urlparse((url or "").strip()).hostname
|
||||
if not host:
|
||||
return False
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, None, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return False
|
||||
try:
|
||||
return all(ipaddress.ip_address(i[4][0]).is_loopback for i in infos)
|
||||
except ValueError:
|
||||
return False
|
||||
+30
-194
@@ -1,183 +1,35 @@
|
||||
"""Phase 9: abuse guards for hosted, multi-user deployments.
|
||||
"""Resource bounds on what a single request or a single story may cost.
|
||||
|
||||
Rate limits and row caps do nothing in local mode, because a single local player
|
||||
should never be throttled by their own app. The values are hardcoded on purpose.
|
||||
They are generous enough that a legitimate player never notices them, and tight
|
||||
enough that a hostile visitor cannot exhaust the demo key, saturate the CPU, or
|
||||
fill the database.
|
||||
Upstream carried three things here, and only one of them belongs in a local
|
||||
single-user product. Per-IP and per-user **rate limiting**, the login-attempt
|
||||
throttle, and the per-user **quotas** were hosted-service policy: they existed
|
||||
to stop a hostile visitor exhausting a shared demo key or filling a shared
|
||||
database. M2 removed all of it. There are no visitors, and throttling the one
|
||||
person who started the application would be a bug rather than a guard.
|
||||
|
||||
What is left is defensive programming, and it applies whatever the deployment:
|
||||
|
||||
* a ceiling on the **request body**, so a malformed or hostile payload cannot
|
||||
be read into memory before anything looks at it;
|
||||
* ceilings on how large **one adventure** may grow, in actions, memories,
|
||||
story cards and branches. These bound storage and the cost of the queries
|
||||
that walk them. They are per-story, not per-user: nothing here counts how
|
||||
many campaigns a person may have.
|
||||
|
||||
An import is checked against the same per-adventure ceilings that live creation
|
||||
uses, so a bundle cannot carry a story past a limit that play could not reach.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from collections import defaultdict, deque
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import auth, models
|
||||
from . import models
|
||||
|
||||
# ---------- Rate limiting ----------
|
||||
# Fixed windows per scope and caller. The windows live in memory, which is
|
||||
# enough for the single-process deployment this app targets. The worst case
|
||||
# after a restart is a brief extra allowance.
|
||||
# ---------- Per-story row caps ----------
|
||||
|
||||
# Maps a scope to (max requests, window seconds).
|
||||
RATE_LIMITS: dict[str, tuple[int, int]] = {
|
||||
"turn": (10, 60), # AI turn generation. The demo key also has a daily cap.
|
||||
"chat": (30, 60), # The AI Chat scratchpad, for power users.
|
||||
"script-test": (30, 60), # Sandboxed, but each run costs up to 2s of CPU.
|
||||
"connection-test": (10, 60), # Outbound HTTP to a user-supplied URL.
|
||||
"import": (30, 60), # Large writes.
|
||||
"auth": (10, 300), # Register and login attempts, per IP.
|
||||
"guest": (30, 300), # New guest users, per IP. Each one is a database row.
|
||||
# Pageview beacons. The limit is generous, because a real reader clicking
|
||||
# around a SPA sends a handful a minute, and it is low enough that nobody
|
||||
# can inflate the traffic numbers faster than by reloading the page.
|
||||
"analytics": (120, 60),
|
||||
}
|
||||
|
||||
_windows: dict[tuple[str, str], deque] = defaultdict(deque)
|
||||
_windows_guard = threading.Lock()
|
||||
|
||||
|
||||
# How many proxy hops sit between the app and the real client. On Render, and on
|
||||
# most platforms, that is one, because the platform's edge appends the connecting
|
||||
# IP to the right of `X-Forwarded-For`. A client can prepend any value on the
|
||||
# left, but it cannot push a value past the edge's own append, so the trustworthy
|
||||
# client IP is the entry that many places from the right rather than uvicorn's
|
||||
# leftmost choice. Trusting the leftmost entry let anyone rotate
|
||||
# `X-Forwarded-For` to get a fresh rate-limit bucket per request and bypass the
|
||||
# auth and guest limits. If the deployment adds more hops, set
|
||||
# `AIDND_TRUSTED_PROXY_HOPS`.
|
||||
TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1))
|
||||
|
||||
|
||||
def client_ip(request: Request) -> str:
|
||||
"""Returns the real client IP, resisting a spoofed `X-Forwarded-For`.
|
||||
|
||||
The function reads the hop the trusted edge appended, which is the rightmost
|
||||
entry minus any extra trusted hops. If no forwarded header is present, which
|
||||
happens locally, in development, and on a direct connection, it falls back to
|
||||
the socket peer.
|
||||
|
||||
The function is public because the access log needs the same answer. Two
|
||||
functions that each decide which address belongs to the caller is how one of
|
||||
them ends up trusting a header it should not.
|
||||
"""
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
if forwarded:
|
||||
parts = [p.strip() for p in forwarded.split(",") if p.strip()]
|
||||
if parts:
|
||||
return parts[-min(TRUSTED_PROXY_HOPS, len(parts))]
|
||||
return request.client.host if request.client else "unknown"
|
||||
|
||||
|
||||
def rate_limit(scope: str, request: Request, user: models.User | None = None) -> None:
|
||||
"""Raises a 429 when the caller exceeds the scope's window.
|
||||
|
||||
The window is keyed per user when a user is known, because an account
|
||||
survives an IP change, and per IP otherwise.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
limit, window_seconds = RATE_LIMITS[scope]
|
||||
key = (scope, f"u{user.id}" if user else f"ip{client_ip(request)}")
|
||||
now = time.time()
|
||||
with _windows_guard:
|
||||
window = _windows[key]
|
||||
while window and window[0] < now - window_seconds:
|
||||
window.popleft()
|
||||
if len(window) >= limit:
|
||||
raise HTTPException(
|
||||
429, "You're doing that too fast — wait a minute and try again."
|
||||
)
|
||||
window.append(now)
|
||||
if len(_windows) > 10_000:
|
||||
_prune(now)
|
||||
|
||||
|
||||
# ---------- Per-account login throttle ----------
|
||||
# This is defense in depth next to the per-IP `auth` limit. A botnet dilutes
|
||||
# that limit, because many real source IPs each get their own bucket, so it
|
||||
# cannot by itself stop a distributed guessing run against one account. This cap
|
||||
# keys on the target email rather than on the caller, so guessing one account's
|
||||
# password stays expensive however many addresses the guesses come from.
|
||||
#
|
||||
# Only failures count, and a correct password clears the record. The window
|
||||
# slides over a short period rather than locking the account, so a user who
|
||||
# mistypes a few times recovers within minutes. The trade-off is that an
|
||||
# attacker can keep a known account throttled, which is an inconvenience and is
|
||||
# preferable to letting the account be brute-forced.
|
||||
LOGIN_FAIL_LIMIT = 8 # Failed attempts per account.
|
||||
LOGIN_FAIL_WINDOW = 900 # The window in seconds, which is 15 minutes.
|
||||
|
||||
_login_fails: dict[str, deque] = defaultdict(deque)
|
||||
_login_guard = threading.Lock()
|
||||
|
||||
|
||||
def check_login_allowed(email: str) -> None:
|
||||
"""Raises a 429 when an account has too many recent failed logins.
|
||||
|
||||
Call this before verifying the password, so that a guess never reaches the
|
||||
hash.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
now = time.time()
|
||||
with _login_guard:
|
||||
window = _login_fails[email]
|
||||
while window and window[0] < now - LOGIN_FAIL_WINDOW:
|
||||
window.popleft()
|
||||
if len(window) >= LOGIN_FAIL_LIMIT:
|
||||
raise HTTPException(
|
||||
429,
|
||||
"Too many failed sign-in attempts for this account — "
|
||||
"wait a few minutes and try again.",
|
||||
)
|
||||
|
||||
|
||||
def note_login_failure(email: str) -> None:
|
||||
"""Records one failed attempt against `email`."""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
now = time.time()
|
||||
with _login_guard:
|
||||
_login_fails[email].append(now)
|
||||
if len(_login_fails) > 10_000: # Bound the map against a flood of unique emails.
|
||||
stale = [
|
||||
key for key, window in _login_fails.items()
|
||||
if not window or window[-1] < now - LOGIN_FAIL_WINDOW
|
||||
]
|
||||
for key in stale:
|
||||
del _login_fails[key]
|
||||
|
||||
|
||||
def note_login_success(email: str) -> None:
|
||||
"""Clears the account's failure record after a correct password."""
|
||||
with _login_guard:
|
||||
_login_fails.pop(email, None)
|
||||
|
||||
|
||||
def _prune(now: float) -> None:
|
||||
"""Drops callers whose whole window has expired, so the per-IP dict stays bounded.
|
||||
|
||||
Call this with the guard held.
|
||||
"""
|
||||
longest = max(seconds for _, seconds in RATE_LIMITS.values())
|
||||
stale = [key for key, window in _windows.items()
|
||||
if not window or window[-1] < now - longest]
|
||||
for key in stale:
|
||||
del _windows[key]
|
||||
|
||||
|
||||
# ---------- Per-user row caps ----------
|
||||
|
||||
MAX_ADVENTURES_PER_USER = 100
|
||||
MAX_SCENARIOS_PER_USER = 200
|
||||
MAX_SCRIPTS_PER_USER = 200
|
||||
MAX_STORY_CARDS_PER_OWNER = 200 # Per scenario or per adventure.
|
||||
MAX_MEMORIES_PER_ADVENTURE = 1000
|
||||
MAX_ACTIONS_PER_ADVENTURE = 5000
|
||||
@@ -200,28 +52,14 @@ def check_row_cap(
|
||||
) -> None:
|
||||
"""Raises a 409 when creating one more row of `kind` would exceed its cap.
|
||||
|
||||
The caller has already checked ownership of the scenario or adventure passed
|
||||
in.
|
||||
Only per-story kinds are capped. `adventures` and `scenarios` were per-user
|
||||
quotas and are no longer checked; the callers still pass them, and they are
|
||||
accepted and ignored so that adding a cap back is a change here rather than
|
||||
at every call site.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
if kind in ("adventures", "scenarios"):
|
||||
return
|
||||
if kind == "adventures":
|
||||
count = _count(db, models.Adventure, models.Adventure.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_ADVENTURES_PER_USER, "adventures",
|
||||
"delete one you no longer play to make room",
|
||||
)
|
||||
elif kind == "scenarios":
|
||||
count = _count(db, models.Scenario, models.Scenario.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_SCENARIOS_PER_USER, "scenarios", "delete one to make room"
|
||||
)
|
||||
elif kind == "scripts":
|
||||
count = _count(db, models.Script, models.Script.user_id == user.id)
|
||||
cap, subject, hint = (
|
||||
MAX_SCRIPTS_PER_USER, "scripts", "delete one to make room"
|
||||
)
|
||||
elif kind == "story_cards":
|
||||
if kind == "story_cards":
|
||||
owner_filter = (
|
||||
models.StoryCard.scenario_id == scenario_id
|
||||
if scenario_id is not None
|
||||
@@ -273,8 +111,6 @@ def check_bundle_lists(**lists) -> None:
|
||||
The keyword arguments are `story_cards`, `memories`, `actions`, and
|
||||
`branches`.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return
|
||||
for name, value in lists.items():
|
||||
cap = _BUNDLE_LIST_CAPS[name]
|
||||
if isinstance(value, list) and len(value) > cap:
|
||||
@@ -286,8 +122,8 @@ def check_bundle_lists(**lists) -> None:
|
||||
|
||||
# ---------- Request body size ----------
|
||||
# The limit is generous enough for the largest legitimate payload, which is an
|
||||
# adventure export holding thousands of actions. It applies in every mode, and no
|
||||
# honest request approaches it.
|
||||
# adventure export holding thousands of actions. No honest request approaches
|
||||
# it.
|
||||
|
||||
MAX_BODY_BYTES = 2 * 1024 * 1024
|
||||
MAX_IMPORT_BODY_BYTES = 20 * 1024 * 1024
|
||||
|
||||
+31
-68
@@ -1,6 +1,5 @@
|
||||
import mimetypes
|
||||
import os
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI
|
||||
@@ -8,15 +7,10 @@ from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
from . import analytics, cleanup
|
||||
from .auth import MULTI_USER
|
||||
from .database import engine
|
||||
from .limits import BodySizeLimitMiddleware
|
||||
from .migrations import bootstrap
|
||||
from .routers import (
|
||||
adventures, analytics as analytics_router, auth, chat, debug, scenarios, scripts,
|
||||
settings, story_cards,
|
||||
)
|
||||
from .routers import adventures, chat, debug, scenarios, settings, story_cards
|
||||
from .seed import seed_public_scenarios
|
||||
|
||||
bootstrap(engine)
|
||||
@@ -24,37 +18,32 @@ seed_public_scenarios(engine)
|
||||
|
||||
# Production serves the SPA same-origin, so CORS only matters for the Vite dev
|
||||
# server; AIDND_CORS_ORIGINS overrides for any other cross-origin setup.
|
||||
#
|
||||
# A wildcard is refused rather than honoured. This API is unauthenticated by
|
||||
# design and bound to loopback, so its only protection from a page the user
|
||||
# happens to have open in another tab is the same-origin policy. `*` would hand
|
||||
# every site on the Internet a write handle on the local campaign database. If the
|
||||
# value is wrong the app refuses to start, because a permissive CORS policy that
|
||||
# nobody notices is worse than one that fails loudly.
|
||||
CORS_ORIGINS = [
|
||||
o.strip()
|
||||
for o in os.environ.get("AIDND_CORS_ORIGINS", "").split(",")
|
||||
if o.strip()
|
||||
] or ["http://localhost:5173", "http://127.0.0.1:5173"]
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
# Sweeps once on boot, then on an interval. Booting is the reliable
|
||||
# trigger on Render's free tier, where the service sleeps after ~15
|
||||
# minutes and a long-running timer rarely gets to fire.
|
||||
sweeper = cleanup.start_sweeper()
|
||||
# Visit counters are buffered in memory and written in batches; this is
|
||||
# what turns them into rows, and stop_flusher writes out the last batch so
|
||||
# a deploy doesn't drop it.
|
||||
flusher = analytics.start_flusher()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await cleanup.stop_sweeper(sweeper)
|
||||
await analytics.stop_flusher(flusher)
|
||||
if any(o == "*" or o.strip() == "*" for o in CORS_ORIGINS):
|
||||
raise RuntimeError(
|
||||
"AIDND_CORS_ORIGINS must not contain '*'. The storyteller API is "
|
||||
"unauthenticated and loopback-bound; a wildcard origin would let any "
|
||||
"web page read and rewrite every campaign. List the exact origins "
|
||||
"instead."
|
||||
)
|
||||
|
||||
|
||||
# The interactive API docs stay local-only: in multi-user mode they just hand
|
||||
# strangers a map of the API surface.
|
||||
app = FastAPI(
|
||||
title="AI D&D",
|
||||
docs_url=None if MULTI_USER else "/docs",
|
||||
title="Adventure Storyteller",
|
||||
docs_url="/docs",
|
||||
redoc_url=None,
|
||||
openapi_url=None if MULTI_USER else "/openapi.json",
|
||||
lifespan=lifespan,
|
||||
openapi_url="/openapi.json",
|
||||
)
|
||||
|
||||
app.add_middleware(
|
||||
@@ -117,50 +106,14 @@ class SecurityHeadersMiddleware:
|
||||
await self.app(scope, receive, send_with_headers)
|
||||
|
||||
|
||||
class ApiErrorMiddleware:
|
||||
"""Counts failed API responses for the analytics dashboard.
|
||||
|
||||
This is middleware rather than an exception handler, because it observes
|
||||
what the client received. A 429 from a rate limiter, a 404 from routing, and
|
||||
a 500 from a handler that never returned all reach it the same way. It is
|
||||
pure ASGI for the same reason as the headers above: an SSE turn must not be
|
||||
buffered on its way out. It watches `/api` only, because a 404 on the SPA
|
||||
mount is a page load rather than a fault.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
self.app = app
|
||||
|
||||
async def __call__(self, scope, receive, send):
|
||||
if scope["type"] != "http" or not scope.get("path", "").startswith("/api"):
|
||||
return await self.app(scope, receive, send)
|
||||
|
||||
async def send_counting(message):
|
||||
if message["type"] == "http.response.start" and message["status"] >= 400:
|
||||
# The router has already put the matched route on the scope by
|
||||
# the time a response starts, so the label can name the
|
||||
# endpoint rather than the caller's path.
|
||||
analytics.record(
|
||||
analytics.M_ERROR,
|
||||
analytics.api_route_label(scope, message["status"]),
|
||||
)
|
||||
await send(message)
|
||||
|
||||
await self.app(scope, receive, send_counting)
|
||||
|
||||
|
||||
app.add_middleware(ApiErrorMiddleware)
|
||||
app.add_middleware(SecurityHeadersMiddleware)
|
||||
|
||||
app.include_router(auth.router)
|
||||
app.include_router(scenarios.router)
|
||||
app.include_router(adventures.router)
|
||||
app.include_router(story_cards.router)
|
||||
app.include_router(scripts.router)
|
||||
app.include_router(settings.router)
|
||||
app.include_router(chat.router)
|
||||
app.include_router(debug.router)
|
||||
app.include_router(analytics_router.router)
|
||||
|
||||
|
||||
@app.get("/api/health")
|
||||
@@ -171,7 +124,8 @@ def health():
|
||||
# In production, serve the built frontend (frontend/dist) as static files.
|
||||
class SPAStaticFiles(StaticFiles):
|
||||
"""Serve index.html for unknown paths so client-side routes (/play/3)
|
||||
survive a page reload. API routes are matched before this mount."""
|
||||
survive a page reload. API routes are matched before this mount, and an
|
||||
unmatched one 404s rather than falling through to the page."""
|
||||
|
||||
async def get_response(self, path, scope):
|
||||
try:
|
||||
@@ -179,11 +133,20 @@ class SPAStaticFiles(StaticFiles):
|
||||
except StarletteHTTPException as exc:
|
||||
if exc.status_code != 404:
|
||||
raise
|
||||
return await super().get_response("index.html", scope)
|
||||
return await self._fallback(path, scope)
|
||||
if response.status_code == 404:
|
||||
return await super().get_response("index.html", scope)
|
||||
return await self._fallback(path, scope)
|
||||
return response
|
||||
|
||||
async def _fallback(self, path, scope):
|
||||
# The mount is a catch-all, so an /api path no router claims — a typo,
|
||||
# or an endpoint this build removed — used to come back as the SPA's
|
||||
# HTML with status 200, and a client asking for JSON parsed a web page
|
||||
# instead of seeing that the route is not there.
|
||||
if path == "api" or path.startswith("api/"):
|
||||
raise StarletteHTTPException(status_code=404)
|
||||
return await super().get_response("index.html", scope)
|
||||
|
||||
|
||||
# Python's mimetypes table has no entry for woff2 on a slim Debian image, so
|
||||
# StaticFiles served the self-hosted fonts as application/octet-stream. Browsers
|
||||
|
||||
@@ -142,26 +142,22 @@ _running: set[int] = set()
|
||||
_tasks: set[asyncio.Task] = set()
|
||||
|
||||
|
||||
# Both factories below use the user's own key by construction. They read the
|
||||
# endpoint and key from `Settings` and never from `auth.DEMO_*`, so
|
||||
# summarization and embedding cannot spend the shared demo key. Their call sites
|
||||
# are also skipped when `using_demo` is true.
|
||||
#
|
||||
# Do not change these to accept a `ProviderConfig`. `summary_model` and
|
||||
# `embedding_model` are free-form user input and are not on the demo allowlist.
|
||||
# Both factories read the endpoint and the model names straight off `Settings`.
|
||||
# They used to also read an API key, which is gone: Ollama does not use one and
|
||||
# M2 removed cloud providers. `summary_model` and `embedding_model` fall back to
|
||||
# the narrator model when the user has not named a separate one.
|
||||
def summary_provider(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
return OpenAICompatibleProvider(
|
||||
settings.endpoint_url,
|
||||
settings.api_key_plain,
|
||||
settings.summary_model or settings.model,
|
||||
settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
|
||||
|
||||
def embedding_provider(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
return OpenAICompatibleProvider(
|
||||
settings.endpoint_url, settings.api_key_plain, settings.embedding_model
|
||||
settings.endpoint_url, settings.embedding_model
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -338,6 +338,10 @@ MIGRATIONS: list[tuple[int, str | dict[str, str]]] = [
|
||||
(74, "ALTER TABLE adventures ADD COLUMN persona_name VARCHAR(80) NOT NULL DEFAULT ''"),
|
||||
(75, "ALTER TABLE adventures ADD COLUMN persona_pronouns VARCHAR(40) NOT NULL DEFAULT ''"),
|
||||
(76, "ALTER TABLE adventures ADD COLUMN persona_desc TEXT NOT NULL DEFAULT ''"),
|
||||
# M2: how long to wait for the model. Upstream hardcoded 120s in the HTTP
|
||||
# client, which a cold model load on a CPU-only machine can exceed. The
|
||||
# default matches `providers.openai_compatible.DEFAULT_READ_TIMEOUT`.
|
||||
(77, "ALTER TABLE settings ADD COLUMN model_timeout_seconds INTEGER NOT NULL DEFAULT 300"),
|
||||
]
|
||||
|
||||
LATEST_VERSION = max((v for v, _ in MIGRATIONS), default=1)
|
||||
@@ -1106,23 +1110,3 @@ def bootstrap(engine: Engine, through: int = LATEST_VERSION) -> None:
|
||||
_backfill_parents(conn)
|
||||
current = version
|
||||
_set_version(conn, current)
|
||||
_encrypt_plaintext_api_keys(conn)
|
||||
|
||||
|
||||
def _encrypt_plaintext_api_keys(conn) -> None:
|
||||
"""Encrypts API keys saved before encryption at rest existed (Phase 8).
|
||||
|
||||
Those keys are stored in plain text, so this pass wraps them in Fernet. Plain
|
||||
SQL cannot do it. The pass runs on every start, and it matches no rows once
|
||||
every row carries the `enc:` prefix.
|
||||
"""
|
||||
from . import security # Deferred: security derives its key from DB_PATH setup.
|
||||
|
||||
rows = conn.execute(text(
|
||||
"SELECT id, api_key FROM settings WHERE api_key != '' AND api_key NOT LIKE 'enc:%'"
|
||||
)).all()
|
||||
for row_id, plain in rows:
|
||||
conn.execute(
|
||||
text("UPDATE settings SET api_key = :key WHERE id = :id"),
|
||||
{"key": security.encrypt_secret(plain), "id": row_id},
|
||||
)
|
||||
|
||||
+33
-183
@@ -1,8 +1,8 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
|
||||
String, Table, Text, UniqueConstraint, event,
|
||||
JSON, Boolean, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
|
||||
String, Text, event,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, Session, mapped_column, relationship
|
||||
|
||||
@@ -37,19 +37,12 @@ class User(Base):
|
||||
is_guest: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
last_seen_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
|
||||
# Shared demo key usage (resets when the UTC date changes).
|
||||
# Was the shared demo key's per-day tally. M2 removed the demo key; these
|
||||
# columns stay so existing databases open unchanged and are never written.
|
||||
demo_turns_used: Mapped[int] = mapped_column(Integer, default=0)
|
||||
demo_turns_date: Mapped[str] = mapped_column(String(10), default="")
|
||||
|
||||
|
||||
scenario_scripts = Table(
|
||||
"scenario_scripts",
|
||||
Base.metadata,
|
||||
Column("scenario_id", ForeignKey("scenarios.id", ondelete="CASCADE"), primary_key=True),
|
||||
Column("script_id", ForeignKey("scripts.id", ondelete="CASCADE"), primary_key=True),
|
||||
)
|
||||
|
||||
|
||||
class Scenario(Base):
|
||||
__tablename__ = "scenarios"
|
||||
|
||||
@@ -89,7 +82,6 @@ class Scenario(Base):
|
||||
back_populates="scenario", cascade="all, delete-orphan"
|
||||
)
|
||||
adventures: Mapped[list["Adventure"]] = relationship(back_populates="scenario")
|
||||
scripts: Mapped[list["Script"]] = relationship(secondary=scenario_scripts)
|
||||
|
||||
|
||||
class Adventure(Base):
|
||||
@@ -121,6 +113,9 @@ class Adventure(Base):
|
||||
persona_name: Mapped[str] = mapped_column(String(80), default="")
|
||||
persona_pronouns: Mapped[str] = mapped_column(String(40), default="")
|
||||
persona_desc: Mapped[str] = mapped_column(Text, default="")
|
||||
# Was the campaign scripting engine's shared `state` object. M2 removed
|
||||
# scripting; the column stays so existing databases open unchanged, and it
|
||||
# is never written with anything but an empty dict.
|
||||
script_state: Mapped[dict] = mapped_column(JSON, default=dict)
|
||||
# Phase 12: live RPG world state (world/player/npc stats + milestones),
|
||||
# instantiated from the scenario's stat_schema. Empty when there's no RPG layer.
|
||||
@@ -177,11 +172,6 @@ class Adventure(Base):
|
||||
cascade="all, delete-orphan",
|
||||
order_by="Action.id",
|
||||
)
|
||||
scripts: Mapped[list["AdventureScript"]] = relationship(
|
||||
back_populates="adventure",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="AdventureScript.position",
|
||||
)
|
||||
memories: Mapped[list["Memory"]] = relationship(
|
||||
back_populates="adventure",
|
||||
cascade="all, delete-orphan",
|
||||
@@ -419,9 +409,10 @@ class Action(Base):
|
||||
# and for re-attaching the emit block when replaying history to the model.
|
||||
# Mirrors the active variant, same as text/reasoning/context_snapshot.
|
||||
world_delta: Mapped[dict | None] = mapped_column(JSON, nullable=True)
|
||||
# Phase 14, SP4: the shared script state and the RPG world state as they
|
||||
# stood after this node was played. These columns record the node's outcome
|
||||
# rather than its starting position.
|
||||
# Phase 14, SP4: the RPG world state as it stood after this node was
|
||||
# played. These columns record the node's outcome rather than its starting
|
||||
# position. (`state_after` held the scripting engine's state, which M2
|
||||
# removed; it is now always written empty.)
|
||||
#
|
||||
# Two operations need this outcome, and neither can use a snapshot taken
|
||||
# before the turn. Switching between siblings must restore the state that
|
||||
@@ -508,52 +499,6 @@ class Action(Base):
|
||||
return out
|
||||
|
||||
|
||||
class Script(Base):
|
||||
__tablename__ = "scripts"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), nullable=True
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(200), default="Untitled Script")
|
||||
description: Mapped[str] = mapped_column(Text, default="")
|
||||
library_js: Mapped[str] = mapped_column(Text, default="")
|
||||
input_js: Mapped[str] = mapped_column(Text, default="")
|
||||
context_js: Mapped[str] = mapped_column(Text, default="")
|
||||
output_js: Mapped[str] = mapped_column(Text, default="")
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, onupdate=utcnow)
|
||||
|
||||
|
||||
class AdventureScript(Base):
|
||||
"""A script copied into an adventure at creation, so library edits don't
|
||||
change running adventures unless the player explicitly re-syncs it from
|
||||
`source_script_id`. `state` lives on Adventure.script_state (one shared
|
||||
state per adventure, as in AI Dungeon)."""
|
||||
|
||||
__tablename__ = "adventure_scripts"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
adventure_id: Mapped[int] = mapped_column(ForeignKey("adventures.id", ondelete="CASCADE"))
|
||||
# The library Script that this copy was made from, which lets the player
|
||||
# re-sync it on demand. The value is NULL for legacy copies that predate
|
||||
# this column, and for demo-derived copies whose source the player does not
|
||||
# own. Those copies fall back to matching by name, or cannot be synced.
|
||||
source_script_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("scripts.id", ondelete="SET NULL"), nullable=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
name: Mapped[str] = mapped_column(String(200), default="Untitled Script")
|
||||
description: Mapped[str] = mapped_column(Text, default="")
|
||||
library_js: Mapped[str] = mapped_column(Text, default="")
|
||||
input_js: Mapped[str] = mapped_column(Text, default="")
|
||||
context_js: Mapped[str] = mapped_column(Text, default="")
|
||||
output_js: Mapped[str] = mapped_column(Text, default="")
|
||||
|
||||
adventure: Mapped[Adventure] = relationship(back_populates="scripts")
|
||||
|
||||
|
||||
class Settings(Base):
|
||||
__tablename__ = "settings"
|
||||
|
||||
@@ -564,8 +509,10 @@ class Settings(Base):
|
||||
ForeignKey("users.id", ondelete="CASCADE"), nullable=True, unique=True
|
||||
)
|
||||
endpoint_url: Mapped[str] = mapped_column(String(500), default="http://localhost:11434/v1")
|
||||
# Encrypted at rest with Fernet, which produces a value that starts with
|
||||
# "enc:". See security.py. To read the key, use `api_key_plain`.
|
||||
# Was a cloud provider's API key, encrypted at rest. Ollama does not use
|
||||
# one and M2 removed cloud providers, so nothing reads or writes this now.
|
||||
# The column stays so existing databases open unchanged; an old value is
|
||||
# left where it is rather than migrated or decrypted.
|
||||
api_key: Mapped[str] = mapped_column(String(500), default="")
|
||||
model: Mapped[str] = mapped_column(String(200), default="")
|
||||
api_mode: Mapped[str] = mapped_column(String(20), default="chat") # chat|completion
|
||||
@@ -573,12 +520,17 @@ class Settings(Base):
|
||||
# 800 leaves room for a full scene; 400 tended to truncate mid-paragraph
|
||||
# and left reasoning models with nothing after their thinking.
|
||||
max_output_tokens: Mapped[int] = mapped_column(Integer, default=800)
|
||||
# Separate thinking budget for reasoning models (OpenRouter-style
|
||||
# `reasoning: {max_tokens}`); 0 = param not sent, -1 = reasoning explicitly
|
||||
# off (`reasoning: {effort: none}`). Added on top of
|
||||
# max_output_tokens so story output keeps its full budget.
|
||||
# Was an OpenRouter-style thinking budget. Ollama's OpenAI-compatible
|
||||
# endpoint ignores the field, so M2 stopped sending it and removed it from
|
||||
# the Settings API and UI. The column stays so existing databases open
|
||||
# unchanged and is never read.
|
||||
reasoning_max_tokens: Mapped[int] = mapped_column(Integer, default=0)
|
||||
context_token_budget: Mapped[int] = mapped_column(Integer, default=16384)
|
||||
# How long to wait for the model, in seconds, before giving up on a turn.
|
||||
# A cold load of a mid-sized model on a CPU-only machine can take minutes,
|
||||
# while the same turn takes seconds once the model is resident. See
|
||||
# `providers.openai_compatible.DEFAULT_READ_TIMEOUT`.
|
||||
model_timeout_seconds: Mapped[int] = mapped_column(Integer, default=300)
|
||||
narrator_prompt: Mapped[str] = mapped_column(
|
||||
Text,
|
||||
default=(
|
||||
@@ -599,120 +551,18 @@ class Settings(Base):
|
||||
memory_bank_capacity: Mapped[int] = mapped_column(Integer, default=80)
|
||||
memory_top_k: Mapped[int] = mapped_column(Integer, default=5)
|
||||
|
||||
@property
|
||||
def has_api_key(self) -> bool:
|
||||
return bool(self.api_key)
|
||||
|
||||
@property
|
||||
def api_key_plain(self) -> str:
|
||||
from . import security # local import: models is imported before security
|
||||
|
||||
return security.decrypt_secret(self.api_key)
|
||||
|
||||
|
||||
# ---------- Visit analytics (see analytics.py) ----------
|
||||
# Two intentionally simple tables. Neither can hold text that a player wrote,
|
||||
# and neither can be joined back to a `users` row, because the visitor column
|
||||
# holds an HMAC and has no foreign key. When guest cleanup deletes an account,
|
||||
# the history that account contributed remains intact and anonymous.
|
||||
|
||||
|
||||
class AnalyticsDaily(Base):
|
||||
"""One counter: how many times `label` happened within `metric` on `day`.
|
||||
|
||||
The table stores a generic triple of metric, label, and hits rather than one
|
||||
column per statistic. Measuring something new therefore costs a constant
|
||||
rather than a migration. The only writer is an UPSERT that runs from a
|
||||
buffer. See `analytics.flush`.
|
||||
"""
|
||||
|
||||
__tablename__ = "analytics_daily"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
day: Mapped[str] = mapped_column(String(10), index=True) # YYYY-MM-DD, UTC
|
||||
metric: Mapped[str] = mapped_column(String(32))
|
||||
label: Mapped[str] = mapped_column(String(80), default="")
|
||||
hits: Mapped[int] = mapped_column(Integer, default=0)
|
||||
|
||||
# The upsert target: one row per bucket per day, created or incremented.
|
||||
__table_args__ = (
|
||||
UniqueConstraint("day", "metric", "label", name="uq_analytics_daily_bucket"),
|
||||
)
|
||||
|
||||
|
||||
class AnalyticsVisitorDay(Base):
|
||||
"""One visitor, one day, and which funnel steps they reached on it.
|
||||
|
||||
This table exists so that the funnel counts people rather than clicks. A
|
||||
player who starts six adventures counts as one person who started an
|
||||
adventure. `is_new` is set when the visitor has no earlier row, which is why
|
||||
the visitor column also has an index of its own.
|
||||
"""
|
||||
|
||||
__tablename__ = "analytics_visitor_days"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
day: Mapped[str] = mapped_column(String(10))
|
||||
# HMAC of the user id under the app secret; not reversible, not a key.
|
||||
visitor: Mapped[str] = mapped_column(String(32))
|
||||
is_new: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
opened: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
created: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
played: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
signed_up: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint("day", "visitor", name="uq_analytics_visitor_day"),
|
||||
Index("ix_analytics_visitor", "visitor"),
|
||||
)
|
||||
|
||||
|
||||
class AccessEvent(Base):
|
||||
"""One sign-in, registration, failed attempt, or session first-seen.
|
||||
|
||||
This table is the counterpart to the two above, and it is kept separate from
|
||||
them on purpose. It identifies people by design, recording address, email,
|
||||
and device. Keeping it in its own table and its own module means that the
|
||||
structure enforces the anonymity of the counters rather than a convention.
|
||||
|
||||
`user_id` is a plain integer with no foreign key. An access log that
|
||||
disappeared when the account did would not serve its purpose, and guest
|
||||
cleanup deletes accounts on a schedule. `who` and `is_guest` are snapshots
|
||||
for the same reason, so a row still reads correctly after the account is
|
||||
gone.
|
||||
"""
|
||||
|
||||
__tablename__ = "access_events"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, index=True)
|
||||
# session | login | register | login_failed
|
||||
kind: Mapped[str] = mapped_column(String(16))
|
||||
user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
# The email for a registered account, or a label such as "Guest #12"
|
||||
# otherwise. For a failed sign-in, this holds the address that was tried,
|
||||
# which is the reason the row exists.
|
||||
who: Mapped[str] = mapped_column(String(320), default="")
|
||||
is_guest: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
ip: Mapped[str] = mapped_column(String(45), default="") # 45 = max IPv6
|
||||
country: Mapped[str] = mapped_column(String(16), default="")
|
||||
device: Mapped[str] = mapped_column(String(16), default="")
|
||||
user_agent: Mapped[str] = mapped_column(String(200), default="")
|
||||
|
||||
|
||||
# Phase 14: the fallback under `tree.place_action`.
|
||||
# The hosted visitor dashboard's two counter tables and the access log that
|
||||
# recorded sign-ins, addresses and devices used to be mapped here. M2 removed
|
||||
# the hosted deployment they served.
|
||||
#
|
||||
# Since SP2, reads filter on `branch_id` and `depth`. A node written without
|
||||
# them is invisible to every page, every context build, and every memory pass.
|
||||
# The failure is silent, because nothing raises an error. Every current writer
|
||||
# places its nodes explicitly, but relying on that would also mean relying on
|
||||
# every fixture, script, and test written from now on. The session therefore
|
||||
# enforces the rule as rows travel to the database.
|
||||
#
|
||||
# This listener is registered here rather than in tree.py so that importing the
|
||||
# models is enough to enable it. The invariant belongs to the rows, not to the
|
||||
# module that usually writes them. The import sits inside the callback because
|
||||
# tree.py imports this module.
|
||||
# The tables are left in the database rather than dropped: they are inert,
|
||||
# nothing reads or writes them, and a destructive migration would risk an
|
||||
# existing campaign database for tidiness alone. They are not product
|
||||
# functionality.
|
||||
|
||||
|
||||
@event.listens_for(Session, "before_flush")
|
||||
def _place_new_nodes_on_the_tree(session, flush_context, instances):
|
||||
from . import tree
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
"""SSRF guard for the one place the server makes an outbound request to a
|
||||
user-supplied address: the BYOK `endpoint_url` (connection test + turns/chat).
|
||||
|
||||
Without this guard, a hosted user could point `endpoint_url` at an internal
|
||||
service or at the cloud metadata endpoint, 169.254.169.254, and have the server
|
||||
fetch it. The connection test even returns part of the response. The guard
|
||||
therefore refuses any URL that resolves to a non-public address.
|
||||
|
||||
The guard does nothing in local mode. A local install talking to
|
||||
http://localhost:11434, which is Ollama, is the intended case. The guard applies
|
||||
only to a hosted, multi-user deployment, where the endpoint comes from an
|
||||
untrusted visitor.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from . import auth
|
||||
|
||||
|
||||
def endpoint_block_reason(url: str) -> str | None:
|
||||
"""A human-readable reason this URL must NOT be fetched server-side, or None
|
||||
if it's allowed. Resolves the host and rejects it if any resulting address
|
||||
is non-public (private, loopback, link-local/metadata, reserved, …).
|
||||
|
||||
Checking at request time (not just on save) is deliberate: it resists a DNS
|
||||
record that flips to a private IP after the value was stored.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
return None
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return "the endpoint URL must start with http:// or https://"
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return "the endpoint URL has no host"
|
||||
port = parsed.port or (443 if parsed.scheme == "https" else 80)
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return "the endpoint host could not be resolved"
|
||||
for info in infos:
|
||||
try:
|
||||
ip = ipaddress.ip_address(info[4][0])
|
||||
except ValueError:
|
||||
return "the endpoint host resolved to an unrecognized address"
|
||||
# is_global is the strict allowlist: private/loopback/link-local/CGNAT
|
||||
# all report False, so this one check covers the metadata IP too.
|
||||
if not ip.is_global or ip.is_multicast or ip.is_reserved:
|
||||
return "the endpoint URL resolves to a non-public address"
|
||||
return None
|
||||
@@ -3,32 +3,27 @@ from typing import AsyncIterator
|
||||
|
||||
import httpx
|
||||
|
||||
from .. import debuglog, netguard, tlstrust
|
||||
from .. import debuglog, endpoints, tlstrust
|
||||
from .base import PromptParts, Provider, ProviderError
|
||||
|
||||
# Appended after the story text in chat mode, so a chat-tuned model continues
|
||||
# the prose rather than replying conversationally.
|
||||
CHAT_CONTINUE_HINT = "\n\n[Continue the story directly. Output only story text.]"
|
||||
|
||||
# OpenRouter serves one model from whichever upstream is available, and every
|
||||
# upstream holds its own prompt cache, so a request routed somewhere new starts
|
||||
# with a cold cache however stable the prompt is. Naming a preferred upstream
|
||||
# makes routing deterministic, which is what allows a cache hit at all.
|
||||
#
|
||||
# `allow_fallbacks` stays at its default of true on purpose, because this is a
|
||||
# preference rather than a restriction. If the named upstream is down, the
|
||||
# request still goes elsewhere and only misses the cache, which is the behavior
|
||||
# without this setting.
|
||||
#
|
||||
# This is a list rather than a value derived from the model slug. The vendor half
|
||||
# of a slug is usually the provider slug, such as "deepseek/..." mapping to
|
||||
# "deepseek", which was verified against /api/v1/providers, but not reliably.
|
||||
# Google's models are served by "google-ai-studio" and "google-vertex", and there
|
||||
# is no "google". Look a vendor up on the model's Providers tab before adding it
|
||||
# here. A slug that does not exist is a routing preference that, at best, does
|
||||
# nothing.
|
||||
_OPENROUTER_HOST = "openrouter.ai"
|
||||
_PREFERRED_UPSTREAM = {"deepseek": "deepseek"}
|
||||
# A machine that is not listening refuses in milliseconds, so a slow connect
|
||||
# means the wrong address rather than a busy model.
|
||||
CONNECT_TIMEOUT = 10.0
|
||||
|
||||
# How long to wait for generation when Settings names no value. Upstream
|
||||
# hardcoded 120s, and M1 measured a *cold* load of a 3B model on a GPU-less
|
||||
# four-core host exceeding it three times while the same turn took 6-9 seconds
|
||||
# once the model was resident. 300s covers a cold start on modest hardware and
|
||||
# is still a number: a wedged endpoint fails rather than hanging forever.
|
||||
DEFAULT_READ_TIMEOUT = 300.0
|
||||
|
||||
# Embeddings are short and never cold-load a large model.
|
||||
EMBED_READ_TIMEOUT = 60.0
|
||||
|
||||
|
||||
|
||||
# Completion endpoints have no roles, so a chat has to be flattened into one
|
||||
@@ -44,74 +39,47 @@ def flatten_messages(messages: list[dict]) -> str:
|
||||
|
||||
|
||||
class OpenAICompatibleProvider(Provider):
|
||||
"""Adapter for any /v1-style endpoint.
|
||||
"""Adapter for Ollama's OpenAI-compatible `/v1` API.
|
||||
|
||||
This covers Ollama, LM Studio, OpenAI, OpenRouter, vLLM, and Groq, among
|
||||
others.
|
||||
The protocol is OpenAI's, which is what the module is named for; the
|
||||
product speaks it to Ollama and to nothing else. `endpoints.py` decides
|
||||
which addresses may be reached, and every request re-checks — the shape of
|
||||
the wire format is not the same thing as permission to use it.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
endpoint_url: str,
|
||||
api_key: str,
|
||||
model: str,
|
||||
api_mode: str = "chat",
|
||||
reasoning_max_tokens: int = 0,
|
||||
read_timeout: float | None = None,
|
||||
):
|
||||
self.base_url = endpoint_url.rstrip("/")
|
||||
self.api_key = api_key
|
||||
self.model = model
|
||||
self.api_mode = api_mode # Either "chat" or "completion".
|
||||
# The thinking budget for reasoning models, on top of `max_tokens`. A
|
||||
# value of 0 means the `reasoning` parameter is not sent, because an
|
||||
# endpoint that does not know the field may reject it. A negative value
|
||||
# asks the endpoint to turn reasoning off.
|
||||
self.reasoning_max_tokens = reasoning_max_tokens
|
||||
# How long to wait for the model, in seconds. Cold-loading a model on a
|
||||
# CPU-only machine can take minutes, and a fixed short timeout reports
|
||||
# that as a failure. See `DEFAULT_READ_TIMEOUT`.
|
||||
self.read_timeout = read_timeout or DEFAULT_READ_TIMEOUT
|
||||
# The token accounting from the last call, when the endpoint reported
|
||||
# any. It holds the prompt and completion counts, plus, on OpenRouter,
|
||||
# `prompt_tokens_details.cached_tokens`, which is the number of prompt
|
||||
# tokens read from cache rather than billed in full. Every request method
|
||||
# writes it, so a caller reads it after the call it made. One provider is
|
||||
# built per request.
|
||||
# any. Every request method writes it, so a caller reads it after the
|
||||
# call it made. One provider is built per request.
|
||||
self.last_usage: dict | None = None
|
||||
|
||||
def _headers(self) -> dict:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self.api_key:
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
return headers
|
||||
# No Authorization header: Ollama does not use one, and this build has
|
||||
# no cloud provider to carry a key for.
|
||||
return {"Content-Type": "application/json"}
|
||||
|
||||
def _apply_reasoning_budget(self, body: dict) -> None:
|
||||
"""Gives reasoning models their own thinking budget, in the OpenRouter style.
|
||||
def _timeout(self, seconds: float | None = None) -> httpx.Timeout:
|
||||
"""Short to connect, patient to read.
|
||||
|
||||
The method raises `max_tokens`, so the output keeps its full budget.
|
||||
|
||||
A negative budget does the opposite. It sends `effort: "none"` to turn
|
||||
reasoning off on a model that reasons by default, such as DeepSeek V4
|
||||
Flash. That differs from `exclude: true`, which still reasons and still
|
||||
bills for it while hiding the trace. Zero still means send nothing, so an
|
||||
endpoint that rejects unknown fields, such as Ollama, keeps working.
|
||||
A machine that is not listening says so in milliseconds, so a slow
|
||||
connect is a wrong address rather than a busy model and should fail
|
||||
fast. Generation is the opposite: the first token can be minutes away
|
||||
while a model loads.
|
||||
"""
|
||||
if self.api_mode != "chat":
|
||||
return
|
||||
if self.reasoning_max_tokens < 0:
|
||||
body["reasoning"] = {"effort": "none"}
|
||||
elif self.reasoning_max_tokens > 0:
|
||||
body["reasoning"] = {"max_tokens": self.reasoning_max_tokens}
|
||||
body["max_tokens"] += self.reasoning_max_tokens
|
||||
|
||||
def _apply_provider_routing(self, body: dict) -> None:
|
||||
"""Prefers one upstream on OpenRouter, so the prompt cache stays warm.
|
||||
|
||||
The method does nothing anywhere else. `provider` is an OpenRouter
|
||||
extension, and Ollama and similar servers reject fields they do not know.
|
||||
The `reasoning` parameter above is written around the same constraint.
|
||||
"""
|
||||
if _OPENROUTER_HOST not in self.base_url:
|
||||
return
|
||||
upstream = _PREFERRED_UPSTREAM.get(self.model.split("/", 1)[0].lower())
|
||||
if upstream:
|
||||
body["provider"] = {"order": [upstream]}
|
||||
return httpx.Timeout(seconds or self.read_timeout, connect=CONNECT_TIMEOUT)
|
||||
|
||||
def _record_usage(self, payload: dict) -> None:
|
||||
"""Records the endpoint's own token accounting, if it reported any.
|
||||
@@ -147,8 +115,6 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": True,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
return url, body
|
||||
|
||||
@staticmethod
|
||||
@@ -227,8 +193,6 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": True,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
async for event in self._stream(url, body):
|
||||
yield event
|
||||
|
||||
@@ -238,17 +202,17 @@ class OpenAICompatibleProvider(Provider):
|
||||
The method POSTs a streaming request, yields `("text", chunk)` and
|
||||
`("reasoning", chunk)` pairs, and logs the exchange.
|
||||
"""
|
||||
# SSRF guard for hosted mode. A user-supplied `endpoint_url` must not
|
||||
# point at an internal or metadata address. This does nothing for a
|
||||
# local install.
|
||||
reason = netguard.endpoint_block_reason(url)
|
||||
# Re-checked on every request, not only when the endpoint was saved: a
|
||||
# hostname that resolved to a LAN address yesterday can resolve
|
||||
# somewhere else today, and a database row can be edited by hand.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
received: list[str] = []
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(120, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
async with client.stream("POST", url, json=body, headers=self._headers()) as resp:
|
||||
if resp.status_code != 200:
|
||||
@@ -351,13 +315,16 @@ class OpenAICompatibleProvider(Provider):
|
||||
"max_tokens": max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
self._apply_reasoning_budget(body)
|
||||
self._apply_provider_routing(body)
|
||||
|
||||
# Same check as `_stream`: every outbound request re-tests the
|
||||
# endpoint, so no path reaches an address the policy refuses.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(120, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
resp = await client.post(url, json=body, headers=self._headers())
|
||||
except httpx.HTTPError as exc:
|
||||
@@ -383,10 +350,15 @@ class OpenAICompatibleProvider(Provider):
|
||||
raise ProviderError("No embedding model configured — set one in Settings.")
|
||||
url = f"{self.base_url}/embeddings"
|
||||
body = {"model": self.model, "input": texts}
|
||||
# Same check as `_stream`: every outbound request re-tests the
|
||||
# endpoint, so no path reaches an address the policy refuses.
|
||||
reason = endpoints.rejection_reason(url)
|
||||
if reason:
|
||||
raise ProviderError(f"This endpoint can't be used — {reason}.")
|
||||
log = debuglog.start_entry(url, self.model, body)
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(60, connect=10), verify=tlstrust.ssl_context()
|
||||
timeout=self._timeout(EMBED_READ_TIMEOUT), verify=tlstrust.ssl_context()
|
||||
) as client:
|
||||
resp = await client.post(url, json=body, headers=self._headers())
|
||||
except httpx.HTTPError as exc:
|
||||
|
||||
@@ -32,7 +32,6 @@ from . import ( # noqa: F401
|
||||
takes,
|
||||
branches,
|
||||
bundle_io,
|
||||
scripts,
|
||||
refresh,
|
||||
insights,
|
||||
memories,
|
||||
|
||||
@@ -7,7 +7,7 @@ only check ownership and hand the work over.
|
||||
from fastapi import Body, Depends, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import analytics, bundle, limits, models, schemas
|
||||
from ... import bundle, limits, models, schemas
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
@@ -34,7 +34,6 @@ def import_adventure(
|
||||
user: models.User = CurrentUser,
|
||||
):
|
||||
version = bundle.check_format(payload)
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("adventures", db, user)
|
||||
limits.check_bundle_lists(
|
||||
story_cards=payload.get("storyCards"),
|
||||
@@ -66,5 +65,4 @@ def import_adventure(
|
||||
# This is not a funnel step. A returning player imports a bundle, so it
|
||||
# says nothing about how far a first-time visitor got. It is counted anyway,
|
||||
# because it is the clearest evidence that anyone uses the export format.
|
||||
analytics.record_event(analytics.EV_IMPORT, user)
|
||||
return adventure
|
||||
|
||||
@@ -9,7 +9,7 @@ from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm.attributes import set_committed_value
|
||||
|
||||
from ... import analytics, attempts, images, limits, memorybank, models, schemas, tree, worldstate
|
||||
from ... import attempts, images, limits, memorybank, models, schemas, tree, worldstate
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
@@ -195,20 +195,6 @@ def create_adventure(
|
||||
if scenario:
|
||||
for ref, spec in scenario_card_specs(scenario, values).items():
|
||||
db.add(models.StoryCard(adventure_id=adventure.id, source_ref=ref, **spec))
|
||||
for position, script in enumerate(scenario.scripts):
|
||||
db.add(
|
||||
models.AdventureScript(
|
||||
adventure_id=adventure.id,
|
||||
source_script_id=script.id,
|
||||
position=position,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
library_js=script.library_js,
|
||||
input_js=script.input_js,
|
||||
context_js=script.context_js,
|
||||
output_js=script.output_js,
|
||||
)
|
||||
)
|
||||
if scenario.prompt.strip():
|
||||
opening = models.Action(
|
||||
adventure_id=adventure.id,
|
||||
@@ -223,12 +209,6 @@ def create_adventure(
|
||||
|
||||
db.commit()
|
||||
db.refresh(adventure)
|
||||
analytics.record_event(analytics.EV_ADVENTURE, user)
|
||||
# Track which shared scenarios players pick. This is the only content this
|
||||
# module records, and it records only public scenarios. A player's own
|
||||
# scenario titles stay private.
|
||||
if scenario is not None and scenario.is_public:
|
||||
analytics.record(analytics.M_SCENARIO, scenario.title)
|
||||
return adventure
|
||||
|
||||
|
||||
@@ -262,20 +242,6 @@ def get_adventure(
|
||||
return out
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/script-state")
|
||||
def get_script_state(
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Returns the scripting `state` object.
|
||||
|
||||
The object holds every variable that scripts read and write through
|
||||
`state.x`, persisted after each hook. It stays `{}` until a script sets a
|
||||
variable.
|
||||
"""
|
||||
state = adventure.script_state if isinstance(adventure.script_state, dict) else {}
|
||||
return {"state": state}
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/world-state")
|
||||
def get_world_state(
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
|
||||
@@ -7,7 +7,7 @@ returns the prompt a turn was actually generated from. Neither writes anything.
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import auth, memorybank, models
|
||||
from ... import memorybank, models
|
||||
from ...context import build_context
|
||||
from ...database import get_db
|
||||
from ..settings import get_settings
|
||||
@@ -23,14 +23,7 @@ async def dry_run_context(
|
||||
):
|
||||
"""Returns what the app would send to the AI if the player continued now."""
|
||||
settings = get_settings(db, user)
|
||||
if auth.resolve_provider_config(settings).using_demo:
|
||||
memories = (
|
||||
{"used": [], "error": "Memory bank is unavailable on the shared demo key."}
|
||||
if adventure.memory_bank_enabled
|
||||
else None
|
||||
)
|
||||
else:
|
||||
memories = await memorybank.retrieve_memories(adventure, settings, update_stats=False)
|
||||
memories = await memorybank.retrieve_memories(adventure, settings, update_stats=False)
|
||||
_, _, report = build_context(adventure, settings, memories)
|
||||
return report
|
||||
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
"""The per-adventure copies of library scripts.
|
||||
|
||||
An adventure snapshots a library `Script` when it starts, so editing the library
|
||||
does not change a story in progress. These endpoints report whether a snapshot
|
||||
has fallen behind its library original, and copy the original over on request.
|
||||
"""
|
||||
|
||||
from fastapi import Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import models, schemas
|
||||
from ...database import get_db
|
||||
|
||||
from .deps import CurrentUser, current_adventure, router
|
||||
|
||||
|
||||
# Fields that are copied from a library Script into its adventure-script
|
||||
# snapshot, and compared to decide whether a copy is out of date.
|
||||
SYNC_FIELDS = ("name", "description", "library_js", "input_js", "context_js", "output_js")
|
||||
|
||||
|
||||
def resolve_library_script(
|
||||
adv_script: models.AdventureScript, db: Session, user: models.User
|
||||
) -> models.Script | None:
|
||||
"""Returns the library Script an adventure script can re-sync from.
|
||||
|
||||
The result is the script this copy was made from. For a legacy copy with no
|
||||
link, it is one of the player's own scripts with the same name. Only the
|
||||
player's own scripts are considered, so a copy derived from a demo scenario
|
||||
has nothing to sync to.
|
||||
"""
|
||||
if adv_script.source_script_id is not None:
|
||||
script = db.get(models.Script, adv_script.source_script_id)
|
||||
if script is not None and script.user_id == user.id:
|
||||
return script
|
||||
return (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.user_id == user.id, models.Script.name == adv_script.name)
|
||||
.order_by(models.Script.updated_at.desc())
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def _mark_out_of_date(
|
||||
adv_script: models.AdventureScript, db: Session, user: models.User
|
||||
) -> models.AdventureScript:
|
||||
"""Attaches a transient `out_of_date` flag, which `AdventureScriptOut` reads.
|
||||
|
||||
The flag is `True` or `False` when a syncable library version exists, and
|
||||
`None` when none exists.
|
||||
"""
|
||||
library = resolve_library_script(adv_script, db, user)
|
||||
adv_script.out_of_date = (
|
||||
None if library is None
|
||||
else any(getattr(adv_script, f) != getattr(library, f) for f in SYNC_FIELDS)
|
||||
)
|
||||
return adv_script
|
||||
|
||||
|
||||
@router.get("/{adventure_id}/scripts", response_model=list[schemas.AdventureScriptOut])
|
||||
def list_adventure_scripts(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
return [_mark_out_of_date(s, db, user) for s in adventure.scripts]
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{adventure_id}/scripts/{adv_script_id}/sync",
|
||||
response_model=schemas.AdventureScriptOut,
|
||||
)
|
||||
def sync_adventure_script(
|
||||
adventure_id: int,
|
||||
adv_script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
"""Overwrites this copy's code with the latest from its library script.
|
||||
|
||||
`enabled`, `position`, and the adventure's shared `script_state` are kept.
|
||||
"""
|
||||
script = db.get(models.AdventureScript, adv_script_id)
|
||||
if script is None or script.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
library = resolve_library_script(script, db, user)
|
||||
if library is None:
|
||||
raise HTTPException(404, "No library script to sync from")
|
||||
for field in SYNC_FIELDS:
|
||||
setattr(script, field, getattr(library, field))
|
||||
# Store the link, so that a name-matched legacy copy syncs by id next
|
||||
# time.
|
||||
script.source_script_id = library.id
|
||||
db.commit()
|
||||
db.refresh(script)
|
||||
return _mark_out_of_date(script, db, user)
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/{adventure_id}/scripts/{adv_script_id}", response_model=schemas.AdventureScriptOut
|
||||
)
|
||||
def update_adventure_script(
|
||||
adventure_id: int,
|
||||
adv_script_id: int,
|
||||
payload: schemas.AdventureScriptUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
script = db.get(models.AdventureScript, adv_script_id)
|
||||
if script is None or script.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
for field, value in payload.model_dump(exclude_unset=True).items():
|
||||
setattr(script, field, value)
|
||||
db.commit()
|
||||
return script
|
||||
@@ -15,7 +15,6 @@ from ... import attempts, limits, memorybank, models, schemas, tree
|
||||
from ...context import cursors
|
||||
from ...context import lineage
|
||||
from ...database import get_db
|
||||
from ...scripting import ScriptPipeline
|
||||
from ...sse import SSE_HEADERS
|
||||
|
||||
from . import turns
|
||||
@@ -39,8 +38,6 @@ def retry_action(
|
||||
attempt is stored as a sibling at the same coordinate. No text the AI wrote
|
||||
is rewritten or deleted.
|
||||
"""
|
||||
limits.rate_limit("turn", request, user)
|
||||
turns.check_demo_cap(db, user)
|
||||
turns.acquire_turn_lock(adventure_id)
|
||||
last_ai = None
|
||||
try:
|
||||
@@ -62,9 +59,7 @@ def retry_action(
|
||||
return StreamingResponse(
|
||||
turns.with_turn_lock(
|
||||
adventure_id,
|
||||
turns.generate_turn(
|
||||
adventure, db, ScriptPipeline(adventure, db), user, retry_of=last_ai
|
||||
),
|
||||
turns.generate_turn(adventure, db, user, retry_of=last_ai),
|
||||
),
|
||||
media_type="text/event-stream",
|
||||
headers=SSE_HEADERS,
|
||||
@@ -254,9 +249,7 @@ def add_take(
|
||||
turn, so the new attempt is written at the same depth under the same parent,
|
||||
and the line it leaves is unchanged. No node below is copied.
|
||||
"""
|
||||
limits.rate_limit("turn", request, user)
|
||||
limits.check_row_cap("actions", db, user, adventure=adventure)
|
||||
turns.check_demo_cap(db, user)
|
||||
action = db.get(models.Action, action_id)
|
||||
if action is None or action.adventure_id != adventure_id:
|
||||
raise HTTPException(404, "Action not found")
|
||||
@@ -292,9 +285,7 @@ def add_take(
|
||||
if action.type == "ai":
|
||||
# There is no player action to write. The action this turn answers is
|
||||
# already on the path, borrowed from the line being left.
|
||||
stream = turns.generate_turn(
|
||||
adventure, db, ScriptPipeline(adventure, db), user, retry_of=retry_of
|
||||
)
|
||||
stream = turns.generate_turn(adventure, db, user, retry_of=retry_of)
|
||||
else:
|
||||
stream = turns.run_player_turn(
|
||||
adventure,
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
Everything a test needs to intercept lives here, and other modules reach it as
|
||||
`turns.<name>` rather than importing it by value. That matters twice. The turn
|
||||
lock guards one set only while one module owns it. And a test that replaces
|
||||
`OpenAICompatibleProvider`, `generate_turn`, or `check_demo_cap` patches this
|
||||
module, which every caller reads through.
|
||||
`OpenAICompatibleProvider` or `generate_turn` patches this module, which every
|
||||
caller reads through.
|
||||
"""
|
||||
import threading
|
||||
|
||||
@@ -13,12 +13,11 @@ from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ... import (
|
||||
analytics, attempts, auth, limits, memorybank, models, schemas, tree, worldstate,
|
||||
attempts, limits, memorybank, models, schemas, tree, worldstate,
|
||||
)
|
||||
from ...context import build_context, cursors
|
||||
from ...database import get_db
|
||||
from ...providers import OpenAICompatibleProvider, PromptParts, ProviderError
|
||||
from ...scripting import ScriptPipeline
|
||||
from ...sse import SSE_HEADERS, sse, turn_error
|
||||
from ..settings import get_settings
|
||||
|
||||
@@ -115,14 +114,11 @@ def action_json(action: models.Action, db: Session | None = None) -> dict:
|
||||
async def generate_turn(
|
||||
adventure: models.Adventure,
|
||||
db: Session,
|
||||
pipeline: ScriptPipeline,
|
||||
user: models.User,
|
||||
retry_of: models.Action | None = None,
|
||||
):
|
||||
"""Streams the AI continuation as SSE, then stores the result.
|
||||
|
||||
The continuation passes through the `context` and `output` script hooks.
|
||||
|
||||
If `retry_of` is set, the result is stored as a sibling of that AI action, at
|
||||
the same turn and the same coordinate, and the discarded attempt stays where
|
||||
it was written. Before calling, the caller must roll the adventure back to
|
||||
@@ -132,15 +128,15 @@ async def generate_turn(
|
||||
"""
|
||||
saved = False
|
||||
try:
|
||||
async for event in _generate_turn(adventure, db, pipeline, user, retry_of):
|
||||
async for event in _generate_turn(adventure, db, user, retry_of):
|
||||
if event is _SAVED:
|
||||
saved = True
|
||||
continue
|
||||
yield event
|
||||
finally:
|
||||
if retry_of is not None and not saved:
|
||||
# The turn failed with a provider error, an empty reply, a script
|
||||
# stop, or a disconnected client. No sibling was written, so the
|
||||
# The turn failed with a provider error, an empty reply, or a
|
||||
# disconnected client. No sibling was written, so the
|
||||
# attempt on screen is still the live one. Restore the state it
|
||||
# produced.
|
||||
attempts.restore_state(adventure, retry_of)
|
||||
@@ -155,55 +151,27 @@ _SAVED = object()
|
||||
async def _generate_turn(
|
||||
adventure: models.Adventure,
|
||||
db: Session,
|
||||
pipeline: ScriptPipeline,
|
||||
user: models.User,
|
||||
retry_of: models.Action | None = None,
|
||||
):
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
# On a retry, the attempt being replaced is still the live node of its turn,
|
||||
# because it stays live until a replacement exists. Filter it out of the
|
||||
# context. Otherwise the model reads the attempt it is replacing as
|
||||
# established story and writes a sequel to it.
|
||||
replacing_id = retry_of.id if retry_of is not None else None
|
||||
if cfg.using_demo:
|
||||
# The server-funded key makes no embedding or summarization calls, so
|
||||
# memory retrieval is skipped. If the bank is on, return a note.
|
||||
memories = (
|
||||
{"used": [], "error": "Memory bank is unavailable on the shared demo key — add your own API key in Settings."}
|
||||
if adventure.memory_bank_enabled
|
||||
else None
|
||||
)
|
||||
else:
|
||||
memories = await memorybank.retrieve_memories(
|
||||
adventure, settings, update_stats=True, exclude_action_id=replacing_id
|
||||
)
|
||||
memories = await memorybank.retrieve_memories(
|
||||
adventure, settings, update_stats=True, exclude_action_id=replacing_id
|
||||
)
|
||||
system_text, story_text, snapshot = build_context(
|
||||
adventure, settings, memories, exclude_action_id=replacing_id
|
||||
)
|
||||
|
||||
# onModelContext: scripts read, and can rewrite, the whole assembled
|
||||
# context.
|
||||
combined = f"{system_text}\n\n{story_text}" if system_text else story_text
|
||||
modified, stop = pipeline.run("context", combined)
|
||||
if stop:
|
||||
yield sse({"type": "stopped", "script": pipeline.report()})
|
||||
return
|
||||
context_changed = modified != combined
|
||||
parts = (
|
||||
PromptParts(system="", story=modified)
|
||||
if context_changed
|
||||
else PromptParts(system=system_text, story=story_text)
|
||||
)
|
||||
snapshot["script"] = pipeline.report() | {
|
||||
"context_changed": context_changed,
|
||||
"context_before": combined if context_changed else None,
|
||||
"context_after": modified if context_changed else None,
|
||||
}
|
||||
parts = PromptParts(system=system_text, story=story_text)
|
||||
|
||||
provider = OpenAICompatibleProvider(
|
||||
cfg.endpoint_url, cfg.api_key, cfg.model, settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.endpoint_url, settings.model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
chunks: list[str] = []
|
||||
reasoning_chunks: list[str] = []
|
||||
@@ -239,13 +207,6 @@ async def _generate_turn(
|
||||
yield turn_error(detail)
|
||||
return
|
||||
|
||||
# onOutput
|
||||
text, _ = pipeline.run("output", text)
|
||||
if not text.strip():
|
||||
yield turn_error("A script's output modifier returned empty text.")
|
||||
return
|
||||
snapshot["script"] = snapshot["script"] | pipeline.report()
|
||||
|
||||
# RPG world state (Phase 12): read the AI's state delta out of the reply,
|
||||
# apply it through the engine, and strip the block from the displayed text.
|
||||
#
|
||||
@@ -304,39 +265,15 @@ async def _generate_turn(
|
||||
tree.place_action(db, adventure, ai_action)
|
||||
db.add(ai_action)
|
||||
adventure.updated_at = models.utcnow()
|
||||
if cfg.using_demo:
|
||||
# Successful demo turns count against the daily cap, which the endpoint
|
||||
# checks before the turn starts. A failed provider call above returns
|
||||
# before this line.
|
||||
auth.count_demo_turn(user)
|
||||
db.commit()
|
||||
# Count the turn here, after every path on which it could still have failed,
|
||||
# so the number means "stories advanced" rather than "requests attempted".
|
||||
# The demo tally counts those same turns as spend on the server-funded key.
|
||||
analytics.record_event(analytics.EV_TURN, user)
|
||||
if cfg.using_demo:
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_DEMO_TURN)
|
||||
db.refresh(ai_action)
|
||||
yield _SAVED
|
||||
yield sse({"type": "done", "action": action_json(ai_action, db), "script": pipeline.report()})
|
||||
yield sse({"type": "done", "action": action_json(ai_action, db)})
|
||||
# Phase 6: schedule summarization and embedding without waiting for them.
|
||||
# The task opens its own database session. It is skipped on the demo key,
|
||||
# because background AI calls are unmetered spend on the server-funded
|
||||
# key.
|
||||
if not cfg.using_demo:
|
||||
memorybank.schedule_post_turn(adventure)
|
||||
# The task opens its own database session.
|
||||
memorybank.schedule_post_turn(adventure)
|
||||
|
||||
|
||||
def check_demo_cap(db: Session, user: models.User) -> None:
|
||||
"""Checks the demo cap before a turn starts.
|
||||
|
||||
Checking first avoids storing a capped player's input and then leaving it
|
||||
without a reply.
|
||||
"""
|
||||
settings = get_settings(db, user)
|
||||
if auth.resolve_provider_config(settings).using_demo and auth.demo_turns_left(user) <= 0:
|
||||
raise HTTPException(429, auth.DEMO_CAP_MESSAGE)
|
||||
|
||||
|
||||
async def run_player_turn(
|
||||
adventure: models.Adventure,
|
||||
@@ -353,29 +290,20 @@ async def run_player_turn(
|
||||
formatted, and a plain edit puts that same text in the box and writes it back
|
||||
verbatim. Formatting it a second time produces `> You > You ...`.
|
||||
"""
|
||||
pipeline = ScriptPipeline(adventure, db)
|
||||
|
||||
# An empty do, say, or story action behaves as a continue.
|
||||
if payload.type != "continue" and payload.text.strip():
|
||||
# onInput reads the formatted text, as in AI Dungeon: "> You ...".
|
||||
formatted = (
|
||||
payload.text.strip() if preformatted
|
||||
else format_player_input(payload.type, payload.text)
|
||||
)
|
||||
modified, stop = pipeline.run("input", formatted)
|
||||
if not modified.strip():
|
||||
yield turn_error("A script's input modifier returned empty text.",
|
||||
script=pipeline.report())
|
||||
return
|
||||
player_action = models.Action(
|
||||
adventure_id=adventure.id,
|
||||
depth=next_depth(adventure),
|
||||
type=payload.type,
|
||||
text=modified,
|
||||
text=formatted,
|
||||
)
|
||||
# The state after the input hook has run. The node leaves this state
|
||||
# behind. The AI turn after it starts here, and a retry of that turn
|
||||
# rolls back to here.
|
||||
# The state this node leaves behind. The AI turn after it starts here,
|
||||
# and a retry of that turn rolls back to here.
|
||||
attempts.snapshot_outcome(adventure, player_action)
|
||||
tree.place_action(db, adventure, player_action)
|
||||
db.add(player_action)
|
||||
@@ -387,12 +315,8 @@ async def run_player_turn(
|
||||
# that was just saved.
|
||||
db.expire(adventure, ["actions"])
|
||||
yield sse({"type": "player", "action": action_json(player_action, db)})
|
||||
if stop:
|
||||
# If onInput returns `{ stop: true }`, skip the AI call.
|
||||
yield sse({"type": "stopped", "script": pipeline.report()})
|
||||
return
|
||||
|
||||
async for event in generate_turn(adventure, db, pipeline, user):
|
||||
async for event in generate_turn(adventure, db, user):
|
||||
yield event
|
||||
|
||||
|
||||
@@ -405,9 +329,7 @@ def create_action(
|
||||
user: models.User = CurrentUser,
|
||||
adventure: models.Adventure = Depends(current_adventure),
|
||||
):
|
||||
limits.rate_limit("turn", request, user)
|
||||
limits.check_row_cap("actions", db, user, adventure=adventure)
|
||||
check_demo_cap(db, user)
|
||||
acquire_turn_lock(adventure_id)
|
||||
try:
|
||||
_move_to_after(db, adventure, payload.after_id)
|
||||
|
||||
@@ -1,140 +0,0 @@
|
||||
"""Visit analytics: one endpoint the browser writes to, one the owner reads.
|
||||
|
||||
The split matters. `/collect` is public and accepts one fact, which page was
|
||||
viewed, because anything a stranger can POST is a number a stranger can invent.
|
||||
Everything the dashboard relies on, meaning turns, adventures, sign-ups, demo
|
||||
spend, and errors, is recorded on the server by the code that performs it, so
|
||||
those counts are as trustworthy as the app itself.
|
||||
|
||||
The two reading endpoints are owner-only and 404 for everyone else, the same
|
||||
way the AI Chat router does: a feature nobody else can use is better off not
|
||||
appearing to exist. `/summary` serves the anonymous counters (analytics.py,
|
||||
which stores nothing that points at a person) and `/access` serves the access
|
||||
log (accesslog.py, which identifies people on purpose).
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import accesslog, analytics, auth, limits, models
|
||||
from ..database import get_db
|
||||
|
||||
router = APIRouter(prefix="/api/analytics", tags=["analytics"])
|
||||
|
||||
|
||||
def owner(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
) -> models.User:
|
||||
"""Gates the reading half. It returns 404 rather than 403. See the module
|
||||
docstring."""
|
||||
if not auth.is_owner(user):
|
||||
raise HTTPException(404, "Not found")
|
||||
return user
|
||||
|
||||
|
||||
Owner = Depends(owner)
|
||||
|
||||
|
||||
class Pageview(BaseModel):
|
||||
"""What the SPA reports on a page load or a route change.
|
||||
|
||||
`first` marks a real page load rather than a client-side navigation. The
|
||||
facts that describe a visit rather than a view, which are where it came from,
|
||||
on what kind of device, and from which country, are recorded only on a page
|
||||
load, so a visitor who clicks through five pages is still one referral.
|
||||
"""
|
||||
|
||||
path: str = Field("", max_length=300)
|
||||
referrer: str = Field("", max_length=500)
|
||||
first: bool = False
|
||||
|
||||
|
||||
@router.post("/collect", status_code=204)
|
||||
def collect(
|
||||
payload: Pageview,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
) -> Response:
|
||||
"""Record one pageview. Always 204, even when nothing was counted: the
|
||||
browser has no business knowing whether it was."""
|
||||
limits.rate_limit("analytics", request)
|
||||
# Resolved by hand rather than through get_current_user: a pageview that
|
||||
# arrives before /auth/me has minted a session should still be counted as a
|
||||
# view, not turned into a 401 the SPA has to handle.
|
||||
user = (
|
||||
auth.resolve_session_user(request, db)
|
||||
if auth.MULTI_USER
|
||||
else auth.local_user(db)
|
||||
)
|
||||
# The operator's own clicks are not traffic. This applies only in
|
||||
# multi-user mode. Locally every user is the owner, and excluding them would
|
||||
# leave the dashboard empty on the machine the app is developed on.
|
||||
if auth.MULTI_USER and user is not None and auth.is_owner(user):
|
||||
return Response(status_code=204)
|
||||
|
||||
analytics.record(analytics.M_PAGE, analytics.normalize_route(payload.path))
|
||||
analytics.record_visit(user)
|
||||
if payload.first:
|
||||
referrer = analytics.normalize_referrer(
|
||||
payload.referrer, request.url.hostname or ""
|
||||
)
|
||||
if referrer: # "" means same-origin, which is not a referral
|
||||
analytics.record(analytics.M_REFERRER, referrer)
|
||||
analytics.record(
|
||||
analytics.M_DEVICE,
|
||||
analytics.device_of(request.headers.get("user-agent", "")),
|
||||
)
|
||||
analytics.record(analytics.M_COUNTRY, analytics.country_of(request.headers))
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.get("/summary")
|
||||
def summary(
|
||||
days: int = Query(30, ge=1, le=365),
|
||||
db: Session = Depends(get_db),
|
||||
_user: models.User = Owner,
|
||||
) -> dict:
|
||||
"""Returns the whole dashboard in one aggregate response.
|
||||
|
||||
The response is a few kilobytes however much traffic is behind it.
|
||||
"""
|
||||
return analytics.summary(db, days)
|
||||
|
||||
|
||||
@router.get("/access")
|
||||
def access_log(
|
||||
limit: int = Query(50, ge=1, le=200),
|
||||
before_id: int | None = Query(None),
|
||||
kind: str | None = Query(None),
|
||||
q: str | None = Query(None, max_length=120),
|
||||
db: Session = Depends(get_db),
|
||||
_user: models.User = Owner,
|
||||
) -> dict:
|
||||
"""A page of the access log, newest first.
|
||||
|
||||
Unlike `/summary`, this returns rows about people, which is what it is for.
|
||||
It is therefore behind the same owner gate, it is paged rather than returned
|
||||
in full, and the people it describes have no endpoint that reaches it.
|
||||
"""
|
||||
page = accesslog.recent(
|
||||
db, limit=limit, before_id=before_id, kind=kind, query=q
|
||||
)
|
||||
return {
|
||||
"events": [
|
||||
{
|
||||
"id": event.id,
|
||||
"at": event.at.isoformat(),
|
||||
"kind": event.kind,
|
||||
"who": event.who,
|
||||
"is_guest": event.is_guest,
|
||||
"ip": event.ip,
|
||||
"country": event.country,
|
||||
"device": event.device,
|
||||
"user_agent": event.user_agent,
|
||||
}
|
||||
for event in page["events"]
|
||||
],
|
||||
"has_more": page["has_more"],
|
||||
}
|
||||
@@ -1,158 +0,0 @@
|
||||
import re
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import (accesslog, analytics, auth, cleanup, limits, models, schemas,
|
||||
security, starter)
|
||||
from ..database import get_db
|
||||
from .settings import get_settings
|
||||
|
||||
router = APIRouter(prefix="/api/auth", tags=["auth"])
|
||||
|
||||
EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
|
||||
|
||||
|
||||
def _set_session_cookie(response: Response, user_id: int) -> None:
|
||||
response.set_cookie(
|
||||
auth.SESSION_COOKIE,
|
||||
security.sign_session(user_id),
|
||||
max_age=auth.COOKIE_MAX_AGE,
|
||||
httponly=True,
|
||||
samesite="lax",
|
||||
secure=auth.COOKIE_SECURE,
|
||||
path="/",
|
||||
)
|
||||
|
||||
|
||||
def me_payload(user: models.User, db: Session) -> dict:
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
return {
|
||||
"multi_user": auth.MULTI_USER,
|
||||
"id": user.id,
|
||||
"email": user.email,
|
||||
"is_guest": user.is_guest,
|
||||
# Trusted testers: unmetered demo turns, plus the AI Chat scratchpad.
|
||||
"power_user": auth.is_power_user(user),
|
||||
# Separate allowlist: shows the visit-analytics page and its nav link.
|
||||
"analytics": auth.is_owner(user),
|
||||
# How long an idle guest is kept before cleanup deletes it (None when
|
||||
# the policy is off). Served rather than hardcoded in the UI so the
|
||||
# number a guest is shown is the number actually enforced.
|
||||
"guest_retention_days": cleanup.RETENTION_DAYS if cleanup.enabled() else None,
|
||||
"demo": {
|
||||
"enabled": auth.demo_enabled(),
|
||||
"using_demo": cfg.using_demo,
|
||||
"model": cfg.model if cfg.using_demo else None,
|
||||
"turns_per_day": auth.DEMO_TURNS_PER_DAY,
|
||||
"turns_left": auth.demo_turns_left(user) if auth.demo_enabled() else None,
|
||||
"models": auth.DEMO_MODELS if auth.demo_enabled() else [],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
def me(request: Request, response: Response, db: Session = Depends(get_db)):
|
||||
"""Returns the current user.
|
||||
|
||||
In multi-user mode this also establishes the session. If the cookie is
|
||||
missing or invalid, the endpoint creates a guest user and sets a cookie. The
|
||||
frontend calls it on load and after any 401.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
user = auth.local_user(db)
|
||||
else:
|
||||
user = auth.resolve_session_user(request, db)
|
||||
if user is None:
|
||||
# Each new guest is a database row, so cap how fast one IP can
|
||||
# create them.
|
||||
limits.rate_limit("guest", request)
|
||||
user = models.User(is_guest=True)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
# The guest is committed first, so a failure while copying the
|
||||
# starter adventure still leaves them with an account.
|
||||
starter.give(db, user)
|
||||
db.commit()
|
||||
_set_session_cookie(response, user.id)
|
||||
# This endpoint is the SPA's bootstrap call, so it is where a session first
|
||||
# shows itself; accesslog thins the rows down to one per day per address.
|
||||
accesslog.note_session(db, user, request)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/register")
|
||||
def register(
|
||||
payload: schemas.AuthCredentials,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Upgrades the current guest in place.
|
||||
|
||||
The `user_id` does not change, so every adventure, scenario, script, and
|
||||
setting they created as a guest is kept.
|
||||
"""
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
limits.rate_limit("auth", request)
|
||||
email = payload.email.strip().lower()
|
||||
if not EMAIL_RE.match(email):
|
||||
raise HTTPException(422, "Enter a valid email address.")
|
||||
if len(payload.password) < 8:
|
||||
raise HTTPException(422, "Password must be at least 8 characters.")
|
||||
if not user.is_guest:
|
||||
raise HTTPException(400, "This session is already registered.")
|
||||
if db.query(models.User).filter(models.User.email == email).first():
|
||||
raise HTTPException(409, "An account with this email already exists — log in instead.")
|
||||
user.email = email
|
||||
user.password_hash = security.hash_password(payload.password)
|
||||
user.is_guest = False
|
||||
db.commit()
|
||||
analytics.record_event(analytics.EV_SIGNUP, user)
|
||||
accesslog.record(db, accesslog.REGISTER, request, user=user)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/login")
|
||||
def login(
|
||||
payload: schemas.AuthCredentials,
|
||||
request: Request,
|
||||
response: Response,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Point this browser's session at an existing account. Any current guest
|
||||
session is simply abandoned (its data stays under the guest user)."""
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
limits.rate_limit("auth", request)
|
||||
email = payload.email.strip().lower()
|
||||
# Per-account throttle: stops distributed guessing against one email even
|
||||
# when the per-IP limit above is diluted across many source addresses.
|
||||
limits.check_login_allowed(email)
|
||||
user = db.query(models.User).filter(models.User.email == email).first()
|
||||
if (
|
||||
user is None
|
||||
or not user.password_hash
|
||||
or not security.verify_password(payload.password, user.password_hash)
|
||||
):
|
||||
limits.note_login_failure(email)
|
||||
# Logged with the address that was tried, not the account that owns it:
|
||||
# a guessing run against an address that has no account is exactly the
|
||||
# thing worth being able to see.
|
||||
accesslog.record(db, accesslog.LOGIN_FAILED, request, who=email)
|
||||
raise HTTPException(401, "Incorrect email or password.")
|
||||
limits.note_login_success(email)
|
||||
_set_session_cookie(response, user.id)
|
||||
analytics.record_event(analytics.EV_LOGIN, user)
|
||||
accesslog.record(db, accesslog.LOGIN, request, user=user)
|
||||
return me_payload(user, db)
|
||||
|
||||
|
||||
@router.post("/logout")
|
||||
def logout(response: Response):
|
||||
if not auth.MULTI_USER:
|
||||
raise HTTPException(400, "Accounts are disabled in local mode.")
|
||||
response.delete_cookie(auth.SESSION_COOKIE, path="/")
|
||||
return {"ok": True}
|
||||
+37
-79
@@ -1,22 +1,23 @@
|
||||
"""AI Chat: a plain scratchpad for talking to a model directly.
|
||||
"""AI Chat: a plain scratchpad for talking to the configured model directly.
|
||||
|
||||
Power users reach it, which means the `AIDND_POWER_USERS` email allowlist. It is
|
||||
deliberately thin. It adds no story context, no scripts, and no world state, and
|
||||
it persists nothing. The conversation lives in the browser and is posted in full
|
||||
on each turn. It exists for testing models, prompts, and endpoints without
|
||||
starting an adventure.
|
||||
Deliberately thin. It adds no story context and no world state, and it persists
|
||||
nothing. The conversation lives in the browser and is posted in full on each
|
||||
turn. It exists for checking a model, a prompt, or an endpoint without starting
|
||||
an adventure — which is exactly the kind of thing a local single-user install
|
||||
wants a page for.
|
||||
|
||||
Model choice is free when the user brought their own API key. On the shared demo
|
||||
key the model stays pinned to the `AIDND_DEMO_MODELS` allowlist, exactly as it is
|
||||
for turns. The server funds that key, so this page must not let it reach paid
|
||||
models.
|
||||
Upstream gated this behind a "power user" email allowlist and pinned the model
|
||||
when a shared demo key was in play. M2 removed both: there is one local user,
|
||||
who owns the endpoint, and there is no server-funded key to protect. The model
|
||||
this page talks to is the one in Settings, or one the user names per request —
|
||||
either way it is their own Ollama.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import auth, limits, models, schemas
|
||||
from .. import auth, models, schemas
|
||||
from ..database import get_db
|
||||
from ..providers import OpenAICompatibleProvider, ProviderError
|
||||
from ..sse import SSE_HEADERS, sse
|
||||
@@ -25,80 +26,41 @@ from .settings import get_settings, list_endpoint_models
|
||||
router = APIRouter(prefix="/api/chat", tags=["chat"])
|
||||
|
||||
|
||||
def power_user(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
) -> models.User:
|
||||
"""Gate for the whole router. 404 rather than 403 so the feature simply
|
||||
doesn't appear to exist for everyone else."""
|
||||
if not auth.is_power_user(user):
|
||||
raise HTTPException(404, "Not found")
|
||||
return user
|
||||
|
||||
|
||||
PowerUser = Depends(power_user)
|
||||
|
||||
|
||||
def _resolve_model(
|
||||
settings: models.Settings, requested: str | None
|
||||
) -> tuple[auth.ProviderConfig, str | None]:
|
||||
"""Returns the provider config for this chat, plus a note when the requested
|
||||
model was not used.
|
||||
|
||||
The pinning rule lives in `resolve_provider_config`. This function only
|
||||
reports the substitution that call made, so one place decides what the demo
|
||||
key may talk to.
|
||||
"""
|
||||
cfg = auth.resolve_provider_config(settings, model_override=requested)
|
||||
wanted = (requested or "").strip()
|
||||
if wanted and wanted != cfg.model:
|
||||
return cfg, (
|
||||
f"'{wanted}' isn't available on the shared demo key — using "
|
||||
f"{cfg.model}. Add your own API key in Settings to use any model."
|
||||
)
|
||||
return cfg, None
|
||||
|
||||
|
||||
@router.get("/config")
|
||||
async def chat_config(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Returns what this page can talk to.
|
||||
|
||||
The response holds the resolved endpoint and model, whether model choice is
|
||||
pinned to the demo allowlist, and the endpoint's model listing. The listing
|
||||
is best effort, and an unreachable endpoint returns an empty list.
|
||||
The model listing is best effort: an unreachable endpoint returns an empty
|
||||
list and the reason, rather than failing the page.
|
||||
"""
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
listing = await list_endpoint_models(cfg)
|
||||
listing = await list_endpoint_models(settings.endpoint_url)
|
||||
return {
|
||||
"endpoint_url": cfg.endpoint_url,
|
||||
"model": cfg.model,
|
||||
"using_demo": cfg.using_demo,
|
||||
"endpoint_url": settings.endpoint_url,
|
||||
"model": settings.model,
|
||||
"api_mode": settings.api_mode,
|
||||
"temperature": settings.temperature,
|
||||
"max_tokens": settings.max_output_tokens,
|
||||
# On the demo key the whitelist IS the list of choices; otherwise it's
|
||||
# whatever the endpoint advertises (suggestions, not a restriction).
|
||||
"models": auth.DEMO_MODELS if cfg.using_demo else listing.get("models", []),
|
||||
# Suggestions from the endpoint, not a restriction.
|
||||
"models": listing.get("models", []),
|
||||
"models_error": None if listing.get("ok") else listing.get("detail"),
|
||||
}
|
||||
|
||||
|
||||
async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload: schemas.ChatRequest,
|
||||
note: str | None, db: Session, user: models.User):
|
||||
async def run_chat(
|
||||
settings: models.Settings, model: str, payload: schemas.ChatRequest
|
||||
):
|
||||
"""Streams the reply as SSE, using the turn stream's event shape.
|
||||
|
||||
The generator emits `reasoning` and `chunk` events while generating and then
|
||||
a `done` event, so the frontend reuses the same code.
|
||||
"""
|
||||
if note:
|
||||
yield sse({"type": "note", "detail": note})
|
||||
provider = OpenAICompatibleProvider(
|
||||
cfg.endpoint_url, cfg.api_key, cfg.model, settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.endpoint_url, model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
messages = [m.model_dump() for m in payload.messages]
|
||||
chunks: list[str] = []
|
||||
@@ -106,7 +68,11 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
try:
|
||||
async for kind, chunk in provider.chat(
|
||||
messages,
|
||||
temperature=payload.temperature if payload.temperature is not None else settings.temperature,
|
||||
temperature=(
|
||||
payload.temperature
|
||||
if payload.temperature is not None
|
||||
else settings.temperature
|
||||
),
|
||||
max_tokens=payload.max_tokens or settings.max_output_tokens,
|
||||
):
|
||||
if kind == "reasoning":
|
||||
@@ -123,33 +89,26 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
if not text:
|
||||
detail = (
|
||||
"The model used its entire token budget on reasoning and returned no "
|
||||
"reply — raise max tokens, cap the reasoning budget in Settings, or "
|
||||
"use a non-reasoning model."
|
||||
"reply — raise max tokens or use a non-reasoning model."
|
||||
if reasoning_chunks
|
||||
else "The AI returned an empty response."
|
||||
)
|
||||
yield sse({"type": "error", "detail": detail})
|
||||
return
|
||||
|
||||
if cfg.using_demo:
|
||||
# Unmetered for power users (count_demo_turn is a no-op for them), but
|
||||
# keep the call so the accounting stays right if the gate ever widens.
|
||||
auth.count_demo_turn(user)
|
||||
db.commit()
|
||||
yield sse({
|
||||
"type": "done",
|
||||
"text": text,
|
||||
"reasoning": "".join(reasoning_chunks).strip() or None,
|
||||
"model": cfg.model,
|
||||
"model": model,
|
||||
})
|
||||
|
||||
|
||||
@router.post("/stream")
|
||||
def chat_stream(
|
||||
payload: schemas.ChatRequest,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
total = sum(len(m.content) for m in payload.messages)
|
||||
if total > schemas.CHAT_TOTAL_MAX:
|
||||
@@ -157,13 +116,12 @@ def chat_stream(
|
||||
413, f"This conversation is too long to send ({total:,} characters) — "
|
||||
"clear it or start a new one."
|
||||
)
|
||||
limits.rate_limit("chat", request, user)
|
||||
settings = get_settings(db, user)
|
||||
cfg, note = _resolve_model(settings, payload.model)
|
||||
if not cfg.model:
|
||||
model = (payload.model or "").strip() or settings.model
|
||||
if not model:
|
||||
raise HTTPException(400, "No model configured — set one in Settings or pick one here.")
|
||||
return StreamingResponse(
|
||||
run_chat(cfg, settings, payload, note, db, user),
|
||||
run_chat(settings, model, payload),
|
||||
media_type="text/event-stream",
|
||||
headers=SSE_HEADERS,
|
||||
)
|
||||
|
||||
@@ -1,19 +1,16 @@
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi import APIRouter
|
||||
|
||||
from .. import auth, debuglog
|
||||
from .. import debuglog
|
||||
|
||||
router = APIRouter(prefix="/api/debug", tags=["debug"])
|
||||
|
||||
|
||||
@router.get("/requests")
|
||||
def recent_requests():
|
||||
"""Most-recent-first log of provider requests/responses (no API keys).
|
||||
"""Most-recent-first log of provider requests and responses.
|
||||
|
||||
The log is a single process-wide ring buffer with no per-user attribution,
|
||||
so in multi-user mode, which is how a hosted deployment runs, it would expose
|
||||
other players' prompts. It is disabled there and available on a local
|
||||
install.
|
||||
A single process-wide ring buffer. It holds the prompts this install sent
|
||||
to its own Ollama, which is exactly what the person running it needs to
|
||||
diagnose a turn, and there is nobody else it could expose them to.
|
||||
"""
|
||||
if auth.MULTI_USER:
|
||||
raise HTTPException(403, "The debug log is only available on local installs.")
|
||||
return debuglog.recent()
|
||||
|
||||
@@ -3,7 +3,7 @@ from fastapi.responses import Response
|
||||
from sqlalchemy import or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import analytics, auth, images, limits, models, schemas
|
||||
from .. import auth, images, limits, models, schemas
|
||||
from ..database import get_db
|
||||
|
||||
router = APIRouter(prefix="/api/scenarios", tags=["scenarios"])
|
||||
@@ -54,12 +54,6 @@ def get_scenario(
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
scenario = get_scenario_or_404(scenario_id, db, user)
|
||||
# A funnel step, recorded for shared scenarios only. Opening one is the
|
||||
# first sign that a visitor is interested, and someone editing their own
|
||||
# scenario is already past this point. Their titles are theirs rather than a
|
||||
# statistic.
|
||||
if scenario.is_public:
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, user)
|
||||
return scenario
|
||||
|
||||
|
||||
@@ -96,18 +90,8 @@ def update_scenario(
|
||||
):
|
||||
scenario = get_scenario_or_404(scenario_id, db, user, edit=True)
|
||||
data = payload.model_dump(exclude_unset=True)
|
||||
script_ids = data.pop("script_ids", None)
|
||||
for field, value in data.items():
|
||||
setattr(scenario, field, value)
|
||||
if script_ids is not None:
|
||||
scripts = (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.id.in_(script_ids), models.Script.user_id == user.id)
|
||||
.all()
|
||||
)
|
||||
if len(scripts) != len(set(script_ids)):
|
||||
raise HTTPException(404, "One or more scripts not found")
|
||||
scenario.scripts = sorted(scripts, key=lambda s: script_ids.index(s.id))
|
||||
db.commit()
|
||||
return scenario
|
||||
|
||||
@@ -148,13 +132,6 @@ def export_scenario(
|
||||
{"type": c.type, "name": c.name, "keys": c.keys, "entry": c.entry, "notes": c.notes}
|
||||
for c in s.story_cards
|
||||
],
|
||||
"scripts": [
|
||||
{
|
||||
"name": sc.name, "description": sc.description, "library": sc.library_js,
|
||||
"input": sc.input_js, "context": sc.context_js, "output": sc.output_js,
|
||||
}
|
||||
for sc in s.scripts
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@@ -185,7 +162,6 @@ def import_scenario(
|
||||
):
|
||||
"""Accepts our export format and AI Dungeon scenario exports best-effort;
|
||||
reports any keys it didn't understand."""
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("scenarios", db, user)
|
||||
fields: dict = {}
|
||||
unmapped: list[str] = []
|
||||
@@ -250,21 +226,6 @@ def import_scenario(
|
||||
)
|
||||
)
|
||||
|
||||
for item in bundle.get("scripts") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
script = models.Script(
|
||||
user_id=user.id,
|
||||
name=str(item.get("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=str(item.get("description") or ""),
|
||||
library_js=str(item.get("library") or item.get("sharedLibrary") or ""),
|
||||
input_js=str(item.get("input") or item.get("onInput") or ""),
|
||||
context_js=str(item.get("context") or item.get("onModelContext") or ""),
|
||||
output_js=str(item.get("output") or item.get("onOutput") or ""),
|
||||
)
|
||||
db.add(script)
|
||||
db.flush()
|
||||
scenario.scripts.append(script)
|
||||
|
||||
db.commit()
|
||||
out = schemas.ScenarioOut.model_validate(scenario).model_dump(mode="json")
|
||||
|
||||
@@ -1,159 +0,0 @@
|
||||
from fastapi import APIRouter, Body, Depends, HTTPException, Request
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import auth, limits, models, schemas
|
||||
from ..database import get_db
|
||||
from ..scripting import run_hook
|
||||
|
||||
router = APIRouter(prefix="/api/scripts", tags=["scripts"])
|
||||
|
||||
HOOK_FIELDS = {"input": "input_js", "context": "context_js", "output": "output_js"}
|
||||
|
||||
|
||||
def get_script_or_404(script_id: int, db: Session, user: models.User) -> models.Script:
|
||||
script = db.get(models.Script, script_id)
|
||||
if script is None or script.user_id != user.id:
|
||||
raise HTTPException(404, "Script not found")
|
||||
return script
|
||||
|
||||
|
||||
@router.get("", response_model=list[schemas.ScriptOut])
|
||||
def list_scripts(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
return (
|
||||
db.query(models.Script)
|
||||
.filter(models.Script.user_id == user.id)
|
||||
.order_by(models.Script.updated_at.desc())
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
@router.post("", response_model=schemas.ScriptOut, status_code=201)
|
||||
def create_script(
|
||||
payload: schemas.ScriptCreate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
limits.check_row_cap("scripts", db, user)
|
||||
script = models.Script(**payload.model_dump(), user_id=user.id)
|
||||
db.add(script)
|
||||
db.commit()
|
||||
return script
|
||||
|
||||
|
||||
@router.get("/{script_id}", response_model=schemas.ScriptOut)
|
||||
def get_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
return get_script_or_404(script_id, db, user)
|
||||
|
||||
|
||||
@router.patch("/{script_id}", response_model=schemas.ScriptOut)
|
||||
def update_script(
|
||||
script_id: int,
|
||||
payload: schemas.ScriptUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
for field, value in payload.model_dump(exclude_unset=True).items():
|
||||
setattr(script, field, value)
|
||||
db.commit()
|
||||
return script
|
||||
|
||||
|
||||
@router.delete("/{script_id}", status_code=204)
|
||||
def delete_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
db.delete(get_script_or_404(script_id, db, user))
|
||||
db.commit()
|
||||
|
||||
|
||||
@router.post("/{script_id}/test")
|
||||
def test_script(
|
||||
script_id: int,
|
||||
payload: schemas.ScriptTestRequest,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Runs one hook against sample text, making no AI call and storing nothing."""
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
limits.rate_limit("script-test", request, user)
|
||||
result = run_hook(
|
||||
script.library_js,
|
||||
getattr(script, HOOK_FIELDS[payload.hook]),
|
||||
payload.text,
|
||||
payload.state,
|
||||
history=[],
|
||||
story_cards=[],
|
||||
info={"actionCount": 0, "characterNames": [], "memoryLength": 0, "maxChars": 0},
|
||||
)
|
||||
return {
|
||||
"text": result.text,
|
||||
"stop": result.stop,
|
||||
"state": result.state,
|
||||
"storyCards": result.story_cards,
|
||||
"logs": result.logs,
|
||||
"error": result.error,
|
||||
}
|
||||
|
||||
|
||||
# ---------- Import / Export ----------
|
||||
|
||||
@router.get("/{script_id}/export")
|
||||
def export_script(
|
||||
script_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""JSON bundle matching how AI Dungeon scripts circulate."""
|
||||
script = get_script_or_404(script_id, db, user)
|
||||
return {
|
||||
"name": script.name,
|
||||
"description": script.description,
|
||||
"library": script.library_js,
|
||||
"input": script.input_js,
|
||||
"context": script.context_js,
|
||||
"output": script.output_js,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/import", response_model=schemas.ScriptOut, status_code=201)
|
||||
def import_script(
|
||||
request: Request,
|
||||
bundle: dict = Body(...),
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Accepts our export bundle; tolerates *_js key names too."""
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_row_cap("scripts", db, user)
|
||||
def pick(*keys: str) -> str:
|
||||
for key in keys:
|
||||
value = bundle.get(key)
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
return ""
|
||||
|
||||
script = models.Script(
|
||||
user_id=user.id,
|
||||
# A raw-dict import bypasses the schemas, so truncate to the VARCHAR
|
||||
# width.
|
||||
name=(pick("name") or "Imported Script")[:schemas.NAME_MAX],
|
||||
description=pick("description"),
|
||||
library_js=pick("library", "library_js", "sharedLibrary"),
|
||||
input_js=pick("input", "input_js", "onInput"),
|
||||
context_js=pick("context", "context_js", "onModelContext"),
|
||||
output_js=pick("output", "output_js", "onOutput"),
|
||||
)
|
||||
db.add(script)
|
||||
db.commit()
|
||||
return script
|
||||
@@ -1,20 +1,33 @@
|
||||
"""The model settings, and the connection test that tells you why they don't work.
|
||||
|
||||
There is one settings row, belonging to the one local user. It describes an
|
||||
Ollama: where it is, which model to narrate with, which to embed with, and how
|
||||
long to wait for it.
|
||||
|
||||
Upstream let this row name any OpenAI-compatible endpoint and carry an
|
||||
encrypted API key for it. M2 narrowed both: `endpoints.py` decides which
|
||||
addresses may be named, and there is no key field, because Ollama does not use
|
||||
one and this build has no cloud provider to carry a key for.
|
||||
"""
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from .. import auth, limits, models, netguard, schemas, security, tlstrust
|
||||
from .. import auth, endpoints, models, schemas, tlstrust
|
||||
from ..database import get_db
|
||||
from ..providers.openai_compatible import CONNECT_TIMEOUT
|
||||
|
||||
router = APIRouter(prefix="/api/settings", tags=["settings"])
|
||||
|
||||
#: The connection test is a listing, not a generation, so it never waits on a
|
||||
#: model load and does not need the turn engine's patience.
|
||||
TEST_TIMEOUT = 15.0
|
||||
|
||||
|
||||
def get_settings(db: Session, user: models.User) -> models.Settings:
|
||||
"""Returns the user's settings row, creating it on first access.
|
||||
|
||||
Phase 8 made settings per user rather than global. They cover the endpoint,
|
||||
the key, the models, and the memory configuration.
|
||||
"""
|
||||
"""Returns the settings row, creating it on first access."""
|
||||
settings = (
|
||||
db.query(models.Settings).filter(models.Settings.user_id == user.id).first()
|
||||
)
|
||||
@@ -34,16 +47,24 @@ def read_settings(
|
||||
|
||||
|
||||
@router.put("", response_model=schemas.SettingsOut)
|
||||
def update_settings(
|
||||
async def update_settings(
|
||||
payload: schemas.SettingsUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
settings = get_settings(db, user)
|
||||
fields = payload.model_dump(exclude_unset=True)
|
||||
# Write-only API key: absent = unchanged, "" = cleared, else encrypted.
|
||||
if "api_key" in fields:
|
||||
fields["api_key"] = security.encrypt_secret(fields["api_key"].strip())
|
||||
|
||||
if "endpoint_url" in fields:
|
||||
# Refused here so the user finds out while they are looking at the
|
||||
# field, rather than on their next turn. The provider re-checks before
|
||||
# every request regardless; this is the friendly half of the same rule.
|
||||
reason = await run_in_threadpool(
|
||||
endpoints.rejection_reason, fields["endpoint_url"]
|
||||
)
|
||||
if reason is not None:
|
||||
raise HTTPException(400, f"That endpoint can't be used — {reason}.")
|
||||
|
||||
embedding_model_changed = (
|
||||
"embedding_model" in fields
|
||||
and fields["embedding_model"] != settings.embedding_model
|
||||
@@ -53,8 +74,6 @@ def update_settings(
|
||||
if embedding_model_changed:
|
||||
# Vectors from the old model have a different dimensionality/space;
|
||||
# clear them so the post-turn task re-embeds with the new model.
|
||||
# This covers only this user's adventures, because settings are per
|
||||
# user now.
|
||||
#
|
||||
# Both columns, and the flag. This is the one place that clears vectors
|
||||
# in bulk rather than through memorybank.set_vector, and when the
|
||||
@@ -80,28 +99,61 @@ def update_settings(
|
||||
return settings
|
||||
|
||||
|
||||
async def list_endpoint_models(cfg: auth.ProviderConfig) -> dict:
|
||||
"""Fetches the endpoint's /models listing.
|
||||
async def list_endpoint_models(endpoint_url: str) -> dict:
|
||||
"""Fetches the endpoint's `/models` listing, and doubles as the connection test.
|
||||
|
||||
The call also serves as a connectivity check, so a failure returns
|
||||
`{"ok": False, "detail": ...}` rather than raising.
|
||||
Returns `{"ok": False, "detail": ...}` rather than raising, because every
|
||||
caller wants to show the reason rather than fail the page.
|
||||
|
||||
The failure cases are told apart on purpose. "Ollama isn't running", "that
|
||||
address isn't allowed", "the certificate doesn't verify" and "it answered,
|
||||
but with an error" need four different things done about them, and a single
|
||||
"connection failed" leaves the user guessing which they have.
|
||||
"""
|
||||
# SSRF guard. Never probe a non-public address the user supplied.
|
||||
reason = await run_in_threadpool(netguard.endpoint_block_reason, cfg.endpoint_url)
|
||||
if reason:
|
||||
return {"ok": False, "detail": f"Can't reach that endpoint — {reason}."}
|
||||
url = cfg.endpoint_url.rstrip("/") + "/models"
|
||||
headers = {}
|
||||
if cfg.api_key:
|
||||
headers["Authorization"] = f"Bearer {cfg.api_key}"
|
||||
reason = await run_in_threadpool(endpoints.rejection_reason, endpoint_url)
|
||||
if reason is not None:
|
||||
return {
|
||||
"ok": False, "kind": "rejected",
|
||||
"detail": f"That endpoint can't be used — {reason}.",
|
||||
}
|
||||
|
||||
url = endpoint_url.rstrip("/") + "/models"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10, verify=tlstrust.ssl_context()) as client:
|
||||
resp = await client.get(url, headers=headers)
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(TEST_TIMEOUT, connect=CONNECT_TIMEOUT),
|
||||
verify=tlstrust.ssl_context(),
|
||||
) as client:
|
||||
resp = await client.get(url)
|
||||
except httpx.ConnectError as exc:
|
||||
# A TLS failure arrives as a ConnectError too, and it needs a different
|
||||
# answer from "nothing is listening": install the CA, don't start Ollama.
|
||||
if "CERTIFICATE_VERIFY" in str(exc).upper() or "SSL" in str(exc).upper():
|
||||
return {
|
||||
"ok": False, "kind": "tls",
|
||||
"detail": (
|
||||
"The endpoint's TLS certificate could not be verified. If it "
|
||||
"uses a private or self-signed CA, install that CA on this "
|
||||
"machine so the system trusts it. Certificate checking is "
|
||||
"not optional."
|
||||
),
|
||||
}
|
||||
return {
|
||||
"ok": False, "kind": "unreachable",
|
||||
"detail": f"Could not connect to {endpoint_url} — is Ollama running there?",
|
||||
}
|
||||
except httpx.TimeoutException:
|
||||
return {
|
||||
"ok": False, "kind": "timeout",
|
||||
"detail": f"{endpoint_url} did not answer within {TEST_TIMEOUT:.0f}s.",
|
||||
}
|
||||
except httpx.HTTPError as exc:
|
||||
return {"ok": False, "detail": f"Connection failed: {exc}"}
|
||||
return {"ok": False, "kind": "error", "detail": f"Connection failed: {exc}"}
|
||||
|
||||
if resp.status_code != 200:
|
||||
return {"ok": False, "detail": f"HTTP {resp.status_code}: {resp.text[:300]}"}
|
||||
return {
|
||||
"ok": False, "kind": "http",
|
||||
"detail": f"HTTP {resp.status_code}: {resp.text[:300]}",
|
||||
}
|
||||
|
||||
models_available: list[str] = []
|
||||
try:
|
||||
@@ -115,15 +167,19 @@ async def list_endpoint_models(cfg: auth.ProviderConfig) -> dict:
|
||||
|
||||
@router.post("/test")
|
||||
async def test_connection(
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Runs a cheap connectivity check against whatever the turn engine would use.
|
||||
|
||||
That includes the shared demo endpoint, when the user has no key of their
|
||||
own.
|
||||
"""
|
||||
limits.rate_limit("connection-test", request, user)
|
||||
"""Checks the endpoint the turn engine would use, and lists its models."""
|
||||
settings = get_settings(db, user)
|
||||
return await list_endpoint_models(auth.resolve_provider_config(settings))
|
||||
result = await list_endpoint_models(settings.endpoint_url)
|
||||
if result.get("ok") and settings.model and settings.model not in result["models"]:
|
||||
# Reachable, but pointed at a model that is not installed there — the
|
||||
# commonest way for a correct endpoint to still fail every turn.
|
||||
return result | {
|
||||
"warning": (
|
||||
f"{settings.endpoint_url} is reachable, but has no model named "
|
||||
f"'{settings.model}'. Pull it there, or pick one from the list."
|
||||
)
|
||||
}
|
||||
return result
|
||||
|
||||
@@ -159,7 +159,6 @@ def import_story_cards(
|
||||
raise HTTPException(422, 'Expected a "cards" array of story cards.')
|
||||
cards_in = [c for c in cards_in if isinstance(c, dict)]
|
||||
|
||||
limits.rate_limit("import", request, user)
|
||||
limits.check_bundle_lists(story_cards=cards_in)
|
||||
existing = len(owner.story_cards)
|
||||
if auth.MULTI_USER and existing + len(cards_in) > limits.MAX_STORY_CARDS_PER_OWNER:
|
||||
|
||||
+5
-76
@@ -14,7 +14,6 @@ NAME_MAX = 200 # Titles and names. VARCHAR(200).
|
||||
TAGS_MAX = 500 # VARCHAR(500).
|
||||
CARD_TYPE_MAX = 100 # VARCHAR(100).
|
||||
PROSE_MAX = 50_000 # Memory, author's note, prompts, entries, and notes.
|
||||
SCRIPT_MAX = 200_000 # One JavaScript source file.
|
||||
ACTION_MAX = 20_000 # One player action.
|
||||
MEMORY_TEXT_MAX = 5_000
|
||||
# A scenario cover image, stored inline as a base64 data URI. A 400x300 WebP at
|
||||
@@ -31,7 +30,6 @@ Name = Annotated[str, Field(max_length=NAME_MAX)]
|
||||
Tags = Annotated[str, Field(max_length=TAGS_MAX)]
|
||||
CardType = Annotated[str, Field(max_length=CARD_TYPE_MAX)]
|
||||
Prose = Annotated[str, Field(max_length=PROSE_MAX)]
|
||||
ScriptSource = Annotated[str, Field(max_length=SCRIPT_MAX)]
|
||||
ActionText = Annotated[str, Field(max_length=ACTION_MAX)]
|
||||
Image = Annotated[str, Field(max_length=IMAGE_MAX)]
|
||||
Icon = Annotated[str, Field(max_length=ICON_MAX)]
|
||||
@@ -106,7 +104,6 @@ class ScenarioUpdate(BaseModel):
|
||||
image: Image | None = None
|
||||
icon: Icon | None = None
|
||||
stat_schema: dict | None = None
|
||||
script_ids: list[int] | None = None
|
||||
|
||||
|
||||
class ScenarioOut(ORMModel, ScenarioBase):
|
||||
@@ -115,7 +112,6 @@ class ScenarioOut(ORMModel, ScenarioBase):
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
story_cards: list[StoryCardOut] = []
|
||||
scripts: list["ScriptOut"] = []
|
||||
|
||||
|
||||
class ScenarioListItem(ORMModel):
|
||||
@@ -374,67 +370,6 @@ class AdventureListItem(ORMModel):
|
||||
icon: str = ""
|
||||
|
||||
|
||||
# ---------- Scripts ----------
|
||||
|
||||
class ScriptBase(BaseModel):
|
||||
name: Name = "Untitled Script"
|
||||
description: Prose = ""
|
||||
library_js: ScriptSource = ""
|
||||
input_js: ScriptSource = ""
|
||||
context_js: ScriptSource = ""
|
||||
output_js: ScriptSource = ""
|
||||
|
||||
|
||||
class ScriptCreate(ScriptBase):
|
||||
pass
|
||||
|
||||
|
||||
class ScriptUpdate(BaseModel):
|
||||
name: Name | None = None
|
||||
description: Prose | None = None
|
||||
library_js: ScriptSource | None = None
|
||||
input_js: ScriptSource | None = None
|
||||
context_js: ScriptSource | None = None
|
||||
output_js: ScriptSource | None = None
|
||||
|
||||
|
||||
class ScriptOut(ORMModel, ScriptBase):
|
||||
id: int
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class ScriptTestRequest(BaseModel):
|
||||
hook: Literal["input", "context", "output"]
|
||||
text: Prose = ""
|
||||
state: dict = {}
|
||||
|
||||
|
||||
class AdventureScriptOut(ORMModel):
|
||||
id: int
|
||||
adventure_id: int
|
||||
position: int
|
||||
enabled: bool
|
||||
name: str
|
||||
description: str
|
||||
library_js: str
|
||||
input_js: str
|
||||
context_js: str
|
||||
output_js: str
|
||||
# The router sets this field, which is not stored. It is `True` when a
|
||||
# syncable library version exists whose code differs from this copy, and
|
||||
# `None` when there is nothing to sync from.
|
||||
out_of_date: bool | None = None
|
||||
|
||||
|
||||
class AdventureScriptUpdate(BaseModel):
|
||||
enabled: bool | None = None
|
||||
library_js: ScriptSource | None = None
|
||||
input_js: ScriptSource | None = None
|
||||
context_js: ScriptSource | None = None
|
||||
output_js: ScriptSource | None = None
|
||||
|
||||
|
||||
# ---------- Auth (Phase 8) ----------
|
||||
|
||||
class AuthCredentials(BaseModel):
|
||||
@@ -448,15 +383,12 @@ class AuthCredentials(BaseModel):
|
||||
|
||||
class SettingsOut(ORMModel):
|
||||
endpoint_url: str
|
||||
# The key itself is never returned. It is encrypted at rest and
|
||||
# write-only.
|
||||
has_api_key: bool
|
||||
model: str
|
||||
api_mode: str
|
||||
temperature: float
|
||||
max_output_tokens: int
|
||||
reasoning_max_tokens: int
|
||||
context_token_budget: int
|
||||
model_timeout_seconds: int
|
||||
narrator_prompt: str
|
||||
summary_model: str
|
||||
embedding_model: str
|
||||
@@ -491,18 +423,15 @@ class ChatRequest(BaseModel):
|
||||
|
||||
class SettingsUpdate(BaseModel):
|
||||
endpoint_url: Annotated[str, Field(max_length=500)] | None = None # VARCHAR(500).
|
||||
# Encryption expands the stored value by about four thirds into the same
|
||||
# VARCHAR(500), so 256 plaintext characters is the largest safe input. The
|
||||
# stored form is "enc:" plus Fernet plus base64.
|
||||
api_key: Annotated[str, Field(max_length=256)] | None = None
|
||||
model: Name | None = None
|
||||
api_mode: Annotated[str, Field(max_length=20)] | None = None
|
||||
temperature: Annotated[float, Field(ge=0, le=5)] | None = None
|
||||
max_output_tokens: Annotated[int, Field(ge=1, le=100_000)] | None = None
|
||||
# A value of -1 turns reasoning off explicitly, which sends
|
||||
# `reasoning: {effort: none}`. A value of 0 sends nothing.
|
||||
reasoning_max_tokens: Annotated[int, Field(ge=-1, le=100_000)] | None = None
|
||||
context_token_budget: Annotated[int, Field(ge=256, le=200_000)] | None = None
|
||||
# Seconds to wait for the model. The floor is high enough that a normal
|
||||
# turn cannot trip it; the ceiling exists so that "wait longer" stays a
|
||||
# number rather than becoming "wait forever".
|
||||
model_timeout_seconds: Annotated[int, Field(ge=30, le=3600)] | None = None
|
||||
narrator_prompt: Prose | None = None
|
||||
summary_model: Name | None = None
|
||||
embedding_model: Name | None = None
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
from .engine import HookResult, run_hook
|
||||
from .pipeline import ScriptPipeline
|
||||
|
||||
__all__ = ["HookResult", "ScriptPipeline", "run_hook"]
|
||||
@@ -1,146 +0,0 @@
|
||||
"""AI Dungeon-compatible script execution in an embedded QuickJS sandbox.
|
||||
|
||||
Each hook run is fully isolated (fresh Context), capped at 16 MB memory and
|
||||
2 seconds CPU, with no filesystem/network/process access (QuickJS has none by
|
||||
default). Scripts follow the AI Dungeon contract: define a `modifier(text)`
|
||||
and call it as the last line; its return value `{ text, stop }` is the result.
|
||||
"""
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import quickjs
|
||||
|
||||
MEMORY_LIMIT = 16 * 1024 * 1024
|
||||
TIME_LIMIT_SECONDS = 2
|
||||
HISTORY_WINDOW = 100 # recent actions exposed as `history`
|
||||
|
||||
# Globals per the official docs: text, state, history, storyCards, info,
|
||||
# log/console.log, story card functions, plus legacy worldInfo aliases.
|
||||
PRELUDE = """
|
||||
"use strict";
|
||||
var __logs = [];
|
||||
var state = __DATA__.state;
|
||||
var text = __DATA__.text;
|
||||
var history = __DATA__.history;
|
||||
var storyCards = __DATA__.storyCards;
|
||||
var info = __DATA__.info;
|
||||
|
||||
function log(msg) {
|
||||
__logs.push(typeof msg === "string" ? msg : JSON.stringify(msg));
|
||||
}
|
||||
var console = { log: log };
|
||||
|
||||
// Returns the new card's index, or false if a card with those keys exists —
|
||||
// matching real AI Dungeon. Note index 0 is falsy; that quirk is upstream's.
|
||||
function addStoryCard(keys, entry, type) {
|
||||
for (var i = 0; i < storyCards.length; i++) {
|
||||
if (storyCards[i].keys === keys) return false;
|
||||
}
|
||||
storyCards.push({ id: null, keys: keys || "", entry: entry || "", type: type || "" });
|
||||
return storyCards.length - 1;
|
||||
}
|
||||
function updateStoryCard(index, keys, entry, type) {
|
||||
var card = storyCards[index];
|
||||
if (!card) throw new Error("Story card not found");
|
||||
card.keys = keys;
|
||||
card.entry = entry;
|
||||
card.type = type;
|
||||
}
|
||||
function removeStoryCard(index) {
|
||||
if (!storyCards[index]) throw new Error("Story card not found");
|
||||
storyCards.splice(index, 1);
|
||||
}
|
||||
|
||||
// Legacy aliases used by older AI Dungeon scripts.
|
||||
var worldInfo = storyCards;
|
||||
var worldEntries = storyCards;
|
||||
function addWorldEntry(keys, entry) { return addStoryCard(keys, entry, ""); }
|
||||
function updateWorldEntry(index, keys, entry) {
|
||||
var card = storyCards[index];
|
||||
if (!card) throw new Error("World entry not found");
|
||||
card.keys = keys;
|
||||
card.entry = entry;
|
||||
}
|
||||
function removeWorldEntry(index) { return removeStoryCard(index); }
|
||||
"""
|
||||
|
||||
COLLECT = """
|
||||
JSON.stringify({
|
||||
result: (typeof __result === "undefined" || __result === null) ? null : __result,
|
||||
state: state,
|
||||
storyCards: storyCards,
|
||||
logs: __logs
|
||||
})
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class HookResult:
|
||||
text: str
|
||||
stop: bool = False
|
||||
state: dict = field(default_factory=dict)
|
||||
story_cards: list = field(default_factory=list)
|
||||
logs: list = field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
def run_hook(
|
||||
library_js: str,
|
||||
hook_js: str,
|
||||
text: str,
|
||||
state: dict,
|
||||
history: list[dict],
|
||||
story_cards: list[dict],
|
||||
info: dict,
|
||||
) -> HookResult:
|
||||
"""Run one modifier hook. This function never raises. Failures return as
|
||||
`.error` with text, state, and cards unchanged, so a bad script cannot
|
||||
break a turn."""
|
||||
unchanged = HookResult(text=text, state=state, story_cards=story_cards)
|
||||
source = f"{library_js}\n;\n{hook_js}" if library_js.strip() else hook_js
|
||||
if not source.strip():
|
||||
return unchanged
|
||||
|
||||
data = {
|
||||
"state": state,
|
||||
"text": text,
|
||||
"history": history[-HISTORY_WINDOW:],
|
||||
"storyCards": story_cards,
|
||||
"info": info,
|
||||
}
|
||||
try:
|
||||
ctx = quickjs.Context()
|
||||
ctx.set_memory_limit(MEMORY_LIMIT)
|
||||
ctx.set_time_limit(TIME_LIMIT_SECONDS)
|
||||
ctx.eval(f"var __DATA__ = {json.dumps(data)};")
|
||||
ctx.eval(PRELUDE)
|
||||
ctx.eval(f"var __SRC__ = {json.dumps(source)};")
|
||||
# Indirect eval keeps the script in global scope, so `modifier(text)` as the
|
||||
# script's final expression statement becomes the completion value.
|
||||
ctx.eval("var __result = (0, eval)(__SRC__);")
|
||||
collected = json.loads(ctx.eval(COLLECT))
|
||||
except quickjs.JSException as exc:
|
||||
unchanged.error = f"Script error: {exc}"
|
||||
return unchanged
|
||||
except Exception as exc: # memory limit, invalid JSON state, engine faults
|
||||
unchanged.error = f"Script execution failed: {exc}"
|
||||
return unchanged
|
||||
|
||||
result = collected.get("result")
|
||||
new_text, stop = text, False
|
||||
if isinstance(result, dict):
|
||||
if isinstance(result.get("text"), str):
|
||||
new_text = result["text"]
|
||||
stop = bool(result.get("stop"))
|
||||
elif isinstance(result, str):
|
||||
new_text = result
|
||||
|
||||
new_state = collected.get("state")
|
||||
return HookResult(
|
||||
text=new_text,
|
||||
stop=stop,
|
||||
state=new_state if isinstance(new_state, dict) else {},
|
||||
story_cards=collected.get("storyCards") or [],
|
||||
logs=collected.get("logs") or [],
|
||||
)
|
||||
@@ -1,112 +0,0 @@
|
||||
"""Runs an adventure's enabled scripts through a turn's hook points, applying
|
||||
state and story-card mutations back to the database after each hook."""
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import models
|
||||
from ..context import history as context_history
|
||||
from .engine import run_hook
|
||||
|
||||
MAX_STORY_CARDS = 5000 # AI Dungeon's per-adventure sanity cap
|
||||
|
||||
|
||||
class ScriptPipeline:
|
||||
def __init__(self, adventure: models.Adventure, db: Session):
|
||||
self.adventure = adventure
|
||||
self.db = db
|
||||
self.logs: list[str] = []
|
||||
self.errors: list[str] = []
|
||||
|
||||
@property
|
||||
def message(self) -> str | None:
|
||||
state = self.adventure.script_state
|
||||
msg = state.get("message") if isinstance(state, dict) else None
|
||||
return msg if isinstance(msg, str) and msg.strip() else None
|
||||
|
||||
def _history(self) -> list[dict]:
|
||||
# Read the path rather than `adventure.actions`. That collection holds
|
||||
# every branch's actions, and this is the documented history API a user
|
||||
# script reads. Giving a script the siblings of the turn it is running on
|
||||
# would be the same bug as building a prompt from them, and visible to
|
||||
# the user.
|
||||
#
|
||||
# `story_actions` also drops rows with blank text, which
|
||||
# `adventure.actions` kept, so this array is shorter than it was for an
|
||||
# adventure that has any such rows. `info.actionCount` counts the same
|
||||
# way. That is intended. A row with no text is this app's bookkeeping, it
|
||||
# has no counterpart in the AI Dungeon history a ported script was
|
||||
# written against, and the prompt has never included one. A script keyed
|
||||
# on every N actions lands on different turns than it did before phase
|
||||
# 14, and no reading of this is compatible with both.
|
||||
return [
|
||||
{"text": a.text, "rawText": a.text, "type": a.type}
|
||||
for a in context_history.story_actions(self.adventure)
|
||||
]
|
||||
|
||||
def _cards(self) -> list[dict]:
|
||||
return [
|
||||
{"id": c.id, "keys": c.keys, "entry": c.entry, "type": c.type}
|
||||
for c in self.adventure.story_cards
|
||||
]
|
||||
|
||||
def _info(self) -> dict:
|
||||
return {
|
||||
"actionCount": context_history.count(self.adventure),
|
||||
"characterNames": [],
|
||||
"memoryLength": len(self.adventure.memory),
|
||||
"maxChars": 0,
|
||||
}
|
||||
|
||||
def _apply_cards(self, returned: list) -> None:
|
||||
existing = {c.id: c for c in self.adventure.story_cards}
|
||||
seen_ids = set()
|
||||
added = 0
|
||||
for item in returned:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
card_id = item.get("id")
|
||||
keys = str(item.get("keys") or "")
|
||||
entry = str(item.get("entry") or "")
|
||||
card_type = str(item.get("type") or "")
|
||||
if card_id in existing:
|
||||
seen_ids.add(card_id)
|
||||
card = existing[card_id]
|
||||
card.keys, card.entry, card.type = keys, entry, card_type
|
||||
elif len(existing) + added < MAX_STORY_CARDS:
|
||||
self.db.add(
|
||||
models.StoryCard(
|
||||
adventure_id=self.adventure.id,
|
||||
keys=keys, entry=entry, type=card_type,
|
||||
)
|
||||
)
|
||||
added += 1
|
||||
for card_id, card in existing.items():
|
||||
if card_id not in seen_ids:
|
||||
self.db.delete(card)
|
||||
|
||||
def run(self, hook: str, text: str) -> tuple[str, bool]:
|
||||
"""Chain `hook` across all enabled scripts. Returns (text, stop)."""
|
||||
state = self.adventure.script_state if isinstance(self.adventure.script_state, dict) else {}
|
||||
for script in self.adventure.scripts:
|
||||
hook_js = getattr(script, f"{hook}_js")
|
||||
if not script.enabled or not hook_js.strip():
|
||||
continue
|
||||
result = run_hook(
|
||||
script.library_js, hook_js, text, state,
|
||||
self._history(), self._cards(), self._info(),
|
||||
)
|
||||
if result.error:
|
||||
self.errors.append(f"{script.name} ({hook}): {result.error}")
|
||||
continue # a broken script never breaks the turn
|
||||
self.logs.extend(f"[{script.name}/{hook}] {line}" for line in result.logs)
|
||||
self._apply_cards(result.story_cards)
|
||||
state = result.state
|
||||
self.adventure.script_state = state
|
||||
self.db.commit()
|
||||
text = result.text
|
||||
if result.stop:
|
||||
return text, True
|
||||
return text, False
|
||||
|
||||
def report(self) -> dict:
|
||||
return {"logs": self.logs, "errors": self.errors, "message": self.message}
|
||||
@@ -1,131 +0,0 @@
|
||||
"""Phase 8: secrets and crypto primitives for optional accounts.
|
||||
|
||||
Everything derives from one server-side secret:
|
||||
|
||||
* Session cookies are HMAC-signed with it.
|
||||
* Stored LLM API keys are Fernet-encrypted with a key derived from it.
|
||||
|
||||
The secret comes from `AIDND_SECRET_KEY`, or it is generated once into
|
||||
`secret.key` next to the database, so a local install and a Docker volume work
|
||||
with no configuration. Losing that file logs everyone out and makes the stored
|
||||
API keys unreadable, and users then re-enter them. A multi-user deployment has
|
||||
to set the environment variable, because a hosted filesystem is ephemeral and a
|
||||
`secret.key` regenerated on every deploy would log out every user each time.
|
||||
|
||||
Passwords use `hashlib.scrypt`, which is in the standard library and backed by
|
||||
OpenSSL, so this needs no separate hashing dependency.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
|
||||
from cryptography.fernet import Fernet, InvalidToken
|
||||
|
||||
from .database import DB_PATH
|
||||
|
||||
_SECRET_FILE = DB_PATH.parent / "secret.key"
|
||||
|
||||
|
||||
def _load_secret() -> bytes:
|
||||
env = os.environ.get("AIDND_SECRET_KEY", "").strip()
|
||||
if env:
|
||||
return env.encode()
|
||||
# Same flag parse as auth.MULTI_USER (auth imports this module, so it
|
||||
# can't be imported from there).
|
||||
if os.environ.get("AIDND_MULTI_USER", "").strip().lower() in ("1", "true", "yes", "on"):
|
||||
raise RuntimeError(
|
||||
"AIDND_SECRET_KEY must be set when AIDND_MULTI_USER is on: an "
|
||||
"auto-generated secret.key on an ephemeral hosted filesystem would "
|
||||
"rotate on every deploy, logging out every user and orphaning "
|
||||
"their stored API keys. Generate one with: "
|
||||
"python -c \"import secrets; print(secrets.token_urlsafe(48))\""
|
||||
)
|
||||
if _SECRET_FILE.exists():
|
||||
return _SECRET_FILE.read_bytes().strip()
|
||||
secret = secrets.token_urlsafe(48).encode()
|
||||
_SECRET_FILE.write_bytes(secret)
|
||||
return secret
|
||||
|
||||
|
||||
SECRET_KEY = _load_secret()
|
||||
_fernet = Fernet(base64.urlsafe_b64encode(hashlib.sha256(SECRET_KEY).digest()))
|
||||
|
||||
|
||||
# ---------- Password hashing (scrypt) ----------
|
||||
|
||||
_SCRYPT_N, _SCRYPT_R, _SCRYPT_P = 2**14, 8, 1
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
salt = secrets.token_bytes(16)
|
||||
key = hashlib.scrypt(
|
||||
password.encode(), salt=salt, n=_SCRYPT_N, r=_SCRYPT_R, p=_SCRYPT_P
|
||||
)
|
||||
return f"scrypt${_SCRYPT_N}${_SCRYPT_R}${_SCRYPT_P}${salt.hex()}${key.hex()}"
|
||||
|
||||
|
||||
def verify_password(password: str, stored: str) -> bool:
|
||||
try:
|
||||
scheme, n, r, p, salt_hex, key_hex = stored.split("$")
|
||||
if scheme != "scrypt":
|
||||
return False
|
||||
key = hashlib.scrypt(
|
||||
password.encode(), salt=bytes.fromhex(salt_hex),
|
||||
n=int(n), r=int(r), p=int(p),
|
||||
)
|
||||
return hmac.compare_digest(key, bytes.fromhex(key_hex))
|
||||
except (ValueError, AttributeError):
|
||||
return False
|
||||
|
||||
|
||||
# ---------- Session tokens ----------
|
||||
# The token is "v1.<user_id>.<hmac>". It does not expire, because a long-lived
|
||||
# guest session is what this is for.
|
||||
|
||||
def sign_session(user_id: int) -> str:
|
||||
payload = f"v1.{user_id}"
|
||||
sig = hmac.new(SECRET_KEY, payload.encode(), hashlib.sha256).hexdigest()
|
||||
return f"{payload}.{sig}"
|
||||
|
||||
|
||||
def verify_session(token: str) -> int | None:
|
||||
try:
|
||||
version, user_id, sig = token.split(".")
|
||||
if version != "v1":
|
||||
return None
|
||||
payload = f"{version}.{user_id}"
|
||||
expected = hmac.new(SECRET_KEY, payload.encode(), hashlib.sha256).hexdigest()
|
||||
if not hmac.compare_digest(sig, expected):
|
||||
return None
|
||||
return int(user_id)
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
# ---------- API-key encryption at rest ----------
|
||||
# Stored values carry an "enc:" prefix so plaintext keys from pre-Phase-8
|
||||
# databases can be recognized and migrated.
|
||||
|
||||
ENC_PREFIX = "enc:"
|
||||
|
||||
|
||||
def encrypt_secret(plain: str) -> str:
|
||||
if not plain:
|
||||
return ""
|
||||
return ENC_PREFIX + _fernet.encrypt(plain.encode()).decode()
|
||||
|
||||
|
||||
def decrypt_secret(stored: str) -> str:
|
||||
"""Returns the plaintext key. Tolerates legacy plaintext values (returned
|
||||
as-is) and undecryptable tokens (secret rotated → treated as unset)."""
|
||||
if not stored:
|
||||
return ""
|
||||
if not stored.startswith(ENC_PREFIX):
|
||||
return stored
|
||||
try:
|
||||
return _fernet.decrypt(stored[len(ENC_PREFIX):].encode()).decode()
|
||||
except (InvalidToken, ValueError):
|
||||
return ""
|
||||
+4
-41
@@ -4,8 +4,7 @@ Every JSON file in ``seed_data/`` describes one demo scenario in the same
|
||||
model-native shape the export endpoint produces. Seeded scenarios have a NULL
|
||||
owner and ``is_public=True``, so every visitor (including guests) sees them and
|
||||
can start an adventure from them, while nobody can edit them. Starting an
|
||||
adventure copies the scenario's story cards and scripts into the adventure, so
|
||||
the seeded scripts run for guests too.
|
||||
adventure copies the scenario's story cards into the adventure.
|
||||
|
||||
Seed files are the source of truth for demo content: a scenario is inserted if
|
||||
missing, reconciled in place when a seed file's content changes, and deleted
|
||||
@@ -13,7 +12,7 @@ when no file claims its title any more, so an edit ships on the next deploy.
|
||||
Rename a seed by changing its `title` and listing the old one under
|
||||
`previous_titles`, which moves the rename onto the existing row. When a seed already matches, nothing is written, so
|
||||
this stays cheap to run on every boot. An adventure already started from a demo
|
||||
keeps its own copied cards and scripts and is unchanged. Only a new adventure
|
||||
keeps its own copied cards and is unchanged. Only a new adventure
|
||||
picks up the updated content.
|
||||
"""
|
||||
|
||||
@@ -35,7 +34,6 @@ SEED_DIR = Path(__file__).resolve().parent / "seed_data"
|
||||
_SCALARS = ("title", "description", "prompt", "memory", "authors_note", "ai_instructions",
|
||||
"tags", "image", "icon")
|
||||
_CARD_FIELDS = ("type", "name", "keys", "entry", "notes")
|
||||
_SCRIPT_FIELDS = ("name", "library_js", "input_js", "context_js", "output_js")
|
||||
|
||||
|
||||
def seed_public_scenarios(engine: Engine) -> None:
|
||||
@@ -98,7 +96,7 @@ def _sweep_unclaimed(db, claimed: set[str]) -> int:
|
||||
nothing anybody created can be reached from here.
|
||||
|
||||
An adventure started from a deleted demo survives. `adventures.scenario_id`
|
||||
is `ON DELETE SET NULL`, so the story, its cards, and its scripts are its
|
||||
is `ON DELETE SET NULL`, so the story and its cards are its
|
||||
own copies and stay; the adventure loses the cover art it inherited.
|
||||
|
||||
The caller skips this when a seed file failed to parse. A file that cannot
|
||||
@@ -117,11 +115,6 @@ def _sweep_unclaimed(db, claimed: set[str]) -> int:
|
||||
)
|
||||
for scenario in stale:
|
||||
logger.info("Removing seeded scenario %r; no seed file claims it.", scenario.title)
|
||||
# The scripts are joined through a secondary table, so nothing cascades
|
||||
# to them. They have a NULL owner and no other reader.
|
||||
for script in list(scenario.scripts):
|
||||
db.delete(script)
|
||||
scenario.scripts = []
|
||||
db.delete(scenario)
|
||||
return len(stale)
|
||||
|
||||
@@ -130,10 +123,6 @@ def _card_tuple(source, get) -> tuple:
|
||||
return tuple(get(source, f) for f in _CARD_FIELDS)
|
||||
|
||||
|
||||
def _script_tuple(source, get) -> tuple:
|
||||
return tuple(get(source, f) for f in _SCRIPT_FIELDS)
|
||||
|
||||
|
||||
def find_seeded(db, title: str) -> models.Scenario | None:
|
||||
"""Returns the seeded scenario with this exact title, if there is one."""
|
||||
return (
|
||||
@@ -178,14 +167,7 @@ def _matches(scenario: models.Scenario, data: dict) -> bool:
|
||||
_card_tuple(c, lambda o, f: o.get(f, ""))
|
||||
for c in (data.get("story_cards") or []) if isinstance(c, dict)
|
||||
)
|
||||
if have_cards != want_cards:
|
||||
return False
|
||||
have_scripts = sorted(_script_tuple(s, lambda o, f: getattr(o, f)) for s in scenario.scripts)
|
||||
want_scripts = sorted(
|
||||
_script_tuple(s, lambda o, f: (o.get(f, "") or ("Script" if f == "name" else "")))
|
||||
for s in (data.get("scripts") or []) if isinstance(s, dict)
|
||||
)
|
||||
return have_scripts == want_scripts
|
||||
return have_cards == want_cards
|
||||
|
||||
|
||||
def _insert_scenario(db, data: dict) -> None:
|
||||
@@ -203,9 +185,6 @@ def _update_scenario(db, scenario: models.Scenario, data: dict) -> None:
|
||||
# adventure foreign keys that point at it, intact.
|
||||
for card in list(scenario.story_cards):
|
||||
db.delete(card)
|
||||
for script in list(scenario.scripts):
|
||||
db.delete(script)
|
||||
scenario.scripts = []
|
||||
db.flush()
|
||||
_populate_children(db, scenario, data)
|
||||
|
||||
@@ -231,19 +210,3 @@ def _populate_children(db, scenario: models.Scenario, data: dict) -> None:
|
||||
notes=card.get("notes", ""),
|
||||
)
|
||||
)
|
||||
|
||||
for item in data.get("scripts") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
script = models.Script(
|
||||
user_id=None,
|
||||
name=item.get("name", "Script"),
|
||||
description=item.get("description", ""),
|
||||
library_js=item.get("library_js", ""),
|
||||
input_js=item.get("input_js", ""),
|
||||
context_js=item.get("context_js", ""),
|
||||
output_js=item.get("output_js", ""),
|
||||
)
|
||||
db.add(script)
|
||||
db.flush()
|
||||
scenario.scripts.append(script)
|
||||
|
||||
+5
-9
@@ -6,11 +6,9 @@ frames, so the format lives here rather than in either one.
|
||||
"""
|
||||
import json
|
||||
|
||||
from . import analytics
|
||||
|
||||
# `no-cache` stops an intermediary from caching the stream. `X-Accel-Buffering`
|
||||
# makes nginx-style reverse proxies, which hosted deploys use, flush each event
|
||||
# immediately rather than buffer it.
|
||||
# makes an nginx-style reverse proxy flush each event immediately rather than
|
||||
# buffer it, which matters if anyone puts one in front of the app.
|
||||
SSE_HEADERS = {"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}
|
||||
|
||||
|
||||
@@ -20,11 +18,9 @@ def sse(obj: dict) -> str:
|
||||
|
||||
|
||||
def turn_error(detail: str, **extra) -> str:
|
||||
"""Returns an SSE error for a turn that could not be produced, and counts it.
|
||||
"""Returns an SSE error for a turn that could not be produced.
|
||||
|
||||
A failed turn is still an HTTP 200 response, so the middleware's status-code
|
||||
tally cannot see it. This metric exists so that a demo whose model refuses
|
||||
every request does not report as healthy.
|
||||
A failed turn is still an HTTP 200 response, because the error is reported
|
||||
inside the stream the client is already reading.
|
||||
"""
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_TURN_ERROR)
|
||||
return sse({"type": "error", "detail": detail, **extra})
|
||||
|
||||
@@ -19,10 +19,8 @@ annotated-doc==0.0.5
|
||||
annotated-types==0.8.0
|
||||
anyio==4.14.2
|
||||
certifi==2026.7.22
|
||||
cffi==2.1.1
|
||||
charset-normalizer==3.5.1
|
||||
click==8.5.0
|
||||
cryptography==50.0.1
|
||||
fastapi==0.141.1
|
||||
greenlet==3.5.5
|
||||
h11==0.16.0
|
||||
@@ -33,16 +31,12 @@ idna==3.19
|
||||
iniconfig==2.3.0
|
||||
packaging==26.3
|
||||
pluggy==1.6.0
|
||||
psycopg==3.3.5
|
||||
psycopg-binary==3.3.5
|
||||
pycparser==3.0
|
||||
pydantic==2.13.5
|
||||
pydantic_core==2.46.5
|
||||
Pygments==2.21.0
|
||||
pytest==9.1.1
|
||||
python-dotenv==1.2.3
|
||||
PyYAML==6.0.3
|
||||
quickjs==1.19.4
|
||||
regex==2026.9.3
|
||||
requests==2.34.2
|
||||
SQLAlchemy==2.0.52
|
||||
|
||||
@@ -8,6 +8,3 @@ httpx>=0.27
|
||||
# code imports it by name.
|
||||
certifi
|
||||
tiktoken>=0.7
|
||||
quickjs>=1.19
|
||||
cryptography>=42
|
||||
psycopg[binary]>=3.2
|
||||
|
||||
@@ -39,3 +39,44 @@ class ScriptedProvider:
|
||||
if isinstance(reply, Exception):
|
||||
raise reply
|
||||
yield ("text", reply)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Deterministic per-turn state instrumentation
|
||||
# ---------------------------------------------------------------------------
|
||||
# Several tests need a value that changes by a fixed amount on every turn, so
|
||||
# that a rollback failure is arithmetic rather than a judgement call: if a take
|
||||
# stacks instead of replacing, the total is off by exactly one turn's worth.
|
||||
#
|
||||
# That instrumentation used to be a JavaScript `output` hook doing
|
||||
# `state.gold += 10` in the QuickJS sandbox. M2 removed campaign scripting, and
|
||||
# the tests below are not about scripting — they are about the state snapshot,
|
||||
# rollback, and branch-isolation machinery in `attempts.py` and `tree.py`,
|
||||
# which is unchanged.
|
||||
#
|
||||
# The counter therefore moved to the world-state engine, which is a real
|
||||
# remaining product path: the model emits a ```state delta block, the referee
|
||||
# applies it, and the result lands in `adventure.world_state`. The fake
|
||||
# provider decides what the model "emits", so it is exactly as deterministic as
|
||||
# the script was, and it exercises production code rather than a test hook.
|
||||
|
||||
#: A schema with a plain unbounded counter. No `max_delta_per_turn` and no
|
||||
#: `cooldown`, so every +10 is applied in full, every turn.
|
||||
GOLD_SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
GOLD_PER_TURN = 10
|
||||
|
||||
|
||||
def gold_reply(text: str, amount: int = GOLD_PER_TURN) -> str:
|
||||
"""A model reply that narrates `text` and banks `amount` gold."""
|
||||
return f'{text}\n```state\n{{"player.gold": {amount}}}\n```'
|
||||
|
||||
|
||||
def gold_replies(prefix: str = "Take", count: int = 40) -> list[str]:
|
||||
"""`count` numbered replies, each banking one turn's worth of gold."""
|
||||
return [gold_reply(f"{prefix} {n}.") for n in range(1, count + 1)]
|
||||
|
||||
@@ -1,208 +0,0 @@
|
||||
"""The access log: app/accesslog.py and GET /api/analytics/access.
|
||||
|
||||
This is the half of the analytics work that identifies people on purpose,
|
||||
so these tests pin the details that would quietly make it wrong. The
|
||||
address recorded must be the hardened one, not a header a client chose.
|
||||
Session rows must be thinned instead of written on every page load. And a
|
||||
row must outlive the account it describes, because guest cleanup deletes
|
||||
accounts on a schedule, and a log that deletes itself is not a log.
|
||||
|
||||
python -m pytest tests/test_accesslog.py -v
|
||||
"""
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import accesslog, auth, limits, models, security
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
|
||||
EDGE = "198.51.100.77" # what the trusted proxy appended
|
||||
SPOOF = "10.0.0.1" # what a client put in front of it
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_state():
|
||||
accesslog._last_session.clear()
|
||||
yield
|
||||
accesslog._last_session.clear()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
owner = models.User(is_guest=False, email="owner@example.com")
|
||||
member = models.User(
|
||||
is_guest=False, email="player@example.com",
|
||||
password_hash=security.hash_password("hunter2long"),
|
||||
)
|
||||
setup.add_all([owner, member])
|
||||
setup.commit()
|
||||
ids = {"owner": owner.id, "member": member.id}
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_login_allowed", lambda *a, **k: None)
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
|
||||
|
||||
# /auth/me resolves its own session, so the cookie flow below is the real
|
||||
# one. Every other endpoint goes through get_current_user, and `act_as`
|
||||
# decides who that is.
|
||||
acting = {"id": ids["owner"]}
|
||||
|
||||
def _current(db=Depends(get_db)):
|
||||
return db.get(models.User, acting["id"])
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current
|
||||
try:
|
||||
test_client = TestClient(app)
|
||||
test_client.ids = ids
|
||||
test_client.act_as = lambda user_id: acting.update(id=user_id)
|
||||
yield test_client
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def visit(client, ip=EDGE, ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64)"):
|
||||
return client.get(
|
||||
"/api/auth/me",
|
||||
headers={"x-forwarded-for": f"{SPOOF}, {ip}", "user-agent": ua},
|
||||
)
|
||||
|
||||
|
||||
def rows(kind=None):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
query = db.query(models.AccessEvent).order_by(models.AccessEvent.id)
|
||||
if kind:
|
||||
query = query.filter_by(kind=kind)
|
||||
return query.all()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def read_log(client, **params):
|
||||
return client.get("/api/analytics/access", params=params)
|
||||
|
||||
|
||||
# ---------- Writing ----------
|
||||
|
||||
def test_a_new_session_is_logged(client):
|
||||
visit(client)
|
||||
logged = rows()
|
||||
assert len(logged) == 1
|
||||
entry = logged[0]
|
||||
assert entry.kind == accesslog.SESSION
|
||||
assert entry.is_guest and entry.who.startswith("Guest #")
|
||||
assert entry.device == "desktop"
|
||||
|
||||
|
||||
def test_the_address_is_the_hardened_one_not_the_clients(client):
|
||||
visit(client)
|
||||
# The client prepended its own value. Only the hop the edge appended counts.
|
||||
# Recording the leftmost value would make every row forgeable, which is
|
||||
# worse for a log than having no log at all.
|
||||
assert rows()[0].ip == EDGE
|
||||
|
||||
|
||||
def test_session_rows_are_thinned_to_one_per_day_per_address(client):
|
||||
for _ in range(4):
|
||||
visit(client)
|
||||
assert len(rows(accesslog.SESSION)) == 1
|
||||
|
||||
|
||||
def test_a_changed_address_writes_a_new_row(client):
|
||||
visit(client)
|
||||
visit(client, ip="203.0.113.9")
|
||||
logged = rows(accesslog.SESSION)
|
||||
assert [entry.ip for entry in logged] == [EDGE, "203.0.113.9"]
|
||||
# Same session throughout, so both rows name the same visitor.
|
||||
assert logged[0].who == logged[1].who
|
||||
|
||||
|
||||
def test_sign_in_and_failure_are_both_logged(client):
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "wrong"},
|
||||
headers={"x-forwarded-for": EDGE})
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
|
||||
headers={"x-forwarded-for": EDGE})
|
||||
kinds = [entry.kind for entry in rows()]
|
||||
assert accesslog.LOGIN_FAILED in kinds and accesslog.LOGIN in kinds
|
||||
|
||||
failure = rows(accesslog.LOGIN_FAILED)[0]
|
||||
# This records the address that was tried, not the account it belongs to.
|
||||
# A failed attempt against an address with no matching account is
|
||||
# exactly what this row exists to capture.
|
||||
assert failure.who == "player@example.com"
|
||||
assert failure.user_id is None
|
||||
assert rows(accesslog.LOGIN)[0].user_id == client.ids["member"]
|
||||
|
||||
|
||||
def test_registering_is_logged_against_the_upgraded_account(client):
|
||||
visit(client) # creates the guest whose session then registers
|
||||
client.act_as(rows()[0].user_id)
|
||||
client.post("/api/auth/register", json={"email": "new@example.com", "password": "hunter2long"})
|
||||
entry = rows(accesslog.REGISTER)[0]
|
||||
assert entry.who == "new@example.com" and not entry.is_guest
|
||||
|
||||
|
||||
def test_a_row_outlives_the_account_it_describes(client):
|
||||
visit(client)
|
||||
entry = rows()[0]
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.delete(db.get(models.User, entry.user_id))
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
# There is no foreign key, and `who` is a snapshot. Guest cleanup deletes
|
||||
# accounts on a schedule, and a log that vanishes along with them is not
|
||||
# a log.
|
||||
survivor = rows()[0]
|
||||
assert survivor.who == entry.who and survivor.ip == EDGE
|
||||
|
||||
|
||||
def test_a_long_user_agent_is_truncated(client):
|
||||
visit(client, ua="Mozilla/" + "x" * 500)
|
||||
assert len(rows()[0].user_agent) == accesslog.MAX_UA
|
||||
|
||||
|
||||
def test_a_logging_failure_does_not_break_the_request(client, monkeypatch):
|
||||
monkeypatch.setattr(accesslog, "_client_ip", lambda request: 1 / 0)
|
||||
# The log observes sign-in. A logging failure must not block the request.
|
||||
assert visit(client).status_code == 200
|
||||
|
||||
|
||||
# ---------- Reading ----------
|
||||
|
||||
def test_the_log_is_invisible_to_everyone_but_the_owner(client):
|
||||
visit(client)
|
||||
assert read_log(client).status_code == 200
|
||||
client.act_as(client.ids["member"])
|
||||
assert read_log(client).status_code == 404
|
||||
|
||||
|
||||
def test_the_log_reads_newest_first_and_pages_backwards(client):
|
||||
for index in range(5):
|
||||
visit(client, ip=f"203.0.113.{index}")
|
||||
first = read_log(client, limit=2).json()
|
||||
assert [event["ip"] for event in first["events"]] == ["203.0.113.4", "203.0.113.3"]
|
||||
assert first["has_more"]
|
||||
|
||||
older = read_log(client, limit=2, before_id=first["events"][-1]["id"]).json()
|
||||
assert [event["ip"] for event in older["events"]] == ["203.0.113.2", "203.0.113.1"]
|
||||
|
||||
|
||||
def test_the_log_filters_by_kind_and_searches(client):
|
||||
visit(client)
|
||||
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
|
||||
headers={"x-forwarded-for": "203.0.113.44"})
|
||||
|
||||
assert len(read_log(client, kind="login").json()["events"]) == 1
|
||||
by_email = read_log(client, q="player@example.com").json()["events"]
|
||||
assert len(by_email) == 1 and by_email[0]["kind"] == "login"
|
||||
by_ip = read_log(client, q="203.0.113.44").json()["events"]
|
||||
assert len(by_ip) == 1
|
||||
assert read_log(client, q="nobody@example.com").json()["events"] == []
|
||||
@@ -48,7 +48,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -1,385 +0,0 @@
|
||||
"""Visit analytics: app/analytics.py and the two endpoints in front of it.
|
||||
|
||||
This file tests three things, and the rest is arithmetic. The counters must
|
||||
survive the buffer/UPSERT round trip: a flush adds to what is already
|
||||
stored instead of replacing it, or every number would show only the last
|
||||
minute. The funnel counts people rather than clicks, which is the only
|
||||
reason the visitor-day table exists. The gate holds: a stranger cannot read
|
||||
the dashboard, and cannot inflate what it reports beyond hitting the page.
|
||||
|
||||
python -m pytest tests/test_analytics.py -v
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import analytics, auth, limits, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_buffer():
|
||||
"""The buffer is process-wide, so a test that leaves counts in it would
|
||||
show up inside the next one's flush."""
|
||||
analytics._counts.clear()
|
||||
analytics._visits.clear()
|
||||
analytics._labels_seen.clear()
|
||||
yield
|
||||
analytics._counts.clear()
|
||||
analytics._visits.clear()
|
||||
analytics._labels_seen.clear()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db():
|
||||
Base.metadata.create_all(bind=engine)
|
||||
session = SessionLocal()
|
||||
try:
|
||||
yield session
|
||||
finally:
|
||||
session.close()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def counter(db, metric, label):
|
||||
row = (
|
||||
db.query(models.AnalyticsDaily)
|
||||
.filter_by(metric=metric, label=label)
|
||||
.one_or_none()
|
||||
)
|
||||
return row.hits if row else 0
|
||||
|
||||
|
||||
def make_user(db, email=None):
|
||||
user = models.User(is_guest=email is None, email=email)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
return user
|
||||
|
||||
|
||||
# ---------- The buffer and its flush ----------
|
||||
|
||||
def test_counts_accumulate_across_flushes(db):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.flush(db)
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
analytics.flush(db)
|
||||
# The second flush has to find the existing row and add to it. Replacing it
|
||||
# would leave every counter showing only the newest minute of traffic.
|
||||
assert counter(db, analytics.M_PAGE, "/") == 3
|
||||
|
||||
|
||||
def test_flush_is_a_no_op_when_nothing_happened(db):
|
||||
analytics.flush(db)
|
||||
assert db.query(models.AnalyticsDaily).count() == 0
|
||||
|
||||
|
||||
def test_a_failed_flush_keeps_the_counts(db, monkeypatch):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
monkeypatch.setattr(analytics, "_write_counts", lambda *a: 1 / 0)
|
||||
analytics.flush(db) # must not raise
|
||||
monkeypatch.undo()
|
||||
analytics.flush(db)
|
||||
assert counter(db, analytics.M_PAGE, "/") == 1
|
||||
|
||||
|
||||
def test_label_cardinality_is_capped(db):
|
||||
for i in range(analytics.MAX_LABELS_PER_METRIC + 25):
|
||||
analytics.record(analytics.M_REFERRER, f"host{i}.example")
|
||||
analytics.flush(db)
|
||||
labels = db.query(models.AnalyticsDaily).filter_by(metric=analytics.M_REFERRER).count()
|
||||
# Everything past the cap is folded into one bucket, so a referrer flood
|
||||
# cannot create unlimited rows.
|
||||
assert labels == analytics.MAX_LABELS_PER_METRIC + 1
|
||||
assert counter(db, analytics.M_REFERRER, analytics.OTHER) == 25
|
||||
|
||||
|
||||
# ---------- Visitors ----------
|
||||
|
||||
def test_visitor_id_is_stable_and_keyed(db, monkeypatch):
|
||||
user = make_user(db)
|
||||
handle = analytics.visitor_id(user)
|
||||
assert handle == analytics.visitor_id(user) # a returning visitor
|
||||
assert handle != analytics.visitor_id(make_user(db)) # is still one visitor
|
||||
assert len(handle) == 32 and int(handle, 16) >= 0 # opaque hex, not an id
|
||||
# Keyed on the app secret, not a bare hash of the user id. Otherwise
|
||||
# anyone holding this table could rebuild the mapping by hashing
|
||||
# sequential ids.
|
||||
monkeypatch.setattr(analytics.security, "SECRET_KEY", b"a-different-secret")
|
||||
assert analytics.visitor_id(user) != handle
|
||||
|
||||
|
||||
def test_a_repeat_visitor_is_new_only_once(db):
|
||||
user = make_user(db)
|
||||
analytics.record_visit(user)
|
||||
analytics.flush(db)
|
||||
rows = db.query(models.AnalyticsVisitorDay).all()
|
||||
assert len(rows) == 1 and rows[0].is_new
|
||||
|
||||
# Same visitor, a later day: seen before, so not new. The row is not
|
||||
# merged into the first day's row either.
|
||||
tomorrow = (models.utcnow().date() + timedelta(days=1)).isoformat()
|
||||
analytics._visits[(tomorrow, analytics.visitor_id(user))] = set()
|
||||
analytics.flush(db)
|
||||
rows = db.query(models.AnalyticsVisitorDay).order_by(models.AnalyticsVisitorDay.day).all()
|
||||
assert [row.is_new for row in rows] == [True, False]
|
||||
|
||||
|
||||
def test_one_row_per_visitor_per_day_however_much_they_do(db):
|
||||
user = make_user(db)
|
||||
for _ in range(5):
|
||||
analytics.record_event(analytics.EV_ADVENTURE, user)
|
||||
analytics.flush(db)
|
||||
assert db.query(models.AnalyticsVisitorDay).count() == 1
|
||||
assert counter(db, analytics.M_EVENT, analytics.EV_ADVENTURE) == 5
|
||||
|
||||
|
||||
def test_funnel_flags_only_ever_turn_on(db):
|
||||
user = make_user(db)
|
||||
analytics.record_event(analytics.EV_TURN, user)
|
||||
analytics.flush(db)
|
||||
# A later visit that reaches no funnel step must not clear the earlier one.
|
||||
analytics.record_visit(user)
|
||||
analytics.flush(db)
|
||||
row = db.query(models.AnalyticsVisitorDay).one()
|
||||
assert row.played and not row.created
|
||||
|
||||
|
||||
def test_purge_drops_only_rows_past_the_horizon(db):
|
||||
old = (models.utcnow().date() - timedelta(days=analytics.RETENTION_DAYS + 1)).isoformat()
|
||||
db.add(models.AnalyticsVisitorDay(day=old, visitor="a" * 32))
|
||||
db.add(models.AnalyticsVisitorDay(day=analytics._today(), visitor="b" * 32))
|
||||
db.commit()
|
||||
assert analytics.purge_old_visitor_days(db) == 1
|
||||
assert [r.visitor for r in db.query(models.AnalyticsVisitorDay)] == ["b" * 32]
|
||||
|
||||
|
||||
# ---------- Normalizing what a browser claims ----------
|
||||
|
||||
@pytest.mark.parametrize("path, expected", [
|
||||
("/", "/"),
|
||||
("/adventures", "/adventures"),
|
||||
("/adventures/", "/adventures"),
|
||||
("/play/12?x=1", "/play/:id"),
|
||||
("/scenarios/9#top", "/scenarios/:id"),
|
||||
("/wp-admin", "(other)"),
|
||||
("/play/../../etc", "(other)"),
|
||||
("", "/"),
|
||||
])
|
||||
def test_route_normalization(path, expected):
|
||||
assert analytics.normalize_route(path) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("referrer, expected", [
|
||||
("", "(direct)"),
|
||||
("https://news.ycombinator.com/item?id=1", "news.ycombinator.com"),
|
||||
("https://www.google.com/", "google.com"),
|
||||
("https://ai-dnd.example/scenarios", ""), # our own host: not a referral
|
||||
("javascript:alert(1)", "(other)"),
|
||||
("https://" + "x" * 200 + ".com", "(other)"),
|
||||
])
|
||||
def test_referrer_normalization(referrer, expected):
|
||||
assert analytics.normalize_referrer(referrer, "ai-dnd.example") == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ua, expected", [
|
||||
("Mozilla/5.0 (iPhone; CPU iPhone OS 17_0) AppleWebKit", "mobile"),
|
||||
("Mozilla/5.0 (iPad; CPU OS 17_0) AppleWebKit", "tablet"),
|
||||
("Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "desktop"),
|
||||
("Googlebot/2.1", "bot"),
|
||||
("", "(unknown)"),
|
||||
])
|
||||
def test_device_detection(ua, expected):
|
||||
assert analytics.device_of(ua) == expected
|
||||
|
||||
|
||||
def test_only_iso_looking_country_headers_are_trusted():
|
||||
assert analytics.country_of({"cf-ipcountry": "de"}) == "DE"
|
||||
assert analytics.country_of({"cf-ipcountry": "Norway"}) == analytics.UNKNOWN
|
||||
assert analytics.country_of({"cf-ipcountry": "XX"}) == analytics.UNKNOWN
|
||||
assert analytics.country_of({}) == analytics.UNKNOWN
|
||||
|
||||
|
||||
def test_error_labels_use_the_route_not_the_path():
|
||||
class Route:
|
||||
path = "/api/adventures/{adventure_id}"
|
||||
|
||||
assert analytics.api_route_label({"route": Route()}, 500) == "500 /api/adventures/{adventure_id}"
|
||||
# An unmatched path is entirely attacker-chosen, so it never becomes a label.
|
||||
assert analytics.api_route_label({}, 404) == "404 (unmatched)"
|
||||
|
||||
|
||||
# ---------- The summary ----------
|
||||
|
||||
def test_summary_counts_people_once_per_step(db):
|
||||
one, two = make_user(db), make_user(db)
|
||||
for _ in range(3):
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, one)
|
||||
analytics.record_event(analytics.EV_TURN, one)
|
||||
analytics.record_event(analytics.EV_SCENARIO_OPEN, two)
|
||||
|
||||
result = analytics.summary(db, days=7)
|
||||
steps = {row["step"]: row["count"] for row in result["funnel"]}
|
||||
assert steps["Visited"] == 2
|
||||
assert steps["Opened a scenario"] == 2
|
||||
assert steps["Played a turn"] == 1 # not 3, because one person made three turns
|
||||
assert steps["Signed up"] == 0
|
||||
# Raw event totals still count every occurrence.
|
||||
assert result["totals"]["turns"] == 3
|
||||
assert result["totals"]["visitors"] == 2
|
||||
|
||||
|
||||
def test_summary_series_covers_every_day_including_empty_ones(db):
|
||||
analytics.record(analytics.M_PAGE, "/")
|
||||
result = analytics.summary(db, days=7)
|
||||
assert len(result["series"]) == 7
|
||||
assert result["series"][-1]["day"] == models.utcnow().date().isoformat()
|
||||
assert result["series"][-1]["pageviews"] == 1
|
||||
assert result["series"][0]["pageviews"] == 0
|
||||
|
||||
|
||||
def test_summary_flushes_before_reading(db):
|
||||
analytics.record(analytics.M_EVENT, analytics.EV_TURN)
|
||||
# Never flushed by hand: the dashboard must not be up to a minute stale.
|
||||
assert analytics.summary(db, days=1)["totals"]["turns"] == 1
|
||||
|
||||
|
||||
def test_summary_reports_pages_referrers_and_errors(db):
|
||||
analytics.record(analytics.M_PAGE, "/play/:id", n=4)
|
||||
analytics.record(analytics.M_REFERRER, "news.ycombinator.com", n=2)
|
||||
analytics.record(analytics.M_ERROR, "500 /api/adventures/{adventure_id}")
|
||||
result = analytics.summary(db, days=30)
|
||||
assert result["pages"][0] == {"label": "/play/:id", "hits": 4}
|
||||
assert result["referrers"][0]["label"] == "news.ycombinator.com"
|
||||
assert result["totals"]["errors"] == 1
|
||||
|
||||
|
||||
# ---------- The endpoints ----------
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
visitor = models.User(is_guest=True)
|
||||
owner = models.User(is_guest=False, email="owner@example.com")
|
||||
setup.add_all([visitor, owner])
|
||||
setup.commit()
|
||||
ids = {"visitor": visitor.id, "owner": owner.id}
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
# Multi-user is what makes the gate mean anything: local mode trusts
|
||||
# whoever is at the keyboard, because it is the operator's own machine.
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
|
||||
|
||||
current = {"id": ids["visitor"]}
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, current["id"])
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
monkeypatch.setattr(
|
||||
auth, "resolve_session_user", lambda request, db: db.get(models.User, current["id"])
|
||||
)
|
||||
try:
|
||||
client = TestClient(app)
|
||||
client.ids, client.current = ids, current
|
||||
yield client
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def read_summary(client, days=30):
|
||||
return client.get(f"/api/analytics/summary?days={days}")
|
||||
|
||||
|
||||
def test_dashboard_is_invisible_to_everyone_but_the_owner(client):
|
||||
assert read_summary(client).status_code == 404
|
||||
client.current["id"] = client.ids["owner"]
|
||||
assert read_summary(client).status_code == 200
|
||||
|
||||
|
||||
def test_collect_records_a_pageview_and_the_visit(client):
|
||||
resp = client.post("/api/analytics/collect", json={"path": "/play/7", "first": True,
|
||||
"referrer": "https://news.ycombinator.com/"})
|
||||
assert resp.status_code == 204
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert body["pages"][0] == {"label": "/play/:id", "hits": 1}
|
||||
assert body["referrers"][0]["label"] == "news.ycombinator.com"
|
||||
assert body["totals"]["visitors"] == 1
|
||||
|
||||
|
||||
def test_referrer_and_device_are_recorded_once_per_visit_not_per_view(client):
|
||||
for path in ("/", "/scenarios", "/adventures"):
|
||||
client.post("/api/analytics/collect", json={"path": path, "first": path == "/"})
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert body["totals"]["pageviews"] == 3
|
||||
# Three views, one visit: the referral and the device are facts about the
|
||||
# visit, so counting them per view would multiply every one of them.
|
||||
assert sum(row["hits"] for row in body["devices"]) == 1
|
||||
|
||||
|
||||
def test_the_owners_own_visits_are_not_traffic(client):
|
||||
client.current["id"] = client.ids["owner"]
|
||||
client.post("/api/analytics/collect", json={"path": "/", "first": True})
|
||||
assert read_summary(client).json()["totals"]["pageviews"] == 0
|
||||
|
||||
|
||||
def test_a_client_cannot_invent_pages_or_events(client):
|
||||
client.post("/api/analytics/collect", json={"path": "/../../admin", "first": True})
|
||||
# There is no field for it, so a made-up event is not even expressible.
|
||||
client.post("/api/analytics/collect", json={"path": "/", "event": "signup"})
|
||||
client.current["id"] = client.ids["owner"]
|
||||
body = read_summary(client).json()
|
||||
assert {row["label"] for row in body["pages"]} == {"(other)", "/"}
|
||||
assert body["totals"]["signups"] == 0
|
||||
|
||||
|
||||
def test_api_errors_are_counted_by_route(client):
|
||||
client.get("/api/adventures/999999")
|
||||
client.current["id"] = client.ids["owner"]
|
||||
errors = read_summary(client).json()["errors"]
|
||||
assert errors and errors[0]["label"].startswith("404 /api/adventures/")
|
||||
|
||||
|
||||
# ---------- The dialect the tests never run on ----------
|
||||
|
||||
def test_the_upserts_compile_for_postgres():
|
||||
"""Prod runs on Neon, but these tests run on SQLite, and a failed flush
|
||||
is caught and logged instead of raised. A dialect mistake would
|
||||
therefore stay invisible until the dashboard quietly stayed empty. This
|
||||
test compiles both statements against Postgres without connecting to
|
||||
one.
|
||||
"""
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.dialects import postgresql
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
session = sessionmaker(bind=create_engine("postgresql+psycopg://u:p@localhost/db"))()
|
||||
compiled = []
|
||||
|
||||
def capture(statement, *args, **kwargs):
|
||||
compiled.append(str(statement.compile(dialect=postgresql.dialect())))
|
||||
|
||||
session.execute = capture
|
||||
session.scalars = lambda *a, **k: []
|
||||
|
||||
analytics._write_counts(session, {("2026-01-01", "pageview", "/"): 2})
|
||||
analytics._write_visits(session, {("2026-01-01", "f" * 32): {"played"}})
|
||||
|
||||
counts, visits = compiled
|
||||
assert "ON CONFLICT (day, metric, label) DO UPDATE" in counts
|
||||
assert "analytics_daily.hits + excluded.hits" in counts
|
||||
assert "ON CONFLICT (day, visitor) DO UPDATE" in visits
|
||||
assert "analytics_visitor_days.played OR excluded.played" in visits
|
||||
# is_new is settled by the first write of a visitor's first day and must
|
||||
# not be in the update clause at all.
|
||||
assert "is_new" not in visits.split("DO UPDATE")[1]
|
||||
@@ -19,17 +19,10 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -45,14 +38,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -61,9 +51,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -30,7 +30,6 @@ from app import auth, limits, models, tree
|
||||
from app.context import history, lineage
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.scripting import ScriptPipeline
|
||||
from tools import dbmeter
|
||||
|
||||
|
||||
@@ -224,24 +223,11 @@ def test_an_already_loaded_collection_is_cut_down_to_the_path(forked):
|
||||
assert labels(history.tail(adventure, 3)) == ["B5", "C6", "C7"]
|
||||
|
||||
|
||||
def test_user_scripts_are_handed_the_path(forked):
|
||||
"""The same risk one layer up, in code visible to users:
|
||||
`pipeline._history()` is the documented scripting history API."""
|
||||
db, adventure, _ = forked
|
||||
list(adventure.actions) # the pipeline's caller has usually loaded these
|
||||
pipeline = ScriptPipeline(adventure, db)
|
||||
assert [h["text"] for h in pipeline._history()] == [
|
||||
"A0", "A1", "A2", "A3", "B4", "B5", "C6", "C7"
|
||||
]
|
||||
assert pipeline._info()["actionCount"] == 8
|
||||
|
||||
|
||||
# ------------------------------------------------------------ over the wire
|
||||
|
||||
@pytest.fixture()
|
||||
def client(forked, monkeypatch):
|
||||
db, adventure, ids = forked
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(session=Depends(get_db)):
|
||||
|
||||
@@ -22,7 +22,7 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# `hp` moves freely. `mana` has a cooldown of 2 turns, so an incorrect
|
||||
# advance shows up as a change the referee should have rejected.
|
||||
@@ -30,16 +30,13 @@ SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"mana": {"min": 0, "max": 50, "initial": 50, "cooldown": 2},
|
||||
# The per-turn counter these tests measure rollbacks with. Unbounded and
|
||||
# uncapped on purpose, so every turn's +10 lands in full. See
|
||||
# `fakes.gold_reply`.
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -55,14 +52,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100, "mana": 50}},
|
||||
world_state={"player": {"hp": 100, "mana": 50, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -71,9 +65,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -121,7 +112,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -322,11 +313,13 @@ def test_forking_a_live_node_on_another_branch_is_refused(client):
|
||||
|
||||
# -------------------------------------------------------------- the state
|
||||
|
||||
def test_switching_restores_the_script_and_world_state(client):
|
||||
def test_switching_restores_the_state_a_branch_left_behind(client):
|
||||
# Two stats move: hp differs per attempt, and gold counts turns. Between
|
||||
# them, a switch that restored the wrong snapshot is visible either way.
|
||||
ScriptedProvider.replies = [
|
||||
"A scratch.\n```state\n{\"player.hp\": -5}\n```",
|
||||
"A beating.\n```state\n{\"player.hp\": -40}\n```",
|
||||
"Onward.",
|
||||
'A scratch.\n```state\n{"player.hp": -5, "player.gold": 10}\n```',
|
||||
'A beating.\n```state\n{"player.hp": -40, "player.gold": 10}\n```',
|
||||
gold_reply("Onward."),
|
||||
]
|
||||
_play(client)
|
||||
_retry(client)
|
||||
@@ -336,9 +329,9 @@ def test_switching_restores_the_script_and_world_state(client):
|
||||
|
||||
discarded = [a.id for a in _rows(client.adv_id) if a.type == "ai" and not a.live][0]
|
||||
_fork(client, discarded)
|
||||
script_state, world_state = _state(client.adv_id)
|
||||
player, world_state = _state(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 95, "the attempt this branch tells"
|
||||
assert script_state == {"gold": 10}, "one turn of gold, not three"
|
||||
assert player["gold"] == 10, "one turn of gold, not three"
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/branches/{parent}/switch")
|
||||
assert _state(client.adv_id) == on_parent
|
||||
|
||||
@@ -55,9 +55,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two.", "Next turn."]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -33,19 +33,12 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn, so the stored gold total tells how many turns the
|
||||
# story behind it played. This makes an after-snapshot visible from outside.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
OPENING = "You enter a cave."
|
||||
|
||||
@@ -63,14 +56,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text=OPENING))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -78,9 +68,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = ["A reply."]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -186,7 +173,7 @@ def _branch_rows(adv_id) -> list[models.Branch]:
|
||||
def _script_state(adv_id) -> dict:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return db.get(models.Adventure, adv_id).script_state
|
||||
return (db.get(models.Adventure, adv_id).world_state or {}).get("player", {})
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -272,7 +259,7 @@ def test_a_switch_in_the_copy_restores_what_that_branch_left_behind(client):
|
||||
# Play one more turn on the fork, so the two tips end up at
|
||||
# genuinely different totals. Turn for turn, both branches earn the
|
||||
# same gold, so a switch that restored nothing would still look right.
|
||||
ScriptedProvider.replies = ["Further still."]
|
||||
ScriptedProvider.replies = [gold_reply("Further still.")]
|
||||
_play(client, original, "press on")
|
||||
|
||||
per_branch = []
|
||||
@@ -398,7 +385,6 @@ def test_a_fork_with_no_depth_is_refused(client):
|
||||
|
||||
|
||||
def test_more_branches_than_the_cap_is_refused(client, monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
payload = {
|
||||
"format": bundle.FORMAT, "title": "Too many",
|
||||
"branches": [{"parent": None, "forkDepth": None}]
|
||||
@@ -558,7 +544,6 @@ def test_the_action_cap_counts_the_rows_a_v1_file_expands_into(client, monkeypat
|
||||
multiple of it. The body-size limit does not help here: the text is
|
||||
tiny, and the row count is the actual cost.
|
||||
"""
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(limits, "MAX_ACTIONS_PER_ADVENTURE", 6)
|
||||
monkeypatch.setattr(limits, "_BUNDLE_LIST_CAPS",
|
||||
{**limits._BUNDLE_LIST_CAPS, "actions": 6})
|
||||
|
||||
+43
-165
@@ -1,8 +1,14 @@
|
||||
"""HTTP tests for the AI Chat scratchpad (power users only).
|
||||
"""HTTP tests for the AI Chat scratchpad.
|
||||
|
||||
Covers the access gate, the streamed reply, and the demo-key model pinning.
|
||||
This pinning must not let a public visitor reach paid models through this
|
||||
page.
|
||||
Most of this file used to be about the shared demo key: an access gate on a
|
||||
"power user" email allowlist, and a pinning rule that stopped a public visitor
|
||||
reaching paid models on a server-funded key. M2 removed the hosted deployment
|
||||
those defended, so the rules they tested no longer exist to be tested. See
|
||||
`planning/reports/M2-*` for the accounting.
|
||||
|
||||
What remains is what the page still does: stream a reply from the configured
|
||||
model, honour a system prompt and a per-request model override, and refuse a
|
||||
conversation too large to send.
|
||||
|
||||
python -m pytest tests/test_chat.py -v
|
||||
"""
|
||||
@@ -10,25 +16,27 @@ import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, limits, models
|
||||
from app import auth, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import chat
|
||||
|
||||
|
||||
class FakeProvider:
|
||||
"""Records what it was constructed with, then streams a fixed reply. Stands
|
||||
in for the real egress point, so asserting on last_key/last_model is
|
||||
asserting on exactly what would have gone over the wire."""
|
||||
"""Records what it was constructed with, then streams a fixed reply.
|
||||
|
||||
It stands in for the real egress point, so asserting on `last_endpoint` and
|
||||
`last_model` is asserting on exactly what would have gone over the wire.
|
||||
There is no `last_key` any more: the provider takes no API key, because
|
||||
Ollama does not use one.
|
||||
"""
|
||||
last_usage = None
|
||||
last_model = None
|
||||
last_key = None
|
||||
last_endpoint = None
|
||||
last_messages = None
|
||||
|
||||
def __init__(self, endpoint_url, api_key, model, api_mode="chat", reasoning_max_tokens=0):
|
||||
def __init__(self, endpoint_url, model, api_mode="chat", read_timeout=None):
|
||||
FakeProvider.last_model = model
|
||||
FakeProvider.last_key = api_key
|
||||
FakeProvider.last_endpoint = endpoint_url
|
||||
|
||||
async def chat(self, messages, *, temperature, max_tokens):
|
||||
@@ -41,68 +49,35 @@ class FakeProvider:
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False, email="power@example.com")
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(chat, "OpenAICompatibleProvider", FakeProvider)
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
# Multi-user mode is what makes the power-user gate meaningful, because
|
||||
# local mode trusts everyone. The allowlist is set per test.
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(auth, "POWER_USERS", {"power@example.com"})
|
||||
# These tests deliberately do not stub resolve_provider_config. The
|
||||
# point is to exercise the real BYOK-vs-demo decision, since that
|
||||
# decision is what keeps the shared key off paid models. Each test
|
||||
# picks a mode with _byok/_demo below.
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield TestClient(app)
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def _send(client, **extra):
|
||||
return client.post("/api/chat/stream", json={"messages": [{"role": "user", "content": "hi"}], **extra})
|
||||
def _send(client, **body):
|
||||
payload = {"messages": [{"role": "user", "content": "hi"}]}
|
||||
payload.update(body)
|
||||
return client.post("/api/chat/stream", json=payload)
|
||||
|
||||
|
||||
def _byok(monkeypatch):
|
||||
"""The user brought their own key: no demo key in play, any model allowed."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: False)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.api_key = "sk-my-own-key" # legacy-plaintext path: used as-is
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _demo(monkeypatch, whitelist=("free/allowed",)):
|
||||
"""The user has no key, so turns run on the server-funded demo key."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", list(whitelist))
|
||||
|
||||
|
||||
def test_non_power_user_gets_404(client, monkeypatch):
|
||||
monkeypatch.setattr(auth, "POWER_USERS", set())
|
||||
assert _send(client).status_code == 404
|
||||
assert client.get("/api/chat/config").status_code == 404
|
||||
|
||||
|
||||
def test_power_user_streams_a_reply(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_the_page_streams_a_reply(client):
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert '"type": "reasoning"' in resp.text
|
||||
@@ -111,136 +86,39 @@ def test_power_user_streams_a_reply(client, monkeypatch):
|
||||
assert FakeProvider.last_messages == [{"role": "user", "content": "hi"}]
|
||||
|
||||
|
||||
def test_system_prompt_and_model_override_are_honoured(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_it_uses_the_configured_endpoint_and_model(client):
|
||||
_send(client)
|
||||
assert FakeProvider.last_model == "test-model"
|
||||
# The default from `models.Settings`, and the only kind of address the
|
||||
# endpoint policy allows without configuration.
|
||||
assert FakeProvider.last_endpoint == "http://localhost:11434/v1"
|
||||
|
||||
|
||||
def test_system_prompt_and_model_override_are_honoured(client):
|
||||
resp = client.post("/api/chat/stream", json={
|
||||
"messages": [
|
||||
{"role": "system", "content": "Be terse."},
|
||||
{"role": "user", "content": "hi"},
|
||||
],
|
||||
"model": "some/other-model",
|
||||
"model": "some-other-model",
|
||||
})
|
||||
assert resp.status_code == 200, resp.text
|
||||
# BYOK: any model the user names is passed straight through, on their key.
|
||||
assert FakeProvider.last_model == "some/other-model"
|
||||
assert FakeProvider.last_key == "sk-my-own-key"
|
||||
assert FakeProvider.last_model == "some-other-model"
|
||||
assert FakeProvider.last_messages[0] == {"role": "system", "content": "Be terse."}
|
||||
|
||||
|
||||
def test_demo_key_pins_model_to_whitelist(client, monkeypatch):
|
||||
_demo(monkeypatch)
|
||||
resp = _send(client, model="expensive/paid-model")
|
||||
assert resp.status_code == 200, resp.text
|
||||
# Refused visibly: the whitelisted model runs instead, with a note. The
|
||||
# paid slug must never reach the wire alongside the server-funded key.
|
||||
assert FakeProvider.last_model == "free/allowed"
|
||||
assert FakeProvider.last_key == "demo-key"
|
||||
assert '"type": "note"' in resp.text
|
||||
|
||||
# A whitelisted model is still selectable on the demo key.
|
||||
_demo(monkeypatch, ["free/allowed", "free/second"])
|
||||
resp = _send(client, model="free/second")
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "free/second"
|
||||
|
||||
|
||||
def test_demo_key_ignores_an_off_whitelist_settings_model(client, monkeypatch):
|
||||
"""The override is not the only untrusted input. `Settings.model` is
|
||||
also user-set, and it must be pinned the same way when there is no
|
||||
BYOK key."""
|
||||
_demo(monkeypatch)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().model = "expensive/paid-model"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "free/allowed"
|
||||
|
||||
|
||||
def test_demo_key_endpoint_cannot_be_redirected(client, monkeypatch):
|
||||
"""A user-controlled `endpoint_url` would leak the key itself, which is
|
||||
worse than spending it. The demo branch pins the URL too."""
|
||||
_demo(monkeypatch)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().endpoint_url = "http://attacker.example/v1"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
assert _send(client).status_code == 200
|
||||
assert FakeProvider.last_endpoint == "http://demo"
|
||||
assert FakeProvider.last_key == "demo-key"
|
||||
|
||||
|
||||
def test_provider_config_refuses_server_funded_paid_model(monkeypatch):
|
||||
"""The structural backstop: a hand-built config (a future code path that
|
||||
forgets to go through resolve_provider_config) cannot run a
|
||||
server-funded turn on an off-whitelist model."""
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
with pytest.raises(ValueError):
|
||||
auth.ProviderConfig("http://demo", "demo-key", "expensive/paid-model", True)
|
||||
auth.ProviderConfig("http://demo", "demo-key", "free/allowed", True) # whitelisted: fine
|
||||
# The user's own key with any model stays fine.
|
||||
auth.ProviderConfig("http://any", "sk-mine", "expensive/paid-model", False)
|
||||
|
||||
|
||||
def test_byok_user_may_reuse_the_demo_keys_value(client, monkeypatch):
|
||||
"""Regression: the demo key is just an OpenRouter key, so a user can paste
|
||||
that same value into their own Settings. That is still BYOK, because
|
||||
the user is paying, and it must not trip the guard. It used to raise
|
||||
on every resolution, which returned a 500 from `GET /auth/me` and
|
||||
broke the entire SPA (no nav, no chat)."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "shared-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
def test_a_request_with_no_model_anywhere_is_refused(client):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.api_key = "shared-key" # same value, but supplied by the user
|
||||
settings.model = "expensive/paid-model" # their spend, their choice
|
||||
settings.model = ""
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
assert client.get("/api/auth/me").status_code == 200
|
||||
assert client.get("/api/chat/config").status_code == 200
|
||||
resp = _send(client)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert FakeProvider.last_model == "expensive/paid-model"
|
||||
assert FakeProvider.last_key == "shared-key"
|
||||
assert _send(client).status_code == 400
|
||||
|
||||
|
||||
def test_resolve_provider_config_is_the_single_choke_point(monkeypatch):
|
||||
"""Turns, AI Chat, and the connection test all resolve through this one
|
||||
function, so pinning it here pins every caller. No DB or HTTP needed."""
|
||||
monkeypatch.setattr(auth, "demo_enabled", lambda: True)
|
||||
monkeypatch.setattr(auth, "DEMO_API_KEY", "demo-key")
|
||||
monkeypatch.setattr(auth, "DEMO_ENDPOINT_URL", "http://demo")
|
||||
monkeypatch.setattr(auth, "DEMO_MODELS", ["free/allowed"])
|
||||
|
||||
# No key of their own: both endpoint and model are pinned, regardless
|
||||
# of what they set.
|
||||
no_key = models.Settings(endpoint_url="http://mine/v1", api_key="", model="expensive/paid")
|
||||
assert auth.resolve_provider_config(no_key) == auth.ProviderConfig(
|
||||
"http://demo", "demo-key", "free/allowed", True)
|
||||
assert auth.resolve_provider_config(
|
||||
no_key, model_override="expensive/paid").model == "free/allowed"
|
||||
assert auth.resolve_provider_config(
|
||||
no_key, model_override="free/allowed").model == "free/allowed"
|
||||
|
||||
# Their own key: their endpoint, their key, their choice of model.
|
||||
byok = models.Settings(endpoint_url="http://mine/v1", api_key="sk-mine", model="expensive/paid")
|
||||
assert auth.resolve_provider_config(byok) == auth.ProviderConfig(
|
||||
"http://mine/v1", "sk-mine", "expensive/paid", False)
|
||||
|
||||
|
||||
def test_oversized_conversation_is_refused(client, monkeypatch):
|
||||
_byok(monkeypatch)
|
||||
def test_oversized_conversation_is_refused(client):
|
||||
huge = "x" * 90_000
|
||||
resp = client.post("/api/chat/stream", json={
|
||||
"messages": [{"role": "user", "content": huge} for _ in range(5)],
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Deleting a turn puts the shared state back.
|
||||
|
||||
`script_state` and `world_state` belong to the adventure, not to the node
|
||||
`world_state` belongs to the adventure, not to the node
|
||||
that changed them. Undo, retry, a take and a branch switch all restore them;
|
||||
the delete endpoint did not. Deleting an AI turn removed the text and left
|
||||
everything the turn did to the numbers standing.
|
||||
@@ -23,7 +23,7 @@ from app import auth, limits, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# `mana` carries a cooldown, so a clock that was not rolled back shows up as
|
||||
# a refusal rather than as a number that is merely off.
|
||||
@@ -31,20 +31,19 @@ SCHEMA = {
|
||||
"player": {
|
||||
"hp": {"min": 0, "max": 100, "initial": 100},
|
||||
"mana": {"min": 0, "max": 50, "initial": 50, "cooldown": 2},
|
||||
# The per-turn counter these tests measure rollbacks with. Unbounded and
|
||||
# uncapped on purpose, so every turn's +10 lands in full. See
|
||||
# `fakes.gold_reply`.
|
||||
"gold": {"min": 0, "max": 1_000_000, "initial": 0},
|
||||
}
|
||||
}
|
||||
|
||||
# Ten gold a turn. A total that only ever climbs makes a missing rollback
|
||||
# obvious: it is off by exactly one turn's worth.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
DRAIN = 'Drained.\n```state\n{"player.mana": -10}\n```'
|
||||
# The same turn, also banking the per-turn counter the rollback tests measure.
|
||||
DRAIN_AND_GOLD = 'Drained.\n```state\n{"player.mana": -10, "player.gold": 10}\n```'
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -60,24 +59,18 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Tower", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100, "mana": 50}},
|
||||
world_state={"player": {"hp": 100, "mana": 50, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You begin."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = [DRAIN]
|
||||
ScriptedProvider.replies = [DRAIN_AND_GOLD]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -116,7 +109,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -168,17 +161,17 @@ def test_the_next_turn_is_not_refused_for_a_deleted_turn_s_cooldown(client):
|
||||
assert [c for c in _last_changes(client.adv_id) if c["kind"] == "rejected"] == []
|
||||
|
||||
|
||||
def test_deleting_the_ai_turn_rewinds_the_script_state(client):
|
||||
"""The same restore, on the other half of the shared state. Without it a
|
||||
replayed turn stacks its script run on top of the deleted one's."""
|
||||
def test_deleting_the_ai_turn_rewinds_the_counter(client):
|
||||
"""The same restore, seen through a value that only ever climbs. Without it
|
||||
a replayed turn stacks its effects on top of the deleted one's."""
|
||||
_play(client)
|
||||
assert _state(client.adv_id)[0] == {"gold": 10}
|
||||
assert _state(client.adv_id)[0]["gold"] == 10
|
||||
|
||||
_delete(client, _ai_rows(client.adv_id)[-1].id)
|
||||
assert _state(client.adv_id)[0] == {}
|
||||
assert _state(client.adv_id)[0]["gold"] == 0
|
||||
|
||||
_continue(client)
|
||||
assert _state(client.adv_id)[0] == {"gold": 10}, "one turn of gold, not two"
|
||||
assert _state(client.adv_id)[0]["gold"] == 10, "one turn of gold, not two"
|
||||
|
||||
|
||||
# ------------------------------------------------- deleting further back
|
||||
@@ -190,7 +183,7 @@ def test_deleting_a_turn_the_story_moved_past_leaves_the_tip_alone(client):
|
||||
_play(client)
|
||||
_play(client, "press on")
|
||||
before = _state(client.adv_id)
|
||||
assert before[0] == {"gold": 20}
|
||||
assert before[0]["gold"] == 20
|
||||
|
||||
first_ai = _ai_rows(client.adv_id)[0]
|
||||
assert _delete(client, first_ai.id).status_code == 204
|
||||
|
||||
@@ -99,7 +99,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -57,7 +57,6 @@ def client(monkeypatch):
|
||||
adv_id, user_id = adventure.id, user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Which inference endpoints the product will and will not talk to.
|
||||
|
||||
The endpoint is where the whole campaign goes: prompts, narration, retrieved
|
||||
memories, embedding inputs. `app/endpoints.py` is the rule that keeps that on
|
||||
the user's own infrastructure, and these tests are what stops the rule quietly
|
||||
widening.
|
||||
|
||||
Nothing here touches the network. Hostnames are resolved through a stub, so the
|
||||
tests are about the *policy*, run identically offline, and cannot be made to
|
||||
pass or fail by whatever DNS the machine happens to have.
|
||||
|
||||
python -m pytest tests/test_endpoint_policy.py -v
|
||||
"""
|
||||
import ipaddress
|
||||
import socket
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, endpoints, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.providers import OpenAICompatibleProvider, ProviderError
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def resolves(monkeypatch):
|
||||
"""Points every hostname at addresses the test names."""
|
||||
table: dict[str, list[str]] = {}
|
||||
|
||||
def fake_getaddrinfo(host, port, *a, **k):
|
||||
# An address literal resolves to itself, as the real resolver does.
|
||||
try:
|
||||
ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
if host not in table:
|
||||
raise socket.gaierror(-2, "Name or service not known")
|
||||
answers = table[host]
|
||||
else:
|
||||
answers = [host]
|
||||
return [
|
||||
(
|
||||
socket.AF_INET6 if ":" in ip else socket.AF_INET,
|
||||
socket.SOCK_STREAM, 6, "", (ip, port or 0),
|
||||
)
|
||||
for ip in answers
|
||||
]
|
||||
|
||||
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
|
||||
return table
|
||||
|
||||
|
||||
# --- allowed: the user's own machine, and the user's own network ----------
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"http://127.0.0.1:11434/v1",
|
||||
"http://[::1]:11434/v1",
|
||||
"http://192.168.1.50:11434/v1", # RFC1918
|
||||
"http://10.0.0.7:11434/v1", # RFC1918
|
||||
"http://172.16.4.4:11434/v1", # RFC1918
|
||||
"https://192.168.1.50:8443/v1", # TLS on the LAN
|
||||
"http://100.100.5.6:11434/v1", # CGNAT, which is what a mesh VPN hands out
|
||||
"http://[fd00::1]:11434/v1", # IPv6 unique-local
|
||||
])
|
||||
def test_local_and_lan_addresses_are_allowed(url, resolves):
|
||||
assert endpoints.rejection_reason(url) is None
|
||||
|
||||
|
||||
def test_a_hostname_resolving_to_the_lan_is_allowed(resolves):
|
||||
resolves["ollama.home.arpa"] = ["192.168.1.50"]
|
||||
assert endpoints.rejection_reason("https://ollama.home.arpa:8443/v1") is None
|
||||
|
||||
|
||||
# --- refused: anywhere else ----------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"http://8.8.8.8:11434/v1",
|
||||
"http://1.1.1.1:11434/v1",
|
||||
"http://[2001:4860:4860::8888]:11434/v1",
|
||||
])
|
||||
def test_public_addresses_are_refused(url, resolves):
|
||||
reason = endpoints.rejection_reason(url)
|
||||
assert reason is not None
|
||||
assert "public Internet address" in reason
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"https://198.51.100.9/v1", # TEST-NET-2, a documentation range
|
||||
"http://0.0.0.0:11434/v1", # "this host", not an address to dial
|
||||
])
|
||||
def test_addresses_that_are_neither_local_nor_public_are_refused(url, resolves):
|
||||
"""Deny by default. `ipaddress` calls both of these "private", which is why
|
||||
the policy names the networks it allows instead of asking that question."""
|
||||
assert endpoints.rejection_reason(url) is not None
|
||||
|
||||
|
||||
def test_a_hostname_resolving_to_the_public_internet_is_refused(resolves):
|
||||
resolves["ollama.example.com"] = ["93.184.216.34"]
|
||||
reason = endpoints.rejection_reason("https://ollama.example.com/v1")
|
||||
assert reason is not None and "public Internet address" in reason
|
||||
|
||||
|
||||
def test_a_split_horizon_answer_is_refused(resolves):
|
||||
"""One public address among the answers is enough. A name that resolves to
|
||||
both a LAN address and a public one must not be usable: which one the
|
||||
request actually reaches is not ours to decide."""
|
||||
resolves["sneaky.example"] = ["192.168.1.50", "93.184.216.34"]
|
||||
assert endpoints.rejection_reason("http://sneaky.example:11434/v1") is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", [
|
||||
"openrouter.ai", "api.openai.com", "api.anthropic.com", "api.groq.com",
|
||||
])
|
||||
def test_known_cloud_providers_are_named_in_the_refusal(host, resolves):
|
||||
"""They would be refused by address anyway — every one resolves publicly.
|
||||
Naming them turns 'your DNS might be broken' into 'this build has no cloud
|
||||
provider support'."""
|
||||
resolves[host] = ["192.168.1.50"] # even if DNS said otherwise
|
||||
reason = endpoints.rejection_reason(f"https://{host}/v1")
|
||||
assert reason is not None
|
||||
assert "cloud inference service" in reason
|
||||
|
||||
|
||||
def test_a_subdomain_of_a_cloud_provider_is_refused(resolves):
|
||||
resolves["eu.api.openai.com"] = ["192.168.1.50"]
|
||||
assert endpoints.rejection_reason("https://eu.api.openai.com/v1") is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"ftp://127.0.0.1/v1",
|
||||
"file:///etc/passwd",
|
||||
"127.0.0.1:11434", # no scheme
|
||||
"",
|
||||
"http://",
|
||||
])
|
||||
def test_things_that_are_not_usable_urls_are_refused(url, resolves):
|
||||
assert endpoints.rejection_reason(url) is not None
|
||||
|
||||
|
||||
def test_an_unresolvable_host_is_refused_with_advice(resolves):
|
||||
reason = endpoints.rejection_reason("http://nope.invalid:11434/v1")
|
||||
assert reason is not None and "could not be resolved" in reason
|
||||
|
||||
|
||||
# --- the rule is applied, not merely available ---------------------------
|
||||
|
||||
@pytest.fixture()
|
||||
def client(monkeypatch):
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def test_saving_a_public_endpoint_is_refused(client, resolves):
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
r = client.put("/api/settings", json={"endpoint_url": "https://openrouter.ai/api/v1"})
|
||||
assert r.status_code == 400, r.text
|
||||
assert "can't be used" in r.json()["detail"]
|
||||
# And it was not written.
|
||||
assert client.get("/api/settings").json()["endpoint_url"] != "https://openrouter.ai/api/v1"
|
||||
|
||||
|
||||
def test_saving_a_lan_endpoint_is_allowed(client, resolves):
|
||||
r = client.put("/api/settings", json={"endpoint_url": "http://192.168.1.50:11434/v1"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.get("/api/settings").json()["endpoint_url"] == "http://192.168.1.50:11434/v1"
|
||||
|
||||
|
||||
def test_the_connection_test_reports_a_refused_endpoint_as_such(client, resolves):
|
||||
"""A row edited by hand, or a name that has started resolving elsewhere,
|
||||
must not simply look 'unreachable'."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.Settings).first().endpoint_url = "https://openrouter.ai/api/v1"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
body = client.post("/api/settings/test").json()
|
||||
assert body["ok"] is False
|
||||
assert body["kind"] == "rejected"
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_the_provider_refuses_before_it_sends_anything(resolves):
|
||||
"""The check that actually matters. Whatever is stored, no request leaves
|
||||
for an address outside the policy — so a database edited behind the app's
|
||||
back cannot turn into an exfiltration path."""
|
||||
resolves["openrouter.ai"] = ["93.184.216.34"]
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "m")
|
||||
with pytest.raises(ProviderError) as exc:
|
||||
async for _ in provider.generate(
|
||||
_parts(), temperature=0.8, max_tokens=10
|
||||
):
|
||||
pass
|
||||
assert "can't be used" in str(exc.value)
|
||||
|
||||
|
||||
def _parts():
|
||||
from app.providers.base import PromptParts
|
||||
return PromptParts(system="s", story="t")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anyio_backend():
|
||||
return "asyncio"
|
||||
@@ -1,205 +0,0 @@
|
||||
"""Guest retention policy: app/cleanup.py.
|
||||
|
||||
These tests cover the two things that matter. Idle guests and their whole
|
||||
data graph must actually be deleted, and nothing else must ever be
|
||||
deleted.
|
||||
|
||||
python -m pytest tests/test_guest_cleanup.py -v
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from app import cleanup, models
|
||||
from app.database import Base
|
||||
from app.migrations import bootstrap
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db(tmp_path):
|
||||
engine = create_engine(f"sqlite:///{tmp_path/'t.db'}", connect_args={"check_same_thread": False})
|
||||
|
||||
@event.listens_for(engine, "connect")
|
||||
def _fk(dbapi_connection, _record):
|
||||
# The whole policy relies on ON DELETE CASCADE. SQLite ignores every
|
||||
# one of them unless this is set, the same as database.py does.
|
||||
cur = dbapi_connection.cursor()
|
||||
cur.execute("PRAGMA foreign_keys=ON")
|
||||
cur.close()
|
||||
|
||||
bootstrap(engine)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
session = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)()
|
||||
yield session
|
||||
session.close()
|
||||
|
||||
|
||||
NOW = models.utcnow().replace(tzinfo=None)
|
||||
|
||||
|
||||
def make_user(db, *, days_idle=None, days_old=0, guest=True, email=None):
|
||||
"""A user last seen `days_idle` ago (None = never seen, only created)."""
|
||||
user = models.User(
|
||||
is_guest=guest,
|
||||
email=email,
|
||||
password_hash=None if email is None else "x",
|
||||
created_at=NOW - timedelta(days=days_old),
|
||||
last_seen_at=None if days_idle is None else NOW - timedelta(days=days_idle),
|
||||
)
|
||||
db.add(user)
|
||||
db.commit()
|
||||
return user
|
||||
|
||||
|
||||
def sweep(db):
|
||||
return cleanup.delete_stale_guests(db, now=NOW)
|
||||
|
||||
|
||||
def alive(db, user_id):
|
||||
# A count, not db.get: the sweep deletes with synchronize_session=False, so
|
||||
# the session's identity map still holds the object and db.get would answer
|
||||
# from memory without ever asking the database.
|
||||
return db.query(models.User).filter(models.User.id == user_id).count() == 1
|
||||
|
||||
|
||||
# ---------- what goes ----------
|
||||
|
||||
def test_deletes_guest_idle_past_the_window(db):
|
||||
user = make_user(db, days_idle=6)
|
||||
assert sweep(db) == 1
|
||||
assert not alive(db, user.id)
|
||||
|
||||
|
||||
def test_keeps_guest_inside_the_window(db):
|
||||
user = make_user(db, days_idle=4)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_boundary_is_not_yet_stale(db):
|
||||
# Exactly 5 days survives. The comparison is strict.
|
||||
user = make_user(db, days_idle=cleanup.RETENTION_DAYS)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_never_seen_guest_falls_back_to_created_at(db):
|
||||
"""last_seen_at is NULL until a guest's second request (auth._touch runs
|
||||
hourly), so a coalesce-less query would delete brand-new visitors."""
|
||||
fresh = make_user(db, days_idle=None, days_old=0)
|
||||
stale = make_user(db, days_idle=None, days_old=9)
|
||||
assert sweep(db) == 1
|
||||
assert alive(db, fresh.id)
|
||||
assert not alive(db, stale.id)
|
||||
|
||||
|
||||
def test_recent_visit_beats_an_old_created_at(db):
|
||||
# A long-standing guest who came back yesterday stays.
|
||||
user = make_user(db, days_idle=1, days_old=90)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
# ---------- what must never go ----------
|
||||
|
||||
def test_spares_registered_users(db):
|
||||
"""Registering upgrades the guest row in place. An idle account here is
|
||||
a real user with real data, which is what signing up is meant to
|
||||
protect."""
|
||||
user = make_user(db, days_idle=400, guest=False, email="a@b.com")
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_spares_the_local_mode_user(db):
|
||||
# email NULL but is_guest False: local mode's implicit owner of everything.
|
||||
user = make_user(db, days_idle=400, guest=False)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_spares_a_guest_flagged_row_that_has_an_email(db):
|
||||
# This row should not exist, but both clauses are checked so it cannot be collected.
|
||||
user = make_user(db, days_idle=400, guest=True, email="odd@b.com")
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_leaves_seeded_public_scenarios_alone(db):
|
||||
"""Seeded demo content has user_id NULL, so it is outside the filter."""
|
||||
seeded = models.Scenario(user_id=None, is_public=True, title="Demo")
|
||||
db.add(seeded)
|
||||
make_user(db, days_idle=30)
|
||||
db.commit()
|
||||
assert sweep(db) == 1
|
||||
assert db.query(models.Scenario).filter(models.Scenario.id == seeded.id).count() == 1
|
||||
|
||||
|
||||
def test_disabled_when_retention_is_zero(db, monkeypatch):
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 0)
|
||||
user = make_user(db, days_idle=999)
|
||||
assert sweep(db) == 0
|
||||
assert alive(db, user.id)
|
||||
|
||||
|
||||
def test_enabled_requires_multi_user(monkeypatch):
|
||||
from app import auth
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
assert cleanup.enabled() is False
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 5)
|
||||
assert cleanup.enabled() is True
|
||||
monkeypatch.setattr(cleanup, "RETENTION_DAYS", 0)
|
||||
assert cleanup.enabled() is False
|
||||
|
||||
|
||||
# ---------- the cascade ----------
|
||||
|
||||
def test_deletes_the_whole_data_graph(db):
|
||||
"""One DELETE must remove the adventure, its actions and memories, the
|
||||
story cards, and the settings row. Nothing is loaded into Python, so if
|
||||
the FK cascade does not reach a table, its rows are silently orphaned,
|
||||
or the statement fails, instead of being removed."""
|
||||
user = make_user(db, days_idle=30)
|
||||
scenario = models.Scenario(user_id=user.id, title="S")
|
||||
db.add(scenario)
|
||||
db.commit()
|
||||
adventure = models.Adventure(user_id=user.id, scenario_id=scenario.id, title="A")
|
||||
db.add(adventure)
|
||||
db.commit()
|
||||
db.add_all([
|
||||
models.Action(adventure_id=adventure.id, type="ai", text="t"),
|
||||
models.Memory(adventure_id=adventure.id, text="m", source_start=0, source_end=0),
|
||||
models.StoryCard(adventure_id=adventure.id, name="c"),
|
||||
models.Settings(user_id=user.id),
|
||||
])
|
||||
db.commit()
|
||||
|
||||
assert sweep(db) == 1
|
||||
|
||||
for model in (models.Scenario, models.Adventure, models.Action,
|
||||
models.Memory, models.StoryCard, models.Settings):
|
||||
assert db.query(model).count() == 0, f"{model.__name__} rows survived"
|
||||
|
||||
|
||||
def test_one_users_cleanup_does_not_touch_another(db):
|
||||
keeper = make_user(db, days_idle=1)
|
||||
keep_adv = models.Adventure(user_id=keeper.id, title="mine")
|
||||
goner = make_user(db, days_idle=30)
|
||||
db.add_all([keep_adv, models.Adventure(user_id=goner.id, title="theirs")])
|
||||
db.commit()
|
||||
|
||||
assert sweep(db) == 1
|
||||
remaining = db.query(models.Adventure).all()
|
||||
assert [a.title for a in remaining] == ["mine"]
|
||||
|
||||
|
||||
def test_sweep_swallows_errors(monkeypatch):
|
||||
"""A broken cleanup must not take the app down (same rule as seeding)."""
|
||||
monkeypatch.setattr(cleanup.auth, "MULTI_USER", True)
|
||||
monkeypatch.setattr(cleanup, "delete_stale_guests",
|
||||
lambda *a, **k: (_ for _ in ()).throw(RuntimeError("boom")))
|
||||
assert cleanup.sweep() == 0
|
||||
@@ -0,0 +1,269 @@
|
||||
"""What M2 removed stays removed, and what it must not break stays working.
|
||||
|
||||
A subtractive milestone needs tests that fail if the surface grows back. These
|
||||
are cheap, blunt, and deliberately not clever: they assert against the running
|
||||
app's route table, the shipped configuration files, and the settings API.
|
||||
|
||||
python -m pytest tests/test_local_only_surface.py -v
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi import Depends
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import auth, models
|
||||
from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.providers.openai_compatible import (
|
||||
CONNECT_TIMEOUT, DEFAULT_READ_TIMEOUT, OpenAICompatibleProvider,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _paths() -> set[str]:
|
||||
"""Every path the app serves, including those inside included routers."""
|
||||
found = set()
|
||||
|
||||
def walk(routes):
|
||||
for r in routes:
|
||||
path = getattr(r, "path", None)
|
||||
if path:
|
||||
found.add(path)
|
||||
walk(getattr(r, "routes", []) or [])
|
||||
|
||||
walk(app.routes)
|
||||
return found
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client():
|
||||
Base.metadata.create_all(bind=engine)
|
||||
setup = SessionLocal()
|
||||
user = models.User(is_guest=False)
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, model="test-model"))
|
||||
setup.commit()
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
return db.get(models.User, user_id)
|
||||
|
||||
app.dependency_overrides[auth.get_current_user] = _current_user
|
||||
c = TestClient(app)
|
||||
try:
|
||||
yield c
|
||||
finally:
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
# --- the removed surfaces ------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("prefix", ["/api/auth", "/api/analytics", "/api/scripts"])
|
||||
def test_no_route_serves_a_removed_subsystem(prefix):
|
||||
"""Accounts, the visitor dashboard, and campaign scripting are gone as
|
||||
routes, not merely hidden behind a flag."""
|
||||
assert not [p for p in _paths() if p.startswith(prefix)], prefix
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", [
|
||||
"/api/auth/me", "/api/auth/login", "/api/auth/register", "/api/auth/logout",
|
||||
"/api/analytics/summary", "/api/analytics/collect", "/api/analytics/access",
|
||||
"/api/scripts", "/api/adventures/1/scripts", "/api/adventures/1/script-state",
|
||||
])
|
||||
def test_a_removed_endpoint_answers_404(client, path):
|
||||
assert client.get(path).status_code == 404, path
|
||||
|
||||
|
||||
def test_the_application_has_no_scripting_engine():
|
||||
with pytest.raises(ImportError):
|
||||
__import__("app.scripting")
|
||||
|
||||
|
||||
def test_no_module_imports_quickjs():
|
||||
"""The dependency is gone from requirements; this catches an import that
|
||||
would put it back."""
|
||||
for py in (REPO / "backend" / "app").rglob("*.py"):
|
||||
assert "import quickjs" not in py.read_text(), py
|
||||
|
||||
|
||||
def test_requirements_carry_no_hosted_dependencies():
|
||||
text = (REPO / "backend" / "requirements.txt").read_text()
|
||||
for gone in ("quickjs", "psycopg", "cryptography"):
|
||||
assert gone not in text, gone
|
||||
|
||||
|
||||
def test_no_render_deployment_config():
|
||||
assert not (REPO / "render.yaml").exists()
|
||||
|
||||
|
||||
# --- no cloud provider, no key ------------------------------------------
|
||||
|
||||
def test_settings_expose_no_api_key_field(client):
|
||||
body = client.get("/api/settings").json()
|
||||
assert "api_key" not in body
|
||||
assert "has_api_key" not in body
|
||||
|
||||
|
||||
def test_an_api_key_cannot_be_set_through_the_api(client):
|
||||
"""Pydantic ignores unknown fields, so this asserts the value does not
|
||||
land rather than that the request is refused."""
|
||||
client.put("/api/settings", json={"api_key": "sk-should-not-stick"})
|
||||
db = SessionLocal()
|
||||
try:
|
||||
assert db.query(models.Settings).first().api_key == ""
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_the_provider_sends_no_authorization_header():
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
assert "Authorization" not in provider._headers()
|
||||
|
||||
|
||||
# --- the model timeout ---------------------------------------------------
|
||||
|
||||
def test_the_default_timeout_is_generous_but_finite():
|
||||
"""M1 measured a cold model load exceeding the inherited hardcoded 120s on
|
||||
a CPU-only host. It must be longer than that, and it must be a number."""
|
||||
assert DEFAULT_READ_TIMEOUT > 120
|
||||
assert DEFAULT_READ_TIMEOUT <= 3600
|
||||
|
||||
|
||||
def test_connect_stays_short_while_reading_stays_patient():
|
||||
"""A wrong address should fail in seconds; a loading model should not."""
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
timeout = provider._timeout()
|
||||
assert timeout.connect == CONNECT_TIMEOUT <= 30
|
||||
assert timeout.read == DEFAULT_READ_TIMEOUT
|
||||
|
||||
|
||||
def test_the_timeout_is_configurable(client):
|
||||
r = client.put("/api/settings", json={"model_timeout_seconds": 900})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.get("/api/settings").json()["model_timeout_seconds"] == 900
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [0, 29, 3601, -1])
|
||||
def test_an_unusable_timeout_is_refused(client, value):
|
||||
"""Not zero, not negative, and not "wait forever" spelled as a big number."""
|
||||
assert client.put(
|
||||
"/api/settings", json={"model_timeout_seconds": value}
|
||||
).status_code == 422
|
||||
|
||||
|
||||
def test_the_provider_honours_the_configured_timeout():
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m", read_timeout=45)
|
||||
assert provider._timeout().read == 45
|
||||
|
||||
|
||||
# --- the storyteller stays on loopback -----------------------------------
|
||||
|
||||
def test_the_native_start_scripts_bind_loopback():
|
||||
for script in ("start.sh", "start.ps1"):
|
||||
text = (REPO / script).read_text(errors="ignore")
|
||||
assert "--host 127.0.0.1" in text, script
|
||||
assert "--host 0.0.0.0" not in text, script
|
||||
|
||||
|
||||
def test_compose_publishes_to_loopback_only():
|
||||
"""The container listens on 0.0.0.0 because a published port cannot reach
|
||||
anything else. What must stay loopback is the *host* side of the mapping."""
|
||||
text = (REPO / "docker-compose.yml").read_text()
|
||||
published = re.findall(r'^\s*-\s*"([^"]+)"', text, re.M)
|
||||
assert published, "no published ports found — has the file moved?"
|
||||
for mapping in published:
|
||||
assert mapping.startswith("127.0.0.1:"), mapping
|
||||
|
||||
|
||||
# --- every provider factory must actually build ---------------------------
|
||||
|
||||
def test_every_provider_factory_builds_from_a_real_settings_row(client):
|
||||
"""M2 shipped with a defect this test would have caught.
|
||||
|
||||
`Settings.api_key_plain` was removed with the API key, but `memorybank`'s
|
||||
two provider factories still read it. Nothing failed at import, and no test
|
||||
noticed, because every memory test stubs those factories out — so the break
|
||||
only appeared at runtime, in a background task, as a swallowed
|
||||
`AttributeError` that silently stopped summaries and embeddings.
|
||||
|
||||
Constructing each factory from a real row is the cheapest thing that would
|
||||
have caught it, and it catches the same shape of mistake next time a
|
||||
Settings column moves.
|
||||
"""
|
||||
from app import memorybank
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
settings = db.query(models.Settings).first()
|
||||
settings.embedding_model = "nomic-embed-text:latest"
|
||||
settings.summary_model = ""
|
||||
db.commit()
|
||||
|
||||
summary = memorybank.summary_provider(settings)
|
||||
assert summary.model == settings.model # falls back to the narrator
|
||||
assert summary.base_url == settings.endpoint_url.rstrip("/")
|
||||
|
||||
embed = memorybank.embedding_provider(settings)
|
||||
assert embed.model == "nomic-embed-text:latest"
|
||||
|
||||
turn = OpenAICompatibleProvider(
|
||||
settings.endpoint_url, settings.model, settings.api_mode,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
assert turn.read_timeout == settings.model_timeout_seconds
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_the_configured_timeout_reaches_every_generating_client(client, monkeypatch):
|
||||
"""The second defect this review caught. The setting was stored, validated
|
||||
and exposed, and then not passed to the provider — so the turn engine kept
|
||||
using the module default and "configurable" was a claim rather than a fact.
|
||||
|
||||
Each generating path is driven for real and the constructed provider is
|
||||
recorded. Embeddings are deliberately excluded: they are short, never
|
||||
cold-load a large model, and keep their own shorter constant.
|
||||
"""
|
||||
from app import memorybank
|
||||
from app.routers import chat as chat_router
|
||||
from app.routers.adventures import turns as turns_router
|
||||
|
||||
seen = []
|
||||
|
||||
class Recorder:
|
||||
last_usage = None
|
||||
|
||||
def __init__(self, endpoint_url, model, api_mode="chat", read_timeout=None):
|
||||
seen.append(read_timeout)
|
||||
|
||||
async def generate(self, *a, **k):
|
||||
yield ("text", "narration")
|
||||
|
||||
async def chat(self, *a, **k):
|
||||
yield ("text", "reply")
|
||||
|
||||
monkeypatch.setattr(turns_router, "OpenAICompatibleProvider", Recorder)
|
||||
monkeypatch.setattr(chat_router, "OpenAICompatibleProvider", Recorder)
|
||||
monkeypatch.setattr(memorybank, "OpenAICompatibleProvider", Recorder)
|
||||
|
||||
assert client.put("/api/settings", json={"model_timeout_seconds": 777}).status_code == 200
|
||||
|
||||
adv = client.post("/api/adventures", json={"title": "T"}).json()["id"]
|
||||
client.post(f"/api/adventures/{adv}/actions", json={"type": "story", "text": "hello"})
|
||||
assert seen and seen[-1] == 777, f"turn engine used {seen[-1]!r}"
|
||||
|
||||
client.post("/api/chat/stream", json={"messages": [{"role": "user", "content": "hi"}]})
|
||||
assert seen[-1] == 777, f"chat used {seen[-1]!r}"
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
memorybank.summary_provider(db.query(models.Settings).first())
|
||||
finally:
|
||||
db.close()
|
||||
assert seen[-1] == 777, f"summarizer used {seen[-1]!r}"
|
||||
@@ -128,7 +128,7 @@ def fill_bank(db, adventure, *, count=2):
|
||||
def options(**overrides):
|
||||
args = dict(write=False, adventure=None, email=None, limit=None,
|
||||
include_forgotten=False, embed=False, endpoint=None,
|
||||
model=None, api_key=None)
|
||||
model=None)
|
||||
args.update(overrides)
|
||||
return argparse.Namespace(**args)
|
||||
|
||||
@@ -248,23 +248,6 @@ def test_the_rewrite_prompt_is_the_one_the_app_sends(db):
|
||||
|
||||
# --------------------------------------------------------------------- the tool
|
||||
|
||||
def test_the_database_line_carries_no_password():
|
||||
"""The report names the database it is about to rewrite. That line ends up
|
||||
in a console, a screenshot or a pasted bug report."""
|
||||
shown = rewrite_memories.safe_dsn(
|
||||
"postgresql://parth:hunter2@ep-cool-frost.us-east-1.aws.neon.tech/aidnd"
|
||||
"?sslmode=require")
|
||||
assert "hunter2" not in shown
|
||||
assert "sslmode" not in shown # a password can be passed there too
|
||||
assert shown == ("postgresql://parth@ep-cool-frost.us-east-1.aws.neon.tech"
|
||||
"/aidnd")
|
||||
|
||||
|
||||
def test_an_unparseable_database_url_shows_nothing_at_all():
|
||||
assert rewrite_memories.safe_dsn("not-a-url") == "(configured)"
|
||||
|
||||
|
||||
|
||||
def test_without_write_nothing_changes(db, monkeypatch):
|
||||
adventure = make_adventure(db)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
@@ -316,9 +299,12 @@ def test_a_hand_written_memory_is_left_alone(db, monkeypatch):
|
||||
assert typed.text == "Kaelen owes a debt."
|
||||
|
||||
|
||||
def test_an_owner_with_no_api_key_is_skipped(db, monkeypatch):
|
||||
"""Summarization spends the user's own key and never the shared demo key."""
|
||||
adventure = make_adventure(db, key="")
|
||||
def test_an_adventure_with_no_model_configured_is_skipped(db, monkeypatch):
|
||||
"""Nothing to summarize with, so the memory is reported rather than
|
||||
silently left as it was. This used to be about the owner's API key; after
|
||||
M2 the thing that can be missing is the model name."""
|
||||
adventure = make_adventure(db)
|
||||
_blank_model(db, adventure)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
stub = StubSummarizer()
|
||||
monkeypatch.setattr(memorybank, "summary_provider", lambda s: stub)
|
||||
@@ -328,10 +314,10 @@ def test_an_owner_with_no_api_key_is_skipped(db, monkeypatch):
|
||||
assert first.text == "You entered the crypt 1."
|
||||
|
||||
|
||||
def test_an_api_key_on_the_command_line_covers_that_owner(db, monkeypatch):
|
||||
"""The way to summarize for an owner who has no key of their own, and the
|
||||
way to point a run at the local Claude shim instead of a paid endpoint."""
|
||||
adventure = make_adventure(db, key="")
|
||||
def test_a_model_on_the_command_line_covers_that_adventure(db, monkeypatch):
|
||||
"""How to run the rewrite against a model other than the configured one."""
|
||||
adventure = make_adventure(db)
|
||||
_blank_model(db, adventure)
|
||||
first, _ = fill_bank(db, adventure)
|
||||
built: list[tuple] = []
|
||||
|
||||
@@ -340,11 +326,22 @@ def test_an_api_key_on_the_command_line_covers_that_owner(db, monkeypatch):
|
||||
return StubSummarizer()
|
||||
|
||||
monkeypatch.setattr("app.providers.OpenAICompatibleProvider", build)
|
||||
assert run_tool(options(write=True, api_key="sk-cli", model="sonnet",
|
||||
assert run_tool(options(write=True, model="other-model",
|
||||
endpoint="http://127.0.0.1:8787/v1")) == 0
|
||||
db.expire_all()
|
||||
assert first.text == "Kaelen entered the crypt 1."
|
||||
assert built[0][:3] == ("http://127.0.0.1:8787/v1", "sk-cli", "sonnet")
|
||||
assert built[0][:2] == ("http://127.0.0.1:8787/v1", "other-model")
|
||||
|
||||
|
||||
def _blank_model(db, adventure) -> None:
|
||||
settings = (
|
||||
db.query(models.Settings)
|
||||
.filter(models.Settings.user_id == adventure.user_id)
|
||||
.first()
|
||||
)
|
||||
settings.model = ""
|
||||
settings.summary_model = ""
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_limit_stops_early(db, monkeypatch):
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
"""Tests for the SSRF guard on the user-supplied BYOK endpoint_url.
|
||||
|
||||
python -m pytest tests/test_netguard.py -v
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from app import auth, netguard
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def hosted(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
|
||||
|
||||
def _resolves_to(monkeypatch, ip: str):
|
||||
"""Pin `getaddrinfo` so the test exercises the address decision, not real DNS."""
|
||||
monkeypatch.setattr(
|
||||
netguard.socket, "getaddrinfo",
|
||||
lambda *a, **k: [(2, 1, 6, "", (ip, 443))],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ip", [
|
||||
"127.0.0.1", # loopback
|
||||
"169.254.169.254", # cloud metadata (link-local)
|
||||
"10.0.0.5", # RFC1918
|
||||
"192.168.1.1", # RFC1918
|
||||
"172.16.0.9", # RFC1918
|
||||
"0.0.0.0", # unspecified
|
||||
"100.64.0.1", # carrier-grade NAT
|
||||
"::1", # IPv6 loopback
|
||||
"fd00::1", # IPv6 unique-local
|
||||
])
|
||||
def test_blocks_non_public_addresses(hosted, monkeypatch, ip):
|
||||
_resolves_to(monkeypatch, ip)
|
||||
assert netguard.endpoint_block_reason("https://evil.example.com/v1") is not None
|
||||
|
||||
|
||||
def test_allows_public_address(hosted, monkeypatch):
|
||||
_resolves_to(monkeypatch, "104.18.0.1") # a public IP
|
||||
assert netguard.endpoint_block_reason("https://openrouter.ai/api/v1") is None
|
||||
|
||||
|
||||
def test_rejects_non_http_scheme(hosted):
|
||||
assert netguard.endpoint_block_reason("file:///etc/passwd") is not None
|
||||
assert netguard.endpoint_block_reason("gopher://x/") is not None
|
||||
|
||||
|
||||
def test_unresolvable_host_is_blocked(hosted, monkeypatch):
|
||||
def boom(*a, **k):
|
||||
raise netguard.socket.gaierror("no such host")
|
||||
monkeypatch.setattr(netguard.socket, "getaddrinfo", boom)
|
||||
assert netguard.endpoint_block_reason("https://nope.invalid/v1") is not None
|
||||
|
||||
|
||||
def test_noop_in_local_mode(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
# Local installs must reach localhost (Ollama). The guard never blocks local mode.
|
||||
assert netguard.endpoint_block_reason("http://localhost:11434/v1") is None
|
||||
assert netguard.endpoint_block_reason("http://127.0.0.1:11434/v1") is None
|
||||
@@ -275,7 +275,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -17,10 +17,10 @@ Three things must hold, and each is easy to undo by accident:
|
||||
* Moving a section out of the system block does not drop it from the
|
||||
token budget. It is still in the prompt.
|
||||
|
||||
This file also covers two request-level concerns: preferring one
|
||||
OpenRouter upstream, since each upstream holds its own cache, and reading
|
||||
back the usage the endpoint reports, so the hit rate is measurable rather
|
||||
than assumed.
|
||||
This file also covers one request-level concern: reading back the usage
|
||||
the endpoint reports, so the hit rate is measurable rather than assumed.
|
||||
It covered OpenRouter upstream pinning too, until M2 removed cloud
|
||||
provider support.
|
||||
|
||||
python -m pytest tests/test_prompt_caching.py -v
|
||||
"""
|
||||
@@ -40,50 +40,11 @@ SCHEMA = {
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------- preferring one upstream
|
||||
|
||||
def _routed(endpoint, model):
|
||||
provider = OpenAICompatibleProvider(endpoint, "k", model, "chat", 0)
|
||||
body = {"max_tokens": 100}
|
||||
provider._apply_provider_routing(body)
|
||||
return body
|
||||
|
||||
|
||||
def test_openrouter_deepseek_pins_the_upstream():
|
||||
"""Each upstream has its own cache, so routing has to be deterministic."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert body["provider"] == {"order": ["deepseek"]}
|
||||
|
||||
|
||||
def test_fallbacks_stay_on():
|
||||
"""A preference, not a restriction: if deepseek is down the turn still runs
|
||||
somewhere else and merely misses the cache."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert "allow_fallbacks" not in body["provider"]
|
||||
|
||||
|
||||
def test_non_openrouter_endpoints_get_no_provider_field():
|
||||
"""Ollama and other providers reject fields they do not know. This is
|
||||
the same problem the `reasoning` param works around."""
|
||||
body = _routed("http://localhost:11434/v1", "deepseek/deepseek-v4-flash-0731")
|
||||
assert "provider" not in body
|
||||
|
||||
|
||||
def test_unknown_vendors_are_left_alone():
|
||||
"""The vendor half of a slug is not reliably a provider slug: Google's
|
||||
models are served by "google-ai-studio", and there is no "google". A guess
|
||||
would be a routing preference naming an upstream that does not exist."""
|
||||
body = _routed("https://openrouter.ai/api/v1", "google/gemma-4-26b-a4b-it:free")
|
||||
assert "provider" not in body
|
||||
|
||||
|
||||
# ------------------------------------------------------ reading usage back
|
||||
|
||||
def test_usage_is_recorded_from_a_final_chunk():
|
||||
"""In a stream, the usage block arrives in a final chunk that carries
|
||||
no choices, which is why it is read separately from the text
|
||||
extraction."""
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "k", "m")
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
assert provider.last_usage is None
|
||||
provider._record_usage({"choices": [{"delta": {"content": "hi"}}]})
|
||||
assert provider.last_usage is None
|
||||
@@ -95,7 +56,7 @@ def test_usage_is_recorded_from_a_final_chunk():
|
||||
|
||||
|
||||
def test_a_later_chunk_without_usage_does_not_erase_it():
|
||||
provider = OpenAICompatibleProvider("https://openrouter.ai/api/v1", "k", "m")
|
||||
provider = OpenAICompatibleProvider("http://127.0.0.1:11434/v1", "m")
|
||||
provider._record_usage({"usage": {"prompt_tokens": 5}})
|
||||
provider._record_usage({"choices": [{"delta": {"content": "x"}}]})
|
||||
provider._record_usage({"usage": {}})
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
"""Regression tests for the X-Forwarded-For rate-limit bypass and the
|
||||
per-account login throttle added to close it.
|
||||
|
||||
Background: uvicorn's `--forwarded-allow-ips "*"` trusted the leftmost
|
||||
`X-Forwarded-For` entry. The client controls that entry, so rotating the
|
||||
header issued a fresh rate-limit bucket on every request. `client_ip` now
|
||||
reads the hop the trusted edge appends, which is the rightmost one. Login
|
||||
also has an email-keyed throttle that no IP trick can weaken.
|
||||
|
||||
python -m pytest tests/test_ratelimit_hardening.py -v
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from app import auth, limits
|
||||
|
||||
|
||||
class _Req:
|
||||
"""Minimal stand-in for starlette's Request: a header lookup and a peer."""
|
||||
|
||||
def __init__(self, xff: str | None, peer: str | None = "10.0.0.1"):
|
||||
self.headers = {} if xff is None else {"x-forwarded-for": xff}
|
||||
self.client = None if peer is None else type("C", (), {"host": peer})()
|
||||
|
||||
|
||||
# ---------- client_ip: the spoof-resistant hop ----------
|
||||
|
||||
def test_client_ip_takes_appended_rightmost_hop(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
|
||||
# An attacker prepends a fake IP. The edge appends the real one on the right.
|
||||
req = _Req("203.0.113.9, 198.51.100.77")
|
||||
assert limits.client_ip(req) == "198.51.100.77"
|
||||
|
||||
|
||||
def test_client_ip_ignores_spoofed_leftmost(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
|
||||
# The keyed IP stays the real hop regardless of what the client adds on
|
||||
# the left, so rotating that value no longer creates a new bucket.
|
||||
a = limits.client_ip(_Req("1.1.1.1, 198.51.100.77"))
|
||||
b = limits.client_ip(_Req("2.2.2.2, 198.51.100.77"))
|
||||
c = limits.client_ip(_Req("evil, junk, 198.51.100.77"))
|
||||
assert a == b == c == "198.51.100.77"
|
||||
|
||||
|
||||
def test_client_ip_honours_extra_trusted_hops(monkeypatch):
|
||||
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 2)
|
||||
# Two trusted hops: real client is second from the right.
|
||||
req = _Req("9.9.9.9, 203.0.113.5, 198.51.100.77")
|
||||
assert limits.client_ip(req) == "203.0.113.5"
|
||||
|
||||
|
||||
def test_client_ip_falls_back_to_socket_peer():
|
||||
assert limits.client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4"
|
||||
assert limits.client_ip(_Req(None, peer=None)) == "unknown"
|
||||
|
||||
|
||||
# ---------- per-account login throttle ----------
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _multi_user(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", True)
|
||||
# Isolate the module-level failure map for each test.
|
||||
from collections import defaultdict, deque
|
||||
monkeypatch.setattr(limits, "_login_fails", defaultdict(deque))
|
||||
|
||||
|
||||
def test_login_throttle_blocks_after_limit():
|
||||
email = "victim@example.com"
|
||||
# Each attempt up to the limit is allowed and recorded as a failure.
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.check_login_allowed(email) # does not raise
|
||||
limits.note_login_failure(email)
|
||||
# One more failure exceeds the limit.
|
||||
with pytest.raises(limits.HTTPException) as exc:
|
||||
limits.check_login_allowed(email)
|
||||
assert exc.value.status_code == 429
|
||||
|
||||
|
||||
def test_login_throttle_is_per_account():
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.note_login_failure("a@example.com")
|
||||
with pytest.raises(limits.HTTPException):
|
||||
limits.check_login_allowed("a@example.com")
|
||||
# A different account is unaffected because the throttle keys on email, not IP.
|
||||
limits.check_login_allowed("b@example.com") # must not raise
|
||||
|
||||
|
||||
def test_successful_login_clears_the_streak():
|
||||
email = "typo@example.com"
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT):
|
||||
limits.note_login_failure(email)
|
||||
limits.note_login_success(email)
|
||||
limits.check_login_allowed(email) # failure streak cleared, must not raise
|
||||
|
||||
|
||||
def test_throttle_is_noop_in_local_mode(monkeypatch):
|
||||
monkeypatch.setattr(auth, "MULTI_USER", False)
|
||||
for _ in range(limits.LOGIN_FAIL_LIMIT * 3):
|
||||
limits.note_login_failure("solo@example.com")
|
||||
limits.check_login_allowed("solo@example.com") # never throttled locally
|
||||
@@ -1,45 +0,0 @@
|
||||
"""What the provider puts in the `reasoning` request field for each budget
|
||||
setting: a positive budget asks for thinking, 0 stays silent, -1 turns it off.
|
||||
|
||||
python -m pytest tests/test_reasoning_param.py -v
|
||||
"""
|
||||
from app.providers.openai_compatible import OpenAICompatibleProvider
|
||||
|
||||
|
||||
def _body(reasoning_max_tokens, api_mode="chat", max_tokens=1000):
|
||||
provider = OpenAICompatibleProvider(
|
||||
"https://openrouter.ai/api/v1", "k", "deepseek/deepseek-v4-flash-0731",
|
||||
api_mode, reasoning_max_tokens,
|
||||
)
|
||||
body = {"max_tokens": max_tokens}
|
||||
provider._apply_reasoning_budget(body)
|
||||
return body
|
||||
|
||||
|
||||
def test_zero_sends_nothing():
|
||||
"""Ollama and other providers reject unknown fields. Sending 0 must not add a `reasoning` field."""
|
||||
assert "reasoning" not in _body(0)
|
||||
|
||||
|
||||
def test_positive_budget_adds_thinking_tokens():
|
||||
body = _body(500)
|
||||
assert body["reasoning"] == {"max_tokens": 500}
|
||||
# the story output keeps its own full budget on top of the thinking budget
|
||||
assert body["max_tokens"] == 1500
|
||||
|
||||
|
||||
def test_negative_turns_reasoning_off():
|
||||
body = _body(-1)
|
||||
assert body["reasoning"] == {"effort": "none"}
|
||||
# "off" must not inflate the output budget
|
||||
assert body["max_tokens"] == 1000
|
||||
|
||||
|
||||
def test_off_is_not_merely_excluded():
|
||||
"""`exclude: true` still generates and bills for reasoning tokens. The off setting must omit the field entirely instead of relying on `exclude`."""
|
||||
assert _body(-1)["reasoning"].get("exclude") is None
|
||||
|
||||
|
||||
def test_completion_mode_never_sends_reasoning():
|
||||
for budget in (-1, 0, 500):
|
||||
assert "reasoning" not in _body(budget, api_mode="completion")
|
||||
@@ -14,18 +14,11 @@ from app.main import app
|
||||
from app.providers import ProviderError
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Each turn spends 10 gold, so a double-applied or un-rolled-back attempt shows.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -41,14 +34,11 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
@@ -57,9 +47,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -80,7 +67,7 @@ def _adv(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -177,15 +164,15 @@ def test_three_attempts_all_kept_in_order(client):
|
||||
|
||||
def test_switching_back_restores_that_attempt_state(client):
|
||||
ScriptedProvider.replies = [
|
||||
"You take a scratch.\n```state\n{\"player.hp\": -5}\n```",
|
||||
"You take a beating.\n```state\n{\"player.hp\": -40}\n```",
|
||||
'You take a scratch.\n```state\n{"player.hp": -5, "player.gold": 10}\n```',
|
||||
'You take a beating.\n```state\n{"player.hp": -40, "player.gold": 10}\n```',
|
||||
]
|
||||
_play(client)
|
||||
assert _adv(client.adv_id)[1]["player"]["hp"] == 95
|
||||
_retry(client)
|
||||
script_state, world_state = _adv(client.adv_id)
|
||||
player, world_state = _adv(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 60
|
||||
assert script_state == {"gold": 10} # rolled back, not stacked to 20
|
||||
assert player["gold"] == 10 # rolled back, not stacked to 20
|
||||
|
||||
last = _actions(client)[-1]
|
||||
r = client.post(
|
||||
@@ -194,9 +181,9 @@ def test_switching_back_restores_that_attempt_state(client):
|
||||
assert r.json()["text"].startswith("You take a scratch")
|
||||
assert r.json()["take_index"] == 0
|
||||
# The stats follow the narration back.
|
||||
script_state, world_state = _adv(client.adv_id)
|
||||
player, world_state = _adv(client.adv_id)
|
||||
assert world_state["player"]["hp"] == 95
|
||||
assert script_state == {"gold": 10}
|
||||
assert player["gold"] == 10
|
||||
|
||||
# And forward again.
|
||||
client.post(f"/api/adventures/{client.adv_id}/actions/{last['id']}/variant",
|
||||
@@ -251,26 +238,26 @@ def test_switching_to_a_missing_index_is_rejected(client):
|
||||
def test_failed_retry_leaves_the_previous_attempt_in_charge(client):
|
||||
"""A provider error mid-retry must undo the rollback, or the stats on
|
||||
screen would silently disagree with the text still shown."""
|
||||
ScriptedProvider.replies = ["Attempt one.", ProviderError("upstream is down")]
|
||||
ScriptedProvider.replies = [gold_reply("Attempt one."), ProviderError("upstream is down")]
|
||||
_play(client)
|
||||
assert _adv(client.adv_id)[0] == {"gold": 10}
|
||||
assert _adv(client.adv_id)[0]["gold"] == 10
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
|
||||
actions = _actions(client)
|
||||
assert actions[-1]["text"] == "Attempt one." # text never lost
|
||||
assert _adv(client.adv_id)[0] == {"gold": 10} # and the state still matches it
|
||||
assert _adv(client.adv_id)[0]["gold"] == 10 # and the state still matches it
|
||||
|
||||
|
||||
def test_undo_removes_the_action_and_its_history(client):
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
_retry(client)
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
# Undo returns the newest window now, not the whole story.
|
||||
assert [a["type"] for a in r.json()["actions"]] == ["start"]
|
||||
assert _adv(client.adv_id)[0] == {}
|
||||
assert _adv(client.adv_id)[0]["gold"] == 0
|
||||
|
||||
|
||||
def test_editing_the_text_updates_the_live_variant(client):
|
||||
|
||||
@@ -125,7 +125,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -34,7 +34,6 @@ def client(monkeypatch):
|
||||
user_id = user.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
"""Tests for undo and retry rolling back the shared `script_state`
|
||||
"""Tests for undo and retry rolling back the shared `world_state`
|
||||
(plan/11-state-revert-and-retry-fix.md).
|
||||
|
||||
The state being rolled back was the scripting engine's `script_state` until
|
||||
M2 removed campaign scripting. The machinery under test — `attempts.restore_state`,
|
||||
`roll_back_before`, and the per-node outcome snapshot — is unchanged; only the
|
||||
column it moves has. `world_state`/`world_state_after` is now the shared state
|
||||
an adventure carries, so that is what these tests exercise.
|
||||
|
||||
Phase 14 SP4 reversed the snapshots. An action used to carry the state as
|
||||
it stood before it ran, and rolling back read the snapshot off the action
|
||||
being removed. Now it carries the state it left behind, and rolling back
|
||||
@@ -36,11 +42,11 @@ def db():
|
||||
adventures.turns._active_turns.clear()
|
||||
|
||||
|
||||
def _make_adventure(db, script_state):
|
||||
def _make_adventure(db, world_state):
|
||||
user = models.User(is_guest=False)
|
||||
db.add(user)
|
||||
db.flush()
|
||||
adv = models.Adventure(user_id=user.id, title="T", script_state=script_state)
|
||||
adv = models.Adventure(user_id=user.id, title="T", world_state=world_state)
|
||||
db.add(adv)
|
||||
db.flush()
|
||||
return user, adv
|
||||
@@ -49,7 +55,7 @@ def _make_adventure(db, script_state):
|
||||
def _add(db, adv, index, type_, text="x", state_after=None):
|
||||
a = models.Action(
|
||||
adventure_id=adv.id, type=type_, text=text,
|
||||
state_after=state_after,
|
||||
world_state_after=state_after,
|
||||
)
|
||||
db.add(a)
|
||||
db.flush()
|
||||
@@ -72,7 +78,7 @@ def _forget_snapshots(db, adv):
|
||||
# ---------------------------------------------------------------- undo
|
||||
|
||||
def test_undo_reverts_state_to_before_the_turn(db):
|
||||
# A turn moved script_state from {gold:0} to {gold:10}. The node in
|
||||
# A turn moved world_state from {gold:0} to {gold:10}. The node in
|
||||
# front of the turn records where it started. The current state is
|
||||
# the mutated one.
|
||||
user, adv = _make_adventure(db, {"gold": 10})
|
||||
@@ -83,7 +89,7 @@ def test_undo_reverts_state_to_before_the_turn(db):
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 0}
|
||||
assert adv.world_state == {"gold": 0}
|
||||
assert [a.type for a in adv.actions] == ["start"]
|
||||
|
||||
|
||||
@@ -97,7 +103,7 @@ def test_undo_of_bare_continue_uses_the_node_in_front(db):
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 0}
|
||||
assert adv.world_state == {"gold": 0}
|
||||
assert [a.type for a in adv.actions] == ["start"]
|
||||
|
||||
|
||||
@@ -112,7 +118,7 @@ def test_undo_leaves_state_untouched_when_snapshot_missing(db):
|
||||
|
||||
adventures.undo_turn(adv.id, db=db, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 10}
|
||||
assert adv.world_state == {"gold": 10}
|
||||
|
||||
|
||||
def test_undo_raises_when_nothing_to_undo(db):
|
||||
@@ -188,24 +194,24 @@ def test_snapshot_outcome_is_an_independent_deep_copy(db):
|
||||
_, adv = _make_adventure(db, {"nested": {"n": 1}})
|
||||
node = models.Action(adventure_id=adv.id, type="ai", text="x")
|
||||
attempts.snapshot_outcome(adv, node)
|
||||
adv.script_state["nested"]["n"] = 99
|
||||
assert node.state_after == {"nested": {"n": 1}} # unaffected by later mutation
|
||||
adv.world_state["nested"]["n"] = 99
|
||||
assert node.world_state_after == {"nested": {"n": 1}} # unaffected by later mutation
|
||||
|
||||
|
||||
def test_snapshot_outcome_handles_non_dict(db):
|
||||
_, adv = _make_adventure(db, {})
|
||||
adv.script_state = None
|
||||
adv.world_state = None
|
||||
node = models.Action(adventure_id=adv.id, type="ai", text="x")
|
||||
attempts.snapshot_outcome(adv, node)
|
||||
assert node.state_after == {}
|
||||
assert node.world_state_after == {}
|
||||
|
||||
|
||||
def test_restore_state_ignores_a_node_with_no_outcome(db):
|
||||
_, adv = _make_adventure(db, {"gold": 7})
|
||||
attempts.restore_state(adv, models.Action(adventure_id=adv.id, type="ai"))
|
||||
assert adv.script_state == {"gold": 7}
|
||||
assert adv.world_state == {"gold": 7}
|
||||
attempts.restore_state(adv, None)
|
||||
assert adv.script_state == {"gold": 7}
|
||||
assert adv.world_state == {"gold": 7}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- retry
|
||||
@@ -220,8 +226,6 @@ def test_retry_restores_the_state_the_turn_started_from(db, monkeypatch):
|
||||
_add(db, adv, 2, "ai", state_after={"gold": 20})
|
||||
db.commit()
|
||||
|
||||
monkeypatch.setattr(adventures.limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(adventures.turns, "check_demo_cap", lambda *a, **k: None)
|
||||
|
||||
async def _noop(*a, **k):
|
||||
if False:
|
||||
@@ -230,12 +234,12 @@ def test_retry_restores_the_state_the_turn_started_from(db, monkeypatch):
|
||||
|
||||
adventures.retry_action(adv.id, request=None, db=db, user=user, adventure=adv)
|
||||
|
||||
assert adv.script_state == {"gold": 10}
|
||||
assert adv.world_state == {"gold": 10}
|
||||
# Nothing is written until a replacement actually arrives: the attempt on
|
||||
# screen is left exactly as it was, and stays the live one.
|
||||
assert [a.type for a in adv.actions] == ["start", "do", "ai"]
|
||||
last = adv.actions[-1]
|
||||
assert last.live is True
|
||||
assert len(attempts.group(db, last)) == 1 # no sibling was filed
|
||||
assert last.state_after == {"gold": 20} # its own outcome, untouched
|
||||
assert last.world_state_after == {"gold": 20} # its own outcome, untouched
|
||||
adventures.turns._active_turns.discard(adv.id)
|
||||
|
||||
@@ -26,20 +26,13 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
# A world-state schema, so the RPG layer is exercised rather than skipped.
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn. A turn that double-applies its hooks, or one that fails to
|
||||
# roll back on retry, shows up here as a wrong total rather than as nothing.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
OPENING = "You enter a cave."
|
||||
|
||||
@@ -59,7 +52,7 @@ def _make_world(monkeypatch, *, seeded_actions: int = 0):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
@@ -70,20 +63,14 @@ def _make_world(monkeypatch, *, seeded_actions: int = 0):
|
||||
type="ai" if i % 2 else "do",
|
||||
text=f"Seeded turn {i}.",
|
||||
))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = ["A reply."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["A reply."]]
|
||||
ScriptedProvider.calls = 0
|
||||
ScriptedProvider.prompts = []
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -145,7 +132,7 @@ def _state(adv_id):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
adv = db.get(models.Adventure, adv_id)
|
||||
return adv.script_state, adv.world_state
|
||||
return (adv.world_state or {}).get("player", {}), adv.world_state
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -172,7 +159,7 @@ def test_short_adventure_returns_everything(client):
|
||||
# --------------------------------------------------------------- the turn
|
||||
|
||||
def test_a_turn_appends_the_player_action_then_the_ai_action(client):
|
||||
ScriptedProvider.replies = ["The dark presses in."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["The dark presses in."]]
|
||||
_play(client, "light a torch")
|
||||
actions = _actions(client)
|
||||
assert [a["type"] for a in actions] == ["start", "do", "ai"]
|
||||
@@ -181,7 +168,7 @@ def test_a_turn_appends_the_player_action_then_the_ai_action(client):
|
||||
|
||||
|
||||
def test_say_and_story_and_continue_all_work(client):
|
||||
ScriptedProvider.replies = ["One.", "Two.", "Three."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two.", "Three."]]
|
||||
_play(client, "hello", type="say")
|
||||
_play(client, "The wind rises.", type="story")
|
||||
_play(client, "", type="continue")
|
||||
@@ -190,7 +177,7 @@ def test_say_and_story_and_continue_all_work(client):
|
||||
|
||||
|
||||
def test_the_story_so_far_is_replayed_into_the_prompt(client):
|
||||
ScriptedProvider.replies = ["First.", "Second."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["First.", "Second."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
story = ScriptedProvider.prompts[-1][1]
|
||||
@@ -201,7 +188,7 @@ def test_the_story_so_far_is_replayed_into_the_prompt(client):
|
||||
|
||||
def test_scripts_run_once_per_turn(client):
|
||||
"""The gold script adds ten a turn. Two turns is twenty — not forty."""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
_play(client)
|
||||
script_state, _ = _state(client.adv_id)
|
||||
@@ -211,7 +198,7 @@ def test_scripts_run_once_per_turn(client):
|
||||
# ------------------------------------------------------------------ retry
|
||||
|
||||
def test_retry_replaces_the_text_and_keeps_the_attempt(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
assert _texts(client)[-1] == "Attempt one."
|
||||
|
||||
@@ -233,7 +220,7 @@ def test_retry_replaces_the_text_and_keeps_the_attempt(client):
|
||||
def test_retry_does_not_stack_script_effects(client):
|
||||
"""The discarded attempt's ten gold is rolled back, so one turn plus one
|
||||
retry is still ten, not twenty."""
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
script_state, _ = _state(client.adv_id)
|
||||
@@ -241,7 +228,7 @@ def test_retry_does_not_stack_script_effects(client):
|
||||
|
||||
|
||||
def test_switching_back_to_an_earlier_attempt_restores_it(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
@@ -259,7 +246,7 @@ def test_switching_back_to_an_earlier_attempt_restores_it(client):
|
||||
def test_only_the_newest_turn_can_be_switched(client):
|
||||
"""An older turn's alternatives stay readable but not selectable. The
|
||||
story after it continues from what is live."""
|
||||
ScriptedProvider.replies = ["One.", "Again.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Again.", "Two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
older_id = _actions(client)[-1]["id"]
|
||||
@@ -274,7 +261,7 @@ def test_only_the_newest_turn_can_be_switched(client):
|
||||
# ------------------------------------------------------------------- undo
|
||||
|
||||
def test_undo_removes_the_whole_turn_and_rolls_state_back(client):
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
assert len(_actions(client)) == 5
|
||||
@@ -365,7 +352,7 @@ def test_paging_past_the_start_reports_the_end(long_client):
|
||||
# ------------------------------------------------------- editing and deleting
|
||||
|
||||
def test_editing_an_action_sticks(client):
|
||||
ScriptedProvider.replies = ["Original."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Original."]]
|
||||
_play(client)
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
r = client.patch(f"/api/adventures/{client.adv_id}/actions/{action_id}",
|
||||
@@ -380,7 +367,7 @@ def test_editing_an_action_sticks(client):
|
||||
def test_editing_a_retried_action_survives_a_reload(client):
|
||||
"""The edit has to reach the live attempt too, or paging away and back
|
||||
reverts it."""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
@@ -390,7 +377,7 @@ def test_editing_a_retried_action_survives_a_reload(client):
|
||||
|
||||
|
||||
def test_deleting_an_action_removes_it(client):
|
||||
ScriptedProvider.replies = ["One."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One."]]
|
||||
_play(client)
|
||||
action_id = _actions(client)[-1]["id"]
|
||||
r = client.delete(f"/api/adventures/{client.adv_id}/actions/{action_id}")
|
||||
@@ -431,7 +418,7 @@ def test_export_carries_the_whole_story(client):
|
||||
same fact: a bundle that stores coordinates has no use for a repeating
|
||||
group. Everything else here still passes unmodified.
|
||||
"""
|
||||
ScriptedProvider.replies = ["One.", "Two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One.", "Two."]]
|
||||
_play(client, "go north")
|
||||
_play(client, "go south")
|
||||
|
||||
@@ -446,7 +433,7 @@ def test_export_carries_the_whole_story(client):
|
||||
|
||||
|
||||
def test_export_round_trips_through_import(client):
|
||||
ScriptedProvider.replies = ["One."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["One."]]
|
||||
_play(client, "go north")
|
||||
bundle = client.get(f"/api/adventures/{client.adv_id}/export").json()
|
||||
|
||||
@@ -461,7 +448,7 @@ def test_export_round_trips_through_import(client):
|
||||
|
||||
|
||||
def test_export_keeps_retry_attempts(client):
|
||||
ScriptedProvider.replies = ["Attempt one.", "Attempt two."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Attempt one.", "Attempt two."]]
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
|
||||
@@ -474,7 +461,7 @@ def test_export_keeps_retry_attempts(client):
|
||||
# -------------------------------------------------------------- world state
|
||||
|
||||
def test_world_state_is_readable_and_survives_a_turn(client):
|
||||
ScriptedProvider.replies = ["Nothing changes."]
|
||||
ScriptedProvider.replies = [gold_reply(t) for t in ["Nothing changes."]]
|
||||
_play(client)
|
||||
r = client.get(f"/api/adventures/{client.adv_id}/world-state")
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
@@ -57,9 +57,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -58,12 +58,6 @@ def client(monkeypatch):
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(
|
||||
auth,
|
||||
"resolve_provider_config",
|
||||
lambda s: auth.ProviderConfig("http://fake", "k", "test-model", False),
|
||||
)
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -24,20 +24,14 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
SCHEMA = {"player": {"hp": {"min": 0, "max": 100, "initial": 100}}}
|
||||
SCHEMA = GOLD_SCHEMA
|
||||
|
||||
# Ten gold a turn, every turn. A number that only ever increases makes a
|
||||
# rollback failure obvious: if a take stacks instead of replacing, the gold
|
||||
# total is off by exactly one turn's worth.
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
# Ten gold a turn, every turn, banked through the world-state engine. A number
|
||||
# that only ever increases makes a rollback failure obvious: if a take stacks
|
||||
# instead of replacing, the gold total is off by exactly one turn's worth. See
|
||||
# `fakes.gold_reply` for why this is not a script any more.
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -53,24 +47,18 @@ def client(monkeypatch):
|
||||
setup.flush()
|
||||
adv = models.Adventure(
|
||||
user_id=user.id, title="Vault", scenario_id=scenario.id,
|
||||
script_state={}, world_state={"player": {"hp": 100}},
|
||||
world_state={"player": {"hp": 100, "gold": 0}},
|
||||
)
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You begin."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold", output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
ScriptedProvider.replies = [f"Take {n}." for n in range(1, 40)]
|
||||
ScriptedProvider.replies = gold_replies("Take", 39)
|
||||
ScriptedProvider.calls = 0
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -107,7 +95,7 @@ def _take(client, action_id, text=""):
|
||||
def _gold(adv_id) -> int:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return (db.get(models.Adventure, adv_id).script_state or {}).get("gold", 0)
|
||||
return (db.get(models.Adventure, adv_id).world_state or {}).get("player", {}).get("gold", 0)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@@ -397,7 +397,6 @@ def client(monkeypatch):
|
||||
user_id, scenario_id = user.id, scenario.id
|
||||
setup.close()
|
||||
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
"""End-to-end HTTP tests for undo and retry state revert. These tests drive
|
||||
real turns through the actual routes and scripting engine, with only the LLM
|
||||
provider mocked.
|
||||
real turns through the actual routes and the world-state engine, with only the
|
||||
LLM provider mocked.
|
||||
|
||||
A script's output hook adds 10 gold each turn. These tests confirm that the
|
||||
adventure's stored gold total stays correct across play, undo, and retry.
|
||||
The model banks 10 gold each turn through its ```state delta block. These tests
|
||||
confirm that the adventure's stored gold total stays correct across play, undo,
|
||||
and retry. It was a JavaScript output hook that added the gold until M2 removed
|
||||
campaign scripting; the state machinery under test is the same either way.
|
||||
|
||||
python -m pytest tests/test_turn_flow_integration.py -v
|
||||
"""
|
||||
@@ -16,15 +18,8 @@ from app.database import Base, SessionLocal, engine, get_db
|
||||
from app.main import app
|
||||
from app.routers import adventures
|
||||
|
||||
from fakes import ScriptedProvider
|
||||
from fakes import GOLD_SCHEMA, ScriptedProvider, gold_replies, gold_reply
|
||||
|
||||
GOLD_SCRIPT = """
|
||||
const modifier = (text) => {
|
||||
state.gold = (state.gold || 0) + 10;
|
||||
return { text };
|
||||
};
|
||||
modifier(text);
|
||||
"""
|
||||
|
||||
|
||||
AI_REPLY = "The torch flickers as you press onward."
|
||||
@@ -38,24 +33,21 @@ def client(monkeypatch):
|
||||
setup.add(user)
|
||||
setup.flush()
|
||||
setup.add(models.Settings(user_id=user.id, api_key="enc:dummy", model="test-model"))
|
||||
adv = models.Adventure(user_id=user.id, title="Cave", script_state={})
|
||||
scenario = models.Scenario(user_id=user.id, title="S", stat_schema=GOLD_SCHEMA)
|
||||
setup.add(scenario)
|
||||
setup.flush()
|
||||
adv = models.Adventure(user_id=user.id, title="Cave", scenario_id=scenario.id,
|
||||
world_state={"player": {"hp": 100, "gold": 0}})
|
||||
setup.add(adv)
|
||||
setup.flush()
|
||||
setup.add(models.Action(adventure_id=adv.id, type="start", text="You enter a cave."))
|
||||
setup.add(models.AdventureScript(
|
||||
adventure_id=adv.id, position=0, enabled=True, name="Gold",
|
||||
output_js=GOLD_SCRIPT,
|
||||
))
|
||||
setup.commit()
|
||||
adv_id, user_id = adv.id, user.id
|
||||
setup.close()
|
||||
|
||||
# Force a real, non-demo turn that uses the fake provider.
|
||||
ScriptedProvider.replies = [AI_REPLY]
|
||||
ScriptedProvider.replies = [gold_reply(AI_REPLY)]
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
@@ -73,10 +65,13 @@ def client(monkeypatch):
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
|
||||
def _state(adv_id):
|
||||
def _gold(adv_id) -> int:
|
||||
"""The banked total. Reading one stat rather than the whole section keeps
|
||||
these assertions about the rollback, not about the schema's shape."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return db.get(models.Adventure, adv_id).script_state
|
||||
player = (db.get(models.Adventure, adv_id).world_state or {}).get("player", {})
|
||||
return player.get("gold", 0)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -88,38 +83,38 @@ def _play(client, type_="do", text="look around"):
|
||||
|
||||
|
||||
def test_play_then_undo_reverts_gold(client):
|
||||
assert _state(client.adv_id) == {}
|
||||
assert _gold(client.adv_id) == 0
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert r.status_code == 200, r.text
|
||||
assert _state(client.adv_id) == {} # gold reverted to zero
|
||||
assert _gold(client.adv_id) == 0 # gold reverted to zero
|
||||
|
||||
|
||||
def test_two_turns_then_undo_reverts_only_last(client):
|
||||
_play(client)
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 20}
|
||||
assert _gold(client.adv_id) == 20
|
||||
|
||||
client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert _state(client.adv_id) == {"gold": 10} # back to after turn 1, not 0
|
||||
assert _gold(client.adv_id) == 10 # back to after turn 1, not 0
|
||||
|
||||
|
||||
def test_retry_does_not_double_apply_gold(client):
|
||||
_play(client)
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
# Before the fix, this produced 20 because the output hook ran twice.
|
||||
# Before the fix, this produced 20 because the turn's effects ran twice.
|
||||
# Now it stays 10.
|
||||
r = client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert r.status_code == 200, r.text
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
|
||||
|
||||
def test_retry_then_undo_still_clean(client):
|
||||
_play(client)
|
||||
client.post(f"/api/adventures/{client.adv_id}/retry")
|
||||
assert _state(client.adv_id) == {"gold": 10}
|
||||
assert _gold(client.adv_id) == 10
|
||||
client.post(f"/api/adventures/{client.adv_id}/undo")
|
||||
assert _state(client.adv_id) == {}
|
||||
assert _gold(client.adv_id) == 0
|
||||
|
||||
@@ -63,9 +63,6 @@ def client(monkeypatch):
|
||||
|
||||
ScriptedProvider.replies = [AI_REPLY]
|
||||
monkeypatch.setattr(adventures.turns, "OpenAICompatibleProvider", ScriptedProvider)
|
||||
monkeypatch.setattr(auth, "resolve_provider_config", lambda s: auth.ProviderConfig(
|
||||
"http://fake", "k", "test-model", False))
|
||||
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
|
||||
monkeypatch.setattr(limits, "check_row_cap", lambda *a, **k: None)
|
||||
|
||||
def _current_user(db=Depends(get_db)):
|
||||
|
||||
@@ -56,66 +56,22 @@ Usage, from `backend/`:
|
||||
python -m tools.rewrite_memories --write --limit 3 # try three of them
|
||||
python -m tools.rewrite_memories --write --embed # the whole backfill
|
||||
|
||||
Without `--write` it makes no model calls, spends nothing, and only reports the
|
||||
scope. Every run reads the database the app reads: `AIDND_DB_PATH`, or
|
||||
`DATABASE_URL` for a hosted Postgres. Take a copy of it first — the old text is
|
||||
overwritten and is not kept anywhere.
|
||||
Without `--write` it makes no model calls and only reports the scope. Every run
|
||||
reads the database the app reads, at `AIDND_DB_PATH`. Take a copy of it first —
|
||||
the old text is overwritten and is not kept anywhere.
|
||||
|
||||
**On the hosted deploy, name whose adventures you mean.** That database holds
|
||||
other people's stories, and each adventure is summarized with *its owner's* key,
|
||||
so an unfiltered `--write` spends other people's money on memories they did not
|
||||
ask to have rewritten. `--email` restricts the run to the accounts you name and
|
||||
`--adventure` to single adventures; a dry run costs nothing and lists both, with
|
||||
the owner of each. Guests have no email and can only be reached by id.
|
||||
|
||||
Two environment variables reach that database from a checkout. In PowerShell,
|
||||
which is where this project is developed, they are set first and then persist
|
||||
for the rest of the session:
|
||||
|
||||
$env:AIDND_DATABASE_URL = Read-Host 'Neon URL'
|
||||
$env:AIDND_SECRET_KEY = Read-Host 'Secret key'
|
||||
.venv\Scripts\python.exe -m tools.rewrite_memories --email you@example.com
|
||||
|
||||
`Read-Host` keeps both values out of the PowerShell history file. On a POSIX
|
||||
shell the same thing is one line:
|
||||
|
||||
AIDND_DATABASE_URL=... AIDND_SECRET_KEY=... \
|
||||
python -m tools.rewrite_memories --email you@example.com
|
||||
|
||||
`AIDND_SECRET_KEY` is not optional there. Stored API keys are encrypted with it,
|
||||
and with the wrong one `decrypt_secret` returns "" and every adventure is skipped
|
||||
as having no key (see `security.py`). The deployed image does not carry this
|
||||
directory — the Dockerfile copies `backend/app` alone — so run it from a
|
||||
checkout against the hosted database rather than from a shell on the box.
|
||||
`--adventure` restricts the run to single adventures, and a dry run lists what
|
||||
would change. The deployed image does not carry this directory — the Dockerfile
|
||||
copies `backend/app` alone — so run it from a checkout.
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from sqlalchemy import func, inspect as sa_inspect, select
|
||||
|
||||
|
||||
def safe_dsn(url: str) -> str:
|
||||
"""A connection string with the credentials taken out.
|
||||
|
||||
The report says which database it is about to rewrite, which is worth
|
||||
printing. The password in a Neon URL is not: this output goes to a console,
|
||||
a screenshot, or a pasted bug report, and the operator has no way to know
|
||||
the line carried a credential until it is somewhere else.
|
||||
"""
|
||||
parsed = urlsplit(url)
|
||||
if not parsed.hostname:
|
||||
return "(configured)"
|
||||
who = f"{parsed.username}@" if parsed.username else ""
|
||||
port = f":{parsed.port}" if parsed.port else ""
|
||||
# The query string is dropped whole. `sslmode` is the only part anyone
|
||||
# wants to see, and some drivers accept a password there too.
|
||||
return f"{parsed.scheme}://{who}{parsed.hostname}{port}{parsed.path}"
|
||||
|
||||
|
||||
def words(text: str) -> int:
|
||||
return len(text.split())
|
||||
|
||||
@@ -127,16 +83,16 @@ def one_line(text: str, width: int = 96) -> str:
|
||||
|
||||
async def main(args) -> int:
|
||||
from app import memorybank, models
|
||||
from app.database import DB_PATH, DATABASE_URL, SessionLocal
|
||||
from app.database import DB_PATH, SessionLocal
|
||||
from app.providers import OpenAICompatibleProvider, ProviderError
|
||||
|
||||
db = SessionLocal()
|
||||
print(f"database: {safe_dsn(DATABASE_URL) if DATABASE_URL else DB_PATH}")
|
||||
print(f"database: {DB_PATH}")
|
||||
if not sa_inspect(db.get_bind()).has_table(models.Adventure.__tablename__):
|
||||
# A mistyped path creates an empty SQLite file rather than failing, so
|
||||
# say what is wrong instead of raising "no such table: adventures".
|
||||
print("There are no tables here. Point AIDND_DB_PATH, or DATABASE_URL "
|
||||
"for a hosted deploy, at the database the app uses.")
|
||||
print("There are no tables here. Point AIDND_DB_PATH at the database "
|
||||
"the app uses.")
|
||||
return 2
|
||||
|
||||
adventures = db.query(models.Adventure).order_by(models.Adventure.id)
|
||||
@@ -177,14 +133,13 @@ async def main(args) -> int:
|
||||
|
||||
def provider_for(settings: models.Settings) -> OpenAICompatibleProvider:
|
||||
"""The adventure owner's own summarizer, unless the run overrides it."""
|
||||
if not (args.endpoint or args.model or args.api_key):
|
||||
if not (args.endpoint or args.model):
|
||||
return memorybank.summary_provider(settings)
|
||||
return OpenAICompatibleProvider(
|
||||
args.endpoint or settings.endpoint_url,
|
||||
args.api_key or settings.api_key_plain,
|
||||
args.model or settings.summary_model or settings.model,
|
||||
settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.model_timeout_seconds,
|
||||
)
|
||||
|
||||
totals = {"rewritten": 0, "would rewrite": 0, "no source": 0,
|
||||
@@ -205,11 +160,11 @@ async def main(args) -> int:
|
||||
continue
|
||||
|
||||
settings = settings_for(adventure.user_id)
|
||||
# An adventure whose owner has no key is reported rather than skipped
|
||||
# silently: it is the one reason a memory this tool can rewrite is left
|
||||
# alone, and the operator can fix it with --api-key.
|
||||
# An adventure with no model configured is reported rather than
|
||||
# skipped silently: it is the one reason a memory this tool can rewrite
|
||||
# is left alone, and the operator can fix it with --model.
|
||||
usable = settings is not None and bool(
|
||||
args.api_key or args.endpoint or settings.api_key_plain
|
||||
args.model or settings.summary_model or settings.model
|
||||
)
|
||||
owner = db.get(models.User, adventure.user_id)
|
||||
who = (owner.email if owner and owner.email
|
||||
@@ -333,7 +288,6 @@ if __name__ == "__main__":
|
||||
help="re-embed here. Stop the app first; see the docstring.")
|
||||
parser.add_argument("--endpoint", help="override the owner's endpoint URL.")
|
||||
parser.add_argument("--model", help="override the owner's summary model.")
|
||||
parser.add_argument("--api-key", help="override the owner's API key.")
|
||||
args = parser.parse_args()
|
||||
|
||||
# A Windows console defaults to cp1252, which cannot encode the arrow this
|
||||
|
||||
Generated
-247
@@ -8,9 +8,6 @@
|
||||
"name": "frontend",
|
||||
"version": "0.0.0",
|
||||
"dependencies": {
|
||||
"@codemirror/lang-javascript": "^6.2.5",
|
||||
"@codemirror/theme-one-dark": "^6.1.3",
|
||||
"@uiw/react-codemirror": "^4.25.10",
|
||||
"react": "^19.2.7",
|
||||
"react-dom": "^19.2.7",
|
||||
"react-router-dom": "^7.18.1"
|
||||
@@ -23,123 +20,6 @@
|
||||
"vite": "^8.1.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/runtime": {
|
||||
"version": "7.29.7",
|
||||
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.7.tgz",
|
||||
"integrity": "sha512-Nq8OhGWiZIZGV6hLHoyAKLLcJihP/xFeBMGJoUrxTX2psI8dCifzLhZISFb+VWS3wFMRDmCGw5R+dOySCqPLhw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6.9.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/autocomplete": {
|
||||
"version": "6.20.3",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.20.3.tgz",
|
||||
"integrity": "sha512-tlosUqb+3BbxCxZdu4tKeRghPFC+QM7q4X5YhKV2eCmPG+1r2F3f4AaSz5sCrFqUtX4Jh20VFTKecl16MgiV9g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/language": "^6.0.0",
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.17.0",
|
||||
"@lezer/common": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/commands": {
|
||||
"version": "6.10.4",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/commands/-/commands-6.10.4.tgz",
|
||||
"integrity": "sha512-Ryk9y9T0FFVF0cUGhAknveAyUOl/A1qReTFi+qPKtOh2Z9F4AUBz3XOrYD4ZEgZirdugVzHvd/2/Wcwy5OliTg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/language": "^6.0.0",
|
||||
"@codemirror/state": "^6.7.0",
|
||||
"@codemirror/view": "^6.27.0",
|
||||
"@lezer/common": "^1.1.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/lang-javascript": {
|
||||
"version": "6.2.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/lang-javascript/-/lang-javascript-6.2.5.tgz",
|
||||
"integrity": "sha512-zD4e5mS+50htS7F+TYjBPsiIFGanfVqg4HyUz6WNFikgOPf2BgKlx+TQedI1w6n/IqRBVBbBWmGFdLB/7uxO4A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/autocomplete": "^6.0.0",
|
||||
"@codemirror/language": "^6.6.0",
|
||||
"@codemirror/lint": "^6.0.0",
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.17.0",
|
||||
"@lezer/common": "^1.0.0",
|
||||
"@lezer/javascript": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/language": {
|
||||
"version": "6.12.4",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/language/-/language-6.12.4.tgz",
|
||||
"integrity": "sha512-1q4PaT+o6PbgpkJt4Q8Fv5XJxTy4FUZ4MWETtyiDw3J0Pyr9E2vqcKL+k9wcvjNTIsauxvE7OfmWj3FRPHQ76A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.23.0",
|
||||
"@lezer/common": "^1.5.0",
|
||||
"@lezer/highlight": "^1.0.0",
|
||||
"@lezer/lr": "^1.0.0",
|
||||
"style-mod": "^4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/lint": {
|
||||
"version": "6.9.7",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/lint/-/lint-6.9.7.tgz",
|
||||
"integrity": "sha512-28/+iWLYxKxsvGYhSYL7zaCZqLz5+FFFDq9tVsvGv9kv8RY4fFAchJ5WX9M3YrrRlTIsECjsXPqeNgnSmNP2dg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.42.0",
|
||||
"crelt": "^1.0.5"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/search": {
|
||||
"version": "6.7.1",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/search/-/search-6.7.1.tgz",
|
||||
"integrity": "sha512-uMe5UO6PamJtSHrXhhHOzSX3ReWtiJrva6GnPMwSOrZtiExb5X5eExhr2OUZQVvdxPsKpY3Ro2mFbQadpPWmHA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.37.0",
|
||||
"crelt": "^1.0.5"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/state": {
|
||||
"version": "6.7.0",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-6.7.0.tgz",
|
||||
"integrity": "sha512-Zbl9NyscLMZkfXPQnNAIIAFftidrA1UbcJEIMp24C0Bukc2I5T8wJS0wsXYsnDOqCFJUeJ1BITGNs5CqPDSmSg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@marijn/find-cluster-break": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/theme-one-dark": {
|
||||
"version": "6.1.3",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/theme-one-dark/-/theme-one-dark-6.1.3.tgz",
|
||||
"integrity": "sha512-NzBdIvEJmx6fjeremiGp3t/okrLPYT0d9orIc7AFun8oZcRk58aejkqhv6spnz4MLAevrKNPMQYXEWMg4s+sKA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/language": "^6.0.0",
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.0.0",
|
||||
"@lezer/highlight": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/view": {
|
||||
"version": "6.43.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.43.5.tgz",
|
||||
"integrity": "sha512-7uT/vUgH6dfXWn3WqOe23KneILMvGy5wQjNMEcRXLKzziJ9NOktpW6tGoyQpwVkBgE5Gj6hKkCcsddbnkaWrOQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^6.7.0",
|
||||
"crelt": "^1.0.6",
|
||||
"style-mod": "^4.1.0",
|
||||
"w3c-keyname": "^2.2.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@emnapi/core": {
|
||||
"version": "1.11.2",
|
||||
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.11.2.tgz",
|
||||
@@ -176,47 +56,6 @@
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@lezer/common": {
|
||||
"version": "1.5.2",
|
||||
"resolved": "https://registry.npmjs.org/@lezer/common/-/common-1.5.2.tgz",
|
||||
"integrity": "sha512-sxQE460fPZyU3sdc8lafxiPwJHBzZRy/udNFynGQky1SePYBdhkBl1kOagA9uT3pxR8K09bOrmTUqA9wb/PjSQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@lezer/highlight": {
|
||||
"version": "1.2.3",
|
||||
"resolved": "https://registry.npmjs.org/@lezer/highlight/-/highlight-1.2.3.tgz",
|
||||
"integrity": "sha512-qXdH7UqTvGfdVBINrgKhDsVTJTxactNNxLk7+UMwZhU13lMHaOBlJe9Vqp907ya56Y3+ed2tlqzys7jDkTmW0g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@lezer/common": "^1.3.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@lezer/javascript": {
|
||||
"version": "1.5.4",
|
||||
"resolved": "https://registry.npmjs.org/@lezer/javascript/-/javascript-1.5.4.tgz",
|
||||
"integrity": "sha512-vvYx3MhWqeZtGPwDStM2dwgljd5smolYD2lR2UyFcHfxbBQebqx8yjmFmxtJ/E6nN6u1D9srOiVWm3Rb4tmcUA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@lezer/common": "^1.2.0",
|
||||
"@lezer/highlight": "^1.1.3",
|
||||
"@lezer/lr": "^1.3.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@lezer/lr": {
|
||||
"version": "1.4.10",
|
||||
"resolved": "https://registry.npmjs.org/@lezer/lr/-/lr-1.4.10.tgz",
|
||||
"integrity": "sha512-rnCpTIBafOx4mRp43xOxDJbFipJm/c0cia/V5TiGlhmMa+wsSdoGmUN3w5Bqrks/09Q/D4tNAmWaT8p6NRi77A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@lezer/common": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@marijn/find-cluster-break": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@marijn/find-cluster-break/-/find-cluster-break-1.0.3.tgz",
|
||||
"integrity": "sha512-FY+MKLBoTsLNJF/eLWaOsXGdz6uh3Iu1axjPf6TUq92IYumcTcXWHoS747JARLkcdlJ/Waiaxc5wQfFO8jC6NA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@napi-rs/wasm-runtime": {
|
||||
"version": "1.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.6.tgz",
|
||||
@@ -887,59 +726,6 @@
|
||||
"@types/react": "^19.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@uiw/codemirror-extensions-basic-setup": {
|
||||
"version": "4.25.10",
|
||||
"resolved": "https://registry.npmjs.org/@uiw/codemirror-extensions-basic-setup/-/codemirror-extensions-basic-setup-4.25.10.tgz",
|
||||
"integrity": "sha512-P3vytLlpE62KYSWrMUnwDCv2lvaQDuDZzyj03mHntuHo5bSl34fRZpjTY3kQTPGuXHxkGSYpoPFFj+hMTqaaMQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/autocomplete": "^6.0.0",
|
||||
"@codemirror/commands": "^6.0.0",
|
||||
"@codemirror/language": "^6.0.0",
|
||||
"@codemirror/lint": "^6.0.0",
|
||||
"@codemirror/search": "^6.0.0",
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://jaywcjlove.github.io/#/sponsor"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@codemirror/autocomplete": ">=6.0.0",
|
||||
"@codemirror/commands": ">=6.0.0",
|
||||
"@codemirror/language": ">=6.0.0",
|
||||
"@codemirror/lint": ">=6.0.0",
|
||||
"@codemirror/search": ">=6.0.0",
|
||||
"@codemirror/state": ">=6.0.0",
|
||||
"@codemirror/view": ">=6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@uiw/react-codemirror": {
|
||||
"version": "4.25.10",
|
||||
"resolved": "https://registry.npmjs.org/@uiw/react-codemirror/-/react-codemirror-4.25.10.tgz",
|
||||
"integrity": "sha512-DzgSMwM5qzB7v1FIb4gEeriYt67iiay756/HIOM9mAbeOVK0MO7rqefHf0O5c0269pJKMW7AH9FjclExD23V9w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/runtime": "^7.18.6",
|
||||
"@codemirror/commands": "^6.1.0",
|
||||
"@codemirror/state": "^6.1.1",
|
||||
"@codemirror/theme-one-dark": "^6.0.0",
|
||||
"@uiw/codemirror-extensions-basic-setup": "4.25.10",
|
||||
"codemirror": "^6.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://jaywcjlove.github.io/#/sponsor"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@babel/runtime": ">=7.11.0",
|
||||
"@codemirror/state": ">=6.0.0",
|
||||
"@codemirror/theme-one-dark": ">=6.0.0",
|
||||
"@codemirror/view": ">=6.0.0",
|
||||
"codemirror": ">=6.0.0",
|
||||
"react": ">=17.0.0",
|
||||
"react-dom": ">=17.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@vitejs/plugin-react": {
|
||||
"version": "6.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@vitejs/plugin-react/-/plugin-react-6.0.3.tgz",
|
||||
@@ -966,21 +752,6 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/codemirror": {
|
||||
"version": "6.0.2",
|
||||
"resolved": "https://registry.npmjs.org/codemirror/-/codemirror-6.0.2.tgz",
|
||||
"integrity": "sha512-VhydHotNW5w1UGK0Qj96BwSk/Zqbp9WbnyK2W/eVMv4QyF41INRGpjUhFJY7/uDNuudSc33a/PKr4iDqRduvHw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@codemirror/autocomplete": "^6.0.0",
|
||||
"@codemirror/commands": "^6.0.0",
|
||||
"@codemirror/language": "^6.0.0",
|
||||
"@codemirror/lint": "^6.0.0",
|
||||
"@codemirror/search": "^6.0.0",
|
||||
"@codemirror/state": "^6.0.0",
|
||||
"@codemirror/view": "^6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/cookie": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/cookie/-/cookie-1.1.1.tgz",
|
||||
@@ -994,12 +765,6 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/crelt": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/crelt/-/crelt-1.0.7.tgz",
|
||||
"integrity": "sha512-aK6BbWfhf4U/wCcLHKPJl/xa6VkVstRaPywWtMKGwuOLc/wZTyQYuoxgvZnNsBvv7Kg3YTBQYYBCggcviQczuA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/csstype": {
|
||||
"version": "3.2.3",
|
||||
"resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz",
|
||||
@@ -1543,12 +1308,6 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/style-mod": {
|
||||
"version": "4.1.3",
|
||||
"resolved": "https://registry.npmjs.org/style-mod/-/style-mod-4.1.3.tgz",
|
||||
"integrity": "sha512-i/n8VsZydrugj3Iuzll8+x/00GH2vnYsk1eomD8QiRrSAeW6ItbCQDtfXCeJHd0iwiNagqjQkvpvREEPtW3IoQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tinyglobby": {
|
||||
"version": "0.2.17",
|
||||
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz",
|
||||
@@ -1651,12 +1410,6 @@
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/w3c-keyname": {
|
||||
"version": "2.2.8",
|
||||
"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz",
|
||||
"integrity": "sha512-dpojBhNsCNN7T82Tm7k26A6G9ML3NkhDsnw9n/eoxSRlVBB4CEtIQ/KTCLI2Fwf3ataSXRhYFkQi3SlnFwPvPQ==",
|
||||
"license": "MIT"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,9 +10,6 @@
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"@codemirror/lang-javascript": "^6.2.5",
|
||||
"@codemirror/theme-one-dark": "^6.1.3",
|
||||
"@uiw/react-codemirror": "^4.25.10",
|
||||
"react": "^19.2.7",
|
||||
"react-dom": "^19.2.7",
|
||||
"react-router-dom": "^7.18.1"
|
||||
|
||||
+15
-85
@@ -1,56 +1,23 @@
|
||||
import { useEffect, useRef, useState } from 'react'
|
||||
import { NavLink, Outlet, useLocation } from 'react-router-dom'
|
||||
import { api } from './api'
|
||||
import { AuthModal, ToastHost } from './components'
|
||||
import { useState } from 'react'
|
||||
import { NavLink, Outlet } from 'react-router-dom'
|
||||
import { ToastHost } from './components'
|
||||
import Embers from './Embers.jsx'
|
||||
|
||||
// The shell: the nav bar and whatever page is routed under it.
|
||||
//
|
||||
// Upstream also carried the hosted deployment's account furniture here — a
|
||||
// /auth/me lookup on mount, guest and sign-up prompts, a log-out button, a
|
||||
// pageview beacon, and two nav links gated on server-side allowlists. M2
|
||||
// removed the deployment those served. There is one local user, nothing to log
|
||||
// in to, and nothing counting.
|
||||
export default function App() {
|
||||
// null until /auth/me resolves; in local mode multi_user=false hides all auth UI.
|
||||
const [me, setMe] = useState(null)
|
||||
const [authMode, setAuthMode] = useState(null) // 'register' | 'login' | null
|
||||
const [navOpen, setNavOpen] = useState(false) // mobile hamburger menu
|
||||
const location = useLocation()
|
||||
const lastPath = useRef(null)
|
||||
|
||||
useEffect(() => {
|
||||
api.getMe().then(setMe).catch(() => {})
|
||||
}, [])
|
||||
|
||||
// One pageview per route the reader actually lands on. Guarded on the path
|
||||
// rather than fired on every render: StrictMode runs effects twice in dev,
|
||||
// and a re-render for unrelated state is not a new page.
|
||||
useEffect(() => {
|
||||
if (lastPath.current === location.pathname) return
|
||||
const first = lastPath.current === null
|
||||
lastPath.current = location.pathname
|
||||
// document.referrer survives client-side navigation, so it is only honest
|
||||
// on the first view — after that this was our own page, not a referral.
|
||||
api.trackPageview(location.pathname, {
|
||||
referrer: first ? document.referrer : '',
|
||||
first,
|
||||
})
|
||||
}, [location.pathname])
|
||||
|
||||
const onAuthed = (newMe, mode) => {
|
||||
setAuthMode(null)
|
||||
if (mode === 'login') {
|
||||
// Different user now — reload so every page refetches its scoped data.
|
||||
window.location.reload()
|
||||
} else {
|
||||
setMe(newMe) // register upgrades the same user in place; data unchanged
|
||||
}
|
||||
}
|
||||
|
||||
const logout = async () => {
|
||||
try { await api.logout() } catch { /* already logged out */ }
|
||||
window.location.reload()
|
||||
}
|
||||
|
||||
return (
|
||||
<ToastHost>
|
||||
<Embers />
|
||||
<nav className="topnav">
|
||||
<span className="brand">⚔ AI D&D</span>
|
||||
<span className="brand">⚔ Adventure Storyteller</span>
|
||||
<button
|
||||
className="nav-hamburger"
|
||||
aria-label="Menu"
|
||||
@@ -69,52 +36,15 @@ export default function App() {
|
||||
<NavLink to="/scenarios" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
Scenarios
|
||||
</NavLink>
|
||||
<NavLink to="/scripts" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
Scripts
|
||||
</NavLink>
|
||||
<NavLink to="/settings" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
Settings
|
||||
</NavLink>
|
||||
{/* Power-user tooling, not part of the game — hidden for everyone else. */}
|
||||
{me?.power_user && (
|
||||
<NavLink to="/chat" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
AI Chat
|
||||
</NavLink>
|
||||
)}
|
||||
{/* Owner only: the site's own traffic, on its own allowlist. */}
|
||||
{me?.analytics && (
|
||||
<NavLink to="/analytics" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
Visitors
|
||||
</NavLink>
|
||||
)}
|
||||
{me?.multi_user && (
|
||||
<div className="nav-account">
|
||||
{me.is_guest ? (
|
||||
<>
|
||||
<span className="guest-nudge"
|
||||
title={me.guest_retention_days
|
||||
? `Guest adventures are deleted after ${me.guest_retention_days} days without a visit.`
|
||||
: undefined}>
|
||||
Playing as guest — sign up to keep your adventures
|
||||
</span>
|
||||
<button onClick={() => setAuthMode('login')}>Log in</button>
|
||||
<button className="primary" onClick={() => setAuthMode('register')}>Sign up</button>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<span className="account-email" title={me.email}>{me.email}</span>
|
||||
<button onClick={logout}>Log out</button>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
<NavLink to="/chat" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
|
||||
AI Chat
|
||||
</NavLink>
|
||||
</div>
|
||||
</nav>
|
||||
<Outlet context={{ me, setMe }} />
|
||||
{authMode && (
|
||||
<AuthModal mode={authMode} onClose={() => setAuthMode(null)} onAuthed={onAuthed}
|
||||
retentionDays={me?.guest_retention_days} />
|
||||
)}
|
||||
<Outlet />
|
||||
</ToastHost>
|
||||
)
|
||||
}
|
||||
|
||||
+5
-63
@@ -1,19 +1,11 @@
|
||||
// Multi-user mode: a 401 means our session cookie is missing/stale. Hitting
|
||||
// /api/auth/me creates a fresh guest session, after which the original call
|
||||
// is retried once.
|
||||
async function ensureSession() {
|
||||
await fetch('/api/auth/me')
|
||||
}
|
||||
|
||||
async function request(path, options = {}, isRetry = false) {
|
||||
// There is no session. The API is single-user and loopback-bound, so a request
|
||||
// either works or reports why. The 401-and-retry dance here served the hosted
|
||||
// deployment's guest cookies, which M2 removed.
|
||||
async function request(path, options = {}) {
|
||||
const resp = await fetch(`/api${path}`, {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
...options,
|
||||
})
|
||||
if (resp.status === 401 && !isRetry && path !== '/auth/me') {
|
||||
await ensureSession()
|
||||
return request(path, options, true)
|
||||
}
|
||||
if (!resp.ok) {
|
||||
let detail = resp.statusText
|
||||
try {
|
||||
@@ -27,17 +19,13 @@ async function request(path, options = {}, isRetry = false) {
|
||||
}
|
||||
|
||||
// POSTs to an SSE endpoint and dispatches events: {type: 'player'|'chunk'|'done'|'error', ...}
|
||||
async function streamSSE(path, payload, onEvent, signal, isRetry = false) {
|
||||
async function streamSSE(path, payload, onEvent, signal) {
|
||||
const resp = await fetch(`/api${path}`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload),
|
||||
signal,
|
||||
})
|
||||
if (resp.status === 401 && !isRetry) {
|
||||
await ensureSession()
|
||||
return streamSSE(path, payload, onEvent, signal, true)
|
||||
}
|
||||
if (!resp.ok) {
|
||||
let detail = resp.statusText
|
||||
try { detail = (await resp.json()).detail || detail } catch { /* non-JSON */ }
|
||||
@@ -61,36 +49,6 @@ async function streamSSE(path, payload, onEvent, signal, isRetry = false) {
|
||||
}
|
||||
|
||||
export const api = {
|
||||
// Analytics. The beacon is deliberately not a `request()`: it must never
|
||||
// retry, never bootstrap a session, and never surface an error — a counter
|
||||
// that can interrupt the app it is counting is worse than no counter.
|
||||
trackPageview: (path, { referrer = '', first = false } = {}) => {
|
||||
try {
|
||||
fetch('/api/analytics/collect', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ path, referrer, first }),
|
||||
keepalive: true,
|
||||
}).catch(() => {})
|
||||
} catch { /* no beacon, no problem */ }
|
||||
},
|
||||
getAnalytics: (days) => request(`/analytics/summary?days=${days}`),
|
||||
getAccessLog: ({ beforeId, kind, q, limit = 50 } = {}) => {
|
||||
const params = new URLSearchParams({ limit })
|
||||
if (beforeId != null) params.set('before_id', beforeId)
|
||||
if (kind) params.set('kind', kind)
|
||||
if (q) params.set('q', q)
|
||||
return request(`/analytics/access?${params}`)
|
||||
},
|
||||
|
||||
// Auth (Phase 8 — no-ops in local mode beyond getMe)
|
||||
getMe: () => request('/auth/me'),
|
||||
register: (email, password) =>
|
||||
request('/auth/register', { method: 'POST', body: JSON.stringify({ email, password }) }),
|
||||
login: (email, password) =>
|
||||
request('/auth/login', { method: 'POST', body: JSON.stringify({ email, password }) }),
|
||||
logout: () => request('/auth/logout', { method: 'POST' }),
|
||||
|
||||
// Scenarios
|
||||
listScenarios: () => request('/scenarios'),
|
||||
getScenario: (id) => request(`/scenarios/${id}`),
|
||||
@@ -101,7 +59,6 @@ export const api = {
|
||||
// Adventures
|
||||
listAdventures: () => request('/adventures'),
|
||||
getAdventure: (id) => request(`/adventures/${id}`),
|
||||
getScriptState: (id) => request(`/adventures/${id}/script-state`),
|
||||
getWorldState: (id) => request(`/adventures/${id}/world-state`),
|
||||
overrideWorldState: (id, overrides) =>
|
||||
request(`/adventures/${id}/world-state`, { method: 'PUT', body: JSON.stringify(overrides) }),
|
||||
@@ -185,21 +142,6 @@ export const api = {
|
||||
deleteMemory: (advId, memoryId) =>
|
||||
request(`/adventures/${advId}/memories/${memoryId}`, { method: 'DELETE' }),
|
||||
|
||||
listAdventureScripts: (advId) => request(`/adventures/${advId}/scripts`),
|
||||
updateAdventureScript: (advId, scriptId, data) =>
|
||||
request(`/adventures/${advId}/scripts/${scriptId}`, { method: 'PATCH', body: JSON.stringify(data) }),
|
||||
syncAdventureScript: (advId, scriptId) =>
|
||||
request(`/adventures/${advId}/scripts/${scriptId}/sync`, { method: 'POST' }),
|
||||
|
||||
// Scripts
|
||||
listScripts: () => request('/scripts'),
|
||||
getScript: (id) => request(`/scripts/${id}`),
|
||||
createScript: (data) => request('/scripts', { method: 'POST', body: JSON.stringify(data) }),
|
||||
updateScript: (id, data) => request(`/scripts/${id}`, { method: 'PATCH', body: JSON.stringify(data) }),
|
||||
deleteScript: (id) => request(`/scripts/${id}`, { method: 'DELETE' }),
|
||||
testScript: (id, data) => request(`/scripts/${id}/test`, { method: 'POST', body: JSON.stringify(data) }),
|
||||
exportScript: (id) => request(`/scripts/${id}/export`),
|
||||
importScript: (bundle) => request('/scripts/import', { method: 'POST', body: JSON.stringify(bundle) }),
|
||||
|
||||
// Scenario import/export
|
||||
exportScenario: (id) => request(`/scenarios/${id}/export`),
|
||||
|
||||
@@ -287,111 +287,6 @@ export function BeginAdventureModal({ title, names = [], onSubmit, onCancel }) {
|
||||
|
||||
// Phase 8: register/login for the hosted multi-user mode. `onAuthed(me)` gets
|
||||
// the fresh /auth/me payload after success.
|
||||
export function AuthModal({ mode: initialMode, onClose, onAuthed, retentionDays }) {
|
||||
const [mode, setMode] = useState(initialMode || 'register')
|
||||
const [email, setEmail] = useState('')
|
||||
const [password, setPassword] = useState('')
|
||||
const [reveal, setReveal] = useState(false)
|
||||
const [error, setError] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
const registering = mode === 'register'
|
||||
|
||||
// Esc dismisses, like the overlay click already does.
|
||||
useEffect(() => {
|
||||
const onKey = (e) => { if (e.key === 'Escape') onClose() }
|
||||
window.addEventListener('keydown', onKey)
|
||||
return () => window.removeEventListener('keydown', onKey)
|
||||
}, [onClose])
|
||||
|
||||
const switchTo = (next) => {
|
||||
if (next === mode) return
|
||||
setMode(next)
|
||||
setError('')
|
||||
setReveal(false)
|
||||
}
|
||||
|
||||
const submit = async (e) => {
|
||||
e.preventDefault()
|
||||
setBusy(true)
|
||||
setError('')
|
||||
try {
|
||||
const address = email.trim()
|
||||
const me = registering
|
||||
? await api.register(address, password)
|
||||
: await api.login(address, password)
|
||||
onAuthed(me, mode)
|
||||
} catch (err) {
|
||||
setError(err.message)
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="modal-overlay" onClick={onClose}>
|
||||
<form className="modal auth-modal" onClick={(e) => e.stopPropagation()} onSubmit={submit}
|
||||
aria-labelledby="auth-title">
|
||||
<div className="auth-crest" aria-hidden="true">❖</div>
|
||||
<h2 id="auth-title">{registering ? 'Create an account' : 'Welcome back'}</h2>
|
||||
<p className="modal-hint auth-hint">
|
||||
{registering
|
||||
? 'Everything you’ve played as a guest stays with your new account, and you can pick it up from any device.'
|
||||
: 'Log in to reach your adventures.'}
|
||||
{/* Guest data really is deleted, so say so where the decision is
|
||||
being made. The window comes from the server (see cleanup.py) so
|
||||
it can't drift from what's enforced. */}
|
||||
{registering && retentionDays ? (
|
||||
<> Guest adventures are deleted after {retentionDays} days without a visit.</>
|
||||
) : null}
|
||||
</p>
|
||||
|
||||
<div className="auth-tabs" role="tablist">
|
||||
<button type="button" role="tab" aria-selected={!registering}
|
||||
className={`auth-tab${registering ? '' : ' active'}`}
|
||||
onClick={() => switchTo('login')}>Log in</button>
|
||||
<button type="button" role="tab" aria-selected={registering}
|
||||
className={`auth-tab${registering ? ' active' : ''}`}
|
||||
onClick={() => switchTo('register')}>Sign up</button>
|
||||
</div>
|
||||
|
||||
<label className="field">
|
||||
<span className="label">Email</span>
|
||||
<input type="email" autoFocus required value={email}
|
||||
autoComplete="email" placeholder="you@example.com"
|
||||
onChange={(e) => setEmail(e.target.value)} />
|
||||
</label>
|
||||
<label className="field">
|
||||
<span className="label">Password</span>
|
||||
<div className="auth-password">
|
||||
<input type={reveal ? 'text' : 'password'} required
|
||||
minLength={registering ? 8 : undefined} value={password}
|
||||
autoComplete={registering ? 'new-password' : 'current-password'}
|
||||
onChange={(e) => setPassword(e.target.value)} />
|
||||
<button type="button" className="auth-reveal" tabIndex={-1}
|
||||
aria-label={reveal ? 'Hide password' : 'Show password'}
|
||||
onClick={() => setReveal((r) => !r)}>
|
||||
{reveal ? 'Hide' : 'Show'}
|
||||
</button>
|
||||
</div>
|
||||
{registering && <span className="auth-help">At least 8 characters.</span>}
|
||||
</label>
|
||||
|
||||
{error && <div className="auth-error" role="alert">{error}</div>}
|
||||
|
||||
<div className="modal-buttons">
|
||||
<button type="button" onClick={onClose}>Cancel</button>
|
||||
<button type="submit" className="primary" disabled={busy}>
|
||||
{busy ? 'Please wait…' : registering ? 'Sign up' : 'Log in'}
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/** A textarea that grows to fit its content instead of sitting at a fixed
|
||||
height. Editing an AI beat means editing a few paragraphs, and a fixed box
|
||||
made that a keyhole. CSS min/max-height still bound it (the cap scrolls),
|
||||
so callers control the floor and ceiling from the stylesheet. */
|
||||
export function AutoTextarea({ value, ...props }) {
|
||||
const ref = useRef(null)
|
||||
|
||||
|
||||
@@ -7,11 +7,8 @@ import Adventures from './pages/Adventures.jsx'
|
||||
import Scenarios from './pages/Scenarios.jsx'
|
||||
import ScenarioEditor from './pages/ScenarioEditor.jsx'
|
||||
import Play from './pages/Play'
|
||||
import Scripts from './pages/Scripts.jsx'
|
||||
import ScriptEditor from './pages/ScriptEditor.jsx'
|
||||
import Settings from './pages/Settings.jsx'
|
||||
import Chat from './pages/Chat.jsx'
|
||||
import Analytics from './pages/Analytics.jsx'
|
||||
import { trackKeyboardInset } from './keyboard.js'
|
||||
import './index.css'
|
||||
|
||||
@@ -27,13 +24,8 @@ const router = createBrowserRouter([
|
||||
{ path: 'scenarios', element: <Scenarios /> },
|
||||
{ path: 'scenarios/:id', element: <ScenarioEditor /> },
|
||||
{ path: 'play/:id', element: <Play /> },
|
||||
{ path: 'scripts', element: <Scripts /> },
|
||||
{ path: 'scripts/:id', element: <ScriptEditor /> },
|
||||
{ path: 'settings', element: <Settings /> },
|
||||
// Power users only — the page redirects home and the API 404s otherwise.
|
||||
{ path: 'chat', element: <Chat /> },
|
||||
// Owner only, by a separate allowlist; same redirect-and-404 treatment.
|
||||
{ path: 'analytics', element: <Analytics /> },
|
||||
],
|
||||
},
|
||||
])
|
||||
|
||||
@@ -1,476 +0,0 @@
|
||||
/* Visit analytics — the owner's view of who came by and what they did.
|
||||
|
||||
Everything on this page arrives in a single aggregate response (see
|
||||
backend/app/analytics.py), so changing the range is one small request, not a
|
||||
scan of anything. The page is hidden from everyone else: the nav link is
|
||||
gated on `me.analytics`, this component bounces, and the API 404s.
|
||||
|
||||
Charts are plain HTML — a flex row of columns, a row of bars — rather than
|
||||
SVG or a charting library. At this size that is less code, responsive for
|
||||
free, and keeps the CSP as tight as it is. */
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { useNavigate, useOutletContext } from 'react-router-dom'
|
||||
import { api } from '../api'
|
||||
|
||||
const RANGES = [
|
||||
{ days: 7, label: '7 days' },
|
||||
{ days: 30, label: '30 days' },
|
||||
{ days: 90, label: '90 days' },
|
||||
]
|
||||
|
||||
const nf = new Intl.NumberFormat()
|
||||
|
||||
// Weekday + day for a short range, day + month for a long one: a 90-day axis
|
||||
// has no room for "Mon".
|
||||
function dayLabel(iso, days) {
|
||||
const date = new Date(`${iso}T00:00:00Z`)
|
||||
const opts = days <= 14
|
||||
? { weekday: 'short', timeZone: 'UTC' }
|
||||
: { day: 'numeric', month: 'short', timeZone: 'UTC' }
|
||||
return date.toLocaleDateString(undefined, opts)
|
||||
}
|
||||
|
||||
function fullDate(iso) {
|
||||
return new Date(`${iso}T00:00:00Z`)
|
||||
.toLocaleDateString(undefined, { dateStyle: 'medium', timeZone: 'UTC' })
|
||||
}
|
||||
|
||||
/* ---------- Pieces ---------- */
|
||||
|
||||
function StatTile({ label, value, hint }) {
|
||||
return (
|
||||
<div className="an-tile" title={hint || undefined}>
|
||||
<div className="an-tile-value">{nf.format(value ?? 0)}</div>
|
||||
<div className="an-tile-label">{label}</div>
|
||||
{hint && <div className="an-tile-hint">{hint}</div>}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/* A day-by-day column chart. `series` names the stacked segments bottom-up;
|
||||
one segment means one plain bar and no legend, since the title already says
|
||||
what it is. */
|
||||
function DayChart({ title, data, series, days }) {
|
||||
const [hover, setHover] = useState(null)
|
||||
const max = Math.max(1, ...data.map((d) => series.reduce((sum, s) => sum + (d[s.key] || 0), 0)))
|
||||
// Only ever three x labels. Every column labelled is unreadable at 90 days
|
||||
// and redundant at 7 — the tooltip carries the exact date either way.
|
||||
const ticks = new Set([0, Math.floor((data.length - 1) / 2), data.length - 1])
|
||||
const empty = data.every((d) => series.every((s) => !d[s.key]))
|
||||
|
||||
return (
|
||||
<section className="an-card an-chart">
|
||||
<header className="an-card-head">
|
||||
<h2>{title}</h2>
|
||||
{series.length > 1 && (
|
||||
<div className="an-legend">
|
||||
{[...series].reverse().map((s) => (
|
||||
<span key={s.key} className="an-legend-item">
|
||||
<i className="an-swatch" style={{ background: s.color }} />
|
||||
{s.label}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</header>
|
||||
<div className="an-plot" onMouseLeave={() => setHover(null)}>
|
||||
<div className="an-gridline" style={{ bottom: '100%' }}><span>{nf.format(max)}</span></div>
|
||||
<div className="an-gridline" style={{ bottom: '50%' }}><span>{nf.format(Math.round(max / 2))}</span></div>
|
||||
<div className="an-columns">
|
||||
{data.map((point, i) => {
|
||||
const total = series.reduce((sum, s) => sum + (point[s.key] || 0), 0)
|
||||
return (
|
||||
<div
|
||||
key={point.day}
|
||||
className={`an-column${hover === i ? ' hot' : ''}`}
|
||||
onMouseEnter={() => setHover(i)}
|
||||
onFocus={() => setHover(i)}
|
||||
tabIndex={0}
|
||||
aria-label={`${fullDate(point.day)}: ${total}`}
|
||||
>
|
||||
<div className="an-stack">
|
||||
{[...series].reverse().map((s) => (
|
||||
(point[s.key] || 0) > 0 && (
|
||||
<div
|
||||
key={s.key}
|
||||
className="an-bar"
|
||||
style={{
|
||||
height: `${((point[s.key] || 0) / max) * 100}%`,
|
||||
background: s.color,
|
||||
}}
|
||||
/>
|
||||
)
|
||||
))}
|
||||
</div>
|
||||
{hover === i && (
|
||||
<div className="an-tip">
|
||||
<strong>{fullDate(point.day)}</strong>
|
||||
{series.map((s) => (
|
||||
<span key={s.key}>
|
||||
<i className="an-swatch" style={{ background: s.color }} />
|
||||
{s.label}: {nf.format(point[s.key] || 0)}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div className="an-axis">
|
||||
{data.map((point, i) => (
|
||||
<span key={point.day}>{ticks.has(i) ? dayLabel(point.day, days) : ''}</span>
|
||||
))}
|
||||
</div>
|
||||
{empty && <div className="an-overlay-empty">Nothing recorded in this range</div>}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
/* The funnel. Each step's bar is drawn against the first step, so the shape of
|
||||
the drop-off is the picture; the percentage beside it is of the step above,
|
||||
which is the number you act on. */
|
||||
function Funnel({ steps }) {
|
||||
const top = steps[0]?.count || 0
|
||||
return (
|
||||
<section className="an-card">
|
||||
<header className="an-card-head">
|
||||
<h2>Where visitors get to</h2>
|
||||
<span className="an-note">people, counted once each</span>
|
||||
</header>
|
||||
{top === 0 ? (
|
||||
<div className="an-empty">No visitors in this range.</div>
|
||||
) : (
|
||||
<div className="an-funnel">
|
||||
{steps.map((step, i) => {
|
||||
const previous = i === 0 ? step.count : steps[i - 1].count
|
||||
const share = previous ? Math.round((step.count / previous) * 100) : 0
|
||||
return (
|
||||
<div key={step.step} className="an-funnel-row">
|
||||
<div className="an-funnel-label">{step.step}</div>
|
||||
<div className="an-funnel-track">
|
||||
<div
|
||||
className="an-funnel-bar"
|
||||
style={{ width: `${top ? (step.count / top) * 100 : 0}%` }}
|
||||
/>
|
||||
</div>
|
||||
<div className="an-funnel-value">
|
||||
{nf.format(step.count)}
|
||||
{i > 0 && <span className="an-funnel-share">{share}%</span>}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
function TopList({ title, rows, note, empty = 'Nothing yet' }) {
|
||||
const max = Math.max(1, ...rows.map((r) => r.hits))
|
||||
return (
|
||||
<section className="an-card">
|
||||
<header className="an-card-head">
|
||||
<h2>{title}</h2>
|
||||
{note && <span className="an-note">{note}</span>}
|
||||
</header>
|
||||
{rows.length === 0 ? (
|
||||
<div className="an-empty">{empty}</div>
|
||||
) : (
|
||||
<ul className="an-list">
|
||||
{rows.map((row) => (
|
||||
<li key={row.label}>
|
||||
{/* The bar is the row's own background, so a long label stays
|
||||
readable on top of it instead of being squeezed beside it. */}
|
||||
<span className="an-list-fill" style={{ width: `${(row.hits / max) * 100}%` }} />
|
||||
<span className="an-list-label" title={row.label}>{row.label}</span>
|
||||
<span className="an-list-value">{nf.format(row.hits)}</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
/* ---------- Access log ---------- */
|
||||
|
||||
const KINDS = [
|
||||
{ key: '', label: 'Everything' },
|
||||
{ key: 'session', label: 'Sessions' },
|
||||
{ key: 'login', label: 'Sign-ins' },
|
||||
{ key: 'register', label: 'Registrations' },
|
||||
{ key: 'login_failed', label: 'Failed' },
|
||||
]
|
||||
|
||||
const KIND_LABEL = {
|
||||
session: 'Session',
|
||||
login: 'Signed in',
|
||||
register: 'Registered',
|
||||
login_failed: 'Failed sign-in',
|
||||
}
|
||||
|
||||
function when(iso) {
|
||||
const date = new Date(iso.endsWith('Z') ? iso : `${iso}Z`)
|
||||
return date.toLocaleString(undefined, { dateStyle: 'medium', timeStyle: 'short' })
|
||||
}
|
||||
|
||||
function AccessLog() {
|
||||
const [kind, setKind] = useState('')
|
||||
const [search, setSearch] = useState('')
|
||||
const [query, setQuery] = useState('')
|
||||
const [page, setPage] = useState(null) // { events, has_more }
|
||||
const [error, setError] = useState(null)
|
||||
const [busy, setBusy] = useState(false)
|
||||
|
||||
// Typing shouldn't fire a request per keystroke against a table scan.
|
||||
useEffect(() => {
|
||||
const timer = setTimeout(() => setQuery(search.trim()), 350)
|
||||
return () => clearTimeout(timer)
|
||||
}, [search])
|
||||
|
||||
useEffect(() => {
|
||||
let live = true
|
||||
setPage(null)
|
||||
api.getAccessLog({ kind, q: query }).then(
|
||||
(result) => { if (live) { setPage(result); setError(null) } },
|
||||
(err) => { if (live) setError(err.message) },
|
||||
)
|
||||
return () => { live = false }
|
||||
}, [kind, query])
|
||||
|
||||
const loadMore = useCallback(async () => {
|
||||
if (!page?.events.length || busy) return
|
||||
setBusy(true)
|
||||
try {
|
||||
// Anchored on the oldest row already on screen, so rows arriving while
|
||||
// this is open can't shift the next page.
|
||||
const next = await api.getAccessLog({
|
||||
kind, q: query, beforeId: page.events[page.events.length - 1].id,
|
||||
})
|
||||
setPage({ events: [...page.events, ...next.events], has_more: next.has_more })
|
||||
} catch (err) {
|
||||
setError(err.message)
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}, [page, kind, query, busy])
|
||||
|
||||
return (
|
||||
<section className="an-card">
|
||||
<header className="an-card-head an-log-head">
|
||||
<div className="an-ranges">
|
||||
{KINDS.map((option) => (
|
||||
<button
|
||||
key={option.key}
|
||||
className={kind === option.key ? 'primary' : ''}
|
||||
onClick={() => setKind(option.key)}
|
||||
>
|
||||
{option.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<input
|
||||
type="text"
|
||||
className="search-input an-log-search"
|
||||
placeholder="Search email, IP, country…"
|
||||
value={search}
|
||||
onChange={(event) => setSearch(event.target.value)}
|
||||
/>
|
||||
</header>
|
||||
|
||||
{error && <div className="an-empty">Couldn’t load the log: {error}</div>}
|
||||
{!page && !error && <div className="an-empty">Reading…</div>}
|
||||
|
||||
{page && (page.events.length === 0 ? (
|
||||
<div className="an-empty">
|
||||
{query || kind ? 'Nothing matches that.' : 'Nothing logged yet.'}
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="an-table-scroll">
|
||||
<table className="an-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>When</th><th>Who</th><th>Event</th>
|
||||
<th>IP</th><th>Country</th><th>Device</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{page.events.map((event) => (
|
||||
<tr key={event.id} className={event.kind === 'login_failed' ? 'failed' : undefined}>
|
||||
<td className="an-cell-dim">{when(event.at)}</td>
|
||||
<td>
|
||||
{event.who}
|
||||
{event.is_guest && <span className="an-tag">guest</span>}
|
||||
</td>
|
||||
<td>{KIND_LABEL[event.kind] || event.kind}</td>
|
||||
<td className="an-cell-mono">{event.ip || '—'}</td>
|
||||
<td>{event.country || '—'}</td>
|
||||
{/* The full user-agent is a wall of text; it lives on the
|
||||
hover instead of in a column that would push the rest
|
||||
of the table off screen. */}
|
||||
<td className="an-cell-dim" title={event.user_agent || undefined}>
|
||||
{event.device || '—'}
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{page.has_more && (
|
||||
<div className="an-more">
|
||||
<button onClick={loadMore} disabled={busy}>
|
||||
{busy ? 'Loading…' : 'Load older'}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
))}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
/* ---------- Page ---------- */
|
||||
|
||||
export default function Analytics() {
|
||||
const { me } = useOutletContext() ?? {}
|
||||
const navigate = useNavigate()
|
||||
const [tab, setTab] = useState('overview')
|
||||
const [days, setDays] = useState(30)
|
||||
const [data, setData] = useState(null)
|
||||
const [error, setError] = useState(null)
|
||||
const loaded = useRef(false)
|
||||
|
||||
// me is null until /auth/me resolves; only bounce once we know.
|
||||
useEffect(() => {
|
||||
if (me && !me.analytics) navigate('/', { replace: true })
|
||||
}, [me, navigate])
|
||||
|
||||
useEffect(() => {
|
||||
if (tab !== 'overview') return undefined
|
||||
let live = true
|
||||
api.getAnalytics(days).then(
|
||||
(result) => { if (live) { setData(result); setError(null); loaded.current = true } },
|
||||
(err) => { if (live) setError(err.message) },
|
||||
)
|
||||
return () => { live = false }
|
||||
}, [days, tab])
|
||||
|
||||
const totals = data?.totals ?? {}
|
||||
const visitorSeries = useMemo(() => ([
|
||||
{ key: 'returning', label: 'Returning', color: 'var(--chart-2)' },
|
||||
{ key: 'new', label: 'New', color: 'var(--chart-1)' },
|
||||
]), [])
|
||||
// The server sends new-vs-total; the chart stacks, so it wants the remainder.
|
||||
const visitorDays = useMemo(
|
||||
() => (data?.series ?? []).map((d) => ({ ...d, returning: d.visitors - d.new })),
|
||||
[data],
|
||||
)
|
||||
|
||||
if (me && !me.analytics) return null
|
||||
|
||||
return (
|
||||
<div className="page an-page">
|
||||
<div className="page-header">
|
||||
<h1>Visitors</h1>
|
||||
{tab === 'overview' && (
|
||||
<div className="an-ranges">
|
||||
{RANGES.map((range) => (
|
||||
<button
|
||||
key={range.days}
|
||||
className={days === range.days ? 'primary' : ''}
|
||||
onClick={() => setDays(range.days)}
|
||||
>
|
||||
{range.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="an-tabs">
|
||||
<button
|
||||
className={tab === 'overview' ? 'active' : ''}
|
||||
onClick={() => setTab('overview')}
|
||||
>
|
||||
Overview
|
||||
</button>
|
||||
<button
|
||||
className={tab === 'access' ? 'active' : ''}
|
||||
onClick={() => setTab('access')}
|
||||
>
|
||||
Access log
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{tab === 'access' && <AccessLog />}
|
||||
|
||||
{tab === 'overview' && error && (
|
||||
<div className="an-empty">Couldn’t load analytics: {error}</div>
|
||||
)}
|
||||
{tab === 'overview' && !data && !error && <div className="an-empty">Counting…</div>}
|
||||
|
||||
{tab === 'overview' && data && (
|
||||
<>
|
||||
<div className="an-tiles">
|
||||
<StatTile label="Visitors" value={totals.visitors}
|
||||
hint={`${nf.format(totals.new_visitors || 0)} first-time`} />
|
||||
<StatTile label="Visits" value={totals.visits}
|
||||
hint={`${totals.pages_per_visit || 0} pages each`} />
|
||||
<StatTile label="Pageviews" value={totals.pageviews} />
|
||||
<StatTile label="Adventures started" value={totals.adventures} />
|
||||
<StatTile label="Turns played" value={totals.turns}
|
||||
hint={`${totals.turns_per_visit || 0} per visit`} />
|
||||
<StatTile label="Sign-ups" value={totals.signups} />
|
||||
<StatTile label="Demo-key turns" value={totals.demo_turns}
|
||||
hint="billed to the shared key" />
|
||||
<StatTile label="Failed turns" value={totals.turn_errors}
|
||||
hint={`${nf.format(totals.errors || 0)} API errors`} />
|
||||
</div>
|
||||
|
||||
<div className="an-grid">
|
||||
<DayChart
|
||||
title="Visitors per day"
|
||||
data={visitorDays}
|
||||
series={visitorSeries}
|
||||
days={days}
|
||||
/>
|
||||
<DayChart
|
||||
title="Turns played per day"
|
||||
data={data.series}
|
||||
series={[{ key: 'turns', label: 'Turns', color: 'var(--chart-3)' }]}
|
||||
days={days}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Funnel steps={data.funnel} />
|
||||
|
||||
<div className="an-grid">
|
||||
<TopList title="Pages" rows={data.pages} />
|
||||
<TopList
|
||||
title="Where they came from"
|
||||
rows={data.referrers}
|
||||
note="per visit"
|
||||
empty="No referrals yet — every visit was typed or bookmarked."
|
||||
/>
|
||||
<TopList title="Scenarios started" rows={data.scenarios}
|
||||
note="shared scenarios only" empty="No adventures started yet." />
|
||||
<TopList title="Countries" rows={data.countries} />
|
||||
<TopList title="Devices" rows={data.devices} />
|
||||
<TopList title="API errors" rows={data.errors} empty="None — clean run." />
|
||||
</div>
|
||||
|
||||
<p className="an-footnote">
|
||||
{fullDate(data.since)} – {fullDate(data.until)}, UTC. Your own visits aren’t
|
||||
counted here. These totals are anonymous — visitors are counted as one-way
|
||||
hashes, and nothing on this tab can be traced back to a player or their
|
||||
stories. The access log tab is the separate, identifying record.
|
||||
</p>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,12 +1,11 @@
|
||||
/* AI Chat — a plain scratchpad for talking to a model, with none of the game's
|
||||
context assembly in the way. Power users only (the backend 404s the routes
|
||||
for everyone else, and the nav link is hidden).
|
||||
/* AI Chat — a plain scratchpad for talking to the configured model, with none
|
||||
of the game's context assembly in the way. Useful for checking a model or a
|
||||
prompt without starting an adventure.
|
||||
|
||||
Deliberately client-side: the conversation lives in localStorage, not the
|
||||
database. Nothing here is part of an adventure, so there's nothing worth a
|
||||
migration — and a refresh still keeps what you were poking at. */
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { useNavigate, useOutletContext } from 'react-router-dom'
|
||||
import { api } from '../api'
|
||||
import { useToast } from '../components'
|
||||
|
||||
@@ -64,8 +63,6 @@ function Message({ message, onDelete }) {
|
||||
}
|
||||
|
||||
export default function Chat() {
|
||||
const { me } = useOutletContext() ?? {}
|
||||
const navigate = useNavigate()
|
||||
const toast = useToast()
|
||||
|
||||
const initial = useRef(load()).current
|
||||
@@ -88,10 +85,6 @@ export default function Chat() {
|
||||
const inputRef = useRef(null)
|
||||
const pinnedRef = useRef(true)
|
||||
|
||||
// me is null until /auth/me resolves; only bounce once we know.
|
||||
useEffect(() => {
|
||||
if (me && !me.power_user) navigate('/', { replace: true })
|
||||
}, [me, navigate])
|
||||
|
||||
useEffect(() => {
|
||||
api.getChatConfig().then(setConfig).catch(() => setConfig(null))
|
||||
@@ -226,7 +219,6 @@ export default function Chat() {
|
||||
}
|
||||
}
|
||||
|
||||
if (me && !me.power_user) return null
|
||||
const waitingForFirstToken = streaming === '' && reasoningStream === null
|
||||
const canRegenerate = !busy && messages.some((m) => m.role === 'user')
|
||||
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
// The left drawer: the script state an adventure's scripts read and write.
|
||||
//
|
||||
// `StateTree` and `StateValue` render a value of any shape, because script
|
||||
// state is whatever the scripts put there.
|
||||
|
||||
import { useCallback, useEffect, useState } from 'react'
|
||||
import { api } from '../../../api'
|
||||
|
||||
// Renders one script-state value. A primitive renders inline, typed and
|
||||
// colored. An object or an array renders as a collapsible indented tree, and
|
||||
// the render recurses, so deep state shows its structure rather than one flat
|
||||
// JSON blob.
|
||||
function StateValue({ value, depth = 0 }) {
|
||||
if (value === null || value === undefined) return <span className="jv-null">null</span>
|
||||
if (typeof value === 'boolean') return <span className="jv-bool">{String(value)}</span>
|
||||
if (typeof value === 'number') return <span className="jv-number">{value}</span>
|
||||
if (typeof value === 'string') return <span className="jv-string">{value}</span>
|
||||
if (Array.isArray(value)) return <StateTree entries={value.map((v, i) => [i, v])} empty="[ ]" depth={depth} />
|
||||
if (typeof value === 'object') return <StateTree entries={Object.entries(value)} empty="{ }" depth={depth} />
|
||||
return <span className="jv-string">{String(value)}</span>
|
||||
}
|
||||
|
||||
// Only the first level is expanded by default (depth 0); nested trees start
|
||||
// collapsed and can be opened on demand.
|
||||
function StateTree({ entries, empty, depth = 0 }) {
|
||||
const [open, setOpen] = useState(depth < 1)
|
||||
if (entries.length === 0) return <span className="jv-empty">{empty}</span>
|
||||
return (
|
||||
<div className="jv-tree">
|
||||
<button className="jv-toggle" onClick={() => setOpen((o) => !o)}>
|
||||
{open ? '▾' : '▸'} {entries.length} {entries.length === 1 ? 'item' : 'items'}
|
||||
</button>
|
||||
{open && (
|
||||
<ul className="jv-children">
|
||||
{entries.map(([k, v]) => (
|
||||
<li key={k}>
|
||||
<span className="jv-key">{k}</span>
|
||||
<StateValue value={v} depth={depth + 1} />
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Collapsible left rail showing the scripting `state` object — every variable
|
||||
// scripts read/write via state.x, refreshed after each turn.
|
||||
function StatusDrawer({ advId, refreshKey }) {
|
||||
const [open, setOpen] = useState(false)
|
||||
const [state, setState] = useState(null)
|
||||
const [failed, setFailed] = useState(false)
|
||||
|
||||
const load = useCallback(() => {
|
||||
api.getScriptState(advId)
|
||||
.then((r) => { setState(r.state || {}); setFailed(false) })
|
||||
.catch(() => setFailed(true))
|
||||
}, [advId])
|
||||
|
||||
// Only fetch while open; re-fetch after each turn so values stay live.
|
||||
useEffect(() => { if (open) load() }, [open, refreshKey, load])
|
||||
|
||||
const entries = state ? Object.entries(state) : []
|
||||
|
||||
return (
|
||||
<div className={`status-drawer ${open ? 'open' : ''}`}>
|
||||
<button className="status-toggle" onClick={() => setOpen((o) => !o)}
|
||||
title="Script state variables">
|
||||
{open ? '‹' : '›'}<span className="status-toggle-label">State</span>
|
||||
</button>
|
||||
{open && (
|
||||
<div className="status-body">
|
||||
<div className="side-panel-header">
|
||||
<h2>Script State</h2>
|
||||
<button onClick={load} title="Refresh">↻</button>
|
||||
</div>
|
||||
{failed ? (
|
||||
<div className="empty">Couldn’t load state.</div>
|
||||
) : entries.length === 0 ? (
|
||||
<div className="empty">
|
||||
No variables yet. Scripts that use <code>state</code> will appear here after a turn.
|
||||
</div>
|
||||
) : (
|
||||
<ul className="status-vars">
|
||||
{entries.map(([k, v]) => (
|
||||
<li key={k}>
|
||||
<span className="status-key">{k}</span>
|
||||
<div className="status-val"><StateValue value={v} /></div>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export { StatusDrawer }
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
const SECTION_LABELS = {
|
||||
narrator: 'Narrator prompt',
|
||||
script_context: 'Script context',
|
||||
ai_instructions: 'AI Instructions',
|
||||
plot_essentials: 'Plot Essentials',
|
||||
story_summary: 'Story Summary',
|
||||
@@ -32,7 +31,6 @@ const SECTION_COLORS = {
|
||||
narrator: '#7d8fc9',
|
||||
ai_instructions: '#9c8fd6',
|
||||
plot_essentials: '#c97dc0',
|
||||
script_context: '#d99ad0',
|
||||
story_summary: '#7dc9a2',
|
||||
used_memories: '#5fb8c9',
|
||||
world_lore: '#c9b47d',
|
||||
|
||||
@@ -17,13 +17,11 @@ import { api } from '../../api'
|
||||
import { AutoTextarea } from '../../components'
|
||||
import { StateChangeChips } from './reports'
|
||||
import { TakePager } from './TakePager'
|
||||
import { StatusDrawer } from './drawers/StatusDrawer'
|
||||
import { WorldStateDrawer } from './drawers/WorldStateDrawer'
|
||||
import { BranchPanel } from './panels/BranchPanel'
|
||||
import { InsightsPanel } from './panels/InsightsPanel'
|
||||
import { MemoryPanel } from './panels/MemoryPanel'
|
||||
import { PlotPanel } from './panels/PlotPanel'
|
||||
import { ScriptsPanel } from './panels/ScriptsPanel'
|
||||
|
||||
const MODES = ['do', 'say', 'story']
|
||||
const PLAYER_TYPES = ['do', 'say', 'story']
|
||||
@@ -175,7 +173,7 @@ export default function Play() {
|
||||
setActions(page.actions)
|
||||
setTotal(page.total)
|
||||
setHasMore(page.has_more)
|
||||
// The script and world state come back to what that branch's tip left
|
||||
// The world state comes back to what that branch's tip left
|
||||
// behind, so anything drawn from them is now showing another line's
|
||||
// numbers until it re-reads.
|
||||
setStateKey((k) => k + 1)
|
||||
@@ -264,15 +262,6 @@ export default function Play() {
|
||||
}
|
||||
}, [actions, streaming, reasoningStream])
|
||||
|
||||
const handleScriptReport = useCallback((script) => {
|
||||
if (!script) return
|
||||
if (script.errors?.length) {
|
||||
setToast({ text: `Script error: ${script.errors[0]}`, isError: true })
|
||||
} else if (script.message) {
|
||||
setToast({ text: script.message, isError: false })
|
||||
}
|
||||
}, [])
|
||||
|
||||
const handleEvent = useCallback((event) => {
|
||||
if (event.type === 'player') {
|
||||
setActions((prev) => [...prev, event.action])
|
||||
@@ -288,17 +277,12 @@ export default function Play() {
|
||||
setReasoningStream(null)
|
||||
setActions((prev) => [...prev, event.action])
|
||||
setTotal((n) => n + 1)
|
||||
handleScriptReport(event.script)
|
||||
} else if (event.type === 'stopped') {
|
||||
setStreaming(null)
|
||||
setReasoningStream(null)
|
||||
handleScriptReport(event.script)
|
||||
} else if (event.type === 'error') {
|
||||
setStreaming(null)
|
||||
setReasoningStream(null)
|
||||
setToast({ text: event.detail, isError: true })
|
||||
}
|
||||
}, [handleScriptReport])
|
||||
}, [])
|
||||
|
||||
async function runTurn(run) {
|
||||
const controller = new AbortController()
|
||||
@@ -483,7 +467,7 @@ export default function Play() {
|
||||
setActions(adv.actions)
|
||||
setTotal(adv.action_count ?? adv.actions.length)
|
||||
setHasMore(adv.actions.length < (adv.action_count ?? adv.actions.length))
|
||||
// The branch, the script state and the world state can all have moved.
|
||||
// The branch and the world state can both have moved.
|
||||
setStateKey((k) => k + 1)
|
||||
} catch { /* stale beats wrong */ }
|
||||
}
|
||||
@@ -516,7 +500,6 @@ export default function Play() {
|
||||
without changing `actions.length` by one. */}
|
||||
<WorldStateDrawer advId={id} refreshKey={`${actions.length}:${stateKey}`}
|
||||
personaName={adventure?.persona_name} />
|
||||
<StatusDrawer advId={id} refreshKey={`${actions.length}:${stateKey}`} />
|
||||
<div className="page play-page">
|
||||
<div className="page-header">
|
||||
<h1>{adventure.title}</h1>
|
||||
@@ -525,8 +508,6 @@ export default function Play() {
|
||||
onClick={() => setPanel(panel === 'plot' ? null : 'plot')}>Plot</button>
|
||||
<button className={panel === 'memory' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'memory' ? null : 'memory')}>Memory</button>
|
||||
<button className={panel === 'scripts' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'scripts' ? null : 'scripts')}>Scripts</button>
|
||||
<button className={panel === 'branches' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'branches' ? null : 'branches')}>Branches</button>
|
||||
<button className={panel === 'insights' ? 'active' : ''}
|
||||
@@ -731,7 +712,7 @@ export default function Play() {
|
||||
{panel && (
|
||||
<div className="side-panel">
|
||||
<div className="side-panel-header">
|
||||
<h2>{{ plot: 'Plot Components', memory: 'Memory Bank', scripts: 'Scripts', branches: 'Branches', insights: 'Insights' }[panel]}</h2>
|
||||
<h2>{{ plot: 'Plot Components', memory: 'Memory Bank', branches: 'Branches', insights: 'Insights' }[panel]}</h2>
|
||||
<button onClick={() => setPanel(null)}>✕</button>
|
||||
</div>
|
||||
{panel === 'plot' ? (
|
||||
@@ -743,8 +724,6 @@ export default function Play() {
|
||||
// but deleting a branch deletes the memories that hung off it,
|
||||
// and that happens without a turn being played.
|
||||
refreshKey={`${actions.length}:${stateKey}`} />
|
||||
) : panel === 'scripts' ? (
|
||||
<ScriptsPanel advId={id} />
|
||||
) : panel === 'branches' ? (
|
||||
<BranchPanel
|
||||
advId={id}
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { api } from '../../../api'
|
||||
import { SECTION_LABELS, pctLabel, sectionColor } from '../format'
|
||||
import { CacheReport, ScriptReport, TokenBreakdown, WorldStateReport } from '../reports'
|
||||
import { CacheReport, TokenBreakdown, WorldStateReport } from '../reports'
|
||||
|
||||
function InsightsPanel({ advId, inspectActionId, onClearInspect, refreshKey }) {
|
||||
const [report, setReport] = useState(null)
|
||||
@@ -104,7 +104,6 @@ function InsightsPanel({ advId, inspectActionId, onClearInspect, refreshKey }) {
|
||||
<pre>{s.text}</pre>
|
||||
</div>
|
||||
))}
|
||||
<ScriptReport script={report.script} />
|
||||
<WorldStateReport worldState={report.world_state} />
|
||||
{report.raw_output && (
|
||||
<div className="ctx-section">
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
// The scripts panel: this adventure's copies of the library scripts.
|
||||
|
||||
import { useEffect, useState } from 'react'
|
||||
import { api } from '../../../api'
|
||||
import { downloadJSON } from '../../../components'
|
||||
|
||||
const SCRIPT_HOOKS = [
|
||||
['Shared library', 'library_js'],
|
||||
['onInput', 'input_js'],
|
||||
['onModelContext', 'context_js'],
|
||||
['onOutput', 'output_js'],
|
||||
]
|
||||
|
||||
function ScriptsPanel({ advId }) {
|
||||
const [scripts, setScripts] = useState(null)
|
||||
const [openId, setOpenId] = useState(null)
|
||||
|
||||
useEffect(() => {
|
||||
api.listAdventureScripts(advId).then(setScripts).catch(() => setScripts([]))
|
||||
}, [advId])
|
||||
|
||||
const [syncingId, setSyncingId] = useState(null)
|
||||
|
||||
const toggle = async (script) => {
|
||||
const updated = await api.updateAdventureScript(advId, script.id, { enabled: !script.enabled })
|
||||
setScripts((prev) => prev.map((s) => (s.id === script.id ? updated : s)))
|
||||
}
|
||||
|
||||
// Pull the latest code from the library script this copy was made from.
|
||||
const sync = async (script) => {
|
||||
setSyncingId(script.id)
|
||||
try {
|
||||
const updated = await api.syncAdventureScript(advId, script.id)
|
||||
setScripts((prev) => prev.map((s) => (s.id === script.id ? updated : s)))
|
||||
} finally {
|
||||
setSyncingId(null)
|
||||
}
|
||||
}
|
||||
|
||||
// Download as an import-compatible bundle (matches /scripts export), so demo
|
||||
// scripts can be forked into your own library via the Scripts page's Import.
|
||||
const download = (s) => {
|
||||
downloadJSON(
|
||||
{
|
||||
name: s.name,
|
||||
description: s.description,
|
||||
library: s.library_js,
|
||||
input: s.input_js,
|
||||
context: s.context_js,
|
||||
output: s.output_js,
|
||||
},
|
||||
`${(s.name || 'script').replace(/\W+/g, '-')}.json`,
|
||||
)
|
||||
}
|
||||
|
||||
if (!scripts) return <div className="empty">Loading…</div>
|
||||
if (scripts.length === 0) {
|
||||
return (
|
||||
<div className="empty">
|
||||
No scripts on this adventure. Attach scripts to a scenario before starting an
|
||||
adventure from it.
|
||||
</div>
|
||||
)
|
||||
}
|
||||
return (
|
||||
<div>
|
||||
{scripts.map((s) => {
|
||||
const hooks = SCRIPT_HOOKS.filter(([, f]) => (s[f] || '').trim())
|
||||
const open = openId === s.id
|
||||
return (
|
||||
<div key={s.id} className="adv-script">
|
||||
<div className="adv-script-head">
|
||||
<label className="script-attach" style={{ margin: 0 }}>
|
||||
<input type="checkbox" checked={s.enabled} onChange={() => toggle(s)} />
|
||||
<span>{s.name}</span>
|
||||
</label>
|
||||
<div style={{ display: 'flex', gap: 12, flexShrink: 0 }}>
|
||||
{s.out_of_date && (
|
||||
<button
|
||||
className="linklike"
|
||||
title="Replace this adventure's copy with the latest from your script library"
|
||||
disabled={syncingId === s.id}
|
||||
onClick={() => sync(s)}
|
||||
>
|
||||
{syncingId === s.id ? 'Syncing…' : '⟳ Sync from library'}
|
||||
</button>
|
||||
)}
|
||||
<button className="linklike" onClick={() => setOpenId(open ? null : s.id)}>
|
||||
{open ? 'Hide code' : 'View code'}
|
||||
</button>
|
||||
<button className="linklike" onClick={() => download(s)}>Download</button>
|
||||
</div>
|
||||
</div>
|
||||
{s.description && <div className="dim adv-script-desc">{s.description}</div>}
|
||||
{open && (
|
||||
<div className="adv-script-code">
|
||||
{hooks.length === 0 ? (
|
||||
<div className="empty">This script has no code.</div>
|
||||
) : (
|
||||
hooks.map(([label, f]) => (
|
||||
<div key={f}>
|
||||
<div className="adv-script-hook">{label}</div>
|
||||
<pre>{s[f]}</pre>
|
||||
</div>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export { ScriptsPanel }
|
||||
@@ -1,5 +1,5 @@
|
||||
// What a turn did, rendered five ways: world-state chips, the world-state
|
||||
// report, the script report, the cache report, and the token breakdown.
|
||||
// What a turn did, rendered four ways: world-state chips, the world-state
|
||||
// report, the cache report, and the token breakdown.
|
||||
//
|
||||
// These read a turn's result and render it. None of them fetch.
|
||||
|
||||
@@ -109,43 +109,6 @@ function WorldStateReport({ worldState }) {
|
||||
)
|
||||
}
|
||||
|
||||
function ScriptReport({ script }) {
|
||||
if (!script || (!script.logs?.length && !script.errors?.length && !script.context_changed)) {
|
||||
return null
|
||||
}
|
||||
return (
|
||||
<div className="script-report">
|
||||
<div className="ctx-header" style={{ padding: '4px 0 2px' }}><span>Scripts</span></div>
|
||||
{script.errors?.map((e, i) => <div key={i} className="script-error">⚠ {e}</div>)}
|
||||
{script.logs?.length > 0 && (
|
||||
<pre className="script-logs">{script.logs.join('\n')}</pre>
|
||||
)}
|
||||
{script.context_changed && (
|
||||
<>
|
||||
<div className="ctx-section" style={{ borderLeftColor: sectionColor('script_context') }}>
|
||||
<div className="ctx-header" style={{ color: sectionColor('script_context') }}>
|
||||
<span>Context before script</span>
|
||||
</div>
|
||||
<pre>{script.context_before}</pre>
|
||||
</div>
|
||||
<div className="ctx-section" style={{ borderLeftColor: sectionColor('script_context') }}>
|
||||
<div className="ctx-header" style={{ color: sectionColor('script_context') }}>
|
||||
<span>Context after script (sent to AI)</span>
|
||||
</div>
|
||||
<pre>{script.context_after}</pre>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// What the endpoint charged for the turn, and how much of the prompt it read
|
||||
// back out of its cache instead of billing in full. Only shown on a past turn:
|
||||
// the "next turn" view has not been sent anywhere yet, so it has no usage. A
|
||||
// cached read costs a tenth of a fresh one, which is the whole reason the
|
||||
// prompt is laid out static-first — so this is the number that says whether
|
||||
// that layout is working.
|
||||
function CacheReport({ usage }) {
|
||||
if (!usage) return null
|
||||
const prompt = usage.prompt_tokens || 0
|
||||
@@ -244,4 +207,4 @@ function TokenBreakdown({ sections, tokens, used, onJump }) {
|
||||
)
|
||||
}
|
||||
|
||||
export { StateChangeChips, WorldStateReport, ScriptReport, CacheReport, TokenBreakdown }
|
||||
export { StateChangeChips, WorldStateReport, CacheReport, TokenBreakdown }
|
||||
|
||||
@@ -10,7 +10,6 @@ export default function ScenarioEditor() {
|
||||
const { id } = useParams()
|
||||
const navigate = useNavigate()
|
||||
const [scenario, setScenario] = useState(null)
|
||||
const [allScripts, setAllScripts] = useState([])
|
||||
const [status, setStatus] = useState('')
|
||||
// Raw text buffer for the stat_schema JSON editor + a live parse error.
|
||||
const [schemaText, setSchemaText] = useState('')
|
||||
@@ -27,7 +26,6 @@ export default function ScenarioEditor() {
|
||||
setSchemaText(s.stat_schema ? JSON.stringify(s.stat_schema, null, 2) : '')
|
||||
setParsedSchema(s.stat_schema || null)
|
||||
}).catch(() => navigate('/scenarios'))
|
||||
api.listScripts().then(setAllScripts).catch(() => {})
|
||||
}, [id, navigate])
|
||||
|
||||
const setField = (field, value) => {
|
||||
@@ -149,15 +147,6 @@ export default function ScenarioEditor() {
|
||||
}
|
||||
}
|
||||
|
||||
const toggleScript = async (scriptId) => {
|
||||
const current = scenario.scripts.map((s) => s.id)
|
||||
const next = current.includes(scriptId)
|
||||
? current.filter((sid) => sid !== scriptId)
|
||||
: [...current, scriptId]
|
||||
const updated = await api.updateScenario(id, { script_ids: next })
|
||||
setScenario(updated)
|
||||
}
|
||||
|
||||
const exportScenario = async () => {
|
||||
downloadJSON(await api.exportScenario(id), `${scenario.title.replace(/\W+/g, '-')}.json`)
|
||||
}
|
||||
@@ -297,26 +286,6 @@ export default function ScenarioEditor() {
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="page-header" style={{ marginTop: 28 }}>
|
||||
<h2 style={{ margin: 0, fontFamily: 'Georgia, serif', fontSize: '1.2rem' }}>Attached Scripts</h2>
|
||||
</div>
|
||||
{allScripts.length === 0 ? (
|
||||
<div className="empty" style={{ padding: '20px 0' }}>
|
||||
No scripts in your library. Create some on the Scripts page.
|
||||
</div>
|
||||
) : (
|
||||
allScripts.map((s) => (
|
||||
<label key={s.id} className="script-attach">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={scenario.scripts.some((att) => att.id === s.id)}
|
||||
onChange={() => toggleScript(s.id)}
|
||||
/>
|
||||
<span>{s.name}</span>
|
||||
<span className="dim">{s.description}</span>
|
||||
</label>
|
||||
))
|
||||
)}
|
||||
</fieldset>
|
||||
</div>
|
||||
)
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { useNavigate, useParams } from 'react-router-dom'
|
||||
import CodeMirror from '@uiw/react-codemirror'
|
||||
import { javascript } from '@codemirror/lang-javascript'
|
||||
import { oneDark } from '@codemirror/theme-one-dark'
|
||||
import { api } from '../api'
|
||||
import { Field, downloadJSON } from '../components'
|
||||
import { useDebouncedSave } from '../hooks/useDebouncedSave'
|
||||
|
||||
const SLOTS = [
|
||||
{ key: 'library_js', label: 'Library', hint: 'Shared code prepended to all three hooks.' },
|
||||
{ key: 'input_js', label: 'Input', hint: "onInput — modifies the player's input before context construction." },
|
||||
{ key: 'context_js', label: 'Context', hint: 'onModelContext — modifies the assembled text sent to the model.' },
|
||||
{ key: 'output_js', label: 'Output', hint: 'onOutput — modifies the model output before it is shown.' },
|
||||
]
|
||||
|
||||
export default function ScriptEditor() {
|
||||
const { id } = useParams()
|
||||
const navigate = useNavigate()
|
||||
const [script, setScript] = useState(null)
|
||||
const [slot, setSlot] = useState('input_js')
|
||||
const [status, setStatus] = useState('')
|
||||
const debounceSave = useDebouncedSave()
|
||||
|
||||
// Test-run state
|
||||
const [testHook, setTestHook] = useState('input')
|
||||
const [testText, setTestText] = useState('> You look around.')
|
||||
const [testState, setTestState] = useState('{}')
|
||||
const [testResult, setTestResult] = useState(null)
|
||||
|
||||
useEffect(() => {
|
||||
api.getScript(id).then(setScript).catch(() => navigate('/scripts'))
|
||||
}, [id, navigate])
|
||||
|
||||
const setField = (field, value) => {
|
||||
setScript((prev) => ({ ...prev, [field]: value }))
|
||||
debounceSave(field, async () => {
|
||||
await api.updateScript(id, { [field]: value })
|
||||
setStatus('Saved')
|
||||
setTimeout(() => setStatus(''), 1500)
|
||||
})
|
||||
}
|
||||
|
||||
const runTest = async () => {
|
||||
let state
|
||||
try {
|
||||
state = JSON.parse(testState || '{}')
|
||||
} catch {
|
||||
setTestResult({ error: 'Test state is not valid JSON' })
|
||||
return
|
||||
}
|
||||
try {
|
||||
setTestResult(await api.testScript(id, { hook: testHook, text: testText, state }))
|
||||
} catch (err) {
|
||||
setTestResult({ error: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
const exportScript = async () => {
|
||||
downloadJSON(await api.exportScript(id), `${script.name.replace(/\W+/g, '-')}.json`)
|
||||
}
|
||||
|
||||
const deleteScript = async () => {
|
||||
if (!confirm('Delete this script permanently?')) return
|
||||
await api.deleteScript(id)
|
||||
navigate('/scripts')
|
||||
}
|
||||
|
||||
if (!script) return null
|
||||
|
||||
const activeSlot = SLOTS.find((s) => s.key === slot)
|
||||
|
||||
return (
|
||||
<div className="page">
|
||||
<div className="page-header">
|
||||
<h1>Edit Script</h1>
|
||||
<div style={{ display: 'flex', gap: 10, alignItems: 'center' }}>
|
||||
<span style={{ color: 'var(--text-dim)', fontSize: '0.85rem' }}>{status}</span>
|
||||
<button onClick={exportScript}>Export</button>
|
||||
<button className="danger" onClick={deleteScript}>Delete</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<Field label="Name" value={script.name} onChange={(v) => setField('name', v)} />
|
||||
<Field label="Description" value={script.description} onChange={(v) => setField('description', v)}
|
||||
textarea rows={2} placeholder="What this script does." />
|
||||
|
||||
<div className="slot-tabs">
|
||||
{SLOTS.map((s) => (
|
||||
<button key={s.key} className={slot === s.key ? 'active' : ''} onClick={() => setSlot(s.key)}>
|
||||
{s.label}{script[s.key].trim() ? ' •' : ''}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<div className="slot-hint">{activeSlot.hint}</div>
|
||||
<CodeMirror
|
||||
value={script[slot]}
|
||||
height="360px"
|
||||
theme={oneDark}
|
||||
extensions={[javascript()]}
|
||||
onChange={(value) => setField(slot, value)}
|
||||
/>
|
||||
|
||||
<div className="page-header" style={{ marginTop: 26 }}>
|
||||
<h2 style={{ margin: 0, fontFamily: 'Georgia, serif', fontSize: '1.2rem' }}>Test Run</h2>
|
||||
</div>
|
||||
<div className="test-run">
|
||||
<div className="row">
|
||||
<label className="field" style={{ flex: '0 0 140px' }}>
|
||||
<span className="label">Hook</span>
|
||||
<select value={testHook} onChange={(e) => setTestHook(e.target.value)}>
|
||||
<option value="input">Input</option>
|
||||
<option value="context">Context</option>
|
||||
<option value="output">Output</option>
|
||||
</select>
|
||||
</label>
|
||||
<label className="field" style={{ flex: 1 }}>
|
||||
<span className="label">Sample text</span>
|
||||
<input type="text" value={testText} onChange={(e) => setTestText(e.target.value)} />
|
||||
</label>
|
||||
</div>
|
||||
<label className="field">
|
||||
<span className="label">State (JSON)</span>
|
||||
<input type="text" value={testState} onChange={(e) => setTestState(e.target.value)} />
|
||||
</label>
|
||||
<button className="primary" onClick={runTest}>Run</button>
|
||||
{testResult && (
|
||||
<div className="test-result">
|
||||
{testResult.error ? (
|
||||
<div className="test-error">{testResult.error}</div>
|
||||
) : (
|
||||
<>
|
||||
<div><span className="label">text</span> <pre>{testResult.text}</pre></div>
|
||||
{testResult.stop && <div><span className="label">stop</span> true — the AI call would be skipped</div>}
|
||||
<div><span className="label">state</span> <pre>{JSON.stringify(testResult.state)}</pre></div>
|
||||
{testResult.storyCards.length > 0 && (
|
||||
<div><span className="label">storyCards</span> <pre>{JSON.stringify(testResult.storyCards, null, 1)}</pre></div>
|
||||
)}
|
||||
{testResult.logs.length > 0 && (
|
||||
<div><span className="label">logs</span> <pre>{testResult.logs.join('\n')}</pre></div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { useNavigate } from 'react-router-dom'
|
||||
import { api } from '../api'
|
||||
import { pickJSONFile, useToast } from '../components'
|
||||
|
||||
export default function Scripts() {
|
||||
const [scripts, setScripts] = useState(null)
|
||||
const navigate = useNavigate()
|
||||
const toast = useToast()
|
||||
|
||||
useEffect(() => {
|
||||
api.listScripts().then(setScripts).catch(() => setScripts([]))
|
||||
}, [])
|
||||
|
||||
const createScript = async () => {
|
||||
const script = await api.createScript({
|
||||
name: 'New Script',
|
||||
input_js: '// onInput — modify the player\'s input\nconst modifier = (text) => {\n return { text }\n}\nmodifier(text)\n',
|
||||
})
|
||||
navigate(`/scripts/${script.id}`)
|
||||
}
|
||||
|
||||
const importScript = async () => {
|
||||
try {
|
||||
const bundle = await pickJSONFile()
|
||||
const script = await api.importScript(bundle)
|
||||
navigate(`/scripts/${script.id}`)
|
||||
} catch (err) {
|
||||
toast(err.message, 'error')
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="page">
|
||||
<div className="page-header">
|
||||
<h1>Scripts</h1>
|
||||
<div style={{ display: 'flex', gap: 10 }}>
|
||||
<button onClick={importScript}>Import</button>
|
||||
<button className="primary" onClick={createScript}>+ New Script</button>
|
||||
</div>
|
||||
</div>
|
||||
{scripts === null ? null : scripts.length === 0 ? (
|
||||
<div className="empty">
|
||||
No scripts yet. Scripts are AI Dungeon-compatible JavaScript modifiers
|
||||
(onInput / onModelContext / onOutput) you can attach to scenarios.
|
||||
</div>
|
||||
) : (
|
||||
<div className="card-grid">
|
||||
{scripts.map((s) => (
|
||||
<div key={s.id} className="card" onClick={() => navigate(`/scripts/${s.id}`)}>
|
||||
<h3>{s.name}</h3>
|
||||
<p>{s.description || 'No description'}</p>
|
||||
<div className="meta">
|
||||
{['library', 'input', 'context', 'output']
|
||||
.filter((slot) => s[`${slot}_js`].trim())
|
||||
.join(' · ') || 'empty'}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,5 +1,4 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { useOutletContext } from 'react-router-dom'
|
||||
import { api } from '../api'
|
||||
|
||||
function DebugLog() {
|
||||
@@ -46,11 +45,7 @@ function DebugLog() {
|
||||
}
|
||||
|
||||
export default function Settings() {
|
||||
const { me, setMe } = useOutletContext() ?? {}
|
||||
const [settings, setSettings] = useState(null)
|
||||
// The API key is write-only: the server only reports has_api_key, and this
|
||||
// holds whatever new key the user has typed (empty = leave unchanged).
|
||||
const [apiKey, setApiKey] = useState('')
|
||||
const [testResult, setTestResult] = useState(null)
|
||||
const [saved, setSaved] = useState('')
|
||||
|
||||
@@ -60,17 +55,11 @@ export default function Settings() {
|
||||
|
||||
const setField = (field, value) => setSettings({ ...settings, [field]: value })
|
||||
|
||||
const buildPayload = () => {
|
||||
const { has_api_key: _hasKey, ...payload } = settings
|
||||
if (apiKey.trim()) payload.api_key = apiKey.trim()
|
||||
return payload
|
||||
}
|
||||
const buildPayload = () => ({ ...settings })
|
||||
|
||||
const afterSave = async () => {
|
||||
const fresh = await api.getSettings()
|
||||
setSettings(fresh)
|
||||
setApiKey('')
|
||||
if (me?.multi_user) api.getMe().then(setMe).catch(() => {}) // demo banner state
|
||||
return fresh
|
||||
}
|
||||
|
||||
@@ -85,17 +74,6 @@ export default function Settings() {
|
||||
setTimeout(() => setSaved(''), 4000)
|
||||
}
|
||||
|
||||
const clearKey = async () => {
|
||||
try {
|
||||
await api.updateSettings({ api_key: '' })
|
||||
await afterSave()
|
||||
setSaved('API key removed')
|
||||
} catch (err) {
|
||||
setSaved(`Failed: ${err.message}`)
|
||||
}
|
||||
setTimeout(() => setSaved(''), 4000)
|
||||
}
|
||||
|
||||
const test = async () => {
|
||||
setTestResult({ pending: true })
|
||||
try {
|
||||
@@ -108,7 +86,6 @@ export default function Settings() {
|
||||
}
|
||||
|
||||
if (!settings) return null
|
||||
const demo = me?.demo
|
||||
|
||||
return (
|
||||
<div className="page" style={{ maxWidth: 640 }}>
|
||||
@@ -117,34 +94,21 @@ export default function Settings() {
|
||||
<span style={{ color: 'var(--text-dim)', fontSize: '0.85rem' }}>{saved}</span>
|
||||
</div>
|
||||
|
||||
{demo?.using_demo && (
|
||||
<div className="demo-banner">
|
||||
<strong>Using the shared demo key</strong> — {demo.turns_left} of {demo.turns_per_day} free
|
||||
turns left today (model: {demo.model}). Add your own API key below for unlimited play,
|
||||
your choice of models, and the memory bank.
|
||||
</div>
|
||||
)}
|
||||
|
||||
<label className="field">
|
||||
<span className="label">Endpoint URL (OpenAI-compatible)</span>
|
||||
<span className="label">Ollama endpoint</span>
|
||||
<input type="text" value={settings.endpoint_url}
|
||||
placeholder="http://localhost:11434/v1"
|
||||
placeholder="http://127.0.0.1:11434/v1"
|
||||
onChange={(e) => setField('endpoint_url', e.target.value)} />
|
||||
</label>
|
||||
<label className="field">
|
||||
<span className="label">API Key {settings.has_api_key ? '(saved — enter a new one to replace it)' : ''}</span>
|
||||
<div style={{ display: 'flex', gap: 8 }}>
|
||||
<input type="password" value={apiKey} style={{ flex: 1 }}
|
||||
placeholder={settings.has_api_key ? '••••••••••••' : 'Leave empty for local endpoints'}
|
||||
onChange={(e) => setApiKey(e.target.value)} />
|
||||
{settings.has_api_key && (
|
||||
<button type="button" onClick={clearKey}>Remove key</button>
|
||||
)}
|
||||
</div>
|
||||
<span className="label" style={{ marginTop: 4 }}>
|
||||
Ollama on this machine, or on a machine on your own network. A public
|
||||
Internet address is refused — your story never leaves your own
|
||||
infrastructure. An https:// endpoint whose certificate comes from your
|
||||
own CA works once that CA is installed on this machine.
|
||||
</span>
|
||||
</label>
|
||||
<label className="field">
|
||||
<span className="label">Model</span>
|
||||
<input type="text" value={settings.model} placeholder="e.g. llama3.1"
|
||||
<input type="text" value={settings.model} placeholder="e.g. qwen2.5:3b-instruct"
|
||||
onChange={(e) => setField('model', e.target.value)} />
|
||||
</label>
|
||||
|
||||
@@ -166,16 +130,13 @@ export default function Settings() {
|
||||
</label>
|
||||
</div>
|
||||
<label className="field">
|
||||
<span className="label">Reasoning budget (tokens)</span>
|
||||
<input type="number" min="-1" value={settings.reasoning_max_tokens}
|
||||
onChange={(e) => setField(
|
||||
'reasoning_max_tokens', Math.max(-1, Number(e.target.value)))} />
|
||||
<span className="label">Model timeout (seconds)</span>
|
||||
<input type="number" min="30" max="3600" value={settings.model_timeout_seconds}
|
||||
onChange={(e) => setField('model_timeout_seconds', Number(e.target.value))} />
|
||||
<span className="label" style={{ marginTop: 4 }}>
|
||||
For reasoning models: separate thinking budget on top of max output tokens,
|
||||
and thinking is shown collapsed above each response. 0 = nothing extra is
|
||||
sent (keep 0 for endpoints without reasoning support, e.g. Ollama).
|
||||
−1 = actively turn reasoning off, for models that think by default
|
||||
(DeepSeek V4 Flash) — saves the thinking tokens rather than just hiding them.
|
||||
How long to wait for a turn. Loading a model for the first time can
|
||||
take minutes on a machine without a GPU, while later turns take
|
||||
seconds. Raise this if turns time out only after Ollama has been idle.
|
||||
</span>
|
||||
</label>
|
||||
|
||||
@@ -232,13 +193,13 @@ export default function Settings() {
|
||||
<div style={{ marginTop: 16, color: testResult.ok ? 'var(--accent)' : 'var(--danger)' }}>
|
||||
{testResult.pending ? 'Testing…'
|
||||
: testResult.ok
|
||||
? `Connected. Models: ${testResult.models?.slice(0, 8).join(', ') || '(none listed)'}`
|
||||
? (testResult.warning
|
||||
|| `Connected. Models: ${testResult.models?.slice(0, 8).join(', ') || '(none listed)'}`)
|
||||
: testResult.detail}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* The provider debug log is a single global buffer — local installs only. */}
|
||||
{me?.multi_user !== true && <DebugLog />}
|
||||
<DebugLog />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Adventure Storyteller — Production Build Milestones
|
||||
|
||||
**Status:** In implementation. M1 complete (2026-09-02); M2 next
|
||||
**Status:** In implementation. M1 and M2 complete (2026-09-02); M3 next
|
||||
**Base:** AI-DnD `d72f7c1bda0f34fccd84afb7a25c34eb01c901de`
|
||||
|
||||
## 1. Purpose
|
||||
@@ -166,6 +166,37 @@ Add:
|
||||
|
||||
The codebase has a narrow single-user/local-only surface and the inherited story foundation still passes its relevant regression suite.
|
||||
|
||||
## Status: COMPLETE
|
||||
|
||||
Accepted 2026-09-02. Evidence: `planning/reports/M2-BASELINE-REPORT.md`
|
||||
(measurements) and `planning/reports/M2-IMPLEMENTATION-REPORT.md` (review);
|
||||
verdict *accept with non-blocking debt, proceed to M3*. Implementation is
|
||||
commit `8c65ae9`, and the three defects the review found are commit `8652fe7`
|
||||
— see the closeout note appended to both reports.
|
||||
|
||||
**Capabilities M2 delivered, which later milestones inherit rather than build:**
|
||||
|
||||
- a single-user product with no accounts, sessions or auth — all four
|
||||
`/api/auth/*` routes are gone, not gated; 52 API routes down to 36,
|
||||
- Ollama as the only inference backend, with no cloud provider code and no API
|
||||
key anywhere in the product,
|
||||
- an **address-based inference endpoint policy**, enforced on save and again
|
||||
before every outbound request, that refuses public addresses even when the
|
||||
database is edited behind the settings API (ADR 011),
|
||||
- M1's trusted-LAN and TLS behaviour carried through the removal intact —
|
||||
verified HTTPS against a real second machine, no bypass option,
|
||||
- SQLite as the only store; Postgres, Neon and the Render deployment path removed,
|
||||
- no campaign scripting: the QuickJS engine and `/api/scripts` are gone,
|
||||
- a materially simpler runtime — 10 environment variables down to two, 6 Python
|
||||
and 21 npm packages removed, and a 933 kB bundle down to 395 kB.
|
||||
|
||||
**Debt carried forward, none of it blocking M3:** `Settings.model` still
|
||||
defaults to `""` with nothing prompting for it (M8); inert legacy tables and
|
||||
columns await a cleanup migration once the schema settles, after M3/M5; there
|
||||
are still no frontend tests (M8); `docs/*.html`, upstream's project site and not
|
||||
served by the app, still links Google Fonts. Full table in the implementation
|
||||
report §P.
|
||||
|
||||
---
|
||||
|
||||
# M3 — Production Non-Destructive History, Redo, and Active-Head Export
|
||||
@@ -310,6 +341,34 @@ Remove or demote:
|
||||
|
||||
Accepted story state is genre-neutral, auditable, reconstructable, and no longer depends on ambiguous relative deltas.
|
||||
|
||||
## Note from M2 — eight rollback tests use the world-state engine as instrumentation
|
||||
|
||||
M2 removed QuickJS. Eight existing rollback/history tests had used a JavaScript
|
||||
counter as deterministic instrumentation — a value they could change on a turn
|
||||
and then assert had been rolled back — and they now use the inherited
|
||||
RPG/world-state delta machinery for the same purpose.
|
||||
|
||||
Read those tests correctly before touching them:
|
||||
|
||||
- **what they exercise** is rollback and state reconstruction across undo,
|
||||
retry, takes and divergence;
|
||||
- **their use of the world-state engine is test instrumentation, not an
|
||||
endorsement** of RPG-shaped state as the target architecture. Nothing about
|
||||
them argues against the typed-event model this milestone installs;
|
||||
- **when M5 replaces or generalizes the world-state protocol, the
|
||||
instrumentation must move with it** to the new narrative-state mechanism. In
|
||||
practice that is one schema entry and one helper in `backend/tests/fakes.py`;
|
||||
- **preserve or rework these tests; do not delete them** merely because their
|
||||
current instrumentation is RPG-shaped. The behaviour they pin is exactly the
|
||||
behaviour M5 is most likely to break.
|
||||
|
||||
`test_state_revert` and `test_delete_state` additionally assert *destructive*
|
||||
undo semantics and are expected to be rewritten by M3; that is separate from
|
||||
this instrumentation point, and they should likewise be rewritten rather than
|
||||
dropped.
|
||||
|
||||
Evidence: `planning/reports/M2-IMPLEMENTATION-REPORT.md` §K.2, §Q.
|
||||
|
||||
---
|
||||
|
||||
# M6 — Branch-Safe Context, Summaries, and Long-Term Story Memory
|
||||
@@ -348,6 +407,28 @@ Align inherited AI-DnD memory/context behavior with the final authority and hist
|
||||
|
||||
Long-running story context is lineage-safe, authority-aware, local, inspectable, and bounded.
|
||||
|
||||
## Note from M2 — background memory failure must be observable
|
||||
|
||||
M2 shipped with the memory bank entirely dead, and the full suite stayed green.
|
||||
Summaries and embeddings raised `AttributeError` inside a fire-and-forget task:
|
||||
no user-visible error, no log a player would read, and no failing test, because
|
||||
every memory test stubs the provider factories out.
|
||||
|
||||
M6 therefore additionally requires:
|
||||
|
||||
- **memory/summarization background failures must be observable** — a
|
||||
fire-and-forget task that dies must leave a record a user or maintainer can
|
||||
actually find, rather than being swallowed;
|
||||
- **tests must exercise at least one real provider-construction/wiring path**,
|
||||
not only mocked factories, so that a moved or removed setting surfaces as a
|
||||
test failure (`TECHNICAL-DESIGN.md` §18.1);
|
||||
- **derived-memory failure must not corrupt accepted story state.** Already in
|
||||
the scope list above; M2's evidence is why it stays there. A dead memory bank
|
||||
degraded the storyteller quietly and left the transcript correct, which is the
|
||||
right failure direction — but it must also be a *visible* one.
|
||||
|
||||
Evidence: `planning/reports/M2-IMPLEMENTATION-REPORT.md` §A.1, §9.1.
|
||||
|
||||
---
|
||||
|
||||
# M7 — First-Class Imported Knowledge Library
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# ADR 011 — Local Inference Endpoint Policy
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-09-02
|
||||
**Implemented in:** M2, `backend/app/endpoints.py`
|
||||
|
||||
## Decision
|
||||
|
||||
v1 supports Ollama only, and will send a story to an inference endpoint **only**
|
||||
when every address that endpoint resolves to lies inside an explicit allowlist
|
||||
of local networks.
|
||||
|
||||
- The storyteller UI/API remains **loopback-bound by default**. Configuring a
|
||||
LAN inference endpoint does not change where the storyteller listens.
|
||||
- Inference may use **same-host loopback** (the default) or an **explicitly
|
||||
configured trusted-LAN/local-network address**.
|
||||
- **Public Internet addresses are denied.**
|
||||
- The policy is **address-based**, using explicit allowed CIDRs rather than
|
||||
Python's generic `is_private` / `is_reserved` classifications.
|
||||
- **Every** resolved address must be allowed; one address outside the allowlist
|
||||
refuses the endpoint.
|
||||
- The policy is checked **on configuration and again before every request**.
|
||||
- **TLS verification is mandatory** for HTTPS and is never traded against this
|
||||
policy.
|
||||
- Arbitrary cloud / OpenAI-compatible endpoints are **intentionally outside v1**.
|
||||
|
||||
## Context
|
||||
|
||||
The endpoint setting is the most consequential in the application. The
|
||||
storyteller sends the player's prose, the assembled context, the retrieved
|
||||
memories and the embedding inputs to whatever address it names: point it
|
||||
somewhere else and the whole campaign goes there.
|
||||
|
||||
The inherited AI-DnD guard could not be reused, because its rule is the
|
||||
**opposite** of this product's. AI-DnD was a hosted service, so its SSRF guard
|
||||
blocked *private* addresses to stop a user reaching the server's internal
|
||||
network. A local storyteller must do exactly the reverse — permit the private
|
||||
ranges and refuse the public Internet. The guard was removed rather than adapted.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Hostname matching / a denylist of cloud providers.** Rejected as the primary
|
||||
rule: it is trivially talked around by spelling a name differently, by a
|
||||
`CNAME`, or by a private DNS entry pointing at a public host. A denylist of
|
||||
known cloud hostnames is retained, but only to make the *error message*
|
||||
explain why — the address rule already refuses all of them.
|
||||
- **Python's `is_private` / `is_reserved`.** Rejected; see below.
|
||||
- **Loopback-only inference.** Rejected: trusted-LAN inference is accepted
|
||||
production behaviour under ADR 002, and a great many users will run Ollama on
|
||||
the one machine in the house that has a GPU.
|
||||
- **UI-only discouragement.** Rejected: a setting that is merely absent from a
|
||||
dropdown is still reachable by editing the database.
|
||||
|
||||
## Reason for Explicit CIDRs
|
||||
|
||||
Generic address classifications do not answer this project's security question,
|
||||
and get it wrong in both directions for addresses this application actually sees:
|
||||
|
||||
- `is_private` is **true** of the documentation ranges and of `0.0.0.0/8`,
|
||||
neither of which is a user's LAN;
|
||||
- `is_reserved` is **true** of IPv6 loopback, so a rule written around it
|
||||
refuses `http://[::1]:11434/v1` — an ordinary same-host Ollama.
|
||||
|
||||
Naming the networks keeps the policy readable, makes it auditable against this
|
||||
document, and makes anything unnamed refused by default:
|
||||
|
||||
```text
|
||||
127.0.0.0/8 this machine ::1/128 this machine, v6
|
||||
10.0.0.0/8 RFC1918 fc00::/7 unique-local, v6
|
||||
172.16.0.0/12 RFC1918 fe80::/10 link-local, v6
|
||||
192.168.0.0/16 RFC1918
|
||||
169.254.0.0/16 link-local
|
||||
100.64.0.0/10 carrier-grade NAT, which mesh VPNs such as Tailscale use
|
||||
```
|
||||
|
||||
Carrier-grade NAT is included deliberately: it is what a mesh VPN such as
|
||||
Tailscale hands out, and such a network is as user-controlled as a LAN.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Validation runs in two places — `routers/settings.py` on save and on the
|
||||
connection test, and `providers/openai_compatible.py` before the generate,
|
||||
chat and embedding requests. Configuration validation alone is not sufficient,
|
||||
and a database edited behind the settings API must not become a way out.
|
||||
- Because a hostname is resolved at check time, an endpoint given as a literal
|
||||
address behaves most predictably.
|
||||
- No cloud provider can be configured, so no API-key storage is needed. M2
|
||||
removed both.
|
||||
- The error strings are user-facing and say what to do, not what failed
|
||||
internally.
|
||||
|
||||
## Accepted Residual Risks
|
||||
|
||||
Documented rather than mitigated, and **not** M3 work:
|
||||
|
||||
1. **A hostile host on a trusted LAN is inside the permitted boundary.** The
|
||||
policy authorizes an address range, not a machine. If an attacker already
|
||||
controls a device on the user's network and the user points the storyteller
|
||||
at it, the story goes there. The defence is the inference host's own firewall
|
||||
and network policy.
|
||||
2. **A DNS-rebinding interval** exists between the policy's resolution of a
|
||||
hostname and the HTTP client's own connection. The two resolutions are
|
||||
separate, so a name that answers with a LAN address for the check and a public
|
||||
address for the connection is theoretically possible. Using a literal address
|
||||
closes it entirely.
|
||||
|
||||
## Scope
|
||||
|
||||
This ADR covers which inference endpoints the product will talk to. It is **not**
|
||||
a general remote-access design: it says nothing about exposing the storyteller
|
||||
itself beyond loopback, which remains out of scope for v1.
|
||||
|
||||
## References
|
||||
|
||||
- `SECURITY-THREAT-MODEL.md` §10A, §71A item 5, §77
|
||||
- `TECHNICAL-DESIGN.md` §5.1 item 4, §5.2
|
||||
- ADR 002 (Ollama-only, and the TLS consequence), ADR 004 (local-only production)
|
||||
- `planning/reports/M2-IMPLEMENTATION-REPORT.md` §F, §K.1
|
||||
+36
-7
@@ -1,7 +1,7 @@
|
||||
# Adventure Storyteller Planning Package
|
||||
|
||||
**Status:** Phase 0 complete; architecture selected; **Milestone M1 implemented and accepted (2026-09-02)**.
|
||||
**Production coding:** Underway, milestone by milestone. M1 is done; M2 is the next milestone to brief.
|
||||
**Status:** Phase 0 complete; architecture selected; **Milestones M1 and M2 implemented and accepted (2026-09-02)**.
|
||||
**Production coding:** Underway, milestone by milestone. M1 and M2 are done; M3 is the next milestone to brief.
|
||||
|
||||
This package contains the current product requirements, final Phase 0 architecture decisions, detailed subsystem designs, acceptance tests, research evidence, and the production milestone plan for the local-only interactive-story project.
|
||||
|
||||
@@ -163,7 +163,13 @@ Milestone M1 COMPLETE (2026-09-02)
|
||||
fork + offline baseline see planning/reports/M1-*.md
|
||||
|
|
||||
v
|
||||
Milestone M2 NEXT — brief not yet prepared
|
||||
Milestone M2 COMPLETE (2026-09-02)
|
||||
local-only surface + endpoint see planning/reports/M2-*.md
|
||||
policy
|
||||
|
|
||||
v
|
||||
Milestone M3 NEXT — brief not yet prepared
|
||||
non-destructive undo/redo
|
||||
|
|
||||
v
|
||||
Implement and review milestone-by-milestone
|
||||
@@ -173,10 +179,33 @@ Implement and review milestone-by-milestone
|
||||
|
||||
**One milestone at a time. Do not begin a milestone before its brief exists.**
|
||||
|
||||
M1 is complete and accepted; its evidence is in `reports/M1-BASELINE-REPORT.md`
|
||||
and `reports/M1-IMPLEMENTATION-REPORT.md`. **No M2 brief has been prepared.**
|
||||
The current action is to review the post-M1 planning corrections below before
|
||||
writing one.
|
||||
M1 and M2 are complete and accepted; the evidence is in `reports/M1-*.md` and
|
||||
`reports/M2-*.md`. **No M3 brief has been prepared.** The current action is to
|
||||
write one, informed by the post-M2 corrections below and by
|
||||
`reports/M2-IMPLEMENTATION-REPORT.md` §Q, which records that M3's chokepoints
|
||||
were left untouched or simplified by M2 and that the Phase 0B undo/redo spike
|
||||
still applies.
|
||||
|
||||
### Post-M2 corrections applied (2026-09-03)
|
||||
|
||||
M2's review recommended six planning changes and reported rather than applied
|
||||
them. All six are now applied, plus three additions drawn from the same evidence:
|
||||
|
||||
| Document | Correction |
|
||||
| --- | --- |
|
||||
| `SECURITY-THREAT-MODEL.md` | New §10A records the inference endpoint policy **as implemented** — address allowlist, enforced on save and before every request, TLS never traded against it — with both residual limits stated. §71A item 5 marked resolved; §77 notes the required defaults are now met. |
|
||||
| `TECHNICAL-DESIGN.md` §5.1 | Items 3 and 4 marked done; all five hardening items are now resolved. |
|
||||
| `TECHNICAL-DESIGN.md` §5.2 | New: the M1/M2 production architecture recorded as fact — SQLite, Ollama-only, loopback storyteller, trusted-LAN inference accepted, public endpoints refused. |
|
||||
| `TECHNICAL-DESIGN.md` §18.1 | New wiring rule from the M2 regressions: test a real consumer path when removing a setting, and prove a new setting reaches its component. |
|
||||
| `DECISIONS/011-local-inference-endpoint-policy.md` | **New ADR.** Address-based allowlist over hostname matching, deny by default, checked twice, mandatory TLS — with the `ipaddress`-classification finding as the reason the CIDRs are spelled out. |
|
||||
| `BUILD-MILESTONES.md` M2 | Marked COMPLETE with the capabilities it delivered and the debt it carried forward. |
|
||||
| `BUILD-MILESTONES.md` M5 | Note: eight rollback tests now use the world-state engine as *instrumentation*, not as endorsement; move the instrumentation when M5 replaces the protocol, and rework rather than delete those tests. |
|
||||
| `BUILD-MILESTONES.md` M6 | Note: background memory failure must be observable, at least one real provider-construction path must be tested, and derived-memory failure must not corrupt accepted story state. |
|
||||
| `V1-ACCEPTANCE-TESTS.md` H10 | Strengthened: a wildcard origin must be rejected at startup, and an unknown `/api/...` path must 404 rather than returning the SPA with HTTP 200. |
|
||||
| `V1-ACCEPTANCE-TESTS.md` H12 | **New.** Inference endpoint enforcement, including the defence-in-depth case: a public endpoint written into the database behind the settings API must still be refused at request time. |
|
||||
|
||||
`SPECIFICATION.md` was deliberately **not** changed. M2 altered no product
|
||||
requirement; it removed capability the specification never asked for.
|
||||
|
||||
### Post-M1 corrections applied (2026-09-02)
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Adventure Storyteller — Security Threat Model
|
||||
|
||||
**Status:** v1.0 — local-only hardening requirements informed by Phase 0B
|
||||
**Status:** v1.1 — local-only hardening requirements informed by Phase 0B, with the
|
||||
inference endpoint policy recorded as implemented in M2 (§10A, §71A item 5, §77)
|
||||
**Purpose:** Define the security and privacy boundaries for a local-only interactive storytelling application.
|
||||
|
||||
## 1. Security Objective
|
||||
@@ -231,6 +232,104 @@ http://inferencebox.local:11434
|
||||
|
||||
The LAN hostname/address must be an intentional user configuration. Do not infer that every non-loopback endpoint is trusted merely because it resolves.
|
||||
|
||||
## 10A. Inference Endpoint Policy As Implemented (M2)
|
||||
|
||||
**Status:** implemented in M2, `backend/app/endpoints.py`. §10 above states the
|
||||
requirement; this section records the rule that now enforces it, and what it
|
||||
does not cover. ADR 011 records the decision.
|
||||
|
||||
This section supersedes the assumption in §71A item 5 that the inherited
|
||||
network guard was the starting point. It was not reusable: AI-DnD's guard was
|
||||
an SSRF guard for a *hosted* deployment, and its rule is the **opposite** of
|
||||
this product's. A hosted server blocks private addresses to stop a user
|
||||
reaching its internal network; a local storyteller must permit exactly those
|
||||
addresses and refuse the public Internet. The inherited guard was removed
|
||||
rather than adapted.
|
||||
|
||||
### The path
|
||||
|
||||
```text
|
||||
Browser -> storyteller on loopback
|
||||
Storyteller -> SQLite / local files
|
||||
Storyteller -> one approved Ollama endpoint
|
||||
```
|
||||
|
||||
The Ollama endpoint is either same-host loopback (the default) or an explicitly
|
||||
configured trusted-LAN/local-network endpoint. Configuring a LAN inference host
|
||||
does not change where the storyteller itself listens: the UI/API remains
|
||||
loopback-bound by default, and the endpoint setting has no influence on the
|
||||
bind address.
|
||||
|
||||
### The rule
|
||||
|
||||
Endpoints are validated **by address against an explicit allowlist of
|
||||
networks**, not by hostname matching:
|
||||
|
||||
```text
|
||||
127.0.0.0/8 this machine ::1/128 this machine, v6
|
||||
10.0.0.0/8 RFC1918 fc00::/7 unique-local, v6
|
||||
172.16.0.0/12 RFC1918 fe80::/10 link-local, v6
|
||||
192.168.0.0/16 RFC1918
|
||||
169.254.0.0/16 link-local
|
||||
100.64.0.0/10 carrier-grade NAT, which mesh VPNs such as Tailscale use
|
||||
```
|
||||
|
||||
- **Every** address the hostname resolves to must fall inside one of these
|
||||
networks. One address outside is enough to refuse the endpoint, so a name
|
||||
resolving to both a private and a public address does not squeak through.
|
||||
- Public Internet addresses are **refused**, not merely discouraged or hidden
|
||||
from a dropdown.
|
||||
- The networks are spelled out rather than derived from Python's `is_private` /
|
||||
`is_reserved` classifications, which do not answer this question: `is_private`
|
||||
is true of the documentation ranges and of `0.0.0.0/8`, and `is_reserved` is
|
||||
true of IPv6 loopback — so a rule built on it would refuse `http://[::1]:11434/v1`,
|
||||
an ordinary same-host Ollama. See ADR 011.
|
||||
- A short list of known cloud inference hostnames is checked first. The address
|
||||
rule already refuses all of them; the list exists only so the error explains
|
||||
*why* rather than leaving the user to suspect broken DNS.
|
||||
|
||||
### Where it is enforced
|
||||
|
||||
Twice, deliberately — configuration validation alone is not sufficient:
|
||||
|
||||
1. **when settings are saved** (`routers/settings.py`), so the user gets an
|
||||
immediate, specific error, and on the connection-test path;
|
||||
2. **before every outbound request** (`providers/openai_compatible.py`, on the
|
||||
generate, chat and embedding paths), because a name that resolved to a LAN
|
||||
address this morning can resolve elsewhere this afternoon — and because a
|
||||
database edited behind the settings API must not become a way out.
|
||||
|
||||
M2 demonstrated the second at runtime: a cloud endpoint written straight into
|
||||
SQLite with `sqlite3`, bypassing the API entirely, was still refused at the wire.
|
||||
|
||||
### TLS
|
||||
|
||||
HTTPS to a trusted-LAN Ollama with a privately issued certificate is supported.
|
||||
TLS verification is **never traded against** the address policy:
|
||||
|
||||
- certificate and hostname verification remain fully enabled,
|
||||
- trust is the machine's own CA store unioned with certifi (`tlstrust.py`, ADR 002),
|
||||
- there is **no `verify=False`, no bypass flag, and no "insecure" option** —
|
||||
however private the address.
|
||||
|
||||
### Residual limits
|
||||
|
||||
Stated plainly, because the policy does not cover them:
|
||||
|
||||
1. **A hostile host on a network the user treats as trusted is inside the
|
||||
permitted boundary.** The policy authorizes an address range, not a machine.
|
||||
If an attacker already controls a device on the user's LAN and the user
|
||||
points the storyteller at it, the story goes there. Defending that is the
|
||||
inference host's own firewall and network policy (§7), not this rule.
|
||||
2. **A DNS-rebinding interval exists** between the policy resolving a hostname
|
||||
and the HTTP client making its own connection. The two resolutions are
|
||||
separate, so a name that answers with a LAN address for the check and a
|
||||
public one for the connection is theoretically possible. Using a literal
|
||||
address rather than a hostname closes it entirely.
|
||||
|
||||
Both are **accepted residual risks for v1**, documented rather than mitigated.
|
||||
Neither is M3 work.
|
||||
|
||||
## 11. Imported Files Must Be Data Only
|
||||
|
||||
Imported files must never be treated as executable application extensions.
|
||||
@@ -1108,9 +1207,10 @@ Phase 0B runtime validation found specific inherited behaviors that production m
|
||||
- executable campaign scripting is outside the v1 trust boundary.
|
||||
- Remove/disable the engine and replace any test-only instrumentation that depended on it.
|
||||
|
||||
5. **Endpoint policy mismatch**
|
||||
- inherited network guarding is aimed at hosted deployment behavior, not at preventing accidental story-data exfiltration.
|
||||
5. **Endpoint policy mismatch** — **resolved in M2.**
|
||||
- inherited network guarding is aimed at hosted deployment behavior, not at preventing accidental story-data exfiltration. Its rule was in fact the *opposite* of this product's, so it was removed rather than adapted.
|
||||
- Production should default to loopback Ollama, explicitly support a configured trusted-LAN Ollama host, and reject/avoid arbitrary public Internet inference endpoints.
|
||||
- **Done.** Implemented as an address-based allowlist enforced on save and again before every request; see §10A and ADR 011.
|
||||
|
||||
6. **Postgres is removable**
|
||||
- Phase 0B found no architectural blocker to dropping Postgres support; SQLite remains the v1 store.
|
||||
@@ -1222,7 +1322,8 @@ local lexical/semantic retrieval
|
||||
optional explicitly configured local media services in the future
|
||||
```
|
||||
|
||||
Required production defaults:
|
||||
Required production defaults. **As of M2 every item below is implemented**;
|
||||
the endpoint rule that enforces the third and fourth is recorded in §10A:
|
||||
|
||||
- storyteller binds loopback by default,
|
||||
- Ollama endpoint is same-host loopback by default,
|
||||
|
||||
@@ -201,8 +201,8 @@ Production defaults must not require:
|
||||
|
||||
### 5.1 Known AI-DnD hardening work
|
||||
|
||||
Phase 0B identified concrete inherited violations, and M1 added a fifth. Items
|
||||
1, 2 and 5 are **resolved**; items 3 and 4 remain open and belong to M2.
|
||||
Phase 0B identified concrete inherited violations, and M1 added a fifth. **All
|
||||
five are now resolved** — items 1, 2 and 5 in M1, items 3 and 4 in M2.
|
||||
|
||||
1. ~~`tiktoken` attempts to download the `cl100k_base` encoding on first use.~~
|
||||
**Done in M1.** The encoding table is vendored in the tree and loaded
|
||||
@@ -211,21 +211,48 @@ Phase 0B identified concrete inherited violations, and M1 added a fifth. Items
|
||||
2. ~~the SPA requests Google Fonts at runtime.~~
|
||||
**Done in M1.** All three families are self-hosted, and the CSP names no
|
||||
remote origin at all.
|
||||
3. hosted/multi-user/auth/demo/analytics/Postgres/cloud-provider/QuickJS paths are unnecessary.
|
||||
3. ~~hosted/multi-user/auth/demo/analytics/Postgres/cloud-provider/QuickJS paths are unnecessary.~~
|
||||
- remove them rather than merely hide them where practical.
|
||||
- **Open — M2.** M1 removed nothing, so this surface is unchanged from the
|
||||
fork point.
|
||||
4. endpoint validation must reflect this product's threat model.
|
||||
- **Done in M2, in full.** Removed rather than hidden: 52 API routes fell to
|
||||
36, and `/api/auth`, `/api/analytics` and `/api/scripts` are gone entirely
|
||||
rather than gated. See §5.2.
|
||||
4. ~~endpoint validation must reflect this product's threat model.~~
|
||||
- same-host loopback Ollama is the default; an explicitly configured trusted-LAN Ollama endpoint is supported; arbitrary public/Internet model endpoints must be rejected or kept outside normal v1 configuration.
|
||||
- inference endpoint configuration must not change the storyteller's own loopback bind behavior.
|
||||
- **Open — M2.** The trusted-LAN path itself works as of M1; what remains is
|
||||
deciding and enforcing which endpoints normal v1 configuration may name.
|
||||
- **Done in M2.** `backend/app/endpoints.py` applies an address-based
|
||||
allowlist on save and again before every outbound request. Endpoint
|
||||
configuration has no influence on the storyteller's own bind address.
|
||||
ADR 011; `SECURITY-THREAT-MODEL.md` §10A.
|
||||
5. ~~outbound TLS verified only against a bundled public-CA list, so a LAN host
|
||||
with a privately issued certificate was refused.~~
|
||||
**Found and fixed in M1.** Not visible to Phase 0B: every run up to that
|
||||
point used plain HTTP over loopback, where certificate verification never
|
||||
happens. See *Transport to a trusted-LAN endpoint* above.
|
||||
|
||||
### 5.2 Production architecture as established by M1 and M2
|
||||
|
||||
The architecture below is no longer a selection; it is what the code does. It is
|
||||
recorded here so later milestones inherit facts rather than intentions.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| Production base | AI-DnD, forked at `d72f7c1` (§2, ADR 009) |
|
||||
| Persistence | SQLite. Postgres, Neon and the Render deployment path are removed |
|
||||
| Inference | Ollama only. No cloud provider code, no API key, no key UI |
|
||||
| Storyteller bind | loopback by default, in every run path including the published Docker port |
|
||||
| Ollama endpoint | same-host loopback by default; an explicitly configured trusted-LAN endpoint is equally supported |
|
||||
| Public endpoints | refused by address, on save and before every request |
|
||||
| Trusted-LAN HTTPS | supported, with full certificate and hostname verification against the machine's CA store; no bypass exists |
|
||||
| Runtime assets | self-contained. Tokenizer table and fonts are vendored; the CSP names no remote origin |
|
||||
|
||||
Removed in M2 rather than hidden: hosted accounts and auth, guest/demo
|
||||
behaviour, hosted analytics, cloud inference providers, API-key storage and its
|
||||
UI, Postgres/Neon/Render support, and QuickJS campaign scripting.
|
||||
|
||||
**A trusted-LAN Ollama endpoint is accepted production behaviour**, not a
|
||||
development convenience. Any statement that the only valid endpoint is literally
|
||||
`127.0.0.1` is stale and should be read against §5 and §10A of the threat model.
|
||||
|
||||
## 6. Browser UI Boundary
|
||||
|
||||
The browser remains a presentation/control layer, not the owner of story authority.
|
||||
@@ -606,6 +633,24 @@ Required categories:
|
||||
|
||||
Acceptance-test IDs in `V1-ACCEPTANCE-TESTS.md` are the black-box release contract.
|
||||
|
||||
### 18.1 Wiring rule, from the M2 regressions
|
||||
|
||||
M2 shipped two defects that a 604-test green suite did not see: a removed
|
||||
`Settings` attribute left two provider factories raising `AttributeError` inside
|
||||
a background task, and a newly added timeout setting was stored, validated,
|
||||
exposed and rendered without ever being passed to the provider that needed it.
|
||||
Both were invisible because the tests at that boundary were mocks.
|
||||
|
||||
> **When removing a setting, attribute or dependency, test at least one real
|
||||
> consumer construction path. When adding a setting, test that the configured
|
||||
> value reaches the component that uses it. A green suite built entirely around
|
||||
> mocks at that boundary is insufficient evidence.**
|
||||
|
||||
The corollary is where to look: subtractive changes and plumbing changes fail in
|
||||
background and fire-and-forget paths, which are exactly the paths that report
|
||||
nothing when they break.
|
||||
|
||||
|
||||
## 19. Removal / Migration Strategy From Upstream
|
||||
|
||||
Production migration should be incremental and test-gated rather than a broad rewrite.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user