M1: make the first story turn work with no Internet

Phase 0B ran the upstream application on a network with no route out and
the first turn died in tiktoken, which downloads its BPE table the first
time anything counts a token. The browser separately fetched three font
families from Google on every page load. Neither is visible on a machine
that has been online once, which is why both now have tests.

The tokenizer table is vendored at
backend/app/context/vendor/cl100k_base.tiktoken and
backend/app/context/encoding.py builds the encoding from it directly,
verifying its SHA-256 against the digest tiktoken itself pins for that
URL. No code path in the tokenizer can reach the network any more —
not a warm cache, not an environment variable a deployment could forget.
The encoding was checked token for token against tiktoken's own.

The three font families are self-hosted as variable fonts under
frontend/public/fonts/ (343 KiB, Latin and Latin Extended), declared in
frontend/src/styles/fonts.css, and re-vendored by
frontend/tools/vendor_fonts.py. Their OFL licences ship beside them.
With no remote asset left, the CSP drops both Google hosts and gains
object-src, base-uri and form-action; woff2 also gets its real media
type, which Python's table lacks on a slim image.

A trusted-LAN Ollama turned out not to work at all over HTTPS. httpx
verifies against the certifi bundle, so an endpoint whose certificate
comes from a CA the user installed on their own machines — a StartOS
server's Ollama, for one — was refused with CERTIFICATE_VERIFY_FAILED
while curl and the browser on the same host accepted it.
app/tlstrust.py builds one context that unions the platform CA store
with certifi's, and all four outbound clients use it. A union rather
than a swap, so an image with an empty system store cannot start failing
on endpoints that worked before. Verification itself is untouched:
CERT_REQUIRED, hostname checking on, and no insecure escape hatch.

The storyteller listener is now loopback by explicit statement rather
than by inheriting uvicorn's default: start.sh, start.ps1, and
docker-compose.yml, which publishes to 127.0.0.1 rather than every
interface. Reaching an Ollama on another machine is outbound and needs
none of that inbound exposure.

backend/requirements.lock pins the exact tested closure;
requirements.txt keeps the ranges. DEVELOPMENT.md covers setup, the
same-host and trusted-LAN Ollama configurations, and how to re-run the
offline proof. PROVENANCE.md records the upstream commit, the MIT terms,
and both vendored assets.

Verified, not just compiled. On an --internal Docker network with
1.1.1.1 unreachable and no name resolving, a campaign was created and
played for six turns through same-host Ollama, restarted, and resumed.
A second run played ten turns through Ollama on a separate physical
machine on the LAN over verified HTTPS, summaries and embeddings
included, with the storyteller's default route deleted so the LAN was
reachable and the Internet was not. Its capture: 893 packets to the
approved host, 730 loopback, zero anywhere else, and zero DNS queries.
Two induced model failures left the accepted story bit-identical. The
inherited SPA was opened in a browser and a campaign read back from it.
Evidence is in planning/reports/M1-BASELINE-REPORT.md, along with the
findings that did not belong in this change.

648 backend tests pass, up from the inherited 632; frontend lint and
build are clean; the image builds. No M2 work is included: the hosted,
cloud, analytics, Postgres and scripting surfaces are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017foPNqFjAJa2Ngebf5mEfL

DEVELOPMENT.md

@@ -0,0 +1,246 @@
+# Development and local operation
+
+This is the Adventure Storyteller production fork of AI-DnD. `PROVENANCE.md`
+records where the code came from; `planning/` holds the product specification
+and milestone plan.
+
+Everything here assumes the local-only rule from
+`planning/DECISIONS/004-local-only-production.md`: after setup, ordinary story
+play must work with **no Internet access at all**. Setup itself downloads
+dependencies and models; playing does not.
+
+## Versions this was built and tested on
+
+| | |
+| --- | --- |
+| OS | Linux (Ubuntu 24.04 userland), x86-64, 4 cores, 15 GB RAM, no GPU |
+| Python | 3.12.3 |
+| Node | 22.23.1, npm 10.9.8 (the Dockerfile builds the SPA on Node 24) |
+| Ollama | `ollama/ollama:latest` in Docker |
+| Models | `qwen2.5:3b-instruct` (narrator), `nomic-embed-text` (memory bank) |
+
+## Setup
+
+```bash
+# Backend, from the exact tested dependency closure.
+python3 -m venv backend/.venv
+backend/.venv/bin/pip install -r backend/requirements.lock
+
+# Frontend.
+cd frontend && npm ci && cd ..
+```
+
+`backend/requirements.lock` pins every version, transitive ones included.
+`backend/requirements.txt` states the ranges the code actually needs and stays
+the file you edit; regenerate the lock after a deliberate upgrade (the header in
+the lock says how).
+
+This is the only step that needs the Internet. It downloads Python and npm
+packages; it does **not** download a tokenizer or a font, because both are
+vendored in the tree — see "What was made offline-safe" below.
+
+You also need the models, once:
+
+```bash
+ollama pull qwen2.5:3b-instruct
+ollama pull nomic-embed-text      # only if you want the memory bank
+```
+
+## Running
+
+**Development** — backend on `:8000`, Vite dev server on `:5173`:
+
+```bash
+./start.sh
+```
+
+**Production-shaped** — one server, SPA served by FastAPI:
+
+```bash
+cd frontend && npm run build && cd ..
+cd backend && .venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000
+```
+
+Then open <http://127.0.0.1:8000>.
+
+**Docker**:
+
+```bash
+docker compose up --build
+```
+
+### The listener is loopback, and stays loopback
+
+`start.sh`, `start.ps1` and the production command above all pass
+`--host 127.0.0.1` explicitly. `docker-compose.yml` publishes
+`127.0.0.1:8000:8000` — the process inside the container listens on `0.0.0.0`
+because a published port cannot reach anything else, but the port is only
+bound on the host's loopback.
+
+That is a requirement, not a preference. In local mode the storyteller API is
+single-user and unauthenticated: anything that can reach it can read and
+rewrite every campaign. Putting Ollama on another machine (below) does **not**
+change this — it is an outbound connection and needs no inbound exposure.
+
+If you publish the port to `0.0.0.0` anyway, you have made a deliberate
+decision that this project's threat model does not cover
+(`planning/SECURITY-THREAT-MODEL.md`).
+
+## Pointing the storyteller at Ollama
+
+The endpoint, model and (unused) API key are **runtime settings stored in the
+database**, not environment variables. Set them on the app's Settings page, or
+with one request:
+
+```bash
+curl -X PUT http://127.0.0.1:8000/api/settings \
+  -H 'Content-Type: application/json' \
+  -d '{"endpoint_url":"http://127.0.0.1:11434/v1","model":"qwen2.5:3b-instruct",
+       "api_mode":"chat","api_key":"","max_output_tokens":200,
+       "context_token_budget":4096}'
+```
+
+`POST /api/settings/test` (the **Test connection** button) returns
+`{"ok": true, "models": [...]}` and is the fastest way to tell a wrong endpoint
+from a missing model.
+
+### Same host (the default)
+
+```text
+endpoint_url = http://127.0.0.1:11434/v1
+```
+
+Nothing else to do. Ollama's own default is to listen on loopback.
+
+### An Ollama on another machine on your trusted LAN
+
+Supported and explicitly configured — never guessed, never discovered.
+
+On the **inference machine**, tell Ollama to accept connections from the LAN,
+because it binds loopback by default:
+
+```bash
+OLLAMA_HOST=0.0.0.0:11434 ollama serve
+```
+
+On the **storyteller machine**, set the endpoint to that host's address:
+
+```text
+endpoint_url = http://192.168.1.50:11434/v1
+```
+
+Use an IP address or a name your own network resolves. Then:
+
+- the storyteller UI/API stays on `127.0.0.1` — do not change the listener;
+- prompts, story text, retrieved memories and embedding inputs all travel to
+  that host, so it has to be one you control, on a network you trust;
+- the inference machine needs the models installed, not the storyteller;
+- no Internet is involved in either direction.
+
+`app/netguard.py` refuses private addresses only in hosted multi-user mode
+(`AIDND_MULTI_USER=1`), which local installs never turn on, so a LAN endpoint
+is accepted as configured.
+
+#### If that endpoint is HTTPS with your own CA
+
+Some inference hosts are only reachable over TLS. A StartOS server is one: it
+serves Ollama over HTTPS with a certificate from its own local CA, and plain
+HTTP redirects to it.
+
+Install that CA on the machine running the storyteller, the same way you would
+for the browser — on Debian and Ubuntu:
+
+```bash
+sudo cp your-ca.crt /usr/local/share/ca-certificates/
+sudo update-ca-certificates
+```
+
+then use the `https://` URL and the hostname the certificate is issued for:
+
+```text
+endpoint_url = https://inference.lan:8443/v1
+```
+
+The application verifies against the machine's CA store **and** the `certifi`
+bundle (`backend/app/tlstrust.py`), so a CA you installed at the OS level is
+honoured, exactly as `curl` and your browser honour it. Public certificates
+keep working unchanged.
+
+There is deliberately **no** setting to skip verification. If a connection is
+refused with `CERTIFICATE_VERIFY_FAILED`, the CA is not installed where the
+storyteller can see it, or the URL's hostname does not match the certificate —
+`openssl s_client -connect host:port` will say which. In a container, remember
+the CA has to be inside the image or bind-mounted; the host's store is not
+visible from within.
+
+## Tests
+
+```bash
+cd backend && .venv/bin/python -m pytest tests/ -q     # 648 tests
+cd frontend && npm run lint && npm run build
+```
+
+Two files are the M1 regression guards.
+
+`test_offline_assets.py` fails if the tokenizer starts fetching its table
+again, if a remote font or stylesheet comes back, or if the CSP names a remote
+origin. Two of its checks read the built SPA under `frontend/dist/` and skip
+when it has not been built, so run `npm run build` before treating a green
+suite as complete evidence.
+
+`test_tls_trust.py` fails if outbound verification is weakened, if a public CA
+is lost from the union, or if a new HTTP client is added without the shared
+verification context.
+
+## What was made offline-safe, and how to check
+
+Two runtime downloads were removed in Milestone M1. Both were invisible on a
+machine that had been online once, which is exactly why they need tests.
+
+**The tokenizer.** `tiktoken.get_encoding("cl100k_base")` downloads a 1.7 MB
+BPE table on first use, and the context builder counts tokens on every turn, so
+the first story turn on an air-gapped install died with a `ConnectionError`.
+The table is vendored at `backend/app/context/vendor/cl100k_base.tiktoken` and
+`backend/app/context/encoding.py` builds the encoding from it, verifying its
+SHA-256 against the digest `tiktoken` itself pins.
+
+**The fonts.** The SPA linked `fonts.googleapis.com` from `index.html`, so
+every page load fetched a stylesheet and font files from Google. The three
+families are self-hosted under `frontend/public/fonts/`, declared in
+`frontend/src/styles/fonts.css`, and re-vendored by
+`python3 frontend/tools/vendor_fonts.py`. The CSP in `backend/app/main.py` now
+names no remote origin at all.
+
+To convince yourself on a machine that has already been online, run the app
+with no route out rather than trusting a cold cache:
+
+```bash
+docker network create --internal offline
+docker run -d --name ollama --network offline -v ollama-models:/root/.ollama ollama/ollama
+docker build -t storyteller .
+# The app shares Ollama's network namespace, so Ollama is on its loopback and
+# neither has a route to the Internet.
+docker run -d --name app --network container:ollama -v story-data:/data \
+  storyteller uvicorn app.main:app --host 127.0.0.1 --port 8000
+docker exec app python -c "import socket; socket.create_connection(('1.1.1.1',443),timeout=4)"
+# -> OSError: Network is unreachable, and story turns still work
+```
+
+`planning/reports/M1-BASELINE-REPORT.md` records the run this procedure is
+taken from, including the packet captures.
+
+## Things inherited from upstream that M1 deliberately did not touch
+
+These are M2's scope (`planning/BUILD-MILESTONES.md`), listed here so nobody
+reports them as new:
+
+- hosted/multi-user/account/demo-key code, analytics tables, Postgres and
+  Render deployment paths, and the OpenRouter default endpoint constant all
+  still exist in the tree. None of them is reachable from a default local run,
+  and none requires a cloud service.
+- `docs/*.html` is upstream's GitHub Pages project site and still links Google
+  Fonts. It is not served by the application and is not part of any build.
+- `.github/workflows/ci.yml` is upstream's GitHub Actions pipeline. This
+  repository lives on a self-hosted Gitea; the workflow is kept for provenance
+  and is not what runs the tests here.
+- QuickJS campaign scripting is still present and still tested.