JesseMarkowitzandClaude Opus 5
480414efe0
A campaign can import local .txt and .md files as Canon, Reference or Inspiration, and the class is load-bearing rather than a label: it decides the words a passage is framed with in the prompt, the weight it carries when passages are ranked, and which budget it competes in when the context is tight. This is a separate subsystem, which is the Phase 0B decision (IMPORTED-KNOWLEDGE-DESIGN.md §73). Story Cards do not carry classification, provenance, content identity, chunking, an index or a lifecycle, and they were not promoted into something that does. Nothing here reads or writes one. The subsystem, in backend/app/knowledge/: classes the three classes, their weights, and the prompt framing chunking deterministic, heading-aware, 60-800 tokens, no overlap fts SQLite FTS5 with porter stemming; scoped and bounded in SQL importer validate, hash, store, chunk, index — in one transaction embeddings local Ollama vectors through the shared provider retrieval query construction, hybrid merge, rerank inject the budgeted cut and the rendered prompt sections Relevance admission is a separate stage from ranking, and that separation is the milestone's most expensive lesson. An independent review found the first implementation deciding relevance with a floor expressed as a share of the best candidate — which the best clears by construction — so a passage was admitted on every turn regardless of the scene. A query about tide tables and container tonnage retrieved all five sources of a fantasy campaign, narrator-only hidden Canon among them. So the pipeline is now: candidate generation -> admission -> ranking -> class weighting -> budget Admission reads raw, candidate-set-independent signals: the cosine the model returned, and how many distinct meaningful query terms a passage contains. Ranking reads normalized ones, because bm25 has no fixed range and cosine's zero is not zero. Normalization decides order among things that matched; it can never decide whether anything matched. Authority is applied after admission, so a class orders what matched and never rescues what did not. Retrieval may therefore return nothing, and on a scene unrelated to the library it does. The other decisions that each replaced an obvious wrong one: - The class multiplies relevance rather than adding to it. An additive bonus satisfies "Canon outranks Reference" and makes "do not include irrelevant Canon" impossible, because a large enough constant wins on its own. - The semantic floor is measured, not guessed: 113 production-path pairs against nomic-embed-text put targeted matches at 0.55-0.85 and off-topic pairs at 0.36-0.56, and 0.58 sits between them. Because it is a property of that model and not of cosine similarity, it is keyed to the model rather than applied to whatever is configured: an embedding model with no measured calibration in this build does not borrow the number. Semantic admission is skipped, the campaign retrieves lexically, and the reason is stated in the knowledge status and in the turn's provenance. Degrading to lexical keeps the library usable; lending the threshold to an unmeasured model is how the admitted-everything defect would return. - One lexical term is not evidence. Two distinct meaningful terms, or one that is neither a standing campaign entity nor a negligible share of the query. The stop list grew from 42 words to 261, all function words — no subject matter, because a stop list that removes subject matter stops finding "The Silver Key". - Lexical retrieval is a production path, not a fallback. It finds the proper nouns and invented terms a setting bible is made of, and the library is fully usable with no embedding model configured. Safety is structural rather than filtered. Imported text reaches the prompt whole, inside a section that says what it is, under a rule stating the authority order in words and refusing every instruction inside it. No endpoint accepts a filesystem path, so H08 has no mechanism to escape from. Nothing renders imported content as HTML, so a script tag is five visible characters and a remote image is never fetched. Import, chunking, indexing, retrieval and a turn open no socket at all; only embeddings do, through the endpoint allowlist the memory bank already uses. Provenance is the rendered text, not a foreign key: deleting a source cannot turn a historical turn's evidence into dangling ids. Schema: knowledge_sources, knowledge_chunks, knowledge_embeddings, and an FTS5 virtual table attached to knowledge_chunks as a DDL hook so it is created and dropped with the table it indexes. Migration 92. A pre-M7 database opens unchanged and needs no sources to play. Bundle: the source content and the reader's judgements about it travel; the passages, index rows and vectors are rebuilt on import, so a restored campaign is searchable immediately without a reindex step. One runtime dependency: python-multipart, Starlette's multipart parser. It is what makes the upload surface possible, and the upload surface is why no pathname is ever accepted. The test doubles were the reason the defect shipped, so they were corrected too. The retrieval stub scored unrelated text at 0.06-0.20 where the real model scores it at 0.43-0.44, and its docstring said it had deliberately removed the constant component that "would put a similarity floor under every pair" — which is exactly the property real models have. The stub now has that floor, one test fails if it is ever removed, and another reproduces the superseded rule and asserts it is still fooled by the same fixture. Run against the pre-corrective implementation, the new suite fails 13 of 18. Tests: 939 passed, 14 skipped (836/7 at M6). 110 new across seven files, one of which mocks nothing between itself and Ollama and re-measures the similarity separation on every run. 43/43 checks in a real Firefox, reproduced. Docker build clean. Four other defects found by review or by the browser run were fixed here rather than carried: an unreachable relevance constant that appeared to enforce something and did not; acceptance tests using the wrong fixture files, so G07's trap was never exercised; a bidirectional override surviving into displayed filenames; and, from the implementation pass, the Insights panel showing M5's two state sections as raw keys and the source inspector refetching on every keystroke. M7 was independently reviewed, which returned PASS WITH CORRECTIVE WORK REQUIRED. Both blocking findings are closed, and closeout resolved the embedding-model calibration boundary the corrective pass had left as debt. planning/reports/M7-IMPLEMENTATION-REPORT.md carries the review, the corrective closeout and the closeout verification in sequence, none overwriting another. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HdaXiFbscatQaLS7dJk6b
15 KiB
Development and local operation
This is the Adventure Storyteller production fork of AI-DnD. PROVENANCE.md
records where the code came from; planning/ holds the product specification
and milestone plan.
Everything here assumes the local-only rule from
planning/DECISIONS/004-local-only-production.md: after setup, ordinary story
play must work with no Internet access at all. Setup itself downloads
dependencies and models; playing does not.
Versions this was built and tested on
| OS | Linux (Ubuntu 24.04 userland), x86-64, 4 cores, 15 GB RAM, no GPU |
| Python | 3.12.3 |
| Node | 22.23.1, npm 10.9.8 (the Dockerfile builds the SPA on Node 24) |
| Ollama | ollama/ollama:latest in Docker |
| Models | qwen2.5:3b-instruct (narrator), nomic-embed-text (memory bank) |
Setup
# Backend, from the exact tested dependency closure.
python3 -m venv backend/.venv
backend/.venv/bin/pip install -r backend/requirements.lock
# Frontend.
cd frontend && npm ci && cd ..
One runtime dependency was added in M7: python-multipart, which is Starlette's
multipart form parser and is how a knowledge source is uploaded. It is pure
Python, Apache-2.0, and has no dependencies of its own, so it adds nothing to
audit beyond itself and no network path at all.
backend/requirements.lock pins every version, transitive ones included.
backend/requirements.txt states the ranges the code actually needs and stays
the file you edit; regenerate the lock after a deliberate upgrade (the header in
the lock says how).
This is the only step that needs the Internet. It downloads Python and npm packages; it does not download a tokenizer or a font, because both are vendored in the tree — see "What was made offline-safe" below.
You also need the models, once:
ollama pull qwen2.5:3b-instruct
ollama pull nomic-embed-text # only if you want the memory bank
There is no account to create and nothing to log in to. The application is single-user: whoever can reach it on loopback is its owner.
Running
Development — backend on :8000, Vite dev server on :5173:
./start.sh
Production-shaped — one server, SPA served by FastAPI:
cd frontend && npm run build && cd ..
cd backend && .venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000
Then open http://127.0.0.1:8000.
Docker:
docker compose up --build
The listener is loopback, and stays loopback
start.sh, start.ps1 and the production command above all pass
--host 127.0.0.1 explicitly. docker-compose.yml publishes
127.0.0.1:8000:8000 — the process inside the container listens on 0.0.0.0
because a published port cannot reach anything else, but the port is only
bound on the host's loopback.
That is a requirement, not a preference. In local mode the storyteller API is single-user and unauthenticated: anything that can reach it can read and rewrite every campaign. Putting Ollama on another machine (below) does not change this — it is an outbound connection and needs no inbound exposure.
If you publish the port to 0.0.0.0 anyway, you have made a deliberate
decision that this project's threat model does not cover
(planning/SECURITY-THREAT-MODEL.md).
Pointing the storyteller at Ollama
The endpoint, the model and the generation parameters are runtime settings stored in the database, not environment variables. There is no API key field: M2 removed it along with the cloud providers, and Ollama does not use one. Set them on the app's Settings page, or with one request:
curl -X PUT http://127.0.0.1:8000/api/settings \
-H 'Content-Type: application/json' \
-d '{"endpoint_url":"http://127.0.0.1:11434/v1","model":"qwen2.5:3b-instruct",
"api_mode":"chat","max_output_tokens":200,
"context_token_budget":4096}'
POST /api/settings/test (the Test connection button) returns
{"ok": true, "models": [...]} and is the fastest way to tell a wrong endpoint
from a missing model. When it fails it says which kind of failure it was, and
they need different things done about them:
kind |
What it means |
|---|---|
rejected |
The endpoint is outside the policy below. Not a network problem. |
unreachable |
Nothing answered. Ollama is not running there, or the port is wrong. |
tls |
The certificate did not verify — install the CA (see below). |
timeout |
It accepted the connection and then said nothing. |
http |
It answered with an error status; the body is included. |
A successful test also warns when the endpoint is reachable but has no model by the configured name, which is the commonest way for a correct endpoint to still fail every turn.
Which endpoints are allowed
backend/app/endpoints.py decides, and it is deliberately narrow: loopback,
your own LAN, or nothing. The allowed networks are 127.0.0.0/8, the three
RFC1918 ranges, link-local, IPv6 loopback and unique-local, and 100.64.0.0/10
(carrier-grade NAT, which is what a mesh VPN such as Tailscale hands out).
Every address the endpoint's hostname resolves to must be in one of them. A public address is refused, a name resolving to both a private and a public address is refused, and known cloud inference hosts are refused by name so the error says why rather than looking like a DNS fault.
The rule is applied when you save the endpoint and again before every outbound request, so a database edited by hand or a hostname that starts resolving somewhere new cannot turn a local install into an exfiltration path. There is no setting to relax it.
Same host (the default)
endpoint_url = http://127.0.0.1:11434/v1
Nothing else to do. Ollama's own default is to listen on loopback.
An Ollama on another machine on your trusted LAN
Supported and explicitly configured — never guessed, never discovered.
On the inference machine, tell Ollama to accept connections from the LAN, because it binds loopback by default:
OLLAMA_HOST=0.0.0.0:11434 ollama serve
On the storyteller machine, set the endpoint to that host's address:
endpoint_url = http://192.168.1.50:11434/v1
Use an IP address or a name your own network resolves. Then:
- the storyteller UI/API stays on
127.0.0.1— do not change the listener; - prompts, story text, retrieved memories and embedding inputs all travel to that host, so it has to be one you control, on a network you trust;
- the inference machine needs the models installed, not the storyteller;
- no Internet is involved in either direction.
A LAN endpoint is accepted because it is on one of the allowed networks above. Nothing else about it is special.
If that endpoint is HTTPS with your own CA
Some inference hosts are only reachable over TLS. A StartOS server is one: it serves Ollama over HTTPS with a certificate from its own local CA, and plain HTTP redirects to it.
Install that CA on the machine running the storyteller, the same way you would for the browser — on Debian and Ubuntu:
sudo cp your-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
then use the https:// URL and the hostname the certificate is issued for:
endpoint_url = https://inference.lan:8443/v1
The application verifies against the machine's CA store and the certifi
bundle (backend/app/tlstrust.py), so a CA you installed at the OS level is
honoured, exactly as curl and your browser honour it. Public certificates
keep working unchanged.
There is deliberately no setting to skip verification. If a connection is
refused with CERTIFICATE_VERIFY_FAILED, the CA is not installed where the
storyteller can see it, or the URL's hostname does not match the certificate —
openssl s_client -connect host:port will say which. In a container, remember
the CA has to be inside the image or bind-mounted; the host's store is not
visible from within.
Tests
cd backend && .venv/bin/python -m pytest tests/ -q # 920 tests, 10 skipped
cd frontend && npm run lint && npm run build
Ten tests skip without something the machine may not have: seven need a second machine or an environment the suite cannot create, and three are M7's real-model tests below.
Two files are the M1 regression guards.
test_offline_assets.py fails if the tokenizer starts fetching its table
again, if a remote font or stylesheet comes back, or if the CSP names a remote
origin. Two of its checks read the built SPA under frontend/dist/ and skip
when it has not been built, so run npm run build before treating a green
suite as complete evidence.
test_tls_trust.py fails if outbound verification is weakened, if a public CA
is lost from the union, or if a new HTTP client is added without the shared
verification context.
M5 added test_narrative_state.py, which fails if the state stops being
genre-neutral, if an event outside the allowlist is ever applied, if a malformed
proposal mutates anything, if campaign canon stops outranking the narration, or
if a turn's narration and its state can be committed apart from each other.
test_narrative_realistic.py is the one suite that needs a real model, and it is
skipped unless you point it at one:
AIDND_TEST_ENDPOINT=http://127.0.0.1:11434/v1 \
AIDND_TEST_MODEL=qwen2.5:3b-instruct \
backend/.venv/bin/python -m pytest backend/tests/test_narrative_realistic.py -v -s
It exists because Phase 0B found that structured-state behaviour can look correct on a small prompt and fail under a full one — and it has already earned its place, catching a case where a model echoed its own instruction into the narration.
M7 added five files. test_imported_knowledge.py is the acceptance contract —
G01-G10, C05, F05/F06's imported halves, I05, H06-H09, campaign isolation,
lexical retrieval without embeddings, a bounded knowledge budget, deletion that
preserves historical prompt evidence, hidden Canon, stale Canon against current
state, and an abandoned line of story failing to influence the retrieval query.
test_knowledge_chunking.py fails if chunking stops being deterministic or
starts producing fragments or giants. test_knowledge_retrieval_quality.py
fails if class stops settling ties, if irrelevant Canon starts winning on class
alone, if the hybrid merge duplicates a passage, or if suppression crosses a
class. test_knowledge_performance.py fails if any knowledge read grows a query
per source or per passage, or if candidates stop being bounded in SQL.
test_knowledge_migration.py fails if a pre-M7 database stops opening, or if the
FTS5 index stops travelling with the table it indexes.
test_knowledge_real_model.py is M7's real-provider test and skips without an
endpoint. It mocks nothing between itself and Ollama: a real Settings row, the
real factory, a real embedding request, real stored vectors, real hybrid
retrieval, and a real prompt.
AIDND_TEST_ENDPOINT=https://inference.lan:8443/v1 \
AIDND_TEST_EMBED_MODEL=nomic-embed-text \
backend/.venv/bin/python -m pytest backend/tests/test_knowledge_real_model.py -v -s
M4 added test_save_points.py, which fails if restoring a Save Point starts
deleting history, stops going through the active head, forks on its own, lets a
Save Point on one campaign be restored through another, or lets deleting a branch
take a Save Point with it. It also fails if listing Save Points goes back to one
query per Save Point, or starts fetching narration to render the list.
test_process_restart.py is the durability guard: it starts the application as a
real subprocess, kills it, and starts a second one against the same database. A
Save Point that survived only because a Python object was still alive would pass
an in-process test and fail a user's restart.
M2 added two more. test_endpoint_policy.py fails if the set of reachable
addresses widens, or if either place the rule is applied stops applying it —
it resolves hostnames through a stub, so it tests the policy rather than
whatever DNS the machine has. test_local_only_surface.py fails if a removed
subsystem comes back as a route, if an API key becomes settable again, if the
model timeout stops being configurable or becomes unbounded, or if a supported
start path stops binding loopback.
What was made offline-safe, and how to check
Two runtime downloads were removed in Milestone M1. Both were invisible on a machine that had been online once, which is exactly why they need tests.
The tokenizer. tiktoken.get_encoding("cl100k_base") downloads a 1.7 MB
BPE table on first use, and the context builder counts tokens on every turn, so
the first story turn on an air-gapped install died with a ConnectionError.
The table is vendored at backend/app/context/vendor/cl100k_base.tiktoken and
backend/app/context/encoding.py builds the encoding from it, verifying its
SHA-256 against the digest tiktoken itself pins.
The fonts. The SPA linked fonts.googleapis.com from index.html, so
every page load fetched a stylesheet and font files from Google. The three
families are self-hosted under frontend/public/fonts/, declared in
frontend/src/styles/fonts.css, and re-vendored by
python3 frontend/tools/vendor_fonts.py. The CSP in backend/app/main.py now
names no remote origin at all.
To convince yourself on a machine that has already been online, run the app with no route out rather than trusting a cold cache:
docker network create --internal offline
docker run -d --name ollama --network offline -v ollama-models:/root/.ollama ollama/ollama
docker build -t storyteller .
# The app shares Ollama's network namespace, so Ollama is on its loopback and
# neither has a route to the Internet.
docker run -d --name app --network container:ollama -v story-data:/data \
storyteller uvicorn app.main:app --host 127.0.0.1 --port 8000
docker exec app python -c "import socket; socket.create_connection(('1.1.1.1',443),timeout=4)"
# -> OSError: Network is unreachable, and story turns still work
planning/archive/milestone-reports/M1-BASELINE-REPORT.md records the run this procedure is
taken from, including the packet captures.
Things still inherited from upstream
M2 removed the hosted, cloud, account, analytics, Postgres/Render and QuickJS
scripting surfaces outright — PROVENANCE.md lists exactly what went. What is
left of upstream that a newcomer might report as a defect:
- Inert legacy tables and columns. Five tables and four columns M2 emptied
of meaning are still in the schema, unmapped, so an M1-era campaign database
opens unchanged. Nothing reads or writes them. A cleanup migration waits for
the schema to settle after M5 (
planning/BUILD-MILESTONES.md). - Dual-dialect migration code.
backend/app/migrations.pystill carries SQLite/Postgres branches from upstream, although Postgres support itself is gone and SQLite is the only store. Same cleanup, same milestone. .github/workflows/ci.ymlis upstream's GitHub Actions pipeline. This repository lives on a self-hosted Gitea; the workflow is kept for provenance and is not what runs the tests here.- No frontend tests.
npm run lint && npm run buildis the whole frontend check. A test runner is M8's job.