Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M2: cut the hosted product away from the local one

Browse Source 
94 files, +1,395 -6,578. Three files are new; twenty-four are gone. The
milestone is subtraction, and what is left is the single-user local
storyteller the specification describes.

Removed in full: campaign scripting and its QuickJS sandbox; multi-user
accounts, guest sessions, login, registration and the shared demo key;
the visitor-analytics tables, dashboard and page beacon; the access log
of sign-ins, addresses and devices; per-IP and per-user rate limiting
and quotas; Render deployment config; Postgres and psycopg; cloud
inference providers, the API-key field and the key encryption that
existed to store it; session-cookie signing. None of it was hidden
behind a flag — the routes are gone and answer 404.

Two things were kept that the brief allowed keeping. The `users` table
and its foreign keys stay as an internal ownership detail, because
rewriting them out means a migration across most of the schema to
delete a column that costs nothing; nothing creates a second user and
no request carries an identity. Five inert tables and four inert
columns stay for the same reason, so an M1 campaign database opens
unchanged.

The one addition is app/endpoints.py, which decides where a story may
be sent. Loopback, RFC1918, link-local, unique-local and CGNAT — an
explicit allowlist of networks, not a guess at what `ipaddress` means
by "private", which calls the documentation ranges private and IPv6
loopback reserved. Every address a hostname resolves to must be in it,
so a split answer does not squeak through, and the rule runs both when
the endpoint is saved and before every outbound request, because a name
that resolved to the LAN this morning can resolve elsewhere this
afternoon. Known cloud hosts are named in the refusal so the error says
why rather than looking like broken DNS. TLS is never traded against
it: M1's shared trust context is intact on all four clients and there
is no way to skip verification.

The hardcoded 120-second model timeout is now a setting. That was not
theoretical — on this GPU-less four-core host a cold load of
qwen2.5:3b-instruct took 648.9 seconds to produce the first turn, while
turns 2 to 5 of the same campaign took 3.6 to 13.1. Connect stays short
at 10s so a wrong address still fails fast; the read timeout defaults
to 300s and is bounded at 3600, because "wait longer" must stay a
number.

Two defects found while testing and fixed here. An unknown /api path
fell through the SPA catch-all and came back as HTML with status 200,
so a client asking for JSON parsed a web page instead of learning the
route was gone. And AIDND_CORS_ORIGINS accepted "*", which on an
unauthenticated loopback API would hand every page on the Internet a
write handle on the campaign database; it now refuses to start.

Verified rather than assumed. Offline, on a network with no route out
and no DNS: five turns, retry with both takes retained, restart with an
identical transcript digest, a failed model call leaving the accepted
AI-turn count untouched, and a capture with zero non-loopback unicast
packets. Against a real second machine on the LAN over HTTPS with a
private CA: four turns, restart, and a capture showing 289 packets to
the approved host, 344 loopback, zero anywhere else, zero DNS queries.
Cloud and public endpoints refused with their reasons; no API key
settable; every removed route 404.

604 backend tests pass, down from 648 by the fifteen retired with the
subsystems they tested and up by the twenty-nine added for the endpoint
policy and the removed surface. The scripting tests were not deleted:
eight files used a JavaScript counter as instrumentation for the state
snapshot and rollback machinery, which M2 does not touch, so the
counter moved to the world-state engine and those tests still assert
what they always did. Frontend lint and build are clean; the image
builds, and its wheel-building stage is gone with quickjs.

No M3 work. Undo is still destructive and there is still no Redo.
This commit is contained in:
JesseMarkowitz
2026-09-02 11:27:14 -04:00
parent 1a28a9a708
commit 8c65ae99de
94 changed files with 1384 additions and 6567 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/tools/rewrite_memories.py
+16 -62
View File
@@ -56,66 +56,22 @@ Usage, from `backend/`:
python -m tools.rewrite_memories --write --limit 3 # try three of them
python -m tools.rewrite_memories --write --embed # the whole backfill
Without `--write` it makes no model calls, spends nothing, and only reports the
scope. Every run reads the database the app reads: `AIDND_DB_PATH`, or
`DATABASE_URL` for a hosted Postgres. Take a copy of it first — the old text is
overwritten and is not kept anywhere.
Without `--write` it makes no model calls and only reports the scope. Every run
reads the database the app reads, at `AIDND_DB_PATH`. Take a copy of it first —
the old text is overwritten and is not kept anywhere.
**On the hosted deploy, name whose adventures you mean.** That database holds
other people's stories, and each adventure is summarized with *its owner's* key,
so an unfiltered `--write` spends other people's money on memories they did not
ask to have rewritten. `--email` restricts the run to the accounts you name and
`--adventure` to single adventures; a dry run costs nothing and lists both, with
the owner of each. Guests have no email and can only be reached by id.
Two environment variables reach that database from a checkout. In PowerShell,
which is where this project is developed, they are set first and then persist
for the rest of the session:
$env:AIDND_DATABASE_URL = Read-Host 'Neon URL'
$env:AIDND_SECRET_KEY = Read-Host 'Secret key'
.venv\Scripts\python.exe -m tools.rewrite_memories --email you@example.com
`Read-Host` keeps both values out of the PowerShell history file. On a POSIX
shell the same thing is one line:
AIDND_DATABASE_URL=... AIDND_SECRET_KEY=... \
python -m tools.rewrite_memories --email you@example.com
`AIDND_SECRET_KEY` is not optional there. Stored API keys are encrypted with it,
and with the wrong one `decrypt_secret` returns "" and every adventure is skipped
as having no key (see `security.py`). The deployed image does not carry this
directory — the Dockerfile copies `backend/app` alone — so run it from a
checkout against the hosted database rather than from a shell on the box.
`--adventure` restricts the run to single adventures, and a dry run lists what
would change. The deployed image does not carry this directory — the Dockerfile
copies `backend/app` alone — so run it from a checkout.
"""
import argparse
import asyncio
import sys
from pathlib import Path
from urllib.parse import urlsplit
from sqlalchemy import func, inspect as sa_inspect, select
def safe_dsn(url: str) -> str:
"""A connection string with the credentials taken out.
The report says which database it is about to rewrite, which is worth
printing. The password in a Neon URL is not: this output goes to a console,
a screenshot, or a pasted bug report, and the operator has no way to know
the line carried a credential until it is somewhere else.
"""
parsed = urlsplit(url)
if not parsed.hostname:
return "(configured)"
who = f"{parsed.username}@" if parsed.username else ""
port = f":{parsed.port}" if parsed.port else ""
# The query string is dropped whole. `sslmode` is the only part anyone
# wants to see, and some drivers accept a password there too.
return f"{parsed.scheme}://{who}{parsed.hostname}{port}{parsed.path}"
def words(text: str) -> int:
return len(text.split())
@@ -127,16 +83,16 @@ def one_line(text: str, width: int = 96) -> str:
async def main(args) -> int:
from app import memorybank, models
from app.database import DB_PATH, DATABASE_URL, SessionLocal
from app.database import DB_PATH, SessionLocal
from app.providers import OpenAICompatibleProvider, ProviderError
db = SessionLocal()
print(f"database: {safe_dsn(DATABASE_URL) if DATABASE_URL else DB_PATH}")
print(f"database: {DB_PATH}")
if not sa_inspect(db.get_bind()).has_table(models.Adventure.__tablename__):
# A mistyped path creates an empty SQLite file rather than failing, so
# say what is wrong instead of raising "no such table: adventures".
print("There are no tables here. Point AIDND_DB_PATH, or DATABASE_URL "
"for a hosted deploy, at the database the app uses.")
print("There are no tables here. Point AIDND_DB_PATH at the database "
"the app uses.")
return 2
adventures = db.query(models.Adventure).order_by(models.Adventure.id)
@@ -177,14 +133,13 @@ async def main(args) -> int:
def provider_for(settings: models.Settings) -> OpenAICompatibleProvider:
"""The adventure owner's own summarizer, unless the run overrides it."""
if not (args.endpoint or args.model or args.api_key):
if not (args.endpoint or args.model):
return memorybank.summary_provider(settings)
return OpenAICompatibleProvider(
args.endpoint or settings.endpoint_url,
args.api_key or settings.api_key_plain,
args.model or settings.summary_model or settings.model,
settings.api_mode,
settings.reasoning_max_tokens,
settings.model_timeout_seconds,
)
totals = {"rewritten": 0, "would rewrite": 0, "no source": 0,
@@ -205,11 +160,11 @@ async def main(args) -> int:
continue
settings = settings_for(adventure.user_id)
# An adventure whose owner has no key is reported rather than skipped
# silently: it is the one reason a memory this tool can rewrite is left
# alone, and the operator can fix it with --api-key.
# An adventure with no model configured is reported rather than
# skipped silently: it is the one reason a memory this tool can rewrite
# is left alone, and the operator can fix it with --model.
usable = settings is not None and bool(
args.api_key or args.endpoint or settings.api_key_plain
args.model or settings.summary_model or settings.model
)
owner = db.get(models.User, adventure.user_id)
who = (owner.email if owner and owner.email
@@ -333,7 +288,6 @@ if __name__ == "__main__":
help="re-embed here. Stop the app first; see the docstring.")
parser.add_argument("--endpoint", help="override the owner's endpoint URL.")
parser.add_argument("--model", help="override the owner's summary model.")
parser.add_argument("--api-key", help="override the owner's API key.")
args = parser.parse_args()
# A Windows console defaults to cp1252, which cannot encode the arrow this