M2: cut the hosted product away from the local one
94 files, +1,395 -6,578. Three files are new; twenty-four are gone. The milestone is subtraction, and what is left is the single-user local storyteller the specification describes. Removed in full: campaign scripting and its QuickJS sandbox; multi-user accounts, guest sessions, login, registration and the shared demo key; the visitor-analytics tables, dashboard and page beacon; the access log of sign-ins, addresses and devices; per-IP and per-user rate limiting and quotas; Render deployment config; Postgres and psycopg; cloud inference providers, the API-key field and the key encryption that existed to store it; session-cookie signing. None of it was hidden behind a flag — the routes are gone and answer 404. Two things were kept that the brief allowed keeping. The `users` table and its foreign keys stay as an internal ownership detail, because rewriting them out means a migration across most of the schema to delete a column that costs nothing; nothing creates a second user and no request carries an identity. Five inert tables and four inert columns stay for the same reason, so an M1 campaign database opens unchanged. The one addition is app/endpoints.py, which decides where a story may be sent. Loopback, RFC1918, link-local, unique-local and CGNAT — an explicit allowlist of networks, not a guess at what `ipaddress` means by "private", which calls the documentation ranges private and IPv6 loopback reserved. Every address a hostname resolves to must be in it, so a split answer does not squeak through, and the rule runs both when the endpoint is saved and before every outbound request, because a name that resolved to the LAN this morning can resolve elsewhere this afternoon. Known cloud hosts are named in the refusal so the error says why rather than looking like broken DNS. TLS is never traded against it: M1's shared trust context is intact on all four clients and there is no way to skip verification. The hardcoded 120-second model timeout is now a setting. That was not theoretical — on this GPU-less four-core host a cold load of qwen2.5:3b-instruct took 648.9 seconds to produce the first turn, while turns 2 to 5 of the same campaign took 3.6 to 13.1. Connect stays short at 10s so a wrong address still fails fast; the read timeout defaults to 300s and is bounded at 3600, because "wait longer" must stay a number. Two defects found while testing and fixed here. An unknown /api path fell through the SPA catch-all and came back as HTML with status 200, so a client asking for JSON parsed a web page instead of learning the route was gone. And AIDND_CORS_ORIGINS accepted "*", which on an unauthenticated loopback API would hand every page on the Internet a write handle on the campaign database; it now refuses to start. Verified rather than assumed. Offline, on a network with no route out and no DNS: five turns, retry with both takes retained, restart with an identical transcript digest, a failed model call leaving the accepted AI-turn count untouched, and a capture with zero non-loopback unicast packets. Against a real second machine on the LAN over HTTPS with a private CA: four turns, restart, and a capture showing 289 packets to the approved host, 344 loopback, zero anywhere else, zero DNS queries. Cloud and public endpoints refused with their reasons; no API key settable; every removed route 404. 604 backend tests pass, down from 648 by the fifteen retired with the subsystems they tested and up by the twenty-nine added for the endpoint policy and the removed surface. The scripting tests were not deleted: eight files used a JavaScript counter as instrumentation for the state snapshot and rollback machinery, which M2 does not touch, so the counter moved to the world-state engine and those tests still assert what they always did. Frontend lint and build are clean; the image builds, and its wheel-building stage is gone with quickjs. No M3 work. Undo is still destructive and there is still no Redo.
This commit is contained in:
JesseMarkowitz
+36
-79
@@ -1,22 +1,23 @@
|
||||
"""AI Chat: a plain scratchpad for talking to a model directly.
|
||||
"""AI Chat: a plain scratchpad for talking to the configured model directly.
|
||||
|
||||
Power users reach it, which means the `AIDND_POWER_USERS` email allowlist. It is
|
||||
deliberately thin. It adds no story context, no scripts, and no world state, and
|
||||
it persists nothing. The conversation lives in the browser and is posted in full
|
||||
on each turn. It exists for testing models, prompts, and endpoints without
|
||||
starting an adventure.
|
||||
Deliberately thin. It adds no story context and no world state, and it persists
|
||||
nothing. The conversation lives in the browser and is posted in full on each
|
||||
turn. It exists for checking a model, a prompt, or an endpoint without starting
|
||||
an adventure — which is exactly the kind of thing a local single-user install
|
||||
wants a page for.
|
||||
|
||||
Model choice is free when the user brought their own API key. On the shared demo
|
||||
key the model stays pinned to the `AIDND_DEMO_MODELS` allowlist, exactly as it is
|
||||
for turns. The server funds that key, so this page must not let it reach paid
|
||||
models.
|
||||
Upstream gated this behind a "power user" email allowlist and pinned the model
|
||||
when a shared demo key was in play. M2 removed both: there is one local user,
|
||||
who owns the endpoint, and there is no server-funded key to protect. The model
|
||||
this page talks to is the one in Settings, or one the user names per request —
|
||||
either way it is their own Ollama.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from .. import auth, limits, models, schemas
|
||||
from .. import auth, models, schemas
|
||||
from ..database import get_db
|
||||
from ..providers import OpenAICompatibleProvider, ProviderError
|
||||
from ..sse import SSE_HEADERS, sse
|
||||
@@ -25,80 +26,40 @@ from .settings import get_settings, list_endpoint_models
|
||||
router = APIRouter(prefix="/api/chat", tags=["chat"])
|
||||
|
||||
|
||||
def power_user(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
) -> models.User:
|
||||
"""Gate for the whole router. 404 rather than 403 so the feature simply
|
||||
doesn't appear to exist for everyone else."""
|
||||
if not auth.is_power_user(user):
|
||||
raise HTTPException(404, "Not found")
|
||||
return user
|
||||
|
||||
|
||||
PowerUser = Depends(power_user)
|
||||
|
||||
|
||||
def _resolve_model(
|
||||
settings: models.Settings, requested: str | None
|
||||
) -> tuple[auth.ProviderConfig, str | None]:
|
||||
"""Returns the provider config for this chat, plus a note when the requested
|
||||
model was not used.
|
||||
|
||||
The pinning rule lives in `resolve_provider_config`. This function only
|
||||
reports the substitution that call made, so one place decides what the demo
|
||||
key may talk to.
|
||||
"""
|
||||
cfg = auth.resolve_provider_config(settings, model_override=requested)
|
||||
wanted = (requested or "").strip()
|
||||
if wanted and wanted != cfg.model:
|
||||
return cfg, (
|
||||
f"'{wanted}' isn't available on the shared demo key — using "
|
||||
f"{cfg.model}. Add your own API key in Settings to use any model."
|
||||
)
|
||||
return cfg, None
|
||||
|
||||
|
||||
@router.get("/config")
|
||||
async def chat_config(
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
"""Returns what this page can talk to.
|
||||
|
||||
The response holds the resolved endpoint and model, whether model choice is
|
||||
pinned to the demo allowlist, and the endpoint's model listing. The listing
|
||||
is best effort, and an unreachable endpoint returns an empty list.
|
||||
The model listing is best effort: an unreachable endpoint returns an empty
|
||||
list and the reason, rather than failing the page.
|
||||
"""
|
||||
settings = get_settings(db, user)
|
||||
cfg = auth.resolve_provider_config(settings)
|
||||
listing = await list_endpoint_models(cfg)
|
||||
listing = await list_endpoint_models(settings.endpoint_url)
|
||||
return {
|
||||
"endpoint_url": cfg.endpoint_url,
|
||||
"model": cfg.model,
|
||||
"using_demo": cfg.using_demo,
|
||||
"endpoint_url": settings.endpoint_url,
|
||||
"model": settings.model,
|
||||
"api_mode": settings.api_mode,
|
||||
"temperature": settings.temperature,
|
||||
"max_tokens": settings.max_output_tokens,
|
||||
# On the demo key the whitelist IS the list of choices; otherwise it's
|
||||
# whatever the endpoint advertises (suggestions, not a restriction).
|
||||
"models": auth.DEMO_MODELS if cfg.using_demo else listing.get("models", []),
|
||||
# Suggestions from the endpoint, not a restriction.
|
||||
"models": listing.get("models", []),
|
||||
"models_error": None if listing.get("ok") else listing.get("detail"),
|
||||
}
|
||||
|
||||
|
||||
async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload: schemas.ChatRequest,
|
||||
note: str | None, db: Session, user: models.User):
|
||||
async def run_chat(
|
||||
settings: models.Settings, model: str, payload: schemas.ChatRequest
|
||||
):
|
||||
"""Streams the reply as SSE, using the turn stream's event shape.
|
||||
|
||||
The generator emits `reasoning` and `chunk` events while generating and then
|
||||
a `done` event, so the frontend reuses the same code.
|
||||
"""
|
||||
if note:
|
||||
yield sse({"type": "note", "detail": note})
|
||||
provider = OpenAICompatibleProvider(
|
||||
cfg.endpoint_url, cfg.api_key, cfg.model, settings.api_mode,
|
||||
settings.reasoning_max_tokens,
|
||||
settings.endpoint_url, model, settings.api_mode
|
||||
)
|
||||
messages = [m.model_dump() for m in payload.messages]
|
||||
chunks: list[str] = []
|
||||
@@ -106,7 +67,11 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
try:
|
||||
async for kind, chunk in provider.chat(
|
||||
messages,
|
||||
temperature=payload.temperature if payload.temperature is not None else settings.temperature,
|
||||
temperature=(
|
||||
payload.temperature
|
||||
if payload.temperature is not None
|
||||
else settings.temperature
|
||||
),
|
||||
max_tokens=payload.max_tokens or settings.max_output_tokens,
|
||||
):
|
||||
if kind == "reasoning":
|
||||
@@ -123,33 +88,26 @@ async def run_chat(cfg: auth.ProviderConfig, settings: models.Settings, payload:
|
||||
if not text:
|
||||
detail = (
|
||||
"The model used its entire token budget on reasoning and returned no "
|
||||
"reply — raise max tokens, cap the reasoning budget in Settings, or "
|
||||
"use a non-reasoning model."
|
||||
"reply — raise max tokens or use a non-reasoning model."
|
||||
if reasoning_chunks
|
||||
else "The AI returned an empty response."
|
||||
)
|
||||
yield sse({"type": "error", "detail": detail})
|
||||
return
|
||||
|
||||
if cfg.using_demo:
|
||||
# Unmetered for power users (count_demo_turn is a no-op for them), but
|
||||
# keep the call so the accounting stays right if the gate ever widens.
|
||||
auth.count_demo_turn(user)
|
||||
db.commit()
|
||||
yield sse({
|
||||
"type": "done",
|
||||
"text": text,
|
||||
"reasoning": "".join(reasoning_chunks).strip() or None,
|
||||
"model": cfg.model,
|
||||
"model": model,
|
||||
})
|
||||
|
||||
|
||||
@router.post("/stream")
|
||||
def chat_stream(
|
||||
payload: schemas.ChatRequest,
|
||||
request: Request,
|
||||
db: Session = Depends(get_db),
|
||||
user: models.User = PowerUser,
|
||||
user: models.User = Depends(auth.get_current_user),
|
||||
):
|
||||
total = sum(len(m.content) for m in payload.messages)
|
||||
if total > schemas.CHAT_TOTAL_MAX:
|
||||
@@ -157,13 +115,12 @@ def chat_stream(
|
||||
413, f"This conversation is too long to send ({total:,} characters) — "
|
||||
"clear it or start a new one."
|
||||
)
|
||||
limits.rate_limit("chat", request, user)
|
||||
settings = get_settings(db, user)
|
||||
cfg, note = _resolve_model(settings, payload.model)
|
||||
if not cfg.model:
|
||||
model = (payload.model or "").strip() or settings.model
|
||||
if not model:
|
||||
raise HTTPException(400, "No model configured — set one in Settings or pick one here.")
|
||||
return StreamingResponse(
|
||||
run_chat(cfg, settings, payload, note, db, user),
|
||||
run_chat(settings, model, payload),
|
||||
media_type="text/event-stream",
|
||||
headers=SSE_HEADERS,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user