Add Phase 0B local validation findings and recommendation
Validates the three finalists by clone, build, test run and live local Ollama inference, then answers the fork question with measurements rather than static review. Recommendation: fork AI-DnD, confidence high. The Phase 0A call holds, but it was wrong that AI-DnD's undo is non-destructive — retry preserves the replaced take, undo hard-deletes it. A follow-up spike fixed that in 3 files (+130/-31): undo now moves a head cursor, redo round-trips, writing below a moved-back head forks and keeps the abandoned line, branch-scoped memory isolation survives, suite 627/632 with all 5 failures asserting the deleted-row behaviour that was replaced. Findings that change the plan: - AI-DnD cannot take a turn air-gapped as shipped; tiktoken fetches its encoding from a CDN. Proven on an internal Docker network, proven fixed by vendoring the file. - ai-adventure needs zero code for Ollama — two config lines — and its turn/head/checkpoint schema is the target model to build to. - Open Dungeon has zero automated tests and a positional summary watermark, making its branch retrofit larger than Phase 0A costed. - The world-state referee takes relative deltas; a 3B model sent absolute values under full context, so a wounded player ended at full health. Validation cannot catch this, so prefer ai-adventure's typed-event vocabulary when generalising narrative state. - Export/import recomputes head depth, so a round-trip silently undoes an undo. Must be fixed alongside the undo work. Docs only; no production code. Working tree from the runs stays untracked under phase0b/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015gUPLuxLs8wypxZPEmccJu
This commit is contained in:
JesseMarkowitz
co-authored by
Claude Opus 5
Claude Opus 5
parent
f011362494
commit
ba737de9b4
@@ -0,0 +1,106 @@
|
||||
# Phase 0B — Offline and Network Behavior
|
||||
|
||||
Method: a Docker network created with `--internal` (no NAT, no DNS to the
|
||||
outside). The Ollama container was attached to it so the app could reach a model
|
||||
while having no path to the Internet. Isolation was verified from inside the
|
||||
app container before testing:
|
||||
|
||||
```
|
||||
blocked ('1.1.1.1', 443) OSError
|
||||
blocked ('openrouter.ai', 443) gaierror
|
||||
blocked ('fonts.googleapis.com', 443) gaierror
|
||||
```
|
||||
|
||||
## AI-DnD — fails offline as shipped; fine once one file is vendored
|
||||
|
||||
First turn on the isolated network died. The SSE stream emitted the `player`
|
||||
event and stopped. Container log:
|
||||
|
||||
```
|
||||
requests.exceptions.ConnectionError: HTTPSConnectionPool(
|
||||
host='openaipublic.blob.core.windows.net', port=443):
|
||||
Max retries exceeded with url: /encodings/cl100k_base.tiktoken
|
||||
(NameResolutionError ... Temporary failure in name resolution)
|
||||
```
|
||||
|
||||
`tiktoken` downloads its BPE encoding on first use, and AI-DnD calls it for
|
||||
context budgeting on every turn. On the host this was invisible, because the
|
||||
file had already been cached to `/tmp/data-gym-cache/9b5ad71b…` during an
|
||||
earlier online run.
|
||||
|
||||
After copying that 1.7 MB file into the container:
|
||||
|
||||
```
|
||||
OFFLINE TURN GENERATED: "I am Vale, an explorer. I journey alone through the
|
||||
darkened depths of the lighthouse..."
|
||||
```
|
||||
|
||||
So: a hard blocker on a clean air-gapped install, and a packaging fix — vendor
|
||||
the encoding (or pre-seed `TIKTOKEN_CACHE_DIR`, or replace the tokenizer). Worth
|
||||
stressing that **static analysis could not have found this**; it took an actually
|
||||
isolated run.
|
||||
|
||||
Other AI-DnD network surface:
|
||||
|
||||
- **Google Fonts at runtime.** The built SPA's `index.html` still contains
|
||||
`fonts.googleapis.com/css2?family=Cinzel...&family=Crimson+Pro...&family=Inter...`,
|
||||
and `main.py`'s CSP explicitly allows `fonts.googleapis.com` /
|
||||
`fonts.gstatic.com`. Violates specification §12. Fix: self-host three families.
|
||||
- Only other remote host in backend Python is `https://openrouter.ai/api/v1`,
|
||||
used as a default endpoint constant and a header-attribution host check. Both
|
||||
removable with the cloud-provider path.
|
||||
- `analytics.py` is a **self-hosted** counter writing to two local tables. No
|
||||
third-party script, no outbound request. It records no IP, no user agent, and
|
||||
hashes the user id with HMAC. Removable, and not a telemetry leak in the
|
||||
meantime.
|
||||
- Connection test and turns honour the configured endpoint only; no automatic URL
|
||||
retrieval or content fetching was observed.
|
||||
|
||||
## Open Dungeon — runtime clean, build and telemetry are not
|
||||
|
||||
- **Next.js telemetry is on by default** and printed its notice on first start.
|
||||
Needs `NEXT_TELEMETRY_DISABLED=1` or `next telemetry disable` in the
|
||||
production configuration.
|
||||
- **Fonts are fine at runtime.** `layout.tsx` uses `next/font/google`, which
|
||||
downloads at *build* time and self-hosts. The served page contains no
|
||||
`fonts.googleapis.com` reference. The trade-off is that the build needs
|
||||
network access.
|
||||
- Remote hosts referenced in `src/`: `openrouter.ai` (preset provider URL and a
|
||||
model list), plus `ko-fi.com` and `github.com/sponsors` donation links in the
|
||||
UI. Everything else is `127.0.0.1` / `localhost` (13 occurrences).
|
||||
- Local story play needs no cloud service: Ollama for text, a local FLUX worker
|
||||
or the user's own ComfyUI for images.
|
||||
- 335 npm packages with 6 high-severity advisories is the largest supply-chain
|
||||
surface of the three.
|
||||
|
||||
## ai-adventure — verifiably local-only
|
||||
|
||||
The strongest posture by a wide margin, and the easiest to audit:
|
||||
|
||||
- **Exactly one outbound call site in the whole codebase**: `urlopen` in
|
||||
`llm/lm_studio.py`. Nothing else in `local_adventure/` opens a socket.
|
||||
- **No hardcoded remote host anywhere.** The only `http` string in the package is
|
||||
the scheme check in `content/models.py`.
|
||||
- **One direct dependency** (`pydantic`), six packages in the closure.
|
||||
- It is the only candidate that already implements specification §12's
|
||||
non-local-endpoint warning:
|
||||
|
||||
```python
|
||||
def _endpoint_warnings(config):
|
||||
hostname = urlparse(config.model.base_url).hostname
|
||||
if hostname not in {"127.0.0.1", "localhost", "::1"}:
|
||||
return ["model.base_url is not loopback; game prompts and content will
|
||||
be sent to that endpoint. Enable API authentication."]
|
||||
```
|
||||
|
||||
- `audit.store_prompts` defaults to `false`, with prompt *hashes* stored instead.
|
||||
- Tests, world validation, session creation, replay, branching and export are all
|
||||
offline by construction.
|
||||
|
||||
## Summary
|
||||
|
||||
| | Local play works offline | Cloud service required | Telemetry / remote assets | Removal difficulty |
|
||||
|---|---|---|---|---|
|
||||
| AI-DnD | **Only after vendoring the tiktoken encoding** | No | Google Fonts at runtime; local-only analytics tables | Low — one vendored file, three self-hosted fonts, delete analytics |
|
||||
| Open Dungeon | Yes at runtime; build needs network | No | Next.js telemetry on by default | Low — one env var; fonts already self-hosted |
|
||||
| ai-adventure | **Yes, unconditionally** | No | **None** | Nothing to remove |
|
||||
Reference in New Issue
Block a user