JesseMarkowitzandClaude Opus 5
ba737de9b4
Validates the three finalists by clone, build, test run and live local Ollama inference, then answers the fork question with measurements rather than static review. Recommendation: fork AI-DnD, confidence high. The Phase 0A call holds, but it was wrong that AI-DnD's undo is non-destructive — retry preserves the replaced take, undo hard-deletes it. A follow-up spike fixed that in 3 files (+130/-31): undo now moves a head cursor, redo round-trips, writing below a moved-back head forks and keeps the abandoned line, branch-scoped memory isolation survives, suite 627/632 with all 5 failures asserting the deleted-row behaviour that was replaced. Findings that change the plan: - AI-DnD cannot take a turn air-gapped as shipped; tiktoken fetches its encoding from a CDN. Proven on an internal Docker network, proven fixed by vendoring the file. - ai-adventure needs zero code for Ollama — two config lines — and its turn/head/checkpoint schema is the target model to build to. - Open Dungeon has zero automated tests and a positional summary watermark, making its branch retrofit larger than Phase 0A costed. - The world-state referee takes relative deltas; a 3B model sent absolute values under full context, so a wounded player ended at full health. Validation cannot catch this, so prefer ai-adventure's typed-event vocabulary when generalising narrative state. - Export/import recomputes head depth, so a round-trip silently undoes an undo. Must be fixed alongside the undo work. Docs only; no production code. Working tree from the runs stays untracked under phase0b/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015gUPLuxLs8wypxZPEmccJu
107 lines
4.9 KiB
Markdown
107 lines
4.9 KiB
Markdown
# Phase 0B — Offline and Network Behavior
|
|
|
|
Method: a Docker network created with `--internal` (no NAT, no DNS to the
|
|
outside). The Ollama container was attached to it so the app could reach a model
|
|
while having no path to the Internet. Isolation was verified from inside the
|
|
app container before testing:
|
|
|
|
```
|
|
blocked ('1.1.1.1', 443) OSError
|
|
blocked ('openrouter.ai', 443) gaierror
|
|
blocked ('fonts.googleapis.com', 443) gaierror
|
|
```
|
|
|
|
## AI-DnD — fails offline as shipped; fine once one file is vendored
|
|
|
|
First turn on the isolated network died. The SSE stream emitted the `player`
|
|
event and stopped. Container log:
|
|
|
|
```
|
|
requests.exceptions.ConnectionError: HTTPSConnectionPool(
|
|
host='openaipublic.blob.core.windows.net', port=443):
|
|
Max retries exceeded with url: /encodings/cl100k_base.tiktoken
|
|
(NameResolutionError ... Temporary failure in name resolution)
|
|
```
|
|
|
|
`tiktoken` downloads its BPE encoding on first use, and AI-DnD calls it for
|
|
context budgeting on every turn. On the host this was invisible, because the
|
|
file had already been cached to `/tmp/data-gym-cache/9b5ad71b…` during an
|
|
earlier online run.
|
|
|
|
After copying that 1.7 MB file into the container:
|
|
|
|
```
|
|
OFFLINE TURN GENERATED: "I am Vale, an explorer. I journey alone through the
|
|
darkened depths of the lighthouse..."
|
|
```
|
|
|
|
So: a hard blocker on a clean air-gapped install, and a packaging fix — vendor
|
|
the encoding (or pre-seed `TIKTOKEN_CACHE_DIR`, or replace the tokenizer). Worth
|
|
stressing that **static analysis could not have found this**; it took an actually
|
|
isolated run.
|
|
|
|
Other AI-DnD network surface:
|
|
|
|
- **Google Fonts at runtime.** The built SPA's `index.html` still contains
|
|
`fonts.googleapis.com/css2?family=Cinzel...&family=Crimson+Pro...&family=Inter...`,
|
|
and `main.py`'s CSP explicitly allows `fonts.googleapis.com` /
|
|
`fonts.gstatic.com`. Violates specification §12. Fix: self-host three families.
|
|
- Only other remote host in backend Python is `https://openrouter.ai/api/v1`,
|
|
used as a default endpoint constant and a header-attribution host check. Both
|
|
removable with the cloud-provider path.
|
|
- `analytics.py` is a **self-hosted** counter writing to two local tables. No
|
|
third-party script, no outbound request. It records no IP, no user agent, and
|
|
hashes the user id with HMAC. Removable, and not a telemetry leak in the
|
|
meantime.
|
|
- Connection test and turns honour the configured endpoint only; no automatic URL
|
|
retrieval or content fetching was observed.
|
|
|
|
## Open Dungeon — runtime clean, build and telemetry are not
|
|
|
|
- **Next.js telemetry is on by default** and printed its notice on first start.
|
|
Needs `NEXT_TELEMETRY_DISABLED=1` or `next telemetry disable` in the
|
|
production configuration.
|
|
- **Fonts are fine at runtime.** `layout.tsx` uses `next/font/google`, which
|
|
downloads at *build* time and self-hosts. The served page contains no
|
|
`fonts.googleapis.com` reference. The trade-off is that the build needs
|
|
network access.
|
|
- Remote hosts referenced in `src/`: `openrouter.ai` (preset provider URL and a
|
|
model list), plus `ko-fi.com` and `github.com/sponsors` donation links in the
|
|
UI. Everything else is `127.0.0.1` / `localhost` (13 occurrences).
|
|
- Local story play needs no cloud service: Ollama for text, a local FLUX worker
|
|
or the user's own ComfyUI for images.
|
|
- 335 npm packages with 6 high-severity advisories is the largest supply-chain
|
|
surface of the three.
|
|
|
|
## ai-adventure — verifiably local-only
|
|
|
|
The strongest posture by a wide margin, and the easiest to audit:
|
|
|
|
- **Exactly one outbound call site in the whole codebase**: `urlopen` in
|
|
`llm/lm_studio.py`. Nothing else in `local_adventure/` opens a socket.
|
|
- **No hardcoded remote host anywhere.** The only `http` string in the package is
|
|
the scheme check in `content/models.py`.
|
|
- **One direct dependency** (`pydantic`), six packages in the closure.
|
|
- It is the only candidate that already implements specification §12's
|
|
non-local-endpoint warning:
|
|
|
|
```python
|
|
def _endpoint_warnings(config):
|
|
hostname = urlparse(config.model.base_url).hostname
|
|
if hostname not in {"127.0.0.1", "localhost", "::1"}:
|
|
return ["model.base_url is not loopback; game prompts and content will
|
|
be sent to that endpoint. Enable API authentication."]
|
|
```
|
|
|
|
- `audit.store_prompts` defaults to `false`, with prompt *hashes* stored instead.
|
|
- Tests, world validation, session creation, replay, branching and export are all
|
|
offline by construction.
|
|
|
|
## Summary
|
|
|
|
| | Local play works offline | Cloud service required | Telemetry / remote assets | Removal difficulty |
|
|
|---|---|---|---|---|
|
|
| AI-DnD | **Only after vendoring the tiktoken encoding** | No | Google Fonts at runtime; local-only analytics tables | Low — one vendored file, three self-hosted fonts, delete analytics |
|
|
| Open Dungeon | Yes at runtime; build needs network | No | Next.js telemetry on by default | Low — one env var; fonts already self-hosted |
|
|
| ai-adventure | **Yes, unconditionally** | No | **None** | Nothing to remove |
|