Initial commit: AI Dungeon clone (FastAPI backend + React frontend)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFsGHju9szibJJa2YJcdbg
This commit is contained in:
parththakkar106
2026-07-06 16:08:19 +05:30
co-authored by Claude Fable 5
commit db9f904222
57 changed files with 7804 additions and 0 deletions
+74
View File
@@ -0,0 +1,74 @@
# AI D&D — Local AI Dungeon Clone: Plan Overview
A locally hosted web app replicating AI Dungeon's core experience: scenarios, adventures,
AI-driven storytelling, AI Dungeon-style memory/context management, JavaScript scripting
(compatible with real AI Dungeon scripts), and full transparency into what is sent to the AI.
## Confirmed decisions
| Area | Decision |
|---|---|
| Backend | Python — FastAPI + SQLAlchemy + SQLite (single-user, local) |
| Frontend | React SPA (Vite), dark AI Dungeon-like theme |
| AI provider | Provider-agnostic adapter layer; first adapter: **OpenAI-compatible** (`/v1/chat/completions`) — covers Ollama, LM Studio, OpenAI, OpenRouter, vLLM, Groq. Endpoint URL, API key, model name all configurable at runtime. |
| Scripting | **JavaScript, AI Dungeon-compatible** (`onInput` / `onModelContext` / `onOutput` modifiers, shared `state`, `worldEntries` API) via an embedded JS engine (quickjs / py-mini-racer). Real AI Dungeon scripts should import and run. |
| Import/export | AI Dungeon-compatible formats for scripts and scenarios; JSON export/import for everything. |
## Architecture at a glance
```
frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI
├─ routers/ (scenarios, adventures, actions, scripts, settings, insights)
├─ models/ (SQLAlchemy: Scenario, Adventure, Action, StoryCard, Script, Settings)
├─ context/ (prompt assembly: memory, author's note, world info, history budget)
├─ scripting/ (JS sandbox, AI Dungeon API surface, per-adventure state)
├─ providers/ (base adapter + openai_compatible.py; streaming)
└─ data.db (SQLite)
```
## Core domain model
- **Scenario** — template: title, description, opening prompt (with `${placeholders}`), memory,
author's note, story cards (world info), attached scripts, tags.
- **Adventure** — a playthrough created from a scenario (or blank). Owns its own copy of memory,
author's note, story cards, script state, and the action list.
- **Action** — one entry in the story: type (`do` / `say` / `story` / `continue` / AI output),
text, timestamp, plus the **context snapshot** (exact prompt sent to the AI) for Insights.
- **Story Card / World Info** — keys (comma-separated keywords), entry text, optional type/notes.
Injected into context only when a key matches recent story text.
- **Script** — JS source per hook (input / context / output modifier), attachable to scenarios;
copied into adventures with persistent `state`.
## The turn pipeline (heart of the app)
```
player input
→ onInput script modifier
→ store player action
→ assemble context: [AI instructions] + [plot essentials] + [story summary]
+ [triggered story cards ("World Lore:")] + [story history, token-budgeted]
+ [author's note inserted N lines from the end] + [player action]
→ onModelContext script modifier
→ snapshot context (Insights)
→ provider adapter → AI (streamed)
→ onOutput script modifier
→ store AI action → render
```
## Phases
1. **[Phase 1 — Foundation](01-phase-foundation.md)**: repo scaffold, FastAPI + SQLite models,
React shell, scenario/adventure CRUD, settings (endpoint config).
2. **[Phase 2 — Play loop + AI](02-phase-play-loop.md)**: provider adapter with streaming,
Do/Say/Story/Continue, Retry/Undo/Edit, the adventure play screen.
3. **[Phase 3 — Context engine + Insights](03-phase-context-insights.md)**: memory, author's note,
story cards with keyword triggering, token budgeting, per-turn prompt snapshots + Insights UI.
4. **[Phase 4 — Scripting](04-phase-scripting.md)**: embedded JS sandbox, AI Dungeon scripting API,
script editor, script + scenario import/export (AI Dungeon-compatible).
5. **[Phase 5 — Polish](05-phase-polish.md)**: AI Dungeon-like theming pass, placeholders on
scenario start, adventure export/import, quality-of-life and hardening.
6. **[Phase 6 — Auto Summarization + Memory Bank](06-phase-memory-bank.md)** *(optional)*:
modern AI Dungeon memory system — AI-generated memories every 6 actions, Story Summary every
15, embedding-based retrieval of relevant memories into context.
Each phase ends with the app runnable and testable end-to-end.
+38
View File
@@ -0,0 +1,38 @@
# Phase 1 — Foundation
**Goal:** runnable skeleton — backend serving a database-backed API, frontend shell with
navigation, scenario & adventure CRUD, and a settings page for the AI endpoint. No AI calls yet.
## Backend
- [x] Project scaffold: `backend/` with FastAPI app, uvicorn entrypoint, `requirements.txt`
(fastapi, uvicorn, sqlalchemy, pydantic, httpx, tiktoken; quickjs deferred to phase 4).
- [x] SQLite via SQLAlchemy; auto-create `data.db` on first run.
- [x] Models:
- `Scenario`: id, title, description, prompt, memory, authors_note, tags, created/updated.
- `StoryCard`: id, owner (scenario or adventure), keys, entry, type, title, description.
- `Adventure`: id, scenario_id (nullable), title, memory, authors_note, script_state (JSON),
created/updated.
- `Action`: id, adventure_id, index, type (`do|say|story|continue|ai|start`), text,
context_snapshot (JSON, nullable), created.
- `Script`: id, name, description, input_js, context_js, output_js, library_js.
- `Settings`: single row — endpoint_url, api_key, model, temperature, max_output_tokens,
context_token_budget, api_mode (`chat|completion`).
- [x] Routers: CRUD for scenarios, adventures (+ create-from-scenario copying memory/AN/cards),
story cards, settings. Consistent JSON errors.
- [x] CORS for the Vite dev server; production mode serves built frontend as static files.
## Frontend
- [x] Vite + React scaffold in `frontend/`; router with pages: Home (adventure list),
Scenarios (list + editor), Play (placeholder), Settings.
- [x] Dark base theme (AI Dungeon-like: near-black background, serif story font, gold/teal accent).
- [x] Scenario editor: title, description, prompt, memory, author's note, story card list editor.
- [x] Settings page: endpoint URL, API key, model name, sampling params; "Test connection" button
(backend proxies a trivial request — wired for real in Phase 2, stub now).
- [x] "New adventure" flow: pick scenario (or blank) → creates adventure → navigates to Play page.
## Exit criteria
Run `uvicorn` + `npm run dev`, create/edit/delete scenarios with story cards, start an adventure
from one, see it listed on Home, and save endpoint settings — all persisted across restarts.
+45
View File
@@ -0,0 +1,45 @@
# Phase 2 — Play loop + AI provider
**Goal:** the core game is playable. Player acts, AI continues the story, streamed live.
## Provider adapter layer
- [x] `providers/base.py`: abstract `Provider` — `generate(prompt_parts, params) -> async stream of text`.
Takes an assembled context object (system text + story text), so providers decide how to
map it to their wire format.
- [x] `providers/openai_compatible.py`:
- Chat mode: system message carries instructions/memory; story history flows in as
user/assistant text continuation framing suited to a chat endpoint. A configurable
"narrator" system prompt frames the AI as a second-person storyteller continuing the text.
- Streaming via SSE from the endpoint, re-streamed to the browser.
- Optional raw completion mode (`/v1/completions`) for pure-continuation models.
- [x] Errors surfaced cleanly (bad key, connection refused, model not found) with retry affordance.
- [x] "Test connection" on Settings now real.
## Turn engine (`POST /adventures/{id}/actions`)
- [x] Input formatting per AI Dungeon conventions:
- **Do** → `> You <text>` (normalized to second person, stripped punctuation as needed)
- **Say** → `> You say "<text>"`
- **Story** → raw text appended
- **Continue** → no player text; AI just continues
- [x] Simple context for this phase: opening prompt + full history, truncated from the top to the
token budget (tiktoken count). Real context engine lands in Phase 3.
- [x] Response streamed to the client via SSE; final text stored as an `ai` action.
- [x] **Retry**: delete last AI action, regenerate with same input.
- [x] **Undo/Erase**: delete last action pair (player + AI) or single action.
- [x] **Edit**: PATCH any action's text in place.
## Play UI
- [x] Story view: continuous prose (not chat bubbles), player actions styled distinctly
(`>` prefix, accent color), auto-scroll, streaming text renders token-by-token.
- [x] Input bar with mode selector (Do / Say / Story) + Continue button; Enter to send.
- [x] Per-turn controls: Retry, Undo, Edit (inline contenteditable or textarea swap).
- [x] Loading/streaming state, error toast with retry.
## Exit criteria
Point Settings at any OpenAI-compatible endpoint (e.g. Ollama or LM Studio locally), start an
adventure, and play a multi-turn story with all four input modes plus retry/undo/edit, with
streaming output.
+62
View File
@@ -0,0 +1,62 @@
# Phase 3 — Context engine + Insights
**Goal:** AI Dungeon-grade context management, and full visibility into every prompt.
## Context assembly (`context/builder.py`)
Assembles the prompt each turn from AI Dungeon's **plot components**
(per help.aidungeon.com/faq/the-memory-system):
```
[AI Instructions] ← behavioral guidance for the model (always included)
[Plot Essentials] ← key facts for constant recall — the classic "Memory" (always)
[Story Summary] ← running summary slot; manual in this phase, auto in Phase 6
[Triggered Story Cards] ← "World Lore: <entry>" for each triggered card (conditional)
[Story history] ← as many recent actions as fit the token budget
[Author's Note] ← injected N lines (default 3) before the end of history
[Latest player action] ← (+ script frontMemory right after it, Phase 4)
```
- [x] **AI Instructions / Plot Essentials / Story Summary / Author's Note**: adventure-level
free-text fields, always included, editable mid-adventure from the side panel.
- [x] **Story cards** — five fields per official docs: **Type** (organizational, not sent to AI),
**Name** (not sent to AI), **Entry** (sent when triggered), **Triggers**, **Notes** (not sent).
- Triggers: comma-separated words/phrases; **case-insensitive but space-sensitive**;
**partial-word matching** (`boat` triggers on `boats`); matched against both player input
and AI output in the recent-story window.
- Not instant: a card triggered mid-response only enters context on the *next* turn; once
triggered, stays active while the triggering text remains in the context window.
- Triggered entries injected once each, prefixed `World Lore:`; story cards are the **first
component dropped** when context is full.
- Editable per-adventure (copied from scenario at creation). Soft-cap sanity limit (AI Dungeon
allows 5,000/adventure).
- [x] **Author's Note**: inserted near the end (strongest steering position), formatted
`[Author's note: <text>]`.
- [x] **Token budgeting** with tiktoken: always-included components (AI Instructions, Plot
Essentials, Story Summary, Author's Note) reserved first; story cards get a capped share
and are dropped first when over budget; remainder goes to story history (newest first).
Budget = `context_token_budget` setting.
- [x] Slots for script-provided memory overrides (Phase 4): `state.memory.context` (prepended),
`state.memory.authorsNote` (replaces/augments author's note), `state.memory.frontMemory`
(inserted immediately after the latest player action).
- [x] Builder returns a structured `ContextReport`: ordered sections, each with source label,
text, token count; plus totals and a list of triggered cards (and which keyword fired).
## Insights
- [x] Every AI turn stores its `ContextReport` on the `Action` row (`context_snapshot`).
- [x] `GET /adventures/{id}/actions/{id}/context` returns it.
- [x] **Insights panel** in Play UI (drawer/tab):
- Exact final prompt text as sent, sectioned and color-coded (memory / world info / history /
author's note / input), with per-section token counts and total vs budget.
- Which story cards triggered and on which keyword; which history got cut off.
- Viewable for the *upcoming* turn (dry-run endpoint: "what would be sent now") and for any
past AI action.
- [x] Adventure side panel: edit memory, author's note, story cards mid-game (AI Dungeon's
right-hand panel equivalent).
## Exit criteria
Create a card with key `dragon`; mention a dragon in play and see the card enter the context in
the Insights panel (and influence the AI); verify memory and author's note appear in the snapshot
in the right positions; long adventures visibly trim oldest history within budget.
+84
View File
@@ -0,0 +1,84 @@
# Phase 4 — Scripting (AI Dungeon-compatible JavaScript)
**Goal:** real AI Dungeon scripts import and run: the three modifier hooks, persistent `state`,
and the scripting API surface.
## JS runtime
- [x] Embed a JS engine in Python: **quickjs** (preferred; check Windows wheel availability at
implementation time; fallback: py-mini-racer, or Node subprocess as last resort).
- [x] Sandbox matching AI Dungeon's documented limits: each hook runs **isolated**, **16 MB
memory cap**, **2-second timeout**; no filesystem/network/process access; script errors
captured and surfaced in the UI, never crash a turn.
## AI Dungeon scripting model (compatibility target)
*(Per official docs: help.aidungeon.com/faq/how-do-i-write-scripts-and-use-scripting)*
Three lifecycle hooks — `onInput`, `onModelContext`, `onOutput`. Each script defines a modifier
and **must call it as its last line**:
```javascript
const modifier = (text) => {
// script logic
return { text, stop }
}
modifier(text)
```
- **onInput** — modifies player input before context construction.
- **onModelContext** — modifies the assembled text sent to the model.
- **onOutput** — modifies the model output before it is shown/stored.
- **Shared Library** — code prepended to all three slots.
Return contract:
- [x] `{ text, stop }`; `stop: true` from onInput prevents the AI call.
- [x] Empty-string `text` from onInput/onOutput → user-facing error (replicate this behavior).
Globals provided (exact names from docs):
- [x] `text` — hook input (player input / context / AI response respectively).
- [x] `state` — persisted per adventure across turns (`Adventure.script_state`); includes
`state.memory`, `state.message` (shown as a UI notice), `state.placeholders`.
- [x] `state.memory` slots: `context` (prepended to context), `authorsNote` (near end, before
latest response), `frontMemory` (inserted right after the player's input).
- [x] `history` — array of recent actions: `{ text, rawText, type }`.
- [x] `storyCards` — array of `{ id, keys, entry, type }`, backed by the adventure's story cards.
- [x] Story card functions: `addStoryCard(keys, entry, type)` → index (or `false` on duplicate),
`updateStoryCard(index, keys, entry, type)` and `removeStoryCard(index)` → throw if absent.
- [x] Legacy aliases for older scripts: `worldInfo` / `worldEntries`, `addWorldEntry`,
`updateWorldEntry`, `removeWorldEntry` mapped onto the storyCards implementation.
- [x] `info` — `{ actionCount, characterNames, memoryLength, maxChars }`.
- [x] `log(message)` / `console.log` — captured per turn, shown in a script log panel.
## Pipeline integration
```
player input → INPUT modifier → format & store
context build → CONTEXT modifier → (snapshot includes pre- and post-script versions in Insights)
AI response → OUTPUT modifier → store & render
```
- [x] Insights (Phase 3) extended: show context before vs after the context modifier (diff view),
and script log output per turn.
## Script management UI
- [x] Scripts page: create/edit scripts with a code editor (CodeMirror), one tab per slot
(Library / Input / Context / Output), description field.
- [x] Attach scripts to scenarios; adventures inherit at creation. Enable/disable per adventure.
- [x] Test-run a script against sample text without an AI call.
## Import / Export
- [x] **Scripts**: export/import as JSON bundle `{ name, library, input, context, output }` and
as raw `.js` files per slot (matching how AI Dungeon scripts circulate — paste or file).
- [x] **Scenarios**: export/import JSON including prompt, memory, author's note, story cards,
and attached scripts. Accept AI Dungeon scenario export JSON where format is known;
map fields best-effort and report anything unmapped.
## Exit criteria
Paste a real AI Dungeon script (e.g. a simple input modifier + state counter + world entry
manipulation) and it runs unmodified across turns; state persists; export a scenario with scripts,
re-import it into a fresh database, and play it.
+38
View File
@@ -0,0 +1,38 @@
# Phase 5 — Polish & quality of life
**Goal:** the app feels like AI Dungeon — cohesive dark UI, smooth flows, safe data handling.
## UI/UX pass
- [x] Theming: refined dark palette, serif story typography, subtle textures/gradients à la
AI Dungeon; consistent buttons, panels, modals; responsive layout.
- [x] Home: adventure cards with scenario name, last-played time, action count; search/filter;
scenario gallery with tags.
- [x] Play screen: collapsible right side panel (Memory / Cards / Scripts / Insights tabs),
keyboard shortcuts (Enter send, Ctrl+Z undo, Ctrl+R retry), smooth streaming autoscroll
that pauses when the user scrolls up.
- [x] Scenario **placeholders**: `${Character name}` style variables in prompt/memory prompt the
player for values when starting an adventure (AI Dungeon behavior).
## Data & robustness
- [x] Adventure export/import (full JSON: actions, memory, cards, script state) — backup/share.
- [x] Delete confirmations (trash/soft-delete skipped — plain confirm dialogs).
- [x] SQLite migrations story (versioned schema bootstrap via PRAGMA user_version —
`backend/app/migrations.py`).
- [x] Request logging + a debug page tailing recent provider requests/responses (bodies redacted
of API key) — "Recent AI requests" on the Settings page.
- [x] Graceful handling: provider timeout/cancel (stop generation button), concurrent turn lock
per adventure.
## Nice-to-haves (only if time/interest)
- [ ] Multiple provider profiles with quick switching (e.g. local Ollama vs OpenRouter).
- [ ] Per-scenario generation params overriding global settings.
- [ ] Retry with "give me something different" (temperature bump / anti-repeat nudge).
- [ ] Basic light theme toggle.
## Exit criteria
A friend could sit down at `localhost`, start a scenario with placeholders, play comfortably,
peek at Insights, and you can back up / restore everything via export files.
+47
View File
@@ -0,0 +1,47 @@
# Phase 6 — Auto Summarization + Memory Bank (optional)
**Goal:** replicate modern AI Dungeon's Memory System (per
help.aidungeon.com/faq/the-memory-system): AI-generated memories, a running Story Summary, and
embedding-based retrieval. This phase makes extra AI calls (summarization + embeddings), so it is
opt-in per adventure and gated on the endpoint supporting it.
## Auto Summarization
- [x] **Memories**: every 6 actions (starting at action 12), summarize that block of
player actions + AI responses into a short "memory" via a background AI call
(same provider, cheap/configurable model override).
- [x] **Story Summary**: every 15 actions, update the Story Summary plot component — a running
overview of the plot — folding in recent memories; compress it when it grows too long.
- [x] Story Summary stays **manually editable**; user edits inform future updates
(they are the base text for the next summarization pass) but are never overwritten silently.
- [x] Summarization failures are non-fatal: log, retry next interval.
(Failed calls appear on the debug page; cursors only advance on success, so the
next turn retries. Implementation: `backend/app/memorybank.py`.)
## Memory Bank
- [x] Store each memory with an **embedding vector** (OpenAI-compatible `/v1/embeddings`;
embedding model configurable in Settings; feature disabled if unavailable).
- [x] Each turn, embed the recent story text and rank memories by cosine similarity;
inject the top-K "Used Memories" into context as their own component
(between Story Summary and story cards in the layout). Pinned memories are always
included; top-K is a setting (default 5).
- [x] Configurable bank capacity (AI Dungeon tiers: 25–400; ours: a setting, default 200);
when full, evict least-recently-used/least-retrieved memories ("Forgotten Memories").
- [x] SQLite storage for vectors (JSON blob + in-memory cosine ranking — pure Python,
no numpy needed at this scale).
## UI
- [x] Memory Bank panel: list memories (used / idle / forgotten), edit or delete, pin favorites,
see which memories were retrieved for a given turn (🔍 on an AI action → Insights snapshot).
- [x] Insights integration: retrieved memories shown as a context section with similarity scores.
- [x] Adventure settings: toggle auto-summarization / memory bank (per adventure, in the Memory
panel); summary + embedding models are chosen globally in Settings (deliberate
simplification — one endpoint config for the whole app).
## Exit criteria
Play a 40+ action adventure: memories appear every 6 actions, the Story Summary updates every 15,
an early-game fact that scrolled out of the raw history gets retrieved via the Memory Bank when it
becomes relevant again, and Insights shows exactly which memories were injected and why.