JesseMarkowitzandClaude Opus 5
d27ee34901
The planning package had grown to where a new agent could not tell what was authoritative. Phase 0 execution prompts sat beside the specification; four completed milestone reports sat beside the current one; and upstream AI-DnD's own `plan/` build log and `docs/` project site still described a hosted, scripted, multi-user product with accounts — every screenshot in it showed a Scripts tab and a Sign up button, none of which has existed since M2. `planning/archive/` now holds the history and says so in its own README: `phase0/` for the research that chose AI-DnD, `milestone-reports/` for M1 and M2, `decisions/` for ADR 008, the Phase-0-before-build gate Phase 0 satisfied. `planning/reports/` holds only the current milestone's report, because that is the one M4 planning has to read; it moves to the archive when M4's replaces it. Deleted rather than archived: the Phase 0B execution prompts and the handoff/status/summary documents, the Phase 0A discovery and triage reports, upstream's `plan/` and `docs/` trees, and `frontend/README.md`, which was Vite's template boilerplate. All of it is in Git history, and the two recommendation reports carry every conclusion the deleted research reached. Archived documents are kept verbatim. Paths written inside them point at where those files were when the document was written, which is the point: an evidence record that has been quietly edited is no longer evidence. Active documentation is corrected where it pointed at the removed trees or described removed capability as present. `DEVELOPMENT.md`'s "things M1 did not touch" list had gone stale at M2 and claimed QuickJS scripting was still tested; its test count was 604 against an actual 638. `README.md` loses the upstream CI badge, which reported upstream's pipeline rather than this fork's, and a reference to `backend/app/worldstate/engine.py`, a file that does not exist. `planning/README.md` is rewritten as the documentation index. New: `planning/PROJECT-SOURCES.md` and `planning/project-sources.txt`, the manifest of what belongs in the ChatGPT project's Sources. Source comments referring to the deleted trees are reworded; no behaviour changes. 638 backend tests pass, frontend lints and builds, and a reference scan over all 48 tracked Markdown files reports no unresolved path in active documentation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCbwH7yLGKsj1rhXXzKSCu
165 lines
7.6 KiB
Markdown
165 lines
7.6 KiB
Markdown
# Phase 0B — Follow-Up Checks
|
||
|
||
Four open questions settled after the undo/redo spike. Ordered by how much each
|
||
changes the plan.
|
||
|
||
---
|
||
|
||
## 1. The world-state referee under a local model
|
||
|
||
**Why it mattered:** the referee is the closest thing AI-DnD has to
|
||
specification §2's "the model proposes, the application owns authoritative
|
||
state", and §7's structured narrative state. It was never exercised in the main
|
||
round, because the RPG path was deliberately left empty to prove genre-neutrality.
|
||
If it only works with a cloud-class model, the principle the architecture rests
|
||
on is at risk locally.
|
||
|
||
**How it works.** `EMIT_RULE` asks for a fenced ` ```state ` block of **relative
|
||
deltas**, with an example (`{"player.hp": -15, ...}`) and a recency reminder
|
||
appended at the end of the prompt. `apply.py` computes `new = old + delta`, then
|
||
clamps by `max_delta_per_turn`, then by `min`/`max`, and reports every change as
|
||
applied / clamped / rejected with corrective text fed back to the model.
|
||
|
||
### Result A — in the full application, `qwen2.5:3b-instruct` sent absolute values
|
||
|
||
Playing the seeded "Bandit Camp (RPG world state)" scenario:
|
||
|
||
```
|
||
turn 1 proposed: {"player.hp": 92, "npc.gwen.trust": 25,
|
||
"player.outfit": "...arrow in shoulder", "npc.gwen.health": 90}
|
||
applied : player.hp 100 -> 100 "did not move. It is already at its
|
||
maximum of 100 ... at most 35 per turn"
|
||
gwen.trust 20 -> 40 (clamped from +25 to +20)
|
||
gwen.health 100 -> 100 (clamped)
|
||
outfit updated correctly
|
||
|
||
turn 2 proposed: {"npc.gwen.trust": 30}
|
||
applied : gwen.trust 40 -> 60 (clamped from +30 to +20)
|
||
```
|
||
|
||
The model meant "hp is now 92". The engine read `+92`, clamped it to `+35`, then
|
||
to the ceiling of 100. **The player took an arrow to the shoulder and finished
|
||
the turn at full health**, with the prose describing a wound.
|
||
|
||
This is the failure mode that matters: it is not corruption, and no validator can
|
||
catch it, because `+92` is a perfectly legal proposal. The application stayed
|
||
authoritative — which is the point of the referee — but the authoritative state
|
||
silently stopped tracking the narration.
|
||
|
||
The free-text stat (`player.outfit`) was handled correctly, because it is marked
|
||
`(free text)` and takes a whole value rather than a delta.
|
||
|
||
### Result B — in isolation, the same model gets it right
|
||
|
||
An isolated probe using AI-DnD's own `EMIT_RULE` and `extract_delta` verbatim, so
|
||
that only the model varies. A short stat guide, live values, one wound scene,
|
||
3 samples each:
|
||
|
||
| Model | Emitted a parsable state block | Correct relative (negative) hp |
|
||
|---|---|---|
|
||
| `qwen2.5:0.5b` | 0/3 | 0/3 |
|
||
| `qwen2.5:3b-instruct` | 3/3 | **3/3** (−20, −20, −15) |
|
||
| `qwen2.5:7b-instruct` | 3/3 | **3/3** (−15, −30, −15) |
|
||
|
||
So the delta protocol is **not** beyond a 3B model. The difference between A and
|
||
B is context: the full application prompt carries scenario prose, persona, a
|
||
stat guide, live values, story cards and history, and under that load the 3B
|
||
model degraded to absolute values. In a short prompt it followed the same
|
||
instruction perfectly.
|
||
|
||
> Recorded because it nearly became a wrong finding: the first run of this probe
|
||
> reported 0/3 for both models. That was a bug in the probe, not the models —
|
||
> `extract_delta` returns `(clean_text, delta)` and the harness was reading
|
||
> element 0. The table above is the corrected run.
|
||
|
||
Note also `qwen2.5:7b-instruct` sample 2 proposing `npc.gwen.trust: +75`, which
|
||
the referee would clamp to +20. Even a capable model proposes disproportionate
|
||
values, which is precisely what the clamp is for.
|
||
|
||
### What this means for the design
|
||
|
||
1. **Not a blocker for the fork.** The referee is per-scenario and opt-in, and we
|
||
are generalizing it into narrative state rather than adopting it as-is.
|
||
2. **Prefer an unambiguous state vocabulary.** A relative-delta protocol has a
|
||
failure mode that validation cannot detect, and it degrades with context
|
||
length. ai-adventure's typed events (`set_flag key=… value=…`) are explicit
|
||
and absolute, so the same mistake is impossible to express. **This is a
|
||
concrete argument for taking ai-adventure's event vocabulary rather than
|
||
AI-DnD's delta vocabulary when generalizing to genre-neutral narrative state.**
|
||
3. **Keep the emission rule close to generation.** AI-DnD already appends
|
||
`EMIT_REMINDER` at the end of the prompt for exactly this reason; whatever we
|
||
build should keep that property and be tested at realistic context length,
|
||
not just in a clean harness.
|
||
4. **Sample sizes are small** (3 per model; 2 parsed in-app turns). Enough to
|
||
establish the failure mode exists and that it is context-dependent, not enough
|
||
to rank models. A proper capability matrix belongs in the build phase.
|
||
|
||
---
|
||
|
||
## 2. Export/import silently redoes an undone story
|
||
|
||
**Status: a real defect the spike introduces into the export path. One-field fix.**
|
||
|
||
`bundle.py:_point_the_head` sets the head on import by recomputing it:
|
||
|
||
```python
|
||
depths = [n["depth"] for n in story["nodes"] if n["branch"] == head]
|
||
if depths:
|
||
adventure.head_depth = max(depths)
|
||
```
|
||
|
||
The export format carries `headBranch` (line 120) but **not** the head depth.
|
||
|
||
With the destructive undo this was correct — undone rows did not exist, so
|
||
`max(depths)` was the head. With the spike's non-destructive undo, an adventure
|
||
exported while its head sits behind the tip comes back with every abandoned turn
|
||
restored. The undo is silently undone by a round-trip.
|
||
|
||
**Fix:** add `headDepth` to the bundle (optional field, or a `v3`), and honour it
|
||
in `_point_the_head`, falling back to `max(depths)` when absent so that existing
|
||
`ai-dnd-adventure-v2` files still import. Small, but it must land in the same
|
||
change as the non-destructive undo, not after it.
|
||
|
||
---
|
||
|
||
## 3. Story cards are not lineage-scoped
|
||
|
||
**Status: a design constraint for imported knowledge, not a present defect.**
|
||
|
||
`models.StoryCard` is keyed by `scenario_id` **or** `adventure_id`, and carries
|
||
**no `branch_id` and no `depth`** — unlike `Memory`, which carries both and is
|
||
therefore filtered by the same capped lineage clause that gives memories their
|
||
branch isolation.
|
||
|
||
`context.builder.match_cards` is pure keyword matching over the adventure's whole
|
||
card list; `routers/story_cards.py` contains no lineage reference at all.
|
||
|
||
This is harmless today, because cards are authored by the user or copied from a
|
||
scenario — nothing derives a card from story content, so an abandoned branch
|
||
cannot invent one.
|
||
|
||
It stops being harmless the moment imported knowledge or any derived-card feature
|
||
lands. **Implication for `IMPORTED-KNOWLEDGE-DESIGN.md`:** any knowledge source
|
||
that can be created or updated from story content must carry `(branch_id, depth)`
|
||
like `Memory` does, and be read through `lineage.Path.clause`. Do that and it
|
||
inherits both the branch isolation and the undo isolation for free — which is the
|
||
strongest argument for a new table rather than extending `story_cards`.
|
||
|
||
---
|
||
|
||
## 4. Postgres/psycopg is cleanly removable
|
||
|
||
**Status: resolved, low effort.**
|
||
|
||
The entire Postgres surface is three places:
|
||
|
||
- `database.py` — the `AIDND_DATABASE_URL` / `DATABASE_URL` branch and
|
||
`_normalize_url`. Deleting the branch leaves the SQLite path, which is already
|
||
the default when neither variable is set.
|
||
- `migrations.py` — two helpers that tolerate SQLite returning a raw JSON string
|
||
where psycopg returns a parsed list. They simplify rather than disappear.
|
||
- `analytics.py` — `from sqlalchemy.dialects.postgresql import insert as pg_insert`,
|
||
which goes with analytics, already scheduled for removal.
|
||
|
||
`psycopg[binary]` then drops out of `requirements.txt`. No blockers.
|