Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ba737de9b438582727160cdb9622b7851fc20300
interactive-story/planning/TEST-CAMPAIGN-FIXTURE.md
T

999 lines
20 KiB
Markdown
Raw Blame History

# Adventure Storyteller — Standard Test Campaign Fixture
**Status:** Draft v0.1
**Purpose:** Provide a deterministic, reusable campaign fixture for Phase 0B candidate comparison and later v1 regression testing.
## 1. Fixture Name
```text
Continuity Test
```
## 2. Purpose
This fixture is designed to expose failures in:
- canon handling,
- character knowledge,
- possession continuity,
- relationship continuity,
- location continuity,
- long-term memory,
- imported knowledge authority,
- branch/undo safety,
- checkpoint restore,
- abandoned-history leakage,
- summary lineage,
- state reconstruction,
- prompt/context provenance.
It is intentionally small.
The goal is not to create an entertaining campaign. The goal is to create a compact story that is easy to verify.
## 3. Campaign Profile
```yaml
title: Continuity Test
genre: fantasy
subgenre: low fantasy
tone: grounded, tense, restrained
style: clear narrative prose
point_of_view: second person
tense: present
```
## 4. Narrator Rules
Use the following durable narrator rules:
```text
1. Do not decide the protagonist's voluntary actions unless required to describe
the immediate consequence of an action already chosen by the user.
2. Preserve established canon and accepted story state.
3. Do not reveal hidden information unless the protagonist has learned it
through accepted story events.
4. Do not treat imported Reference or Inspiration material as campaign canon.
5. If uncertain about an established fact, avoid contradicting it.
6. Keep responses concise enough for testing. Prefer approximately 2-5 paragraphs
unless the user explicitly asks for more.
7. Magic exists, but resurrection is impossible.
8. Do not introduce modern technology.
```
## 5. Initial Protagonist
### Aldric
```yaml
name: Aldric
type: character
role: protagonist
description: >
A traveling investigator accustomed to dangerous roads and old ruins.
current_location: Crooked Lantern Tavern
condition:
- healthy
goals:
- find Edrin
possessions:
- Silver Key
```
### Visual Profile
```yaml
apparent_age: late 30s
build: lean
hair: dark brown
clothing: weathered green traveling cloak
distinctive_features:
- narrow scar across left eyebrow
```
## 6. Supporting Characters
### Mara
```yaml
name: Mara
type: character
role: tavern keeper
current_location: Crooked Lantern Tavern
relationship_to_aldric: cautious trust
knows:
- Edrin disappeared recently
- Edrin often visited the Old Abbey
does_not_know:
- the Silver Key was found in Edrin's desk
- Aldric currently possesses the Silver Key, until Aldric reveals it
```
### Hidden Canon — Mara
```text
Mara once saw the same broken-circle symbol on a sealed cellar door beneath
the Crooked Lantern.
Mara has not told anyone about the cellar door.
Mara is not a spy.
```
This hidden fact is intended to test:
- narrator-only knowledge,
- delayed revelation,
- abandoned-branch leakage.
### Edrin
```yaml
name: Edrin
type: character
role: missing scholar
current_location: unknown
status: missing
```
### Hidden Canon — Edrin
```text
Edrin discovered that the Silver Key opens the sealed cellar door beneath
the Crooked Lantern.
Edrin disappeared before he could tell Mara.
```
The user should not know this at campaign start.
## 7. Locations
### Crooked Lantern Tavern
```yaml
name: Crooked Lantern Tavern
type: location
description: >
An old timber-framed tavern near the north road. It has a stone hearth,
dark beams, shared tables, and a cellar beneath the main room.
```
Hidden location fact:
```text
A sealed cellar door beneath the tavern bears a broken-circle symbol.
```
### Old Abbey
```yaml
name: Old Abbey
type: location
description: >
A ruined abbey five miles north of Westhaven. Its crypt bears a
broken-circle symbol.
```
## 8. Item
### Silver Key
```yaml
name: Silver Key
type: item
description: >
A small silver key bearing a broken-circle symbol.
current_owner: Aldric
origin: Edrin's desk
```
Hidden function:
```text
The Silver Key opens the sealed cellar door beneath the Crooked Lantern.
```
## 9. Initial Relationships
```text
Aldric -> trusts -> Mara
Mara -> cautiously_trusts -> Aldric
Mara -> knows -> Edrin
Edrin -> frequently_visited -> Old Abbey
Aldric -> possesses -> Silver Key
```
## 10. Initial Story Thread
```yaml
title: Find Edrin
status: open
description: Determine what happened to Edrin.
```
## 11. Global Canon
These facts are authoritative from campaign start.
```text
1. Magic exists.
2. Resurrection is impossible.
3. The Old Abbey lies five miles north of Westhaven.
4. The Silver Key was found in Edrin's desk.
5. The Silver Key bears a broken-circle symbol.
6. The Old Abbey crypt bears the same broken-circle symbol.
7. Mara has never visited the Old Abbey.
8. Mara does not initially know where the Silver Key was found.
9. Mara is not a spy.
10. Aldric begins the campaign carrying the Silver Key.
```
## 12. Imported Knowledge Files
Create exactly these three files.
---
### File A — `canon.md`
Classification:
```text
Canon
```
Contents:
```markdown
# Campaign Canon
The Old Abbey lies five miles north of Westhaven.
The abbey crypt bears a symbol shaped like a broken circle.
Magic exists in this world, but resurrection is impossible.
Mara has never visited the Old Abbey.
```
Expected behavior:
- treated as authoritative,
- may be retrieved selectively,
- cannot be overridden by lower-authority sources.
---
### File B — `reference.md`
Classification:
```text
Reference
```
Contents:
```markdown
# Tavern Reference
Medieval roadside taverns commonly used timber framing, stone hearths,
wooden benches, shared tables, candles, and oil lamps.
Cellars were often used for ale, food storage, and secure storage.
Old buildings frequently accumulated renovations, blocked passages, and
sealed storage areas over generations.
```
Expected behavior:
- may influence environmental detail,
- must not establish that the Crooked Lantern definitely has a secret tunnel,
- must not override campaign canon.
---
### File C — `inspiration.md`
Classification:
```text
Inspiration
```
Contents:
```markdown
# Atmospheric Inspiration
A traveler entered a silent hall while rain tapped against dark shutters.
A single lantern illuminated the room.
Beneath an old house, a forgotten doorway waited behind a wall of barrels.
A frightened innkeeper concealed a dangerous political secret from a stranger.
```
Expected behavior:
- may influence atmosphere,
- must not establish that Mara is concealing a political secret,
- must not turn Mara into a spy,
- must not create a hidden doorway unless accepted story events establish one.
## 13. Deliberate Continuity Traps
The fixture contains several traps.
### Trap 1 — Mara's knowledge
Mara initially does not know:
- where the key was found,
- that Aldric possesses it.
If the narrator gives Mara this knowledge before Aldric reveals it, continuity failed.
### Trap 2 — Mara has never visited Old Abbey
If Mara claims personal experience inside the abbey without a later accepted explanation, canon failed.
### Trap 3 — Inspiration says an innkeeper hides a political secret
Mara is explicitly not a spy.
If Inspiration causes the narrator to establish Mara as a political spy, authority handling failed.
### Trap 4 — Reference describes sealed passages
This is descriptive reference only.
It must not automatically create unrelated secret tunnels.
### Trap 5 — Resurrection
Any retrieved text or pretrained knowledge suggesting resurrection must lose to global canon.
### Trap 6 — Abandoned branch secret
One test branch will reveal:
- Mara has seen the broken-circle symbol in the cellar.
After undo/divergence, the new active branch must not know that revelation occurred.
### Trap 7 — Possession
Aldric starts with the Silver Key.
The key must not vanish or move owners without an accepted event.
## 14. Initial Expected State
At campaign creation:
```yaml
active_location: Crooked Lantern Tavern
aldric:
possesses:
- Silver Key
knows:
- Silver Key was found in Edrin's desk
- Edrin is missing
mara:
knows:
- Edrin is missing
- Edrin often visited Old Abbey
does_not_know:
- key origin
- Aldric possesses key
threads:
- Find Edrin: open
```
## 15. Core Scripted Test Sequence
The following sequence should be used for candidate comparison.
The narrator's exact prose will vary.
The important part is state and continuity.
---
## Turn 1
User:
```text
I enter the Crooked Lantern and look for Mara.
```
Expected:
- Mara is present or plausibly becomes available.
- Current location remains Crooked Lantern.
- No hidden canon is revealed automatically.
---
## Turn 2
User:
```text
I ask Mara, "Have you heard anything about Edrin?"
```
Expected:
- Mara may say Edrin is missing.
- Mara may mention his interest in Old Abbey.
- Mara should not mention the Silver Key unless Aldric reveals it.
- Mara should not mention the cellar symbol yet.
---
## Turn 3
User:
```text
I ask whether Mara has ever been to the Old Abbey.
```
Expected:
- Mara says no or equivalent.
- Any statement that she personally visited the abbey is a failure.
Create named checkpoint:
```text
Before revealing the key
```
Expected checkpoint state:
- Mara still does not know Aldric has the key.
- Mara still does not know where it was found.
---
## Turn 4A — Primary Test Path
User:
```text
I show Mara the Silver Key but do not tell her where I found it.
```
Expected state:
- Mara now knows Aldric possesses the Silver Key.
- Mara still does not know it came from Edrin's desk.
Expected narrative opportunity:
- Mara may recognize the broken-circle symbol.
- If she reveals she has seen it beneath the tavern, that becomes accepted story knowledge.
For deterministic testing, if the narrator does not volunteer the recognition, continue with:
```text
Does the symbol mean anything to you?
```
Expected:
- Mara may reveal that she saw the symbol on a sealed cellar door.
- This revelation is now accepted on Path A.
Record this as:
```text
Path A Secret Revealed:
Mara has seen the broken-circle symbol beneath the tavern.
```
---
## Turn 5A
User:
```text
I tell Mara that I found the key in Edrin's desk.
```
Expected state:
- Mara now knows the key origin.
- New clue may connect Edrin, key, and tavern cellar.
---
## Turn 6A
User:
```text
I ask Mara to take me to the cellar door.
```
Expected:
- location may change into tavern cellar,
- active thread may gain clue,
- key remains with Aldric unless explicitly handed over.
## 16. Undo / Divergence Test
After completing Path A through Turn 6A:
Restore checkpoint:
```text
Before revealing the key
```
Expected:
- active story returns to post-Turn-3 state,
- Mara does not know Aldric has key,
- Mara does not know key origin,
- accepted Path A future becomes abandoned/disposable,
- checkpoint remains.
Create new continuation.
---
## Turn 4B — Divergent Path
User:
```text
I decide not to mention the key. I ask Mara what she remembers about Edrin's last visit.
```
Expected:
- Mara does not know Aldric has the key.
- Mara does not know the key origin.
- the prior Path A revelation about the cellar symbol must not be treated as something already said.
---
## Turn 5B
User:
```text
I ask whether Edrin ever spoke about unusual symbols.
```
Expected:
- narrator may choose a plausible response consistent with current canon,
- must not phrase the Path A cellar revelation as something already discussed,
- may reveal the cellar symbol now if the narrator decides it is narratively appropriate.
Important:
If the system retrieves:
```text
Mara already told Aldric about the cellar symbol.
```
from abandoned Path A, the candidate fails lineage safety.
## 17. Redo Test
Before entering Turn 4B, test ordinary Redo after checkpoint restore/undo if supported.
Expected:
- Redo may return into Path A only until a new Turn 4B is accepted.
- once Turn 4B is accepted, ordinary Redo into Path A should be invalidated.
Path A remains retained/disposable internally.
## 18. Retry Test
On a fresh/appropriate turn, use:
```text
I open the cellar door.
```
Receive narrator Take A.
Then Retry.
Receive narrator Take B.
Expected:
- both narrator takes are retained,
- user may select either before continuing,
- continuing from Take B makes Take A inactive/disposable,
- Take A does not influence later active context.
## 19. Narrator Edit Test
Create a narrator response containing:
```text
Mara wears a red cloak.
```
Edit it to:
```text
Mara wears a green cloak.
```
Expected:
- green cloak becomes active continuity,
- old red-cloak version remains historical/disposable,
- structured visual/state data is re-evaluated if the system tracks clothing.
## 20. Manual State Correction Test
Introduce or simulate an incorrect fact:
```text
Mara knows the Silver Key came from Edrin's desk.
```
Use manual state/canon correction:
```text
Mara does not know where the Silver Key was found.
```
Expected:
- correction is authoritative,
- correction has user/manual provenance,
- future narrator behavior respects correction.
## 21. Long-Term Memory Plant
Near the beginning of the active branch, establish:
```text
Mara says Edrin always tapped twice on the table before mentioning something he feared.
```
This detail is intentionally minor but specific.
Do not mention it for many turns.
At least 30-50 turns later, ask:
```text
I think back to Mara's description of Edrin when he was frightened. Was there any distinctive habit she mentioned?
```
Expected:
- system should retrieve or reconstruct the two-tap habit,
- full transcript should not need to be in prompt.
This is the standard long-term memory recall fact.
## 22. Promise Memory Plant
Establish:
```text
Aldric promises Mara he will return before sunrise.
```
Later, after enough turns for direct context to expire, ask or create a situation near dawn.
Expected:
- promise should be retrievable as a high-value commitment memory.
## 23. Possession Transfer Test
Later in the campaign:
User:
```text
I hand the Silver Key to Mara and ask her to hold it.
```
Expected:
- Mara now possesses key,
- Aldric no longer possesses key.
Several turns later:
```text
I reach for the Silver Key.
```
Expected:
- narrator should not act as if Aldric still has it.
Then:
```text
I ask Mara to give the key back.
```
Expected:
- possession returns to Aldric after accepted transfer.
## 24. Location Continuity Test
Move from tavern to Old Abbey.
Expected:
- active location updates.
Ask about an object clearly located at the tavern without returning.
Expected:
- narrator should not imply the protagonist is still physically at the tavern.
## 25. Resurrection Canon Test
Introduce:
```text
I ask whether any known magic could bring Edrin back if we find him dead.
```
Expected:
- narrator maintains resurrection is impossible,
- lower-authority inspiration/reference/pretrained knowledge cannot override this.
## 26. Reference Retrieval Test
At tavern:
```text
I look around the room carefully. What is the place physically like?
```
Expected:
- reference may contribute timber framing, hearth, benches, lighting,
- narrator should not say these details came from campaign canon unless actually established.
## 27. Inspiration Authority Test
Because Inspiration contains:
```text
A frightened innkeeper concealed a dangerous political secret.
```
ask:
```text
I watch Mara carefully. Does she seem like someone involved in political intrigue?
```
Expected:
- narrator may describe ambiguity,
- must not establish Mara is a spy solely because Inspiration contains that trope,
- global canon says Mara is not a spy.
## 28. Hidden Canon / Spoiler Test
Before the active branch discovers the sealed cellar door, ask:
```text
What do I know about the purpose of the Silver Key?
```
Expected player-facing answer:
- Aldric does not yet know its purpose.
The narrator may have hidden canon saying it opens the cellar door, but must not reveal this as player knowledge.
## 29. Checkpoint Persistence Test
Create named checkpoint:
```text
Before entering the abbey
```
Restart application.
Expected:
- checkpoint still exists,
- restoring it recreates correct state.
## 30. Export / Import Test
After:
- at least one abandoned path,
- at least two named checkpoints,
- imported knowledge,
- possession changes,
- long-term memories,
export campaign.
Import into a fresh data directory.
Expected:
- active transcript restored,
- current state restored,
- checkpoints restored,
- disposable history retained,
- imported knowledge classifications retained,
- memory provenance retained where required.
## 31. Science-Fiction Variant
The same engine should also run this compact alternate fixture without schema changes.
Campaign:
```text
Persephone Test
```
Canon:
```text
1. FTL does not exist.
2. Persephone is a fusion-powered survey ship.
3. Artificial gravity is available only through thrust or rotation.
4. Dr. Vale has never visited Europa.
5. The encrypted data crystal belongs to Captain Imani.
```
Entities:
- Captain Imani — protagonist
- Dr. Vale — scientist
- Persephone — vehicle
- Ceres Station — location
- Europa — location
- encrypted data crystal — item
- Helios Dynamics — organization
Purpose:
- verify genre-neutral entities,
- verify hard technology canon,
- verify possession,
- verify character knowledge,
- verify reference retrieval.
No database/schema changes should be required relative to Continuity Test.
## 32. Expected State Checkpoints
### Checkpoint S0 — Campaign Start
```yaml
location: Crooked Lantern Tavern
key_owner: Aldric
mara_knows_key_possession: false
mara_knows_key_origin: false
mara_cellar_symbol_revealed_to_aldric: false
find_edrin: open
```
### Checkpoint S1 — Before Revealing Key
Same as S0, after initial conversation.
```yaml
mara_knows_key_possession: false
mara_knows_key_origin: false
```
### Checkpoint S2A — After Showing Key
```yaml
key_owner: Aldric
mara_knows_key_possession: true
mara_knows_key_origin: false
```
### Checkpoint S3A — After Revealing Origin
```yaml
mara_knows_key_possession: true
mara_knows_key_origin: true
```
### Checkpoint S2B — Divergent Path
After restoring S1 and continuing without mentioning key:
```yaml
key_owner: Aldric
mara_knows_key_possession: false
mara_knows_key_origin: false
```
Path A revelations must not appear as accepted Path B state.
## 33. Required Test Evidence
For candidate comparison, record at important turns:
- active turn/head ID,
- visible transcript,
- current structured state,
- summary text,
- retrieved memories,
- retrieved imported chunks,
- prompt/context inspection,
- database lineage if accessible,
- checkpoint IDs,
- network activity if security test is running.
## 34. Candidate Comparison Procedure
For each finalist:
1. create the same Continuity Test fixture,
2. use the same imported files,
3. follow the same scripted turns where supported,
4. record deviations,
5. do not compensate manually for missing architecture unless the purpose is a documented experiment.
Rate each capability:
```text
PASS
PARTIAL
FAIL
NOT IMPLEMENTED
```
## 35. Fixture Success Criteria
A production v1 implementation passes the fixture if:
- no canon trap is violated,
- Mara's knowledge remains correct,
- possession remains correct,
- Undo/Restore returns to correct state,
- abandoned Path A facts do not leak into Path B,
- Retry alternatives do not contaminate active history,
- named checkpoints persist,
- old planted memories can be retrieved,
- imported Reference and Inspiration remain lower authority than Canon,
- hidden canon is not exposed prematurely,
- export/import preserves active and retained history,
- science-fiction variant requires no schema redesign.
## 36. Fixture Files
The canonical fixture package should eventually contain:
```text
continuity-test/
├── campaign.yaml
├── canon.md
├── reference.md
├── inspiration.md
├── expected-state.yaml
├── scripted-turns.md
└── README.md
```
For Phase 0B, this document is sufficient to create those files manually or through a small fixture setup script.
## 37. Current Recommendation
Use this fixture as the standard narrative test bed throughout the project.
Do not replace it with ad hoc stories for each candidate.
A stable fixture makes it possible to distinguish:
```text
model randomness
```
from:
```text
actual persistence / memory / canon bugs
```
The prose may vary.
The expected state, authority, and lineage rules should not.
Reference in New Issue View Git Blame Copy Permalink