Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M6: branch-safe context, summaries and long-term story memory

Browse Source 
Aligns the inherited AI-DnD memory and context foundation with the history,
authority and state model M3-M5 established. Long stories now reach the narrator
through a bounded, lineage-safe, inspectable context rather than a growing
transcript.

This commit includes the corrective work that followed the independent review in
planning/reports/M6-IMPLEMENTATION-REPORT.md. The first implementation reported
E03 as passing and it was not; the report records that history rather than
hiding it.

What was already correct, and was kept rather than rebuilt

  Memory lineage. Memories already carried (branch_id, depth) and retrieval
  already filtered through the capped-path clause; the ten-step negative control
  was measured passing against b7005e6 before any change here. M6 adds the
  regression tests that pin it, plus provenance and authority on the result.

Summary lineage — both halves

  A summary is a row carrying the coordinate of the last node it covers, and
  eligibility is the same head-capped lineage clause memories use. That alone
  was not enough: generation was seeded from adventures.story_summary, a
  campaign-global column with no lineage, so after a divergence the summariser
  was handed the abandoned line's prose and asked to update it. The row it
  produced was correctly anchored and therefore looked safe while its sentences
  described a story the reader had left.

  Generation is now seeded from summaries.current — the same question the
  context builder asks — so the input and the output are scoped by one rule.
  adventures.story_summary remains a reader-facing mirror for the Plot panel and
  the export bundle, kept in step when a summary is written and when the head
  moves, and nothing authoritative reads it.

Retrieval redundancy

  With a real embedding model, four near-identical memories crowded out the one
  distinctive clue, which survived only because the default memory_top_k is 5.
  Retrieval now drops a candidate that repeats one already chosen, never across
  authority classes, at a threshold measured against the configured embedding
  model. The clue is retrieved at top_k 5, 4 and 3. Ranking itself is unchanged;
  the further factors CONTEXT-AND-MEMORY §20 contemplates remain unimplemented
  and are recorded as such.

Memory authority, budgeting, observability

  Memory.authority is accepted_story or heuristic, classified by the application
  and marked in the prompt; retrieval never writes state. The reply is reserved
  out of the context budget, and an impossible configuration fails clearly
  instead of overflowing. Each derived pass records ok/idle/failed per campaign,
  served by GET /adventures/{id}/derived and shown in Insights, so the M2
  failure — a dead memory bank with a green suite — is visible if it recurs.
  Provider-wiring tests mock no factory.

Also: two pre-existing test-suite leaks fixed; two fixtures that stored one
vector in every memory now use distinct ones, so lineage assertions stay
readable alongside redundancy suppression.

Planning: CONTEXT-AND-MEMORY, TECHNICAL-DESIGN, DATA-MODEL, V1-ACCEPTANCE-TESTS,
BUILD-MILESTONES, VERSION and planning/README updated to describe what exists,
including that a valid E03 test must regenerate a summary after diverging. The
M5 report was rotated to planning/archive/milestone-reports/. No new ADR — every
choice implements a decision the package had already settled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PWU4gTfLYY6Qq9U7aa9Qw2
This commit is contained in:
JesseMarkowitz
2026-09-06 03:00:33 -04:00
co-authored by Claude Opus 5
parent b7005e6fdd
commit a6e9c7a32b
32 changed files with 4040 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/models.py
+104 -1
View File
@@ -2,7 +2,7 @@ from datetime import datetime, timezone
from sqlalchemy import (
JSON, Boolean, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
String, Text, event,
String, Text, UniqueConstraint, event,
)
from sqlalchemy.orm import Mapped, Session, mapped_column, relationship
@@ -98,6 +98,19 @@ class Adventure(Base):
memory: Mapped[str] = mapped_column(Text, default="")
authors_note: Mapped[str] = mapped_column(Text, default="")
ai_instructions: Mapped[str] = mapped_column(Text, default="")
# A convenience mirror of whichever summary is eligible at the current
# position, and **never** an input to anything authoritative (M6 corrective,
# review finding M6-F1).
#
# It exists because the Plot panel lets a reader read and edit the summary
# and the export bundle carries it. It is not a store: `summaries` rows are,
# and `summaries.current` decides which one the story is entitled to. This
# column has no lineage of its own, so anything that reads it as truth
# inherits whatever was written last, on whatever line — which is exactly
# how abandoned prose reached an active prompt before the correction.
#
# Kept in step by `summaries.record` when one is written and by
# `attempts.restore_state` when the head moves.
story_summary: Mapped[str] = mapped_column(Text, default="")
# Phase 18: who the player is playing as. The AI never writes these — they
# are user-only, which is what lets them sit in the cached system block
@@ -198,6 +211,17 @@ class Adventure(Base):
cascade="all, delete-orphan",
order_by="Memory.id",
)
# M6: the lineage-anchored generated summaries, newest last.
summaries: Mapped[list["Summary"]] = relationship(
back_populates="adventure",
cascade="all, delete-orphan",
order_by="Summary.id",
)
derived_status: Mapped[list["DerivedStatus"]] = relationship(
back_populates="adventure",
cascade="all, delete-orphan",
order_by="DerivedStatus.id",
)
class Branch(Base):
@@ -467,6 +491,14 @@ class Memory(Base):
# current. Readers need only the yes-or-no answer, and fetching six
# kilobytes of vector to get it is too expensive.
embedded: Mapped[bool] = mapped_column(Boolean, default=False)
# M6: how much weight the narrator should give this memory
# (`CONTEXT-AND-MEMORY.md` §14). `accepted_story` is something the story
# actually established; `heuristic` is an interpretation of it. The
# application owns this classification — the extractor may hint, but
# `memorybank.classify_authority` decides — so a guess can never become
# canon merely by being written down. Authoritative state changes still go
# only through the M5 event path (ADR 013).
authority: Mapped[str] = mapped_column(String(20), default="accepted_story")
pinned: Mapped[bool] = mapped_column(Boolean, default=False)
forgotten: Mapped[bool] = mapped_column(Boolean, default=False) # evicted, kept for UI
use_count: Mapped[int] = mapped_column(Integer, default=0)
@@ -476,6 +508,77 @@ class Memory(Base):
adventure: Mapped[Adventure] = relationship(back_populates="memories")
class Summary(Base):
"""M6: one generated rolling summary, anchored to the story it summarizes.
The inherited design kept the summary in a single `adventures.story_summary`
column with a lineage cursor recording how far it had read. The cursor was
lineage-aware; the prose was not. After an Undo and a divergence the column
still held sentences describing the abandoned line, and the context builder
injected it unconditionally — the leak `STORY-BRANCH-SEMANTICS.md` §32 and
acceptance test E03 forbid.
A summary is therefore a row on a path, exactly as a `Memory` is, and it is
filtered through the same `lineage.Path.clause` chokepoint. `branch_id` and
`depth` are the coordinate it was written at; `source_start`/`source_end`
are the stretch of story it covers. A summary whose coordinate is not on the
active capped lineage is not eligible, and is never deleted for it — the
abandoned line keeps its own derived data (§11).
"""
__tablename__ = "summaries"
id: Mapped[int] = mapped_column(primary_key=True)
adventure_id: Mapped[int] = mapped_column(ForeignKey("adventures.id", ondelete="CASCADE"))
text: Mapped[str] = mapped_column(Text, default="")
# The coordinate this summary was written at: the last node it covers.
branch_id: Mapped[int | None] = mapped_column(
ForeignKey("branches.id", ondelete="CASCADE"), nullable=True
)
depth: Mapped[int | None] = mapped_column(Integer, nullable=True)
# The stretch of story it summarizes, as depths on `branch_id`.
source_start: Mapped[int | None] = mapped_column(Integer, nullable=True)
source_end: Mapped[int | None] = mapped_column(Integer, nullable=True)
# Why it was generated: "interval" for the automatic pass, "manual" when the
# reader wrote or edited it themselves.
trigger: Mapped[str] = mapped_column(String(20), default="interval")
model_name: Mapped[str] = mapped_column(String(200), default="")
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
adventure: Mapped[Adventure] = relationship(back_populates="summaries")
class DerivedStatus(Base):
"""M6: the outcome of one kind of background derived work, per campaign.
M2 shipped with the whole memory bank dead and the suite green: the
summariser and the embedder raised inside a fire-and-forget task, and
nothing recorded it (`BUILD-MILESTONES.md`, note from M2). Derived work is
allowed to fail — the accepted turn, the state and the head must all
survive it — but it is not allowed to fail *invisibly*.
One row per (adventure, kind), rewritten in place. This is deliberately not
a job queue: it records what happened last, so a reader can see that
memories stopped being written and why, and so a maintainer can retry.
"""
__tablename__ = "derived_status"
__table_args__ = (UniqueConstraint("adventure_id", "kind", name="uq_derived_kind"),)
id: Mapped[int] = mapped_column(primary_key=True)
adventure_id: Mapped[int] = mapped_column(ForeignKey("adventures.id", ondelete="CASCADE"))
# "memory", "summary" or "embedding".
kind: Mapped[str] = mapped_column(String(20))
# "ok" (did work), "idle" (ran, nothing pending) or "failed".
status: Mapped[str] = mapped_column(String(20), default="ok")
detail: Mapped[str] = mapped_column(Text, default="")
failures: Mapped[int] = mapped_column(Integer, default=0)
last_attempt_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
last_success_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
adventure: Mapped[Adventure] = relationship(back_populates="derived_status")
class StoryCard(Base):
"""Owned by either a scenario or an adventure (exactly one set)."""