Read a window of the story per turn instead of all of it
Two reads still grew without bound after the snapshot fix. `Action.variants` holds every discarded retry attempt, but a list response only needs how many there are — so each retry permanently added ~5 KB to every later load of that adventure. Defer the column and keep the count beside it (migration 37, backfilled server-side), with set_variants() as the one write path that keeps the two in step. `story_actions()` walked adventure.actions, then every caller threw almost all of it away: the builder concatenates the story and immediately cuts it back to the token budget, the NPC check looks at the last 6, retrieval at the last 4, the cursor clamp only wants a count. A turn on a 200-action adventure read 839 KB to use ~70 KB, and grew with every turn played. app/context/ history.py serves those shapes from SQL; window_covering() measures the actions it fetched and projects how many more it needs, fetching only the part it does not already hold. Memorybank cursors move to position_of_index() and settled_count()/settled_slice() — same arithmetic, no full list. The scripting pipeline still receives the whole history per AI Dungeon's API, and every helper reuses adventure.actions when it is already loaded, so a scripted adventure pays what it always did and never twice. Measured at production shape: retry tax 5.1 KB -> 0; turn 200 839 KB -> 129 KB and flat from ~turn 50; a 200-turn playthrough 84.5 MB -> 23.0 MB; a delete 115 KB -> 5 KB. Verified the window builds a byte-identical prompt to the full story across budgets from 1K to 100K tokens, with and without the retry exclusion - this is a cost change and nothing else. Cursor helpers checked against the old list arithmetic, including after deleting a middle action. Counts are real SELECT count(...): Query.count() wraps the entity select in a subquery, so the SQL named every deferred column and the egress guard could not tell it apart from a bulk fetch. 139 tests pass; the four new guards verified by sabotage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeQVy5bEjLhfgWNc27Efet
This commit is contained in:
co-authored by
Claude Opus 5
parent
f1bd099ec8
commit
47e33fa311
@@ -234,7 +234,15 @@ class Action(Base):
|
||||
# is its own only version. `variant_index` says which entry `text`,
|
||||
# `reasoning` and `context_snapshot` currently mirror; retry appends and
|
||||
# points here instead of deleting the row, so nothing is lost.
|
||||
variants: Mapped[list | None] = mapped_column(JSON, nullable=True)
|
||||
#
|
||||
# Deferred for the same reason as context_snapshot: a list response only
|
||||
# ever needs the *count* (see variant_count below), but the column holds
|
||||
# every discarded attempt's full narration, so loading it in bulk made each
|
||||
# retry a permanent tax on every later page load of that adventure.
|
||||
variants: Mapped[list | None] = mapped_column(JSON, nullable=True, deferred=True)
|
||||
# len(variants), maintained on write by set_variants() so the deferred
|
||||
# column above never has to be fetched just to count it. 0 = never retried.
|
||||
variant_count: Mapped[int] = mapped_column(Integer, default=0)
|
||||
variant_index: Mapped[int] = mapped_column(Integer, default=0)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=utcnow)
|
||||
|
||||
@@ -268,12 +276,6 @@ class Action(Base):
|
||||
out.append({"kind": "stat", "label": label, "delta": delta, "value": new})
|
||||
return out
|
||||
|
||||
@property
|
||||
def variant_count(self) -> int:
|
||||
"""How many attempts exist for this turn. 0 (not 1) when the action was
|
||||
never retried — the UI shows its pager only above 1 either way."""
|
||||
return len(self.variants) if isinstance(self.variants, list) else 0
|
||||
|
||||
|
||||
class Script(Base):
|
||||
__tablename__ = "scripts"
|
||||
|
||||
Reference in New Issue
Block a user