Jesse.Markowitz/interactive-story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M10: the seam for media, and no media
CI / Backend tests (push) Canceled after 0s
Details
CI / Frontend lint + build (push) Canceled after 0s
Details
CI / Docker image builds (push) Canceled after 0s
Details

Browse Source 
The media extension contract asks for a scene snapshot a future image or video
provider could be handed: location, who is present, what they hold, what must
stay true, and where in the story it sits. Building one was the milestone's
obvious first task, and it was the wrong one. That snapshot has existed since
M5. `narrative_state["scene"]` holds the summary, the location, the cast and the
coordinate it was written at; a validated `set_scene` event writes it, every
position snapshots it, and every head move restores it. It survives Undo, Redo,
Retry, divergence, Save Point restore and a process restart because it is the
authoritative state rather than a copy of it.

So there is no scenes table here. A second scene store would have been a second
answer to "where is the story now", with its own lineage rules to get wrong —
and the lineage rules are the expensive part, which is the argument for reusing
the ones that already work rather than against it. The Scene Packet is derived
on read, and its identity is computed from the campaign and the position rather
than allocated: the same position yields the same id in another process, after a
restart, and after the packet is thrown away and rebuilt, with no row to keep in
step. That is the part of a future media_assets table that would be expensive to
retrofit, so it is fixed now even though the table is not built.

One table, then: visual_profiles, the only thing the contract's scene list asks
for that nothing already stored. Campaign-scoped and not per-position, because a
character does not change appearance when the story forks — a reader who
diverged would otherwise lose their cast, and the same descriptors would land in
every per-position snapshot, measured at 245 copies of 367 bytes in a 120-turn
campaign to say something that never varies. Keyed by the M5 entity key rather
than a new identity namespace, and one table for characters, locations and items
alike, because a location is an entity with a type and splitting them would
reintroduce the genre shape M5 spent a milestone removing.

What the packet leaves out is the more interesting half. Not the transcript, and
not imported knowledge — none of it, not merely the sources marked hidden. The
rule is what the story established at this position, not everything the narrator
was told, and drawing it by class is what makes it hold for a secret nobody
thought to mark. A hidden Canon source proves it, with a positive control
showing the narrator did receive the sentinel the packet does not carry. Once a
validated event puts the observer in the room, the observer is in the packet:
that is no longer narrator-only knowledge, and a packet that hid it would be
hiding the story from itself.

The providers are contracts and nothing else. Protocols for image, video, audio,
speech and transcription, an empty registry, no adapter, no dependency, no
socket, and no media setting to point anywhere — a setting that exists can be
pointed at a cloud by mistake. A future provider endpoint must be loopback,
stricter than narration's trusted-LAN allowance, because a picture of a scene
carries the scene with it. Transcription returns an editable draft with no
commit method, so STT structurally cannot bypass the authoritative path.

Nothing here can write the story. Not by convention: no module under media/
imports the code that writes state, no media event type exists in the state
vocabulary, and every test in the authority suite compares the authoritative
document byte for byte either side of a media operation — including one where a
provider insists Alice is in a red coat in a corridor, and the campaign goes on
disagreeing.

One defect, found by the milestone's own tests. M10 first added a migration
creating an index that create_all already builds from the column, so an upgraded
database ended up with two indexes and a fresh install with one. Comparing the
two schemas is what caught it; neither database examined alone would have. The
migration is gone rather than renamed, and the right number of migrations for a
new table whose indexes are declared on its columns is zero.

Backend 1,191 passed / 14 skipped / 0 failed, 89 of them M10's. Frontend 145
passed. Lint, production build and Docker build clean. No frontend file changed:
M10 adds no reader-facing surface, and ordinary play — turns, state, memory,
knowledge, Undo, Redo, Retry, Save Point restore, restart — runs with no media
configuration, no warning, no connection attempt and no media row written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qyn3oRd4D6pi72nKBG725B
This commit is contained in:
JesseMarkowitz
2026-09-07 03:41:04 -04:00
co-authored by Claude Opus 5
parent 44edece67e
commit 1013c94eb1
27 changed files with 5235 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/media/__init__.py
+66
View File
@@ -0,0 +1,66 @@
"""M10: the seam a future media provider plugs into, and nothing behind it.
This package is **readiness, not media**. Nothing here generates an image, a
video, audio, speech or a transcription; nothing here opens a socket; nothing
here is required for the storyteller to run. A campaign plays exactly as it did
in M9 with none of this configured, which is M10's central acceptance
condition — see `test_m10_no_media.py`.
## What M10 found already built, and therefore did not build again
The largest finding of the milestone is how little of it needed inventing.
`MEDIA-EXTENSION-CONTRACT.md` §5 asks the story system to persist a structured
scene snapshot with a campaign, a lineage, a source position, a location and the
characters present. **All of that already exists**, and has since M5:
state["scene"] = {"summary": …, "location": <entity key>,
"present": [<entity keys>],
"at": {"branch_id": …, "depth": …}}
written only by the validated `set_scene` typed event (ADR 010), snapshotted per
node in `actions.narrative_state_after` (M5), restored on every head movement by
`attempts.restore_state` (M3/M4), and carried per position in the M9 v3 bundle.
So it is already authoritative, already lineage-safe, already survives Undo,
Redo, Save Point restore, divergence and restart, and already round-trips into a
clean data directory.
Building a `scenes` table beside that would have been a second representation of
information the application already stores authoritatively — the one thing the
M10 brief forbids — and it would have needed its own lineage rules, its own
restore path and its own bundle carriage, each a chance to disagree with the
state document. **So M10 stores no scene rows.** It reads the scene that is
already there.
## What was actually missing
Three things, and this package is each of them:
* `profiles.py` — **visual profiles.** Stable descriptors for how an entity
*looks*, which nothing recorded. Campaign-scoped rather than per-position,
because a character does not change appearance when the story forks (K02, K03).
* `packet.py` — **the Scene Packet.** A bounded, provider-neutral,
hidden-information-safe view of one scene, built on demand from authoritative
state. Persisted nowhere, because it is a pure function of things that are.
* `providers.py` — **the provider contracts.** Types and protocols for image,
video, audio, TTS and STT, with no provider vocabulary anywhere in them, plus
the loopback-only endpoint rule the media contract asks for.
## The authority direction, which never reverses
accepted story -> narrative state -> scene packet -> future provider
Every arrow points away from authority. A visual profile is not a story fact; a
scene packet is a read; a future asset would be a depiction. Nothing in this
package writes `narrative_state`, emits a state event, or moves the head — and
`test_m10_authority.py` asserts that by running each operation and comparing the
authoritative document byte for byte either side.
That is the rule `MEDIA-EXTENSION-CONTRACT.md` §35 and §49 state, and the reason
it is enforced structurally rather than by convention: the only code that may
change authoritative state is the M5 event pipeline, and nothing here imports
it.
"""
from . import packet, profiles, providers
__all__ = ["packet", "profiles", "providers"]
backend/app/media/packet.py
+328
View File
@@ -0,0 +1,328 @@
"""M10: the Scene Packet — one accepted scene, bounded, for a future provider.
`MEDIA-EXTENSION-CONTRACT.md` §10-12 asks for a normalised, provider-independent
description of a scene, and asks explicitly that a provider **not** normally
receive the campaign transcript. This module builds that description.
## It is constructed, never stored
A packet is a pure function of things that are already persisted: the
authoritative state document at a position, the entity records inside it, and
the campaign's visual profiles. Storing one would create a second copy of all of
that, which could then disagree with the first — and the packet has no field the
source of truth does not already hold.
So there is no `scene_packets` table, nothing to migrate, nothing to keep in
step with the head, and nothing to carry in a bundle. Rebuilding it costs one
state read and one profile query. That is the same reasoning M9 applied to the
FTS index and the knowledge passages, applied to a smaller thing.
## Scene identity, without a scenes table
`MEDIA-EXTENSION-CONTRACT.md` §10 shows a `scene_id`, and the M10 brief asks
that a future asset be able to name unambiguously:
campaign -> lineage/story position -> source turn or turn range -> scene
That is a **coordinate**, and the application already has one. So the identity
is derived rather than allocated:
c<adventure>:b<branch>:<start>-<end>
Two properties follow, and both matter more than a surrogate key would have:
* it is **stable** — the same scene yields the same id on any machine, before
and after an export, without a row having to travel;
* it is **resolvable** — a future asset holding this string can be turned back
into the exact accepted position it depicts, with no lookup table.
A surrogate `scene_id` would have needed a table, a lineage column, a restore
path and bundle carriage, all to name something the coordinate already names.
## Ranges, because a video is not a turn
`build` takes a range, not a position. §30-31 of the contract describe a video
covering several accepted turns, and the M10 brief is explicit that neither
"one turn == one scene" nor "one scene == one asset" may be assumed.
So `start` and `end` are depths on one branch, the identity carries both, and a
single-turn image is the case where they are equal rather than a different kind
of request. Several future assets may name the same identity; nothing here
allocates or records them, so nothing constrains how many there are.
## What is deliberately not in a packet
**The transcript.** Not a summarised version of it either. The packet carries
the scene's own summary — the one sentence the story itself accepted through
`set_scene` — and the entities present. A provider that needs to depict a room
does not need to have read the campaign.
**Imported knowledge, of any class.** Not canon, not reference, not
inspiration, and emphatically not a narrator-only source. This is the hidden
information boundary and it is drawn structurally: this module never reads
`knowledge_sources`, so there is no filter to get wrong and no marker to
overlook. A secret reaches a packet only if the *story* put it into accepted
state through a validated event — which is the correct rule, because at that
point it is something that happened rather than something the narrator knows.
**Memories and summaries.** Derived narrative text about the campaign's past,
which is not what depicting a present moment needs.
**Facts, relationships and threads.** These are the campaign's reasoning about
itself. A `continuity_constraints` list carries the few that bear on depiction —
what a character is holding, where they are — and nothing else.
The result is that the honest answer to "what could leak through a packet" is
"what the accepted scene contains", which is what a picture of that scene would
show anyway.
"""
from __future__ import annotations
from sqlalchemy.orm import Session
from .. import models
from ..context import lineage
from ..narrative import model as narrative_model
from ..narrative import store as narrative_store
from . import profiles as visual_profiles
#: How many entities one packet will describe. A scene is a moment with people
#: in it; a request naming two hundred is a runaway state document rather than a
#: picture, and the bound keeps a future provider's prompt finite.
MAX_CHARACTERS = 24
MAX_OBJECTS = 24
MAX_CONSTRAINTS = 24
def scene_id(adventure_id: int, branch_id: int | None, start: int, end: int) -> str:
"""The derived, stable identity for one scene. See the module docstring."""
branch = branch_id if branch_id is not None else 0
return f"c{adventure_id}:b{branch}:{start}-{end}"
def parse_scene_id(value: str) -> dict | None:
"""Turns a scene identity back into the coordinate it names, or `None`.
The half that makes the derived identity worth having: a future asset
holding this string can be resolved to an accepted position without a table.
"""
try:
campaign, branch, span = str(value).split(":")
start, end = span.split("-")
return {
"adventure_id": int(campaign.lstrip("c")),
"branch_id": int(branch.lstrip("b")),
"start": int(start),
"end": int(end),
}
except (ValueError, AttributeError):
return None
def build(
db: Session,
adventure: models.Adventure,
*,
start: int | None = None,
end: int | None = None,
) -> dict:
"""The Scene Packet for a range of accepted story on the active branch.
Defaults to the scene at the active head, which is the ordinary case: an
image of what is happening now. `start` and `end` are depths on the active
branch; passing both describes a stretch, which is what a future video
would ask for.
Reads. Writes nothing, and cannot: this module imports no writer, emits no
event and does not touch the head. `test_m10_authority.py` asserts the
authoritative document is byte-identical either side of a build.
"""
state = narrative_store.current(adventure)
scene = state.get("scene") if isinstance(state.get("scene"), dict) else {}
branch_id = adventure.head_branch_id
head_depth = adventure.head_depth
# The scene's own coordinate is the position `set_scene` last ran at, which
# is where the depiction belongs. It can sit behind the head — the story may
# have moved on without re-establishing the scene — and that is correct: the
# picture is of the moment the scene was set, not of a later turn that did
# not change it.
at = scene.get("at") if isinstance(scene.get("at"), dict) else {}
scene_branch = at.get("branch_id") if at.get("branch_id") is not None else branch_id
scene_depth = at.get("depth") if _is_int(at.get("depth")) else head_depth
first = start if _is_int(start) else scene_depth
last = end if _is_int(end) else max(first, scene_depth)
if last < first:
first, last = last, first
profiles = visual_profiles.by_key(db, adventure)
location_key = scene.get("location") if isinstance(scene.get("location"), str) else None
present = [k for k in (scene.get("present") or []) if isinstance(k, str)]
return {
"scene_id": scene_id(adventure.id, scene_branch, first, last),
"campaign": {"id": adventure.id, "title": adventure.title},
# Where in the story this is, in the vocabulary the application already
# uses internally. A future provider does not read these; a future
# coordinator resolving an asset back to its source does.
"turn_range": {"branch_id": scene_branch, "start": first, "end": last},
"lineage": _lineage_of(db, adventure),
"location": _entity_view(state, profiles, location_key),
"characters": [
view for key in present[:MAX_CHARACTERS]
if (view := _entity_view(state, profiles, key)) is not None
],
"objects": _objects(state, profiles, present, location_key),
"action_summary": str(scene.get("summary") or ""),
"continuity_constraints": _constraints(state, present, location_key),
# Present, empty, and deliberately so — see `_ambience`.
"ambience": _ambience(scene),
"source": {
# What produced this, so a future asset's provenance can say which
# build's rules bounded the packet it was made from.
"packet_version": PACKET_VERSION,
"head_depth": head_depth,
},
}
#: The packet's own shape version. A future provider adapter can branch on it if
#: the packet gains fields; nothing in the story engine reads it.
PACKET_VERSION = 1
def _lineage_of(db: Session, adventure: models.Adventure) -> list[dict]:
"""The capped lineage this scene sits on, as provenance.
Read through `lineage.path_of`, the same helper every story read uses, so a
packet cannot describe a position the story could not. M10 builds no media
head: there is one head, and this follows it.
"""
try:
path = lineage.path_of(db, adventure)
except Exception: # noqa: BLE001 - a packet is a read; it does not raise
return []
entries = getattr(path, "entries", None)
if not entries:
return []
return [
{"branch_id": branch_id, "through_depth": cap}
for branch_id, cap in entries
]
def _entity_view(state: dict, profiles: dict, key: str | None) -> dict | None:
"""One entity as a packet describes it: what it is, plus how it looks."""
if not key:
return None
found = narrative_model.entity(state, key)
if found is None:
return None
return {
"key": key,
"name": narrative_model.entity_name(state, key),
"type": found.get("type") or "other",
"status": found.get("status") or "active",
"description": found.get("description") or "",
# `None` rather than an empty profile, so a provider can tell "nobody
# said how this looks" from "somebody said it looks like nothing".
"visual_profile": profiles.get(key),
}
def _objects(
state: dict, profiles: dict, present: list[str], location_key: str | None
) -> list[dict]:
"""The things visibly in the scene, from what the present entities hold.
Possession is the only relation in the state document that says an object is
*somewhere*, so it is the honest source for "what would be in the picture".
An item nobody in the scene is carrying is not depicted, which is the same
rule a reader would apply looking at the room.
"""
possessions = state.get("possessions")
if not isinstance(possessions, dict):
return []
holders = set(present) | ({location_key} if location_key else set())
out: list[dict] = []
for item_key, holder in possessions.items():
if holder not in holders or not isinstance(item_key, str):
continue
view = _entity_view(state, profiles, item_key)
if view is None:
continue
view["held_by"] = holder
out.append(view)
if len(out) >= MAX_OBJECTS:
break
return out
def _constraints(
state: dict, present: list[str], location_key: str | None
) -> list[str]:
"""The few facts that bear on depicting *this* scene, as sentences.
Deliberately narrow. The state document's `facts` list is the campaign's
reasoning about itself and most of it has nothing to do with a picture;
forwarding all of it would make the packet a state dump with a different
name, and would be the route by which something the scene has not exposed
reached a provider.
So only two kinds are carried: where the present entities are, and what they
are holding. Both are already visible in the scene by construction.
"""
out: list[str] = []
for key in present:
found = narrative_model.entity(state, key)
if found is None:
continue
name = narrative_model.entity_name(state, key)
status = found.get("status")
if status and status != "active":
out.append(f"{name} is {status}.")
if len(out) >= MAX_CONSTRAINTS:
return out
possessions = state.get("possessions")
if isinstance(possessions, dict):
for item_key, holder in possessions.items():
if holder not in present:
continue
out.append(
f"{narrative_model.entity_name(state, holder)} is carrying "
f"{narrative_model.entity_name(state, item_key)}."
)
if len(out) >= MAX_CONSTRAINTS:
break
return out
def _ambience(scene: dict) -> dict:
"""Time of day, lighting and mood — present in the shape, empty in v1.
`MEDIA-EXTENSION-CONTRACT.md` §5 lists these among a scene snapshot's
conceptual fields, and M10 **does not** add them to the `set_scene` event
that would establish them.
That is a deliberate deferral rather than an oversight. Adding them would
mean extending M5's typed-event vocabulary, which means teaching the
narrator to emit them, which means changing the prompt — and M10's central
acceptance condition is that ordinary story flow is *unchanged*. Buying
three optional fields at the price of touching every narration was the wrong
trade for a milestone whose deliverable is a seam.
So the keys are here and are `None`, read from the scene document if a later
milestone starts recording them. A provider adapter written today against
this shape keeps working when they arrive.
"""
return {
"time_of_day": scene.get("time_of_day") or None,
"lighting": scene.get("lighting") or None,
"mood": scene.get("mood") or None,
}
def _is_int(value) -> bool:
return isinstance(value, int) and not isinstance(value, bool)
backend/app/media/profiles.py
+220
View File
@@ -0,0 +1,220 @@
"""M10: reading and writing how an entity looks.
`models.VisualProfile` carries the design reasoning — why these rows are
campaign-scoped rather than per-position, why there is one table for characters,
locations and items, and why nothing here is story state. This module is the
narrow set of operations on them, and its own job is to make two things true:
* **a profile can only name an entity the campaign actually has**, so a typo
produces an error rather than a row describing nobody;
* **writing one changes nothing authoritative**, which is guaranteed by this
module not importing anything that could.
## Why the entity is checked against the current head
An entity key means something only in a state document, and a campaign has a
different document at every position. The check is made against the state at
the **active head** — the story the reader is on — for the same reason
`narrative/validate.py` resolves its `refs` there: it is the only position the
reader is looking at, and a key that means nothing there is a mistake, not a
branch subtlety.
The row that results is campaign-scoped anyway, so a profile written while
standing on one branch is visible from every branch. That asymmetry is
deliberate and is the continuity the profile exists for: the check is *"does
this name someone"*, and the storage answers *"what do they look like"*, which
does not vary by path.
"""
from __future__ import annotations
from sqlalchemy import select
from sqlalchemy.orm import Session
from .. import models
from ..narrative import model as narrative_model
from ..narrative import store as narrative_store
#: How many descriptors one profile may carry, and how long each may be. A
#: profile is a handful of stable traits, not a document: the bound exists so a
#: future provider's prompt cannot be grown without limit through this door, and
#: so one campaign cannot store an essay per entity.
MAX_DESCRIPTORS = 40
MAX_FEATURES = 40
MAX_VALUE = 400
MAX_STYLE_NOTES = 2_000
MAX_KEY = 200
class ProfileError(ValueError):
"""A visual profile could not be written, and why."""
def entity_exists(state: dict, entity_key: str) -> bool:
"""Whether the state document names this entity."""
return narrative_model.entity(state, entity_key) is not None
def set_profile(
db: Session,
adventure: models.Adventure,
entity_key: str,
*,
descriptors: dict | None = None,
features: list | None = None,
style_notes: str | None = None,
) -> models.VisualProfile:
"""Records how `entity_key` looks, creating or replacing the profile.
Replaces rather than merges. A profile is one answer to "what does this look
like", and merging would make it impossible to *remove* a descriptor — the
caller would be able to add "wearing a red coat" and never take it off,
which for continuity metadata is the wrong default. A caller that wants to
amend one reads it first.
Raises `ProfileError` if the campaign's state at the active head does not
name the entity, or if the profile is malformed. It writes nothing in either
case, and it writes nothing to `narrative_state` in any case.
"""
key = _checked_key(entity_key)
state = narrative_store.current(adventure)
if not entity_exists(state, key):
raise ProfileError(
f"This campaign has no entity called {key!r}, so there is nothing "
f"for a visual profile to describe. Profiles attach to the "
f"campaign's own entities, not to names."
)
row = get_profile(db, adventure, key)
if row is None:
row = models.VisualProfile(adventure_id=adventure.id, entity_key=key)
db.add(row)
row.descriptors = _checked_descriptors(descriptors)
row.features = _checked_features(features)
row.style_notes = _checked_notes(style_notes)
return row
def get_profile(
db: Session, adventure: models.Adventure, entity_key: str
) -> models.VisualProfile | None:
return db.execute(
select(models.VisualProfile).where(
models.VisualProfile.adventure_id == adventure.id,
models.VisualProfile.entity_key == entity_key,
)
).scalars().first()
def all_for(db: Session, adventure: models.Adventure) -> list[models.VisualProfile]:
return list(db.execute(
select(models.VisualProfile)
.where(models.VisualProfile.adventure_id == adventure.id)
.order_by(models.VisualProfile.entity_key)
).scalars().all())
def by_key(db: Session, adventure: models.Adventure) -> dict[str, dict]:
"""Every profile in the campaign, keyed by entity, as plain dictionaries.
One query, because the Scene Packet needs several profiles at once and
fetching them per entity would be a query per character in the scene.
"""
return {row.entity_key: as_dict(row) for row in all_for(db, adventure)}
def as_dict(row: models.VisualProfile) -> dict:
"""One profile as it appears in a Scene Packet."""
return {
"descriptors": dict(row.descriptors or {}),
"features": list(row.features or []),
"style_notes": row.style_notes or "",
}
def delete_profile(
db: Session, adventure: models.Adventure, entity_key: str
) -> bool:
"""Removes a profile. Returns whether there was one.
Deleting a profile removes a *description*, never the entity: the entity
lives in the authoritative state document and nothing here can reach it.
"""
row = get_profile(db, adventure, entity_key)
if row is None:
return False
db.delete(row)
return True
# ------------------------------------------------------------- the checking
def _checked_key(entity_key) -> str:
if not isinstance(entity_key, str) or not entity_key.strip():
raise ProfileError("A visual profile has to name an entity.")
key = entity_key.strip()
if len(key) > MAX_KEY:
raise ProfileError(f"Entity keys are at most {MAX_KEY} characters.")
return key
def _checked_descriptors(descriptors) -> dict:
"""Trait -> value, both short strings.
Values are text rather than arbitrary JSON on purpose. A descriptor is
something a future provider will put in a prompt, and a nested structure
would either be flattened by whoever does that — inconsistently — or
smuggle a provider-shaped payload through a story-side field, which is the
boundary this package exists to keep.
"""
if descriptors is None:
return {}
if not isinstance(descriptors, dict):
raise ProfileError("`descriptors` must be a map of trait to value.")
if len(descriptors) > MAX_DESCRIPTORS:
raise ProfileError(
f"A profile may carry at most {MAX_DESCRIPTORS} descriptors."
)
out: dict[str, str] = {}
for trait, value in descriptors.items():
if not isinstance(trait, str) or not trait.strip():
raise ProfileError("Every descriptor needs a name.")
if not isinstance(value, str):
raise ProfileError(
f"The value for {trait!r} must be text — a profile describes "
f"how something looks, in words a person could read back."
)
if len(value) > MAX_VALUE:
raise ProfileError(
f"The value for {trait!r} is longer than {MAX_VALUE} characters."
)
out[trait.strip()[:MAX_KEY]] = value
return out
def _checked_features(features) -> list:
if features is None:
return []
if not isinstance(features, list):
raise ProfileError("`features` must be a list of short phrases.")
if len(features) > MAX_FEATURES:
raise ProfileError(f"A profile may carry at most {MAX_FEATURES} features.")
out = []
for feature in features:
if not isinstance(feature, str) or not feature.strip():
raise ProfileError("Every feature must be a non-empty phrase.")
if len(feature) > MAX_VALUE:
raise ProfileError(f"A feature is longer than {MAX_VALUE} characters.")
out.append(feature.strip())
return out
def _checked_notes(style_notes) -> str:
if style_notes is None:
return ""
if not isinstance(style_notes, str):
raise ProfileError("`style_notes` must be text.")
if len(style_notes) > MAX_STYLE_NOTES:
raise ProfileError(
f"Style notes are longer than {MAX_STYLE_NOTES} characters."
)
return style_notes.strip()
backend/app/media/providers.py
+332
View File
@@ -0,0 +1,332 @@
"""M10: what a future media provider must satisfy, and nothing that satisfies it.
No provider is implemented here, none is registered by default, and nothing in
this module opens a socket. What it defines is the shape of the boundary, so
that adding a real image, video, audio, TTS or STT provider later is writing an
adapter rather than editing the story engine.
## The rule these types exist to enforce
`MEDIA-EXTENSION-CONTRACT.md` §3: the Story Engine must not call ComfyUI, Stable
Diffusion, a video pipeline, a TTS engine or a third-party media API. It states
that as a recommendation; this module makes it structural. Everything crossing
the boundary is expressed in this vocabulary:
MediaKind image | video | audio | tts | stt
MediaRequest a scene packet, a kind, and neutral hints
MediaResult bytes-or-path, a type, and provenance
DraftTranscription STT's deliberately different answer (see below)
**No provider vocabulary appears anywhere in this file or in any story module.**
There is no workflow JSON, no sampler name, no CFG scale, no LoRA, no
`num_inference_steps`, no Whisper option and no voice id. A provider adapter
owns that translation, in its own package, and the story engine never learns it.
`test_m10_providers.py` greps the story modules for that vocabulary so the rule
cannot rot quietly.
## Why Protocols rather than base classes
A future adapter should not have to import from here to be usable — it should
merely have to *fit*. `typing.Protocol` gives a structural contract that a test
double satisfies as readily as a real ComfyUI adapter, which keeps the seam
honest: if the only way to satisfy the interface were to inherit from it, the
interface would be describing this codebase rather than the boundary.
## STT is deliberately shaped differently, and that is the point
Every other provider returns a `MediaResult` — a depiction of something the
story already established. STT returns a `DraftTranscription`, which is a
different type on purpose, because it flows the other way:
audio -> local STT -> draft text -> the reader edits it -> normal submission
`MEDIA-EXTENSION-CONTRACT.md` §24A states the rule as *"STT output is draft user
input, not an accepted story event."* A shared return type would have made it
possible to hand a transcription to something expecting a finished artefact, and
the asymmetry would have survived only as a comment. `DraftTranscription`
carries `editable = True` and has no path into the turn pipeline: the reader's
edited text enters through the ordinary action endpoint like anything they
typed, and is validated, refereed and snapshotted exactly the same way.
M10 implements no microphone capture and no transcription. The type boundary is
the deliverable.
## Endpoints: loopback only, and stricter than the narrator's on purpose
`endpoints.py` already decides which *inference* endpoints this product will
talk to, and allows an explicitly configured trusted LAN as well as loopback
(ADR 011). Media is not given that latitude. `MEDIA-EXTENSION-CONTRACT.md` §27
and §28 set the media default at loopback, with any future LAN extension
explicit and user-controlled — so `check_endpoint` below reuses the existing,
tested address machinery and then applies the stricter rule on top.
Reusing rather than reimplementing matters: a second endpoint validator would be
a second place for the policy to be wrong, and this one inherits the property
that makes the first one hard to talk around — it judges the address a host
actually resolves to, not the name.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Protocol, runtime_checkable
from .. import endpoints
#: The kinds of media this architecture is required to accommodate. A string
#: enum rather than free text, so a typo is a failure here rather than a request
#: nothing will ever service.
IMAGE = "image"
VIDEO = "video"
AUDIO = "audio"
TTS = "tts"
STT = "stt"
MEDIA_KINDS: tuple[str, ...] = (IMAGE, VIDEO, AUDIO, TTS, STT)
def is_media_kind(value) -> bool:
return isinstance(value, str) and value in MEDIA_KINDS
class MediaProviderError(RuntimeError):
"""A provider could not do what was asked.
Deliberately its own type, and deliberately not caught anywhere in the story
path: nothing in a turn calls a provider, so there is no code path where
this could reach an accepted narration. If a future coordinator catches it,
it does so on its own side of the boundary — a failed depiction must leave
the story exactly as it was (`MEDIA-EXTENSION-CONTRACT.md` §50).
"""
class EndpointRejected(endpoints.EndpointRejected):
"""A media endpoint outside the loopback-only media policy.
Subclasses the inference rejection so that a caller which already handles
"this endpoint is not allowed" keeps working, while a caller that wants to
tell the two policies apart still can.
"""
def endpoint_rejection_reason(url: str) -> str | None:
"""Why this URL may not be a media endpoint, or `None` if it may.
Two rules, in order, and the first is somebody else's:
1. the existing inference policy — an address in an allowed private network,
judged by resolution rather than by name (`endpoints.py`);
2. **and** loopback specifically, which is the media contract's stricter
default (§27, §28).
So a trusted-LAN address that an Ollama may legitimately use is refused here.
That is not an oversight: narrator inference is a deployment the user has
already reasoned about and configured, whereas a media endpoint is a new
surface with no v1 use, and the safe default for a surface nobody needs yet
is the narrowest one. A future milestone may widen it, explicitly and off by
default, which is what §27 requires of any such change.
"""
reason = endpoints.rejection_reason(url)
if reason is not None:
return reason
if not endpoints.is_loopback(url):
return (
"A media provider endpoint must be on this machine. "
f"{url!r} resolves somewhere else — media generation has no "
"trusted-LAN mode, and adding one would be an explicit, "
"off-by-default change rather than a setting."
)
return None
def check_endpoint(url: str) -> None:
"""Raises `EndpointRejected` unless `url` is an allowed media endpoint."""
reason = endpoint_rejection_reason(url)
if reason is not None:
raise EndpointRejected(reason)
# ----------------------------------------------------------------- the types
@dataclass(frozen=True)
class ProviderCapabilities:
"""What one provider can do, in neutral terms.
Deliberately small. `MEDIA-EXTENSION-CONTRACT.md` §25 shows a richer example
— seeds, reference images, inpainting — and M10 does not model those,
because every one of them is a guess until a provider exists to be asked.
What is here is what a coordinator would need in order to choose *whether*
to route to this provider at all; anything finer belongs to the adapter and
its own capability document.
"""
provider_id: str
kinds: tuple[str, ...] = ()
#: Free-form, provider-owned, and never interpreted by story code. It exists
#: so an adapter can advertise what it supports without this module growing
#: a field per feature the ecosystem invents.
details: dict = field(default_factory=dict)
def supports(self, kind: str) -> bool:
return kind in self.kinds
@dataclass(frozen=True)
class MediaRequest:
"""What a coordinator would hand a provider: a scene, a kind, and hints.
`scene` is a Scene Packet (`packet.build`) — a bounded description of one
accepted scene, not the transcript. That is the whole point of the packet
existing (`MEDIA-EXTENSION-CONTRACT.md` §12): a provider is given what it
needs to depict a moment and no more, which bounds prompt size, keeps
providers interchangeable, and means swapping one does not hand a new
process the campaign's history.
`hints` is provider-neutral and optional — an aspect ratio, a duration, a
count. It is **not** where a workflow graph or a sampler setting goes; those
belong to the adapter, which knows what it is talking to.
"""
kind: str
scene: dict
hints: dict = field(default_factory=dict)
def __post_init__(self):
if not is_media_kind(self.kind):
raise ValueError(
f"{self.kind!r} is not one of {', '.join(MEDIA_KINDS)}"
)
@dataclass(frozen=True)
class MediaResult:
"""What a provider hands back: a depiction, and where it came from.
Bytes *or* a path, never both, and the caller says which it wanted. Neither
is interpreted here; M10 registers no provider, so nothing constructs one of
these outside a test.
`provenance` carries the scene identity the request named, so that a future
asset can always be traced to the accepted position it depicts
(`MEDIA-EXTENSION-CONTRACT.md` §48). It is a record of what was asked for —
it does not make the depiction true.
"""
kind: str
media_type: str
provenance: dict = field(default_factory=dict)
data: bytes | None = None
path: str | None = None
details: dict = field(default_factory=dict)
@dataclass(frozen=True)
class DraftTranscription:
"""STT's answer, and deliberately not a `MediaResult`.
See the module docstring. This is **draft user input**: text the reader is
expected to read, correct and submit themselves. It is not an accepted turn,
not a state event, not canon, and it has no route into the story that the
reader's own typing does not also take.
`editable` is `True` and there is no constructor that sets it otherwise —
it is a statement about what this type *is* rather than a setting, and a
reader that finds it false has been handed something that is not a draft.
"""
text: str
editable: bool = True
confidence: float | None = None
details: dict = field(default_factory=dict)
# ------------------------------------------------------------- the protocols
@runtime_checkable
class MediaProvider(Protocol):
"""Anything that can depict an accepted scene.
One protocol covers image, video and audio because the boundary is the same
for all three: a bounded scene in, a depiction out, nothing written to the
story. What differs between them is entirely inside the adapter.
"""
def capabilities(self) -> ProviderCapabilities: ...
async def generate(self, request: MediaRequest) -> MediaResult: ...
@runtime_checkable
class SpeechProvider(Protocol):
"""Text to speech: still a depiction, of prose the story already accepted."""
def capabilities(self) -> ProviderCapabilities: ...
async def speak(self, text: str, hints: dict | None = None) -> MediaResult: ...
@runtime_checkable
class TranscriptionProvider(Protocol):
"""Speech to text, which runs the other way and returns a draft.
The signature is the asymmetry: it takes audio and returns
`DraftTranscription`, so no coordinator can hand its output to something
expecting a finished artefact, and nothing can mistake it for an accepted
turn.
"""
def capabilities(self) -> ProviderCapabilities: ...
async def transcribe(
self, audio: bytes, hints: dict | None = None
) -> DraftTranscription: ...
# -------------------------------------------------------------- the registry
#: Registered providers, by id. **Empty, and empty on purpose.**
#:
#: M10 ships no provider, so nothing is registered at import, nothing is
#: required at startup, and no configuration is read. `test_m10_no_media.py`
#: asserts this is empty after the application has been imported and a campaign
#: has been played — media readiness has to be inert until something explicitly
#: uses it.
_REGISTRY: dict[str, object] = {}
def register(provider_id: str, provider: object) -> None:
"""Makes a provider available to a future coordinator.
Exists to prove the claim in M10's Definition of Done — that a provider can
be added *without modifying story authority or history* — by being the only
thing an adapter has to call. Nothing in `app/routers`, `app/narrative`,
`app/context` or `app/tree` imports this module, so registering one cannot
reach them.
"""
if not isinstance(provider_id, str) or not provider_id.strip():
raise ValueError("a provider needs an id")
_REGISTRY[provider_id] = provider
def unregister(provider_id: str) -> None:
_REGISTRY.pop(provider_id, None)
def registered() -> dict[str, object]:
"""The registry, copied — callers must not mutate it in place."""
return dict(_REGISTRY)
def for_kind(kind: str) -> list[object]:
"""Every registered provider advertising `kind`. Empty in v1."""
out = []
for provider in _REGISTRY.values():
caps = getattr(provider, "capabilities", None)
if caps is None:
continue
try:
if caps().supports(kind):
out.append(provider)
except Exception: # noqa: BLE001 - a broken adapter is not this layer's
continue
return out