M7: a first-class imported knowledge library
A campaign can import local .txt and .md files as Canon, Reference or Inspiration, and the class is load-bearing rather than a label: it decides the words a passage is framed with in the prompt, the weight it carries when passages are ranked, and which budget it competes in when the context is tight. This is a separate subsystem, which is the Phase 0B decision (IMPORTED-KNOWLEDGE-DESIGN.md §73). Story Cards do not carry classification, provenance, content identity, chunking, an index or a lifecycle, and they were not promoted into something that does. Nothing here reads or writes one. The subsystem, in backend/app/knowledge/: classes the three classes, their weights, and the prompt framing chunking deterministic, heading-aware, 60-800 tokens, no overlap fts SQLite FTS5 with porter stemming; scoped and bounded in SQL importer validate, hash, store, chunk, index — in one transaction embeddings local Ollama vectors through the shared provider retrieval query construction, hybrid merge, rerank inject the budgeted cut and the rendered prompt sections Relevance admission is a separate stage from ranking, and that separation is the milestone's most expensive lesson. An independent review found the first implementation deciding relevance with a floor expressed as a share of the best candidate — which the best clears by construction — so a passage was admitted on every turn regardless of the scene. A query about tide tables and container tonnage retrieved all five sources of a fantasy campaign, narrator-only hidden Canon among them. So the pipeline is now: candidate generation -> admission -> ranking -> class weighting -> budget Admission reads raw, candidate-set-independent signals: the cosine the model returned, and how many distinct meaningful query terms a passage contains. Ranking reads normalized ones, because bm25 has no fixed range and cosine's zero is not zero. Normalization decides order among things that matched; it can never decide whether anything matched. Authority is applied after admission, so a class orders what matched and never rescues what did not. Retrieval may therefore return nothing, and on a scene unrelated to the library it does. The other decisions that each replaced an obvious wrong one: - The class multiplies relevance rather than adding to it. An additive bonus satisfies "Canon outranks Reference" and makes "do not include irrelevant Canon" impossible, because a large enough constant wins on its own. - The semantic floor is measured, not guessed: 113 production-path pairs against nomic-embed-text put targeted matches at 0.55-0.85 and off-topic pairs at 0.36-0.56, and 0.58 sits between them. Because it is a property of that model and not of cosine similarity, it is keyed to the model rather than applied to whatever is configured: an embedding model with no measured calibration in this build does not borrow the number. Semantic admission is skipped, the campaign retrieves lexically, and the reason is stated in the knowledge status and in the turn's provenance. Degrading to lexical keeps the library usable; lending the threshold to an unmeasured model is how the admitted-everything defect would return. - One lexical term is not evidence. Two distinct meaningful terms, or one that is neither a standing campaign entity nor a negligible share of the query. The stop list grew from 42 words to 261, all function words — no subject matter, because a stop list that removes subject matter stops finding "The Silver Key". - Lexical retrieval is a production path, not a fallback. It finds the proper nouns and invented terms a setting bible is made of, and the library is fully usable with no embedding model configured. Safety is structural rather than filtered. Imported text reaches the prompt whole, inside a section that says what it is, under a rule stating the authority order in words and refusing every instruction inside it. No endpoint accepts a filesystem path, so H08 has no mechanism to escape from. Nothing renders imported content as HTML, so a script tag is five visible characters and a remote image is never fetched. Import, chunking, indexing, retrieval and a turn open no socket at all; only embeddings do, through the endpoint allowlist the memory bank already uses. Provenance is the rendered text, not a foreign key: deleting a source cannot turn a historical turn's evidence into dangling ids. Schema: knowledge_sources, knowledge_chunks, knowledge_embeddings, and an FTS5 virtual table attached to knowledge_chunks as a DDL hook so it is created and dropped with the table it indexes. Migration 92. A pre-M7 database opens unchanged and needs no sources to play. Bundle: the source content and the reader's judgements about it travel; the passages, index rows and vectors are rebuilt on import, so a restored campaign is searchable immediately without a reindex step. One runtime dependency: python-multipart, Starlette's multipart parser. It is what makes the upload surface possible, and the upload surface is why no pathname is ever accepted. The test doubles were the reason the defect shipped, so they were corrected too. The retrieval stub scored unrelated text at 0.06-0.20 where the real model scores it at 0.43-0.44, and its docstring said it had deliberately removed the constant component that "would put a similarity floor under every pair" — which is exactly the property real models have. The stub now has that floor, one test fails if it is ever removed, and another reproduces the superseded rule and asserts it is still fooled by the same fixture. Run against the pre-corrective implementation, the new suite fails 13 of 18. Tests: 939 passed, 14 skipped (836/7 at M6). 110 new across seven files, one of which mocks nothing between itself and Ollama and re-measures the similarity separation on every run. 43/43 checks in a real Firefox, reproduced. Docker build clean. Four other defects found by review or by the browser run were fixed here rather than carried: an unreachable relevance constant that appeared to enforce something and did not; acceptance tests using the wrong fixture files, so G07's trap was never exercised; a bidirectional override surviving into displayed filenames; and, from the implementation pass, the Insights panel showing M5's two state sections as raw keys and the source inspector refetching on every keystroke. M7 was independently reviewed, which returned PASS WITH CORRECTIVE WORK REQUIRED. Both blocking findings are closed, and closeout resolved the embedding-model calibration boundary the corrective pass had left as debt. planning/reports/M7-IMPLEMENTATION-REPORT.md carries the review, the corrective closeout and the closeout verification in sequence, none overwriting another. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HdaXiFbscatQaLS7dJk6b
This commit is contained in:
JesseMarkowitz
co-authored by
Claude Opus 5
Claude Opus 5
parent
a6e9c7a32b
commit
480414efe0
@@ -166,6 +166,56 @@ export const api = {
|
||||
}),
|
||||
getActionContext: (advId, actionId) => request(`/adventures/${advId}/actions/${actionId}/context`),
|
||||
|
||||
// Imported knowledge (M7). Campaign-scoped: every one of these is under
|
||||
// /adventures/{id}, and the server checks the source belongs to that campaign
|
||||
// as well as checking the campaign belongs to the caller. The browser does no
|
||||
// filtering of its own, and nothing here would work if it did.
|
||||
listKnowledge: (advId) => request(`/adventures/${advId}/knowledge`),
|
||||
getKnowledgeSource: (advId, sourceId) =>
|
||||
request(`/adventures/${advId}/knowledge/${sourceId}`),
|
||||
getKnowledgeChunks: (advId, sourceId) =>
|
||||
request(`/adventures/${advId}/knowledge/${sourceId}/chunks`),
|
||||
updateKnowledgeSource: (advId, sourceId, data) =>
|
||||
request(`/adventures/${advId}/knowledge/${sourceId}`, {
|
||||
method: 'PATCH', body: JSON.stringify(data),
|
||||
}),
|
||||
deleteKnowledgeSource: (advId, sourceId) =>
|
||||
request(`/adventures/${advId}/knowledge/${sourceId}`, { method: 'DELETE' }),
|
||||
reindexKnowledge: (advId, { sourceId, semantic = true } = {}) => {
|
||||
const params = new URLSearchParams()
|
||||
if (sourceId != null) params.set('source_id', sourceId)
|
||||
params.set('semantic', semantic ? 'true' : 'false')
|
||||
return request(`/adventures/${advId}/knowledge/reindex?${params}`, { method: 'POST' })
|
||||
},
|
||||
getKnowledgeStatus: (advId) => request(`/adventures/${advId}/knowledge-status`),
|
||||
// The file goes up as multipart, which is the only way a file reaches this
|
||||
// API — there is no endpoint that takes a pathname, so there is no path for a
|
||||
// traversal to escape from. `request` is bypassed because it sets a JSON
|
||||
// content type; the browser has to set the multipart boundary itself.
|
||||
importKnowledge: async (advId, file, fields) => {
|
||||
const body = new FormData()
|
||||
body.append('file', file)
|
||||
Object.entries(fields).forEach(([key, value]) => body.append(key, String(value)))
|
||||
const resp = await fetch(`/api/adventures/${advId}/knowledge`, { method: 'POST', body })
|
||||
if (!resp.ok) {
|
||||
let detail = resp.statusText
|
||||
let conflict = null
|
||||
try {
|
||||
const payload = (await resp.json()).detail
|
||||
if (payload && typeof payload === 'object') {
|
||||
detail = payload.message || detail
|
||||
conflict = payload.conflict || null
|
||||
} else if (payload) {
|
||||
detail = payload
|
||||
}
|
||||
} catch { /* non-JSON error body */ }
|
||||
const error = new Error(detail)
|
||||
error.conflict = conflict
|
||||
throw error
|
||||
}
|
||||
return resp.json()
|
||||
},
|
||||
|
||||
// Memory bank
|
||||
listMemories: (advId) => request(`/adventures/${advId}/memories`),
|
||||
createMemory: (advId, text) =>
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
@import './styles/drawers.css'; /* the status drawer and the world-state drawer */
|
||||
@import './styles/schema-editor.css'; /* the stat-schema form and the NPC roster */
|
||||
@import './styles/insights.css'; /* insights, scripts, and the memory bank */
|
||||
@import './styles/knowledge.css'; /* the imported knowledge library (M7) */
|
||||
@import './styles/modals.css'; /* the filter bar, tags, and modals */
|
||||
@import './styles/auth.css'; /* log in, sign up, and the settings debug log */
|
||||
@import './styles/banners.css'; /* the Play screen's persistent banner */
|
||||
|
||||
@@ -10,6 +10,21 @@ const SECTION_LABELS = {
|
||||
plot_essentials: 'Plot Essentials',
|
||||
story_summary: 'Story Summary',
|
||||
used_memories: 'Used Memories (memory bank)',
|
||||
// M5's two state-protocol sections had no entry here, so the Insights panel
|
||||
// showed their raw keys — `state_rule` and `state_reminder` — beside every
|
||||
// other section's readable name. Found by M7's browser run.
|
||||
state_rule: 'Narrative State (reporting rule)',
|
||||
state_reminder: 'Narrative State (emit reminder)',
|
||||
campaign_canon: 'Campaign Canon',
|
||||
narrative_state: 'Narrative State (current)',
|
||||
state_refusals: 'Narrative State (corrections)',
|
||||
persona: 'Player Character',
|
||||
script_context: 'Scenario context',
|
||||
knowledge_rule: 'Imported Knowledge (rules for using it)',
|
||||
imported_canon_always: 'Imported Canon (always in force)',
|
||||
imported_canon: 'Imported Canon (retrieved)',
|
||||
imported_reference: 'Imported Reference (retrieved)',
|
||||
imported_inspiration: 'Imported Inspiration (retrieved)',
|
||||
world_state_guide: 'World State (stat guide)',
|
||||
world_state: 'World State (RPG)',
|
||||
world_state_rule: 'World State (reporting rule)',
|
||||
@@ -33,6 +48,18 @@ const SECTION_COLORS = {
|
||||
plot_essentials: '#c97dc0',
|
||||
story_summary: '#7dc9a2',
|
||||
used_memories: '#5fb8c9',
|
||||
campaign_canon: '#c98fb4',
|
||||
state_rule: '#9d7a52',
|
||||
state_reminder: '#8a6f52',
|
||||
narrative_state: '#d79a63',
|
||||
state_refusals: '#b8834a',
|
||||
persona: '#8fb0c9',
|
||||
script_context: '#7d9c8f',
|
||||
knowledge_rule: '#8d94b0',
|
||||
imported_canon_always: '#d2688a',
|
||||
imported_canon: '#c76f9c',
|
||||
imported_reference: '#8fa8d1',
|
||||
imported_inspiration: '#a99ad6',
|
||||
world_lore: '#c9b47d',
|
||||
world_state: '#d79a63',
|
||||
world_state_guide: '#b8834a',
|
||||
|
||||
@@ -20,6 +20,7 @@ import { TakePager } from './TakePager'
|
||||
import { WorldStateDrawer } from './drawers/WorldStateDrawer'
|
||||
import { BranchPanel } from './panels/BranchPanel'
|
||||
import { InsightsPanel } from './panels/InsightsPanel'
|
||||
import { KnowledgePanel } from './panels/KnowledgePanel'
|
||||
import { MemoryPanel } from './panels/MemoryPanel'
|
||||
import { PlotPanel } from './panels/PlotPanel'
|
||||
import { SavePointPanel } from './panels/SavePointPanel'
|
||||
@@ -70,7 +71,7 @@ export default function Play() {
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [toast, setToast] = useState(null)
|
||||
const [editing, setEditing] = useState(null)
|
||||
const [panel, setPanel] = useState(null) // null | 'state' | 'plot' | 'memory' | 'branches' | 'savepoints' | 'insights'
|
||||
const [panel, setPanel] = useState(null) // null | 'state' | 'plot' | 'memory' | 'knowledge' | 'branches' | 'savepoints' | 'insights'
|
||||
// Bumped when something outside the turn loop changes the drawers' state
|
||||
// (currently "Update from scenario"), which no action count would reflect.
|
||||
const [stateKey, setStateKey] = useState(0)
|
||||
@@ -560,6 +561,8 @@ export default function Play() {
|
||||
onClick={() => setPanel(panel === 'plot' ? null : 'plot')}>Plot</button>
|
||||
<button className={panel === 'memory' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'memory' ? null : 'memory')}>Memory</button>
|
||||
<button className={panel === 'knowledge' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'knowledge' ? null : 'knowledge')}>Knowledge</button>
|
||||
<button className={panel === 'branches' ? 'active' : ''}
|
||||
onClick={() => setPanel(panel === 'branches' ? null : 'branches')}>Branches</button>
|
||||
<button className={panel === 'savepoints' ? 'active' : ''}
|
||||
@@ -780,7 +783,8 @@ export default function Play() {
|
||||
<div className="side-panel-header">
|
||||
<h2>{{
|
||||
state: 'Story State', plot: 'Plot Components', memory: 'Memory Bank',
|
||||
branches: 'Branches', savepoints: 'Save Points', insights: 'Insights',
|
||||
knowledge: 'Imported Knowledge', branches: 'Branches',
|
||||
savepoints: 'Save Points', insights: 'Insights',
|
||||
}[panel]}</h2>
|
||||
<button onClick={() => setPanel(null)}>✕</button>
|
||||
</div>
|
||||
@@ -803,6 +807,18 @@ export default function Play() {
|
||||
// but deleting a branch deletes the memories that hung off it,
|
||||
// and that happens without a turn being played.
|
||||
refreshKey={`${actions.length}:${stateKey}`} />
|
||||
) : panel === 'knowledge' ? (
|
||||
// M7. The library is campaign-scoped rather than lineage-scoped —
|
||||
// an imported file does not become a different file because the
|
||||
// story forked — so unlike the panels above it does not have to
|
||||
// re-read when the head moves. It keys on `stateKey` all the same,
|
||||
// because deleting a branch or restoring a Save Point is exactly
|
||||
// when a reader looks at what the narrator is being given.
|
||||
<KnowledgePanel
|
||||
advId={id}
|
||||
refreshKey={`${actions.length}:${stateKey}`}
|
||||
onError={(message) => setToast({ text: message, isError: true })}
|
||||
/>
|
||||
) : panel === 'savepoints' ? (
|
||||
// Restoring one moves the story exactly as Undo and Redo do, so it
|
||||
// adopts the returned window the same way a branch switch does —
|
||||
|
||||
@@ -108,6 +108,91 @@ function InsightsPanel({ advId, inspectActionId, onClearInspect, refreshKey }) {
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
{/* M7: which imported passages the narrator was given, and why each of
|
||||
them won. This is F05's "retrieved knowledge" row and F06's imported
|
||||
half: the file, the class, the visibility, the passage, how it was
|
||||
found, what each retrieval path scored it, and what it cost.
|
||||
|
||||
Rendered as text, never as markup — the passage text below is a
|
||||
React child in a <pre>, so imported script or a javascript: URL is
|
||||
inert here exactly as it is in the Knowledge panel (H06, H07). */}
|
||||
{/* M7 corrective: "nothing matched" is a real answer and has to be
|
||||
said. Retrieval can now return no passages at all, and a panel that
|
||||
simply showed nothing would be indistinguishable from a library that
|
||||
was never searched. */}
|
||||
{report.knowledge && report.knowledge.generated > 0
|
||||
&& report.knowledge.used?.length === 0 && (
|
||||
<div className="insights-cards" data-testid="insights-knowledge-none">
|
||||
<div className="dim">
|
||||
▸ Imported knowledge: {report.knowledge.generated} passage(s)
|
||||
{' '}considered, none relevant enough to this scene to be supplied.
|
||||
{report.knowledge.terms?.length > 0 && (
|
||||
<> Searched on: {report.knowledge.terms.slice(0, 10).join(', ')}.</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{report.knowledge && (report.knowledge.used?.length > 0
|
||||
|| report.knowledge.suppressed?.length > 0
|
||||
|| report.knowledge.dropped?.length > 0) && (
|
||||
<div className="insights-cards" data-testid="insights-knowledge">
|
||||
{report.knowledge.used?.map((k) => (
|
||||
<div key={`k${k.chunk_id}`} className="knowledge-used"
|
||||
data-chunk-id={k.chunk_id} data-classification={k.classification}>
|
||||
▸ <b>{k.filename || k.title}</b>
|
||||
<span className={`knowledge-class knowledge-class-${k.classification}`}>
|
||||
{k.classification}
|
||||
</span>
|
||||
{k.heading_path && <span className="dim"> · {k.heading_path}</span>}
|
||||
<span className="dim"> · passage {k.chunk_index + 1}</span>
|
||||
{k.visibility === 'hidden' && (
|
||||
<span className="knowledge-badge" title="Supplied to the narrator only">
|
||||
narrator only
|
||||
</span>
|
||||
)}
|
||||
{k.always_include
|
||||
? <span className="dim"> · always included</span>
|
||||
: (
|
||||
<span className="dim">
|
||||
{' '}· {k.mode} match
|
||||
{k.matched_terms?.length > 0
|
||||
&& ` on ${k.matched_terms.slice(0, 6).join(', ')}`}
|
||||
{k.cosine > 0 && ` · similarity ${k.cosine.toFixed(2)}`}
|
||||
{' '}· rank {k.score.toFixed(2)}
|
||||
</span>
|
||||
)}
|
||||
<span className="dim"> · {k.prompt_tokens} tok</span>
|
||||
<pre className="knowledge-text">{k.text}</pre>
|
||||
</div>
|
||||
))}
|
||||
{report.knowledge.suppressed?.map((k) => (
|
||||
<div key={`ks${k.chunk_id}`} className="dropped">
|
||||
▸ {k.filename} passage {k.chunk_index + 1} — set aside as
|
||||
repeating passage {k.duplicate_of}
|
||||
</div>
|
||||
))}
|
||||
{report.knowledge.dropped?.map((k) => (
|
||||
<div key={`kd${k.chunk_id}`} className="dropped">
|
||||
▸ {k.filename} passage {k.chunk_index + 1} — {k.reason}
|
||||
{' '}({k.tokens} tok)
|
||||
</div>
|
||||
))}
|
||||
<div className="dim">
|
||||
{report.knowledge.generated} passage(s) considered
|
||||
{report.knowledge.rejected > 0
|
||||
&& `, ${report.knowledge.rejected} not relevant enough`}
|
||||
{report.knowledge.spent > 0 && (
|
||||
<>, {report.knowledge.spent} of {report.knowledge.budget} knowledge tokens used</>
|
||||
)}
|
||||
{report.knowledge.terms?.length > 0 && (
|
||||
<> · searched on: {report.knowledge.terms.slice(0, 10).join(', ')}</>
|
||||
)}
|
||||
</div>
|
||||
{report.knowledge.semantic_note && (
|
||||
<div className="dim">{report.knowledge.semantic_note}</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
{/* M6: which summary was used, and what stretch of story it covers, so
|
||||
"what history did that summary cover?" is answerable here. */}
|
||||
{report.summary ? (
|
||||
|
||||
@@ -0,0 +1,381 @@
|
||||
// M7: the imported knowledge library — import, classify, inspect, disable, delete.
|
||||
//
|
||||
// Functional rather than finished. M8 owns the designed knowledge surface; what
|
||||
// this has to do is make every M7 behaviour reachable in a browser without
|
||||
// anyone opening the database, which is the milestone's own standard.
|
||||
//
|
||||
// **Nothing here renders imported text as HTML.** Source text and passage text
|
||||
// both go into a `<pre>` as React children, which React escapes — so a
|
||||
// `<script>` in a file is five visible characters and a `javascript:` URL is
|
||||
// never an href, on first inspection and on every reopen (H06, H07). A remote
|
||||
// Markdown image reference is likewise just characters: no `<img>` is created,
|
||||
// so no request is made (G09). Adding a Markdown renderer would buy appearance
|
||||
// and cost exactly those three properties. `SECURITY-THREAT-MODEL.md` §14 names
|
||||
// sanitized presentation text as the safer default; rendering no markup at all
|
||||
// is one step safer still.
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { api } from '../../../api'
|
||||
|
||||
const CLASSES = [
|
||||
{ value: 'canon', label: 'Canon', hint: 'Authoritative truth for this campaign.' },
|
||||
{ value: 'reference', label: 'Reference', hint: 'Supporting information; does not establish story truth.' },
|
||||
{ value: 'inspiration', label: 'Inspiration', hint: 'Creative and style influence only.' },
|
||||
]
|
||||
|
||||
const CLASS_LABEL = Object.fromEntries(CLASSES.map((c) => [c.value, c.label]))
|
||||
|
||||
function bytes(n) {
|
||||
if (n < 1024) return `${n} B`
|
||||
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)} kB`
|
||||
return `${(n / 1024 / 1024).toFixed(2)} MB`
|
||||
}
|
||||
|
||||
function when(iso) {
|
||||
if (!iso) return ''
|
||||
const d = new Date(iso)
|
||||
return Number.isNaN(d.getTime()) ? '' : d.toLocaleString()
|
||||
}
|
||||
|
||||
function SourceDetail({ advId, source, onError }) {
|
||||
const [detail, setDetail] = useState(null)
|
||||
const [chunks, setChunks] = useState(null)
|
||||
const [showText, setShowText] = useState(false)
|
||||
const [showChunks, setShowChunks] = useState(false)
|
||||
// The error reporter reaches this component through a ref rather than
|
||||
// through the effect's dependencies. It arrives as a fresh arrow function on
|
||||
// every render of the Play screen — which re-renders on every keystroke in
|
||||
// the story box — so depending on it would refetch the source, and the
|
||||
// source text, once per character typed. The ref keeps the *current*
|
||||
// reporter without making it a reason to re-run.
|
||||
const report = useRef(onError)
|
||||
report.current = onError
|
||||
|
||||
useEffect(() => {
|
||||
let stale = false // a slow earlier request must not clobber a newer one
|
||||
setDetail(null)
|
||||
api.getKnowledgeSource(advId, source.id)
|
||||
.then((r) => { if (!stale) setDetail(r) })
|
||||
.catch((e) => { if (!stale) report.current(e.message) })
|
||||
return () => { stale = true }
|
||||
}, [advId, source.id])
|
||||
|
||||
const loadChunks = () => {
|
||||
setShowChunks((open) => !open)
|
||||
if (chunks === null) {
|
||||
api.getKnowledgeChunks(advId, source.id)
|
||||
.then(setChunks).catch((e) => report.current(e.message))
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="knowledge-detail">
|
||||
<div className="dim knowledge-facts">
|
||||
<div>File: {source.original_filename}</div>
|
||||
<div>Imported: {when(source.imported_at)}</div>
|
||||
<div>Size: {bytes(source.byte_size)} · {source.media_type}</div>
|
||||
<div>Passages: {source.chunk_count} · embedded {source.embedded_count}</div>
|
||||
<div>Parser v{source.parser_version} · chunking v{source.chunking_version}</div>
|
||||
{/* The content identity, in full rather than shortened: it is here to be
|
||||
compared against another copy, and half a digest compares nothing. */}
|
||||
<div className="knowledge-hash">SHA-256: {source.content_hash}</div>
|
||||
</div>
|
||||
<div className="knowledge-detail-buttons">
|
||||
<button onClick={() => setShowText((open) => !open)}>
|
||||
{showText ? 'Hide source text' : 'Inspect source text'}
|
||||
</button>
|
||||
<button onClick={loadChunks}>
|
||||
{showChunks ? 'Hide passages' : `Inspect passages (${source.chunk_count})`}
|
||||
</button>
|
||||
</div>
|
||||
{showText && (
|
||||
detail
|
||||
? <pre className="knowledge-text" data-testid="knowledge-source-text">{detail.content}</pre>
|
||||
: <div className="empty">Loading…</div>
|
||||
)}
|
||||
{showChunks && (
|
||||
chunks
|
||||
? chunks.map((chunk) => (
|
||||
<div key={chunk.id} className="knowledge-chunk">
|
||||
<div className="dim">
|
||||
passage {chunk.chunk_index + 1}
|
||||
{chunk.heading_path && ` · ${chunk.heading_path}`}
|
||||
{' '}· {chunk.token_count} tok
|
||||
{chunk.embedded ? ` · embedded (${chunk.embedding_model})` : ' · not embedded'}
|
||||
</div>
|
||||
<pre className="knowledge-text">{chunk.text}</pre>
|
||||
</div>
|
||||
))
|
||||
: <div className="empty">Loading…</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function SourceRow({ advId, source, onChange, onDelete, onError }) {
|
||||
const [open, setOpen] = useState(false)
|
||||
const [confirming, setConfirming] = useState(false)
|
||||
|
||||
return (
|
||||
<div className={`knowledge-row ${source.enabled ? '' : 'disabled'}`}
|
||||
data-source-id={source.id} data-classification={source.classification}>
|
||||
<div className="knowledge-head">
|
||||
<button className="knowledge-title" onClick={() => setOpen((o) => !o)}>
|
||||
{open ? '▾' : '▸'} {source.title}
|
||||
</button>
|
||||
{/* The filename beside the title, not only inside the detail. A title
|
||||
defaults to the filename without its extension, so two files that
|
||||
differ only by type would otherwise be indistinguishable in the
|
||||
list — and the filename is the name the reader knows the file by. */}
|
||||
{source.original_filename && source.original_filename !== source.title && (
|
||||
<span className="dim knowledge-filename">{source.original_filename}</span>
|
||||
)}
|
||||
<span className={`knowledge-class knowledge-class-${source.classification}`}>
|
||||
{CLASS_LABEL[source.classification]}
|
||||
</span>
|
||||
{!source.enabled && <span className="knowledge-badge">disabled</span>}
|
||||
{source.visibility === 'hidden' && (
|
||||
<span className="knowledge-badge" title="Given to the narrator; the protagonist does not know it">
|
||||
narrator only
|
||||
</span>
|
||||
)}
|
||||
{source.always_include && <span className="knowledge-badge">always included</span>}
|
||||
{source.index_state === 'failed' && (
|
||||
<span className="knowledge-badge failed" title={source.index_detail}>index failed</span>
|
||||
)}
|
||||
{source.embed_state === 'failed' && (
|
||||
<span className="knowledge-badge failed" title={source.embed_detail}>
|
||||
semantic failed — lexical search still works
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="knowledge-controls">
|
||||
<label>
|
||||
Class{' '}
|
||||
<select value={source.classification}
|
||||
onChange={(e) => onChange(source, { classification: e.target.value })}>
|
||||
{CLASSES.map((c) => <option key={c.value} value={c.value}>{c.label}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label>
|
||||
<input type="checkbox" checked={source.enabled}
|
||||
onChange={(e) => onChange(source, { enabled: e.target.checked })} />
|
||||
{' '}Enabled
|
||||
</label>
|
||||
<label>
|
||||
<input type="checkbox" checked={source.visibility === 'hidden'}
|
||||
onChange={(e) => onChange(source, {
|
||||
visibility: e.target.checked ? 'hidden' : 'normal',
|
||||
})} />
|
||||
{' '}Narrator only
|
||||
</label>
|
||||
{/* Canon's alone. The server enforces it too — this only stops the
|
||||
control offering something that would be silently ignored. */}
|
||||
{source.classification === 'canon' && (
|
||||
<label>
|
||||
<input type="checkbox" checked={source.always_include}
|
||||
onChange={(e) => onChange(source, { always_include: e.target.checked })} />
|
||||
{' '}Always include
|
||||
</label>
|
||||
)}
|
||||
{confirming ? (
|
||||
<span className="knowledge-confirm">
|
||||
Delete “{source.title}”? It stops being used from now on. Turns that
|
||||
already used it keep their record of what they were given.{' '}
|
||||
<button className="danger" onClick={() => onDelete(source)}>Delete</button>
|
||||
<button onClick={() => setConfirming(false)}>Cancel</button>
|
||||
</span>
|
||||
) : (
|
||||
<button className="danger" onClick={() => setConfirming(true)}>Delete</button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{open && (
|
||||
<SourceDetail advId={advId} source={source} onError={onError} />
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// A stable no-op for the `onError`-less case. Defined once, at module scope,
|
||||
// because an inline `onError || (() => {})` would hand every row a different
|
||||
// function on every render — which is the same identity problem `SourceDetail`
|
||||
// guards against, arriving from the other side.
|
||||
const NO_OP = () => {}
|
||||
|
||||
function KnowledgePanel({ advId, refreshKey, onError }) {
|
||||
const [sources, setSources] = useState(null)
|
||||
const [status, setStatus] = useState(null)
|
||||
const [file, setFile] = useState(null)
|
||||
const [classification, setClassification] = useState('canon')
|
||||
const [title, setTitle] = useState('')
|
||||
const [hidden, setHidden] = useState(false)
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [notice, setNotice] = useState(null)
|
||||
const [duplicate, setDuplicate] = useState(null)
|
||||
|
||||
const load = useCallback(() => {
|
||||
api.listKnowledge(advId).then(setSources).catch(() => setSources([]))
|
||||
api.getKnowledgeStatus(advId).then(setStatus).catch(() => setStatus(null))
|
||||
}, [advId])
|
||||
|
||||
useEffect(() => { load() }, [load, refreshKey])
|
||||
|
||||
const doImport = async (allowDuplicate = false) => {
|
||||
if (!file) return
|
||||
setBusy(true)
|
||||
setNotice(null)
|
||||
try {
|
||||
await api.importKnowledge(advId, file, {
|
||||
classification,
|
||||
title,
|
||||
visibility: hidden ? 'hidden' : 'normal',
|
||||
allow_duplicate: allowDuplicate,
|
||||
})
|
||||
setFile(null)
|
||||
setTitle('')
|
||||
setDuplicate(null)
|
||||
// The input is uncontrolled (a file input cannot be controlled), so it is
|
||||
// cleared through the DOM. Without this, re-picking the same file after a
|
||||
// failed import fires no change event and the button does nothing.
|
||||
const input = document.getElementById('knowledge-file')
|
||||
if (input) input.value = ''
|
||||
setNotice('Imported.')
|
||||
load()
|
||||
} catch (err) {
|
||||
if (err.conflict) setDuplicate(err.conflict)
|
||||
setNotice(err.message)
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
const change = async (source, data) => {
|
||||
try {
|
||||
const updated = await api.updateKnowledgeSource(advId, source.id, data)
|
||||
setSources((prev) => prev.map((s) => (s.id === source.id ? updated : s)))
|
||||
} catch (err) { onError?.(err.message) }
|
||||
}
|
||||
|
||||
const remove = async (source) => {
|
||||
try {
|
||||
await api.deleteKnowledgeSource(advId, source.id)
|
||||
setSources((prev) => prev.filter((s) => s.id !== source.id))
|
||||
load()
|
||||
} catch (err) { onError?.(err.message) }
|
||||
}
|
||||
|
||||
const reindex = async () => {
|
||||
setBusy(true)
|
||||
setNotice(null)
|
||||
try {
|
||||
const out = await api.reindexKnowledge(advId)
|
||||
setNotice(
|
||||
`Rebuilt ${out.chunks} passage${out.chunks === 1 ? '' : 's'} across `
|
||||
+ `${out.sources} source${out.sources === 1 ? '' : 's'}`
|
||||
+ (out.semantic ? `, embedded ${out.embedded}.` : '. Semantic index not configured.')
|
||||
)
|
||||
load()
|
||||
} catch (err) {
|
||||
setNotice(err.message)
|
||||
} finally { setBusy(false) }
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="knowledge-panel">
|
||||
<p className="dim">
|
||||
Local <code>.txt</code> and <code>.md</code> files this campaign can draw on.
|
||||
Nothing is uploaded anywhere, no link in a file is ever fetched, and text
|
||||
in a source is never treated as an instruction to the application.
|
||||
</p>
|
||||
|
||||
<div className="knowledge-import">
|
||||
<input id="knowledge-file" type="file" accept=".txt,.md,text/plain,text/markdown"
|
||||
onChange={(e) => { setFile(e.target.files?.[0] || null); setDuplicate(null) }} />
|
||||
<label>
|
||||
Class{' '}
|
||||
<select value={classification} onChange={(e) => setClassification(e.target.value)}>
|
||||
{CLASSES.map((c) => <option key={c.value} value={c.value}>{c.label}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<div className="dim">{CLASSES.find((c) => c.value === classification)?.hint}</div>
|
||||
<input type="text" placeholder="Title (optional)" value={title}
|
||||
onChange={(e) => setTitle(e.target.value)} />
|
||||
<label>
|
||||
<input type="checkbox" checked={hidden} onChange={(e) => setHidden(e.target.checked)} />
|
||||
{' '}Narrator only — the protagonist does not know this
|
||||
</label>
|
||||
<button className="primary" id="knowledge-import" disabled={!file || busy}
|
||||
onClick={() => doImport(false)}>
|
||||
{busy ? 'Working…' : 'Import file'}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{notice && <div className="knowledge-notice" id="knowledge-notice">{notice}</div>}
|
||||
{duplicate && (
|
||||
<div className="knowledge-notice">
|
||||
Already imported as “{duplicate.title}” ({CLASS_LABEL[duplicate.classification]}).{' '}
|
||||
<button onClick={() => doImport(true)}>Import a second copy anyway</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{status && (
|
||||
<div className="dim knowledge-status">
|
||||
{status.sources} source{status.sources === 1 ? '' : 's'},
|
||||
{' '}{status.enabled_sources} enabled ·{' '}
|
||||
{status.semantic_enabled
|
||||
? `semantic search on (${status.embedding_model})`
|
||||
+ (status.pending_embeddings
|
||||
? `, ${status.pending_embeddings} passage(s) still to embed`
|
||||
: '')
|
||||
: 'semantic search off — lexical search only, which is a supported setup'}
|
||||
{/* A configured but uncalibrated model is neither "on" nor simply
|
||||
"off": the reader chose a model and it is not being used for
|
||||
retrieval, so the reason has to be visible. */}
|
||||
{status.embedding_model && !status.semantic_calibrated && (
|
||||
<div className="dropped" data-testid="knowledge-uncalibrated">
|
||||
⚠ {status.semantic_note} Calibrated in this build:{' '}
|
||||
{(status.calibrated_models || []).join(', ')}.
|
||||
</div>
|
||||
)}
|
||||
{status.failed_index.length > 0 && (
|
||||
<div className="dropped">
|
||||
⚠ {status.failed_index.length} source(s) failed to index and cannot be retrieved.
|
||||
</div>
|
||||
)}
|
||||
{status.failed_embedding.length > 0 && (
|
||||
<div className="dropped">
|
||||
⚠ {status.failed_embedding.length} source(s) failed to embed. Lexical
|
||||
retrieval still works for them; Rebuild indexes to retry.
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="page-header" style={{ marginTop: 16 }}>
|
||||
<h3 style={{ margin: 0 }}>Sources {sources && `(${sources.length})`}</h3>
|
||||
<span>
|
||||
<button onClick={reindex} disabled={busy} title="Rebuild passages and search indexes from the stored text. Story history is untouched.">
|
||||
Rebuild indexes
|
||||
</button>
|
||||
<button onClick={load} style={{ marginLeft: 6 }}>Refresh</button>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{!sources && <div className="empty" style={{ padding: '12px 0' }}>Loading…</div>}
|
||||
{sources && sources.length === 0 && (
|
||||
<div className="empty" style={{ padding: '12px 0' }}>
|
||||
Nothing imported yet. Add a setting bible, character notes, a research
|
||||
file or a passage you want the prose to feel like.
|
||||
</div>
|
||||
)}
|
||||
{sources?.map((source) => (
|
||||
<SourceRow key={source.id} advId={advId} source={source}
|
||||
onChange={change} onDelete={remove} onError={onError || NO_OP} />
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export { KnowledgePanel }
|
||||
@@ -0,0 +1,133 @@
|
||||
/* M7: the imported knowledge library.
|
||||
*
|
||||
* Utilitarian by design. M8 owns the finished knowledge surface; what this has
|
||||
* to do is make every M7 behaviour legible and reachable — which class a source
|
||||
* carries, whether it is enabled, whether it is narrator-only, what its
|
||||
* passages are, and what an indexing failure was.
|
||||
*
|
||||
* The three class colours are the ones the Insights token bar uses for the
|
||||
* matching prompt sections (`pages/Play/format.js`), so a Canon badge here and
|
||||
* a Canon slice there are the same colour. Two views of one thing should not
|
||||
* need a reader to learn two palettes.
|
||||
*/
|
||||
|
||||
.knowledge-panel > p { margin-top: 0; }
|
||||
|
||||
.knowledge-import {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
padding: 12px;
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
}
|
||||
.knowledge-import input[type='text'] { width: 100%; }
|
||||
.knowledge-import .dim { font-size: 0.8rem; margin-top: -4px; }
|
||||
|
||||
.knowledge-notice {
|
||||
margin: 10px 0;
|
||||
padding: 8px 10px;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
|
||||
.knowledge-status { margin-top: 10px; font-size: 0.8rem; line-height: 1.5; }
|
||||
|
||||
.knowledge-row {
|
||||
border-top: 1px solid var(--border);
|
||||
padding: 10px 0;
|
||||
}
|
||||
/* A disabled source is still listed, still inspectable and still editable — it
|
||||
is simply out of retrieval. Fading it says "not in play" without saying
|
||||
"gone", which is the distinction disable exists to make. */
|
||||
.knowledge-row.disabled { opacity: 0.62; }
|
||||
|
||||
.knowledge-head { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
|
||||
.knowledge-title {
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
color: var(--text);
|
||||
cursor: pointer;
|
||||
}
|
||||
.knowledge-title:hover { color: var(--accent); }
|
||||
|
||||
.knowledge-class {
|
||||
font-size: 0.7rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.06em;
|
||||
border-radius: 999px;
|
||||
padding: 1px 8px;
|
||||
border: 1px solid currentColor;
|
||||
}
|
||||
.knowledge-class-canon { color: #c76f9c; }
|
||||
.knowledge-class-reference { color: #8fa8d1; }
|
||||
.knowledge-class-inspiration { color: #a99ad6; }
|
||||
|
||||
.knowledge-badge {
|
||||
font-size: 0.7rem;
|
||||
color: var(--text-dim);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 999px;
|
||||
padding: 0 8px;
|
||||
}
|
||||
.knowledge-badge.failed { color: var(--danger); border-color: var(--danger); }
|
||||
|
||||
.knowledge-controls {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 6px;
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
.knowledge-controls label { display: inline-flex; align-items: center; gap: 4px; }
|
||||
.knowledge-confirm {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
flex-wrap: wrap;
|
||||
color: var(--danger);
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
|
||||
.knowledge-detail { margin-top: 8px; padding-left: 12px; border-left: 2px solid var(--border); }
|
||||
.knowledge-facts { font-size: 0.78rem; line-height: 1.55; }
|
||||
/* The digest is shown whole so it can be compared against another copy, which
|
||||
means it has to be allowed to wrap. */
|
||||
.knowledge-hash { font-family: var(--font-mono, monospace); word-break: break-all; }
|
||||
.knowledge-detail-buttons { display: flex; gap: 8px; margin: 8px 0; flex-wrap: wrap; }
|
||||
|
||||
/* Imported text, everywhere it appears: the source inspector, the passage
|
||||
inspector, and the Insights provenance rows.
|
||||
*
|
||||
* `<pre>` with wrapping, and never `dangerouslySetInnerHTML` anywhere near it.
|
||||
* This is where H06 and H07 are actually decided — a `<script>` in an imported
|
||||
* file is text in a text node, and a `javascript:` URL is characters rather
|
||||
* than an href, because nothing turns either of them into markup. */
|
||||
.knowledge-text {
|
||||
background: var(--bg-input);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
padding: 8px 10px;
|
||||
margin: 4px 0;
|
||||
font-family: var(--font-mono, monospace);
|
||||
font-size: 0.76rem;
|
||||
line-height: 1.5;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
max-height: 22rem;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.knowledge-chunk { margin: 8px 0; }
|
||||
.knowledge-used { margin-bottom: 10px; }
|
||||
.knowledge-used .knowledge-class { margin-left: 6px; }
|
||||
|
||||
/* The filename beside the title in the list. Dim, because the title is what the
|
||||
reader named it and this is what the file was called. */
|
||||
.knowledge-filename { font-size: 0.78rem; }
|
||||
Reference in New Issue
Block a user