Store embeddings as packed float32 instead of a JSON list
A 1536-dimension vector spelled out as JSON decimals is ~31 KB. The same
numbers packed as float32 are 6,144 bytes, and the whole bank is read on
every turn, so those bytes are paid over and over.
It is a format change, not a precision trade: the endpoints compute in
float32 and render that into JSON, so converting back recovers the original
bits exactly. Nothing is re-embedded and no API call is made -- migration 38
is a pure repack of what is already stored.
Unlike migrations 36 and 37 this backfill cannot be expressed in portable
SQL, so it comes through Python, batched, and pays a one-time read of every
vector to stop paying three megabytes a turn.
The JSON column stays, still written through set_vector, so a rollback finds
the vectors intact. Reading from the blob comes next; a follow-up migration
drops the old column once that is verified.
Migration SQL can now be a {dialect: sql} map -- BLOB and BYTEA have no
common spelling, and every Postgres deploy replays this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
This commit is contained in:
co-authored by
Claude Opus 5
parent
7ee5ceea6c
commit
c56864877a
@@ -1488,7 +1488,7 @@ def update_memory(
|
||||
raise HTTPException(404, "Memory not found")
|
||||
fields = {k: v for k, v in payload.model_dump(exclude_unset=True).items() if v is not None}
|
||||
if "text" in fields and fields["text"].strip() != memory.text:
|
||||
memory.embedding = None # re-embed on the next post-turn pass
|
||||
memorybank.set_vector(memory, None) # re-embed on the next post-turn pass
|
||||
for field, value in fields.items():
|
||||
setattr(memory, field, value)
|
||||
db.commit()
|
||||
|
||||
Reference in New Issue
Block a user