Store embeddings as packed float32 instead of a JSON list

A 1536-dimension vector spelled out as JSON decimals is ~31 KB. The same
numbers packed as float32 are 6,144 bytes, and the whole bank is read on
every turn, so those bytes are paid over and over.

It is a format change, not a precision trade: the endpoints compute in
float32 and render that into JSON, so converting back recovers the original
bits exactly. Nothing is re-embedded and no API call is made -- migration 38
is a pure repack of what is already stored.

Unlike migrations 36 and 37 this backfill cannot be expressed in portable
SQL, so it comes through Python, batched, and pays a one-time read of every
vector to stop paying three megabytes a turn.

The JSON column stays, still written through set_vector, so a rollback finds
the vectors intact. Reading from the blob comes next; a follow-up migration
drops the old column once that is verified.

Migration SQL can now be a {dialect: sql} map -- BLOB and BYTEA have no
common spelling, and every Postgres deploy replays this one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
This commit is contained in:
parththakkar106
2026-08-16 21:17:44 +05:30
co-authored by Claude Opus 5
parent 7ee5ceea6c
commit c56864877a
7 changed files with 404 additions and 23 deletions
+8 -3
View File
@@ -173,13 +173,18 @@ def build_fixture(args, rng: random.Random) -> tuple[int, int]:
))
for i in range(args.memories):
db.add(models.Memory(
memory = models.Memory(
adventure_id=adventure.id,
text=f"{MEMORY_TEXT} ({i})",
embedding=[rng.uniform(-1.0, 1.0) for _ in range(EMBEDDING_DIMS)],
source_start=i * memorybank.MEMORY_INTERVAL,
source_end=i * memorybank.MEMORY_INTERVAL + memorybank.MEMORY_INTERVAL - 1,
))
)
# Through the same door the app uses, so the fixture cannot end up
# storing vectors in a shape production never produces.
memorybank.set_vector(
memory, [rng.uniform(-1.0, 1.0) for _ in range(EMBEDDING_DIMS)]
)
db.add(memory)
db.commit()
return adventure.id, user.id