Store embeddings as packed float32 instead of a JSON list
A 1536-dimension vector spelled out as JSON decimals is ~31 KB. The same
numbers packed as float32 are 6,144 bytes, and the whole bank is read on
every turn, so those bytes are paid over and over.
It is a format change, not a precision trade: the endpoints compute in
float32 and render that into JSON, so converting back recovers the original
bits exactly. Nothing is re-embedded and no API call is made -- migration 38
is a pure repack of what is already stored.
Unlike migrations 36 and 37 this backfill cannot be expressed in portable
SQL, so it comes through Python, batched, and pays a one-time read of every
vector to stop paying three megabytes a turn.
The JSON column stays, still written through set_vector, so a rollback finds
the vectors intact. Reading from the blob comes next; a follow-up migration
drops the old column once that is verified.
Migration SQL can now be a {dialect: sql} map -- BLOB and BYTEA have no
common spelling, and every Postgres deploy replays this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CYEJKobJ2Re4Dv7qUoSA7
This commit is contained in:
co-authored by
Claude Opus 5
parent
7ee5ceea6c
commit
c56864877a
@@ -173,13 +173,18 @@ def build_fixture(args, rng: random.Random) -> tuple[int, int]:
|
||||
))
|
||||
|
||||
for i in range(args.memories):
|
||||
db.add(models.Memory(
|
||||
memory = models.Memory(
|
||||
adventure_id=adventure.id,
|
||||
text=f"{MEMORY_TEXT} ({i})",
|
||||
embedding=[rng.uniform(-1.0, 1.0) for _ in range(EMBEDDING_DIMS)],
|
||||
source_start=i * memorybank.MEMORY_INTERVAL,
|
||||
source_end=i * memorybank.MEMORY_INTERVAL + memorybank.MEMORY_INTERVAL - 1,
|
||||
))
|
||||
)
|
||||
# Through the same door the app uses, so the fixture cannot end up
|
||||
# storing vectors in a shape production never produces.
|
||||
memorybank.set_vector(
|
||||
memory, [rng.uniform(-1.0, 1.0) for _ in range(EMBEDDING_DIMS)]
|
||||
)
|
||||
db.add(memory)
|
||||
|
||||
db.commit()
|
||||
return adventure.id, user.id
|
||||
|
||||
Reference in New Issue
Block a user