parththakkar106andClaude Opus 5
ae6e5af6c7
One column is 89% of the database and the free tier allows 512 MB. Reads were already solved -- the column is deferred, so a page load never touches it and one screen fetches one row at a time -- but nothing had costed storage, and storage is the constraint with a cliff: 99.6 MB used, ~94 kB of disk per action, so the ceiling arrives around 5,400 actions and 944 are stored. Postgres already compresses it and only gets 1.7x. pglz is tuned for fast decompression of data a query might filter on, and nothing has ever filtered on an assembled prompt -- it is written once and read whole, rarely, by the Insights viewer. zlib gets 3.5x on the same text for a decompress on a request that already made an LLM call. Done as a TypeDecorator rather than a second column, so every call site still writes a dict and reads a dict back, and deferred/undefer/load_only keep naming the same attribute. Only the storage format moves. Migrations 43-45: add the bytea, convert into it, drop the original, rename. The backfill is the one destructive step in the file -- 44 removes the only other copy -- so it decompresses every row and compares it against what went in, and a row that fails aborts the run. The whole loop is one transaction, so an abort rolls the DROP back and the prompts are still there. Verified on real Postgres, replaying 43-45 from a pre-43 schema on a throwaway Neon database: 720,864 B of JSON became 204,293 B of bytea, 3.53x, the column came out named context_snapshot, every snapshot compared equal and the one NULL stayed NULL. Postgres does not return the disk by itself: DROP COLUMN only marks the column gone and the backfill leaves a dead tuple per row, so the table peaks near twice its size before settling. The deploy needs one VACUUM FULL to collect it; the migration comment says so. The egress fixture's snapshots are prose now rather than "x" * 20_000, and the prose generator moved to tools/fakeprose.py so the harness and the tests share one definition. A repeated character compresses a thousandfold: against the old fixture a compressed column looked free and the byte ceilings would have been guarding nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Dvvqn9ZDR4ixeFPHNbww7
71 lines
2.7 KiB
Python
71 lines
2.7 KiB
Python
"""Storing a JSON column compressed.
|
|
|
|
`actions.context_snapshot` holds the entire assembled prompt for a turn. It is
|
|
89% of the database — 150.8 MB of JSON across 944 actions on production, and
|
|
232 KB a row on the longest adventure — and the free tier this deploys to
|
|
allows 512 MB. Reads are not the problem: the column is deferred, so a page
|
|
load never touches it and exactly one endpoint fetches one row of it at a
|
|
time. Storage is the problem, and storage has a cliff.
|
|
|
|
Postgres already compresses it. TOAST brings 150.8 MB down to ~89 MB, a factor
|
|
of 1.7 — pglz is chosen for decompression speed on data a query might filter
|
|
on, which this never is. Nothing filters on a prompt; it is written once and
|
|
read whole, occasionally, by one screen. zlib at the application layer gets
|
|
three to four times on the same text, and the cost is a decompress on a
|
|
request that already costs an LLM call.
|
|
|
|
Doing it as a TypeDecorator rather than a second column keeps every call site
|
|
writing `action.context_snapshot = {...}` and reading a dict back, and keeps
|
|
`deferred=True`, `undefer()` and `load_only()` naming the same attribute they
|
|
named before. The storage format changes; nothing else does.
|
|
|
|
Level 6 is zlib's default and the knee of the curve here: 9 spends noticeably
|
|
more CPU on prompt text for about a percent more space.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import zlib
|
|
|
|
from sqlalchemy import LargeBinary
|
|
from sqlalchemy.types import TypeDecorator
|
|
|
|
LEVEL = 6
|
|
|
|
|
|
def pack(value) -> bytes:
|
|
"""A JSON-able value as compressed UTF-8."""
|
|
raw = json.dumps(value, separators=(",", ":"), default=str).encode("utf-8")
|
|
return zlib.compress(raw, LEVEL)
|
|
|
|
|
|
def unpack(blob: bytes) -> object:
|
|
"""The value `pack` was given."""
|
|
return json.loads(zlib.decompress(bytes(blob)).decode("utf-8"))
|
|
|
|
|
|
class CompressedJSON(TypeDecorator):
|
|
"""A JSON column stored as zlib-compressed UTF-8 in a BLOB/BYTEA.
|
|
|
|
`cache_ok = True`: the type carries no per-instance configuration, so
|
|
SQLAlchemy may reuse a compiled statement across instances of it.
|
|
"""
|
|
|
|
impl = LargeBinary
|
|
cache_ok = True
|
|
|
|
def process_bind_param(self, value, dialect):
|
|
return None if value is None else pack(value)
|
|
|
|
def process_result_value(self, value, dialect):
|
|
# Tolerate a row the backfill has not reached yet, or one written
|
|
# before the conversion: a snapshot that cannot be read back is worth
|
|
# less than the screen that shows it, and never worth a 500 on the
|
|
# turn that happens to load it.
|
|
if value is None:
|
|
return None
|
|
try:
|
|
return unpack(value)
|
|
except (zlib.error, UnicodeDecodeError, ValueError):
|
|
return None
|