Files
interactive-story/backend/app/limits.py
T
parththakkar106andClaude Opus 5 041f9e25f3 Count the visits, and say whether anyone got anywhere
A hosted demo raises a question a local app never does: is anyone using it,
and do they reach the part that matters? `/analytics` answers it — visitors,
pages, referrers, countries, devices, which shared scenarios get played, turns
and demo-key spend, API and turn errors, and a funnel from visited to played a
turn to signed up.

Not a third-party script, for reasons specific to this one. The CSP allows
`script-src 'self'`, so a tracker means loosening it; adblockers eat the
popular ones, which silently biases exactly the technical audience this
project gets shown to; and none of them can see the measurement that actually
matters here, which is a turn, not a pageview.

**A visit is a write and never a read.** After the 189x egress fix it would be
perverse to add a feature that reads rows per request, so counts accumulate in
a process-local dict and flush every 60s as UPSERTs. Storage is a generic
`(day, metric, label) -> hits` counter, so measuring something new later costs
a constant rather than a migration, plus one row per visitor per day for the
funnel flags. Every dashboard query is a GROUP BY returning tens of rows
however much traffic sits behind it; a month reads back in a few kilobytes.
The buffer's cost is that a hard restart can lose up to a minute — the flusher
also runs on shutdown, and a tier that sleeps when idle sleeps on an empty
buffer anyway.

**The counters are anonymous; the access log beside them is not, on purpose.**
A visitor is `HMAC(secret, "visitor:<user id>")` truncated to 32 chars —
one-way, so `analytics_daily` and `analytics_visitor_days` cannot be joined
back to `users`, and keyed, so no client can compute one. Story content never
reaches that module, and the only content it ever names is a seeded public
scenario's title; a player's own titles are theirs. `accesslog.py` is the
identifying half and is a separate module writing a separate table so that
separation is a property of the code rather than a convention: `access_events`
records sessions, sign-ins, registrations and failed attempts with address,
email and device, read on a second tab of the same page behind the same gate.

Both halves are gated on `AIDND_ANALYTICS_EMAILS`, not `POWER_USERS`. An
unmetered tester is not automatically someone who should see the traffic. The
route 404s and the nav link is absent for everyone else, the same treatment
AI Chat gets; unset in a hosted deploy means nobody sees it, including me.

Three things came out of building it that a test would not have suggested.

**A failed turn is an HTTP 200 with a bad ending.** The status-code middleware
cannot see one, so a demo whose model had started refusing every request would
look perfectly healthy from outside. All five SSE error paths in
`_generate_turn` now go through a `turn_error()` helper that counts on the way
out. Error buckets elsewhere are labelled by the matched route template rather
than the requested path — one bucket per endpoint instead of one per adventure
id, and, the reason it isn't merely tidier, an unmatched path is entirely
attacker-chosen, so labelling by it would let anyone mint rows.

**The funnel counts people, not clicks.** A player who starts six adventures
is one person who started an adventure. That is the whole reason the
per-visitor-day table exists; its flags only ever turn on, and `is_new` is
settled by the first write of a visitor's first day.

**The tests run on SQLite and production is Neon.** A flush that raises is
caught and logged, so a dialect mistake in the UPSERTs would have stayed
invisible until the dashboard quietly never filled.
`test_the_upserts_compile_for_postgres` compiles both statements against the
Postgres dialect without connecting to one.

Two things this leans on elsewhere. `limits._client_ip` is now public
`client_ip`: the access log needs the same answer, and two functions both
deciding which hop is the caller's is how one of them ends up trusting a
header it shouldn't. And the cleanup sweeper now starts if *either* job has
work — a deployment can keep every guest forever and still want its
visitor-day rows aged out.

No migration. Both tables are new and `bootstrap()` calls `create_all` on
existing databases too, the route `branches` took in Phase 14, so
`LATEST_VERSION` is still 64.

497 tests green, frontend lint and build clean, driven by hand against a
synthetic 90-day fixture at 1568px. The narrow-screen layout follows the
existing 720px block but is unverified: `resize_window` is ignored on a
maximized Chrome and `frame-ancestors 'none'` rules out checking it in a sized
iframe. Also repaired here: a rename in test_ratelimit_hardening.py had run
through the test names themselves, leaving `testclient_ip_*` — still collected
by pytest, which is why it passed unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
2026-08-22 16:24:42 +05:30

321 lines
13 KiB
Python

"""Phase 9 — abuse guards for hosted (multi-user) deployments.
Rate limits and row caps are no-ops in local mode: a single local player
should never be throttled by their own app. Values are hardcoded on purpose —
generous enough that a legitimate player never notices, tight enough that a
hostile visitor can't burn the demo key, peg the CPU, or bloat the database.
"""
import json
import os
import threading
import time
from collections import defaultdict, deque
from fastapi import HTTPException, Request
from sqlalchemy import func
from sqlalchemy.orm import Session
from . import auth, models
# ---------- Rate limiting ----------
# Fixed windows per (scope, caller). In-memory: fine for the single-process
# deployment this app targets (and the worst case after a restart is a brief
# extra allowance).
# scope -> (max requests, window seconds)
RATE_LIMITS: dict[str, tuple[int, int]] = {
"turn": (10, 60), # AI turn generation (demo key also has a daily cap)
"chat": (30, 60), # AI Chat scratchpad (power users only)
"script-test": (30, 60), # sandboxed, but each run costs up to 2s CPU
"connection-test": (10, 60), # outbound HTTP to a user-supplied URL
"import": (30, 60), # large writes
"auth": (10, 300), # register/login attempts, per IP
"guest": (30, 300), # new guest users, per IP (each is a DB row)
# Pageview beacons. Generous — a real reader clicking around a SPA fires a
# handful a minute — but low enough that nobody can inflate the traffic
# numbers faster than they could by actually reloading the page.
"analytics": (120, 60),
}
_windows: dict[tuple[str, str], deque] = defaultdict(deque)
_windows_guard = threading.Lock()
# How many proxy hops sit between the app and the real client. On Render (and
# most PaaS) that's one: the platform's edge appends the connecting IP to the
# RIGHT of X-Forwarded-For. A client can prepend anything it likes to the left,
# but it cannot push a value past the edge's own append — so the trustworthy
# client IP is the (hops)-th entry from the right, NOT uvicorn's leftmost pick.
# Trusting the leftmost let anyone rotate X-Forwarded-For to mint a fresh
# rate-limit bucket per request and bypass the auth/guest limits entirely.
# Override with AIDND_TRUSTED_PROXY_HOPS if the deployment adds more hops.
TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1))
def client_ip(request: Request) -> str:
"""The real client IP, resistant to a spoofed X-Forwarded-For. Takes the
hop the trusted edge appended (rightmost minus any extra trusted hops);
falls back to the socket peer when no forwarded header is present
(local/dev, or a direct connection).
Public because the access log needs the same answer, and two functions that
both decide "which address is the caller's" is how one of them ends up
trusting a header it shouldn't."""
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
parts = [p.strip() for p in forwarded.split(",") if p.strip()]
if parts:
return parts[-min(TRUSTED_PROXY_HOPS, len(parts))]
return request.client.host if request.client else "unknown"
def rate_limit(scope: str, request: Request, user: models.User | None = None) -> None:
"""429 when the caller exceeds the scope's window. Keyed per user when one
is known (accounts survive IP changes), per IP otherwise."""
if not auth.MULTI_USER:
return
limit, window_seconds = RATE_LIMITS[scope]
key = (scope, f"u{user.id}" if user else f"ip{client_ip(request)}")
now = time.time()
with _windows_guard:
window = _windows[key]
while window and window[0] < now - window_seconds:
window.popleft()
if len(window) >= limit:
raise HTTPException(
429, "You're doing that too fast — wait a minute and try again."
)
window.append(now)
if len(_windows) > 10_000:
_prune(now)
# ---------- Per-account login throttle ----------
# Defense in depth beside the per-IP `auth` limit: that one can be diluted by a
# botnet (many real source IPs, one bucket each), so it can't by itself stop a
# distributed guessing run against a single account. This cap keys on the target
# email instead of the caller, so guessing ONE account's password stays
# expensive regardless of how many addresses it comes from. Failures only — a
# correct password clears the record — and it's a short sliding window, not a
# hard lock, so a user mistyping a few times recovers on their own in minutes.
# Tradeoff: an attacker can keep a known account throttled (a nuisance), which
# is strictly preferable to letting it be brute-forced.
LOGIN_FAIL_LIMIT = 8 # failed attempts per account...
LOGIN_FAIL_WINDOW = 900 # ...within this many seconds (15 min)
_login_fails: dict[str, deque] = defaultdict(deque)
_login_guard = threading.Lock()
def check_login_allowed(email: str) -> None:
"""429 when an account has too many recent failed logins. Call before
verifying the password so guesses don't even reach the hash."""
if not auth.MULTI_USER:
return
now = time.time()
with _login_guard:
window = _login_fails[email]
while window and window[0] < now - LOGIN_FAIL_WINDOW:
window.popleft()
if len(window) >= LOGIN_FAIL_LIMIT:
raise HTTPException(
429,
"Too many failed sign-in attempts for this account — "
"wait a few minutes and try again.",
)
def note_login_failure(email: str) -> None:
"""Record one failed attempt against `email`."""
if not auth.MULTI_USER:
return
now = time.time()
with _login_guard:
_login_fails[email].append(now)
if len(_login_fails) > 10_000: # bound the map on a flood of unique emails
stale = [
key for key, window in _login_fails.items()
if not window or window[-1] < now - LOGIN_FAIL_WINDOW
]
for key in stale:
del _login_fails[key]
def note_login_success(email: str) -> None:
"""A correct password wipes the account's failure streak."""
with _login_guard:
_login_fails.pop(email, None)
def _prune(now: float) -> None:
"""Drop callers whose whole window has expired (call with guard held) so
the per-IP dict can't grow without bound."""
longest = max(seconds for _, seconds in RATE_LIMITS.values())
stale = [key for key, window in _windows.items()
if not window or window[-1] < now - longest]
for key in stale:
del _windows[key]
# ---------- Per-user row caps ----------
MAX_ADVENTURES_PER_USER = 100
MAX_SCENARIOS_PER_USER = 200
MAX_SCRIPTS_PER_USER = 200
MAX_STORY_CARDS_PER_OWNER = 200 # per scenario or adventure
MAX_MEMORIES_PER_ADVENTURE = 1000
MAX_ACTIONS_PER_ADVENTURE = 5000
# Phase 14, SP6. A branch per divergence somebody built a story on, so a tree
# with more of them than a story has turns is a file, not a game. Import-only
# for now: forking is a POST that adds one row and has no cap of its own, and
# the cap that matters there is `MAX_ACTIONS_PER_ADVENTURE` above it.
MAX_BRANCHES_PER_ADVENTURE = 1000
def check_row_cap(
kind: str,
db: Session,
user: models.User,
*,
adventure: models.Adventure | None = None,
scenario_id: int | None = None,
adventure_id: int | None = None,
) -> None:
"""409 with a friendly message when creating one more row of `kind` would
exceed its cap. Ownership of the passed scenario/adventure has already
been checked by the caller."""
if not auth.MULTI_USER:
return
if kind == "adventures":
count = _count(db, models.Adventure, models.Adventure.user_id == user.id)
cap, subject, hint = (
MAX_ADVENTURES_PER_USER, "adventures",
"delete one you no longer play to make room",
)
elif kind == "scenarios":
count = _count(db, models.Scenario, models.Scenario.user_id == user.id)
cap, subject, hint = (
MAX_SCENARIOS_PER_USER, "scenarios", "delete one to make room"
)
elif kind == "scripts":
count = _count(db, models.Script, models.Script.user_id == user.id)
cap, subject, hint = (
MAX_SCRIPTS_PER_USER, "scripts", "delete one to make room"
)
elif kind == "story_cards":
owner_filter = (
models.StoryCard.scenario_id == scenario_id
if scenario_id is not None
else models.StoryCard.adventure_id == adventure_id
)
count = _count(db, models.StoryCard, owner_filter)
cap, subject, hint = (
MAX_STORY_CARDS_PER_OWNER, "story cards here", "delete one to make room"
)
elif kind == "memories":
count = _count(db, models.Memory, models.Memory.adventure_id == adventure.id)
cap, subject, hint = (
MAX_MEMORIES_PER_ADVENTURE, "memories in this adventure",
"delete some to make room",
)
elif kind == "actions":
# Every action of the adventure — the whole tree, not the path being
# played. That is the number that costs storage, and nothing is ever
# auto-pruned, so it is the right one to cap on. It does mean a heavily
# branched adventure reaches the cap while its *story* is shorter than
# the cap, which is why the message counts "actions in this adventure"
# rather than turns.
count = _count(db, models.Action, models.Action.adventure_id == adventure.id)
cap, subject, hint = (
MAX_ACTIONS_PER_ADVENTURE, "actions in this adventure",
"export it and continue in a new adventure",
)
else: # pragma: no cover — programming error, not user input
raise ValueError(f"Unknown row cap kind: {kind}")
if count >= cap:
raise HTTPException(409, f"You've reached the limit of {cap} {subject} — {hint}.")
def _count(db: Session, model, condition) -> int:
return db.query(func.count(model.id)).filter(condition).scalar() or 0
_BUNDLE_LIST_CAPS = {
"story_cards": MAX_STORY_CARDS_PER_OWNER,
"memories": MAX_MEMORIES_PER_ADVENTURE,
"actions": MAX_ACTIONS_PER_ADVENTURE,
"branches": MAX_BRANCHES_PER_ADVENTURE,
}
def check_bundle_lists(**lists) -> None:
"""409 when an import bundle's lists exceed the same caps live creation
enforces (kwargs: story_cards=, memories=, actions=, branches=)."""
if not auth.MULTI_USER:
return
for name, value in lists.items():
cap = _BUNDLE_LIST_CAPS[name]
if isinstance(value, list) and len(value) > cap:
noun = name.replace("_", " ")
raise HTTPException(
409, f"This file contains {len(value)} {noun} — the limit is {cap}."
)
# ---------- Request body size ----------
# Generous enough for the biggest legitimate payload (an adventure export with
# thousands of actions), applied in every mode — no honest request comes close.
MAX_BODY_BYTES = 2 * 1024 * 1024
MAX_IMPORT_BODY_BYTES = 20 * 1024 * 1024
class BodySizeLimitMiddleware:
"""Rejects oversized request bodies by declared Content-Length. Pure ASGI
(not BaseHTTPMiddleware) so SSE responses stream through untouched.
Chunked uploads without a length are refused — every real client of this
API (browser fetch, curl with a file) sends Content-Length."""
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] == "http" and scope.get("method") in ("POST", "PUT", "PATCH"):
headers = {k.decode("latin-1").lower(): v.decode("latin-1")
for k, v in scope.get("headers", [])}
limit = (
MAX_IMPORT_BODY_BYTES
if scope.get("path", "").endswith("/import")
else MAX_BODY_BYTES
)
length = headers.get("content-length")
problem = None
if length is None:
if "chunked" in headers.get("transfer-encoding", "").lower():
problem = (411, "Content-Length is required.")
else:
try:
if int(length) > limit:
problem = (
413,
f"Request too large (limit {limit // (1024 * 1024)} MB).",
)
except ValueError:
problem = (400, "Invalid Content-Length.")
if problem:
await _send_json_error(send, *problem)
return
await self.app(scope, receive, send)
async def _send_json_error(send, status: int, detail: str) -> None:
body = json.dumps({"detail": detail}).encode()
await send({
"type": "http.response.start",
"status": status,
"headers": [(b"content-type", b"application/json"),
(b"content-length", str(len(body)).encode())],
})
await send({"type": "http.response.body", "body": body})