Files
interactive-story/backend/app/routers/analytics.py
T
parththakkar106andClaude Opus 5 041f9e25f3 Count the visits, and say whether anyone got anywhere
A hosted demo raises a question a local app never does: is anyone using it,
and do they reach the part that matters? `/analytics` answers it — visitors,
pages, referrers, countries, devices, which shared scenarios get played, turns
and demo-key spend, API and turn errors, and a funnel from visited to played a
turn to signed up.

Not a third-party script, for reasons specific to this one. The CSP allows
`script-src 'self'`, so a tracker means loosening it; adblockers eat the
popular ones, which silently biases exactly the technical audience this
project gets shown to; and none of them can see the measurement that actually
matters here, which is a turn, not a pageview.

**A visit is a write and never a read.** After the 189x egress fix it would be
perverse to add a feature that reads rows per request, so counts accumulate in
a process-local dict and flush every 60s as UPSERTs. Storage is a generic
`(day, metric, label) -> hits` counter, so measuring something new later costs
a constant rather than a migration, plus one row per visitor per day for the
funnel flags. Every dashboard query is a GROUP BY returning tens of rows
however much traffic sits behind it; a month reads back in a few kilobytes.
The buffer's cost is that a hard restart can lose up to a minute — the flusher
also runs on shutdown, and a tier that sleeps when idle sleeps on an empty
buffer anyway.

**The counters are anonymous; the access log beside them is not, on purpose.**
A visitor is `HMAC(secret, "visitor:<user id>")` truncated to 32 chars —
one-way, so `analytics_daily` and `analytics_visitor_days` cannot be joined
back to `users`, and keyed, so no client can compute one. Story content never
reaches that module, and the only content it ever names is a seeded public
scenario's title; a player's own titles are theirs. `accesslog.py` is the
identifying half and is a separate module writing a separate table so that
separation is a property of the code rather than a convention: `access_events`
records sessions, sign-ins, registrations and failed attempts with address,
email and device, read on a second tab of the same page behind the same gate.

Both halves are gated on `AIDND_ANALYTICS_EMAILS`, not `POWER_USERS`. An
unmetered tester is not automatically someone who should see the traffic. The
route 404s and the nav link is absent for everyone else, the same treatment
AI Chat gets; unset in a hosted deploy means nobody sees it, including me.

Three things came out of building it that a test would not have suggested.

**A failed turn is an HTTP 200 with a bad ending.** The status-code middleware
cannot see one, so a demo whose model had started refusing every request would
look perfectly healthy from outside. All five SSE error paths in
`_generate_turn` now go through a `turn_error()` helper that counts on the way
out. Error buckets elsewhere are labelled by the matched route template rather
than the requested path — one bucket per endpoint instead of one per adventure
id, and, the reason it isn't merely tidier, an unmatched path is entirely
attacker-chosen, so labelling by it would let anyone mint rows.

**The funnel counts people, not clicks.** A player who starts six adventures
is one person who started an adventure. That is the whole reason the
per-visitor-day table exists; its flags only ever turn on, and `is_new` is
settled by the first write of a visitor's first day.

**The tests run on SQLite and production is Neon.** A flush that raises is
caught and logged, so a dialect mistake in the UPSERTs would have stayed
invisible until the dashboard quietly never filled.
`test_the_upserts_compile_for_postgres` compiles both statements against the
Postgres dialect without connecting to one.

Two things this leans on elsewhere. `limits._client_ip` is now public
`client_ip`: the access log needs the same answer, and two functions both
deciding which hop is the caller's is how one of them ends up trusting a
header it shouldn't. And the cleanup sweeper now starts if *either* job has
work — a deployment can keep every guest forever and still want its
visitor-day rows aged out.

No migration. Both tables are new and `bootstrap()` calls `create_all` on
existing databases too, the route `branches` took in Phase 14, so
`LATEST_VERSION` is still 64.

497 tests green, frontend lint and build clean, driven by hand against a
synthetic 90-day fixture at 1568px. The narrow-screen layout follows the
existing 720px block but is unverified: `resize_window` is ignored on a
maximized Chrome and `frame-ancestors 'none'` rules out checking it in a sized
iframe. Also repaired here: a rename in test_ratelimit_hardening.py had run
through the test names themselves, leaving `testclient_ip_*` — still collected
by pytest, which is why it passed unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
2026-08-22 16:24:42 +05:30

138 lines
5.0 KiB
Python

"""Visit analytics: one endpoint the browser writes to, one the owner reads.
The split matters. `/collect` is public and takes exactly one fact — which
page was viewed — because anything a stranger can POST is a number a stranger
can invent. Everything the dashboard actually relies on (turns, adventures,
sign-ups, demo spend, errors) is recorded server-side by the code performing
it, so those counts are as trustworthy as the app itself.
The two reading endpoints are owner-only and 404 for everyone else, the same
way the AI Chat router does: a feature nobody else can use is better off not
appearing to exist. `/summary` serves the anonymous counters (analytics.py,
which stores nothing that points at a person) and `/access` serves the access
log (accesslog.py, which identifies people on purpose).
"""
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from .. import accesslog, analytics, auth, limits, models
from ..database import get_db
router = APIRouter(prefix="/api/analytics", tags=["analytics"])
def owner(
db: Session = Depends(get_db),
user: models.User = Depends(auth.get_current_user),
) -> models.User:
"""Gate for the reading half. 404, not 403 — see the module docstring."""
if not auth.is_owner(user):
raise HTTPException(404, "Not found")
return user
Owner = Depends(owner)
class Pageview(BaseModel):
"""What the SPA reports on a page load or a route change.
`first` marks a real page load rather than a client-side navigation: the
things that describe a *visit* rather than a *view* — where it came from,
on what kind of device, from which country — are recorded only then, so a
visitor who clicks around five pages is still one referral.
"""
path: str = Field("", max_length=300)
referrer: str = Field("", max_length=500)
first: bool = False
@router.post("/collect", status_code=204)
def collect(
payload: Pageview,
request: Request,
db: Session = Depends(get_db),
) -> Response:
"""Record one pageview. Always 204, even when nothing was counted: the
browser has no business knowing whether it was."""
limits.rate_limit("analytics", request)
# Resolved by hand rather than through get_current_user: a pageview that
# arrives before /auth/me has minted a session should still be counted as a
# view, not turned into a 401 the SPA has to handle.
user = (
auth.resolve_session_user(request, db)
if auth.MULTI_USER
else auth.local_user(db)
)
# The operator's own clicking is not traffic. Only in multi-user mode —
# locally everyone is the owner, and excluding them would leave the
# dashboard permanently empty on the machine it is developed on.
if auth.MULTI_USER and user is not None and auth.is_owner(user):
return Response(status_code=204)
analytics.record(analytics.M_PAGE, analytics.normalize_route(payload.path))
analytics.record_visit(user)
if payload.first:
referrer = analytics.normalize_referrer(
payload.referrer, request.url.hostname or ""
)
if referrer: # "" means same-origin, which is not a referral
analytics.record(analytics.M_REFERRER, referrer)
analytics.record(
analytics.M_DEVICE,
analytics.device_of(request.headers.get("user-agent", "")),
)
analytics.record(analytics.M_COUNTRY, analytics.country_of(request.headers))
return Response(status_code=204)
@router.get("/summary")
def summary(
days: int = Query(30, ge=1, le=365),
db: Session = Depends(get_db),
_user: models.User = Owner,
) -> dict:
"""The whole dashboard in one aggregate response — a few kilobytes however
much traffic sits behind it."""
return analytics.summary(db, days)
@router.get("/access")
def access_log(
limit: int = Query(50, ge=1, le=200),
before_id: int | None = Query(None),
kind: str | None = Query(None),
q: str | None = Query(None, max_length=120),
db: Session = Depends(get_db),
_user: models.User = Owner,
) -> dict:
"""A page of the access log, newest first.
Unlike /summary this returns rows about people, which is the whole point of
it — so it is behind the same owner gate, paged rather than dumped, and has
no counterpart the people it describes can reach.
"""
page = accesslog.recent(
db, limit=limit, before_id=before_id, kind=kind, query=q
)
return {
"events": [
{
"id": event.id,
"at": event.at.isoformat(),
"kind": event.kind,
"who": event.who,
"is_guest": event.is_guest,
"ip": event.ip,
"country": event.country,
"device": event.device,
"user_agent": event.user_agent,
}
for event in page["events"]
],
"has_more": page["has_more"],
}