From 041f9e25f389ef01a857b0bdf77dec7896217a49 Mon Sep 17 00:00:00 2001 From: parththakkar106 Date: Sat, 22 Aug 2026 16:24:42 +0530 Subject: [PATCH] Count the visits, and say whether anyone got anywhere MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A hosted demo raises a question a local app never does: is anyone using it, and do they reach the part that matters? `/analytics` answers it — visitors, pages, referrers, countries, devices, which shared scenarios get played, turns and demo-key spend, API and turn errors, and a funnel from visited to played a turn to signed up. Not a third-party script, for reasons specific to this one. The CSP allows `script-src 'self'`, so a tracker means loosening it; adblockers eat the popular ones, which silently biases exactly the technical audience this project gets shown to; and none of them can see the measurement that actually matters here, which is a turn, not a pageview. **A visit is a write and never a read.** After the 189x egress fix it would be perverse to add a feature that reads rows per request, so counts accumulate in a process-local dict and flush every 60s as UPSERTs. Storage is a generic `(day, metric, label) -> hits` counter, so measuring something new later costs a constant rather than a migration, plus one row per visitor per day for the funnel flags. Every dashboard query is a GROUP BY returning tens of rows however much traffic sits behind it; a month reads back in a few kilobytes. The buffer's cost is that a hard restart can lose up to a minute — the flusher also runs on shutdown, and a tier that sleeps when idle sleeps on an empty buffer anyway. **The counters are anonymous; the access log beside them is not, on purpose.** A visitor is `HMAC(secret, "visitor:")` truncated to 32 chars — one-way, so `analytics_daily` and `analytics_visitor_days` cannot be joined back to `users`, and keyed, so no client can compute one. Story content never reaches that module, and the only content it ever names is a seeded public scenario's title; a player's own titles are theirs. `accesslog.py` is the identifying half and is a separate module writing a separate table so that separation is a property of the code rather than a convention: `access_events` records sessions, sign-ins, registrations and failed attempts with address, email and device, read on a second tab of the same page behind the same gate. Both halves are gated on `AIDND_ANALYTICS_EMAILS`, not `POWER_USERS`. An unmetered tester is not automatically someone who should see the traffic. The route 404s and the nav link is absent for everyone else, the same treatment AI Chat gets; unset in a hosted deploy means nobody sees it, including me. Three things came out of building it that a test would not have suggested. **A failed turn is an HTTP 200 with a bad ending.** The status-code middleware cannot see one, so a demo whose model had started refusing every request would look perfectly healthy from outside. All five SSE error paths in `_generate_turn` now go through a `turn_error()` helper that counts on the way out. Error buckets elsewhere are labelled by the matched route template rather than the requested path — one bucket per endpoint instead of one per adventure id, and, the reason it isn't merely tidier, an unmatched path is entirely attacker-chosen, so labelling by it would let anyone mint rows. **The funnel counts people, not clicks.** A player who starts six adventures is one person who started an adventure. That is the whole reason the per-visitor-day table exists; its flags only ever turn on, and `is_new` is settled by the first write of a visitor's first day. **The tests run on SQLite and production is Neon.** A flush that raises is caught and logged, so a dialect mistake in the UPSERTs would have stayed invisible until the dashboard quietly never filled. `test_the_upserts_compile_for_postgres` compiles both statements against the Postgres dialect without connecting to one. Two things this leans on elsewhere. `limits._client_ip` is now public `client_ip`: the access log needs the same answer, and two functions both deciding which hop is the caller's is how one of them ends up trusting a header it shouldn't. And the cleanup sweeper now starts if *either* job has work — a deployment can keep every guest forever and still want its visitor-day rows aged out. No migration. Both tables are new and `bootstrap()` calls `create_all` on existing databases too, the route `branches` took in Phase 14, so `LATEST_VERSION` is still 64. 497 tests green, frontend lint and build clean, driven by hand against a synthetic 90-day fixture at 1568px. The narrow-screen layout follows the existing 720px block but is unverified: `resize_window` is ignored on a maximized Chrome and `frame-ancestors 'none'` rules out checking it in a sized iframe. Also repaired here: a rename in test_ratelimit_hardening.py had run through the test names themselves, leaving `testclient_ip_*` — still collected by pytest, which is why it passed unnoticed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY --- README.md | 23 +- backend/.env.example | 14 + backend/app/accesslog.py | 160 +++++++ backend/app/analytics.py | 556 ++++++++++++++++++++++ backend/app/auth.py | 19 + backend/app/cleanup.py | 36 +- backend/app/limits.py | 20 +- backend/app/main.py | 46 +- backend/app/models.py | 89 +++- backend/app/routers/adventures.py | 41 +- backend/app/routers/analytics.py | 137 ++++++ backend/app/routers/auth.py | 15 +- backend/app/routers/scenarios.py | 10 +- backend/tests/test_accesslog.py | 215 +++++++++ backend/tests/test_analytics.py | 392 +++++++++++++++ backend/tests/test_ratelimit_hardening.py | 18 +- docs/GUIDE.md | 38 ++ frontend/src/App.jsx | 27 +- frontend/src/api.js | 22 + frontend/src/index.css | 309 ++++++++++++ frontend/src/main.jsx | 3 + frontend/src/pages/Analytics.jsx | 476 ++++++++++++++++++ plan/STATUS.md | 70 ++- render.yaml | 7 + 24 files changed, 2698 insertions(+), 45 deletions(-) create mode 100644 backend/app/accesslog.py create mode 100644 backend/app/analytics.py create mode 100644 backend/app/routers/analytics.py create mode 100644 backend/tests/test_accesslog.py create mode 100644 backend/tests/test_analytics.py create mode 100644 frontend/src/pages/Analytics.jsx diff --git a/README.md b/README.md index e1b2c70..5740404 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ player input ``` frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI - ├─ routers/ auth, scenarios, adventures, story cards, scripts, chat, settings, debug + ├─ routers/ auth, scenarios, adventures, story cards, scripts, chat, settings, analytics, debug ├─ models.py SQLAlchemy: User, Scenario, Adventure, Branch, Action, StoryCard, Script, Settings, Memory ├─ migrations.py hand-rolled, versioned via PRAGMA user_version (64 and counting) ├─ auth.py guest/registered users, sessions, shared demo key @@ -162,6 +162,7 @@ frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI ├─ worldstate/ the stat engine: clamps, cooldowns, bands, milestones ├─ scripting/ quickjs sandbox + AI Dungeon API surface ├─ memorybank.py auto-summarization + embedding retrieval + ├─ analytics.py buffered visit counters + the owner's dashboard query ├─ bundle.py the export/import formats, v2 (tree) and a v1 reader ├─ providers/ OpenAI-compatible adapter, streaming └─ data.db SQLite (path overridable via AIDND_DB_PATH) @@ -172,7 +173,7 @@ development Vite proxies `/api` to FastAPI. ## Tests -440 backend tests — unit plus full HTTP integration through the real quickjs scripting engine, +497 backend tests — unit plus full HTTP integration through the real quickjs scripting engine, with the LLM provider mocked. CI runs them on every push, alongside the frontend lint/build and a Docker image build. @@ -202,6 +203,21 @@ the most interesting engineering in the repo: stay cheap because the lineage is windowed like the history is, so the number of SQL clauses is bounded by the context window rather than by the number of forks. +## Visit analytics + +The hosted demo keeps its own analytics: an owner-only dashboard at `/analytics` showing +traffic, which shared scenarios get played, turns and demo-key spend, errors, and a funnel +from *visited* to *played a turn* to *signed up*. It is visible only to the emails listed in +`AIDND_ANALYTICS_EMAILS`, and the route 404s for everyone else. + +Built into the app rather than bolted on with a third-party script, for reasons specific to +this one: the CSP allows `script-src 'self'`, adblockers eat the popular trackers, and none +of them can see the measurement that actually matters here — a turn. Counts are aggregated +in memory and flushed as UPSERTs, so a visit is a write and never a read, and every dashboard +query is a `GROUP BY` returning tens of rows however much traffic sits behind it. That last +part is not incidental; see the egress note above for what reading rows per request costs on +this stack. + ## Deploy (Render) The repo ships a [`render.yaml`](render.yaml) blueprint: one Docker web service that @@ -211,7 +227,8 @@ serves the SPA and API same-origin, backed by external [Neon](https://neon.tech) 1. Create a **Neon** project and copy its pooled connection string. 2. In Render: **New → Blueprint**, point it at this repo. Render reads `render.yaml`. 3. Fill the secrets it prompts for (`sync: false` vars): `AIDND_DATABASE_URL` (the Neon - string) and, to offer a no-signup demo, `AIDND_DEMO_API_KEY` / `AIDND_DEMO_MODELS`. + string); to offer a no-signup demo, `AIDND_DEMO_API_KEY` / `AIDND_DEMO_MODELS`; and to + see the Visitors dashboard, `AIDND_ANALYTICS_EMAILS` (your own account's email). `AIDND_SECRET_KEY` is generated automatically and kept stable across deploys. 4. Deploy. Pushes to `main` auto-deploy thereafter. Health check: `/api/health`. diff --git a/backend/.env.example b/backend/.env.example index c4403a5..335e2f7 100644 --- a/backend/.env.example +++ b/backend/.env.example @@ -66,6 +66,20 @@ AIDND_DEMO_TURNS_PER_DAY= # Local (single-user) installs are always treated as power users. AIDND_POWER_USERS= +# --- Visit analytics --- +# Comma-separated emails allowed to see the Visitors dashboard (/analytics) and +# its nav link. Deliberately separate from AIDND_POWER_USERS: a trusted tester +# gets unmetered turns, which is no reason to hand them the traffic numbers. +# Unset = nobody sees it in a hosted deploy. Local installs always can, and are +# the only mode where the viewer's own visits are still counted (excluding them +# would leave the page permanently empty on the machine it's developed on). +# Collection itself is always on; only the dashboard is gated. +AIDND_ANALYTICS_EMAILS= +# Days to keep the one-row-per-visitor-per-day table that makes the funnel +# count people rather than clicks. The daily counters are aggregate and kept +# forever. Default: 400. Set 0 to keep visitor-days forever. +AIDND_ANALYTICS_RETENTION_DAYS= + # --- Guest retention (only active when AIDND_MULTI_USER=1) --- # Every first visit mints a guest account, so a public demo collects one row # per visitor. A guest with no activity for this many days is deleted along diff --git a/backend/app/accesslog.py b/backend/app/accesslog.py new file mode 100644 index 0000000..df7a051 --- /dev/null +++ b/backend/app/accesslog.py @@ -0,0 +1,160 @@ +"""The access log: who arrived, when, and from where. + +The deliberate opposite of analytics.py. That module counts and stores nothing +that points at a person; this one records addresses, email addresses and +devices, because an access log that cannot identify the access is not an access +log. They are kept in separate modules and separate tables on purpose — the +anonymity of the counters is then a property of the code rather than of a +convention someone has to remember. + +Owner-only, and never shown to the people it records. + +Four kinds of row: + +- `session` a browser that has a session made a request — for a guest, + their first visit; +- `login` an existing account signed in; +- `register` a guest upgraded to an account; +- `login_failed` a password attempt that didn't match, with the address tried. + +Session rows are the only ones that need thinning: `/auth/me` runs on every +page load, and a row per load would be noise rather than a log. One is written +when the day or the address changes for that user, which is the granularity a +log is actually read at — "seen on the 3rd from 1.2.3.4" — and it still catches +someone moving networks mid-day. +""" + +import logging +import threading + +from sqlalchemy import desc, or_, select +from sqlalchemy.orm import Session + +from . import analytics, models + +logger = logging.getLogger(__name__) + +SESSION = "session" +LOGIN = "login" +REGISTER = "register" +LOGIN_FAILED = "login_failed" + +MAX_UA = 200 + +# user id -> (day, ip) of the last session row written for them. Process-local +# like the rate limiter's windows, and for the same reason: this is a single +# process, and the worst case after a restart is one redundant row per user. +_last_session: dict[int, tuple[str, str]] = {} +_guard = threading.Lock() +_MAX_TRACKED = 10_000 + + +def _client_ip(request) -> str: + # Deferred: limits imports auth, which is imported by the routers that call + # this, so a module-level import here would close the loop. The spoof + # resistance lives there and must not be reimplemented — a second, laxer + # copy of "what is the client's address" is exactly how one of them ends up + # trusting a header it shouldn't. + from . import limits + + return limits.client_ip(request) + + +def describe(user: models.User) -> str: + """How a user is named in the log. Guests have no email, and their id is + the only handle anyone has for them. The third case is a local install's + implicit single user, which is also email-less but is the operator rather + than a visitor — calling that one "Guest #1" would be a small lie in the + one row they are certain to read.""" + if user.email: + return user.email + return f"Guest #{user.id}" if user.is_guest else f"Local user #{user.id}" + + +def _country(request) -> str: + """The edge's country header, or "" when there isn't one. Blank rather than + the counters' "(unknown)" label: a table column reads better as an em dash + than as a word, and an empty string is the honest value for "not known".""" + country = analytics.country_of(request.headers) + return "" if country == analytics.UNKNOWN else country + + +def record( + db: Session, + kind: str, + request, + *, + user: models.User | None = None, + who: str | None = None, +) -> None: + """Write one row. Never raises: the log watches sign-in, it doesn't guard + it, and a logging failure must not be able to lock anyone out.""" + try: + event = models.AccessEvent( + kind=kind, + user_id=user.id if user is not None else None, + who=(who if who is not None else describe(user) if user else "")[:320], + is_guest=bool(user.is_guest) if user is not None else False, + ip=_client_ip(request)[:45], + country=_country(request), + device=analytics.device_of(request.headers.get("user-agent", "")), + user_agent=(request.headers.get("user-agent") or "")[:MAX_UA], + ) + db.add(event) + db.commit() + except Exception: # pragma: no cover - defensive + db.rollback() + logger.exception("Access log write failed; continuing.") + + +def note_session(db: Session, user: models.User, request) -> None: + """A session made a request. Thinned to one row per day per address.""" + try: + today = analytics._today() + ip = _client_ip(request) + with _guard: + if _last_session.get(user.id) == (today, ip): + return + _last_session[user.id] = (today, ip) + if len(_last_session) > _MAX_TRACKED: + # Nothing here is worth persisting; dropping the map costs at + # most one extra row per active user. + _last_session.clear() + _last_session[user.id] = (today, ip) + except Exception: # pragma: no cover - defensive + logger.exception("Access log session check failed; continuing.") + return + record(db, SESSION, request, user=user) + + +def recent( + db: Session, + *, + limit: int = 50, + before_id: int | None = None, + kind: str | None = None, + query: str | None = None, +) -> dict: + """A page of the log, newest first. + + Anchored on a row id rather than an offset, like the story pager: rows keep + arriving while it is being read, and an offset would shift the page under + whoever is reading it. + """ + statement = select(models.AccessEvent).order_by(desc(models.AccessEvent.id)) + if before_id is not None: + statement = statement.where(models.AccessEvent.id < before_id) + if kind: + statement = statement.where(models.AccessEvent.kind == kind) + if query: + like = f"%{query.strip()}%" + statement = statement.where(or_( + models.AccessEvent.who.ilike(like), + models.AccessEvent.ip.ilike(like), + models.AccessEvent.country.ilike(like), + )) + # One extra row answers "is there more" without a second COUNT over the + # whole table. + rows = list(db.scalars(statement.limit(limit + 1))) + has_more = len(rows) > limit + return {"events": rows[:limit], "has_more": has_more} diff --git a/backend/app/analytics.py b/backend/app/analytics.py new file mode 100644 index 0000000..1e882c5 --- /dev/null +++ b/backend/app/analytics.py @@ -0,0 +1,556 @@ +"""Visit analytics for the hosted demo. + +A small self-hosted counter answering "did anyone visit, and did they play?", +built into the app rather than bolted on with a third-party script: the CSP in +main.py allows scripts from 'self' only, adblockers eat the popular trackers, +and none of them can see the things actually worth knowing here (turns taken, +demo-key spend, which seeded scenario people pick). + +Three rules shaped it: + +1. **Nothing personal is stored.** No IP addresses, no user agents, no user + ids, no title of anything a player wrote. A visitor appears only as an HMAC + of their user id — one-way and salted with the app's secret key, so these + tables cannot be joined back to an account even by someone holding the + database. Story content never reaches this module at all. What a *specific* + person did is deliberately unanswerable; only totals are. +2. **Egress is the budget.** Neon bills for bytes leaving the database and this + project has already paid for forgetting that once. So counts are aggregated + in memory and flushed as UPSERTs — a visit is a write, never a read — and + every dashboard query is a GROUP BY returning tens of rows, never per-visit + rows. A month of traffic costs a few kilobytes to read back. +3. **The numbers are the server's, not the browser's.** The client reports one + thing: which page was viewed. Everything that *means* something ("a turn + happened", "an account was created") is recorded by the code that does it, + where it can be neither faked by a stranger nor blocked by an extension. + +Storage is two tables, both bounded. `analytics_daily` is one row per (day, +metric, label) counter — a few dozen a day. `analytics_visitor_days` is one row +per visitor per day carrying the funnel flags, which is what makes the funnel +count *people* rather than clicks; it is the only table that grows with traffic +and cleanup ages it out. +""" + +import hmac +import logging +import os +import re +import threading +from datetime import timedelta +from hashlib import sha256 +from urllib.parse import urlsplit + +from sqlalchemy import case, func, or_, select +from sqlalchemy.dialects.postgresql import insert as pg_insert +from sqlalchemy.dialects.sqlite import insert as sqlite_insert +from sqlalchemy.orm import Session + +from . import models, security + +logger = logging.getLogger(__name__) + +# ---------- Metrics ---------- +# `metric` is the family, `label` the bucket within it. One generic counter +# table beats a column per thing measured: adding a new question later is a +# constant, not a migration. + +M_PAGE = "pageview" +M_EVENT = "event" +M_REFERRER = "referrer" +M_DEVICE = "device" +M_COUNTRY = "country" +M_SCENARIO = "scenario" # which seeded/public scenario got played +M_ERROR = "error" # " " for 4xx/5xx on /api + +EV_SCENARIO_OPEN = "scenario_opened" +EV_ADVENTURE = "adventure_created" +EV_IMPORT = "adventure_imported" +EV_TURN = "turn" +EV_DEMO_TURN = "demo_turn" # a turn billed to the shared demo key +EV_TURN_ERROR = "turn_error" +EV_SIGNUP = "signup" +EV_LOGIN = "login" + +# Events that are also funnel steps: recording one flips a flag on the +# visitor's row for the day, so the funnel counts distinct people-days instead +# of repeat clicks. The name -> column map is the whole definition of the +# funnel; the dashboard reads it back in this order. +FUNNEL_FLAGS = { + EV_SCENARIO_OPEN: "opened", + EV_ADVENTURE: "created", + EV_TURN: "played", + EV_SIGNUP: "signed_up", +} + +OTHER = "(other)" +NONE_LABEL = "(direct)" +UNKNOWN = "(unknown)" + +# ---------- Bounds ---------- +# Everything below exists so a hostile visitor can add rows to these tables no +# faster than an honest one. The only label a client can influence is the +# referrer, and these caps together mean the worst it can do is fill one day's +# referrer list with junk and then be folded into "(other)". + +MAX_LABEL_LEN = 80 +MAX_LABELS_PER_METRIC = 200 # distinct labels per metric per day, then OTHER +MAX_PENDING = 4000 # buffered entries before an inline flush +FLUSH_INTERVAL_SECONDS = 60 + +# How long the per-visitor-day rows are kept. The daily counters are tiny and +# kept forever; these are the ones that scale with traffic. A visitor whose +# last visit falls off the end counts as new again — a fair trade at this +# horizon, and it keeps the table from being a permanent record of anyone. +RETENTION_DAYS = int(os.environ.get("AIDND_ANALYTICS_RETENTION_DAYS", "400") or 400) + +_HOST_OK = re.compile(r"^[a-z0-9.-]+$") +_COUNTRY_OK = re.compile(r"^[A-Z]{2}$") +_NUMERIC_SEGMENT = re.compile(r"^\d+$") + +# SPA routes, in the shape the dashboard should show them. Anything else a +# client claims to have viewed becomes OTHER, so the page list can neither be +# polluted nor accidentally record which adventure someone is reading. +KNOWN_ROUTES = { + "/", "/adventures", "/scenarios", "/scenarios/:id", "/play/:id", + "/scripts", "/scripts/:id", "/settings", "/chat", "/analytics", +} + +# ---------- In-process buffer ---------- +# Single-process deployment (same assumption as limits.py), so a plain dict +# under a lock is the whole design. Losing up to a minute of counts to a hard +# restart is acceptable for traffic numbers, and the flusher also runs on +# shutdown; on Render's free tier the service is idle when it sleeps, so the +# buffer it sleeps on is empty anyway. + +_counts: dict[tuple[str, str, str], int] = {} +_visits: dict[tuple[str, str], set[str]] = {} # (day, visitor) -> flags +_labels_seen: dict[tuple[str, str], set[str]] = {} # (day, metric) -> labels +_guard = threading.Lock() + + +def _today() -> str: + return models.utcnow().date().isoformat() + + +def record(metric: str, label: str = "", *, n: int = 1) -> None: + """Add `n` to one counter. Never raises: analytics must not be able to + fail a request it is only watching.""" + try: + day = _today() + label = (label or "").strip()[:MAX_LABEL_LEN] + with _guard: + seen = _labels_seen.setdefault((day, metric), set()) + if label not in seen: + if len(seen) >= MAX_LABELS_PER_METRIC: + label = OTHER + else: + seen.add(label) + key = (day, metric, label) + _counts[key] = _counts.get(key, 0) + n + pending = len(_counts) + len(_visits) + except Exception: # pragma: no cover - defensive + logger.exception("Analytics counter failed; continuing.") + return + if pending >= MAX_PENDING: + flush() + + +def visitor_id(user: models.User) -> str: + """A stable but one-way handle for one visitor. + + HMAC of the user id under the app's secret key. Stable, so a returning + visitor can be told from a new one; one-way, so nothing in the analytics + tables points back at an account; keyed, so a client cannot compute one and + claim to be somebody else. One consequence worth knowing: rotating + AIDND_SECRET_KEY makes every returning visitor look new again. + """ + digest = hmac.new(security.SECRET_KEY, f"visitor:{user.id}".encode(), sha256) + return digest.hexdigest()[:32] + + +def record_visit(user: models.User | None, *, flag: str | None = None) -> None: + """Note that this visitor was here today, optionally flipping one funnel + flag. A no-op without a user: a page loaded before a session exists still + counts as a pageview, just not as a person.""" + if user is None: + return + try: + with _guard: + flags = _visits.setdefault((_today(), visitor_id(user)), set()) + if flag: + flags.add(flag) + except Exception: # pragma: no cover - defensive + logger.exception("Analytics visit failed; continuing.") + + +def record_event(name: str, user: models.User | None = None) -> None: + """One thing that happened: counted, and — if it is a funnel step — + credited to the visitor's day. This is the call sites' whole interface.""" + record(M_EVENT, name) + record_visit(user, flag=FUNNEL_FLAGS.get(name)) + + +# ---------- Normalizing what the browser reports ---------- + +def normalize_route(path: str) -> str: + """A client-reported path, reduced to one of KNOWN_ROUTES. + + Numeric segments become ":id" — both to bound the label count and because + *which* adventure someone opened is their business, not a statistic. + """ + path = (path or "/").split("?")[0].split("#")[0] + if not path.startswith("/"): + path = "/" + path + if len(path) > 1: + path = path.rstrip("/") + parts = [":id" if _NUMERIC_SEGMENT.match(p) else p for p in path.split("/")] + route = "/".join(parts) or "/" + return route if route in KNOWN_ROUTES else OTHER + + +def normalize_referrer(referrer: str, own_host: str = "") -> str: + """The sending site as a bare host. Our own host means an internal + navigation, which is not a referral — "" tells the caller to skip it.""" + if not referrer: + return NONE_LABEL + host = (urlsplit(referrer).hostname or "").lower().lstrip(".") + if not host or not _HOST_OK.match(host) or len(host) > MAX_LABEL_LEN: + return OTHER + if host == (own_host or "").lower() or host in ("localhost", "127.0.0.1"): + return "" + return host[4:] if host.startswith("www.") else host + + +def api_route_label(scope: dict, status: int) -> str: + """An error bucket like "500 /api/adventures/{adventure_id}". + + The route *template* is used, never the request path: it keeps one bucket + per endpoint instead of one per adventure id, and — the reason it is not + merely tidier — an unmatched path is entirely attacker-chosen, so labelling + by it would let anyone mint rows by requesting nonsense. + """ + template = getattr(scope.get("route"), "path", None) + return f"{status} {template}" if template else f"{status} (unmatched)" + + +def device_of(user_agent: str) -> str: + """Mobile / tablet / desktop, and nothing finer. The UA string itself is + never stored — it is a fingerprint, and the answer worth having is one + word.""" + ua = (user_agent or "").lower() + if not ua: + return UNKNOWN + if any(bot in ua for bot in ("bot", "crawler", "spider", "headless", "preview")): + return "bot" + if "ipad" in ua or "tablet" in ua or ("android" in ua and "mobile" not in ua): + return "tablet" + if any(m in ua for m in ("mobi", "iphone", "ipod", "android", "phone")): + return "mobile" + return "desktop" + + +# Geo headers an edge may add. Render fronts services with a CDN that can set +# cf-ipcountry; the others cost nothing to look for. A value is trusted only if +# it looks like an ISO code, since a client can send any header it likes — the +# worst case is therefore a wrong country, never an unbounded label. +_GEO_HEADERS = ("cf-ipcountry", "x-vercel-ip-country", "x-geo-country", "x-country-code") + + +def country_of(headers) -> str: + for name in _GEO_HEADERS: + value = (headers.get(name) or "").strip().upper() + if _COUNTRY_OK.match(value) and value != "XX": + return value + return UNKNOWN + + +# ---------- Flushing ---------- + +def _insert(db: Session): + return sqlite_insert if db.get_bind().dialect.name == "sqlite" else pg_insert + + +def _drain() -> tuple[dict, dict]: + with _guard: + counts, visits = _counts.copy(), _visits.copy() + _counts.clear() + _visits.clear() + # The label sets only bound cardinality within a day, so let yesterday's + # go rather than growing a map that never shrinks. + today = _today() + for key in [k for k in _labels_seen if k[0] != today]: + del _labels_seen[key] + return counts, visits + + +def _restore(counts: dict, visits: dict) -> None: + """Put a failed flush's work back so the next one retries it.""" + with _guard: + for key, n in counts.items(): + _counts[key] = _counts.get(key, 0) + n + for key, flags in visits.items(): + _visits.setdefault(key, set()).update(flags) + + +def flush(db: Session | None = None) -> None: + """Write the buffer out. Safe to call from anywhere; never raises.""" + counts, visits = _drain() + if not counts and not visits: + return + own_session = db is None + if own_session: + from .database import SessionLocal + db = SessionLocal() + try: + _write_counts(db, counts) + _write_visits(db, visits) + db.commit() + except Exception: + db.rollback() + _restore(counts, visits) + logger.exception("Analytics flush failed; counts held for the next one.") + finally: + if own_session: + db.close() + + +def _write_counts(db: Session, counts: dict) -> None: + if not counts: + return + table = models.AnalyticsDaily.__table__ + rows = [ + {"day": day, "metric": metric, "label": label, "hits": hits} + for (day, metric, label), hits in counts.items() + ] + stmt = _insert(db)(table).values(rows) + db.execute(stmt.on_conflict_do_update( + index_elements=["day", "metric", "label"], + set_={"hits": table.c.hits + stmt.excluded.hits}, + )) + + +def _write_visits(db: Session, visits: dict) -> None: + if not visits: + return + table = models.AnalyticsVisitorDay.__table__ + ids = {visitor for _, visitor in visits} + # One indexed lookup settles new-vs-returning for the whole batch. It is + # the only read this module does off the dashboard, and it returns short + # hashes for visitors who are active right now — bounded by the batch. + known = set(db.scalars( + select(models.AnalyticsVisitorDay.visitor) + .where(models.AnalyticsVisitorDay.visitor.in_(ids)) + .distinct() + )) + rows = [ + { + "day": day, + "visitor": visitor, + "is_new": visitor not in known, + **{column: column in flags for column in FUNNEL_FLAGS.values()}, + } + for (day, visitor), flags in visits.items() + ] + stmt = _insert(db)(table).values(rows) + db.execute(stmt.on_conflict_do_update( + index_elements=["day", "visitor"], + # Flags only ever turn on, and `is_new` is deliberately absent: the + # first write of a visitor's first day is the one that decided it. + set_={ + column: or_(table.c[column], stmt.excluded[column]) + for column in FUNNEL_FLAGS.values() + }, + )) + + +def purge_old_visitor_days(db: Session) -> int: + """Drop visitor-day rows past the retention horizon. Called by the cleanup + sweeper; the daily counters are never purged — they are aggregate, tiny, + and a portfolio project wants to keep its history.""" + if RETENTION_DAYS <= 0: + return 0 + cutoff = (models.utcnow().date() - timedelta(days=RETENTION_DAYS)).isoformat() + removed = db.query(models.AnalyticsVisitorDay).filter( + models.AnalyticsVisitorDay.day < cutoff + ).delete(synchronize_session=False) + db.commit() + return removed or 0 + + +# ---------- Reading it back ---------- +# Every query below is an aggregate: the database does the counting and ships +# back tens of rows, whatever the traffic behind them. Nothing here can return +# a row that belongs to one visitor. + +TOP_N = 12 + + +def _top(rows: list[dict], limit: int = TOP_N) -> list[dict]: + return rows[:limit] + + +def summary(db: Session, days: int = 30) -> dict: + """Everything the dashboard shows, for the last `days` days (today + included). Flushes first so the numbers include the last minute.""" + flush(db) + today = models.utcnow().date() + since = (today - timedelta(days=days - 1)).isoformat() + daily = models.AnalyticsDaily + visitor = models.AnalyticsVisitorDay + + # 1. Every counter in the window, folded to (metric, label) totals: the + # page/referrer/country/device/scenario/error tables all come from this + # one pass rather than a query each. + by_metric: dict[str, list[dict]] = {} + for metric, label, hits in db.execute( + select(daily.metric, daily.label, func.sum(daily.hits)) + .where(daily.day >= since) + .group_by(daily.metric, daily.label) + ): + by_metric.setdefault(metric, []).append({"label": label, "hits": int(hits)}) + for rows in by_metric.values(): + rows.sort(key=lambda row: -row["hits"]) + events = {row["label"]: row["hits"] for row in by_metric.get(M_EVENT, [])} + + # 2. Two per-day series worth drawing. + pageviews_by_day = { + day: int(hits) + for day, hits in db.execute( + select(daily.day, func.sum(daily.hits)) + .where(daily.day >= since, daily.metric == M_PAGE) + .group_by(daily.day) + ) + } + turns_by_day = { + day: int(hits) + for day, hits in db.execute( + select(daily.day, func.sum(daily.hits)) + .where(daily.day >= since, daily.metric == M_EVENT, daily.label == EV_TURN) + .group_by(daily.day) + ) + } + + # 3. People, per day. One row per visitor per day means COUNT(*) is already + # the day's unique visitors — no DISTINCT needed here. + visitors_by_day: dict[str, dict] = {} + for day, total, fresh in db.execute( + select( + visitor.day, + func.count(), + func.sum(case((visitor.is_new, 1), else_=0)), + ) + .where(visitor.day >= since) + .group_by(visitor.day) + ): + visitors_by_day[day] = {"visitors": int(total), "new": int(fresh or 0)} + + # 4. The funnel, over the whole window, counting *people* once each: + # COUNT(DISTINCT CASE WHEN flag THEN visitor END) ignores the NULLs the + # CASE leaves for everyone who didn't reach that step. + unique, unique_new, *reached = db.execute( + select( + func.count(func.distinct(visitor.visitor)), + func.count(func.distinct(case((visitor.is_new, visitor.visitor)))), + *[ + func.count(func.distinct(case((visitor.__table__.c[column], visitor.visitor)))) + for column in FUNNEL_FLAGS.values() + ], + ).where(visitor.day >= since) + ).one() + + series = [] + for offset in range(days): + day = (today - timedelta(days=days - 1 - offset)).isoformat() + counted = visitors_by_day.get(day, {}) + series.append({ + "day": day, + "visitors": counted.get("visitors", 0), + "new": counted.get("new", 0), + "pageviews": pageviews_by_day.get(day, 0), + "turns": turns_by_day.get(day, 0), + }) + + visits = sum(row["visitors"] for row in series) + pageviews = sum(pageviews_by_day.values()) + turns = events.get(EV_TURN, 0) + errors = by_metric.get(M_ERROR, []) + return { + "days": days, + "since": since, + "until": today.isoformat(), + "generated_at": models.utcnow().isoformat(), + "totals": { + # `visitors` counts each person once for the window; `visits` counts + # them once per day they came back, which is the closest honest + # thing to "sessions" without tracking sessions. + "visitors": int(unique), + "new_visitors": int(unique_new), + "visits": visits, + "pageviews": pageviews, + "turns": turns, + "demo_turns": events.get(EV_DEMO_TURN, 0), + "adventures": events.get(EV_ADVENTURE, 0), + "signups": events.get(EV_SIGNUP, 0), + "logins": events.get(EV_LOGIN, 0), + "turn_errors": events.get(EV_TURN_ERROR, 0), + "errors": sum(row["hits"] for row in errors), + "turns_per_visit": round(turns / visits, 1) if visits else 0, + "pages_per_visit": round(pageviews / visits, 1) if visits else 0, + }, + "series": series, + # Step 0 is everyone who showed up, so the drop-off between it and + # "Opened a scenario" is visible as a step like any other. + "funnel": [{"step": "Visited", "count": int(unique)}] + [ + {"step": step, "count": int(count)} + for step, count in zip( + ["Opened a scenario", "Started an adventure", "Played a turn", "Signed up"], + reached, + ) + ], + "pages": _top(by_metric.get(M_PAGE, [])), + "referrers": _top(by_metric.get(M_REFERRER, [])), + "countries": _top(by_metric.get(M_COUNTRY, [])), + "devices": by_metric.get(M_DEVICE, []), + "scenarios": _top(by_metric.get(M_SCENARIO, [])), + "errors": _top(errors), + "events": by_metric.get(M_EVENT, []), + } + + +# ---------- Background flusher ---------- +# Mirrors cleanup's start/stop pair so main.py's lifespan reads the same way +# for both. The interval is what bounds how much a hard restart can lose. + +async def _flush_loop() -> None: + import asyncio + + from starlette.concurrency import run_in_threadpool + + while True: + await asyncio.sleep(FLUSH_INTERVAL_SECONDS) + # Blocking DB work: keep it off the event loop, which is also serving + # SSE turn streams. + await run_in_threadpool(flush) + + +def start_flusher(): + import asyncio + + return asyncio.create_task(_flush_loop()) + + +async def stop_flusher(task) -> None: + """Cancel the loop and write out whatever it was holding — a deploy is the + one restart that is both frequent and predictable, so it should not be the + thing that loses a minute of counts.""" + import asyncio + + from starlette.concurrency import run_in_threadpool + + if task is not None: + task.cancel() + try: + await task + except asyncio.CancelledError: + pass + await run_in_threadpool(flush) diff --git a/backend/app/auth.py b/backend/app/auth.py index be3090f..36499b1 100644 --- a/backend/app/auth.py +++ b/backend/app/auth.py @@ -66,6 +66,16 @@ POWER_USERS = { if e.strip() } +# Who can see the visit analytics. Deliberately its own list rather than +# POWER_USERS: a trusted tester gets unmetered turns and the AI Chat page, +# which is not a reason to hand them the site's traffic numbers. Empty (the +# default) means nobody sees the dashboard in a hosted deployment. +ANALYTICS_EMAILS = { + e.strip().lower() + for e in os.environ.get("AIDND_ANALYTICS_EMAILS", "").split(",") + if e.strip() +} + DEMO_CAP_MESSAGE = ( f"You've used all {DEMO_TURNS_PER_DAY} free demo turns for today. " "Add your own API key in Settings to keep playing (it resets tomorrow)." @@ -148,6 +158,15 @@ def is_power_user(user: models.User) -> bool: return bool(user.email) and user.email.lower() in POWER_USERS +def is_owner(user: models.User) -> bool: + """May this user see the visit analytics? Local installs always can — it is + the operator's own machine and their own visits, same reasoning as the + provider debug log; hosted deployments check AIDND_ANALYTICS_EMAILS.""" + if not MULTI_USER: + return True + return bool(user.email) and user.email.lower() in ANALYTICS_EMAILS + + def demo_turns_left(user: models.User) -> int: # Power users are never capped; report the full cap so the banner reads # "N of N" rather than a decrementing count. diff --git a/backend/app/cleanup.py b/backend/app/cleanup.py index 762ca3b..518838c 100644 --- a/backend/app/cleanup.py +++ b/backend/app/cleanup.py @@ -42,7 +42,7 @@ from sqlalchemy import delete, func from sqlalchemy.orm import Session from starlette.concurrency import run_in_threadpool -from . import auth, models +from . import analytics, auth, models from .database import SessionLocal logger = logging.getLogger(__name__) @@ -72,6 +72,13 @@ def enabled() -> bool: return auth.MULTI_USER and RETENTION_DAYS > 0 +def anything_to_sweep() -> bool: + """Whether the periodic task is worth starting at all. The two jobs it runs + are independent: a deployment can keep every guest forever and still want + its analytics rows aged out, and vice versa.""" + return enabled() or analytics.RETENTION_DAYS > 0 + + def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int: """Delete guests idle for RETENTION_DAYS or more. Returns the row count. @@ -105,12 +112,19 @@ def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int: def sweep() -> int: """One pass, with its own session. Never raises: a failed cleanup must not - be able to take the app down (same rule as seeding).""" - if not enabled(): + be able to take the app down (same rule as seeding). Returns the guest + count, which is the number worth logging about.""" + if not anything_to_sweep(): return 0 db = SessionLocal() try: - removed = delete_stale_guests(db) + # Ages out the per-visitor analytics rows, on its own terms: it is not + # about guests, and it must still happen on a deployment that has + # chosen to keep every account it ever minted. + aged = analytics.purge_old_visitor_days(db) + if aged: + logger.info("Aged out %d analytics visitor-day row(s).", aged) + removed = delete_stale_guests(db) if enabled() else 0 if removed: logger.info( "Cleaned up %d guest account(s) idle for %d+ days.", @@ -135,16 +149,18 @@ async def _sweep_loop() -> None: def start_sweeper() -> asyncio.Task | None: - """Kick off the periodic sweep; None when the policy is off.""" + """Kick off the periodic sweep; None when there is nothing to sweep.""" if not enabled(): logger.info("Guest cleanup disabled (multi_user=%s, retention_days=%d).", auth.MULTI_USER, RETENTION_DAYS) + else: + logger.info( + "Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).", + RETENTION_DAYS, + SWEEP_INTERVAL_SECONDS // 3600, + ) + if not anything_to_sweep(): return None - logger.info( - "Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).", - RETENTION_DAYS, - SWEEP_INTERVAL_SECONDS // 3600, - ) return asyncio.create_task(_sweep_loop()) diff --git a/backend/app/limits.py b/backend/app/limits.py index 4f48b49..b4433e8 100644 --- a/backend/app/limits.py +++ b/backend/app/limits.py @@ -32,6 +32,10 @@ RATE_LIMITS: dict[str, tuple[int, int]] = { "import": (30, 60), # large writes "auth": (10, 300), # register/login attempts, per IP "guest": (30, 300), # new guest users, per IP (each is a DB row) + # Pageview beacons. Generous — a real reader clicking around a SPA fires a + # handful a minute — but low enough that nobody can inflate the traffic + # numbers faster than they could by actually reloading the page. + "analytics": (120, 60), } _windows: dict[tuple[str, str], deque] = defaultdict(deque) @@ -49,11 +53,15 @@ _windows_guard = threading.Lock() TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1)) -def _client_ip(request: Request) -> str: - """The real client IP for rate-limit keying, resistant to a spoofed - X-Forwarded-For. Takes the hop the trusted edge appended (rightmost minus - any extra trusted hops); falls back to the socket peer when no forwarded - header is present (local/dev, or a direct connection).""" +def client_ip(request: Request) -> str: + """The real client IP, resistant to a spoofed X-Forwarded-For. Takes the + hop the trusted edge appended (rightmost minus any extra trusted hops); + falls back to the socket peer when no forwarded header is present + (local/dev, or a direct connection). + + Public because the access log needs the same answer, and two functions that + both decide "which address is the caller's" is how one of them ends up + trusting a header it shouldn't.""" forwarded = request.headers.get("x-forwarded-for") if forwarded: parts = [p.strip() for p in forwarded.split(",") if p.strip()] @@ -68,7 +76,7 @@ def rate_limit(scope: str, request: Request, user: models.User | None = None) -> if not auth.MULTI_USER: return limit, window_seconds = RATE_LIMITS[scope] - key = (scope, f"u{user.id}" if user else f"ip{_client_ip(request)}") + key = (scope, f"u{user.id}" if user else f"ip{client_ip(request)}") now = time.time() with _windows_guard: window = _windows[key] diff --git a/backend/app/main.py b/backend/app/main.py index 3156656..c84b113 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -7,12 +7,15 @@ from fastapi.middleware.cors import CORSMiddleware from fastapi.staticfiles import StaticFiles from starlette.exceptions import HTTPException as StarletteHTTPException -from . import cleanup +from . import analytics, cleanup from .auth import MULTI_USER from .database import engine from .limits import BodySizeLimitMiddleware from .migrations import bootstrap -from .routers import adventures, auth, chat, debug, scenarios, scripts, settings, story_cards +from .routers import ( + adventures, analytics as analytics_router, auth, chat, debug, scenarios, scripts, + settings, story_cards, +) from .seed import seed_public_scenarios bootstrap(engine) @@ -32,10 +35,15 @@ async def lifespan(_app: FastAPI): # trigger on Render's free tier, where the service sleeps after ~15 # minutes and a long-running timer rarely gets to fire. sweeper = cleanup.start_sweeper() + # Visit counters are buffered in memory and written in batches; this is + # what turns them into rows, and stop_flusher writes out the last batch so + # a deploy doesn't drop it. + flusher = analytics.start_flusher() try: yield finally: await cleanup.stop_sweeper(sweeper) + await analytics.stop_flusher(flusher) # The interactive API docs stay local-only: in multi-user mode they just hand @@ -95,6 +103,39 @@ class SecurityHeadersMiddleware: await self.app(scope, receive, send_with_headers) +class ApiErrorMiddleware: + """Counts failed API responses for the analytics dashboard. + + Here rather than in an exception handler because it sees what the client + actually got: a 429 from a rate limiter, a 404 from routing, a 500 from a + handler that never returned, all the same way. Pure ASGI for the same + reason as the headers above — an SSE turn must not be buffered on its way + out. Only /api is watched; a 404 on the SPA mount is a page load, not a + fault. + """ + + def __init__(self, app): + self.app = app + + async def __call__(self, scope, receive, send): + if scope["type"] != "http" or not scope.get("path", "").startswith("/api"): + return await self.app(scope, receive, send) + + async def send_counting(message): + if message["type"] == "http.response.start" and message["status"] >= 400: + # The router has already put the matched route on the scope by + # the time a response starts, so the label can name the + # endpoint rather than the caller's path. + analytics.record( + analytics.M_ERROR, + analytics.api_route_label(scope, message["status"]), + ) + await send(message) + + await self.app(scope, receive, send_counting) + + +app.add_middleware(ApiErrorMiddleware) app.add_middleware(SecurityHeadersMiddleware) app.include_router(auth.router) @@ -105,6 +146,7 @@ app.include_router(scripts.router) app.include_router(settings.router) app.include_router(chat.router) app.include_router(debug.router) +app.include_router(analytics_router.router) @app.get("/api/health") diff --git a/backend/app/models.py b/backend/app/models.py index 78bbc9b..f849170 100644 --- a/backend/app/models.py +++ b/backend/app/models.py @@ -2,7 +2,7 @@ from datetime import datetime, timezone from sqlalchemy import ( JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, LargeBinary, - String, Table, Text, event, + String, Table, Text, UniqueConstraint, event, ) from sqlalchemy.orm import Mapped, Session, mapped_column, relationship @@ -575,6 +575,93 @@ class Settings(Base): return security.decrypt_secret(self.api_key) +# ---------- Visit analytics (see analytics.py) ---------- +# Two deliberately dumb tables. Neither can hold anything a player wrote, and +# neither can be joined back to a `users` row: the visitor column is an HMAC, +# with no foreign key, so guest cleanup deleting an account leaves the history +# it contributed to intact and anonymous. + + +class AnalyticsDaily(Base): + """One counter: how many times `label` happened within `metric` on `day`. + + A generic (metric, label, hits) triple rather than a column per statistic, + so measuring something new later costs a constant instead of a migration. + Written only by UPSERT, from a buffer — see analytics.flush. + """ + + __tablename__ = "analytics_daily" + + id: Mapped[int] = mapped_column(primary_key=True) + day: Mapped[str] = mapped_column(String(10), index=True) # YYYY-MM-DD, UTC + metric: Mapped[str] = mapped_column(String(32)) + label: Mapped[str] = mapped_column(String(80), default="") + hits: Mapped[int] = mapped_column(Integer, default=0) + + # The upsert target: one row per bucket per day, created or incremented. + __table_args__ = ( + UniqueConstraint("day", "metric", "label", name="uq_analytics_daily_bucket"), + ) + + +class AnalyticsVisitorDay(Base): + """One visitor, one day, and which funnel steps they reached on it. + + Exists so the funnel counts people rather than clicks — a player who starts + six adventures is one person who started an adventure. `is_new` is set when + the visitor has no earlier row, which is also why the visitor column is + indexed on its own. + """ + + __tablename__ = "analytics_visitor_days" + + id: Mapped[int] = mapped_column(primary_key=True) + day: Mapped[str] = mapped_column(String(10)) + # HMAC of the user id under the app secret; not reversible, not a key. + visitor: Mapped[str] = mapped_column(String(32)) + is_new: Mapped[bool] = mapped_column(Boolean, default=False) + opened: Mapped[bool] = mapped_column(Boolean, default=False) + created: Mapped[bool] = mapped_column(Boolean, default=False) + played: Mapped[bool] = mapped_column(Boolean, default=False) + signed_up: Mapped[bool] = mapped_column(Boolean, default=False) + + __table_args__ = ( + UniqueConstraint("day", "visitor", name="uq_analytics_visitor_day"), + Index("ix_analytics_visitor", "visitor"), + ) + + +class AccessEvent(Base): + """One sign-in, registration, failed attempt, or session first-seen. + + The counterpart to the two tables above, and deliberately not mixed in with + them: this one identifies people on purpose — address, email, device — so + keeping it in its own table (and its own module) means the anonymity of the + counters stays a property of the code rather than of a convention. + + `user_id` is a plain integer with no foreign key. An access log that + disappeared when the account did would not be an access log, and guest + cleanup deletes accounts on a schedule; `who` and `is_guest` are snapshots + for the same reason, so a row still reads correctly afterwards. + """ + + __tablename__ = "access_events" + + id: Mapped[int] = mapped_column(primary_key=True) + at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, index=True) + # session | login | register | login_failed + kind: Mapped[str] = mapped_column(String(16)) + user_id: Mapped[int | None] = mapped_column(Integer, nullable=True) + # Email for a registered account, "Guest #12" otherwise; for a failed + # sign-in, the address that was tried — which is the point of the row. + who: Mapped[str] = mapped_column(String(320), default="") + is_guest: Mapped[bool] = mapped_column(Boolean, default=False) + ip: Mapped[str] = mapped_column(String(45), default="") # 45 = max IPv6 + country: Mapped[str] = mapped_column(String(16), default="") + device: Mapped[str] = mapped_column(String(16), default="") + user_agent: Mapped[str] = mapped_column(String(200), default="") + + # Phase 14 — the floor under `tree.place_action`. # # From SP2 a read selects on (branch_id, depth): a node written without them is diff --git a/backend/app/routers/adventures.py b/backend/app/routers/adventures.py index f2d9405..b2ca1bf 100644 --- a/backend/app/routers/adventures.py +++ b/backend/app/routers/adventures.py @@ -9,8 +9,8 @@ from sqlalchemy.orm import Session, load_only, undefer from sqlalchemy.orm.attributes import set_committed_value from .. import ( - attempts, auth, bundle, images, limits, memorybank, models, schemas, tree, - worldstate, + analytics, attempts, auth, bundle, images, limits, memorybank, models, schemas, + tree, worldstate, ) from ..context import build_context, cursors from ..context import history as context_history @@ -411,6 +411,12 @@ def create_adventure( db.commit() db.refresh(adventure) + analytics.record_event(analytics.EV_ADVENTURE, user) + # Which of the shared scenarios people actually pick — the one piece of + # content this module ever names, and only ever a seeded/public one. A + # player's own scenario titles are theirs. + if scenario is not None and scenario.is_public: + analytics.record(analytics.M_SCENARIO, scenario.title) return adventure @@ -589,6 +595,15 @@ def sse(obj: dict) -> str: return f"data: {json.dumps(obj)}\n\n" +def turn_error(detail: str, **extra) -> str: + """An SSE error for a turn that could not be produced, counted on the way + out. Worth its own event: a failed turn is an HTTP 200 with a bad ending, + so the middleware's status-code tally cannot see it — a demo whose model + has started refusing every request looks perfectly healthy from outside.""" + analytics.record(analytics.M_EVENT, analytics.EV_TURN_ERROR) + return sse({"type": "error", "detail": detail, **extra}) + + # no-cache defeats any intermediary caching; X-Accel-Buffering makes # nginx-style reverse proxies (hosted deploys) flush each event immediately # instead of buffering the stream. @@ -745,7 +760,7 @@ async def _generate_turn( chunks.append(chunk) yield sse({"type": "chunk", "text": chunk}) except ProviderError as exc: - yield sse({"type": "error", "detail": str(exc)}) + yield turn_error(str(exc)) return text = "".join(chunks).strip() @@ -763,13 +778,13 @@ async def _generate_turn( ) else: detail = "The AI returned an empty response." - yield sse({"type": "error", "detail": detail}) + yield turn_error(detail) return # onOutput text, _ = pipeline.run("output", text) if not text.strip(): - yield sse({"type": "error", "detail": "A script's output modifier returned empty text."}) + yield turn_error("A script's output modifier returned empty text.") return snapshot["script"] = snapshot["script"] | pipeline.report() @@ -785,7 +800,7 @@ async def _generate_turn( if worldstate.has_schema(stat_schema): text, delta = worldstate.extract_delta(text) if not text.strip(): - yield sse({"type": "error", "detail": "The AI returned only a state update and no story text."}) + yield turn_error("The AI returned only a state update and no story text.") return new_world_state, ws_report = worldstate.apply_delta( adventure.world_state, stat_schema, delta, ai_depth @@ -832,6 +847,12 @@ async def _generate_turn( # in the endpoint); failed provider calls above don't reach here. auth.count_demo_turn(user) db.commit() + # Counted here, past every way the turn could still have failed, so the + # number means "stories advanced" rather than "requests attempted". The + # demo tally is those same turns seen as spend on the server-funded key. + analytics.record_event(analytics.EV_TURN, user) + if cfg.using_demo: + analytics.record(analytics.M_EVENT, analytics.EV_DEMO_TURN) db.refresh(ai_action) yield _SAVED yield sse({"type": "done", "action": action_json(ai_action, db), "script": pipeline.report()}) @@ -876,8 +897,8 @@ async def run_player_turn( ) modified, stop = pipeline.run("input", formatted) if not modified.strip(): - yield sse({"type": "error", "detail": "A script's input modifier returned empty text.", - "script": pipeline.report()}) + yield turn_error("A script's input modifier returned empty text.", + script=pipeline.report()) return player_action = models.Action( adventure_id=adventure.id, @@ -1800,6 +1821,10 @@ def import_adventure( db.commit() db.refresh(adventure) + # Not a funnel step: importing a bundle is something a returning player + # does, not a sign a first-time visitor got anywhere. Counted anyway, + # because it is the clearest evidence anyone is using the export format. + analytics.record_event(analytics.EV_IMPORT, user) return adventure diff --git a/backend/app/routers/analytics.py b/backend/app/routers/analytics.py new file mode 100644 index 0000000..8759a66 --- /dev/null +++ b/backend/app/routers/analytics.py @@ -0,0 +1,137 @@ +"""Visit analytics: one endpoint the browser writes to, one the owner reads. + +The split matters. `/collect` is public and takes exactly one fact — which +page was viewed — because anything a stranger can POST is a number a stranger +can invent. Everything the dashboard actually relies on (turns, adventures, +sign-ups, demo spend, errors) is recorded server-side by the code performing +it, so those counts are as trustworthy as the app itself. + +The two reading endpoints are owner-only and 404 for everyone else, the same +way the AI Chat router does: a feature nobody else can use is better off not +appearing to exist. `/summary` serves the anonymous counters (analytics.py, +which stores nothing that points at a person) and `/access` serves the access +log (accesslog.py, which identifies people on purpose). +""" + +from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response +from pydantic import BaseModel, Field +from sqlalchemy.orm import Session + +from .. import accesslog, analytics, auth, limits, models +from ..database import get_db + +router = APIRouter(prefix="/api/analytics", tags=["analytics"]) + + +def owner( + db: Session = Depends(get_db), + user: models.User = Depends(auth.get_current_user), +) -> models.User: + """Gate for the reading half. 404, not 403 — see the module docstring.""" + if not auth.is_owner(user): + raise HTTPException(404, "Not found") + return user + + +Owner = Depends(owner) + + +class Pageview(BaseModel): + """What the SPA reports on a page load or a route change. + + `first` marks a real page load rather than a client-side navigation: the + things that describe a *visit* rather than a *view* — where it came from, + on what kind of device, from which country — are recorded only then, so a + visitor who clicks around five pages is still one referral. + """ + + path: str = Field("", max_length=300) + referrer: str = Field("", max_length=500) + first: bool = False + + +@router.post("/collect", status_code=204) +def collect( + payload: Pageview, + request: Request, + db: Session = Depends(get_db), +) -> Response: + """Record one pageview. Always 204, even when nothing was counted: the + browser has no business knowing whether it was.""" + limits.rate_limit("analytics", request) + # Resolved by hand rather than through get_current_user: a pageview that + # arrives before /auth/me has minted a session should still be counted as a + # view, not turned into a 401 the SPA has to handle. + user = ( + auth.resolve_session_user(request, db) + if auth.MULTI_USER + else auth.local_user(db) + ) + # The operator's own clicking is not traffic. Only in multi-user mode — + # locally everyone is the owner, and excluding them would leave the + # dashboard permanently empty on the machine it is developed on. + if auth.MULTI_USER and user is not None and auth.is_owner(user): + return Response(status_code=204) + + analytics.record(analytics.M_PAGE, analytics.normalize_route(payload.path)) + analytics.record_visit(user) + if payload.first: + referrer = analytics.normalize_referrer( + payload.referrer, request.url.hostname or "" + ) + if referrer: # "" means same-origin, which is not a referral + analytics.record(analytics.M_REFERRER, referrer) + analytics.record( + analytics.M_DEVICE, + analytics.device_of(request.headers.get("user-agent", "")), + ) + analytics.record(analytics.M_COUNTRY, analytics.country_of(request.headers)) + return Response(status_code=204) + + +@router.get("/summary") +def summary( + days: int = Query(30, ge=1, le=365), + db: Session = Depends(get_db), + _user: models.User = Owner, +) -> dict: + """The whole dashboard in one aggregate response — a few kilobytes however + much traffic sits behind it.""" + return analytics.summary(db, days) + + +@router.get("/access") +def access_log( + limit: int = Query(50, ge=1, le=200), + before_id: int | None = Query(None), + kind: str | None = Query(None), + q: str | None = Query(None, max_length=120), + db: Session = Depends(get_db), + _user: models.User = Owner, +) -> dict: + """A page of the access log, newest first. + + Unlike /summary this returns rows about people, which is the whole point of + it — so it is behind the same owner gate, paged rather than dumped, and has + no counterpart the people it describes can reach. + """ + page = accesslog.recent( + db, limit=limit, before_id=before_id, kind=kind, query=q + ) + return { + "events": [ + { + "id": event.id, + "at": event.at.isoformat(), + "kind": event.kind, + "who": event.who, + "is_guest": event.is_guest, + "ip": event.ip, + "country": event.country, + "device": event.device, + "user_agent": event.user_agent, + } + for event in page["events"] + ], + "has_more": page["has_more"], + } diff --git a/backend/app/routers/auth.py b/backend/app/routers/auth.py index 9aa43ce..9b2c08c 100644 --- a/backend/app/routers/auth.py +++ b/backend/app/routers/auth.py @@ -3,7 +3,7 @@ import re from fastapi import APIRouter, Depends, HTTPException, Request, Response from sqlalchemy.orm import Session -from .. import auth, cleanup, limits, models, schemas, security +from .. import accesslog, analytics, auth, cleanup, limits, models, schemas, security from ..database import get_db from .settings import get_settings @@ -34,6 +34,8 @@ def me_payload(user: models.User, db: Session) -> dict: "is_guest": user.is_guest, # Trusted testers: unmetered demo turns, plus the AI Chat scratchpad. "power_user": auth.is_power_user(user), + # Separate allowlist: shows the visit-analytics page and its nav link. + "analytics": auth.is_owner(user), # How long an idle guest is kept before cleanup deletes it (None when # the policy is off). Served rather than hardcoded in the UI so the # number a guest is shown is the number actually enforced. @@ -65,6 +67,9 @@ def me(request: Request, response: Response, db: Session = Depends(get_db)): db.add(user) db.commit() _set_session_cookie(response, user.id) + # This endpoint is the SPA's bootstrap call, so it is where a session first + # shows itself; accesslog thins the rows down to one per day per address. + accesslog.note_session(db, user, request) return me_payload(user, db) @@ -93,6 +98,8 @@ def register( user.password_hash = security.hash_password(payload.password) user.is_guest = False db.commit() + analytics.record_event(analytics.EV_SIGNUP, user) + accesslog.record(db, accesslog.REGISTER, request, user=user) return me_payload(user, db) @@ -119,9 +126,15 @@ def login( or not security.verify_password(payload.password, user.password_hash) ): limits.note_login_failure(email) + # Logged with the address that was tried, not the account that owns it: + # a guessing run against an address that has no account is exactly the + # thing worth being able to see. + accesslog.record(db, accesslog.LOGIN_FAILED, request, who=email) raise HTTPException(401, "Incorrect email or password.") limits.note_login_success(email) _set_session_cookie(response, user.id) + analytics.record_event(analytics.EV_LOGIN, user) + accesslog.record(db, accesslog.LOGIN, request, user=user) return me_payload(user, db) diff --git a/backend/app/routers/scenarios.py b/backend/app/routers/scenarios.py index 8b3a2ae..553a44e 100644 --- a/backend/app/routers/scenarios.py +++ b/backend/app/routers/scenarios.py @@ -3,7 +3,7 @@ from fastapi.responses import Response from sqlalchemy import or_ from sqlalchemy.orm import Session -from .. import auth, images, limits, models, schemas +from .. import analytics, auth, images, limits, models, schemas from ..database import get_db router = APIRouter(prefix="/api/scenarios", tags=["scenarios"]) @@ -53,7 +53,13 @@ def get_scenario( db: Session = Depends(get_db), user: models.User = Depends(auth.get_current_user), ): - return get_scenario_or_404(scenario_id, db, user) + scenario = get_scenario_or_404(scenario_id, db, user) + # Funnel step. Only for shared scenarios: opening one is the first sign a + # visitor is interested, whereas someone editing their own is already past + # this point — and their titles are theirs, not a statistic. + if scenario.is_public: + analytics.record_event(analytics.EV_SCENARIO_OPEN, user) + return scenario @router.get("/{scenario_id}/image") diff --git a/backend/tests/test_accesslog.py b/backend/tests/test_accesslog.py new file mode 100644 index 0000000..9311d7d --- /dev/null +++ b/backend/tests/test_accesslog.py @@ -0,0 +1,215 @@ +"""The access log — app/accesslog.py and GET /api/analytics/access. + +This is the half of the analytics work that identifies people on purpose, so +the things worth pinning are the ones that would quietly make it wrong: that +the address recorded is the hardened one and not a header a client chose, that +session rows are thinned instead of written per page load, and that a row +outlives the account it describes — guest cleanup runs on a schedule, and a log +that deletes itself is not a log. + + python -m pytest tests/test_accesslog.py -v +""" +import os +import tempfile + +_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False) +_tmp.close() +os.environ["AIDND_DB_PATH"] = _tmp.name +os.environ.pop("AIDND_DATABASE_URL", None) +os.environ.pop("DATABASE_URL", None) + +import pytest +from fastapi import Depends +from fastapi.testclient import TestClient + +from app import accesslog, auth, limits, models, security +from app.database import Base, SessionLocal, engine, get_db +from app.main import app + +EDGE = "198.51.100.77" # what the trusted proxy appended +SPOOF = "10.0.0.1" # what a client put in front of it + + +@pytest.fixture(autouse=True) +def clean_state(): + accesslog._last_session.clear() + yield + accesslog._last_session.clear() + + +@pytest.fixture() +def client(monkeypatch): + Base.metadata.create_all(bind=engine) + setup = SessionLocal() + owner = models.User(is_guest=False, email="owner@example.com") + member = models.User( + is_guest=False, email="player@example.com", + password_hash=security.hash_password("hunter2long"), + ) + setup.add_all([owner, member]) + setup.commit() + ids = {"owner": owner.id, "member": member.id} + setup.close() + + monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None) + monkeypatch.setattr(limits, "check_login_allowed", lambda *a, **k: None) + monkeypatch.setattr(auth, "MULTI_USER", True) + monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"}) + + # /auth/me resolves its own session, so the cookie flow below is the real + # one; every other endpoint goes through get_current_user, and `act_as` + # decides who that is. + acting = {"id": ids["owner"]} + + def _current(db=Depends(get_db)): + return db.get(models.User, acting["id"]) + + app.dependency_overrides[auth.get_current_user] = _current + try: + test_client = TestClient(app) + test_client.ids = ids + test_client.act_as = lambda user_id: acting.update(id=user_id) + yield test_client + finally: + app.dependency_overrides.clear() + Base.metadata.drop_all(bind=engine) + + +def visit(client, ip=EDGE, ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64)"): + return client.get( + "/api/auth/me", + headers={"x-forwarded-for": f"{SPOOF}, {ip}", "user-agent": ua}, + ) + + +def rows(kind=None): + db = SessionLocal() + try: + query = db.query(models.AccessEvent).order_by(models.AccessEvent.id) + if kind: + query = query.filter_by(kind=kind) + return query.all() + finally: + db.close() + + +def read_log(client, **params): + return client.get("/api/analytics/access", params=params) + + +# ---------- Writing ---------- + +def test_a_new_session_is_logged(client): + visit(client) + logged = rows() + assert len(logged) == 1 + entry = logged[0] + assert entry.kind == accesslog.SESSION + assert entry.is_guest and entry.who.startswith("Guest #") + assert entry.device == "desktop" + + +def test_the_address_is_the_hardened_one_not_the_clients(client): + visit(client) + # The client prepended its own value; only the hop the edge appended counts. + # Recording the leftmost would make every row forgeable, which for a log is + # worse than having no log. + assert rows()[0].ip == EDGE + + +def test_session_rows_are_thinned_to_one_per_day_per_address(client): + for _ in range(4): + visit(client) + assert len(rows(accesslog.SESSION)) == 1 + + +def test_a_changed_address_writes_a_new_row(client): + visit(client) + visit(client, ip="203.0.113.9") + logged = rows(accesslog.SESSION) + assert [entry.ip for entry in logged] == [EDGE, "203.0.113.9"] + # Same session throughout, so both rows name the same visitor. + assert logged[0].who == logged[1].who + + +def test_sign_in_and_failure_are_both_logged(client): + client.post("/api/auth/login", json={"email": "player@example.com", "password": "wrong"}, + headers={"x-forwarded-for": EDGE}) + client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"}, + headers={"x-forwarded-for": EDGE}) + kinds = [entry.kind for entry in rows()] + assert accesslog.LOGIN_FAILED in kinds and accesslog.LOGIN in kinds + + failure = rows(accesslog.LOGIN_FAILED)[0] + # The address tried, not the account that owns it: a run against an address + # with no account behind it is exactly what this row is for. + assert failure.who == "player@example.com" + assert failure.user_id is None + assert rows(accesslog.LOGIN)[0].user_id == client.ids["member"] + + +def test_registering_is_logged_against_the_upgraded_account(client): + visit(client) # mints the guest whose session registers + client.act_as(rows()[0].user_id) + client.post("/api/auth/register", json={"email": "new@example.com", "password": "hunter2long"}) + entry = rows(accesslog.REGISTER)[0] + assert entry.who == "new@example.com" and not entry.is_guest + + +def test_a_row_outlives_the_account_it_describes(client): + visit(client) + entry = rows()[0] + db = SessionLocal() + try: + db.delete(db.get(models.User, entry.user_id)) + db.commit() + finally: + db.close() + # No foreign key, and `who` is a snapshot — guest cleanup deletes accounts + # on a schedule, and a log that vanishes with them is not a log. + survivor = rows()[0] + assert survivor.who == entry.who and survivor.ip == EDGE + + +def test_a_long_user_agent_is_truncated(client): + visit(client, ua="Mozilla/" + "x" * 500) + assert len(rows()[0].user_agent) == accesslog.MAX_UA + + +def test_a_logging_failure_does_not_break_the_request(client, monkeypatch): + monkeypatch.setattr(accesslog, "_client_ip", lambda request: 1 / 0) + # The log watches sign-in; it must not be able to stand in its way. + assert visit(client).status_code == 200 + + +# ---------- Reading ---------- + +def test_the_log_is_invisible_to_everyone_but_the_owner(client): + visit(client) + assert read_log(client).status_code == 200 + client.act_as(client.ids["member"]) + assert read_log(client).status_code == 404 + + +def test_the_log_reads_newest_first_and_pages_backwards(client): + for index in range(5): + visit(client, ip=f"203.0.113.{index}") + first = read_log(client, limit=2).json() + assert [event["ip"] for event in first["events"]] == ["203.0.113.4", "203.0.113.3"] + assert first["has_more"] + + older = read_log(client, limit=2, before_id=first["events"][-1]["id"]).json() + assert [event["ip"] for event in older["events"]] == ["203.0.113.2", "203.0.113.1"] + + +def test_the_log_filters_by_kind_and_searches(client): + visit(client) + client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"}, + headers={"x-forwarded-for": "203.0.113.44"}) + + assert len(read_log(client, kind="login").json()["events"]) == 1 + by_email = read_log(client, q="player@example.com").json()["events"] + assert len(by_email) == 1 and by_email[0]["kind"] == "login" + by_ip = read_log(client, q="203.0.113.44").json()["events"] + assert len(by_ip) == 1 + assert read_log(client, q="nobody@example.com").json()["events"] == [] diff --git a/backend/tests/test_analytics.py b/backend/tests/test_analytics.py new file mode 100644 index 0000000..dd343cd --- /dev/null +++ b/backend/tests/test_analytics.py @@ -0,0 +1,392 @@ +"""Visit analytics — app/analytics.py and the two endpoints in front of it. + +Three things are worth testing here and the rest is arithmetic. That the +counters survive the buffer/UPSERT round trip (a flush must add to what is +already stored, not replace it, or every number is only ever the last minute). +That the funnel counts *people* rather than clicks, which is the only reason +the visitor-day table exists. And that the gate holds: a stranger cannot read +the dashboard, and cannot inflate what it says beyond hitting the page. + + python -m pytest tests/test_analytics.py -v +""" +import os +import tempfile + +_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False) +_tmp.close() +os.environ["AIDND_DB_PATH"] = _tmp.name +os.environ.pop("AIDND_DATABASE_URL", None) +os.environ.pop("DATABASE_URL", None) + +from datetime import timedelta + +import pytest +from fastapi import Depends +from fastapi.testclient import TestClient + +from app import analytics, auth, limits, models +from app.database import Base, SessionLocal, engine, get_db +from app.main import app + + +@pytest.fixture(autouse=True) +def clean_buffer(): + """The buffer is process-wide, so a test that leaves counts in it would + show up inside the next one's flush.""" + analytics._counts.clear() + analytics._visits.clear() + analytics._labels_seen.clear() + yield + analytics._counts.clear() + analytics._visits.clear() + analytics._labels_seen.clear() + + +@pytest.fixture() +def db(): + Base.metadata.create_all(bind=engine) + session = SessionLocal() + try: + yield session + finally: + session.close() + Base.metadata.drop_all(bind=engine) + + +def counter(db, metric, label): + row = ( + db.query(models.AnalyticsDaily) + .filter_by(metric=metric, label=label) + .one_or_none() + ) + return row.hits if row else 0 + + +def make_user(db, email=None): + user = models.User(is_guest=email is None, email=email) + db.add(user) + db.commit() + return user + + +# ---------- The buffer and its flush ---------- + +def test_counts_accumulate_across_flushes(db): + analytics.record(analytics.M_PAGE, "/") + analytics.record(analytics.M_PAGE, "/") + analytics.flush(db) + analytics.record(analytics.M_PAGE, "/") + analytics.flush(db) + # The second flush has to find the existing row and add to it. Replacing it + # would leave every counter showing only the newest minute of traffic. + assert counter(db, analytics.M_PAGE, "/") == 3 + + +def test_flush_is_a_no_op_when_nothing_happened(db): + analytics.flush(db) + assert db.query(models.AnalyticsDaily).count() == 0 + + +def test_a_failed_flush_keeps_the_counts(db, monkeypatch): + analytics.record(analytics.M_PAGE, "/") + monkeypatch.setattr(analytics, "_write_counts", lambda *a: 1 / 0) + analytics.flush(db) # must not raise + monkeypatch.undo() + analytics.flush(db) + assert counter(db, analytics.M_PAGE, "/") == 1 + + +def test_label_cardinality_is_capped(db): + for i in range(analytics.MAX_LABELS_PER_METRIC + 25): + analytics.record(analytics.M_REFERRER, f"host{i}.example") + analytics.flush(db) + labels = db.query(models.AnalyticsDaily).filter_by(metric=analytics.M_REFERRER).count() + # Everything past the cap is folded into one bucket, so a referrer flood + # cannot mint rows without limit. + assert labels == analytics.MAX_LABELS_PER_METRIC + 1 + assert counter(db, analytics.M_REFERRER, analytics.OTHER) == 25 + + +# ---------- Visitors ---------- + +def test_visitor_id_is_stable_and_keyed(db, monkeypatch): + user = make_user(db) + handle = analytics.visitor_id(user) + assert handle == analytics.visitor_id(user) # a returning visitor + assert handle != analytics.visitor_id(make_user(db)) # is still one visitor + assert len(handle) == 32 and int(handle, 16) >= 0 # opaque hex, not an id + # Keyed on the app secret, not a bare hash of the user id: otherwise anyone + # holding this table could rebuild the mapping by hashing 1, 2, 3, … + monkeypatch.setattr(analytics.security, "SECRET_KEY", b"a-different-secret") + assert analytics.visitor_id(user) != handle + + +def test_a_repeat_visitor_is_new_only_once(db): + user = make_user(db) + analytics.record_visit(user) + analytics.flush(db) + rows = db.query(models.AnalyticsVisitorDay).all() + assert len(rows) == 1 and rows[0].is_new + + # Same visitor, a later day: seen before, so not new — and not merged into + # the first day's row either. + tomorrow = (models.utcnow().date() + timedelta(days=1)).isoformat() + analytics._visits[(tomorrow, analytics.visitor_id(user))] = set() + analytics.flush(db) + rows = db.query(models.AnalyticsVisitorDay).order_by(models.AnalyticsVisitorDay.day).all() + assert [row.is_new for row in rows] == [True, False] + + +def test_one_row_per_visitor_per_day_however_much_they_do(db): + user = make_user(db) + for _ in range(5): + analytics.record_event(analytics.EV_ADVENTURE, user) + analytics.flush(db) + assert db.query(models.AnalyticsVisitorDay).count() == 1 + assert counter(db, analytics.M_EVENT, analytics.EV_ADVENTURE) == 5 + + +def test_funnel_flags_only_ever_turn_on(db): + user = make_user(db) + analytics.record_event(analytics.EV_TURN, user) + analytics.flush(db) + # A later visit that reaches no funnel step must not clear the earlier one. + analytics.record_visit(user) + analytics.flush(db) + row = db.query(models.AnalyticsVisitorDay).one() + assert row.played and not row.created + + +def test_purge_drops_only_rows_past_the_horizon(db): + old = (models.utcnow().date() - timedelta(days=analytics.RETENTION_DAYS + 1)).isoformat() + db.add(models.AnalyticsVisitorDay(day=old, visitor="a" * 32)) + db.add(models.AnalyticsVisitorDay(day=analytics._today(), visitor="b" * 32)) + db.commit() + assert analytics.purge_old_visitor_days(db) == 1 + assert [r.visitor for r in db.query(models.AnalyticsVisitorDay)] == ["b" * 32] + + +# ---------- Normalizing what a browser claims ---------- + +@pytest.mark.parametrize("path, expected", [ + ("/", "/"), + ("/adventures", "/adventures"), + ("/adventures/", "/adventures"), + ("/play/12?x=1", "/play/:id"), + ("/scenarios/9#top", "/scenarios/:id"), + ("/wp-admin", "(other)"), + ("/play/../../etc", "(other)"), + ("", "/"), +]) +def test_route_normalization(path, expected): + assert analytics.normalize_route(path) == expected + + +@pytest.mark.parametrize("referrer, expected", [ + ("", "(direct)"), + ("https://news.ycombinator.com/item?id=1", "news.ycombinator.com"), + ("https://www.google.com/", "google.com"), + ("https://ai-dnd.example/scenarios", ""), # our own host: not a referral + ("javascript:alert(1)", "(other)"), + ("https://" + "x" * 200 + ".com", "(other)"), +]) +def test_referrer_normalization(referrer, expected): + assert analytics.normalize_referrer(referrer, "ai-dnd.example") == expected + + +@pytest.mark.parametrize("ua, expected", [ + ("Mozilla/5.0 (iPhone; CPU iPhone OS 17_0) AppleWebKit", "mobile"), + ("Mozilla/5.0 (iPad; CPU OS 17_0) AppleWebKit", "tablet"), + ("Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "desktop"), + ("Googlebot/2.1", "bot"), + ("", "(unknown)"), +]) +def test_device_detection(ua, expected): + assert analytics.device_of(ua) == expected + + +def test_only_iso_looking_country_headers_are_trusted(): + assert analytics.country_of({"cf-ipcountry": "de"}) == "DE" + assert analytics.country_of({"cf-ipcountry": "Norway"}) == analytics.UNKNOWN + assert analytics.country_of({"cf-ipcountry": "XX"}) == analytics.UNKNOWN + assert analytics.country_of({}) == analytics.UNKNOWN + + +def test_error_labels_use_the_route_not_the_path(): + class Route: + path = "/api/adventures/{adventure_id}" + + assert analytics.api_route_label({"route": Route()}, 500) == "500 /api/adventures/{adventure_id}" + # An unmatched path is entirely attacker-chosen, so it never becomes a label. + assert analytics.api_route_label({}, 404) == "404 (unmatched)" + + +# ---------- The summary ---------- + +def test_summary_counts_people_once_per_step(db): + one, two = make_user(db), make_user(db) + for _ in range(3): + analytics.record_event(analytics.EV_SCENARIO_OPEN, one) + analytics.record_event(analytics.EV_TURN, one) + analytics.record_event(analytics.EV_SCENARIO_OPEN, two) + + result = analytics.summary(db, days=7) + steps = {row["step"]: row["count"] for row in result["funnel"]} + assert steps["Visited"] == 2 + assert steps["Opened a scenario"] == 2 + assert steps["Played a turn"] == 1 # not 3 — one person, three turns + assert steps["Signed up"] == 0 + # Raw event totals still count every occurrence. + assert result["totals"]["turns"] == 3 + assert result["totals"]["visitors"] == 2 + + +def test_summary_series_covers_every_day_including_empty_ones(db): + analytics.record(analytics.M_PAGE, "/") + result = analytics.summary(db, days=7) + assert len(result["series"]) == 7 + assert result["series"][-1]["day"] == models.utcnow().date().isoformat() + assert result["series"][-1]["pageviews"] == 1 + assert result["series"][0]["pageviews"] == 0 + + +def test_summary_flushes_before_reading(db): + analytics.record(analytics.M_EVENT, analytics.EV_TURN) + # Never flushed by hand: the dashboard must not be up to a minute stale. + assert analytics.summary(db, days=1)["totals"]["turns"] == 1 + + +def test_summary_reports_pages_referrers_and_errors(db): + analytics.record(analytics.M_PAGE, "/play/:id", n=4) + analytics.record(analytics.M_REFERRER, "news.ycombinator.com", n=2) + analytics.record(analytics.M_ERROR, "500 /api/adventures/{adventure_id}") + result = analytics.summary(db, days=30) + assert result["pages"][0] == {"label": "/play/:id", "hits": 4} + assert result["referrers"][0]["label"] == "news.ycombinator.com" + assert result["totals"]["errors"] == 1 + + +# ---------- The endpoints ---------- + +@pytest.fixture() +def client(monkeypatch): + Base.metadata.create_all(bind=engine) + setup = SessionLocal() + visitor = models.User(is_guest=True) + owner = models.User(is_guest=False, email="owner@example.com") + setup.add_all([visitor, owner]) + setup.commit() + ids = {"visitor": visitor.id, "owner": owner.id} + setup.close() + + monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None) + # Multi-user is what makes the gate mean anything: local mode trusts + # whoever is at the keyboard, because it is the operator's own machine. + monkeypatch.setattr(auth, "MULTI_USER", True) + monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"}) + + current = {"id": ids["visitor"]} + + def _current_user(db=Depends(get_db)): + return db.get(models.User, current["id"]) + + app.dependency_overrides[auth.get_current_user] = _current_user + monkeypatch.setattr( + auth, "resolve_session_user", lambda request, db: db.get(models.User, current["id"]) + ) + try: + client = TestClient(app) + client.ids, client.current = ids, current + yield client + finally: + app.dependency_overrides.clear() + Base.metadata.drop_all(bind=engine) + + +def read_summary(client, days=30): + return client.get(f"/api/analytics/summary?days={days}") + + +def test_dashboard_is_invisible_to_everyone_but_the_owner(client): + assert read_summary(client).status_code == 404 + client.current["id"] = client.ids["owner"] + assert read_summary(client).status_code == 200 + + +def test_collect_records_a_pageview_and_the_visit(client): + resp = client.post("/api/analytics/collect", json={"path": "/play/7", "first": True, + "referrer": "https://news.ycombinator.com/"}) + assert resp.status_code == 204 + client.current["id"] = client.ids["owner"] + body = read_summary(client).json() + assert body["pages"][0] == {"label": "/play/:id", "hits": 1} + assert body["referrers"][0]["label"] == "news.ycombinator.com" + assert body["totals"]["visitors"] == 1 + + +def test_referrer_and_device_are_recorded_once_per_visit_not_per_view(client): + for path in ("/", "/scenarios", "/adventures"): + client.post("/api/analytics/collect", json={"path": path, "first": path == "/"}) + client.current["id"] = client.ids["owner"] + body = read_summary(client).json() + assert body["totals"]["pageviews"] == 3 + # Three views, one visit: the referral and the device are facts about the + # visit, so counting them per view would multiply every one of them. + assert sum(row["hits"] for row in body["devices"]) == 1 + + +def test_the_owners_own_visits_are_not_traffic(client): + client.current["id"] = client.ids["owner"] + client.post("/api/analytics/collect", json={"path": "/", "first": True}) + assert read_summary(client).json()["totals"]["pageviews"] == 0 + + +def test_a_client_cannot_invent_pages_or_events(client): + client.post("/api/analytics/collect", json={"path": "/../../admin", "first": True}) + # There is no field for it, so a made-up event is not even expressible. + client.post("/api/analytics/collect", json={"path": "/", "event": "signup"}) + client.current["id"] = client.ids["owner"] + body = read_summary(client).json() + assert {row["label"] for row in body["pages"]} == {"(other)", "/"} + assert body["totals"]["signups"] == 0 + + +def test_api_errors_are_counted_by_route(client): + client.get("/api/adventures/999999") + client.current["id"] = client.ids["owner"] + errors = read_summary(client).json()["errors"] + assert errors and errors[0]["label"].startswith("404 /api/adventures/") + + +# ---------- The dialect the tests never run on ---------- + +def test_the_upserts_compile_for_postgres(): + """Prod is Neon; these tests are SQLite, and a failed flush is caught and + logged rather than raised. A dialect mistake would therefore be invisible + until the dashboard quietly stayed empty — so compile both statements + against Postgres without ever connecting to one. + """ + from sqlalchemy import create_engine + from sqlalchemy.dialects import postgresql + from sqlalchemy.orm import sessionmaker + + session = sessionmaker(bind=create_engine("postgresql+psycopg://u:p@localhost/db"))() + compiled = [] + + def capture(statement, *args, **kwargs): + compiled.append(str(statement.compile(dialect=postgresql.dialect()))) + + session.execute = capture + session.scalars = lambda *a, **k: [] + + analytics._write_counts(session, {("2026-01-01", "pageview", "/"): 2}) + analytics._write_visits(session, {("2026-01-01", "f" * 32): {"played"}}) + + counts, visits = compiled + assert "ON CONFLICT (day, metric, label) DO UPDATE" in counts + assert "analytics_daily.hits + excluded.hits" in counts + assert "ON CONFLICT (day, visitor) DO UPDATE" in visits + assert "analytics_visitor_days.played OR excluded.played" in visits + # is_new is settled by the first write of a visitor's first day and must + # not be in the update clause at all. + assert "is_new" not in visits.split("DO UPDATE")[1] diff --git a/backend/tests/test_ratelimit_hardening.py b/backend/tests/test_ratelimit_hardening.py index aaff05e..05b7a60 100644 --- a/backend/tests/test_ratelimit_hardening.py +++ b/backend/tests/test_ratelimit_hardening.py @@ -3,7 +3,7 @@ per-account login throttle added to close it. Background: uvicorn's --forwarded-allow-ips "*" trusted the LEFTMOST X-Forwarded-For entry, which the client controls, so rotating the header -handed out a fresh rate-limit bucket per request. _client_ip now reads the +handed out a fresh rate-limit bucket per request. client_ip now reads the hop the trusted edge appends (rightmost), and login has an email-keyed throttle that no IP trick can dilute. @@ -31,22 +31,22 @@ class _Req: self.client = None if peer is None else type("C", (), {"host": peer})() -# ---------- _client_ip: the spoof-resistant hop ---------- +# ---------- client_ip: the spoof-resistant hop ---------- def test_client_ip_takes_appended_rightmost_hop(monkeypatch): monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1) # Attacker prepends a fake IP; the edge appends the real one on the right. req = _Req("203.0.113.9, 198.51.100.77") - assert limits._client_ip(req) == "198.51.100.77" + assert limits.client_ip(req) == "198.51.100.77" def test_client_ip_ignores_spoofed_leftmost(monkeypatch): monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1) # Whatever the client stuffs to the left, the keyed IP stays the real hop — # so rotating it no longer mints a new bucket. - a = limits._client_ip(_Req("1.1.1.1, 198.51.100.77")) - b = limits._client_ip(_Req("2.2.2.2, 198.51.100.77")) - c = limits._client_ip(_Req("evil, junk, 198.51.100.77")) + a = limits.client_ip(_Req("1.1.1.1, 198.51.100.77")) + b = limits.client_ip(_Req("2.2.2.2, 198.51.100.77")) + c = limits.client_ip(_Req("evil, junk, 198.51.100.77")) assert a == b == c == "198.51.100.77" @@ -54,12 +54,12 @@ def test_client_ip_honours_extra_trusted_hops(monkeypatch): monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 2) # Two trusted hops: real client is second from the right. req = _Req("9.9.9.9, 203.0.113.5, 198.51.100.77") - assert limits._client_ip(req) == "203.0.113.5" + assert limits.client_ip(req) == "203.0.113.5" def test_client_ip_falls_back_to_socket_peer(): - assert limits._client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4" - assert limits._client_ip(_Req(None, peer=None)) == "unknown" + assert limits.client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4" + assert limits.client_ip(_Req(None, peer=None)) == "unknown" # ---------- per-account login throttle ---------- diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 1718c01..ebcb328 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -23,6 +23,7 @@ what makes it a service rather than a demo. Part 4 is the web plumbing, kept sho - [1.8 Why there is no agent framework](#18-why-there-is-no-agent-framework) - [Part 2 — Data and correctness](#part-2--data-and-correctness) - [Part 3 — Production concerns](#part-3--production-concerns) + - [3.6 Counting visits](#36-counting-visits) - [Part 4 — The web plumbing, briefly](#part-4--the-web-plumbing-briefly) - [Part 5 — Measured results and known limitations](#part-5--measured-results-and-known-limitations) @@ -1048,6 +1049,43 @@ Two things worth knowing about the free tier: CI runs the backend tests, the frontend lint and build, and a Docker image build on every push. +## 3.6 Counting visits + +A hosted demo raises a question a local app never does: is anyone using it, and do they get +anywhere? The answer is an owner-only dashboard at `/analytics`, gated on +`AIDND_ANALYTICS_EMAILS` — a list kept separate from `AIDND_POWER_USERS`, since an unmetered +tester is not automatically someone who should see the traffic. + +**Why it isn't a third-party script.** The CSP allows `script-src 'self'`, so a tracker would +mean loosening it; adblockers eat the popular ones, which silently biases exactly the +technical audience this project is shown to; and none of them can see the measurement that +matters here — a *turn*. The interesting funnel step is not a pageview. + +**Egress is the budget.** After the 189x fix (§2.5) it would be perverse to add a feature +that reads rows per request. So counts accumulate in a process-local dict and flush every 60 +seconds as UPSERTs: **a visit is a write and never a read**. Storage is a generic +`(day, metric, label) -> hits` counter plus one row per visitor per day for the funnel flags. +Every dashboard query is a `GROUP BY` that returns tens of rows regardless of the traffic +behind it, so a month costs a few kilobytes to read back. The cost of the buffer is that a +hard restart can lose up to a minute; the flusher also runs on shutdown, and on a tier that +sleeps when idle the buffer it sleeps on is empty anyway. + +**The numbers are the server's, not the browser's.** The client reports one fact — which page +was viewed — and even that is normalized to a route (`/play/12` → `/play/:id`) against a +whitelist, so the page list cannot be polluted by anything a stranger posts. Everything that +means something — a turn, an adventure, a sign-up — is recorded by the code that performs it. +That also fixes a blind spot: a failed turn is an HTTP 200 with a bad ending, so a +status-code tally cannot see it, and a demo whose model has started refusing looks perfectly +healthy from outside. `turn_error` is counted where the SSE error is written. + +The funnel counts **people, not clicks** — a player who starts six adventures is one person +who started an adventure — which is the entire reason the per-visitor-day table exists. + +One smaller decision worth naming: error buckets are labelled by the matched *route template*, +never the requested path. That gives one bucket per endpoint instead of one per adventure id, +and — the reason it isn't merely tidier — an unmatched path is entirely attacker-chosen, so +labelling by it would let anyone mint rows. + --- # Part 4 — The web plumbing, briefly diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index 6d07100..a8da339 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -1,5 +1,5 @@ -import { useEffect, useState } from 'react' -import { NavLink, Outlet } from 'react-router-dom' +import { useEffect, useRef, useState } from 'react' +import { NavLink, Outlet, useLocation } from 'react-router-dom' import { api } from './api' import { AuthModal, ToastHost } from './components' import Embers from './Embers.jsx' @@ -9,11 +9,28 @@ export default function App() { const [me, setMe] = useState(null) const [authMode, setAuthMode] = useState(null) // 'register' | 'login' | null const [navOpen, setNavOpen] = useState(false) // mobile hamburger menu + const location = useLocation() + const lastPath = useRef(null) useEffect(() => { api.getMe().then(setMe).catch(() => {}) }, []) + // One pageview per route the reader actually lands on. Guarded on the path + // rather than fired on every render: StrictMode runs effects twice in dev, + // and a re-render for unrelated state is not a new page. + useEffect(() => { + if (lastPath.current === location.pathname) return + const first = lastPath.current === null + lastPath.current = location.pathname + // document.referrer survives client-side navigation, so it is only honest + // on the first view — after that this was our own page, not a referral. + api.trackPageview(location.pathname, { + referrer: first ? document.referrer : '', + first, + }) + }, [location.pathname]) + const onAuthed = (newMe, mode) => { setAuthMode(null) if (mode === 'login') { @@ -64,6 +81,12 @@ export default function App() { AI Chat )} + {/* Owner only: the site's own traffic, on its own allowlist. */} + {me?.analytics && ( + `navlink${isActive ? ' active' : ''}`}> + Visitors + + )} {me?.multi_user && (
{me.is_guest ? ( diff --git a/frontend/src/api.js b/frontend/src/api.js index 49bf762..6287efd 100644 --- a/frontend/src/api.js +++ b/frontend/src/api.js @@ -61,6 +61,28 @@ async function streamSSE(path, payload, onEvent, signal, isRetry = false) { } export const api = { + // Analytics. The beacon is deliberately not a `request()`: it must never + // retry, never bootstrap a session, and never surface an error — a counter + // that can interrupt the app it is counting is worse than no counter. + trackPageview: (path, { referrer = '', first = false } = {}) => { + try { + fetch('/api/analytics/collect', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ path, referrer, first }), + keepalive: true, + }).catch(() => {}) + } catch { /* no beacon, no problem */ } + }, + getAnalytics: (days) => request(`/analytics/summary?days=${days}`), + getAccessLog: ({ beforeId, kind, q, limit = 50 } = {}) => { + const params = new URLSearchParams({ limit }) + if (beforeId != null) params.set('before_id', beforeId) + if (kind) params.set('kind', kind) + if (q) params.set('q', q) + return request(`/analytics/access?${params}`) + }, + // Auth (Phase 8 — no-ops in local mode beyond getMe) getMe: () => request('/auth/me'), register: (email, password) => diff --git a/frontend/src/index.css b/frontend/src/index.css index e76941e..15fa0ed 100644 --- a/frontend/src/index.css +++ b/frontend/src/index.css @@ -13,6 +13,13 @@ --accent-glow: rgba(212, 169, 78, 0.25); --danger: #d06565; --player: #9fc7d1; + /* Chart hues (analytics dashboard). Deeper and more saturated than --accent + and --player, which are tuned for text and borders and turn muddy once + they are a 10px bar: these are checked against --bg-panel for lightness, + chroma, contrast and colour-blind separation as a set. */ + --chart-1: #b58a30; + --chart-2: #3d8ac4; + --chart-3: #a8608f; --font-display: 'Cinzel', Georgia, serif; --font-story: 'Crimson Pro', Georgia, 'Times New Roman', serif; --font-ui: 'Inter', 'Segoe UI', system-ui, sans-serif; @@ -2333,6 +2340,293 @@ button.primary.compact { padding: 3px 12px; font-size: 0.76rem; margin-left: aut .card.tome:hover { transform: none; } } +/* ---------- Visit analytics (owner dashboard) ---------- */ + +/* Wider than the reading pages: this one is a grid of small figures, not + prose, and 960px puts two charts and eight tiles into a column. */ +.an-page { max-width: 1180px; } +.an-ranges { display: flex; gap: 6px; } + +/* Four across, not auto-fit: there are eight tiles, and letting them flow + leaves a single orphan on the second row at most widths. */ +.an-tiles { + display: grid; + grid-template-columns: repeat(4, minmax(0, 1fr)); + gap: 12px; + margin-bottom: 18px; +} +@media (max-width: 900px) { + .an-tiles { grid-template-columns: repeat(2, minmax(0, 1fr)); } +} +.an-tile { + background: var(--bg-panel); + border: 1px solid var(--border); + border-radius: 10px; + padding: 14px 16px; +} +.an-tile-value { + font-family: var(--font-display); + font-size: 1.7rem; + line-height: 1.1; + color: var(--accent-bright); +} +.an-tile-label { font-size: 0.82rem; color: var(--text); margin-top: 4px; } +.an-tile-hint { font-size: 0.72rem; color: var(--text-dim); margin-top: 2px; } + +.an-grid { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(340px, 1fr)); + gap: 16px; + margin-bottom: 16px; +} +.an-card { + position: relative; + background: var(--bg-panel); + border: 1px solid var(--border); + border-radius: 12px; + padding: 16px 18px 14px; + margin-bottom: 16px; +} +.an-grid .an-card { margin-bottom: 0; } +.an-card-head { + display: flex; + align-items: baseline; + justify-content: space-between; + gap: 12px; + margin-bottom: 14px; +} +.an-card-head h2 { + margin: 0; + font-family: var(--font-display); + font-size: 0.95rem; + letter-spacing: 0.06em; + color: var(--accent); +} +.an-note { font-size: 0.72rem; color: var(--text-dim); } +.an-empty { color: var(--text-dim); font-style: italic; padding: 18px 0; text-align: center; } + +.an-legend { display: flex; gap: 12px; font-size: 0.72rem; color: var(--text-dim); } +.an-legend-item { display: inline-flex; align-items: center; gap: 5px; } +.an-swatch { width: 9px; height: 9px; border-radius: 2px; display: inline-block; flex: none; } + +/* ----- Day columns ----- */ + +.an-plot { position: relative; height: 170px; margin-top: 12px; } +.an-gridline { + position: absolute; + left: 0; + right: 0; + border-top: 1px dashed var(--border); + pointer-events: none; +} +.an-gridline span { + position: absolute; + right: 0; + top: -0.62em; + font-size: 0.68rem; + color: var(--text-dim); + background: var(--bg-panel); + padding: 0 4px; + /* The dashed rule belongs behind the bars; its value does not. Columns are + positioned, so without this the tallest bar swallows the label. */ + z-index: 3; +} +.an-columns { display: flex; align-items: flex-end; gap: 2px; height: 100%; } +.an-column { + position: relative; + flex: 1 1 0; + min-width: 0; + height: 100%; + display: flex; + align-items: flex-end; + border-radius: 4px 4px 0 0; + outline: none; +} +.an-column.hot { background: rgba(212, 169, 78, 0.07); } +.an-stack { + width: 100%; + /* Capped so a 7-day range draws bars and not slabs; the column around it + stays full width, so the hover target does not shrink with the mark. */ + max-width: 44px; + margin: 0 auto; + height: 100%; + display: flex; + flex-direction: column; + justify-content: flex-end; + /* A 2px gap of surface between stacked segments: the boundary reads as a + boundary without a border, which would eat a thin bar entirely. */ + gap: 2px; +} +.an-bar { width: 100%; transition: filter 0.15s; } +.an-stack > .an-bar:first-child { border-radius: 4px 4px 0 0; } +.an-column.hot .an-bar { filter: brightness(1.2); } +.an-tip { + position: absolute; + bottom: calc(100% + 8px); + left: 50%; + transform: translateX(-50%); + display: flex; + flex-direction: column; + gap: 3px; + padding: 8px 10px; + background: var(--bg-panel); + border: 1px solid var(--border-bright); + border-radius: 8px; + box-shadow: 0 10px 26px rgba(0, 0, 0, 0.55); + font-size: 0.74rem; + white-space: nowrap; + pointer-events: none; + z-index: 4; +} +.an-tip strong { font-weight: 600; color: var(--accent-bright); } +.an-tip span { display: inline-flex; align-items: center; gap: 6px; color: var(--text-dim); } + +.an-axis { display: flex; gap: 2px; margin-top: 7px; } +.an-axis span { + flex: 1 1 0; + min-width: 0; + font-size: 0.68rem; + color: var(--text-dim); + text-align: center; + white-space: nowrap; +} +.an-axis span:first-child { text-align: left; } +.an-axis span:last-child { text-align: right; } + +.an-overlay-empty { + position: absolute; + inset: 0; + display: flex; + align-items: center; + justify-content: center; + color: var(--text-dim); + font-style: italic; + font-size: 0.85rem; + pointer-events: none; +} + +/* ----- Funnel ----- */ + +.an-funnel { display: flex; flex-direction: column; gap: 10px; } +.an-funnel-row { + display: grid; + grid-template-columns: 168px 1fr 96px; + align-items: center; + gap: 12px; +} +.an-funnel-label { font-size: 0.84rem; } +.an-funnel-track { height: 14px; background: var(--bg-input); border-radius: 7px; overflow: hidden; } +.an-funnel-bar { + height: 100%; + min-width: 2px; + background: var(--chart-1); + border-radius: 7px; + transition: width 0.4s ease; +} +.an-funnel-value { + text-align: right; + font-size: 0.86rem; + display: flex; + justify-content: flex-end; + align-items: baseline; + gap: 8px; +} +.an-funnel-share { font-size: 0.72rem; color: var(--text-dim); } + +/* ----- Ranked lists ----- */ + +.an-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 3px; } +.an-list li { + position: relative; + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + padding: 7px 10px; + border-radius: 6px; + font-size: 0.84rem; +} +/* The bar sits behind the row rather than beside it, so a long label keeps the + full width and the magnitude is still visible at a glance. */ +.an-list-fill { + position: absolute; + top: 0; + bottom: 0; + left: 0; + background: rgba(181, 138, 48, 0.2); + border-radius: 6px; +} +.an-list-label { + position: relative; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.an-list-value { position: relative; color: var(--text-dim); flex: none; } + +/* ----- Tabs, and the access log table ----- */ + +.an-tabs { display: flex; gap: 4px; margin-bottom: 16px; border-bottom: 1px solid var(--border); } +.an-tabs button { + background: none; + border: none; + border-bottom: 2px solid transparent; + border-radius: 0; + padding: 8px 14px; + color: var(--text-dim); + font-family: var(--font-display); + font-size: 0.82rem; + letter-spacing: 0.06em; + cursor: pointer; +} +.an-tabs button:hover { color: var(--text); } +.an-tabs button.active { color: var(--accent-bright); border-bottom-color: var(--accent); } + +.an-log-head { flex-wrap: wrap; row-gap: 10px; } +.an-log-search { max-width: 260px; } + +/* The table is the one thing here with a minimum width — six columns of real + data don't compress. It scrolls inside its own box so the page never does. */ +.an-table-scroll { overflow-x: auto; } +.an-table { width: 100%; min-width: 620px; border-collapse: collapse; font-size: 0.82rem; } +.an-table th { + text-align: left; + font-weight: 500; + font-size: 0.72rem; + letter-spacing: 0.08em; + text-transform: uppercase; + color: var(--text-dim); + padding: 0 10px 8px; + border-bottom: 1px solid var(--border); + white-space: nowrap; +} +.an-table td { padding: 8px 10px; border-bottom: 1px solid var(--border); white-space: nowrap; } +.an-table tr:last-child td { border-bottom: none; } +.an-table tbody tr:hover { background: rgba(212, 169, 78, 0.05); } +/* A failed attempt is the row you are scanning for; it gets the only colour in + the table, on a border rather than the text, which stays readable. */ +.an-table tr.failed td:first-child { box-shadow: inset 2px 0 var(--danger); } +.an-table tr.failed td:nth-child(3) { color: var(--danger); } +.an-cell-dim { color: var(--text-dim); } +.an-cell-mono { font-family: ui-monospace, 'Cascadia Code', Consolas, monospace; } +.an-tag { + margin-left: 8px; + padding: 1px 6px; + border: 1px solid var(--border-bright); + border-radius: 10px; + font-size: 0.66rem; + color: var(--text-dim); +} +.an-more { display: flex; justify-content: center; padding-top: 14px; } + +.an-footnote { + color: var(--text-dim); + font-size: 0.74rem; + line-height: 1.6; + margin: 22px 0 0; + max-width: 70ch; +} + /* ============================================================ Mobile / narrow screens (≤ 720px) The desktop Play screen is a horizontal row of up to four columns @@ -2536,4 +2830,19 @@ button.primary.compact { padding: 3px 12px; font-size: 0.76rem; margin-left: aut .branch-map-canvas { padding: 4px 6px 8px; } .branch-map-detail { padding: 12px 16px 16px; } .branch-map-hint { padding: 0 16px 10px; } + + /* ---------- Analytics dashboard ---------- */ + .an-page .page-header { flex-direction: column; align-items: flex-start; gap: 10px; } + .an-tiles { gap: 8px; } + .an-tile { padding: 10px 12px; } + .an-tile-value { font-size: 1.35rem; } + .an-grid { grid-template-columns: 1fr; } + .an-plot { height: 140px; } + .an-tabs button { padding: 8px 10px; font-size: 0.78rem; } + .an-log-search { max-width: none; width: 100%; } + /* Three columns don't fit; the label goes above its own bar instead. */ + .an-funnel-row { grid-template-columns: 1fr auto; gap: 4px 10px; } + .an-funnel-track { grid-column: 1; } + .an-funnel-label { grid-column: 1 / -1; } + .an-funnel-value { grid-column: 2; } } diff --git a/frontend/src/main.jsx b/frontend/src/main.jsx index 9350bb0..bb6f621 100644 --- a/frontend/src/main.jsx +++ b/frontend/src/main.jsx @@ -11,6 +11,7 @@ import Scripts from './pages/Scripts.jsx' import ScriptEditor from './pages/ScriptEditor.jsx' import Settings from './pages/Settings.jsx' import Chat from './pages/Chat.jsx' +import Analytics from './pages/Analytics.jsx' import { trackKeyboardInset } from './keyboard.js' import './index.css' @@ -31,6 +32,8 @@ const router = createBrowserRouter([ { path: 'settings', element: }, // Power users only — the page redirects home and the API 404s otherwise. { path: 'chat', element: }, + // Owner only, by a separate allowlist; same redirect-and-404 treatment. + { path: 'analytics', element: }, ], }, ]) diff --git a/frontend/src/pages/Analytics.jsx b/frontend/src/pages/Analytics.jsx new file mode 100644 index 0000000..4b512a5 --- /dev/null +++ b/frontend/src/pages/Analytics.jsx @@ -0,0 +1,476 @@ +/* Visit analytics — the owner's view of who came by and what they did. + + Everything on this page arrives in a single aggregate response (see + backend/app/analytics.py), so changing the range is one small request, not a + scan of anything. The page is hidden from everyone else: the nav link is + gated on `me.analytics`, this component bounces, and the API 404s. + + Charts are plain HTML — a flex row of columns, a row of bars — rather than + SVG or a charting library. At this size that is less code, responsive for + free, and keeps the CSP as tight as it is. */ +import { useCallback, useEffect, useMemo, useRef, useState } from 'react' +import { useNavigate, useOutletContext } from 'react-router-dom' +import { api } from '../api' + +const RANGES = [ + { days: 7, label: '7 days' }, + { days: 30, label: '30 days' }, + { days: 90, label: '90 days' }, +] + +const nf = new Intl.NumberFormat() + +// Weekday + day for a short range, day + month for a long one: a 90-day axis +// has no room for "Mon". +function dayLabel(iso, days) { + const date = new Date(`${iso}T00:00:00Z`) + const opts = days <= 14 + ? { weekday: 'short', timeZone: 'UTC' } + : { day: 'numeric', month: 'short', timeZone: 'UTC' } + return date.toLocaleDateString(undefined, opts) +} + +function fullDate(iso) { + return new Date(`${iso}T00:00:00Z`) + .toLocaleDateString(undefined, { dateStyle: 'medium', timeZone: 'UTC' }) +} + +/* ---------- Pieces ---------- */ + +function StatTile({ label, value, hint }) { + return ( +
+
{nf.format(value ?? 0)}
+
{label}
+ {hint &&
{hint}
} +
+ ) +} + +/* A day-by-day column chart. `series` names the stacked segments bottom-up; + one segment means one plain bar and no legend, since the title already says + what it is. */ +function DayChart({ title, data, series, days }) { + const [hover, setHover] = useState(null) + const max = Math.max(1, ...data.map((d) => series.reduce((sum, s) => sum + (d[s.key] || 0), 0))) + // Only ever three x labels. Every column labelled is unreadable at 90 days + // and redundant at 7 — the tooltip carries the exact date either way. + const ticks = new Set([0, Math.floor((data.length - 1) / 2), data.length - 1]) + const empty = data.every((d) => series.every((s) => !d[s.key])) + + return ( +
+
+

{title}

+ {series.length > 1 && ( +
+ {[...series].reverse().map((s) => ( + + + {s.label} + + ))} +
+ )} +
+
setHover(null)}> +
{nf.format(max)}
+
{nf.format(Math.round(max / 2))}
+
+ {data.map((point, i) => { + const total = series.reduce((sum, s) => sum + (point[s.key] || 0), 0) + return ( +
setHover(i)} + onFocus={() => setHover(i)} + tabIndex={0} + aria-label={`${fullDate(point.day)}: ${total}`} + > +
+ {[...series].reverse().map((s) => ( + (point[s.key] || 0) > 0 && ( +
+ ) + ))} +
+ {hover === i && ( +
+ {fullDate(point.day)} + {series.map((s) => ( + + + {s.label}: {nf.format(point[s.key] || 0)} + + ))} +
+ )} +
+ ) + })} +
+
+
+ {data.map((point, i) => ( + {ticks.has(i) ? dayLabel(point.day, days) : ''} + ))} +
+ {empty &&
Nothing recorded in this range
} +
+ ) +} + +/* The funnel. Each step's bar is drawn against the first step, so the shape of + the drop-off is the picture; the percentage beside it is of the step above, + which is the number you act on. */ +function Funnel({ steps }) { + const top = steps[0]?.count || 0 + return ( +
+
+

Where visitors get to

+ people, counted once each +
+ {top === 0 ? ( +
No visitors in this range.
+ ) : ( +
+ {steps.map((step, i) => { + const previous = i === 0 ? step.count : steps[i - 1].count + const share = previous ? Math.round((step.count / previous) * 100) : 0 + return ( +
+
{step.step}
+
+
+
+
+ {nf.format(step.count)} + {i > 0 && {share}%} +
+
+ ) + })} +
+ )} +
+ ) +} + +function TopList({ title, rows, note, empty = 'Nothing yet' }) { + const max = Math.max(1, ...rows.map((r) => r.hits)) + return ( +
+
+

{title}

+ {note && {note}} +
+ {rows.length === 0 ? ( +
{empty}
+ ) : ( +
    + {rows.map((row) => ( +
  • + {/* The bar is the row's own background, so a long label stays + readable on top of it instead of being squeezed beside it. */} + + {row.label} + {nf.format(row.hits)} +
    • + ))} +
+ )} +
+ ) +} + +/* ---------- Access log ---------- */ + +const KINDS = [ + { key: '', label: 'Everything' }, + { key: 'session', label: 'Sessions' }, + { key: 'login', label: 'Sign-ins' }, + { key: 'register', label: 'Registrations' }, + { key: 'login_failed', label: 'Failed' }, +] + +const KIND_LABEL = { + session: 'Session', + login: 'Signed in', + register: 'Registered', + login_failed: 'Failed sign-in', +} + +function when(iso) { + const date = new Date(iso.endsWith('Z') ? iso : `${iso}Z`) + return date.toLocaleString(undefined, { dateStyle: 'medium', timeStyle: 'short' }) +} + +function AccessLog() { + const [kind, setKind] = useState('') + const [search, setSearch] = useState('') + const [query, setQuery] = useState('') + const [page, setPage] = useState(null) // { events, has_more } + const [error, setError] = useState(null) + const [busy, setBusy] = useState(false) + + // Typing shouldn't fire a request per keystroke against a table scan. + useEffect(() => { + const timer = setTimeout(() => setQuery(search.trim()), 350) + return () => clearTimeout(timer) + }, [search]) + + useEffect(() => { + let live = true + setPage(null) + api.getAccessLog({ kind, q: query }).then( + (result) => { if (live) { setPage(result); setError(null) } }, + (err) => { if (live) setError(err.message) }, + ) + return () => { live = false } + }, [kind, query]) + + const loadMore = useCallback(async () => { + if (!page?.events.length || busy) return + setBusy(true) + try { + // Anchored on the oldest row already on screen, so rows arriving while + // this is open can't shift the next page. + const next = await api.getAccessLog({ + kind, q: query, beforeId: page.events[page.events.length - 1].id, + }) + setPage({ events: [...page.events, ...next.events], has_more: next.has_more }) + } catch (err) { + setError(err.message) + } finally { + setBusy(false) + } + }, [page, kind, query, busy]) + + return ( +
+
+
+ {KINDS.map((option) => ( + + ))} +
+ setSearch(event.target.value)} + /> +
+ + {error &&
Couldn’t load the log: {error}
} + {!page && !error &&
Reading…
} + + {page && (page.events.length === 0 ? ( +
+ {query || kind ? 'Nothing matches that.' : 'Nothing logged yet.'} +
+ ) : ( + <> +
+ + + + + + + + + {page.events.map((event) => ( + + + + + + + {/* The full user-agent is a wall of text; it lives on the + hover instead of in a column that would push the rest + of the table off screen. */} + + + ))} + +
WhenWhoEventIPCountryDevice
{when(event.at)} + {event.who} + {event.is_guest && guest} + {KIND_LABEL[event.kind] || event.kind}{event.ip || '—'}{event.country || '—'} + {event.device || '—'} +
+
+ {page.has_more && ( +
+ +
+ )} + + ))} +
+ ) +} + +/* ---------- Page ---------- */ + +export default function Analytics() { + const { me } = useOutletContext() ?? {} + const navigate = useNavigate() + const [tab, setTab] = useState('overview') + const [days, setDays] = useState(30) + const [data, setData] = useState(null) + const [error, setError] = useState(null) + const loaded = useRef(false) + + // me is null until /auth/me resolves; only bounce once we know. + useEffect(() => { + if (me && !me.analytics) navigate('/', { replace: true }) + }, [me, navigate]) + + useEffect(() => { + if (tab !== 'overview') return undefined + let live = true + api.getAnalytics(days).then( + (result) => { if (live) { setData(result); setError(null); loaded.current = true } }, + (err) => { if (live) setError(err.message) }, + ) + return () => { live = false } + }, [days, tab]) + + const totals = data?.totals ?? {} + const visitorSeries = useMemo(() => ([ + { key: 'returning', label: 'Returning', color: 'var(--chart-2)' }, + { key: 'new', label: 'New', color: 'var(--chart-1)' }, + ]), []) + // The server sends new-vs-total; the chart stacks, so it wants the remainder. + const visitorDays = useMemo( + () => (data?.series ?? []).map((d) => ({ ...d, returning: d.visitors - d.new })), + [data], + ) + + if (me && !me.analytics) return null + + return ( +
+
+

Visitors

+ {tab === 'overview' && ( +
+ {RANGES.map((range) => ( + + ))} +
+ )} +
+ +
+ + +
+ + {tab === 'access' && } + + {tab === 'overview' && error && ( +
Couldn’t load analytics: {error}
+ )} + {tab === 'overview' && !data && !error &&
Counting…
} + + {tab === 'overview' && data && ( + <> +
+ + + + + + + + +
+ +
+ + +
+ + + +
+ + + + + + +
+ +

+ {fullDate(data.since)} – {fullDate(data.until)}, UTC. Your own visits aren’t + counted here. These totals are anonymous — visitors are counted as one-way + hashes, and nothing on this tab can be traced back to a player or their + stories. The access log tab is the separate, identifying record. +

+ + )} +
+ ) +} diff --git a/plan/STATUS.md b/plan/STATUS.md index 00b99bb..3f43546 100644 --- a/plan/STATUS.md +++ b/plan/STATUS.md @@ -3,7 +3,7 @@ Read this first when picking the project back up. Updated at the end of a working session; the per-phase plan files hold the detail, this holds the thread. -**Last updated: 2026-08-20.** +**Last updated: 2026-08-21.** --- @@ -195,6 +195,74 @@ an automated version; see SP7's entry in `plan/14`. --- +## What happened on 2026-08-21 — visit analytics + +The hosted demo can now answer whether anyone is using it. `/analytics` is a dashboard — +visitors, pages, referrers, countries, devices, which shared scenarios get played, turns and +demo-key spend, API and turn errors, and a funnel from *visited* to *played a turn* to +*signed up*. **Built, green (497 tests), driven by hand against a synthetic 90-day fixture. +Not committed, not deployed.** + +**It is gated on its own allowlist, `AIDND_ANALYTICS_EMAILS`** — not `AIDND_POWER_USERS`. +An unmetered tester is not automatically someone who sees the traffic numbers. The route +404s and the nav link is absent for everyone else, same treatment as AI Chat. Set the var in +the Render dashboard or the page is invisible to everybody, including you. + +**The counters are anonymous; the access log beside them is not, on purpose.** A visitor in +`analytics_daily`/`analytics_visitor_days` is `HMAC(secret, "visitor:")` truncated +to 32 chars — one-way, so those two tables cannot be joined back to `users`, and keyed, so no +client can compute one. Story content never reaches that module. The operator's own visits +are not counted there (multi-user only — excluding them locally would leave the page +permanently empty on the machine it is developed on). + +**`accesslog.py` is the identifying half, added the same week.** `access_events` records +sessions, sign-ins, registrations and failed attempts with address, email (or `Guest #n`), +country and device, read on an "Access log" tab of the same page and behind the same owner +gate. It is a separate module and a separate table so the anonymity of the counters stays a +property of the code rather than a convention. Three things it leans on: the address comes +from `limits.client_ip` — now public, and the only place that decides which hop to trust, so +a spoofed `X-Forwarded-For` cannot forge a row; `user_id` carries **no foreign key** and +`who`/`is_guest` are snapshots, so a row outlives the guest cleanup that deletes the account; +and session rows are thinned to one per day per address, since `/auth/me` runs on every page +load. Nothing is purged — that was the deliberate choice. The published docs describe the +analytics generally and do not enumerate this. + +**A visit is a write and never a read.** Counts accumulate in a process-local dict and flush +every 60s as UPSERTs into `analytics_daily` — a generic `(day, metric, label) -> hits` +counter — plus one row per visitor per day in `analytics_visitor_days` for the funnel flags. +Every dashboard query is a `GROUP BY` returning tens of rows however much traffic sits +behind it. Deliberate, given §2.5: adding a feature that reads rows per request would have +undone the egress work. + +**No migration was needed.** Both tables are new, and `bootstrap()` calls `create_all` on +existing databases too — the same route `branches` took in Phase 14. Nothing was appended to +`MIGRATIONS`, so `LATEST_VERSION` is still 64. + +Three things worth remembering out of building it: + +- **The funnel counts people, not clicks.** A player who starts six adventures is one person + who started an adventure. That is the entire reason the per-visitor-day table exists; its + flags only ever turn on, and `is_new` is settled by the first write of a visitor's first + day and never updated after. +- **A failed turn is an HTTP 200 with a bad ending.** The status-code middleware cannot see + one, so a demo whose model started refusing every request would look perfectly healthy. + `turn_error` is counted in a new `turn_error()` helper that all five SSE error paths in + `_generate_turn` now go through. +- **The tests run on SQLite; production is Neon.** A flush that raises is caught and logged, + so a dialect mistake in the UPSERTs would have been invisible until the dashboard quietly + stayed empty. `test_the_upserts_compile_for_postgres` compiles both statements against the + Postgres dialect without connecting to one. + +**Still not verified: the narrow-screen layout.** The CSS follows the existing `max-width: +720px` block (single-column grids, funnel label above its bar) but `resize_window` is +ignored on a maximized Chrome, and the app sends `frame-ancestors 'none'` so it cannot be +checked in a sized iframe either. Desktop was driven by hand at 1568px. + +**`docs/guide.html` is now behind `docs/GUIDE.md`,** which gained §3.6. Nothing in the repo +regenerates it. + +--- + ## What happened on 2026-08-20 — the branch map The Branches panel gained a **⌗ See the tree** button opening a full-screen map: one diff --git a/render.yaml b/render.yaml index a365227..d4de6d1 100644 --- a/render.yaml +++ b/render.yaml @@ -51,6 +51,13 @@ services: - key: AIDND_POWER_USERS sync: false + # Comma-separated emails allowed to see the Visitors dashboard + # (/analytics). Separate from AIDND_POWER_USERS on purpose — an unmetered + # tester is not automatically someone who sees the traffic. Leave unset + # and the page is invisible to everyone. Set it in the dashboard. + - key: AIDND_ANALYTICS_EMAILS + sync: false + # Guest retention: one account is minted per first-time visitor, so idle # ones are collected (with their adventures) to keep the free-tier # Postgres from filling with abandoned demo data. Registered accounts are