Count the visits, and say whether anyone got anywhere

A hosted demo raises a question a local app never does: is anyone using it,
and do they reach the part that matters? `/analytics` answers it — visitors,
pages, referrers, countries, devices, which shared scenarios get played, turns
and demo-key spend, API and turn errors, and a funnel from visited to played a
turn to signed up.

Not a third-party script, for reasons specific to this one. The CSP allows
`script-src 'self'`, so a tracker means loosening it; adblockers eat the
popular ones, which silently biases exactly the technical audience this
project gets shown to; and none of them can see the measurement that actually
matters here, which is a turn, not a pageview.

**A visit is a write and never a read.** After the 189x egress fix it would be
perverse to add a feature that reads rows per request, so counts accumulate in
a process-local dict and flush every 60s as UPSERTs. Storage is a generic
`(day, metric, label) -> hits` counter, so measuring something new later costs
a constant rather than a migration, plus one row per visitor per day for the
funnel flags. Every dashboard query is a GROUP BY returning tens of rows
however much traffic sits behind it; a month reads back in a few kilobytes.
The buffer's cost is that a hard restart can lose up to a minute — the flusher
also runs on shutdown, and a tier that sleeps when idle sleeps on an empty
buffer anyway.

**The counters are anonymous; the access log beside them is not, on purpose.**
A visitor is `HMAC(secret, "visitor:<user id>")` truncated to 32 chars —
one-way, so `analytics_daily` and `analytics_visitor_days` cannot be joined
back to `users`, and keyed, so no client can compute one. Story content never
reaches that module, and the only content it ever names is a seeded public
scenario's title; a player's own titles are theirs. `accesslog.py` is the
identifying half and is a separate module writing a separate table so that
separation is a property of the code rather than a convention: `access_events`
records sessions, sign-ins, registrations and failed attempts with address,
email and device, read on a second tab of the same page behind the same gate.

Both halves are gated on `AIDND_ANALYTICS_EMAILS`, not `POWER_USERS`. An
unmetered tester is not automatically someone who should see the traffic. The
route 404s and the nav link is absent for everyone else, the same treatment
AI Chat gets; unset in a hosted deploy means nobody sees it, including me.

Three things came out of building it that a test would not have suggested.

**A failed turn is an HTTP 200 with a bad ending.** The status-code middleware
cannot see one, so a demo whose model had started refusing every request would
look perfectly healthy from outside. All five SSE error paths in
`_generate_turn` now go through a `turn_error()` helper that counts on the way
out. Error buckets elsewhere are labelled by the matched route template rather
than the requested path — one bucket per endpoint instead of one per adventure
id, and, the reason it isn't merely tidier, an unmatched path is entirely
attacker-chosen, so labelling by it would let anyone mint rows.

**The funnel counts people, not clicks.** A player who starts six adventures
is one person who started an adventure. That is the whole reason the
per-visitor-day table exists; its flags only ever turn on, and `is_new` is
settled by the first write of a visitor's first day.

**The tests run on SQLite and production is Neon.** A flush that raises is
caught and logged, so a dialect mistake in the UPSERTs would have stayed
invisible until the dashboard quietly never filled.
`test_the_upserts_compile_for_postgres` compiles both statements against the
Postgres dialect without connecting to one.

Two things this leans on elsewhere. `limits._client_ip` is now public
`client_ip`: the access log needs the same answer, and two functions both
deciding which hop is the caller's is how one of them ends up trusting a
header it shouldn't. And the cleanup sweeper now starts if *either* job has
work — a deployment can keep every guest forever and still want its
visitor-day rows aged out.

No migration. Both tables are new and `bootstrap()` calls `create_all` on
existing databases too, the route `branches` took in Phase 14, so
`LATEST_VERSION` is still 64.

497 tests green, frontend lint and build clean, driven by hand against a
synthetic 90-day fixture at 1568px. The narrow-screen layout follows the
existing 720px block but is unverified: `resize_window` is ignored on a
maximized Chrome and `frame-ancestors 'none'` rules out checking it in a sized
iframe. Also repaired here: a rename in test_ratelimit_hardening.py had run
through the test names themselves, leaving `testclient_ip_*` — still collected
by pytest, which is why it passed unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfMCsN1KBLsTqMkj5hSgrY
This commit is contained in:
parththakkar106
2026-08-22 16:24:42 +05:30
co-authored by Claude Opus 5
parent 3b9e6b3d50
commit 041f9e25f3
24 changed files with 2698 additions and 45 deletions
+20 -3
View File
@@ -151,7 +151,7 @@ player input
``` ```
frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI
├─ routers/ auth, scenarios, adventures, story cards, scripts, chat, settings, debug ├─ routers/ auth, scenarios, adventures, story cards, scripts, chat, settings, analytics, debug
├─ models.py SQLAlchemy: User, Scenario, Adventure, Branch, Action, StoryCard, Script, Settings, Memory ├─ models.py SQLAlchemy: User, Scenario, Adventure, Branch, Action, StoryCard, Script, Settings, Memory
├─ migrations.py hand-rolled, versioned via PRAGMA user_version (64 and counting) ├─ migrations.py hand-rolled, versioned via PRAGMA user_version (64 and counting)
├─ auth.py guest/registered users, sessions, shared demo key ├─ auth.py guest/registered users, sessions, shared demo key
@@ -162,6 +162,7 @@ frontend/ React + Vite SPA ──HTTP/SSE──► backend/ FastAPI
├─ worldstate/ the stat engine: clamps, cooldowns, bands, milestones ├─ worldstate/ the stat engine: clamps, cooldowns, bands, milestones
├─ scripting/ quickjs sandbox + AI Dungeon API surface ├─ scripting/ quickjs sandbox + AI Dungeon API surface
├─ memorybank.py auto-summarization + embedding retrieval ├─ memorybank.py auto-summarization + embedding retrieval
├─ analytics.py buffered visit counters + the owner's dashboard query
├─ bundle.py the export/import formats, v2 (tree) and a v1 reader ├─ bundle.py the export/import formats, v2 (tree) and a v1 reader
├─ providers/ OpenAI-compatible adapter, streaming ├─ providers/ OpenAI-compatible adapter, streaming
└─ data.db SQLite (path overridable via AIDND_DB_PATH) └─ data.db SQLite (path overridable via AIDND_DB_PATH)
@@ -172,7 +173,7 @@ development Vite proxies `/api` to FastAPI.
## Tests ## Tests
440 backend tests — unit plus full HTTP integration through the real quickjs scripting engine, 497 backend tests — unit plus full HTTP integration through the real quickjs scripting engine,
with the LLM provider mocked. CI runs them on every push, alongside the frontend lint/build and with the LLM provider mocked. CI runs them on every push, alongside the frontend lint/build and
a Docker image build. a Docker image build.
@@ -202,6 +203,21 @@ the most interesting engineering in the repo:
stay cheap because the lineage is windowed like the history is, so the number of SQL clauses is stay cheap because the lineage is windowed like the history is, so the number of SQL clauses is
bounded by the context window rather than by the number of forks. bounded by the context window rather than by the number of forks.
## Visit analytics
The hosted demo keeps its own analytics: an owner-only dashboard at `/analytics` showing
traffic, which shared scenarios get played, turns and demo-key spend, errors, and a funnel
from *visited* to *played a turn* to *signed up*. It is visible only to the emails listed in
`AIDND_ANALYTICS_EMAILS`, and the route 404s for everyone else.
Built into the app rather than bolted on with a third-party script, for reasons specific to
this one: the CSP allows `script-src 'self'`, adblockers eat the popular trackers, and none
of them can see the measurement that actually matters here — a turn. Counts are aggregated
in memory and flushed as UPSERTs, so a visit is a write and never a read, and every dashboard
query is a `GROUP BY` returning tens of rows however much traffic sits behind it. That last
part is not incidental; see the egress note above for what reading rows per request costs on
this stack.
## Deploy (Render) ## Deploy (Render)
The repo ships a [`render.yaml`](render.yaml) blueprint: one Docker web service that The repo ships a [`render.yaml`](render.yaml) blueprint: one Docker web service that
@@ -211,7 +227,8 @@ serves the SPA and API same-origin, backed by external [Neon](https://neon.tech)
1. Create a **Neon** project and copy its pooled connection string. 1. Create a **Neon** project and copy its pooled connection string.
2. In Render: **New → Blueprint**, point it at this repo. Render reads `render.yaml`. 2. In Render: **New → Blueprint**, point it at this repo. Render reads `render.yaml`.
3. Fill the secrets it prompts for (`sync: false` vars): `AIDND_DATABASE_URL` (the Neon 3. Fill the secrets it prompts for (`sync: false` vars): `AIDND_DATABASE_URL` (the Neon
string) and, to offer a no-signup demo, `AIDND_DEMO_API_KEY` / `AIDND_DEMO_MODELS`. string); to offer a no-signup demo, `AIDND_DEMO_API_KEY` / `AIDND_DEMO_MODELS`; and to
see the Visitors dashboard, `AIDND_ANALYTICS_EMAILS` (your own account's email).
`AIDND_SECRET_KEY` is generated automatically and kept stable across deploys. `AIDND_SECRET_KEY` is generated automatically and kept stable across deploys.
4. Deploy. Pushes to `main` auto-deploy thereafter. Health check: `/api/health`. 4. Deploy. Pushes to `main` auto-deploy thereafter. Health check: `/api/health`.
+14
View File
@@ -66,6 +66,20 @@ AIDND_DEMO_TURNS_PER_DAY=
# Local (single-user) installs are always treated as power users. # Local (single-user) installs are always treated as power users.
AIDND_POWER_USERS= AIDND_POWER_USERS=
# --- Visit analytics ---
# Comma-separated emails allowed to see the Visitors dashboard (/analytics) and
# its nav link. Deliberately separate from AIDND_POWER_USERS: a trusted tester
# gets unmetered turns, which is no reason to hand them the traffic numbers.
# Unset = nobody sees it in a hosted deploy. Local installs always can, and are
# the only mode where the viewer's own visits are still counted (excluding them
# would leave the page permanently empty on the machine it's developed on).
# Collection itself is always on; only the dashboard is gated.
AIDND_ANALYTICS_EMAILS=
# Days to keep the one-row-per-visitor-per-day table that makes the funnel
# count people rather than clicks. The daily counters are aggregate and kept
# forever. Default: 400. Set 0 to keep visitor-days forever.
AIDND_ANALYTICS_RETENTION_DAYS=
# --- Guest retention (only active when AIDND_MULTI_USER=1) --- # --- Guest retention (only active when AIDND_MULTI_USER=1) ---
# Every first visit mints a guest account, so a public demo collects one row # Every first visit mints a guest account, so a public demo collects one row
# per visitor. A guest with no activity for this many days is deleted along # per visitor. A guest with no activity for this many days is deleted along
+160
View File
@@ -0,0 +1,160 @@
"""The access log: who arrived, when, and from where.
The deliberate opposite of analytics.py. That module counts and stores nothing
that points at a person; this one records addresses, email addresses and
devices, because an access log that cannot identify the access is not an access
log. They are kept in separate modules and separate tables on purpose — the
anonymity of the counters is then a property of the code rather than of a
convention someone has to remember.
Owner-only, and never shown to the people it records.
Four kinds of row:
- `session` a browser that has a session made a request — for a guest,
their first visit;
- `login` an existing account signed in;
- `register` a guest upgraded to an account;
- `login_failed` a password attempt that didn't match, with the address tried.
Session rows are the only ones that need thinning: `/auth/me` runs on every
page load, and a row per load would be noise rather than a log. One is written
when the day or the address changes for that user, which is the granularity a
log is actually read at — "seen on the 3rd from 1.2.3.4" — and it still catches
someone moving networks mid-day.
"""
import logging
import threading
from sqlalchemy import desc, or_, select
from sqlalchemy.orm import Session
from . import analytics, models
logger = logging.getLogger(__name__)
SESSION = "session"
LOGIN = "login"
REGISTER = "register"
LOGIN_FAILED = "login_failed"
MAX_UA = 200
# user id -> (day, ip) of the last session row written for them. Process-local
# like the rate limiter's windows, and for the same reason: this is a single
# process, and the worst case after a restart is one redundant row per user.
_last_session: dict[int, tuple[str, str]] = {}
_guard = threading.Lock()
_MAX_TRACKED = 10_000
def _client_ip(request) -> str:
# Deferred: limits imports auth, which is imported by the routers that call
# this, so a module-level import here would close the loop. The spoof
# resistance lives there and must not be reimplemented — a second, laxer
# copy of "what is the client's address" is exactly how one of them ends up
# trusting a header it shouldn't.
from . import limits
return limits.client_ip(request)
def describe(user: models.User) -> str:
"""How a user is named in the log. Guests have no email, and their id is
the only handle anyone has for them. The third case is a local install's
implicit single user, which is also email-less but is the operator rather
than a visitor — calling that one "Guest #1" would be a small lie in the
one row they are certain to read."""
if user.email:
return user.email
return f"Guest #{user.id}" if user.is_guest else f"Local user #{user.id}"
def _country(request) -> str:
"""The edge's country header, or "" when there isn't one. Blank rather than
the counters' "(unknown)" label: a table column reads better as an em dash
than as a word, and an empty string is the honest value for "not known"."""
country = analytics.country_of(request.headers)
return "" if country == analytics.UNKNOWN else country
def record(
db: Session,
kind: str,
request,
*,
user: models.User | None = None,
who: str | None = None,
) -> None:
"""Write one row. Never raises: the log watches sign-in, it doesn't guard
it, and a logging failure must not be able to lock anyone out."""
try:
event = models.AccessEvent(
kind=kind,
user_id=user.id if user is not None else None,
who=(who if who is not None else describe(user) if user else "")[:320],
is_guest=bool(user.is_guest) if user is not None else False,
ip=_client_ip(request)[:45],
country=_country(request),
device=analytics.device_of(request.headers.get("user-agent", "")),
user_agent=(request.headers.get("user-agent") or "")[:MAX_UA],
)
db.add(event)
db.commit()
except Exception: # pragma: no cover - defensive
db.rollback()
logger.exception("Access log write failed; continuing.")
def note_session(db: Session, user: models.User, request) -> None:
"""A session made a request. Thinned to one row per day per address."""
try:
today = analytics._today()
ip = _client_ip(request)
with _guard:
if _last_session.get(user.id) == (today, ip):
return
_last_session[user.id] = (today, ip)
if len(_last_session) > _MAX_TRACKED:
# Nothing here is worth persisting; dropping the map costs at
# most one extra row per active user.
_last_session.clear()
_last_session[user.id] = (today, ip)
except Exception: # pragma: no cover - defensive
logger.exception("Access log session check failed; continuing.")
return
record(db, SESSION, request, user=user)
def recent(
db: Session,
*,
limit: int = 50,
before_id: int | None = None,
kind: str | None = None,
query: str | None = None,
) -> dict:
"""A page of the log, newest first.
Anchored on a row id rather than an offset, like the story pager: rows keep
arriving while it is being read, and an offset would shift the page under
whoever is reading it.
"""
statement = select(models.AccessEvent).order_by(desc(models.AccessEvent.id))
if before_id is not None:
statement = statement.where(models.AccessEvent.id < before_id)
if kind:
statement = statement.where(models.AccessEvent.kind == kind)
if query:
like = f"%{query.strip()}%"
statement = statement.where(or_(
models.AccessEvent.who.ilike(like),
models.AccessEvent.ip.ilike(like),
models.AccessEvent.country.ilike(like),
))
# One extra row answers "is there more" without a second COUNT over the
# whole table.
rows = list(db.scalars(statement.limit(limit + 1)))
has_more = len(rows) > limit
return {"events": rows[:limit], "has_more": has_more}
+556
View File
@@ -0,0 +1,556 @@
"""Visit analytics for the hosted demo.
A small self-hosted counter answering "did anyone visit, and did they play?",
built into the app rather than bolted on with a third-party script: the CSP in
main.py allows scripts from 'self' only, adblockers eat the popular trackers,
and none of them can see the things actually worth knowing here (turns taken,
demo-key spend, which seeded scenario people pick).
Three rules shaped it:
1. **Nothing personal is stored.** No IP addresses, no user agents, no user
ids, no title of anything a player wrote. A visitor appears only as an HMAC
of their user id — one-way and salted with the app's secret key, so these
tables cannot be joined back to an account even by someone holding the
database. Story content never reaches this module at all. What a *specific*
person did is deliberately unanswerable; only totals are.
2. **Egress is the budget.** Neon bills for bytes leaving the database and this
project has already paid for forgetting that once. So counts are aggregated
in memory and flushed as UPSERTs — a visit is a write, never a read — and
every dashboard query is a GROUP BY returning tens of rows, never per-visit
rows. A month of traffic costs a few kilobytes to read back.
3. **The numbers are the server's, not the browser's.** The client reports one
thing: which page was viewed. Everything that *means* something ("a turn
happened", "an account was created") is recorded by the code that does it,
where it can be neither faked by a stranger nor blocked by an extension.
Storage is two tables, both bounded. `analytics_daily` is one row per (day,
metric, label) counter — a few dozen a day. `analytics_visitor_days` is one row
per visitor per day carrying the funnel flags, which is what makes the funnel
count *people* rather than clicks; it is the only table that grows with traffic
and cleanup ages it out.
"""
import hmac
import logging
import os
import re
import threading
from datetime import timedelta
from hashlib import sha256
from urllib.parse import urlsplit
from sqlalchemy import case, func, or_, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from sqlalchemy.orm import Session
from . import models, security
logger = logging.getLogger(__name__)
# ---------- Metrics ----------
# `metric` is the family, `label` the bucket within it. One generic counter
# table beats a column per thing measured: adding a new question later is a
# constant, not a migration.
M_PAGE = "pageview"
M_EVENT = "event"
M_REFERRER = "referrer"
M_DEVICE = "device"
M_COUNTRY = "country"
M_SCENARIO = "scenario" # which seeded/public scenario got played
M_ERROR = "error" # "<status> <route>" for 4xx/5xx on /api
EV_SCENARIO_OPEN = "scenario_opened"
EV_ADVENTURE = "adventure_created"
EV_IMPORT = "adventure_imported"
EV_TURN = "turn"
EV_DEMO_TURN = "demo_turn" # a turn billed to the shared demo key
EV_TURN_ERROR = "turn_error"
EV_SIGNUP = "signup"
EV_LOGIN = "login"
# Events that are also funnel steps: recording one flips a flag on the
# visitor's row for the day, so the funnel counts distinct people-days instead
# of repeat clicks. The name -> column map is the whole definition of the
# funnel; the dashboard reads it back in this order.
FUNNEL_FLAGS = {
EV_SCENARIO_OPEN: "opened",
EV_ADVENTURE: "created",
EV_TURN: "played",
EV_SIGNUP: "signed_up",
}
OTHER = "(other)"
NONE_LABEL = "(direct)"
UNKNOWN = "(unknown)"
# ---------- Bounds ----------
# Everything below exists so a hostile visitor can add rows to these tables no
# faster than an honest one. The only label a client can influence is the
# referrer, and these caps together mean the worst it can do is fill one day's
# referrer list with junk and then be folded into "(other)".
MAX_LABEL_LEN = 80
MAX_LABELS_PER_METRIC = 200 # distinct labels per metric per day, then OTHER
MAX_PENDING = 4000 # buffered entries before an inline flush
FLUSH_INTERVAL_SECONDS = 60
# How long the per-visitor-day rows are kept. The daily counters are tiny and
# kept forever; these are the ones that scale with traffic. A visitor whose
# last visit falls off the end counts as new again — a fair trade at this
# horizon, and it keeps the table from being a permanent record of anyone.
RETENTION_DAYS = int(os.environ.get("AIDND_ANALYTICS_RETENTION_DAYS", "400") or 400)
_HOST_OK = re.compile(r"^[a-z0-9.-]+$")
_COUNTRY_OK = re.compile(r"^[A-Z]{2}$")
_NUMERIC_SEGMENT = re.compile(r"^\d+$")
# SPA routes, in the shape the dashboard should show them. Anything else a
# client claims to have viewed becomes OTHER, so the page list can neither be
# polluted nor accidentally record which adventure someone is reading.
KNOWN_ROUTES = {
"/", "/adventures", "/scenarios", "/scenarios/:id", "/play/:id",
"/scripts", "/scripts/:id", "/settings", "/chat", "/analytics",
}
# ---------- In-process buffer ----------
# Single-process deployment (same assumption as limits.py), so a plain dict
# under a lock is the whole design. Losing up to a minute of counts to a hard
# restart is acceptable for traffic numbers, and the flusher also runs on
# shutdown; on Render's free tier the service is idle when it sleeps, so the
# buffer it sleeps on is empty anyway.
_counts: dict[tuple[str, str, str], int] = {}
_visits: dict[tuple[str, str], set[str]] = {} # (day, visitor) -> flags
_labels_seen: dict[tuple[str, str], set[str]] = {} # (day, metric) -> labels
_guard = threading.Lock()
def _today() -> str:
return models.utcnow().date().isoformat()
def record(metric: str, label: str = "", *, n: int = 1) -> None:
"""Add `n` to one counter. Never raises: analytics must not be able to
fail a request it is only watching."""
try:
day = _today()
label = (label or "").strip()[:MAX_LABEL_LEN]
with _guard:
seen = _labels_seen.setdefault((day, metric), set())
if label not in seen:
if len(seen) >= MAX_LABELS_PER_METRIC:
label = OTHER
else:
seen.add(label)
key = (day, metric, label)
_counts[key] = _counts.get(key, 0) + n
pending = len(_counts) + len(_visits)
except Exception: # pragma: no cover - defensive
logger.exception("Analytics counter failed; continuing.")
return
if pending >= MAX_PENDING:
flush()
def visitor_id(user: models.User) -> str:
"""A stable but one-way handle for one visitor.
HMAC of the user id under the app's secret key. Stable, so a returning
visitor can be told from a new one; one-way, so nothing in the analytics
tables points back at an account; keyed, so a client cannot compute one and
claim to be somebody else. One consequence worth knowing: rotating
AIDND_SECRET_KEY makes every returning visitor look new again.
"""
digest = hmac.new(security.SECRET_KEY, f"visitor:{user.id}".encode(), sha256)
return digest.hexdigest()[:32]
def record_visit(user: models.User | None, *, flag: str | None = None) -> None:
"""Note that this visitor was here today, optionally flipping one funnel
flag. A no-op without a user: a page loaded before a session exists still
counts as a pageview, just not as a person."""
if user is None:
return
try:
with _guard:
flags = _visits.setdefault((_today(), visitor_id(user)), set())
if flag:
flags.add(flag)
except Exception: # pragma: no cover - defensive
logger.exception("Analytics visit failed; continuing.")
def record_event(name: str, user: models.User | None = None) -> None:
"""One thing that happened: counted, and — if it is a funnel step —
credited to the visitor's day. This is the call sites' whole interface."""
record(M_EVENT, name)
record_visit(user, flag=FUNNEL_FLAGS.get(name))
# ---------- Normalizing what the browser reports ----------
def normalize_route(path: str) -> str:
"""A client-reported path, reduced to one of KNOWN_ROUTES.
Numeric segments become ":id" — both to bound the label count and because
*which* adventure someone opened is their business, not a statistic.
"""
path = (path or "/").split("?")[0].split("#")[0]
if not path.startswith("/"):
path = "/" + path
if len(path) > 1:
path = path.rstrip("/")
parts = [":id" if _NUMERIC_SEGMENT.match(p) else p for p in path.split("/")]
route = "/".join(parts) or "/"
return route if route in KNOWN_ROUTES else OTHER
def normalize_referrer(referrer: str, own_host: str = "") -> str:
"""The sending site as a bare host. Our own host means an internal
navigation, which is not a referral — "" tells the caller to skip it."""
if not referrer:
return NONE_LABEL
host = (urlsplit(referrer).hostname or "").lower().lstrip(".")
if not host or not _HOST_OK.match(host) or len(host) > MAX_LABEL_LEN:
return OTHER
if host == (own_host or "").lower() or host in ("localhost", "127.0.0.1"):
return ""
return host[4:] if host.startswith("www.") else host
def api_route_label(scope: dict, status: int) -> str:
"""An error bucket like "500 /api/adventures/{adventure_id}".
The route *template* is used, never the request path: it keeps one bucket
per endpoint instead of one per adventure id, and — the reason it is not
merely tidier — an unmatched path is entirely attacker-chosen, so labelling
by it would let anyone mint rows by requesting nonsense.
"""
template = getattr(scope.get("route"), "path", None)
return f"{status} {template}" if template else f"{status} (unmatched)"
def device_of(user_agent: str) -> str:
"""Mobile / tablet / desktop, and nothing finer. The UA string itself is
never stored — it is a fingerprint, and the answer worth having is one
word."""
ua = (user_agent or "").lower()
if not ua:
return UNKNOWN
if any(bot in ua for bot in ("bot", "crawler", "spider", "headless", "preview")):
return "bot"
if "ipad" in ua or "tablet" in ua or ("android" in ua and "mobile" not in ua):
return "tablet"
if any(m in ua for m in ("mobi", "iphone", "ipod", "android", "phone")):
return "mobile"
return "desktop"
# Geo headers an edge may add. Render fronts services with a CDN that can set
# cf-ipcountry; the others cost nothing to look for. A value is trusted only if
# it looks like an ISO code, since a client can send any header it likes — the
# worst case is therefore a wrong country, never an unbounded label.
_GEO_HEADERS = ("cf-ipcountry", "x-vercel-ip-country", "x-geo-country", "x-country-code")
def country_of(headers) -> str:
for name in _GEO_HEADERS:
value = (headers.get(name) or "").strip().upper()
if _COUNTRY_OK.match(value) and value != "XX":
return value
return UNKNOWN
# ---------- Flushing ----------
def _insert(db: Session):
return sqlite_insert if db.get_bind().dialect.name == "sqlite" else pg_insert
def _drain() -> tuple[dict, dict]:
with _guard:
counts, visits = _counts.copy(), _visits.copy()
_counts.clear()
_visits.clear()
# The label sets only bound cardinality within a day, so let yesterday's
# go rather than growing a map that never shrinks.
today = _today()
for key in [k for k in _labels_seen if k[0] != today]:
del _labels_seen[key]
return counts, visits
def _restore(counts: dict, visits: dict) -> None:
"""Put a failed flush's work back so the next one retries it."""
with _guard:
for key, n in counts.items():
_counts[key] = _counts.get(key, 0) + n
for key, flags in visits.items():
_visits.setdefault(key, set()).update(flags)
def flush(db: Session | None = None) -> None:
"""Write the buffer out. Safe to call from anywhere; never raises."""
counts, visits = _drain()
if not counts and not visits:
return
own_session = db is None
if own_session:
from .database import SessionLocal
db = SessionLocal()
try:
_write_counts(db, counts)
_write_visits(db, visits)
db.commit()
except Exception:
db.rollback()
_restore(counts, visits)
logger.exception("Analytics flush failed; counts held for the next one.")
finally:
if own_session:
db.close()
def _write_counts(db: Session, counts: dict) -> None:
if not counts:
return
table = models.AnalyticsDaily.__table__
rows = [
{"day": day, "metric": metric, "label": label, "hits": hits}
for (day, metric, label), hits in counts.items()
]
stmt = _insert(db)(table).values(rows)
db.execute(stmt.on_conflict_do_update(
index_elements=["day", "metric", "label"],
set_={"hits": table.c.hits + stmt.excluded.hits},
))
def _write_visits(db: Session, visits: dict) -> None:
if not visits:
return
table = models.AnalyticsVisitorDay.__table__
ids = {visitor for _, visitor in visits}
# One indexed lookup settles new-vs-returning for the whole batch. It is
# the only read this module does off the dashboard, and it returns short
# hashes for visitors who are active right now — bounded by the batch.
known = set(db.scalars(
select(models.AnalyticsVisitorDay.visitor)
.where(models.AnalyticsVisitorDay.visitor.in_(ids))
.distinct()
))
rows = [
{
"day": day,
"visitor": visitor,
"is_new": visitor not in known,
**{column: column in flags for column in FUNNEL_FLAGS.values()},
}
for (day, visitor), flags in visits.items()
]
stmt = _insert(db)(table).values(rows)
db.execute(stmt.on_conflict_do_update(
index_elements=["day", "visitor"],
# Flags only ever turn on, and `is_new` is deliberately absent: the
# first write of a visitor's first day is the one that decided it.
set_={
column: or_(table.c[column], stmt.excluded[column])
for column in FUNNEL_FLAGS.values()
},
))
def purge_old_visitor_days(db: Session) -> int:
"""Drop visitor-day rows past the retention horizon. Called by the cleanup
sweeper; the daily counters are never purged — they are aggregate, tiny,
and a portfolio project wants to keep its history."""
if RETENTION_DAYS <= 0:
return 0
cutoff = (models.utcnow().date() - timedelta(days=RETENTION_DAYS)).isoformat()
removed = db.query(models.AnalyticsVisitorDay).filter(
models.AnalyticsVisitorDay.day < cutoff
).delete(synchronize_session=False)
db.commit()
return removed or 0
# ---------- Reading it back ----------
# Every query below is an aggregate: the database does the counting and ships
# back tens of rows, whatever the traffic behind them. Nothing here can return
# a row that belongs to one visitor.
TOP_N = 12
def _top(rows: list[dict], limit: int = TOP_N) -> list[dict]:
return rows[:limit]
def summary(db: Session, days: int = 30) -> dict:
"""Everything the dashboard shows, for the last `days` days (today
included). Flushes first so the numbers include the last minute."""
flush(db)
today = models.utcnow().date()
since = (today - timedelta(days=days - 1)).isoformat()
daily = models.AnalyticsDaily
visitor = models.AnalyticsVisitorDay
# 1. Every counter in the window, folded to (metric, label) totals: the
# page/referrer/country/device/scenario/error tables all come from this
# one pass rather than a query each.
by_metric: dict[str, list[dict]] = {}
for metric, label, hits in db.execute(
select(daily.metric, daily.label, func.sum(daily.hits))
.where(daily.day >= since)
.group_by(daily.metric, daily.label)
):
by_metric.setdefault(metric, []).append({"label": label, "hits": int(hits)})
for rows in by_metric.values():
rows.sort(key=lambda row: -row["hits"])
events = {row["label"]: row["hits"] for row in by_metric.get(M_EVENT, [])}
# 2. Two per-day series worth drawing.
pageviews_by_day = {
day: int(hits)
for day, hits in db.execute(
select(daily.day, func.sum(daily.hits))
.where(daily.day >= since, daily.metric == M_PAGE)
.group_by(daily.day)
)
}
turns_by_day = {
day: int(hits)
for day, hits in db.execute(
select(daily.day, func.sum(daily.hits))
.where(daily.day >= since, daily.metric == M_EVENT, daily.label == EV_TURN)
.group_by(daily.day)
)
}
# 3. People, per day. One row per visitor per day means COUNT(*) is already
# the day's unique visitors — no DISTINCT needed here.
visitors_by_day: dict[str, dict] = {}
for day, total, fresh in db.execute(
select(
visitor.day,
func.count(),
func.sum(case((visitor.is_new, 1), else_=0)),
)
.where(visitor.day >= since)
.group_by(visitor.day)
):
visitors_by_day[day] = {"visitors": int(total), "new": int(fresh or 0)}
# 4. The funnel, over the whole window, counting *people* once each:
# COUNT(DISTINCT CASE WHEN flag THEN visitor END) ignores the NULLs the
# CASE leaves for everyone who didn't reach that step.
unique, unique_new, *reached = db.execute(
select(
func.count(func.distinct(visitor.visitor)),
func.count(func.distinct(case((visitor.is_new, visitor.visitor)))),
*[
func.count(func.distinct(case((visitor.__table__.c[column], visitor.visitor))))
for column in FUNNEL_FLAGS.values()
],
).where(visitor.day >= since)
).one()
series = []
for offset in range(days):
day = (today - timedelta(days=days - 1 - offset)).isoformat()
counted = visitors_by_day.get(day, {})
series.append({
"day": day,
"visitors": counted.get("visitors", 0),
"new": counted.get("new", 0),
"pageviews": pageviews_by_day.get(day, 0),
"turns": turns_by_day.get(day, 0),
})
visits = sum(row["visitors"] for row in series)
pageviews = sum(pageviews_by_day.values())
turns = events.get(EV_TURN, 0)
errors = by_metric.get(M_ERROR, [])
return {
"days": days,
"since": since,
"until": today.isoformat(),
"generated_at": models.utcnow().isoformat(),
"totals": {
# `visitors` counts each person once for the window; `visits` counts
# them once per day they came back, which is the closest honest
# thing to "sessions" without tracking sessions.
"visitors": int(unique),
"new_visitors": int(unique_new),
"visits": visits,
"pageviews": pageviews,
"turns": turns,
"demo_turns": events.get(EV_DEMO_TURN, 0),
"adventures": events.get(EV_ADVENTURE, 0),
"signups": events.get(EV_SIGNUP, 0),
"logins": events.get(EV_LOGIN, 0),
"turn_errors": events.get(EV_TURN_ERROR, 0),
"errors": sum(row["hits"] for row in errors),
"turns_per_visit": round(turns / visits, 1) if visits else 0,
"pages_per_visit": round(pageviews / visits, 1) if visits else 0,
},
"series": series,
# Step 0 is everyone who showed up, so the drop-off between it and
# "Opened a scenario" is visible as a step like any other.
"funnel": [{"step": "Visited", "count": int(unique)}] + [
{"step": step, "count": int(count)}
for step, count in zip(
["Opened a scenario", "Started an adventure", "Played a turn", "Signed up"],
reached,
)
],
"pages": _top(by_metric.get(M_PAGE, [])),
"referrers": _top(by_metric.get(M_REFERRER, [])),
"countries": _top(by_metric.get(M_COUNTRY, [])),
"devices": by_metric.get(M_DEVICE, []),
"scenarios": _top(by_metric.get(M_SCENARIO, [])),
"errors": _top(errors),
"events": by_metric.get(M_EVENT, []),
}
# ---------- Background flusher ----------
# Mirrors cleanup's start/stop pair so main.py's lifespan reads the same way
# for both. The interval is what bounds how much a hard restart can lose.
async def _flush_loop() -> None:
import asyncio
from starlette.concurrency import run_in_threadpool
while True:
await asyncio.sleep(FLUSH_INTERVAL_SECONDS)
# Blocking DB work: keep it off the event loop, which is also serving
# SSE turn streams.
await run_in_threadpool(flush)
def start_flusher():
import asyncio
return asyncio.create_task(_flush_loop())
async def stop_flusher(task) -> None:
"""Cancel the loop and write out whatever it was holding — a deploy is the
one restart that is both frequent and predictable, so it should not be the
thing that loses a minute of counts."""
import asyncio
from starlette.concurrency import run_in_threadpool
if task is not None:
task.cancel()
try:
await task
except asyncio.CancelledError:
pass
await run_in_threadpool(flush)
+19
View File
@@ -66,6 +66,16 @@ POWER_USERS = {
if e.strip() if e.strip()
} }
# Who can see the visit analytics. Deliberately its own list rather than
# POWER_USERS: a trusted tester gets unmetered turns and the AI Chat page,
# which is not a reason to hand them the site's traffic numbers. Empty (the
# default) means nobody sees the dashboard in a hosted deployment.
ANALYTICS_EMAILS = {
e.strip().lower()
for e in os.environ.get("AIDND_ANALYTICS_EMAILS", "").split(",")
if e.strip()
}
DEMO_CAP_MESSAGE = ( DEMO_CAP_MESSAGE = (
f"You've used all {DEMO_TURNS_PER_DAY} free demo turns for today. " f"You've used all {DEMO_TURNS_PER_DAY} free demo turns for today. "
"Add your own API key in Settings to keep playing (it resets tomorrow)." "Add your own API key in Settings to keep playing (it resets tomorrow)."
@@ -148,6 +158,15 @@ def is_power_user(user: models.User) -> bool:
return bool(user.email) and user.email.lower() in POWER_USERS return bool(user.email) and user.email.lower() in POWER_USERS
def is_owner(user: models.User) -> bool:
"""May this user see the visit analytics? Local installs always can — it is
the operator's own machine and their own visits, same reasoning as the
provider debug log; hosted deployments check AIDND_ANALYTICS_EMAILS."""
if not MULTI_USER:
return True
return bool(user.email) and user.email.lower() in ANALYTICS_EMAILS
def demo_turns_left(user: models.User) -> int: def demo_turns_left(user: models.User) -> int:
# Power users are never capped; report the full cap so the banner reads # Power users are never capped; report the full cap so the banner reads
# "N of N" rather than a decrementing count. # "N of N" rather than a decrementing count.
+26 -10
View File
@@ -42,7 +42,7 @@ from sqlalchemy import delete, func
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from starlette.concurrency import run_in_threadpool from starlette.concurrency import run_in_threadpool
from . import auth, models from . import analytics, auth, models
from .database import SessionLocal from .database import SessionLocal
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -72,6 +72,13 @@ def enabled() -> bool:
return auth.MULTI_USER and RETENTION_DAYS > 0 return auth.MULTI_USER and RETENTION_DAYS > 0
def anything_to_sweep() -> bool:
"""Whether the periodic task is worth starting at all. The two jobs it runs
are independent: a deployment can keep every guest forever and still want
its analytics rows aged out, and vice versa."""
return enabled() or analytics.RETENTION_DAYS > 0
def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int: def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int:
"""Delete guests idle for RETENTION_DAYS or more. Returns the row count. """Delete guests idle for RETENTION_DAYS or more. Returns the row count.
@@ -105,12 +112,19 @@ def delete_stale_guests(db: Session, *, now: datetime | None = None) -> int:
def sweep() -> int: def sweep() -> int:
"""One pass, with its own session. Never raises: a failed cleanup must not """One pass, with its own session. Never raises: a failed cleanup must not
be able to take the app down (same rule as seeding).""" be able to take the app down (same rule as seeding). Returns the guest
if not enabled(): count, which is the number worth logging about."""
if not anything_to_sweep():
return 0 return 0
db = SessionLocal() db = SessionLocal()
try: try:
removed = delete_stale_guests(db) # Ages out the per-visitor analytics rows, on its own terms: it is not
# about guests, and it must still happen on a deployment that has
# chosen to keep every account it ever minted.
aged = analytics.purge_old_visitor_days(db)
if aged:
logger.info("Aged out %d analytics visitor-day row(s).", aged)
removed = delete_stale_guests(db) if enabled() else 0
if removed: if removed:
logger.info( logger.info(
"Cleaned up %d guest account(s) idle for %d+ days.", "Cleaned up %d guest account(s) idle for %d+ days.",
@@ -135,16 +149,18 @@ async def _sweep_loop() -> None:
def start_sweeper() -> asyncio.Task | None: def start_sweeper() -> asyncio.Task | None:
"""Kick off the periodic sweep; None when the policy is off.""" """Kick off the periodic sweep; None when there is nothing to sweep."""
if not enabled(): if not enabled():
logger.info("Guest cleanup disabled (multi_user=%s, retention_days=%d).", logger.info("Guest cleanup disabled (multi_user=%s, retention_days=%d).",
auth.MULTI_USER, RETENTION_DAYS) auth.MULTI_USER, RETENTION_DAYS)
else:
logger.info(
"Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).",
RETENTION_DAYS,
SWEEP_INTERVAL_SECONDS // 3600,
)
if not anything_to_sweep():
return None return None
logger.info(
"Guest cleanup on: deleting guests idle %d+ days, every %d hour(s).",
RETENTION_DAYS,
SWEEP_INTERVAL_SECONDS // 3600,
)
return asyncio.create_task(_sweep_loop()) return asyncio.create_task(_sweep_loop())
+14 -6
View File
@@ -32,6 +32,10 @@ RATE_LIMITS: dict[str, tuple[int, int]] = {
"import": (30, 60), # large writes "import": (30, 60), # large writes
"auth": (10, 300), # register/login attempts, per IP "auth": (10, 300), # register/login attempts, per IP
"guest": (30, 300), # new guest users, per IP (each is a DB row) "guest": (30, 300), # new guest users, per IP (each is a DB row)
# Pageview beacons. Generous — a real reader clicking around a SPA fires a
# handful a minute — but low enough that nobody can inflate the traffic
# numbers faster than they could by actually reloading the page.
"analytics": (120, 60),
} }
_windows: dict[tuple[str, str], deque] = defaultdict(deque) _windows: dict[tuple[str, str], deque] = defaultdict(deque)
@@ -49,11 +53,15 @@ _windows_guard = threading.Lock()
TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1)) TRUSTED_PROXY_HOPS = max(1, int(os.environ.get("AIDND_TRUSTED_PROXY_HOPS", "1") or 1))
def _client_ip(request: Request) -> str: def client_ip(request: Request) -> str:
"""The real client IP for rate-limit keying, resistant to a spoofed """The real client IP, resistant to a spoofed X-Forwarded-For. Takes the
X-Forwarded-For. Takes the hop the trusted edge appended (rightmost minus hop the trusted edge appended (rightmost minus any extra trusted hops);
any extra trusted hops); falls back to the socket peer when no forwarded falls back to the socket peer when no forwarded header is present
header is present (local/dev, or a direct connection).""" (local/dev, or a direct connection).
Public because the access log needs the same answer, and two functions that
both decide "which address is the caller's" is how one of them ends up
trusting a header it shouldn't."""
forwarded = request.headers.get("x-forwarded-for") forwarded = request.headers.get("x-forwarded-for")
if forwarded: if forwarded:
parts = [p.strip() for p in forwarded.split(",") if p.strip()] parts = [p.strip() for p in forwarded.split(",") if p.strip()]
@@ -68,7 +76,7 @@ def rate_limit(scope: str, request: Request, user: models.User | None = None) ->
if not auth.MULTI_USER: if not auth.MULTI_USER:
return return
limit, window_seconds = RATE_LIMITS[scope] limit, window_seconds = RATE_LIMITS[scope]
key = (scope, f"u{user.id}" if user else f"ip{_client_ip(request)}") key = (scope, f"u{user.id}" if user else f"ip{client_ip(request)}")
now = time.time() now = time.time()
with _windows_guard: with _windows_guard:
window = _windows[key] window = _windows[key]
+44 -2
View File
@@ -7,12 +7,15 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from starlette.exceptions import HTTPException as StarletteHTTPException from starlette.exceptions import HTTPException as StarletteHTTPException
from . import cleanup from . import analytics, cleanup
from .auth import MULTI_USER from .auth import MULTI_USER
from .database import engine from .database import engine
from .limits import BodySizeLimitMiddleware from .limits import BodySizeLimitMiddleware
from .migrations import bootstrap from .migrations import bootstrap
from .routers import adventures, auth, chat, debug, scenarios, scripts, settings, story_cards from .routers import (
adventures, analytics as analytics_router, auth, chat, debug, scenarios, scripts,
settings, story_cards,
)
from .seed import seed_public_scenarios from .seed import seed_public_scenarios
bootstrap(engine) bootstrap(engine)
@@ -32,10 +35,15 @@ async def lifespan(_app: FastAPI):
# trigger on Render's free tier, where the service sleeps after ~15 # trigger on Render's free tier, where the service sleeps after ~15
# minutes and a long-running timer rarely gets to fire. # minutes and a long-running timer rarely gets to fire.
sweeper = cleanup.start_sweeper() sweeper = cleanup.start_sweeper()
# Visit counters are buffered in memory and written in batches; this is
# what turns them into rows, and stop_flusher writes out the last batch so
# a deploy doesn't drop it.
flusher = analytics.start_flusher()
try: try:
yield yield
finally: finally:
await cleanup.stop_sweeper(sweeper) await cleanup.stop_sweeper(sweeper)
await analytics.stop_flusher(flusher)
# The interactive API docs stay local-only: in multi-user mode they just hand # The interactive API docs stay local-only: in multi-user mode they just hand
@@ -95,6 +103,39 @@ class SecurityHeadersMiddleware:
await self.app(scope, receive, send_with_headers) await self.app(scope, receive, send_with_headers)
class ApiErrorMiddleware:
"""Counts failed API responses for the analytics dashboard.
Here rather than in an exception handler because it sees what the client
actually got: a 429 from a rate limiter, a 404 from routing, a 500 from a
handler that never returned, all the same way. Pure ASGI for the same
reason as the headers above — an SSE turn must not be buffered on its way
out. Only /api is watched; a 404 on the SPA mount is a page load, not a
fault.
"""
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or not scope.get("path", "").startswith("/api"):
return await self.app(scope, receive, send)
async def send_counting(message):
if message["type"] == "http.response.start" and message["status"] >= 400:
# The router has already put the matched route on the scope by
# the time a response starts, so the label can name the
# endpoint rather than the caller's path.
analytics.record(
analytics.M_ERROR,
analytics.api_route_label(scope, message["status"]),
)
await send(message)
await self.app(scope, receive, send_counting)
app.add_middleware(ApiErrorMiddleware)
app.add_middleware(SecurityHeadersMiddleware) app.add_middleware(SecurityHeadersMiddleware)
app.include_router(auth.router) app.include_router(auth.router)
@@ -105,6 +146,7 @@ app.include_router(scripts.router)
app.include_router(settings.router) app.include_router(settings.router)
app.include_router(chat.router) app.include_router(chat.router)
app.include_router(debug.router) app.include_router(debug.router)
app.include_router(analytics_router.router)
@app.get("/api/health") @app.get("/api/health")
+88 -1
View File
@@ -2,7 +2,7 @@ from datetime import datetime, timezone
from sqlalchemy import ( from sqlalchemy import (
JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, LargeBinary, JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, LargeBinary,
String, Table, Text, event, String, Table, Text, UniqueConstraint, event,
) )
from sqlalchemy.orm import Mapped, Session, mapped_column, relationship from sqlalchemy.orm import Mapped, Session, mapped_column, relationship
@@ -575,6 +575,93 @@ class Settings(Base):
return security.decrypt_secret(self.api_key) return security.decrypt_secret(self.api_key)
# ---------- Visit analytics (see analytics.py) ----------
# Two deliberately dumb tables. Neither can hold anything a player wrote, and
# neither can be joined back to a `users` row: the visitor column is an HMAC,
# with no foreign key, so guest cleanup deleting an account leaves the history
# it contributed to intact and anonymous.
class AnalyticsDaily(Base):
"""One counter: how many times `label` happened within `metric` on `day`.
A generic (metric, label, hits) triple rather than a column per statistic,
so measuring something new later costs a constant instead of a migration.
Written only by UPSERT, from a buffer — see analytics.flush.
"""
__tablename__ = "analytics_daily"
id: Mapped[int] = mapped_column(primary_key=True)
day: Mapped[str] = mapped_column(String(10), index=True) # YYYY-MM-DD, UTC
metric: Mapped[str] = mapped_column(String(32))
label: Mapped[str] = mapped_column(String(80), default="")
hits: Mapped[int] = mapped_column(Integer, default=0)
# The upsert target: one row per bucket per day, created or incremented.
__table_args__ = (
UniqueConstraint("day", "metric", "label", name="uq_analytics_daily_bucket"),
)
class AnalyticsVisitorDay(Base):
"""One visitor, one day, and which funnel steps they reached on it.
Exists so the funnel counts people rather than clicks — a player who starts
six adventures is one person who started an adventure. `is_new` is set when
the visitor has no earlier row, which is also why the visitor column is
indexed on its own.
"""
__tablename__ = "analytics_visitor_days"
id: Mapped[int] = mapped_column(primary_key=True)
day: Mapped[str] = mapped_column(String(10))
# HMAC of the user id under the app secret; not reversible, not a key.
visitor: Mapped[str] = mapped_column(String(32))
is_new: Mapped[bool] = mapped_column(Boolean, default=False)
opened: Mapped[bool] = mapped_column(Boolean, default=False)
created: Mapped[bool] = mapped_column(Boolean, default=False)
played: Mapped[bool] = mapped_column(Boolean, default=False)
signed_up: Mapped[bool] = mapped_column(Boolean, default=False)
__table_args__ = (
UniqueConstraint("day", "visitor", name="uq_analytics_visitor_day"),
Index("ix_analytics_visitor", "visitor"),
)
class AccessEvent(Base):
"""One sign-in, registration, failed attempt, or session first-seen.
The counterpart to the two tables above, and deliberately not mixed in with
them: this one identifies people on purpose — address, email, device — so
keeping it in its own table (and its own module) means the anonymity of the
counters stays a property of the code rather than of a convention.
`user_id` is a plain integer with no foreign key. An access log that
disappeared when the account did would not be an access log, and guest
cleanup deletes accounts on a schedule; `who` and `is_guest` are snapshots
for the same reason, so a row still reads correctly afterwards.
"""
__tablename__ = "access_events"
id: Mapped[int] = mapped_column(primary_key=True)
at: Mapped[datetime] = mapped_column(DateTime, default=utcnow, index=True)
# session | login | register | login_failed
kind: Mapped[str] = mapped_column(String(16))
user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# Email for a registered account, "Guest #12" otherwise; for a failed
# sign-in, the address that was tried — which is the point of the row.
who: Mapped[str] = mapped_column(String(320), default="")
is_guest: Mapped[bool] = mapped_column(Boolean, default=False)
ip: Mapped[str] = mapped_column(String(45), default="") # 45 = max IPv6
country: Mapped[str] = mapped_column(String(16), default="")
device: Mapped[str] = mapped_column(String(16), default="")
user_agent: Mapped[str] = mapped_column(String(200), default="")
# Phase 14 — the floor under `tree.place_action`. # Phase 14 — the floor under `tree.place_action`.
# #
# From SP2 a read selects on (branch_id, depth): a node written without them is # From SP2 a read selects on (branch_id, depth): a node written without them is
+33 -8
View File
@@ -9,8 +9,8 @@ from sqlalchemy.orm import Session, load_only, undefer
from sqlalchemy.orm.attributes import set_committed_value from sqlalchemy.orm.attributes import set_committed_value
from .. import ( from .. import (
attempts, auth, bundle, images, limits, memorybank, models, schemas, tree, analytics, attempts, auth, bundle, images, limits, memorybank, models, schemas,
worldstate, tree, worldstate,
) )
from ..context import build_context, cursors from ..context import build_context, cursors
from ..context import history as context_history from ..context import history as context_history
@@ -411,6 +411,12 @@ def create_adventure(
db.commit() db.commit()
db.refresh(adventure) db.refresh(adventure)
analytics.record_event(analytics.EV_ADVENTURE, user)
# Which of the shared scenarios people actually pick — the one piece of
# content this module ever names, and only ever a seeded/public one. A
# player's own scenario titles are theirs.
if scenario is not None and scenario.is_public:
analytics.record(analytics.M_SCENARIO, scenario.title)
return adventure return adventure
@@ -589,6 +595,15 @@ def sse(obj: dict) -> str:
return f"data: {json.dumps(obj)}\n\n" return f"data: {json.dumps(obj)}\n\n"
def turn_error(detail: str, **extra) -> str:
"""An SSE error for a turn that could not be produced, counted on the way
out. Worth its own event: a failed turn is an HTTP 200 with a bad ending,
so the middleware's status-code tally cannot see it — a demo whose model
has started refusing every request looks perfectly healthy from outside."""
analytics.record(analytics.M_EVENT, analytics.EV_TURN_ERROR)
return sse({"type": "error", "detail": detail, **extra})
# no-cache defeats any intermediary caching; X-Accel-Buffering makes # no-cache defeats any intermediary caching; X-Accel-Buffering makes
# nginx-style reverse proxies (hosted deploys) flush each event immediately # nginx-style reverse proxies (hosted deploys) flush each event immediately
# instead of buffering the stream. # instead of buffering the stream.
@@ -745,7 +760,7 @@ async def _generate_turn(
chunks.append(chunk) chunks.append(chunk)
yield sse({"type": "chunk", "text": chunk}) yield sse({"type": "chunk", "text": chunk})
except ProviderError as exc: except ProviderError as exc:
yield sse({"type": "error", "detail": str(exc)}) yield turn_error(str(exc))
return return
text = "".join(chunks).strip() text = "".join(chunks).strip()
@@ -763,13 +778,13 @@ async def _generate_turn(
) )
else: else:
detail = "The AI returned an empty response." detail = "The AI returned an empty response."
yield sse({"type": "error", "detail": detail}) yield turn_error(detail)
return return
# onOutput # onOutput
text, _ = pipeline.run("output", text) text, _ = pipeline.run("output", text)
if not text.strip(): if not text.strip():
yield sse({"type": "error", "detail": "A script's output modifier returned empty text."}) yield turn_error("A script's output modifier returned empty text.")
return return
snapshot["script"] = snapshot["script"] | pipeline.report() snapshot["script"] = snapshot["script"] | pipeline.report()
@@ -785,7 +800,7 @@ async def _generate_turn(
if worldstate.has_schema(stat_schema): if worldstate.has_schema(stat_schema):
text, delta = worldstate.extract_delta(text) text, delta = worldstate.extract_delta(text)
if not text.strip(): if not text.strip():
yield sse({"type": "error", "detail": "The AI returned only a state update and no story text."}) yield turn_error("The AI returned only a state update and no story text.")
return return
new_world_state, ws_report = worldstate.apply_delta( new_world_state, ws_report = worldstate.apply_delta(
adventure.world_state, stat_schema, delta, ai_depth adventure.world_state, stat_schema, delta, ai_depth
@@ -832,6 +847,12 @@ async def _generate_turn(
# in the endpoint); failed provider calls above don't reach here. # in the endpoint); failed provider calls above don't reach here.
auth.count_demo_turn(user) auth.count_demo_turn(user)
db.commit() db.commit()
# Counted here, past every way the turn could still have failed, so the
# number means "stories advanced" rather than "requests attempted". The
# demo tally is those same turns seen as spend on the server-funded key.
analytics.record_event(analytics.EV_TURN, user)
if cfg.using_demo:
analytics.record(analytics.M_EVENT, analytics.EV_DEMO_TURN)
db.refresh(ai_action) db.refresh(ai_action)
yield _SAVED yield _SAVED
yield sse({"type": "done", "action": action_json(ai_action, db), "script": pipeline.report()}) yield sse({"type": "done", "action": action_json(ai_action, db), "script": pipeline.report()})
@@ -876,8 +897,8 @@ async def run_player_turn(
) )
modified, stop = pipeline.run("input", formatted) modified, stop = pipeline.run("input", formatted)
if not modified.strip(): if not modified.strip():
yield sse({"type": "error", "detail": "A script's input modifier returned empty text.", yield turn_error("A script's input modifier returned empty text.",
"script": pipeline.report()}) script=pipeline.report())
return return
player_action = models.Action( player_action = models.Action(
adventure_id=adventure.id, adventure_id=adventure.id,
@@ -1800,6 +1821,10 @@ def import_adventure(
db.commit() db.commit()
db.refresh(adventure) db.refresh(adventure)
# Not a funnel step: importing a bundle is something a returning player
# does, not a sign a first-time visitor got anywhere. Counted anyway,
# because it is the clearest evidence anyone is using the export format.
analytics.record_event(analytics.EV_IMPORT, user)
return adventure return adventure
+137
View File
@@ -0,0 +1,137 @@
"""Visit analytics: one endpoint the browser writes to, one the owner reads.
The split matters. `/collect` is public and takes exactly one fact — which
page was viewed — because anything a stranger can POST is a number a stranger
can invent. Everything the dashboard actually relies on (turns, adventures,
sign-ups, demo spend, errors) is recorded server-side by the code performing
it, so those counts are as trustworthy as the app itself.
The two reading endpoints are owner-only and 404 for everyone else, the same
way the AI Chat router does: a feature nobody else can use is better off not
appearing to exist. `/summary` serves the anonymous counters (analytics.py,
which stores nothing that points at a person) and `/access` serves the access
log (accesslog.py, which identifies people on purpose).
"""
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from .. import accesslog, analytics, auth, limits, models
from ..database import get_db
router = APIRouter(prefix="/api/analytics", tags=["analytics"])
def owner(
db: Session = Depends(get_db),
user: models.User = Depends(auth.get_current_user),
) -> models.User:
"""Gate for the reading half. 404, not 403 — see the module docstring."""
if not auth.is_owner(user):
raise HTTPException(404, "Not found")
return user
Owner = Depends(owner)
class Pageview(BaseModel):
"""What the SPA reports on a page load or a route change.
`first` marks a real page load rather than a client-side navigation: the
things that describe a *visit* rather than a *view* — where it came from,
on what kind of device, from which country — are recorded only then, so a
visitor who clicks around five pages is still one referral.
"""
path: str = Field("", max_length=300)
referrer: str = Field("", max_length=500)
first: bool = False
@router.post("/collect", status_code=204)
def collect(
payload: Pageview,
request: Request,
db: Session = Depends(get_db),
) -> Response:
"""Record one pageview. Always 204, even when nothing was counted: the
browser has no business knowing whether it was."""
limits.rate_limit("analytics", request)
# Resolved by hand rather than through get_current_user: a pageview that
# arrives before /auth/me has minted a session should still be counted as a
# view, not turned into a 401 the SPA has to handle.
user = (
auth.resolve_session_user(request, db)
if auth.MULTI_USER
else auth.local_user(db)
)
# The operator's own clicking is not traffic. Only in multi-user mode —
# locally everyone is the owner, and excluding them would leave the
# dashboard permanently empty on the machine it is developed on.
if auth.MULTI_USER and user is not None and auth.is_owner(user):
return Response(status_code=204)
analytics.record(analytics.M_PAGE, analytics.normalize_route(payload.path))
analytics.record_visit(user)
if payload.first:
referrer = analytics.normalize_referrer(
payload.referrer, request.url.hostname or ""
)
if referrer: # "" means same-origin, which is not a referral
analytics.record(analytics.M_REFERRER, referrer)
analytics.record(
analytics.M_DEVICE,
analytics.device_of(request.headers.get("user-agent", "")),
)
analytics.record(analytics.M_COUNTRY, analytics.country_of(request.headers))
return Response(status_code=204)
@router.get("/summary")
def summary(
days: int = Query(30, ge=1, le=365),
db: Session = Depends(get_db),
_user: models.User = Owner,
) -> dict:
"""The whole dashboard in one aggregate response — a few kilobytes however
much traffic sits behind it."""
return analytics.summary(db, days)
@router.get("/access")
def access_log(
limit: int = Query(50, ge=1, le=200),
before_id: int | None = Query(None),
kind: str | None = Query(None),
q: str | None = Query(None, max_length=120),
db: Session = Depends(get_db),
_user: models.User = Owner,
) -> dict:
"""A page of the access log, newest first.
Unlike /summary this returns rows about people, which is the whole point of
it — so it is behind the same owner gate, paged rather than dumped, and has
no counterpart the people it describes can reach.
"""
page = accesslog.recent(
db, limit=limit, before_id=before_id, kind=kind, query=q
)
return {
"events": [
{
"id": event.id,
"at": event.at.isoformat(),
"kind": event.kind,
"who": event.who,
"is_guest": event.is_guest,
"ip": event.ip,
"country": event.country,
"device": event.device,
"user_agent": event.user_agent,
}
for event in page["events"]
],
"has_more": page["has_more"],
}
+14 -1
View File
@@ -3,7 +3,7 @@ import re
from fastapi import APIRouter, Depends, HTTPException, Request, Response from fastapi import APIRouter, Depends, HTTPException, Request, Response
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from .. import auth, cleanup, limits, models, schemas, security from .. import accesslog, analytics, auth, cleanup, limits, models, schemas, security
from ..database import get_db from ..database import get_db
from .settings import get_settings from .settings import get_settings
@@ -34,6 +34,8 @@ def me_payload(user: models.User, db: Session) -> dict:
"is_guest": user.is_guest, "is_guest": user.is_guest,
# Trusted testers: unmetered demo turns, plus the AI Chat scratchpad. # Trusted testers: unmetered demo turns, plus the AI Chat scratchpad.
"power_user": auth.is_power_user(user), "power_user": auth.is_power_user(user),
# Separate allowlist: shows the visit-analytics page and its nav link.
"analytics": auth.is_owner(user),
# How long an idle guest is kept before cleanup deletes it (None when # How long an idle guest is kept before cleanup deletes it (None when
# the policy is off). Served rather than hardcoded in the UI so the # the policy is off). Served rather than hardcoded in the UI so the
# number a guest is shown is the number actually enforced. # number a guest is shown is the number actually enforced.
@@ -65,6 +67,9 @@ def me(request: Request, response: Response, db: Session = Depends(get_db)):
db.add(user) db.add(user)
db.commit() db.commit()
_set_session_cookie(response, user.id) _set_session_cookie(response, user.id)
# This endpoint is the SPA's bootstrap call, so it is where a session first
# shows itself; accesslog thins the rows down to one per day per address.
accesslog.note_session(db, user, request)
return me_payload(user, db) return me_payload(user, db)
@@ -93,6 +98,8 @@ def register(
user.password_hash = security.hash_password(payload.password) user.password_hash = security.hash_password(payload.password)
user.is_guest = False user.is_guest = False
db.commit() db.commit()
analytics.record_event(analytics.EV_SIGNUP, user)
accesslog.record(db, accesslog.REGISTER, request, user=user)
return me_payload(user, db) return me_payload(user, db)
@@ -119,9 +126,15 @@ def login(
or not security.verify_password(payload.password, user.password_hash) or not security.verify_password(payload.password, user.password_hash)
): ):
limits.note_login_failure(email) limits.note_login_failure(email)
# Logged with the address that was tried, not the account that owns it:
# a guessing run against an address that has no account is exactly the
# thing worth being able to see.
accesslog.record(db, accesslog.LOGIN_FAILED, request, who=email)
raise HTTPException(401, "Incorrect email or password.") raise HTTPException(401, "Incorrect email or password.")
limits.note_login_success(email) limits.note_login_success(email)
_set_session_cookie(response, user.id) _set_session_cookie(response, user.id)
analytics.record_event(analytics.EV_LOGIN, user)
accesslog.record(db, accesslog.LOGIN, request, user=user)
return me_payload(user, db) return me_payload(user, db)
+8 -2
View File
@@ -3,7 +3,7 @@ from fastapi.responses import Response
from sqlalchemy import or_ from sqlalchemy import or_
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from .. import auth, images, limits, models, schemas from .. import analytics, auth, images, limits, models, schemas
from ..database import get_db from ..database import get_db
router = APIRouter(prefix="/api/scenarios", tags=["scenarios"]) router = APIRouter(prefix="/api/scenarios", tags=["scenarios"])
@@ -53,7 +53,13 @@ def get_scenario(
db: Session = Depends(get_db), db: Session = Depends(get_db),
user: models.User = Depends(auth.get_current_user), user: models.User = Depends(auth.get_current_user),
): ):
return get_scenario_or_404(scenario_id, db, user) scenario = get_scenario_or_404(scenario_id, db, user)
# Funnel step. Only for shared scenarios: opening one is the first sign a
# visitor is interested, whereas someone editing their own is already past
# this point — and their titles are theirs, not a statistic.
if scenario.is_public:
analytics.record_event(analytics.EV_SCENARIO_OPEN, user)
return scenario
@router.get("/{scenario_id}/image") @router.get("/{scenario_id}/image")
+215
View File
@@ -0,0 +1,215 @@
"""The access log — app/accesslog.py and GET /api/analytics/access.
This is the half of the analytics work that identifies people on purpose, so
the things worth pinning are the ones that would quietly make it wrong: that
the address recorded is the hardened one and not a header a client chose, that
session rows are thinned instead of written per page load, and that a row
outlives the account it describes — guest cleanup runs on a schedule, and a log
that deletes itself is not a log.
python -m pytest tests/test_accesslog.py -v
"""
import os
import tempfile
_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
_tmp.close()
os.environ["AIDND_DB_PATH"] = _tmp.name
os.environ.pop("AIDND_DATABASE_URL", None)
os.environ.pop("DATABASE_URL", None)
import pytest
from fastapi import Depends
from fastapi.testclient import TestClient
from app import accesslog, auth, limits, models, security
from app.database import Base, SessionLocal, engine, get_db
from app.main import app
EDGE = "198.51.100.77" # what the trusted proxy appended
SPOOF = "10.0.0.1" # what a client put in front of it
@pytest.fixture(autouse=True)
def clean_state():
accesslog._last_session.clear()
yield
accesslog._last_session.clear()
@pytest.fixture()
def client(monkeypatch):
Base.metadata.create_all(bind=engine)
setup = SessionLocal()
owner = models.User(is_guest=False, email="owner@example.com")
member = models.User(
is_guest=False, email="player@example.com",
password_hash=security.hash_password("hunter2long"),
)
setup.add_all([owner, member])
setup.commit()
ids = {"owner": owner.id, "member": member.id}
setup.close()
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
monkeypatch.setattr(limits, "check_login_allowed", lambda *a, **k: None)
monkeypatch.setattr(auth, "MULTI_USER", True)
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
# /auth/me resolves its own session, so the cookie flow below is the real
# one; every other endpoint goes through get_current_user, and `act_as`
# decides who that is.
acting = {"id": ids["owner"]}
def _current(db=Depends(get_db)):
return db.get(models.User, acting["id"])
app.dependency_overrides[auth.get_current_user] = _current
try:
test_client = TestClient(app)
test_client.ids = ids
test_client.act_as = lambda user_id: acting.update(id=user_id)
yield test_client
finally:
app.dependency_overrides.clear()
Base.metadata.drop_all(bind=engine)
def visit(client, ip=EDGE, ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64)"):
return client.get(
"/api/auth/me",
headers={"x-forwarded-for": f"{SPOOF}, {ip}", "user-agent": ua},
)
def rows(kind=None):
db = SessionLocal()
try:
query = db.query(models.AccessEvent).order_by(models.AccessEvent.id)
if kind:
query = query.filter_by(kind=kind)
return query.all()
finally:
db.close()
def read_log(client, **params):
return client.get("/api/analytics/access", params=params)
# ---------- Writing ----------
def test_a_new_session_is_logged(client):
visit(client)
logged = rows()
assert len(logged) == 1
entry = logged[0]
assert entry.kind == accesslog.SESSION
assert entry.is_guest and entry.who.startswith("Guest #")
assert entry.device == "desktop"
def test_the_address_is_the_hardened_one_not_the_clients(client):
visit(client)
# The client prepended its own value; only the hop the edge appended counts.
# Recording the leftmost would make every row forgeable, which for a log is
# worse than having no log.
assert rows()[0].ip == EDGE
def test_session_rows_are_thinned_to_one_per_day_per_address(client):
for _ in range(4):
visit(client)
assert len(rows(accesslog.SESSION)) == 1
def test_a_changed_address_writes_a_new_row(client):
visit(client)
visit(client, ip="203.0.113.9")
logged = rows(accesslog.SESSION)
assert [entry.ip for entry in logged] == [EDGE, "203.0.113.9"]
# Same session throughout, so both rows name the same visitor.
assert logged[0].who == logged[1].who
def test_sign_in_and_failure_are_both_logged(client):
client.post("/api/auth/login", json={"email": "player@example.com", "password": "wrong"},
headers={"x-forwarded-for": EDGE})
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
headers={"x-forwarded-for": EDGE})
kinds = [entry.kind for entry in rows()]
assert accesslog.LOGIN_FAILED in kinds and accesslog.LOGIN in kinds
failure = rows(accesslog.LOGIN_FAILED)[0]
# The address tried, not the account that owns it: a run against an address
# with no account behind it is exactly what this row is for.
assert failure.who == "player@example.com"
assert failure.user_id is None
assert rows(accesslog.LOGIN)[0].user_id == client.ids["member"]
def test_registering_is_logged_against_the_upgraded_account(client):
visit(client) # mints the guest whose session registers
client.act_as(rows()[0].user_id)
client.post("/api/auth/register", json={"email": "new@example.com", "password": "hunter2long"})
entry = rows(accesslog.REGISTER)[0]
assert entry.who == "new@example.com" and not entry.is_guest
def test_a_row_outlives_the_account_it_describes(client):
visit(client)
entry = rows()[0]
db = SessionLocal()
try:
db.delete(db.get(models.User, entry.user_id))
db.commit()
finally:
db.close()
# No foreign key, and `who` is a snapshot — guest cleanup deletes accounts
# on a schedule, and a log that vanishes with them is not a log.
survivor = rows()[0]
assert survivor.who == entry.who and survivor.ip == EDGE
def test_a_long_user_agent_is_truncated(client):
visit(client, ua="Mozilla/" + "x" * 500)
assert len(rows()[0].user_agent) == accesslog.MAX_UA
def test_a_logging_failure_does_not_break_the_request(client, monkeypatch):
monkeypatch.setattr(accesslog, "_client_ip", lambda request: 1 / 0)
# The log watches sign-in; it must not be able to stand in its way.
assert visit(client).status_code == 200
# ---------- Reading ----------
def test_the_log_is_invisible_to_everyone_but_the_owner(client):
visit(client)
assert read_log(client).status_code == 200
client.act_as(client.ids["member"])
assert read_log(client).status_code == 404
def test_the_log_reads_newest_first_and_pages_backwards(client):
for index in range(5):
visit(client, ip=f"203.0.113.{index}")
first = read_log(client, limit=2).json()
assert [event["ip"] for event in first["events"]] == ["203.0.113.4", "203.0.113.3"]
assert first["has_more"]
older = read_log(client, limit=2, before_id=first["events"][-1]["id"]).json()
assert [event["ip"] for event in older["events"]] == ["203.0.113.2", "203.0.113.1"]
def test_the_log_filters_by_kind_and_searches(client):
visit(client)
client.post("/api/auth/login", json={"email": "player@example.com", "password": "hunter2long"},
headers={"x-forwarded-for": "203.0.113.44"})
assert len(read_log(client, kind="login").json()["events"]) == 1
by_email = read_log(client, q="player@example.com").json()["events"]
assert len(by_email) == 1 and by_email[0]["kind"] == "login"
by_ip = read_log(client, q="203.0.113.44").json()["events"]
assert len(by_ip) == 1
assert read_log(client, q="nobody@example.com").json()["events"] == []
+392
View File
@@ -0,0 +1,392 @@
"""Visit analytics — app/analytics.py and the two endpoints in front of it.
Three things are worth testing here and the rest is arithmetic. That the
counters survive the buffer/UPSERT round trip (a flush must add to what is
already stored, not replace it, or every number is only ever the last minute).
That the funnel counts *people* rather than clicks, which is the only reason
the visitor-day table exists. And that the gate holds: a stranger cannot read
the dashboard, and cannot inflate what it says beyond hitting the page.
python -m pytest tests/test_analytics.py -v
"""
import os
import tempfile
_tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
_tmp.close()
os.environ["AIDND_DB_PATH"] = _tmp.name
os.environ.pop("AIDND_DATABASE_URL", None)
os.environ.pop("DATABASE_URL", None)
from datetime import timedelta
import pytest
from fastapi import Depends
from fastapi.testclient import TestClient
from app import analytics, auth, limits, models
from app.database import Base, SessionLocal, engine, get_db
from app.main import app
@pytest.fixture(autouse=True)
def clean_buffer():
"""The buffer is process-wide, so a test that leaves counts in it would
show up inside the next one's flush."""
analytics._counts.clear()
analytics._visits.clear()
analytics._labels_seen.clear()
yield
analytics._counts.clear()
analytics._visits.clear()
analytics._labels_seen.clear()
@pytest.fixture()
def db():
Base.metadata.create_all(bind=engine)
session = SessionLocal()
try:
yield session
finally:
session.close()
Base.metadata.drop_all(bind=engine)
def counter(db, metric, label):
row = (
db.query(models.AnalyticsDaily)
.filter_by(metric=metric, label=label)
.one_or_none()
)
return row.hits if row else 0
def make_user(db, email=None):
user = models.User(is_guest=email is None, email=email)
db.add(user)
db.commit()
return user
# ---------- The buffer and its flush ----------
def test_counts_accumulate_across_flushes(db):
analytics.record(analytics.M_PAGE, "/")
analytics.record(analytics.M_PAGE, "/")
analytics.flush(db)
analytics.record(analytics.M_PAGE, "/")
analytics.flush(db)
# The second flush has to find the existing row and add to it. Replacing it
# would leave every counter showing only the newest minute of traffic.
assert counter(db, analytics.M_PAGE, "/") == 3
def test_flush_is_a_no_op_when_nothing_happened(db):
analytics.flush(db)
assert db.query(models.AnalyticsDaily).count() == 0
def test_a_failed_flush_keeps_the_counts(db, monkeypatch):
analytics.record(analytics.M_PAGE, "/")
monkeypatch.setattr(analytics, "_write_counts", lambda *a: 1 / 0)
analytics.flush(db) # must not raise
monkeypatch.undo()
analytics.flush(db)
assert counter(db, analytics.M_PAGE, "/") == 1
def test_label_cardinality_is_capped(db):
for i in range(analytics.MAX_LABELS_PER_METRIC + 25):
analytics.record(analytics.M_REFERRER, f"host{i}.example")
analytics.flush(db)
labels = db.query(models.AnalyticsDaily).filter_by(metric=analytics.M_REFERRER).count()
# Everything past the cap is folded into one bucket, so a referrer flood
# cannot mint rows without limit.
assert labels == analytics.MAX_LABELS_PER_METRIC + 1
assert counter(db, analytics.M_REFERRER, analytics.OTHER) == 25
# ---------- Visitors ----------
def test_visitor_id_is_stable_and_keyed(db, monkeypatch):
user = make_user(db)
handle = analytics.visitor_id(user)
assert handle == analytics.visitor_id(user) # a returning visitor
assert handle != analytics.visitor_id(make_user(db)) # is still one visitor
assert len(handle) == 32 and int(handle, 16) >= 0 # opaque hex, not an id
# Keyed on the app secret, not a bare hash of the user id: otherwise anyone
# holding this table could rebuild the mapping by hashing 1, 2, 3, …
monkeypatch.setattr(analytics.security, "SECRET_KEY", b"a-different-secret")
assert analytics.visitor_id(user) != handle
def test_a_repeat_visitor_is_new_only_once(db):
user = make_user(db)
analytics.record_visit(user)
analytics.flush(db)
rows = db.query(models.AnalyticsVisitorDay).all()
assert len(rows) == 1 and rows[0].is_new
# Same visitor, a later day: seen before, so not new — and not merged into
# the first day's row either.
tomorrow = (models.utcnow().date() + timedelta(days=1)).isoformat()
analytics._visits[(tomorrow, analytics.visitor_id(user))] = set()
analytics.flush(db)
rows = db.query(models.AnalyticsVisitorDay).order_by(models.AnalyticsVisitorDay.day).all()
assert [row.is_new for row in rows] == [True, False]
def test_one_row_per_visitor_per_day_however_much_they_do(db):
user = make_user(db)
for _ in range(5):
analytics.record_event(analytics.EV_ADVENTURE, user)
analytics.flush(db)
assert db.query(models.AnalyticsVisitorDay).count() == 1
assert counter(db, analytics.M_EVENT, analytics.EV_ADVENTURE) == 5
def test_funnel_flags_only_ever_turn_on(db):
user = make_user(db)
analytics.record_event(analytics.EV_TURN, user)
analytics.flush(db)
# A later visit that reaches no funnel step must not clear the earlier one.
analytics.record_visit(user)
analytics.flush(db)
row = db.query(models.AnalyticsVisitorDay).one()
assert row.played and not row.created
def test_purge_drops_only_rows_past_the_horizon(db):
old = (models.utcnow().date() - timedelta(days=analytics.RETENTION_DAYS + 1)).isoformat()
db.add(models.AnalyticsVisitorDay(day=old, visitor="a" * 32))
db.add(models.AnalyticsVisitorDay(day=analytics._today(), visitor="b" * 32))
db.commit()
assert analytics.purge_old_visitor_days(db) == 1
assert [r.visitor for r in db.query(models.AnalyticsVisitorDay)] == ["b" * 32]
# ---------- Normalizing what a browser claims ----------
@pytest.mark.parametrize("path, expected", [
("/", "/"),
("/adventures", "/adventures"),
("/adventures/", "/adventures"),
("/play/12?x=1", "/play/:id"),
("/scenarios/9#top", "/scenarios/:id"),
("/wp-admin", "(other)"),
("/play/../../etc", "(other)"),
("", "/"),
])
def test_route_normalization(path, expected):
assert analytics.normalize_route(path) == expected
@pytest.mark.parametrize("referrer, expected", [
("", "(direct)"),
("https://news.ycombinator.com/item?id=1", "news.ycombinator.com"),
("https://www.google.com/", "google.com"),
("https://ai-dnd.example/scenarios", ""), # our own host: not a referral
("javascript:alert(1)", "(other)"),
("https://" + "x" * 200 + ".com", "(other)"),
])
def test_referrer_normalization(referrer, expected):
assert analytics.normalize_referrer(referrer, "ai-dnd.example") == expected
@pytest.mark.parametrize("ua, expected", [
("Mozilla/5.0 (iPhone; CPU iPhone OS 17_0) AppleWebKit", "mobile"),
("Mozilla/5.0 (iPad; CPU OS 17_0) AppleWebKit", "tablet"),
("Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "desktop"),
("Googlebot/2.1", "bot"),
("", "(unknown)"),
])
def test_device_detection(ua, expected):
assert analytics.device_of(ua) == expected
def test_only_iso_looking_country_headers_are_trusted():
assert analytics.country_of({"cf-ipcountry": "de"}) == "DE"
assert analytics.country_of({"cf-ipcountry": "Norway"}) == analytics.UNKNOWN
assert analytics.country_of({"cf-ipcountry": "XX"}) == analytics.UNKNOWN
assert analytics.country_of({}) == analytics.UNKNOWN
def test_error_labels_use_the_route_not_the_path():
class Route:
path = "/api/adventures/{adventure_id}"
assert analytics.api_route_label({"route": Route()}, 500) == "500 /api/adventures/{adventure_id}"
# An unmatched path is entirely attacker-chosen, so it never becomes a label.
assert analytics.api_route_label({}, 404) == "404 (unmatched)"
# ---------- The summary ----------
def test_summary_counts_people_once_per_step(db):
one, two = make_user(db), make_user(db)
for _ in range(3):
analytics.record_event(analytics.EV_SCENARIO_OPEN, one)
analytics.record_event(analytics.EV_TURN, one)
analytics.record_event(analytics.EV_SCENARIO_OPEN, two)
result = analytics.summary(db, days=7)
steps = {row["step"]: row["count"] for row in result["funnel"]}
assert steps["Visited"] == 2
assert steps["Opened a scenario"] == 2
assert steps["Played a turn"] == 1 # not 3 — one person, three turns
assert steps["Signed up"] == 0
# Raw event totals still count every occurrence.
assert result["totals"]["turns"] == 3
assert result["totals"]["visitors"] == 2
def test_summary_series_covers_every_day_including_empty_ones(db):
analytics.record(analytics.M_PAGE, "/")
result = analytics.summary(db, days=7)
assert len(result["series"]) == 7
assert result["series"][-1]["day"] == models.utcnow().date().isoformat()
assert result["series"][-1]["pageviews"] == 1
assert result["series"][0]["pageviews"] == 0
def test_summary_flushes_before_reading(db):
analytics.record(analytics.M_EVENT, analytics.EV_TURN)
# Never flushed by hand: the dashboard must not be up to a minute stale.
assert analytics.summary(db, days=1)["totals"]["turns"] == 1
def test_summary_reports_pages_referrers_and_errors(db):
analytics.record(analytics.M_PAGE, "/play/:id", n=4)
analytics.record(analytics.M_REFERRER, "news.ycombinator.com", n=2)
analytics.record(analytics.M_ERROR, "500 /api/adventures/{adventure_id}")
result = analytics.summary(db, days=30)
assert result["pages"][0] == {"label": "/play/:id", "hits": 4}
assert result["referrers"][0]["label"] == "news.ycombinator.com"
assert result["totals"]["errors"] == 1
# ---------- The endpoints ----------
@pytest.fixture()
def client(monkeypatch):
Base.metadata.create_all(bind=engine)
setup = SessionLocal()
visitor = models.User(is_guest=True)
owner = models.User(is_guest=False, email="owner@example.com")
setup.add_all([visitor, owner])
setup.commit()
ids = {"visitor": visitor.id, "owner": owner.id}
setup.close()
monkeypatch.setattr(limits, "rate_limit", lambda *a, **k: None)
# Multi-user is what makes the gate mean anything: local mode trusts
# whoever is at the keyboard, because it is the operator's own machine.
monkeypatch.setattr(auth, "MULTI_USER", True)
monkeypatch.setattr(auth, "ANALYTICS_EMAILS", {"owner@example.com"})
current = {"id": ids["visitor"]}
def _current_user(db=Depends(get_db)):
return db.get(models.User, current["id"])
app.dependency_overrides[auth.get_current_user] = _current_user
monkeypatch.setattr(
auth, "resolve_session_user", lambda request, db: db.get(models.User, current["id"])
)
try:
client = TestClient(app)
client.ids, client.current = ids, current
yield client
finally:
app.dependency_overrides.clear()
Base.metadata.drop_all(bind=engine)
def read_summary(client, days=30):
return client.get(f"/api/analytics/summary?days={days}")
def test_dashboard_is_invisible_to_everyone_but_the_owner(client):
assert read_summary(client).status_code == 404
client.current["id"] = client.ids["owner"]
assert read_summary(client).status_code == 200
def test_collect_records_a_pageview_and_the_visit(client):
resp = client.post("/api/analytics/collect", json={"path": "/play/7", "first": True,
"referrer": "https://news.ycombinator.com/"})
assert resp.status_code == 204
client.current["id"] = client.ids["owner"]
body = read_summary(client).json()
assert body["pages"][0] == {"label": "/play/:id", "hits": 1}
assert body["referrers"][0]["label"] == "news.ycombinator.com"
assert body["totals"]["visitors"] == 1
def test_referrer_and_device_are_recorded_once_per_visit_not_per_view(client):
for path in ("/", "/scenarios", "/adventures"):
client.post("/api/analytics/collect", json={"path": path, "first": path == "/"})
client.current["id"] = client.ids["owner"]
body = read_summary(client).json()
assert body["totals"]["pageviews"] == 3
# Three views, one visit: the referral and the device are facts about the
# visit, so counting them per view would multiply every one of them.
assert sum(row["hits"] for row in body["devices"]) == 1
def test_the_owners_own_visits_are_not_traffic(client):
client.current["id"] = client.ids["owner"]
client.post("/api/analytics/collect", json={"path": "/", "first": True})
assert read_summary(client).json()["totals"]["pageviews"] == 0
def test_a_client_cannot_invent_pages_or_events(client):
client.post("/api/analytics/collect", json={"path": "/../../admin", "first": True})
# There is no field for it, so a made-up event is not even expressible.
client.post("/api/analytics/collect", json={"path": "/", "event": "signup"})
client.current["id"] = client.ids["owner"]
body = read_summary(client).json()
assert {row["label"] for row in body["pages"]} == {"(other)", "/"}
assert body["totals"]["signups"] == 0
def test_api_errors_are_counted_by_route(client):
client.get("/api/adventures/999999")
client.current["id"] = client.ids["owner"]
errors = read_summary(client).json()["errors"]
assert errors and errors[0]["label"].startswith("404 /api/adventures/")
# ---------- The dialect the tests never run on ----------
def test_the_upserts_compile_for_postgres():
"""Prod is Neon; these tests are SQLite, and a failed flush is caught and
logged rather than raised. A dialect mistake would therefore be invisible
until the dashboard quietly stayed empty — so compile both statements
against Postgres without ever connecting to one.
"""
from sqlalchemy import create_engine
from sqlalchemy.dialects import postgresql
from sqlalchemy.orm import sessionmaker
session = sessionmaker(bind=create_engine("postgresql+psycopg://u:p@localhost/db"))()
compiled = []
def capture(statement, *args, **kwargs):
compiled.append(str(statement.compile(dialect=postgresql.dialect())))
session.execute = capture
session.scalars = lambda *a, **k: []
analytics._write_counts(session, {("2026-01-01", "pageview", "/"): 2})
analytics._write_visits(session, {("2026-01-01", "f" * 32): {"played"}})
counts, visits = compiled
assert "ON CONFLICT (day, metric, label) DO UPDATE" in counts
assert "analytics_daily.hits + excluded.hits" in counts
assert "ON CONFLICT (day, visitor) DO UPDATE" in visits
assert "analytics_visitor_days.played OR excluded.played" in visits
# is_new is settled by the first write of a visitor's first day and must
# not be in the update clause at all.
assert "is_new" not in visits.split("DO UPDATE")[1]
+9 -9
View File
@@ -3,7 +3,7 @@ per-account login throttle added to close it.
Background: uvicorn's --forwarded-allow-ips "*" trusted the LEFTMOST Background: uvicorn's --forwarded-allow-ips "*" trusted the LEFTMOST
X-Forwarded-For entry, which the client controls, so rotating the header X-Forwarded-For entry, which the client controls, so rotating the header
handed out a fresh rate-limit bucket per request. _client_ip now reads the handed out a fresh rate-limit bucket per request. client_ip now reads the
hop the trusted edge appends (rightmost), and login has an email-keyed throttle hop the trusted edge appends (rightmost), and login has an email-keyed throttle
that no IP trick can dilute. that no IP trick can dilute.
@@ -31,22 +31,22 @@ class _Req:
self.client = None if peer is None else type("C", (), {"host": peer})() self.client = None if peer is None else type("C", (), {"host": peer})()
# ---------- _client_ip: the spoof-resistant hop ---------- # ---------- client_ip: the spoof-resistant hop ----------
def test_client_ip_takes_appended_rightmost_hop(monkeypatch): def test_client_ip_takes_appended_rightmost_hop(monkeypatch):
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1) monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
# Attacker prepends a fake IP; the edge appends the real one on the right. # Attacker prepends a fake IP; the edge appends the real one on the right.
req = _Req("203.0.113.9, 198.51.100.77") req = _Req("203.0.113.9, 198.51.100.77")
assert limits._client_ip(req) == "198.51.100.77" assert limits.client_ip(req) == "198.51.100.77"
def test_client_ip_ignores_spoofed_leftmost(monkeypatch): def test_client_ip_ignores_spoofed_leftmost(monkeypatch):
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1) monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 1)
# Whatever the client stuffs to the left, the keyed IP stays the real hop — # Whatever the client stuffs to the left, the keyed IP stays the real hop —
# so rotating it no longer mints a new bucket. # so rotating it no longer mints a new bucket.
a = limits._client_ip(_Req("1.1.1.1, 198.51.100.77")) a = limits.client_ip(_Req("1.1.1.1, 198.51.100.77"))
b = limits._client_ip(_Req("2.2.2.2, 198.51.100.77")) b = limits.client_ip(_Req("2.2.2.2, 198.51.100.77"))
c = limits._client_ip(_Req("evil, junk, 198.51.100.77")) c = limits.client_ip(_Req("evil, junk, 198.51.100.77"))
assert a == b == c == "198.51.100.77" assert a == b == c == "198.51.100.77"
@@ -54,12 +54,12 @@ def test_client_ip_honours_extra_trusted_hops(monkeypatch):
monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 2) monkeypatch.setattr(limits, "TRUSTED_PROXY_HOPS", 2)
# Two trusted hops: real client is second from the right. # Two trusted hops: real client is second from the right.
req = _Req("9.9.9.9, 203.0.113.5, 198.51.100.77") req = _Req("9.9.9.9, 203.0.113.5, 198.51.100.77")
assert limits._client_ip(req) == "203.0.113.5" assert limits.client_ip(req) == "203.0.113.5"
def test_client_ip_falls_back_to_socket_peer(): def test_client_ip_falls_back_to_socket_peer():
assert limits._client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4" assert limits.client_ip(_Req(None, peer="172.16.0.4")) == "172.16.0.4"
assert limits._client_ip(_Req(None, peer=None)) == "unknown" assert limits.client_ip(_Req(None, peer=None)) == "unknown"
# ---------- per-account login throttle ---------- # ---------- per-account login throttle ----------
+38
View File
@@ -23,6 +23,7 @@ what makes it a service rather than a demo. Part 4 is the web plumbing, kept sho
- [1.8 Why there is no agent framework](#18-why-there-is-no-agent-framework) - [1.8 Why there is no agent framework](#18-why-there-is-no-agent-framework)
- [Part 2 — Data and correctness](#part-2--data-and-correctness) - [Part 2 — Data and correctness](#part-2--data-and-correctness)
- [Part 3 — Production concerns](#part-3--production-concerns) - [Part 3 — Production concerns](#part-3--production-concerns)
- [3.6 Counting visits](#36-counting-visits)
- [Part 4 — The web plumbing, briefly](#part-4--the-web-plumbing-briefly) - [Part 4 — The web plumbing, briefly](#part-4--the-web-plumbing-briefly)
- [Part 5 — Measured results and known limitations](#part-5--measured-results-and-known-limitations) - [Part 5 — Measured results and known limitations](#part-5--measured-results-and-known-limitations)
@@ -1048,6 +1049,43 @@ Two things worth knowing about the free tier:
CI runs the backend tests, the frontend lint and build, and a Docker image build on every CI runs the backend tests, the frontend lint and build, and a Docker image build on every
push. push.
## 3.6 Counting visits
A hosted demo raises a question a local app never does: is anyone using it, and do they get
anywhere? The answer is an owner-only dashboard at `/analytics`, gated on
`AIDND_ANALYTICS_EMAILS` — a list kept separate from `AIDND_POWER_USERS`, since an unmetered
tester is not automatically someone who should see the traffic.
**Why it isn't a third-party script.** The CSP allows `script-src 'self'`, so a tracker would
mean loosening it; adblockers eat the popular ones, which silently biases exactly the
technical audience this project is shown to; and none of them can see the measurement that
matters here — a *turn*. The interesting funnel step is not a pageview.
**Egress is the budget.** After the 189x fix (§2.5) it would be perverse to add a feature
that reads rows per request. So counts accumulate in a process-local dict and flush every 60
seconds as UPSERTs: **a visit is a write and never a read**. Storage is a generic
`(day, metric, label) -> hits` counter plus one row per visitor per day for the funnel flags.
Every dashboard query is a `GROUP BY` that returns tens of rows regardless of the traffic
behind it, so a month costs a few kilobytes to read back. The cost of the buffer is that a
hard restart can lose up to a minute; the flusher also runs on shutdown, and on a tier that
sleeps when idle the buffer it sleeps on is empty anyway.
**The numbers are the server's, not the browser's.** The client reports one fact — which page
was viewed — and even that is normalized to a route (`/play/12` → `/play/:id`) against a
whitelist, so the page list cannot be polluted by anything a stranger posts. Everything that
means something — a turn, an adventure, a sign-up — is recorded by the code that performs it.
That also fixes a blind spot: a failed turn is an HTTP 200 with a bad ending, so a
status-code tally cannot see it, and a demo whose model has started refusing looks perfectly
healthy from outside. `turn_error` is counted where the SSE error is written.
The funnel counts **people, not clicks** — a player who starts six adventures is one person
who started an adventure — which is the entire reason the per-visitor-day table exists.
One smaller decision worth naming: error buckets are labelled by the matched *route template*,
never the requested path. That gives one bucket per endpoint instead of one per adventure id,
and — the reason it isn't merely tidier — an unmatched path is entirely attacker-chosen, so
labelling by it would let anyone mint rows.
--- ---
# Part 4 — The web plumbing, briefly # Part 4 — The web plumbing, briefly
+25 -2
View File
@@ -1,5 +1,5 @@
import { useEffect, useState } from 'react' import { useEffect, useRef, useState } from 'react'
import { NavLink, Outlet } from 'react-router-dom' import { NavLink, Outlet, useLocation } from 'react-router-dom'
import { api } from './api' import { api } from './api'
import { AuthModal, ToastHost } from './components' import { AuthModal, ToastHost } from './components'
import Embers from './Embers.jsx' import Embers from './Embers.jsx'
@@ -9,11 +9,28 @@ export default function App() {
const [me, setMe] = useState(null) const [me, setMe] = useState(null)
const [authMode, setAuthMode] = useState(null) // 'register' | 'login' | null const [authMode, setAuthMode] = useState(null) // 'register' | 'login' | null
const [navOpen, setNavOpen] = useState(false) // mobile hamburger menu const [navOpen, setNavOpen] = useState(false) // mobile hamburger menu
const location = useLocation()
const lastPath = useRef(null)
useEffect(() => { useEffect(() => {
api.getMe().then(setMe).catch(() => {}) api.getMe().then(setMe).catch(() => {})
}, []) }, [])
// One pageview per route the reader actually lands on. Guarded on the path
// rather than fired on every render: StrictMode runs effects twice in dev,
// and a re-render for unrelated state is not a new page.
useEffect(() => {
if (lastPath.current === location.pathname) return
const first = lastPath.current === null
lastPath.current = location.pathname
// document.referrer survives client-side navigation, so it is only honest
// on the first view — after that this was our own page, not a referral.
api.trackPageview(location.pathname, {
referrer: first ? document.referrer : '',
first,
})
}, [location.pathname])
const onAuthed = (newMe, mode) => { const onAuthed = (newMe, mode) => {
setAuthMode(null) setAuthMode(null)
if (mode === 'login') { if (mode === 'login') {
@@ -64,6 +81,12 @@ export default function App() {
AI Chat AI Chat
</NavLink> </NavLink>
)} )}
{/* Owner only: the site's own traffic, on its own allowlist. */}
{me?.analytics && (
<NavLink to="/analytics" className={({ isActive }) => `navlink${isActive ? ' active' : ''}`}>
Visitors
</NavLink>
)}
{me?.multi_user && ( {me?.multi_user && (
<div className="nav-account"> <div className="nav-account">
{me.is_guest ? ( {me.is_guest ? (
+22
View File
@@ -61,6 +61,28 @@ async function streamSSE(path, payload, onEvent, signal, isRetry = false) {
} }
export const api = { export const api = {
// Analytics. The beacon is deliberately not a `request()`: it must never
// retry, never bootstrap a session, and never surface an error — a counter
// that can interrupt the app it is counting is worse than no counter.
trackPageview: (path, { referrer = '', first = false } = {}) => {
try {
fetch('/api/analytics/collect', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path, referrer, first }),
keepalive: true,
}).catch(() => {})
} catch { /* no beacon, no problem */ }
},
getAnalytics: (days) => request(`/analytics/summary?days=${days}`),
getAccessLog: ({ beforeId, kind, q, limit = 50 } = {}) => {
const params = new URLSearchParams({ limit })
if (beforeId != null) params.set('before_id', beforeId)
if (kind) params.set('kind', kind)
if (q) params.set('q', q)
return request(`/analytics/access?${params}`)
},
// Auth (Phase 8 — no-ops in local mode beyond getMe) // Auth (Phase 8 — no-ops in local mode beyond getMe)
getMe: () => request('/auth/me'), getMe: () => request('/auth/me'),
register: (email, password) => register: (email, password) =>
+309
View File
@@ -13,6 +13,13 @@
--accent-glow: rgba(212, 169, 78, 0.25); --accent-glow: rgba(212, 169, 78, 0.25);
--danger: #d06565; --danger: #d06565;
--player: #9fc7d1; --player: #9fc7d1;
/* Chart hues (analytics dashboard). Deeper and more saturated than --accent
and --player, which are tuned for text and borders and turn muddy once
they are a 10px bar: these are checked against --bg-panel for lightness,
chroma, contrast and colour-blind separation as a set. */
--chart-1: #b58a30;
--chart-2: #3d8ac4;
--chart-3: #a8608f;
--font-display: 'Cinzel', Georgia, serif; --font-display: 'Cinzel', Georgia, serif;
--font-story: 'Crimson Pro', Georgia, 'Times New Roman', serif; --font-story: 'Crimson Pro', Georgia, 'Times New Roman', serif;
--font-ui: 'Inter', 'Segoe UI', system-ui, sans-serif; --font-ui: 'Inter', 'Segoe UI', system-ui, sans-serif;
@@ -2333,6 +2340,293 @@ button.primary.compact { padding: 3px 12px; font-size: 0.76rem; margin-left: aut
.card.tome:hover { transform: none; } .card.tome:hover { transform: none; }
} }
/* ---------- Visit analytics (owner dashboard) ---------- */
/* Wider than the reading pages: this one is a grid of small figures, not
prose, and 960px puts two charts and eight tiles into a column. */
.an-page { max-width: 1180px; }
.an-ranges { display: flex; gap: 6px; }
/* Four across, not auto-fit: there are eight tiles, and letting them flow
leaves a single orphan on the second row at most widths. */
.an-tiles {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 12px;
margin-bottom: 18px;
}
@media (max-width: 900px) {
.an-tiles { grid-template-columns: repeat(2, minmax(0, 1fr)); }
}
.an-tile {
background: var(--bg-panel);
border: 1px solid var(--border);
border-radius: 10px;
padding: 14px 16px;
}
.an-tile-value {
font-family: var(--font-display);
font-size: 1.7rem;
line-height: 1.1;
color: var(--accent-bright);
}
.an-tile-label { font-size: 0.82rem; color: var(--text); margin-top: 4px; }
.an-tile-hint { font-size: 0.72rem; color: var(--text-dim); margin-top: 2px; }
.an-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(340px, 1fr));
gap: 16px;
margin-bottom: 16px;
}
.an-card {
position: relative;
background: var(--bg-panel);
border: 1px solid var(--border);
border-radius: 12px;
padding: 16px 18px 14px;
margin-bottom: 16px;
}
.an-grid .an-card { margin-bottom: 0; }
.an-card-head {
display: flex;
align-items: baseline;
justify-content: space-between;
gap: 12px;
margin-bottom: 14px;
}
.an-card-head h2 {
margin: 0;
font-family: var(--font-display);
font-size: 0.95rem;
letter-spacing: 0.06em;
color: var(--accent);
}
.an-note { font-size: 0.72rem; color: var(--text-dim); }
.an-empty { color: var(--text-dim); font-style: italic; padding: 18px 0; text-align: center; }
.an-legend { display: flex; gap: 12px; font-size: 0.72rem; color: var(--text-dim); }
.an-legend-item { display: inline-flex; align-items: center; gap: 5px; }
.an-swatch { width: 9px; height: 9px; border-radius: 2px; display: inline-block; flex: none; }
/* ----- Day columns ----- */
.an-plot { position: relative; height: 170px; margin-top: 12px; }
.an-gridline {
position: absolute;
left: 0;
right: 0;
border-top: 1px dashed var(--border);
pointer-events: none;
}
.an-gridline span {
position: absolute;
right: 0;
top: -0.62em;
font-size: 0.68rem;
color: var(--text-dim);
background: var(--bg-panel);
padding: 0 4px;
/* The dashed rule belongs behind the bars; its value does not. Columns are
positioned, so without this the tallest bar swallows the label. */
z-index: 3;
}
.an-columns { display: flex; align-items: flex-end; gap: 2px; height: 100%; }
.an-column {
position: relative;
flex: 1 1 0;
min-width: 0;
height: 100%;
display: flex;
align-items: flex-end;
border-radius: 4px 4px 0 0;
outline: none;
}
.an-column.hot { background: rgba(212, 169, 78, 0.07); }
.an-stack {
width: 100%;
/* Capped so a 7-day range draws bars and not slabs; the column around it
stays full width, so the hover target does not shrink with the mark. */
max-width: 44px;
margin: 0 auto;
height: 100%;
display: flex;
flex-direction: column;
justify-content: flex-end;
/* A 2px gap of surface between stacked segments: the boundary reads as a
boundary without a border, which would eat a thin bar entirely. */
gap: 2px;
}
.an-bar { width: 100%; transition: filter 0.15s; }
.an-stack > .an-bar:first-child { border-radius: 4px 4px 0 0; }
.an-column.hot .an-bar { filter: brightness(1.2); }
.an-tip {
position: absolute;
bottom: calc(100% + 8px);
left: 50%;
transform: translateX(-50%);
display: flex;
flex-direction: column;
gap: 3px;
padding: 8px 10px;
background: var(--bg-panel);
border: 1px solid var(--border-bright);
border-radius: 8px;
box-shadow: 0 10px 26px rgba(0, 0, 0, 0.55);
font-size: 0.74rem;
white-space: nowrap;
pointer-events: none;
z-index: 4;
}
.an-tip strong { font-weight: 600; color: var(--accent-bright); }
.an-tip span { display: inline-flex; align-items: center; gap: 6px; color: var(--text-dim); }
.an-axis { display: flex; gap: 2px; margin-top: 7px; }
.an-axis span {
flex: 1 1 0;
min-width: 0;
font-size: 0.68rem;
color: var(--text-dim);
text-align: center;
white-space: nowrap;
}
.an-axis span:first-child { text-align: left; }
.an-axis span:last-child { text-align: right; }
.an-overlay-empty {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
color: var(--text-dim);
font-style: italic;
font-size: 0.85rem;
pointer-events: none;
}
/* ----- Funnel ----- */
.an-funnel { display: flex; flex-direction: column; gap: 10px; }
.an-funnel-row {
display: grid;
grid-template-columns: 168px 1fr 96px;
align-items: center;
gap: 12px;
}
.an-funnel-label { font-size: 0.84rem; }
.an-funnel-track { height: 14px; background: var(--bg-input); border-radius: 7px; overflow: hidden; }
.an-funnel-bar {
height: 100%;
min-width: 2px;
background: var(--chart-1);
border-radius: 7px;
transition: width 0.4s ease;
}
.an-funnel-value {
text-align: right;
font-size: 0.86rem;
display: flex;
justify-content: flex-end;
align-items: baseline;
gap: 8px;
}
.an-funnel-share { font-size: 0.72rem; color: var(--text-dim); }
/* ----- Ranked lists ----- */
.an-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 3px; }
.an-list li {
position: relative;
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
padding: 7px 10px;
border-radius: 6px;
font-size: 0.84rem;
}
/* The bar sits behind the row rather than beside it, so a long label keeps the
full width and the magnitude is still visible at a glance. */
.an-list-fill {
position: absolute;
top: 0;
bottom: 0;
left: 0;
background: rgba(181, 138, 48, 0.2);
border-radius: 6px;
}
.an-list-label {
position: relative;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.an-list-value { position: relative; color: var(--text-dim); flex: none; }
/* ----- Tabs, and the access log table ----- */
.an-tabs { display: flex; gap: 4px; margin-bottom: 16px; border-bottom: 1px solid var(--border); }
.an-tabs button {
background: none;
border: none;
border-bottom: 2px solid transparent;
border-radius: 0;
padding: 8px 14px;
color: var(--text-dim);
font-family: var(--font-display);
font-size: 0.82rem;
letter-spacing: 0.06em;
cursor: pointer;
}
.an-tabs button:hover { color: var(--text); }
.an-tabs button.active { color: var(--accent-bright); border-bottom-color: var(--accent); }
.an-log-head { flex-wrap: wrap; row-gap: 10px; }
.an-log-search { max-width: 260px; }
/* The table is the one thing here with a minimum width — six columns of real
data don't compress. It scrolls inside its own box so the page never does. */
.an-table-scroll { overflow-x: auto; }
.an-table { width: 100%; min-width: 620px; border-collapse: collapse; font-size: 0.82rem; }
.an-table th {
text-align: left;
font-weight: 500;
font-size: 0.72rem;
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--text-dim);
padding: 0 10px 8px;
border-bottom: 1px solid var(--border);
white-space: nowrap;
}
.an-table td { padding: 8px 10px; border-bottom: 1px solid var(--border); white-space: nowrap; }
.an-table tr:last-child td { border-bottom: none; }
.an-table tbody tr:hover { background: rgba(212, 169, 78, 0.05); }
/* A failed attempt is the row you are scanning for; it gets the only colour in
the table, on a border rather than the text, which stays readable. */
.an-table tr.failed td:first-child { box-shadow: inset 2px 0 var(--danger); }
.an-table tr.failed td:nth-child(3) { color: var(--danger); }
.an-cell-dim { color: var(--text-dim); }
.an-cell-mono { font-family: ui-monospace, 'Cascadia Code', Consolas, monospace; }
.an-tag {
margin-left: 8px;
padding: 1px 6px;
border: 1px solid var(--border-bright);
border-radius: 10px;
font-size: 0.66rem;
color: var(--text-dim);
}
.an-more { display: flex; justify-content: center; padding-top: 14px; }
.an-footnote {
color: var(--text-dim);
font-size: 0.74rem;
line-height: 1.6;
margin: 22px 0 0;
max-width: 70ch;
}
/* ============================================================ /* ============================================================
Mobile / narrow screens (≤ 720px) Mobile / narrow screens (≤ 720px)
The desktop Play screen is a horizontal row of up to four columns The desktop Play screen is a horizontal row of up to four columns
@@ -2536,4 +2830,19 @@ button.primary.compact { padding: 3px 12px; font-size: 0.76rem; margin-left: aut
.branch-map-canvas { padding: 4px 6px 8px; } .branch-map-canvas { padding: 4px 6px 8px; }
.branch-map-detail { padding: 12px 16px 16px; } .branch-map-detail { padding: 12px 16px 16px; }
.branch-map-hint { padding: 0 16px 10px; } .branch-map-hint { padding: 0 16px 10px; }
/* ---------- Analytics dashboard ---------- */
.an-page .page-header { flex-direction: column; align-items: flex-start; gap: 10px; }
.an-tiles { gap: 8px; }
.an-tile { padding: 10px 12px; }
.an-tile-value { font-size: 1.35rem; }
.an-grid { grid-template-columns: 1fr; }
.an-plot { height: 140px; }
.an-tabs button { padding: 8px 10px; font-size: 0.78rem; }
.an-log-search { max-width: none; width: 100%; }
/* Three columns don't fit; the label goes above its own bar instead. */
.an-funnel-row { grid-template-columns: 1fr auto; gap: 4px 10px; }
.an-funnel-track { grid-column: 1; }
.an-funnel-label { grid-column: 1 / -1; }
.an-funnel-value { grid-column: 2; }
} }
+3
View File
@@ -11,6 +11,7 @@ import Scripts from './pages/Scripts.jsx'
import ScriptEditor from './pages/ScriptEditor.jsx' import ScriptEditor from './pages/ScriptEditor.jsx'
import Settings from './pages/Settings.jsx' import Settings from './pages/Settings.jsx'
import Chat from './pages/Chat.jsx' import Chat from './pages/Chat.jsx'
import Analytics from './pages/Analytics.jsx'
import { trackKeyboardInset } from './keyboard.js' import { trackKeyboardInset } from './keyboard.js'
import './index.css' import './index.css'
@@ -31,6 +32,8 @@ const router = createBrowserRouter([
{ path: 'settings', element: <Settings /> }, { path: 'settings', element: <Settings /> },
// Power users only — the page redirects home and the API 404s otherwise. // Power users only — the page redirects home and the API 404s otherwise.
{ path: 'chat', element: <Chat /> }, { path: 'chat', element: <Chat /> },
// Owner only, by a separate allowlist; same redirect-and-404 treatment.
{ path: 'analytics', element: <Analytics /> },
], ],
}, },
]) ])
+476
View File
@@ -0,0 +1,476 @@
/* Visit analytics — the owner's view of who came by and what they did.
Everything on this page arrives in a single aggregate response (see
backend/app/analytics.py), so changing the range is one small request, not a
scan of anything. The page is hidden from everyone else: the nav link is
gated on `me.analytics`, this component bounces, and the API 404s.
Charts are plain HTML — a flex row of columns, a row of bars — rather than
SVG or a charting library. At this size that is less code, responsive for
free, and keeps the CSP as tight as it is. */
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { useNavigate, useOutletContext } from 'react-router-dom'
import { api } from '../api'
const RANGES = [
{ days: 7, label: '7 days' },
{ days: 30, label: '30 days' },
{ days: 90, label: '90 days' },
]
const nf = new Intl.NumberFormat()
// Weekday + day for a short range, day + month for a long one: a 90-day axis
// has no room for "Mon".
function dayLabel(iso, days) {
const date = new Date(`${iso}T00:00:00Z`)
const opts = days <= 14
? { weekday: 'short', timeZone: 'UTC' }
: { day: 'numeric', month: 'short', timeZone: 'UTC' }
return date.toLocaleDateString(undefined, opts)
}
function fullDate(iso) {
return new Date(`${iso}T00:00:00Z`)
.toLocaleDateString(undefined, { dateStyle: 'medium', timeZone: 'UTC' })
}
/* ---------- Pieces ---------- */
function StatTile({ label, value, hint }) {
return (
<div className="an-tile" title={hint || undefined}>
<div className="an-tile-value">{nf.format(value ?? 0)}</div>
<div className="an-tile-label">{label}</div>
{hint && <div className="an-tile-hint">{hint}</div>}
</div>
)
}
/* A day-by-day column chart. `series` names the stacked segments bottom-up;
one segment means one plain bar and no legend, since the title already says
what it is. */
function DayChart({ title, data, series, days }) {
const [hover, setHover] = useState(null)
const max = Math.max(1, ...data.map((d) => series.reduce((sum, s) => sum + (d[s.key] || 0), 0)))
// Only ever three x labels. Every column labelled is unreadable at 90 days
// and redundant at 7 — the tooltip carries the exact date either way.
const ticks = new Set([0, Math.floor((data.length - 1) / 2), data.length - 1])
const empty = data.every((d) => series.every((s) => !d[s.key]))
return (
<section className="an-card an-chart">
<header className="an-card-head">
<h2>{title}</h2>
{series.length > 1 && (
<div className="an-legend">
{[...series].reverse().map((s) => (
<span key={s.key} className="an-legend-item">
<i className="an-swatch" style={{ background: s.color }} />
{s.label}
</span>
))}
</div>
)}
</header>
<div className="an-plot" onMouseLeave={() => setHover(null)}>
<div className="an-gridline" style={{ bottom: '100%' }}><span>{nf.format(max)}</span></div>
<div className="an-gridline" style={{ bottom: '50%' }}><span>{nf.format(Math.round(max / 2))}</span></div>
<div className="an-columns">
{data.map((point, i) => {
const total = series.reduce((sum, s) => sum + (point[s.key] || 0), 0)
return (
<div
key={point.day}
className={`an-column${hover === i ? ' hot' : ''}`}
onMouseEnter={() => setHover(i)}
onFocus={() => setHover(i)}
tabIndex={0}
aria-label={`${fullDate(point.day)}: ${total}`}
>
<div className="an-stack">
{[...series].reverse().map((s) => (
(point[s.key] || 0) > 0 && (
<div
key={s.key}
className="an-bar"
style={{
height: `${((point[s.key] || 0) / max) * 100}%`,
background: s.color,
}}
/>
)
))}
</div>
{hover === i && (
<div className="an-tip">
<strong>{fullDate(point.day)}</strong>
{series.map((s) => (
<span key={s.key}>
<i className="an-swatch" style={{ background: s.color }} />
{s.label}: {nf.format(point[s.key] || 0)}
</span>
))}
</div>
)}
</div>
)
})}
</div>
</div>
<div className="an-axis">
{data.map((point, i) => (
<span key={point.day}>{ticks.has(i) ? dayLabel(point.day, days) : ''}</span>
))}
</div>
{empty && <div className="an-overlay-empty">Nothing recorded in this range</div>}
</section>
)
}
/* The funnel. Each step's bar is drawn against the first step, so the shape of
the drop-off is the picture; the percentage beside it is of the step above,
which is the number you act on. */
function Funnel({ steps }) {
const top = steps[0]?.count || 0
return (
<section className="an-card">
<header className="an-card-head">
<h2>Where visitors get to</h2>
<span className="an-note">people, counted once each</span>
</header>
{top === 0 ? (
<div className="an-empty">No visitors in this range.</div>
) : (
<div className="an-funnel">
{steps.map((step, i) => {
const previous = i === 0 ? step.count : steps[i - 1].count
const share = previous ? Math.round((step.count / previous) * 100) : 0
return (
<div key={step.step} className="an-funnel-row">
<div className="an-funnel-label">{step.step}</div>
<div className="an-funnel-track">
<div
className="an-funnel-bar"
style={{ width: `${top ? (step.count / top) * 100 : 0}%` }}
/>
</div>
<div className="an-funnel-value">
{nf.format(step.count)}
{i > 0 && <span className="an-funnel-share">{share}%</span>}
</div>
</div>
)
})}
</div>
)}
</section>
)
}
function TopList({ title, rows, note, empty = 'Nothing yet' }) {
const max = Math.max(1, ...rows.map((r) => r.hits))
return (
<section className="an-card">
<header className="an-card-head">
<h2>{title}</h2>
{note && <span className="an-note">{note}</span>}
</header>
{rows.length === 0 ? (
<div className="an-empty">{empty}</div>
) : (
<ul className="an-list">
{rows.map((row) => (
<li key={row.label}>
{/* The bar is the row's own background, so a long label stays
readable on top of it instead of being squeezed beside it. */}
<span className="an-list-fill" style={{ width: `${(row.hits / max) * 100}%` }} />
<span className="an-list-label" title={row.label}>{row.label}</span>
<span className="an-list-value">{nf.format(row.hits)}</span>
</li>
))}
</ul>
)}
</section>
)
}
/* ---------- Access log ---------- */
const KINDS = [
{ key: '', label: 'Everything' },
{ key: 'session', label: 'Sessions' },
{ key: 'login', label: 'Sign-ins' },
{ key: 'register', label: 'Registrations' },
{ key: 'login_failed', label: 'Failed' },
]
const KIND_LABEL = {
session: 'Session',
login: 'Signed in',
register: 'Registered',
login_failed: 'Failed sign-in',
}
function when(iso) {
const date = new Date(iso.endsWith('Z') ? iso : `${iso}Z`)
return date.toLocaleString(undefined, { dateStyle: 'medium', timeStyle: 'short' })
}
function AccessLog() {
const [kind, setKind] = useState('')
const [search, setSearch] = useState('')
const [query, setQuery] = useState('')
const [page, setPage] = useState(null) // { events, has_more }
const [error, setError] = useState(null)
const [busy, setBusy] = useState(false)
// Typing shouldn't fire a request per keystroke against a table scan.
useEffect(() => {
const timer = setTimeout(() => setQuery(search.trim()), 350)
return () => clearTimeout(timer)
}, [search])
useEffect(() => {
let live = true
setPage(null)
api.getAccessLog({ kind, q: query }).then(
(result) => { if (live) { setPage(result); setError(null) } },
(err) => { if (live) setError(err.message) },
)
return () => { live = false }
}, [kind, query])
const loadMore = useCallback(async () => {
if (!page?.events.length || busy) return
setBusy(true)
try {
// Anchored on the oldest row already on screen, so rows arriving while
// this is open can't shift the next page.
const next = await api.getAccessLog({
kind, q: query, beforeId: page.events[page.events.length - 1].id,
})
setPage({ events: [...page.events, ...next.events], has_more: next.has_more })
} catch (err) {
setError(err.message)
} finally {
setBusy(false)
}
}, [page, kind, query, busy])
return (
<section className="an-card">
<header className="an-card-head an-log-head">
<div className="an-ranges">
{KINDS.map((option) => (
<button
key={option.key}
className={kind === option.key ? 'primary' : ''}
onClick={() => setKind(option.key)}
>
{option.label}
</button>
))}
</div>
<input
type="text"
className="search-input an-log-search"
placeholder="Search email, IP, country…"
value={search}
onChange={(event) => setSearch(event.target.value)}
/>
</header>
{error && <div className="an-empty">Couldn’t load the log: {error}</div>}
{!page && !error && <div className="an-empty">Reading…</div>}
{page && (page.events.length === 0 ? (
<div className="an-empty">
{query || kind ? 'Nothing matches that.' : 'Nothing logged yet.'}
</div>
) : (
<>
<div className="an-table-scroll">
<table className="an-table">
<thead>
<tr>
<th>When</th><th>Who</th><th>Event</th>
<th>IP</th><th>Country</th><th>Device</th>
</tr>
</thead>
<tbody>
{page.events.map((event) => (
<tr key={event.id} className={event.kind === 'login_failed' ? 'failed' : undefined}>
<td className="an-cell-dim">{when(event.at)}</td>
<td>
{event.who}
{event.is_guest && <span className="an-tag">guest</span>}
</td>
<td>{KIND_LABEL[event.kind] || event.kind}</td>
<td className="an-cell-mono">{event.ip || '—'}</td>
<td>{event.country || '—'}</td>
{/* The full user-agent is a wall of text; it lives on the
hover instead of in a column that would push the rest
of the table off screen. */}
<td className="an-cell-dim" title={event.user_agent || undefined}>
{event.device || '—'}
</td>
</tr>
))}
</tbody>
</table>
</div>
{page.has_more && (
<div className="an-more">
<button onClick={loadMore} disabled={busy}>
{busy ? 'Loading…' : 'Load older'}
</button>
</div>
)}
</>
))}
</section>
)
}
/* ---------- Page ---------- */
export default function Analytics() {
const { me } = useOutletContext() ?? {}
const navigate = useNavigate()
const [tab, setTab] = useState('overview')
const [days, setDays] = useState(30)
const [data, setData] = useState(null)
const [error, setError] = useState(null)
const loaded = useRef(false)
// me is null until /auth/me resolves; only bounce once we know.
useEffect(() => {
if (me && !me.analytics) navigate('/', { replace: true })
}, [me, navigate])
useEffect(() => {
if (tab !== 'overview') return undefined
let live = true
api.getAnalytics(days).then(
(result) => { if (live) { setData(result); setError(null); loaded.current = true } },
(err) => { if (live) setError(err.message) },
)
return () => { live = false }
}, [days, tab])
const totals = data?.totals ?? {}
const visitorSeries = useMemo(() => ([
{ key: 'returning', label: 'Returning', color: 'var(--chart-2)' },
{ key: 'new', label: 'New', color: 'var(--chart-1)' },
]), [])
// The server sends new-vs-total; the chart stacks, so it wants the remainder.
const visitorDays = useMemo(
() => (data?.series ?? []).map((d) => ({ ...d, returning: d.visitors - d.new })),
[data],
)
if (me && !me.analytics) return null
return (
<div className="page an-page">
<div className="page-header">
<h1>Visitors</h1>
{tab === 'overview' && (
<div className="an-ranges">
{RANGES.map((range) => (
<button
key={range.days}
className={days === range.days ? 'primary' : ''}
onClick={() => setDays(range.days)}
>
{range.label}
</button>
))}
</div>
)}
</div>
<div className="an-tabs">
<button
className={tab === 'overview' ? 'active' : ''}
onClick={() => setTab('overview')}
>
Overview
</button>
<button
className={tab === 'access' ? 'active' : ''}
onClick={() => setTab('access')}
>
Access log
</button>
</div>
{tab === 'access' && <AccessLog />}
{tab === 'overview' && error && (
<div className="an-empty">Couldn’t load analytics: {error}</div>
)}
{tab === 'overview' && !data && !error && <div className="an-empty">Counting…</div>}
{tab === 'overview' && data && (
<>
<div className="an-tiles">
<StatTile label="Visitors" value={totals.visitors}
hint={`${nf.format(totals.new_visitors || 0)} first-time`} />
<StatTile label="Visits" value={totals.visits}
hint={`${totals.pages_per_visit || 0} pages each`} />
<StatTile label="Pageviews" value={totals.pageviews} />
<StatTile label="Adventures started" value={totals.adventures} />
<StatTile label="Turns played" value={totals.turns}
hint={`${totals.turns_per_visit || 0} per visit`} />
<StatTile label="Sign-ups" value={totals.signups} />
<StatTile label="Demo-key turns" value={totals.demo_turns}
hint="billed to the shared key" />
<StatTile label="Failed turns" value={totals.turn_errors}
hint={`${nf.format(totals.errors || 0)} API errors`} />
</div>
<div className="an-grid">
<DayChart
title="Visitors per day"
data={visitorDays}
series={visitorSeries}
days={days}
/>
<DayChart
title="Turns played per day"
data={data.series}
series={[{ key: 'turns', label: 'Turns', color: 'var(--chart-3)' }]}
days={days}
/>
</div>
<Funnel steps={data.funnel} />
<div className="an-grid">
<TopList title="Pages" rows={data.pages} />
<TopList
title="Where they came from"
rows={data.referrers}
note="per visit"
empty="No referrals yet — every visit was typed or bookmarked."
/>
<TopList title="Scenarios started" rows={data.scenarios}
note="shared scenarios only" empty="No adventures started yet." />
<TopList title="Countries" rows={data.countries} />
<TopList title="Devices" rows={data.devices} />
<TopList title="API errors" rows={data.errors} empty="None — clean run." />
</div>
<p className="an-footnote">
{fullDate(data.since)} – {fullDate(data.until)}, UTC. Your own visits aren’t
counted here. These totals are anonymous — visitors are counted as one-way
hashes, and nothing on this tab can be traced back to a player or their
stories. The access log tab is the separate, identifying record.
</p>
</>
)}
</div>
)
}
+69 -1
View File
@@ -3,7 +3,7 @@
Read this first when picking the project back up. Updated at the end of a working Read this first when picking the project back up. Updated at the end of a working
session; the per-phase plan files hold the detail, this holds the thread. session; the per-phase plan files hold the detail, this holds the thread.
**Last updated: 2026-08-20.** **Last updated: 2026-08-21.**
--- ---
@@ -195,6 +195,74 @@ an automated version; see SP7's entry in `plan/14`.
--- ---
## What happened on 2026-08-21 — visit analytics
The hosted demo can now answer whether anyone is using it. `/analytics` is a dashboard —
visitors, pages, referrers, countries, devices, which shared scenarios get played, turns and
demo-key spend, API and turn errors, and a funnel from *visited* to *played a turn* to
*signed up*. **Built, green (497 tests), driven by hand against a synthetic 90-day fixture.
Not committed, not deployed.**
**It is gated on its own allowlist, `AIDND_ANALYTICS_EMAILS`** — not `AIDND_POWER_USERS`.
An unmetered tester is not automatically someone who sees the traffic numbers. The route
404s and the nav link is absent for everyone else, same treatment as AI Chat. Set the var in
the Render dashboard or the page is invisible to everybody, including you.
**The counters are anonymous; the access log beside them is not, on purpose.** A visitor in
`analytics_daily`/`analytics_visitor_days` is `HMAC(secret, "visitor:<user id>")` truncated
to 32 chars — one-way, so those two tables cannot be joined back to `users`, and keyed, so no
client can compute one. Story content never reaches that module. The operator's own visits
are not counted there (multi-user only — excluding them locally would leave the page
permanently empty on the machine it is developed on).
**`accesslog.py` is the identifying half, added the same week.** `access_events` records
sessions, sign-ins, registrations and failed attempts with address, email (or `Guest #n`),
country and device, read on an "Access log" tab of the same page and behind the same owner
gate. It is a separate module and a separate table so the anonymity of the counters stays a
property of the code rather than a convention. Three things it leans on: the address comes
from `limits.client_ip` — now public, and the only place that decides which hop to trust, so
a spoofed `X-Forwarded-For` cannot forge a row; `user_id` carries **no foreign key** and
`who`/`is_guest` are snapshots, so a row outlives the guest cleanup that deletes the account;
and session rows are thinned to one per day per address, since `/auth/me` runs on every page
load. Nothing is purged — that was the deliberate choice. The published docs describe the
analytics generally and do not enumerate this.
**A visit is a write and never a read.** Counts accumulate in a process-local dict and flush
every 60s as UPSERTs into `analytics_daily` — a generic `(day, metric, label) -> hits`
counter — plus one row per visitor per day in `analytics_visitor_days` for the funnel flags.
Every dashboard query is a `GROUP BY` returning tens of rows however much traffic sits
behind it. Deliberate, given §2.5: adding a feature that reads rows per request would have
undone the egress work.
**No migration was needed.** Both tables are new, and `bootstrap()` calls `create_all` on
existing databases too — the same route `branches` took in Phase 14. Nothing was appended to
`MIGRATIONS`, so `LATEST_VERSION` is still 64.
Three things worth remembering out of building it:
- **The funnel counts people, not clicks.** A player who starts six adventures is one person
who started an adventure. That is the entire reason the per-visitor-day table exists; its
flags only ever turn on, and `is_new` is settled by the first write of a visitor's first
day and never updated after.
- **A failed turn is an HTTP 200 with a bad ending.** The status-code middleware cannot see
one, so a demo whose model started refusing every request would look perfectly healthy.
`turn_error` is counted in a new `turn_error()` helper that all five SSE error paths in
`_generate_turn` now go through.
- **The tests run on SQLite; production is Neon.** A flush that raises is caught and logged,
so a dialect mistake in the UPSERTs would have been invisible until the dashboard quietly
stayed empty. `test_the_upserts_compile_for_postgres` compiles both statements against the
Postgres dialect without connecting to one.
**Still not verified: the narrow-screen layout.** The CSS follows the existing `max-width:
720px` block (single-column grids, funnel label above its bar) but `resize_window` is
ignored on a maximized Chrome, and the app sends `frame-ancestors 'none'` so it cannot be
checked in a sized iframe either. Desktop was driven by hand at 1568px.
**`docs/guide.html` is now behind `docs/GUIDE.md`,** which gained §3.6. Nothing in the repo
regenerates it.
---
## What happened on 2026-08-20 — the branch map ## What happened on 2026-08-20 — the branch map
The Branches panel gained a **⌗ See the tree** button opening a full-screen map: one The Branches panel gained a **⌗ See the tree** button opening a full-screen map: one
+7
View File
@@ -51,6 +51,13 @@ services:
- key: AIDND_POWER_USERS - key: AIDND_POWER_USERS
sync: false sync: false
# Comma-separated emails allowed to see the Visitors dashboard
# (/analytics). Separate from AIDND_POWER_USERS on purpose — an unmetered
# tester is not automatically someone who sees the traffic. Leave unset
# and the page is invisible to everyone. Set it in the dashboard.
- key: AIDND_ANALYTICS_EMAILS
sync: false
# Guest retention: one account is minted per first-time visitor, so idle # Guest retention: one account is minted per first-time visitor, so idle
# ones are collected (with their adventures) to keep the free-tier # ones are collected (with their adventures) to keep the free-tier
# Postgres from filling with abandoned demo data. Registered accounts are # Postgres from filling with abandoned demo data. Registered accounts are