diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a6bd05..147dac4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,18 @@ All notable changes to this project will be documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). +## [0.7.0] - 2026-06-28 + +### Added +- `canary` command — probes the ChatGPT/Claude web APIs for schema drift against the fields the parser actually depends on (one listing page + one conversation per provider), not the full response shape. Flags the silent-failure risks for a backup tool: a renamed retrieval-tool author bypassing the hidden-content collapse, a new `content_type`, drifted message fields, or non-empty Claude `attachments`/`files` the normalizer ignores. ERROR findings exit non-zero; WARN findings are surfaced but non-fatal. See `FUTURE.md` §10. +- `doctor` now reports a real ChatGPT token-health check ("ChatGPT token active") based on the `/api/auth/session` `error` field. ChatGPT session tokens are JWEs whose `exp` is encrypted and unreadable client-side; the previous decode path could never yield an expiry. The honest signal is `error == "RefreshAccessTokenError"` (verified live), which means the session token is dead even though `expires`/`accessToken` still echo stale values. See `FUTURE.md` §9. + +### Fixed +- `ChatGPTProvider._fetch_access_token` now checks the `/api/auth/session` `error` field and fails fast with a clear "refresh your token" message. Previously it returned the stale `accessToken` present on a dead session, producing a confusing downstream 401 instead of an actionable auth error. + +### Removed +- `auth --from-browser` and the `browser-cookie3` dependency (`src/browser_tokens.py`). Browser cookie auto-extraction is not viable: modern Chromium App-Bound Encryption (Chrome 127+/current Brave) keys cookies off a SYSTEM-level layer that can't be decrypted off disk without Administrator rights and AV-flagged SYSTEM impersonation, and fails on Brave specifically (symptom: `Unable to get key for cookie decryption`). Auth is manual (DevTools wizard) only. See `FUTURE.md` §2 for the full rationale. + ## [0.6.0] - 2026-06-12 First tagged release. The project was developed through several internal diff --git a/FUTURE.md b/FUTURE.md index d86c8b2..7f09d93 100644 --- a/FUTURE.md +++ b/FUTURE.md @@ -17,14 +17,18 @@ of these additions straightforward. Priorities reflect the tool's primary purpose — a trustworthy backup so that conversation data is not lost if a provider account is ever closed — plus the -day-to-day friction of weekly ChatGPT token refresh, and the long-term goal -of running headless as a StartOS service. +day-to-day friction of the weekly ChatGPT token refresh. The tool stays a +local, manually-run CLI; the headless/StartOS direction was dropped +2026-06-28 (see #7 and #8), which also retires the token-freshness problem +(manual refresh is sufficient). **Now (in order):** 1. ~~Collapse tool retrieval dumps & hidden context~~ — **shipped in v0.6.0** - (full-archive re-export still pending; see entry below) -2. ~~Brave cookie auto-extraction~~ — **shipped in v0.6.0** (note: runs on - whichever machine hosts the browser — Brave is not on this Linux box) + (full-archive `export --force` re-export completed 2026-06-13) +2. ~~Brave cookie auto-extraction~~ — **shipped in v0.6.0, removed afterward.** + Not viable: modern Chromium App-Bound Encryption (Chrome 127+/current Brave) + needs admin + SYSTEM impersonation that AV flags as credential theft, and + fails on Brave specifically. Auth is manual (DevTools) — see entry below. 3. ~~Claude Code session provider~~ — **shipped in v0.6.0** 4. ~~Archive hygiene — `prune` + `doctor` integrity check~~ — **shipped in v0.6.0** @@ -32,12 +36,23 @@ of running headless as a StartOS service. 5. ~~Binary content downloads~~ — **shipped in v0.6.0** 6. ~~Per-session download limiter + polite pacing~~ — **shipped in v0.6.0** -7. Scheduled / watch mode → StartOS service packaging (long-term destination) +7. ~~Scheduled / watch mode~~ — **dropped 2026-06-28**; the tool stays a + manually-run CLI, so no in-app polling loop is needed +8. ~~StartOS service packaging~~ — **dropped 2026-06-28**; the local CLI is + sufficient (source convos live in the cloud and can be re-downloaded; + Joplin already syncs encrypted to an offsite S3 provider). Dropping this + also retires the headless token-freshness problem — manual weekly refresh + is fine. + +**Active:** +9. ~~Surface remaining token validity on `doctor`~~ — **IMPLEMENTED 2026-06-28** + via the `/api/auth/session` `error` field (not `expires`). See §9. +10. ~~Provider API-drift detection~~ — **IMPLEMENTED 2026-06-28** as the + `canary` command. See §10. **Deprioritized** (entries kept at the bottom of this file; revisit on demand): `--force` flags, per-conversation cache reset, official export-ZIP -fallback, o1/o3 reasoning reclassification, Obsidian output, token expiry -notifications (largely superseded by cookie auto-extraction), search command. +fallback, o1/o3 reasoning reclassification, Obsidian output, search command. Additional web providers (Gemini/Grok/Perplexity) are explicitly out of scope — no significant usage to archive. @@ -50,8 +65,8 @@ from live recon: the retrieval dumps are NOT flagged `is_visually_hidden_from_conversation` — `author.name == "file_search"` is the discriminator (the hidden flag only marks Custom Instructions and small system stubs). Verified live: worst files shrink 93% (524KB → 36KB). -Remaining step: full-archive re-export (`cache --clear` + `export`) + -Joplin re-sync, at the user's chosen time. +Full-archive re-export (the `export --force` campaign) + Joplin re-sync +completed 2026-06-13. **Problem (measured 2026-06-12 against a full fresh export):** 45% of the entire 11.2MB archive (260 files) is tool-role messages; 29 files are @@ -89,37 +104,38 @@ file contains zero hidden-context blocks. Re-export workflow after shipping: `cache --clear` + `export` (same as the v0.4.0 migration). -## 2. Brave Cookie Auto-Extraction — SHIPPED v0.6.0 +## 2. Brave Cookie Auto-Extraction — REMOVED (not viable) -**Implemented 2026-06-12**: `auth --from-browser [browser]` -(`src/browser_tokens.py`, browser-cookie3 dependency, defaults to brave; -chrome/chromium/edge/firefox also supported). Extracted tokens are validated -against the live API before `.env` is touched. **Discovered during -implementation: Brave is not installed on this machine** — the logged-in -browser lives elsewhere, so the flag only helps when the exporter runs on -that machine (verified the graceful-failure path here; the happy path is -covered by mocked tests). This strengthens the case for the StartOS -token-push mechanism (entry 8). +**Shipped v0.6.0 (2026-06-12), removed 2026-06-27.** `auth --from-browser` +plus `src/browser_tokens.py` and the `browser-cookie3` dependency are gone. +Auth is manual (DevTools) only. Do not re-attempt without a fundamentally +different mechanism (see below). -Pull session tokens directly from the Brave browser profile instead of the -weekly manual DevTools dance. `auth` gains an "extract from browser" path -(with the manual flow kept as fallback); a later iteration could let -`doctor` or a 401 handler suggest/perform a re-extract automatically. +**Why it doesn't work.** Modern Chromium browsers encrypt cookies on Windows +with **App-Bound Encryption** (Chrome 127+, July 2024; current Brave). +Cookies are written with a `v20` prefix and keyed off a secret wrapped in a +**SYSTEM-level** DPAPI layer plus app validation. `browser-cookie3` only +knows the legacy `v10`/DPAPI key, so its AES-GCM MAC check fails — the exact +symptom hit in the field: -- Brave on Linux follows the Chromium pattern: cookies SQLite at - `~/.config/BraveSoftware/Brave-Browser/Default/Cookies`, values - AES-128-CBC encrypted with a key derived from the OS keyring secret - ("Brave Safe Storage" via SecretService/kwallet, or the `peanuts` v10 - fallback). Well-trodden territory — evaluate a small dependency - (`browser_cookie3`-style) vs. a focused in-repo implementation. -- Extract `__Secure-next-auth.session-token.0` / `.1` from `chatgpt.com` - and `sessionKey` from `claude.ai`; write to `.env` through the existing - auth-wizard write path (tokens never echoed). -- Brave must be the user's logged-in browser (it is); detect a locked / - unreadable cookie DB (Brave running with the DB exclusively locked) and - fall back to the manual flow with a clear message. -- Does **not** solve headless/StartOS — no browser on the server. That - needs a token-push mechanism, tracked under the StartOS entry. +``` +ChatGPT: Could not read brave cookies for chatgpt.com: Unable to get key for cookie decryption. +``` + +Decrypting `v20` at all requires unwrapping the SYSTEM layer, which means +running as SYSTEM (e.g. a PsExec-style service) — i.e. **Administrator +rights** and behavior that AV/EDR flags as infostealer activity. The one +maintained Python option (`rookiepy`) needs admin from Chrome v130+, was +**archived 2026-06-07**, and has an unresolved bug where **Brave returns 0 +cookies** even after the key is retrieved. ABE is *designed* to stop exactly +this, so no off-disk reader is a reliable, non-invasive fit. + +**If ever revisited:** the only non-admin path is Chrome Remote Debugging +(launch the browser with `--remote-debugging-port`, read cookies via +`Network.getAllCookies` — the running browser decrypts for you). Heavier and +intrusive; not worth it for a weekly token refresh that takes 30 seconds by +hand. With the headless/StartOS direction dropped (#8), manual DevTools +refresh is the accepted approach — no automated extraction is needed. ## 3. Claude Code Session Provider — SHIPPED v0.6.0 @@ -257,43 +273,174 @@ Note: the conversation *listing* (paginated, 100/page) still runs in full each time so the cache comparison works — the cap applies to the heavy per-conversation detail fetches, which dominate request volume. -Together with watch mode, this is a stepping stone to the StartOS service: -a scheduled, capped, politely-paced export is the traffic profile a -headless deployment needs. +This is a stepping stone to the StartOS service: a capped, politely-paced +export — scheduled by the host (cron/StartOS), not an in-app loop — is the +traffic profile a headless deployment needs. -## 7. Scheduled / Watch Mode +## 7. Scheduled / Watch Mode — DROPPED (2026-06-28) -Add a `watch` command (or cron integration helper) to run exports automatically -on a schedule: +An in-app `watch`/scheduler loop is not worth building. Scheduling belongs to +whatever hosts the tool: a user cron line locally, and on the long-term +StartOS target the platform's own scheduling. Either way the tool only needs +to do one capped, politely-paced `export` + `joplin` run and exit — which it +already does. If cron ergonomics ever feel clunky, a thin `sync` subcommand +that chains `export` then `joplin` for a single cron line is a trivial +add-on, but the polling loop itself is off the roadmap. -```bash -python -m src.main watch --interval 6h # poll every 6 hours -``` +## 8. StartOS Service Packaging — DROPPED (2026-06-28) -This would run `export` + `joplin` in sequence, then sleep. Alternatively, -provide a `cron` command that prints the correct crontab line for the user's -setup. +Not pursuing a headless StartOS service. The local, manually-run CLI is +sufficient: the source conversations live in the providers' clouds and can be +re-downloaded, and Joplin already syncs (encrypted) to an offsite S3 provider, +so durability is covered without a server in the loop. -Implementation: simple loop with `time.sleep()`, or emit a crontab entry -string that calls the export and joplin commands in sequence. A `--once` -flag would do a single run then exit (useful for cron itself). +Dropping this also retires the one genuinely hard sub-problem it carried — +session-token freshness without a browser. There is no headless context to +keep fresh; the weekly manual DevTools refresh is acceptable. (Local cookie +extraction remains a dead end regardless — see #2.) -Stepping stone to the StartOS service (below). +## 9. Token Validity on `doctor` — IMPLEMENTED (2026-06-28) -## 8. StartOS Service Packaging (long-term destination) +Shipped: `doctor` now adds a "ChatGPT token active" check via +`ChatGPTProvider.session_health()` (reads `/api/auth/session`, passes iff +`error` is falsy and `accessToken` is present), the never-working JWE/`exp` +decode path was removed, and `_fetch_access_token` now fails fast on a set +`error` instead of returning a stale token. Tests in +`tests/test_providers.py::TestChatGPTSessionHealth`. Investigation trail +below for the record. -Run the exporter headless as a StartOS service: scheduled export + sync to -a Joplin Server instance, so the backup happens without a desktop in the -loop. +Goal: if it's cheap to tell how much longer a token will work, show it on +`doctor`. Findings from live recon: -- Building blocks needed first: watch mode (#7), session limiter (#6), - and a Joplin Server (vs. desktop Web Clipper) sync target. -- **Hard problem, flagged early:** session-token freshness without a - browser. ChatGPT tokens last ~7 days; there is no browser on the server - to re-extract from. Candidate mechanisms: a service config action the - user pastes fresh tokens into periodically, or a future browser - extension that pushes tokens to the server. Cookie auto-extraction (#2) - only solves the desktop case. +- **Not readable from the token itself.** ChatGPT's `CHATGPT_SESSION_TOKEN` + is a **JWE** (header `{"alg":"dir","enc":"A256GCM"}`, `eyJ…` prefix is just + the encrypted protected header) — the `exp` claim is AES-256-GCM encrypted + with an OpenAI-only key, so it cannot be decoded client-side. Claude's + `sk-…` key is fully opaque. The existing `doctor` JWT-decode path therefore + never yields an expiry for the real tokens (falls to the "not decodable" + branch). +- **`/api/auth/session` exposes an `expires`** (the provider already calls + this endpoint in `_fetch_access_token`; the response includes `expires` + alongside `accessToken`). Live value observed 2026-06-28: + `2026-09-26` — **~90 days out**. This contradicts both the code's ~7-day + assumption and the lived weekly-refresh cadence, so it is almost certainly + the NextAuth **rolling session window** (re-extended on every call), not + the point at which the pasted token actually 401s. Displaying it verbatim + would give false confidence. +- **RESOLVED 2026-06-28 by a live 401 data point.** When the ChatGPT token + was actually dead (conversations API → 401), `/api/auth/session` still + returned **HTTP 200** with `expires: 2026-09-26` (~90 days out) — and that + `expires` *advanced* between two calls seconds apart (`04:20:08` → `04:30:37`). + So `expires` is a **rolling session window that rolls forward on every call + even for a dead token**; displaying it would actively lie. The same response + carried `error: "RefreshAccessTokenError"` and a stale `accessToken`. +- **The real signal is `error`, not `expires` or `accessToken`.** On a healthy + token `error` is absent/null; when the session token is dead NextAuth can't + refresh and sets `error: "RefreshAccessTokenError"` while still echoing a + rolling `expires` and a stale `accessToken`. This is an exact, free, binary + health check. +- **Design (ready to build):** + 1. `doctor` ChatGPT check → read `/api/auth/session`; pass iff `error` is + falsy and `accessToken` present; on `RefreshAccessTokenError` report + "token expired — refresh". Drop the JWE/`exp` decode path (it can never + work) and do NOT surface `expires`. + 2. Latent bug to fix alongside: `_fetch_access_token` reads `accessToken` + without checking `error`, so it proceeds with a stale token and yields a + confusing downstream 401 instead of a clear "refresh your token" message. + Check `error` there and fail fast. + 3. Claude stays a 401-only signal (opaque `sk-`, no equivalent endpoint). + +## 10. Provider API-Drift Detection — IMPLEMENTED (2026-06-28) + +Shipped: the `canary` command + `BaseProvider.check_drift()` (overridden by +ChatGPT and Claude). It fetches one listing page + one conversation per +provider and asserts only the normalizer's load-bearing fields, emitting +`DRIFT_OK/WARN/ERROR` findings (`src/providers/base.py`). Severity badges +print as a Rich table; ERROR exits non-zero, WARN is non-fatal so a backup +run is never blocked. Drift vocabularies (`_KNOWN_TOOL_AUTHORS`, +`_HANDLED_CONTENT_TYPES`) live in `chatgpt.py` next to the collapse set they +guard. Tests: `TestChatGPTDriftCanary`, `TestClaudeDriftCanary`, +`TestCanaryCommand`. Verified live 2026-06-28 — both providers OK. Recon +trail below for the record. + +## 10b. Provider API-Drift Detection — investigation (2026-06-28) + +The export depends on undocumented internal web APIs (ChatGPT/Claude) that can +change shape without notice. The worst failure for a backup tool is *silent*: +a response-schema change that makes the exporter skip or mis-parse content +without erroring. `doctor` currently checks token validity, reachability, and +manifest↔disk integrity — but not "does the provider's response still look +like what the parser expects." + +To investigate: a lightweight schema/shape assertion on a known-good sample +of each provider's listing + conversation-detail responses (presence and type +of the fields the normalizers rely on), surfaced as a `doctor` check or a +dedicated canary. + +**Live recon — Claude captured 2026-06-28 (ChatGPT pending a token refresh):** + +- **Dependency surface (assert ONLY these — see below for why):** + - listing item: `uuid`, `name`, `updated_at`/`created_at`, `project.name`. + - conversation detail: `uuid`/`id`, `name`, `created_at`, `updated_at`, + `project.name`, `chat_messages[]`. + - message: `sender` (`human`/`assistant`), `text` (string) or `content` + (list of typed blocks), `created_at`. +- **Key finding — full-shape diffing is the wrong design.** Claude's + `settings` object is full of volatile internal codenames that churn + constantly: `enabled_bananagrams`, `enabled_sourdough`, `enabled_foccacia`, + `enabled_saffron`, `enabled_turmeric`, `enabled_monkeys_in_a_barrel`, + `paprika_mode`, `enabled_megaminds`, … A "any new/removed key = drift" + canary would fire on every UI experiment. The canary MUST target the + normalizer's load-bearing fields only, not the whole response. (Aligns with + the drop-noise-don't-retain-it principle.) +- **Real Claude messages are flat `text`/`sender`** — in this archive every + message had a string `text` and NO `content` block list (0 rich blocks + observed). So `_extract_claude_blocks` / `_dispatch_claude_block` (tool_use, + thinking, image, …) is an **unexercised theoretical path**; drift there + can't be "caught" by a canary because it never runs on real data — it's a + safety net for if Claude ever switches to block content. The canary should + assert the flat shape and *warn if `content` ever appears as a list* (that + itself is the drift event that would activate the dormant code). +- **Possible silent-loss spot (separate from drift):** Claude messages carry + `attachments` and `files` arrays (empty in this sample) that the normalizer + ignores entirely. If a user ever attaches files in Claude, they'd be + dropped without a LossReport entry. Worth a follow-up check. +**Live recon — ChatGPT captured 2026-06-28:** + +- **Dependency surface (assert ONLY these):** + - listing item: `id`, `title`, `update_time`/`create_time`. + - conversation detail: `conversation_id`/`id`, `title`, `create_time`, + `update_time`, `mapping` (non-empty). + - mapping node: `message`, `children` (the tree walk depends on both); + message: `author.role`, `author.name`, `content.content_type`, + `content.parts`, `metadata.is_visually_hidden_from_conversation`. +- **content_type vocabulary observed (all currently handled):** `text`, + `model_editable_context`, `multimodal_text`, `thoughts`, `code`, + `execution_output`, `reasoning_recap`, `user_editable_context`, + `tether_browsing_display`. A *new* content_type already degrades gracefully + (visible `unknown` block + WARNING + LossReport tally) — so content_type + drift is **already non-silent**. The canary just needs to confirm the known + set still parses to non-empty blocks. +- **The genuinely silent drift risk — `author.name` collapse keys.** + `_COLLAPSE_TOOL_AUTHORS = {"file_search", "myfiles_browser"}`. Recon + confirms `file_search` is live (and `web.run`/`python` are correctly left + un-collapsed). If OpenAI renames `file_search`, the collapse **silently + stops** and the archive re-bloats with no error or LossReport entry. This is + the top canary target: assert that retrieval-dump tool authors are still + recognized, or at least flag unfamiliar `(role="tool", author.name)` pairs. +- **Second silent risk — empty `content.parts`.** A `text` message whose + `parts` field is renamed/emptied yields zero blocks and is skipped with only + a debug log = silent loss. Canary should assert a sampled `text` message + produces a non-empty block. + +**Recon complete for both providers. Canary design (ready to build):** a +`doctor` check (or dedicated `canary` command) that, per provider, fetches one +listing page + one conversation and asserts the dependency-surface fields +above by presence+type — NOT full shape (Claude `settings` codenames prove +full-shape diffing is pure noise). Specific tripwires: (ChatGPT) unfamiliar +`(tool, author.name)` pair and empty `parts` on a text message; (Claude) +`content` appearing as a list, and non-empty `attachments`/`files`. Failures +surface as a warning, never a hard error (a backup tool must still run). --- @@ -369,16 +516,13 @@ structure matching the user's Obsidian setup. No API needed — just file I/O. ## Token Expiry Notifications -Proactively warn when a token is close to expiry (within 48h for ChatGPT), -rather than only surfacing the warning at startup. Options: - -- Add an `expiry` subcommand that prints token status and exits non-zero if - any token is expired or expiring soon (useful in scripts/cron) -- Send a desktop notification via `notify-send` (Linux) or `osascript` (macOS) - when a token is within 24h of expiry - -Largely superseded by Brave cookie auto-extraction (#2): once refresh is -one command (or automatic), expiry warnings matter much less. +Moved to the active roadmap as §9 (Token Validity on `doctor`). The original +"proactively notify before expiry" idea is blocked by the same finding: a +reliable expiry time isn't available client-side (ChatGPT token is encrypted, +Claude's is opaque, and `/api/auth/session`'s `expires` looks like a rolling +window rather than the real refresh cadence). Any heads-up — a `doctor` +line, an `expiry` subcommand, or a `notify-send` nudge — depends on first +resolving the §9 open question of what signal is actually trustworthy. ## Search Command diff --git a/README.md b/README.md index 7417607..c21aad0 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,17 @@ Before anything else, validate your setup: ai-chat-exporter doctor ``` -This checks token presence, format, expiry, directory permissions, disk space, and live API connectivity. Fix any failures before proceeding. +This checks token presence, format, token health (via the `/api/auth/session` `error` field), directory permissions, disk space, and live API connectivity. Fix any failures before proceeding. + +### Checking for API drift + +The exporter reads ChatGPT's and Claude's undocumented internal web APIs, which can change shape without notice. The worst failure for a backup tool is *silent* — a response change that makes the exporter skip or mis-parse content without erroring. Run the canary to catch that early: + +```bash +ai-chat-exporter canary +``` + +It probes one conversation per provider and checks only the fields the parser depends on (a renamed retrieval-tool author that would bypass the content-collapse, a new content type, drifted message fields, ignored attachments). `OK` means the live schema still matches; `WARN`/`ERROR` tells you exactly what changed. Worth running before a large export or on a schedule. --- diff --git a/pyproject.toml b/pyproject.toml index 432f989..77d8081 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "ai-chat-exporter" -version = "0.6.0" +version = "0.7.0" description = "Export ChatGPT and Claude conversation history to Markdown for personal archival in Joplin" requires-python = ">=3.11" dependencies = [ diff --git a/src/main.py b/src/main.py index a58d5a9..e387b22 100644 --- a/src/main.py +++ b/src/main.py @@ -18,7 +18,7 @@ from src.cache import Cache, CacheError from src.config import ConfigError from src.logging_config import setup_logging from src.loss_report import LossReport -from src.providers.base import ProviderError +from src.providers.base import DRIFT_ERROR, DRIFT_OK, DRIFT_WARN, ProviderError console = Console() err_console = Console(stderr=True) @@ -332,6 +332,91 @@ def doctor(ctx: click.Context) -> None: sys.exit(1) +@cli.command() +@click.option( + "--provider", + type=click.Choice(["chatgpt", "claude", "all"]), + default="all", + help="Which web-API provider(s) to probe.", +) +@click.pass_context +def canary(ctx: click.Context, provider: str) -> None: + """Probe provider APIs for schema drift against the fields the parser needs. + + Fetches one listing page + one conversation per provider and checks only + the normalizer's load-bearing fields (not the full response shape, which + churns harmlessly). Surfaces the silent-failure risks for a backup tool: + a renamed retrieval-tool author bypassing the hidden-content collapse, a + new content_type, drifted message fields, or ignored attachments. + + Exits non-zero if any ERROR finding is reported (a load-bearing field is + missing/mistyped). WARN findings are surfaced but do not fail. See + FUTURE.md §10. + """ + from dotenv import load_dotenv + + load_dotenv(override=False) + + chatgpt_token = os.getenv("CHATGPT_SESSION_TOKEN", "").strip() or None + claude_key = os.getenv("CLAUDE_SESSION_KEY", "").strip() or None + + targets: list[tuple[str, object]] = [] + if provider in ("chatgpt", "all") and chatgpt_token: + from src.providers.chatgpt import ChatGPTProvider + + ct1 = os.getenv("CHATGPT_SESSION_TOKEN_1", "").strip() or None + targets.append( + ("chatgpt", lambda: ChatGPTProvider(chatgpt_token, session_token_1=ct1)) + ) + if provider in ("claude", "all") and claude_key: + from src.providers.claude import ClaudeProvider + + targets.append(("claude", lambda: ClaudeProvider(claude_key))) + + if not targets: + err_console.print( + "[yellow]No web-API provider tokens configured " + "(set CHATGPT_SESSION_TOKEN / CLAUDE_SESSION_KEY).[/yellow]" + ) + sys.exit(1) + + rows: list[tuple[str, dict]] = [] + for name, factory in targets: + try: + findings = factory().check_drift() + except ProviderError as e: + findings = [{"severity": DRIFT_ERROR, "check": "probe", "detail": str(e.original)[:100]}] + except Exception as e: + findings = [{"severity": DRIFT_ERROR, "check": "probe", "detail": str(e)[:100]}] + for f in findings: + rows.append((name, f)) + + _print_canary_table(rows) + if any(f["severity"] == DRIFT_ERROR for _, f in rows): + sys.exit(1) + + +def _print_canary_table(rows: list[tuple[str, dict]]) -> None: + table = Table(title="API Drift Canary", show_header=True) + table.add_column("Provider", style="bold") + table.add_column("Severity", justify="center") + table.add_column("Check") + table.add_column("Detail") + badge = { + DRIFT_OK: "[green]OK[/green]", + DRIFT_WARN: "[yellow]WARN[/yellow]", + DRIFT_ERROR: "[red]ERROR[/red]", + } + for name, f in rows: + table.add_row( + name, + badge.get(f["severity"], f["severity"]), + f.get("check", ""), + f.get("detail", ""), + ) + console.print(table) + + def _run_doctor_checks(cache: Cache | None = None) -> list[dict]: """Run all doctor checks and return results. diff --git a/src/providers/base.py b/src/providers/base.py index 4bacf6d..7f46196 100644 --- a/src/providers/base.py +++ b/src/providers/base.py @@ -50,6 +50,27 @@ VALID_HIDDEN_CONTENT_POLICIES = { HIDDEN_CONTENT_OMIT, } +# --------------------------------------------------------------------------- +# API-drift canary (FUTURE.md §10) +# --------------------------------------------------------------------------- +# The export depends on undocumented internal web APIs that can change shape +# without notice. The canary fetches a small live sample and asserts only the +# normalizer's load-bearing fields (NOT full shape — provider settings objects +# churn constantly and would be pure noise). Findings carry a severity: +# ERROR — a load-bearing field is missing/mistyped; the parser will break or +# silently lose data. The backup is no longer trustworthy. +# WARN — something unfamiliar appeared (new content_type, new tool author, +# a dormant field went live). Investigate; not necessarily broken. +# OK — the assertion held. +DRIFT_ERROR = "error" +DRIFT_WARN = "warn" +DRIFT_OK = "ok" + + +def drift_finding(severity: str, check: str, detail: str = "") -> dict: + """Build one canary finding.""" + return {"severity": severity, "check": check, "detail": detail} + def resolve_hidden_content_policy() -> str: """Read EXPORTER_HIDDEN_CONTENT from the environment, defaulting to placeholder.""" @@ -159,6 +180,15 @@ class BaseProvider(ABC): isolation, e.g. from tests, doesn't crash). """ + def check_drift(self) -> list[dict]: + """Probe the live API and assert the normalizer's load-bearing fields. + + Returns a list of ``drift_finding`` dicts. The default is a no-op + (used by providers with no remote schema risk, e.g. local Claude Code + transcripts); web-API providers override this. See FUTURE.md §10. + """ + return [] + # ------------------------------------------------------------------ # Concrete helpers # ------------------------------------------------------------------ diff --git a/src/providers/chatgpt.py b/src/providers/chatgpt.py index 5ecd367..82b9872 100644 --- a/src/providers/chatgpt.py +++ b/src/providers/chatgpt.py @@ -45,12 +45,16 @@ from src.blocks import ( from src.loss_report import LossReport from src.providers.base import ( BaseProvider, + DRIFT_ERROR, + DRIFT_OK, + DRIFT_WARN, HIDDEN_CONTENT_FULL, HIDDEN_CONTENT_OMIT, HIDDEN_CONTENT_PLACEHOLDER, ProviderError, REQUEST_TIMEOUT, VALID_HIDDEN_CONTENT_POLICIES, + drift_finding, resolve_hidden_content_policy, ) @@ -91,6 +95,20 @@ def parse_asset_file_id(ref: str) -> str | None: # myfiles_browser is the legacy name for the same retrieval tool. _COLLAPSE_TOOL_AUTHORS = {"file_search", "myfiles_browser"} +# ── API-drift canary vocabularies (FUTURE.md §10) ────────────────────────── +# Tool authors seen live 2026-06-28. The collapse keys off author.name, so a +# RENAMED retrieval tool would silently bypass the collapse and re-bloat the +# archive — the canary flags any unfamiliar (role="tool", author.name) pair. +_KNOWN_TOOL_AUTHORS = _COLLAPSE_TOOL_AUTHORS | {"web.run", "python"} +# content_types the normalizer dispatches on (keep in sync with +# _extract_blocks_for_content). A new value already degrades to an `unknown` +# block + LossReport at export time; the canary flags it proactively. +_HANDLED_CONTENT_TYPES = frozenset({ + "text", "multimodal_text", "execution_output", "system_error", + "tether_browsing_display", "code", "thoughts", "reasoning_recap", + "user_editable_context", "model_editable_context", "image_asset_pointer", +}) + class ChatGPTProvider(BaseProvider): """Provider for ChatGPT conversations via the internal web API. @@ -766,6 +784,84 @@ class ChatGPTProvider(BaseProvider): "messages": messages, } + def check_drift(self) -> list[dict]: + """Probe one listing page + one conversation; assert load-bearing fields. + + See FUTURE.md §10. Asserts only the fields the normalizer depends on — + not the full response shape (which churns harmlessly). The top silent + risk is a renamed retrieval tool author bypassing the collapse. + """ + findings: list[dict] = [] + page = self.list_conversations(offset=0, limit=5) + if not page: + return [drift_finding(DRIFT_WARN, "listing", + "empty listing — cannot verify shape")] + + item = page[0] + for f in ("id", "title"): + if f not in item: + findings.append(drift_finding( + DRIFT_ERROR, "listing", f"summary item missing '{f}'")) + if not (item.get("update_time") or item.get("create_time")): + findings.append(drift_finding( + DRIFT_WARN, "listing", "no update_time/create_time on summary")) + + conv_id = item.get("id") + if not conv_id: + return findings or [drift_finding(DRIFT_ERROR, "listing", "no id to fetch")] + raw = self.get_conversation(conv_id) + if not (raw.get("conversation_id") or raw.get("id")): + findings.append(drift_finding( + DRIFT_ERROR, "detail", "no conversation_id/id on detail")) + mapping = raw.get("mapping") + if not isinstance(mapping, dict) or not mapping: + findings.append(drift_finding( + DRIFT_ERROR, "detail", "mapping missing/empty — tree walk will yield 0 messages")) + return findings + + saw_text_with_parts = False + seen_new_ct: set[str] = set() + seen_new_tool: set[str] = set() + for node in mapping.values(): + if "children" not in node: + findings.append(drift_finding( + DRIFT_ERROR, "mapping", "node missing 'children' — tree walk breaks")) + break + msg = node.get("message") + if not msg: + continue + author = msg.get("author") or {} + role, name = author.get("role"), author.get("name") + content = msg.get("content") or {} + ct = content.get("content_type") + if ct and ct not in _HANDLED_CONTENT_TYPES: + seen_new_ct.add(ct) + if role == "tool" and name and name not in _KNOWN_TOOL_AUTHORS: + seen_new_tool.add(name) + if ct == "text": + parts = content.get("parts") or [] + if any(isinstance(p, str) and p.strip() for p in parts): + saw_text_with_parts = True + + for name in sorted(seen_new_tool): + findings.append(drift_finding( + DRIFT_WARN, "collapse", + f"unfamiliar tool author {name!r} — if it's a retrieval dump it " + "is NOT being collapsed (silent archive bloat); update " + "_COLLAPSE_TOOL_AUTHORS / _KNOWN_TOOL_AUTHORS")) + for ct in sorted(seen_new_ct): + findings.append(drift_finding( + DRIFT_WARN, "content_type", + f"new content_type {ct!r} — exported as an unknown block; add a handler")) + if not saw_text_with_parts: + findings.append(drift_finding( + DRIFT_WARN, "parts", + "no text message yielded non-empty parts — content.parts may have drifted")) + + if not findings: + findings.append(drift_finding(DRIFT_OK, "schema", "all load-bearing fields present")) + return findings + # --------------------------------------------------------------------------- # Internal helpers diff --git a/src/providers/claude.py b/src/providers/claude.py index 0e9d66a..a96b48f 100644 --- a/src/providers/claude.py +++ b/src/providers/claude.py @@ -16,7 +16,14 @@ from src.blocks import ( make_unknown_block, ) from src.loss_report import LossReport -from src.providers.base import BaseProvider, ProviderError +from src.providers.base import ( + BaseProvider, + DRIFT_ERROR, + DRIFT_OK, + DRIFT_WARN, + ProviderError, + drift_finding, +) logger = logging.getLogger(__name__) @@ -232,6 +239,70 @@ class ClaudeProvider(BaseProvider): "messages": messages, } + def check_drift(self) -> list[dict]: + """Probe one listing page + one conversation; assert load-bearing fields. + + See FUTURE.md §10. Real Claude messages are flat ``text``/``sender``; + the rich ``content`` block path is dormant, so its activation (content + becoming a list) is itself a drift event worth flagging. ``attachments`` + / ``files`` are ignored by the normalizer — non-empty values are a + silent-loss risk. + """ + findings: list[dict] = [] + page = self.list_conversations(offset=0, limit=5) + if not page: + return [drift_finding(DRIFT_WARN, "listing", + "empty listing — cannot verify shape")] + + item = page[0] + if not (item.get("uuid") or item.get("id")): + findings.append(drift_finding(DRIFT_ERROR, "listing", "summary missing uuid/id")) + if not (item.get("name") or item.get("title")): + findings.append(drift_finding(DRIFT_WARN, "listing", "no name/title on summary")) + if not (item.get("updated_at") or item.get("update_time")): + findings.append(drift_finding(DRIFT_WARN, "listing", "no updated_at on summary")) + + conv_id = item.get("uuid") or item.get("id") + if not conv_id: + return findings or [drift_finding(DRIFT_ERROR, "listing", "no id to fetch")] + raw = self.get_conversation(conv_id) + if not (raw.get("uuid") or raw.get("id")): + findings.append(drift_finding(DRIFT_ERROR, "detail", "no uuid/id on detail")) + msgs = raw.get("chat_messages") or raw.get("messages") + if not isinstance(msgs, list) or not msgs: + findings.append(drift_finding( + DRIFT_ERROR, "detail", "chat_messages missing/empty — 0 messages")) + return findings + + saw_sender = saw_text = False + flagged_list = flagged_attach = False + for m in msgs: + if m.get("sender") or m.get("role"): + saw_sender = True + content = m.get("content") + if isinstance(content, list) and not flagged_list: + findings.append(drift_finding( + DRIFT_WARN, "content", + "message.content is now a LIST — Claude switched to typed blocks; " + "the dormant _dispatch_claude_block path is now live, verify rendering")) + flagged_list = True + if m.get("text") or content: + saw_text = True + if (m.get("attachments") or m.get("files")) and not flagged_attach: + findings.append(drift_finding( + DRIFT_WARN, "attachments", + "message has non-empty attachments/files — the normalizer ignores " + "these (silent loss); add handling")) + flagged_attach = True + + if not saw_sender: + findings.append(drift_finding(DRIFT_ERROR, "messages", "no sender/role on any message")) + if not saw_text: + findings.append(drift_finding(DRIFT_WARN, "messages", "no text/content found")) + if not findings: + findings.append(drift_finding(DRIFT_OK, "schema", "all load-bearing fields present")) + return findings + # --------------------------------------------------------------------------- # Internal helpers diff --git a/tests/test_cli.py b/tests/test_cli.py index bcbbdf7..0de89a5 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -275,3 +275,25 @@ class TestPrune: assert result.exit_code == 0 assert "Aborted" in result.output assert stale.exists() + + +class TestCanaryCommand: + """`canary` wiring — no live API calls (no tokens configured).""" + + def test_no_tokens_exits_nonzero(self, tmp_path): + cache = Cache(tmp_path / "cache") + cache.acknowledge_tos() + runner = CliRunner(mix_stderr=True) + result = runner.invoke( + cli, + ["--no-log-file", "canary"], + env={ + "CACHE_DIR": str(tmp_path / "cache"), + "EXPORT_DIR": str(tmp_path / "exports"), + # Empty so .env (override=False) can't repopulate real tokens. + "CHATGPT_SESSION_TOKEN": "", + "CLAUDE_SESSION_KEY": "", + }, + ) + assert result.exit_code == 1 + assert "No web-API provider tokens" in result.output diff --git a/tests/test_providers.py b/tests/test_providers.py index f25e48d..6028103 100644 --- a/tests/test_providers.py +++ b/tests/test_providers.py @@ -973,3 +973,146 @@ class TestChatGPTSessionHealth: ok, detail = p.session_health() assert ok is False assert "unreachable" in detail.lower() + + +# --------------------------------------------------------------------------- +# API-drift canary (§10): assert load-bearing fields, flag silent risks. +# --------------------------------------------------------------------------- + + +def _sev(findings): + return {f["severity"] for f in findings} + + +def _checks(findings, severity): + return {f["check"] for f in findings if f["severity"] == severity} + + +class TestChatGPTDriftCanary: + def _provider(self, page, detail): + from src.providers.chatgpt import ChatGPTProvider + + p = ChatGPTProvider.__new__(ChatGPTProvider) + p.list_conversations = lambda offset=0, limit=5: page + p.get_conversation = lambda cid: detail + return p + + @staticmethod + def _detail(mapping): + return { + "conversation_id": "c1", "title": "T", + "create_time": 1.0, "update_time": 2.0, "mapping": mapping, + } + + @staticmethod + def _node(role="user", name=None, content_type="text", parts=("hello",)): + return { + "id": "n", "parent": None, "children": [], + "message": { + "author": {"role": role, "name": name}, + "content": {"content_type": content_type, "parts": list(parts)}, + "metadata": {}, + }, + } + + def test_healthy_shape_is_ok_only(self): + from src.providers.base import DRIFT_OK + + p = self._provider([{"id": "c1", "title": "T", "update_time": "x"}], + self._detail({"n1": self._node()})) + assert _sev(p.check_drift()) == {DRIFT_OK} + + def test_missing_listing_id_is_error(self): + from src.providers.base import DRIFT_ERROR + + p = self._provider([{"title": "T"}], self._detail({"n1": self._node()})) + assert DRIFT_ERROR in _sev(p.check_drift()) + + def test_empty_mapping_is_error(self): + from src.providers.base import DRIFT_ERROR + + p = self._provider([{"id": "c1", "title": "T", "update_time": "x"}], + self._detail({})) + findings = p.check_drift() + assert "detail" in _checks(findings, DRIFT_ERROR) + + def test_unfamiliar_tool_author_warns_collapse(self): + from src.providers.base import DRIFT_WARN + + mapping = { + "n1": self._node(), # a healthy text msg so parts check passes + "n2": self._node(role="tool", name="brand_new_retriever"), + } + p = self._provider([{"id": "c1", "title": "T", "update_time": "x"}], + self._detail(mapping)) + assert "collapse" in _checks(p.check_drift(), DRIFT_WARN) + + def test_new_content_type_warns(self): + from src.providers.base import DRIFT_WARN + + mapping = { + "n1": self._node(), + "n2": self._node(content_type="hologram_v2", parts=()), + } + p = self._provider([{"id": "c1", "title": "T", "update_time": "x"}], + self._detail(mapping)) + assert "content_type" in _checks(p.check_drift(), DRIFT_WARN) + + def test_known_tool_author_not_flagged(self): + # file_search is a known collapse author — must NOT warn. + mapping = { + "n1": self._node(), + "n2": self._node(role="tool", name="file_search"), + } + p = self._provider([{"id": "c1", "title": "T", "update_time": "x"}], + self._detail(mapping)) + assert "collapse" not in _checks(p.check_drift(), "warn") + + +class TestClaudeDriftCanary: + def _provider(self, page, detail): + from src.providers.claude import ClaudeProvider + + p = ClaudeProvider.__new__(ClaudeProvider) + p.list_conversations = lambda offset=0, limit=5: page + p.get_conversation = lambda cid: detail + return p + + @staticmethod + def _detail(msgs): + return {"uuid": "u1", "name": "N", "created_at": "x", "updated_at": "y", + "chat_messages": msgs} + + def test_healthy_flat_shape_is_ok(self): + from src.providers.base import DRIFT_OK + + msgs = [{"uuid": "m1", "sender": "human", "text": "hi", + "created_at": "x", "attachments": [], "files": []}] + p = self._provider([{"uuid": "u1", "name": "N", "updated_at": "z"}], + self._detail(msgs)) + assert _sev(p.check_drift()) == {DRIFT_OK} + + def test_content_as_list_warns(self): + from src.providers.base import DRIFT_WARN + + msgs = [{"uuid": "m1", "sender": "assistant", + "content": [{"type": "text", "text": "hi"}], "created_at": "x"}] + p = self._provider([{"uuid": "u1", "name": "N", "updated_at": "z"}], + self._detail(msgs)) + assert "content" in _checks(p.check_drift(), DRIFT_WARN) + + def test_nonempty_attachments_warn(self): + from src.providers.base import DRIFT_WARN + + msgs = [{"uuid": "m1", "sender": "human", "text": "hi", + "created_at": "x", "attachments": [{"file_name": "a.pdf"}]}] + p = self._provider([{"uuid": "u1", "name": "N", "updated_at": "z"}], + self._detail(msgs)) + assert "attachments" in _checks(p.check_drift(), DRIFT_WARN) + + def test_empty_messages_is_error(self): + from src.providers.base import DRIFT_ERROR + + p = self._provider([{"uuid": "u1", "name": "N", "updated_at": "z"}], + self._detail([])) + assert DRIFT_ERROR in _sev(p.check_drift())