canary: detect provider API schema drift; release v0.7.0

The export reads ChatGPT's and Claude's undocumented internal web APIs,
which can change shape without notice; the worst failure for a backup tool
is a silent one (skipped/mis-parsed content with no error). Add a `canary`
command + BaseProvider.check_drift() (overridden by ChatGPT/Claude) that
fetches one listing page + one conversation per provider and asserts only
the normalizer's load-bearing fields — not the full response shape, which
churns harmlessly. The top silent risk it guards is a renamed retrieval-tool
author bypassing the hidden-content collapse. ERROR findings exit non-zero;
WARN findings are surfaced but non-fatal so a backup run is never blocked.

Also retires the in-app watch mode and headless StartOS direction from the
roadmap (tool stays a local, manually-run CLI) and updates docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.env.example

@@ -6,9 +6,19 @@
 
 # --- ChatGPT ---
 # How to get: open chatgpt.com in Chrome → F12 → Application tab
-# → Cookies → https://chatgpt.com → find "__Secure-next-auth.session-token" → copy Value
-# Token type: JWT (starts with "eyJ"). Typically valid for ~7 days.
+# → Cookies → https://chatgpt.com → find the two cookie chunks:
+#   __Secure-next-auth.session-token.0  (starts with "eyJ") → CHATGPT_SESSION_TOKEN
+#   __Secure-next-auth.session-token.1  (the remainder)     → CHATGPT_SESSION_TOKEN_1
+# Token type: JWE. Typically valid for ~7 days.
 CHATGPT_SESSION_TOKEN=
+CHATGPT_SESSION_TOKEN_1=
+
+# ChatGPT Projects (optional): comma-separated list of project gizmo IDs.
+# Project conversations are NOT included in the default /conversations listing.
+# How to find: open chatgpt.com → click a Project → look at the browser URL:
+#   https://chatgpt.com/g/g-p-<ID>-<slug>/project  →  copy "g-p-<ID>"
+# Example: CHATGPT_PROJECT_IDS=g-p-68c2b2b3037c8191890036fb4ae3ed9f,g-p-anotherproject
+CHATGPT_PROJECT_IDS=
 
 # --- Claude ---
 # How to get: open claude.ai in Chrome → F12 → Application tab
@@ -26,10 +36,47 @@ EXPORT_DIR=./exports
 #   provider/year          → exports/claude/2024/file.md (ignores projects)
 OUTPUT_STRUCTURE=provider/project/year
 
+# What to do with content that was invisible in the provider's web UI
+# (file-retrieval tool dumps, hidden context like Custom Instructions).
+# These dumps can be 90% of a conversation's bytes. Options:
+#   placeholder  (default) → one-line placeholder with tool name and size
+#   full                   → keep everything (pre-v0.6.0 behavior)
+#   omit                   → drop entirely (still counted in the run summary)
+EXPORTER_HIDDEN_CONTENT=placeholder
+
+# Download conversation assets (images, audio) next to the Markdown, into a
+# media/ folder, and inline them. Options:
+#   images  (default) → images only
+#   all               → also audio/voice clips and other files
+#   off               → keep text placeholders, download nothing
+# Downloaded media is uploaded to Joplin as resources on the next `joplin` run.
+EXPORTER_DOWNLOAD_MEDIA=images
+
+# Cap how many conversations are downloaded per export run (per provider).
+# Runs are resumable — a capped run continues where it stopped next time.
+# Keeps big backfills from looking like scraper traffic. Unset = unlimited.
+#MAX_CONVERSATIONS_PER_RUN=25
+
+# Seconds between consecutive API requests (small random jitter is added).
+# Default 1.0; set 0 to disable pacing.
+#REQUEST_DELAY=1.0
+
+# --- Joplin ---
+# Automate importing exported conversations into Joplin as notes.
+# Requires Joplin desktop running with the Web Clipper service enabled.
+# How to get the token:
+#   Joplin → Tools → Options → Web Clipper → copy "Authorization token"
+JOPLIN_API_TOKEN=
+# API URL (default port is 41184; change only if you've customised it)
+JOPLIN_API_URL=http://localhost:41184
+# Request timeout in seconds (default: 30). Increase if Joplin times out on
+# large conversations. Example: JOPLIN_REQUEST_TIMEOUT=60
+# JOPLIN_REQUEST_TIMEOUT=30
+
 # --- Cache ---
-# Where the sync manifest and logs are stored (default: ~/.ai-chat-exporter)
-CACHE_DIR=~/.ai-chat-exporter
+# Where the sync manifest is stored (default: ./cache, inside the install directory)
+CACHE_DIR=./cache
 
 # --- Logging ---
 # Log file path. Set to "none" to disable file logging.
-LOG_FILE=~/.ai-chat-exporter/logs/exporter.log
+LOG_FILE=./cache/logs/exporter.log

.gitignore

@@ -25,10 +25,14 @@ exports/
 !CHANGELOG.md
 
 # Cache and logs
+cache/
 .ai-chat-exporter/
 logs/
 *.log
 
+# Test tracking
+test-plan.csv
+
 # Editor / OS
 .DS_Store
 .idea/
@@ -36,3 +40,6 @@ logs/
 *.swp
 *.swo
 Thumbs.db
+
+# HTTP traffic captures — may contain auth cookies and session tokens
+*.har

CHANGELOG.md

@@ -3,9 +3,83 @@
 All notable changes to this project will be documented here.
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
-## [0.1.0] - Unreleased
+## [0.7.0] - 2026-06-28
+
 ### Added
-- Initial implementation: ChatGPT and Claude export via internal web APIs
-- Markdown and JSON exporters
-- Local cache/manifest for incremental sync
-- CLI with export, list, cache, doctor, and auth commands
+- `canary` command — probes the ChatGPT/Claude web APIs for schema drift against the fields the parser actually depends on (one listing page + one conversation per provider), not the full response shape. Flags the silent-failure risks for a backup tool: a renamed retrieval-tool author bypassing the hidden-content collapse, a new `content_type`, drifted message fields, or non-empty Claude `attachments`/`files` the normalizer ignores. ERROR findings exit non-zero; WARN findings are surfaced but non-fatal. See `FUTURE.md` §10.
+- `doctor` now reports a real ChatGPT token-health check ("ChatGPT token active") based on the `/api/auth/session` `error` field. ChatGPT session tokens are JWEs whose `exp` is encrypted and unreadable client-side; the previous decode path could never yield an expiry. The honest signal is `error == "RefreshAccessTokenError"` (verified live), which means the session token is dead even though `expires`/`accessToken` still echo stale values. See `FUTURE.md` §9.
+
+### Fixed
+- `ChatGPTProvider._fetch_access_token` now checks the `/api/auth/session` `error` field and fails fast with a clear "refresh your token" message. Previously it returned the stale `accessToken` present on a dead session, producing a confusing downstream 401 instead of an actionable auth error.
+
+### Removed
+- `auth --from-browser` and the `browser-cookie3` dependency (`src/browser_tokens.py`). Browser cookie auto-extraction is not viable: modern Chromium App-Bound Encryption (Chrome 127+/current Brave) keys cookies off a SYSTEM-level layer that can't be decrypted off disk without Administrator rights and AV-flagged SYSTEM impersonation, and fails on Brave specifically (symptom: `Unable to get key for cookie decryption`). Auth is manual (DevTools wizard) only. See `FUTURE.md` §2 for the full rationale.
+
+## [0.6.0] - 2026-06-12
+
+First tagged release. The project was developed through several internal
+milestones (0.1.0–0.5.0) that were never published; their changes are
+consolidated here.
+
+### Added
+
+**Core export & sync**
+- ChatGPT and Claude export via their internal web APIs, with a local cache/manifest for incremental, resumable sync (only new or updated conversations are fetched; each is written to the manifest immediately).
+- Markdown and JSON exporters. Markdown rendering happens at exporter-write time (providers produce typed blocks; exporters render them), which keeps the door open for future Obsidian/HTML output.
+- CLI: `export`, `list`, `cache`, `doctor`, `auth`, `joplin`, `prune`.
+- `--project` filter on `export`/`list`/`joplin` (case-insensitive substring, or `none`).
+- ChatGPT Projects support via `CHATGPT_PROJECT_IDS` (project conversations are fetched separately from the default listing).
+
+**Joplin integration**
+- `joplin` command syncs exported Markdown to Joplin as notes; notebooks are created automatically and nested per provider+project. Re-running is safe — the Joplin note ID is stored in the manifest, so notes are updated, not duplicated.
+- `JOPLIN_API_TOKEN`, `JOPLIN_API_URL`, `JOPLIN_REQUEST_TIMEOUT` config, with actionable error messages on timeout/connection failure.
+
+**Rich content (typed blocks)**
+- Messages carry an ordered `blocks` list: text, code, thinking, tool_use, tool_result, citation, image_placeholder, file_placeholder, unknown.
+- ChatGPT voice mode: `audio_transcription` parts render as text; audio asset pointers render as `📎 File attached` placeholders with size/duration. Custom Instructions (`user_editable_context`/`model_editable_context`) now appear (were silently dropped) with a `> ℹ️ Hidden context` marker.
+- ChatGPT `execution_output`, `system_error`, and `tether_browsing_display` render as `tool_result` blocks (with `tool_name`, `is_error`, optional `summary`); transient browse spinners skip silently.
+- Defensive Claude block extraction (text/thinking/tool_use/tool_result/image, including nested-block flattening).
+- `_safe_fence` picks a backtick fence longer than any run in the content, so embedded triple-backticks can't corrupt rendering (verified live in Joplin).
+- Data-loss visibility: a `LossReport` summary at the end of every `export` run breaks down `unknown blocks` and `extraction failures` by raw type; `unknown` blocks render as visible `> ⚠️ Unsupported content` with the raw type and observed keys.
+
+**Invisible-content collapse (`EXPORTER_HIDDEN_CONTENT=full|placeholder|omit`, default `placeholder`; `export --hidden-content`)**
+- Messages that were invisible in the ChatGPT web UI are collapsed to one-line placeholders instead of exported in full. Two triggers: (a) tool-role retrieval dumps identified by `author.name` (`file_search`, `myfiles_browser`) — ChatGPT re-injects the full text of attached/project files on every tool run, measured at 86% of all content bytes across the three largest conversations and **not** hidden-flagged; (b) messages flagged `is_visually_hidden_from_conversation` (Custom Instructions). Code-execution results, web-search output, and all dialogue are untouched.
+- New `collapsed` block renders as `> 🔧 **Tool output** — file_search (15.0 KB) — omitted (EXPORTER_HIDDEN_CONTENT=full to keep)` (ℹ️ Hidden context variant for hidden-flagged messages). The `LossReport` gains a `collapsed by policy` section (count per origin + total KB) so the omission stays visible.
+
+**Claude Code session provider (`--provider claude-code`)**
+- Archives local agent transcripts from `~/.claude/projects/*/*.jsonl` (override with `CLAUDE_CODE_DIR`) — no tokens, no rate limits, no ToS exposure. Prose-only by default per the hidden-content policy: dialogue and deliverables kept; tool traffic grouped into one placeholder per activity run (`> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted`); thinking dropped (counted in the summary); subagent (`isSidechain`) and harness (`isMeta`, snapshots, command tags) records stripped. Titles from the last `ai-title` record; project from the working-directory basename. Joplin notebooks nest under the `AI-Claude` parent. Measured: 19MB of JSONL → 804KB of Markdown across 25 sessions.
+
+**Binary/image downloads (`EXPORTER_DOWNLOAD_MEDIA=images|all|off`, default `images`; `export --download-media`)**
+- Downloads ChatGPT conversation assets into a `media/` folder beside each export and inlines them — images become real `![](media/…)` embeds, files become local links. Two-hop fetch via `/backend-api/files/{id}/download` → signed URL (verified live). `all` also pulls voice-mode audio (≈162MB of clips in this archive, transcripts already in text — hence the images-only default). Idempotent (skips assets already on disk), atomic writes, `600` perms. Expired/missing assets (old generated images 404) keep their placeholder and are counted as `media failed` — never fatal.
+- Joplin resource upload: the `joplin` command uploads downloaded media as resources and rewrites `media/…` links to `:/resourceId`, so images render inside notes (verified live: resources created with correct mime/size and linked to the note). Resource IDs are tracked per-conversation in the manifest, so re-syncs reuse them instead of duplicating.
+
+**Token setup & throughput**
+- `auth --from-browser [brave|chrome|chromium|edge|firefox]`: extracts ChatGPT session-token cookies and the Claude `sessionKey` straight from a local browser's cookie store (via `browser-cookie3`; defaults to Brave), validates each against the live API, and writes `.env`. Tokens that fail validation never overwrite a working entry; clear fallback when the browser is absent or the cookie DB is locked.
+- Session limiter: `--max-conversations N` / `MAX_CONVERSATIONS_PER_RUN` cap downloads per run (per provider); capped-out conversations are reported as "deferred" and resumed next run.
+- Polite pacing: `REQUEST_DELAY` (default 1.0s, ±25% jitter, `0` disables) between consecutive API requests, with the existing 429 backoff as the reactive net.
+
+**Re-rendering**
+- `export --force` re-exports every conversation even if cached and unchanged, so the whole archive can be re-rendered after a formatting/feature change without `cache --clear`. Runs as a tracked campaign (a stamped start time in the manifest): each run re-renders the least-recently-exported conversations, the "still to go" count shrinks toward zero, finished providers do no further work, and the campaign auto-completes (reporting "Force re-render complete"). Combines with `--max-conversations` to spread the work across runs.
+- `mark_exported` now preserves `joplin_note_id` / `joplin_synced_at` / `joplin_resources` across re-exports (refreshing `exported_at`), so a re-export followed by `joplin` updates the existing notes instead of creating duplicates.
+- Fix: `.env` is now loaded at the very start of every command, so `CACHE_DIR` and `LOG_FILE` are honored (previously the cache silently used `./cache` regardless of `CACHE_DIR`).
+
+**Archive hygiene**
+- `prune` deletes export files not referenced by the manifest (old-layout trees, no-ID orphans), with listing, confirmation, `--dry-run`, `-y`, and empty-directory sweep. Refuses to run when the manifest references no files (so `cache --clear` + `prune` can't wipe the archive). First live run removed 420 stale files (9.4 MB).
+- `doctor` verifies manifest ↔ disk integrity: every recorded `file_path` must exist on disk.
+
+### Changed
+- ChatGPT role filter that dropped `tool`/`system` messages is **lifted**; all roles route through normal extraction (truly empty messages skip via the empty-content guard).
+- `BaseProvider.normalize_conversation` accepts an optional `LossReport` parameter.
+- `ChatGPTProvider.normalize_conversation` reads `conversation_id` as a fallback for `id` (live detail responses use `conversation_id`; fixtures use `id`).
+- `Config` validates `EXPORTER_HIDDEN_CONTENT`, `EXPORTER_DOWNLOAD_MEDIA`, `MAX_CONVERSATIONS_PER_RUN`, and `REQUEST_DELAY`, and logs the active values at startup.
+- New dependency: `browser-cookie3==0.20.1`.
+
+### Fixed
+- Custom Instructions (`user_editable_context`/`model_editable_context`) were silently dropped from every conversation (parts-vs-direct-fields mismatch).
+
+### Migration
+- JSON exports: messages contain typed `blocks` and may omit the legacy `content` field — external consumers should prefer `blocks`.
+- To re-render existing exports with the current rendering and collapse policy: `python -m src.main cache --clear` then `python -m src.main export`, followed by `joplin` to update notes (and upload media resources). Per-conversation message counts may increase as previously-dropped Custom Instructions, image-only turns, and tool-only turns now appear.
+
+### Test suite
+- 264 tests, all passing.

FUTURE.md

@@ -1,42 +1,479 @@
 # Planned Future Work
 
-These items are explicitly out of scope for v0.1.0 but have been designed for.
-The codebase is structured to make each of these additions straightforward.
+Items completed in each release are moved to the changelog. Items here are
+designed for but not yet implemented. The codebase is structured to make each
+of these additions straightforward.
 
-## Export --force Flag (v0.1.x)
-Add `--force` to the `export` command to re-export already-cached conversations
-without permanently clearing the entire manifest. Useful for re-generating files
-after changing the Markdown template or output structure.
+**Completed:**
+- v0.1.0 — Core export: ChatGPT + Claude, incremental sync, Markdown + JSON output
+- v0.2.0 — Joplin import automation (`joplin` command, create/update notes, notebook auto-creation)
+- v0.4.0 — Rich content support: typed message blocks (text, code, thinking, tool_use, tool_result, image_placeholder, file_placeholder, unknown); ChatGPT voice transcripts as text + audio placeholders; Custom Instructions extraction; data-loss visibility via `LossReport` summary and visible `unknown` blocks
+- v0.5.0 — Nested Joplin notebooks, date-prefixed note titles, flat year folders
+- v0.6.0 — Collapse tool retrieval dumps & hidden context (`EXPORTER_HIDDEN_CONTENT` policy; roadmap item 1); session limiter + request pacing (`MAX_CONVERSATIONS_PER_RUN`, `REQUEST_DELAY`; roadmap item 6)
 
-Implementation: pass a `force=True` flag to `cache.get_new_or_updated()`, which
-returns all conversations regardless of cache state when force is True.
+---
 
-Current workaround: `python -m src.main cache --clear` then re-run export.
+# Roadmap (decided 2026-06-12)
 
-## Joplin Integration (v0.2.0)
-Automate importing exported Markdown files into Joplin as new notes.
-Joplin exposes a local REST API (requires Joplin desktop running with Web Clipper enabled).
+Priorities reflect the tool's primary purpose — a trustworthy backup so that
+conversation data is not lost if a provider account is ever closed — plus the
+day-to-day friction of the weekly ChatGPT token refresh. The tool stays a
+local, manually-run CLI; the headless/StartOS direction was dropped
+2026-06-28 (see #7 and #8), which also retires the token-freshness problem
+(manual refresh is sufficient).
 
-Approach: after export, iterate exported files and POST each to
-`http://localhost:41184/notes` with the appropriate notebook ID.
+**Now (in order):**
+1. ~~Collapse tool retrieval dumps & hidden context~~ — **shipped in v0.6.0**
+   (full-archive `export --force` re-export completed 2026-06-13)
+2. ~~Brave cookie auto-extraction~~ — **shipped in v0.6.0, removed afterward.**
+   Not viable: modern Chromium App-Bound Encryption (Chrome 127+/current Brave)
+   needs admin + SYSTEM impersonation that AV flags as credential theft, and
+   fails on Brave specifically. Auth is manual (DevTools) — see entry below.
+3. ~~Claude Code session provider~~ — **shipped in v0.6.0**
+4. ~~Archive hygiene — `prune` + `doctor` integrity check~~ — **shipped in v0.6.0**
 
-The output folder structure maps directly to Joplin notebooks:
-- exports/chatgpt/my-project/    → Joplin notebook "ChatGPT - My Project"
-- exports/claude/my-project/     → Joplin notebook "Claude - My Project"
-- exports/chatgpt/no-project/    → Joplin notebook "ChatGPT - No Project"
-- exports/claude/no-project/     → Joplin notebook "Claude - No Project"
+**Soon, but later:**
 
-Prerequisites:
-- Joplin desktop must be running with Web Clipper enabled
-- `JOPLIN_API_TOKEN` env var (get from Joplin → Tools → Web Clipper Options)
-- The Joplin import script will need to create notebooks if they don't exist,
-  then POST each note into the correct notebook
+5. ~~Binary content downloads~~ — **shipped in v0.6.0**
+6. ~~Per-session download limiter + polite pacing~~ — **shipped in v0.6.0**
+7. ~~Scheduled / watch mode~~ — **dropped 2026-06-28**; the tool stays a
+   manually-run CLI, so no in-app polling loop is needed
+8. ~~StartOS service packaging~~ — **dropped 2026-06-28**; the local CLI is
+   sufficient (source convos live in the cloud and can be re-downloaded;
+   Joplin already syncs encrypted to an offsite S3 provider). Dropping this
+   also retires the headless token-freshness problem — manual weekly refresh
+   is fine.
 
-Note: The default OUTPUT_STRUCTURE of provider/project/year is assumed when
-implementing the import script. If the user has changed OUTPUT_STRUCTURE,
-the import script will need updating accordingly.
+**Active:**
+9. ~~Surface remaining token validity on `doctor`~~ — **IMPLEMENTED 2026-06-28**
+   via the `/api/auth/session` `error` field (not `expires`). See §9.
+10. ~~Provider API-drift detection~~ — **IMPLEMENTED 2026-06-28** as the
+    `canary` command. See §10.
+
+**Deprioritized** (entries kept at the bottom of this file; revisit on
+demand): `--force` flags, per-conversation cache reset, official export-ZIP
+fallback, o1/o3 reasoning reclassification, Obsidian output, search command.
+Additional web providers (Gemini/Grok/Perplexity) are explicitly out of
+scope — no significant usage to archive.
+
+---
+
+## 1. Collapse Tool Retrieval Dumps & Hidden Context — SHIPPED v0.6.0
+
+**Implemented 2026-06-12** as designed below, with one scoping correction
+from live recon: the retrieval dumps are NOT flagged
+`is_visually_hidden_from_conversation` — `author.name == "file_search"` is
+the discriminator (the hidden flag only marks Custom Instructions and small
+system stubs). Verified live: worst files shrink 93% (524KB → 36KB).
+Full-archive re-export (the `export --force` campaign) + Joplin re-sync
+completed 2026-06-13.
+
+**Problem (measured 2026-06-12 against a full fresh export):** 45% of the
+entire 11.2MB archive (260 files) is tool-role messages; 29 files are
+majority tool-dump. Worst case: a 524KB conversation where 495KB (94%, 64
+messages) is ChatGPT's file-retrieval tool re-injecting the full text of
+the user's own attached documents, the same files dumped dozens of times
+per conversation. These messages were invisible in the ChatGPT web UI;
+they appear in exports because v0.4.0 lifted the role filter to fix silent
+data loss. Custom Instructions hidden-context blocks are a minor secondary
+case (~2KB, once per conversation) — the originally planned
+`EXPORTER_INCLUDE_HIDDEN_CONTEXT` toggle alone would not help: the worst
+file contains zero hidden-context blocks.
+
+**Fix: collapse, don't drop** (consistent with the no-silent-drop rule):
+
+- `EXPORTER_HIDDEN_CONTENT=full|placeholder|omit` env var, default
+  `placeholder`, plus a `--hidden-content` CLI override on `export`.
+- `placeholder` renders affected messages as one line with type and size:
+  `> 🔧 Tool output (file_search, 24KB) — omitted
+  (EXPORTER_HIDDEN_CONTENT=full to keep)`. Expected effect: archive
+  roughly halves; worst files shrink ~90%.
+- **Scope — collapse:** (a) tool-role retrieval dumps, identified by raw
+  `author.name` (`file_search`, `myfiles_browser`, …) in the API response
+  (the rendered Markdown only shows a generic "🔧 Tool" label, so the
+  decision must happen in the provider, not the renderer); (b) messages
+  flagged `is_visually_hidden_from_conversation`, including
+  `user_editable_context` / `model_editable_context` (Custom
+  Instructions) — subsumes the old suppress-hidden-context idea.
+- **Scope — keep at full size:** code-execution `tool_result` blocks and
+  web-search results; those are usually content the user wants.
+- Count collapsed messages in the post-export summary so the omission
+  stays visible (mirror the LossReport presentation, but as intentional
+  policy, not loss).
+
+Re-export workflow after shipping: `cache --clear` + `export` (same as
+the v0.4.0 migration).
+
+## 2. Brave Cookie Auto-Extraction — REMOVED (not viable)
+
+**Shipped v0.6.0 (2026-06-12), removed 2026-06-27.** `auth --from-browser`
+plus `src/browser_tokens.py` and the `browser-cookie3` dependency are gone.
+Auth is manual (DevTools) only. Do not re-attempt without a fundamentally
+different mechanism (see below).
+
+**Why it doesn't work.** Modern Chromium browsers encrypt cookies on Windows
+with **App-Bound Encryption** (Chrome 127+, July 2024; current Brave).
+Cookies are written with a `v20` prefix and keyed off a secret wrapped in a
+**SYSTEM-level** DPAPI layer plus app validation. `browser-cookie3` only
+knows the legacy `v10`/DPAPI key, so its AES-GCM MAC check fails — the exact
+symptom hit in the field:
+
+```
+ChatGPT: Could not read brave cookies for chatgpt.com: Unable to get key for cookie decryption.
+```
+
+Decrypting `v20` at all requires unwrapping the SYSTEM layer, which means
+running as SYSTEM (e.g. a PsExec-style service) — i.e. **Administrator
+rights** and behavior that AV/EDR flags as infostealer activity. The one
+maintained Python option (`rookiepy`) needs admin from Chrome v130+, was
+**archived 2026-06-07**, and has an unresolved bug where **Brave returns 0
+cookies** even after the key is retrieved. ABE is *designed* to stop exactly
+this, so no off-disk reader is a reliable, non-invasive fit.
+
+**If ever revisited:** the only non-admin path is Chrome Remote Debugging
+(launch the browser with `--remote-debugging-port`, read cookies via
+`Network.getAllCookies` — the running browser decrypts for you). Heavier and
+intrusive; not worth it for a weekly token refresh that takes 30 seconds by
+hand. With the headless/StartOS direction dropped (#8), manual DevTools
+refresh is the accepted approach — no automated extraction is needed.
+
+## 3. Claude Code Session Provider — SHIPPED v0.6.0
+
+**Implemented 2026-06-12** as designed below (`src/providers/claude_code.py`,
+`--provider claude-code`, `CLAUDE_CODE_DIR` override). Additional findings
+during implementation: `isSidechain` records are subagent transcripts (skipped),
+`isMeta` marks harness-generated user records (skipped), and listing/normalized
+`updated_at` must both use file mtime or the cache would re-export every
+session every run. First export: 25 sessions, 19MB JSONL → 804KB Markdown.
+
+Archive local Claude Code session transcripts. No tokens, no rate limits,
+no ToS risk — the data is already on disk but lives in a single JSONL per
+session that Claude Code may clean up, and it contains deliverables
+(reviews, plans, analyses) that exist nowhere else.
+
+Decisions (2026-06-12):
+- **Rendering: prose-only.** Keep user prompts and assistant text
+  (including full deliverable write-ups); collapse tool activity to
+  one-line placeholders (`> 🔧 Tool activity — 14 calls (Read ×9, Bash ×2),
+  86KB — omitted`); **exclude thinking blocks**.
+- **Joplin: sync enabled.** Each coding project becomes a notebook nested
+  under an **"AI-Claude"** parent notebook (nested-notebook support shipped
+  in v0.5.0).
+
+Data facts (measured 2026-06-12):
+- Source: `~/.claude/projects/<munged-cwd>/<session-uuid>.jsonl`.
+  Currently 29 sessions, 18.8MB total, largest 3.8MB.
+- Representative 2.1MB session: tool_result 436KB, tool_use 122KB,
+  thinking 104KB, dialogue prose only ~24KB (~4%) — collapsing tool
+  activity is what makes these exports readable.
+- Record types: `user` / `assistant` (Anthropic-style `message.content`
+  block arrays) plus harness records: `ai-title` (use for note title and
+  filename slug), `last-prompt`, `file-history-snapshot`, `attachment`,
+  `permission-mode`, `system` (skip). Strip harness noise from user
+  messages (`<local-command-caveat>`, `<command-name>` blocks).
+
+Implementation shape: new `src/providers/claude_code.py` implementing the
+`BaseProvider` interface — `list_conversations` scans project dirs,
+`get_conversation` parses the JSONL, `normalize_conversation` maps onto the
+existing block schema (content is already block-shaped: text / tool_use /
+tool_result / thinking). Incremental sync via file mtime/size recorded in
+the existing manifest. Project name derives from the munged cwd dirname.
+
+## 4. Archive Hygiene: `prune` Command + Manifest Integrity — SHIPPED v0.6.0
+
+**Implemented 2026-06-12** as designed below, plus an empty-manifest guard
+(refuses to prune right after `cache --clear`). First live run removed 420
+stale files (9.4 MB, old-layout trees + `_.md` orphans); doctor now reports
+manifest↔disk integrity (293/293 after the run).
+
+A backup is only trustworthy if the on-disk tree matches the manifest.
+Observed 2026-06-12: pre-v0.5.0 layout trees (`tspc-expertcouncil/2025/`)
+and `_.md` no-ID orphans (from the empty-conversation-id bug fixed in
+v0.4.1) sit alongside current exports and would double-sync into Joplin.
+
+- `prune` command: delete export files not referenced by the manifest.
+  `--dry-run` (default off, but always print the list before deleting)
+  shows what would be removed and why (old layout / orphan / unknown).
+- `doctor` extension: verify every manifest entry's `file_path` exists on
+  disk; report missing files (re-export candidates) and unreferenced files
+  (prune candidates).
+
+## 5. Binary Content Downloads — SHIPPED v0.6.0
+
+**Implemented 2026-06-12** (`src/media.py`, `EXPORTER_DOWNLOAD_MEDIA`,
+`download_asset`/`parse_asset_file_id` on the ChatGPT provider, Joplin
+`create_resource` + `upload_media_and_rewrite`). Live recon settled the
+download mechanism: `GET /backend-api/files/{id}/download` returns a signed
+`download_url`; a second GET yields the bytes (works for user uploads;
+older AI-generated images 404 — expired server-side, handled gracefully).
+Asset refs come in three shapes — `sediment://file_…`,
+`sediment://<hash>#file_…#p_N.png` (generated), `file-service://…`. Archive
+scan: 14 images (4 uploads / 10 generated) + 556 audio clips ≈162MB, so
+images-only is the default and audio is opt-in via `all`.
+
+Original notes below.
+
+**Priority note (2026-06-12): "later, but soon" — under the
+backup-if-account-closes goal, embedded images are part of the data that
+would be lost; placeholders alone don't preserve them.**
+
+v0.4.0 ships placeholders for images and audio assets but does not download
+the binary content. The `_safe_fence`-wrapped placeholders include the asset
+reference (`sediment://...` or `file-service://...`), MIME type, size, and
+duration where available; the actual bytes are not preserved.
+
+Next steps:
+- Download attached images alongside the Markdown export, save under a
+  `media/` sibling directory with a stable filename derived from the asset
+  reference.
+- Replace `image_placeholder` rendering with an inline `![](relative/path)`
+  reference once the file is on disk.
+- Joplin integration: upload binaries as Joplin resources via `POST /resources`,
+  rewrite the rendered Markdown to use `:/resourceId` references, and track
+  the resource ID in the cache manifest so re-syncs stay idempotent.
+- DALL-E images on the assistant side: not observed in this user's data; the
+  code path exists (`source = "model_generated"`) but is untested.
+
+The block-level schema is already in place — only the file-fetch + rewrite
+layer needs to be added. See the `image_placeholder` and `file_placeholder`
+block definitions in `src/blocks.py`.
+
+## 6. Per-Session Download Limiter — SHIPPED v0.6.0
+
+**Implemented 2026-06-12** as designed below: `--max-conversations N` /
+`MAX_CONVERSATIONS_PER_RUN` session cap with deferred-count reporting, and
+`REQUEST_DELAY` pacing (default 1.0s ±25% jitter) in `BaseProvider._request`.
+Verified live: a 2-pending run with cap 1 exported one, deferred one, and
+the re-run picked it up.
+
+Cap how many conversations are downloaded in a single `export` run so the
+tool never hammers the ChatGPT/Claude internal APIs with a large burst —
+most importantly on the very first export, which otherwise fetches the
+entire conversation history in one session. Because every run is resumable
+(the manifest records each conversation immediately), a capped run simply
+exports the first N pending conversations and the next run picks up where
+it left off. This keeps traffic looking like a human-paced session rather
+than a scraper, reducing the risk of rate limiting or account flags.
+
+Two complementary pieces:
+
+1. **Session cap** — `--max-conversations N` flag (and
+   `MAX_CONVERSATIONS_PER_RUN` env default). Implementation: in the
+   `export` command, slice the pending list after the cache filter:
+   `to_export = to_export[:n]`. On exit, print exported-vs-remaining
+   counts (reuse the message format from the 429 early-exit path) and
+   remind the user to re-run to continue.
+2. **Polite pacing** — `REQUEST_DELAY` env var (seconds, with small
+   random jitter) slept between per-conversation detail fetches in
+   `BaseProvider`, so even a capped run doesn't fire requests
+   back-to-back. The existing 429 backoff in `_request` stays as the
+   reactive safety net.
+
+Note: the conversation *listing* (paginated, 100/page) still runs in full
+each time so the cache comparison works — the cap applies to the heavy
+per-conversation detail fetches, which dominate request volume.
+
+This is a stepping stone to the StartOS service: a capped, politely-paced
+export — scheduled by the host (cron/StartOS), not an in-app loop — is the
+traffic profile a headless deployment needs.
+
+## 7. Scheduled / Watch Mode — DROPPED (2026-06-28)
+
+An in-app `watch`/scheduler loop is not worth building. Scheduling belongs to
+whatever hosts the tool: a user cron line locally, and on the long-term
+StartOS target the platform's own scheduling. Either way the tool only needs
+to do one capped, politely-paced `export` + `joplin` run and exit — which it
+already does. If cron ergonomics ever feel clunky, a thin `sync` subcommand
+that chains `export` then `joplin` for a single cron line is a trivial
+add-on, but the polling loop itself is off the roadmap.
+
+## 8. StartOS Service Packaging — DROPPED (2026-06-28)
+
+Not pursuing a headless StartOS service. The local, manually-run CLI is
+sufficient: the source conversations live in the providers' clouds and can be
+re-downloaded, and Joplin already syncs (encrypted) to an offsite S3 provider,
+so durability is covered without a server in the loop.
+
+Dropping this also retires the one genuinely hard sub-problem it carried —
+session-token freshness without a browser. There is no headless context to
+keep fresh; the weekly manual DevTools refresh is acceptable. (Local cookie
+extraction remains a dead end regardless — see #2.)
+
+## 9. Token Validity on `doctor` — IMPLEMENTED (2026-06-28)
+
+Shipped: `doctor` now adds a "ChatGPT token active" check via
+`ChatGPTProvider.session_health()` (reads `/api/auth/session`, passes iff
+`error` is falsy and `accessToken` is present), the never-working JWE/`exp`
+decode path was removed, and `_fetch_access_token` now fails fast on a set
+`error` instead of returning a stale token. Tests in
+`tests/test_providers.py::TestChatGPTSessionHealth`. Investigation trail
+below for the record.
+
+Goal: if it's cheap to tell how much longer a token will work, show it on
+`doctor`. Findings from live recon:
+
+- **Not readable from the token itself.** ChatGPT's `CHATGPT_SESSION_TOKEN`
+  is a **JWE** (header `{"alg":"dir","enc":"A256GCM"}`, `eyJ…` prefix is just
+  the encrypted protected header) — the `exp` claim is AES-256-GCM encrypted
+  with an OpenAI-only key, so it cannot be decoded client-side. Claude's
+  `sk-…` key is fully opaque. The existing `doctor` JWT-decode path therefore
+  never yields an expiry for the real tokens (falls to the "not decodable"
+  branch).
+- **`/api/auth/session` exposes an `expires`** (the provider already calls
+  this endpoint in `_fetch_access_token`; the response includes `expires`
+  alongside `accessToken`). Live value observed 2026-06-28:
+  `2026-09-26` — **~90 days out**. This contradicts both the code's ~7-day
+  assumption and the lived weekly-refresh cadence, so it is almost certainly
+  the NextAuth **rolling session window** (re-extended on every call), not
+  the point at which the pasted token actually 401s. Displaying it verbatim
+  would give false confidence.
+- **RESOLVED 2026-06-28 by a live 401 data point.** When the ChatGPT token
+  was actually dead (conversations API → 401), `/api/auth/session` still
+  returned **HTTP 200** with `expires: 2026-09-26` (~90 days out) — and that
+  `expires` *advanced* between two calls seconds apart (`04:20:08` → `04:30:37`).
+  So `expires` is a **rolling session window that rolls forward on every call
+  even for a dead token**; displaying it would actively lie. The same response
+  carried `error: "RefreshAccessTokenError"` and a stale `accessToken`.
+- **The real signal is `error`, not `expires` or `accessToken`.** On a healthy
+  token `error` is absent/null; when the session token is dead NextAuth can't
+  refresh and sets `error: "RefreshAccessTokenError"` while still echoing a
+  rolling `expires` and a stale `accessToken`. This is an exact, free, binary
+  health check.
+- **Design (ready to build):**
+  1. `doctor` ChatGPT check → read `/api/auth/session`; pass iff `error` is
+     falsy and `accessToken` present; on `RefreshAccessTokenError` report
+     "token expired — refresh". Drop the JWE/`exp` decode path (it can never
+     work) and do NOT surface `expires`.
+  2. Latent bug to fix alongside: `_fetch_access_token` reads `accessToken`
+     without checking `error`, so it proceeds with a stale token and yields a
+     confusing downstream 401 instead of a clear "refresh your token" message.
+     Check `error` there and fail fast.
+  3. Claude stays a 401-only signal (opaque `sk-`, no equivalent endpoint).
+
+## 10. Provider API-Drift Detection — IMPLEMENTED (2026-06-28)
+
+Shipped: the `canary` command + `BaseProvider.check_drift()` (overridden by
+ChatGPT and Claude). It fetches one listing page + one conversation per
+provider and asserts only the normalizer's load-bearing fields, emitting
+`DRIFT_OK/WARN/ERROR` findings (`src/providers/base.py`). Severity badges
+print as a Rich table; ERROR exits non-zero, WARN is non-fatal so a backup
+run is never blocked. Drift vocabularies (`_KNOWN_TOOL_AUTHORS`,
+`_HANDLED_CONTENT_TYPES`) live in `chatgpt.py` next to the collapse set they
+guard. Tests: `TestChatGPTDriftCanary`, `TestClaudeDriftCanary`,
+`TestCanaryCommand`. Verified live 2026-06-28 — both providers OK. Recon
+trail below for the record.
+
+## 10b. Provider API-Drift Detection — investigation (2026-06-28)
+
+The export depends on undocumented internal web APIs (ChatGPT/Claude) that can
+change shape without notice. The worst failure for a backup tool is *silent*:
+a response-schema change that makes the exporter skip or mis-parse content
+without erroring. `doctor` currently checks token validity, reachability, and
+manifest↔disk integrity — but not "does the provider's response still look
+like what the parser expects."
+
+To investigate: a lightweight schema/shape assertion on a known-good sample
+of each provider's listing + conversation-detail responses (presence and type
+of the fields the normalizers rely on), surfaced as a `doctor` check or a
+dedicated canary.
+
+**Live recon — Claude captured 2026-06-28 (ChatGPT pending a token refresh):**
+
+- **Dependency surface (assert ONLY these — see below for why):**
+  - listing item: `uuid`, `name`, `updated_at`/`created_at`, `project.name`.
+  - conversation detail: `uuid`/`id`, `name`, `created_at`, `updated_at`,
+    `project.name`, `chat_messages[]`.
+  - message: `sender` (`human`/`assistant`), `text` (string) or `content`
+    (list of typed blocks), `created_at`.
+- **Key finding — full-shape diffing is the wrong design.** Claude's
+  `settings` object is full of volatile internal codenames that churn
+  constantly: `enabled_bananagrams`, `enabled_sourdough`, `enabled_foccacia`,
+  `enabled_saffron`, `enabled_turmeric`, `enabled_monkeys_in_a_barrel`,
+  `paprika_mode`, `enabled_megaminds`, … A "any new/removed key = drift"
+  canary would fire on every UI experiment. The canary MUST target the
+  normalizer's load-bearing fields only, not the whole response. (Aligns with
+  the drop-noise-don't-retain-it principle.)
+- **Real Claude messages are flat `text`/`sender`** — in this archive every
+  message had a string `text` and NO `content` block list (0 rich blocks
+  observed). So `_extract_claude_blocks` / `_dispatch_claude_block` (tool_use,
+  thinking, image, …) is an **unexercised theoretical path**; drift there
+  can't be "caught" by a canary because it never runs on real data — it's a
+  safety net for if Claude ever switches to block content. The canary should
+  assert the flat shape and *warn if `content` ever appears as a list* (that
+  itself is the drift event that would activate the dormant code).
+- **Possible silent-loss spot (separate from drift):** Claude messages carry
+  `attachments` and `files` arrays (empty in this sample) that the normalizer
+  ignores entirely. If a user ever attaches files in Claude, they'd be
+  dropped without a LossReport entry. Worth a follow-up check.
+**Live recon — ChatGPT captured 2026-06-28:**
+
+- **Dependency surface (assert ONLY these):**
+  - listing item: `id`, `title`, `update_time`/`create_time`.
+  - conversation detail: `conversation_id`/`id`, `title`, `create_time`,
+    `update_time`, `mapping` (non-empty).
+  - mapping node: `message`, `children` (the tree walk depends on both);
+    message: `author.role`, `author.name`, `content.content_type`,
+    `content.parts`, `metadata.is_visually_hidden_from_conversation`.
+- **content_type vocabulary observed (all currently handled):** `text`,
+  `model_editable_context`, `multimodal_text`, `thoughts`, `code`,
+  `execution_output`, `reasoning_recap`, `user_editable_context`,
+  `tether_browsing_display`. A *new* content_type already degrades gracefully
+  (visible `unknown` block + WARNING + LossReport tally) — so content_type
+  drift is **already non-silent**. The canary just needs to confirm the known
+  set still parses to non-empty blocks.
+- **The genuinely silent drift risk — `author.name` collapse keys.**
+  `_COLLAPSE_TOOL_AUTHORS = {"file_search", "myfiles_browser"}`. Recon
+  confirms `file_search` is live (and `web.run`/`python` are correctly left
+  un-collapsed). If OpenAI renames `file_search`, the collapse **silently
+  stops** and the archive re-bloats with no error or LossReport entry. This is
+  the top canary target: assert that retrieval-dump tool authors are still
+  recognized, or at least flag unfamiliar `(role="tool", author.name)` pairs.
+- **Second silent risk — empty `content.parts`.** A `text` message whose
+  `parts` field is renamed/emptied yields zero blocks and is skipped with only
+  a debug log = silent loss. Canary should assert a sampled `text` message
+  produces a non-empty block.
+
+**Recon complete for both providers. Canary design (ready to build):** a
+`doctor` check (or dedicated `canary` command) that, per provider, fetches one
+listing page + one conversation and asserts the dependency-surface fields
+above by presence+type — NOT full shape (Claude `settings` codenames prove
+full-shape diffing is pure noise). Specific tripwires: (ChatGPT) unfamiliar
+`(tool, author.name)` pair and empty `parts` on a text message; (Claude)
+`content` appearing as a list, and non-empty `attachments`/`files`. Failures
+surface as a warning, never a hard error (a backup tool must still run).
+
+---
+
+# Deprioritized
+
+Kept for reference; not on the active roadmap.
+
+## Export `--force` Flag — SHIPPED v0.6.0
+
+Implemented 2026-06-12: `export --force` passes `force=True` to
+`cache.get_new_or_updated()`. Shipped alongside a `mark_exported` fix that
+preserves Joplin links across re-exports, so a forced re-render + `joplin`
+updates existing notes instead of duplicating them.
+
+## Joplin `--force` Flag
+
+Similarly, add `--force` to the `joplin` command to re-sync all cached
+conversations to Joplin regardless of whether they've been synced before.
+Useful after making formatting changes to the Markdown exporter.
+
+Implementation: in `get_joplin_pending()`, return all entries that have a
+`file_path` when `force=True`, ignoring `joplin_synced_at`.
+
+## Per-Conversation Cache Reset
+
+Add `cache --reset --conversation <id>` to force re-export or re-sync of a
+single conversation without clearing the entire provider cache.
+
+Current workaround: manually edit `~/.ai-chat-exporter/manifest.json` and
+delete the entry, then re-run export.
+
+## Official API Fallback
 
-## Official API Migration (v0.3.0)
 If the unofficial internal web API approach breaks, migrate to official export
 file parsing as a fallback:
 - ChatGPT: parse `conversations.json` from Settings → Export Data
@@ -44,29 +481,58 @@ file parsing as a fallback:
 
 The `BaseProvider` abstract class is intentionally designed so that a
 `FileProvider` subclass can implement the same interface
-(list_conversations, get_conversation, normalize_conversation)
+(`list_conversations`, `get_conversation`, `normalize_conversation`)
 without any changes to cache, exporters, or CLI code.
 
 To add this: implement `src/providers/file_chatgpt.py` and
 `src/providers/file_claude.py`, then add `--input-file` flag to the
 export command to accept a pre-downloaded export ZIP or JSON.
 
-## Rich Content Support (v0.4.0)
-Currently only text content is exported. Future versions should handle:
+Deprioritized 2026-06-12: the official ChatGPT export does not cover what
+this user needs (project data), so it isn't a real fallback here.
 
-### Claude
-- Artifacts (code, documents, HTML) — export as separate files, link from Markdown
-- Uploaded images — download and embed or link
-- Extended thinking/reasoning blocks — include as collapsible sections
-- Tool call results and web search citations — include as footnotes or appendices
+## Reclassify o1/o3 Reasoning Subparts
 
-### ChatGPT
-- DALL-E generated images — download and embed or link
-- Code Interpreter outputs — export code and results
-- File attachments — download and reference
-- Voice transcripts — include as text
+v0.4.0 leaves dict parts inside `text` content_type messages with shape
+`{"summary": ..., "content": ...}` rendered as plain text (defensive — the
+shape was inferred from a code comment, not captured live). Once a real
+reasoning conversation is captured, reclassify these as `thinking` blocks.
 
-Implementation note: the normalized message schema already includes a
-`content_type` field placeholder. When this work begins, extend the schema
-rather than replacing it. In v0.1.0, log a WARNING whenever non-text content
-is encountered so users know what was skipped.
+## Obsidian Vault Output
+
+Add an `obsidian` command (or `--target obsidian` flag) to sync exported
+conversations into an Obsidian vault directory. The current Markdown format
+is already largely compatible; the main differences are:
+
+- Obsidian uses YAML frontmatter `properties` (same format, already supported)
+- Tags should use `#tag` inline or `tags:` list in frontmatter (already done)
+- Wikilinks (`[[Title]]`) instead of Markdown links — optional, Obsidian
+  supports both
+
+Implementation: the existing `MarkdownExporter` output is already valid in
+Obsidian. An `ObsidianSyncer` class (mirroring `JoplinClient`) would simply
+copy files to the vault directory and maintain a flat or nested folder
+structure matching the user's Obsidian setup. No API needed — just file I/O.
+
+## Token Expiry Notifications
+
+Moved to the active roadmap as §9 (Token Validity on `doctor`). The original
+"proactively notify before expiry" idea is blocked by the same finding: a
+reliable expiry time isn't available client-side (ChatGPT token is encrypted,
+Claude's is opaque, and `/api/auth/session`'s `expires` looks like a rolling
+window rather than the real refresh cadence). Any heads-up — a `doctor`
+line, an `expiry` subcommand, or a `notify-send` nudge — depends on first
+resolving the §9 open question of what signal is actually trustworthy.
+
+## Search Command
+
+Add a `search` command to full-text search across all exported Markdown files:
+
+```bash
+python -m src.main search "kubernetes ingress"
+python -m src.main search "kubernetes ingress" --provider claude --project devops
+```
+
+Implementation: `grep`/`ripgrep` over `EXPORT_DIR`, display results with
+conversation title, date, and a snippet. No index needed — Markdown files are
+small enough to grep directly.

README.md

@@ -0,0 +1,530 @@
+# AI Chat Exporter
+
+A personal backup tool for ChatGPT and Claude conversation history. Exports your chats to Markdown files and syncs them to [Joplin](https://joplinapp.org/) as notes. Each conversation becomes a single `.md` file with YAML frontmatter, organised into folders that map directly to Joplin notebooks.
+
+Supports incremental sync — only new or updated conversations are exported on each run. Every run is resumable: if interrupted, re-running picks up exactly where it left off.
+
+---
+
+## ⚠️ Terms of Service Warning
+
+**Read this before using this tool.**
+
+This tool works by accessing **unofficial, undocumented internal web API endpoints** used by the ChatGPT and Claude web apps. These endpoints are not publicly supported by OpenAI or Anthropic and are subject to change or removal without notice.
+
+**Use of this tool may conflict with their Terms of Service:**
+- OpenAI: https://openai.com/policies/terms-of-use
+- Anthropic: https://www.anthropic.com/legal/consumer-terms
+
+**By using this tool, you accept that:**
+- You are using it entirely at your own risk
+- Your account could potentially be suspended for automated or scripted access
+- The internal APIs this tool relies on may break at any time without notice
+- This tool is for **personal archival use only** — not commercial use
+
+This tool is designed for a single user backing up their own conversations. Do not use it to scrape data at scale or for any commercial purpose.
+
+---
+
+## Installation
+
+### Linux / macOS
+
+```bash
+git clone <repo-url>
+cd ai-chat-exporter
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+### Windows
+
+No admin access required. Run these in **Command Prompt** (`cmd.exe`) — it's the simplest option on Windows because it doesn't have PowerShell's script execution policy restrictions.
+
+```bat
+git clone <repo-url>
+cd ai-chat-exporter
+python -m venv .venv
+.venv\Scripts\activate
+pip install -e ".[dev]"
+```
+
+All `ai-chat-exporter` commands work identically in Command Prompt.
+
+**Using PowerShell instead?** If you prefer PowerShell, you may need to allow script execution first (one-time, current user only):
+
+```powershell
+Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
+```
+
+Then activate the venv and run commands the same way.
+
+**Prerequisites:**
+- Python 3.11 or later — install from [python.org](https://www.python.org/downloads/windows/). During installation, tick **"Add Python to PATH"**.
+- Git — install from [git-scm.com](https://git-scm.com/) if not already present.
+
+**Notes:**
+- The cache manifest and logs are stored in `cache\` inside the install directory — the same as on Linux.
+- File permission hardening (`chmod 600`) is silently ignored on Windows — not a concern for single-user desktop use.
+- Joplin Web Clipper runs on `localhost:41184` on all platforms; no configuration changes needed.
+
+---
+
+## First Run: Run Doctor
+
+Before anything else, validate your setup:
+
+```bash
+ai-chat-exporter doctor
+```
+
+This checks token presence, format, token health (via the `/api/auth/session` `error` field), directory permissions, disk space, and live API connectivity. Fix any failures before proceeding.
+
+### Checking for API drift
+
+The exporter reads ChatGPT's and Claude's undocumented internal web APIs, which can change shape without notice. The worst failure for a backup tool is *silent* — a response change that makes the exporter skip or mis-parse content without erroring. Run the canary to catch that early:
+
+```bash
+ai-chat-exporter canary
+```
+
+It probes one conversation per provider and checks only the fields the parser depends on (a renamed retrieval-tool author that would bypass the content-collapse, a new content type, drifted message fields, ignored attachments). `OK` means the live schema still matches; `WARN`/`ERROR` tells you exactly what changed. Worth running before a large export or on a schedule.
+
+---
+
+## Getting Your Session Tokens
+
+Session tokens are how your browser stays logged in. This tool uses them to access your chat history on your behalf.
+
+Tokens are entered manually — copy them from your browser's DevTools and run the wizard:
+
+```bash
+ai-chat-exporter auth
+```
+
+The wizard detects your OS, shows the correct DevTools shortcut, and writes the values to `.env` without echoing them to the terminal.
+
+> **Why no auto-extraction from the browser?** Modern Chromium browsers (Chrome 127+, current Brave) encrypt cookies on Windows with **App-Bound Encryption**: the key is bound to the browser through a SYSTEM-level service, so reading cookies off disk requires Administrator rights and a PsExec-style SYSTEM impersonation that antivirus flags as credential theft — and it still fails on Brave specifically. There is no reliable, non-invasive way to pull these cookies automatically, so the manual DevTools flow below is the supported path.
+
+### Token Lifetimes
+
+| Provider | Cookie Name | Lifetime | Expiry Detection |
+|----------|-------------|----------|-----------------|
+| ChatGPT | `__Secure-next-auth.session-token.0` + `.1` | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. |
+| Claude | `sessionKey` | ~30 days | Opaque token — only detectable via 401 response |
+
+### Finding Tokens in Chrome DevTools
+
+1. Open the provider's website and make sure you're logged in
+2. Press **F12** (Windows/Linux) or **Cmd+Option+I** (macOS) to open DevTools
+3. Click the **Application** tab
+4. In the left panel, expand **Cookies** and click the site URL
+5. Find the cookie by name and copy its **Value**
+
+**ChatGPT:** go to `https://chatgpt.com` → find **two** cookies:
+- `__Secure-next-auth.session-token.0` — copy Value (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN`
+- `__Secure-next-auth.session-token.1` — copy Value → `CHATGPT_SESSION_TOKEN_1`
+
+ChatGPT splits large session tokens across two cookies to stay under the browser's 4KB cookie limit. Both are required.
+
+**Claude:** go to `https://claude.ai` → find `sessionKey` → copy Value
+
+### When Tokens Expire
+
+When a token expires you'll see a `401 Unauthorized` error. To refresh:
+- Re-run the `auth` wizard: `ai-chat-exporter auth`
+- Or manually update the value in your `.env` file
+
+---
+
+## The `auth` Command
+
+The easiest way to configure tokens is the interactive wizard:
+
+```bash
+ai-chat-exporter auth
+```
+
+This walks you through finding your token, validates it, shows the expiry date (ChatGPT only), and offers to write it to your `.env` automatically. Tokens are never echoed to the terminal.
+
+---
+
+## `.env` Setup
+
+Copy `.env.example` to `.env` and fill in your values:
+
+```bash
+cp .env.example .env
+```
+
+### Provider tokens
+
+| Variable | Description |
+|----------|-------------|
+| `CHATGPT_SESSION_TOKEN` | ChatGPT session token chunk `.0` (starts with `eyJ…`) |
+| `CHATGPT_SESSION_TOKEN_1` | ChatGPT session token chunk `.1` (the remainder) |
+| `CHATGPT_PROJECT_IDS` | Comma-separated ChatGPT project IDs (see below) |
+| `CLAUDE_SESSION_KEY` | Your Claude session key |
+
+### Output
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `EXPORT_DIR` | `./exports` | Where to write exported Markdown files |
+| `OUTPUT_STRUCTURE` | `provider/project/year` | Folder structure (see below) |
+| `EXPORTER_HIDDEN_CONTENT` | `placeholder` | What to do with content invisible in the provider's web UI (file-retrieval tool dumps, Custom Instructions): `placeholder` collapses to a one-line note with size, `full` keeps everything, `omit` drops it. Dumps can be 90% of a conversation's bytes. |
+| `EXPORTER_DOWNLOAD_MEDIA` | `images` | Download conversation assets into a `media/` folder beside each export and inline them: `images` (images only), `all` (also audio/voice clips), `off` (text placeholders only). Downloaded media is uploaded to Joplin as resources on the next `joplin` run. |
+| `MAX_CONVERSATIONS_PER_RUN` | unlimited | Cap downloads per export run (per provider). Runs are resumable, so re-running continues where the cap stopped — useful to spread a large first export across several sessions. |
+| `REQUEST_DELAY` | `1.0` | Seconds between consecutive API requests, with small random jitter, so traffic stays human-paced. Set `0` to disable. |
+
+### Joplin
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `JOPLIN_API_TOKEN` | — | Authorization token from Joplin Web Clipper settings |
+| `JOPLIN_API_URL` | `http://localhost:41184` | Joplin API URL (change only if you've customised the port) |
+| `JOPLIN_REQUEST_TIMEOUT` | `30` | Seconds before an API call times out. Increase for very large conversations. |
+
+### Cache & logging
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `CACHE_DIR` | `./cache` | Where to store the sync manifest |
+| `LOG_FILE` | `./cache/logs/exporter.log` | Log file path (`none` to disable) |
+
+---
+
+## ChatGPT Projects
+
+ChatGPT project conversations are stored separately from your main conversation list and require extra configuration.
+
+### Finding your project IDs
+
+1. Open ChatGPT and click a Project in the left sidebar
+2. Look at the browser URL — it will look like:
+   `https://chatgpt.com/g/g-p-68c2b2b3037c8191890036fb4ae3ed9f-my-project/project`
+3. Copy the `g-p-…` part (everything up to but not including the slug after the second `-`)
+
+Add all your project IDs to `.env` as a comma-separated list:
+
+```
+CHATGPT_PROJECT_IDS=g-p-68c2b2b3037c8191890036fb4ae3ed9f,g-p-anotherprojectid
+```
+
+The `auth` wizard can also guide you through this step interactively.
+
+---
+
+## Claude Code Sessions
+
+The `claude-code` provider archives your local [Claude Code](https://claude.com/claude-code) agent transcripts — no tokens, no API, no ToS exposure. Sessions are read from `~/.claude/projects/` (override with `CLAUDE_CODE_DIR` in `.env`).
+
+```bash
+ai-chat-exporter export --provider claude-code
+ai-chat-exporter joplin --provider claude-code
+```
+
+Exports are prose-only by default: your prompts and Claude's write-ups are kept, tool activity is grouped into one-line placeholders (`> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted`), and internal reasoning is dropped (counted in the run summary). Set `EXPORTER_HIDDEN_CONTENT=full` to keep everything. Each coding project becomes a Joplin notebook under the `AI-Claude` parent. The provider is included in `--provider all` whenever the sessions directory exists.
+
+---
+
+## Output Structure
+
+All exported files go under `EXPORT_DIR`. The folder structure maps directly to Joplin notebooks.
+
+### Default: `provider/project/year`
+
+```
+exports/
+├── chatgpt/
+│   ├── no-project/
+│   │   └── 2024/
+│   │       └── 2024-03-15_my-conversation_abc12345.md
+│   └── learning-python/
+│       └── 2024/
+│           └── 2024-03-15_async-tutorial_def67890.md
+└── claude/
+    ├── no-project/
+    │   └── 2024/
+    │       └── 2024-06-01_docker-explained_ghi11111.md
+    └── startos-packaging/
+        └── 2024/
+            └── 2024-06-10_manifest-setup_jkl22222.md
+```
+
+### Joplin Notebook Mapping
+
+Each provider+project combination maps to a flat Joplin notebook created automatically by the `joplin` command:
+
+| Export folder | Joplin notebook |
+|---------------|-----------------|
+| `exports/chatgpt/learning-python/` | `ChatGPT - Learning Python` |
+| `exports/claude/startos-packaging/` | `Claude - Startos Packaging` |
+| `exports/chatgpt/no-project/` | `ChatGPT - No Project` |
+| `exports/claude/no-project/` | `Claude - No Project` |
+
+### Other `OUTPUT_STRUCTURE` options
+
+| Value | Result |
+|-------|--------|
+| `provider/project/year` (default) | `exports/claude/my-project/2024/file.md` |
+| `provider/project` | `exports/claude/my-project/file.md` |
+| `provider/year` | `exports/claude/2024/file.md` (projects ignored) |
+
+### Filename format
+
+`YYYY-MM-DD_{title-slug}_{id[:8]}.md` — e.g. `2024-06-10_manifest-setup_jkl22222.md`
+
+---
+
+## CLI Reference
+
+### Global flags
+
+```
+--verbose / -v     DEBUG output to console
+--quiet / -q       WARNING and above only
+--debug            DEBUG + full tracebacks + redacted API response bodies
+--no-log-file      Disable file logging
+--version          Print version and exit
+```
+
+### `auth` — Interactive token setup
+
+```bash
+ai-chat-exporter auth                  # manual wizard (DevTools flow)
+```
+
+Guided wizard to find and save session tokens and ChatGPT project IDs. Detects OS and shows the correct DevTools shortcut, then writes the values to `.env`.
+
+### `doctor` — Health check
+
+```bash
+ai-chat-exporter doctor
+```
+
+Checks: token presence, JWT validity and expiry, directory permissions, disk space, live API reachability. Exits with code 0 if all pass, 1 if any fail.
+
+### `export` — Export conversations
+
+```bash
+# Export everything (new/updated only)
+ai-chat-exporter export
+
+# Single provider
+ai-chat-exporter export --provider claude
+
+# JSON output
+ai-chat-exporter export --format json
+
+# Both Markdown and JSON
+ai-chat-exporter export --format both
+
+# Only conversations updated since a date
+ai-chat-exporter export --since 2024-06-01
+
+# Only conversations in a specific project (case-insensitive substring)
+ai-chat-exporter export --project "learning python"
+
+# Only conversations outside any project
+ai-chat-exporter export --project none
+
+# Write to a custom directory
+ai-chat-exporter export --output /path/to/my/notes
+
+# Preview without writing anything
+ai-chat-exporter export --dry-run
+```
+
+Options: `--provider [chatgpt|claude|claude-code|all]`, `--format [markdown|json|both]`, `--output PATH`, `--since YYYY-MM-DD`, `--project NAME`, `--hidden-content [full|placeholder|omit]`, `--download-media [images|all|off]`, `--max-conversations N`, `--force`, `--dry-run`
+
+**Re-rendering the whole archive after an upgrade.** New formatting or features (collapse policy, media downloads) only change conversations as they're re-exported. To re-render everything you already have, use `--force` — it re-exports every conversation even if unchanged, **without** `cache --clear`, so your Joplin note links are preserved (a later `joplin` run updates the existing notes instead of duplicating them).
+
+A force re-render runs as a tracked "campaign": each run re-renders the least-recently-exported conversations, the "still to go" count shrinks each run, and finished providers do no further work. The simplest approach is one uncapped pass:
+
+```bash
+ai-chat-exporter export --force      # re-renders everything in one paced pass
+ai-chat-exporter joplin              # update notes + upload media
+```
+
+Or spread the load with `--max-conversations`, re-running until it reports "Force re-render complete":
+
+```bash
+ai-chat-exporter export --force --max-conversations 50   # repeat until complete
+```
+
+### `list` — List conversations
+
+```bash
+# List all conversations for all providers
+ai-chat-exporter list
+
+# Single provider
+ai-chat-exporter list --provider chatgpt
+
+# Filter by project
+ai-chat-exporter list --project "learning python"
+
+# Only conversations outside any project
+ai-chat-exporter list --project none
+```
+
+Fetches and displays all conversations without exporting them. Useful for verifying what the tool can see before running an export.
+
+### `joplin` — Sync to Joplin
+
+```bash
+# Sync all pending conversations to Joplin
+ai-chat-exporter joplin
+
+# Preview what would be synced without sending anything
+ai-chat-exporter joplin --dry-run
+
+# Sync a single provider
+ai-chat-exporter joplin --provider chatgpt
+
+# Sync only conversations in a specific project
+ai-chat-exporter joplin --project "learning python"
+
+# Sync only conversations outside any project
+ai-chat-exporter joplin --project none
+```
+
+Reads the local export cache and pushes each exported Markdown file to Joplin as a note. Notebooks are created automatically. Re-running is safe — notes are updated (not duplicated).
+
+**Prerequisites:**
+1. Run `export` first to generate the Markdown files
+2. Open Joplin → Tools → Options → Web Clipper → enable the service
+3. Copy the Authorization token and add `JOPLIN_API_TOKEN=<token>` to your `.env`
+4. Joplin desktop must be open when you run this command
+
+Options: `--provider [chatgpt|claude|all]`, `--project NAME`, `--dry-run`
+
+### `prune` — Delete stale export files
+
+```bash
+# Preview what would be deleted
+ai-chat-exporter prune --dry-run
+
+# Delete (asks for confirmation; -y skips the prompt)
+ai-chat-exporter prune
+```
+
+Deletes export files no longer referenced by the cache manifest — leftovers
+from old folder layouts or fixed bugs that would otherwise double-sync into
+Joplin. Refuses to run when the manifest is empty (e.g. right after
+`cache --clear`) so it can never wipe a freshly cleared archive. The `doctor`
+command separately verifies that every manifest entry's file exists on disk.
+
+### `cache` — Manage the sync manifest
+
+```bash
+# Show statistics
+ai-chat-exporter cache --show
+
+# Clear all cached entries (forces full re-export next run)
+ai-chat-exporter cache --clear
+
+# Clear a single provider
+ai-chat-exporter cache --clear --provider claude
+```
+
+---
+
+## How the Cache Works
+
+The cache manifest lives at `cache/manifest.json` (inside the install directory) and records every exported conversation: its title, project, `updated_at` timestamp, output file path, and (after Joplin sync) the Joplin note ID.
+
+On every `export` run:
+1. Fetch the full conversation list from the provider
+2. Compare each conversation's `updated_at` against the manifest
+3. Export only conversations that are new or have been updated
+4. Write each successfully exported conversation to the manifest **immediately** (not batched)
+
+On every `joplin` run:
+1. Read the manifest to find conversations not yet synced to Joplin, or re-exported since last sync
+2. Push each pending Markdown file to Joplin (create or update)
+3. Store the Joplin note ID in the manifest so subsequent runs update rather than duplicate
+
+**This design makes every run inherently resumable.** If the tool is interrupted for any reason — rate limit, network drop, Ctrl+C, crash — simply re-run the same command. It will skip already-processed conversations and continue from where it stopped.
+
+To force a full re-export: `ai-chat-exporter cache --clear` then re-run export.
+
+---
+
+## Troubleshooting
+
+### `401 Unauthorized`
+Your session token has expired.
+- Run `ai-chat-exporter auth` to get a new token interactively
+- Or manually copy a fresh cookie value into your `.env` file
+
+Note: Claude's `sessionKey` is an opaque string — the only way to know it's expired is the 401 error. ChatGPT JWTs have an `exp` claim that the `doctor` command can decode and display.
+
+### `429 Rate Limited`
+The tool automatically pauses, saves progress, and exits with a clear message showing how many conversations were exported vs remaining. Just re-run the same export command to resume — the cache picks up exactly where it left off.
+
+### Joplin: "JOPLIN_API_TOKEN is not set"
+You need to configure the token before running the `joplin` command:
+1. Open Joplin desktop
+2. Go to Tools → Options → Web Clipper
+3. Enable the Web Clipper service
+4. Copy the Authorization token shown on that page
+5. Add `JOPLIN_API_TOKEN=<token>` to your `.env` file
+
+### Joplin: "Joplin is not responding"
+Joplin desktop must be running when you run the `joplin` command. The Web Clipper service shuts down when Joplin is closed.
+
+### Joplin: "Joplin rejected the API token (HTTP 401)"
+The token in `JOPLIN_API_TOKEN` doesn't match what Joplin expects. Get a fresh token from Joplin → Tools → Options → Web Clipper → Authorization token.
+
+### Joplin: note timed out
+If you see a timeout error, Joplin took longer than `JOPLIN_REQUEST_TIMEOUT` seconds (default: 30) to respond. Possible causes:
+- The conversation is very large and Joplin is slow to index it
+- Joplin is busy syncing or loading a large library
+- Joplin has frozen — try restarting it
+
+To increase the timeout: add `JOPLIN_REQUEST_TIMEOUT=60` to your `.env`.
+
+### ChatGPT project conversations not appearing
+Make sure you've added the project IDs to `CHATGPT_PROJECT_IDS` in your `.env`. See [ChatGPT Projects](#chatgpt-projects) for how to find them. Project conversations are not included in the default conversation listing — they must be fetched separately.
+
+### Schema warnings in logs (`Unexpected API response shape`)
+The provider's internal API may have changed. Run with `--debug`, sanitize the output (remove any personal content), and check the project's GitHub Issues for known fixes.
+
+### Non-text content warnings
+Since v0.4.0, rich content is preserved as typed blocks in the export. ChatGPT voice transcripts render as text and audio assets as `📎 File attached` placeholders with size and duration metadata. Anything the extractor doesn't recognise renders as a visible `> ⚠️ Unsupported content` block naming the type and observed keys, *and* increments a counter in the post-export summary so you can tell whether real content is being silently skipped.
+
+### Embedded images and media
+Since v0.6.0, image attachments are downloaded by default into a `media/` folder beside each export and inlined as `![](media/…)`. Audio and other files download only with `EXPORTER_DOWNLOAD_MEDIA=all`. On the next `joplin` run these are uploaded as Joplin resources so they render inside the note. Assets that have expired on the provider's side (older AI-generated images often do) keep their text placeholder and are tallied as `media failed` in the run summary — they're never fatal to the export. Set `EXPORTER_DOWNLOAD_MEDIA=off` to skip downloads entirely.
+
+### Tool output / hidden context collapsed
+Since v0.6.0, content that was invisible in the ChatGPT web UI is collapsed by default to one-line placeholders like `> 🔧 Tool output — file_search (15.0 KB) — omitted`. This is ChatGPT's file-retrieval tool re-injecting your attached files on every run — it can be 90% of a conversation's bytes while containing none of the dialogue. Custom Instructions collapse the same way (`> ℹ️ Hidden context`). The post-export summary lists every collapsed message by origin with the total KB omitted. To keep everything, set `EXPORTER_HIDDEN_CONTENT=full` in `.env` or pass `--hidden-content full`.
+
+### Empty export / all conversations skipped
+No new or updated conversations since your last run. To verify: `ai-chat-exporter cache --show`. To force a full re-export: `ai-chat-exporter cache --clear`.
+
+### Filing a bug report
+1. Run with `--debug`: `ai-chat-exporter export --debug 2>&1 | tee debug.log`
+2. Remove any personal conversation content from `debug.log`
+3. Open a GitHub Issue with the sanitized log and the exact command you ran
+
+---
+
+## Future Work
+
+See `FUTURE.md` for the full roadmap. Current priorities:
+
+- **Watch/scheduled mode** on the way to a headless StartOS service
+
+---
+
+## Security Notes
+
+- All exported data is stored **locally only** — nothing is sent anywhere except to your local Joplin instance
+- Exported files and the cache manifest are created with `600` permissions (owner read/write only)
+- `.env` is in `.gitignore` — **never commit it**
+- Session tokens are never logged, printed, or included in error messages
+- The Joplin API token is only ever sent to `localhost` — it never leaves your machine
+- If you accidentally commit `.env`: immediately log out and back in to invalidate the token, then remove it from git history using [BFG Repo Cleaner](https://rtyley.github.io/bfg-repo-cleaner/) or `git filter-branch`

pyproject.toml

@@ -4,11 +4,12 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ai-chat-exporter"
-version = "0.1.0"
+version = "0.7.0"
 description = "Export ChatGPT and Claude conversation history to Markdown for personal archival in Joplin"
 requires-python = ">=3.11"
 dependencies = [
     "requests==2.31.0",
+    "curl_cffi==0.14.0",
     "click==8.1.7",
     "python-dotenv==1.0.1",
     "rich==13.7.1",

requirements.txt

@@ -1,14 +1,17 @@
 # Editable Git install with no remote (ai-chat-exporter==0.1.0)
 -e /home/jesse/services/ai-chatexport
 certifi==2026.2.25
+cffi==2.0.0
 charset-normalizer==3.4.4
 click==8.1.7
+curl_cffi==0.14.0
 idna==3.11
 iniconfig==2.3.0
 markdown-it-py==4.0.0
 mdurl==0.1.2
 packaging==26.0
 pluggy==1.6.0
+pycparser==3.0
 Pygments==2.19.2
 PyJWT==2.8.0
 pytest==8.1.1

src/blocks.py

@@ -0,0 +1,399 @@
+"""Typed content blocks for normalized messages.
+
+Providers produce ordered lists of blocks; exporters render them. Living outside
+``src/providers/`` deliberately — blocks are a separate concern from extraction
+or rendering, shared by both layers.
+
+Block dicts always have ``type`` set to one of the BLOCK_TYPE_* constants.
+Construct via the ``make_*`` helpers; never build dicts by hand. The ``unknown``
+block constructor REQUIRES a corresponding WARNING log + ``LossReport`` tally
+at the call site — see plan §Data-loss visibility.
+"""
+
+import json
+from pathlib import Path
+from typing import Any
+
+BLOCK_TYPE_TEXT = "text"
+BLOCK_TYPE_CODE = "code"
+BLOCK_TYPE_THINKING = "thinking"
+BLOCK_TYPE_TOOL_USE = "tool_use"
+BLOCK_TYPE_TOOL_RESULT = "tool_result"
+BLOCK_TYPE_CITATION = "citation"
+BLOCK_TYPE_IMAGE_PLACEHOLDER = "image_placeholder"
+BLOCK_TYPE_FILE_PLACEHOLDER = "file_placeholder"
+BLOCK_TYPE_UNKNOWN = "unknown"
+BLOCK_TYPE_HIDDEN_CONTEXT_MARKER = "hidden_context_marker"
+BLOCK_TYPE_COLLAPSED = "collapsed"
+
+COLLAPSED_KIND_TOOL_DUMP = "tool_dump"
+COLLAPSED_KIND_HIDDEN_CONTEXT = "hidden_context"
+
+UNKNOWN_REASON_UNKNOWN_TYPE = "unknown_type"
+UNKNOWN_REASON_EXTRACTION_FAILED = "extraction_failed"
+UNKNOWN_REASON_ALL_BLOCKS_FAILED = "all_blocks_failed"
+UNKNOWN_REASON_UNKNOWN_FIELD_IN_KNOWN_TYPE = "unknown_field_in_known_type"
+
+_OBSERVED_KEYS_LIMIT = 10
+
+
+# ---------------------------------------------------------------------------
+# Constructors
+# ---------------------------------------------------------------------------
+
+
+def make_text_block(text: str) -> dict | None:
+    """Return a text block, or None if the text is empty/whitespace-only.
+
+    Returning None lets callers do ``if block: blocks.append(block)`` and prune
+    empty blocks at construction time. See plan §Finalizing the message dict.
+    """
+    if not isinstance(text, str) or not text.strip():
+        return None
+    return {"type": BLOCK_TYPE_TEXT, "text": text}
+
+
+def make_code_block(code: str, language: str = "") -> dict | None:
+    """Return a code block, or None if code is empty."""
+    if not isinstance(code, str) or not code.strip():
+        return None
+    return {"type": BLOCK_TYPE_CODE, "language": language or "", "code": code}
+
+
+def make_thinking_block(text: str) -> dict | None:
+    """Return a thinking block, or None if empty."""
+    if not isinstance(text, str) or not text.strip():
+        return None
+    return {"type": BLOCK_TYPE_THINKING, "text": text}
+
+
+def make_tool_use_block(name: str, input_data: Any, tool_id: str | None = None) -> dict:
+    """Return a tool_use block.
+
+    Always returns a block (no None) — tool calls are meaningful even with
+    empty inputs.
+    """
+    return {
+        "type": BLOCK_TYPE_TOOL_USE,
+        "name": name or "",
+        "input": input_data if input_data is not None else {},
+        "tool_id": tool_id,
+    }
+
+
+def make_tool_result_block(
+    output: str,
+    tool_name: str | None = None,
+    is_error: bool = False,
+    summary: str | None = None,
+) -> dict:
+    """Return a tool_result block.
+
+    ``summary`` is an optional short human label rendered between header and
+    fence (e.g. ChatGPT's ``metadata.reasoning_title`` for execution_output).
+    """
+    return {
+        "type": BLOCK_TYPE_TOOL_RESULT,
+        "tool_name": tool_name,
+        "output": output if isinstance(output, str) else str(output),
+        "is_error": bool(is_error),
+        "summary": summary,
+    }
+
+
+def make_citation_block(
+    url: str,
+    title: str | None = None,
+    snippet: str | None = None,
+) -> dict | None:
+    if not url:
+        return None
+    return {
+        "type": BLOCK_TYPE_CITATION,
+        "url": url,
+        "title": title,
+        "snippet": snippet,
+    }
+
+
+def make_image_placeholder(
+    ref: str,
+    source: str = "unknown",
+    mime: str | None = None,
+) -> dict:
+    """source ∈ {'user_upload', 'model_generated', 'unknown'}."""
+    return {
+        "type": BLOCK_TYPE_IMAGE_PLACEHOLDER,
+        "ref": ref or "",
+        "source": source,
+        "mime": mime,
+    }
+
+
+def make_file_placeholder(
+    ref: str,
+    filename: str | None = None,
+    mime: str | None = None,
+    size_bytes: int | None = None,
+    duration_seconds: float | None = None,
+) -> dict:
+    return {
+        "type": BLOCK_TYPE_FILE_PLACEHOLDER,
+        "ref": ref or "",
+        "filename": filename,
+        "mime": mime,
+        "size_bytes": size_bytes,
+        "duration_seconds": duration_seconds,
+    }
+
+
+def make_unknown_block(
+    raw_type: str,
+    observed_keys: list[str] | None = None,
+    reason: str = UNKNOWN_REASON_UNKNOWN_TYPE,
+    summary: str | None = None,
+) -> dict:
+    """Construct an unknown block.
+
+    Every call site MUST also emit a WARNING log and increment a LossReport
+    tally — see plan §Data-loss visibility. The block surfaces the loss at
+    read time; the WARNING surfaces it at run time. Both signals matter.
+    """
+    keys = list(observed_keys or [])[:_OBSERVED_KEYS_LIMIT]
+    return {
+        "type": BLOCK_TYPE_UNKNOWN,
+        "raw_type": raw_type,
+        "observed_keys": keys,
+        "reason": reason,
+        "summary": summary,
+    }
+
+
+def make_collapsed_block(
+    origin: str,
+    content_type: str,
+    size_bytes: int,
+    kind: str,
+) -> dict:
+    """One-line stand-in for a message omitted by the EXPORTER_HIDDEN_CONTENT policy.
+
+    ``kind`` ∈ {COLLAPSED_KIND_TOOL_DUMP, COLLAPSED_KIND_HIDDEN_CONTEXT}.
+    Unlike ``unknown`` blocks (unexpected loss), a collapsed block records an
+    intentional policy decision — the content was invisible in the provider's
+    web UI and the user chose not to keep it. Every call site MUST tally via
+    ``LossReport.record_collapsed`` so the omission stays visible in the
+    post-export summary.
+    """
+    return {
+        "type": BLOCK_TYPE_COLLAPSED,
+        "origin": origin or "",
+        "content_type": content_type or "",
+        "size_bytes": int(size_bytes or 0),
+        "kind": kind,
+    }
+
+
+def make_hidden_context_marker(content_type: str) -> dict:
+    """A short prepend block that flags the surrounding message as hidden context.
+
+    Driven by the ``metadata.is_visually_hidden_from_conversation`` flag, not by
+    content_type matching. The marker tells the reader "this message is
+    hidden in the source UI; we're showing it here for archival fidelity."
+    """
+    return {
+        "type": BLOCK_TYPE_HIDDEN_CONTEXT_MARKER,
+        "content_type": content_type or "",
+    }
+
+
+# ---------------------------------------------------------------------------
+# Rendering
+# ---------------------------------------------------------------------------
+
+
+def render_blocks_to_markdown(blocks: list[dict]) -> str:
+    """Render an ordered list of blocks to a single Markdown string.
+
+    Blocks are joined with one blank line between them. Pure function; no I/O.
+    """
+    if not blocks:
+        return ""
+
+    rendered: list[str] = []
+    for block in blocks:
+        chunk = _render_one(block)
+        if chunk:
+            rendered.append(chunk)
+
+    return "\n\n".join(rendered)
+
+
+def _render_one(block: dict) -> str:
+    btype = block.get("type", "")
+    if btype == BLOCK_TYPE_TEXT:
+        return block.get("text", "")
+    if btype == BLOCK_TYPE_CODE:
+        lang = block.get("language") or ""
+        code = block.get("code", "")
+        fence = _safe_fence(code)
+        return f"{fence}{lang}\n{code}\n{fence}"
+    if btype == BLOCK_TYPE_THINKING:
+        text = block.get("text", "")
+        quoted = _blockquote_prefix(text)
+        return f"**💭 Reasoning**\n\n{quoted}"
+    if btype == BLOCK_TYPE_TOOL_USE:
+        name = block.get("name", "")
+        input_data = block.get("input", {})
+        body_json = json.dumps(input_data, indent=2, sort_keys=False, default=str, ensure_ascii=False)
+        fence = _safe_fence(body_json)
+        body = f"{fence}json\n{body_json}\n{fence}"
+        quoted = _blockquote_prefix(f"🔧 **Tool: {name}**\n{body}")
+        return quoted
+    if btype == BLOCK_TYPE_TOOL_RESULT:
+        output = block.get("output", "")
+        is_error = bool(block.get("is_error"))
+        tool_name = block.get("tool_name") or ""
+        summary = block.get("summary") or ""
+        icon = "❌" if is_error else "📤"
+        label = "Result (error)" if is_error else "Result"
+        if tool_name:
+            header = f"{icon} **{label}: {tool_name}**"
+        else:
+            header = f"{icon} **{label}**"
+        fence = _safe_fence(output)
+        body = f"{fence}\n{output}\n{fence}"
+        if summary:
+            inner = f"{header}\n*{summary}*\n{body}"
+        else:
+            inner = f"{header}\n{body}"
+        return _blockquote_prefix(inner)
+    if btype == BLOCK_TYPE_CITATION:
+        url = block.get("url", "")
+        title = block.get("title") or url
+        return f"[{title}]({url})"
+    if btype == BLOCK_TYPE_IMAGE_PLACEHOLDER:
+        ref = block.get("ref", "")
+        source = block.get("source", "unknown")
+        mime = block.get("mime")
+        local_path = block.get("local_path")
+        if local_path:
+            # Downloaded — render as a real inline image.
+            return f"![{source or 'image'}]({local_path})"
+        meta_parts = [source] if source else []
+        if mime:
+            meta_parts.append(mime)
+        meta_parts.append("content not preserved in this export")
+        meta = ", ".join(meta_parts)
+        return f"> 🖼️ **Image attached** — `{ref}` ({meta})"
+    if btype == BLOCK_TYPE_FILE_PLACEHOLDER:
+        ref = block.get("ref", "")
+        filename = block.get("filename")
+        label = filename or ref
+        mime = block.get("mime")
+        size_bytes = block.get("size_bytes")
+        duration = block.get("duration_seconds")
+        local_path = block.get("local_path")
+        meta_parts: list[str] = []
+        if mime:
+            meta_parts.append(mime)
+        size_label = _format_size(size_bytes)
+        if size_label:
+            meta_parts.append(size_label)
+        if isinstance(duration, (int, float)) and duration > 0:
+            meta_parts.append(f"{duration:.2f}s")
+        if local_path:
+            # Downloaded — render as a link to the local copy.
+            meta = ", ".join(meta_parts) if meta_parts else ""
+            meta_str = f" ({meta})" if meta else ""
+            return f"> 📎 **File attached** — [{Path(local_path).name}]({local_path}){meta_str}"
+        meta_parts.append("content not preserved in this export")
+        meta = ", ".join(meta_parts)
+        return f"> 📎 **File attached** — `{label}` ({meta})"
+    if btype == BLOCK_TYPE_UNKNOWN:
+        raw_type = block.get("raw_type", "?")
+        reason = block.get("reason", UNKNOWN_REASON_UNKNOWN_TYPE)
+        keys = block.get("observed_keys") or []
+        summary = block.get("summary")
+        first_line = f"⚠️ **Unsupported content** — type `{raw_type}` ({reason})"
+        lines = [first_line]
+        if summary:
+            lines.append(summary)
+        if keys:
+            keys_str = ", ".join(f"`{k}`" for k in keys)
+            lines.append(f"Keys observed: {keys_str}")
+        return _blockquote_prefix("\n".join(lines))
+    if btype == BLOCK_TYPE_HIDDEN_CONTEXT_MARKER:
+        ctype = block.get("content_type", "")
+        return f"> ℹ️ **Hidden context** — `{ctype}`"
+    if btype == BLOCK_TYPE_COLLAPSED:
+        origin = block.get("origin", "")
+        kind = block.get("kind", "")
+        if kind == COLLAPSED_KIND_HIDDEN_CONTEXT:
+            icon, label = "ℹ️", "Hidden context"
+        else:
+            icon, label = "🔧", "Tool output"
+        size_label = _format_size(block.get("size_bytes"))
+        size_part = f" ({size_label})" if size_label else ""
+        return (
+            f"> {icon} **{label}** — `{origin}`{size_part} — omitted "
+            "(EXPORTER_HIDDEN_CONTENT=full to keep)"
+        )
+
+    # Defensive: a block of unrecognised local type (shouldn't happen if
+    # constructors are used). Render as visible warning rather than dropping.
+    return f"> ⚠️ **Internal: unrecognised block type** — `{btype}`"
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _format_size(size_bytes: Any) -> str:
+    """Human-readable size ('24.1 KB', '1.05 MB'), or '' when absent/zero."""
+    if not isinstance(size_bytes, int) or size_bytes <= 0:
+        return ""
+    kb = size_bytes / 1024
+    return f"{kb:.1f} KB" if kb < 1024 else f"{kb / 1024:.2f} MB"
+
+
+def _safe_fence(text: str) -> str:
+    """Return a backtick fence longer than the longest run of backticks in ``text``.
+
+    CommonMark requires the closing fence to be at least as long as the opening
+    fence. Picking N+1 (where N = longest run in content) ensures the content's
+    own backticks are inert. Minimum is 3.
+
+    Verified live against Joplin during planning — see plan
+    §Backtick-corruption defense.
+    """
+    if not isinstance(text, str):
+        return "```"
+    longest_run = 0
+    current_run = 0
+    for ch in text:
+        if ch == "`":
+            current_run += 1
+            if current_run > longest_run:
+                longest_run = current_run
+        else:
+            current_run = 0
+    fence_len = max(3, longest_run + 1)
+    return "`" * fence_len
+
+
+def _blockquote_prefix(text: str) -> str:
+    """Prefix every line of ``text`` with ``> `` so the whole block renders as a quote.
+
+    Empty source lines become ``>`` (no trailing space) so blockquote continuity
+    is preserved without trailing-whitespace noise.
+    """
+    if not isinstance(text, str):
+        return ""
+    out_lines: list[str] = []
+    for line in text.split("\n"):
+        if line == "":
+            out_lines.append(">")
+        else:
+            out_lines.append(f"> {line}")
+    return "\n".join(out_lines)

src/cache.py

@@ -1,4 +1,4 @@
-"""Local cache manifest for tracking exported conversations."""
+"""Local cache manifest for tracking exported and Joplin-synced conversations."""
 
 import json
 import logging
@@ -18,11 +18,17 @@ class CacheError(Exception):
 
 
 class Cache:
-    """Manages the local JSON manifest of exported conversations.
+    """Manages the local JSON manifest of exported and Joplin-synced conversations.
 
-    The manifest is the single source of truth for what has been exported.
-    Every run compares the provider's full conversation list against this
-    manifest to determine what is new or updated.
+    The manifest is the single source of truth for what has been exported and
+    synced. Every export run compares the provider's full conversation list
+    against this manifest to determine what is new or updated. The Joplin sync
+    run reads it to find conversations not yet pushed to Joplin (or re-exported
+    since the last sync).
+
+    Each entry tracks:
+      title, project, updated_at, exported_at, file_path,
+      joplin_note_id (after first sync), joplin_synced_at (after first sync)
 
     File security:
     - Permissions: 600 (owner read/write only)
@@ -78,27 +84,70 @@ class Cache:
         if provider not in self._data:
             self._data[provider] = {}
 
-        self._data[provider][conv_id] = {
+        # Preserve the Joplin link across re-exports. Without this, re-rendering
+        # a conversation (e.g. a forced re-export to pick up new formatting)
+        # would drop joplin_note_id, and the next sync would create a duplicate
+        # note instead of updating the existing one. exported_at is refreshed,
+        # so get_joplin_pending() still flags the note for an update.
+        prev = self._data[provider].get(conv_id) or {}
+        entry = {
             "title": metadata.get("title", ""),
             "project": metadata.get("project"),
+            "created_at": metadata.get("created_at", ""),
             "updated_at": metadata.get("updated_at", ""),
             "exported_at": datetime.now(tz=timezone.utc).isoformat(),
             "file_path": metadata.get("file_path", ""),
         }
+        for carried in ("joplin_note_id", "joplin_synced_at", "joplin_resources"):
+            if carried in prev:
+                entry[carried] = prev[carried]
+
+        self._data[provider][conv_id] = entry
         self._data["last_run"] = datetime.now(tz=timezone.utc).isoformat()
         self._save()
 
-    def get_new_or_updated(self, provider: str, conversations: list[dict]) -> list[dict]:
+    def get_new_or_updated(
+        self,
+        provider: str,
+        conversations: list[dict],
+        force: bool = False,
+        campaign_at: str | None = None,
+    ) -> list[dict]:
         """Filter a conversation list to only new or updated conversations.
 
         Args:
             provider: "chatgpt" or "claude"
             conversations: List of raw conversation dicts from the provider.
                            Each must have an ``id``/``uuid`` and ``updated_at``/``update_time``.
+            force: When True, return conversations to re-render regardless of
+                   cache freshness — used by ``export --force``.
+            campaign_at: In force mode, exclude conversations already re-exported
+                   at/after this ISO timestamp (the re-render campaign start), so
+                   each run advances and finished providers do nothing.
 
         Returns:
-            Subset that needs to be exported.
+            Subset that needs to be exported, ordered oldest-export-first in
+            force mode.
         """
+        if force:
+            # Re-export for a re-render campaign, ordered least-recently-exported
+            # first (never-exported sorts first). When ``campaign_at`` is given,
+            # conversations already re-exported during this campaign
+            # (exported_at >= campaign_at) are excluded — so finished providers
+            # do no work and the remaining count shrinks to zero. Each capped run
+            # re-renders the next-oldest batch. Conversations without an id drop.
+            entries = self._data.get(provider, {})
+
+            def _exported_at(conv: dict) -> str:
+                conv_id = conv.get("id") or conv.get("uuid", "")
+                entry = entries.get(conv_id) or {}
+                return entry.get("exported_at") or ""
+
+            candidates = [c for c in conversations if c.get("id") or c.get("uuid")]
+            if campaign_at is not None:
+                candidates = [c for c in candidates if _exported_at(c) < campaign_at]
+            return sorted(candidates, key=_exported_at)
+
         result = []
         for conv in conversations:
             conv_id = conv.get("id") or conv.get("uuid", "")
@@ -150,10 +199,117 @@ class Cache:
         """Return all cached entries for a provider (for --cache --show)."""
         return dict(self._data.get(provider, {}))
 
+    def mark_joplin_synced(self, provider: str, conv_id: str, note_id: str) -> None:
+        """Record a successful Joplin sync for a conversation.
+
+        Adds ``joplin_note_id`` and ``joplin_synced_at`` to the manifest entry
+        and writes atomically to disk.
+        """
+        entry = self._data.get(provider, {}).get(conv_id)
+        if entry is None:
+            logger.warning(
+                "[cache] mark_joplin_synced: no cache entry for %s/%s", provider, conv_id[:8]
+            )
+            return
+        entry["joplin_note_id"] = note_id
+        entry["joplin_synced_at"] = datetime.now(tz=timezone.utc).isoformat()
+        self._save()
+
+    def get_joplin_resources(self, provider: str, conv_id: str) -> dict[str, str]:
+        """Return the {relative_media_path: joplin_resource_id} map for a conversation."""
+        entry = self._data.get(provider, {}).get(conv_id)
+        if not isinstance(entry, dict):
+            return {}
+        resources = entry.get("joplin_resources")
+        return dict(resources) if isinstance(resources, dict) else {}
+
+    def set_joplin_resources(self, provider: str, conv_id: str, resources: dict[str, str]) -> None:
+        """Persist the {relative_media_path: joplin_resource_id} map for a conversation."""
+        entry = self._data.get(provider, {}).get(conv_id)
+        if entry is None:
+            return
+        entry["joplin_resources"] = dict(resources)
+        self._save()
+
+    def get_joplin_pending(self, provider: str) -> list[tuple[str, dict]]:
+        """Return (conv_id, entry) pairs that need to be synced to Joplin.
+
+        A conversation is pending when:
+        - It has never been synced (no ``joplin_note_id``), OR
+        - It was re-exported after the last Joplin sync
+          (``exported_at`` > ``joplin_synced_at``).
+
+        Returns:
+            List of (conv_id, entry_dict) tuples, where entry_dict includes
+            ``file_path``, ``title``, ``project``, and optionally ``joplin_note_id``.
+        """
+        pending = []
+        for conv_id, entry in self._data.get(provider, {}).items():
+            if not isinstance(entry, dict):
+                continue
+            if not entry.get("file_path"):
+                continue
+
+            note_id = entry.get("joplin_note_id")
+            if not note_id:
+                pending.append((conv_id, entry))
+                continue
+
+            # Re-sync if the file was re-exported after the last Joplin sync
+            exported_at = entry.get("exported_at", "")
+            synced_at = entry.get("joplin_synced_at", "")
+            if exported_at and synced_at:
+                try:
+                    from src.utils import _parse_dt
+                    if _parse_dt(exported_at) > _parse_dt(synced_at):
+                        pending.append((conv_id, entry))
+                except Exception:
+                    pass
+
+        return pending
+
+    def all_file_paths(self) -> set[str]:
+        """Return every non-empty ``file_path`` in the manifest, across providers.
+
+        Used by ``prune`` (files on disk but not here are stale) and by the
+        doctor integrity check (files here but not on disk are missing).
+        """
+        paths: set[str] = set()
+        for key, entries in self._data.items():
+            if not isinstance(entries, dict) or key in (
+                "version",
+                "last_run",
+                "tos_acknowledged_at",
+            ):
+                continue
+            for entry in entries.values():
+                if isinstance(entry, dict) and entry.get("file_path"):
+                    paths.add(entry["file_path"])
+        return paths
+
     def last_run(self) -> str | None:
         """Return the ISO8601 timestamp of the last export run, or None."""
         return self._data.get("last_run")
 
+    def exported_at(self, provider: str, conv_id: str) -> str:
+        """Return the recorded ``exported_at`` for a conversation, or '' if absent."""
+        entry = self._data.get(provider, {}).get(conv_id)
+        return entry.get("exported_at", "") if isinstance(entry, dict) else ""
+
+    # Force re-render campaign marker (top-level scalar; skipped by stats/clear,
+    # which only act on dict-valued provider keys).
+    def get_force_campaign(self) -> str | None:
+        """Return the active force-re-render campaign start timestamp, or None."""
+        return self._data.get("force_campaign_at")
+
+    def set_force_campaign(self, started_at: str) -> None:
+        self._data["force_campaign_at"] = started_at
+        self._save()
+
+    def clear_force_campaign(self) -> None:
+        if self._data.pop("force_campaign_at", None) is not None:
+            self._save()
+
     # ------------------------------------------------------------------
     # Private helpers
     # ------------------------------------------------------------------

src/config.py

@@ -20,6 +20,12 @@ _CLAUDE_PLACEHOLDER = ""
 # Valid OUTPUT_STRUCTURE values
 VALID_STRUCTURES = {"provider/project/year", "provider/project", "provider/year"}
 
+# Valid EXPORTER_HIDDEN_CONTENT values (see src/providers/base.py)
+VALID_HIDDEN_CONTENT = {"full", "placeholder", "omit"}
+
+# Valid EXPORTER_DOWNLOAD_MEDIA values (see src/media.py)
+VALID_DOWNLOAD_MEDIA = {"images", "all", "off"}
+
 
 class ConfigError(Exception):
     """Raised when required configuration is missing or invalid."""
@@ -28,6 +34,7 @@ class ConfigError(Exception):
 @dataclass
 class Config:
     chatgpt_session_token: str | None
+    chatgpt_session_token_1: str | None
     claude_session_key: str | None
     export_dir: Path
     output_structure: str
@@ -35,6 +42,23 @@ class Config:
     log_file: str
     # Decoded ChatGPT JWT expiry (None if token absent or not a JWT)
     chatgpt_token_expiry: datetime | None = field(default=None, repr=False)
+    # ChatGPT Project gizmo IDs (g-p-xxx) — project conversations are not
+    # included in the default /conversations listing; they must be fetched
+    # separately via /backend-api/gizmos/{id}/conversations.
+    chatgpt_project_ids: list[str] = field(default_factory=list)
+    # Joplin local REST API settings (Web Clipper service)
+    joplin_api_token: str | None = None
+    joplin_api_url: str = "http://localhost:41184"
+    # Policy for content invisible in the provider web UI (retrieval dumps,
+    # hidden context): full | placeholder | omit
+    hidden_content: str = "placeholder"
+    # Session cap: max conversations downloaded per export run (None = unlimited).
+    # Every run is resumable, so a capped run just continues next time.
+    max_conversations: int | None = None
+    # Seconds between consecutive API requests (politeness pacing; 0 disables)
+    request_delay: float = 1.0
+    # Asset download policy: images | all | off
+    download_media: str = "images"
 
 
 def load_config() -> Config:
@@ -48,11 +72,36 @@ def load_config() -> Config:
     load_dotenv(override=False)
 
     chatgpt_token = os.getenv("CHATGPT_SESSION_TOKEN", "").strip() or None
+    chatgpt_token_1 = os.getenv("CHATGPT_SESSION_TOKEN_1", "").strip() or None
     claude_key = os.getenv("CLAUDE_SESSION_KEY", "").strip() or None
     export_dir = Path(os.getenv("EXPORT_DIR", "./exports")).expanduser()
     output_structure = os.getenv("OUTPUT_STRUCTURE", "provider/project/year").strip()
-    cache_dir = Path(os.getenv("CACHE_DIR", "~/.ai-chat-exporter")).expanduser()
-    log_file = os.getenv("LOG_FILE", "~/.ai-chat-exporter/logs/exporter.log").strip()
+    cache_dir = Path(os.getenv("CACHE_DIR", "./cache")).expanduser()
+    log_file = os.getenv("LOG_FILE", "./cache/logs/exporter.log").strip()
+
+    # Joplin
+    joplin_token = os.getenv("JOPLIN_API_TOKEN", "").strip() or None
+    joplin_url = os.getenv("JOPLIN_API_URL", "http://localhost:41184").strip()
+
+    hidden_content = os.getenv("EXPORTER_HIDDEN_CONTENT", "").strip().lower() or "placeholder"
+
+    max_conversations_raw = os.getenv("MAX_CONVERSATIONS_PER_RUN", "").strip()
+    request_delay_raw = os.getenv("REQUEST_DELAY", "").strip()
+    download_media = os.getenv("EXPORTER_DOWNLOAD_MEDIA", "").strip().lower() or "images"
+
+    # Parse CHATGPT_PROJECT_IDS — comma-separated list of gizmo IDs (g-p-xxx)
+    _project_ids_raw = os.getenv("CHATGPT_PROJECT_IDS", "").strip()
+    chatgpt_project_ids = [
+        pid.strip()
+        for pid in _project_ids_raw.split(",")
+        if pid.strip() and pid.strip().startswith("g-p-")
+    ] if _project_ids_raw else []
+    if _project_ids_raw and not chatgpt_project_ids:
+        logger.warning(
+            "CHATGPT_PROJECT_IDS is set but contains no valid project IDs. "
+            "Each ID should start with 'g-p-' (e.g. g-p-68c2b2b3037c8191890036fb4ae3ed9f). "
+            "Find your project ID in the browser URL when viewing a project."
+        )
 
     errors: list[str] = []
 
@@ -63,6 +112,46 @@ def load_config() -> Config:
             f"Must be one of: {', '.join(sorted(VALID_STRUCTURES))}"
         )
 
+    # Validate hidden-content policy
+    if hidden_content not in VALID_HIDDEN_CONTENT:
+        errors.append(
+            f"EXPORTER_HIDDEN_CONTENT '{hidden_content}' is invalid. "
+            f"Must be one of: {', '.join(sorted(VALID_HIDDEN_CONTENT))}"
+        )
+
+    # Validate media download policy
+    if download_media not in VALID_DOWNLOAD_MEDIA:
+        errors.append(
+            f"EXPORTER_DOWNLOAD_MEDIA '{download_media}' is invalid. "
+            f"Must be one of: {', '.join(sorted(VALID_DOWNLOAD_MEDIA))}"
+        )
+
+    # Validate session cap
+    max_conversations: int | None = None
+    if max_conversations_raw:
+        try:
+            max_conversations = int(max_conversations_raw)
+        except ValueError:
+            errors.append(
+                f"MAX_CONVERSATIONS_PER_RUN '{max_conversations_raw}' is not an integer."
+            )
+        else:
+            if max_conversations < 1:
+                errors.append(
+                    f"MAX_CONVERSATIONS_PER_RUN must be at least 1 (got {max_conversations})."
+                )
+
+    # Validate request pacing
+    request_delay = 1.0
+    if request_delay_raw:
+        try:
+            request_delay = float(request_delay_raw)
+        except ValueError:
+            errors.append(f"REQUEST_DELAY '{request_delay_raw}' is not a number.")
+        else:
+            if request_delay < 0:
+                errors.append(f"REQUEST_DELAY must be >= 0 (got {request_delay}).")
+
     # Validate and decode ChatGPT JWT
     chatgpt_expiry: datetime | None = None
     if chatgpt_token:
@@ -76,7 +165,7 @@ def load_config() -> Config:
     if not chatgpt_token and not claude_key:
         logger.warning(
             "Neither CHATGPT_SESSION_TOKEN nor CLAUDE_SESSION_KEY is set. "
-            "Run 'python -m src.main auth' to configure credentials."
+            "Run 'ai-chat-exporter auth' to configure credentials."
         )
 
     # Create and validate output directory
@@ -102,12 +191,20 @@ def load_config() -> Config:
 
     config = Config(
         chatgpt_session_token=chatgpt_token,
+        chatgpt_session_token_1=chatgpt_token_1,
         claude_session_key=claude_key,
         export_dir=export_dir,
         output_structure=output_structure,
         cache_dir=cache_dir,
         log_file=log_file,
         chatgpt_token_expiry=chatgpt_expiry,
+        chatgpt_project_ids=chatgpt_project_ids,
+        joplin_api_token=joplin_token,
+        joplin_api_url=joplin_url,
+        hidden_content=hidden_content,
+        max_conversations=max_conversations,
+        request_delay=request_delay,
+        download_media=download_media,
     )
 
     _log_startup_summary(config)
@@ -125,8 +222,12 @@ def _validate_chatgpt_token(token: str) -> datetime | None:
 
     try:
         payload = jwt.decode(token, options={"verify_signature": False})
-    except jwt.DecodeError as e:
-        logger.warning("CHATGPT_SESSION_TOKEN could not be decoded as JWT: %s", e)
+    except jwt.DecodeError:
+        # JWE (encrypted JWT, alg=dir) — cannot be decoded without the server key.
+        # This is normal for ChatGPT's current token format. Token is valid; expiry unknown.
+        logger.debug(
+            "CHATGPT_SESSION_TOKEN is an encrypted JWE token — expiry cannot be decoded client-side."
+        )
         return None
 
     exp = payload.get("exp")
@@ -141,7 +242,7 @@ def _validate_chatgpt_token(token: str) -> datetime | None:
     if delta.total_seconds() < 0:
         logger.warning(
             "CHATGPT_SESSION_TOKEN expired at %s. "
-            "Run 'python -m src.main auth' to refresh it.",
+            "Run 'ai-chat-exporter auth' to refresh it.",
             expiry.strftime("%Y-%m-%d %H:%M UTC"),
         )
     elif delta.total_seconds() < 86400:
@@ -178,17 +279,30 @@ def _log_startup_summary(cfg: Config) -> None:
     """Log a single INFO line summarising the active configuration."""
     chatgpt_status = format_token_status(cfg.chatgpt_session_token, cfg.chatgpt_token_expiry)
     claude_status = format_token_status(cfg.claude_session_key)
+    joplin_status = "configured" if cfg.joplin_api_token else "not configured"
 
     logger.info(
         "Config loaded | "
         "ChatGPT: %s | "
         "Claude: %s | "
+        "chatgpt_projects: %d | "
+        "Joplin: %s | "
         "export_dir=%s | "
         "structure=%s | "
-        "cache_dir=%s",
+        "cache_dir=%s | "
+        "hidden_content=%s | "
+        "max_conversations=%s | "
+        "request_delay=%.1fs | "
+        "download_media=%s",
         chatgpt_status,
         claude_status,
+        len(cfg.chatgpt_project_ids),
+        joplin_status,
         cfg.export_dir,
         cfg.output_structure,
         cfg.cache_dir,
+        cfg.hidden_content,
+        cfg.max_conversations if cfg.max_conversations is not None else "unlimited",
+        cfg.request_delay,
+        cfg.download_media,
     )

src/exporters/markdown.py

@@ -6,6 +6,7 @@ from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any
 
+from src.blocks import render_blocks_to_markdown
 from src.utils import build_export_path, generate_filename
 
 logger = logging.getLogger(__name__)
@@ -15,6 +16,7 @@ _ROLE_LABELS = {
     "user": ("🧑 Human", "user"),
     "assistant": ("🤖 Assistant", "assistant"),
     "system": ("⚙️ System", "system"),
+    "tool": ("🔧 Tool", "tool"),
 }
 
 
@@ -125,10 +127,17 @@ class MarkdownExporter:
         # Messages
         for msg in messages:
             role = msg.get("role", "user")
-            content = msg.get("content", "")
+            blocks = msg.get("blocks") or []
             timestamp = msg.get("timestamp")
 
-            if not content or not content.strip():
+            # Prefer rendering from blocks (v0.4.0+). Backward-compat fallback:
+            # if blocks is missing/empty AND content exists, render content as-is.
+            if blocks:
+                body = render_blocks_to_markdown(blocks)
+            else:
+                body = msg.get("content", "") or ""
+
+            if not body or not body.strip():
                 logger.warning(
                     "[markdown] Skipping empty/whitespace message in conversation %s",
                     conv_id[:8],
@@ -143,7 +152,7 @@ class MarkdownExporter:
             else:
                 lines.append("")
 
-            lines.append(content)
+            lines.append(body)
             lines.append("")
             lines.append("---")
             lines.append("")

src/joplin.py

@@ -0,0 +1,416 @@
+"""Joplin Data API client for importing notes into Joplin desktop."""
+
+import json
+import logging
+import os
+import re
+from pathlib import Path
+from typing import Any
+
+import requests
+
+logger = logging.getLogger(__name__)
+
+# HTTP timeout for regular API calls (seconds). Notes can be large Markdown
+# files so we allow more time than a typical JSON API call.
+# Override with JOPLIN_REQUEST_TIMEOUT env var if you have very large conversations.
+_REQUEST_TIMEOUT: int = int(os.getenv("JOPLIN_REQUEST_TIMEOUT", "30"))
+
+
+class JoplinError(Exception):
+    """Raised when the Joplin API returns an error or is unreachable."""
+
+
+class JoplinClient:
+    """HTTP client for the Joplin local REST API (Web Clipper service).
+
+    Requires Joplin desktop to be running with the Web Clipper service enabled.
+    Get your API token from: Joplin → Tools → Options → Web Clipper.
+
+    Args:
+        base_url: Joplin API base URL (default: http://localhost:41184).
+        token: API authorization token from Joplin Web Clipper settings.
+    """
+
+    def __init__(self, base_url: str, token: str) -> None:
+        self._base_url = base_url.rstrip("/")
+        self._token = token
+        # In-memory cache: (parent_id | None, title) → folder ID
+        self._notebook_cache: dict[tuple[str | None, str], str] = {}
+        self._notebooks_loaded = False
+        logger.debug("[joplin] Client initialised with base_url=%s", self._base_url)
+
+    # ------------------------------------------------------------------
+    # Connectivity
+    # ------------------------------------------------------------------
+
+    def ping(self) -> bool:
+        """Return True if the Joplin API is reachable and responding.
+
+        Note: /ping does not require authentication. A successful ping only
+        confirms Joplin is running — not that the token is valid. Call
+        ``validate_token()`` to confirm authentication separately.
+
+        Raises:
+            JoplinError: If the API returns an unexpected non-connection error.
+        """
+        url = f"{self._base_url}/ping"
+        logger.debug("[joplin] GET %s", url)
+        try:
+            resp = requests.get(url, timeout=5)
+            resp.raise_for_status()
+            ok = "JoplinClipperServer" in resp.text
+            logger.debug("[joplin] ping → %s (body: %r)", "OK" if ok else "unexpected response", resp.text[:80])
+            return ok
+        except requests.exceptions.ConnectionError:
+            logger.debug("[joplin] ping → connection refused at %s", url)
+            return False
+        except requests.exceptions.Timeout:
+            logger.debug("[joplin] ping → timed out after 5s at %s", url)
+            return False
+        except requests.exceptions.RequestException as e:
+            raise JoplinError(f"Joplin ping failed: {e}") from e
+
+    def validate_token(self) -> None:
+        """Verify the API token is accepted by Joplin.
+
+        Does a minimal authenticated call (GET /folders?limit=1) and raises
+        ``JoplinError`` if authentication fails.
+
+        Raises:
+            JoplinError: If the token is rejected (401) or Joplin is unreachable.
+        """
+        logger.debug("[joplin] Validating API token…")
+        self._get("/folders", params={"limit": 1, "fields": "id"})
+        logger.debug("[joplin] Token validated OK")
+
+    # ------------------------------------------------------------------
+    # Notebooks (folders)
+    # ------------------------------------------------------------------
+
+    def list_notebooks(self) -> list[dict]:
+        """Return all Joplin notebooks (folders), handling pagination.
+
+        Returns:
+            List of folder dicts with at least ``id``, ``title``, and ``parent_id`` keys.
+        """
+        results: list[dict] = []
+        page = 1
+        while True:
+            logger.debug("[joplin] GET /folders page=%d", page)
+            resp = self._get("/folders", params={"page": page, "fields": "id,title,parent_id"})
+            items = resp.get("items", [])
+            results.extend(items)
+            logger.debug("[joplin] /folders page=%d → %d items, has_more=%s", page, len(items), resp.get("has_more"))
+            if not resp.get("has_more"):
+                break
+            page += 1
+        return results
+
+    def get_or_create_notebook(self, title: str, parent_id: str | None = None) -> str:
+        """Return the Joplin folder ID for ``title`` under ``parent_id``, creating if needed.
+
+        Args:
+            title: Notebook display name.
+            parent_id: ID of the parent folder, or None for a root notebook.
+
+        Returns:
+            Joplin folder ID string.
+        """
+        if not self._notebooks_loaded:
+            self._load_notebook_cache()
+
+        key = (parent_id, title)
+        if key in self._notebook_cache:
+            folder_id = self._notebook_cache[key]
+            logger.debug("[joplin] Notebook cache hit: %r (parent=%s) → %s", title, parent_id, folder_id)
+            return folder_id
+
+        # Not found — create it
+        logger.info("[joplin] Creating notebook: %r (parent=%s)", title, parent_id)
+        data: dict = {"title": title}
+        if parent_id:
+            data["parent_id"] = parent_id
+        resp = self._post("/folders", data)
+        folder_id = resp["id"]
+        self._notebook_cache[key] = folder_id
+        logger.debug("[joplin] Notebook created: %r → %s", title, folder_id)
+        return folder_id
+
+    def get_or_create_notebook_path(self, path: list[str]) -> str:
+        """Ensure a nested notebook path exists and return the leaf folder ID.
+
+        Creates intermediate notebooks as needed.
+
+        Args:
+            path: Ordered list of notebook names, e.g. ["AI-ChatGPT", "No Project"].
+
+        Returns:
+            Joplin folder ID of the deepest (leaf) notebook.
+        """
+        parent_id: str | None = None
+        for name in path:
+            parent_id = self.get_or_create_notebook(name, parent_id)
+        assert parent_id is not None
+        return parent_id
+
+    # ------------------------------------------------------------------
+    # Notes
+    # ------------------------------------------------------------------
+
+    def create_note(self, title: str, body: str, parent_id: str) -> str:
+        """Create a new note in the specified notebook.
+
+        Args:
+            title: Note title.
+            body: Note body (Markdown).
+            parent_id: Notebook (folder) ID.
+
+        Returns:
+            ID of the created note.
+        """
+        logger.debug(
+            "[joplin] Creating note: %r in notebook %s (%d chars)",
+            title, parent_id, len(body),
+        )
+        resp = self._post("/notes", {"title": title, "body": body, "parent_id": parent_id})
+        note_id = resp["id"]
+        logger.info("[joplin] Note created: %r → %s", title, note_id)
+        return note_id
+
+    def update_note(self, note_id: str, title: str, body: str) -> None:
+        """Update the title and body of an existing note.
+
+        Args:
+            note_id: Joplin note ID.
+            title: New note title.
+            body: New note body (Markdown).
+        """
+        logger.debug(
+            "[joplin] Updating note %s: %r (%d chars)",
+            note_id, title, len(body),
+        )
+        self._put(f"/notes/{note_id}", {"title": title, "body": body})
+        logger.info("[joplin] Note updated: %r (%s)", title, note_id)
+
+    # ------------------------------------------------------------------
+    # Resources (embedded media)
+    # ------------------------------------------------------------------
+
+    def create_resource(self, file_path: "Path", title: str | None = None) -> str:
+        """Upload a local file as a Joplin resource and return its ID.
+
+        Joplin's ``POST /resources`` is multipart: a ``data`` file part plus a
+        ``props`` JSON part. The returned ID is referenced from note bodies as
+        ``:/<id>``.
+        """
+        url = f"{self._base_url}/resources"
+        props = json.dumps({"title": title or file_path.name})
+        logger.debug("[joplin] POST /resources (%s)", file_path.name)
+        try:
+            with file_path.open("rb") as fh:
+                resp = requests.post(
+                    url,
+                    params={"token": self._token},
+                    files={
+                        "data": (file_path.name, fh),
+                        "props": (None, props),
+                    },
+                    timeout=_REQUEST_TIMEOUT,
+                )
+            resp.raise_for_status()
+            resource_id = resp.json()["id"]
+            logger.info("[joplin] Resource created: %s → %s", file_path.name, resource_id)
+            return resource_id
+        except requests.exceptions.ConnectionError as e:
+            raise JoplinError(
+                "Cannot connect to Joplin. Is Joplin desktop running with Web Clipper enabled?"
+            ) from e
+        except requests.exceptions.Timeout as e:
+            raise JoplinError(_timeout_message("POST", "/resources")) from e
+        except requests.exceptions.HTTPError as e:
+            raise JoplinError(_http_error_message("POST", "/resources", e)) from e
+        except (requests.exceptions.RequestException, KeyError, OSError) as e:
+            raise JoplinError(f"Joplin resource upload failed for {file_path.name}: {e}") from e
+
+    # ------------------------------------------------------------------
+    # HTTP helpers
+    # ------------------------------------------------------------------
+
+    def _get(self, path: str, params: dict | None = None) -> dict[str, Any]:
+        url = f"{self._base_url}{path}"
+        query = {"token": self._token, **(params or {})}
+        logger.debug("[joplin] GET %s params=%s", path, {k: v for k, v in (params or {}).items()})
+        try:
+            resp = requests.get(url, params=query, timeout=_REQUEST_TIMEOUT)
+            logger.debug("[joplin] GET %s → HTTP %d", path, resp.status_code)
+            resp.raise_for_status()
+            return resp.json()
+        except requests.exceptions.ConnectionError as e:
+            raise JoplinError(
+                "Cannot connect to Joplin. Is Joplin desktop running with Web Clipper enabled?"
+            ) from e
+        except requests.exceptions.Timeout as e:
+            raise JoplinError(_timeout_message("GET", path)) from e
+        except requests.exceptions.HTTPError as e:
+            raise JoplinError(_http_error_message("GET", path, e)) from e
+        except requests.exceptions.RequestException as e:
+            raise JoplinError(f"Joplin GET {path} failed: {e}") from e
+
+    def _post(self, path: str, data: dict) -> dict[str, Any]:
+        url = f"{self._base_url}{path}"
+        logger.debug("[joplin] POST %s", path)
+        try:
+            resp = requests.post(url, params={"token": self._token}, json=data, timeout=_REQUEST_TIMEOUT)
+            logger.debug("[joplin] POST %s → HTTP %d", path, resp.status_code)
+            resp.raise_for_status()
+            return resp.json()
+        except requests.exceptions.ConnectionError as e:
+            raise JoplinError(
+                "Cannot connect to Joplin. Is Joplin desktop running with Web Clipper enabled?"
+            ) from e
+        except requests.exceptions.Timeout as e:
+            raise JoplinError(_timeout_message("POST", path)) from e
+        except requests.exceptions.HTTPError as e:
+            raise JoplinError(_http_error_message("POST", path, e)) from e
+        except requests.exceptions.RequestException as e:
+            raise JoplinError(f"Joplin POST {path} failed: {e}") from e
+
+    def _put(self, path: str, data: dict) -> dict[str, Any]:
+        url = f"{self._base_url}{path}"
+        logger.debug("[joplin] PUT %s", path)
+        try:
+            resp = requests.put(url, params={"token": self._token}, json=data, timeout=_REQUEST_TIMEOUT)
+            logger.debug("[joplin] PUT %s → HTTP %d", path, resp.status_code)
+            resp.raise_for_status()
+            return resp.json()
+        except requests.exceptions.ConnectionError as e:
+            raise JoplinError(
+                "Cannot connect to Joplin. Is Joplin desktop running with Web Clipper enabled?"
+            ) from e
+        except requests.exceptions.Timeout as e:
+            raise JoplinError(_timeout_message("PUT", path)) from e
+        except requests.exceptions.HTTPError as e:
+            raise JoplinError(_http_error_message("PUT", path, e)) from e
+        except requests.exceptions.RequestException as e:
+            raise JoplinError(f"Joplin PUT {path} failed: {e}") from e
+
+    def _load_notebook_cache(self) -> None:
+        logger.debug("[joplin] Loading notebook list from Joplin…")
+        notebooks = self.list_notebooks()
+        self._notebook_cache = {
+            (nb.get("parent_id") or None, nb["title"]): nb["id"]
+            for nb in notebooks
+        }
+        self._notebooks_loaded = True
+        logger.debug("[joplin] Notebook cache loaded: %d notebooks", len(self._notebook_cache))
+        for (parent_id, title), folder_id in self._notebook_cache.items():
+            logger.debug("[joplin]   (%s) %r → %s", parent_id or "root", title, folder_id)
+
+
+# ------------------------------------------------------------------
+# Error message helper
+# ------------------------------------------------------------------
+
+
+def _timeout_message(method: str, path: str) -> str:
+    """Build a clear timeout error message with actionable suggestions."""
+    return (
+        f"Joplin {method} {path} timed out after {_REQUEST_TIMEOUT}s. "
+        "Possible causes:\n"
+        "  • The note body is very large and Joplin is slow to process it.\n"
+        "  • Joplin is busy (syncing, indexing, or loading a large library).\n"
+        "  • Joplin has frozen — try restarting it.\n"
+        f"If this happens repeatedly, increase JOPLIN_REQUEST_TIMEOUT in your .env "
+        f"(currently {_REQUEST_TIMEOUT}s)."
+    )
+
+
+def _http_error_message(method: str, path: str, e: requests.exceptions.HTTPError) -> str:
+    """Build a human-friendly error message from an HTTP error, with auth hint on 401."""
+    resp = e.response
+    status = resp.status_code if resp is not None else "?"
+    if status == 401:
+        return (
+            f"Joplin rejected the API token (HTTP 401 on {method} {path}). "
+            "Check that JOPLIN_API_TOKEN is correct: "
+            "Joplin → Tools → Options → Web Clipper → Authorization token."
+        )
+    if status == 404:
+        return f"Joplin resource not found (HTTP 404 on {method} {path}). The note may have been deleted in Joplin."
+    body_snippet = ""
+    if resp is not None:
+        try:
+            body_snippet = f" — {resp.text[:120]}"
+        except Exception:
+            pass
+    return f"Joplin {method} {path} failed: HTTP {status}{body_snippet}"
+
+
+# ------------------------------------------------------------------
+# Embedded-media rewriting
+# ------------------------------------------------------------------
+
+# Markdown image/link targets pointing at the local media/ sibling directory,
+# e.g. ![alt](media/file_x.png) or [name](media/clip.wav).
+_MEDIA_LINK_RE = re.compile(r"(!?\[[^\]]*\])\((media/[^)\s]+)\)")
+
+
+def upload_media_and_rewrite(
+    body: str,
+    note_dir: Path,
+    client: "JoplinClient",
+    resource_map: dict[str, str],
+) -> tuple[str, dict[str, str]]:
+    """Rewrite local ``media/…`` links to Joplin ``:/resourceId`` references.
+
+    Uploads each referenced file as a Joplin resource (once), rewriting both
+    image embeds and file links. ``resource_map`` (relative-path → resource_id)
+    is the idempotency record from the cache: known paths are reused, new ones
+    are added and returned so the caller can persist them. Missing files are
+    left as-is so the link still points at the on-disk copy.
+    """
+    new_map = dict(resource_map)
+
+    def replace(match: re.Match) -> str:
+        label, rel_path = match.group(1), match.group(2)
+        resource_id = new_map.get(rel_path)
+        if resource_id is None:
+            file_path = (note_dir / rel_path).resolve()
+            if not file_path.is_file():
+                logger.warning("[joplin] Media file missing, leaving link: %s", rel_path)
+                return match.group(0)
+            resource_id = client.create_resource(file_path)
+            new_map[rel_path] = resource_id
+        return f"{label}(:/{resource_id})"
+
+    return _MEDIA_LINK_RE.sub(replace, body), new_map
+
+
+# ------------------------------------------------------------------
+# Notebook naming helper
+# ------------------------------------------------------------------
+
+
+_PROVIDER_DISPLAY = {
+    "chatgpt": "AI-ChatGPT",
+    "claude": "AI-Claude",
+    # Decision 2026-06-12: Claude Code coding projects nest under the same
+    # AI-Claude parent, alongside Claude web projects.
+    "claude-code": "AI-Claude",
+}
+
+
+def notebook_path(provider: str, project: str | None) -> tuple[str, str]:
+    """Return (parent_notebook, child_notebook) for the given provider and project.
+
+    The parent is the top-level provider notebook; the child is the project name.
+
+    Examples:
+        notebook_path("chatgpt", None)             → ("AI-ChatGPT", "No Project")
+        notebook_path("chatgpt", "no-project")     → ("AI-ChatGPT", "No Project")
+        notebook_path("claude", "budget-tracker")  → ("AI-Claude", "Budget Tracker")
+    """
+    parent = _PROVIDER_DISPLAY.get(provider, f"AI-{provider.capitalize()}")
+    child = (project or "no-project").replace("-", " ").title()
+    return parent, child

src/loss_report.py

@@ -0,0 +1,123 @@
+"""Per-export-run tally for content that was dropped or partially extracted.
+
+Surfaces the loss visibility that the rest of the system promises in its
+output (visible ``unknown`` blocks). The summary emitted at the end of
+each export is the load-bearing operator-facing signal: if a real content
+type starts being silently dropped, this is where it shows up.
+
+Pass a single instance through ``BaseProvider.normalize_conversation`` and
+read it back in ``src/main.py`` after the export loop. No global state.
+"""
+
+from collections import Counter
+from dataclasses import dataclass, field
+
+_TOP_N_BREAKDOWN = 5
+
+
+@dataclass
+class LossReport:
+    """Counters for things that didn't render cleanly in an export run."""
+
+    # Type-keyed counters. Values are int counts.
+    unknown_blocks: Counter = field(default_factory=Counter)
+    extraction_failures: Counter = field(default_factory=Counter)
+    filtered_roles: Counter = field(default_factory=Counter)
+    # Messages collapsed/omitted by the EXPORTER_HIDDEN_CONTENT policy —
+    # intentional, but still surfaced so the omission is never invisible.
+    collapsed: Counter = field(default_factory=Counter)
+    collapsed_bytes: int = 0
+
+    # Aggregate counters
+    messages_rendered: int = 0
+    conversations: int = 0
+
+    # Media downloads (EXPORTER_DOWNLOAD_MEDIA): successes / skips / failures
+    media_downloaded: int = 0
+    media_downloaded_bytes: int = 0
+    media_failed: Counter = field(default_factory=Counter)
+
+    # Recording -------------------------------------------------------------
+
+    def record_unknown(self, raw_type: str) -> None:
+        self.unknown_blocks[raw_type or "?"] += 1
+
+    def record_extraction_failure(self, raw_type: str) -> None:
+        self.extraction_failures[raw_type or "?"] += 1
+
+    def record_filtered_role(self, role: str) -> None:
+        self.filtered_roles[role or "?"] += 1
+
+    def record_collapsed(self, origin: str, size_bytes: int = 0) -> None:
+        self.collapsed[origin or "?"] += 1
+        if isinstance(size_bytes, int) and size_bytes > 0:
+            self.collapsed_bytes += size_bytes
+
+    def record_media_downloaded(self, size_bytes: int = 0) -> None:
+        self.media_downloaded += 1
+        if isinstance(size_bytes, int) and size_bytes > 0:
+            self.media_downloaded_bytes += size_bytes
+
+    def record_media_failed(self, reason: str) -> None:
+        self.media_failed[reason or "?"] += 1
+
+    def record_message(self) -> None:
+        self.messages_rendered += 1
+
+    def record_conversation(self) -> None:
+        self.conversations += 1
+
+    # Summary ---------------------------------------------------------------
+
+    def format_summary(self) -> str:
+        """Return a multi-line summary table suitable for INFO logging.
+
+        Format pinned by plan §Post-export summary — "(none)" sentinel when a
+        counter is empty, top-5 breakdown with "+ N more types" overflow.
+        """
+        lines: list[str] = ["[export] Run summary:"]
+        lines.append(f"  conversations:        {self.conversations}")
+        lines.append(f"  messages rendered:    {self.messages_rendered}")
+        lines.extend(_format_section("unknown blocks:      ", self.unknown_blocks))
+        lines.extend(_format_section("extraction failures: ", self.extraction_failures))
+        lines.extend(_format_section("collapsed by policy: ", self.collapsed))
+        if self.collapsed:
+            lines.append(
+                f"                        ≈{self.collapsed_bytes / 1024:.0f} KB omitted "
+                "(intentional — EXPORTER_HIDDEN_CONTENT=full to keep)"
+            )
+        if self.media_downloaded or self.media_failed:
+            lines.append(
+                f"  media downloaded:     {self.media_downloaded} "
+                f"(≈{self.media_downloaded_bytes / 1024:.0f} KB)"
+            )
+            if self.media_failed:
+                total_failed = sum(self.media_failed.values())
+                lines.append(f"  media failed:         {total_failed}")
+                for reason, count in self.media_failed.most_common(_TOP_N_BREAKDOWN):
+                    lines.append(f"                        {reason}={count}")
+        lines.append(
+            "  filtered roles:       "
+            "(filter lifted in v0.4.0 — counter retained for future use, expected 0)"
+        )
+        if self.filtered_roles:
+            for role, count in self.filtered_roles.most_common(_TOP_N_BREAKDOWN):
+                lines.append(f"                        {role}={count}")
+        return "\n".join(lines)
+
+
+def _format_section(label: str, counter: Counter) -> list[str]:
+    """Render one counter section: header line + indented breakdown lines."""
+    total = sum(counter.values())
+    header = f"  {label} {total}"
+    if total == 0:
+        return [header, "                        (none)"]
+
+    lines = [header]
+    most_common = counter.most_common()
+    for raw_type, count in most_common[:_TOP_N_BREAKDOWN]:
+        lines.append(f"                        {raw_type}={count}")
+    if len(most_common) > _TOP_N_BREAKDOWN:
+        remainder = len(most_common) - _TOP_N_BREAKDOWN
+        lines.append(f"                        + {remainder} more types")
+    return lines

src/media.py

@@ -0,0 +1,202 @@
+"""Media resolution — download conversation assets next to the export files.
+
+Walks a normalized conversation's placeholder blocks, downloads each asset via
+the provider, saves it under a ``media/`` directory beside the conversation's
+Markdown file, and annotates the block with a relative ``local_path`` so the
+renderer emits a real inline image / file link instead of a placeholder.
+
+Policy (``EXPORTER_DOWNLOAD_MEDIA``):
+  images (default) — image_placeholder blocks only
+  all              — also file_placeholder blocks (voice-mode audio etc.;
+                     measured 2026-06-12: 556 clips ≈ 162MB whose transcripts
+                     are already in the exports as text)
+  off              — leave placeholders untouched
+
+Failures (expired assets, network) keep the existing placeholder and are
+counted in the run summary — never fatal to the export.
+"""
+
+import logging
+import os
+import tempfile
+from pathlib import Path
+
+from src.blocks import BLOCK_TYPE_FILE_PLACEHOLDER, BLOCK_TYPE_IMAGE_PLACEHOLDER
+from src.loss_report import LossReport
+from src.providers.base import ProviderError
+from src.utils import build_export_path, generate_filename
+
+logger = logging.getLogger(__name__)
+
+MEDIA_IMAGES = "images"
+MEDIA_ALL = "all"
+MEDIA_OFF = "off"
+VALID_MEDIA_POLICIES = {MEDIA_IMAGES, MEDIA_ALL, MEDIA_OFF}
+
+_EXT_BY_MIME = {
+    "image/png": ".png",
+    "image/jpeg": ".jpg",
+    "image/gif": ".gif",
+    "image/webp": ".webp",
+    "audio/wav": ".wav",
+    "audio/x-wav": ".wav",
+    "audio/mpeg": ".mp3",
+    "audio/mp4": ".m4a",
+    "audio/aac": ".aac",
+    "application/pdf": ".pdf",
+}
+
+
+def resolve_media_policy() -> str:
+    """Read EXPORTER_DOWNLOAD_MEDIA from the environment, defaulting to images."""
+    value = os.getenv("EXPORTER_DOWNLOAD_MEDIA", "").strip().lower()
+    if not value:
+        return MEDIA_IMAGES
+    if value not in VALID_MEDIA_POLICIES:
+        logger.warning(
+            "EXPORTER_DOWNLOAD_MEDIA=%r is invalid (expected images|all|off) "
+            "— using 'images'.",
+            value,
+        )
+        return MEDIA_IMAGES
+    return value
+
+
+def resolve_media(
+    normalized: dict,
+    provider,
+    export_base: Path,
+    structure: str,
+    policy: str,
+    report: LossReport,
+) -> int:
+    """Download assets for one normalized conversation. Returns download count.
+
+    Mutates placeholder blocks in place (adds ``local_path``). Idempotent:
+    an asset already on disk is annotated without an API call.
+    """
+    if policy == MEDIA_OFF:
+        return 0
+
+    download = getattr(provider, "download_asset", None)
+    if download is None:
+        # Provider has no remote assets (e.g. claude-code local transcripts).
+        return 0
+
+    wanted_types = {BLOCK_TYPE_IMAGE_PLACEHOLDER}
+    if policy == MEDIA_ALL:
+        wanted_types.add(BLOCK_TYPE_FILE_PLACEHOLDER)
+
+    targets = [
+        block
+        for message in normalized.get("messages", [])
+        for block in message.get("blocks", [])
+        if block.get("type") in wanted_types and block.get("ref")
+    ]
+    if not targets:
+        return 0
+
+    # Same path computation the exporter uses, so media/ lands beside the .md.
+    filename = generate_filename(
+        normalized.get("title", "Untitled"),
+        normalized.get("id", ""),
+        normalized.get("created_at") or "2000-01-01",
+    )
+    conv_dir = build_export_path(
+        export_base,
+        normalized.get("provider", ""),
+        normalized.get("project"),
+        normalized.get("created_at") or "2000-01-01",
+        filename,
+        structure,
+    ).parent
+    media_dir = conv_dir / "media"
+
+    downloaded = 0
+    for block in targets:
+        ref = block["ref"]
+        file_id = _safe_asset_name(provider, ref)
+        if not file_id:
+            report.record_media_failed("unparseable-ref")
+            continue
+
+        existing = _find_existing(media_dir, file_id)
+        if existing is not None:
+            block["local_path"] = f"media/{existing.name}"
+            continue
+
+        try:
+            content, mime, file_name = download(ref)
+        except ProviderError as e:
+            logger.warning(
+                "[media] Could not download %s: %s", ref[:60], e.original
+            )
+            reason = "expired-or-missing" if "404" in str(e.original) or "not found" in str(
+                e.original
+            ).lower() else "download-error"
+            report.record_media_failed(reason)
+            continue
+
+        ext = _pick_extension(mime, file_name)
+        target = media_dir / f"{file_id}{ext}"
+        try:
+            _write_atomic(target, content)
+        except OSError as e:
+            logger.error("[media] Could not write %s: %s", target, e)
+            report.record_media_failed("write-error")
+            continue
+
+        block["local_path"] = f"media/{target.name}"
+        report.record_media_downloaded(len(content))
+        downloaded += 1
+
+    return downloaded
+
+
+def _safe_asset_name(provider, ref: str) -> str | None:
+    """A stable, filesystem-safe name for the asset (the provider file ID)."""
+    parser = getattr(provider, "parse_asset_file_id", None)
+    if parser is None:
+        from src.providers.chatgpt import parse_asset_file_id as parser
+    file_id = parser(ref)
+    if not file_id:
+        return None
+    return "".join(c if c.isalnum() or c in "-_." else "_" for c in file_id)
+
+
+def _find_existing(media_dir: Path, file_id: str) -> Path | None:
+    """Return an already-downloaded file for this asset, if present."""
+    if not media_dir.is_dir():
+        return None
+    for candidate in media_dir.glob(f"{file_id}.*"):
+        if candidate.is_file() and candidate.stat().st_size > 0:
+            return candidate
+    return None
+
+
+def _pick_extension(mime: str | None, file_name: str | None) -> str:
+    if mime:
+        base_mime = mime.split(";")[0].strip().lower()
+        if base_mime in _EXT_BY_MIME:
+            return _EXT_BY_MIME[base_mime]
+    if file_name:
+        suffix = Path(file_name).suffix
+        if suffix and len(suffix) <= 8:
+            return suffix.lower()
+    return ".bin"
+
+
+def _write_atomic(target: Path, content: bytes) -> None:
+    target.parent.mkdir(parents=True, exist_ok=True)
+    fd, tmp_name = tempfile.mkstemp(dir=target.parent, suffix=".tmp")
+    try:
+        with os.fdopen(fd, "wb") as fh:
+            fh.write(content)
+        os.chmod(tmp_name, 0o600)
+        os.replace(tmp_name, target)
+    except OSError:
+        try:
+            os.unlink(tmp_name)
+        except OSError:
+            pass
+        raise

src/providers/base.py

@@ -1,6 +1,7 @@
 """Abstract base class for AI chat providers."""
 
 import logging
+import os
 import random
 import time
 from abc import ABC, abstractmethod
@@ -9,8 +10,24 @@ from typing import Any
 
 import requests
 
+from src.loss_report import LossReport
 from src.utils import redact_secrets
 
+# curl_cffi has its own exception hierarchy (rooted at CurlError → OSError),
+# completely separate from requests.exceptions. Import them so _make_request
+# can catch both when a curl_cffi session is in use.
+try:
+    from curl_cffi.requests.exceptions import (
+        HTTPError as _CurlHTTPError,
+        ConnectionError as _CurlConnectionError,
+        Timeout as _CurlTimeout,
+    )
+except ImportError:
+    # Fall back to requests types — catching them twice is harmless.
+    _CurlHTTPError = requests.HTTPError  # type: ignore[misc,assignment]
+    _CurlConnectionError = requests.ConnectionError  # type: ignore[misc,assignment]
+    _CurlTimeout = requests.Timeout  # type: ignore[misc,assignment]
+
 logger = logging.getLogger(__name__)
 
 # Request timeouts (connect, read) in seconds
@@ -21,6 +38,83 @@ MAX_RETRIES = 3
 BACKOFF_BASE = 2.0
 BACKOFF_MAX = 60.0
 
+# EXPORTER_HIDDEN_CONTENT policy — what to do with content that was invisible
+# in the provider's UI (retrieval dumps, hidden-flagged context, agent tool
+# traffic). Shared by all providers.
+HIDDEN_CONTENT_FULL = "full"
+HIDDEN_CONTENT_PLACEHOLDER = "placeholder"
+HIDDEN_CONTENT_OMIT = "omit"
+VALID_HIDDEN_CONTENT_POLICIES = {
+    HIDDEN_CONTENT_FULL,
+    HIDDEN_CONTENT_PLACEHOLDER,
+    HIDDEN_CONTENT_OMIT,
+}
+
+# ---------------------------------------------------------------------------
+# API-drift canary (FUTURE.md §10)
+# ---------------------------------------------------------------------------
+# The export depends on undocumented internal web APIs that can change shape
+# without notice. The canary fetches a small live sample and asserts only the
+# normalizer's load-bearing fields (NOT full shape — provider settings objects
+# churn constantly and would be pure noise). Findings carry a severity:
+#   ERROR — a load-bearing field is missing/mistyped; the parser will break or
+#           silently lose data. The backup is no longer trustworthy.
+#   WARN  — something unfamiliar appeared (new content_type, new tool author,
+#           a dormant field went live). Investigate; not necessarily broken.
+#   OK    — the assertion held.
+DRIFT_ERROR = "error"
+DRIFT_WARN = "warn"
+DRIFT_OK = "ok"
+
+
+def drift_finding(severity: str, check: str, detail: str = "") -> dict:
+    """Build one canary finding."""
+    return {"severity": severity, "check": check, "detail": detail}
+
+
+def resolve_hidden_content_policy() -> str:
+    """Read EXPORTER_HIDDEN_CONTENT from the environment, defaulting to placeholder."""
+    value = os.getenv("EXPORTER_HIDDEN_CONTENT", "").strip().lower()
+    if not value:
+        return HIDDEN_CONTENT_PLACEHOLDER
+    if value not in VALID_HIDDEN_CONTENT_POLICIES:
+        logger.warning(
+            "EXPORTER_HIDDEN_CONTENT=%r is invalid (expected full|placeholder|omit) "
+            "— using 'placeholder'.",
+            value,
+        )
+        return HIDDEN_CONTENT_PLACEHOLDER
+    return value
+
+
+# Default polite pacing between consecutive API requests (seconds). Keeps
+# traffic human-paced instead of bursty. REQUEST_DELAY=0 disables.
+DEFAULT_REQUEST_DELAY = 1.0
+
+
+def resolve_request_delay() -> float:
+    """Read REQUEST_DELAY from the environment, defaulting to DEFAULT_REQUEST_DELAY."""
+    raw = os.getenv("REQUEST_DELAY", "").strip()
+    if not raw:
+        return DEFAULT_REQUEST_DELAY
+    try:
+        value = float(raw)
+    except ValueError:
+        logger.warning(
+            "REQUEST_DELAY=%r is not a number — using default %.1fs.",
+            raw,
+            DEFAULT_REQUEST_DELAY,
+        )
+        return DEFAULT_REQUEST_DELAY
+    if value < 0:
+        logger.warning(
+            "REQUEST_DELAY=%r is negative — using default %.1fs.",
+            raw,
+            DEFAULT_REQUEST_DELAY,
+        )
+        return DEFAULT_REQUEST_DELAY
+    return value
+
 # Realistic Chrome User-Agent
 USER_AGENT = (
     "Mozilla/5.0 (X11; Linux x86_64) "
@@ -60,6 +154,9 @@ class BaseProvider(ABC):
                 "Accept-Language": "en-US,en;q=0.9",
             }
         )
+        # Polite pacing between consecutive requests (REQUEST_DELAY env).
+        self._request_delay = resolve_request_delay()
+        self._last_request_at: float | None = None
 
     # ------------------------------------------------------------------
     # Abstract interface — subclasses must implement these
@@ -74,13 +171,45 @@ class BaseProvider(ABC):
         """Return the full conversation detail for a single ID."""
 
     @abstractmethod
-    def normalize_conversation(self, raw: dict) -> dict:
-        """Transform provider-specific schema to the common normalized schema."""
+    def normalize_conversation(self, raw: dict, loss_report: LossReport | None = None) -> dict:
+        """Transform provider-specific schema to the common normalized schema.
+
+        ``loss_report`` accumulates counts of dropped/unhandled content so the
+        export loop can surface a single summary at the end. When None, providers
+        construct a throwaway local report (so calling normalize_conversation in
+        isolation, e.g. from tests, doesn't crash).
+        """
+
+    def check_drift(self) -> list[dict]:
+        """Probe the live API and assert the normalizer's load-bearing fields.
+
+        Returns a list of ``drift_finding`` dicts. The default is a no-op
+        (used by providers with no remote schema risk, e.g. local Claude Code
+        transcripts); web-API providers override this. See FUTURE.md §10.
+        """
+        return []
 
     # ------------------------------------------------------------------
     # Concrete helpers
     # ------------------------------------------------------------------
 
+    def _pace(self) -> None:
+        """Sleep so consecutive requests are at least REQUEST_DELAY apart (±25% jitter).
+
+        getattr fallback: tests construct providers via ``__new__``, skipping
+        ``__init__`` — those run unpaced.
+        """
+        delay = getattr(self, "_request_delay", 0)
+        if delay <= 0:
+            return
+        target = delay * random.uniform(0.75, 1.25)
+        last = getattr(self, "_last_request_at", None)
+        if last is not None:
+            wait = target - (time.monotonic() - last)
+            if wait > 0:
+                time.sleep(wait)
+        self._last_request_at = time.monotonic()
+
     def fetch_all_conversations(self, since: datetime | None = None) -> list[dict]:
         """Fetch every conversation, handling pagination automatically.
 
@@ -186,6 +315,7 @@ class BaseProvider(ABC):
 
         while attempt <= MAX_RETRIES:
             attempt += 1
+            self._pace()
             start = time.monotonic()
 
             try:
@@ -271,7 +401,7 @@ class BaseProvider(ABC):
             except ProviderError:
                 raise
 
-            except (requests.ConnectionError, requests.Timeout) as e:
+            except (requests.ConnectionError, requests.Timeout, _CurlConnectionError, _CurlTimeout) as e:
                 last_exc = e
                 if attempt > MAX_RETRIES:
                     raise ProviderError(
@@ -293,7 +423,7 @@ class BaseProvider(ABC):
                 )
                 time.sleep(wait)
 
-            except requests.HTTPError as e:
+            except (requests.HTTPError, _CurlHTTPError) as e:
                 raise ProviderError(
                     self.provider_name, f"{method} {url}", e
                 ) from e
@@ -311,7 +441,7 @@ class BaseProvider(ABC):
         msg = (
             f"[{self.provider_name}] Authentication failed (401 Unauthorized). "
             "Your session token has likely expired. "
-            "Run 'python -m src.main auth' to refresh your token."
+            "Run 'ai-chat-exporter auth' to refresh your token."
         )
         logger.error(msg)
         raise ProviderError(

src/providers/claude.py

@@ -3,25 +3,53 @@
 import logging
 import os
 
-from src.providers.base import BaseProvider, ProviderError
+from curl_cffi import requests as curl_requests
+
+from src.blocks import (
+    UNKNOWN_REASON_EXTRACTION_FAILED,
+    UNKNOWN_REASON_UNKNOWN_TYPE,
+    make_image_placeholder,
+    make_text_block,
+    make_thinking_block,
+    make_tool_result_block,
+    make_tool_use_block,
+    make_unknown_block,
+)
+from src.loss_report import LossReport
+from src.providers.base import (
+    BaseProvider,
+    DRIFT_ERROR,
+    DRIFT_OK,
+    DRIFT_WARN,
+    ProviderError,
+    drift_finding,
+)
 
 logger = logging.getLogger(__name__)
 
 BASE_URL = "https://claude.ai/api"
+IMPERSONATE = "chrome120"
 
 
 class ClaudeProvider(BaseProvider):
     """Provider for Claude conversations via the internal web API.
 
-    Authentication: Cookie: sessionKey=<CLAUDE_SESSION_KEY>
-    Token: sessionKey cookie value from claude.ai.
-    Typical validity: ~30 days (opaque; expiry cannot be decoded client-side).
+    Uses curl_cffi to impersonate Chrome's TLS fingerprint, bypassing
+    Cloudflare's bot detection (same issue as chatgpt.com).
+
+    Authentication: sessionKey cookie (~30 day lifetime, opaque string).
+    Expiry cannot be decoded client-side — a 401 is the only signal.
     """
 
     provider_name = "claude"
 
     def __init__(self, session_key: str | None = None) -> None:
-        super().__init__()
+        cf_session = curl_requests.Session(impersonate=IMPERSONATE)
+        super().__init__(session=cf_session)  # type: ignore[arg-type]
+
+        # Remove base class User-Agent so curl_cffi uses its Chrome-matched UA
+        self._session.headers.pop("User-Agent", None)
+
         key = session_key or os.getenv("CLAUDE_SESSION_KEY", "").strip()
         if not key:
             raise ProviderError(
@@ -29,19 +57,19 @@ class ClaudeProvider(BaseProvider):
                 "init",
                 RuntimeError(
                     "CLAUDE_SESSION_KEY is not set. "
-                    "Run 'python -m src.main auth' to configure it."
+                    "Run 'ai-chat-exporter auth' to configure it."
                 ),
             )
-        # Set cookie header; never log the key value
+        # Set sessionKey in the cookie jar
+        self._session.cookies.set("sessionKey", key, domain="claude.ai", path="/")
         self._session.headers.update(
             {
-                "Cookie": f"sessionKey={key}",
                 "Referer": "https://claude.ai/",
                 "Origin": "https://claude.ai",
             }
         )
         self._org_id: str | None = None  # cached per session
-        logger.debug("[claude] Session initialised (key: [REDACTED])")
+        logger.debug("[claude] Session initialised with Chrome TLS impersonation (key: [REDACTED])")
 
     def _handle_401(self) -> None:
         msg = (
@@ -50,7 +78,7 @@ class ClaudeProvider(BaseProvider):
             "Note: Claude session keys are opaque — a 401 is the only expiry signal. "
             "To refresh: open claude.ai in Chrome → F12 → Application → Cookies "
             "→ find 'sessionKey' → copy the value. "
-            "Then run 'python -m src.main auth' or update CLAUDE_SESSION_KEY in .env."
+            "Then run 'ai-chat-exporter auth' or update CLAUDE_SESSION_KEY in .env."
         )
         logger.error(msg)
         raise ProviderError(
@@ -151,8 +179,9 @@ class ClaudeProvider(BaseProvider):
 
         return data
 
-    def normalize_conversation(self, raw: dict) -> dict:
+    def normalize_conversation(self, raw: dict, loss_report: LossReport | None = None) -> dict:
         """Transform Claude raw schema to the common normalized schema."""
+        report = loss_report if loss_report is not None else LossReport()
         conv_id = raw.get("uuid") or raw.get("id", "")
         title = raw.get("name") or raw.get("title") or "Untitled"
         created_at = raw.get("created_at") or raw.get("create_time") or ""
@@ -168,40 +197,37 @@ class ClaudeProvider(BaseProvider):
 
         # Messages
         raw_messages = raw.get("chat_messages") or raw.get("messages") or []
-        messages = []
+        messages: list[dict] = []
 
         for msg in raw_messages:
             role = _map_role(msg.get("sender") or msg.get("role", ""))
             if not role:
                 continue
 
-            # Content can be a string or a list of content blocks
-            content_raw = msg.get("content") or msg.get("text") or ""
-            content, skipped_types = _extract_claude_text(content_raw, conv_id)
-
-            for ctype in skipped_types:
-                logger.warning(
-                    "[claude] Skipping %s content in conversation %s "
-                    "— rich content not yet supported (see FUTURE.md)",
-                    ctype,
-                    conv_id[:8],
-                )
+            content_raw = msg.get("content") if "content" in msg else msg.get("text", "")
+            blocks = _extract_claude_blocks(content_raw, conv_id, report)
 
             timestamp = msg.get("created_at") or msg.get("timestamp") or None
 
-            if content is None:
+            if not blocks:
                 logger.debug("[claude] Skipping empty message in conversation %s", conv_id[:8])
                 continue
 
+            content_type = "text"
+
             messages.append(
                 {
                     "role": role,
-                    "content": content,
-                    "content_type": "text",
+                    "content_type": content_type,
                     "timestamp": timestamp,
+                    "blocks": blocks,
                 }
             )
 
+        for _ in messages:
+            report.record_message()
+        report.record_conversation()
+
         return {
             "id": conv_id,
             "title": title,
@@ -213,6 +239,70 @@ class ClaudeProvider(BaseProvider):
             "messages": messages,
         }
 
+    def check_drift(self) -> list[dict]:
+        """Probe one listing page + one conversation; assert load-bearing fields.
+
+        See FUTURE.md §10. Real Claude messages are flat ``text``/``sender``;
+        the rich ``content`` block path is dormant, so its activation (content
+        becoming a list) is itself a drift event worth flagging. ``attachments``
+        / ``files`` are ignored by the normalizer — non-empty values are a
+        silent-loss risk.
+        """
+        findings: list[dict] = []
+        page = self.list_conversations(offset=0, limit=5)
+        if not page:
+            return [drift_finding(DRIFT_WARN, "listing",
+                                  "empty listing — cannot verify shape")]
+
+        item = page[0]
+        if not (item.get("uuid") or item.get("id")):
+            findings.append(drift_finding(DRIFT_ERROR, "listing", "summary missing uuid/id"))
+        if not (item.get("name") or item.get("title")):
+            findings.append(drift_finding(DRIFT_WARN, "listing", "no name/title on summary"))
+        if not (item.get("updated_at") or item.get("update_time")):
+            findings.append(drift_finding(DRIFT_WARN, "listing", "no updated_at on summary"))
+
+        conv_id = item.get("uuid") or item.get("id")
+        if not conv_id:
+            return findings or [drift_finding(DRIFT_ERROR, "listing", "no id to fetch")]
+        raw = self.get_conversation(conv_id)
+        if not (raw.get("uuid") or raw.get("id")):
+            findings.append(drift_finding(DRIFT_ERROR, "detail", "no uuid/id on detail"))
+        msgs = raw.get("chat_messages") or raw.get("messages")
+        if not isinstance(msgs, list) or not msgs:
+            findings.append(drift_finding(
+                DRIFT_ERROR, "detail", "chat_messages missing/empty — 0 messages"))
+            return findings
+
+        saw_sender = saw_text = False
+        flagged_list = flagged_attach = False
+        for m in msgs:
+            if m.get("sender") or m.get("role"):
+                saw_sender = True
+            content = m.get("content")
+            if isinstance(content, list) and not flagged_list:
+                findings.append(drift_finding(
+                    DRIFT_WARN, "content",
+                    "message.content is now a LIST — Claude switched to typed blocks; "
+                    "the dormant _dispatch_claude_block path is now live, verify rendering"))
+                flagged_list = True
+            if m.get("text") or content:
+                saw_text = True
+            if (m.get("attachments") or m.get("files")) and not flagged_attach:
+                findings.append(drift_finding(
+                    DRIFT_WARN, "attachments",
+                    "message has non-empty attachments/files — the normalizer ignores "
+                    "these (silent loss); add handling"))
+                flagged_attach = True
+
+        if not saw_sender:
+            findings.append(drift_finding(DRIFT_ERROR, "messages", "no sender/role on any message"))
+        if not saw_text:
+            findings.append(drift_finding(DRIFT_WARN, "messages", "no text/content found"))
+        if not findings:
+            findings.append(drift_finding(DRIFT_OK, "schema", "all load-bearing fields present"))
+        return findings
+
 
 # ---------------------------------------------------------------------------
 # Internal helpers
@@ -232,43 +322,134 @@ def _map_role(sender: str) -> str | None:
     return mapping.get(sender.lower()) if sender else None
 
 
-def _extract_claude_text(
-    content: str | list | dict, conv_id: str
-) -> tuple[str | None, list[str]]:
-    """Extract plain text from a Claude content field.
+def _extract_claude_blocks(
+    content: str | list | dict | None, conv_id: str, report: LossReport
+) -> list[dict]:
+    """Extract typed blocks from a Claude content field.
 
-    Returns:
-        (text_or_None, list_of_skipped_content_types)
+    Defensive dispatch — zero observed cases of rich Claude content in the
+    user's archive at planning time, so this is theory-only. Real shapes
+    will be locked in v0.4.1 once captured. Any unrecognised block type
+    surfaces as an `unknown` block + WARNING + tally.
     """
-    skipped: list[str] = []
+    if content is None:
+        return []
 
     if isinstance(content, str):
-        text = content.strip()
-        return (text if text else None), skipped
+        block = make_text_block(content)
+        return [block] if block else []
 
     if isinstance(content, list):
-        parts: list[str] = []
-        for block in content:
-            if isinstance(block, str):
-                parts.append(block)
-            elif isinstance(block, dict):
-                btype = block.get("type", "text")
-                if btype == "text":
-                    t = block.get("text", "").strip()
-                    if t:
-                        parts.append(t)
-                else:
-                    skipped.append(btype)
-        text = "\n".join(parts).strip()
-        return (text if text else None), skipped
+        blocks: list[dict] = []
+        for item in content:
+            if isinstance(item, str):
+                block = make_text_block(item)
+                if block:
+                    blocks.append(block)
+            elif isinstance(item, dict):
+                blocks.extend(_dispatch_claude_block(item, conv_id, report))
+        return blocks
 
     if isinstance(content, dict):
-        btype = content.get("type", "text")
-        if btype == "text":
-            text = content.get("text", "").strip()
-            return (text if text else None), skipped
-        else:
-            skipped.append(btype)
-            return None, skipped
+        return _dispatch_claude_block(content, conv_id, report)
 
-    return None, skipped
+    return []
+
+
+def _dispatch_claude_block(block: dict, conv_id: str, report: LossReport) -> list[dict]:
+    """Translate one raw Claude content block into normalized blocks."""
+    btype = block.get("type", "text")
+
+    if btype == "text":
+        block_obj = make_text_block(block.get("text", "") or "")
+        return [block_obj] if block_obj else []
+
+    if btype == "thinking":
+        # Claude extended-thinking blocks may use 'thinking' or 'text' field.
+        text = block.get("thinking") or block.get("text") or ""
+        block_obj = make_thinking_block(text)
+        return [block_obj] if block_obj else []
+
+    if btype == "tool_use":
+        return [
+            make_tool_use_block(
+                name=block.get("name", "") or "",
+                input_data=block.get("input"),
+                tool_id=block.get("id"),
+            )
+        ]
+
+    if btype == "tool_result":
+        # ``content`` may be a string or a list of nested blocks (recursive).
+        nested = block.get("content")
+        output = _flatten_tool_result_content(nested, conv_id, report)
+        return [
+            make_tool_result_block(
+                output=output,
+                tool_name=None,
+                is_error=bool(block.get("is_error")),
+            )
+        ]
+
+    if btype == "image":
+        # Source shape is unverified; try the most likely fields.
+        source = block.get("source") or {}
+        ref = ""
+        if isinstance(source, dict):
+            ref = (
+                source.get("file_uuid")
+                or source.get("media_type")
+                or source.get("url")
+                or ""
+            )
+        return [make_image_placeholder(ref=ref or "(unknown)", source="user_upload")]
+
+    # Unknown block type
+    keys = list(block.keys())
+    logger.warning(
+        "[claude] Unknown block type %r in conversation %s "
+        "— see plan §Data-loss visibility (rendering as unknown block)",
+        btype,
+        conv_id[:8],
+    )
+    report.record_unknown(btype or "?")
+    return [
+        make_unknown_block(
+            raw_type=btype or "?",
+            observed_keys=keys,
+            reason=UNKNOWN_REASON_UNKNOWN_TYPE,
+        )
+    ]
+
+
+def _flatten_tool_result_content(
+    nested: object, conv_id: str, report: LossReport
+) -> str:
+    """Flatten Claude tool_result content (string OR list of nested blocks) to text.
+
+    Recurses into nested text blocks; any non-text nested block becomes a
+    visible inline marker so non-text content isn't silently dropped.
+    """
+    if nested is None:
+        return ""
+    if isinstance(nested, str):
+        return nested
+    if isinstance(nested, list):
+        chunks: list[str] = []
+        for item in nested:
+            if isinstance(item, str):
+                chunks.append(item)
+            elif isinstance(item, dict):
+                btype = item.get("type", "text")
+                if btype == "text":
+                    chunks.append(item.get("text", "") or "")
+                else:
+                    keys = list(item.keys())[:10]
+                    report.record_extraction_failure(f"tool_result.{btype}")
+                    chunks.append(
+                        f"[Unsupported nested {btype} block; keys={keys}]"
+                    )
+        return "\n".join(c for c in chunks if c)
+    if isinstance(nested, dict):
+        return _flatten_tool_result_content([nested], conv_id, report)
+    return str(nested)

src/providers/claude_code.py

@@ -0,0 +1,476 @@
+"""Claude Code session provider — archives local agent transcripts.
+
+Reads JSONL session files from ``~/.claude/projects/<munged-cwd>/<uuid>.jsonl``
+(override with ``CLAUDE_CODE_DIR``). No tokens, no rate limits, no ToS risk —
+but the data lives in single files Claude Code may clean up, and it contains
+deliverables (reviews, plans, analyses) that exist nowhere else.
+
+Rendering follows the EXPORTER_HIDDEN_CONTENT policy (decided 2026-06-12):
+prose-only by default. User prompts and assistant text are kept; tool_use /
+tool_result traffic is collapsed to one grouped placeholder per activity run
+(measured: dialogue prose is ~4% of session bytes); thinking blocks are
+dropped (counted in the run summary, no placeholder). ``full`` keeps
+everything.
+
+Record types in a session file: ``user`` / ``assistant`` carry the dialogue
+(Anthropic-style ``message.content`` block arrays); ``ai-title`` carries the
+evolving session title (last one wins); ``last-prompt``,
+``file-history-snapshot``, ``attachment``, ``permission-mode``, ``system``
+are harness records and are skipped. Records flagged ``isSidechain`` are
+subagent transcripts; ``isMeta`` are harness-generated user records — both
+skipped.
+"""
+
+import json
+import logging
+import os
+import re
+from collections import Counter
+from datetime import datetime, timezone
+from pathlib import Path
+
+from src.blocks import (
+    COLLAPSED_KIND_TOOL_DUMP,
+    UNKNOWN_REASON_UNKNOWN_TYPE,
+    make_collapsed_block,
+    make_image_placeholder,
+    make_text_block,
+    make_thinking_block,
+    make_tool_result_block,
+    make_tool_use_block,
+    make_unknown_block,
+)
+from src.loss_report import LossReport
+from src.providers.base import (
+    BaseProvider,
+    HIDDEN_CONTENT_FULL,
+    ProviderError,
+    VALID_HIDDEN_CONTENT_POLICIES,
+    resolve_hidden_content_policy,
+)
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_PROJECTS_DIR = "~/.claude/projects"
+
+# Harness-injected tags inside user message text. Stripped so exports contain
+# the dialogue, not the CLI plumbing. A record that is nothing but tags
+# (e.g. a /model invocation) ends up empty and is skipped.
+_HARNESS_TAG_RE = re.compile(
+    r"<local-command-caveat>.*?</local-command-caveat>"
+    r"|<command-name>.*?</command-name>"
+    r"|<command-message>.*?</command-message>"
+    r"|<command-args>.*?</command-args>"
+    r"|<command-contents>.*?</command-contents>"
+    r"|<local-command-stdout>.*?</local-command-stdout>"
+    r"|<system-reminder>.*?</system-reminder>",
+    re.DOTALL,
+)
+
+# How many distinct tool names to list in a collapsed-activity placeholder.
+_TOOL_NAMES_SHOWN = 4
+
+
+def _strip_harness_noise(text: str) -> str:
+    if not isinstance(text, str):
+        return ""
+    return _HARNESS_TAG_RE.sub("", text).strip()
+
+
+class ClaudeCodeProvider(BaseProvider):
+    """Local-file provider over Claude Code session transcripts."""
+
+    provider_name = "claude-code"
+
+    def __init__(
+        self,
+        projects_dir: str | Path | None = None,
+        hidden_content: str | None = None,
+    ) -> None:
+        super().__init__()
+        self._projects_dir = Path(
+            projects_dir or os.getenv("CLAUDE_CODE_DIR", DEFAULT_PROJECTS_DIR)
+        ).expanduser()
+        self._hidden_content = (
+            hidden_content
+            if hidden_content in VALID_HIDDEN_CONTENT_POLICIES
+            else resolve_hidden_content_policy()
+        )
+        # conv_id → session file path, populated by _scan()
+        self._path_map: dict[str, Path] = {}
+
+    # ------------------------------------------------------------------
+    # BaseProvider interface
+    # ------------------------------------------------------------------
+
+    def list_conversations(self, offset: int = 0, limit: int = 100) -> list[dict]:
+        full = self._scan()
+        return full[offset : offset + limit]
+
+    def fetch_all_conversations(self, since: datetime | None = None) -> list[dict]:
+        convs = self._scan()
+        if since is not None:
+            since_aware = since if since.tzinfo else since.replace(tzinfo=timezone.utc)
+            convs = [
+                c for c in convs
+                if datetime.fromisoformat(c["updated_at"]) >= since_aware
+            ]
+        logger.info(
+            "[claude-code] Found %d session(s) under %s", len(convs), self._projects_dir
+        )
+        return convs
+
+    def get_conversation(self, conv_id: str) -> dict:
+        path = self._path_map.get(conv_id)
+        if path is None:
+            # Direct call without a prior listing (e.g. tests) — scan first.
+            self._scan()
+            path = self._path_map.get(conv_id)
+        if path is None or not path.exists():
+            raise ProviderError(
+                self.provider_name,
+                f"get_conversation({conv_id[:8]})",
+                FileNotFoundError(f"No session file for id {conv_id}"),
+            )
+
+        records: list[dict] = []
+        bad_lines = 0
+        for line in path.read_text(encoding="utf-8").splitlines():
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                records.append(json.loads(line))
+            except json.JSONDecodeError:
+                bad_lines += 1
+        if bad_lines:
+            logger.warning(
+                "[claude-code] %s: skipped %d unparseable line(s)", path.name, bad_lines
+            )
+
+        mtime = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
+        return {
+            "id": conv_id,
+            "_path": str(path),
+            "_records": records,
+            # Listing and normalized updated_at must match, or the cache
+            # staleness comparison would re-export every session every run.
+            "_mtime_iso": mtime.isoformat(),
+        }
+
+    def normalize_conversation(self, raw: dict, loss_report: LossReport | None = None) -> dict:
+        report = loss_report if loss_report is not None else LossReport()
+        policy = getattr(self, "_hidden_content", None) or resolve_hidden_content_policy()
+        conv_id = raw.get("id") or ""
+        records: list[dict] = raw.get("_records") or []
+
+        title = _extract_title(records)
+        project = _extract_project(records, raw.get("_path"))
+        created_at = next(
+            (r.get("timestamp") for r in records if r.get("timestamp")), ""
+        )
+        updated_at = raw.get("_mtime_iso") or next(
+            (r.get("timestamp") for r in reversed(records) if r.get("timestamp")), ""
+        )
+
+        messages = _extract_messages(records, conv_id, report, policy)
+        for _ in messages:
+            report.record_message()
+        report.record_conversation()
+
+        return {
+            "id": conv_id,
+            "title": title,
+            "provider": self.provider_name,
+            "project": project,
+            "created_at": created_at or "",
+            "updated_at": updated_at or "",
+            "message_count": len(messages),
+            "messages": messages,
+        }
+
+    # ------------------------------------------------------------------
+    # Scanning
+    # ------------------------------------------------------------------
+
+    def _scan(self) -> list[dict]:
+        if not self._projects_dir.is_dir():
+            logger.warning(
+                "[claude-code] Projects directory %s does not exist", self._projects_dir
+            )
+            return []
+
+        convs: list[dict] = []
+        for proj_dir in sorted(p for p in self._projects_dir.iterdir() if p.is_dir()):
+            for session_file in sorted(proj_dir.glob("*.jsonl")):
+                try:
+                    stat = session_file.stat()
+                except OSError:
+                    continue
+                if stat.st_size == 0:
+                    continue
+                conv_id = session_file.stem
+                self._path_map[conv_id] = session_file
+                title, project, created = _read_session_meta(session_file)
+                convs.append(
+                    {
+                        "id": conv_id,
+                        "title": title,
+                        "project": project,
+                        # The --project filter and dry-run table read the
+                        # listing dict, not the normalized conversation.
+                        "_project_name": project,
+                        "created_at": created,
+                        "updated_at": datetime.fromtimestamp(
+                            stat.st_mtime, tz=timezone.utc
+                        ).isoformat(),
+                        "_path": str(session_file),
+                        "_project_dir": proj_dir.name,
+                    }
+                )
+        return convs
+
+
+# ---------------------------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------------------------
+
+
+def _read_session_meta(path: Path) -> tuple[str, str | None, str]:
+    """Light single-pass scan for listing metadata: (title, project, created_at).
+
+    Substring guards keep this cheap — only candidate lines are JSON-parsed.
+    The full-fidelity extraction happens later in normalize_conversation.
+    """
+    title = ""
+    project: str | None = None
+    created = ""
+    try:
+        with path.open(encoding="utf-8") as fh:
+            for line in fh:
+                if '"ai-title"' in line:
+                    try:
+                        rec = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    if rec.get("type") == "ai-title" and rec.get("aiTitle"):
+                        title = str(rec["aiTitle"])  # last one wins
+                    continue
+                if (not created or project is None) and (
+                    '"timestamp"' in line or '"cwd"' in line
+                ):
+                    try:
+                        rec = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    if not created and rec.get("timestamp"):
+                        created = str(rec["timestamp"])
+                    if project is None and rec.get("cwd"):
+                        project = Path(rec["cwd"]).name or None
+    except OSError as e:
+        logger.warning("[claude-code] Could not read %s: %s", path, e)
+    return title, project, created
+
+
+def _extract_title(records: list[dict]) -> str:
+    """Last ai-title record wins; fall back to the first real user prompt."""
+    title = ""
+    for rec in records:
+        if rec.get("type") == "ai-title" and rec.get("aiTitle"):
+            title = str(rec["aiTitle"])
+    if title:
+        return title
+
+    for rec in records:
+        if rec.get("type") != "user" or rec.get("isMeta") or rec.get("isSidechain"):
+            continue
+        content = (rec.get("message") or {}).get("content")
+        if isinstance(content, str):
+            text = _strip_harness_noise(content)
+        elif isinstance(content, list):
+            text = " ".join(
+                _strip_harness_noise(item.get("text", ""))
+                for item in content
+                if isinstance(item, dict) and item.get("type") == "text"
+            ).strip()
+        else:
+            text = ""
+        if text:
+            return text[:80]
+    return "Untitled session"
+
+
+def _extract_project(records: list[dict], path: str | None) -> str | None:
+    """Project = basename of the session's working directory."""
+    for rec in records:
+        cwd = rec.get("cwd")
+        if cwd:
+            name = Path(cwd).name
+            if name:
+                return name
+    # Fallback: the munged directory name (cannot be reliably de-munged
+    # because '-' is both the path separator and a legal name character).
+    if path:
+        return Path(path).parent.name.lstrip("-") or None
+    return None
+
+
+def _extract_messages(
+    records: list[dict], conv_id: str, report: LossReport, policy: str
+) -> list[dict]:
+    messages: list[dict] = []
+    # Pending collapsed tool activity: name → call count, plus total bytes.
+    pending_tools: Counter = Counter()
+    pending_bytes = 0
+
+    def flush_pending() -> None:
+        nonlocal pending_bytes
+        if not pending_tools:
+            return
+        shown = ", ".join(
+            f"{name} ×{count}" for name, count in pending_tools.most_common(_TOOL_NAMES_SHOWN)
+        )
+        if len(pending_tools) > _TOOL_NAMES_SHOWN:
+            shown += ", …"
+        calls = sum(pending_tools.values())
+        block = make_collapsed_block(
+            origin=f"{calls} calls: {shown}",
+            content_type="tool_activity",
+            size_bytes=pending_bytes,
+            kind=COLLAPSED_KIND_TOOL_DUMP,
+        )
+        if messages and messages[-1]["role"] == "assistant":
+            messages[-1]["blocks"].append(block)
+        else:
+            messages.append(
+                {"role": "tool", "content_type": "text", "timestamp": None, "blocks": [block]}
+            )
+        pending_tools.clear()
+        pending_bytes = 0
+
+    for rec in records:
+        if rec.get("type") not in ("user", "assistant"):
+            continue
+        if rec.get("isSidechain") or rec.get("isMeta"):
+            continue
+
+        msg = rec.get("message") or {}
+        role = msg.get("role") or rec["type"]
+        content = msg.get("content")
+        blocks: list[dict] = []
+        # This record's own tool traffic — merged into pending only after the
+        # record's message is appended, so the placeholder lands on it (not on
+        # the previous message).
+        local_tools: Counter = Counter()
+        local_bytes = 0
+
+        if isinstance(content, str):
+            text = _strip_harness_noise(content)
+            block = make_text_block(text)
+            if block:
+                blocks.append(block)
+        elif isinstance(content, list):
+            for item in content:
+                if not isinstance(item, dict):
+                    continue
+                item_type = item.get("type", "")
+                if item_type == "text":
+                    block = make_text_block(_strip_harness_noise(item.get("text", "")))
+                    if block:
+                        blocks.append(block)
+                elif item_type in ("thinking", "redacted_thinking"):
+                    if policy == HIDDEN_CONTENT_FULL:
+                        block = make_thinking_block(
+                            item.get("thinking") or item.get("text") or ""
+                        )
+                        if block:
+                            blocks.append(block)
+                    else:
+                        # Decision 2026-06-12: thinking is dropped without a
+                        # placeholder, but stays visible in the run summary.
+                        report.record_collapsed(
+                            "thinking", len(json.dumps(item, default=str))
+                        )
+                elif item_type == "tool_use":
+                    if policy == HIDDEN_CONTENT_FULL:
+                        blocks.append(
+                            make_tool_use_block(
+                                item.get("name", ""), item.get("input"), item.get("id")
+                            )
+                        )
+                    else:
+                        name = item.get("name") or "tool"
+                        size = len(json.dumps(item, default=str))
+                        local_tools[name] += 1
+                        local_bytes += size
+                        report.record_collapsed(name, size)
+                elif item_type == "tool_result":
+                    if policy == HIDDEN_CONTENT_FULL:
+                        blocks.append(
+                            make_tool_result_block(
+                                _stringify_tool_result(item.get("content")),
+                                is_error=bool(item.get("is_error")),
+                            )
+                        )
+                    else:
+                        size = len(json.dumps(item, default=str))
+                        local_bytes += size
+                        report.record_collapsed("tool_result", size)
+                elif item_type == "image":
+                    blocks.append(
+                        make_image_placeholder(ref="embedded image", source="user_upload")
+                    )
+                else:
+                    logger.warning(
+                        "[claude-code] Unknown content block type %r in session %s",
+                        item_type,
+                        conv_id[:8],
+                    )
+                    report.record_unknown(f"claude-code.{item_type or '?'}")
+                    blocks.append(
+                        make_unknown_block(
+                            raw_type=f"claude-code.{item_type or '?'}",
+                            observed_keys=list(item.keys()),
+                            reason=UNKNOWN_REASON_UNKNOWN_TYPE,
+                        )
+                    )
+
+        if not blocks:
+            # Tool-result-only records (and similar): traffic accumulates and
+            # is attached to the message that initiated it.
+            pending_tools.update(local_tools)
+            pending_bytes += local_bytes
+            continue
+
+        # Attach previous records' tool activity to the previous message
+        # before starting a new one, so the placeholder lands between the
+        # dialogue turns it actually occurred between.
+        flush_pending()
+        messages.append(
+            {
+                "role": role,
+                "content_type": "text",
+                "timestamp": rec.get("timestamp"),
+                "blocks": blocks,
+            }
+        )
+        pending_tools.update(local_tools)
+        pending_bytes += local_bytes
+
+    flush_pending()
+    return messages
+
+
+def _stringify_tool_result(content) -> str:
+    """tool_result content may be a string or a list of text blocks."""
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts = []
+        for item in content:
+            if isinstance(item, dict) and item.get("type") == "text":
+                parts.append(item.get("text", ""))
+            else:
+                parts.append(json.dumps(item, default=str))
+        return "\n".join(parts)
+    return json.dumps(content, default=str) if content is not None else ""

src/utils.py

@@ -50,7 +50,7 @@ def build_export_path(
         created_at: ISO8601 creation timestamp (used for year folder).
         filename: Already-generated filename from generate_filename().
         structure: OUTPUT_STRUCTURE value. One of:
-            "provider/project/year" (default)
+            "provider/project/year" (default) — project and year combined, e.g. no-project.2025/
             "provider/project"
             "provider/year"
 
@@ -64,14 +64,14 @@ def build_export_path(
     parts: list[str] = [provider]
 
     if structure == "provider/project/year":
-        parts += [project_slug, year]
+        parts += [f"{project_slug}.{year}"]
     elif structure == "provider/project":
         parts += [project_slug]
     elif structure == "provider/year":
         parts += [year]
     else:
         # Unknown structure — fall back to default
-        parts += [project_slug, year]
+        parts += [f"{project_slug}.{year}"]
 
     return base_dir.joinpath(*parts) / filename
 

tests/fixtures/chatgpt_conversation.json

@@ -8,12 +8,30 @@
     "node-root": {
       "id": "node-root",
       "parent": null,
-      "children": ["node-1"],
+      "children": ["node-uec"],
       "message": null
     },
+    "node-uec": {
+      "id": "node-uec",
+      "parent": "node-root",
+      "children": ["node-1"],
+      "message": {
+        "id": "node-uec",
+        "author": {"role": "user"},
+        "create_time": null,
+        "content": {
+          "content_type": "user_editable_context",
+          "user_profile": "Preferred name: Jesse",
+          "user_instructions": "The user provided the additional info about how they would like you to respond:\n```Always cite sources.```"
+        },
+        "metadata": {
+          "is_visually_hidden_from_conversation": true
+        }
+      }
+    },
     "node-1": {
       "id": "node-1",
-      "parent": "node-root",
+      "parent": "node-uec",
       "children": ["node-2"],
       "message": {
         "id": "node-1",
@@ -28,7 +46,7 @@
     "node-2": {
       "id": "node-2",
       "parent": "node-1",
-      "children": ["node-3"],
+      "children": ["node-mm-user"],
       "message": {
         "id": "node-2",
         "author": {"role": "assistant"},
@@ -39,19 +57,140 @@
         }
       }
     },
-    "node-3": {
-      "id": "node-3",
+    "node-mm-user": {
+      "id": "node-mm-user",
       "parent": "node-2",
-      "children": [],
+      "children": ["node-mm-assistant"],
       "message": {
-        "id": "node-3",
+        "id": "node-mm-user",
         "author": {"role": "user"},
         "create_time": 1704067300.0,
         "content": {
-          "content_type": "image_asset_pointer",
-          "parts": [{"content_type": "image_asset_pointer", "asset_pointer": "file://some-image"}]
+          "content_type": "multimodal_text",
+          "parts": [
+            {"content_type": "audio_transcription", "text": "What is the capital of France?", "direction": "in", "decoding_id": null},
+            {"content_type": "real_time_user_audio_video_asset_pointer", "frames_asset_pointers": [], "video_container_asset_pointer": null, "audio_asset_pointer": {"content_type": "audio_asset_pointer", "asset_pointer": "sediment://file_user001", "size_bytes": 50000, "format": "wav", "metadata": {"start": 0.0, "end": 2.5}}, "audio_start_timestamp": 1.0}
+          ]
+        },
+        "metadata": {"voice_mode_message": true}
+      }
+    },
+    "node-mm-assistant": {
+      "id": "node-mm-assistant",
+      "parent": "node-mm-user",
+      "children": ["node-mm-user-rev"],
+      "message": {
+        "id": "node-mm-assistant",
+        "author": {"role": "assistant"},
+        "create_time": 1704067305.0,
+        "content": {
+          "content_type": "multimodal_text",
+          "parts": [
+            {"content_type": "audio_transcription", "text": "The capital of France is Paris.", "direction": "out", "decoding_id": null},
+            {"content_type": "audio_asset_pointer", "asset_pointer": "sediment://file_assistant001", "size_bytes": 80000, "format": "wav", "metadata": {"start": 0.0, "end": 3.2}}
+          ]
         }
       }
+    },
+    "node-mm-user-rev": {
+      "id": "node-mm-user-rev",
+      "parent": "node-mm-assistant",
+      "children": ["node-image-only"],
+      "message": {
+        "id": "node-mm-user-rev",
+        "author": {"role": "user"},
+        "create_time": 1704067400.0,
+        "content": {
+          "content_type": "multimodal_text",
+          "parts": [
+            {"content_type": "real_time_user_audio_video_asset_pointer", "frames_asset_pointers": [], "video_container_asset_pointer": null, "audio_asset_pointer": {"content_type": "audio_asset_pointer", "asset_pointer": "sediment://file_user002", "size_bytes": 30000, "format": "wav", "metadata": {"start": 0.0, "end": 1.5}}, "audio_start_timestamp": 5.0},
+            {"content_type": "audio_transcription", "text": "Tell me more please.", "direction": "in", "decoding_id": null}
+          ]
+        }
+      }
+    },
+    "node-image-only": {
+      "id": "node-image-only",
+      "parent": "node-mm-user-rev",
+      "children": ["node-exec-output"],
+      "message": {
+        "id": "node-image-only",
+        "author": {"role": "user"},
+        "create_time": 1704067500.0,
+        "content": {
+          "content_type": "multimodal_text",
+          "parts": [
+            {"content_type": "image_asset_pointer", "asset_pointer": "file-service://image001"}
+          ]
+        }
+      }
+    },
+    "node-exec-output": {
+      "id": "node-exec-output",
+      "parent": "node-image-only",
+      "children": ["node-exec-output-empty"],
+      "message": {
+        "id": "node-exec-output",
+        "author": {"role": "tool", "name": "container.exec", "metadata": {}},
+        "create_time": 1704067600.0,
+        "content": {
+          "content_type": "execution_output",
+          "text": "Hello from container.exec\nLine 2 of output"
+        },
+        "metadata": {
+          "aggregate_result": {"status": "success", "messages": []},
+          "reasoning_title": "Reading skill documentation"
+        }
+      }
+    },
+    "node-exec-output-empty": {
+      "id": "node-exec-output-empty",
+      "parent": "node-exec-output",
+      "children": ["node-system-error"],
+      "message": {
+        "id": "node-exec-output-empty",
+        "author": {"role": "tool", "name": "python", "metadata": {}},
+        "create_time": 1704067610.0,
+        "content": {
+          "content_type": "execution_output",
+          "text": ""
+        },
+        "metadata": {}
+      }
+    },
+    "node-system-error": {
+      "id": "node-system-error",
+      "parent": "node-exec-output-empty",
+      "children": ["node-tether-spinner"],
+      "message": {
+        "id": "node-system-error",
+        "author": {"role": "tool", "name": "web", "metadata": {}},
+        "create_time": 1704067620.0,
+        "content": {
+          "content_type": "system_error",
+          "name": "tool_error",
+          "text": "Error: Error from browse service: Error calling browse service: 503"
+        },
+        "metadata": {}
+      }
+    },
+    "node-tether-spinner": {
+      "id": "node-tether-spinner",
+      "parent": "node-system-error",
+      "children": [],
+      "message": {
+        "id": "node-tether-spinner",
+        "author": {"role": "tool", "name": "file_search", "metadata": {}},
+        "create_time": 1704067630.0,
+        "content": {
+          "content_type": "tether_browsing_display",
+          "result": "",
+          "summary": "",
+          "assets": null,
+          "tether_id": null
+        },
+        "metadata": {"command": "spinner", "status": "running"}
+      }
     }
   }
 }

tests/fixtures/claude_conversation.json

@@ -30,6 +30,15 @@
       "sender": "human",
       "created_at": "2024-06-10T14:45:00.000Z",
       "content": "Thank you, that helped!"
+    },
+    {
+      "uuid": "msg-004",
+      "sender": "human",
+      "created_at": "2024-06-10T14:50:00.000Z",
+      "content": [
+        {"type": "text", "text": "What about this image?"},
+        {"type": "image", "source": {"file_uuid": "claude-image-uuid-1", "media_type": "image/png"}}
+      ]
     }
   ]
 }

tests/test_cache.py

@@ -151,3 +151,106 @@ class TestGetNewOrUpdated:
         convs = [{"id": "a", "updated_at": "2024-06-01T00:00:00Z"}]
         result = tmp_cache.get_new_or_updated("claude", convs)
         assert len(result) == 1
+
+    def test_force_returns_all_cached_and_unchanged(self, tmp_cache):
+        tmp_cache.mark_exported("claude", "a", {"updated_at": "2024-01-01T00:00:00Z"})
+        convs = [
+            {"id": "a", "updated_at": "2024-01-01T00:00:00Z"},  # cached, unchanged
+            {"id": "b", "updated_at": "2024-01-02T00:00:00Z"},  # new
+        ]
+        assert len(tmp_cache.get_new_or_updated("claude", convs)) == 1
+        assert len(tmp_cache.get_new_or_updated("claude", convs, force=True)) == 2
+
+    def test_force_orders_least_recently_exported_first(self, tmp_cache):
+        # 'a' was exported long ago; 'b' just now; 'c' never.
+        tmp_cache.mark_exported("claude", "a", {"updated_at": "2024-01-01T00:00:00Z"})
+        tmp_cache._data["claude"]["a"]["exported_at"] = "2024-01-01T00:00:00Z"
+        tmp_cache.mark_exported("claude", "b", {"updated_at": "2024-01-01T00:00:00Z"})
+        tmp_cache._data["claude"]["b"]["exported_at"] = "2026-06-12T00:00:00Z"
+        convs = [{"id": "a"}, {"id": "b"}, {"id": "c"}]
+        ordered = [c["id"] for c in tmp_cache.get_new_or_updated("claude", convs, force=True)]
+        # never-exported ('c', exported_at "") and oldest ('a') come before 'b'
+        assert ordered == ["c", "a", "b"]
+
+    def test_force_capped_run_makes_progress(self, tmp_cache):
+        """The bug: force + cap repeated the same head every run. Now each
+        capped run re-exports the next-oldest batch and converges."""
+        convs = [{"id": f"c{i}", "updated_at": "2024-01-01T00:00:00Z"} for i in range(5)]
+        seen = set()
+        for _ in range(3):  # cap=2 over 3 runs should cover all 5
+            batch = tmp_cache.get_new_or_updated("claude", convs, force=True)[:2]
+            for c in batch:
+                seen.add(c["id"])
+                tmp_cache.mark_exported("claude", c["id"], {"file_path": f"/{c['id']}.md"})
+        assert seen == {f"c{i}" for i in range(5)}
+
+    def test_campaign_excludes_already_refreshed(self, tmp_cache):
+        """With a campaign stamp, conversations re-exported during the campaign
+        (exported_at >= stamp) drop out, so the remaining count shrinks to 0."""
+        convs = [{"id": f"c{i}"} for i in range(5)]
+        campaign = "2026-06-12T00:00:00+00:00"
+        # Nothing exported yet → all 5 are candidates.
+        assert len(tmp_cache.get_new_or_updated("claude", convs, force=True,
+                                                campaign_at=campaign)) == 5
+        # Re-export 2 "during" the campaign (after the stamp).
+        for cid in ("c0", "c1"):
+            tmp_cache.mark_exported("claude", cid, {"file_path": f"/{cid}.md"})
+            tmp_cache._data["claude"][cid]["exported_at"] = "2026-06-12T01:00:00+00:00"
+        remaining = tmp_cache.get_new_or_updated("claude", convs, force=True,
+                                                 campaign_at=campaign)
+        assert {c["id"] for c in remaining} == {"c2", "c3", "c4"}
+        # Finish them → none left.
+        for cid in ("c2", "c3", "c4"):
+            tmp_cache.mark_exported("claude", cid, {"file_path": f"/{cid}.md"})
+            tmp_cache._data["claude"][cid]["exported_at"] = "2026-06-12T02:00:00+00:00"
+        assert tmp_cache.get_new_or_updated("claude", convs, force=True,
+                                            campaign_at=campaign) == []
+
+    def test_force_campaign_marker_roundtrip(self, tmp_cache):
+        assert tmp_cache.get_force_campaign() is None
+        tmp_cache.set_force_campaign("2026-06-12T00:00:00+00:00")
+        assert tmp_cache.get_force_campaign() == "2026-06-12T00:00:00+00:00"
+        # Survives reload, and is not mistaken for a provider by stats().
+        reopened = Cache(tmp_cache._dir)
+        assert reopened.get_force_campaign() == "2026-06-12T00:00:00+00:00"
+        assert "force_campaign_at" not in reopened.stats()
+        tmp_cache.clear_force_campaign()
+        assert tmp_cache.get_force_campaign() is None
+
+
+class TestReexportPreservesJoplinLink:
+    """A re-export must not drop the Joplin note link, or re-syncing would
+    create duplicate notes instead of updating the existing ones."""
+
+    def test_joplin_fields_carried_across_reexport(self, tmp_cache):
+        tmp_cache.mark_exported("chatgpt", "c1", {
+            "title": "T", "updated_at": "2024-01-01T00:00:00Z", "file_path": "/x.md",
+        })
+        tmp_cache.mark_joplin_synced("chatgpt", "c1", "note-123")
+        tmp_cache.set_joplin_resources("chatgpt", "c1", {"media/a.png": "res-1"})
+
+        # Re-export the same conversation (new render, new file_path).
+        tmp_cache.mark_exported("chatgpt", "c1", {
+            "title": "T", "updated_at": "2024-01-01T00:00:00Z", "file_path": "/x2.md",
+        })
+
+        entry = tmp_cache.get_all_entries("chatgpt")["c1"]
+        assert entry["joplin_note_id"] == "note-123"
+        assert entry["joplin_resources"] == {"media/a.png": "res-1"}
+        assert entry["file_path"] == "/x2.md"
+
+    def test_reexport_marks_note_for_update_not_create(self, tmp_cache):
+        tmp_cache.mark_exported("chatgpt", "c1", {
+            "updated_at": "2024-01-01T00:00:00Z", "file_path": "/x.md",
+        })
+        tmp_cache.mark_joplin_synced("chatgpt", "c1", "note-123")
+        # Nothing pending right after sync.
+        assert tmp_cache.get_joplin_pending("chatgpt") == []
+        # Re-export bumps exported_at past joplin_synced_at → pending for update,
+        # carrying the existing note id so the sync updates rather than creates.
+        tmp_cache.mark_exported("chatgpt", "c1", {
+            "updated_at": "2024-01-01T00:00:00Z", "file_path": "/x.md",
+        })
+        pending = tmp_cache.get_joplin_pending("chatgpt")
+        assert len(pending) == 1
+        assert pending[0][1]["joplin_note_id"] == "note-123"

tests/test_claude_code.py

@@ -0,0 +1,230 @@
+"""Unit tests for the Claude Code session provider."""
+
+import json
+
+import pytest
+
+from src.blocks import (
+    BLOCK_TYPE_COLLAPSED,
+    BLOCK_TYPE_TEXT,
+    BLOCK_TYPE_THINKING,
+    BLOCK_TYPE_TOOL_RESULT,
+    BLOCK_TYPE_TOOL_USE,
+    render_blocks_to_markdown,
+)
+from src.loss_report import LossReport
+from src.providers.claude_code import ClaudeCodeProvider
+
+
+def _write_session(tmp_path, records, project_dir="-home-jesse-myproj", name="abc-123"):
+    proj = tmp_path / project_dir
+    proj.mkdir(parents=True, exist_ok=True)
+    f = proj / f"{name}.jsonl"
+    f.write_text("\n".join(json.dumps(r) for r in records), encoding="utf-8")
+    return f
+
+
+def _records():
+    """A representative session: title, harness noise, dialogue, tool traffic."""
+    return [
+        {"type": "ai-title", "aiTitle": "First title", "sessionId": "abc-123"},
+        {
+            "type": "user",
+            "cwd": "/home/jesse/myproj",
+            "timestamp": "2026-05-01T10:00:00.000Z",
+            "message": {
+                "role": "user",
+                "content": (
+                    "<local-command-caveat>Caveat: ignore</local-command-caveat>"
+                    "<command-name>/model</command-name>"
+                    "<local-command-stdout>Set model</local-command-stdout>"
+                    "Please review my project."
+                ),
+            },
+        },
+        {
+            "type": "assistant",
+            "timestamp": "2026-05-01T10:00:05.000Z",
+            "message": {
+                "role": "assistant",
+                "content": [
+                    {"type": "thinking", "thinking": "Let me think about this."},
+                    {"type": "text", "text": "I'll review it now."},
+                    {"type": "tool_use", "id": "t1", "name": "Read", "input": {"file_path": "/x"}},
+                    {"type": "tool_use", "id": "t2", "name": "Read", "input": {"file_path": "/y"}},
+                    {"type": "tool_use", "id": "t3", "name": "Bash", "input": {"command": "ls"}},
+                ],
+            },
+        },
+        {
+            "type": "user",
+            "timestamp": "2026-05-01T10:00:08.000Z",
+            "message": {
+                "role": "user",
+                "content": [
+                    {"type": "tool_result", "tool_use_id": "t1", "content": "file contents " * 50},
+                ],
+            },
+        },
+        {
+            "type": "assistant",
+            "timestamp": "2026-05-01T10:00:20.000Z",
+            "message": {
+                "role": "assistant",
+                "content": [{"type": "text", "text": "Here is my review: all good."}],
+            },
+        },
+        # Harness records — skipped
+        {"type": "file-history-snapshot", "snapshot": {"big": "blob"}},
+        {"type": "permission-mode", "mode": "plan"},
+        # Subagent and meta records — skipped
+        {
+            "type": "assistant",
+            "isSidechain": True,
+            "message": {"role": "assistant", "content": [{"type": "text", "text": "subagent noise"}]},
+        },
+        {
+            "type": "user",
+            "isMeta": True,
+            "message": {"role": "user", "content": "meta noise"},
+        },
+        # Later title wins
+        {"type": "ai-title", "aiTitle": "Review my project", "sessionId": "abc-123"},
+    ]
+
+
+class TestClaudeCodeProvider:
+    def _provider(self, tmp_path, policy="placeholder"):
+        return ClaudeCodeProvider(projects_dir=tmp_path, hidden_content=policy)
+
+    def test_listing_metadata(self, tmp_path):
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path)
+        convs = p.fetch_all_conversations()
+        assert len(convs) == 1
+        c = convs[0]
+        assert c["id"] == "abc-123"
+        assert c["title"] == "Review my project"  # last ai-title wins
+        assert c["_project_name"] == "myproj"  # from cwd basename
+        assert c["created_at"] == "2026-05-01T10:00:00.000Z"
+        assert c["updated_at"]  # file mtime
+
+    def test_empty_files_skipped(self, tmp_path):
+        proj = tmp_path / "-home-x"
+        proj.mkdir()
+        (proj / "empty.jsonl").write_text("")
+        p = self._provider(tmp_path)
+        assert p.fetch_all_conversations() == []
+
+    def test_normalize_prose_only_default(self, tmp_path):
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path)
+        p.fetch_all_conversations()
+        raw = p.get_conversation("abc-123")
+        result = p.normalize_conversation(raw)
+
+        assert result["provider"] == "claude-code"
+        assert result["title"] == "Review my project"
+        assert result["project"] == "myproj"
+
+        roles = [m["role"] for m in result["messages"]]
+        assert roles == ["user", "assistant", "assistant"]
+
+        # Harness tags stripped, dialogue kept
+        user_text = result["messages"][0]["blocks"][0]["text"]
+        assert user_text == "Please review my project."
+        assert "Caveat" not in user_text
+
+        # Tool activity collapsed into one grouped placeholder on the
+        # assistant message that ran the tools (includes the tool_result
+        # bytes from the following user record).
+        first_assistant = result["messages"][1]
+        types = [b["type"] for b in first_assistant["blocks"]]
+        assert types == [BLOCK_TYPE_TEXT, BLOCK_TYPE_COLLAPSED]
+        collapsed = first_assistant["blocks"][1]
+        assert "3 calls" in collapsed["origin"]
+        assert "Read ×2" in collapsed["origin"]
+        assert "Bash ×1" in collapsed["origin"]
+        assert collapsed["size_bytes"] > 500
+
+        # Thinking dropped without placeholder; sidechain/meta absent
+        all_blocks = [b for m in result["messages"] for b in m["blocks"]]
+        assert not any(b["type"] == BLOCK_TYPE_THINKING for b in all_blocks)
+        assert not any(
+            "subagent noise" in (b.get("text") or "") or "meta noise" in (b.get("text") or "")
+            for b in all_blocks
+        )
+
+    def test_collapsed_counted_in_loss_report(self, tmp_path):
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path)
+        p.fetch_all_conversations()
+        report = LossReport()
+        p.normalize_conversation(p.get_conversation("abc-123"), report)
+        assert report.collapsed["Read"] == 2
+        assert report.collapsed["Bash"] == 1
+        assert report.collapsed["tool_result"] == 1
+        assert report.collapsed["thinking"] == 1
+        assert report.collapsed_bytes > 500
+
+    def test_full_policy_keeps_everything(self, tmp_path):
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path, policy="full")
+        p.fetch_all_conversations()
+        result = p.normalize_conversation(p.get_conversation("abc-123"))
+        all_blocks = [b for m in result["messages"] for b in m["blocks"]]
+        types = {b["type"] for b in all_blocks}
+        assert BLOCK_TYPE_THINKING in types
+        assert BLOCK_TYPE_TOOL_USE in types
+        assert BLOCK_TYPE_TOOL_RESULT in types
+        assert BLOCK_TYPE_COLLAPSED not in types
+
+    def test_updated_at_consistent_with_listing(self, tmp_path):
+        """Listing and normalized updated_at must match or the cache would
+        consider every session stale on every run (mtime vs last timestamp)."""
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path)
+        listing = p.fetch_all_conversations()[0]
+        normalized = p.normalize_conversation(p.get_conversation("abc-123"))
+        assert normalized["updated_at"] == listing["updated_at"]
+
+    def test_title_falls_back_to_first_prompt(self, tmp_path):
+        records = [r for r in _records() if r.get("type") != "ai-title"]
+        _write_session(tmp_path, records)
+        p = self._provider(tmp_path)
+        p.fetch_all_conversations()
+        result = p.normalize_conversation(p.get_conversation("abc-123"))
+        assert result["title"] == "Please review my project."
+
+    def test_unknown_block_type_surfaces(self, tmp_path):
+        records = [
+            {
+                "type": "assistant",
+                "timestamp": "2026-05-01T10:00:00.000Z",
+                "cwd": "/home/jesse/myproj",
+                "message": {
+                    "role": "assistant",
+                    "content": [
+                        {"type": "text", "text": "hello"},
+                        {"type": "future_block_xyz", "data": 1},
+                    ],
+                },
+            },
+        ]
+        _write_session(tmp_path, records)
+        p = self._provider(tmp_path)
+        p.fetch_all_conversations()
+        report = LossReport()
+        result = p.normalize_conversation(p.get_conversation("abc-123"), report)
+        assert report.unknown_blocks["claude-code.future_block_xyz"] == 1
+        rendered = render_blocks_to_markdown(result["messages"][0]["blocks"])
+        assert "Unsupported content" in rendered
+
+    def test_collapsed_placeholder_renders_grouped_line(self, tmp_path):
+        _write_session(tmp_path, _records())
+        p = self._provider(tmp_path)
+        p.fetch_all_conversations()
+        result = p.normalize_conversation(p.get_conversation("abc-123"))
+        rendered = render_blocks_to_markdown(result["messages"][1]["blocks"])
+        assert "> 🔧 **Tool output** — `3 calls: Read ×2, Bash ×1`" in rendered
+        assert "omitted (EXPORTER_HIDDEN_CONTENT=full to keep)" in rendered

tests/test_cli.py

@@ -0,0 +1,299 @@
+"""CLI-level tests using Click's CliRunner — no live API calls required."""
+
+import pytest
+from click.testing import CliRunner
+
+from src.cache import Cache
+from src.main import _filter_by_project, cli
+
+
+# ---------------------------------------------------------------------------
+# _filter_by_project  (T-27)
+# ---------------------------------------------------------------------------
+
+
+class TestFilterByProject:
+    """Unit tests for the project filter logic used by export/list/joplin."""
+
+    # ChatGPT conversations use the _project_name annotation key
+    def _chatgpt(self, conv_id, project_name):
+        return {"id": conv_id, "_project_name": project_name}
+
+    # Claude conversations use the project dict key
+    def _claude(self, conv_id, project_name):
+        proj = {"name": project_name} if project_name else None
+        return {"id": conv_id, "project": proj}
+
+    def test_none_filter_keeps_no_project_chatgpt(self):
+        convs = [self._chatgpt("a", None), self._chatgpt("b", "Python Course")]
+        result = _filter_by_project(convs, "none")
+        assert len(result) == 1
+        assert result[0]["id"] == "a"
+
+    def test_none_filter_keeps_no_project_claude(self):
+        convs = [self._claude("a", None), self._claude("b", "Python Course")]
+        result = _filter_by_project(convs, "none")
+        assert len(result) == 1
+        assert result[0]["id"] == "a"
+
+    def test_name_filter_case_insensitive(self):
+        convs = [
+            self._chatgpt("a", "Python Course"),
+            self._chatgpt("b", "Java Course"),
+            self._chatgpt("c", None),
+        ]
+        result = _filter_by_project(convs, "PYTHON")
+        assert len(result) == 1
+        assert result[0]["id"] == "a"
+
+    def test_name_filter_substring_match(self):
+        convs = [
+            self._chatgpt("a", "Python Advanced Course"),
+            self._chatgpt("b", "Python Basics"),
+            self._chatgpt("c", "JavaScript"),
+        ]
+        result = _filter_by_project(convs, "python")
+        assert len(result) == 2
+        assert {c["id"] for c in result} == {"a", "b"}
+
+    def test_no_matches_returns_empty(self):
+        convs = [self._chatgpt("a", "Python Course"), self._chatgpt("b", None)]
+        result = _filter_by_project(convs, "ruby")
+        assert result == []
+
+    def test_none_filter_excludes_all_with_projects(self):
+        convs = [self._chatgpt("a", "Project A"), self._chatgpt("b", "Project B")]
+        result = _filter_by_project(convs, "none")
+        assert result == []
+
+    def test_empty_string_project_treated_as_no_project(self):
+        convs = [{"id": "a", "_project_name": ""}, {"id": "b", "_project_name": "Real"}]
+        result = _filter_by_project(convs, "none")
+        assert len(result) == 1
+        assert result[0]["id"] == "a"
+
+    def test_claude_project_string_matched(self):
+        # Claude can also have project as a plain string
+        convs = [{"id": "a", "project": "python-course"}, {"id": "b", "project": None}]
+        result = _filter_by_project(convs, "python")
+        assert len(result) == 1
+        assert result[0]["id"] == "a"
+
+
+# ---------------------------------------------------------------------------
+# export --since validation  (T-25)
+# ---------------------------------------------------------------------------
+
+
+class TestExportSinceValidation:
+    """Test that --since with an invalid date exits cleanly with an error message."""
+
+    def _pre_populated_cache(self, tmp_path) -> Cache:
+        """Create a cache that passes the ToS gate and first-run doctor check."""
+        cache = Cache(tmp_path)
+        cache.acknowledge_tos()
+        cache.mark_exported("chatgpt", "dummy-conv", {"updated_at": "2024-01-01T00:00:00Z"})
+        return cache
+
+    def test_invalid_since_date_exits_with_error(self, tmp_path):
+        self._pre_populated_cache(tmp_path)
+
+        runner = CliRunner(mix_stderr=True)
+        result = runner.invoke(
+            cli,
+            ["--no-log-file", "export", "--since", "notadate"],
+            env={
+                "CHATGPT_SESSION_TOKEN": "eyJtesttoken",
+                "CACHE_DIR": str(tmp_path),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+            },
+        )
+        assert result.exit_code == 1
+        assert "Invalid --since date" in result.output
+        assert "YYYY-MM-DD" in result.output
+
+    def test_valid_since_date_does_not_error(self, tmp_path):
+        """A valid date should not produce the invalid-date error (may fail later on API)."""
+        self._pre_populated_cache(tmp_path)
+
+        runner = CliRunner(mix_stderr=True)
+        result = runner.invoke(
+            cli,
+            ["--no-log-file", "export", "--since", "2024-01-01"],
+            env={
+                "CHATGPT_SESSION_TOKEN": "eyJtesttoken",
+                "CACHE_DIR": str(tmp_path),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+                # This test hits the real (failing) auth endpoint with retries;
+                # don't add politeness pacing on top of the backoff sleeps.
+                "REQUEST_DELAY": "0",
+            },
+        )
+        assert "Invalid --since date" not in result.output
+
+    def test_max_conversations_zero_rejected(self, tmp_path):
+        """--max-conversations uses IntRange(min=1); 0 must be rejected by click."""
+        self._pre_populated_cache(tmp_path)
+
+        runner = CliRunner(mix_stderr=True)
+        result = runner.invoke(
+            cli,
+            ["--no-log-file", "export", "--max-conversations", "0"],
+            env={
+                "CHATGPT_SESSION_TOKEN": "eyJtesttoken",
+                "CACHE_DIR": str(tmp_path),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+            },
+        )
+        assert result.exit_code == 2
+        assert "max-conversations" in result.output
+
+
+# ---------------------------------------------------------------------------
+# LossReport summary
+# ---------------------------------------------------------------------------
+
+
+class TestLossReportSummary:
+    """The LossReport's format_summary() pinned format covers zero, top-5, and overflow cases."""
+
+    def test_zero_summary_uses_none_sentinel(self):
+        from src.loss_report import LossReport
+
+        report = LossReport()
+        out = report.format_summary()
+        assert "[export] Run summary:" in out
+        assert "conversations:        0" in out
+        assert "messages rendered:    0" in out
+        # All three "(none)" sentinels present — never empty parens
+        # (unknown blocks, extraction failures, collapsed by policy)
+        assert out.count("(none)") == 3
+
+    def test_top_5_breakdown(self):
+        from src.loss_report import LossReport
+
+        report = LossReport()
+        for raw_type in ("a", "b", "c", "d", "e", "f", "g"):
+            report.record_unknown(raw_type)
+            if raw_type == "a":
+                # Make 'a' the most common
+                for _ in range(4):
+                    report.record_unknown("a")
+        out = report.format_summary()
+        # Top entry shown
+        assert "a=5" in out
+        # Overflow line present (7 types, top 5 + 2 more)
+        assert "+ 2 more types" in out
+
+    def test_messages_and_conversations_recorded(self):
+        from src.loss_report import LossReport
+
+        report = LossReport()
+        report.record_conversation()
+        report.record_message()
+        report.record_message()
+        out = report.format_summary()
+        assert "conversations:        1" in out
+        assert "messages rendered:    2" in out
+
+
+# ---------------------------------------------------------------------------
+# prune command
+# ---------------------------------------------------------------------------
+
+
+class TestPrune:
+    """prune deletes export files not referenced by the manifest."""
+
+    def _setup(self, tmp_path):
+        """Cache with one referenced file; one stale file + empty-dir candidate."""
+        cache = Cache(tmp_path / "cache")
+        cache.acknowledge_tos()
+        export_dir = tmp_path / "exports"
+        keep = export_dir / "chatgpt" / "proj.2026" / "keep.md"
+        keep.parent.mkdir(parents=True)
+        keep.write_text("kept")
+        stale = export_dir / "chatgpt" / "proj" / "2026" / "old-layout.md"
+        stale.parent.mkdir(parents=True)
+        stale.write_text("stale")
+        cache.mark_exported("chatgpt", "conv-1", {"file_path": str(keep)})
+        return export_dir, keep, stale
+
+    def _invoke(self, tmp_path, *args):
+        runner = CliRunner(mix_stderr=True)
+        return runner.invoke(
+            cli,
+            ["--no-log-file", "prune", *args],
+            env={
+                "CACHE_DIR": str(tmp_path / "cache"),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+            },
+        )
+
+    def test_dry_run_lists_but_keeps_files(self, tmp_path):
+        export_dir, keep, stale = self._setup(tmp_path)
+        result = self._invoke(tmp_path, "--dry-run")
+        assert result.exit_code == 0
+        assert "old-layout.md" in result.output
+        assert "Dry run" in result.output
+        assert stale.exists() and keep.exists()
+
+    def test_yes_deletes_stale_keeps_referenced_sweeps_dirs(self, tmp_path):
+        export_dir, keep, stale = self._setup(tmp_path)
+        result = self._invoke(tmp_path, "--yes")
+        assert result.exit_code == 0
+        assert not stale.exists()
+        assert keep.exists()
+        # Old-layout dirs are now empty and swept
+        assert not (export_dir / "chatgpt" / "proj").exists()
+
+    def test_refuses_with_empty_manifest(self, tmp_path):
+        """Footgun guard: after cache --clear, prune must not wipe the archive."""
+        cache = Cache(tmp_path / "cache")
+        cache.acknowledge_tos()
+        export_dir = tmp_path / "exports"
+        f = export_dir / "chatgpt" / "a.md"
+        f.parent.mkdir(parents=True)
+        f.write_text("data")
+        result = self._invoke(tmp_path, "--yes")
+        assert result.exit_code == 1
+        assert "Refusing to prune" in result.output
+        assert f.exists()
+
+    def test_aborts_without_confirmation(self, tmp_path):
+        export_dir, keep, stale = self._setup(tmp_path)
+        runner = CliRunner(mix_stderr=True)
+        result = runner.invoke(
+            cli,
+            ["--no-log-file", "prune"],
+            input="n\n",
+            env={
+                "CACHE_DIR": str(tmp_path / "cache"),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+            },
+        )
+        assert result.exit_code == 0
+        assert "Aborted" in result.output
+        assert stale.exists()
+
+
+class TestCanaryCommand:
+    """`canary` wiring — no live API calls (no tokens configured)."""
+
+    def test_no_tokens_exits_nonzero(self, tmp_path):
+        cache = Cache(tmp_path / "cache")
+        cache.acknowledge_tos()
+        runner = CliRunner(mix_stderr=True)
+        result = runner.invoke(
+            cli,
+            ["--no-log-file", "canary"],
+            env={
+                "CACHE_DIR": str(tmp_path / "cache"),
+                "EXPORT_DIR": str(tmp_path / "exports"),
+                # Empty so .env (override=False) can't repopulate real tokens.
+                "CHATGPT_SESSION_TOKEN": "",
+                "CLAUDE_SESSION_KEY": "",
+            },
+        )
+        assert result.exit_code == 1
+        assert "No web-API provider tokens" in result.output

tests/test_config.py

@@ -0,0 +1,118 @@
+"""Tests for src/config.py — token validation logic (T-14)."""
+
+import logging
+import time
+
+import jwt
+import pytest
+
+from src.config import _validate_chatgpt_token
+
+
+class TestValidateChatGPTToken:
+    def test_expired_token_logs_warning(self, caplog):
+        # T-14: expired JWT must produce a clear warning
+        payload = {"exp": int(time.time()) - 3600}  # expired 1 hour ago
+        token = jwt.encode(payload, "secret", algorithm="HS256")
+        with caplog.at_level(logging.WARNING, logger="src.config"):
+            result = _validate_chatgpt_token(token)
+        assert any("expired" in r.message.lower() for r in caplog.records)
+        assert result is not None  # still returns the expiry datetime
+
+    def test_expiring_within_24h_logs_warning(self, caplog):
+        payload = {"exp": int(time.time()) + 3600}  # expires in 1 hour
+        token = jwt.encode(payload, "secret", algorithm="HS256")
+        with caplog.at_level(logging.WARNING, logger="src.config"):
+            _validate_chatgpt_token(token)
+        assert any("less than 24 hours" in r.message for r in caplog.records)
+
+    def test_valid_token_no_expiry_warning(self, caplog):
+        payload = {"exp": int(time.time()) + 86400 * 5}  # valid for 5 days
+        token = jwt.encode(payload, "secret", algorithm="HS256")
+        with caplog.at_level(logging.WARNING, logger="src.config"):
+            result = _validate_chatgpt_token(token)
+        assert not any("expired" in r.message.lower() for r in caplog.records)
+        assert result is not None
+
+    def test_token_without_exp_claim_logs_warning(self, caplog):
+        payload = {"sub": "user123"}  # no exp
+        token = jwt.encode(payload, "secret", algorithm="HS256")
+        with caplog.at_level(logging.WARNING, logger="src.config"):
+            result = _validate_chatgpt_token(token)
+        assert any("'exp'" in r.message or "no 'exp'" in r.message for r in caplog.records)
+        assert result is None
+
+    def test_jwe_encrypted_token_returns_none(self, caplog):
+        # JWE tokens (alg=dir) cannot be decoded client-side — this is normal for ChatGPT
+        jwe_like = "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIn0.fake.token.data.here"
+        with caplog.at_level(logging.DEBUG, logger="src.config"):
+            result = _validate_chatgpt_token(jwe_like)
+        assert result is None  # cannot decode, but not an error
+
+    def test_non_jwt_string_logs_warning(self, caplog):
+        with caplog.at_level(logging.WARNING, logger="src.config"):
+            result = _validate_chatgpt_token("notajwttoken")
+        assert any("does not look like a JWT" in r.message for r in caplog.records)
+        assert result is None
+
+
+class TestSessionLimiterConfig:
+    """MAX_CONVERSATIONS_PER_RUN and REQUEST_DELAY parsing in load_config."""
+
+    def _load(self, monkeypatch, tmp_path, **env):
+        from src.config import load_config
+        monkeypatch.setenv("EXPORT_DIR", str(tmp_path / "exports"))
+        monkeypatch.setenv("CACHE_DIR", str(tmp_path / "cache"))
+        for key in (
+            "MAX_CONVERSATIONS_PER_RUN",
+            "REQUEST_DELAY",
+            "EXPORTER_HIDDEN_CONTENT",
+            "EXPORTER_DOWNLOAD_MEDIA",
+        ):
+            monkeypatch.delenv(key, raising=False)
+        for key, value in env.items():
+            monkeypatch.setenv(key, value)
+        return load_config()
+
+    def test_defaults(self, monkeypatch, tmp_path):
+        cfg = self._load(monkeypatch, tmp_path)
+        assert cfg.max_conversations is None
+        assert cfg.request_delay == 1.0
+        assert cfg.hidden_content == "placeholder"
+        assert cfg.download_media == "images"
+
+    def test_download_media_valid(self, monkeypatch, tmp_path):
+        cfg = self._load(monkeypatch, tmp_path, EXPORTER_DOWNLOAD_MEDIA="all")
+        assert cfg.download_media == "all"
+
+    def test_download_media_invalid_raises(self, monkeypatch, tmp_path):
+        from src.config import ConfigError
+        with pytest.raises(ConfigError, match="EXPORTER_DOWNLOAD_MEDIA"):
+            self._load(monkeypatch, tmp_path, EXPORTER_DOWNLOAD_MEDIA="sometimes")
+
+    def test_valid_values(self, monkeypatch, tmp_path):
+        cfg = self._load(
+            monkeypatch, tmp_path,
+            MAX_CONVERSATIONS_PER_RUN="25", REQUEST_DELAY="0.5",
+        )
+        assert cfg.max_conversations == 25
+        assert cfg.request_delay == 0.5
+
+    def test_zero_delay_allowed(self, monkeypatch, tmp_path):
+        cfg = self._load(monkeypatch, tmp_path, REQUEST_DELAY="0")
+        assert cfg.request_delay == 0.0
+
+    def test_non_integer_cap_raises(self, monkeypatch, tmp_path):
+        from src.config import ConfigError
+        with pytest.raises(ConfigError, match="MAX_CONVERSATIONS_PER_RUN"):
+            self._load(monkeypatch, tmp_path, MAX_CONVERSATIONS_PER_RUN="lots")
+
+    def test_zero_cap_raises(self, monkeypatch, tmp_path):
+        from src.config import ConfigError
+        with pytest.raises(ConfigError, match="at least 1"):
+            self._load(monkeypatch, tmp_path, MAX_CONVERSATIONS_PER_RUN="0")
+
+    def test_negative_delay_raises(self, monkeypatch, tmp_path):
+        from src.config import ConfigError
+        with pytest.raises(ConfigError, match="REQUEST_DELAY"):
+            self._load(monkeypatch, tmp_path, REQUEST_DELAY="-1")

tests/test_exporters.py

@@ -1,4 +1,4 @@
-"""Unit tests for src/exporters/."""
+"""Unit tests for src/exporters/ and src/blocks.py."""
 
 import json
 import os
@@ -7,6 +7,23 @@ from pathlib import Path
 
 import pytest
 
+from src.blocks import (
+    BLOCK_TYPE_TEXT,
+    UNKNOWN_REASON_EXTRACTION_FAILED,
+    UNKNOWN_REASON_UNKNOWN_TYPE,
+    _blockquote_prefix,
+    _safe_fence,
+    make_code_block,
+    make_file_placeholder,
+    make_hidden_context_marker,
+    make_image_placeholder,
+    make_text_block,
+    make_thinking_block,
+    make_tool_result_block,
+    make_tool_use_block,
+    make_unknown_block,
+    render_blocks_to_markdown,
+)
 from src.exporters.markdown import MarkdownExporter, _yaml_escape, _format_timestamp
 from src.exporters.json_export import JSONExporter
 
@@ -122,7 +139,7 @@ class TestMarkdownFilenameGeneration:
     def test_year_in_path(self, tmp_path):
         exp = MarkdownExporter(tmp_path)
         path = exp.export(SAMPLE_CONV)
-        assert "/2024/" in str(path)
+        assert ".2024/" in str(path)
 
     def test_output_structure_provider_project(self, tmp_path):
         exp = MarkdownExporter(tmp_path, output_structure="provider/project")
@@ -199,6 +216,34 @@ class TestJSONExporter:
         assert "  " in raw
 
 
+class TestBothFormats:
+    """T-38: Markdown and JSON exporters produce matching filenames for the same conversation."""
+
+    def test_both_formats_produce_files(self, tmp_path):
+        md_exp = MarkdownExporter(tmp_path)
+        json_exp = JSONExporter(tmp_path)
+        md_path = md_exp.export(SAMPLE_CONV)
+        json_path = json_exp.export(SAMPLE_CONV)
+        assert md_path.exists()
+        assert json_path.exists()
+
+    def test_both_formats_have_matching_stems(self, tmp_path):
+        md_exp = MarkdownExporter(tmp_path)
+        json_exp = JSONExporter(tmp_path)
+        md_path = md_exp.export(SAMPLE_CONV)
+        json_path = json_exp.export(SAMPLE_CONV)
+        assert md_path.suffix == ".md"
+        assert json_path.suffix == ".json"
+        assert md_path.stem == json_path.stem
+
+    def test_both_formats_same_directory(self, tmp_path):
+        md_exp = MarkdownExporter(tmp_path)
+        json_exp = JSONExporter(tmp_path)
+        md_path = md_exp.export(SAMPLE_CONV)
+        json_path = json_exp.export(SAMPLE_CONV)
+        assert md_path.parent == json_path.parent
+
+
 class TestYamlEscape:
     def test_escapes_double_quotes(self):
         assert _yaml_escape('Say "hello"') == 'Say \\"hello\\"'
@@ -222,3 +267,271 @@ class TestFormatTimestamp:
 
     def test_empty_string(self):
         assert _format_timestamp("") == ""
+
+
+# ---------------------------------------------------------------------------
+# Block helpers and rendering
+# ---------------------------------------------------------------------------
+
+
+class TestSafeFence:
+    def test_minimum_three_backticks(self):
+        assert _safe_fence("plain text") == "```"
+
+    def test_four_backticks_when_three_in_content(self):
+        assert _safe_fence("here ``` is a fence") == "````"
+
+    def test_five_backticks_when_four_in_content(self):
+        assert _safe_fence("here ```` is four") == "`````"
+
+    def test_handles_empty_string(self):
+        assert _safe_fence("") == "```"
+
+    def test_handles_run_at_end(self):
+        # Trailing run still counted
+        assert _safe_fence("text ending in ```") == "````"
+
+
+class TestBlockquotePrefix:
+    def test_single_line(self):
+        assert _blockquote_prefix("hello") == "> hello"
+
+    def test_multi_line(self):
+        assert _blockquote_prefix("a\nb\nc") == "> a\n> b\n> c"
+
+    def test_empty_lines_become_naked_quote_marker(self):
+        assert _blockquote_prefix("a\n\nb") == "> a\n>\n> b"
+
+    def test_empty_string(self):
+        assert _blockquote_prefix("") == ">"
+
+
+class TestBlockConstructors:
+    def test_make_text_block_returns_none_for_empty(self):
+        assert make_text_block("") is None
+        assert make_text_block("   ") is None
+
+    def test_make_text_block_returns_dict(self):
+        b = make_text_block("hello")
+        assert b == {"type": "text", "text": "hello"}
+
+    def test_make_code_block_returns_none_for_empty(self):
+        assert make_code_block("") is None
+
+    def test_make_thinking_block_returns_none_for_empty(self):
+        assert make_thinking_block("") is None
+
+
+class TestRenderBlocks:
+    def test_text_block_renders_as_paragraph(self):
+        out = render_blocks_to_markdown([make_text_block("Hello world")])
+        assert out == "Hello world"
+
+    def test_blocks_separated_by_blank_line(self):
+        out = render_blocks_to_markdown(
+            [make_text_block("first"), make_text_block("second")]
+        )
+        assert out == "first\n\nsecond"
+
+    def test_code_block_with_language(self):
+        out = render_blocks_to_markdown([make_code_block("print(1)", language="python")])
+        assert "```python" in out
+        assert "print(1)" in out
+
+    def test_thinking_block_uses_blockquote(self):
+        out = render_blocks_to_markdown([make_thinking_block("step 1\nstep 2")])
+        assert "**💭 Reasoning**" in out
+        assert "> step 1" in out
+        assert "> step 2" in out
+
+    def test_tool_use_renders_as_blockquote_with_safe_fence(self):
+        out = render_blocks_to_markdown(
+            [make_tool_use_block("search", {"query": "test"})]
+        )
+        assert "> 🔧 **Tool: search**" in out
+        # Every line of the body is blockquote-prefixed
+        assert "> ```json" in out
+        assert "> }" in out
+
+    def test_tool_use_with_multiline_input(self):
+        out = render_blocks_to_markdown(
+            [make_tool_use_block("complex", {"a": 1, "b": [{"x": "y"}]})]
+        )
+        # Prefix every line of multi-line JSON
+        for line in out.split("\n"):
+            assert line.startswith(">") or line == ""
+
+    def test_tool_result_success_uses_outbox_icon(self):
+        out = render_blocks_to_markdown([make_tool_result_block("OK")])
+        assert "📤 **Result**" in out
+        assert "❌" not in out
+
+    def test_tool_result_error_uses_x_icon(self):
+        out = render_blocks_to_markdown([make_tool_result_block("oops", is_error=True)])
+        assert "❌ **Result (error)**" in out
+        assert "📤" not in out
+
+    def test_tool_result_with_tool_name_in_header(self):
+        out = render_blocks_to_markdown(
+            [make_tool_result_block("done", tool_name="container.exec")]
+        )
+        assert "📤 **Result: container.exec**" in out
+
+    def test_tool_result_error_with_tool_name(self):
+        out = render_blocks_to_markdown(
+            [make_tool_result_block("503", tool_name="web", is_error=True)]
+        )
+        assert "❌ **Result (error): web**" in out
+
+    def test_tool_result_summary_renders_as_italic_line(self):
+        out = render_blocks_to_markdown(
+            [
+                make_tool_result_block(
+                    "output",
+                    tool_name="container.exec",
+                    summary="Reading skill documentation",
+                )
+            ]
+        )
+        # Summary line is italic, lives between header and fence,
+        # all inside the blockquote prefix.
+        assert "> *Reading skill documentation*" in out
+        # Order: header before summary before fence
+        header_idx = out.index("Result: container.exec")
+        summary_idx = out.index("Reading skill documentation")
+        fence_idx = out.index("output")
+        assert header_idx < summary_idx < fence_idx
+
+    def test_image_placeholder_rendering(self):
+        out = render_blocks_to_markdown(
+            [make_image_placeholder(ref="file-123", source="user_upload")]
+        )
+        assert "🖼️ **Image attached**" in out
+        assert "`file-123`" in out
+        assert "user_upload" in out
+        assert "content not preserved" in out
+
+    def test_file_placeholder_with_metadata(self):
+        out = render_blocks_to_markdown(
+            [make_file_placeholder(ref="sediment://x", mime="audio/wav", size_bytes=10240, duration_seconds=2.5)]
+        )
+        assert "📎 **File attached**" in out
+        assert "audio/wav" in out
+        assert "KB" in out
+        assert "2.50s" in out
+
+    def test_unknown_block_renders_with_keys(self):
+        out = render_blocks_to_markdown(
+            [
+                make_unknown_block(
+                    raw_type="future_x",
+                    observed_keys=["foo", "bar"],
+                    reason=UNKNOWN_REASON_UNKNOWN_TYPE,
+                )
+            ]
+        )
+        assert "⚠️ **Unsupported content**" in out
+        assert "future_x" in out
+        assert "`foo`" in out
+        assert "`bar`" in out
+
+    def test_unknown_extraction_failed_includes_summary(self):
+        out = render_blocks_to_markdown(
+            [
+                make_unknown_block(
+                    raw_type="audio_transcription",
+                    observed_keys=["asset_pointer"],
+                    reason=UNKNOWN_REASON_EXTRACTION_FAILED,
+                    summary="expected key 'text' not found",
+                )
+            ]
+        )
+        assert "extraction_failed" in out
+        assert "expected key 'text' not found" in out
+
+    def test_hidden_context_marker(self):
+        out = render_blocks_to_markdown(
+            [make_hidden_context_marker("user_editable_context")]
+        )
+        assert "ℹ️ **Hidden context**" in out
+        assert "`user_editable_context`" in out
+
+    def test_safe_fence_prevents_runaway_code_block(self):
+        # Content contains an unbalanced opening fence — without _safe_fence
+        # this would corrupt downstream rendering.
+        evil_content = "before\n```Follow\ntext\nraw is: \"```"
+        block = make_code_block(evil_content)
+        out = render_blocks_to_markdown([block, make_text_block("after")])
+        # The 4-backtick wrap should be present
+        assert "````" in out
+        # The "after" text should appear OUTSIDE any code block — it follows
+        # the closing ```` fence.
+        assert out.endswith("after")
+
+    def test_block_order_preserved(self):
+        blocks = [
+            make_text_block("a"),
+            make_image_placeholder(ref="r1", source="user_upload"),
+            make_text_block("b"),
+        ]
+        out = render_blocks_to_markdown(blocks)
+        assert out.index("a") < out.index("Image attached")
+        assert out.index("Image attached") < out.index("b")
+
+
+# ---------------------------------------------------------------------------
+# Markdown exporter with blocks
+# ---------------------------------------------------------------------------
+
+
+SAMPLE_CONV_BLOCKS = {
+    "id": "blocks12345",
+    "title": "Blocks Conversation",
+    "provider": "claude",
+    "project": None,
+    "created_at": "2024-06-10T14:32:00Z",
+    "updated_at": "2024-06-10T15:00:00Z",
+    "message_count": 1,
+    "messages": [
+        {
+            "role": "assistant",
+            "content_type": "text",
+            "timestamp": None,
+            "blocks": [
+                {"type": "text", "text": "Here is the answer."},
+                {"type": "tool_use", "name": "search", "input": {"q": "x"}, "tool_id": "t1"},
+            ],
+        }
+    ],
+}
+
+
+class TestMarkdownExporterWithBlocks:
+    def test_renders_blocks(self, tmp_path):
+        exp = MarkdownExporter(tmp_path)
+        path = exp.export(SAMPLE_CONV_BLOCKS)
+        body = path.read_text()
+        assert "Here is the answer." in body
+        assert "🔧 **Tool: search**" in body
+
+    def test_falls_back_to_content_when_blocks_missing(self, tmp_path):
+        # Backward-compat: messages with `content` only (no `blocks`) still render.
+        exp = MarkdownExporter(tmp_path)
+        path = exp.export(SAMPLE_CONV)  # SAMPLE_CONV has content only, no blocks
+        body = path.read_text()
+        assert "Hello, how are you?" in body
+
+    def test_skips_messages_with_neither_blocks_nor_content(self, tmp_path):
+        conv = {
+            **SAMPLE_CONV_BLOCKS,
+            "messages": [
+                {"role": "user", "content_type": "text", "timestamp": None, "blocks": []},
+                {"role": "assistant", "content_type": "text", "timestamp": None, "blocks": [
+                    {"type": "text", "text": "I am here."}
+                ]},
+            ],
+        }
+        exp = MarkdownExporter(tmp_path)
+        path = exp.export(conv)
+        body = path.read_text()
+        assert "I am here." in body

tests/test_joplin.py

@@ -0,0 +1,434 @@
+"""Unit tests for src/joplin.py (JoplinClient)."""
+
+from unittest.mock import MagicMock, patch
+
+import pytest
+import requests
+
+from src.joplin import JoplinClient, JoplinError, _http_error_message, _timeout_message, notebook_path
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _make_client() -> JoplinClient:
+    return JoplinClient(base_url="http://localhost:41184", token="test-token")
+
+
+def _mock_response(json_data=None, text="", status_code=200):
+    resp = MagicMock()
+    resp.status_code = status_code
+    resp.text = text
+    resp.json.return_value = json_data or {}
+    resp.raise_for_status = MagicMock()
+    if status_code >= 400:
+        resp.raise_for_status.side_effect = requests.exceptions.HTTPError(
+            response=resp
+        )
+    return resp
+
+
+# ---------------------------------------------------------------------------
+# notebook_path helper
+# ---------------------------------------------------------------------------
+
+
+class TestNotebookPath:
+    def test_no_project(self):
+        assert notebook_path("chatgpt", None) == ("AI-ChatGPT", "No Project")
+
+    def test_no_project_string(self):
+        assert notebook_path("chatgpt", "no-project") == ("AI-ChatGPT", "No Project")
+
+    def test_project_with_hyphens(self):
+        assert notebook_path("chatgpt", "my-project") == ("AI-ChatGPT", "My Project")
+
+    def test_claude_provider(self):
+        assert notebook_path("claude", "budget-tracker") == ("AI-Claude", "Budget Tracker")
+
+    def test_multi_word_project(self):
+        assert notebook_path("claude", "ai-research-notes") == ("AI-Claude", "Ai Research Notes")
+
+    def test_returns_tuple(self):
+        result = notebook_path("chatgpt", "some-project")
+        assert isinstance(result, tuple) and len(result) == 2
+
+
+# ---------------------------------------------------------------------------
+# ping
+# ---------------------------------------------------------------------------
+
+
+class TestPing:
+    def test_ping_success(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(text="JoplinClipperServer")
+            assert client.ping() is True
+
+    def test_ping_not_joplin(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(text="SomeOtherServer")
+            assert client.ping() is False
+
+    def test_ping_connection_refused(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = requests.exceptions.ConnectionError()
+            assert client.ping() is False
+
+    def test_ping_timeout_returns_false(self):
+        """Ping timeout is not an error — Joplin just isn't responding."""
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = requests.exceptions.Timeout()
+            assert client.ping() is False
+
+    def test_ping_invalid_url_raises_joplin_error(self):
+        """Non-connection, non-timeout errors (e.g. invalid URL) surface as JoplinError."""
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = requests.exceptions.InvalidURL("bad url")
+            with pytest.raises(JoplinError):
+                client.ping()
+
+
+class TestValidateToken:
+    def test_validate_token_success(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(json_data={"items": [], "has_more": False})
+            client.validate_token()  # should not raise
+
+    def test_validate_token_401_raises_joplin_error(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(status_code=401)
+            with pytest.raises(JoplinError, match="401"):
+                client.validate_token()
+
+
+class TestTimeoutMessage:
+    def test_includes_timeout_duration(self):
+        import src.joplin as joplin_module
+        msg = _timeout_message("POST", "/notes")
+        assert "POST" in msg
+        assert "/notes" in msg
+        assert str(joplin_module._REQUEST_TIMEOUT) in msg
+
+    def test_includes_actionable_hints(self):
+        msg = _timeout_message("PUT", "/notes/abc")
+        assert "JOPLIN_REQUEST_TIMEOUT" in msg
+        # Should mention at least one cause
+        assert "large" in msg.lower() or "busy" in msg.lower() or "frozen" in msg.lower()
+
+
+class TestTimeoutHandling:
+    def test_get_timeout_raises_joplin_error_with_clear_message(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = requests.exceptions.Timeout()
+            with pytest.raises(JoplinError) as exc_info:
+                client._get("/folders")
+        assert "timed out" in str(exc_info.value).lower()
+        assert "JOPLIN_REQUEST_TIMEOUT" in str(exc_info.value)
+
+    def test_post_timeout_raises_joplin_error_with_clear_message(self):
+        client = _make_client()
+        with patch("requests.post") as mock_post:
+            mock_post.side_effect = requests.exceptions.Timeout()
+            with pytest.raises(JoplinError) as exc_info:
+                client._post("/notes", {"title": "Test"})
+        assert "timed out" in str(exc_info.value).lower()
+
+    def test_put_timeout_raises_joplin_error_with_clear_message(self):
+        client = _make_client()
+        with patch("requests.put") as mock_put:
+            mock_put.side_effect = requests.exceptions.Timeout()
+            with pytest.raises(JoplinError) as exc_info:
+                client._put("/notes/abc", {"title": "Test"})
+        assert "timed out" in str(exc_info.value).lower()
+
+    def test_create_note_timeout_propagates(self):
+        """Timeout on create_note surfaces as JoplinError, not raw requests exception."""
+        client = _make_client()
+        with patch("requests.post") as mock_post:
+            mock_post.side_effect = requests.exceptions.Timeout()
+            with pytest.raises(JoplinError, match="timed out"):
+                client.create_note("Big Note", "x" * 100_000, "nb-123")
+
+    def test_update_note_timeout_propagates(self):
+        client = _make_client()
+        with patch("requests.put") as mock_put:
+            mock_put.side_effect = requests.exceptions.Timeout()
+            with pytest.raises(JoplinError, match="timed out"):
+                client.update_note("note-id", "Big Note", "x" * 100_000)
+
+
+class TestHttpErrorMessage:
+    def test_401_gives_token_hint(self):
+        resp = MagicMock()
+        resp.status_code = 401
+        resp.text = "Unauthorized"
+        e = requests.exceptions.HTTPError(response=resp)
+        msg = _http_error_message("GET", "/folders", e)
+        assert "401" in msg
+        assert "token" in msg.lower()
+
+    def test_404_gives_deleted_note_hint(self):
+        resp = MagicMock()
+        resp.status_code = 404
+        resp.text = "Not Found"
+        e = requests.exceptions.HTTPError(response=resp)
+        msg = _http_error_message("PUT", "/notes/abc", e)
+        assert "404" in msg
+        assert "deleted" in msg.lower()
+
+    def test_other_error_includes_status_and_body(self):
+        resp = MagicMock()
+        resp.status_code = 500
+        resp.text = "Internal Server Error"
+        e = requests.exceptions.HTTPError(response=resp)
+        msg = _http_error_message("POST", "/notes", e)
+        assert "500" in msg
+
+
+# ---------------------------------------------------------------------------
+# list_notebooks
+# ---------------------------------------------------------------------------
+
+
+class TestListNotebooks:
+    def test_list_notebooks_single_page(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(
+                json_data={"items": [{"id": "nb1", "title": "ChatGPT - No Project"}], "has_more": False}
+            )
+            result = client.list_notebooks()
+        assert len(result) == 1
+        assert result[0]["id"] == "nb1"
+
+    def test_list_notebooks_paginated(self):
+        client = _make_client()
+        page1 = _mock_response(
+            json_data={"items": [{"id": "nb1", "title": "A"}], "has_more": True}
+        )
+        page2 = _mock_response(
+            json_data={"items": [{"id": "nb2", "title": "B"}], "has_more": False}
+        )
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = [page1, page2]
+            result = client.list_notebooks()
+        assert len(result) == 2
+        assert {nb["id"] for nb in result} == {"nb1", "nb2"}
+
+    def test_list_notebooks_connection_error(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.side_effect = requests.exceptions.ConnectionError()
+            with pytest.raises(JoplinError, match="Joplin"):
+                client.list_notebooks()
+
+
+# ---------------------------------------------------------------------------
+# get_or_create_notebook
+# ---------------------------------------------------------------------------
+
+
+class TestGetOrCreateNotebook:
+    def test_returns_existing_root_notebook_id(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(
+                json_data={
+                    "items": [{"id": "nb-existing", "title": "AI-ChatGPT", "parent_id": ""}],
+                    "has_more": False,
+                }
+            )
+            nb_id = client.get_or_create_notebook("AI-ChatGPT")
+        assert nb_id == "nb-existing"
+
+    def test_returns_existing_child_notebook_id(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(
+                json_data={
+                    "items": [{"id": "nb-child", "title": "No Project", "parent_id": "nb-parent"}],
+                    "has_more": False,
+                }
+            )
+            nb_id = client.get_or_create_notebook("No Project", parent_id="nb-parent")
+        assert nb_id == "nb-child"
+
+    def test_creates_new_notebook_when_not_found(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get, patch("requests.post") as mock_post:
+            mock_get.return_value = _mock_response(
+                json_data={"items": [], "has_more": False}
+            )
+            mock_post.return_value = _mock_response(
+                json_data={"id": "nb-new", "title": "AI-ChatGPT"}
+            )
+            nb_id = client.get_or_create_notebook("AI-ChatGPT")
+        assert nb_id == "nb-new"
+        mock_post.assert_called_once()
+
+    def test_creates_child_notebook_with_parent_id(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get, patch("requests.post") as mock_post:
+            mock_get.return_value = _mock_response(
+                json_data={"items": [], "has_more": False}
+            )
+            mock_post.return_value = _mock_response(
+                json_data={"id": "nb-child", "title": "My Project"}
+            )
+            nb_id = client.get_or_create_notebook("My Project", parent_id="nb-parent")
+        assert nb_id == "nb-child"
+        _, kwargs = mock_post.call_args
+        assert kwargs["json"]["parent_id"] == "nb-parent"
+
+    def test_does_not_include_parent_id_for_root(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get, patch("requests.post") as mock_post:
+            mock_get.return_value = _mock_response(json_data={"items": [], "has_more": False})
+            mock_post.return_value = _mock_response(json_data={"id": "nb-root", "title": "AI-Claude"})
+            client.get_or_create_notebook("AI-Claude")
+        _, kwargs = mock_post.call_args
+        assert "parent_id" not in kwargs["json"]
+
+    def test_caches_notebook_after_first_load(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(
+                json_data={
+                    "items": [{"id": "nb1", "title": "AI-Claude", "parent_id": ""}],
+                    "has_more": False,
+                }
+            )
+            # Call twice — GET /folders should only happen once
+            client.get_or_create_notebook("AI-Claude")
+            client.get_or_create_notebook("AI-Claude")
+        assert mock_get.call_count == 1
+
+    def test_different_parent_ids_are_distinct_cache_entries(self):
+        """Same title under different parents are different notebooks."""
+        client = _make_client()
+        with patch("requests.get") as mock_get:
+            mock_get.return_value = _mock_response(
+                json_data={
+                    "items": [
+                        {"id": "nb-a", "title": "No Project", "parent_id": "parent-chatgpt"},
+                        {"id": "nb-b", "title": "No Project", "parent_id": "parent-claude"},
+                    ],
+                    "has_more": False,
+                }
+            )
+            id_a = client.get_or_create_notebook("No Project", parent_id="parent-chatgpt")
+            id_b = client.get_or_create_notebook("No Project", parent_id="parent-claude")
+        assert id_a == "nb-a"
+        assert id_b == "nb-b"
+
+
+class TestGetOrCreateNotebookPath:
+    def test_creates_two_level_path(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get, patch("requests.post") as mock_post:
+            mock_get.return_value = _mock_response(json_data={"items": [], "has_more": False})
+            mock_post.side_effect = [
+                _mock_response(json_data={"id": "nb-parent", "title": "AI-ChatGPT"}),
+                _mock_response(json_data={"id": "nb-child", "title": "No Project"}),
+            ]
+            leaf_id = client.get_or_create_notebook_path(["AI-ChatGPT", "No Project"])
+        assert leaf_id == "nb-child"
+        assert mock_post.call_count == 2
+        # Second POST should use the parent's ID
+        _, kwargs = mock_post.call_args_list[1]
+        assert kwargs["json"]["parent_id"] == "nb-parent"
+
+    def test_reuses_existing_parent_for_new_child(self):
+        client = _make_client()
+        with patch("requests.get") as mock_get, patch("requests.post") as mock_post:
+            mock_get.return_value = _mock_response(
+                json_data={
+                    "items": [{"id": "nb-parent", "title": "AI-Claude", "parent_id": ""}],
+                    "has_more": False,
+                }
+            )
+            mock_post.return_value = _mock_response(
+                json_data={"id": "nb-child", "title": "Budget Tracker"}
+            )
+            leaf_id = client.get_or_create_notebook_path(["AI-Claude", "Budget Tracker"])
+        assert leaf_id == "nb-child"
+        # Only one POST — the parent already existed
+        assert mock_post.call_count == 1
+        _, kwargs = mock_post.call_args
+        assert kwargs["json"]["parent_id"] == "nb-parent"
+
+
+# ---------------------------------------------------------------------------
+# create_note
+# ---------------------------------------------------------------------------
+
+
+class TestCreateNote:
+    def test_create_note_returns_id(self):
+        client = _make_client()
+        with patch("requests.post") as mock_post:
+            mock_post.return_value = _mock_response(
+                json_data={"id": "note-123", "title": "My Note"}
+            )
+            note_id = client.create_note("My Note", "Note body", "nb-456")
+        assert note_id == "note-123"
+        _, kwargs = mock_post.call_args
+        assert kwargs["json"]["title"] == "My Note"
+        assert kwargs["json"]["body"] == "Note body"
+        assert kwargs["json"]["parent_id"] == "nb-456"
+
+    def test_create_note_connection_error(self):
+        client = _make_client()
+        with patch("requests.post") as mock_post:
+            mock_post.side_effect = requests.exceptions.ConnectionError()
+            with pytest.raises(JoplinError, match="Joplin"):
+                client.create_note("Title", "Body", "nb-id")
+
+    def test_create_note_http_error(self):
+        client = _make_client()
+        with patch("requests.post") as mock_post:
+            mock_post.return_value = _mock_response(status_code=401)
+            with pytest.raises(JoplinError):
+                client.create_note("Title", "Body", "nb-id")
+
+
+# ---------------------------------------------------------------------------
+# update_note
+# ---------------------------------------------------------------------------
+
+
+class TestUpdateNote:
+    def test_update_note_calls_put(self):
+        client = _make_client()
+        with patch("requests.put") as mock_put:
+            mock_put.return_value = _mock_response(json_data={"id": "note-123"})
+            client.update_note("note-123", "New Title", "New Body")
+        mock_put.assert_called_once()
+        _, kwargs = mock_put.call_args
+        assert kwargs["json"]["title"] == "New Title"
+        assert kwargs["json"]["body"] == "New Body"
+
+    def test_update_note_connection_error(self):
+        client = _make_client()
+        with patch("requests.put") as mock_put:
+            mock_put.side_effect = requests.exceptions.ConnectionError()
+            with pytest.raises(JoplinError, match="Joplin"):
+                client.update_note("note-id", "Title", "Body")
+
+    def test_update_note_http_error(self):
+        client = _make_client()
+        with patch("requests.put") as mock_put:
+            mock_put.return_value = _mock_response(status_code=404)
+            with pytest.raises(JoplinError):
+                client.update_note("note-id", "Title", "Body")

tests/test_media.py

@@ -0,0 +1,248 @@
+"""Tests for media downloads and Joplin resource rewriting."""
+
+from pathlib import Path
+
+import pytest
+
+from src.blocks import (
+    make_file_placeholder,
+    make_image_placeholder,
+    render_blocks_to_markdown,
+)
+from src.loss_report import LossReport
+from src.media import resolve_media, resolve_media_policy
+from src.providers.base import ProviderError
+from src.providers.chatgpt import parse_asset_file_id
+
+
+# ---------------------------------------------------------------------------
+# Asset reference parsing
+# ---------------------------------------------------------------------------
+
+
+class TestParseAssetFileId:
+    def test_plain_sediment(self):
+        assert parse_asset_file_id("sediment://file_00000000245c71fda5") == "file_00000000245c71fda5"
+
+    def test_generated_image_with_hash_and_page(self):
+        ref = "sediment://8456107fc383a53#file_00000000979c71f685#p_6.png"
+        assert parse_asset_file_id(ref) == "file_00000000979c71f685"
+
+    def test_file_service_scheme(self):
+        assert parse_asset_file_id("file-service://file-AbCdEf") == "file-AbCdEf"
+
+    def test_unrecognised(self):
+        assert parse_asset_file_id("https://example.com/x.png") is None
+        assert parse_asset_file_id("") is None
+        assert parse_asset_file_id(None) is None
+
+
+# ---------------------------------------------------------------------------
+# Media policy
+# ---------------------------------------------------------------------------
+
+
+class TestMediaPolicy:
+    def test_default(self, monkeypatch):
+        monkeypatch.delenv("EXPORTER_DOWNLOAD_MEDIA", raising=False)
+        assert resolve_media_policy() == "images"
+
+    def test_valid(self, monkeypatch):
+        monkeypatch.setenv("EXPORTER_DOWNLOAD_MEDIA", "all")
+        assert resolve_media_policy() == "all"
+
+    def test_invalid_falls_back(self, monkeypatch, caplog):
+        monkeypatch.setenv("EXPORTER_DOWNLOAD_MEDIA", "bogus")
+        assert resolve_media_policy() == "images"
+
+
+# ---------------------------------------------------------------------------
+# resolve_media
+# ---------------------------------------------------------------------------
+
+
+class _FakeProvider:
+    """Minimal provider exposing download_asset + the ref parser."""
+
+    def __init__(self, assets=None, fail_refs=None):
+        self._assets = assets or {}
+        self._fail_refs = fail_refs or {}
+        self.calls = []
+
+    def parse_asset_file_id(self, ref):
+        return parse_asset_file_id(ref)
+
+    def download_asset(self, ref):
+        self.calls.append(ref)
+        if ref in self._fail_refs:
+            raise ProviderError("chatgpt", "download_asset", self._fail_refs[ref])
+        return self._assets[ref]  # (content, mime, file_name)
+
+
+def _conv_with(blocks):
+    return {
+        "id": "conv-1",
+        "title": "Has Media",
+        "provider": "chatgpt",
+        "project": None,
+        "created_at": "2026-05-20T00:00:00+00:00",
+        "messages": [{"role": "user", "blocks": blocks}],
+    }
+
+
+class TestResolveMedia:
+    def test_downloads_image_and_inlines(self, tmp_path):
+        ref = "sediment://file_img1"
+        provider = _FakeProvider({ref: (b"\x89PNG\r\n", "image/png", "x.png")})
+        block = make_image_placeholder(ref=ref, source="user_upload")
+        conv = _conv_with([block])
+        report = LossReport()
+
+        n = resolve_media(conv, provider, tmp_path, "provider/project/year", "images", report)
+
+        assert n == 1
+        assert report.media_downloaded == 1
+        assert block["local_path"] == "media/file_img1.png"
+        rendered = render_blocks_to_markdown([block])
+        assert rendered == "![user_upload](media/file_img1.png)"
+        # File written under the conversation's media/ dir
+        written = list(tmp_path.rglob("media/file_img1.png"))
+        assert written and written[0].read_bytes() == b"\x89PNG\r\n"
+
+    def test_images_policy_skips_files(self, tmp_path):
+        ref = "sediment://file_audio1"
+        provider = _FakeProvider({ref: (b"RIFF", "audio/wav", "a.wav")})
+        block = make_file_placeholder(ref=ref, mime="audio/wav", size_bytes=1000)
+        conv = _conv_with([block])
+        report = LossReport()
+
+        n = resolve_media(conv, provider, tmp_path, "provider/project/year", "images", report)
+        assert n == 0
+        assert "local_path" not in block
+        assert provider.calls == []
+
+    def test_all_policy_downloads_files(self, tmp_path):
+        ref = "sediment://file_audio1"
+        provider = _FakeProvider({ref: (b"RIFFdata", "audio/wav", "a.wav")})
+        block = make_file_placeholder(ref=ref, mime="audio/wav", size_bytes=8)
+        conv = _conv_with([block])
+        report = LossReport()
+
+        n = resolve_media(conv, provider, tmp_path, "provider/project/year", "all", report)
+        assert n == 1
+        assert block["local_path"] == "media/file_audio1.wav"
+        rendered = render_blocks_to_markdown([block])
+        assert "media/file_audio1.wav" in rendered and rendered.startswith("> 📎")
+
+    def test_off_policy_noop(self, tmp_path):
+        provider = _FakeProvider({"sediment://file_x": (b"x", "image/png", None)})
+        block = make_image_placeholder(ref="sediment://file_x", source="user_upload")
+        conv = _conv_with([block])
+        report = LossReport()
+        assert resolve_media(conv, provider, tmp_path, "provider/project/year", "off", report) == 0
+        assert provider.calls == []
+
+    def test_idempotent_uses_disk(self, tmp_path):
+        ref = "sediment://file_img1"
+        provider = _FakeProvider({ref: (b"\x89PNG", "image/png", "x.png")})
+        conv = _conv_with([make_image_placeholder(ref=ref, source="user_upload")])
+        report = LossReport()
+        resolve_media(conv, provider, tmp_path, "provider/project/year", "images", report)
+        assert len(provider.calls) == 1
+
+        # Second run, fresh blocks: file already on disk → no new API call.
+        conv2 = _conv_with([make_image_placeholder(ref=ref, source="user_upload")])
+        resolve_media(conv2, provider, tmp_path, "provider/project/year", "images", LossReport())
+        assert len(provider.calls) == 1  # unchanged
+        assert conv2["messages"][0]["blocks"][0]["local_path"] == "media/file_img1.png"
+
+    def test_failure_keeps_placeholder_and_counts(self, tmp_path):
+        ref = "sediment://file_gone"
+        provider = _FakeProvider(fail_refs={ref: RuntimeError("Signed URL returned HTTP 404")})
+        block = make_image_placeholder(ref=ref, source="model_generated")
+        conv = _conv_with([block])
+        report = LossReport()
+
+        n = resolve_media(conv, provider, tmp_path, "provider/project/year", "images", report)
+        assert n == 0
+        assert "local_path" not in block
+        assert report.media_failed["expired-or-missing"] == 1
+        # Still renders as a placeholder, not a broken image link
+        assert render_blocks_to_markdown([block]).startswith("> 🖼️")
+
+    def test_provider_without_download_asset(self, tmp_path):
+        """claude-code has no remote assets — resolve_media must no-op."""
+        class NoDownload:
+            pass
+        block = make_image_placeholder(ref="sediment://file_x", source="user_upload")
+        conv = _conv_with([block])
+        assert resolve_media(conv, NoDownload(), tmp_path, "provider/project/year", "images", LossReport()) == 0
+
+
+# ---------------------------------------------------------------------------
+# Joplin media rewriting
+# ---------------------------------------------------------------------------
+
+
+class _FakeJoplin:
+    def __init__(self):
+        self.uploaded = []
+        self._n = 0
+
+    def create_resource(self, file_path, title=None):
+        self.uploaded.append(Path(file_path).name)
+        self._n += 1
+        return f"res{self._n}"
+
+
+class TestUploadMediaAndRewrite:
+    def _note(self, tmp_path, body):
+        media = tmp_path / "media"
+        media.mkdir()
+        (media / "file_img1.png").write_bytes(b"\x89PNG")
+        (media / "clip.wav").write_bytes(b"RIFF")
+        return body
+
+    def test_rewrites_image_and_file_links(self, tmp_path):
+        from src.joplin import upload_media_and_rewrite
+        body = self._note(
+            tmp_path,
+            "Look: ![user_upload](media/file_img1.png)\n"
+            "> 📎 **File attached** — [clip.wav](media/clip.wav) (audio/wav)",
+        )
+        client = _FakeJoplin()
+        new_body, res_map = upload_media_and_rewrite(body, tmp_path, client, {})
+
+        assert "![user_upload](:/res1)" in new_body
+        assert "(:/res2)" in new_body
+        assert "media/" not in new_body
+        assert res_map == {"media/file_img1.png": "res1", "media/clip.wav": "res2"}
+        assert len(client.uploaded) == 2
+
+    def test_reuses_known_resource_ids(self, tmp_path):
+        from src.joplin import upload_media_and_rewrite
+        body = self._note(tmp_path, "![x](media/file_img1.png)")
+        client = _FakeJoplin()
+        existing = {"media/file_img1.png": "existing-res"}
+        new_body, res_map = upload_media_and_rewrite(body, tmp_path, client, existing)
+        assert "(:/existing-res)" in new_body
+        assert client.uploaded == []  # no re-upload
+        assert res_map == existing
+
+    def test_missing_file_left_as_link(self, tmp_path):
+        from src.joplin import upload_media_and_rewrite
+        (tmp_path / "media").mkdir()
+        body = "![x](media/gone.png)"
+        client = _FakeJoplin()
+        new_body, res_map = upload_media_and_rewrite(body, tmp_path, client, {})
+        assert new_body == body
+        assert res_map == {}
+        assert client.uploaded == []
+
+    def test_no_media_links_untouched(self, tmp_path):
+        from src.joplin import upload_media_and_rewrite
+        body = "Just text with [a link](https://example.com) and `media/foo` in code."
+        client = _FakeJoplin()
+        new_body, res_map = upload_media_and_rewrite(body, tmp_path, client, {})
+        assert new_body == body
+        assert res_map == {}

tests/test_utils.py

@@ -0,0 +1,147 @@
+"""Tests for src/utils.py — filename generation, path building, redaction."""
+
+from pathlib import Path
+
+import pytest
+
+from src.utils import (
+    build_export_path,
+    format_token_status,
+    generate_filename,
+    redact_secrets,
+)
+
+
+class TestGenerateFilename:
+    def test_basic_format(self):
+        name = generate_filename("Hello World", "abc12345def", "2024-06-10T14:00:00Z")
+        assert name == "2024-06-10_hello-world_abc12345.md"
+
+    def test_special_chars_slugified(self):
+        # T-36: titles with punctuation must produce safe, OS-compatible filenames
+        name = generate_filename("What's this?! A test.", "abc12345", "2024-06-01T00:00:00Z")
+        assert "?" not in name
+        assert "!" not in name
+        assert "'" not in name
+        assert " " not in name
+        assert name.startswith("2024-06-01_")
+        assert name.endswith("_abc12345.md")
+
+    def test_unicode_chars_handled(self):
+        name = generate_filename("Héllo Wörld", "abc12345", "2024-06-01T00:00:00Z")
+        assert " " not in name
+        assert name.endswith("_abc12345.md")
+
+    def test_empty_title_becomes_untitled(self):
+        name = generate_filename("", "abc12345", "2024-06-01T00:00:00Z")
+        assert "untitled" in name
+
+    def test_id_truncated_to_8_chars(self):
+        name = generate_filename("Test", "abcdefghijklmnop", "2024-06-01T00:00:00Z")
+        assert name.endswith("_abcdefgh.md")
+
+    def test_long_title_truncated(self):
+        long_title = "a" * 200
+        name = generate_filename(long_title, "abc12345", "2024-06-01T00:00:00Z")
+        # Slug is capped at 60 chars by max_length
+        slug_part = name.split("_")[1]
+        assert len(slug_part) <= 60
+
+    def test_date_comes_from_created_at(self):
+        name = generate_filename("Test", "abc12345", "2023-11-25T00:00:00Z")
+        assert name.startswith("2023-11-25_")
+
+
+class TestBuildExportPath:
+    def test_default_structure_provider_project_year(self):
+        path = build_export_path(
+            Path("/exports"), "claude", "my-project", "2024-06-01T00:00:00Z", "file.md"
+        )
+        assert str(path) == "/exports/claude/my-project.2024/file.md"
+
+    def test_no_project_uses_no_project_slug(self):
+        path = build_export_path(
+            Path("/exports"), "chatgpt", None, "2024-06-01T00:00:00Z", "file.md"
+        )
+        assert "no-project.2024" in str(path)
+
+    def test_provider_project_structure_omits_year(self):
+        path = build_export_path(
+            Path("/exports"), "claude", "proj", "2024-06-01T00:00:00Z", "file.md",
+            structure="provider/project",
+        )
+        assert "2024" not in str(path)
+        assert "proj" in str(path)
+
+    def test_provider_year_structure_omits_project(self):
+        path = build_export_path(
+            Path("/exports"), "claude", "proj", "2024-06-01T00:00:00Z", "file.md",
+            structure="provider/year",
+        )
+        assert "proj" not in str(path)
+        assert "2024" in str(path)
+
+    def test_project_name_with_spaces_is_slugified(self):
+        path = build_export_path(
+            Path("/exports"), "claude", "My Project Name!", "2024-06-01T00:00:00Z", "file.md"
+        )
+        assert " " not in str(path)
+        assert "!" not in str(path)
+
+
+class TestRedactSecrets:
+    def test_token_value_redacted(self):
+        data = {"token": "supersecret"}
+        result = redact_secrets(data)
+        assert result["token"] == "[REDACTED]"
+
+    def test_session_key_redacted(self):
+        result = redact_secrets({"sessionKey": "abc123"})
+        assert result["sessionKey"] == "[REDACTED]"
+
+    def test_non_sensitive_key_unchanged(self):
+        result = redact_secrets({"title": "My Chat", "id": "abc123"})
+        assert result["title"] == "My Chat"
+        assert result["id"] == "abc123"
+
+    def test_nested_dict_redacted(self):
+        data = {"user": {"token": "secret", "name": "Alice"}}
+        result = redact_secrets(data)
+        assert result["user"]["token"] == "[REDACTED]"
+        assert result["user"]["name"] == "Alice"
+
+    def test_list_of_dicts(self):
+        data = [{"password": "p@ss"}, {"title": "chat"}]
+        result = redact_secrets(data)
+        assert result[0]["password"] == "[REDACTED]"
+        assert result[1]["title"] == "chat"
+
+
+class TestFormatTokenStatus:
+    def test_none_token_returns_not_set(self):
+        assert format_token_status(None) == "[NOT SET]"
+
+    def test_empty_token_returns_not_set(self):
+        assert format_token_status("") == "[NOT SET]"
+
+    def test_set_token_no_expiry(self):
+        assert format_token_status("sometoken") == "[SET]"
+
+    def test_expired_token(self):
+        from datetime import datetime, timezone, timedelta
+        expiry = datetime.now(tz=timezone.utc) - timedelta(days=1)
+        result = format_token_status("tok", expiry)
+        assert "EXPIRED" in result
+
+    def test_expiring_today_shows_hours(self):
+        from datetime import datetime, timezone, timedelta
+        expiry = datetime.now(tz=timezone.utc) + timedelta(hours=3)
+        result = format_token_status("tok", expiry)
+        assert "expires in" in result
+        assert "h" in result
+
+    def test_expiring_in_days(self):
+        from datetime import datetime, timezone, timedelta
+        expiry = datetime.now(tz=timezone.utc) + timedelta(days=10, hours=12)
+        result = format_token_status("tok", expiry)
+        assert "10 days" in result