AIChatExporter/CHANGELOG.md

Changelog

All notable changes to this project will be documented here. Format follows Keep a Changelog.

[0.7.0] - 2026-06-28

Added

• canary command — probes the ChatGPT/Claude web APIs for schema drift against the fields the parser actually depends on (one listing page + one conversation per provider), not the full response shape. Flags the silent-failure risks for a backup tool: a renamed retrieval-tool author bypassing the hidden-content collapse, a new content_type, drifted message fields, or non-empty Claude attachments/files the normalizer ignores. ERROR findings exit non-zero; WARN findings are surfaced but non-fatal. See FUTURE.md §10.
• doctor now reports a real ChatGPT token-health check ("ChatGPT token active") based on the /api/auth/session error field. ChatGPT session tokens are JWEs whose exp is encrypted and unreadable client-side; the previous decode path could never yield an expiry. The honest signal is error == "RefreshAccessTokenError" (verified live), which means the session token is dead even though expires/accessToken still echo stale values. See FUTURE.md §9.

Fixed

• ChatGPTProvider._fetch_access_token now checks the /api/auth/session error field and fails fast with a clear "refresh your token" message. Previously it returned the stale accessToken present on a dead session, producing a confusing downstream 401 instead of an actionable auth error.

Removed

• auth --from-browser and the browser-cookie3 dependency (src/browser_tokens.py). Browser cookie auto-extraction is not viable: modern Chromium App-Bound Encryption (Chrome 127+/current Brave) keys cookies off a SYSTEM-level layer that can't be decrypted off disk without Administrator rights and AV-flagged SYSTEM impersonation, and fails on Brave specifically (symptom: Unable to get key for cookie decryption). Auth is manual (DevTools wizard) only. See FUTURE.md §2 for the full rationale.

[0.6.0] - 2026-06-12

First tagged release. The project was developed through several internal milestones (0.1.0–0.5.0) that were never published; their changes are consolidated here.

Added

Core export & sync

• ChatGPT and Claude export via their internal web APIs, with a local cache/manifest for incremental, resumable sync (only new or updated conversations are fetched; each is written to the manifest immediately).
• Markdown and JSON exporters. Markdown rendering happens at exporter-write time (providers produce typed blocks; exporters render them), which keeps the door open for future Obsidian/HTML output.
• CLI: export, list, cache, doctor, auth, joplin, prune.
• --project filter on export/list/joplin (case-insensitive substring, or none).
• ChatGPT Projects support via CHATGPT_PROJECT_IDS (project conversations are fetched separately from the default listing).

Joplin integration

• joplin command syncs exported Markdown to Joplin as notes; notebooks are created automatically and nested per provider+project. Re-running is safe — the Joplin note ID is stored in the manifest, so notes are updated, not duplicated.
• JOPLIN_API_TOKEN, JOPLIN_API_URL, JOPLIN_REQUEST_TIMEOUT config, with actionable error messages on timeout/connection failure.

Rich content (typed blocks)

• Messages carry an ordered blocks list: text, code, thinking, tool_use, tool_result, citation, image_placeholder, file_placeholder, unknown.
• ChatGPT voice mode: audio_transcription parts render as text; audio asset pointers render as 📎 File attached placeholders with size/duration. Custom Instructions (user_editable_context/model_editable_context) now appear (were silently dropped) with a > ℹ️ Hidden context marker.
• ChatGPT execution_output, system_error, and tether_browsing_display render as tool_result blocks (with tool_name, is_error, optional summary); transient browse spinners skip silently.
• Defensive Claude block extraction (text/thinking/tool_use/tool_result/image, including nested-block flattening).
• _safe_fence picks a backtick fence longer than any run in the content, so embedded triple-backticks can't corrupt rendering (verified live in Joplin).
• Data-loss visibility: a LossReport summary at the end of every export run breaks down unknown blocks and extraction failures by raw type; unknown blocks render as visible > ⚠️ Unsupported content with the raw type and observed keys.

Invisible-content collapse (EXPORTER_HIDDEN_CONTENT=full|placeholder|omit, default placeholder; export --hidden-content)

• Messages that were invisible in the ChatGPT web UI are collapsed to one-line placeholders instead of exported in full. Two triggers: (a) tool-role retrieval dumps identified by author.name (file_search, myfiles_browser) — ChatGPT re-injects the full text of attached/project files on every tool run, measured at 86% of all content bytes across the three largest conversations and not hidden-flagged; (b) messages flagged is_visually_hidden_from_conversation (Custom Instructions). Code-execution results, web-search output, and all dialogue are untouched.
• New collapsed block renders as > 🔧 **Tool output** — file_search (15.0 KB) — omitted (EXPORTER_HIDDEN_CONTENT=full to keep) (ℹ️ Hidden context variant for hidden-flagged messages). The LossReport gains a collapsed by policy section (count per origin + total KB) so the omission stays visible.

Claude Code session provider (--provider claude-code)

• Archives local agent transcripts from ~/.claude/projects/*/*.jsonl (override with CLAUDE_CODE_DIR) — no tokens, no rate limits, no ToS exposure. Prose-only by default per the hidden-content policy: dialogue and deliverables kept; tool traffic grouped into one placeholder per activity run (> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted); thinking dropped (counted in the summary); subagent (isSidechain) and harness (isMeta, snapshots, command tags) records stripped. Titles from the last ai-title record; project from the working-directory basename. Joplin notebooks nest under the AI-Claude parent. Measured: 19MB of JSONL → 804KB of Markdown across 25 sessions.

Binary/image downloads (EXPORTER_DOWNLOAD_MEDIA=images|all|off, default images; export --download-media)

• Downloads ChatGPT conversation assets into a media/ folder beside each export and inlines them — images become real ![](media/…) embeds, files become local links. Two-hop fetch via /backend-api/files/{id}/download → signed URL (verified live). all also pulls voice-mode audio (≈162MB of clips in this archive, transcripts already in text — hence the images-only default). Idempotent (skips assets already on disk), atomic writes, 600 perms. Expired/missing assets (old generated images 404) keep their placeholder and are counted as media failed — never fatal.
• Joplin resource upload: the joplin command uploads downloaded media as resources and rewrites media/… links to :/resourceId, so images render inside notes (verified live: resources created with correct mime/size and linked to the note). Resource IDs are tracked per-conversation in the manifest, so re-syncs reuse them instead of duplicating.

Token setup & throughput

• auth --from-browser [brave|chrome|chromium|edge|firefox]: extracts ChatGPT session-token cookies and the Claude sessionKey straight from a local browser's cookie store (via browser-cookie3; defaults to Brave), validates each against the live API, and writes .env. Tokens that fail validation never overwrite a working entry; clear fallback when the browser is absent or the cookie DB is locked.
• Session limiter: --max-conversations N / MAX_CONVERSATIONS_PER_RUN cap downloads per run (per provider); capped-out conversations are reported as "deferred" and resumed next run.
• Polite pacing: REQUEST_DELAY (default 1.0s, ±25% jitter, 0 disables) between consecutive API requests, with the existing 429 backoff as the reactive net.

Re-rendering

• export --force re-exports every conversation even if cached and unchanged, so the whole archive can be re-rendered after a formatting/feature change without cache --clear. Runs as a tracked campaign (a stamped start time in the manifest): each run re-renders the least-recently-exported conversations, the "still to go" count shrinks toward zero, finished providers do no further work, and the campaign auto-completes (reporting "Force re-render complete"). Combines with --max-conversations to spread the work across runs.
• mark_exported now preserves joplin_note_id / joplin_synced_at / joplin_resources across re-exports (refreshing exported_at), so a re-export followed by joplin updates the existing notes instead of creating duplicates.
• Fix: .env is now loaded at the very start of every command, so CACHE_DIR and LOG_FILE are honored (previously the cache silently used ./cache regardless of CACHE_DIR).

Archive hygiene

• prune deletes export files not referenced by the manifest (old-layout trees, no-ID orphans), with listing, confirmation, --dry-run, -y, and empty-directory sweep. Refuses to run when the manifest references no files (so cache --clear + prune can't wipe the archive). First live run removed 420 stale files (9.4 MB).
• doctor verifies manifest ↔ disk integrity: every recorded file_path must exist on disk.

Changed

• ChatGPT role filter that dropped tool/system messages is lifted; all roles route through normal extraction (truly empty messages skip via the empty-content guard).
• BaseProvider.normalize_conversation accepts an optional LossReport parameter.
• ChatGPTProvider.normalize_conversation reads conversation_id as a fallback for id (live detail responses use conversation_id; fixtures use id).
• Config validates EXPORTER_HIDDEN_CONTENT, EXPORTER_DOWNLOAD_MEDIA, MAX_CONVERSATIONS_PER_RUN, and REQUEST_DELAY, and logs the active values at startup.
• New dependency: browser-cookie3==0.20.1.

Fixed

• Custom Instructions (user_editable_context/model_editable_context) were silently dropped from every conversation (parts-vs-direct-fields mismatch).

Migration

• JSON exports: messages contain typed blocks and may omit the legacy content field — external consumers should prefer blocks.
• To re-render existing exports with the current rendering and collapse policy: python -m src.main cache --clear then python -m src.main export, followed by joplin to update notes (and upload media resources). Per-conversation message counts may increase as previously-dropped Custom Instructions, image-only turns, and tool-only turns now appear.

Test suite

• 264 tests, all passing.