Changelog

All notable changes to this project will be documented here. Format follows Keep a Changelog.

[0.6.0] - 2026-06-12

First tagged release. The project was developed through several internal milestones (0.1.0–0.5.0) that were never published; their changes are consolidated here.

Added

Core export & sync

ChatGPT and Claude export via their internal web APIs, with a local cache/manifest for incremental, resumable sync (only new or updated conversations are fetched; each is written to the manifest immediately).
Markdown and JSON exporters. Markdown rendering happens at exporter-write time (providers produce typed blocks; exporters render them), which keeps the door open for future Obsidian/HTML output.
CLI: export, list, cache, doctor, auth, joplin, prune.
--project filter on export/list/joplin (case-insensitive substring, or none).
ChatGPT Projects support via CHATGPT_PROJECT_IDS (project conversations are fetched separately from the default listing).

Joplin integration

joplin command syncs exported Markdown to Joplin as notes; notebooks are created automatically and nested per provider+project. Re-running is safe — the Joplin note ID is stored in the manifest, so notes are updated, not duplicated.
JOPLIN_API_TOKEN, JOPLIN_API_URL, JOPLIN_REQUEST_TIMEOUT config, with actionable error messages on timeout/connection failure.

Rich content (typed blocks)

Messages carry an ordered blocks list: text, code, thinking, tool_use, tool_result, citation, image_placeholder, file_placeholder, unknown.
ChatGPT voice mode: audio_transcription parts render as text; audio asset pointers render as 📎 File attached placeholders with size/duration. Custom Instructions (user_editable_context/model_editable_context) now appear (were silently dropped) with a > ℹ️ Hidden context marker.
ChatGPT execution_output, system_error, and tether_browsing_display render as tool_result blocks (with tool_name, is_error, optional summary); transient browse spinners skip silently.
Defensive Claude block extraction (text/thinking/tool_use/tool_result/image, including nested-block flattening).
_safe_fence picks a backtick fence longer than any run in the content, so embedded triple-backticks can't corrupt rendering (verified live in Joplin).
Data-loss visibility: a LossReport summary at the end of every export run breaks down unknown blocks and extraction failures by raw type; unknown blocks render as visible > ⚠️ Unsupported content with the raw type and observed keys.

Invisible-content collapse (EXPORTER_HIDDEN_CONTENT=full|placeholder|omit, default placeholder; export --hidden-content)

Messages that were invisible in the ChatGPT web UI are collapsed to one-line placeholders instead of exported in full. Two triggers: (a) tool-role retrieval dumps identified by author.name (file_search, myfiles_browser) — ChatGPT re-injects the full text of attached/project files on every tool run, measured at 86% of all content bytes across the three largest conversations and not hidden-flagged; (b) messages flagged is_visually_hidden_from_conversation (Custom Instructions). Code-execution results, web-search output, and all dialogue are untouched.
New collapsed block renders as > 🔧 **Tool output** — file_search (15.0 KB) — omitted (EXPORTER_HIDDEN_CONTENT=full to keep) (ℹ️ Hidden context variant for hidden-flagged messages). The LossReport gains a collapsed by policy section (count per origin + total KB) so the omission stays visible.

Claude Code session provider (--provider claude-code)

Archives local agent transcripts from ~/.claude/projects/*/*.jsonl (override with CLAUDE_CODE_DIR) — no tokens, no rate limits, no ToS exposure. Prose-only by default per the hidden-content policy: dialogue and deliverables kept; tool traffic grouped into one placeholder per activity run (> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted); thinking dropped (counted in the summary); subagent (isSidechain) and harness (isMeta, snapshots, command tags) records stripped. Titles from the last ai-title record; project from the working-directory basename. Joplin notebooks nest under the AI-Claude parent. Measured: 19MB of JSONL → 804KB of Markdown across 25 sessions.

Binary/image downloads (EXPORTER_DOWNLOAD_MEDIA=images|all|off, default images; export --download-media)

Downloads ChatGPT conversation assets into a media/ folder beside each export and inlines them — images become real ![](media/…) embeds, files become local links. Two-hop fetch via /backend-api/files/{id}/download → signed URL (verified live). all also pulls voice-mode audio (≈162MB of clips in this archive, transcripts already in text — hence the images-only default). Idempotent (skips assets already on disk), atomic writes, 600 perms. Expired/missing assets (old generated images 404) keep their placeholder and are counted as media failed — never fatal.
Joplin resource upload: the joplin command uploads downloaded media as resources and rewrites media/… links to :/resourceId, so images render inside notes (verified live: resources created with correct mime/size and linked to the note). Resource IDs are tracked per-conversation in the manifest, so re-syncs reuse them instead of duplicating.

Token setup & throughput

auth --from-browser [brave|chrome|chromium|edge|firefox]: extracts ChatGPT session-token cookies and the Claude sessionKey straight from a local browser's cookie store (via browser-cookie3; defaults to Brave), validates each against the live API, and writes .env. Tokens that fail validation never overwrite a working entry; clear fallback when the browser is absent or the cookie DB is locked.
Session limiter: --max-conversations N / MAX_CONVERSATIONS_PER_RUN cap downloads per run (per provider); capped-out conversations are reported as "deferred" and resumed next run.
Polite pacing: REQUEST_DELAY (default 1.0s, ±25% jitter, 0 disables) between consecutive API requests, with the existing 429 backoff as the reactive net.

Archive hygiene

prune deletes export files not referenced by the manifest (old-layout trees, no-ID orphans), with listing, confirmation, --dry-run, -y, and empty-directory sweep. Refuses to run when the manifest references no files (so cache --clear + prune can't wipe the archive). First live run removed 420 stale files (9.4 MB).
doctor verifies manifest ↔ disk integrity: every recorded file_path must exist on disk.

Changed

ChatGPT role filter that dropped tool/system messages is lifted; all roles route through normal extraction (truly empty messages skip via the empty-content guard).
BaseProvider.normalize_conversation accepts an optional LossReport parameter.
ChatGPTProvider.normalize_conversation reads conversation_id as a fallback for id (live detail responses use conversation_id; fixtures use id).
Config validates EXPORTER_HIDDEN_CONTENT, EXPORTER_DOWNLOAD_MEDIA, MAX_CONVERSATIONS_PER_RUN, and REQUEST_DELAY, and logs the active values at startup.
New dependency: browser-cookie3==0.20.1.

Fixed

Custom Instructions (user_editable_context/model_editable_context) were silently dropped from every conversation (parts-vs-direct-fields mismatch).

Migration

JSON exports: messages contain typed blocks and may omit the legacy content field — external consumers should prefer blocks.
To re-render existing exports with the current rendering and collapse policy: python -m src.main cache --clear then python -m src.main export, followed by joplin to update notes (and upload media resources). Per-conversation message counts may increase as previously-dropped Custom Instructions, image-only turns, and tool-only turns now appear.

Test suite

264 tests, all passing.

7.9 KiB Raw Blame History Unescape Escape