# Changelog

All notable changes to this project will be documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [0.6.0] - 2026-06-12

First tagged release. The project was developed through several internal
milestones (0.1.0–0.5.0) that were never published; their changes are
consolidated here.

### Added

**Core export & sync**
- ChatGPT and Claude export via their internal web APIs, with a local cache/manifest for incremental, resumable sync (only new or updated conversations are fetched; each is written to the manifest immediately).
- Markdown and JSON exporters. Markdown rendering happens at exporter-write time (providers produce typed blocks; exporters render them), which keeps the door open for future Obsidian/HTML output.
- CLI: `export`, `list`, `cache`, `doctor`, `auth`, `joplin`, `prune`.
- `--project` filter on `export`/`list`/`joplin` (case-insensitive substring, or `none`).
- ChatGPT Projects support via `CHATGPT_PROJECT_IDS` (project conversations are fetched separately from the default listing).

**Joplin integration**
- `joplin` command syncs exported Markdown to Joplin as notes; notebooks are created automatically and nested per provider+project. Re-running is safe — the Joplin note ID is stored in the manifest, so notes are updated, not duplicated.
- `JOPLIN_API_TOKEN`, `JOPLIN_API_URL`, `JOPLIN_REQUEST_TIMEOUT` config, with actionable error messages on timeout/connection failure.

**Rich content (typed blocks)**
- Messages carry an ordered `blocks` list: text, code, thinking, tool_use, tool_result, citation, image_placeholder, file_placeholder, unknown.
- ChatGPT voice mode: `audio_transcription` parts render as text; audio asset pointers render as `📎 File attached` placeholders with size/duration. Custom Instructions (`user_editable_context`/`model_editable_context`) now appear (were silently dropped) with a `> ℹ️ Hidden context` marker.
- ChatGPT `execution_output`, `system_error`, and `tether_browsing_display` render as `tool_result` blocks (with `tool_name`, `is_error`, optional `summary`); transient browse spinners skip silently.
- Defensive Claude block extraction (text/thinking/tool_use/tool_result/image, including nested-block flattening).
- `_safe_fence` picks a backtick fence longer than any run in the content, so embedded triple-backticks can't corrupt rendering (verified live in Joplin).
- Data-loss visibility: a `LossReport` summary at the end of every `export` run breaks down `unknown blocks` and `extraction failures` by raw type; `unknown` blocks render as visible `> ⚠️ Unsupported content` with the raw type and observed keys.

**Invisible-content collapse (`EXPORTER_HIDDEN_CONTENT=full|placeholder|omit`, default `placeholder`; `export --hidden-content`)**
- Messages that were invisible in the ChatGPT web UI are collapsed to one-line placeholders instead of exported in full. Two triggers: (a) tool-role retrieval dumps identified by `author.name` (`file_search`, `myfiles_browser`) — ChatGPT re-injects the full text of attached/project files on every tool run, measured at 86% of all content bytes across the three largest conversations and **not** hidden-flagged; (b) messages flagged `is_visually_hidden_from_conversation` (Custom Instructions). Code-execution results, web-search output, and all dialogue are untouched.
- New `collapsed` block renders as `> 🔧 **Tool output** — file_search (15.0 KB) — omitted (EXPORTER_HIDDEN_CONTENT=full to keep)` (ℹ️ Hidden context variant for hidden-flagged messages). The `LossReport` gains a `collapsed by policy` section (count per origin + total KB) so the omission stays visible.

**Claude Code session provider (`--provider claude-code`)**
- Archives local agent transcripts from `~/.claude/projects/*/*.jsonl` (override with `CLAUDE_CODE_DIR`) — no tokens, no rate limits, no ToS exposure. Prose-only by default per the hidden-content policy: dialogue and deliverables kept; tool traffic grouped into one placeholder per activity run (`> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted`); thinking dropped (counted in the summary); subagent (`isSidechain`) and harness (`isMeta`, snapshots, command tags) records stripped. Titles from the last `ai-title` record; project from the working-directory basename. Joplin notebooks nest under the `AI-Claude` parent. Measured: 19MB of JSONL → 804KB of Markdown across 25 sessions.

**Binary/image downloads (`EXPORTER_DOWNLOAD_MEDIA=images|all|off`, default `images`; `export --download-media`)**
- Downloads ChatGPT conversation assets into a `media/` folder beside each export and inlines them — images become real `![](media/…)` embeds, files become local links. Two-hop fetch via `/backend-api/files/{id}/download` → signed URL (verified live). `all` also pulls voice-mode audio (≈162MB of clips in this archive, transcripts already in text — hence the images-only default). Idempotent (skips assets already on disk), atomic writes, `600` perms. Expired/missing assets (old generated images 404) keep their placeholder and are counted as `media failed` — never fatal.
- Joplin resource upload: the `joplin` command uploads downloaded media as resources and rewrites `media/…` links to `:/resourceId`, so images render inside notes (verified live: resources created with correct mime/size and linked to the note). Resource IDs are tracked per-conversation in the manifest, so re-syncs reuse them instead of duplicating.

**Token setup & throughput**
- `auth --from-browser [brave|chrome|chromium|edge|firefox]`: extracts ChatGPT session-token cookies and the Claude `sessionKey` straight from a local browser's cookie store (via `browser-cookie3`; defaults to Brave), validates each against the live API, and writes `.env`. Tokens that fail validation never overwrite a working entry; clear fallback when the browser is absent or the cookie DB is locked.
- Session limiter: `--max-conversations N` / `MAX_CONVERSATIONS_PER_RUN` cap downloads per run (per provider); capped-out conversations are reported as "deferred" and resumed next run.
- Polite pacing: `REQUEST_DELAY` (default 1.0s, ±25% jitter, `0` disables) between consecutive API requests, with the existing 429 backoff as the reactive net.

**Re-rendering**
- `export --force` re-exports every conversation even if cached and unchanged, so the whole archive can be re-rendered after a formatting/feature change without `cache --clear`. Combines with `--max-conversations` to spread the work across runs.
- `mark_exported` now preserves `joplin_note_id` / `joplin_synced_at` / `joplin_resources` across re-exports (refreshing `exported_at`), so a re-export followed by `joplin` updates the existing notes instead of creating duplicates.

**Archive hygiene**
- `prune` deletes export files not referenced by the manifest (old-layout trees, no-ID orphans), with listing, confirmation, `--dry-run`, `-y`, and empty-directory sweep. Refuses to run when the manifest references no files (so `cache --clear` + `prune` can't wipe the archive). First live run removed 420 stale files (9.4 MB).
- `doctor` verifies manifest ↔ disk integrity: every recorded `file_path` must exist on disk.

### Changed
- ChatGPT role filter that dropped `tool`/`system` messages is **lifted**; all roles route through normal extraction (truly empty messages skip via the empty-content guard).
- `BaseProvider.normalize_conversation` accepts an optional `LossReport` parameter.
- `ChatGPTProvider.normalize_conversation` reads `conversation_id` as a fallback for `id` (live detail responses use `conversation_id`; fixtures use `id`).
- `Config` validates `EXPORTER_HIDDEN_CONTENT`, `EXPORTER_DOWNLOAD_MEDIA`, `MAX_CONVERSATIONS_PER_RUN`, and `REQUEST_DELAY`, and logs the active values at startup.
- New dependency: `browser-cookie3==0.20.1`.

### Fixed
- Custom Instructions (`user_editable_context`/`model_editable_context`) were silently dropped from every conversation (parts-vs-direct-fields mismatch).

### Migration
- JSON exports: messages contain typed `blocks` and may omit the legacy `content` field — external consumers should prefer `blocks`.
- To re-render existing exports with the current rendering and collapse policy: `python -m src.main cache --clear` then `python -m src.main export`, followed by `joplin` to update notes (and upload media resources). Per-conversation message counts may increase as previously-dropped Custom Instructions, image-only turns, and tool-only turns now appear.

### Test suite
- 264 tests, all passing.