The archive does what it was built to do, unattended, on both machines:
four providers, incremental and resumable, Markdown plus Joplin, on a
schedule, with results pushed to a phone and failures that announce
themselves. The one item left on the roadmap — a StartOS service that
consolidates both machines into one corpus — is an optional convenience,
not a gap. Hence 1.0.0 rather than another 0.x.
The code in this release shipped across b6ce636..8c3e00a: the claude.ai
403 session-invalid fix, the fork-subagent recursion fix, the FAILED-push
backstops on both platforms, and asset-download retries with cross-run
recovery, a circuit breaker and media counts in the notification. This
commit cuts the release and pays a documentation debt.
The README had reached 939 lines / 6,490 words / 48 KB / 96 headings,
doing four unrelated jobs at once, and FUTURE.md asked for the split to
ride along with the next substantial README edit rather than break every
existing anchor for its own sake. A release is that edit. README.md is
now 303 lines and keeps only what you need to install and configure the
tool; the rest moved verbatim into docs/providers.md, docs/cli.md,
docs/scheduling.md and docs/troubleshooting.md. No section was rewritten
or dropped and every original heading still exists, so each anchor still
resolves somewhere — but bookmarks into moved sections now need the
docs/ path, which is the cost this deferral was protecting against.
.gitignore needed the fix FUTURE.md predicted: the blanket *.md rule
(exported conversations are Markdown and may be private) matched all
four new pages, and git check-ignore confirmed every one would have been
silently ignored. !docs/*.md is added with a comment saying why. Gitea's
rendering of relative docs/ links cannot be checked before pushing, so
it still needs confirming after.
Also corrects two stale claims in the changelog: the Windows backstop is
no longer "not yet run under Windows PowerShell 5.1 or Task Scheduler"
(confirmed on winmini today, LastTaskResult 0 — only the FAILED push
itself remains unexercised there), and the test count is 414, not 409.
414 tests passing. No code changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012gCWSknsWvhLnMFwa7qnGp
An asset download is two hops — mint a signed URL, then fetch the bytes —
and only the first went through _make_request. The second was a bare
session.request with no 5xx handling, which lost two images in the
2026-10-20 09:00 run: one to a 500 the retried first hop had already
recovered from, one to a 503. Both re-downloaded cleanly an hour later
(106 KB and 1.3 MB), so the bytes were there all along.
A third asset timed out and aborted its whole conversation. curl_cffi's
RequestException subclasses OSError, so the un-wrapped timeout slipped
past resolve_media's except ProviderError into the export loop's
except OSError and was reported as "File write failed for conversation
6a945155" — a conversation that had already downloaded ~20 images.
Nothing was wrong with any file write.
The retry loop is factored out of _make_request into _send_with_retries,
which returns the raw response so a bytes caller gets the same handling
as a JSON one. Hop 2 goes through it: 5xx retried, Retry-After honoured,
timeouts surfacing as ProviderError for resolve_media to placeholder.
The hop is now logged, which it never was. Assets get their own ladder —
an immediate retry, then ~2/4/8/16s with jitter kept, since a storage
incident fails many assets at once — bounded by a per-asset wall-clock
budget, because the timeout case costs 40s a go.
Beyond the retry:
- Transient failures (5xx, timeout, network) are recorded as
pending_media so a later run re-fetches just that asset; capped at
3 runs. 404/403 stay permanent — settled 2026-08-17 for 19 assets.
- A circuit breaker stops downloads after 5 consecutive transient
failures, recording the rest as retryable. Permanent failures never
trip it; any success resets the streak.
- Media failures now appear in the set ids.
The run that lost two assets pushed a clean "OK".
Keeps FUTURE.md's per-conversation cache reset correctly closed. The
README split (FUTURE.md §2) stays otial
enough to justify breaking every anchor link.
37 new tests (414 passing). Verified live: both lost assets recovered,
doctor green, a read-only chatgpt dns.
The cross-run retry and breaker are unit-tested only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/cwa7qnGp
Claude-Session: https://claude.ai/code/session_012gCWSknsWvhLnMFwa7qnGp
The task now runs scheduling\run-sync.ps1 instead of an inline cmd.exe
loop: every provider is attempted, a run that dies without the app's
own report pushes a high-priority FAILED with the exception class only,
and each run's output is appended to cache\logs\scheduled-sync.log
because Task Scheduler keeps none. Verified under PowerShell 7.6 on
Linux with a stand-in cmd.exe; not yet run under Windows PowerShell 5.1
or Task Scheduler.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rwKTmug58sEsJ72AuWsEi
sync pushes its result only from the end of a run it finished, so a crash or an early exit sent nothing, while the providers that survived kept pushing OK. The systemd unit now runs scheduling/run-sync.sh, which keeps the per-provider loop and pushes a high-priority failure, with the exception class only, for any run that exited non-zero without the app's own report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
claude.ai answers an invalid or expired sessionKey with 403 account_session_invalid, never 401, so the refresh-your-cookie message could not fire for Claude. Auth detection is now a provider decision (_is_auth_failure); Claude matches the 403 on its error code so a real permission error still reports as itself. README and .env.example no longer claim both ChatGPT cookie chunks are required.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
FUTURE.md had become a 648-line archaeological record: six shipped roadmap
items, two dropped, two implemented, an investigation trail for each, and a
backlog closed as not needed — with the two genuinely planned items buried
at the bottom. It is now 139 lines, planned work first.
- Archives the old file verbatim as FUTURE-ARCHIVE.md. It carries the recon
behind decisions now recorded in one line each — why Brave cookie
extraction is not viable, why the ChatGPT token's expiry cannot be read
client-side, how the drift canary was designed — which would be expensive
to rediscover.
- Roadmap is now two items: the StartOS service (including the 2026-08-18
decision that clients upload to StartOS storage and the service owns the
Joplin connection), and the README split.
- Everything shipped or dropped is off the roadmap. Decisions not to build
survive as a one-line table so they are not re-proposed.
- Status notes for v0.7.0 and v0.8.0 move into Completed, joined by v0.9.0.
Releases v0.9.0: pyproject 0.8.0 -> 0.9.0, and the changelog's [Unreleased]
section becomes [0.9.0] - 2026-08-18. Covers the Codex provider, the
launcher scripts, `sync`, daily scheduling on Linux and Windows, ntfy
notifications, gizmo_id project attribution and the `projects` command, and
the splitlines data-loss fix in both local providers.
Whitelists FUTURE-ARCHIVE.md in .gitignore. `*.md` is ignored on purpose —
exported conversations are Markdown and may contain private content — with
each doc re-included by name, so a new doc is silently untracked rather than
rejected. The archive would not have been committed at all. Recorded in the
README-split roadmap item, since `docs/*.md` will hit exactly this.
Also refreshes the package description, which still named only ChatGPT and
Claude after two local providers were added.
Carries the documentation and table-of-contents work from earlier today.
Reading gizmo_id during normalization fixed attribution, but it cannot
answer "which projects am I missing?": normalization only runs on
conversations being exported, and a normal run skips everything already
cached. Discovering the gaps would have meant --force re-exporting the
whole archive.
`ai-chat-exporter projects` does it directly. It lists conversations,
collects the g-p- ids they belong to, resolves display names, and prints a
table marking which are absent from .env plus a paste-ready
CHATGPT_PROJECT_IDS line. --write applies it; --deep falls back to one
detail request per conversation when the listing does not carry gizmo_id
(unverified which shape this account returns, so the command reports which
path it took rather than assuming).
This matters beyond tidiness: attribution is now self-correcting, but the
listing pass still needs the ids. Conversations that live only inside a
project never appear in the default listing, so an unlisted project's chats
are not merely misfiled — they are never fetched.
6 CLI tests: reporting, the already-configured case, the g-p- guard, --deep,
the hint when --deep is needed, and --write. 324 pass.
- Subagents: fold Task-tool transcripts (subagents/*.jsonl) inline as
collapsible <details> blocks at the spawn point; their own tool traffic
collapses under the same policy; nesting handled via toolUseId matching
- Joplin: Claude Code gets its own top-level AI-ClaudeCode notebook;
update_note sets parent_id so notes self-heal/relocate on re-sync
- Repo [tags] in titles via git-root detection (nearest .git ancestor),
home-wide and cross-workspace; CLAUDE_CODE_REPO_TAG_IGNORE escape hatch
- Multi-root scanning: CLAUDE_CODE_DIR as os.pathsep list + CLAUDE_CONFIG_DIR;
merged by launch-folder, newer-mtime wins on UUID collision
- 298 tests passing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The export reads ChatGPT's and Claude's undocumented internal web APIs,
which can change shape without notice; the worst failure for a backup tool
is a silent one (skipped/mis-parsed content with no error). Add a `canary`
command + BaseProvider.check_drift() (overridden by ChatGPT/Claude) that
fetches one listing page + one conversation per provider and asserts only
the normalizer's load-bearing fields — not the full response shape, which
churns harmlessly. The top silent risk it guards is a renamed retrieval-tool
author bypassing the hidden-content collapse. ERROR findings exit non-zero;
WARN findings are surfaced but non-fatal so a backup run is never blocked.
Also retires the in-app watch mode and headless StartOS direction from the
roadmap (tool stays a local, manually-run CLI) and updates docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Browser cookie auto-extraction is not viable: modern Chromium App-Bound
Encryption (Chrome 127+/current Brave) keys cookies off a SYSTEM-level
layer that cannot be decrypted off disk without admin rights and AV-flagged
SYSTEM impersonation, and fails on Brave specifically. Drop the
`--from-browser` flag, `_auth_from_browser`, `src/browser_tokens.py`, its
test, and the browser-cookie3 dependency. Auth is manual (DevTools wizard)
only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extracts per-message content into a typed `blocks` list (text, code,
thinking, tool_use, tool_result, image_placeholder, file_placeholder,
unknown) and renders them at exporter write time. Voice transcripts,
Custom Instructions, and image references now appear in exports
instead of being silently dropped.
Foundation:
- src/blocks.py: pure block constructors, _safe_fence (fence-corruption
defense, verified live in Joplin), _blockquote_prefix, render
- src/loss_report.py: per-run tally surfaced as INFO summary at end of
export so silently-dropped data becomes visible
Providers:
- ChatGPT: dispatch on content_type produces typed blocks; voice shapes
(audio_transcription, audio_asset_pointer, real_time_user_audio_video_
asset_pointer) locked from live DevTools capture; Custom Instructions
bug fix (parts-vs-direct-fields); role filter lifted; hidden-context
marker driven by is_visually_hidden_from_conversation flag
- Claude: defensive dispatch for text/thinking/tool_use/tool_result/image
with recursive nested-block flattening; untested against real rich-
content data — fix-forward in v0.4.1
Exporter:
- Markdown renders from blocks at write time via render_blocks_to_markdown;
backward-compat fallback to content for any pre-v0.4.0 cached data
Tests:
- 27 new tests across providers, exporters, CLI; fixtures rebuilt with
real-shape ChatGPT voice + Custom Instructions cases
- 181/181 pass
Behavior changes (intentional):
- JSON output omits content; consumers should read blocks
- Per-conversation message counts increase (Custom Instructions, image-
only, tool-only messages now appear)
- Existing exports not auto-re-rendered; users wanting fresh output run
cache --clear then export
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Core features:
- Add `joplin` command: syncs exported Markdown to Joplin via local REST API
- Notebooks auto-created per provider+project (e.g. "ChatGPT - My Project")
- Idempotent: notes updated (not duplicated) on re-run; note ID tracked in manifest
- Add `--project` filter to `export` and `list` commands (substring or 'none')
- Add ChatGPT Projects support via CHATGPT_PROJECT_IDS env var
Config:
- Add JOPLIN_API_TOKEN, JOPLIN_API_URL, JOPLIN_REQUEST_TIMEOUT
- Version now read from importlib.metadata (single source of truth: pyproject.toml)
- Bump version to 0.2.0
Quality:
- Explicit Timeout handling in JoplinClient with actionable error messages
- token validation (validate_token) separate from connectivity (ping)
- Remove debug_auth.py, debug_claude.py, and untracked .har file
- Add *.har to .gitignore (may contain auth cookies/session tokens)
- Update README, CHANGELOG, FUTURE.md to reflect v0.2.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
