release: v1.0.0, and split the README into docs/
The archive does what it was built to do, unattended, on both machines: four providers, incremental and resumable, Markdown plus Joplin, on a schedule, with results pushed to a phone and failures that announce themselves. The one item left on the roadmap — a StartOS service that consolidates both machines into one corpus — is an optional convenience, not a gap. Hence 1.0.0 rather than another 0.x. The code in this release shipped across b6ce636..8c3e00a: the claude.ai 403 session-invalid fix, the fork-subagent recursion fix, the FAILED-push backstops on both platforms, and asset-download retries with cross-run recovery, a circuit breaker and media counts in the notification. This commit cuts the release and pays a documentation debt. The README had reached 939 lines / 6,490 words / 48 KB / 96 headings, doing four unrelated jobs at once, and FUTURE.md asked for the split to ride along with the next substantial README edit rather than break every existing anchor for its own sake. A release is that edit. README.md is now 303 lines and keeps only what you need to install and configure the tool; the rest moved verbatim into docs/providers.md, docs/cli.md, docs/scheduling.md and docs/troubleshooting.md. No section was rewritten or dropped and every original heading still exists, so each anchor still resolves somewhere — but bookmarks into moved sections now need the docs/ path, which is the cost this deferral was protecting against. .gitignore needed the fix FUTURE.md predicted: the blanket *.md rule (exported conversations are Markdown and may be private) matched all four new pages, and git check-ignore confirmed every one would have been silently ignored. !docs/*.md is added with a comment saying why. Gitea's rendering of relative docs/ links cannot be checked before pushing, so it still needs confirming after. Also corrects two stale claims in the changelog: the Windows backstop is no longer "not yet run under Windows PowerShell 5.1 or Task Scheduler" (confirmed on winmini today, LastTaskResult 0 — only the FAILED push itself remains unexercised there), and the test count is 414, not 409. 414 tests passing. No code changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012gCWSknsWvhLnMFwa7qnGp
This commit is contained in:
JesseMarkowitz
co-authored by
Claude Opus 5
Claude Opus 5
parent
8c3e00a520
commit
04ebb8987c
@@ -0,0 +1,134 @@
|
||||
# Troubleshooting
|
||||
|
||||
What the cache does, what each failure means, and how to file a bug report.
|
||||
|
||||
← [README](../README.md) · [Providers and Tokens](providers.md) · [CLI Reference](cli.md) · [Scheduling](scheduling.md)
|
||||
|
||||
---
|
||||
|
||||
## How the Cache Works
|
||||
|
||||
The cache manifest lives at `cache/manifest.json` (inside the install directory) and records every exported conversation: its title, project, `updated_at` timestamp, output file path, and (after Joplin sync) the Joplin note ID.
|
||||
|
||||
On every `export` run:
|
||||
1. Fetch the full conversation list from the provider
|
||||
2. Compare each conversation's `updated_at` against the manifest
|
||||
3. Export only conversations that are new or have been updated
|
||||
4. Write each successfully exported conversation to the manifest **immediately** (not batched)
|
||||
|
||||
On every `joplin` run:
|
||||
1. Read the manifest to find conversations not yet synced to Joplin, or re-exported since last sync
|
||||
2. Push each pending Markdown file to Joplin (create or update)
|
||||
3. Store the Joplin note ID in the manifest so subsequent runs update rather than duplicate
|
||||
|
||||
**This design makes every run inherently resumable.** If the tool is interrupted for any reason — rate limit, network drop, Ctrl+C, crash — simply re-run the same command. It will skip already-processed conversations and continue from where it stopped.
|
||||
|
||||
A conversation whose assets failed transiently also carries `pending_media` (the
|
||||
asset refs still missing) and `pending_media_attempts`. While `pending_media` is
|
||||
set and the attempt cap is not reached, step 2 treats the conversation as stale
|
||||
so the next run re-fetches those assets — see
|
||||
[Embedded images and media](#embedded-images-and-media). A run that recovers them
|
||||
clears the key.
|
||||
|
||||
To force a full re-export: `ai-chat-exporter cache --clear` then re-run export.
|
||||
|
||||
To re-export a single conversation, delete its entry from `cache/manifest.json`
|
||||
and re-run. (There is deliberately no `cache --reset --conversation` flag; see
|
||||
`FUTURE.md`.)
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### `Authentication failed` (401, or 403 from Claude)
|
||||
Your session token has expired.
|
||||
- Run `ai-chat-exporter auth` to get a new token interactively
|
||||
- Or manually copy a fresh cookie value into your `.env` file
|
||||
|
||||
Note: neither token's expiry can be read client-side. Claude's `sessionKey` is an opaque string, and claude.ai reports an invalid one as **403** "Invalid authorization" (`account_session_invalid`), not 401. ChatGPT's token is an encrypted JWE; `doctor` reads the `error` field of `/api/auth/session` instead. See [When Tokens Expire](providers.md#when-tokens-expire).
|
||||
|
||||
### `429 Rate Limited`
|
||||
The tool automatically pauses, saves progress, and exits with a clear message showing how many conversations were exported vs remaining. Just re-run the same export command to resume — the cache picks up exactly where it left off.
|
||||
|
||||
### `HTTP 500` / `503` on an asset download
|
||||
The provider's asset storage, not your setup. Retries and the cross-run recovery
|
||||
described in [Embedded images and media](#embedded-images-and-media) handle this;
|
||||
the run reports `media failed` and the next run re-fetches. Measured 2026-10-20:
|
||||
two assets lost this way to a 500 and a 503 both downloaded cleanly an hour later.
|
||||
|
||||
### Joplin: "JOPLIN_API_TOKEN is not set"
|
||||
You need to configure the token before running the `joplin` command:
|
||||
1. Open Joplin desktop
|
||||
2. Go to Tools → Options → Web Clipper
|
||||
3. Enable the Web Clipper service
|
||||
4. Copy the Authorization token shown on that page
|
||||
5. Add `JOPLIN_API_TOKEN=<token>` to your `.env` file
|
||||
|
||||
### Joplin: "Joplin is not responding"
|
||||
Joplin desktop must be running when you run the `joplin` command. The Web Clipper service shuts down when Joplin is closed.
|
||||
|
||||
### Joplin: "Joplin rejected the API token (HTTP 401)"
|
||||
The token in `JOPLIN_API_TOKEN` doesn't match what Joplin expects. Get a fresh token from Joplin → Tools → Options → Web Clipper → Authorization token.
|
||||
|
||||
### Joplin: note timed out
|
||||
If you see a timeout error, Joplin took longer than `JOPLIN_REQUEST_TIMEOUT` seconds (default: 30) to respond. Possible causes:
|
||||
- The conversation is very large and Joplin is slow to index it
|
||||
- Joplin is busy syncing or loading a large library
|
||||
- Joplin has frozen — try restarting it
|
||||
|
||||
To increase the timeout: add `JOPLIN_REQUEST_TIMEOUT=60` to your `.env`.
|
||||
|
||||
### ChatGPT project conversations not appearing
|
||||
Make sure you've added the project IDs to `CHATGPT_PROJECT_IDS` in your `.env`. See [ChatGPT Projects](providers.md#chatgpt-projects) for how to find them. Project conversations are not included in the default conversation listing — they must be fetched separately.
|
||||
|
||||
### Schema warnings in logs (`Unexpected API response shape`)
|
||||
The provider's internal API may have changed. Run with `--debug`, sanitize the output (remove any personal content), and check the project's GitHub Issues for known fixes.
|
||||
|
||||
### Non-text content warnings
|
||||
Since v0.4.0, rich content is preserved as typed blocks in the export. ChatGPT voice transcripts render as text and audio assets as `📎 File attached` placeholders with size and duration metadata. Anything the extractor doesn't recognise renders as a visible `> ⚠️ Unsupported content` block naming the type and observed keys, *and* increments a counter in the post-export summary so you can tell whether real content is being silently skipped.
|
||||
|
||||
### Embedded images and media
|
||||
Since v0.6.0, image attachments are downloaded by default into a `media/` folder beside each export and inlined as ``. Audio and other files download only with `EXPORTER_DOWNLOAD_MEDIA=all`. On the next `joplin` run these are uploaded as Joplin resources so they render inside the note. Set `EXPORTER_DOWNLOAD_MEDIA=off` to skip downloads entirely.
|
||||
|
||||
A failed asset **never fails the export** — the conversation is archived with its
|
||||
text placeholder, and the failure is tallied as `media failed` in the run summary
|
||||
and in the push notification. What happens next depends on *why* it failed:
|
||||
|
||||
| Bucket | Meaning | Retried? |
|
||||
|---|---|---|
|
||||
| `expired-or-missing` | 404 — the upload was deleted server-side | No. The bytes are gone. |
|
||||
| `forbidden` | 403 — metadata survives, bytes do not (see the 2026-08-17 investigation in `CHANGELOG.md`) | No. |
|
||||
| `download-error` | 5xx, a timeout, a network blip | **Yes** — within the run, then on up to 3 later runs |
|
||||
| `skipped-provider-unwell` | the run's circuit breaker was open | **Yes**, on a later run |
|
||||
| `unparseable-ref` / `write-error` | local problem, not the provider's | No |
|
||||
|
||||
**Within a run**, an asset download is two requests — one for a signed URL, one
|
||||
for the bytes — and each is retried on a 5xx: immediately once, then after about
|
||||
2, 4, 8 and 16 seconds with jitter, honouring `Retry-After` when the server sends
|
||||
one. Retries for a single asset are capped by a wall-clock budget so a hanging
|
||||
endpoint cannot consume the run's whole execution window.
|
||||
|
||||
**Across runs**, an asset that failed transiently is recorded against its
|
||||
conversation in the cache manifest, and the next run re-exports that conversation
|
||||
to re-fetch it. Only the missing asset is fetched — media already on disk is
|
||||
reused, and Joplin resource IDs are keyed by path, so the re-sync uploads just
|
||||
the new file. After 3 unsuccessful runs the asset is treated as lost and the
|
||||
conversation stops being re-exported.
|
||||
|
||||
**Circuit breaker.** If 5 transient failures land back to back, asset downloads
|
||||
stop for the rest of the run and the remaining assets are recorded for a later
|
||||
attempt. A provider's storage having a bad day should cost one quiet morning, not
|
||||
a run that overruns its two-hour scheduled window. Permanent failures never count
|
||||
toward this — a deleted upload says nothing about the provider's health.
|
||||
|
||||
### Tool output / hidden context collapsed
|
||||
Since v0.6.0, content that was invisible in the ChatGPT web UI is collapsed by default to one-line placeholders like `> 🔧 Tool output — file_search (15.0 KB) — omitted`. This is ChatGPT's file-retrieval tool re-injecting your attached files on every run — it can be 90% of a conversation's bytes while containing none of the dialogue. Custom Instructions collapse the same way (`> ℹ️ Hidden context`). The post-export summary lists every collapsed message by origin with the total KB omitted. To keep everything, set `EXPORTER_HIDDEN_CONTENT=full` in `.env` or pass `--hidden-content full`.
|
||||
|
||||
### Empty export / all conversations skipped
|
||||
No new or updated conversations since your last run. To verify: `ai-chat-exporter cache --show`. To force a full re-export: `ai-chat-exporter cache --clear`.
|
||||
|
||||
### Filing a bug report
|
||||
1. Run with `--debug`: `ai-chat-exporter export --debug 2>&1 | tee debug.log`
|
||||
2. Remove any personal conversation content from `debug.log`
|
||||
3. Open a GitHub Issue with the sanitized log and the exact command you ran
|
||||
|
||||
Reference in New Issue
Block a user