feat: v0.6.0 — collapse policy, session limiter, Claude Code provider, prune, browser auth, media downloads
This commit is contained in:
JesseMarkowitz
@@ -87,6 +87,17 @@ This checks token presence, format, expiry, directory permissions, disk space, a
|
||||
|
||||
Session tokens are how your browser stays logged in. This tool uses them to access your chat history on your behalf.
|
||||
|
||||
### The fast way: extract from your browser
|
||||
|
||||
If the browser you're logged in with runs on the same machine as this tool:
|
||||
|
||||
```bash
|
||||
ai-chat-exporter auth --from-browser # Brave (default)
|
||||
ai-chat-exporter auth --from-browser firefox # or chrome, chromium, edge
|
||||
```
|
||||
|
||||
This reads the session cookies straight from the browser's cookie store (decrypted via the OS keyring/DPAPI/Keychain), validates each token against the live API, and writes `.env` — no DevTools, no copy-pasting. A token that fails validation never overwrites a working one. If extraction fails (browser not installed here, cookie DB locked, logged out), fall back to the manual flow below.
|
||||
|
||||
### Token Lifetimes
|
||||
|
||||
| Provider | Cookie Name | Lifetime | Expiry Detection |
|
||||
@@ -153,6 +164,10 @@ cp .env.example .env
|
||||
|----------|---------|-------------|
|
||||
| `EXPORT_DIR` | `./exports` | Where to write exported Markdown files |
|
||||
| `OUTPUT_STRUCTURE` | `provider/project/year` | Folder structure (see below) |
|
||||
| `EXPORTER_HIDDEN_CONTENT` | `placeholder` | What to do with content invisible in the provider's web UI (file-retrieval tool dumps, Custom Instructions): `placeholder` collapses to a one-line note with size, `full` keeps everything, `omit` drops it. Dumps can be 90% of a conversation's bytes. |
|
||||
| `EXPORTER_DOWNLOAD_MEDIA` | `images` | Download conversation assets into a `media/` folder beside each export and inline them: `images` (images only), `all` (also audio/voice clips), `off` (text placeholders only). Downloaded media is uploaded to Joplin as resources on the next `joplin` run. |
|
||||
| `MAX_CONVERSATIONS_PER_RUN` | unlimited | Cap downloads per export run (per provider). Runs are resumable, so re-running continues where the cap stopped — useful to spread a large first export across several sessions. |
|
||||
| `REQUEST_DELAY` | `1.0` | Seconds between consecutive API requests, with small random jitter, so traffic stays human-paced. Set `0` to disable. |
|
||||
|
||||
### Joplin
|
||||
|
||||
@@ -192,6 +207,19 @@ The `auth` wizard can also guide you through this step interactively.
|
||||
|
||||
---
|
||||
|
||||
## Claude Code Sessions
|
||||
|
||||
The `claude-code` provider archives your local [Claude Code](https://claude.com/claude-code) agent transcripts — no tokens, no API, no ToS exposure. Sessions are read from `~/.claude/projects/` (override with `CLAUDE_CODE_DIR` in `.env`).
|
||||
|
||||
```bash
|
||||
ai-chat-exporter export --provider claude-code
|
||||
ai-chat-exporter joplin --provider claude-code
|
||||
```
|
||||
|
||||
Exports are prose-only by default: your prompts and Claude's write-ups are kept, tool activity is grouped into one-line placeholders (`> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted`), and internal reasoning is dropped (counted in the run summary). Set `EXPORTER_HIDDEN_CONTENT=full` to keep everything. Each coding project becomes a Joplin notebook under the `AI-Claude` parent. The provider is included in `--provider all` whenever the sessions directory exists.
|
||||
|
||||
---
|
||||
|
||||
## Output Structure
|
||||
|
||||
All exported files go under `EXPORT_DIR`. The folder structure maps directly to Joplin notebooks.
|
||||
@@ -256,10 +284,11 @@ Each provider+project combination maps to a flat Joplin notebook created automat
|
||||
### `auth` — Interactive token setup
|
||||
|
||||
```bash
|
||||
ai-chat-exporter auth
|
||||
ai-chat-exporter auth # manual wizard (DevTools flow)
|
||||
ai-chat-exporter auth --from-browser # extract from Brave's cookie store
|
||||
```
|
||||
|
||||
Guided wizard to find and save session tokens and ChatGPT project IDs. Detects OS and shows the correct DevTools shortcut.
|
||||
Guided wizard to find and save session tokens and ChatGPT project IDs. Detects OS and shows the correct DevTools shortcut. `--from-browser [brave|chrome|chromium|edge|firefox]` skips the wizard and pulls tokens directly from a local browser, validating them live before writing `.env`.
|
||||
|
||||
### `doctor` — Health check
|
||||
|
||||
@@ -300,7 +329,7 @@ ai-chat-exporter export --output /path/to/my/notes
|
||||
ai-chat-exporter export --dry-run
|
||||
```
|
||||
|
||||
Options: `--provider [chatgpt|claude|all]`, `--format [markdown|json|both]`, `--output PATH`, `--since YYYY-MM-DD`, `--project NAME`, `--dry-run`
|
||||
Options: `--provider [chatgpt|claude|claude-code|all]`, `--format [markdown|json|both]`, `--output PATH`, `--since YYYY-MM-DD`, `--project NAME`, `--hidden-content [full|placeholder|omit]`, `--download-media [images|all|off]`, `--max-conversations N`, `--dry-run`
|
||||
|
||||
### `list` — List conversations
|
||||
|
||||
@@ -349,6 +378,22 @@ Reads the local export cache and pushes each exported Markdown file to Joplin as
|
||||
|
||||
Options: `--provider [chatgpt|claude|all]`, `--project NAME`, `--dry-run`
|
||||
|
||||
### `prune` — Delete stale export files
|
||||
|
||||
```bash
|
||||
# Preview what would be deleted
|
||||
ai-chat-exporter prune --dry-run
|
||||
|
||||
# Delete (asks for confirmation; -y skips the prompt)
|
||||
ai-chat-exporter prune
|
||||
```
|
||||
|
||||
Deletes export files no longer referenced by the cache manifest — leftovers
|
||||
from old folder layouts or fixed bugs that would otherwise double-sync into
|
||||
Joplin. Refuses to run when the manifest is empty (e.g. right after
|
||||
`cache --clear`) so it can never wipe a freshly cleared archive. The `doctor`
|
||||
command separately verifies that every manifest entry's file exists on disk.
|
||||
|
||||
### `cache` — Manage the sync manifest
|
||||
|
||||
```bash
|
||||
@@ -426,7 +471,13 @@ Make sure you've added the project IDs to `CHATGPT_PROJECT_IDS` in your `.env`.
|
||||
The provider's internal API may have changed. Run with `--debug`, sanitize the output (remove any personal content), and check the project's GitHub Issues for known fixes.
|
||||
|
||||
### Non-text content warnings
|
||||
Since v0.4.0, rich content is preserved as typed blocks in the export. ChatGPT voice transcripts render as text and audio assets as `📎 File attached` placeholders with size and duration metadata. Images render as `🖼️ Image attached` placeholders showing the asset reference. Custom Instructions appear under a `> ℹ️ Hidden context` marker. Anything the extractor doesn't recognise renders as a visible `> ⚠️ Unsupported content` block naming the type and observed keys, *and* increments a counter in the post-export summary so you can tell whether real content is being silently skipped. Binary downloads (the actual image/audio bytes) are still deferred — see `FUTURE.md` v0.5.0.
|
||||
Since v0.4.0, rich content is preserved as typed blocks in the export. ChatGPT voice transcripts render as text and audio assets as `📎 File attached` placeholders with size and duration metadata. Anything the extractor doesn't recognise renders as a visible `> ⚠️ Unsupported content` block naming the type and observed keys, *and* increments a counter in the post-export summary so you can tell whether real content is being silently skipped.
|
||||
|
||||
### Embedded images and media
|
||||
Since v0.6.0, image attachments are downloaded by default into a `media/` folder beside each export and inlined as ``. Audio and other files download only with `EXPORTER_DOWNLOAD_MEDIA=all`. On the next `joplin` run these are uploaded as Joplin resources so they render inside the note. Assets that have expired on the provider's side (older AI-generated images often do) keep their text placeholder and are tallied as `media failed` in the run summary — they're never fatal to the export. Set `EXPORTER_DOWNLOAD_MEDIA=off` to skip downloads entirely.
|
||||
|
||||
### Tool output / hidden context collapsed
|
||||
Since v0.6.0, content that was invisible in the ChatGPT web UI is collapsed by default to one-line placeholders like `> 🔧 Tool output — file_search (15.0 KB) — omitted`. This is ChatGPT's file-retrieval tool re-injecting your attached files on every run — it can be 90% of a conversation's bytes while containing none of the dialogue. Custom Instructions collapse the same way (`> ℹ️ Hidden context`). The post-export summary lists every collapsed message by origin with the total KB omitted. To keep everything, set `EXPORTER_HIDDEN_CONTENT=full` in `.env` or pass `--hidden-content full`.
|
||||
|
||||
### Empty export / all conversations skipped
|
||||
No new or updated conversations since your last run. To verify: `ai-chat-exporter cache --show`. To force a full re-export: `ai-chat-exporter cache --clear`.
|
||||
@@ -440,12 +491,9 @@ No new or updated conversations since your last run. To verify: `ai-chat-exporte
|
||||
|
||||
## Future Work
|
||||
|
||||
See `FUTURE.md` for planned features:
|
||||
See `FUTURE.md` for the full roadmap. Current priorities:
|
||||
|
||||
- **v0.2.x** — `export --force` flag; `joplin --force` flag; per-conversation cache reset
|
||||
- **v0.3.0** — Official API fallback: parse export ZIP files from ChatGPT/Claude settings
|
||||
- **v0.4.x / v0.5.0** — Binary content downloads (images, audio bytes) and Joplin resource upload; reclassify o1/o3 reasoning subparts; optional `EXPORTER_INCLUDE_HIDDEN_CONTEXT` toggle
|
||||
- **v0.5.0** — Watch/scheduled mode; Obsidian vault output
|
||||
- **Watch/scheduled mode** on the way to a headless StartOS service
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user