JesseMarkowitzandClaude Opus 5
04ebb8987c
The archive does what it was built to do, unattended, on both machines: four providers, incremental and resumable, Markdown plus Joplin, on a schedule, with results pushed to a phone and failures that announce themselves. The one item left on the roadmap — a StartOS service that consolidates both machines into one corpus — is an optional convenience, not a gap. Hence 1.0.0 rather than another 0.x. The code in this release shipped across b6ce636..8c3e00a: the claude.ai 403 session-invalid fix, the fork-subagent recursion fix, the FAILED-push backstops on both platforms, and asset-download retries with cross-run recovery, a circuit breaker and media counts in the notification. This commit cuts the release and pays a documentation debt. The README had reached 939 lines / 6,490 words / 48 KB / 96 headings, doing four unrelated jobs at once, and FUTURE.md asked for the split to ride along with the next substantial README edit rather than break every existing anchor for its own sake. A release is that edit. README.md is now 303 lines and keeps only what you need to install and configure the tool; the rest moved verbatim into docs/providers.md, docs/cli.md, docs/scheduling.md and docs/troubleshooting.md. No section was rewritten or dropped and every original heading still exists, so each anchor still resolves somewhere — but bookmarks into moved sections now need the docs/ path, which is the cost this deferral was protecting against. .gitignore needed the fix FUTURE.md predicted: the blanket *.md rule (exported conversations are Markdown and may be private) matched all four new pages, and git check-ignore confirmed every one would have been silently ignored. !docs/*.md is added with a comment saying why. Gitea's rendering of relative docs/ links cannot be checked before pushing, so it still needs confirming after. Also corrects two stale claims in the changelog: the Windows backstop is no longer "not yet run under Windows PowerShell 5.1 or Task Scheduler" (confirmed on winmini today, LastTaskResult 0 — only the FAILED push itself remains unexercised there), and the test count is 414, not 409. 414 tests passing. No code changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012gCWSknsWvhLnMFwa7qnGp
172 lines
10 KiB
Markdown
172 lines
10 KiB
Markdown
# Providers and Tokens
|
||
|
||
How to obtain and refresh the session tokens for the two web providers, and the
|
||
per-provider setup for ChatGPT Projects, Claude Code and Codex.
|
||
|
||
← [README](../README.md) · [Configuration](../README.md#env-setup) · [CLI Reference](cli.md) · [Scheduling](scheduling.md) · [Troubleshooting](troubleshooting.md)
|
||
|
||
---
|
||
|
||
## Getting Your Session Tokens
|
||
|
||
Session tokens are how your browser stays logged in. This tool uses them to access your chat history on your behalf.
|
||
|
||
Tokens are entered manually — copy them from your browser's DevTools and run the wizard:
|
||
|
||
```bash
|
||
ai-chat-exporter auth
|
||
```
|
||
|
||
The wizard detects your OS, shows the correct DevTools shortcut, and writes the values to `.env` without echoing them to the terminal.
|
||
|
||
> **Why no auto-extraction from the browser?** Modern Chromium browsers (Chrome 127+, current Brave) encrypt cookies on Windows with **App-Bound Encryption**: the key is bound to the browser through a SYSTEM-level service, so reading cookies off disk requires Administrator rights and a PsExec-style SYSTEM impersonation that antivirus flags as credential theft — and it still fails on Brave specifically. There is no reliable, non-invasive way to pull these cookies automatically, so the manual DevTools flow below is the supported path.
|
||
|
||
### Token Lifetimes
|
||
|
||
| Provider | Cookie Name | Lifetime | Expiry Detection |
|
||
|----------|-------------|----------|-----------------|
|
||
| ChatGPT | `__Secure-next-auth.session-token` (split into `.0` + `.1` when over ~4KB) | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. |
|
||
| Claude | `sessionKey` | ~30 days | Opaque token — only detectable from an API rejection. claude.ai answers an invalid session with **403** `permission_error` / `account_session_invalid`, **not** 401; `doctor` reports it on the "Claude API reachable" row. |
|
||
|
||
### Finding Tokens in Chrome DevTools
|
||
|
||
1. Open the provider's website and make sure you're logged in
|
||
2. Press **F12** (Windows/Linux) or **Cmd+Option+I** (macOS) to open DevTools
|
||
3. Click the **Application** tab
|
||
4. In the left panel, expand **Cookies** and click the site URL
|
||
5. Find the cookie by name and copy its **Value**
|
||
|
||
**ChatGPT:** go to `https://chatgpt.com` → find the session token cookie. You will
|
||
see **one of two layouts**, depending on how large your session token is:
|
||
|
||
- **One cookie**, `__Secure-next-auth.session-token` — copy Value → `CHATGPT_SESSION_TOKEN`, and leave `CHATGPT_SESSION_TOKEN_1` empty.
|
||
- **Two cookies**, `__Secure-next-auth.session-token.0` and `.1` — copy `.0` (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN`, and `.1` → `CHATGPT_SESSION_TOKEN_1`.
|
||
|
||
Chrome splits a cookie only when it exceeds ~4KB, so a larger session is chunked
|
||
and a smaller one is not — the same account can differ from machine to machine.
|
||
`CHATGPT_SESSION_TOKEN_1` is optional; both layouts authenticate, because the
|
||
server reassembles a complete value sent under the `.0` name.
|
||
|
||
What does *not* work is sending a **partial** chunk — `.0` on its own when a `.1`
|
||
exists. That fails silently: `/api/auth/session` answers HTTP 200 with no
|
||
`accessToken` rather than an error. If you see two cookies, copy both.
|
||
|
||
**Claude:** go to `https://claude.ai` → find `sessionKey` → copy Value
|
||
|
||
### When Tokens Expire
|
||
|
||
An expired token shows up as an authentication error naming the cookie to
|
||
refresh and how. The status differs by provider — ChatGPT reports 401, while
|
||
claude.ai reports **403 "Invalid authorization"** (`account_session_invalid`) —
|
||
so don't read a 403 from Claude as a permissions problem with your account.
|
||
|
||
To refresh:
|
||
- Re-run the `auth` wizard: `ai-chat-exporter auth`
|
||
- Or manually update the value in your `.env` file
|
||
|
||
`ai-chat-exporter doctor` is the quickest check: the "token set" rows only test
|
||
that a value is present, so an expired credential passes those and fails on the
|
||
"API reachable" row.
|
||
|
||
---
|
||
|
||
## The `auth` Command
|
||
|
||
The easiest way to configure tokens is the interactive wizard:
|
||
|
||
```bash
|
||
ai-chat-exporter auth
|
||
```
|
||
|
||
This walks you through finding your token, validates it, shows the expiry date (ChatGPT only), and offers to write it to your `.env` automatically. Tokens are never echoed to the terminal.
|
||
|
||
---
|
||
|
||
## ChatGPT Projects
|
||
|
||
ChatGPT project conversations are stored separately from your main conversation list and require extra configuration.
|
||
|
||
### Finding your project IDs
|
||
|
||
The quickest way is to let the exporter find them:
|
||
|
||
```
|
||
ai-chat-exporter projects
|
||
```
|
||
|
||
It lists every project your conversations actually belong to, marks which are
|
||
missing from `.env`, and prints a paste-ready `CHATGPT_PROJECT_IDS=` line
|
||
(`--write` updates `.env` for you). If your ChatGPT account does not include
|
||
the project on conversation summaries, add `--deep` and it reads each
|
||
conversation's detail instead — slower, one request per conversation, but
|
||
complete.
|
||
|
||
Why it matters: project attribution is resolved from each conversation's own
|
||
`gizmo_id`, so exports file correctly whether or not a project is configured.
|
||
But the *listing* pass still needs `CHATGPT_PROJECT_IDS` — conversations that
|
||
live only inside a project never appear in the default conversation list, so an
|
||
unlisted project's chats are never fetched at all. Every export run also names
|
||
any unconfigured project it encounters.
|
||
|
||
To find them by hand instead:
|
||
|
||
1. Open ChatGPT and click a Project in the left sidebar
|
||
2. Look at the browser URL — it will look like:
|
||
`https://chatgpt.com/g/g-p-68c2b2b3037c8191890036fb4ae3ed9f-my-project/project`
|
||
3. Copy the `g-p-…` part (everything up to but not including the slug after the second `-`)
|
||
|
||
Add all your project IDs to `.env` as a comma-separated list:
|
||
|
||
```
|
||
CHATGPT_PROJECT_IDS=g-p-68c2b2b3037c8191890036fb4ae3ed9f,g-p-anotherprojectid
|
||
```
|
||
|
||
The `auth` wizard can also guide you through this step interactively.
|
||
|
||
---
|
||
|
||
## Claude Code Sessions
|
||
|
||
The `claude-code` provider archives your local [Claude Code](https://claude.com/claude-code) agent transcripts — no tokens, no API, no ToS exposure. Sessions are read from `~/.claude/projects/`.
|
||
|
||
```bash
|
||
ai-chat-exporter export --provider claude-code
|
||
ai-chat-exporter joplin --provider claude-code
|
||
```
|
||
|
||
Exports are prose-only by default: your prompts and Claude's write-ups are kept, tool activity is grouped into one-line placeholders (`> 🔧 Tool output — 14 calls: Read ×9, Bash ×2 (86KB) — omitted`), and internal reasoning is dropped (counted in the run summary). Set `EXPORTER_HIDDEN_CONTENT=full` to keep everything. Sessions become notes under their own top-level **`AI-ClaudeCode`** Joplin notebook, in a sub-notebook per launch folder. The provider is included in `--provider all` whenever a sessions directory exists.
|
||
|
||
**Subagents.** Claude Code stores subagent (Task-tool) transcripts as separate files under `<session>/subagents/`; each is folded into its parent session inline, as a collapsible `<details>` block labeled with the subagent's type and description (its own tool traffic is collapsed like the main dialogue).
|
||
|
||
**Repo tags.** Sessions launched from a workspace root all share one folder-named notebook, so titles carry the repos each session touched — `Resume StartWRT project work [start-technologies]` — for at-a-glance scanning and search. A file's repo is the git repository it lives in (nearest ancestor with a `.git`), resolved from the tool paths in the transcript, so work is tagged wherever it happened — even across workspaces — and config/one-off files are ignored (they aren't repos). Repos are frequency-ordered and capped at 3. To never tag specific repos, set `CLAUDE_CODE_REPO_TAG_IGNORE` (comma-separated). Note this reads your current git layout, so a repo you later delete or move drops from the tag on re-export.
|
||
|
||
**Multiple locations.** By default the provider scans `~/.claude/projects/` plus `$CLAUDE_CONFIG_DIR/projects` when `CLAUDE_CONFIG_DIR` is set. To scan additional roots (e.g. other machines' sessions copied onto this box), set `CLAUDE_CODE_DIR` to a `:`-separated list:
|
||
|
||
```bash
|
||
CLAUDE_CODE_DIR="$HOME/.claude/projects:/mnt/backup/laptop/.claude/projects"
|
||
```
|
||
|
||
Sessions from all roots are merged by folder (no per-machine label); if the same session UUID appears in two roots, the newer copy wins. Note the exporter only sees this machine's disk and cannot recover sessions Claude Code has already pruned — run it regularly.
|
||
|
||
---
|
||
|
||
## Codex Sessions
|
||
|
||
The `codex` provider archives your local [Codex CLI](https://chatgpt.com/codex) agent transcripts — same deal as Claude Code: no tokens, no API, no ToS exposure. Rollout files are read from `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl`.
|
||
|
||
```bash
|
||
ai-chat-exporter export --provider codex
|
||
ai-chat-exporter joplin --provider codex
|
||
```
|
||
|
||
Exports are prose-only by default, with the same placeholder format as Claude Code, into their own top-level **`AI-Codex`** Joplin notebook. Repo tags work the same way (`CODEX_REPO_TAG_IGNORE` to suppress), and additional roots can be scanned with `CODEX_DIR` (`:`-separated); `CODEX_HOME`'s `sessions/` is picked up automatically when that variable is set.
|
||
|
||
Three things differ from Claude Code, all forced by how Codex stores its data:
|
||
|
||
**Reasoning cannot be exported.** Codex encrypts it at rest — every reasoning record carries `encrypted_content` with no plaintext summary in any layer of the file. It is always dropped and counted; `EXPORTER_HIDDEN_CONTENT=full` cannot bring it back.
|
||
|
||
**Incomplete tool calls are reported.** Codex writes each session twice in one file: its own typed items (what actually ran) and the raw model-facing wire format (everything attempted). The provider reads the typed layer — it is already decoded, and it omits harness plumbing that would otherwise need stripping — but cross-checks the raw layer for calls that never produced a result, so placeholders read `3 calls: exec_command ×3 (+2 did not complete)`. Those are commands that failed to launch, that you aborted, or that were still running when the turn ended.
|
||
|
||
**Cloud tasks are out of scope.** `codex cloud` tasks run server-side and are reachable at `chatgpt.com/backend-api/api/codex/tasks`, but local CLI sessions are never uploaded there, so the cloud API is not an alternative source for these transcripts and this provider stays entirely offline. If you start using `codex cloud exec`, those transcripts would be cloud-only and would need separate work.
|
||
|