3 Commits
Author SHA1 Message Date
JesseMarkowitzandClaude Opus 5.5 23c6e1f512 fix: push a FAILED notification when a scheduled run dies without reporting
sync pushes its result only from the end of a run it finished, so a crash or an early exit sent nothing, while the providers that survived kept pushing OK. The systemd unit now runs scheduling/run-sync.sh, which keeps the per-provider loop and pushes a high-priority failure, with the exception class only, for any run that exited non-zero without the app's own report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:16 -04:00
JesseMarkowitzandClaude Opus 5.5 64068bb19b fix: fork subagents recursed forever in the Claude Code export
A fork subagent's transcript opens with a copy of the parent turn that spawned it, its own Agent call included; folding that call re-entered the same fork until RecursionError, failing every daily sync since 2026-09-24. Skip a spawn call while its own subagent is being expanded, and strip the <fork-boilerplate> preamble.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:16 -04:00
JesseMarkowitzandClaude Opus 5.5 b6ce636891 fix: detect claude.ai's 403 session-invalid as an expired key; ChatGPT .1 cookie is optional
claude.ai answers an invalid or expired sessionKey with 403 account_session_invalid, never 401, so the refresh-your-cookie message could not fire for Claude. Auth detection is now a provider decision (_is_auth_failure); Claude matches the 403 on its error code so a real permission error still reports as itself. README and .env.example no longer claim both ChatGPT cookie chunks are required.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:01 -04:00
12 changed files with 441 additions and 42 deletions
+10 -2
View File
@@ -6,10 +6,18 @@
# --- ChatGPT --- # --- ChatGPT ---
# How to get: open chatgpt.com in Chrome → F12 → Application tab # How to get: open chatgpt.com in Chrome → F12 → Application tab
# → Cookies → https://chatgpt.com → find the two cookie chunks: # → Cookies → https://chatgpt.com → find the session token cookie. Chrome splits a
# cookie only above ~4KB, so you will see ONE of these two layouts:
#
# __Secure-next-auth.session-token (the whole value) → CHATGPT_SESSION_TOKEN
# (leave _1 empty)
# or, when the token was large enough to be split:
# __Secure-next-auth.session-token.0 (starts with "eyJ") → CHATGPT_SESSION_TOKEN # __Secure-next-auth.session-token.0 (starts with "eyJ") → CHATGPT_SESSION_TOKEN
# __Secure-next-auth.session-token.1 (the remainder) → CHATGPT_SESSION_TOKEN_1 # __Secure-next-auth.session-token.1 (the remainder) → CHATGPT_SESSION_TOKEN_1
# Token type: JWE. Typically valid for ~7 days. #
# CHATGPT_SESSION_TOKEN_1 is OPTIONAL — leave it empty when there is no .1 cookie.
# But if a .1 cookie does exist, you must copy it: a partial .0 fails silently
# (HTTP 200 with no accessToken). Token type: JWE. Typically valid for ~7 days.
CHATGPT_SESSION_TOKEN= CHATGPT_SESSION_TOKEN=
CHATGPT_SESSION_TOKEN_1= CHATGPT_SESSION_TOKEN_1=
+26
View File
@@ -6,6 +6,32 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased] ## [Unreleased]
### Fixed ### Fixed
- **Every scheduled Claude Code sync since 2026-09-24 died with `RecursionError`.** Claude Code's `fork` subagents write a transcript that opens with a copy of the parent turn that spawned them — the fork's own `Agent` call included. `_extract_messages` folds a subagent inline whenever it meets its spawn call, so it met that copy inside the fork, folded the same fork again, and recursed until Python's limit. One fork anywhere in `~/.claude/projects` was enough to fail the whole provider; seven existed across three sessions, and the Codex half of the run, unaffected, hid the cause behind a generic exit 1.
The recursive pass now carries the spawn ids being expanded around it and skips a tool_use whose id is among them — the enclosing subagent block already stands for that call. Because the set accumulates, a longer cycle (A spawns B, whose transcript re-spawns A) stops too. The fork's `<fork-boilerplate>` preamble — generic worker rules the harness prepends — is now stripped with the other harness tags, so a fork block opens with its actual directive.
`TestSubagentFold.test_fork_containing_its_own_spawn_call` reproduces the on-disk shape (a `fork-context-ref` record, the copied spawn call, the boilerplate-wrapped directive) and fails with the original `RecursionError` against the unfixed code. Verified against the real archive: all 85 sessions normalize, the seven forks each render as one subagent block.
- **A scheduled run that crashed sent no notification, and the providers that survived said "OK".** `sync` pushes its ntfy result from the end of a run it finished, so the `RecursionError` above — and any crash, any exit before the sync starts (the ToS gate, a cache error), a launcher that cannot build its venv — sent nothing at all. Worse, the unit runs each provider as its own `sync`, so codex kept pushing a low-priority "AI archive OK" every morning for the eleven days claude-code was dead: a broken provider was indistinguishable from a quiet day.
The systemd unit's `ExecStart` is now `scheduling/run-sync.sh`, which carries the old per-provider loop and pushes a high-priority **FAILED** notification for any run that exits non-zero without the app's "Sync completed with failures" banner (printed right after the app's own push, so an app-reported failure is not reported twice). The push names the provider and, for a crash, the exception class only — `claude-code: crashed (RecursionError)` — never its message, holding to `src/notify.py`'s counts-only rule for a topic anyone can read. It honours `NTFY_NOTIFY=off` and reads `NTFY_*` the way the app does: environment first, then `.env`. Re-run `install-systemd-timer.sh` to pick it up. The Windows task has no equivalent yet.
Verified against a fake launcher and a local capture server: a crash pushes with the class and without the message, an exit before the sync pushes, an app-reported failure and a success push nothing extra, `off` pushes nothing, and the run still exits 1 if any provider failed.
- **An expired Claude session key reported a raw JSON dump instead of how to fix it.** `_make_request` routed only **401** to the auth handler (`src/providers/base.py`), and claude.ai does not use 401 — an invalid or expired `sessionKey` comes back as `403 permission_error` with `details.error_code = account_session_invalid`. So the one message that names the cookie, its ~30-day lifetime and the DevTools path to refresh it could never fire for Claude. What the user got instead was the generic 4xx path: `HTTP 403 — error: {'type': 'permission_error', 'message': 'Invalid authorization'…}`, which reads like a permissions problem with the account and not like "your key expired, here is how to replace it."
Measured live 2026-09-20 against `GET /api/organizations`: a valid key returns 200, while an expired key, a deliberately malformed key and **no cookie at all** return byte-identical 403s carrying that code — i.e. the API treats a dead session as an absent one. This is the same mistake as the ChatGPT media 403s below: assuming 403 means "forbidden" when the service uses it for "unauthenticated."
Auth detection is now a provider decision rather than a hardcoded status. `BaseProvider._is_auth_failure(response)` defaults to 401 and `ClaudeProvider` overrides it to add 403 **matched on `account_session_invalid`**, not on the bare status — so a genuine permission error, which carries a different code, is still reported as itself rather than being mislabelled an expired key. `_handle_401` is renamed `_handle_auth_failure` and takes the response, because a handler named for one status that must handle two is how this stayed hidden; its messages now state the status actually observed instead of asserting "401 Unauthorized". ChatGPT is unaffected: it does not override the default, so the deleted-asset 403 path is untouched.
Seven regression tests cover the split (`TestAuthFailureDetection`), including the two that matter most: a Claude 403 with a *different* error code must **not** be treated as an auth failure, and a ChatGPT 403 must not either. The docs that repeated the wrong premise — `README.md`'s expiry table and "When Tokens Expire" section, the `auth` wizard's on-screen note, and the `ClaudeProvider` docstring — are corrected in the same change.
- **The docs claimed both ChatGPT cookie chunks were required; they are not.** `README.md` stated flatly that "ChatGPT splits large session tokens across two cookies to stay under the browser's 4KB cookie limit. Both are required," and `.env.example` documented only the chunked layout — so a machine whose session token happens to fit in a single `__Secure-next-auth.session-token` cookie looked broken, with the user hunting for a `.1` that does not exist. Chrome splits a cookie only above ~4KB, so the layout varies by session size and the *same account* can be chunked on one machine and not on another.
The code was already correct: `CHATGPT_SESSION_TOKEN_1` is optional (`src/providers/chatgpt.py:162`) and the `auth` wizard already told you to paste a lone cookie into `.0` and leave `.1` blank. Only the reference docs were wrong, and they are the ones read when setting up a new machine.
Measured 2026-09-20 against `/api/auth/session`, reassembling a real 4089-byte token to test each naming: chunked `.0`+`.1` → 200 with an `accessToken`; the whole value under the unchunked name → 200 with an `accessToken`; the whole value under `.0` alone → 200 with an `accessToken`. The server reassembles a complete value sent under `.0`, so both layouts authenticate as the code already assumed. A *partial* `.0` with its `.1` omitted is the one combination that fails, and it fails **silently** — HTTP 200 with no `accessToken` rather than an error — which is now documented in both files alongside the correction.
- **The test suite sent real push notifications to the developer's phone.** `TestSyncCommand` invokes the actual `sync` command, which calls `load_config()`, which calls `load_dotenv()` — so the real `.env` was loaded and its live `NTFY_TOPIC` used for the POST. Every `pytest` run fired three or four pushes, including a fabricated "codex: 3 conversation(s) failed to export" straight out of a fixture, which is worse than noise: it reports a failure that never happened. Nothing appeared in `cache/logs/exporter.log` to explain it, because every test invocation passes `--no-log-file`. - **The test suite sent real push notifications to the developer's phone.** `TestSyncCommand` invokes the actual `sync` command, which calls `load_config()`, which calls `load_dotenv()` — so the real `.env` was loaded and its live `NTFY_TOPIC` used for the POST. Every `pytest` run fired three or four pushes, including a fabricated "codex: 3 conversation(s) failed to export" straight out of a fixture, which is worse than noise: it reports a failure that never happened. Nothing appeared in `cache/logs/exporter.log` to explain it, because every test invocation passes `--no-log-file`.
A `tests/conftest.py` autouse fixture now neutralises the environment for every test: `NTFY_TOPIC`/`NTFY_TOKEN` are emptied, and `NTFY_SERVER` and `JOPLIN_API_URL` are pointed at a closed local port, so a stray topic cannot reach the internet and a test cannot write notes into a real Joplin instance. The values are **emptied rather than deleted** — `load_dotenv(override=False)` skips only keys already present, so deleting one lets `.env` put it back. Verified by instrumenting `requests` across a full run: zero outbound requests, where the same instrumentation without the fixture records POSTs to the live ntfy topic. A `tests/conftest.py` autouse fixture now neutralises the environment for every test: `NTFY_TOPIC`/`NTFY_TOKEN` are emptied, and `NTFY_SERVER` and `JOPLIN_API_URL` are pointed at a closed local port, so a stray topic cannot reach the internet and a test cannot write notes into a real Joplin instance. The values are **emptied rather than deleted** — `load_dotenv(override=False)` skips only keys already present, so deleting one lets `.env` put it back. Verified by instrumenting `requests` across a full run: zero outbound requests, where the same instrumentation without the fixture records POSTs to the live ntfy topic.
+39 -10
View File
@@ -169,8 +169,8 @@ The wizard detects your OS, shows the correct DevTools shortcut, and writes the
| Provider | Cookie Name | Lifetime | Expiry Detection | | Provider | Cookie Name | Lifetime | Expiry Detection |
|----------|-------------|----------|-----------------| |----------|-------------|----------|-----------------|
| ChatGPT | `__Secure-next-auth.session-token.0` + `.1` | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. | | ChatGPT | `__Secure-next-auth.session-token` (split into `.0` + `.1` when over ~4KB) | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. |
| Claude | `sessionKey` | ~30 days | Opaque token — only detectable via 401 response | | Claude | `sessionKey` | ~30 days | Opaque token — only detectable from an API rejection. claude.ai answers an invalid session with **403** `permission_error` / `account_session_invalid`, **not** 401; `doctor` reports it on the "Claude API reachable" row. |
### Finding Tokens in Chrome DevTools ### Finding Tokens in Chrome DevTools
@@ -180,20 +180,38 @@ The wizard detects your OS, shows the correct DevTools shortcut, and writes the
4. In the left panel, expand **Cookies** and click the site URL 4. In the left panel, expand **Cookies** and click the site URL
5. Find the cookie by name and copy its **Value** 5. Find the cookie by name and copy its **Value**
**ChatGPT:** go to `https://chatgpt.com` → find **two** cookies: **ChatGPT:** go to `https://chatgpt.com` → find the session token cookie. You will
- `__Secure-next-auth.session-token.0` — copy Value (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN` see **one of two layouts**, depending on how large your session token is:
- `__Secure-next-auth.session-token.1` — copy Value → `CHATGPT_SESSION_TOKEN_1`
ChatGPT splits large session tokens across two cookies to stay under the browser's 4KB cookie limit. Both are required. - **One cookie**, `__Secure-next-auth.session-token` — copy Value → `CHATGPT_SESSION_TOKEN`, and leave `CHATGPT_SESSION_TOKEN_1` empty.
- **Two cookies**, `__Secure-next-auth.session-token.0` and `.1` — copy `.0` (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN`, and `.1` → `CHATGPT_SESSION_TOKEN_1`.
Chrome splits a cookie only when it exceeds ~4KB, so a larger session is chunked
and a smaller one is not — the same account can differ from machine to machine.
`CHATGPT_SESSION_TOKEN_1` is optional; both layouts authenticate, because the
server reassembles a complete value sent under the `.0` name.
What does *not* work is sending a **partial** chunk — `.0` on its own when a `.1`
exists. That fails silently: `/api/auth/session` answers HTTP 200 with no
`accessToken` rather than an error. If you see two cookies, copy both.
**Claude:** go to `https://claude.ai` → find `sessionKey` → copy Value **Claude:** go to `https://claude.ai` → find `sessionKey` → copy Value
### When Tokens Expire ### When Tokens Expire
When a token expires you'll see a `401 Unauthorized` error. To refresh: An expired token shows up as an authentication error naming the cookie to
refresh and how. The status differs by provider — ChatGPT reports 401, while
claude.ai reports **403 "Invalid authorization"** (`account_session_invalid`) —
so don't read a 403 from Claude as a permissions problem with your account.
To refresh:
- Re-run the `auth` wizard: `ai-chat-exporter auth` - Re-run the `auth` wizard: `ai-chat-exporter auth`
- Or manually update the value in your `.env` file - Or manually update the value in your `.env` file
`ai-chat-exporter doctor` is the quickest check: the "token set" rows only test
that a value is present, so an expired credential passes those and fails on the
"API reachable" row.
--- ---
## The `auth` Command ## The `auth` Command
@@ -411,7 +429,8 @@ cache on the next run that finds Joplin up.
providers in a single action rather than one action each, because systemd providers in a single action rather than one action each, because systemd
`oneshot` stops at the first failing `ExecStart` and Task Scheduler reports only `oneshot` stops at the first failing `ExecStart` and Task Scheduler reports only
the last action's result. Every provider is attempted; the run still exits the last action's result. Every provider is attempted; the run still exits
non-zero if any failed. non-zero if any failed. On Linux the loop is `scheduling/run-sync.sh`, which the
unit's `ExecStart` calls.
### Getting notified ### Getting notified
@@ -439,6 +458,16 @@ high priority with an alert tag, so a failed archive is distinguishable from a
quiet one on your phone. `NTFY_NOTIFY=failure` notifies only on failure; `off` quiet one on your phone. `NTFY_NOTIFY=failure` notifies only on failure; `off`
disables it; `--notify` / `--no-notify` override per run. disables it; `--notify` / `--no-notify` override per run.
`sync` can only push from the end of a run it finished. A crash, an exit before
the sync starts (the ToS gate, a cache error) or a launcher that can't build its
venv sends nothing — and since each provider pushes separately, the ones that
succeeded still say "OK", so a dead provider looks like a quiet day. On Linux,
`scheduling/run-sync.sh` closes that gap: any run that exits non-zero without the
app having reported it gets a high-priority **FAILED** push naming the provider
and, for a crash, the exception's class (`claude-code: crashed (RecursionError)`)
— the class only, never its message, which can carry a conversation title. The
traceback is in the journal. The Windows task has no such backstop yet.
The message includes the **machine name**, which matters because both machines The message includes the **machine name**, which matters because both machines
archive into one topic. It contains counts only — never conversation titles. A archive into one topic. It contains counts only — never conversation titles. A
topic on public ntfy.sh is readable by anyone who knows its name, so if you want topic on public ntfy.sh is readable by anyone who knows its name, so if you want
@@ -776,12 +805,12 @@ To force a full re-export: `ai-chat-exporter cache --clear` then re-run export.
## Troubleshooting ## Troubleshooting
### `401 Unauthorized` ### `Authentication failed` (401, or 403 from Claude)
Your session token has expired. Your session token has expired.
- Run `ai-chat-exporter auth` to get a new token interactively - Run `ai-chat-exporter auth` to get a new token interactively
- Or manually copy a fresh cookie value into your `.env` file - Or manually copy a fresh cookie value into your `.env` file
Note: Claude's `sessionKey` is an opaque string — the only way to know it's expired is the 401 error. ChatGPT JWTs have an `exp` claim that the `doctor` command can decode and display. Note: neither token's expiry can be read client-side. Claude's `sessionKey` is an opaque string, and claude.ai reports an invalid one as **403** "Invalid authorization" (`account_session_invalid`), not 401. ChatGPT's token is an encrypted JWE; `doctor` reads the `error` field of `/api/auth/session` instead. See [When Tokens Expire](#when-tokens-expire).
### `429 Rate Limited` ### `429 Rate Limited`
The tool automatically pauses, saves progress, and exits with a clear message showing how many conversations were exported vs remaining. Just re-run the same export command to resume — the cache picks up exactly where it left off. The tool automatically pauses, saves progress, and exits with a clear message showing how many conversations were exported vs remaining. Just re-run the same export command to resume — the cache picks up exactly where it left off.
+8 -8
View File
@@ -44,10 +44,12 @@ fi
[ ${#PROVIDERS[@]} -eq 0 ] && PROVIDERS=("all") [ ${#PROVIDERS[@]} -eq 0 ] && PROVIDERS=("all")
if [ ! -x "$REPO/ai-chat-exporter" ]; then for f in "$REPO/ai-chat-exporter" "$REPO/scheduling/run-sync.sh"; do
echo "error: $REPO/ai-chat-exporter is missing or not executable." >&2 if [ ! -x "$f" ]; then
echo "error: $f is missing or not executable." >&2
exit 1 exit 1
fi fi
done
mkdir -p "$UNIT_DIR" mkdir -p "$UNIT_DIR"
@@ -65,12 +67,10 @@ mkdir -p "$UNIT_DIR"
echo "Type=oneshot" echo "Type=oneshot"
echo "WorkingDirectory=$REPO" echo "WorkingDirectory=$REPO"
echo "Environment=AI_CHAT_EXPORTER_QUIET_CWD=1" echo "Environment=AI_CHAT_EXPORTER_QUIET_CWD=1"
# One ExecStart per provider would stop at the first failure, silently echo "Environment=AICHAT_SYNC_UNIT=$NAME"
# skipping the rest — an expired ChatGPT token would mean codex never runs. # run-sync.sh attempts every provider even after one fails, and pushes a
# Loop instead, so every provider is attempted and the unit still reports # FAILED notification for any run that died without sending its own.
# failure if any of them failed. echo "ExecStart=$REPO/scheduling/run-sync.sh ${PROVIDERS[*]}"
printf 'ExecStart=/bin/sh -c '\''rc=0; for p in %s; do "$0" sync --provider "$p" --joplin-optional || rc=1; done; exit $rc'\'' %s\n' \
"${PROVIDERS[*]}" "$REPO/ai-chat-exporter"
} > "$UNIT_DIR/$NAME.service" } > "$UNIT_DIR/$NAME.service"
# Persistent=true runs a missed schedule at the next boot — the machine being # Persistent=true runs a missed schedule at the next boot — the machine being
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
# Run `ai-chat-exporter sync` once per provider — the ExecStart of the systemd
# unit that install-systemd-timer.sh writes.
#
# ./scheduling/run-sync.sh claude-code codex
#
# Every provider is attempted even after one fails, and the exit code is
# non-zero if any failed — one ExecStart per provider would stop at the first.
#
# The app pushes its own ntfy result, but only from the end of a run it
# finished. A crash, a non-zero exit before the sync starts (the terms-of-service
# gate, a cache error) or a launcher that can't build its venv sends nothing, and
# because each provider pushes separately, the providers that did succeed still
# send "OK" — so a broken one looks like a quiet day. This script pushes a FAILED
# notification for any run that exited non-zero without the app having reported
# it. (Its "Sync completed with failures" banner prints right after its push.)
#
# The push carries the provider, the exit code and, for a crash, the exception's
# class name — never its message. Same counts-only rule as src/notify.py: on a
# public ntfy topic anyone who guesses the name can read it, and exception text
# can carry conversation titles. The full traceback is in the journal.
set -uo pipefail
REPO="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
LAUNCHER="$REPO/ai-chat-exporter"
# NTFY_* as the app resolves them: the environment wins, then .env.
env_value() {
local name=$1 value=${!1:-}
if [ -z "$value" ] && [ -f "$REPO/.env" ]; then
value=$(sed -n "s/^[[:space:]]*$name[[:space:]]*=[[:space:]]*//p" "$REPO/.env" | tail -n 1)
value=${value%%[[:space:]]#*}
value=${value%"${value##*[![:space:]]}"}
value=${value#[\"\']}
value=${value%[\"\']}
fi
printf '%s' "$value"
}
push_failure() {
local body=$1 topic server token policy
topic=$(env_value NTFY_TOPIC)
policy=$(env_value NTFY_NOTIFY | tr '[:upper:]' '[:lower:]')
if [ -z "$topic" ] || [ "$policy" = "off" ]; then
return 0
fi
server=$(env_value NTFY_SERVER)
server=${server:-https://ntfy.sh}
token=$(env_value NTFY_TOKEN)
local args=(-fsS --max-time 15 -o /dev/null
-H "Title: AI archive FAILED - $(hostname -s)"
-H "Tags: rotating_light" -H "Priority: high"
--data-binary "$body")
[ -n "$token" ] && args+=(-H "Authorization: Bearer $token")
curl "${args[@]}" "${server%/}/$topic" \
|| echo "run-sync: could not send the failure notification" >&2
}
[ $# -eq 0 ] && set -- all
rc=0
for provider in "$@"; do
out=$(mktemp)
"$LAUNCHER" sync --provider "$provider" --joplin-optional 2>&1 | tee "$out"
status=${PIPESTATUS[0]}
if [ "$status" -ne 0 ]; then
rc=1
if ! grep -q "Sync completed with failures" "$out"; then
crash=$(grep -oE '^[A-Za-z_][A-Za-z0-9_.]*(Error|Exception)\b' "$out" | tail -n 1)
if [ -n "$crash" ]; then
reason="crashed ($crash)"
else
reason="exited $status before reporting a result"
fi
push_failure "$provider: $reason
journalctl --user -u ${AICHAT_SYNC_UNIT:-aichat-sync} -n 100"
fi
fi
rm -f "$out"
done
exit "$rc"
+4 -1
View File
@@ -258,7 +258,10 @@ def _auth_claude(os_name: str) -> None:
console.print("2. Press [bold]F12[/bold] to open DevTools → Application tab.") console.print("2. Press [bold]F12[/bold] to open DevTools → Application tab.")
console.print("3. Expand [bold]Cookies[/bold] → [bold]https://claude.ai[/bold]") console.print("3. Expand [bold]Cookies[/bold] → [bold]https://claude.ai[/bold]")
console.print("4. Find [bold]sessionKey[/bold] → copy the Value.") console.print("4. Find [bold]sessionKey[/bold] → copy the Value.")
console.print(" (Note: Claude tokens expire after ~30 days; a 401 error is the only signal.)") console.print(
" (Note: Claude tokens expire after ~30 days; an API rejection — 403 "
"'Invalid authorization' — is the only signal.)"
)
console.print("5. Paste it below (input is hidden).\n") console.print("5. Paste it below (input is hidden).\n")
key = click.prompt("Claude session key", hide_input=True, default="", show_default=False).strip() key = click.prompt("Claude session key", hide_input=True, default="", show_default=False).strip()
+29 -10
View File
@@ -335,7 +335,8 @@ class BaseProvider(ABC):
Parsed JSON response body. Parsed JSON response body.
Raises: Raises:
ProviderError: On 401, exhausted retries, or unrecoverable errors. ProviderError: On an authentication failure, exhausted retries, or
unrecoverable errors.
""" """
kwargs.setdefault("timeout", REQUEST_TIMEOUT) kwargs.setdefault("timeout", REQUEST_TIMEOUT)
@@ -361,14 +362,19 @@ class BaseProvider(ABC):
elapsed_ms, elapsed_ms,
) )
# ── 401: token expired / invalid ────────────────────────── # ── Auth failure: token expired / invalid ─────────────────
if response.status_code == 401: # Not keyed on 401 alone: a provider is free to answer an
self._handle_401() # invalid session with some other status, and claude.ai does
# _handle_401 raises ProviderError — this line never runs # (403). Ask the provider rather than assuming.
if self._is_auth_failure(response):
self._handle_auth_failure(response)
# _handle_auth_failure raises — this line never runs
raise ProviderError( raise ProviderError(
self.provider_name, self.provider_name,
f"{method} {url}", f"{method} {url}",
RuntimeError("401 Unauthorized"), RuntimeError(
f"HTTP {response.status_code} — not authenticated"
),
) )
# ── 429: rate limited ────────────────────────────────────── # ── 429: rate limited ──────────────────────────────────────
@@ -476,11 +482,22 @@ class BaseProvider(ABC):
last_exc or RuntimeError("Unknown error"), last_exc or RuntimeError("Unknown error"),
) )
def _handle_401(self) -> None: def _is_auth_failure(self, response: Any) -> bool:
"""Log a clear human-readable message for a 401 and raise ProviderError.""" """Whether this response means the stored credential is not valid.
Defaults to 401. Providers whose API reports an invalid session with a
different status override this — claude.ai returns 403, so keying auth
handling on 401 alone reports an expired key as a generic permission
error and never tells the user to refresh it.
"""
return bool(response.status_code == 401)
def _handle_auth_failure(self, response: Any) -> None:
"""Log a clear human-readable message for an auth failure and raise."""
# Subclasses override to include provider-specific cookie name # Subclasses override to include provider-specific cookie name
msg = ( msg = (
f"[{self.provider_name}] Authentication failed (401 Unauthorized). " f"[{self.provider_name}] Authentication failed "
f"(HTTP {response.status_code}). "
"Your session token has likely expired. " "Your session token has likely expired. "
"Run 'ai-chat-exporter auth' to refresh your token." "Run 'ai-chat-exporter auth' to refresh your token."
) )
@@ -488,7 +505,9 @@ class BaseProvider(ABC):
raise ProviderError( raise ProviderError(
self.provider_name, self.provider_name,
"authentication", "authentication",
RuntimeError("401 Unauthorized — token expired"), RuntimeError(
f"HTTP {response.status_code} — session token expired or invalid"
),
) )
@staticmethod @staticmethod
+5 -3
View File
@@ -307,9 +307,9 @@ class ChatGPTProvider(BaseProvider):
) )
return access_token return access_token
def _handle_401(self) -> None: def _handle_auth_failure(self, response: Any) -> None:
msg = ( msg = (
"[chatgpt] Authentication failed (401 Unauthorized). " f"[chatgpt] Authentication failed (HTTP {response.status_code}). "
"Your __Secure-next-auth.session-token has likely expired (~7 day lifetime). " "Your __Secure-next-auth.session-token has likely expired (~7 day lifetime). "
"The session token is used to obtain a short-lived access token via /api/auth/session. " "The session token is used to obtain a short-lived access token via /api/auth/session. "
"To refresh: open chatgpt.com in Chrome → F12 → Application → Cookies " "To refresh: open chatgpt.com in Chrome → F12 → Application → Cookies "
@@ -320,7 +320,9 @@ class ChatGPTProvider(BaseProvider):
raise ProviderError( raise ProviderError(
self.provider_name, self.provider_name,
"authentication", "authentication",
RuntimeError("401 Unauthorized — ChatGPT token expired"), RuntimeError(
f"HTTP {response.status_code} — ChatGPT session token expired"
),
) )
# ------------------------------------------------------------------ # ------------------------------------------------------------------
+40 -5
View File
@@ -2,6 +2,7 @@
import logging import logging
import os import os
from typing import Any
from curl_cffi import requests as curl_requests from curl_cffi import requests as curl_requests
@@ -38,7 +39,9 @@ class ClaudeProvider(BaseProvider):
Cloudflare's bot detection (same issue as chatgpt.com). Cloudflare's bot detection (same issue as chatgpt.com).
Authentication: sessionKey cookie (~30 day lifetime, opaque string). Authentication: sessionKey cookie (~30 day lifetime, opaque string).
Expiry cannot be decoded client-side — a 401 is the only signal. Expiry cannot be decoded client-side, so an API rejection is the only
signal — and claude.ai sends 403 permission_error / account_session_invalid
for an invalid session, not 401. See ``_is_auth_failure``.
""" """
provider_name = "claude" provider_name = "claude"
@@ -71,11 +74,41 @@ class ClaudeProvider(BaseProvider):
self._org_id: str | None = None # cached per session self._org_id: str | None = None # cached per session
logger.debug("[claude] Session initialised with Chrome TLS impersonation (key: [REDACTED])") logger.debug("[claude] Session initialised with Chrome TLS impersonation (key: [REDACTED])")
def _handle_401(self) -> None: # claude.ai answers an invalid or expired sessionKey with 403
# permission_error / account_session_invalid — never 401. Verified live
# 2026-09-20 against GET /api/organizations: a valid key returns 200, while
# an expired key, a garbage key and no cookie at all return byte-identical
# 403s carrying this code. Matching on the code rather than on the bare
# status keeps a genuine permission problem (which would carry a different
# code) reported as itself.
_SESSION_INVALID_CODE = "account_session_invalid"
def _is_auth_failure(self, response: Any) -> bool:
if response.status_code == 401:
return True
if response.status_code != 403:
return False
try:
body = response.json()
except Exception:
return False
if not isinstance(body, dict):
return False
error = body.get("error")
if not isinstance(error, dict):
return False
details = error.get("details")
if not isinstance(details, dict):
return False
return details.get("error_code") == self._SESSION_INVALID_CODE
def _handle_auth_failure(self, response: Any) -> None:
msg = ( msg = (
"[claude] Authentication failed (401 Unauthorized). " f"[claude] Authentication failed (HTTP {response.status_code}). "
"Your sessionKey has likely expired (~30 day lifetime). " "Your sessionKey has likely expired (~30 day lifetime). "
"Note: Claude session keys are opaque — a 401 is the only expiry signal. " "Note: Claude session keys are opaque, and claude.ai reports an "
"invalid one as 403 'Invalid authorization', not 401 — an API "
"rejection is the only expiry signal. "
"To refresh: open claude.ai in Chrome → F12 → Application → Cookies " "To refresh: open claude.ai in Chrome → F12 → Application → Cookies "
"→ find 'sessionKey' → copy the value. " "→ find 'sessionKey' → copy the value. "
"Then run 'ai-chat-exporter auth' or update CLAUDE_SESSION_KEY in .env." "Then run 'ai-chat-exporter auth' or update CLAUDE_SESSION_KEY in .env."
@@ -84,7 +117,9 @@ class ClaudeProvider(BaseProvider):
raise ProviderError( raise ProviderError(
self.provider_name, self.provider_name,
"authentication", "authentication",
RuntimeError("401 Unauthorized — Claude session key expired"), RuntimeError(
f"HTTP {response.status_code} — Claude session key expired or invalid"
),
) )
def _get_org_id(self) -> str: def _get_org_id(self) -> str:
+15 -1
View File
@@ -118,7 +118,10 @@ _HARNESS_TAG_RE = re.compile(
r"|<command-args>.*?</command-args>" r"|<command-args>.*?</command-args>"
r"|<command-contents>.*?</command-contents>" r"|<command-contents>.*?</command-contents>"
r"|<local-command-stdout>.*?</local-command-stdout>" r"|<local-command-stdout>.*?</local-command-stdout>"
r"|<system-reminder>.*?</system-reminder>", r"|<system-reminder>.*?</system-reminder>"
# A fork subagent's first user turn: generic worker rules ahead of the
# fork's actual "Your directive: …", which is kept.
r"|<fork-boilerplate>.*?</fork-boilerplate>",
re.DOTALL, re.DOTALL,
) )
@@ -530,6 +533,7 @@ def _extract_messages(
policy: str, policy: str,
subagents: dict | None = None, subagents: dict | None = None,
include_sidechain: bool = False, include_sidechain: bool = False,
expanding: frozenset[str] = frozenset(),
) -> list[dict]: ) -> list[dict]:
"""Normalize Claude Code records into messages. """Normalize Claude Code records into messages.
@@ -540,6 +544,13 @@ def _extract_messages(
those recursive subagent passes (subagent records are flagged those recursive subagent passes (subagent records are flagged
``isSidechain``); the top-level pass keeps skipping sidechain records so a ``isSidechain``); the top-level pass keeps skipping sidechain records so a
subagent is never also emitted as a stray top-level turn. subagent is never also emitted as a stray top-level turn.
``expanding`` holds the spawn ids of the subagents being folded around
this pass. A ``fork`` subagent's transcript opens with a copy of the parent
turn that spawned it, its own spawn call included; that copy is dropped,
since the enclosing subagent block already stands for it. Expanding it
again recursed without end (RecursionError, every daily sync from
2026-09-24).
""" """
subagents = subagents or {} subagents = subagents or {}
messages: list[dict] = [] messages: list[dict] = []
@@ -620,6 +631,8 @@ def _extract_messages(
elif item_type == "tool_use": elif item_type == "tool_use":
name = item.get("name") or "tool" name = item.get("name") or "tool"
tool_id = item.get("id") tool_id = item.get("id")
if tool_id in expanding:
continue
if name in _SUBAGENT_TOOL_NAMES and tool_id in subagents: if name in _SUBAGENT_TOOL_NAMES and tool_id in subagents:
# A Task/Agent spawn: fold its separate transcript inline # A Task/Agent spawn: fold its separate transcript inline
# instead of collapsing it. Its own tool traffic is # instead of collapsing it. Its own tool traffic is
@@ -633,6 +646,7 @@ def _extract_messages(
policy, policy,
subagents, subagents,
include_sidechain=True, include_sidechain=True,
expanding=expanding | {tool_id},
) )
blocks.append( blocks.append(
make_subagent_block( make_subagent_block(
+53
View File
@@ -370,6 +370,59 @@ class TestSubagentFold:
] ]
assert "stray sidechain" not in top_text assert "stray sidechain" not in top_text
def test_fork_containing_its_own_spawn_call(self, tmp_path):
# The shape Claude Code writes for a `fork` subagent: a context-ref
# record, a copy of the parent turn holding the fork's own spawn call,
# then the directive behind <fork-boilerplate>. Folding that copied
# call recursed until RecursionError.
fork_records = [
{"type": "fork-context-ref", "agentId": "a1", "parentLastUuid": "u0"},
{
"type": "assistant", "isSidechain": True,
"message": {"role": "assistant", "content": [
{"type": "tool_use", "id": "toolu_sub1", "name": "Agent",
"input": {"description": "research"}},
]},
},
{
"type": "user", "isSidechain": True,
"timestamp": "2026-05-01T10:00:06.000Z",
"message": {"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "toolu_sub1",
"content": [{"type": "text", "text": "Fork started"}]},
{"type": "text", "text":
"<fork-boilerplate>\nYou are a worker fork.\n"
"</fork-boilerplate>\n\nYour directive: research X"},
]},
},
{
"type": "assistant", "isSidechain": True,
"timestamp": "2026-05-01T10:00:07.000Z",
"message": {"role": "assistant", "content": [
{"type": "text", "text": "Fork result."},
]},
},
]
f = _write_session(tmp_path, _parent_with_task(), name="parent-3")
_write_subagent(
f, "toolu_sub1", fork_records,
agent_type="fork", description="research X",
)
p = self._provider(tmp_path)
p.fetch_all_conversations()
conv = p.normalize_conversation(p.get_conversation("parent-3"))
subs = [
b for m in conv["messages"] for b in m["blocks"] if b["type"] == "subagent"
]
assert len(subs) == 1
inner = [bb for m in subs[0]["messages"] for bb in m["blocks"]]
assert not any(b["type"] == "subagent" for b in inner)
inner_text = [b.get("text") for b in inner]
assert "Your directive: research X" in inner_text
assert "Fork result." in inner_text
assert not any("worker fork" in (t or "") for t in inner_text)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Repo tags in title (Part C) # Repo tags in title (Part C)
+127
View File
@@ -1184,6 +1184,133 @@ class TestErrorBodyDiagnostics:
assert "[REDACTED]" in described assert "[REDACTED]" in described
# ---------------------------------------------------------------------------
# Auth failure detection: claude.ai reports an invalid/expired sessionKey as
# 403 permission_error / account_session_invalid, never 401. Verified live
# 2026-09-20 against GET /api/organizations — a valid key returned 200, while
# an expired key, a garbage key and no cookie at all returned byte-identical
# 403s. Keying auth handling on 401 alone hid the refresh instructions behind
# a raw JSON dump.
# ---------------------------------------------------------------------------
class TestAuthFailureDetection:
class _Resp:
reason = ""
headers: dict = {}
def __init__(self, status, payload=None):
self.status_code = status
self.ok = 200 <= status < 400
self._payload = payload
self.text = ""
def json(self):
if self._payload is None:
raise ValueError("not json")
return self._payload
@staticmethod
def _bare(cls, response):
p = cls.__new__(cls)
p._request_delay = 0
p._last_request_at = None
p._session = type("S", (), {"request": lambda *a, **k: response})()
return p
@staticmethod
def _session_invalid(message="Invalid authorization"):
return {
"type": "error",
"error": {
"type": "permission_error",
"message": message,
"details": {
"error_code": "account_session_invalid",
"error_visibility": "user_facing",
},
},
}
def test_claude_403_session_invalid_is_an_auth_failure(self):
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, self._session_invalid())
prov = self._bare(ClaudeProvider, resp)
assert prov._is_auth_failure(resp) is True
with pytest.raises(ProviderError) as exc:
prov._make_request("GET", "https://claude.ai/api/organizations")
# The actionable message, not a dump of the response body.
assert exc.value.operation == "authentication"
assert "session key expired or invalid" in str(exc.value.original)
def test_claude_403_auth_failure_tells_the_user_how_to_refresh(self, caplog):
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, self._session_invalid())
with caplog.at_level(logging.ERROR):
with pytest.raises(ProviderError):
self._bare(ClaudeProvider, resp)._make_request(
"GET", "https://claude.ai/api/organizations"
)
logged = caplog.text
assert "sessionKey" in logged
assert "CLAUDE_SESSION_KEY" in logged
# States the status it actually saw, rather than claiming 401.
assert "403" in logged
assert "401 Unauthorized" not in logged
def test_claude_403_with_another_error_code_is_not_an_auth_failure(self):
"""A genuine permission problem must stay reported as itself."""
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
payload = {
"type": "error",
"error": {
"type": "permission_error",
"message": "Organization access denied",
"details": {"error_code": "org_access_denied"},
},
}
resp = self._Resp(403, payload)
prov = self._bare(ClaudeProvider, resp)
assert prov._is_auth_failure(resp) is False
with pytest.raises(ProviderError) as exc:
prov._make_request("GET", "https://claude.ai/api/organizations")
assert exc.value.operation != "authentication"
assert "Organization access denied" in str(exc.value.original)
def test_claude_403_with_unparseable_body_is_not_an_auth_failure(self):
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, None)
assert self._bare(ClaudeProvider, resp)._is_auth_failure(resp) is False
def test_claude_401_is_still_an_auth_failure(self):
from src.providers.claude import ClaudeProvider
resp = self._Resp(401, {})
assert self._bare(ClaudeProvider, resp)._is_auth_failure(resp) is True
def test_chatgpt_403_is_not_an_auth_failure(self):
"""Guards the deleted-asset path: a media 403 is not an expired token."""
from src.providers.chatgpt import ChatGPTProvider
resp = self._Resp(403, {"detail": "Forbidden"})
assert self._bare(ChatGPTProvider, resp)._is_auth_failure(resp) is False
def test_chatgpt_401_is_an_auth_failure(self):
from src.providers.chatgpt import ChatGPTProvider
resp = self._Resp(401, {})
assert self._bare(ChatGPTProvider, resp)._is_auth_failure(resp) is True
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Deleted assets: ChatGPT's download endpoint answers a missing upload with # Deleted assets: ChatGPT's download endpoint answers a missing upload with
# 403 Forbidden, not 404. Measured 2026-08-17 over 18 such assets — every one # 403 Forbidden, not 404. Measured 2026-08-17 over 18 such assets — every one