3 Commits
Author SHA1 Message Date
JesseMarkowitzandClaude Opus 5.5 23c6e1f512 fix: push a FAILED notification when a scheduled run dies without reporting
sync pushes its result only from the end of a run it finished, so a crash or an early exit sent nothing, while the providers that survived kept pushing OK. The systemd unit now runs scheduling/run-sync.sh, which keeps the per-provider loop and pushes a high-priority failure, with the exception class only, for any run that exited non-zero without the app's own report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:16 -04:00
JesseMarkowitzandClaude Opus 5.5 64068bb19b fix: fork subagents recursed forever in the Claude Code export
A fork subagent's transcript opens with a copy of the parent turn that spawned it, its own Agent call included; folding that call re-entered the same fork until RecursionError, failing every daily sync since 2026-09-24. Skip a spawn call while its own subagent is being expanded, and strip the <fork-boilerplate> preamble.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:16 -04:00
JesseMarkowitzandClaude Opus 5.5 b6ce636891 fix: detect claude.ai's 403 session-invalid as an expired key; ChatGPT .1 cookie is optional
claude.ai answers an invalid or expired sessionKey with 403 account_session_invalid, never 401, so the refresh-your-cookie message could not fire for Claude. Auth detection is now a provider decision (_is_auth_failure); Claude matches the 403 on its error code so a real permission error still reports as itself. README and .env.example no longer claim both ChatGPT cookie chunks are required.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbnmGHnFqDjyhPcCg1SEfF
2026-10-05 07:14:01 -04:00
12 changed files with 441 additions and 42 deletions
+10 -2
View File
@@ -6,10 +6,18 @@
# --- ChatGPT ---
# How to get: open chatgpt.com in Chrome → F12 → Application tab
# → Cookies → https://chatgpt.com → find the two cookie chunks:
# → Cookies → https://chatgpt.com → find the session token cookie. Chrome splits a
# cookie only above ~4KB, so you will see ONE of these two layouts:
#
# __Secure-next-auth.session-token (the whole value) → CHATGPT_SESSION_TOKEN
# (leave _1 empty)
# or, when the token was large enough to be split:
# __Secure-next-auth.session-token.0 (starts with "eyJ") → CHATGPT_SESSION_TOKEN
# __Secure-next-auth.session-token.1 (the remainder) → CHATGPT_SESSION_TOKEN_1
# Token type: JWE. Typically valid for ~7 days.
#
# CHATGPT_SESSION_TOKEN_1 is OPTIONAL — leave it empty when there is no .1 cookie.
# But if a .1 cookie does exist, you must copy it: a partial .0 fails silently
# (HTTP 200 with no accessToken). Token type: JWE. Typically valid for ~7 days.
CHATGPT_SESSION_TOKEN=
CHATGPT_SESSION_TOKEN_1=
+26
View File
@@ -6,6 +6,32 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased]
### Fixed
- **Every scheduled Claude Code sync since 2026-09-24 died with `RecursionError`.** Claude Code's `fork` subagents write a transcript that opens with a copy of the parent turn that spawned them — the fork's own `Agent` call included. `_extract_messages` folds a subagent inline whenever it meets its spawn call, so it met that copy inside the fork, folded the same fork again, and recursed until Python's limit. One fork anywhere in `~/.claude/projects` was enough to fail the whole provider; seven existed across three sessions, and the Codex half of the run, unaffected, hid the cause behind a generic exit 1.
The recursive pass now carries the spawn ids being expanded around it and skips a tool_use whose id is among them — the enclosing subagent block already stands for that call. Because the set accumulates, a longer cycle (A spawns B, whose transcript re-spawns A) stops too. The fork's `<fork-boilerplate>` preamble — generic worker rules the harness prepends — is now stripped with the other harness tags, so a fork block opens with its actual directive.
`TestSubagentFold.test_fork_containing_its_own_spawn_call` reproduces the on-disk shape (a `fork-context-ref` record, the copied spawn call, the boilerplate-wrapped directive) and fails with the original `RecursionError` against the unfixed code. Verified against the real archive: all 85 sessions normalize, the seven forks each render as one subagent block.
- **A scheduled run that crashed sent no notification, and the providers that survived said "OK".** `sync` pushes its ntfy result from the end of a run it finished, so the `RecursionError` above — and any crash, any exit before the sync starts (the ToS gate, a cache error), a launcher that cannot build its venv — sent nothing at all. Worse, the unit runs each provider as its own `sync`, so codex kept pushing a low-priority "AI archive OK" every morning for the eleven days claude-code was dead: a broken provider was indistinguishable from a quiet day.
The systemd unit's `ExecStart` is now `scheduling/run-sync.sh`, which carries the old per-provider loop and pushes a high-priority **FAILED** notification for any run that exits non-zero without the app's "Sync completed with failures" banner (printed right after the app's own push, so an app-reported failure is not reported twice). The push names the provider and, for a crash, the exception class only — `claude-code: crashed (RecursionError)` — never its message, holding to `src/notify.py`'s counts-only rule for a topic anyone can read. It honours `NTFY_NOTIFY=off` and reads `NTFY_*` the way the app does: environment first, then `.env`. Re-run `install-systemd-timer.sh` to pick it up. The Windows task has no equivalent yet.
Verified against a fake launcher and a local capture server: a crash pushes with the class and without the message, an exit before the sync pushes, an app-reported failure and a success push nothing extra, `off` pushes nothing, and the run still exits 1 if any provider failed.
- **An expired Claude session key reported a raw JSON dump instead of how to fix it.** `_make_request` routed only **401** to the auth handler (`src/providers/base.py`), and claude.ai does not use 401 — an invalid or expired `sessionKey` comes back as `403 permission_error` with `details.error_code = account_session_invalid`. So the one message that names the cookie, its ~30-day lifetime and the DevTools path to refresh it could never fire for Claude. What the user got instead was the generic 4xx path: `HTTP 403 — error: {'type': 'permission_error', 'message': 'Invalid authorization'…}`, which reads like a permissions problem with the account and not like "your key expired, here is how to replace it."
Measured live 2026-09-20 against `GET /api/organizations`: a valid key returns 200, while an expired key, a deliberately malformed key and **no cookie at all** return byte-identical 403s carrying that code — i.e. the API treats a dead session as an absent one. This is the same mistake as the ChatGPT media 403s below: assuming 403 means "forbidden" when the service uses it for "unauthenticated."
Auth detection is now a provider decision rather than a hardcoded status. `BaseProvider._is_auth_failure(response)` defaults to 401 and `ClaudeProvider` overrides it to add 403 **matched on `account_session_invalid`**, not on the bare status — so a genuine permission error, which carries a different code, is still reported as itself rather than being mislabelled an expired key. `_handle_401` is renamed `_handle_auth_failure` and takes the response, because a handler named for one status that must handle two is how this stayed hidden; its messages now state the status actually observed instead of asserting "401 Unauthorized". ChatGPT is unaffected: it does not override the default, so the deleted-asset 403 path is untouched.
Seven regression tests cover the split (`TestAuthFailureDetection`), including the two that matter most: a Claude 403 with a *different* error code must **not** be treated as an auth failure, and a ChatGPT 403 must not either. The docs that repeated the wrong premise — `README.md`'s expiry table and "When Tokens Expire" section, the `auth` wizard's on-screen note, and the `ClaudeProvider` docstring — are corrected in the same change.
- **The docs claimed both ChatGPT cookie chunks were required; they are not.** `README.md` stated flatly that "ChatGPT splits large session tokens across two cookies to stay under the browser's 4KB cookie limit. Both are required," and `.env.example` documented only the chunked layout — so a machine whose session token happens to fit in a single `__Secure-next-auth.session-token` cookie looked broken, with the user hunting for a `.1` that does not exist. Chrome splits a cookie only above ~4KB, so the layout varies by session size and the *same account* can be chunked on one machine and not on another.
The code was already correct: `CHATGPT_SESSION_TOKEN_1` is optional (`src/providers/chatgpt.py:162`) and the `auth` wizard already told you to paste a lone cookie into `.0` and leave `.1` blank. Only the reference docs were wrong, and they are the ones read when setting up a new machine.
Measured 2026-09-20 against `/api/auth/session`, reassembling a real 4089-byte token to test each naming: chunked `.0`+`.1` → 200 with an `accessToken`; the whole value under the unchunked name → 200 with an `accessToken`; the whole value under `.0` alone → 200 with an `accessToken`. The server reassembles a complete value sent under `.0`, so both layouts authenticate as the code already assumed. A *partial* `.0` with its `.1` omitted is the one combination that fails, and it fails **silently** — HTTP 200 with no `accessToken` rather than an error — which is now documented in both files alongside the correction.
- **The test suite sent real push notifications to the developer's phone.** `TestSyncCommand` invokes the actual `sync` command, which calls `load_config()`, which calls `load_dotenv()` — so the real `.env` was loaded and its live `NTFY_TOPIC` used for the POST. Every `pytest` run fired three or four pushes, including a fabricated "codex: 3 conversation(s) failed to export" straight out of a fixture, which is worse than noise: it reports a failure that never happened. Nothing appeared in `cache/logs/exporter.log` to explain it, because every test invocation passes `--no-log-file`.
A `tests/conftest.py` autouse fixture now neutralises the environment for every test: `NTFY_TOPIC`/`NTFY_TOKEN` are emptied, and `NTFY_SERVER` and `JOPLIN_API_URL` are pointed at a closed local port, so a stray topic cannot reach the internet and a test cannot write notes into a real Joplin instance. The values are **emptied rather than deleted** — `load_dotenv(override=False)` skips only keys already present, so deleting one lets `.env` put it back. Verified by instrumenting `requests` across a full run: zero outbound requests, where the same instrumentation without the fixture records POSTs to the live ntfy topic.
+39 -10
View File
@@ -169,8 +169,8 @@ The wizard detects your OS, shows the correct DevTools shortcut, and writes the
| Provider | Cookie Name | Lifetime | Expiry Detection |
|----------|-------------|----------|-----------------|
| ChatGPT | `__Secure-next-auth.session-token.0` + `.1` | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. |
| Claude | `sessionKey` | ~30 days | Opaque token — only detectable via 401 response |
| ChatGPT | `__Secure-next-auth.session-token` (split into `.0` + `.1` when over ~4KB) | refresh ~weekly | `error` field of `/api/auth/session` — `doctor` reports "ChatGPT token active". The token is an encrypted JWE, so its `exp` is **not** readable client-side, and the `expires` field is a misleading rolling window; the `error` (`RefreshAccessTokenError` when dead) is the honest signal. |
| Claude | `sessionKey` | ~30 days | Opaque token — only detectable from an API rejection. claude.ai answers an invalid session with **403** `permission_error` / `account_session_invalid`, **not** 401; `doctor` reports it on the "Claude API reachable" row. |
### Finding Tokens in Chrome DevTools
@@ -180,20 +180,38 @@ The wizard detects your OS, shows the correct DevTools shortcut, and writes the
4. In the left panel, expand **Cookies** and click the site URL
5. Find the cookie by name and copy its **Value**
**ChatGPT:** go to `https://chatgpt.com` → find **two** cookies:
- `__Secure-next-auth.session-token.0` — copy Value (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN`
- `__Secure-next-auth.session-token.1` — copy Value → `CHATGPT_SESSION_TOKEN_1`
**ChatGPT:** go to `https://chatgpt.com` → find the session token cookie. You will
see **one of two layouts**, depending on how large your session token is:
ChatGPT splits large session tokens across two cookies to stay under the browser's 4KB cookie limit. Both are required.
- **One cookie**, `__Secure-next-auth.session-token` — copy Value → `CHATGPT_SESSION_TOKEN`, and leave `CHATGPT_SESSION_TOKEN_1` empty.
- **Two cookies**, `__Secure-next-auth.session-token.0` and `.1` — copy `.0` (starts with `eyJ`) → `CHATGPT_SESSION_TOKEN`, and `.1` → `CHATGPT_SESSION_TOKEN_1`.
Chrome splits a cookie only when it exceeds ~4KB, so a larger session is chunked
and a smaller one is not — the same account can differ from machine to machine.
`CHATGPT_SESSION_TOKEN_1` is optional; both layouts authenticate, because the
server reassembles a complete value sent under the `.0` name.
What does *not* work is sending a **partial** chunk — `.0` on its own when a `.1`
exists. That fails silently: `/api/auth/session` answers HTTP 200 with no
`accessToken` rather than an error. If you see two cookies, copy both.
**Claude:** go to `https://claude.ai` → find `sessionKey` → copy Value
### When Tokens Expire
When a token expires you'll see a `401 Unauthorized` error. To refresh:
An expired token shows up as an authentication error naming the cookie to
refresh and how. The status differs by provider — ChatGPT reports 401, while
claude.ai reports **403 "Invalid authorization"** (`account_session_invalid`) —
so don't read a 403 from Claude as a permissions problem with your account.
To refresh:
- Re-run the `auth` wizard: `ai-chat-exporter auth`
- Or manually update the value in your `.env` file
`ai-chat-exporter doctor` is the quickest check: the "token set" rows only test
that a value is present, so an expired credential passes those and fails on the
"API reachable" row.
---
## The `auth` Command
@@ -411,7 +429,8 @@ cache on the next run that finds Joplin up.
providers in a single action rather than one action each, because systemd
`oneshot` stops at the first failing `ExecStart` and Task Scheduler reports only
the last action's result. Every provider is attempted; the run still exits
non-zero if any failed.
non-zero if any failed. On Linux the loop is `scheduling/run-sync.sh`, which the
unit's `ExecStart` calls.
### Getting notified
@@ -439,6 +458,16 @@ high priority with an alert tag, so a failed archive is distinguishable from a
quiet one on your phone. `NTFY_NOTIFY=failure` notifies only on failure; `off`
disables it; `--notify` / `--no-notify` override per run.
`sync` can only push from the end of a run it finished. A crash, an exit before
the sync starts (the ToS gate, a cache error) or a launcher that can't build its
venv sends nothing — and since each provider pushes separately, the ones that
succeeded still say "OK", so a dead provider looks like a quiet day. On Linux,
`scheduling/run-sync.sh` closes that gap: any run that exits non-zero without the
app having reported it gets a high-priority **FAILED** push naming the provider
and, for a crash, the exception's class (`claude-code: crashed (RecursionError)`)
— the class only, never its message, which can carry a conversation title. The
traceback is in the journal. The Windows task has no such backstop yet.
The message includes the **machine name**, which matters because both machines
archive into one topic. It contains counts only — never conversation titles. A
topic on public ntfy.sh is readable by anyone who knows its name, so if you want
@@ -776,12 +805,12 @@ To force a full re-export: `ai-chat-exporter cache --clear` then re-run export.
## Troubleshooting
### `401 Unauthorized`
### `Authentication failed` (401, or 403 from Claude)
Your session token has expired.
- Run `ai-chat-exporter auth` to get a new token interactively
- Or manually copy a fresh cookie value into your `.env` file
Note: Claude's `sessionKey` is an opaque string — the only way to know it's expired is the 401 error. ChatGPT JWTs have an `exp` claim that the `doctor` command can decode and display.
Note: neither token's expiry can be read client-side. Claude's `sessionKey` is an opaque string, and claude.ai reports an invalid one as **403** "Invalid authorization" (`account_session_invalid`), not 401. ChatGPT's token is an encrypted JWE; `doctor` reads the `error` field of `/api/auth/session` instead. See [When Tokens Expire](#when-tokens-expire).
### `429 Rate Limited`
The tool automatically pauses, saves progress, and exits with a clear message showing how many conversations were exported vs remaining. Just re-run the same export command to resume — the cache picks up exactly where it left off.
+8 -8
View File
@@ -44,10 +44,12 @@ fi
[ ${#PROVIDERS[@]} -eq 0 ] && PROVIDERS=("all")
if [ ! -x "$REPO/ai-chat-exporter" ]; then
echo "error: $REPO/ai-chat-exporter is missing or not executable." >&2
for f in "$REPO/ai-chat-exporter" "$REPO/scheduling/run-sync.sh"; do
if [ ! -x "$f" ]; then
echo "error: $f is missing or not executable." >&2
exit 1
fi
done
mkdir -p "$UNIT_DIR"
@@ -65,12 +67,10 @@ mkdir -p "$UNIT_DIR"
echo "Type=oneshot"
echo "WorkingDirectory=$REPO"
echo "Environment=AI_CHAT_EXPORTER_QUIET_CWD=1"
# One ExecStart per provider would stop at the first failure, silently
# skipping the rest — an expired ChatGPT token would mean codex never runs.
# Loop instead, so every provider is attempted and the unit still reports
# failure if any of them failed.
printf 'ExecStart=/bin/sh -c '\''rc=0; for p in %s; do "$0" sync --provider "$p" --joplin-optional || rc=1; done; exit $rc'\'' %s\n' \
"${PROVIDERS[*]}" "$REPO/ai-chat-exporter"
echo "Environment=AICHAT_SYNC_UNIT=$NAME"
# run-sync.sh attempts every provider even after one fails, and pushes a
# FAILED notification for any run that died without sending its own.
echo "ExecStart=$REPO/scheduling/run-sync.sh ${PROVIDERS[*]}"
} > "$UNIT_DIR/$NAME.service"
# Persistent=true runs a missed schedule at the next boot — the machine being
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
# Run `ai-chat-exporter sync` once per provider — the ExecStart of the systemd
# unit that install-systemd-timer.sh writes.
#
# ./scheduling/run-sync.sh claude-code codex
#
# Every provider is attempted even after one fails, and the exit code is
# non-zero if any failed — one ExecStart per provider would stop at the first.
#
# The app pushes its own ntfy result, but only from the end of a run it
# finished. A crash, a non-zero exit before the sync starts (the terms-of-service
# gate, a cache error) or a launcher that can't build its venv sends nothing, and
# because each provider pushes separately, the providers that did succeed still
# send "OK" — so a broken one looks like a quiet day. This script pushes a FAILED
# notification for any run that exited non-zero without the app having reported
# it. (Its "Sync completed with failures" banner prints right after its push.)
#
# The push carries the provider, the exit code and, for a crash, the exception's
# class name — never its message. Same counts-only rule as src/notify.py: on a
# public ntfy topic anyone who guesses the name can read it, and exception text
# can carry conversation titles. The full traceback is in the journal.
set -uo pipefail
REPO="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
LAUNCHER="$REPO/ai-chat-exporter"
# NTFY_* as the app resolves them: the environment wins, then .env.
env_value() {
local name=$1 value=${!1:-}
if [ -z "$value" ] && [ -f "$REPO/.env" ]; then
value=$(sed -n "s/^[[:space:]]*$name[[:space:]]*=[[:space:]]*//p" "$REPO/.env" | tail -n 1)
value=${value%%[[:space:]]#*}
value=${value%"${value##*[![:space:]]}"}
value=${value#[\"\']}
value=${value%[\"\']}
fi
printf '%s' "$value"
}
push_failure() {
local body=$1 topic server token policy
topic=$(env_value NTFY_TOPIC)
policy=$(env_value NTFY_NOTIFY | tr '[:upper:]' '[:lower:]')
if [ -z "$topic" ] || [ "$policy" = "off" ]; then
return 0
fi
server=$(env_value NTFY_SERVER)
server=${server:-https://ntfy.sh}
token=$(env_value NTFY_TOKEN)
local args=(-fsS --max-time 15 -o /dev/null
-H "Title: AI archive FAILED - $(hostname -s)"
-H "Tags: rotating_light" -H "Priority: high"
--data-binary "$body")
[ -n "$token" ] && args+=(-H "Authorization: Bearer $token")
curl "${args[@]}" "${server%/}/$topic" \
|| echo "run-sync: could not send the failure notification" >&2
}
[ $# -eq 0 ] && set -- all
rc=0
for provider in "$@"; do
out=$(mktemp)
"$LAUNCHER" sync --provider "$provider" --joplin-optional 2>&1 | tee "$out"
status=${PIPESTATUS[0]}
if [ "$status" -ne 0 ]; then
rc=1
if ! grep -q "Sync completed with failures" "$out"; then
crash=$(grep -oE '^[A-Za-z_][A-Za-z0-9_.]*(Error|Exception)\b' "$out" | tail -n 1)
if [ -n "$crash" ]; then
reason="crashed ($crash)"
else
reason="exited $status before reporting a result"
fi
push_failure "$provider: $reason
journalctl --user -u ${AICHAT_SYNC_UNIT:-aichat-sync} -n 100"
fi
fi
rm -f "$out"
done
exit "$rc"
+4 -1
View File
@@ -258,7 +258,10 @@ def _auth_claude(os_name: str) -> None:
console.print("2. Press [bold]F12[/bold] to open DevTools → Application tab.")
console.print("3. Expand [bold]Cookies[/bold] → [bold]https://claude.ai[/bold]")
console.print("4. Find [bold]sessionKey[/bold] → copy the Value.")
console.print(" (Note: Claude tokens expire after ~30 days; a 401 error is the only signal.)")
console.print(
" (Note: Claude tokens expire after ~30 days; an API rejection — 403 "
"'Invalid authorization' — is the only signal.)"
)
console.print("5. Paste it below (input is hidden).\n")
key = click.prompt("Claude session key", hide_input=True, default="", show_default=False).strip()
+29 -10
View File
@@ -335,7 +335,8 @@ class BaseProvider(ABC):
Parsed JSON response body.
Raises:
ProviderError: On 401, exhausted retries, or unrecoverable errors.
ProviderError: On an authentication failure, exhausted retries, or
unrecoverable errors.
"""
kwargs.setdefault("timeout", REQUEST_TIMEOUT)
@@ -361,14 +362,19 @@ class BaseProvider(ABC):
elapsed_ms,
)
# ── 401: token expired / invalid ──────────────────────────
if response.status_code == 401:
self._handle_401()
# _handle_401 raises ProviderError — this line never runs
# ── Auth failure: token expired / invalid ─────────────────
# Not keyed on 401 alone: a provider is free to answer an
# invalid session with some other status, and claude.ai does
# (403). Ask the provider rather than assuming.
if self._is_auth_failure(response):
self._handle_auth_failure(response)
# _handle_auth_failure raises — this line never runs
raise ProviderError(
self.provider_name,
f"{method} {url}",
RuntimeError("401 Unauthorized"),
RuntimeError(
f"HTTP {response.status_code} — not authenticated"
),
)
# ── 429: rate limited ──────────────────────────────────────
@@ -476,11 +482,22 @@ class BaseProvider(ABC):
last_exc or RuntimeError("Unknown error"),
)
def _handle_401(self) -> None:
"""Log a clear human-readable message for a 401 and raise ProviderError."""
def _is_auth_failure(self, response: Any) -> bool:
"""Whether this response means the stored credential is not valid.
Defaults to 401. Providers whose API reports an invalid session with a
different status override this — claude.ai returns 403, so keying auth
handling on 401 alone reports an expired key as a generic permission
error and never tells the user to refresh it.
"""
return bool(response.status_code == 401)
def _handle_auth_failure(self, response: Any) -> None:
"""Log a clear human-readable message for an auth failure and raise."""
# Subclasses override to include provider-specific cookie name
msg = (
f"[{self.provider_name}] Authentication failed (401 Unauthorized). "
f"[{self.provider_name}] Authentication failed "
f"(HTTP {response.status_code}). "
"Your session token has likely expired. "
"Run 'ai-chat-exporter auth' to refresh your token."
)
@@ -488,7 +505,9 @@ class BaseProvider(ABC):
raise ProviderError(
self.provider_name,
"authentication",
RuntimeError("401 Unauthorized — token expired"),
RuntimeError(
f"HTTP {response.status_code} — session token expired or invalid"
),
)
@staticmethod
+5 -3
View File
@@ -307,9 +307,9 @@ class ChatGPTProvider(BaseProvider):
)
return access_token
def _handle_401(self) -> None:
def _handle_auth_failure(self, response: Any) -> None:
msg = (
"[chatgpt] Authentication failed (401 Unauthorized). "
f"[chatgpt] Authentication failed (HTTP {response.status_code}). "
"Your __Secure-next-auth.session-token has likely expired (~7 day lifetime). "
"The session token is used to obtain a short-lived access token via /api/auth/session. "
"To refresh: open chatgpt.com in Chrome → F12 → Application → Cookies "
@@ -320,7 +320,9 @@ class ChatGPTProvider(BaseProvider):
raise ProviderError(
self.provider_name,
"authentication",
RuntimeError("401 Unauthorized — ChatGPT token expired"),
RuntimeError(
f"HTTP {response.status_code} — ChatGPT session token expired"
),
)
# ------------------------------------------------------------------
+40 -5
View File
@@ -2,6 +2,7 @@
import logging
import os
from typing import Any
from curl_cffi import requests as curl_requests
@@ -38,7 +39,9 @@ class ClaudeProvider(BaseProvider):
Cloudflare's bot detection (same issue as chatgpt.com).
Authentication: sessionKey cookie (~30 day lifetime, opaque string).
Expiry cannot be decoded client-side — a 401 is the only signal.
Expiry cannot be decoded client-side, so an API rejection is the only
signal — and claude.ai sends 403 permission_error / account_session_invalid
for an invalid session, not 401. See ``_is_auth_failure``.
"""
provider_name = "claude"
@@ -71,11 +74,41 @@ class ClaudeProvider(BaseProvider):
self._org_id: str | None = None # cached per session
logger.debug("[claude] Session initialised with Chrome TLS impersonation (key: [REDACTED])")
def _handle_401(self) -> None:
# claude.ai answers an invalid or expired sessionKey with 403
# permission_error / account_session_invalid — never 401. Verified live
# 2026-09-20 against GET /api/organizations: a valid key returns 200, while
# an expired key, a garbage key and no cookie at all return byte-identical
# 403s carrying this code. Matching on the code rather than on the bare
# status keeps a genuine permission problem (which would carry a different
# code) reported as itself.
_SESSION_INVALID_CODE = "account_session_invalid"
def _is_auth_failure(self, response: Any) -> bool:
if response.status_code == 401:
return True
if response.status_code != 403:
return False
try:
body = response.json()
except Exception:
return False
if not isinstance(body, dict):
return False
error = body.get("error")
if not isinstance(error, dict):
return False
details = error.get("details")
if not isinstance(details, dict):
return False
return details.get("error_code") == self._SESSION_INVALID_CODE
def _handle_auth_failure(self, response: Any) -> None:
msg = (
"[claude] Authentication failed (401 Unauthorized). "
f"[claude] Authentication failed (HTTP {response.status_code}). "
"Your sessionKey has likely expired (~30 day lifetime). "
"Note: Claude session keys are opaque — a 401 is the only expiry signal. "
"Note: Claude session keys are opaque, and claude.ai reports an "
"invalid one as 403 'Invalid authorization', not 401 — an API "
"rejection is the only expiry signal. "
"To refresh: open claude.ai in Chrome → F12 → Application → Cookies "
"→ find 'sessionKey' → copy the value. "
"Then run 'ai-chat-exporter auth' or update CLAUDE_SESSION_KEY in .env."
@@ -84,7 +117,9 @@ class ClaudeProvider(BaseProvider):
raise ProviderError(
self.provider_name,
"authentication",
RuntimeError("401 Unauthorized — Claude session key expired"),
RuntimeError(
f"HTTP {response.status_code} — Claude session key expired or invalid"
),
)
def _get_org_id(self) -> str:
+15 -1
View File
@@ -118,7 +118,10 @@ _HARNESS_TAG_RE = re.compile(
r"|<command-args>.*?</command-args>"
r"|<command-contents>.*?</command-contents>"
r"|<local-command-stdout>.*?</local-command-stdout>"
r"|<system-reminder>.*?</system-reminder>",
r"|<system-reminder>.*?</system-reminder>"
# A fork subagent's first user turn: generic worker rules ahead of the
# fork's actual "Your directive: …", which is kept.
r"|<fork-boilerplate>.*?</fork-boilerplate>",
re.DOTALL,
)
@@ -530,6 +533,7 @@ def _extract_messages(
policy: str,
subagents: dict | None = None,
include_sidechain: bool = False,
expanding: frozenset[str] = frozenset(),
) -> list[dict]:
"""Normalize Claude Code records into messages.
@@ -540,6 +544,13 @@ def _extract_messages(
those recursive subagent passes (subagent records are flagged
``isSidechain``); the top-level pass keeps skipping sidechain records so a
subagent is never also emitted as a stray top-level turn.
``expanding`` holds the spawn ids of the subagents being folded around
this pass. A ``fork`` subagent's transcript opens with a copy of the parent
turn that spawned it, its own spawn call included; that copy is dropped,
since the enclosing subagent block already stands for it. Expanding it
again recursed without end (RecursionError, every daily sync from
2026-09-24).
"""
subagents = subagents or {}
messages: list[dict] = []
@@ -620,6 +631,8 @@ def _extract_messages(
elif item_type == "tool_use":
name = item.get("name") or "tool"
tool_id = item.get("id")
if tool_id in expanding:
continue
if name in _SUBAGENT_TOOL_NAMES and tool_id in subagents:
# A Task/Agent spawn: fold its separate transcript inline
# instead of collapsing it. Its own tool traffic is
@@ -633,6 +646,7 @@ def _extract_messages(
policy,
subagents,
include_sidechain=True,
expanding=expanding | {tool_id},
)
blocks.append(
make_subagent_block(
+53
View File
@@ -370,6 +370,59 @@ class TestSubagentFold:
]
assert "stray sidechain" not in top_text
def test_fork_containing_its_own_spawn_call(self, tmp_path):
# The shape Claude Code writes for a `fork` subagent: a context-ref
# record, a copy of the parent turn holding the fork's own spawn call,
# then the directive behind <fork-boilerplate>. Folding that copied
# call recursed until RecursionError.
fork_records = [
{"type": "fork-context-ref", "agentId": "a1", "parentLastUuid": "u0"},
{
"type": "assistant", "isSidechain": True,
"message": {"role": "assistant", "content": [
{"type": "tool_use", "id": "toolu_sub1", "name": "Agent",
"input": {"description": "research"}},
]},
},
{
"type": "user", "isSidechain": True,
"timestamp": "2026-05-01T10:00:06.000Z",
"message": {"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "toolu_sub1",
"content": [{"type": "text", "text": "Fork started"}]},
{"type": "text", "text":
"<fork-boilerplate>\nYou are a worker fork.\n"
"</fork-boilerplate>\n\nYour directive: research X"},
]},
},
{
"type": "assistant", "isSidechain": True,
"timestamp": "2026-05-01T10:00:07.000Z",
"message": {"role": "assistant", "content": [
{"type": "text", "text": "Fork result."},
]},
},
]
f = _write_session(tmp_path, _parent_with_task(), name="parent-3")
_write_subagent(
f, "toolu_sub1", fork_records,
agent_type="fork", description="research X",
)
p = self._provider(tmp_path)
p.fetch_all_conversations()
conv = p.normalize_conversation(p.get_conversation("parent-3"))
subs = [
b for m in conv["messages"] for b in m["blocks"] if b["type"] == "subagent"
]
assert len(subs) == 1
inner = [bb for m in subs[0]["messages"] for bb in m["blocks"]]
assert not any(b["type"] == "subagent" for b in inner)
inner_text = [b.get("text") for b in inner]
assert "Your directive: research X" in inner_text
assert "Fork result." in inner_text
assert not any("worker fork" in (t or "") for t in inner_text)
# ---------------------------------------------------------------------------
# Repo tags in title (Part C)
+127
View File
@@ -1184,6 +1184,133 @@ class TestErrorBodyDiagnostics:
assert "[REDACTED]" in described
# ---------------------------------------------------------------------------
# Auth failure detection: claude.ai reports an invalid/expired sessionKey as
# 403 permission_error / account_session_invalid, never 401. Verified live
# 2026-09-20 against GET /api/organizations — a valid key returned 200, while
# an expired key, a garbage key and no cookie at all returned byte-identical
# 403s. Keying auth handling on 401 alone hid the refresh instructions behind
# a raw JSON dump.
# ---------------------------------------------------------------------------
class TestAuthFailureDetection:
class _Resp:
reason = ""
headers: dict = {}
def __init__(self, status, payload=None):
self.status_code = status
self.ok = 200 <= status < 400
self._payload = payload
self.text = ""
def json(self):
if self._payload is None:
raise ValueError("not json")
return self._payload
@staticmethod
def _bare(cls, response):
p = cls.__new__(cls)
p._request_delay = 0
p._last_request_at = None
p._session = type("S", (), {"request": lambda *a, **k: response})()
return p
@staticmethod
def _session_invalid(message="Invalid authorization"):
return {
"type": "error",
"error": {
"type": "permission_error",
"message": message,
"details": {
"error_code": "account_session_invalid",
"error_visibility": "user_facing",
},
},
}
def test_claude_403_session_invalid_is_an_auth_failure(self):
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, self._session_invalid())
prov = self._bare(ClaudeProvider, resp)
assert prov._is_auth_failure(resp) is True
with pytest.raises(ProviderError) as exc:
prov._make_request("GET", "https://claude.ai/api/organizations")
# The actionable message, not a dump of the response body.
assert exc.value.operation == "authentication"
assert "session key expired or invalid" in str(exc.value.original)
def test_claude_403_auth_failure_tells_the_user_how_to_refresh(self, caplog):
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, self._session_invalid())
with caplog.at_level(logging.ERROR):
with pytest.raises(ProviderError):
self._bare(ClaudeProvider, resp)._make_request(
"GET", "https://claude.ai/api/organizations"
)
logged = caplog.text
assert "sessionKey" in logged
assert "CLAUDE_SESSION_KEY" in logged
# States the status it actually saw, rather than claiming 401.
assert "403" in logged
assert "401 Unauthorized" not in logged
def test_claude_403_with_another_error_code_is_not_an_auth_failure(self):
"""A genuine permission problem must stay reported as itself."""
from src.providers.base import ProviderError
from src.providers.claude import ClaudeProvider
payload = {
"type": "error",
"error": {
"type": "permission_error",
"message": "Organization access denied",
"details": {"error_code": "org_access_denied"},
},
}
resp = self._Resp(403, payload)
prov = self._bare(ClaudeProvider, resp)
assert prov._is_auth_failure(resp) is False
with pytest.raises(ProviderError) as exc:
prov._make_request("GET", "https://claude.ai/api/organizations")
assert exc.value.operation != "authentication"
assert "Organization access denied" in str(exc.value.original)
def test_claude_403_with_unparseable_body_is_not_an_auth_failure(self):
from src.providers.claude import ClaudeProvider
resp = self._Resp(403, None)
assert self._bare(ClaudeProvider, resp)._is_auth_failure(resp) is False
def test_claude_401_is_still_an_auth_failure(self):
from src.providers.claude import ClaudeProvider
resp = self._Resp(401, {})
assert self._bare(ClaudeProvider, resp)._is_auth_failure(resp) is True
def test_chatgpt_403_is_not_an_auth_failure(self):
"""Guards the deleted-asset path: a media 403 is not an expired token."""
from src.providers.chatgpt import ChatGPTProvider
resp = self._Resp(403, {"detail": "Forbidden"})
assert self._bare(ChatGPTProvider, resp)._is_auth_failure(resp) is False
def test_chatgpt_401_is_an_auth_failure(self):
from src.providers.chatgpt import ChatGPTProvider
resp = self._Resp(401, {})
assert self._bare(ChatGPTProvider, resp)._is_auth_failure(resp) is True
# ---------------------------------------------------------------------------
# Deleted assets: ChatGPT's download endpoint answers a missing upload with
# 403 Forbidden, not 404. Measured 2026-08-17 over 18 such assets — every one