fix: retry asset downloads on 5xx and recover them across runs

An asset download is two hops — mint a signed URL, then fetch the bytes —
and only the first went through _make_request. The second was a bare
session.request with no 5xx handling, which lost two images in the
2026-10-20 09:00 run: one to a 500 the retried first hop had already
recovered from, one to a 503. Both re-downloaded cleanly an hour later
(106 KB and 1.3 MB), so the bytes were there all along.

A third asset timed out and aborted its whole conversation. curl_cffi's
RequestException subclasses OSError, so the un-wrapped timeout slipped
past resolve_media's except ProviderError into the export loop's
except OSError and was reported as "File write failed for conversation
6a945155" — a conversation that had already downloaded ~20 images.
Nothing was wrong with any file write.

The retry loop is factored out of _make_request into _send_with_retries,
which returns the raw response so a bytes caller gets the same handling
as a JSON one. Hop 2 goes through it: 5xx retried, Retry-After honoured,
timeouts surfacing as ProviderError for resolve_media to placeholder.
The hop is now logged, which it never was. Assets get their own ladder —
an immediate retry, then ~2/4/8/16s with jitter kept, since a storage
incident fails many assets at once — bounded by a per-asset wall-clock
budget, because the timeout case costs 40s a go.

Beyond the retry:

- Transient failures (5xx, timeout, network) are recorded as
  pending_media so a later run re-fetches just that asset; capped at
  3 runs. 404/403 stay permanent — settled 2026-08-17 for 19 assets.
- A circuit breaker stops downloads after 5 consecutive transient
  failures, recording the rest as retryable. Permanent failures never
  trip it; any success resets the streak.
- Media failures now appear in the set ids.
  The run that lost two assets pushed a clean "OK".

Keeps FUTURE.md's per-conversation cache reset correctly closed. The
README split (FUTURE.md §2) stays otial
enough to justify breaking every anchor link.

37 new tests (414 passing). Verified live: both lost assets recovered,
doctor green, a read-only chatgpt dns.
The cross-run retry and breaker are unit-tested only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/cwa7qnGp
Claude-Session: https://claude.ai/code/session_012gCWSknsWvhLnMFwa7qnGp

CHANGELOG.md

@@ -6,6 +6,35 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ## [Unreleased]
 
 ### Fixed
+- **A transient 5xx on an image download lost the image permanently, and a timeout lost the whole conversation.** An asset download is two hops: `GET /files/{id}/download` mints a signed URL, then that URL is fetched for the bytes. Only the first went through `_make_request`. The second was a bare `session.request` with no 5xx handling at all, which produced two distinct failures in the 2026-10-20 09:00 run:
+
+  ```
+  09:04:05  .../files/<id>/download → 500   Retry 1/3 in 2.2s
+  09:04:17  → 500                           Retry 2/3 in 4.3s
+  09:04:32  → 500                           Retry 3/3 in 9.1s
+  09:04:42  → 200                            ← hop 1 recovered on attempt 4
+  09:04:47  WARNING  Signed URL returned HTTP 500   ← hop 2, one try, gone
+  ```
+
+  Hop 1 was patiently retried and won; hop 2 failed once and gave up. A second asset was lost the same way to a **503**, so this was never specific to 500. Both assets downloaded cleanly on 2026-10-20 at 10:00 — 106 KB and 1.3 MB — confirming the bytes were there all along and only the retry policy lost them.
+
+  Worse, a third asset **timed out**, and that aborted the entire conversation. `curl_cffi`'s `RequestException` subclasses `OSError`, so an un-wrapped timeout on hop 2 slipped past `resolve_media`'s `except ProviderError` and was caught by the export loop's `except OSError`, which reported it as **"File write failed for conversation 6a945155"** — a conversation that had already downloaded ~20 images. Nothing was wrong with any file write. The design intent in `media.py` ("Failures keep placeholders; never fatal") was bypassed because the exception took a path nobody had anticipated.
+
+  The retry loop is now factored out of `_make_request` into `_send_with_retries`, which returns the raw response so a caller wanting **bytes** gets identical handling to one wanting JSON. Hop 2 goes through it, which fixes both bugs at once: 5xx is retried, `Retry-After` is honoured, and a timeout arrives as `ProviderError` — which `resolve_media` already knows how to placeholder. The hop is also now logged, which it never was; its absence from the log is what made this take a debug run to find. `_pace()` is no longer called by hand there, because `_send_with_retries` paces every attempt — pacing twice would have added `REQUEST_DELAY` to all 88 assets in that run.
+
+  Assets get a more patient ladder than the conversation API, since they fail for their own reasons: an immediate retry (free, and a single bad backend node is the commonest shape of a transient 5xx), then ~2/4/8/16s with jitter. **Jitter is deliberately kept** rather than using fixed steps — a storage incident fails many assets at once, and un-jittered retries would hit a recovering backend in lockstep. The ladder is bounded by a wall-clock budget per asset, and the final attempt's read timeout is clamped to what remains, because the timeout case costs 40s a go and six unbounded attempts of it would eat a scheduled run's execution window (two hours, per the Windows task definition).
+
+### Added
+- **Transiently failed assets are re-attempted on later runs.** Previously `mark_exported` ran regardless, so a conversation whose image 500'd was "up to date" forever and the image was unrecoverable without hand-editing the manifest. Failures are now split by whether retrying can possibly help: `expired-or-missing` (404) and `forbidden` (403) are permanent — that was settled exhaustively on 2026-08-17 for 19 assets whose bytes OpenAI lost in a Library migration — while 5xx, timeouts and network blips are transient. Transient refs are recorded against the conversation as `pending_media`, which makes `is_cached` treat it as stale so the next run re-fetches them. Re-exporting is cheap: media already on disk is reused and Joplin resource IDs are keyed by path, so only the missing asset is fetched and only the new resource uploaded. Capped at 3 runs (`MAX_MEDIA_RETRY_RUNS`), because an asset that 5xxs every morning would otherwise re-export and re-sync its conversation forever — one of today's conversations took 3m15s and 50 resource uploads. The refs are kept after the cap as a record of what was lost.
+
+  This leaves the "per-conversation cache reset" item in `FUTURE.md` correctly closed: the case that would have motivated it — recovering one conversation after a transient asset failure — now heals itself, and the documented workaround (edit the manifest) still covers the rest.
+- **A media circuit breaker, so a bad day at the provider costs one quiet morning.** Five consecutive *transient* asset failures stop downloads for the remainder of the run; the skipped assets are recorded as retryable rather than abandoned, so the next run collects them. Permanent failures never count toward tripping it — a deleted upload says nothing about the provider's health, and this archive holds 19 of those that would otherwise trip it on a perfectly healthy server. Any success resets the streak, so ordinary scattered losses never trip it either. Assets already on disk are still linked while the breaker is open, since that needs no network.
+- **Media failures appear in the push notification.** `format_summary` now adds `N media failed` per provider. The 2026-10-20 run lost two assets and pushed a clean "AI archive OK" with no hint of it — the counts existed only in the log file, which is somewhere you have to think to go and look. Counts only, never asset ids, for the same reason conversation titles are excluded: a topic on public ntfy.sh is readable by anyone who knows its name. Media failures deliberately do **not** raise the notification's priority or mark the run failed — the transcript is archived and a transient asset recovers by itself.
+
+### Notes
+- **`FUTURE.md` §2 (split the README) is still open and was deliberately not done here.** It asks to be folded into "the next substantial README edit", and this edit is a new subsection plus two cross-references — not substantial enough to justify breaking every existing link into a README anchor. The new material is written to survive the split: it lives under *Embedded images and media*, which that plan assigns to `docs/troubleshooting.md`.
+- Verified live rather than only in tests: the two assets lost in the morning run were re-downloaded successfully through the new path (`doctor` all green, 409 tests passing, a read-only `export --dry-run --provider chatgpt` exercising the refactored request core across 4 listing pages / 379 conversations). The cross-run retry and the breaker are covered by unit tests only — reproducing a multi-day storage incident against the live provider is not something this change waited for.
+
 - **Every scheduled Claude Code sync since 2026-09-24 died with `RecursionError`.** Claude Code's `fork` subagents write a transcript that opens with a copy of the parent turn that spawned them — the fork's own `Agent` call included. `_extract_messages` folds a subagent inline whenever it meets its spawn call, so it met that copy inside the fork, folded the same fork again, and recursed until Python's limit. One fork anywhere in `~/.claude/projects` was enough to fail the whole provider; seven existed across three sessions, and the Codex half of the run, unaffected, hid the cause behind a generic exit 1.
 
   The recursive pass now carries the spawn ids being expanded around it and skips a tool_use whose id is among them — the enclosing subagent block already stands for that call. Because the set accumulates, a longer cycle (A spawns B, whose transcript re-spawns A) stops too. The fork's `<fork-boilerplate>` preamble — generic worker rules the harness prepends — is now stripped with the other harness tags, so a fork block opens with its actual directive.