diff --git a/CHANGELOG.md b/CHANGELOG.md index faef6df..1923ffd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,10 +14,14 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - **A scheduled run that crashed sent no notification, and the providers that survived said "OK".** `sync` pushes its ntfy result from the end of a run it finished, so the `RecursionError` above — and any crash, any exit before the sync starts (the ToS gate, a cache error), a launcher that cannot build its venv — sent nothing at all. Worse, the unit runs each provider as its own `sync`, so codex kept pushing a low-priority "AI archive OK" every morning for the eleven days claude-code was dead: a broken provider was indistinguishable from a quiet day. - The systemd unit's `ExecStart` is now `scheduling/run-sync.sh`, which carries the old per-provider loop and pushes a high-priority **FAILED** notification for any run that exits non-zero without the app's "Sync completed with failures" banner (printed right after the app's own push, so an app-reported failure is not reported twice). The push names the provider and, for a crash, the exception class only — `claude-code: crashed (RecursionError)` — never its message, holding to `src/notify.py`'s counts-only rule for a topic anyone can read. It honours `NTFY_NOTIFY=off` and reads `NTFY_*` the way the app does: environment first, then `.env`. Re-run `install-systemd-timer.sh` to pick it up. The Windows task has no equivalent yet. + The systemd unit's `ExecStart` is now `scheduling/run-sync.sh`, which carries the old per-provider loop and pushes a high-priority **FAILED** notification for any run that exits non-zero without the app's "Sync completed with failures" banner (printed right after the app's own push, so an app-reported failure is not reported twice). The push names the provider and, for a crash, the exception class only — `claude-code: crashed (RecursionError)` — never its message, holding to `src/notify.py`'s counts-only rule for a topic anyone can read. It honours `NTFY_NOTIFY=off` and reads `NTFY_*` the way the app does: environment first, then `.env`. Re-run `install-systemd-timer.sh` to pick it up. Verified against a fake launcher and a local capture server: a crash pushes with the class and without the message, an exit before the sync pushes, an app-reported failure and a success push nothing extra, `off` pushes nothing, and the run still exits 1 if any provider failed. + The Windows task gets the same backstop. Its action is now `scheduling\run-sync.ps1` (via `powershell.exe -File`, window hidden) in place of the inline `cmd.exe` loop, with the same rules: every provider attempted, exit 1 if any failed, a FAILED push carrying the class only for a run that died unreported, `NTFY_NOTIFY=off` honoured, `NTFY_*` read from the environment first. Because Task Scheduler keeps no output, each run's output is appended to `cache\logs\scheduled-sync.log` (one rollover to `.1` past 1 MB), and the push points there. The wrapper sets `PYTHONIOENCODING=utf-8`: with stdout redirected to a file rather than a console, Python would otherwise encode in the ANSI code page and fail on the first `✓`. Re-run `Register-AiChatSyncTask.ps1` with your providers to pick it up. + + Verified on Linux under PowerShell 7.6 with a stand-in `cmd.exe`, the same fake launcher and capture server: the same five outcomes as above, plus a bearer token, an environment `NTFY_TOPIC` overriding `.env`, an unreachable server (warning, exit still 1) and the `chatgpt,claude` single-argument form. Not yet run under Windows PowerShell 5.1 or Task Scheduler. + - **An expired Claude session key reported a raw JSON dump instead of how to fix it.** `_make_request` routed only **401** to the auth handler (`src/providers/base.py`), and claude.ai does not use 401 — an invalid or expired `sessionKey` comes back as `403 permission_error` with `details.error_code = account_session_invalid`. So the one message that names the cookie, its ~30-day lifetime and the DevTools path to refresh it could never fire for Claude. What the user got instead was the generic 4xx path: `HTTP 403 — error: {'type': 'permission_error', 'message': 'Invalid authorization'…}`, which reads like a permissions problem with the account and not like "your key expired, here is how to replace it." Measured live 2026-09-20 against `GET /api/organizations`: a valid key returns 200, while an expired key, a deliberately malformed key and **no cookie at all** return byte-identical 403s carrying that code — i.e. the API treats a dead session as an absent one. This is the same mistake as the ChatGPT media 403s below: assuming 403 means "forbidden" when the service uses it for "unauthenticated." diff --git a/README.md b/README.md index f9b8d71..bba0a48 100644 --- a/README.md +++ b/README.md @@ -407,7 +407,13 @@ Check on it with `systemctl --user list-timers aichat-sync.timer` and ``` Per-user task, no admin rights needed. `-StartWhenAvailable` is the counterpart -of systemd's `Persistent=true`. +of systemd's `Persistent=true`. The task runs `scheduling\run-sync.ps1` with its +window hidden; Task Scheduler keeps no output, so each run's output is appended +to `cache\logs\scheduled-sync.log` (rolled over to `.1` past 1 MB). + +Check on it with `Get-ScheduledTaskInfo -TaskName AiChatExporterSync` (look at +`LastTaskResult`: 0 is success) and +`Get-Content cache\logs\scheduled-sync.log -Tail 50`. ### What to know before you rely on it @@ -429,15 +435,15 @@ cache on the next run that finds Joplin up. providers in a single action rather than one action each, because systemd `oneshot` stops at the first failing `ExecStart` and Task Scheduler reports only the last action's result. Every provider is attempted; the run still exits -non-zero if any failed. On Linux the loop is `scheduling/run-sync.sh`, which the -unit's `ExecStart` calls. +non-zero if any failed. The loop is `scheduling/run-sync.sh` on Linux (the +unit's `ExecStart`) and `scheduling\run-sync.ps1` on Windows (the task's action). ### Getting notified A scheduled run is silent by default. Output goes to three pull-only places: the exporter's own log (`cache/logs/exporter.log`), the systemd journal on Linux -(`journalctl --user -u aichat-sync.service`), and Task Scheduler's -`LastTaskResult` on Windows. +(`journalctl --user -u aichat-sync.service`), and on Windows Task Scheduler's +`LastTaskResult` plus `cache\logs\scheduled-sync.log`. To have runs report back, set an [ntfy](https://ntfy.sh) topic in `.env`: @@ -461,12 +467,14 @@ disables it; `--notify` / `--no-notify` override per run. `sync` can only push from the end of a run it finished. A crash, an exit before the sync starts (the ToS gate, a cache error) or a launcher that can't build its venv sends nothing — and since each provider pushes separately, the ones that -succeeded still say "OK", so a dead provider looks like a quiet day. On Linux, -`scheduling/run-sync.sh` closes that gap: any run that exits non-zero without the +succeeded still say "OK", so a dead provider looks like a quiet day. The +scheduler wrappers (`scheduling/run-sync.sh`, `scheduling\run-sync.ps1`) close +that gap: any run that exits non-zero without the app having reported it gets a high-priority **FAILED** push naming the provider and, for a crash, the exception's class (`claude-code: crashed (RecursionError)`) — the class only, never its message, which can carry a conversation title. The -traceback is in the journal. The Windows task has no such backstop yet. +traceback is in the journal on Linux and in `cache\logs\scheduled-sync.log` on +Windows. The message includes the **machine name**, which matters because both machines archive into one topic. It contains counts only — never conversation titles. A diff --git a/scheduling/Register-AiChatSyncTask.ps1 b/scheduling/Register-AiChatSyncTask.ps1 index 8883e8c..3452e83 100644 --- a/scheduling/Register-AiChatSyncTask.ps1 +++ b/scheduling/Register-AiChatSyncTask.ps1 @@ -35,6 +35,7 @@ $ErrorActionPreference = 'Stop' $repo = Split-Path -Parent $PSScriptRoot $launcher = Join-Path $repo 'ai-chat-exporter.cmd' +$wrapper = Join-Path $PSScriptRoot 'run-sync.ps1' if ($Unregister) { Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false -ErrorAction SilentlyContinue @@ -42,22 +43,22 @@ if ($Unregister) { return } -if (-not (Test-Path $launcher)) { - throw "Launcher not found at $launcher" +foreach ($f in $launcher, $wrapper) { + if (-not (Test-Path $f)) { throw "Not found: $f" } } -# A single action looping over the providers, rather than one action each. -# Task Scheduler runs multiple actions in order but reports only the last one's -# result, so a failure in an earlier provider would be invisible. The loop keeps -# going after a failure and propagates a non-zero exit code. +# One action running run-sync.ps1, rather than one action per provider: Task +# Scheduler runs multiple actions in order but reports only the last one's +# result, so a failure in an earlier provider would be invisible. The script +# keeps going after a failure, exits non-zero if any provider failed, and pushes +# a FAILED notification for any run that died without sending its own. # -# /v:on and !RC! are required, not stylistic: cmd expands every %VAR% on a -# command line *before* running any of it, so "exit /b %RC%" would report the -# value RC had before the loop ever ran - i.e. always success. Delayed expansion -# reads it at the point of use. -$loop = ($Provider | ForEach-Object { "`"$launcher`" sync --provider $_ --joplin-optional || set RC=1" }) -join ' & ' -$taskArgs = "/v:on /c set RC=0 & $loop & exit /b !RC!" -$actions = New-ScheduledTaskAction -Execute 'cmd.exe' ` +# Providers go as separate arguments: under -File, "chatgpt,claude" would arrive +# as one string. -WindowStyle Hidden because the output now goes to +# cache\logs\scheduled-sync.log, so the console window would only ever be blank. +$taskArgs = "-NoProfile -NonInteractive -ExecutionPolicy Bypass -WindowStyle Hidden " + + "-File `"$wrapper`" $($Provider -join ' ')" +$actions = New-ScheduledTaskAction -Execute 'powershell.exe' ` -Argument $taskArgs ` -WorkingDirectory $repo @@ -82,12 +83,12 @@ Register-ScheduledTask -TaskName $TaskName ` Write-Host "Registered '$TaskName' - daily at $Time for: $($Provider -join ', ')" Write-Host '' Write-Host 'Command the task will run:' -Write-Host " cmd.exe $taskArgs" +Write-Host " powershell.exe $taskArgs" Write-Host " (working directory: $repo)" Write-Host '' Write-Host 'Next steps:' Write-Host " * Run it once now: Start-ScheduledTask -TaskName $TaskName" Write-Host " * Check the result: Get-ScheduledTaskInfo -TaskName $TaskName" -Write-Host " * Read the log: Get-Content '$repo\cache\logs\exporter.log' -Tail 50" +Write-Host " * Read the run output: Get-Content '$repo\cache\logs\scheduled-sync.log' -Tail 50" Write-Host ' * The terms-of-service notice must have been acknowledged' Write-Host ' interactively once on this machine, or the task exits 1.' diff --git a/scheduling/run-sync.ps1 b/scheduling/run-sync.ps1 new file mode 100644 index 0000000..d385df5 --- /dev/null +++ b/scheduling/run-sync.ps1 @@ -0,0 +1,150 @@ +<# +.SYNOPSIS + Run `ai-chat-exporter sync` once per provider - the action of the scheduled + task that Register-AiChatSyncTask.ps1 creates. + +.DESCRIPTION + The Windows counterpart to run-sync.sh, and there for the same reason. + + Every provider is attempted even after one fails, and the exit code is + non-zero if any failed - Task Scheduler reports only the last action's + result, so one action per provider would hide an earlier failure. + + The app pushes its own ntfy result, but only from the end of a run it + finished. A crash, a non-zero exit before the sync starts (the terms-of- + service gate, a cache error) or a launcher that can't build its venv sends + nothing, and because each provider pushes separately, the providers that did + succeed still send "OK" - so a broken one looks like a quiet day. This + script pushes a FAILED notification for any run that exited non-zero without + the app having reported it. (Its "Sync completed with failures" banner + prints right after its push.) + + The push carries the provider, the exit code and, for a crash, the + exception's class name - never its message. Same counts-only rule as + src/notify.py: on a public ntfy topic anyone who guesses the name can read + it, and exception text can carry conversation titles. + + Task Scheduler keeps no output, so each run's output is appended to + cache\logs\scheduled-sync.log (rolled over to .1 past 1 MB). That is where + the traceback is. + +.EXAMPLE + powershell -NoProfile -ExecutionPolicy Bypass -File .\scheduling\run-sync.ps1 chatgpt claude +#> +[CmdletBinding()] +param( + [Parameter(ValueFromRemainingArguments = $true)] + [string[]]$Provider +) + +$ErrorActionPreference = 'Stop' + +$repo = Split-Path -Parent $PSScriptRoot +$launcher = Join-Path $repo 'ai-chat-exporter.cmd' +$envFile = Join-Path $repo '.env' +$logDir = Join-Path $repo 'cache\logs' +$runLog = Join-Path $logDir 'scheduled-sync.log' + +# -File passes "chatgpt,claude" as one string; accept it as well as a list. +$Provider = @($Provider | ForEach-Object { $_ -split ',' } | Where-Object { $_ }) +if ($Provider.Count -eq 0) { $Provider = @('all') } +foreach ($p in $Provider) { + # Each name is spliced into a cmd.exe command line below. + if ($p -notmatch '^[A-Za-z0-9_-]+$') { throw "Invalid provider name: '$p'" } +} + +# Under Task Scheduler stdout is a file, not a console, so Python would encode +# it in the ANSI code page and die on the first character outside it. +$env:PYTHONIOENCODING = 'utf-8' + +# NTFY_* as the app resolves them: the environment wins, then .env. +function Get-EnvValue([string]$Name) { + $value = [Environment]::GetEnvironmentVariable($Name) + if (-not $value -and (Test-Path -LiteralPath $envFile)) { + foreach ($line in Get-Content -LiteralPath $envFile) { + if ($line -match "^\s*$Name\s*=\s*(.*)$") { $value = $Matches[1] } + } + if ($value) { $value = ($value -replace '\s+#.*$', '').Trim().Trim('"', "'") } + } + return "$value" +} + +function Send-FailurePush([string]$Body) { + $topic = Get-EnvValue 'NTFY_TOPIC' + $policy = (Get-EnvValue 'NTFY_NOTIFY').ToLower() + if (-not $topic -or $policy -eq 'off') { return } + $server = Get-EnvValue 'NTFY_SERVER' + if (-not $server) { $server = 'https://ntfy.sh' } + $token = Get-EnvValue 'NTFY_TOKEN' + + # The same short name src/notify.py puts in its titles. + $machine = [System.Net.Dns]::GetHostName().Split('.')[0] + $headers = @{ + Title = "AI archive FAILED - $machine" + Tags = 'rotating_light' + Priority = 'high' + } + if ($token) { $headers.Authorization = "Bearer $token" } + + try { + # Windows PowerShell 5.1 on an older .NET may not offer TLS 1.2 by default. + [Net.ServicePointManager]::SecurityProtocol = ` + [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12 + Invoke-WebRequest -UseBasicParsing -Method Post -TimeoutSec 15 ` + -Uri "$($server.TrimEnd('/'))/$topic" -Headers $headers ` + -ContentType 'text/plain; charset=utf-8' ` + -Body ([Text.Encoding]::UTF8.GetBytes($Body)) | Out-Null + } catch { + Write-Warning "run-sync: could not send the failure notification ($($_.Exception.Message))" + } +} + +New-Item -ItemType Directory -Force -Path $logDir | Out-Null +if ((Test-Path -LiteralPath $runLog) -and (Get-Item -LiteralPath $runLog).Length -gt 1MB) { + Move-Item -LiteralPath $runLog -Destination "$runLog.1" -Force +} + +$rc = 0 +foreach ($p in $Provider) { + $out = [IO.Path]::GetTempFileName() + try { + # cmd.exe does the redirect, so stdout and stderr stay interleaved and + # PowerShell's native-command stderr handling never sees either. /s strips + # exactly the outer pair of quotes, leaving the quoted paths inside intact. + $psi = New-Object System.Diagnostics.ProcessStartInfo + $psi.FileName = if ($env:ComSpec) { $env:ComSpec } else { 'cmd.exe' } + $psi.Arguments = "/d /s /c `"`"$launcher`" sync --provider $p --joplin-optional > `"$out`" 2>&1`"" + $psi.WorkingDirectory = $repo + $psi.UseShellExecute = $false + $proc = [Diagnostics.Process]::Start($psi) + $proc.WaitForExit() + $status = $proc.ExitCode + + $text = [IO.File]::ReadAllText($out, [Text.Encoding]::UTF8) + Write-Host $text + Add-Content -LiteralPath $runLog -Encoding UTF8 -Value ( + "===== $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss') sync --provider $p (exit $status) =====`r`n$text") + + if ($status -ne 0) { + $rc = 1 + if (-not $text.Contains('Sync completed with failures')) { + $crash = [regex]::Matches($text, '(?m)^[A-Za-z_][A-Za-z0-9_.]*(Error|Exception)\b') | + Select-Object -Last 1 + if ($crash) { + $reason = "crashed ($($crash.Value))" + } else { + $reason = "exited $status before reporting a result" + } + Send-FailurePush "${p}: $reason`ncache\logs\scheduled-sync.log" + } + } + } catch { + # This script failing is a run that reports nothing, too. + $rc = 1 + Write-Warning "run-sync: $p - $($_.Exception.Message)" + Send-FailurePush "${p}: run-sync.ps1 failed ($($_.Exception.GetType().Name))" + } finally { + Remove-Item -LiteralPath $out -Force -ErrorAction SilentlyContinue + } +} +exit $rc