fix: surface the response body on 4xx so 403s are diagnosable
Media downloads logged "HTTP Error 403:" with no reason. That string is
curl_cffi's raise_for_status() format, "HTTP Error {code}: {reason}", and
HTTP/2 carries no reason phrase — so the message said nothing, and
_make_request threw the response body away. The provider's JSON `detail`
is the only explanation available for a refused asset.
- base._make_request: end non-retryable statuses with a ProviderError
carrying the body's detail/error/message (redacted, truncated to 300
chars) instead of a bare raise_for_status().
- media: bucket 403 as `forbidden` in the run summary, separately from
`download-error` — "the asset is gone" and "we were refused" are
different problems.
- utils.redact_secrets: match secret key names per word. Exact matching
let access_token, api_key, and session-token through into logged
bodies; "keywords"/"monkey"/"tokenizer" stay intact.
- tests/test_config.py: test_defaults depended on the absence of a local
.env — load_config() calls load_dotenv(override=False), which restored
the variable the test had just deleted. Stub dotenv discovery.
305 tests pass.
This commit is contained in:
+19
-3
@@ -76,11 +76,27 @@ def build_export_path(
|
||||
return base_dir.joinpath(*parts) / filename
|
||||
|
||||
|
||||
def _is_sensitive_key(key: object) -> bool:
|
||||
"""True if a mapping key names a secret.
|
||||
|
||||
Matches the whole key and each of its underscore/dash-separated words, so
|
||||
compound names carry too: exact-match alone let ``access_token`` and
|
||||
``api_key`` through into logged response bodies. Word-level matching keeps
|
||||
innocent keys ("keywords", "monkey") intact.
|
||||
"""
|
||||
if not isinstance(key, str):
|
||||
return False
|
||||
lowered = key.lower()
|
||||
if lowered in _SENSITIVE_KEYS:
|
||||
return True
|
||||
return any(part in _SENSITIVE_KEYS for part in re.split(r"[^a-z0-9]+", lowered))
|
||||
|
||||
|
||||
def redact_secrets(data: object) -> object:
|
||||
"""Recursively redact sensitive values from a dict/list for safe logging.
|
||||
|
||||
Keys matching _SENSITIVE_KEYS (case-insensitive) have their values
|
||||
replaced with "[REDACTED]".
|
||||
Keys naming a secret (see _is_sensitive_key) have their values replaced
|
||||
with "[REDACTED]".
|
||||
|
||||
Args:
|
||||
data: Any JSON-serializable object.
|
||||
@@ -90,7 +106,7 @@ def redact_secrets(data: object) -> object:
|
||||
"""
|
||||
if isinstance(data, dict):
|
||||
return {
|
||||
k: "[REDACTED]" if k.lower() in _SENSITIVE_KEYS else redact_secrets(v)
|
||||
k: "[REDACTED]" if _is_sensitive_key(k) else redact_secrets(v)
|
||||
for k, v in data.items()
|
||||
}
|
||||
if isinstance(data, list):
|
||||
|
||||
Reference in New Issue
Block a user