feat(cot): Bug 2034520 improve json download and parsing error handling#791
Open
feat(cot): Bug 2034520 improve json download and parsing error handling#791
Conversation
load_json_or_yaml's default exception message now names the file when
is_path=True ("Failed to load json from /path/to/x.json: ..." instead
of just "Failed to load json: ..."). load_json_or_yaml_from_url passes
a custom message that also includes the (loggable) source URL alongside
the cache path.
Motivation: a chain-of-trust verification failure surfaced only the
JSONDecodeError offset, forcing a manual trace through several
concurrent downloads to identify the bad file. The extra context pins
the culprit immediately.
…lure Combine download and parse into a single retry unit in load_json_or_yaml_from_url: if the downloaded file fails to parse as JSON/YAML, the cache is invalidated and retry_async triggers a fresh download (up to the default 5 attempts). This turns transient artifact- server glitches (e.g. GCS gunzip-transcoding returning 'Bad Request' with 200 OK, HTML error pages from hg, truncated responses) from hard failures into self-healing retries. download_file gains an optional expected_content_type kwarg that rejects 'text/html' responses when the caller asked for something else, failing fast with a clearer error before writing a byte. DownloadError is reused so retry_async picks it up if the HTML turns out to be transient. The misleading 'overwrite' parameter semantics are preserved (overwrite=True keeps any existing cached file; overwrite=False always redownloads) and flagged with a comment for future cleanup. Tests added for HTML rejection and the parse-failure-then-retry-succeeds flow.
331b32b to
c5ca4ce
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On
load_json_or_yaml:On
load_json_or_yaml_from_url: