fix(tags): report server 4xx rejections instead of masking them#132
fix(tags): report server 4xx rejections instead of masking them#132ryanhayame wants to merge 5 commits into
Conversation
The backend (server-private, staging-v2) now rejects with HTTP 400 any
create / tag-update payload whose tags array carries more than one
group:* tag ({"error":"A run can have at most one group:* tag."}).
The SDK handled the rejection badly: the write client _try retried the
400 with backoff, then either silently no-op'd (tags/update) or raised a
misleading ConnectionError("...Check connection...") (create). The real
reason was only ever in a buried warning log.
This makes the SDK surface server validation errors loudly and honestly,
without second-guessing or rewriting the caller's tags:
- iface.py: add PlutoRequestError + _server_error_message(). _try now
treats 4xx as terminal (no retry, no wasted backoff) and, when
raise_on_error=True, raises PlutoRequestError carrying the server's
reason (parsed from the JSON "error" field). Only 5xx and
network/timeout errors are retried — mirroring query.py.
- op.py: create/resume pass raise_on_error=True. A server rejection now
raises RuntimeError("Failed to create run: <server reason>"); a genuine
unreachable server still raises the ConnectionError. add_tags/
remove_tags surface a rejection at WARNING instead of burying it at
DEBUG.
- sync/process.py: the sync-subprocess uploader (_post_with_retry) — the
primary path for run.add_tags(), which sends the full tags array — also
treats 4xx as terminal and raises PlutoRequestError with the server's
reason, so the sync loop logs the actual message rather than a bare
"400 Bad Request".
No client-side tag rewriting: if a caller sends two group:* tags, the
request fails and the server's reason is reported, rather than the SDK
quietly dropping one.
Tests (tests/test_iface_errors.py): _try does not retry a 400 and raises
PlutoRequestError with the server message (not ConnectionError), retries
5xx, and returns None on network errors; the sync uploader does not retry
a 400 and surfaces the reason, and still retries 5xx.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces PlutoRequestError and a helper function _server_error_message to capture and surface human-readable server validation errors (such as 4xx responses) instead of raising generic connection errors. It updates the ServerInterface and the sync uploader to immediately halt retries on 4xx client errors, while adding comprehensive unit tests to verify this behavior. The reviewer suggests an improvement to extract and propagate the actual HTTP status code from the error message when raising PlutoRequestError on persistent server errors (e.g., 5xx errors) after exhausting retries, along with a corresponding test assertion to verify this behavior.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 63bc5a2. Configure here.
…n path Addresses PR review: on the persistent-5xx exhaustion path _try raised PlutoRequestError with status_code=None, inconsistent with the 4xx path which sets the real code. Thread the last HTTP status through the retry recursion (last_status) so the exception reports the actual code (e.g. 500) instead of None. Prefer threading over parsing it back out of the formatted error_info string, which would couple the value to the log format. Assert the code is populated in the 5xx test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…atus_code The previous rewrite of _post_with_retry read response.status_code directly, which broke ~11 existing sync-uploader tests: they mock the response with a no-op raise_for_status() and never set status_code, so `MagicMock() < 400` raised TypeError. Restore the original contract — call raise_for_status(); success if it doesn't raise — and branch on the httpx.HTTPStatusError it raises: 4xx is terminal and raises PlutoRequestError with the server's reason, 5xx and network/timeout errors stay retryable. Same external behavior as intended, compatible with the existing mocks. Update the two new sync tests to attach a request to their canned httpx.Response objects (raise_for_status needs one to build its error). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Making *all* 4xx terminal regressed resilience against transient auth
failures. In CI, the auth/token-validation step intermittently times out
and the create endpoint returns a transient 401; previously _try retried
all non-2xx up to 5x so it recovered, but the "4xx is terminal" change
hard-failed on the first 401 (RuntimeError: Failed to create run:
Unauthorized).
Introduce RETRYABLE_STATUS_CODES = {401, 408, 429} — auth races, server
timeouts, rate limits — and treat only the *other* 4xx (400, 403, 404,
409, 422, ...) as terminal. Genuine client/validation errors like 400
"A run can have at most one group:* tag." still fail fast with the
server's reason; transient auth/timeout/rate-limit 4xx retry like 5xx and,
if they never clear, still raise PlutoRequestError with the real code
after exhausting retries. Applied to both the write client (_try) and the
sync uploader (_post_with_retry).
Tests: transient 401 that clears on retry succeeds; persistent 401 is
retried then raises with status_code 401; 400 stays terminal/no-retry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a user logs a list of media at one step —
pluto.log({"eval/img": [img0, img1, img2]}, step=s) — the server sorted a
step's files by fileName, losing the logged order. Send an explicit
0-based sampleIndex per file so the server can restore it.
Contract (matches the server): each object in the POST /files `files: [...]`
array gets a non-negative integer `sampleIndex` = the file's position within
a single (logName, step) log() call. A list of N media → 0..N-1; a scalar
(single, non-list) media → 0. Missing sampleIndex is treated as 0 server-side
(older SDKs keep working), so no version gate is needed.
Transparent to users — no new public API. The index is derived purely from
list position via enumerate(), captured at log() time (op.py, where order is
known) and threaded through the sync DB to the upload payload:
- op.py: enumerate items in the log loop → _process_log_item_sync →
_enqueue_file_sync → SyncProcessManager.enqueue_file.
- sync/store.py: new `sample_index` column (additive migration, defaults 0),
persisted on enqueue and read back into FileRecord.
- sync/process.py: `_get_presigned_urls` adds `sampleIndex` to each entry.
- api.py: make_compat_file_v1 (legacy builder) enumerates each logName list.
Deriving from stored position (not from the batch) is deliberate — sync
batches can split or mix steps, so numbering the batch would be wrong.
Tests: store round-trip (list → 0..N-1, default 0); sync presign payload
carries sampleIndex; make_compat_file_v1 numbers a list 0..N-1 and a single
media 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This PR does two independent, forward-compatible things: (1) makes the SDK report server errors honestly instead of masking/retrying them, and (2) sends a
sampleIndexso lists of media keep their logged order in the UI.Both are safe to merge before any server/DB change — see "Merge safety" at the bottom.
Part 1 — report server 4xx rejections instead of masking them
Problem. On a non-2xx write,
iface._tryretried everything (incl. 4xx) with backoff, then either silently no-op'd (/api/runs/tags/update— a failedrun.add_tags(...)looked like it worked) or raised a misleadingConnectionError("...Check connection...")(/api/runs/create— blamed the network). The server's real reason lived only in a buried warning log. This first surfaced with the server's new "a run can have at most onegroup:*tag" 400.Change. Surface the server's reason; don't rewrite the caller's request.
iface.py: newPlutoRequestError(+_server_error_message, parses the JSONerrorfield)._trytreats permanent 4xx as terminal (no retry, no wasted backoff) and, whenraise_on_error=True, raisesPlutoRequestErrorwith the server's message. Transient 4xx —401/408/429(RETRYABLE_STATUS_CODES) — and 5xx/network stay retryable (auth races, timeouts, rate limits), mirroringquery.py.op.py: create/resume passraise_on_error=True→ a rejection raisesRuntimeError("Failed to create run: <server reason>"); a genuine unreachable server still raisesConnectionError.add_tags/remove_tagssurface a rejection at WARNING instead of DEBUG.sync/process.py: the sync-subprocess uploader (_post_with_retry, the primaryrun.add_tagspath) usesraise_for_status()and applies the same terminal-4xx / retryable-401/408/429policy, so the sync loop logs the real reason instead of a bare400.No client-side tag rewriting — if a caller sends two
group:*tags, the request fails and the server's reason is reported (we do not silently drop one).Part 2 —
sampleIndexso media lists keep logged orderProblem.
pluto.log({"eval/img": [img0, img1, img2]}, step=s)came back out of order in the UI because the server sorts a step's files byfileNameand nothing recorded each image's position.Change (SDK side). Each object in the
POST /filesfiles: [...]array now carries a non-negative, 0-based integersampleIndex= the file's position within a single(logName, step)log()call.0..N-1; a single (non-list) media →0.enumerate(), captured atlog()time (op.py, where order is known) and threaded through the sync DB to the payload:op.py:enumerateitems in the log loop →_process_log_item_sync→_enqueue_file_sync→SyncProcessManager.enqueue_filesync/store.py: newsample_indexcolumn (additive migration, defaults0), persisted on enqueue and read back intoFileRecordsync/process.py:_get_presigned_urlsaddssampleIndexto each entryapi.py:make_compat_file_v1(legacy builder) enumerates each logName's listTests
tests/test_iface_errors.py:_trydoesn't retry a 400 and raisesPlutoRequestErrorwith the server message (notConnectionError); persistent 5xx/401 retry then raise with the realstatus_code; transient 401 that clears on retry succeeds; network error returnsNone; the sync uploader doesn't retry a 400 and surfaces the reason.tests/test_sync_process.py: store round-trip (sample_indexlist →0..N-1, default0); presign payload carriessampleIndex;make_compat_file_v1numbers a list0..N-1and a single media0.Ruff + mypy clean on changed files.
Merge safety (verified against
server-private)Safe to merge before the backend/DB changes:
group:*rule, there's simply no 400 to surface; behavior is unchanged.ingest/src/models/files.rs,FileInput) has no#[serde(deny_unknown_fields)], so serde silently ignores the unknownsampleIndex— same pattern as the already-shippedcaptionfield. Until the backend addssample_index(column + sort), media just keeps its currentfileNamesort; after, ordering works. No coordinated deploy required.🤖 Generated with Claude Code