Conversation
sjawhar
marked this pull request as draft
August 29, 2026 22:48
…N logs Three symptom groups of the same shape: a client or network condition the pipeline already handles correctly still surfaces as a Sentry error. - ClientDisconnect (starlette.requests): raised while reading the client request body after the client has already left. Completely unhandled anywhere in this codebase — there's no one to send a response to, and no proxying behavior depends on it. Dropped in _sentry_before_send by (module, type) so an unrelated same-named exception isn't swallowed by accident. Silences LUTHIEN-3/4/E. - Mid-stream httpx.TransportError (RemoteProtocolError, ReadTimeout, ReadError, ...): raised directly by httpx, not wrapped in AnthropicConnectionError, when the upstream connection drops while _build_error_event's caller is mid-read of a streaming response. Added as a new branch logging at warning, matching the existing AnthropicConnectionError treatment right above it. Silences LUTHIEN-A/B/G. - Malformed JSON in an incoming request: the client's fault, already returned to the client as an HTTP 400 by the surrounding code. Downgraded logger.error to logger.warning at the one call site. Silences LUTHIEN-7/9. Genuine proxy bugs are unaffected: the generic except-Exception branch in _build_error_event still logs at error for anything that isn't one of the above, and a new test pins that. No behavior change to proxying itself — response codes, retry behavior, and client-visible errors are unchanged; only what Sentry captures changes.
…ientDisconnect _sentry_before_send dropped the whole event if ANY exception entry matched ClientDisconnect, so a mixed ExceptionGroup(ClientDisconnect, RuntimeError) would be swallowed even though the RuntimeError is a genuine proxy-side failure riding alongside the disconnect. Require every non-group-wrapper (leaf) exception entry to match _DROPPED_EXCEPTION_TYPES instead, treating ExceptionGroup/BaseExceptionGroup wrapper entries as structural. Addresses thermonuclear-deep-review Medium finding on PR LuthienResearch#814.
legion-implementer Bot
pushed a commit
to trajectory-labs-pbc/luthien-proxy
that referenced
this pull request
Aug 29, 2026
The httpx.TransportError warning-downgrade lived in _build_error_event, which also receives policy-origin exceptions via _handle_execution_streaming's broad except around the emissions iterator, so a policy's own httpx.TransportError got misclassified as an upstream network flap; the non-streaming path (_handle_anthropic_error) didn't classify TransportError at all and it surfaced as an unclassified 500. Move the classification to the upstream client boundary: AnthropicClient.complete/stream now wrap a genuine httpx.TransportError from the actual Anthropic call into a new AnthropicUpstreamTransportError. _build_error_event (streaming) and _handle_anthropic_error (non-streaming) both downgrade that type to a warning-level 502/api_connection_error, matching the existing AnthropicConnectionError handling. A raw httpx.TransportError from anywhere else (e.g. a policy's own outbound call) is not this type and stays on the error-level internal path. Addresses thermonuclear-deep-review High-quality finding on PR LuthienResearch#814.
legion-implementer
Bot
force-pushed
the
fix/sentry-expected-client-and-network-noise
branch
from
August 29, 2026 23:32
3227f60 to
d4ce431
Compare
The httpx.TransportError warning-downgrade lived in _build_error_event, which also receives policy-origin exceptions via _handle_execution_streaming's broad except around the emissions iterator, so a policy's own httpx.TransportError got misclassified as an upstream network flap; the non-streaming path (_handle_anthropic_error) didn't classify TransportError at all and it surfaced as an unclassified 500. Move the classification to the upstream client boundary: AnthropicClient.complete/stream now wrap a genuine httpx.TransportError from the actual Anthropic call into a new AnthropicUpstreamTransportError. _build_error_event (streaming) and _handle_anthropic_error (non-streaming) both downgrade that type to a warning-level 502/api_connection_error, matching the existing AnthropicConnectionError handling. A raw httpx.TransportError from anywhere else (e.g. a policy's own outbound call) is not this type and stays on the error-level internal path. Addresses thermonuclear-deep-review High-quality finding on PR LuthienResearch#814.
…nal_status The mid-stream exception handler in _handle_execution_streaming only special-cased AnthropicStatusError and AnthropicConnectionError before falling through to `else: final_status = 500`. AnthropicUpstreamTransportError (AnthropicClient's own wrapper around a real upstream network failure, added earlier in this PR) fell into that generic branch, so a mid- stream Anthropic network outage recorded final_status=500 in the completion webhook and request-log rows — misclassifying an upstream outage as a proxy bug, even though the emitted SSE error event already correctly classified it as api_connection_error. Fold AnthropicUpstreamTransportError into the same branch as AnthropicConnectionError (503), matching how both are already treated identically in _build_error_event and _handle_anthropic_error.
legion-implementer
Bot
force-pushed
the
fix/sentry-expected-client-and-network-noise
branch
from
August 30, 2026 00:26
d4ce431 to
ddd3bbc
Compare
The 4 code comments that attributed these regression tests to an internal review tool name are rewritten to describe the finding generically. No behavior or test-logic change. red: grep -rn "thermonuclear-deep-review" tests/luthien_proxy/unit_tests/llm/test_anthropic_client.py tests/luthien_proxy/unit_tests/pipeline/test_anthropic_processor.py tests/luthien_proxy/unit_tests/test_sentry_scrubbing.py -> 4 hits (test_anthropic_client.py:441, test_anthropic_processor.py:879+1169, test_sentry_scrubbing.py:202) green: same grep -> 0 hits Omp-Session: 01a081d4-2514-7000-b8e2-ff8548776146
…iProvider DirectApiProvider.complete() calls AnthropicClient.complete(), which this PR's earlier commit changed to raise AnthropicUpstreamTransportError (a plain, gateway-owned Exception subclass — not an anthropic.APIConnectionError) for a raw httpx.TransportError from the actual upstream call. DirectApiProvider's catch chain only translated anthropic.APIConnectionError into InferenceProviderError, so the new type escaped the documented provider error boundary as an unclassified exception instead of InferenceProviderError, same as every other backend-connection failure this provider handles. Adds the missing except branch, mirroring the existing APIConnectionError one. red: uv run pytest tests/luthien_proxy/unit_tests/inference/test_direct_api.py::TestErrorTranslation::test_upstream_transport_error_becomes_provider_error -> AnthropicUpstreamTransportError escapes uncaught (1 failed) green: same command -> 1 passed; full test_direct_api.py -> 29 passed Omp-Session: 01a081d4-2514-7000-b8e2-ff8548776146
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With Sentry enabled, three more categories of expected client/network conditions still surface as error-level events, on top of the two already-open fixes (#807 OTel logger noise, #809 upstream provider status codes):
ClientDisconnect(starlette.requests) — raised while reading the client's request body after the client has already disconnected. Unhandled anywhere in this codebase; the client is gone, there's nothing to respond to. Three separate Sentry issues (LUTHIEN-3/4/E) for the three call sites that read the body (/v1/messages,/v1/{path},/openai/{path}).httpx.RemoteProtocolError,ReadTimeout,ReadErrorraised directly by httpx (not wrapped inAnthropicConnectionError) when the upstream connection drops while we're reading a streaming response. These fell through_build_error_event's genericelsebranch, which logs aterror— that branch is meant for genuine proxy bugs, but it also caught this backend-network condition (LUTHIEN-A/B/G).logger.errorcall at that site was the only reason it shows up in Sentry (LUTHIEN-7/9).None of these are proxy defects. Combined they're several dozen events across six issues, not huge individually, but stacking with the other two fixes to help clear the org's exhausted error quota.
Fix
_sentry_before_send: new_DROPPED_EXCEPTION_TYPESset matched by(module, type)against the built event's exception list (nothint["exc_info"], which for a client-disconnect is the outerExceptionGroupanyio's TaskGroup wraps it in — the SDK has already flattened the group's members intoevent["exception"]["values"]by the timebefore_sendruns). Currently just("starlette.requests", "ClientDisconnect")._build_error_event: newelif isinstance(e, AnthropicUpstreamTransportError)branch, sitting next to the existingAnthropicConnectionErrorbranch it mirrors — sameerror_type/message,logger.warninginstead of the generic branch'slogger.error. (AnthropicClientwraps the rawhttpx.TransportErrorintoAnthropicUpstreamTransportErrorbefore it reaches this function — see the client-boundary bullet below — so this branch matches the wrapper type, not the raw httpx exception.)_process_request: theMalformed JSON in Anthropic requestlog call downgraded fromlogger.errortologger.warning.AnthropicClient.complete()/.stream(): rawhttpx.TransportErrorraised during the actual upstream call is now wrapped in a newAnthropicUpstreamTransportErrortype at the client boundary, so the pipeline can tell a genuine upstream network flap apart from a rawhttpx.TransportErrorraised elsewhere (e.g. a policy's own outbound HTTP call), which must stay on the generic error-level path._handle_execution_streaming's mid-stream exception handler):AnthropicUpstreamTransportErrorjoins the existingAnthropicConnectionErrorbranch that sets the internal completionfinal_statusto503(previously fell to the genericelse: final_status = 500) — this feeds the completion webhook'shttp_statusfield and the request-log recorder's inbound/outbound status, not an HTTP status returned to the client (headers are already flushed for an in-progress stream; the client only sees the mid-stream SSE error event from_build_error_eventabove)._handle_anthropic_errorclassifiesAnthropicUpstreamTransportErrorinto a 502BackendAPIError— previously this type wasn't classified at all in the non-streaming path and propagated as an unclassified 500. This one is the literal HTTP status returned to the client, since no response has been sent yet at this point.DirectApiProvider.complete(): newexcept AnthropicUpstreamTransportErrorbranch translating it intoInferenceProviderError, mirroring the existinganthropic.APIConnectionErrorbranch — the new type isn't ananthropic.APIConnectionErrorsubclass, so without this it escaped the provider's error boundary unclassified instead of following the same connection-error path every other backend failure in this provider takes.Genuine proxy bugs are unaffected — the generic
except Exception/elsebranches still log aterror, pinned bytest_builds_generic_error_event_logs_at_errorandtest_builds_generic_error_event_for_policy_origin_transport_error.No behavior change to proxying itself for a successful response, and no change to what bytes reach the client mid-stream: the SSE error event's JSON shape is identical, only its
error.type/message and log level change. Two internal status values used for accounting (not returned to the client) do change for this exception class: the non-streaming path's unclassified 500 becomes the correct 502, and the streaming completion webhook/request-log's unclassified 500 becomes 503. This only changes what Sentry captures plus that internal accounting.Verification
uv run pytest tests/luthien_proxy/unit_tests/test_sentry_scrubbing.py tests/luthien_proxy/unit_tests/pipeline/test_anthropic_processor.py tests/luthien_proxy/unit_tests/llm/test_anthropic_client.py— 164 passed (new tests:test_drops_client_disconnect,test_drops_client_disconnect_flattened_from_exception_group,test_keeps_same_named_exception_from_different_module,test_keeps_mixed_exception_group_with_non_client_disconnect_member,test_builds_transport_error_event_and_logs_at_warning,test_builds_generic_error_event_logs_at_error,test_builds_generic_error_event_for_policy_origin_transport_error,test_upstream_transport_error_raises_backend_api_error, plus theTestAnthropicClientTransportErrorWrappingclass intest_anthropic_client.pyand a caplog assertion added totest_malformed_json_returns_400).uv run pytest tests/luthien_proxy/unit_tests/inference/test_direct_api.py— 29 passed (new:test_upstream_transport_error_becomes_provider_error).ruff format --check/ruff checkclean on every touched file.pyright: 0 errors, 0 warnings, 0 informations on the touched source files (pyrightconfig'sincludeissrc/saas_infraonly; the touched test files are outside pyright's scope in this repo).Issues silenced
LUTHIEN-3, LUTHIEN-4, LUTHIEN-E (ClientDisconnect), LUTHIEN-A, LUTHIEN-B, LUTHIEN-G (mid-stream transport errors), LUTHIEN-7, LUTHIEN-9 (malformed JSON).
Related
Part of the same Sentry-quota bug sweep as #807 (OTel logger noise → broadened to
opentelemetry.*wildcard in a follow-up commit on that branch, also silences LUTHIEN-C/F) and #809 (upstream provider status codes → broadened to include 400/404/401 in a follow-up commit on that branch, also silences LUTHIEN-6/2/D).