feat(telemetry): observability spans + attrs (validation, DB-write, body-size, first-stream-event, Sentry) - #805
Open
sjawhar wants to merge 2 commits into
Conversation
…ody-size, first-stream-event, Sentry) Adds OpenTelemetry spans and span attributes across the proxy's hot paths so operators can see validation, DB-write, and streaming behaviour in traces. All attributes are low-cardinality (sizes, counts, durations, status, booleans) — no request/response content, credentials, or high-cardinality ids are recorded. - credential_manager: span around credential validation. - observability/emitter + request_log/recorder: DB-write spans with duration_ms and error status on failure. - request_log/recorder: request/response body-size attributes and a body_truncated flag. - pipeline/anthropic_processor: time-to-first-streamed-event attribute. - observability/sentry: opt-in Sentry init (no-op when DSN unset), with tests. Tests cover each new span/attribute and the disabled/no-op paths.
legion-implementer
Bot
force-pushed
the
feat/telemetry-observability
branch
from
July 8, 2026 22:55
266add0 to
b2a80f2
Compare
…stdout event dump - request_log.write span now record_exception on DatabaseWriteError (previously set ERROR status with no exception -> error=null in Datadog). - Instrument asyncpg + psycopg (instrument_db, wired at startup) so slow DB queries surface as spans. API-call latency was already traced via anthropic.complete/anthropic.stream. - Gate the observability emitter's full-payload stdout dump behind a new OBSERVABILITY_STDOUT_ENABLED setting (default true; set false in prod). The raw print() of full request/response payloads to stdout drove ~$450/mo of CloudWatch ingestion (/ecs/luthien-proxy ~910 GB/mo); events still persist to the DB and event publisher, so no observability is lost.
legion-implementer
Bot
force-pushed
the
feat/telemetry-observability
branch
from
July 19, 2026 16:22
d496638 to
36a12b5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds OpenTelemetry spans and span attributes across the proxy's hot paths so operators can see validation, DB-write, and streaming behaviour in traces.
All attributes are low-cardinality (sizes, counts, durations, status, booleans). No request/response content, credentials, or high-cardinality ids are recorded.
Changes
duration_msand error status on failure.body_truncatedflag.Testing
Unit tests cover each new span/attribute plus the disabled/no-op paths (emitter, recorder, anthropic_processor, credential_manager, sentry init).
pytestgreen locally.Notes
Content-free by construction — every span attribute is a size/count/duration/status/boolean. Safe to enable in any OTLP backend.