Skip to content

feat(instrumentation): defer input value resolution - #3716

Open
caroger wants to merge 1 commit into
mainfrom
codex/deferred-input-value
Open

caroger wants to merge 1 commit into
mainfrom
codex/deferred-input-value

Conversation

@caroger

@caroger caroger commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Why

This PR does not upload images from input.value. It adds a recording gate so a later provider callback can do that work only after the sampler keeps the span.

Provider instrumentors serialize input.value as a JSON copy of the request. That copy is a different attribute from the flattened llm.input_messages.*.image.url keys that #3409 already made recording-safe.

TraceConfig.mask() never walks JSON, so calling BlobUploader while building input.value would upload before the sampler decides whether the span is kept. This PR adds OITracer.start_span(..., input_value=callback) so providers finish input.value only after the span is recording.

TraceConfig.externalize_blob() is the shared upload-or-redact helper for already-decoded bytes. The existing data-URI mask path now uses it too.

#3717 is the first consumer. It passes input_value=lambda: serialize_request_input(...). That JSON walk is not in this PR.

The two copies of the same image

LLM request with a base64 image
              |
              +-- flattened keys  (already recording-safe since #3409)
              |     llm.input_messages.0.message.contents.0
              |         .message_content.image.image.url
              |     OpenInferenceSpan.set_attribute()
              |       -> TraceConfig.mask()
              |       -> BlobUploader only if the span is recording
              |
              +-- input.value  (JSON string of the whole request)
                    mask() does not parse this string
                    providers currently build it before start_span()

This PR only opens a safe seam for the input.value copy. It does not walk provider JSON and it does not upload from input.value by itself. Follow-up PRs (OpenAI #3717, then Anthropic and Google GenAI) pass the callback.

Before

Naive upload inside the provider serializer, which is the pattern this seam exists to stop. On main today OpenAI redacts oversized images in that serializer and does not upload. There is still no recording-safe place to upload from input.value.

provider walks the request
  decode image bytes
  BlobUploader.upload()          <--- TOO EARLY
  serialize JSON
        |
        v
OITracer.start_span(
    attributes={"input.value": "<json or uri>"}
)
        |
        |  mask_without_externalization()
        v
OTel sampler.should_sample()
        |
        +-- DROP / suppressed ---- span discarded
        |                          blob already uploaded
        |
        +-- RECORD --------------- span exported
                                   accidental success

Flattened image keys still upload later at finish_tracing when the span is recording.

After

OITracer.start_span(..., input_value=callback)
        |
        |  Phase 1 (always, no callback)
        |    attributes["input.value"] = "__REDACTED__"
        |    mask_without_externalization()
        v
OTel sampler.should_sample()
        |
        +-- suppressed, DROP, ended, NoOp, or hide_inputs=True
        |     callback is never called
        |     zero uploads
        |     span keeps "__REDACTED__" or is not exported
        |
        +-- is_recording() and hide_inputs is false
              Phase 2 (exactly once)
                result = callback()
                span.set_attribute("input.value", result)
                if callback raises, leave "__REDACTED__"

The callback is where a provider may walk JSON, honor hide_input_images, and call TraceConfig.externalize_blob(). Core does not inspect image leaves. hide_input_images is a provider-callback concern. Core only skips the callback when hide_inputs is true or the span is not recording.

externalize_blob()

TraceConfig.externalize_blob(data, mime_type, attribute_key)
        |
        +-- no uploader -------------------- "__REDACTED__"
        +-- uploader.upload() raises ------- "__REDACTED__"
        +-- missing, relative, or invalid URI
        |                                    "__REDACTED__"
        +-- absolute URI (https://, s3://, ...)
                                             return URI

Blob.modality is "image". Failures never raise into the instrumented SDK call.

Scope

  • Add recording-safe input_value callbacks to OITracer.start_span() and OITracer.start_as_current_span().
  • Add TraceConfig.externalize_blob() and reuse it for existing data URI masking.
  • Cover sampled, dropped, suppressed, hidden, invalid, ended, and custom-mask paths.

Tradeoffs

The sampler receives __REDACTED__ instead of serialized provider input. This avoids provider traversal and upload work before the recording decision.

The callback is specific to input.value. This keeps ordinary OpenTelemetry attribute values and masking contracts unchanged.

Blast radius

The new callback and upload helper are opt-in. Existing spans and flattened image attributes keep their current behavior.

Verification

  • uvx tox run -e ruff-mypy-test-instrumentation passed formatting, Ruff, mypy, and 8,524 tests.

Closes #3537

@github-actions

Copy link
Copy Markdown
Contributor

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[python] Add recording-safe BlobUploader support for input.value sanitizers

1 participant