Skip to content

[opentelemetry-util-genai] gen_ai.tool.definitions keeps description and parameters when content capture is off #655

Description

@sfc-gh-zeningchen

Describe your environment

OS: Ubuntu
Python version: Python 3.12.14
Package version: opentelemetry-util-genai 1.2b0.dev (main, 3e6a57d)
GenAI library (e.g. anthropic, openai) and version: n/a, reproduces through the util alone

What happened?

get_content_attributes in util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py
takes an early return when content capture is disabled. Its comment says the intent is to emit the
attribute while leaving the optional properties out:

# Tool definitions are always captured, the sem conv recommends adding params / description only
# when the content capture mode is set..
if mode not in allowed_modes:
    return (
        {GenAI.GEN_AI_TOOL_DEFINITIONS: serialize(tool_definitions)}
        if tool_definitions
        else {}
    )

serialize() is the same full asdict() serialization the capture-enabled path uses, so
description and parameters go out either way. The code does not do what its comment says.

model/gen-ai/gen-ai-tool-definitions.json in semantic-conventions-genai requires only type and
name. Both the schema and the gen_ai.tool.definitions entry in model/gen-ai/registry.yaml say:

Since this attribute could be large, it's NOT RECOMMENDED to populate non-required properties by
default. Instrumentations MAY provide a way to enable populating optional properties.

parameters holds the tool's full JSON Schema, so it is the large part of the payload, and the
registry entry also carries a "may contain sensitive information" warning. Someone who disables
content capture still receives it.

Steps to Reproduce

import os

os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "NO_CONTENT"

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
    InMemorySpanExporter,
)
from opentelemetry.util.genai.handler import TelemetryHandler
from opentelemetry.util.genai.types import FunctionToolDefinition

exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))

handler = TelemetryHandler(tracer_provider=provider)
invocation = handler.inference(provider="anthropic", request_model="a-model")
invocation.tool_definitions = [
    FunctionToolDefinition(
        name="get_weather",
        description="Get weather by city",
        parameters={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    )
]
invocation.stop()

span = exporter.get_finished_spans()[0]
print(span.attributes["gen_ai.tool.definitions"])

Expected Result

Only the properties the schema requires:

[{"name":"get_weather","type":"function"}]

Actual Result

[{"name":"get_weather","description":"Get weather by city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]},"type":"function"}]

Additional context

There is a second question behind this one, and answering it first may change the fix.

model/gen-ai/spans.yaml marks gen_ai.tool.definitions requirement_level: opt_in and puts it in
the attributes.gen_ai.content group next to gen_ai.input.messages, gen_ai.output.messages and
gen_ai.system_instructions. That placement reads as "the attribute is content and follows the
content-capture setting", which is the opposite of "always captured".

The instrumentations in this repo are split on it. genai-openai gates tool_definitions behind
capture_content in both utils.py (Chat Completions) and response_extractors.py (Responses API).
genai-langchain, genai-agno, genai-portkey and genai-smolagents set it regardless. The same
call therefore yields a different attribute depending on which provider you use.

Suggested order:

  1. Decide whether the attribute follows the content-capture setting, given opt_in and its
    membership in the content attribute group.
  2. Either way, make the capture-disabled path serialize only type and name rather than the whole
    object, so the code matches its comment and the semconv guidance.
  3. Align the instrumentations on the outcome.

References:

Would you like to implement a fix?

No

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions