Skip to content

Add optional OpenTelemetry tracing for agent runs - #1

Closed
cyberd4ch wants to merge 1 commit into
mainfrom
feat/otel-tracing
Closed

cyberd4ch wants to merge 1 commit into
mainfrom
feat/otel-tracing

Conversation

@cyberd4ch

@cyberd4ch cyberd4ch commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

What this adds

Observability is the missing half of an agent harness: the audit log records what happened, but nothing emits it as a trace, so an agent run is invisible in Grafana/Jaeger/Tempo.

  • One span per model run: invoke_agent txn-agent, with gen_ai.operation.name, gen_ai.agent.name and gen_ai.request.model.
  • Children per turn and per tool: chat <model> spans carry gen_ai.usage.input_tokens / output_tokens; execute_tool <name> spans carry gen_ai.tool.name and an error attribute when the router fails soft.
  • The money path: txn_agent.purchase and txn_agent.checkout_cart carry the gate decision, its reasons, cart total, outcome and the idempotency key; txn_agent.approval records the verdict.
  • Optional dependency: extra txn-agent[otel] (opentelemetry-api). Nothing is imported at module load, so the stdlib-only core invariant holds and an unconfigured provider means every helper is a no-op.
  • Privacy by default: tool arguments and prompt text are never recorded as attributes; only short, bounded facts are (decisions, reasons, token counts, a hashed key).

Why the attribute names live in one file

txn_agent/telemetry.py centralises every span and attribute name. The GenAI semantic conventions are still moving, so when they rename something the fix is one file, not a search across the package.

Verification

  • ruff check . clean
  • mypy txn_agent — strict, no issues (20 files)
  • pytest — 74 passed, 1 skipped, coverage 87.36% against the 85% floor
  • Manually probed with the real SDK + InMemorySpanExporter: chat and execute_tool spans report invoke_agent txn-agent as their parent, and a purchase records txn_agent.gate.decision=auto_buy, txn_agent.outcome=purchased and a 32-char idempotency key.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff noreply@codebuff.com

Summary by Sourcery

Add optional, privacy-conscious OpenTelemetry tracing across agent execution and transaction workflows.

New Features:

  • Add optional OpenTelemetry tracing for agent runs, including parent agent spans, model-turn and tool spans, and transaction and approval spans with bounded operational attributes.

Enhancements:

  • Keep tracing dependency-free and non-disruptive by lazy-loading OpenTelemetry, supporting no-op behavior without a configured provider, and excluding prompts and tool arguments from telemetry.

Build:

  • Add the txn-agent[otel] optional dependency and include the OpenTelemetry SDK in development dependencies.

Documentation:

  • Document optional tracing installation, provider configuration, emitted spans, and privacy behavior.

Tests:

  • Add coverage for span contracts, transaction outcomes, approval verdicts, token usage, tool errors, privacy protections, and the no-op path.

Chores:

  • Record the tracing capability in the unreleased changelog.

Observability is the missing half of an agent harness: the audit log records
what happened, but nothing emits it as a trace. Each model run is now an
`invoke_agent` span with `chat <model>` and `execute_tool <name>` children, and
the money path gets `txn_agent.purchase` / `txn_agent.checkout_cart` spans
carrying the gate decision, its reasons, the outcome and the idempotency key.
Approval verdicts get their own child span.

The core stays stdlib-only: OpenTelemetry is imported lazily, so an unconfigured
provider means every helper is a no-op, and tool arguments and prompts are never
recorded as attributes.
@sourcery-ai

sourcery-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

API

Add the optional tracing module and its public configuration, span, and attribute helpers for agent runs.

  • txn_agent/telemetry.py L1-205

Uses of the API

Instrument checkout and single-offer purchase paths with gate, approval, outcome, and idempotency details.

  • txn_agent/agent.py L103-120, L129-153, L208-245 (+3 small edits)

Wrap model runs, chat turns, and tool calls in spans while recording model usage and run metadata.

  • txn_agent/llm.py L17, L60-104

Instrument tool execution while excluding arguments and marking fail-soft errors.

  • txn_agent/tools.py L8, L125-142

Configuration

Add the OpenTelemetry API extra and include its SDK in the development dependencies for testing.

  • pyproject.toml L33-36

Documentation

Document the optional tracing setup, emitted spans, privacy behavior, and provider configuration.

  • CHANGELOG.md L6-17
  • README.md L157-178

Tests

Cover configured and no-op tracing, checkout and approval facts, tool errors, and model usage spans.

  • tests/test_telemetry.py L1-207

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 9 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="txn_agent/tools.py" line_range="128" />
<code_context>
+        # Every tool execution is a span: the arguments are deliberately NOT recorded
+        # (they can carry user data), only the tool name, the outcome and an error
+        # class when the call failed.
+        with telemetry.span(telemetry.tool_span_name(name), {
+            telemetry.ATTR_OPERATION: "execute_tool",
+            telemetry.ATTR_TOOL_NAME: name,
</code_context>
<issue_to_address>
**Sensitive details reach traces**

When tool arguments, gate reasons, or payment or checkout errors contain caller- or provider-supplied text, tool names, verticals, error text, gate reasons, and payment or checkout messages are copied into span attributes or events, so trace exporters retain dynamic details excluded by the privacy guarantees.

Record only fixed classifications and bounded facts in telemetry, omitting or sanitizing caller- and provider-supplied text.

Also at `txn_agent/tools.py:130-131`, `txn_agent/tools.py:141-142`, `txn_agent/agent.py:118`, `txn_agent/agent.py:134`, `txn_agent/agent.py:159-164`, `txn_agent/agent.py:210`, `txn_agent/agent.py:233`, `txn_agent/agent.py:238`.
</issue_to_address>

### Comment 2
<location path="txn_agent/agent.py" line_range="107" />
<code_context>
+            telemetry.ATTR_CART_ID: cart.cart_id,
+            telemetry.ATTR_MERCHANT: cart.merchant,
+            telemetry.ATTR_VERTICAL: cart.vertical.value,
+            telemetry.ATTR_CART_TOTAL: cart.total,
+            telemetry.ATTR_CART_LINES: len(cart.items),
+        }) as active:
</code_context>
<issue_to_address>
**Purchase traces report the wrong total**

When revalidation changes an offer price and the purchase proceeds, the purchase spans retain the original cart or quote total while checkout and payment use the revalidated live total, so traces report a different amount from the one charged.

Update the purchase and checkout span totals to use the revalidated live total.

Also at `txn_agent/agent.py:192`.
</issue_to_address>

### Comment 3
<location path="txn_agent/telemetry.py" line_range="115" />
<code_context>
+    except ImportError:
+        _TRACER = _NullTracer()
+    else:
+        _TRACER = trace.get_tracer(INSTRUMENTATION_NAME)
+    return _TRACER
+
</code_context>
<issue_to_address>
**Configured tracer gets overwritten**

When the first `_tracer()` lookup runs concurrently with `configure()` or `reset()`, `_tracer()` can assign `trace.get_tracer(...)` after `configure()` installs its tracer, so later spans go to the wrong provider and the configured exporter misses them.

Synchronize tracer initialization and configuration so an in-flight lookup cannot overwrite a configured tracer.

Also at `txn_agent/telemetry.py:121-122`.
</issue_to_address>

### Comment 4
<location path="tests/test_telemetry.py" line_range="201-203" />
<code_context>
+    assert chat.attributes[telemetry.ATTR_OUTPUT_TOKENS] == 30
+
+
+@pytest.mark.skipif(importlib.util.find_spec("opentelemetry") is not None,
+                    reason="opentelemetry-api is installed; the no-op path is not the default")
+def test_span_is_a_noop_without_opentelemetry() -> None:
+    with telemetry.span("anything", {"a": 1}) as active:
+        active.set_attribute("b", 2)
</code_context>
<issue_to_address>
**No-op path goes untested**

When the test suite runs with the project's development dependencies installed, the OpenTelemetry package is installed, so the skip condition skips the no-op test and CI does not detect regressions in that path.

Run the no-op assertions in an environment or test setup that exercises the absent-SDK path even when OpenTelemetry is installed.
</issue_to_address>

### Comment 5
<location path="README.md" line_range="177" />
<code_context>
+
+```bash
+pip install 'txn-agent[otel]'   # opentelemetry-api; bring your own exporter/SDK
+```
+
+Every agent run becomes a trace — `invoke_agent txn-agent` → `chat <model>` and
</code_context>
<issue_to_address>
**README setup snippet raises NameError**

When a reader runs the tracing setup snippet as shown, `txn_agent.telemetry.reset()` references `txn_agent` without importing or binding it, so the snippet raises `NameError` instead of resetting telemetry.

Import or otherwise bind `txn_agent` in the setup snippet before calling `txn_agent.telemetry.reset()`.

Also at `README.md:176`.
</issue_to_address>

### Comment 6
<location path="txn_agent/agent.py" line_range="159" />
<code_context>
             payment_ref, charged = self._pay(live, key, live.cart_id)
         except PaymentError as e:
-            return Outcome(Status.FAILED, live.cart_id, cart=live, reasons=(f"payment: {e}",))
+            telemetry.add_event(active, "txn_agent.payment_failed", {"error": str(e)})
+            return done(Outcome(Status.FAILED, live.cart_id, cart=live, reasons=(f"payment: {e}",)))
         try:
</code_context>
<issue_to_address>
**Failed purchases look like successful spans**

When a payment authorization or merchant checkout fails, `_checkout_cart` and `_purchase` catch payment and checkout exceptions and return normally from the span context; recording an exception only adds an event, so these failed purchase spans keep an unset status and standard OpenTelemetry error queries miss them.

Set the active span status to OpenTelemetry ERROR for failed payment and checkout outcomes, including payment failures that only add an event.

Also at `txn_agent/agent.py:164`, `txn_agent/agent.py:233`, `txn_agent/agent.py:238`.
</issue_to_address>

### Comment 7
<location path="txn_agent/telemetry.py" line_range="139-140" />
<code_context>
+    """
+    try:
+        ctx = _tracer().start_as_current_span(name)
+    except Exception:  # pragma: no cover - defensive: a broken tracer is not fatal
+        yield _NullSpan()
+        return
+    with _ctx(ctx) as active:
</code_context>
<issue_to_address>
**Tracing failures disappear silently**

When a configured tracer raises while creating a span or writing attributes, `telemetry.span` substitutes a null span when span creation fails, and `set_attributes` silently skips failed writes; a configured but broken tracer therefore produces no useful trace or diagnostic signal, leaving operators unaware that tracing has stopped.

Keep tracing failures isolated from purchases but emit a rate-limited diagnostic or expose a failure callback so operators can detect broken tracing.

Also at `txn_agent/telemetry.py:179`.
</issue_to_address>

### Comment 8
<location path="txn_agent/llm.py" line_range="78" />
<code_context>
+                resp = client.messages.create(model=model, max_tokens=1024, system=SYSTEM_PROMPT,
+                                              messages=messages, tools=TOOL_SCHEMAS)
+                usage = getattr(resp, "usage", None)
+                if usage is not None:
+                    telemetry.set_attributes(chat_span, {
+                        telemetry.ATTR_INPUT_TOKENS: getattr(usage, "input_tokens", None),
</code_context>
<issue_to_address>
**OpenAI chat spans omit token usage**

When the agent uses the OpenAI-compatible adapter and the endpoint returns token usage, when the OpenAI-compatible endpoint returns usage, `LLMResponse` does not expose it and the adapter discards it, so this check sees `None` and the chat span has no input or output token attributes.

Add usage to `LLMResponse` and copy the endpoint's usage fields into it so the shared tracing path can record them.
</issue_to_address>

### Comment 9
<location path="tests/test_telemetry.py" line_range="102" />
<code_context>
+    assert span.attributes[telemetry.ATTR_OUTCOME] == "purchased"
+    assert span.attributes[telemetry.ATTR_MERCHANT] == "AppliancePartsCo"
+    # The key is a hash, not a cart dump: safe to put on a span.
+    assert len(str(span.attributes[telemetry.ATTR_IDEMPOTENCY_KEY])) == 32
+    # Decimals are stringified so the attribute type stays valid.
+    assert isinstance(span.attributes[telemetry.ATTR_CART_TOTAL], str)
</code_context>
<issue_to_address>
**Wrong idempotency key goes unnoticed**

When the span records any incorrect key that is still 32 characters long, `test_checkout_records_gate_decision_outcome_and_key` only checks the key's length on line 102, so a constant or otherwise incorrect 32-character value passes and the test misses incorrect telemetry.

Assert that the recorded attribute equals the idempotency key computed for the checkout cart.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 8 findings to address first, and with a configured exporter, traces include cart IDs, the idempotency key, and payment or checkout exception details, which may expose transaction information outside the service. Reverting stops future traces but cannot retract data already exported.

Blocking findings: txn_agent/tools.py:128, txn_agent/agent.py:107, txn_agent/telemetry.py:115, tests/test_telemetry.py:203, txn_agent/agent.py:159, and 3 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread txn_agent/tools.py
# Every tool execution is a span: the arguments are deliberately NOT recorded
# (they can carry user data), only the tool name, the outcome and an error
# class when the call failed.
with telemetry.span(telemetry.tool_span_name(name), {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Sensitive details reach traces

When tool arguments, gate reasons, or payment or checkout errors contain caller- or provider-supplied text, tool names, verticals, error text, gate reasons, and payment or checkout messages are copied into span attributes or events, so trace exporters retain dynamic details excluded by the privacy guarantees.

Record only fixed classifications and bounded facts in telemetry, omitting or sanitizing caller- and provider-supplied text.

Also at txn_agent/tools.py:130-131, txn_agent/tools.py:141-142, txn_agent/agent.py:118, txn_agent/agent.py:134, txn_agent/agent.py:159-164, txn_agent/agent.py:210, txn_agent/agent.py:233, txn_agent/agent.py:238.

Prompt for AI agents
In `txn_agent/tools.py` at line 128:

**Sensitive details reach traces**

When tool arguments, gate reasons, or payment or checkout errors contain caller- or provider-supplied text, tool names, verticals, error text, gate reasons, and payment or checkout messages are copied into span attributes or events, so trace exporters retain dynamic details excluded by the privacy guarantees.

Record only fixed classifications and bounded facts in telemetry, omitting or sanitizing caller- and provider-supplied text.

Also at `txn_agent/tools.py:130-131`, `txn_agent/tools.py:141-142`, `txn_agent/agent.py:118`, `txn_agent/agent.py:134`, `txn_agent/agent.py:159-164`, `txn_agent/agent.py:210`, `txn_agent/agent.py:233`, `txn_agent/agent.py:238`.

Comment thread txn_agent/agent.py
telemetry.ATTR_CART_ID: cart.cart_id,
telemetry.ATTR_MERCHANT: cart.merchant,
telemetry.ATTR_VERTICAL: cart.vertical.value,
telemetry.ATTR_CART_TOTAL: cart.total,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Purchase traces report the wrong total

When revalidation changes an offer price and the purchase proceeds, the purchase spans retain the original cart or quote total while checkout and payment use the revalidated live total, so traces report a different amount from the one charged.

Update the purchase and checkout span totals to use the revalidated live total.

Also at txn_agent/agent.py:192.

Prompt for AI agents
In `txn_agent/agent.py` at line 107:

**Purchase traces report the wrong total**

When revalidation changes an offer price and the purchase proceeds, the purchase spans retain the original cart or quote total while checkout and payment use the revalidated live total, so traces report a different amount from the one charged.

Update the purchase and checkout span totals to use the revalidated live total.

Also at `txn_agent/agent.py:192`.

Comment thread txn_agent/telemetry.py
except ImportError:
_TRACER = _NullTracer()
else:
_TRACER = trace.get_tracer(INSTRUMENTATION_NAME)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Configured tracer gets overwritten

When the first _tracer() lookup runs concurrently with configure() or reset(), _tracer() can assign trace.get_tracer(...) after configure() installs its tracer, so later spans go to the wrong provider and the configured exporter misses them.

Synchronize tracer initialization and configuration so an in-flight lookup cannot overwrite a configured tracer.

Also at txn_agent/telemetry.py:121-122.

Prompt for AI agents
In `txn_agent/telemetry.py` at line 115:

**Configured tracer gets overwritten**

When the first `_tracer()` lookup runs concurrently with `configure()` or `reset()`, `_tracer()` can assign `trace.get_tracer(...)` after `configure()` installs its tracer, so later spans go to the wrong provider and the configured exporter misses them.

Synchronize tracer initialization and configuration so an in-flight lookup cannot overwrite a configured tracer.

Also at `txn_agent/telemetry.py:121-122`.

Comment thread tests/test_telemetry.py
Comment on lines +201 to +203
@pytest.mark.skipif(importlib.util.find_spec("opentelemetry") is not None,
reason="opentelemetry-api is installed; the no-op path is not the default")
def test_span_is_a_noop_without_opentelemetry() -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · No-op path goes untested

When the test suite runs with the project's development dependencies installed, the OpenTelemetry package is installed, so the skip condition skips the no-op test and CI does not detect regressions in that path.

Run the no-op assertions in an environment or test setup that exercises the absent-SDK path even when OpenTelemetry is installed.

Prompt for AI agents
In `tests/test_telemetry.py` at lines 201-203:

**No-op path goes untested**

When the test suite runs with the project's development dependencies installed, the OpenTelemetry package is installed, so the skip condition skips the no-op test and CI does not detect regressions in that path.

Run the no-op assertions in an environment or test setup that exercises the absent-SDK path even when OpenTelemetry is installed.

Comment thread README.md

trace.set_tracer_provider(TracerProvider()) # your exporter of choice
txn_agent.telemetry.reset() # re-detect the configured provider
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Low · README setup snippet raises NameError

When a reader runs the tracing setup snippet as shown, txn_agent.telemetry.reset() references txn_agent without importing or binding it, so the snippet raises NameError instead of resetting telemetry.

Import or otherwise bind txn_agent in the setup snippet before calling txn_agent.telemetry.reset().

Also at README.md:176.

Prompt for AI agents
In `README.md` at line 177:

**README setup snippet raises NameError**

When a reader runs the tracing setup snippet as shown, `txn_agent.telemetry.reset()` references `txn_agent` without importing or binding it, so the snippet raises `NameError` instead of resetting telemetry.

Import or otherwise bind `txn_agent` in the setup snippet before calling `txn_agent.telemetry.reset()`.

Also at `README.md:176`.

Comment thread txn_agent/agent.py
payment_ref, charged = self._pay(live, key, live.cart_id)
except PaymentError as e:
return Outcome(Status.FAILED, live.cart_id, cart=live, reasons=(f"payment: {e}",))
telemetry.add_event(active, "txn_agent.payment_failed", {"error": str(e)})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Failed purchases look like successful spans

When a payment authorization or merchant checkout fails, _checkout_cart and _purchase catch payment and checkout exceptions and return normally from the span context; recording an exception only adds an event, so these failed purchase spans keep an unset status and standard OpenTelemetry error queries miss them.

Set the active span status to OpenTelemetry ERROR for failed payment and checkout outcomes, including payment failures that only add an event.

Also at txn_agent/agent.py:164, txn_agent/agent.py:233, txn_agent/agent.py:238.

Prompt for AI agents
In `txn_agent/agent.py` at line 159:

**Failed purchases look like successful spans**

When a payment authorization or merchant checkout fails, `_checkout_cart` and `_purchase` catch payment and checkout exceptions and return normally from the span context; recording an exception only adds an event, so these failed purchase spans keep an unset status and standard OpenTelemetry error queries miss them.

Set the active span status to OpenTelemetry ERROR for failed payment and checkout outcomes, including payment failures that only add an event.

Also at `txn_agent/agent.py:164`, `txn_agent/agent.py:233`, `txn_agent/agent.py:238`.

Comment thread txn_agent/telemetry.py
Comment on lines +139 to +140
except Exception: # pragma: no cover - defensive: a broken tracer is not fatal
yield _NullSpan()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Tracing failures disappear silently

When a configured tracer raises while creating a span or writing attributes, telemetry.span substitutes a null span when span creation fails, and set_attributes silently skips failed writes; a configured but broken tracer therefore produces no useful trace or diagnostic signal, leaving operators unaware that tracing has stopped.

Keep tracing failures isolated from purchases but emit a rate-limited diagnostic or expose a failure callback so operators can detect broken tracing.

Also at txn_agent/telemetry.py:179.

Prompt for AI agents
In `txn_agent/telemetry.py` at lines 139-140:

**Tracing failures disappear silently**

When a configured tracer raises while creating a span or writing attributes, `telemetry.span` substitutes a null span when span creation fails, and `set_attributes` silently skips failed writes; a configured but broken tracer therefore produces no useful trace or diagnostic signal, leaving operators unaware that tracing has stopped.

Keep tracing failures isolated from purchases but emit a rate-limited diagnostic or expose a failure callback so operators can detect broken tracing.

Also at `txn_agent/telemetry.py:179`.

Comment thread txn_agent/llm.py
resp = client.messages.create(model=model, max_tokens=1024, system=SYSTEM_PROMPT,
messages=messages, tools=TOOL_SCHEMAS)
usage = getattr(resp, "usage", None)
if usage is not None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · OpenAI chat spans omit token usage

When the agent uses the OpenAI-compatible adapter and the endpoint returns token usage, when the OpenAI-compatible endpoint returns usage, LLMResponse does not expose it and the adapter discards it, so this check sees None and the chat span has no input or output token attributes.

Add usage to LLMResponse and copy the endpoint's usage fields into it so the shared tracing path can record them.

Prompt for AI agents
In `txn_agent/llm.py` at line 78:

**OpenAI chat spans omit token usage**

When the agent uses the OpenAI-compatible adapter and the endpoint returns token usage, when the OpenAI-compatible endpoint returns usage, `LLMResponse` does not expose it and the adapter discards it, so this check sees `None` and the chat span has no input or output token attributes.

Add usage to `LLMResponse` and copy the endpoint's usage fields into it so the shared tracing path can record them.

Comment thread tests/test_telemetry.py
assert span.attributes[telemetry.ATTR_OUTCOME] == "purchased"
assert span.attributes[telemetry.ATTR_MERCHANT] == "AppliancePartsCo"
# The key is a hash, not a cart dump: safe to put on a span.
assert len(str(span.attributes[telemetry.ATTR_IDEMPOTENCY_KEY])) == 32

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Wrong idempotency key goes unnoticed

When the span records any incorrect key that is still 32 characters long, test_checkout_records_gate_decision_outcome_and_key only checks the key's length on line 102, so a constant or otherwise incorrect 32-character value passes and the test misses incorrect telemetry.

Assert that the recorded attribute equals the idempotency key computed for the checkout cart.

Prompt for AI agents
In `tests/test_telemetry.py` at line 102:

**Wrong idempotency key goes unnoticed**

When the span records any incorrect key that is still 32 characters long, `test_checkout_records_gate_decision_outcome_and_key` only checks the key's length on line 102, so a constant or otherwise incorrect 32-character value passes and the test misses incorrect telemetry.

Assert that the recorded attribute equals the idempotency key computed for the checkout cart.

@cyberd4ch cyberd4ch closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant