[openai] Record cache-write and modality token usage by 1fanwang · Pull Request #681 · open-telemetry/opentelemetry-python-genai

1fanwang · 2026-09-11T18:34:30Z

Description

Chat Completions drops optional cache-write and modality counts. This exports them, with streaming details applied at cleanup.

OpenAI 3.1.0 defines the text/image fields. Missing and zero values remain omitted.

Fixes #603.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How has this been tested?

Client mocks drive real instrumentation with typed SDK responses.

Test A

From the configured latest tox environment:

P=instrumentation/opentelemetry-instrumentation-genai-openai
T=tests/test_chat_token_usage.py
PY="$PWD/.tox/py312-test-instrumentation-genai-openai-latest/bin/python"
B=$(mktemp -d)
git archive 7ca90a33f163baa6146749335438488ee90d4fba "$P/src" "$P/tests" | tar -x -C "$B"
cp "$P/$T" "$B/$P/$T"
PYTHONPATH="$B/$P/src" "$PY" -m pytest "$B/$P/$T" -k all-details -q -s
"$PY" -m pytest "$P/$T" -k all-details -q -s

Raw logs

Before:

E         Right contains 6 more items:
E         {'gen_ai.usage.audio.input_tokens': 10,

After:

{"exported_usage": {"gen_ai.usage.audio.input_tokens": 10, "gen_ai.usage.audio.output_tokens": 2, "gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.cache_write.input_tokens": 10, "gen_ai.usage.image.input_tokens": 20, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.text.input_tokens": 70, "gen_ai.usage.text.output_tokens": 18}}

These runs do not call the provider or exercise Weaver.

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

Followed the style guidelines of this project
Changelog updated if the change requires an entry
Unit tests added
Documentation updated

Signed-off-by: 1fanwang <1fannnw@gmail.com>

opentelemetry-pr-dashboard · 2026-09-11T18:35:09Z

Pull request dashboard status

Waiting on the author · refreshed 2026-09-13 22:36 UTC

Respond to 2 review items (e.g. link a commit, explain why not, ask a follow-up):

Inline threads: 1
Top-level threads: 2

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

Signed-off-by: 1fanwang <1fannnw@gmail.com>

Copilot

🔵 Needs a closer look

Add integer type assertions for exported usage attributes in the regression tests.

Pull request overview

Adds OpenAI Chat Completions cache-write and modality token usage recording for synchronous, asynchronous, and streaming calls.

Changes:

Extracts detailed usage fields into telemetry.
Adds local HTTP/SSE regression coverage.
Adds a changelog entry.

File summaries

File	Description
`instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_token_usage.py`	Tests usage-detail variants across call modes.
`instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/utils.py`	Extracts detailed usage fields.
`instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch.py`	Applies extraction to non-streaming responses.
`instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/chat_wrappers.py`	Applies extraction to streaming responses.
`instrumentation/opentelemetry-instrumentation-genai-openai/.changelog/603.added`	Documents the feature.

Review details

Suppressed comments (1)

instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_token_usage.py:222

This only checks equality, so a bool or float usage value would still pass (True == 1 and 1.0 == 1). The new semconv usage attributes must be integers; assert the types of the exported values here so the regression test catches invalid attribute values.

    assert actual == expected

Files reviewed: 5/5 changed files
Comments generated: 0
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lmolkova

Please make sure to follow existing test practices in this repo

lmolkova · 2026-09-11T22:00:50Z

+    )
+    invocation.text_input_tokens = get_property_value(
+        prompt_details, "text_tokens"
+    )


OpenAI Chat Completions does not expose text_tokens or image_tokens in prompt_tokens_details or completion_tokens_details. Only cache_write_tokens and audio_tokens exist on PromptTokensDetails, and audio_tokens on CompletionTokensDetails. Please remove the non-existent modality mappings.

https://github.com/openai/openai-python/blob/10ee3f0da2ac6f93345c1204bd7bb1a2faa79ff2/src/openai/types/completion_usage.py#L33-L53

lmolkova · 2026-09-11T22:00:50Z

                self._self_cached_prompt_tokens = get_property_value(
                    prompt_tokens_details, "cached_tokens"
                )
+            set_chat_usage_details(


Do not mutate invocation directly during chunk processing. Buffer the token details on the wrapper instance with a _self_ prefix and assign them to self._self_invocation in _cleanup().

Done in 7ff833e.

lmolkova · 2026-09-11T22:00:50Z

+                        "choices": [
+                            {
+                                "index": 0,
+                                "delta": {


Do not spin up a custom ThreadingHTTPServer with synthetic responses. Use standard VCR cassettes or client unit mocks consistent with the rest of the test suite.

Done in 7ff833e.

Signed-off-by: 1fanwang <1fannnw@gmail.com>

lmolkova · 2026-09-12T23:39:57Z

    return getattr(obj, property_name, None)


+def set_chat_usage_details(


The token-detail extraction is duplicated between this helper and the stream wrapper. Please share extraction across both paths, buffering the values until stream cleanup.

Please also wait for #674 to land, then reuse its set_input_tokens() and set_output_tokens() helpers for modality breakdowns. Keep the aggregate cache-write count separate.

[openai] Record cache-write and modality token usage

7ba7531

Signed-off-by: 1fanwang <1fannnw@gmail.com>

Copilot AI lite review requested due to automatic review settings September 11, 2026 18:34

1fanwang requested a review from a team as a code owner September 11, 2026 18:34

github-project-automation Bot added this to GenAI Semantic Conventions and Instrumentation libraries Sep 11, 2026

Copilot started reviewing on behalf of 1fanwang September 11, 2026 18:34 View session

opentelemetry-pr-dashboard Bot mentioned this pull request Sep 11, 2026

Pull Request Dashboard #171

Open

1fanwang added 2 commits September 11, 2026 11:35

Associate the changelog entry with its pull request

6fdd9f7

Signed-off-by: 1fanwang <1fannnw@gmail.com>

Show exported usage in the HTTP regression output

dd9d211

Signed-off-by: 1fanwang <1fannnw@gmail.com>

Copilot AI reviewed Sep 11, 2026

View reviewed changes

lmolkova requested changes Sep 11, 2026

View reviewed changes

lmolkova mentioned this pull request Sep 11, 2026

[openai] Capture detailed token usage (cache write and modality breakdown) #603

Open

1fanwang added 2 commits September 11, 2026 16:28

Merge main into Chat Completions usage fix

bfe2bf0

Signed-off-by: 1fanwang <1fannnw@gmail.com>

Buffer Chat Completions usage until stream cleanup

7ff833e

Signed-off-by: 1fanwang <1fannnw@gmail.com>

lmolkova reviewed Sep 12, 2026

View reviewed changes

Navigation Menu

Uh oh!

[openai] Record cache-write and modality token usage - #681

[openai] Record cache-write and modality token usage#681
1fanwang wants to merge 5 commits into
open-telemetry:mainfrom
1fanwang:1fannnw/openai-token-details-603

1fanwang commented Sep 11, 2026 •

edited

Loading

Uh oh!

opentelemetry-pr-dashboard Bot commented Sep 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

lmolkova left a comment

Uh oh!

lmolkova Sep 11, 2026

Uh oh!

1fanwang Sep 12, 2026

Uh oh!

lmolkova Sep 11, 2026

Uh oh!

1fanwang Sep 12, 2026

Uh oh!

lmolkova Sep 11, 2026

Uh oh!

1fanwang Sep 12, 2026

Uh oh!

lmolkova Sep 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return getattr(obj, property_name, None)


		def set_chat_usage_details(

Uh oh!

Conversation

1fanwang commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How has this been tested?

Checklist

Uh oh!

opentelemetry-pr-dashboard Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (1)

Uh oh!

lmolkova left a comment

Choose a reason for hiding this comment

Uh oh!

lmolkova Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

1fanwang Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

1fanwang Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

1fanwang Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

1fanwang commented Sep 11, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Sep 11, 2026 •

edited

Loading