fix(groq): keep multimodal input message content on spans - #3758
Harsh23Kashyap wants to merge 5 commits into
Conversation
|
Ran one request through the three packages' own message-attribute functions at The message is a user turn with three content parts:
So the same message comes out with different Two smaller notes from the same pass, both optional and both arguably out of scope here:
Disclosure: I am not a maintainer and have no commit rights here. I work on the groq instrumentor in this repo (my open PR #3782 touches |
|
Thanks for the careful pass, and for the concrete repro. Fixed in ea5917b and eb11835: groq now enumerates content parts by their position in the original list, matching together, portkey, and the existing openai instrumentor, so a part with no attributes leaves a gap instead of renumbering the parts after it. Each of the three packages also has a new test with an input_audio part pinning that gap. On the smaller notes: agreed that an input_audio or file part contributing nothing at all is the same class of silent drop, and that behaviour predates this PR in the openai instrumentor too, so a separate issue is the right scope for it. The isinstance(content, list) check is a fair flag; a tuple of parts behaves the same as before this PR in all three packages, so I left it untouched here. |
|
Re-ran your two commits myself at
Agreed on leaving the |
eb11835 to
6732248
Compare
|
@caroger This PR includes changes for Groq, Portkey, and TogetherAI. Please let me know if you'd prefer me to split them into three separate PRs. |
|
@satyadevai yes please separate |
…spans Content-part arrays (vision requests) were yielded as a raw list of dicts on llm.input_messages.N.message.content. OpenTelemetry rejects attribute values that are sequences of dicts, so the whole attribute was dropped and the user message content silently vanished from the span. Flatten typed content parts into message.contents.* instead, mirroring the openai instrumentor, with a JSON fallback for non-mapping parts. Fail-before/pass-after regression tests added for all three instrumentors.
The multimodal input flattening compacted the message.contents indices whenever a content part contributed no attributes (for example input_audio), shifting every later part one slot down. The together and portkey instrumentors and the existing openai instrumentor keep the original list positions, leaving a gap instead. Align groq with that convention and add a test pinning the gap for an unsupported part.
5eec3ff to
df49e05
Compare
Part of #3757
The groq instrumentor writes a request message's
contentontollm.input_messages.N.message.contentas-is. For multimodal requestscontentis a list of typed parts (text,image_url), and the OpenTelemetry SDK drops the attribute because sequences may only hold scalars, so the user message silently disappears from the span.Content parts are now flattened into
message.contents.N.message_content.*, the same way the openai instrumentor does it:textparts becometype=textwithmessage_content.textimage_urlparts becometype=imagewithmessage_content.image.image.urldocumentpart in newer groq releases) is skipped and leaves a gap instead of shifting later partsTests cover list, tuple and generator content with exhaustive attribute assertions, and a skipped part keeping later indices in place. The generator test also checks the request body the SDK sent.
Validation:
py310-ci-groqandpy310-ci-groq-latest(ruff, mypy, 20 tests) passqwen/qwen3.8-27b) with list and tuple content; both spans in Phoenix show the text and image parts undermessage.contents