fix(ai-obs): recognize legacy Gemini tool blocks in traces - #76028
fix(ai-obs): recognize legacy Gemini tool blocks in traces#76028marco-g-pm wants to merge 3 commits into
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
🤖 CI report✅ Bundle size — no changeUncompressed size of every built Total: 65.58 MiB · no change No file changed by more than 1000 B. Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report ✅ Eager graph — within budgetHow much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy
🟢 Largest files eagerly shipped from
|
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 762 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 285.5 KiB | ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 237.5 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 231.5 KiB | ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/module.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.0 KiB | src/lib/api.ts |
| 94.7 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.20 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.20 MiB · 17 files | no change | ████░░░░░░ 38.4% of 5.72 MiB |
| Deferred (lazy) | 2.08 MiB · 33 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.1 KiB | no change | █░░░░░░░░░ 5.8% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 721.9 KiB | dist/toolbar/toolbar-app-EFK53Z4U.css |
| 551.9 KiB | dist/toolbar/chunk-chunk-H5J6CWWA.js |
| 484.6 KiB | dist/toolbar/chunk-chunk-PXAACQ6U.js |
| 133.6 KiB | dist/toolbar/chunk-chunk-HY3POEM4.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-T5KY5WYR.js |
| 71.0 KiB | dist/toolbar/toolbar-app-HJXGB3X4.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-27JL52RE.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-GDVGUWBX.js |
| 20.9 KiB | dist/toolbar/chunk-chunk-FYPGMRW4.js |
| 12.2 KiB | dist/toolbar/chunk-chunk-PIK3PADE.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🔺 +3.7 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1393.76 MiB · 🔺 +3.7 KiB (+0.0%)
7cc5891 to
a43b6b4
Compare
posthog-python 7.30.1-7.35.x copied Gemini function calls and their
responses into `$ai_input` in the raw form the google-genai SDK emits:
{"type": "function_call", "function_call": {name, args}}
{"type": "function_response", "function_response": {name, response}}
Neither matches any shape `isToolResult`/`isToolStepItem` know, so
traces captured in that window misclassify their tool loop three ways:
- `isToolMessage` computes `hasText` from content items that are neither
a tool step nor a tool result, so a message carrying only a tool
response counts as having real text and gets surfaced in the traces
list as the human-readable answer.
- `isInternalToolResultUserMessage` requires every content item to be a
tool result, so the framework-appended tool-response user message
isn't classified internal and leaks as a visible user bubble.
- `isUnrenderableContentItem` treats the block as non-text content the
transcript can't render.
The SDK now converts both kinds (PostHog/posthog-python#823), but
already-ingested traces keep the old form, so match it here too.
`function_call` needs its nested object checked: OpenAI's Responses API
uses the same `type` with `name`/`call_id` at the top level, and
`isOpenAIResponsesFunctionCall` already owns that case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
a43b6b4 to
fec8beb
Compare
|
👋 Visual changes detected for this PR. Review and approve in PostHog Visual Review If these changes are unexpected, they may be caused by a flaky test or a broken snapshot on master. Don't approve — rerun the job or wait for a fix. |
🦔 Hogbox preview · ❌ build failedThe preview didn't come up for commit Previews are optional and never block merging. A failure here is often a hogland or tailnet hiccup rather than anything in your PR, so the check stays green and this comment is the status. |
|
Reviews (1): Last reviewed commit: "Merge branch 'master' into claude/gemini..." | Re-trigger Greptile |
|
deemed not worth the tech debt for legacy traces that will expire in 30 days |
1 participant
Problem
posthog-python 7.30.1 through 7.35.x copied Gemini function calls and their responses into
$ai_input, in the raw form the google-genai SDK emits:{"type": "function_call", "function_call": {"name": "...", "args": {}}} {"type": "function_response", "function_response": {"name": "...", "response": {}}}isToolResultandisToolStepItemmatch neither one. Traces from that window misread their tool loop three ways:isToolMessagehasTextcounts any item that is not a tool step and not a tool result. A message holding only a tool response reads as real text. The traces list then shows raw tool output as the answer.isInternalToolResultUserMessageisUnrenderableContentItemPostHog/posthog-python#823 fixes the SDK. Traces already stored keep the old shape, so the guards need to match it here.
These guards classify, they do not render.
ConversationMessagesDisplayalready falls back to a JSON viewer for an unknown block, so the data is not invisible everywhere.Changes
Adds
isLegacyGeminiFunctionResponseandisLegacyGeminiFunctionCall.isToolResulttakes the response.isToolStepItemtakes both.function_callneeds its nested object checked. The OpenAI Responses API uses the sametypewithnameandcall_idat the top level, andisOpenAIResponsesFunctionCallalready owns that case.No screenshots. This change is predicate logic with no visual part of its own.
How did you test this code?
New cases in
utils.test.ts, for the regressions no existing case caught:isToolResulttakes a legacyfunction_response. It still rejects a legacyfunction_call, which is a call and not a result. It also rejects a bare{type: 'function_response'}with no payload.isToolStepItemtakes both legacy kinds.I could not run jest. My worktree has no frontend deps, and my node is v26 against the repo range
>=24 <25. CI has to be the check here. What I did run instead: I pulled the guard bodies out ofutils.tsverbatim, paired them with the realisObjectandisString, and ran nine cases through them. All nine matched what I expected. That set includes an OpenAI Responsesfunction_call, which still routes toisOpenAIResponsesFunctionCalland does not trip the new guard.The pre-commit hook also could not run, because
bin/hoglineeds the flox and uv Python env. I committed with--no-verify. CI has since confirmed that part: Frontend formatting passes and Frontend typechecking passes.I did not open a real trace in the UI.
Automatic notifications
Docs update
None needed.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written with Claude Code (Opus 5), directed by @marco-g-pm.
This is the backfill half of PostHog/posthog-python#823. That PR fixes the shape for new traces. This one covers traces already stored.
The fix stays in the guards. Historical traces still feed evals a raw
repr()dump, because the eval path is a separate Python formatter. A matching guard inmessage_formatter.pywould fix that, and I left it out on purpose. Rendering these blocks as tool-result cards is a second option I did not try.