Skip to content

fix(ai): preserve Anthropic cache-write TTL breakdowns - #4236

Merged
richardsolomou merged 13 commits into
PostHog:mainfrom
gouveags:fix/ai-anthropic-cache-ttl
Aug 6, 2026
Merged

fix(ai): preserve Anthropic cache-write TTL breakdowns#4236
richardsolomou merged 13 commits into
PostHog:mainfrom
gouveags:fix/ai-anthropic-cache-ttl

Conversation

@gouveags

@gouveags gouveags commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Problem

@posthog/ai can drop Anthropic's 5-minute and 1-hour cache-write breakdown before capturing an event.

The Anthropic streaming wrapper replaces the usage captured from message_start with the final usage delta, which may contain only output tokens. The LangChain callback also preserves the aggregate cache-write count but misses the TTL breakdown from both direct Anthropic responses and Bedrock Converse responses.

Bedrock Converse retains this data under response_metadata.usage.cacheDetails for non-streaming calls and response_metadata.metadata.usage.cacheDetails after streaming, but the PostHog callback did not read either shape.

Without those fields, ingestion cannot apply the TTL-specific pricing added in PostHog/posthog#71323.

Changes

  • Merge Anthropic streaming usage updates so fields from message_start survive later deltas that omit them.
  • Read cache-write TTLs from direct Anthropic and Bedrock Converse LangChain metadata, including the aggregated streaming shape.
  • Validate and sum repeated 5-minute and 1-hour entries before emitting both TTL counts alongside the existing aggregate.
  • Keep the aggregate-only fallback when the breakdown is incomplete, invalid, or does not match the aggregate.
  • Treat empty usage containers as missing while preserving explicit zero-valued counts.
  • Preserve existing usage-source precedence and avoid subtracting Anthropic cache tokens twice.
  • Add regression coverage for Anthropic streaming, direct LangChain usage, both Bedrock Converse shapes, TTL edge cases, aggregate-only payloads, and non-Anthropic precedence.

The Vercel integration already preserves raw usage and is unchanged.

How did you test this code?

The regression tests consume the mocked Anthropic stream through the wrapper and pass direct Anthropic and Bedrock Converse AIMessage usage shapes through the LangChain callback to the captured $ai_generation event.

  • Focused LangChain callback suite — 31 passed
  • pnpm --filter @posthog/ai test:unit — 517 passed, 84 skipped
  • pnpm --filter @posthog/ai lint
  • pnpm --filter @posthog/ai build
  • pnpm check:public-api
  • pnpm build
  • pnpm lint
  • pnpm lint:playground
  • pnpm test:functional

The root pnpm test:unit run hit two unrelated five-second timeouts in posthog-node under parallel load. Both affected suites passed together in isolation: 31/31 tests.

Release info Sub-libraries affected

Libraries affected

  • All of them
  • posthog-js (web)
  • posthog-js-lite (web lite)
  • posthog-node
  • posthog-react-native
  • @posthog/react-native-plugin
  • @posthog/react
  • @posthog/ai
  • @posthog/convex
  • @posthog/next
  • @posthog/nextjs-config
  • @posthog/nuxt
  • @posthog/openfeature-node-provider
  • @posthog/openfeature-web-provider
  • @posthog/rollup-plugin
  • @posthog/webpack-plugin
  • @posthog/types
  • @posthog/browser-common

Checklist

  • Tests for new code
  • Accounted for the impact of any changes across different platforms
  • Accounted for backwards compatibility of any changes (no breaking changes!)
  • Took care not to unnecessarily increase the bundle size

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

Related work

Deployment note

The new properties are optional, and the existing aggregate cache-write count remains available. The @posthog/ai release should follow PostHog/posthog#71323 so ingestion can price the TTL breakdown from the first event that includes it.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Gabriel directed the change. I used Codex to inspect the provider payloads, implement the fix, review regression risks, and run validation; no public session link is available. We kept the change inside the existing Anthropic and LangChain integrations: Vercel is unchanged, and pricing remains an ingestion responsibility.

@gouveags

Copy link
Copy Markdown
Contributor Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "fix(ai): preserve Anthropic cache-write ..." | Re-trigger Greptile

@gouveags

gouveags commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@richardsolomou here is the JS fix you asked for during the review on the Python PR.

I'm keeping this one as a draft until PostHog/posthog#71323 lands. Does this cover the JS side you had in mind, or is there anything else you'd like me to include?

@gouveags

Copy link
Copy Markdown
Contributor Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "fix(ai): preserve Bedrock cache TTL brea..." | Re-trigger Greptile

Comment thread packages/ai/src/langchain/callbacks.ts
Comment thread packages/ai/src/langchain/callbacks.ts
Comment thread packages/ai/src/langchain/callbacks.ts Outdated
@gouveags

Copy link
Copy Markdown
Contributor Author

@richardsolomou all fixed!

Comment thread packages/ai/src/langchain/callbacks.ts Outdated
@gouveags

Copy link
Copy Markdown
Contributor Author

Done!

@gouveags
gouveags marked this pull request as ready for review July 27, 2026 15:32
Comment thread packages/ai/src/langchain/callbacks.ts Outdated
@gouveags

Copy link
Copy Markdown
Contributor Author

@richardsolomou we still have some pending checks

@richardsolomou

Copy link
Copy Markdown
Member

@gouveags "Commits must have verified signatures" 😬😅

@marandaneto
marandaneto force-pushed the fix/ai-anthropic-cache-ttl branch from 3a2a789 to 5896d55 Compare August 2, 2026 17:36
@marandaneto

Copy link
Copy Markdown
Member

i forced pushed signed commits
theres this open question, can this be merged as is @richardsolomou @PostHog/team-ai-observability

@marandaneto
marandaneto requested review from a team August 2, 2026 18:52
@gouveags

gouveags commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the help, @marandaneto! Things have been hectic on my side, so I haven’t really had time to look at this before.

Align the two conflicted AI files before updating the branch, so the main sync can be created through GitHub with a verified signature. The final TTL resolution follows in the next signed commit.
Keep current main streaming and callback behavior while restoring the reviewed Anthropic and Bedrock cache TTL extraction, including the normalized generation fallback required by current LangChain result shapes.
@gouveags

gouveags commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

@richardsolomou I resolved the conflicts with the latest main and reran the focused @posthog/ai checks. The branch is mergeable again, and the new commits are verified. Could you take another look when you have a chance?

@gouveags

gouveags commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

@marandaneto maybe you can help

@richardsolomou
richardsolomou merged commit d36a421 into PostHog:main Aug 6, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants