fix(aio): price Anthropic cache writes by TTL - #71323
Conversation
richardsolomou
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Could we add one TTL-breakdown case using numeric strings? The validator accepts them, but the pricing-path tests only use numbers, so a regression in hasNumericProperty could silently fall back to aggregate pricing.
|
@richardsolomou Thanks for catching this. I added a numeric-string TTL breakdown case in |
|
@richardsolomou Your last review prompted me to take another close pass over the provider payloads, and I noticed we weren't covering Vercel's Amazon Bedrock I've added support for its 5-minute and 1-hour TTL buckets under both Let me know if you'd like me to cover anything else. |
richardsolomou
left a comment
There was a problem hiding this comment.
Could we also cover rawUsage.providerMetadata.amazonBedrock? That supported nested path currently has no regression test.
|
I added the missing |
|
@richardsolomou Thanks again for the reviews. It seems like the remaining step is for you or another maintainer to approve the check workflows so the required pipeline can run, and then merge if everything passes. I also opened the companion draft PR PostHog/posthog-python#746. I have been running into this exact issue in our application, and we need both the ingestion and Python SDK sides covered so the same reporting bugs do not continue through that path. We use PostHog to track our users' LLM usage and inform product and business decisions, so accurate cost data is important to us. If there is someone on the Python SDK or LLM analytics side I should coordinate with, I would really appreciate a pointer. Please let me know if there is anything else you have in mind or any other way I can help. Thanks for the whole review process and for the chance of contributing! |
|
@gouveags Ran CI, couple small issues to sort out then should be good to go |
|
@richardsolomou No worries! It looks like I can’t do much from my side, so I’ll wait for the checks to be rerun. Thanks again for all the help! |
|
@richardsolomou this seems finally ready to be merged |
337a032 to
db5b57d
Compare
|
@richardsolomou I didn't know that all commits were required to be verified, really sorry. I'll keep this in mind for future contributions. I've fixed the branch history. Can you check again? |
Problem
Anthropic prices 5-minute cache writes at
1.25 ×the input-token rate and 1-hour writes at2 ×. PostHog currently collapses both into one aggregate count, so automatic costing applies the 5-minute rate to 1-hour writes.Closes #71306
Changes
@posthog/ai's Vercel wrapper (usage.rawandproviderMetadata.anthropic.usage).providerMetadata.amazonBedrock.usage.cacheDetails, with legacybedrockalias support and no double-counting when both aliases are present.For custom pricing,
$ai_cache_write_1h_token_pricewins for 1-hour writes. If it is absent,$ai_cache_write_token_priceapplies to both TTLs. Catalog pricing keeps the existing 5-minute rate and derives the 1-hour rate from2 ×the resolved prompt price. No model prices are hardcoded.Note
This corrects core ingestion when an event supplies a complete TTL breakdown through raw Anthropic usage, Vercel Amazon Bedrock metadata, or both explicit properties. SDKs that emit only the aggregate keep the existing pricing path until they propagate the breakdown.
How did you test this code?
$ai_usageremoval, process-level pricing, and invalid-property normalization.git diff --check.I did not run a full PostHog stack or a full production build. The change is covered at the calculation, extraction, validation, and
processAiEventboundaries.👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Automatic notifications
Docs update
Updated the AI generation event reference with the TTL-specific count fields, complete-pair fallback rule, aggregate invariant, and custom-price precedence. Registered the fields in the event taxonomy.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
I directed the fix and used Codex for implementation, tests, validation, and review. The repository's
/writing-testsskill shaped the regression-first test contract. No public session link is available.The implementation stays in the existing ingestion pipeline and continues using the existing model catalog. It derives the 1-hour rate from the resolved prompt price, preserves aggregate-only events, and leaves SDK-specific propagation as follow-up work.
Companion Python SDK work
PostHog/posthog-python#746 preserves the TTL breakdown across supported Python integrations. It is intentionally a draft until this server contract merges. The SDK gap is tracked in PostHog/posthog-python#745.