fix(aio): use bedrock inference-profile region for cost matching - #74489
Closed
posthog[bot] wants to merge 1 commit into
Closed
fix(aio): use bedrock inference-profile region for cost matching#74489posthog[bot] wants to merge 1 commit into
posthog[bot] wants to merge 1 commit into
Conversation
Bedrock cross-region inference profiles prefix the model ID with the region the request ran in (`bedrock/us.anthropic.claude-sonnet-5`). The cost matcher resolved the model to a catalog row first and then handed only `$ai_provider` (`bedrock`) to the provider resolver, so the region was gone by then. With no exact `amazon-bedrock` key the resolver fell into its partial-match loop and returned whichever `amazon-bedrock-*` key happened to come first in `llm-costs.json`, regardless of region. Extract the region token from the raw model string and thread it through as a preferred-region hint, tried ahead of both the un-regioned provider key and the partial-match fallback. Candidate keys are sorted so the pick no longer depends on key ordering in the generated catalog. Models without a region prefix keep their current behavior. Generated-By: PostHog Code Task-Id: 9370f695-5848-462e-97e9-7268c09d6f4c
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
Radu-Raicea
marked this pull request as ready for review
August 3, 2026 19:07
Member
|
@Radu-Raicea #74380 already merged the same fix and closed #74366, so #74489 is superseded and can be closed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Closes #74366
AWS Bedrock cross-region inference profiles put the region the request ran in at the front of the model ID, so events arrive as
bedrock/us.anthropic.claude-sonnet-5.findCostFromModelresolved that to a catalog row first and then handed only$ai_provider(bedrock) to the provider resolver, so the region was already gone. With no exactamazon-bedrockkey, the resolver fell into its partial-match loop and didObject.keys(providerCosts).find(key => key.includes('amazon-bedrock'))- it returned whicheveramazon-bedrock-*key happened to sit first inllm-costs.json, not the one for the region the request actually ran in.Two consequences:
us.anthropic.claude-sonnet-5pickedamazon-bedrock-claude-on-awsinstead ofamazon-bedrock-us-east-1, a flat 9.09% understatement across input, output, cache read, and cache write (and the 1-hour cache write bucket, which is derived from the selected input price).anthropic/claude-sonnet-4.6only hasamazon-bedrock-eu-west-1andamazon-bedrock-globalrows, so aus.request was being billed at EU rates.The failure is silent: no errors, no warnings, just quietly wrong numbers in the metric AI observability exists to get right.
Changes
extractInferenceProfileRegionpulls the region token off the raw model string andresolveModelCostForProvidertakes it as a preferred-region hint. Handlesus.,eu.,global., andapac., mapping onto keys shaped likeamazon-bedrock-us-east-1,amazon-bedrock-us,amazon-bedrock-eu-west-1, andamazon-bedrock-global.Details worth calling out for review:
provider/segment, so we read the last slash-delimited segment first. That also covers inference-profile ARNs.eu.request on a model with bothamazon-bedrockandamazon-bedrock-eu-west-1rows now gets the EU row.usnever matches something likeamazon-bedrock-usw.globalbefore giving up. That is what fixes the sonnet-4.6 case above: with no US row, the un-regioned list price is a closer stand-in than another region's premium, and it makes the pick deterministic instead of dependent on catalog key order.llm-costs.jsonkey order is regenerated by a scheduled job and must not decide which region we bill at.$ai_providerstays authoritative, and models with no region prefix keep today's behavior.Note
This changes attributed cost for Bedrock inference-profile events going forward. Already-ingested events are not backfilled.
How did you test this code?
Automated only - I (Claude) did not run the app or ingest live events.
New cases, and the regression each catches:
extractInferenceProfileRegionis parameterized over the prefix shapes. The load-bearing rows arebedrock/us.…(a naivestartsWith('us.')misses it, which is the trap here) and the negative rowsclaude-opus-4.5/openai/gpt-4.1-mini, which would start reading a version segment as a region if the split logic regressed.resolveModelCostForProviderregion cases: region beats another provider-specific key (the 9.09% bug), region beats the un-regioned key,globalfallback instead of a different region (the EU-rates bug), no match on a shared prefix, and unprefixed models unchanged. These are distinct branches rather than data variations, so they are separate cases.findCostFromModelcase as a wiring guard: the region has to actually be threaded from the raw model string into the resolver. Every resolver test would still pass if that thread were dropped.Per the issue, tests assert the correct region is chosen rather than pinning today's specific key ordering.
The wider
src/ingestion/pipelines/aisuite passes too (958 tests), except pre-existing local-environment failures unrelated to this change: the S3 blob-store integration tests need object storage running, andconsumer.test.ts/pipeline.test.tsneed thenode-rdkafkanative binding.tsc --noEmitand eslint are clean overai/costs.I also ran the resolver against the real catalog with and without the hint, to bound the blast radius. Only Bedrock rows move, and they move the way you would want:
Resolution diff across the real catalog, per region token
Automatic notifications
Docs update
No user-facing docs cover the regional pricing-row selection, so nothing to update.
🤖 Agent context
Autonomy: Fully autonomous
Written by Claude (Claude Code) from a PostHog inbox report. Skills invoked:
/writing-tests.Two judgment calls a reviewer may want to push back on. First, the hint is tried before the exact un-regioned provider key, not only before the partial-match fallback. The narrower reading of the issue would leave
eu.amazon.nova-lite-v1on the genericamazon-bedrockrow even though anamazon-bedrock-eu-west-1row exists; if a request names its region and we have that region's price, using it seemed clearly more correct. Second, theglobalfallback per region token. Without it,us.anthropic.claude-sonnet-4-6(the highest-volume prefixed model) stays on EU rates and the pick stays hostage to catalog key ordering. I considered leaving unmatched regions on the old partial match and rejected it for exactly that reason.I did not touch the unprefixed-model path, which still takes the first partial match. It is the same class of nondeterminism, but changing it is a broader behavior change than this fix needs.
Created with PostHog Desktop from this inbox report.