You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Haiku 5.5's prompt-length pricing tiers aren't modelled — long chats under-estimate by up to 5× #1003
Follow-up from #1002, which adds Claude Haiku 5.5 to the model catalog.
The gap
Haiku 5.5 is the only model in the catalog priced by prompt length rather than at a flat per-token rate (pricing docs):
Prompt size
Input
Output
5m cache write
Cache read
≤ 100,000 tokens
$0.10
$0.50
$0.125
$0.01
> 100,000 tokens
$0.50
$2.50
$0.625
$0.05
A request's prompt length counts all of its input tokens, including cache reads and writes. Each request is priced on its own, so a request over the threshold pays the higher rate even when most of its prompt was a cache hit.
estimateCostUsd in packages/server/src/models.ts prices a chat's cumulativeTokenTotals, which no longer carry per-request prompt lengths. It therefore can't tell the tiers apart. #1002 pins Haiku 5.5 at the base tier, which is exact for short chats and under-estimates a long one by up to 5×. A test in test/unit/usage.test.ts pins this so a change is deliberate rather than accidental.
Why it's fixable
The tier is decidable where the totals are built. readSessionTokenUsageFile (packages/server/src/usage.ts) already computes each turn's prompt length — it's the same input + cacheCreation + cacheRead sum it uses for contextTokens — so each turn could be bucketed into a tier at read time.
Sketch, not a decision
One shape that fits the existing seams:
ModelPricing gains an optional longContext?: { thresholdTokens, inputPer1M, outputPer1M }.
TokenTotals splits into a base TokenCounts plus an optional longContext?: TokenCounts holding the over-threshold portion.
usage.ts picks the bucket per turn; estimateCostUsd prices each at its own rate.
mergeByModelInto in subagents.ts folds the nested counts too, so sub-agent roll-ups stay correct.
The cost is that TokenTotals stops being four flat numbers, which several call sites construct literally. Worth weighing against how much Haiku 5.5 actually gets used for long chats before building it.
Filed for a design decision before implementation rather than folding it into #1002.
Activity
feat(server): offer Claude Sonnet 5.5 and Haiku 5.5 alongside Fable 5.2
feat(server): offer Claude Fable 5.2, Sonnet 5.5 and Haiku 5.5 in the…