fix(pricing): general vendor-rename normalizer + fuzzy-pricing UI disclosure - #194
Merged
pitimon merged 1 commit intoAug 22, 2026
Merged
Conversation
…closure Two follow-ups from the #187/#188 cross-verification review. 1. matcher.js: add a scoped denoise retry (issue #193). #188 fixed the fable/mythos canary regression with a per-name curated fuzzy rule, but left the structural gap open — any future auto-pilot/canary rename of an already-priced model would reproduce the same bug and require another one-line patch. The denoise retry strips the '-auto-pilot-' infix and trailing '-vN-canary'/'-canary' suffix and retries the full lookup ladder exactly once, only after every other tier has missed on the raw id, so a genuine exact match on the raw id (however unlikely) is never shadowed. Also narrows the #188 fuzzy rules from bare 'fable'/'mythos' substrings to 'claude-fable'/'claude-mythos'. Verified against the published pricing seed that a real collision risk existed: claude-mythos-preview is a genuine LiteLLM-priced model that happens to contain the substring 'mythos' — the bare rule would have silently mispriced any future model in the same shape that lacks its own exact LiteLLM/curated entry. 2. dashboard: wire the already-computed fuzzyPricingModels (from model-breakdown.ts) into ProviderBreakdownCard (issue #192). The provider card already discloses '{{count}} pricing missing' for unpriced models; it now also discloses '{{count}} pricing estimated' when any model resolved via substring/fuzzy match rather than an exact price, so a user sees when a total like $3,164.69 is a curated estimate, not a reconciled invoice figure. Verified: node --test test/pricing.test.js (39/39), dashboard ProviderBreakdownCard suite (10/10, incl. 2 new), npm run ci:local full gate (dashboard 320/320, product suite 1059/1059, all validators) — all green. Addresses #193, #192.
|
To use Codex here, create a Codex account and connect to github. |
pitimon
deleted the
fix/193-192-vendor-rename-normalizer-and-fuzzy-disclosure
branch
August 22, 2026 15:27
pitimon
added a commit
that referenced
this pull request
Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Goal (one sentence)
Close two gaps a cross-verification review found in #187/#188: the fable/mythos fix was a per-name patch with no general handling and a bare-substring collision risk (#193), and fuzzy-matched prices are computed but never disclosed to the user (#192).
Scope
src/)dashboard/)Checklist
npm run ci:localpasses, or focused test commands are listed below with rationaledashboard/src/content/copy.csv(usage.overview.provider_fuzzy_pricing)feat:/fix:/refactor:/docs:/chore:/test:/ci:)chore(release)PR per the established pattern (chore(release): 0.39.53 #189 after fix(pricing): resolve fable/mythos canary variants via fuzzy match #188)Why
A cross-verification review of #187/#188 (17-question pre-ship checklist) flagged three gaps after independently reproducing the original bug fix against the published npm artifacts and 76 real production queue rows. #191 needs external billing data and is tracked separately. This PR resolves the two gaps that are resolvable from source:
#193 — no general normalizer, and a real collision risk in the narrow fix
#188s fix added
{match:"fable", ref:"claude-fable-5"}/{match:"mythos", ref:"claude-mythos-5"}as bare curated-fuzzy substring rules. Two problems verified independently against the published pricing seed:claude-auto-pilot-opus-4-8-v1-canary) reproduces the exact Claude auto-pilot/canary model ids (fable/mythos) bill $0 — fuzzy pricing match misses non-contiguous substrings #187 bug and needs another one-line curated patch. Verified: before this PR, that hypothetical id resolvesmiss; after, it resolveslitellm:exactvia the new denoise retry with no curated entry added.claude-mythos-previewis a genuine LiteLLM-priced model in the published seed (generated_at: 2026-08-22T15:00:43.511Z) that contains the bare substring"mythos". It currently survives becauselitellm:exactruns beforecurated:fuzzyin the resolve order, but the bare rule was one design change away from silently mispricing it (or any future model with the same shape and no exact entry) at the Opus-tier Fable/Mythos rate. Narrowed the rules to"claude-fable"/"claude-mythos"to remove that risk while keeping the Claude auto-pilot/canary model ids (fable/mythos) bill $0 — fuzzy pricing match misses non-contiguous substrings #187 fix intact.Fix:
matcher.jsgains astripVendorRenameNoise()denoise retry — strips-auto-pilot-infix and trailing-vN-canary/-canary, then re-runs the full 7-tier lookup ladder exactly once, only after every tier has already missed on the raw id (so a genuine exact match on the noisy raw id is never shadowed).#192 — fuzzy pricing computed but never shown to the user
model-breakdown.tsalready computesfuzzyPricingModelsper provider andplan-value.tsalready uses it to mark the plan-value comparison as a "floor," butProviderBreakdownCard.jsxonly ever renderedmissingPricingModels("N pricing missing"). A user looking at a provider total like $3,164.69 (the real post-#188 total forclaude-auto-pilot-fable-v1-canaryacross 76 production rows) had no way to tell it apart from an exactly-priced total.Fix: wire the existing
fuzzyPricingModelsinto the same card, mirroring the missing-pricing pattern exactly — new copy keyusage.overview.provider_fuzzy_pricing("{{count}} pricing estimated").Codex Context (required when requesting @codex review)
_denoisedguard), never fires when stripping is a no-op, and never shadows a real exact match on the raw (noisy) id. The narrowed fuzzy rules still resolve every id fix(pricing): resolve fable/mythos canary variants via fuzzy match #188 fixed; they no longer match models that merely contain "fable"/"mythos" without the "claude-" prefix.some-vendor-fable-mini) now correctly misses;claude-mythos-preview(real seed collision) still resolves vialitellm:exact, unaffected.node --test test/pricing.test.js→ 39/39 pass (5 new:stripVendorRenameNoiseunit test, denoise-retry-resolves-without-curated-rule, denoise-retry-does-not-shadow-raw-exact-match, denoise-retry-no-loop-on-clean-string, fuzzy-rule-scoped-not-bare-substring)npm --prefix dashboard run test -- ProviderBreakdownCard→ 10/10 pass (2 new: shows-fuzzy-caveat, does-not-show-fuzzy-caveat-when-exact)npm run ci:local→ full pass: dashboard build + 320/320 dashboard tests, 1059/1059 product-suite tests, all validators (copy, ui-hardcode, guardrails, curated-expiry, outbound, openwiki)Risk Layer Trigger (if any)
(No risk-layer trigger — pure pricing-lookup logic + a read-only UI disclosure, no I/O/auth/exposure surface touched.)
Regression Test Gate
Most likely regression surface
Verification method (choose at least one)
curated:exacton the noisy id itself, notcurated:fuzzyvia the stripped form).claude-auto-pilot-fable-v1-canary,claude-mythos-v1-canary) still resolve correctly under the narrowed rule.Uncovered scope