Skip to content

Fix #524: prevent retired models from inheriting reseller pricing - #527

Open
willwashburn wants to merge 7 commits into
mainfrom
issue-524-reseller-precedence
Open

Fix #524: prevent retired models from inheriting reseller pricing#527
willwashburn wants to merge 7 commits into
mainfrom
issue-524-reseller-precedence

Conversation

@willwashburn

@willwashburn willwashburn commented Aug 3, 2026

Copy link
Copy Markdown
Member

Retired first-party models now report as unpriced when their authoritative provider removes them from the bundled snapshot. Reseller tariffs are accepted only for model IDs that have never appeared under a primary pricing provider.

The bundled append-only ownership catalog is seeded from the pre-refresh and current snapshots. Pricing refreshes union retained, outgoing, and incoming primary-provider IDs, while a Rust synchronization test rejects snapshot changes that omit a current primary model from the catalog. The internal codex-auto-review alias is protected from future direct reseller entries.

Every production query verb loads the authoritative models.dev.json override adjacent to the active ledger. This makes $RELAYBURN_HOME/models.dev.json effective for the CLI and preserves custom ledger homes for embedders. User overrides can explicitly restore rates for retired models.

The honest unpriced path covers all 29 retired-primary reseller fallbacks without changing reseller-exclusive models. Grouped summaries expose unpriced turns and models, summary buckets expose unpricedTurns, and compare retains its existing per-cell turns versus pricedTurns signal.

Golden summary output reports gpt-5-codex at zero cost with explicit unpricedTurns and unpricedModels fields and a human warning. Current Claude first-party rows remain unchanged. Two stale state-status snapshots now match the existing schema version 6.

Fixes #524
Fixes #509

🤖 Generated with Claude Code.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@willwashburn, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 56 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c3b5ebcc-46be-4c78-9884-5a7ff243e205

📥 Commits

Reviewing files that changed from the base of the PR and between 08fb206 and 5d3e0d9.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • crates/relayburn-cli/src/commands/summary/human.rs
  • tests/fixtures/cli-golden/snapshots/summary.stderr.txt
📝 Walkthrough

Walkthrough

The change protects retired first-party models from reseller pricing, loads ledger-local pricing overrides, and reports unpriced turns and models in summaries. It adds model catalog maintenance, regression tests, changelog notes, and updated CLI fixtures.

Changes

Pricing and unpriced usage

Layer / File(s) Summary
Primary model protection and catalog generation
crates/relayburn-sdk/data/primary-model-ids.json, crates/relayburn-sdk/src/analyze/pricing.rs, scripts/update-pricing.mjs
Primary model history prevents retired models from inheriting reseller tariffs. The pricing update script maintains the protected model catalog. User overrides can restore pricing.
Ledger-local pricing resolution
crates/relayburn-sdk/src/query_verbs/mod.rs, crates/relayburn-sdk/src/query_verbs/{compare,flow,hotspots,overhead,sessions}.rs, crates/relayburn-sdk/src/query_verbs/summary/mod.rs
Query paths load pricing from the active ledger’s adjacent models.dev.json file. Summary buckets count unpriced turns.
Unpriced usage reporting and validation
crates/relayburn-cli/src/commands/summary/{json,mod}.rs, crates/relayburn-sdk/src/query_verbs/tests.rs, tests/fixtures/cli-golden/snapshots/*, CHANGELOG.md
Grouped JSON exposes unpriced turn and model fields. Tests and CLI snapshots cover retired-model usage, zero cost, warnings, and override pricing.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

Poem

A rabbit checks the model list,
Retired rates no longer twist.
Local prices can now return,
Summaries count each unpriced turn.
Zero costs stand clear and bright—
Safe accounting through the night.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR satisfies #524 and override loading for #509, but it does not show the required models.dev.json refresh for claude-opus-5 and claude-sonnet-5. Update the vendored models.dev.json snapshot to include current first-party pricing for claude-opus-5 and claude-sonnet-5, with regression coverage.
Out of Scope Changes check ⚠️ Warning The two state-status snapshot schema updates are unrelated to the linked pricing and override objectives. Remove the unrelated state-status fixture changes or link them to the schema-version change.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: preventing retired models from inheriting reseller pricing.
Description check ✅ Passed The description directly explains the pricing fix, override loading, ownership catalog, reporting changes, and related tests.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-524-reseller-precedence

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39617925da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/relayburn-sdk/src/analyze/pricing.rs
Comment thread crates/relayburn-sdk/src/analyze/pricing.rs

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread crates/relayburn-sdk/data/primary-model-ids.json
Comment thread crates/relayburn-sdk/src/analyze/pricing.rs
Comment thread crates/relayburn-cli/src/commands/summary/mod.rs Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 13 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/relayburn-sdk/src/query_verbs/summary/mod.rs">

<violation number="1" location="crates/relayburn-sdk/src/query_verbs/summary/mod.rs:624">
P2: Human `summary --bucket` output still silently presents unpriced buckets as `$0`. Include an unpriced-turn indicator or warning in the time-series renderer when `bucket.unpriced_turns > 0`, matching the grouped-summary warning.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread crates/relayburn-sdk/src/query_verbs/summary/mod.rs
Comment thread crates/relayburn-sdk/src/query_verbs/tests.rs Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 8: Update the changelog entry to state that protected first-party models,
including codex-auto-review, are treated as unpriced rather than inheriting any
reseller tariff, regardless of cache-rate completeness; retain the explicit
models.dev.json pricing override and keep the wording concise and impact-first.

In `@tests/fixtures/cli-golden/snapshots/summary.stderr.txt`:
- Around line 1-2: Update the warning text generated by load_pricing_for_ledger
to reference the active ledger home rather than RELAYBURN_HOME, instructing
users to add the override at the resolved ledger directory’s models.dev.json
path. Refresh the affected CLI golden snapshot accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 73ff9e44-46b4-402a-b1e7-40b0b1855380

📥 Commits

Reviewing files that changed from the base of the PR and between 5b531d0 and 08fb206.

📒 Files selected for processing (19)
  • CHANGELOG.md
  • crates/relayburn-cli/src/commands/summary/json.rs
  • crates/relayburn-cli/src/commands/summary/mod.rs
  • crates/relayburn-sdk/data/primary-model-ids.json
  • crates/relayburn-sdk/src/analyze/pricing.rs
  • crates/relayburn-sdk/src/query_verbs/compare.rs
  • crates/relayburn-sdk/src/query_verbs/flow.rs
  • crates/relayburn-sdk/src/query_verbs/hotspots.rs
  • crates/relayburn-sdk/src/query_verbs/mod.rs
  • crates/relayburn-sdk/src/query_verbs/overhead.rs
  • crates/relayburn-sdk/src/query_verbs/sessions.rs
  • crates/relayburn-sdk/src/query_verbs/summary/mod.rs
  • crates/relayburn-sdk/src/query_verbs/tests.rs
  • scripts/update-pricing.mjs
  • tests/fixtures/cli-golden/snapshots/state-status-json.stdout.txt
  • tests/fixtures/cli-golden/snapshots/state-status.stdout.txt
  • tests/fixtures/cli-golden/snapshots/summary-json.stdout.txt
  • tests/fixtures/cli-golden/snapshots/summary.stderr.txt
  • tests/fixtures/cli-golden/snapshots/summary.stdout.txt

Comment thread CHANGELOG.md Outdated
Comment thread tests/fixtures/cli-golden/snapshots/summary.stderr.txt Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant