feat: add UTF-8 byte billing mode - #6598
Open
twodogegg wants to merge 1 commit into
Open
Conversation
Contributor
WalkthroughThe PR adds ChangesUTF-8 byte billing
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Relay
participant EstimateRequestToken
participant calculateTextQuotaSummary
participant UsageLog
Relay->>EstimateRequestToken: Estimate UTF-8 input bytes
EstimateRequestToken-->>Relay: Return prompt byte count
Relay->>calculateTextQuotaSummary: Settle local byte usage
calculateTextQuotaSummary->>UsageLog: Record UTF-8 billing path
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Warning |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
web/src/features/usage-logs/components/dialogs/details-dialog.tsx (1)
228-267: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winExclude non-input billing rows for UTF-8 byte logs.
isUTF8Bytesdoes not guard the cache, audio, and image row conditions at Lines 302-362. If generic ratio fields exist in a UTF-8 log, the dialog shows prices that this billing mode does not apply. Add!isUTF8Bytesto both enclosing conditions.Proposed fix
- if (!isTieredExpr && isClaude && hasAnyCacheTokens(other)) { + if (!isTieredExpr && !isUTF8Bytes && isClaude && hasAnyCacheTokens(other)) { ... - if (!isTieredExpr) { + if (!isTieredExpr && !isUTF8Bytes) {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/features/usage-logs/components/dialogs/details-dialog.tsx` around lines 228 - 267, Update the cache, audio, and image row conditions in the details dialog to also require !isUTF8Bytes, preventing generic ratio-based billing rows from appearing for UTF-8 byte logs while preserving the existing input row.web/src/features/pricing/components/pricing-columns.tsx (1)
178-219: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winHide cached pricing for UTF-8 byte models.
The new branch hides output pricing, but the
cached_pricecell still formatscache_ratiofor UTF-8 models. This conflicts withmodel-card.tsxandmodel-details.tsx, which suppress unsupported cache pricing. Add anisUTF8BytesModel(model)check to the cached-price empty condition.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/features/pricing/components/pricing-columns.tsx` around lines 178 - 219, Update the cached-price cell’s empty-condition in the pricing column renderer to also treat isUTF8BytesModel(model) as unsupported. Preserve the existing cache_ratio formatting for models that support cached pricing, while returning the same empty display used by the other unsupported cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@service/token_counter.go`:
- Around line 185-201: Move the RelayFormatOpenAIRealtime zero-return check
before the BillingModeUTF8Bytes branch in the token-counting flow. Update the
surrounding logic in the relevant token-counting function so realtime requests
return zero without billing based on the empty or non-representative
meta.CombineText, while preserving existing behavior for other relay formats.
In `@web/src/features/system-settings/models/model-pricing-snapshots.ts`:
- Around line 83-85: Update getModeLabel so the “UTF-8 bytes” label is produced
through the existing i18next translation mechanism by accepting and using a t
function, or translate it at every render call site. Keep the existing
mode-label behavior unchanged for other values and ensure no user-facing English
string bypasses localization.
---
Outside diff comments:
In `@web/src/features/pricing/components/pricing-columns.tsx`:
- Around line 178-219: Update the cached-price cell’s empty-condition in the
pricing column renderer to also treat isUTF8BytesModel(model) as unsupported.
Preserve the existing cache_ratio formatting for models that support cached
pricing, while returning the same empty display used by the other unsupported
cases.
In `@web/src/features/usage-logs/components/dialogs/details-dialog.tsx`:
- Around line 228-267: Update the cache, audio, and image row conditions in the
details dialog to also require !isUTF8Bytes, preventing generic ratio-based
billing rows from appearing for UTF-8 byte logs while preserving the existing
input row.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4f32289a-0aa4-4689-b1bf-94891ed3b48c
📒 Files selected for processing (31)
controller/model_list_test.gocontroller/ratio_sync.gocontroller/relay.gomodel/pricing.gorelay/common/relay_info.gorelay/helper/price.gorelay/helper/price_test.goservice/text_quota.goservice/token_counter.goservice/utf8_bytes_billing_test.gosetting/billing_setting/tiered_billing.goweb/src/features/pricing/components/model-billing-mode-badge.tsxweb/src/features/pricing/components/model-card.tsxweb/src/features/pricing/components/model-details.tsxweb/src/features/pricing/components/pricing-columns.tsxweb/src/features/pricing/lib/model-helpers.tsweb/src/features/pricing/types.tsweb/src/features/system-settings/models/__tests__/utf8-bytes-pricing.test.tsweb/src/features/system-settings/models/model-pricing-core.tsweb/src/features/system-settings/models/model-pricing-sheet.tsxweb/src/features/system-settings/models/model-pricing-snapshots.tsweb/src/features/system-settings/models/model-ratio-visual-editor.tsxweb/src/features/usage-logs/components/columns/common-logs-columns.tsxweb/src/features/usage-logs/components/dialogs/details-dialog.tsxweb/src/i18n/locales/en.jsonweb/src/i18n/locales/fr.jsonweb/src/i18n/locales/ja.jsonweb/src/i18n/locales/ru.jsonweb/src/i18n/locales/vi.jsonweb/src/i18n/locales/zh-TW.jsonweb/src/i18n/locales/zh.json
Comment on lines
+185
to
201
| if info.BillingMode == "" { | ||
| info.BillingMode = billing_setting.GetBillingMode(info.OriginModelName) | ||
| } | ||
| if info.BillingMode == billing_setting.BillingModeUTF8Bytes { | ||
| count := len(meta.CombineText) | ||
| common.SetContextKey(c, constant.ContextKeyPromptTokens, count) | ||
| return count, nil | ||
| } | ||
|
|
||
| // 是否统计token | ||
| if !constant.CountToken { | ||
| return 0, nil | ||
| } | ||
|
|
||
| if meta == nil { | ||
| return 0, errors.New("token count meta is nil") | ||
| } | ||
|
|
||
| if info.RelayFormat == types.RelayFormatOpenAIRealtime { | ||
| return 0, nil | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check how GetTokenCountMeta is implemented for realtime/openai-realtime request types.
rg -n -C5 'func.*GetTokenCountMeta' --type=go
rg -n -C5 'RelayFormatOpenAIRealtime' --type=go relayRepository: QuantumNous/new-api
Length of output: 157
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== git status/stat =="
git status --short
git diff --stat || true
echo "== locate relevant files =="
fd -a 'token_counter.go|relay.go|types|openai.*realtime|billing_setting' . | sed 's#^\./##' | head -200
echo "== search key identifiers =="
rg -n -C4 'GetTokenCountMeta|RelayFormatOpenAIRealtime|BillingModeUTF8Bytes|CombineText|CountToken' . --glob '*.go' --glob '*.ts' --glob '*.tsx' | head -300Repository: QuantumNous/new-api
Length of output: 20999
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== service/token_counter.go outline =="
ast-grep outline service/token_counter.go --view compact || true
echo "== service/token_counter.go relevant section =="
sed -n '160,230p' service/token_counter.go
echo "== billing_setting UTF8 bytes definitions/usages =="
rg -n -C4 'BillingModeUTF8Bytes|BilingMode|BillingMode' setting --glob '*.go' | head -200
fd -a '*.go' setting | xargs -r rg -n 'BillingModeUTF8Bytes|AddBillingMode|GetBillingMode|BillingMode' | head -200
echo "== relay valid request relevant section =="
sed -n '35,60p' relay/helper/valid_request.go
echo "== relay pricing relevant section =="
sed -n '80,115p' relay/helper/price.goRepository: QuantumNous/new-api
Length of output: 7023
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== GetAndValidAudioRequest implementation =="
rg -n -C8 'GetAndValidAudioRequest|Build.*Audio' --glob '*.go' | head -220
echo "== realtime/audio token count metadata in OpenAI relay files =="
fd -a 'openai.*realtime|openai.*audio|audio' relaykit relay controller | sed 's#^\./##'
for f in $(fd -a 'openai.*realtime|openai.*audio|audio' relaykit relay controller); do
echo "--- $f"
rg -n -C6 'GetTokenCountMeta|Input|CombineText|openai_realtime|OpenAIRealtime|Audio' "$f" || true
done
echo "== pricing helper pre-consumption context =="
rg -n -C8 'SetEstimatePromptTokens|GetEstimatePromptTokens|PreConsumedTokens|promptTokens|BillingModeUTF8Bytes|RelayFormatOpenAIRealtime' relay --glob '*.go' | head -260
echo "== token_counter EstimateRequestToken continuation =="
sed -n '185,270p' service/token_counter.goRepository: QuantumNous/new-api
Length of output: 202
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files mentioning token count meta for realtime/audio =="
rg -n -C6 'RelayFormatOpenAIRealtime|RelayFormatOpenAIAudio|GetTokenCountMeta|CombineText|SetEstimatePromptTokens|GetEstimatePromptTokens' relay --glob '*.go' | head -400
echo "== list relevant relay dto types =="
fd -a 'openai.*realtime|realtime|openai.*audio|audio' relaykit relaykit/dto relay --print0 | xargs -0 -r -I{} sh -c 'echo "--- {}"; wc -l "{}" | sed "s#$# file:"; sed -n "1,220p" "{}"'
echo "== price go relevant =="
sed -n '85,120p' relay/helper/price.goRepository: QuantumNous/new-api
Length of output: 50375
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== realtime DTO real file =="
sed -n '1,220p' relaykit/dto/realtime.go
echo "== realtime implementation relevant =="
sed -n '1,250p' relay/channel/openai/relay_realtime.go
echo "== pricing preConsume context =="
rg -n 'func preConsumeUsage|preConsumeUsage\(' relay --glob '*.go' -C8
echo "== realtime price/consume helpers =="
rg -n 'RealtimeUsage|audioToken|textToken|input_tokens|output_tokens|InputTokenDetails|Post.*Consume|quota' relay/helper relay channel/relay --glob '*.go' -C5 | head -400
echo "== relevant token counter section exact =="
sed -n '175,205p' service/token_counter.goRepository: QuantumNous/new-api
Length of output: 41529
Preserve the realtime zero pre-consumer before UTF-8 byte billing.
RelayFormatOpenAIRealtime uses websocket usage accounting through preConsumeUsage; realtime request validation also creates a BaseRequest, so meta.CombineText is empty or not representative. Keep the info.RelayFormat == types.RelayFormatOpenAIRealtime zero-return before the BillingModeUTF8Bytes path, or guard it as a non-billable realtime case.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@service/token_counter.go` around lines 185 - 201, Move the
RelayFormatOpenAIRealtime zero-return check before the BillingModeUTF8Bytes
branch in the token-counting flow. Update the surrounding logic in the relevant
token-counting function so realtime requests return zero without billing based
on the empty or non-representative meta.CombineText, while preserving existing
behavior for other relay formats.
Comment on lines
83
to
+85
| export const getModeLabel = (mode?: string) => { | ||
| if (mode === 'per-request') return 'Per-request' | ||
| if (mode === 'utf8_bytes') return 'UTF-8 bytes' |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Localize the UTF-8 billing-mode label.
Line 85 adds a user-facing English string without translation. Pass t into getModeLabel, or translate this value at its render call site.
As per coding guidelines, “Frontend user-facing text must use i18next/react-i18next.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/src/features/system-settings/models/model-pricing-snapshots.ts` around
lines 83 - 85, Update getModeLabel so the “UTF-8 bytes” label is produced
through the existing i18next translation mechanism by accepting and using a t
function, or translate it at every render call site. Keep the existing
mode-label behavior unchanged for other values and ensure no user-facing English
string bypasses localization.
Source: Coding guidelines
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
新增通用的
utf8_bytes计费模式：请求用量按输入文本的 UTF-8 字节数统计，价格仍按每 100 万单位配置。预扣费与结算使用请求级计费模式快照，不受请求期间配置热更新影响，也不会混入输出、缓存、图片或音频 token 用量。管理后台新增 UTF-8 bytes 定价选项；模型广场和使用日志会显示对应计费单位，并继续使用现有的统一货币换算。
$15 / 1M UTF-8 bytes对应现有ModelRatio配置值7.5。本 PR 由 AI 辅助实现，提交者已检查变更范围和测试结果。
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
make testGOWORK=off go vet ./...GOWORK=off go build -buildvcs=false ./...cd relaykit && GOWORK=off go vet ./... && GOWORK=off go build ./...cd web && bun test src/features/system-settings/models/__tests__/utf8-bytes-pricing.test.tscd web && bun run typecheckcd web && bun run build && bun run i18n:syncs2.1-pro在模型广场显示$15 / 1M UTF-8 字节，详情中不显示输出 token 价格Summary by CodeRabbit
New Features
Bug Fixes