Repository navigation
[PR] fix(#5199): network.throttle429Scope — model-scoped 429 holds for marker-less gateway bodies - #5889
Merged
Conversation
…less gateway 429s Implements #5853 decision 1 option A (recommended); decision 2 unchanged (streak-1 arming kept). Default "credential" preserves #5696 semantics — zero behavior change without opt-in. - knobs: BILI_THROTTLE_429_SCOPE > network.throttle429Scope > "credential", read per settlement (hot flip) - upstream-throttle: throttle429ModelScope() — recognized KeyRing/bili-relay markers win as before; scope=model settles marker-less 429s (RPM/quota text, opaque relay rewrites) on the request model; model-less sends stay credential-wide - relay/loop/degenerate-retry lanes settle through the wrapper - docs: CONFIGURATION en/zh + website seed regenerated, LIFECYCLE env row - tests: tests/issue5199-throttle429-scope.test.ts (6 rows, both directions)
Contributor
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8519-pr.5889.20eee93This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5889.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Owner
Author
CI: the 9 failing
|
Contributor
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1075-master.6945a35 # this exact build
npm install -g billion-context@master # always the newest merged build
|
This was referenced Nov 7, 2026
Louie0806
pushed a commit
to Louie0806/billion-context
that referenced
this pull request
Nov 8, 2026
…-value guard, media-refs carry-forward filter & cloud prefix families, oversize refusal messages that name the real fix, dsh durable lines with readable numbers, website streamKeepAliveMs typo, 5755 ord-crossing guard kept with regression tests - upstream-throttle: a saturated hold map (64/64 account-wide, no model-scoped hold to evict) silently dropped the next model-scoped 429 — now it warns once and the send is documented as ungoverned (ranxianglei#5199/ranxianglei#5889) - knobs: network.throttle429Scope matches its value EXACTLY; a typo now warns once and falls back to credential instead of silently doing so (ranxianglei#5199) - kernel media-refs: block summaries written by older builds can carry refs looksEphemeralSignedUrl would reject today; the carry-forward into merged blocks now applies the same filter so a dead signed URL cannot survive forever (ranxianglei#5865 review). EPHEMERAL_QUERY_PARAM also gains the Huawei OBS x-obs-*, Tencent COS x-cos-* and Baidu BCE x-bce-* prefix families. - native-intercept/dsh-native oversize refusals now say what to DO (restart host / fix the model window declaration), and dsh durable lines format numbers with thousands separators (ranxianglei#2924/ranxianglei#4068) - website: streamKeepaliveMs → streamKeepAliveMs (the real knob name) across main.js, docs/index.html, config-reference/process.yaml - CONFIGURATION.md/.zh-CN.md: throttle429Scope exact-match + model-string aliasing caveat documented - store/persist: ranxianglei#5755 refs ord-crossing guard (touching rows stay in the old-refs set until the new-refs commit lands) kept as defense-in-depth with a regression test kernel-change: 0.0.186 → 0.0.187 (media-refs carry-forward filter + cloud prefix families; compress.ts starvation-bound comment corrected)
This was referenced Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements #5853 decision 1 = option A (recommended) and closes #5199: a new knob
network.throttle429Scope: "credential" | "model"(default "credential", zero behavior change) lets deployments whose gateway fronts many independently-limited models behind one credential (LiteLLM / one-api / opencode.ai-style relays) opt into model-scoped 429 holds for marker-less bodies — the gap PR #5696 deliberately left (it scopes only thekeyring_model_temporarily_unavailablemarker and bili-relaybili_throttle_holdscope=model markers).#5853 decision 2 = option A: streak-1 arming unchanged — no code change.
Field evidence for the marker-less shape (both in #5199):
{"error":{"code":null,"message":"qnaigc: rate limit reached for RPM (request_id: …)"}}— one model's RPM pool froze every model on the credential; 10-min window: 5 real 429s, 49 local 429 rejects.{"error":{"message":"[prefill_cap_refused] the cell is at capacity; retry shortly (after 30 s)"}}— opencode.ai lane: a big session's prefill-cap 429 froze a fresh session's title-gen sidecar.How
src/knobs.ts:throttle429Scope()—BILI_THROTTLE_429_SCOPE>network.throttle429Scope> "credential"; exact"model"match, everything else falls back to "credential"; read per settlement so it flips live.src/upstream-throttle.ts:throttle429ModelScope(status, requestModel, responseBody)— recognized KeyRing/bili-relay markers win exactly as before; otherwise scope=model + identifiable request model → the request model; model-less or non-429 → legacy credential key. Worst-case protection never widens beyond legacy.src/server/relay.ts(main lane),src/loop/core.ts(replay lane),src/degenerate-retry.ts(provisional settler). Governor internals (gate/peek/resolved/probe ownership) already threadrequestModelpost-[PR] fix: isolate gateway model-scoped 429 holds #5696 — a model-scoped hold skips sibling-model and model-less sends by construction, and only the matching model's resolved response clears it.config surface
network.throttle429Scope("credential" | "model", default "credential"; envBILI_THROTTLE_429_SCOPE). Extends the existingnetwork.throttle429*family (enabled / maxConsecutive / baseDelayMs / maxDelayMs), same env-naming convention, hot-reloaded.Tests
tests/issue5199-throttle429-scope.test.ts— 6 rows, both directions per gate discipline: knob read (live flip, exact match, empty/invalid → credential); default mode keeps marker-less bodies credential-wide (#5696 row) while the recognized marker still grants model scope; scope=model settles RPM/prefill-cap/malformed bodies on the request model; governor integration — exhausted model held, sibling + model-less sends eligible, sibling 200 doesn't clear, matching 200 clears; model-less 429 stays credential-wide (storm protection intact); mid-flight scope flip changes only future settlements.Existing suites pass unchanged (default behavior identical): issue4204 (61 tests incl. 4725/4743/5795/upstream-error-log lanes), typecheck clean.
Docs
CONFIGURATION.md / CONFIGURATION.zh-CN.md + website seed (
process.yaml) regenerated viatools/gen-config-docs.mjs generate(coverage check green); LIFECYCLE.md env-table row extended.No kernel/ or pi-subagents/ changes.