Repository navigation
Description
Description
When the client sends a shrunken output budget (max_tokens: 1), bili logs that it restored the budget, but the restored value never reaches the upstream: the forwarded body still carries max_tokens: 1. The upstream then streams a single token, and the client reports "output token limit reached, answer truncated".
Repro:
- dsh (DeepSeek Harness) sends a turn on a large session — 2487 messages in,
max_tokens: 1(shrunk from its own raw-history estimate). - bili logs
output budget restored 1 -> 64000 (#546 …). - The forwarded body (captured with
diagnostics.dumpBody) still hasmax_tokens: 1and nomax_completion_tokens. - The upstream records
completion_tokens = 1for that request; every subsequent turn repeats the same way.
Expected: whatever restoreOutputBudget() decides is what gets forwarded.
bili version
0.1.214 (plugin native, embedded in dsh)
Usage mode
Plugin native (bili runs inside the dsh profile)
Client and version
dsh 0.2.2-rc.3 (DeepSeek Harness)
Operating system
Linux — Ubuntu-based container on a self-hosted NAS
Upstream and model
Self-hosted new-api v1.0.0-rc.48 (one-api fork) → SGLang serving DeepSeek-Flash, declared in bili as { "context": 512000, "output": 32768 }
Config
{
"providers": {
"https://<host>:23000": {
"models": { "Deepseek-Flash": { "context": 512000, "output": 32768 } }
}
}
}Log excerpt
18:54:28 [info] [session-…] output budget restored 1 -> 64000 (#546: client shrank it from its raw-history estimate; high-water 64000 below known output ceiling — floored (#1665/#1840))
18:57:12 [info] [session-…] output budget restored 1 -> 64000 (#546: …)
18:57:13 [info] [sess=session-…] forward POST → https://<host>:23000/v1/chat/completions
The body that was actually forwarded at 18:57:13:
keys: model,messages,stream,stream_options,store,max_tokens,tools,thinking,reasoning_effort
max_tokens: 1
max_completion_tokens: (absent)
messages: 270 (folded from 2487)
tools: 73
reasoning_effort: "max"
Where it looks like it goes wrong
Byte offsets in the published 0.1.214 dist/index.js (single-line bundle):
| symbol | offset |
|---|---|
wireObj=parsed |
5,660,237 |
const serialized=JSON.stringify(wireObj); wireBody=serialized; |
5,660,463 |
commitGraph() — all four call sites |
5,661,400 … 5,664,339 |
restoreOutputBudget(parsed, …) — the only call site |
5,777,288 |
So the request body is serialized into the wireBody string, and every commitGraph() runs, before restoreOutputBudget() is called. restoreOutputBudget() mutates the parsed object, but nothing re-serializes it into wireBody afterwards — and wireBody is what is forwarded. The log2("info", … restored …) line is emitted regardless, so the failure is invisible.
This is consistent with [compat] dropFields working, since that path explicitly handles both representations: const graph = separatorManifest !== void 0 ? wireGraph() : null; → dropCompatFieldsJson(graph) + commitGraph(), otherwise applyCompatDropFields(wireBody).
Possibly related: the restore lands after the last commit, so even a commitGraph() there would need the mutation to target the same object the graph holds.
Workaround
"compat": { "dropFields": ["max_tokens", "max_completion_tokens"] }plus a restart. The upstream then applies its own default output limit and completion_tokens immediately returned to normal (649 / 1183 / 355 / 117 across the next 8 requests, end_reason: done). The cost is that bili's own output-budget management is bypassed for that route.
(Side note, not filed separately: after saving compat.dropFields on a running instance there was no [compat] dropped line and no change in the forwarded body until the process restarted. We did not try POST /__bili/config/reload — it may well be the intended path; if so, a pointer in the docs would have saved a restart cycle.)
Activity
test: pin the output-budget re-export identity in the side-lane suite (…
fix: carry the restored output budget across the watermark rebind (ra…
test: pin the output-budget re-export identity in the side-lane suite (…
SAXEM1997 commentedon Nov 17, 2026
Confirming on 0.1.215 (installed today): the same code shape is still there, so this is not fixed in the release.
Byte offsets in the published
0.1.215dist/index.js(single-line bundle, 6,263,710 bytes):wireBody=assignmentwireBody=assignmentcommitGraph()commitGraph()restoreOutputBudget(parsed, …)— only call sitewireBodyreads after the restoreSame conclusion as 0.1.214: every write to
wireBodyand everycommitGraph()happens ~113 KB beforerestoreOutputBudget()runs, and afterwardswireBodyis only ever read. The restored value still cannot reach the forwarded body.I can't observe the failure end-to-end on 0.1.215 because the workaround is still active on this instance, but the path is unchanged.
Two PRs reference this issue — #6766 and #6827. Happy to re-test and drop the workaround as soon as either lands.
Minor nit: when the budget field is listed in
compat.dropFields, theoutput budget restored 1 -> 64000line is still logged even though the field is dropped moments later. That line sent me looking at the wrong layer for a while — suppressing it (or noting the drop) would make the log honest.ranxianglei commentedon Nov 17, 2026
[bot] 🏷 Thank you for the 0.1.215 confirmation and the precise byte-offset table — that pins the persistence of the defect across the release.
Root cause (clarified by PR #6766): the issue title says "mutates the body after it is serialized," but the actual mechanism is different. On a watermark fastpath hit,
prepareOpenaire-parses every scalar field (includingmax_tokens) from the raw wire text, silently discarding the restored budget thatrestoreOutputBudgethad written into theparsedobject. The log fires at restore time, before the rebind, so it reports success while the wire still carries the client's shrunk value. Your byte-offset analysis correctly identified that the restored value never reaches the forwarded body — the rebind is the producer, not a serialization-order bug.Status of the two PRs:
fix: carry the restored output budget across the watermark rebind) — open, mergeable. Snapshots the restored budget before the rebind and writes it back after. Includes a hermetic A/B regression test (rollout off vs on) that pins the fix.wave-11 batch: 9 community fixes) — open, mergeable. Includes [PR] fix: carry the restored output budget across the watermark rebind (#6752) #6766 as one of the 9 fixes, merged as a real two-parent merge commit.Either landing will unblock your re-test. #6766 is the targeted fix; #6827 is the batch vehicle.
On the log-honesty nit (dropFields): valid observation. The
output budget restored 1 -> 64000line atside-request.ts:871fires unconditionally at restore time; thecompat.dropFieldsdrop happens later inrelay.ts:683–709and logs its own line ([compat] dropped 1 field(s) per compat.dropFields (max_tokens)). Both lines appear, but the first one is misleading when the field is going to be dropped. I'll add a note to the drop log in relay.ts so it explicitly calls out that it overrides the earlier restore — a one-line change I'll fold into #6766's branch.fix: note the output-budget override in the compat.dropFields log (ra…
[-]Output-budget restore mutates the body after it is serialized, so the restored max_tokens never reaches upstream while the log reports success[/-][+]Output-budget restore is discarded by the watermark fastpath rebind, so the restored max_tokens never reaches upstream while the log reports success[/+]SAXEM1997 commentedon Nov 17, 2026
Thanks for the correction — the watermark fastpath rebind explains what my offsets only showed as a symptom. I've renamed the issue to match the real mechanism, so anyone searching later lands on the right path:
Keeping
compat.dropFieldsin place until #6766 (or the #6827 batch) lands; I'll re-test and drop it as soon as a release with the fix is out, and report back either way.ranxianglei commentedon Nov 17, 2026
[bot] 🏷 Good — the title now names the mechanism, which will save the next reader the same offset-hunt you went through. The log-honesty fix is already on #6766's branch (commit
1fcc31d114), so the drop log will be honest in the same release that fixes the rebind. Standing by for your re-test result.fix: carry the restored output budget across the watermark rebind (#6752
test: pin the output-budget re-export identity in the side-lane suite (…
fix: carry the restored output budget across the watermark rebind (#6752