Skip to content

[Bug][dsh] 20k-message / 81 MiB-inbound session: host aborted 6× in 2 h — the #3050 resident-AST OOM is not bounded by sessions.hardMaxBytes (Windows, 0.1.210) #5403

Open

Description

@xiaweiliang060035

Duplicate check performed first. I searched open+closed issues before filing. This is not a duplicate of #5138 (that is event-loop blocking / port-hold / twin-collision on 300 MB sessions — I hit none of those), nor of #5392 (fold drift — my resent history drifted and NO persisted blocks counts are both 0). It is a new data point for #3050 / #4967 / #5190: the resident-AST OOM at a session size roughly 4× the size described in #4967, where the configured per-session hard cap provably does not bound it. I filed it because #5190 shows the #4967 fix is blocked on a design decision, and this field data bears on that decision.

TL;DR

On DSH desktop (Windows), one long-lived session caused six host aborts in under three hours (exit 134, FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory). The crash interval kept shrinking: 57 min → 64 min → 8 min → 39 min → 10 min.

The affected session is 20,037 messages / 81.3 MiB inbound per turn. Its resident estimate sits at ~838 MB against a 512 MB sessions.hardMaxBytes, and the discharge path reports success while the estimate never decreases — it walks upward monotonically (837,363,567 → 838,036,463 B). Every attempt to bound it produces an evict → cold-reload cycle instead: 202 evictions / 217 cold reloads / 24 hard-cap warnings in one day.

Environment

Item Value
billion-context 0.1.210 (latest; npm registry check confirmed up to date)
Host @deepseek-ai/dsh 0.2.1-rc.2, desktop bundle @deepseek-ai/dsh-desktop 0.2.1-rc.3
Lane bili-native plugin lane (dsh-client / native-client)
Runtime Electron 44.7.0, Node 24.21.0 (ELECTRON_RUN_AS_NODE), win32 x64, 16 GB RAM
Storage sqlite (sessions.backend default)
Relevant config all at defaults: sessions.residentMaxBytes = sessions.hardMaxBytes = 536,870,912 (512 MB), sessions.warmMax = 0, compress.* default

The startup compatibility warning appears on every run and is noted here for completeness rather than as the claimed cause:

@deepseek-ai/dsh v0.2.1-rc.2 — NEWER than the newest build billion-context has tested against (v0.2.0-rc.2) … if you hit compression conflicts or lose history, roll the client back to v0.2.0-rc.2.

I do not think version drift is the root cause, because the failure reproduces exactly the #4967 mechanism at a larger scale, and the hard cap fails to bound it on the same build throughout.

The session

Metric Value
Session session-eb61c255-39fe-454d-8449-1cf2145be966
Messages per turn 20,037 → 20,071 over the reporting window
Inbound per turn 81.2 → 81.3 MiB
Outbound per turn ~1.0 MiB
Session file 95 MB session.v4.jsonl.zstd → 316 MB / ~57,300 lines decompressed
bili snapshot 224.9 MB (sessions/openai/127.0.0.1_acaa….json)
Fold state 28,120 ref(s), 739 block(s) (50 active), baseline 257,387 tok, compressions 739
System prompt 463,075 → 485,739 chars (see note below)

The .zstd file grows on every crash: 94,900,942 B (11:40) → 94,957,918 B (12:19) → larger again at 12:30.

System-prompt note. Four fold-reconcile lines in this window show the system prompt itself oscillating across ~46–49 KB:

10:56:11  system prompt changed (463075 → 431888 chars); fold state unaffected (#1921)
10:57:39  system prompt changed (431888 → 485739 chars)
11:15:59  system prompt changed (485739 → 479204 chars)
11:24:45  system prompt changed (479204 → 481771 chars)

These are not drift events (no drifted line), but a ~480 KB system prompt is a meaningful share of an 81 MiB turn and may be relevant to why this session crosses the ceiling earlier than #4967's 11 MiB example. Flagging as an observation, not a claim.

Symptom 1 — the hard cap fires and reclaims nothing

[info]  [session] evicted session-eb61c255-… from resident pool
        (weight 838036463 bytes, 0 session(s) remain, budget 536870912 bytes)
[warn]  [persist] session-eb61c255-…: resident estimate 838036463 B exceeds the
        per-session hard cap 536870912 B (#4788 D1-A) — discharging droppable fields
        back to disk now, including pool-exempt sessions;
        remainder (state/transient) is non-droppable

Immediately followed by:

[info]  [persist] cold-loaded session-eb61c255-… from legacy file:
        28120 ref(s), 739 block(s) (50 active), baseline 257387 tok,
        compressions 739, savedAt ~0 min ago

The estimate does not fall after discharge. Successive warnings, all above the cap and rising:

837,582,636   (+286.78 MB over cap)
837,667,225   (+286.86 MB)
837,715,454   (+286.91 MB)
837,766,853   (+286.96 MB)
837,843,432   (+287.03 MB)
837,925,586   (+287.11 MB)
837,989,305   (+287.17 MB)
838,036,463   (+287.21 MB)

remainder (state/transient) is non-droppable is doing all the work here: if state/transient alone can exceed the hard cap, the cap bounds nothing, and the discharge message ("discharging droppable fields back to disk now") is misleading to an operator reading the log.

Symptom 2 — evict/cold-reload churn, 212 cycles/day

One day, one session, one machine:

Counter Count
evicted … from resident pool 202
cold-loaded … from legacy file 217
exceeds the per-session hard cap 24

Back-to-back cycles at ~40 s spacing:

12:12:16  cold-loaded  (28072 refs, baseline 243416 tok)
12:13:31  evicted      (weight 837582636)
12:13:33  cold-loaded  (28078 refs, baseline 244562 tok)
12:14:08  evicted      (weight 837629957)
12:14:10  cold-loaded  (28081 refs, baseline 245424 tok)
12:14:45  cold-loaded  (again)

That is ~212 full decompress-and-rebuild passes over a 316 MB history in a single day. This is the same churn family reported in #5138, but on my machine it terminates in a host abort rather than an event-loop wedge — no port hold, no twin-collision, no database is locked.

Symptom 3 — per-turn cost is already O(messages)

[info] request: 20068 msgs, inbound=81.3MiB, outbound=1012.7KiB, local=5630ms
[warn] slow-session: 20068 msgs / inbound=81.3MiB took local=5630ms — per-turn O(messages) synchronous

local= on effectively identical payloads: 5.3 s, 5.5 s, 5.6 s, 6.5 s, 6.6 s, 7.5 s, 14.4 s, 15.2 s, 16.0 s, 17.4 s, 19.1 s, 48.4 s. The 48.4 s outlier coincides with the host reaching ~4.0 GB RSS just before an abort.

Symptom 4 — host abort

[13880:…]  408254 ms: Mark-Compact (reduce) 1418.8 (1534.3) -> 1418.4 (1428.6) MB … last resort; GC in old space requested
[13880:…]  408835 ms: Mark-Compact (reduce) 1418.4 (1428.6) -> 1417.7 (1426.8) MB … last resort; GC in old space requested
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

All six aborts, with the heap level at which each tripped:

# Time (UTC) Heap at abort Interval since previous
1 2026-11-01T01:30:46Z 1605.8 (1644.0) MB —
2 2026-11-01T02:28:05Z 1604.9 (1622.2) MB 57 min
3 2026-11-01T03:32:31Z 2589.4 (2219.5) MB 64 min
4 2026-11-01T03:40:40Z 1623.4 (1647.2) MB 8 min
5 2026-11-01T04:19:38Z 2212.9 (2237.2) MB 39 min
6 2026-11-01T04:30:04Z 1417.7 (1426.8) MB 10 min

Native stack includes Cr_z_deflateEnd — the zstd/inflate path that loads the session file. bili's own watcher confirms the host death from outside:

[watchdog] kill-0 probe failed for pid 13880 (code=ESRCH) — requiring 3 consecutive failures before believing it (#3890)
[watchdog] pid 13880 dead confirmed after 3 consecutive failed kill-0 probes (last code=ESRCH) — removing
[parent-gone (pid 13880) received — flushing sessions…

Restarting the client only helps until the session is opened: the frontend auto-restores it (dsh.sessions.current), bili cold-loads it, and the heap climbs again from ~2.6 GB within ~60 s. Measured after one restart, before any user interaction:

12:13:33  host=2643MB   bili child=1662MB
12:13:59  host=3134MB   bili child=1744MB
12:14:11  host=3225MB   bili child=1991MB
12:14:36  host=3428MB   bili child=2041MB

How I read the cause

The evidence points at the #4967 mechanism rather than at compression ratios: session.lastMessages keeps the multi-megabyte unpruned AST resident across turns, the #3050 R3 ②a fold-on-save path does not fire often enough at this churn rate, and the hardMaxBytes discharge path cannot reduce what remains (state/transient is non-droppable). #4967 describes exactly this with a 5,000-message / ~11 MiB transcript; my case is ~20,000 messages / 81 MiB, and it produces six aborts in 2 h with the cap reporting as breached-but-unenforced throughout.

I am not claiming a specific code line is wrong — I have not read the source. I am reporting that the observable contract of sessions.hardMaxBytes ("constrains a single session", per CONFIGURATION.zh-CN.md line 206) is not met in this configuration, and that the failure mode is a silent evict/reload loop rather than a bounded stop.

Relation to existing issues

Issue Same as this? Why / why not
#3050 / #4967 / #5190 Same mechanism, larger scale Resident unpruned AST → heap exhaustion. #4967 was measured at 5k msgs / 11 MiB; this is 20k / 81 MiB. #5190 shows the fix is blocked on a design choice (A vs C). This report is evidence that the failure is already in the wild at 4× that size, on Windows, and repeatedly.
#5138 Adjacent, different terminal symptom Same cold-load storm family (their 104 loads/day vs my 217), but theirs wedges the event loop and holds the port; mine aborts the host. No twin-collision or DB lock on my machine.
#5392 No resent history drifted = 0, NO persisted blocks = 0 on my log.
#5111 Related, different path That is compress returning no-visible-context when over budget; my compress path is not erroring, the churn is in persist/evict.

What I would find most useful

  1. On decision request: #4967 OOM fix folds session history before it is saved — pick the rework option (A vs C) #5190 (A vs C): does this field data change the calculus? My case is a clean argument that "discharge later in the debounce window" (option C) would still be too slow at 81 MiB in / ~40 s turns — six aborts in 2 h suggests the fold needs to happen closer to releaseInFlight than to save completion, which is option A's territory. If you agree, I can hold this session intact and run whatever measurement you want (RSS sampling, --expose-gc heap snapshots, before/after fold instrumentation).
  2. Is sessions.hardMaxBytes supposed to bound state/transient at all? If not, the doc line should say so explicitly and the warn text should not read as "we just reclaimed this". If yes, this is the bug.
  3. Should 20k messages / 81 MiB be supported? The README advertises month-long single sessions; if there is an operating ceiling where the plugin should refuse rather than loop, a loud advisory (and ideally a hard stop) before the host dies would be far better than exit 134.

Workaround I'm applying

Archiving this session and starting fresh (per maintainer advice in #5067 option A: "export + new session"). I have not tuned sessions.hardMaxBytes downward yet — happy to test a specific value if you want the result, since a smaller cap might convert the loop into an early, visible refusal instead of an abort.

Attachments available on request

  • Full bili.log for the window (2.4 MB)
  • All six crash logs (137 KB total)
  • dsh-desktop renderer console excerpt (437 __bili/upstreams CORS failures, unrelated but visible)

Say the word and I will attach them or paste specific slices.

Activity

  1. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    ework

  2. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    标签: bug / dsh / scope/storage / p1(宿主反复 abort = 用户不能用;且属「驻留内存只增不减失控增长」类,命中 owner 2026-10-19「一律 p1」)

    查重:同家族(#3050 / #4967 / #5190),不重复立项。 症状相同(驻留 AST 撑爆堆 → 宿主 exit 134)、机制相同(#4967 的驻留未折叠 lastMessages)。你的报告是这个家族的新数据点:规模约 4×(#4967 是 5k msg / 11 MiB,你是 20k / 81 MiB)、平台 Windows、且暴露了两个既有 issue 没点名的面(见第 2、4 问)。按 owner 2026-10-13「同症状 = 一个 issue」,归并到该家族清单管理,#5190 仍是 A/C 拍板点,本条作为其补充证据与新增子缺陷。我不关闭任何条目(agent 无权关单),只做交叉链接。

    我独立核过 v0.1.210(= origin/master)源码,不是照抄你的描述。按八问:

    1. 是什么问题
    单个超长会话的驻留内存没有任何有效上界:池软上限只能逐出「别的冷会话」(你只有这一个会话,无可逐出对象),per-session hard cap 又够不到「非可丢弃地板」,于是唯一结局是 evict→cold-reload 空转 + 堆涨到 OOM。你观察到的 838 MB 对 512 MB、估算值单调不降、一天 202 evict / 217 cold-load,全部由这个机制解释。

    2. 什么原因导致(逐行核过)
    根因分两层,你说的「cap 没约束住」是对的,但精确失效点在非可丢弃字段:

    • sessionWeight(src/session.ts:1141)= deepSizeOf(整个 session),把 lastMessages / state / metadata / blockContents / contentStore / pluginSnapshot 全算进去。
    • hard cap 放电路径 spillBetweenTurns→runIdleSpill(src/persist.ts:3030–3076)只丢四个字段:foldResidentTail / dropResidentBlockContents / dropResidentMetadata / dropResidentContentStore。state(压缩状态,每轮热路径要用)不在其中。代码注释自己写死了:persist.ts:3025–3027 「The non-droppable remainder (state/transient) is out of scope here — state is hot every turn」。
    • 「估算值放电后不降」有两条都能解释的因果链,我都不排除:① 地板超限——若 state 自身 ≥ 512 MB,怎么折 lastMessages 都压不到线以下;② churn 期间门控挡掉所有 drop——runIdleSpill 有 durableRevision 落盘证明门(persist.ts:3071),你 ~40 s 一次 evict/cold-load,cold-load 后新副本还没干净落盘就被再次逐出,门一直不满足,连可丢字段都折不动。两条都指向同一结论:cap 无法可靠 bound 该会话,最终 OOM。要区分哪条主导,一条 /__bili/memory 的 per-field breakdown(见末段)就能定,比贴 2.4 MB 日志便宜得多。

    3. 影响范围与严重程度
    触发条件:单一长会话,且其不可丢地板(state)或 churn 时序使 cap 失效——你这种 20k msg / 81 MiB-inbound 正中。严重度 p1:宿主反复 abort(exit 134),会话实际不可用;重启后前端自动恢复同一会话立即复发。

    4. 三个问题分别答

    • Q1(A/C 计算):你的数据表面偏向选项 A,但更关键的是它揭示 A/C 都不解决真正的堵点——A/C 争的是「lastMessages 何时折叠」,而你这条里卡住的是非可丢 state 地板 + churn,跟折叠时机无关。即便选 C 完美落地,只要 state 地板 > cap,cap 照样失效。所以本条给 decision request: #4967 OOM fix folds session history before it is saved — pick the rework option (A vs C) #5190 的真正增量是:光定 A/C 不够,还得决定「非可丢地板本身超限时怎么办」(拒绝 / 硬停 / 降级)——这是 decision request: #4967 OOM fix folds session history before it is saved — pick the rework option (A vs C) #5190 目前没列的第三选项。
    • Q2(hardMaxBytes 该不该 bound state?):当前设计故意不 bound state(每轮热路径要用,丢了要重算),EN 文档 env 段也声明了 non-droppable。但有两处文档失真我现在就能修(不改行为):① 汇总表头那句「bounds ONE session」(ZH:「约束单个会话」)夸大了保证——它读起来像「会话会被压到线内」,实际只压得动可丢部分;② CONFIGURATION.md:2051 把 media content store 也列为 non-droppable,但 phase-2 代码已让它默认可丢(dropResidentContentStore,persist.ts:3262;sessions.contentStorePin 才是钉住开关)——这行是过期文案。ZH 表行还漏了 non-droppable 说明,是 zh/en 漂移。
    • Q3(要不要支持 20k/81 MiB、死前要不要告警):这是新功能(feat),需你拍板方向,我不擅自造。我的建议与其让它在 ~4 GB RSS 时静默 exit 134,不如在「地板已超 cap 且放电净回收≈0」这个可检测状态上打一条响亮告警 + 可选硬停(拒绝继续为该会话冷加载,提示归档/新开会话)——比 abort 体面得多。归入 decision request: #4967 OOM fix folds session history before it is saved — pick the rework option (A vs C) #5190 的第三选项待你定。

    5. 是否有回归:本轮我只做文档/日志措辞诚实性修正(见交付),零行为变更,无回归风险。真正的机制修复(地板超限处置)仍卡在 owner 拍板,未动。
    6. 是否破坏缓存:不涉及。
    7. 是否改设计方向:本轮不涉及;地板超限处置若采纳会是新的边界行为,随 #5190 一起定。
    8. 是否引入新配置:本轮无;Q3 硬停若要实现大概率复用现有 cap 或加 opt-in 开关,届时按配置面纪律单独报批。

    我现在就做的(无决策依赖,不等 A/C 拍板):一个小 PR(关联 #5406 + #5190,不关闭任一)收掉两处诚实性缺陷——① 文档措辞:明说 cap 只压「可丢部分」、当不可丢地板自身超限时无法把会话压到线内;修正 content-store 已过期的「non-droppable」表述;补齐 ZH 缺失说明消除漂移。② persist.ts:3045 那条 warn:不再读起来像「刚回收成功」,如实说明「已强制回写可丢字段;剩余 state 等不可丢地板本次未能降到线以下」。纯文档/日志,零行为变更。

    需要你拍板的(阻塞项,我只提交不实施):#5190 的 A/C;以及新增第三选项——非可丢地板超限时是否「响亮告警 + 可选硬停」替代静默 OOM abort(Q3)。

    附件先不用贴——机制已从代码侧钉死。等你定了地板处置方向,若需要我用那份 20k 会话做验证,给我一条 /__bili/memory 的 per-field breakdown(能直接看出是 state 还是 lastMessages 主导那 838 MB)即可,不必整份日志。

  3. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    问题是为啥 state 有 500m 这么大?我理解下 原文存储我们已经做过优化 让原文尽可能在数据库而不是内存 尽量保证内存都是 state 状态 。
    问题是 state 本身为啥也这么大?我理解 state 本身不约定于一个实时快照吗?

  4. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    你这一问点到我上一条的漏洞:我把「非可丢地板」直接归到 state,那是从代码 drop 清单反推的结构判断,不是测量。核了一遍 state 到底能有多大:

    state(CompressionState)里会增长的集合(kernel/src/types.ts):messageRefs(byRaw/byRef)、tsStamps(每 raw id 一条时间戳,永不剪)、tokenSnapshot、lastPassIds(上一轮全部入站 id)、deadRefs/renderDeferredRefs、viewLatches(每 ref 的变换后视图字节,折叠时整体重置)、blocks。

    按你这个会话量级估算——28,120 refs / 20,037 msg——这些集合基本都是「每 ref / 每 msg 一条小记录」(几百字节封顶)。粗算 messageRefs + tsStamps + tokenSnapshot + lastPassIds + 各 refs 数组 ≈ 几十 MB 量级,viewLatches/blocks 在频繁折叠下也是有界的。结论:按构造 state 到不了 500 MB。你的直觉对——state 就该是个有界快照。

    所以那 838 MB 几乎可以肯定不是 state 撑的,而是可丢弃字段没被真正折掉:最可能是 lastMessages(原始 AST,81 MiB wire 解析成 JS 对象后膨胀到 200–400+ MB)± pluginSnapshot。它们的设计就是「存证落盘后折走、内存只留 state」,但在这个会话里没折掉——要么 evict/cold-reload 空转期间 durableRevision !== revisionEpoch 那道门一直不放行(每次 cold-load 又把完整 lastMessages 重新物化),要么就是 #4967 那个折叠时机问题本身。两条都指回 #4967/#5190 的折叠路径没跟上,而不是 state 太大。

    这也修正了我上一条 Q1:A/C(折叠时机)恰恰是相关杠杆——如果胖的是 lastMessages,「何时折」直接决定它能不能在下轮前瘦下来;我之前说「堵点在 state 地板、跟 A/C 无关」是错的,收回。相应地 hardMaxBytes 那条 cap 对这个会话基本是红鲱鱼:它触发的放电和常规折叠走同一套门控,门不开它也折不动 lastMessages。

    定论只差一个数据点:给我那份会话的 GET /__bili/memory 输出(per-field breakdown:lastMessages / pluginSnapshot / state / contentStore / metadata / other 各自字节数)。一行就能确认是 lastMessages 主导、还是真·state 异常——后者才是值得单查的独立新 bug。不用贴日志。

    (补充:我已落的文档/日志措辞修正是对的、保留——它写的是「当不可丢地板自身超限时 cap 压不下去」这个条件式结构事实,不是断言 state 通常很大。)

  5. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    你是说lastMessages 这么大?还是会话启动后原始会话做了 json parser 变得这么大?我怀疑是 json 解析后撑大的?没有丢弃?

  6. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    对,两个说法是同一件事:lastMessages 存的就是入站历史被 JSON 解析后的内存 AST。dsh 是无状态重发宿主——你日志里「每轮 inbound 81 MiB / 20k msg」就是证据,每轮把全部历史重发一遍——所以 bili 每轮都要把 81 MiB wire 解析成 CoreMessage[]。V8 对象/字符串开销会把解析后体积放大数倍(经验 2–4×),落到几百 MB 量级常驻。这就是「lastMessages 大」=「JSON 解析后撑大」,同一个东西。

    关键是它没被折回去。自持环路三行代码(都在 src/persist.ts):

    • :3824–3826 保存时 record.messages = view.lastMessages 原样落盘——折过(lastMessagesFolded=true)就存小的 bounded snapshot;没折就存完整原始 AST。
    • :3769–3770 cold-load 时 lastMessages = parsed.messages 原样物化回内存——上次存的是全量,这次加载又是全量几百 MB。
    • :3071 折叠 foldResidentTail 只在 durableRevision === revisionEpoch 才跑,即必须先有一次「已证明落盘」的保存;noteDurable 只在写成功时触发(:2892/:2939/:3129…)。

    死循环:某轮 lastMessages=全量 AST → 要折它得先有落盘成功 → 但那次落盘本身序列化的还是全量(首折之前它还没折)→ 若保存慢或失败(Windows 上 Defender/索引锁 sessions 目录报 EPERM 最典型,README 专门点过),noteDurable 不推进 → 门一直关着 → 折不掉 → evict 时 flushSync 失败又拒绝逐出(src/session.ts:1820 见 !ok 就保留)→ 下次 cold-reload 又把全量物化回来。重量钉在 ~838 MB 单调爬升、永不下降,正是你日志现象。

    还有一层独立于折叠的固有成本:因为 dsh 每轮重发 81 MiB,每一轮都重新解析出一份几百 MB 的 AST——即使折叠完全正常,也只能压住「两轮之间」的地板,压不住「单轮之内」的峰值。六次 abort 间隔越缩越短,跟「每轮把堆顶上去、GC 赶在下一次分配尖峰前收不回来」完全吻合。

    所以修正上一条:主导项基本可断定是 lastMessages(解析后 AST),pluginSnapshot 次之(受 plugin.snapshotCapBytes 默认 100 MiB 封顶);state 到不了这个量级。

    定论要两个便宜数据点(都不用整份日志):

    1. GET /__bili/memory 的 per-field breakdown —— 确认 lastMessages 占比;
    2. 窗口内有没有 [persist] … EPERM / flushSync 失败 / save failed 的行 —— 区分两条分支:有 → 根因在 Windows 落盘受阻(给 %USERPROFILE%\.local\share\billion-context\ 加 AV 排除即可缓解);无 → 回到 [PR] fix(session): discharge raw lastMessages AST on request release to prevent OOM (#3050) #4967 折叠时机(A/C,此时 A 更合适,因为它在 release 时刻立即折、不等 debounce)。
  7. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    现在要彻底解决这个问题 ,需要服务端 1.不能再 parse 这个 json?看有没有办法不 parser 的前提下搞定?我理解我们本来就发送 sessions 和最后几条 msgid 这样 是不是我们按照这个拿库里的即可?

  8. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    你描述的东西已经存在——就是增量车道(#3248,POST /__bili/plugin/turn):native 客户端只发 conversationId + sinceSeq + 增量 delta(+ msgIds),服务端从自己的 unified store 重建前缀(readCanonicalFromStore,src/incremental-lane.ts:770),再拼上本轮 delta。#5155 还做了前缀预序列化拼接,每轮只重渲染 delta。所以「按 session + seq/msgid 拿库里的、不重发全量」在设计上就是增量车道的样子。

    但这个会话现在没走它,是全量重发。 你日志里每轮 inbound 81 MiB / 20k msg 是铁证——整个历史每轮都过 wire。原因:两道门默认都关着(src/incremental-lane.ts:636-643):

    1. rollout.features.pluginIncrementalTurns 默认 {percent:0}(src/rollout.ts:55),config 里没显式开就是关;
    2. 且要 getStore().replicasAvailable()=true(unified sidecar 活着,shadow/full;legacy-only 不行)。
      客户端探测拿到 enabled:false 就回退全量重发。所以「我们本来就发 session+最后几条 msgid」是目标设计,不是这个 build 当前的实际行为。

    关键判断——切到增量能消掉最狠的部分,但不是 O(1):

    所以「彻底」分两层:

    • 第 1 层(大概率就止住 OOM):让该 dsh 会话上增量车道 → 稳定的几百 MB 取代 838 MB 尖峰。前置:config 开 pluginIncrementalTurns + unified 存储活着 + dsh 侧接了增量客户端(共用逻辑在 src/agent/incremental-client.ts,具体到 dsh 这条 lane 是否接线需确认)。
    • 第 2 层(真正的 O(delta),即你要的「彻底不 parser / 不物化」):server-authored-prefix fast path——直接从 store 行渲染压缩输出,全程不建完整内存历史数组。这才是根上的解,属大工程。

    我需要你确认 + 建议(按「无决策依赖的先修」拆两条并行):

    1. 先答我:这台机器当时的 storage.mode 是不是 unified(shadow/full)?若 legacy-only,车道根本不可用,得先把存储升上去——这决定第 1 层能不能直接做。
    2. 立即线:若已 unified,就把该用户纳入 pluginIncrementalTurns、让他上增量车道——低成本、大概率止 OOM。(注意:翻 rollout 默认值 / 加 config 项都是 owner 拍板域,我不擅自动,等你点头。)
    3. 根治线:把 server-authored-prefix fast path(O(delta) 物化)立成独立 issue(来源标记 [PR] fix(native): refuse oversized direct model requests (#5405) #5406),作为真正根因方案,跟 [PR] fix(session): discharge raw lastMessages AST on request release to prevent OOM (#3050) #4967/decision request: #4967 OOM fix folds session history before it is saved — pick the rework option (A vs C) #5190 的折叠时机区分开——那条治的是「折得不够快」,这条治的是「根本不该每轮物化全量」。

    这样既不停在分析,也把「彻底」的路钉死。要不要我先做第 1 层的 rollout/config 改动(等你确认 storage.mode + 点头)?

  9. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    补两个我刚核实完、直接影响「第 1 层要多少工作量」的点(接着上一条的两层方案):

    ① dsh 侧已经接好了增量客户端——第 1 层不用改 dsh。 src/agent/dsh-native.ts:2483 探测(probeIncrementalTurns)、:2508 planIncrementalTurn,整条 delta/seed 路径都在。所以让该会话上增量车道纯粹是 bili 服务端的开关,客户端零改动。

    ② 一个之前没提的客户端门——只有 Anthropic wire 才走增量。 src/agent/dsh-native.ts:2480:if (!wireMessagesLookAnthropic(parsed.messages)) return; // anthropic wire only — OpenAI tool shapes (tool_calls / role:"tool") resend full (#3433)。也就是说,即使服务端把 rollout 打开,如果这个会话的 wire 是 OpenAI tool 形状,它照样全量重发。这解释了为什么「开了也不一定生效」。

    所以第 1 层的完整前置条件是三个(缺一不可):

    1. config 开 rollout.features.pluginIncrementalTurns(owner 拍板域,我不擅动);
    2. unified 存储活着(replicasAvailable(),shadow/full;legacy-only 不行)——这条等你确认 storage.mode;
    3. 该会话 wire 形状 = Anthropic(否则 [bug] dsh-native not wired to wireMessagesLookAnthropic: Enabling pluginIncrementalTurns on the OpenAI lane silently dro | [bug] dsh-native 未接入 wireMessagesLookAnthropic:OpenAI 车道启用 pluginIncrementalTurns 会静默丢弃 tool_calls / role:"tool" #3433 强制全量重发)。

    第 3 条我可以直接从你的日志/任一请求体判断 role 是 tool 还是 Anthropic 的 tool_use/role:"user"+content-blocks;你要么告诉我 wire 类型,要么给我一行请求体特征。确认这三条后,第 1 层基本就是「翻一个开关」级别的改动,大概率止住 OOM。 根治那条(server-authored-prefix fast path,O(delta) 物化)仍建议单独立 issue,跟折叠时机分开治。

  10. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    增量还没怎么验证过 如果不开增量 是不是也很简单的可以不 parser http header 这个里面不是可以放一个 Message ID 吗?有这个的话就能知道是哪个会话,直接从数据库里面构建是不就行了?

  11. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    你这个思路对,而且它其实就是增量车道的最小内核——不用新发明。「header 放会话/消息 ID → 服务端据此知道是哪个会话 → 直接从库构建」= 增量车道干的事:conversationId 就是那个 ID,readCanonicalFromStore 就是「从库构建」(src/incremental-lane.ts:770)。它的价值在于:可以只做这个最小版,刻意跳过 #3248 那套未充分验证的重握手(ack / sinceSeq / resync / single-flight)——正好绕开你说「增量还没怎么验证过」的风险。

    但有两个绕不开的点,先讲清免得落空:

    (a) 不能零负载。 每一轮的「新内容」(用户最新消息、工具结果)此刻还不在库里——它就是客户端正在发的东西。所以每轮至少要把 delta(新消息,几 KB)送过来:header 管「哪个会话」,body 只装 delta。纯靠库拿不到本轮新增。(dsh 现有增量客户端本来就构造这种 delta 体,格式可复用。)

    (b)「从库构建」不会缩小常驻内存。 库里那 20k 行和解析出的 body 是同一份数据。把来源从「wire 上 JSON.parse」换成「从 store 读进内存数组」,省的是每轮重新 parse 的尖峰 + 81 MiB 网络传输,但对象图本身仍是 O(全量历史)≈几百 MB。真正压到 O(delta)(你要的「彻底不 parser/不物化」)只有一条路:不组装完整内存数组、直接从 store 行流式渲染压缩输出——就是代码里标的 not-in-v1 fast path。

    所以拆成两个正交轴,别混:

    务实路径(解耦止血与根治,都不依赖未验证的 #3248):

    1. 先做轴 A 最小版(header+delta+回退),让该会话稳定持有 O(全量),取代现在每轮 838 MB 尖峰;
    2. 抓一份 /__bili/memory 看稳态多少 MB——远低于堆上限则轴 B 的 fast path 降为独立 follow-up issue 慢慢做;仍顶线再上 fast path。

    需要你拍板:轴 A 走「最小 header+delta」(不等 #3248 验证完)这个方向你认吗?认的话我立一个 issue(来源 #5406)写清最小协议 + 回退 + 与 #3248 的关系,代码先不动——动的是 wire 形状 + 依赖 unified 存储,属行为变更/feat 域,按规矩等你点头再实现。

  12. 41 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions