Skip to content

[PR] fix(keepalive): arm cache keepalive when cache_control has no ttl (#5667) - #5669

Closed
ranxianglei wants to merge 0 commit into
masterfrom
2026-11-04_keepalive-arm-no-ttl
Closed

ranxianglei wants to merge 0 commit into
masterfrom
2026-11-04_keepalive-arm-no-ttl

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Problem

With cacheKeepalive.enabled: true, keepalive never armed for sessions whose Anthropic-wire breakpoints use the default form cache_control: { type: "ephemeral" } (no explicit ttl). No [keepalive] armed line appeared at all.

Root cause

detectKeepaliveTier (src/cache-keepalive.ts) only set a tier when a cache_control breakpoint carried an explicit ttl of "1h" or "5m". A breakpoint without ttl returned undefined, so armInternal reached its final return and exited silently. On the Anthropic API an ephemeral cache_control defaults to a 5-minute TTL, so such breakpoints are 5m-tier breakpoints — the detection was simply missing them. This contradicts the documented cadence (CONFIGURATION.md: "tier is 1h when any breakpoint declares ttl: \"1h\", else 5m").

Fix

Treat any cache_control breakpoint without an explicit ttl as the 5m tier; only an explicit ttl: "1h" lifts the tier. One-line logic change in detectKeepaliveTier.

Behavior change (old -> new)

  • Old: a session whose breakpoints omit ttl NEVER armed the keepalive (silent no-op), even with cacheKeepalive.enabled: true.
  • New: such sessions arm at the 5m cadence (tier x ttlFactor 0.9 ~= 270s).
  • Unchanged: bodies with ZERO cache_control breakpoints still require assumeTtl / armWithoutBreakpoints to arm; an explicit ttl: "1h" still yields the 1h tier; intervalMs still wins over everything.

This aligns the code with the already-documented behavior; no CONFIGURATION.md edit is required (the docs were correct, the code was the defect).

Tests

Added regression coverage in tests/cache-keepalive.test.ts:

  • detectKeepaliveTier: no-ttl -> "5m"; no-ttl mixed with an explicit 5m -> "5m"; no-ttl alongside an explicit 1h -> "1h"; a zero-breakpoint body is still undefined.
  • New arm-gate test: a no-ttl body arms at cadenceMs=270000, tierLabel="5m".

Verified red against the pre-fix source (both new assertions fail) and green after the fix (35/35 pass in the file). npm run typecheck and npm run build are clean on this head. Full-suite verdict comes from remote CI.

Closes #5665

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-04_keepalive-arm-no-ttl (e661032)

Latest build — install this exact version

npm install -g billion-context@0.0.8128-pr.5669.95dc39b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
e661032 billion-context@0.0.8128-pr.5669.95dc39b build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5669.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(keepalive): arm cache keepalive when cache_control has no ttl (#5667) [PR] fix(keepalive): arm cache keepalive when cache_control has no ttl (#5667) Nov 4, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重:不重复 —— 本 PR 正是对原始 bug 报告(#5665,marcantoinepelletier 报的「cache keepalive never arms when cache_control has no ttl」,镜像 #5667)的修复。相关但不重复的条目:镜像 #5668(keepalive timer 让 pi -p/sub-agent 短命进程活到 maxDurationMs)——同功能域但症状与机制都不同(事件循环被 timer 拖住 vs 层级检测漏判),属独立问题;#4233 是全局 cacheKeepalive 块的配置校验,无关。

标签: bug / scope/wiring / p2 —— opt-in 功能在默认断点形态下静默完全失效、有绕行(显式 ttl / assumeTtl / intervalMs),定 p2;缺陷在宿主侧请求管道(src/cache-keepalive.ts),非 kernel。

状态: 识别为 agent 提交的 PR(ework-agent-pr 标记)→ 不重做,进入核验流程:

  • 已拉取分支 origin/2026-11-04_keepalive-arm-no-ttl(单提交 e661032),diff 干净:src/cache-keepalive.ts +4/-1、tests/cache-keepalive.test.ts +32,无跑题文件;
  • base 新鲜:merge-base == master head(23a826d37),无需 rebase;
  • 代码核对:detectKeepaliveTier 顶部 found === "1h" 早退保证 1h 优先级不被后续无 ttl 断点降级;零 cache_control 的 body 仍返回 undefined(仍需 assumeTtl/armWithoutBreakpoints);intervalMs > assumeTtl > breakpointTier 阶梯未动;CONFIGURATION.md:691 的文档表述(「tier is 1h when any breakpoint declares ttl: "1h", else 5m」)与新行为一致,PR 描述与代码事实相符。

正在本地对 PR head 跑定向测试 tests/cache-keepalive.test.ts 并读镜像 CI 缓存,随后给结构化八问结论。

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 4, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

核验完成(识别为 agent 提交的 PR,未重做)。结构化八问:

  1. 是什么问题:cacheKeepalive.enabled: true 时,Anthropic-wire 断点用默认形态 cache_control: {type:"ephemeral"}(不带显式 ttl)的会话从不武装空闲 keepalive——静默 no-op,连 [keepalive] armed 日志都没有。
  2. 什么原因导致的:detectKeepaliveTier(src/cache-keepalive.ts)只认显式 ttl: "5m"/"1h";缺 ttl → 返回 undefined → armInternal 落到最后的静默 return。Anthropic 的 ephemeral 断点默认就是 5 分钟 TTL,这些断点本就是 5m 层级——检测漏掉了它们,代码与文档化 cadence(CONFIGURATION.md:691「tier is 1h when any breakpoint declares ttl: "1h", else 5m」)矛盾。
  3. 影响范围与严重程度:开了 keepalive、但 relay/客户端发无 ttl 断点(最常见形态)的用户一个 ping 都拿不到 → 空闲期前缀缓存过期、下次请求全价冷重建。纯成本影响(无数据/正确性损失),有绕行(显式 ttl / assumeTtl / intervalMs / armWithoutBreakpoints)→ p2。
  4. 怎么修复的:一行逻辑变更——任何无显式 ttl 的 cache_control 断点按 5m 层级处理,仅显式 ttl:"1h" 提升层级。让代码对齐既有文档,无需改文档。
  5. 是否有回归问题:触碰路径的既有行为逐项清点——① 1h 优先级保持(walk 顶部 found==="1h" 早退 + 显式 1h 分支 return,mixed 断言钉住);② 零断点 body 仍需 assumeTtl/armWithoutBreakpoints 才武装(tests/cache-keepalive.test.ts:346 既有测试钉住);③ intervalMs > assumeTtl > breakpointTier 阶梯未动(:351/:359/:365 各钉一档);④ ping 机制(body 只改 max_tokens/stream、verdict、并发/时长上限)未触碰。未发现回归。一处诚实边界记录:现在任何带 cache_control 键的对象(哪怕嵌在用户内容里)都计为 5m 断点——该暴露面类别变更前已存在(显式 ttl 值此前在 body 任意位置同样会设层级),影响限于 opt-in keepalive 的会话,最坏是多几个近零成本 ping(maxConcurrentPings=4 / 24h 上限封顶)。
  6. 是否破坏缓存:不涉及——改动只是让更多会话按文档武装(ping 用于刷新上游前缀缓存 TTL),bili 侧存储/缓存/重试策略未动。
  7. 是否改了设计方向:没有——同一机制,检测对齐文档契约。
  8. 是否引入新配置:没有。

核验证据:

判定:可合 —— 满足 §7.4 自动合入门槛(单模块纯 fix、红/绿回归钉、rebased head 上 CI 全绿、无配置/持久化/wire/kernel 变更、干净 diff、引用 issue)。合并留给 owner:#5669

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

判定:APPROVE —— 单点修复,把代码对齐到既有文档行为;回归测试有牙(本地 red 验证过);既有行为无回归。无需要顺手修的小问题。

结构化八问

  1. 是什么问题:cacheKeepalive.enabled: true 时,Anthropic 线上断点用默认形态 cache_control: {"type":"ephemeral"}(不带显式 ttl)的会话,保活永远不武装——静默 no-op,连 [keepalive] armed 日志都没有。而 no-ttl 恰是最常见的断点形态,所以大多数开了保活的用户这个功能实际不生效。
  2. 什么原因导致的:detectKeepaliveTier 只认显式 ttl:"5m"/"1h";不带 ttl 的断点返回 undefined,armInternal 落到末尾 return。而 Anthropic API 默认 ephemeral TTL 就是 5 分钟——这些本来就是 5 分钟档断点,被检测漏掉。代码与文档声明的节奏规则矛盾(CONFIGURATION.md:705:“tier is 1h when any breakpoint declares ttl: \"1h\", else 5m”)。
  3. 影响范围与严重程度:opt-in 功能(默认关),无数据丢失、无安全面。受影响面 = 开了保活且宿主发 no-ttl 断点的用户——对他们功能完全失效(静默);有绕行(assumeTtl/intervalMs/显式 ttl)。p2 级。
  4. 怎么修复的:detectKeepaliveTier 中,任何不带显式 ttl 的断点按 5m 档处理;保留显式 "1h" 的提前返回,1h 优先语义不变。一行逻辑改动 + 引 #5667 的解释注释(src/cache-keepalive.ts detectKeepaliveTier,:119-144)。
  5. 是否有回归问题:逐项审过既有行为——显式 5m→5m 不变;显式 1h→1h 且优先级保持(新增子断言 oneHourAmongNoTtl 钉住,tests/cache-keepalive.test.ts:232-236);零断点 body 仍返回 undefined、仍需 assumeTtl/armWithoutBreakpoints(既有负门 :346-349 不变);intervalMs > assumeTtl > breakpointTier 阶梯未动(:365-371);武装门(non-anthropic/GET/passthrough/resign/buffer,:332-344)与 ping 生命周期代码全部未动。一处小的可观测漂移(披露):带 no-ttl 断点且 armWithoutBreakpoints:true 的 body,tierLabel 从 5m(no-breakpoints) 变为 5m(节奏同为 270s)——语义上更准确,仅外观。Red 验证:换回修复前源码恰好两条新断言失败(33/35),修复后 35/35。
  6. 是否破坏缓存:否。ping 重放逐字一致的前缀(只改缓存区之外的 max_tokens=1/stream=false);新武装的会话多出的周期性 TTL 刷新 ping 正是该功能的目的;解除规则(cache_read=0 / 429 / 连续 3 错 / 时长上限)未变。
  7. 是否改了设计方向:否。代码向既有文档契约对齐;CONFIGURATION.md en+zh 已在 head 核对一致,无需文档改动(文档一直是对的,坏的是代码)。
  8. 是否引入新配置:无(无新 env/config/默认值)。

验证记录

  • 本地定向跑(head e661032):tests/cache-keepalive.test.ts 35/35 通过。
  • Red check:换入 master(23a826d) 版 src/cache-keepalive.ts → 恰好两条新断言失败(detectKeepaliveTier: 5m / 1h / mixed / nested / none :216、arm: no-ttl breakpoints arm at the default 5m tier :323),跑完还原——回归测试抓得住原 bug。
  • CI(镜像缓存):failed 0 / pending 0;guards 全绿(kernel-guard/docs-sync/docs-coverage/config-surface-guard 等);base = master head,无需 rebase。
  • 调用面:detectKeepaliveTier 唯一调用点在 armInternal(src/cache-keepalive.ts:282);仓内其他 “breakpoint” 提及属 bili 自己的 cache_control stamping 机制,与本改动无关。

非阻塞观察(无需动作)

  • 递归 walk 会把 body 任意位置名为 cache_control 的对象键(包括用户内容里嵌入的同名 JSON)计为断点——这是修复前就有的 naive-walk 固有属性;修复后误报面略宽(任何 cache_control 对象 vs 之前需精确 ttl 匹配),但后果更轻(误武装 5m 档 vs 之前可能误武装 1h 档),且 ping 有上游验证、miss 即解除,成本有界。

结论:可合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) 审毕:PR #5669 上游已发 review(判定 APPROVE,结构化八问 + 本地 red 验证记录;因同账号自审限制以 COMMENT 事件提交)。结论:可合。

ranxianglei added a commit that referenced this pull request Nov 7, 2026
…tier so keepalive actually arms (#5665)

detectKeepaliveTier only armed when a breakpoint carried an explicit ttl of 1h/5m; the default form cache_control:{type:ephemeral} (no ttl) returned undefined and armInternal exited silently — contradicting CONFIGURATION.md (which said: 1h when any breakpoint declares ttl:1h, else 5m). Now any breakpoint without explicit ttl is 5m-tier; explicit ttl:1h still lifts to 1h; intervalMs still wins.

Picked up from PR #5669 (issue #5665).
@ranxianglei
ranxianglei force-pushed the 2026-11-04_keepalive-arm-no-ttl branch from e661032 to 58a8796 Compare November 10, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] cache keepalive never arms when cache_control has no ttl

1 participant