Skip to content

[Bug] cache keepalive never arms when cache_control has no ttl #5665

Closed

Description

@marcantoinepelletier

Version: 0.1.212 (also on current master 23a826d), inline lane with pi, Anthropic messages wire.

With cacheKeepalive.enabled: true the keepalive never arms for my sessions, no [keepalive] armed line at all.

From what I see the client sends breakpoints as "cache_control": {"type": "ephemeral"} with no ttl (which is 5m by default on the Anthropic API). detectKeepaliveTier only set a tier when ttl is explicitly "1h" or "5m", so it return undefined and armInternal exit silently:

export function detectKeepaliveTier(body: unknown): "5m" | "1h" | undefined {
let found: "5m" | "1h" | undefined;
const walk = (n: unknown): void => {
if (found === "1h" || n === null || typeof n !== "object") return;
if (Array.isArray(n)) {
for (const item of n) walk(item);
return;
}
const o = n as Record<string, unknown>;
const cc = o["cache_control"];
if (cc && typeof cc === "object" && !Array.isArray(cc)) {
const ttl = (cc as Record<string, unknown>)["ttl"];
if (ttl === "1h") {
found = "1h";
return;
}
if (ttl === "5m") found = "5m";
}
for (const v of Object.values(o)) walk(v);
};
walk(body);
return found;

if (ttl === "1h") {
    found = "1h";
    return;
}
if (ttl === "5m") found = "5m";

The docs say otherwise (tier is 1h when any breakpoint declares ttl: "1h", else 5m):

- **Cadence.** `tier × ttlFactor`, where tier is `1h` when any breakpoint declares `ttl: "1h"`, else `5m`; default factor `0.9` → roughly one ping per 4.5 min / 54 min. Override with `assumeTtl` (skip body detection — for relays that inject or rewrite `cache_control` TTL downstream of bili) or absolute `intervalMs` (wins over everything); `armWithoutBreakpoints: true` arms even breakpoint-less bodies (pairs with `assumeTtl`).

Repro:

  1. "cacheKeepalive": { "enabled": true } in billion-context.json
  2. Send any request where the cache_control blocks have no ttl
  3. Nothing arms, the session row has no keepalive state

Setting assumeTtl: "5m" works around it (armed tier=5m(assumed) cadence=270s), but then I hit #5666.

I guess the fix is to treat a cache_control object without ttl as 5m.

Activity

  1. ranxianglei commented on Nov 4, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

    查重: 不重复。keepalive 特性本体是 #3376/#3390(已合并关闭);本条是它的一个独立缺陷——无 ttl 的断点从不武装。与你用 assumeTtl 绕过后撞到的 #5666 是另一机制(保活定时器让短命进程活到 maxDurationMs),不在本条范围。

    标签: bug · pi · scope/wiring · p2(缺陷;pi 车道 Anthropic wire;保活属请求管线/接线面;有 assumeTtl 绕行故 p2)

    PR: #5669 (Closes #5665)


    八问

    1. 是什么问题:开了 cacheKeepalive.enabled: true,但断点写成 Anthropic 默认形式 cache_control: { type: "ephemeral" }(不带 ttl)时,保活从不武装,日志里没有任何 [keepalive] armed。
    2. 什么原因导致的:detectKeepaliveTier(src/cache-keepalive.ts)只在断点显式带 ttl: "1h" 或 "5m" 时才定档;不带 ttl 时返回 undefined,armInternal 走到最后那个 return 静默退出。而 Anthropic 上 ephemeral 断点缺省就是 5 分钟 TTL——这种断点本就该算 5m 档,检测漏掉了它。CONFIGURATION.md 写的正是「任一断点声明 ttl: "1h" 则 1h,否则 5m」——文档是对的,代码错了。
    3. 影响范围与严重程度:所有开启 keepalive、且断点不带显式 ttl 的会话(这是最常见的写法)——保活完全失效,空闲期后下次请求付全价冷重建。有绕行(assumeTtl: "5m"),但会撞到 [Bug] cache keepalive timer keeps short lived processes (pi -p, sub-agents) alive until maxDurationMs #5666。属「个别功能异常 + 需绕行」= p2。
    4. 怎么修复的:把「不带 ttl 的 cache_control 断点」一律按 5m 档处理,只有显式 ttl: "1h" 才升档。detectKeepaliveTier 里一行逻辑改动(if (ttl === "5m") found = "5m" → 无条件 found = "5m")。
    5. 是否有回归问题:既有行为逐项过——显式 5m → 仍 5m;显式 1h → 仍 1h(提前返回);零断点 body → 仍 undefined(仍靠 assumeTtl / armWithoutBreakpoints 武装);intervalMs 优先级不变;ping 体构造 buildPingBody 未动。新增回归测试在旧代码上红(两条断言 not ok)、修复后绿(文件内 35/35 通过);typecheck + build 干净,全量以远程 CI 为准。
    6. 是否破坏缓存:否。不动任何缓存键/字节/失效策略。保活 ping 本就走旁路(不进 /acp-cache 命中账本),本次只改「该不该武装、按哪档节拍」。
    7. 是否改了设计方向:否。只是让检测与既有文档/设计对齐,无架构/边界/数据流转向。
    8. 是否引入了新的配置:否。无新 env / config / 默认值。

    行为变更披露(旧 → 新):断点省略 ttl 的会话,过去从不武装 keepalive(静默无操作);现在按 5m 节拍武装(tier × 0.9 ≈ 270s)。零 cache_control 断点的 body 行为不变。无需改 CONFIGURATION.md(文档本就正确)。

    判定:可合。纯单模块 fix:、带回归钉、diff 干净、不动内核 / wire 形状 / 配置面 / 版本号。合并留给你。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions