Repository navigation
fix: TTL-cache the opencode-native per-request health probe (#928) - #930
Conversation
createNativeRoute probed GET /__bili/health (1500ms timeout) before every model request with no caching, while respawn in the same module is already rate-limited by RESPAWN_COOLDOWN_MS=15s. Per-request probing contradicted that stability assumption: a healthy proxy paid a loopback RTT on every request; a freshly dead one added up to 1.5s before degrading. Wrap the probe in a per-origin TTL cache (HEALTH_PROBE_TTL_MS=2s, injectable via deps.probeTtlMs). Both verdicts are cached: a healthy hit removes the steady-state RTT; a dead hit avoids re-paying the timeout across the death+cooldown window. Stale entries are evicted on insert so the map stays bounded. Detection latency is bounded by one TTL, consistent with (and far tighter than) the 15s cooldown horizon. Adds 3 regression tests: single probe across N steady-state requests, re-probe after TTL expiry (not a permanent latch), and death detected within one TTL degrading to direct. Pre-flight: typecheck clean, opencode-native.test.ts 27/27, full suite 1696 pass / 0 fail / 2 skipped, tsup build success.
[bot] 🏷 Reviewed PR #930 (this issue's tracked PR). It was already opened and is green, so I did not redo the work — I pulled the branch, cross-checked the code against the description, confirmed CI, and re-ran the pre-flight on the rebased head ( Code matches the description. In
I traced the logic: a hit within the TTL returns the cached verdict without touching the network; a miss/stale entry re-probes, then evicts every older entry before storing. Correct, and orthogonal to the two compression modes (it's an agent-liveness-gate concern, not wire/message shape). Tests match too. The three Pre-flight re-run on the rebased head (isolated checkout, deps installed fresh):
All four match what's documented in the PR body. CI: on head Auto-merge gate (§7.4): this reads as a clean single-module Two things to flag before it lands (neither blocks correctness):
Minor nit, non-blocking: Conclusion: code, tests, and pre-flight all check out; CI green. Ready for a human to merge into 中文摘要:本 PR 给 opencode-native 的每请求健康探测加了 2s 的按-origin TTL 缓存(稳态零 RTT、死亡检测有界),代码与描述一致、测试与预检全绿、CI 通过,可合并进集成分支 |
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-930Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr930.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Summary
createNativeRoute(src/agent/opencode-native.ts) synchronously probedGET /__bili/health(1500ms timeout) before every model request with no caching of the result, whilerespawnin the same module is already rate-limited byRESPAWN_COOLDOWN_MS = 15_000. That per-request probing contradicted the stability assumption the cooldown already concedes: a healthy proxy paid a loopback RTT on every request, and a freshly-dead one added up to 1.5s of latency before degrading.Change
Wrap the probe in a per-origin TTL cache:
HEALTH_PROBE_TTL_MS = 2_000— deliberately far below the 15s cooldown, so the detection horizon is consistent with the cooldown's own stability assumption while removing the per-request cost.withProbeTtl(...)wraps both the defaultprobeHealthand any test-injected probe; it caches both verdicts (a healthy hit removes the steady-state RTT; a dead hit avoids re-paying the full timeout across the death+cooldown window); stale entries are evicted on insert so the map stays bounded.deps.probeTtlMsfor tests; defaults toHEALTH_PROBE_TTL_MS.createNativeRoute, covering all three probe call sites uniformly.Detection latency is now bounded by one TTL (<=2s). On the issue's question — neither "per-request freshness" nor the 15s cooldown is the hard constraint; the real requirement is bounded death-detection plus zero steady-state cost, and a few-second TTL satisfies both.
Tests
3 regression tests in
tests/opencode-native.test.ts:Pre-flight
npm run typecheck— cleantests/opencode-native.test.ts— 27/27 passnpm run build(tsup) — successNote on stacking
This code lives on the unmerged integration branch
feat/pi-native-plugin(not yet onmaster), so this PR targets that branch rather thanmaster. It should land together with / ahead of the other opencode-native PRs (#921, #923) to avoid churn.Fixes #928