Skip to content

fix: TTL-cache the opencode-native per-request health probe (#928) - #930

Merged
ranxianglei merged 2 commits into
masterfrom
2026-09-18_opencode-native-health-probe-ttl
Sep 18, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-09-18_opencode-native-health-probe-ttl

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

createNativeRoute (src/agent/opencode-native.ts) synchronously probed GET /__bili/health (1500ms timeout) before every model request with no caching of the result, while respawn in the same module is already rate-limited by RESPAWN_COOLDOWN_MS = 15_000. That per-request probing contradicted the stability assumption the cooldown already concedes: a healthy proxy paid a loopback RTT on every request, and a freshly-dead one added up to 1.5s of latency before degrading.

Change

Wrap the probe in a per-origin TTL cache:

  • New constant HEALTH_PROBE_TTL_MS = 2_000 — deliberately far below the 15s cooldown, so the detection horizon is consistent with the cooldown's own stability assumption while removing the per-request cost.
  • withProbeTtl(...) wraps both the default probeHealth and any test-injected probe; it caches both verdicts (a healthy hit removes the steady-state RTT; a dead hit avoids re-paying the full timeout across the death+cooldown window); stale entries are evicted on insert so the map stays bounded.
  • Injectable via deps.probeTtlMs for tests; defaults to HEALTH_PROBE_TTL_MS.
  • Wired once in createNativeRoute, covering all three probe call sites uniformly.

Detection latency is now bounded by one TTL (<=2s). On the issue's question — neither "per-request freshness" nor the 15s cooldown is the hard constraint; the real requirement is bounded death-detection plus zero steady-state cost, and a few-second TTL satisfies both.

Tests

3 regression tests in tests/opencode-native.test.ts:

  1. healthy origin -> exactly one health probe across N (5) steady-state requests
  2. re-probes after the TTL elapses (not a permanent latch)
  3. a proxy that dies is detected within one TTL and degrades to direct (unrewritten)

Pre-flight

  • npm run typecheck — clean
  • tests/opencode-native.test.ts — 27/27 pass
  • full suite — 1696 pass / 0 fail / 2 skipped
  • npm run build (tsup) — success

Note on stacking

This code lives on the unmerged integration branch feat/pi-native-plugin (not yet on master), so this PR targets that branch rather than master. It should land together with / ahead of the other opencode-native PRs (#921, #923) to avoid churn.

Fixes #928

createNativeRoute probed GET /__bili/health (1500ms timeout) before every model request with no caching, while respawn in the same module is already rate-limited by RESPAWN_COOLDOWN_MS=15s. Per-request probing contradicted that stability assumption: a healthy proxy paid a loopback RTT on every request; a freshly dead one added up to 1.5s before degrading.

Wrap the probe in a per-origin TTL cache (HEALTH_PROBE_TTL_MS=2s, injectable via deps.probeTtlMs). Both verdicts are cached: a healthy hit removes the steady-state RTT; a dead hit avoids re-paying the timeout across the death+cooldown window. Stale entries are evicted on insert so the map stays bounded. Detection latency is bounded by one TTL, consistent with (and far tighter than) the 15s cooldown horizon.

Adds 3 regression tests: single probe across N steady-state requests, re-probe after TTL expiry (not a permanent latch), and death detected within one TTL degrading to direct.

Pre-flight: typecheck clean, opencode-native.test.ts 27/27, full suite 1696 pass / 0 fail / 2 skipped, tsup build success.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Reviewed PR #930 (this issue's tracked PR). It was already opened and is green, so I did not redo the work — I pulled the branch, cross-checked the code against the description, confirmed CI, and re-ran the pre-flight on the rebased head (9444ba6).

Code matches the description. In src/agent/opencode-native.ts:

  • HEALTH_PROBE_TTL_MS = 2_000 added (opencode-native.ts:104), deliberately far below RESPAWN_COOLDOWN_MS = 15_000.
  • withProbeTtl(probe, ttlMs) wraps both the default probeHealth and any injected probe; caches both verdicts (a healthy hit removes the steady-state loopback RTT; a dead hit avoids re-paying the full 1.5 s timeout across the death+cooldown window); evicts stale entries on insert so the map stays bounded.
  • Wired once in createNativeRoute (opencode-native.ts:146) via deps.probeTtlMs ?? HEALTH_PROBE_TTL_MS, so all three probe call sites (lines 155 / 170 / 213) share the same cached probe.
  • Header comment updated to describe the TTL behavior.

I traced the logic: a hit within the TTL returns the cached verdict without touching the network; a miss/stale entry re-probes, then evicts every older entry before storing. Correct, and orthogonal to the two compression modes (it's an agent-liveness-gate concern, not wire/message shape).

Tests match too. The three #928 regression tests in tests/opencode-native.test.ts cover: (1) exactly one probe across 5 steady-state requests, (2) a re-probe after the TTL elapses (no permanent latch), (3) a proxy that dies is caught within one TTL and degrades to an unrewritten (direct) request.

Pre-flight re-run on the rebased head (isolated checkout, deps installed fresh):

  • npm run typecheck — clean
  • tests/opencode-native.test.ts — 27/27 (the last 3 are the new #928 ones)
  • full suite — 1696 pass / 0 fail / 2 skipped
  • npm run build — success

All four match what's documented in the PR body.

CI: on head 9444ba6, the check real codex vs deterministic fake upstream (zero tokens) = completed / success; GitHub reports mergeable_state = clean.

Auto-merge gate (§7.4): this reads as a clean single-module fix: — scoped to opencode-native.ts, regression tests reproduce the old per-request behavior and now pass, green on the rebased head, no config/persistence/wire/cross-repo changes, clean diff, Fixes #928, nothing load-bearing touched.

Two things to flag before it lands (neither blocks correctness):

  1. Target is the integration branch feat/pi-native-plugin, not master. Nothing here reaches production until that branch lands; the standard "auto-merge to master" framing doesn't apply — it's meant to land with/ahead of the sibling opencode-native work.
  2. Hot-file contention with feat: v1 legacy-lane routing — opencode-acp old sessions keep working, new sessions go to bili (#920) #921. PR feat: v1 legacy-lane routing — opencode-acp old sessions keep working, new sessions go to bili (#920) #921 (also base feat/pi-native-plugin, v1 legacy-lane routing) also edits src/agent/opencode-native.ts. Recommend landing fix: TTL-cache the opencode-native per-request health probe (#928) #930 ahead of feat: v1 legacy-lane routing — opencode-acp old sessions keep working, new sessions go to bili (#920) #921 (as its body suggests) and having the merger rebase + re-run tests to resolve any semantic conflict. (No feat: opencode v1 — keep opencode-acp legacy sessions working under bili (#920) #923 is currently open.)

Minor nit, non-blocking: withProbeTtl has no single-flight dedup, so two truly-concurrent requests inside a cache-miss window could both fire a probe. Negligible in practice (sub-ms, cold-start only; steady state always hits the cache) — not worth changing.

Conclusion: code, tests, and pre-flight all check out; CI green. Ready for a human to merge into feat/pi-native-plugin (ahead of #921).

中文摘要:本 PR 给 opencode-native 的每请求健康探测加了 2s 的按-origin TTL 缓存(稳态零 RTT、死亡检测有界),代码与描述一致、测试与预检全绿、CI 通过,可合并进集成分支 feat/pi-native-plugin(建议先于 #921)。

Base automatically changed from feat/pi-native-plugin to master September 18, 2026 07:54
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-18_opencode-native-health-probe-ttl (7b3c004)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-930

Each push to this PR publishes a new version under the pr-930 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr930.tgz
npm install -g package

Install instructions are refreshed on each push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant