Skip to content

feat(shared): add run budget contract and runaway detectors (step 1 of #17) - #47

Merged
helsome merged 1 commit into
helsome:mainfrom
kbkb628:feat/17-run-budget-contract
Sep 11, 2026
Merged

feat(shared): add run budget contract and runaway detectors (step 1 of #17)#47
helsome merged 1 commit into
helsome:mainfrom
kbkb628:feat/17-run-budget-contract

Conversation

@kbkb628

@kbkb628 kbkb628 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

背景

#17 要求把"Agent 最多能做多少事"变成 runtime 契约,而不是各 workflow 自己 hardcode。这条 PR 是 step 1/2:只落地契约与判定逻辑,不接线,便于先评审形状;真实 provider 的 E2E 放在 step 2。

改动

文件 内容
packages/shared/src/kernel/run-budget.ts 预算契约:7 类预算键(wall-clock / model calls / tool calls / search iterations / input・output tokens / cost)、StopReasonresolveBudget(defaults ← per-run override ← 系统 ceiling 强制夹紧并回报被夹紧的键)、checkBudget固定顺序取第一个耗尽键)、addUsage(不可变累加)、budgetStop
packages/shared/src/kernel/runaway-detector.ts 四类确定性检测器:重复 tool call(参数键序无关,支持 toolCallInclude/Exclude 通配,未跟踪工具透明)、近似重复 search query(token 包含度)、同一证据集连续无新增、窗口内 retry storm;状态是可序列化纯数据,每次判定带 evidence;runawayStop 映射到 stop reason
两个 .test.ts 32 个用例
kernel/index.tssrc/index.ts barrel re-export(显式,照仓库既有风格)

设计取舍

  • fail-loud:非法 limit(0 / 负数 / NaN / Infinity)、负的 usage delta、会导致"第一次信号就熔断"的阈值(< 2)、越界的 similarity 全部加载或调用即抛错,不静默生效。
  • 用包含度而非 Jaccard"nvidia earnings 2026""…2026 q2" 在 Jaccard 下相似度只有 0.8,会被"每次加一个词"绕过;包含度 |A∩B| / min(|A|,|B|) 在该场景恒为 1.0,而无关查询仍接近 0。
  • 确定性:无模型调用、无 I/O、不读墙上时钟(wallClockMs 由调用方以"距运行开始的绝对耗时"提供),因此可重放、可解释。
  • 只做 step 1:不碰 RunManager,避免在形状定稿前改变运行时行为。

验证

bun install --frozen-lockfile          # 563 packages
bun test packages/shared/src/kernel    # 47 pass / 0 fail(含既有 agent-kernel / run-manager)
cd packages/shared && tsc --noEmit     # exit 0
# barrel 实跑:kernel/index.ts 与 src/index.ts 均导出全部 13 个新符号

不在本 PR 内(step 2,对齐 #17 验收标准)

本 PR 覆盖验收清单中的:统一 budget contract、四类 runaway 检测、每类预算与检测器的单元/边界测试。

我目前是协作者,这条走 fork 分支(kbkb628:feat/17-run-budget-contract)。如果你更希望同仓分支,我可以重开一条到上游仓库,说一声即可。

Step 1 of helsome#17: the runtime contract the agent loop can enforce, without
wiring it into RunManager yet.

- run-budget.ts: RunBudgetLimits / RunBudgetUsage over wall-clock, model
  calls, tool calls, search iterations, tokens and cost. resolveBudget
  applies defaults, then per-run overrides, always clamped by the system
  ceiling and reporting which keys it clamped; checkBudget reports the
  first exhausted key in a fixed order; budgetStop turns exhaustion into a
  machine-readable stop_reason with its detail. Invalid limits and negative
  usage deltas fail loudly instead of being silently ignored.
- runaway-detector.ts: deterministic observers for repeated identical tool
  calls (argument order independent, with tool include/exclude scope),
  near-identical search queries (token overlap, so widening a query with
  more words still counts), iterations reporting the same evidence set, and
  retry storms inside a rolling window. State is plain data and every
  detection carries its evidence; runawayStop maps a detection to a stop
  reason.
- 32 unit tests cover every budget key, the ceiling/override rules and each
  detector boundary.

Wiring into RunManager, partial-result preservation and the real-provider
E2E (a deliberately tiny budget plus an induced repeat-search case) follow
in step 2.

Refs helsome#17
@kbkb628

kbkb628 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

CI 结果说明(避免误判成本 PR 的代码问题):

Check 结果
Typecheck ✅ pass (53s)
Secret scan ✅ pass (6s)
Unit tests 1192 pass / 7 fail

这 7 个失败不是本 PR 引入的,全部位于 packages/longbridge-tools 的账户 fixture 用例:

(fail) phase-2 parser normalization > parses positions fixture / parses assets fixture / parses cash-flow fixture
(fail) phase-2 argv construction > getAccountPositions takes no symbol / getAssets forwards optional currency / getCashFlow forwards date range
(fail) real CLI fixture > maps the real `longbridge portfolio --format json` body (totals + holdings + base USD)

复现方式:干净 clone 上 bun install --frozen-lockfile && bun test --isolate,得到逐字相同
1192 pass / 7 fail / 5638 expect() / 1199 tests across 134 files,失败用例名完全一致。这些账户 fixture
.gitignore 约定不入库,全新环境必然缺失;#10 正是修这个问题的

本 PR 自身范围(packages/shared/src/kernel/*)的 47 个用例全绿,Typecheck 与 Secret scan 也已通过。

@helsome
helsome merged commit 1a5279a into helsome:main Sep 11, 2026
2 of 3 checks passed
helsome added a commit that referenced this pull request Sep 11, 2026
Replay the completed #49 implementation onto clean main after #47 was squash-merged.

Preserves runtime budget enforcement, partial-result persistence, runaway detection wiring, AgentKernel configuration, tests, and the real-provider smoke harness.

Refs #17
Supersedes #49

Co-authored-by: kbkb628 <278338969+kbkb628@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants