Skip to content

fix(preflight): calibrate local estimates against provider usage, provenance-gate the baseline, scale the fold budget (#1933) - #1940

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-02_preflight-gate-metering
Oct 3, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-02_preflight-gate-metering

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

What / why

Issue #1933: on DSH (OpenAI-compatible relay, full-history resend every turn) the gate logged context ~257K tokens reached preflight target 256000 while upstream usage measured only 151–161K (59–63% of the window); after a mid-session provider switch a stale cross-provider usage baseline fired preflight even on small payloads; worst case (issue Evidence D): a 1.5M-token resend was folded for 12 rounds until the fixed 16-call summarization budget exhausted → fail-fast 502 on a payload the provider had just billed at ~50K.

Root causes (verified against v0.1.180 source dd7631c1; the issue's code quote matches master exactly):

  1. Trigger caliber. The gate judges max(usage-baseline, local-estimate), but the local chars/4 estimator's ratio to real billing varies per upstream — observed in the issue log: 1.30–2.47× on one relay vs ~1.0× on another in the same session. On a relay that bills below the estimator, the raw estimate alone crosses the target while real usage sits far under it.
  2. Baseline provenance. lastInputTokens carries no record of which upstream measured it, so it survives provider/window switches and fires the gate on another provider's scale.
  3. Fixed fold budget. The folder gets 16 summarization calls regardless of how far over the window the entry payload is; large resends exhaust the budget mid-shrink and hard-fail instead of converging.

Changes (F1–F4)

  • F1 — calibration factor (k-hat). Each settled request pairs the local billed-caliber estimate (core messages + system/tools overhead + image reserve, recorded in prepare*) with its usage report. A sample is admitted only when plausible (estimate ≥ 2000 tok, ratio in [0.2, 5] — outside it the report demonstrably does not correspond to this payload). Up to 3 same-route samples are kept in a ring; the factor publishes only once two agree within ×2 (clamped mean, [0.25, 1] — one-way, deflate only; see Review round 3). Disagreement or a route switch clears it back to raw behavior, so degenerate billing (placeholder reports, relay echoes, mocks) can never poison a judgment. Applied at every preflight judgment site that uses the local estimate — gate trigger, zero-range fast-forward fit, folder per-round exit/final fit, post-fold fit — so all sites share one scale.
  • F2 — baseline provenance. settleUsageReport records which upstream origin measured lastInputTokens; when the request now routes to a different origin, the usage-baseline floor is demoted to untrusted (info log) and the gate judges by this payload's own calibrated estimate.
  • F3 — budget scaling. Preflight summarization/round budgets scale with entry overshoot (entry tokens ÷ limit): base 16, cap 32, loud warn when raised. A payload ~11× over the window now gets 32 folds instead of dying at 16.
  • F4 — trigger log. The trigger warn now carries both measurement scales (usage-baseline, calibrated local est, raw est + k-hat) for field diagnosis.

Config surface

No new config fields, env vars, or CLI flags. Only additive session-stats persistence fields (lastInputTokensOrigin, calibratedEstimate, calibratedEstimateOrigin, calibrationRing, lastLocalTextEstimate + lastLocalTextEstimateOrigin) — all optional, absent on legacy sessions, ignored when missing. No config schema change, no persistence format version bump.

Behavior changes (old → new)

  1. Trigger & fit caliber: max(raw-local-estimate, usage-baseline) → max(k̂·raw-local-estimate, usage-baseline). k̂ ∈ [0.25, 1] — one-way by design (Review round 3: the correction may only deflate the estimate, never inflate it), published only after ≥2 consistent same-route samples (each: est ≥ 2000 tok, ratio ∈ [0.2, 5]; ring ≤ 3, spread ≤ ×2), cleared on disagreement or route change. Until k̂ publishes (legacy sessions, first turns, unknown/mismatched routes) behavior is byte-identical to today.
  2. Usage-baseline floor: previously authoritative regardless of which upstream measured it → now demoted to untrusted (with info log) when the request routes to a different origin than the measurer.
  3. Preflight budgets: fixed 16 rounds / 16 summary calls → scaled by entry overshoot, capped 32 / 32 (warn log when raised). Worst case across two protection regimes: 64 total summary calls (was 32).
  4. New log lines: trigger warn gains the dual-scale detail; new demotion info line; new budget-raise warn line.

Not implemented (tracked separately): the Evidence-D remainder — trigger object ≠ fold object on protected-heavy resends, where no budget converges and fail-fast still hard-fails — is filed as #1950 (suggested direction: generalize the #496 forward-once-then-learn pattern with evidence gating). Downgrading fail-fast to forward-with-warning and last-resort emergency truncation remain owner decisions inside that issue.

Relation to #1849 / #1929

Orthogonal lines: #1849/#1929 fix the window denominator (host window delivery + credibility visibility); this PR fixes the trigger numerator (what the gate measures). This incident's window was explicitly configured (compress.modelContextLimit), so the #1929 path does not touch it. No shared code regions (#1929: window ladder, stats panel, dsh-native, client-config; here: preflight gate, cache-ledger, prefolder). If #1929 merges first I will rebase onto it.

Tests

  • New tests/issue1933-preflight-calibration.test.ts (11 tests): calibration-ring unit pins (publish-at-two-consistent, cross-route clearing, plausibility band, clamps, no-upstream safety, compaction-boundary reset) plus three e2e regressions through the real proxy: (a) a relay billing at 0.5× the estimator teaches k̂≈0.42, after which an over-raw payload that would have fired preflight on master forwards with zero summary calls; (b) a route switch demotes the stale baseline so a small post-switch request is not folded; (c) overflow-arm provenance (Review round 2): usage settles on route A, the session switches to B, B rejects an unfolded payload with a stated window — the arm stamps B's origin so the baseline stays authoritative and the 使用pi 压缩后就提示prompt is too long #1195 same-request refold rescue delivers 200 to the client.
  • tests/preflight-round-budget.test.ts / tests/preflight-multi-range.test.ts: expectations updated to the scaled budgets, fixtures sized so the raised cap is genuinely exercised.
  • Self-review caught a regression class the naive single-sample learner would have introduced (degenerate pairs like "bill 800 for a 14K payload" poisoning k̂ and breaking the fix: Codex preflight summaries fail with unsupported max_output_tokens #663 suite) — the plausibility band + consistency gate exist because of it; tests/preflight-max-output-tokens.test.ts and tests/preflight-compress.test.ts pass unchanged.

Pre-flight (local, targeted per repo testing discipline — full suite runs in CI): typecheck ✅ · build ✅ · targeted runs: all tests/preflight-*.test.ts + new file + metering family (cache-ledger, nudge, usage frame/abort/totals/observability) = 225/225 pass.

Review round 2 (2026-10-03)

Owner review findings addressed directly on this branch:

  1. Arm provenance gap (real bug found in review): armOverflowShrink wrote lastInputTokens + source="overflow-arm" but never lastInputTokensOrigin. After a mid-session provider switch, the F2 gate compared the arm against the previous route's origin, demoted this route's own rejection evidence to untrusted, and the 使用pi 压缩后就提示prompt is too long #1195 in-request refold rescue never fired — the client got the raw overflow 400 on exactly the scenario the arm protects. Both arm branches now stamp normalizeUpstreamOrigin(route) alongside the baseline write (unknown route degrades to the legacy unprovenanced behavior). New regression e2e (F2b in the calibration suite): usage settles on A → switch to B → B rejects an unfolded payload with a stated window → armed baseline stays authoritative on B → same-request refold summarizes, re-sends, client sees 200; asserts the baseline origin is B and post-rescue traffic keeps flowing. Verified red without the fix, green with it.
  2. Dead param: settleUsageReport's localTextEstimate argument had no caller passing it (prepare* records the pair directly) — dropped; the roll-forward became an explicit clear of the pending pair, which is the actual invariant (one pair per turn, consumed or cleared by exactly one report).
  3. Known caliber difference (acknowledged, unchanged): k̂'s pairing divisor uses countSystemAndToolsTokens(system, tools) while the gate applies the factor to estimateWireOverhead(protocol, body) — a constant additive offset that shrinks on large payloads and is floored by CALIBRATION_MIN_ESTIMATE = 2000 on small ones; accepted as-is per review.
  4. Rebased onto current master (a863e935, includes fix(web): protectedTools row must address compress.protectedTools (#1948) #1949 — web-only, zero overlap); targeted suites re-run green (calibration 11/11 incl. new F2b, preflight round-budget + multi-range, overflow/refold family 30/30, cache-ledger + zero-usage 45/45), typecheck clean. Full regression runs in CI.

Review round 3 (2026-10-03)

Owner-driven tightening (0ea4a174), verified and rebased onto current master (f2b18022, includes #1956 — its src/server.ts hunks sit at lines ~108–2336, zero overlap with this PR's regions; clean rebase, no conflicts):

  1. One-way k̂ (behavior change, declared): CALIBRATION_CLAMP_MAX 4 → 1. The published factor can now only deflate the estimate (trigger no earlier than the raw proxy would); routes whose billing runs above the local estimate publish k̂=1 = exactly legacy behavior, still covered by the overflow arm / learn-on-failure ladder. Rationale: risk asymmetry — every observed DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 harm was over-triggering, while late triggering already has backstops. This eliminates the failure class "calibration itself causes an earlier trigger" outright. Learning side (plausibility band [0.2,5], ring ≤3, ≥2 agreement within ×2) unchanged.
  2. Apply-site invalid-factor guard: applyEstimateCalibration degrades null / 0 / NaN / ±Infinity / negative to no-correction at the single shared choke point — verified all five k̂ consumers go through it (gate trigger src/server.ts:5365, final fit src/server.ts:5583, fit sites src/preflight.ts:855 / :955). A corrupted persisted session.stats.calibratedEstimate can no longer blind (raw×0) or poison (NaN) the estimate arm. Healthy pipelines cannot produce such values (clamp + admission guarantee [0.25,1]), so this is pure hardening against data corruption.
  3. Residual (flagged, self-healing): a session persisted under a pre-tightening build could carry a stored factor >1 until its next settle re-publishes under the new clamp — bounded to one report cycle. Optional follow-up hardening: re-clamp at the apply site as well.
  4. Verification on the rebased head: typecheck ✅ · targeted suites 45/45 ✅ (calibration 11/11 incl. updated clamp expectations + new invalid-factor cases, preflight round-budget/multi-range, overflow/refold family). Full regression runs in CI.

Fixes #1933

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-02_preflight-gate-metering (75f70a8)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1968-pr.1940.e698d77

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1940.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

PR review 完成(拉分支逐段核对代码 + 机制验证 + CI 全量确认)。结论:可合,CI 15/15 全绿,合并由人来。

审核方式说明:不是只读 diff——把 a5e68d28 检出到临时 worktree,核对了 settle 配对时序(JSON/SSE/plugin 三条 settle 路径的 origin 归一化与同请求配对保证)、outboundPayloadBreakdown 其余消费者、fast-forward 路径的 compressibleRanges===0 前置条件(F2 测试有效性取决于它)、relax 时 summaryCalls=0 重置(「per protection regime」措辞属实)、master 领先 PR base 的两个提交(纯 AGENTS.md 文档瘦身,与本 PR 8 个文件零重叠,无冲突、无需 rebase)。

结构化七问

  1. 是什么问题:preflight 触发门用 max(provider计费基线, 本地chars/4估算) 混了两把刻度——估算相对真实计费的比值随上游漂移(事故日志 A 路由 1.30–2.47×、B 路由 0.94–1.05×),估算单独越线而真实 usage 只有 59–63%;切 provider 后旧基线继续垫底;极端 resend 撞上固定 16 次摘要预算 → 假 502。
  2. 什么原因导致的:触发判据取两种刻度的最大值且本地估算无校准机制(根因);usage 基线不记录测量出处(切换放大器);折叠预算与超窗幅度无关(极端案例致死)。切换本身不是独立原因——与你「≥27s 乃至数小时后也触发」的观察一致。
  3. 怎么修复的:修在决策点而非传输层掩症状。F1 k̂ 按路由学习(plausibility band [0.2,5] 过滤退化样本、≥2 个同路由样本 ×2 内一致才发布、钳制 [0.25,4]、不一致/换路由即清除回退 raw),统一作用于四个判定点(gate 触发、零范围 fast-forward fit、folder 每轮退出、post-fold final fit)——gate 与 folder 同刻度是原 issue 方向 3 的落点;F2 基线记录测量 origin,请求改道即降级为不可信(legacy 会话无 origin 字段 → 保持旧行为);F3 预算随入口超窗比缩放(基础 16、上限 32、提升时 warn);F4 触发日志携带双刻度明细。
  4. 是否有回归问题:无破坏。逐项过既有行为:无 k̂ 的会话(前两轮/legacy/未知或换路由)判定与现状字节级一致(applyEstimateCalibration 任一缺失/不匹配即透传 raw,单测钉住);unknownBaseline 匿名路径保持 raw 上界不动;fail-fast/dead-end cooldown([Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726)/nudge/cache ledger/wire 字节全部不变(diff 未触碰任何 rebuild/injection 路径);持久化纯增量可选字段,legacy 文件缺失即忽略,无格式版本升级。三个已知残留角落(均保守方向、不阻塞):(a) openai/responses lane 若配置了独立 image-billing host(billingUpstream ≠ 转发目标),pair origin 对不上 → k̂ 在该 lane 静默不学习,退回 raw 行为——设计取舍而非缺陷,如要覆盖可后续单独处理;(b) folder 内部 baselineFloor 未按 origin 降级(仅 gate 降级)——陈旧基线 + 真超窗触发的角落会多烧几轮,但 server 侧 post-fold final-fit 救援保证不会假 502;(c) zero-total 报告(placeholder billing)会清掉 pending pair——丢样本不学错,与 [Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793 哲学一致。
  5. 是否破坏缓存:否。k̂ 只是判定侧乘子,不改任何出网字节;prompt cache key、prefix affinity、seam forensics、去重键全部不受影响。
  6. 是否改了设计方向:无架构转向——仍是「provider 实测为准、本地估算兜底」,只是把本地估算刻度校正到与计费一致。issue 里两项未实现项(fail-fast 降为 forward-with-warning、最后手段紧急截断)仍待你拍板,要的话在 DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 线程说一声另开 follow-up。
  7. 是否引入新配置:无 config/env/CLI 新增。session-stats 持久化新增 6 个可选字段(k̂+origin、采样环、基线 origin、pending pair+origin),缺失即忽略,无需迁移。

测试有效性(④关)

  • 单位半钉死学习规则本身:单样本不发布、不一致清除、[0.25,4] 钳制、ring 上限 3、跨路由隔离(新环+旧因子退役)、无路由报告不扰动、compaction 边界重置六字段——全是带牙断言。
  • F1-e2e 走真代理复现事故算术:raw ~22k 越 20k target、calibrated ~10k → 必须原样转发且零摘要调用(pre-fix 必折,能抓回归)。
  • F2-e2e 复现陈旧基线折叠:我专门验证了它有牙——小 payload 有 compressible ranges,fast-forward(要求零 range)走不到,pre-fix 必然进 preflightCompress 产生摘要调用,断言抓得住。
  • 两个预算更新测试不是恒真改写:fixture 放大到真正触及提升后的上限(200 turns ≈55 ranges > 32 calls;~11× 超窗 → 32 rounds),断言精确等于提升后上限并匹配 budget-exhausted 文案变体;若缩放逻辑回退成固定 16 会直接红。

CI 与 diff 整洁度

  • CI:15/15 success——ubuntu/windows × node 22/24 全量测试、e2e codex、hermetic registry ×2 OS、dsh/pi/opencode 实客户端 lane、real-image billing、build-artifact、version-guard。
  • diff:8 文件全部服务于本 PR 目的,无无关改动、无空白噪音、无 version/lockfile/acp-kernel pin/src/update.ts 触碰(auto-merge gate 的 load-bearing 清单一项未碰)。

判定:可合。 行为变更已在 PR 描述声明(刻度 max(raw,usage)→max(k̂·raw,usage)、基线跨路由降级、预算 16→≤32、新日志行),均为有意变更且 legacy 路径保守回退。合并操作留给人:#1940

@ranxianglei

Copy link
Copy Markdown
Owner Author

Reviewed against issue #1933's four defects + master hot-path context. Verdict: high quality, fixes defects 1–2 cleanly, softens 3, defers 3/4 — one real gap found.

Verified locally: calibration suite 10/10, preflight-round-budget + multi-range green, typecheck clean; CI 15/15.

What's sound

  • F1's guards are the right shape: plausibility band [0.2,5] + ≥2 same-route samples agreeing within ×2 + ring of 3 + clamp [0.25,4] means degenerate billing (placeholder reports, relay echoes, mocks) can never publish a factor, and disagreement clears back to raw. Applied at every judgment site (gate trigger, zero-range fit, per-round exit, final fit) so there's no mid-loop scale drift — exactly the consistency the incident lacked.
  • F2 demotion semantics are right: a usage baseline is only authoritative on the route that measured it. Legacy unprovenanced baselines keep legacy behavior (fail-open, documented).
  • F3 stays bounded (2× cap, loud warn) and only raises a ceiling.
  • No config-surface change; additive optional session-stats fields only.
  • The two e2e regressions (relay billing 0.5× teaches k̂≈0.42 → over-raw payload forwards with zero summary calls; route switch demotes stale baseline) are the sharp edge of the incident — good.

Finding 1 (real gap): armOverflowShrink never writes lastInputTokensOrigin — the arm gets demoted on exactly the provider-switch scenario it protects.

Sequence: usage settles on route A (lastInputTokensOrigin = http://A) → session switches to route B → B rejects with overflow → the arm sets lastInputTokens + source = "overflow-arm" but the origin stays A (server.ts, both branches of the info.window / no-window arm) → the next request on B sees baselineOrigin(A) ≠ currentOrigin(B) and demotes the floor to 0 — discarding B's own rejection evidence. That silently disables the #1839 in-request refold / #987 emergency-shrink trigger precisely in the mid-session-switch case this issue is about.

Fix is one line per branch: s.stats.lastInputTokensOrigin = normalizeUpstreamOrigin(route?.upstream); (route is in scope in the forward() closure). Worth adding a test: settle usage on A → arm on B → assert the B-routed baseline stays authoritative (and an A-routed one gets demoted).

Finding 2 (scope): Fixes #1933 will auto-close the issue, but Evidence D (trigger object ≠ fold object → fail-fast 502) is not fixed — F3 only softens it (budget 16→32); a 681-msg resend whose content is mostly protected still cannot converge, and forward-with-warning is explicitly deferred pending owner decision. Recommend either splitting the remaining defects into a follow-up issue and referencing both here, or noting the remainder in #1933 so it isn't lost on auto-close.

Nits

  • k̂'s pairing divisor uses countSystemAndToolsTokens(system, tools) while the gate applies it against estimateWireOverhead(protocol, body) — a constant additive caliber difference; the bias % shrinks on large payloads and CALIBRATION_MIN_ESTIMATE = 2000 floors the small ones, so acceptable, but worth knowing it's not the same estimator.
  • settleUsageReport's new localTextEstimate parameter has no caller passing it (the pair is written by prepare* directly) — dead parameter today.
  • mergeable_state: behind — rebase before merge.

@ranxianglei

Copy link
Copy Markdown
Owner Author

Follow-up for Finding 2 filed: #1950 now carries the Evidence-D remainder (object mismatch -> fail-fast 502; suggested #496-pattern generalization), so this PR's auto-close of #1933 loses no scope. Finding 1 (arm-origin demotion gap) still stands for this PR.

@ranxianglei
ranxianglei force-pushed the 2026-10-02_preflight-gate-metering branch from a5e68d2 to 706a96c Compare October 3, 2026 01:55
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review round 2 findings 已全部在本分支直接修掉并推送(rebase 到当前 master a863e935,含 #1949——web-only,与本 PR 零文件交集)。新 head:706a96ce,两个标注 commit。

Finding 1(arm-origin gap)— 确认成立,已修。 你的序列分析完全准确:armOverflowShrink 写 lastInputTokens + source="overflow-arm" 但从不写 lastInputTokensOrigin,F2 的 provenance gate 只覆盖了 settle 路径、漏了 arm 这条写入点——换路由后 B 自己的拒绝证据被当成 A 的陈旧基线降级,#1195 同请求 refold 救援在「mid-session switch」这个它服务的场景里静默失效。修法按你建议:两个分支各一行 s.stats.lastInputTokensOrigin = normalizeUpstreamOrigin(route?.upstream)(route 未知时退化为 legacy 无溯源行为,fail-open 不变)。

回归 e2e(calibration suite 新增 F2b,全真实 proxy 链路):A 上 settle usage → 切 B 带全量历史重发 → B 对未折叠 payload 返回带窗口的 overflow 400 → 断言 ①救援生效(refold 折叠后重发,client 见 200 而非裸 400)②基线 origin === B(sharp pin,pre-fix 停在 A)③救援后流量继续流动(无 502 锁死——即事件终态的回归守卫)。已验证 red-without-fix(去掉两行 → 400 passthrough 断言失败)、green-with-fix。测试设计过程中还排掉两个坑:mock 若按内容 marker 拒绝会撞上 kernel 的 minSummaryLength 与范围分组机制;最终用「payload 是否含折叠载体」作拒绝判据,与 #1195 的产出物精确对齐、无算术边界。

Nits: ①死参数 settleUsageReport(localTextEstimate?) 已删——无任何 caller 传值(pairing input 由 prepare* 直记),roll-forward 改为显式 clear pending pair,这才是实际不变量(每 turn 一对,被恰好一个 report 消费或清除);session.ts 字段文档同步更正。②k̂ 配对分母(countSystemAndToolsTokens)与 gate 应用口径(estimateWireOverhead)的常数加性差——按你判断接受现状,已在 PR 正文记录备查。

七问逐项:

  1. 是什么问题:F2 provenance 机制漏了 overflow-arm 这条基线写入点——换 provider 后本路由自己的溢出拒绝证据被误降级,同请求救援失效。
  2. 什么原因导致的:基线有两个生产者(usage settle / overflow arm),provenance 只给前者接了线;arm 的 source 标记(任何一轮没有拿到 usage 报告的请求,都会把上下文分母放大 12–20×(nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟) #1839)与 origin 溯源(DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 F2)是两次独立演进,交汇处的契约没人写。
  3. 怎么修复的:在决策点的上游补写入(arm 落基线时同步落 origin),不在 gate 侧加特例豁免——gate 的「origin 不匹配即降级」语义保持纯净。
  4. 是否有回归问题:逐项过——arm 路径纯增量写字段;gate 降级条件要求双方 origin 均非空,legacy 无溯源会话行为字节级不变;同路由场景 origin 恒等、无降级、行为不变;zero-total report 不消费 pair 的语义保持;删参等价于原「赋 undefined」路径(无 caller 传过值)。定向 91/91 绿(calibration 11/11 含 F2b、preflight-round-budget/multi-range、overflow/refold 族 30/30、cache-ledger+zero-usage 45/45)+ typecheck 干净;全量交 CI。
  5. 是否破坏缓存:否。零 wire 字节变化,k̂/降级/arm-origin 全是判定侧。
  6. 是否改了设计方向:无。只补齐 F2 最后一个缺失写入点,无新机制、无新配置。
  7. 是否引入新配置:无。

Scope:#1950(你已立)承接 Evidence-D 剩余(对象不匹配→任何预算不收敛→fail-fast 硬失败),PR 正文已更新引用,#1933 的 auto-close 不丢范围。CI 正在新 head 上跑,绿了我会再报一句。合并仍由人来。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI 全绿:head 706a96ce 上 15/15 check-runs success(含 e2e、hermetic registry、real client lanes),mergeable: true,mergeable_state: clean(rebase 后不再 behind)。

当前状态汇总:

PR ready for merge — 合并由人来:#1940

@ranxianglei
ranxianglei force-pushed the 2026-10-02_preflight-gate-metering branch from 706a96c to c110edb Compare October 3, 2026 04:20
@ranxianglei

Copy link
Copy Markdown
Owner Author

追加收紧(0ea4a17):k̂ 校准改为单向 —— clamp [0.25,4] → [0.25,1],外加 apply 站点无效因子防护。

改了什么

  1. CALIBRATION_CLAMP_MAX 4 → 1(src/cache-ledger.ts):发布后的 k̂ 只能把估算往下修(读数 ≤ raw,触发只会比今天更晚);计费高于本地估算的路由(样本 >1)发布 k̂=1 = 保持原行为。环形入环/plausibility band/≥2 一致性等学习侧全部不动,只有最终 clamp 收口。
  2. applyEstimateCalibration(src/util.ts)加无效因子防护:null / 0 / NaN / ±Infinity / 负数 一律当"不校正"原样返回 raw——持久化 session.stats 损坏字段不会把估算臂乘成 0 或 NaN(raw×0 会把 gate 弄盲)。4 个判定点(触发、零区间快进 fit、folder 逐轮/最终 fit、折后 fit)共用这一个收口。

为什么单向

  • 风险不对称:DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 的伤害面全是"过度提前触发"(37% 有效窗口税、63 分钟 7 折、Evidence D 502);反方向错误(晚触发)已有 overflow-arm / learn-on-failure / 上游 400 兜底。单向化把"校准本身导致更早触发"这一类故障整个消灭——校准后的 gate 读数在数学上不可能高于 raw 估算。
  • 注意单向并不能防"学歪的 k̂ 偏大"(真实 1.35 学成 2.0 仍然 >1),那一侧继续由 ≥2 一致性、×2 spread、plausibility band 兜着;clamp 上界 1 现在是语义边界而非主要保险。
  • 放弃的:估算偏低路由(真实计费高于估算)不再校正——它们本来就是今天的行为,且有兜底,净损失≈0。

验证:clamp/cross-route 单测改期望(k̂>1 → 1),apply 站点新增 0/null/NaN/∞/负 用例;e2e F1(0.5 放气路径)不受影响仍绿。typecheck ✔、校准相关 5 个套件 86 用例全绿 ✔、build ✔。(dsh-native.test.ts 在本沙箱 25 例超时是网络边界所致,在不含本改动的 PR head 上同样超时,CI 会正常跑。)

ework-agent and others added 2 commits October 3, 2026 13:57
…venance-gate the baseline, scale the fold budget (#1933)

F1: learn a per-route estimate-calibration factor (k-hat) from settled pairs of the local billed-caliber estimate and the upstream usage report. A sample is admitted only when plausible (estimate >= 2000 tok, ratio in [0.2, 5] — outside it the report demonstrably does not correspond to this payload); up to 3 same-route samples are kept in a ring and the factor publishes only once two agree within x2 (clamped mean, [0.25, 4]). Disagreement or a route switch clears it back to raw behavior, so degenerate billing (placeholder reports, relay echoes, mocks) can never poison a judgment. Applied at every preflight judgment site that uses the local estimate (gate trigger/fit, folder per-round exit/final fit, post-fold fit) so all sites share one scale.

F2: record which upstream origin measured lastInputTokens; when the request now routes to a different origin the usage-baseline floor is demoted to untrusted (info log) instead of firing preflight on another provider's scale.

F3: scale the preflight summarization/round budgets with entry overshoot (base 16, cap 2x) so a payload far over the window gets the folds it needs instead of exhausting a fixed 16-call budget mid-shrink into a fail-fast 502.

F4: the trigger log carries both measurement scales (usage-baseline, calibrated local est, raw est + k-hat) for field diagnosis.

Fixes #1933
…apply-site factor (#1933)

Calibration may now only DEFLATE the estimate (fire later than the raw
proxy would), never inflate it: routes billing above the local estimate
(samples > 1) publish k̂ = 1 — legacy raw behavior, still covered by the
overflow arm / learn-on-failure ladder. This eliminates the class
"calibration itself causes an earlier trigger" outright; the #1933
damage was over-triggering, and the opposite error already has backstops.

applyEstimateCalibration now degrades invalid factors (null / 0 / NaN /
±Infinity, e.g. corrupted persisted session.stats) to no-correction at
the single shared apply site — raw × 0 would blind the estimate arm.
@ranxianglei
ranxianglei force-pushed the 2026-10-02_preflight-gate-metering branch from 0ea4a17 to 75f70a8 Compare October 3, 2026 05:59
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 追加收紧(0ea4a174)已审核、验证,并 rebase 到当前 master。结论:方向正确,实现干净,可合入路径已铺好。

审核要点

  • 单向化论证成立:DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 的伤害面全部是「过度提前触发」(窗口税、fold churn、fail-fast 502),晚触发一侧本来就有 overflow-arm / learn-on-failure / 上游 400 兜底——单向 clamp 把「校准本身导致更早触发」这一故障类从数学上消灭,而放弃的(低估路由不再校正)恰好就是 pre-PR 的 status quo,净损失≈0。收口放在最终 publish clamp、学习侧(band [0.2,5] / ring ≤3 / ≥2 ×2 一致性)不动——分层正确,clamp 上界 1 是语义边界而非主要保险,与你的说明一致。
  • guard 实现核对:!Number.isFinite(k) || k <= 0 用严格检查(无类型强转)——null 由 Number.isFinite(null)=false 捕获,±Inf/NaN 同路,-0/负数落 k<=0;五个用例(0/null/NaN/+Inf/-0.5)全有断言。我 grep 了全部 k̂ 消费点:5 处全走 applyEstimateCalibration 单一 helper(gate trigger src/server.ts:5365、final fit :5583、fit sites src/preflight.ts:855/:955),无内联乘法——单点收口 by construction 成立,不存在漏网判定点。
  • 测试牙齿:k̂>1→1 期望改对了;新增的两个 apply 断言(clamped hi = identity、lo = deflate)把「读数只能低于 raw」这个不变量直接钉死,比单测 clamp 值更强。
  • 一个 residual(已写入 PR 正文,自愈合):旧构建下持久化的 session 可能带着 >1 的存储因子,直到下一次 settle 按新 clamp 重发布——上限一个 report cycle。可选加固:apply 站点再 clamp 一次(一行)。这是你刚定的设计线,我不擅动,留你拍板。

七问逐项:

  1. 是什么问题:不是修 bug,是校准语义收紧——消灭「校准导致更早触发」故障类 + 防持久化字段损坏毒化估算臂。
  2. 什么原因导致的:原设计允许双向校正 [0.25,4];风险不对称评估后,上行方向无净收益(低估路由今天就跑 raw 且有兜底),下行才是事件治疗路径。
  3. 怎么修复的:publish 端 clamp 上界 4→1(学习机制不动)+ apply 单点加无效因子降级(raw 直通,而非清零/NaN 投毒)。
  4. 是否有回归问题:逐项过——高估路由(ratio<1,即事件类 relay)发布值完全不变([0.25] 下界未动);低估路由从最多 4× 膨胀变为 1(= pre-PR 行为,零新增风险);退化计费多一层保险;损坏字段从静默致盲变 raw 直通(健康管线不可能产出此类值,纯防御)。两种压缩模式对称:k̂ 只作用于 proxy 侧判定,不碰工具注入/nudge/summary carrier。本地定向 45/45 绿 + typecheck 干净;全量交 CI。
  5. 是否破坏缓存:否。零 wire 字节变化,值域收窄不影响 prefix cache。
  6. 是否改了设计方向:无转向——F1 内部语义收窄(双向→单向),owner 决策,理由已落 commit message 与 PR 正文。
  7. 是否引入新配置:无。

状态:rebase 到 master f2b18022(#1956 dsh-persona-fingerprint 已并入——其 server.ts hunks 在 ~108–2336 行区,与本 PR 触碰区域零交集,干净 rebase 无冲突)。新 head 75f70a80 已推,PR 正文已同步(clamp 区间、11 tests、round-3 节)。CI 在新 head 上跑,绿了再报一句。合并仍由人来。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI 全绿:head 75f70a80(rebase 后)上 15/15 check-runs success,mergeable: true,mergeable_state: clean。

当前 PR 状态:owner 追加收紧(单向 k̂ + apply 站点防护)已审核验证、已 rebase 到含 #1956 的 master、PR 正文已同步。PR ready for merge — 合并由人来:#1940

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩

1 participant