Skip to content

fix(agent-runtime): 避免长任务中的重试预算累计耗尽 - #713

Closed
hui455 wants to merge 49 commits into
vastsa:mainfrom
hui455:fix/issue-699-network-recovery
Closed

hui455 wants to merge 49 commits into
vastsa:mainfrom
hui455:fix/issue-699-network-recovery

Conversation

@hui455

@hui455 hui455 commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

问题与修复

长任务中,网络故障即使已经重试成功,次数仍按整个用户回合累计。上游 aad46adb 的桌面模拟中,完成 10 次文件读取后,第 11 次独立断网直接终止任务。

现在主会话和内置子代理在完整模型响应成功后重置网络错误及限流的重试预算,包括成功返回工具调用。下一次请求从第 1 次重试开始;只收到响应头、部分文字或切换故障阶段不会重置预算。

终止诊断改为读取对应预算的计数,避免临时重试状态清理后漏报 retryAttempt

范围与兼容性

  • 连续故障仍保留每类最多 10 次重试,沿用退避等待、Retry-After 和取消规则;不会重放已完成的工具调用。
  • 不改变 IPC、数据库、权限或提供商配置,不新增无限重试开关。
  • 同步运行时、错误码、中英文相关 E2E 规格与 ADR 0206;中文页更正旧上限,并以英文补充新增规范条目。

验证

基线 main:aad46adbc1aa8120659681125ae11a28014868bf;Windows x64、Electron 43.6.0、Node 24.2.0。使用重新构建的桌面及 runtime、隔离数据目录和本机故障模拟 API。

  • 修复前复现累计预算耗尽;修复后经历 12 次独立断网、完成 11 次真实 Read,最终正常结束。每次独立恢复均从第 1 次重试开始。
  • node scripts/e2e-provider-recovery.mjs 通过:请求断线、输出中断、Responses 恢复、连续故障耗尽及恢复后点击“继续”。持续断线仍在 10 次重试、约 63 秒后终止。
  • 新增真实 Agent 循环测试 10 项通过,覆盖主会话、子代理、网络错误、429、流中断、成功后再次故障、混合阶段及取消。相关 runtime、subagent、provider-retry、provider-transport-recovery、agent-errors 套件共 299 项通过。
  • Runtime 及 Desktop 构建、两者 typecheck、pnpm lint、架构检查、pnpm docs:checkpnpm docs:build、agent-policy 和 diff 检查通过。

验证边界

Runtime 全量测试 642 项通过、1 项失败:未改动的 native-pi-session.test.ts:759 使用 / 拆分 Windows 路径,导致文件名比较失败。该测试及对应实现与上游一致,未纳入本次修复,不将全量结果标为通过。

未验证原生 macOS/Linux、真实 ChatGPT OAuth 或原报告者的网络环境。本次解决已复现的预算累计问题,不能据此认定原报告的具体网络根因。与 #476 的可选无限重试功能范围不同。远程 CI 以 PR 发布后的实际运行结果为准。

Refs #699

CI 当前结果

本候选的 JS 构建/类型检查/lint/架构/测试及 Docs checks 已通过。Rust 的格式检查和 Clippy 通过,测试阶段 558 项通过、1 项失败:plugins::tests::an_install_reports_progress_and_honours_a_cancel 报告 PLUGIN_INTEGRITY: package is 0 bytes, 6540 were announced运行日志)。

本 PR 未修改 Rust 源码或 Cargo 文件;基线 aad46adb 的 Rust CI 通过。相同失败用例在本候选的 Windows 本地单独复测通过,尚不能确定 Linux 并行测试的完整根因。已尝试重跑失败任务,但 GitHub 返回需要仓库管理员权限。该 Rust 检查仍为失败,需项目方重跑或继续定位。Vercel 预览另需部署授权,不属于代码测试失败。

zeroy1024 and others added 30 commits September 20, 2026 19:37
The combined model x reasoning menu carried a native range slider whose thumb travel did not match the tick label grid: the thumb centers run 7 + i*(W-14)/(n-1) while the labels sat at flex-column centers, so the end stops were off by roughly a character width in each direction and the middle stops drifted the opposite way (vastsa#417 follow-up).

Make the ticks row an n-column grid and inset the range input on both sides by half a column minus the thumb radius, so the native thumb center lands on the matching column center for every stop count. Each label now centers in its column and carries a tick dot; the selected stop uses the accent token and the rest a muted token, per the D458 contract.

Long ladders cannot fit every canonical word, so when more than four stops are listed only the selected stop and its immediate neighbours keep visible text; the rest collapse to their tick dot but stay clickable and keep their tooltip. Four or fewer stops show every label.

Renderer only: no protocol, storage, host, or migration change. The range input remains the accessible control; tick labels are still not tab stops. Updates the source-contract test, D458 in both decisions logs, and E2E-050.
The previous stop alignment kept the tick dots in the label row below the track and collapsed labels on long ladders, neither of which matched the intended design. The dots belong on the rail itself, one per stop with the thumb gliding over them, and every label should stay visible.

Restructure the slider into a rail (dots row plus the overlaid range input) and a separate labels row. Both are full-width n-column grids keyed to --stop-count, so dot i, label i and the thumb at index i share one column center W/(2n) + i·W/n. The input is inset by half a column minus the thumb radius to move its native stops onto those centers; its own track is transparent so the rail line and dots show through. The rail line spans the stop-to-stop distance and the fill uses i/(n-1), so it ends exactly at the active stop.

Track dots use the muted token with the accent token for the selected stop (the thumb covers it); every label stays visible and ellipsizes in its column. Renderer only: the range input remains the accessible control and labels are still not tab stops. Updates the source-contract test, D458 in both decisions logs, and E2E-050.
The slider now gives feedback when a level change lands without giving up direct manipulation. While a drag is in flight the thumb, track dots and accent fill track the pointer with no transition so the control stays responsive; once the store confirms and the drag lead clears, the thumb plays a single overshoot pop and the fill slides to the stop.

A settling flag marks the brief window after a drag commit lands, so the thumb pop only runs then — never mid-drag and never on open. The accent fill is split into its own ::after layer whose width transitions between stops, and the track dots cross-fade between muted and accent. The fill percentage is i/n so it reaches exactly the active stop's column center under the new layering. All of it is disabled under prefers-reduced-motion.

Renderer only: no change to the accessible control, commit path, or persistence.
Keep the popover anchored when the trigger is pressed and move the visual
thumb smoothly between selected levels without delaying native dragging.
Preserve optimistic selection while writes settle and link label hover
with unfilled stops without disturbing the filled track.

Cover geometry, repeated menu opening, motion, hover, keyboard input,
dragging, and reduced motion in an isolated Electron fixture.
Let long-running sessions and subagents recover from independent provider
outages without spending retries from earlier successful responses. Keep
continuous failures bounded and preserve cancellation and tool execution.

Report terminal retry counts from the relevant budget and cover recovery
through real agent loops and isolated desktop fault injection.

Refs vastsa#699
pi-ai only clamps max_tokens on the streamSimple path; the low-level
stream used for thinkingLevel "omit" sends options.maxTokens untouched,
and pi-ai's chars/4 estimate under-counts CJK text. Both routes can push
input + output past the model window and hit a maximum-context-length
400/413/503.

Add a shared request-side clamp (output-cap.ts) applied in the parent
runtime and subagent stream fns: CJK-aware input estimate plus a
window-scaled reserve, so the clamp holds for streamSimple and the
low-level stream alike. Unit tests are model-agnostic.
Use the desktop runtime's OS trust support so locally trusted inspection
roots work without bypassing TLS validation. Preserve certificate causes
through session and delegate recovery to stop futile retries and explain
the failure in the transcript.

Cover the real sidecar launch and local HTTPS path, synchronize the
runtime specifications, and document the trust decision.

fixes vastsa#714
fix(agent-runtime): cap output tokens to the model window
Plugin utility processes and plugin-declared stdio MCP servers inherited
a closed environment that omitted HOME, USER, and USERPROFILE. A spawned
binary that resolves ~ through $HOME then looked for its state under
TMPDIR and exited 1.

Both spawn sites now share one allowlist so the identity variables
cannot drift out of only one of them. Provider keys still do not cross.

fixes vastsa#717
fix(plugins): pass HOME/USER into plugin child processes
Merge the reviewed provider certificate trust fix.\n\nThe Vercel preview check was not available because deployment authorization is required.
Only clear relative file references when the workspace actually changes. Returning from Settings must retain the restored session draft.

Cover remount preservation and workspace-change cleanup in the existing real React composer regression fixture.
The session runtime's budget math (hardLimit, requestHeadroom,
keepRecentTokens) is about to be needed by subagents as well (ADR 0299,
issue vastsa#708). Two drifting copies of the same safety boundary would let a
delegate issue requests its parent already considers unsafe, so the
formula moves to a dependency-light module both sides import.

Pure extraction: contextBudgetFor() and retainedUserMessageBudget()
reproduce the previous numbers exactly, which the unchanged
runtime.test.ts budget assertions verify.
Record the decision to extend the session's compaction contract to
delegates (issue vastsa#708): a delegate gets its own budget-derived turn
boundary compaction, a degraded tail-only retry, and a dedicated
SUBAGENT_CONTEXT_OVERFLOW terminal failure with actionable guidance
instead of a raw provider error.

The commit also declares the new error code in the shared registry and
its localized strings ahead of the runtime implementation, which lands
in the follow-up commit. Specs (agent runtime, error codes, E2E plan)
are updated in both languages per the change checklist.
A delegate ran its Agent with none of the session's context protection:
no budget, no turn-boundary compaction, no degradation, and a provider
overflow ended the run as an unactionable CONTEXT_TOO_LARGE that the
parent model could only read as a raw provider sentence. The feature
whose purpose is keeping large reads out of the session window was the
one path with no window protection at all (closes vastsa#708, ADR 0299).

SubagentRun now wires prepareNextTurnWithContext and evaluates the
shared budget formula against the run's resolved model. At or above the
hard limit the run compacts synchronously through pi-agent-core's
prepareCompaction/generateSummaryWithUsage, in memory only; if no
fitting summary exists it degrades to the task brief plus the most
recent messages and says so on its result; if even that does not fit
the run fails with SUBAGENT_CONTEXT_OVERFLOW naming the three remedies
the parent can act on. Fallback alternatives are re-evaluated against
their own windows before switching, and resume seeding truncates the
oldest tool call/result pairs first so a resumed chain starts below its
limit instead of overflowing on arrival.
Keep the app default when it remains in the provider model list.
Only fall back to the first binding when the selected model is removed.
Refresh the pinned pi runtime packages and rebase the Desktop-specific adapter patches onto the new upstream artifacts. Keep provider transcript compatibility and expose the upstream Meta/Muse OAuth catalog through the existing dynamic vendor flow.
Preserve captured certificate causes through one-shot completion so certificate errors remain terminal and diagnosable. Synchronize the Chinese runtime and E2E documentation with the pi 0.86.1 system-message contract.
fix(agent-runtime): give delegates the session's context budget
# Conflicts:
#	packages/agent-runtime/src/delegation-history.ts
Merge pi 0.86.1 sync, provider adapter fixes, and context-budget compatibility into main.
# Conflicts:
#	docs/spec/06-delivery/04-e2e-test-plan.md
#	packages/agent-runtime/src/runtime.ts

@vastsa vastsa left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感谢这个重试预算修复,代码逻辑和新增 agent-loop 覆盖看起来是对的。当前不能合并:PR 处于 DIRTY,且 Rust 检查失败,cargo test -p host-core --locked 为 558 passed / 1 failed,失败是 plugins::tests::an_install_reports_progress_and_honours_a_cancel,报 PLUGIN_INTEGRITY: package is 0 bytes, 6540 were announced

请先基于当前 main 解决冲突,再重跑完整 Rust 检查;如果确认是基础设施竞态,也请提供可复现的分类和新的绿色结果。谢谢。

vastsa and others added 13 commits September 21, 2026 04:34
…lider-alignment

fix(composer): stabilize reasoning menu and animate slider interactions
…omposer

fix(composer): 遮住输入框下方的会话正文
fix(settings): preserve selected default model when saving providers
fix(composer): preserve existing attachments when switching chats during paste
Use the PluginManager data directory when the built-in marketplace catalog is materialized instead of consulting process-global test state. Add a regression test that keeps an unrelated data-directory override from redirecting package writes.
Keep beta and rc version surfaces aligned with the package version while checking their stable release documentation and changelog entry. Cover the preview resolution rules with focused Node tests so the release preflight matches the documented workflow.
Bump every package, Cargo, protocol, and bundled model-catalog surface for the next test release. Add stable 0.15.2 notes in every shipped locale and declare the 0.15.x release line in both READMEs.
All repository CI and docs gates passed. Vercel preview failed independently and is not a protected branch check.
Preserve both the workspace-remount regression and the pending-paste
regression after upstream integration of vastsa#732. Keep both draft
contracts in the shared Composer specification.
Cache the workspace associated with a composer draft so a remount can distinguish an unchanged workspace from a project switch while ChatSurface is unmounted. Extend the real composer fixture and cache tests to cover the transition.
…unt-726

fix(composer): preserve file references across workspace remounts
Preserve provider failure diagnostics while reporting the exhausted retry
budget. Give the pi loop a message-array snapshot so stream recovery after
tool calls does not duplicate assistant history.

Incorporate the upstream marketplace data-path isolation fix and retain
contributor history. Stabilize Windows E2E screenshots without weakening
recovery assertions.
@hui455

hui455 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

已按建议更新到当前 main,解决冲突并推送至 b31cbf46

main 已包含插件市场数据目录隔离修复;本次重新运行的完整 Rust CI 已通过,原失败用例通过,JS 和文档检查也均通过。

集成新版 pi 时还发现消息数组共享导致流恢复失败,已修复并通过回归测试。隔离桌面 E2E 全部通过,包括连续故障耗尽、点击“继续”恢复,以及多次独立断网后的长任务。

当前 PR 已无冲突,仅 Vercel 预览仍需部署授权。麻烦再看一下,谢谢!

@vastsa

vastsa commented Sep 21, 2026

Copy link
Copy Markdown
Owner

已核实该网络错误恢复问题确实存在,但修复内容已经随当前 main 的提交 68534f661fix(agent-runtime): reset retry budgets after successful responses)落地,覆盖主 agent、subagent、重试计数重置及回归/E2E 测试。当前 PR 分支是旧基线,合入会带入已过时的整批变更,没有需要单独合入的唯一增量,因此按“已修复/重复 PR”关闭本 PR。

@vastsa vastsa closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants