Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ determines which versions can connect.

### Fixed

- Eliminated frequent false `"Page interaction coverage changed; wait and synchronize"` (`control_context_invalid` with `effect_state: "unknown"` and `IntentWal` tab freezes) across SPA sites and pages with dynamic subframes: separated same-document root URL updates (`Page.navigatedWithinDocument` and `FrameState.url`) from cross-document root destruction via `ChromiumCdp.onRouteChanged` / `CooperationController.invalidateRoute` (restoring `rootFrameId`, reconciling child frames across `soft_detached` reattach and post-dialog checks on same-document route updates, and stopping same-process child-iframe `DOM.documentUpdated` events from firing root `onDocumentChanged`) so live collectors stay available, multi-step inputs (`fill`, `scroll-to`, `wheel`, `press`, `select`, `click`, `drag`) and `bsk plan` batches survive same-pathname query/anchor updates (while consuming strict baselines for untargeted actions) and pathname/hash-router transitions revoke stale contexts cleanly after action settlement (`effect_state: "none"`), completed `fill` Step 3 across both `onBeforeNavigate` and instant-commit auto-submitting forms (`finalValueAlreadyNotified`) as well as post-input `Runtime.callFunctionOn` / `DOM.getBoxModel` visibility measurement across main and child frames, sequenced background `observeTab` behind `armCommittedFrame` while initializing collection leases on `COOPERATION_ARM` and `COOPERATION_RESUME` for newly mounted child frames, allowing `collectCooperationFrames` replacement passes past proven-hidden unobservable subframes, and isolating in-flight `state.working` operations and `target` contexts from non-owner `refresh` failures, and preserved explicit handler errors (`navigate`/`reload` errors and structured `result.data.reason` errors such as `download_capture_failed` and `fill_target_changed`) while revoking stale implicit `target` context auto-injection in `ImplicitScopeRegistry` (without dropping cached `last_vom` / alias hydration for `unchanged` and `delta` syncs) and allowing `navigate`/`reload`/`navigate_back`/`navigate_forward` recovery off `page_load_failed` error pages (#762).
- Fixed `tool.wheel` skipping compositor paint readiness (`waitForInputPaint`) under `ToolDispatcher` persistent background leases on unfocused or minimized windows by decoupling `ReadyInput.hidden` (`lacksForegroundPresentation`) from temporary `restoreFocus` and wiring `windowsApi` in `ToolDispatcher`; stopped `target`-policy `--selector` actions in `CooperationController` from polluting `session.refStore`, emitting `used_ref` (while preserving `used_selector` on `--selector` action receipts without polluting `tool.drag`'s `DragResult` or `tool.press`'s `PressResult`), or leaking hallucinated `@eN` references in `handleSelect` custom-dropdown error messages (`withPinnedSelectorTargets`); refreshed live frame activity inside `CooperationController.beginDomMutation()` under `target` policy before scripted DOM writes while preserving `state.baseline` (`observation_id`) when pre-write validation fails with `effect_state: "none"`; fixed `clearRecordingForSession` to use the caller/dispatcher's `deps` (`RECORD_CANCEL` + `bypassOverlay(tabId, false)`), required explicit `isAgentWindowId` in `returnBorrowedTab` so returning a borrowed user tab never routes into another session's Agent Window, replaced module-global download locking with instance-scoped `DownloadConcurrencyGate` (`session_busy`, `effect_state: "none"`, `dispatched: false`), and cleaned up `observeIdentityStreaks`, `hoverLatches`, and `hoverBypassTabs` on session teardown and `tabs.onRemoved`; enabled Canvas `[visual:screenshot]` `@eN` parity on `cooperate sync`, budgeted the `renderDoubleLayer` `L2 page … occluded by L1` footer line within `maxTokens`, and moved popup `[expanded]` / `[has-submenu]` tags out of `vom.name` to post-quote VOM tags so custom dropdowns no longer emit duplicate `[expanded]` suffixes.
- Eliminated redundant back-to-back frame collection sweeps in `guardedRunner.runBrowserMutation` (`action-guard.ts`), skipped mutation fencing on passive `inspect` probes in `handleFullPageScreenshot`, and raised the per-frame cooperation IPC reply deadline (`FRAME_REPLY_TIMEOUT_MS` / `OWNER_REPLY_MS`) from `1200ms` to `3000ms` so cold high-DPR / software-rasterized full-page captures and Tier-A overlay mounts do not spuriously fail with `FrameUnresponsiveError` (#761).
- Added `--max-tokens` (`max_tokens` in MCP) to `bsk step`, `bsk plan`, `bsk_interact` (with `sync_after` or `until_visible`), and `bsk_plan`, forwarding it to the post-action `tool.cooperate` sync instead of hardcoding `None` so `--max-tokens` baselines stay reusable across `delta` and `unchanged` syncs without forcing an unbounded page re-capture (#755).
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ CLI 和扩展可以分别发布,能否连接由协议兼容性决定。更新

### 修复

- 修复现代 SPA 页面与含动态子 `<iframe>` 站点上高频误报 `"Page interaction coverage changed; wait and synchronize"`(`control_context_invalid` + `effect_state: "unknown"` 并触发守护进程 `IntentWal` 锁表)的系统性问题:通过 `ChromiumCdp.onRouteChanged`、`CooperationController.invalidateRoute` 与顶层帧 `FrameState.url` 将同文档 URL 更新(`pushState`、`replaceState` 与 `#hash`)同跨文档销毁解耦(同时在 `soft_detached` 重连与对话框关闭复核时恢复 `rootFrameId`、补齐同文档路由更新下的子帧对账,并移除同进程子 `<iframe>` 挂载/导航派发无参 `DOM.documentUpdated` 对顶层 `onDocumentChanged` 的误触发),不再将存活采集器误标为 `unavailable`,使多步动作(`fill`、`scroll-to`、`wheel`、`press`、`select`、`click`、`drag`)及 `bsk plan` 批处理在同路径查询参数或锚点更新下完整执行并保持 `target` 上下文可用(同时消费无锚点动作所需的严格基线),而在跨路径/Hash 路由跳转时待当前动作正常结算后再以 `effect_state: "none"` 失效旧上下文与引用;支持自动提交表单的 `fill` 在 `onBeforeNavigate` 及瞬时提交销毁旧执行上下文(`finalValueAlreadyNotified`)两种时序下正常完成 Step 3 回执,并放行已启动原生输入动作在 `onBeforeNavigate` 后的主帧/子帧收尾测量(`Runtime.callFunctionOn` 与 `DOM.getBoxModel`);将后台 `observeTab` 排在 `armCommittedFrame` 之后、支持 `COOPERATION_ARM` 与 `COOPERATION_RESUME` 为新挂载子帧初始化采集租约、允许 `collectCooperationFrames` 在已证隐藏的受限子帧旁完成兄弟子帧替换重采,并隔离非持有方并发 `refresh` 失败(`observeTab`、`status`、`wait` 及带参被动 `read`)对在途 `state.working` 操作与 `target` 上下文的污染;同时保留导航失败及携带结构化 `result.data.reason`(如 `download_capture_failed`、`fill_target_changed`)的真实错误回执,在守护进程 `ImplicitScopeRegistry` 中精准吊销已失效的隐式 `target` 自动注入上下文(完整保留 `last_vom` 与短别名以支持后续 `unchanged`/`delta` 同步回填),并支持在 `page_load_failed` 错误页上通过 `navigate`/`reload`/`navigate_back`/`navigate_forward` 直接恢复标签页(#762)。
- 修复 `ToolDispatcher` 持久后台执行租约下失焦或最小化窗口上的 `tool.wheel` 因耦合临时 `restoreFocus` 而跳过合成器渲染就绪检查(`waitForInputPaint`)的问题(将 `ReadyInput.hidden` 拆耦为 `lacksForegroundPresentation` 并在 `ToolDispatcher` 注入 `windowsApi`);修复 `CooperationController` 在 `target` 策略下执行 `--selector` 动作时污染 `session.refStore`、误输出 `used_ref`(同时保留 `--selector` 动作回执的 `used_selector` 且不污染 `tool.drag` 的 `DragResult` 与 `tool.press` 的 `PressResult`)或在 `handleSelect` 自定义下拉报错引导中泄漏伪造 `@eN` 引用的问题(改用请求级 `withPinnedSelectorTargets`);在 `target` 策略脚本写 DOM 前于 `CooperationController.beginDomMutation()` 内主动刷新实时帧活动,并在写入前校验以 `effect_state: "none"` 失败时保留 `state.baseline`(`observation_id`);修复 `clearRecordingForSession` 未使用调用方/派发器注入 `deps`(`RECORD_CANCEL` + `bypassOverlay(tabId, false)`)的问题,将 `returnBorrowedTab` 的 `isAgentWindowId` 设为必填契约以防归还借用标签页误落入其他会话的 Agent 窗口,以实例级 `DownloadConcurrencyGate`(`session_busy`、`effect_state: "none"`、`dispatched: false`)替换模块级全局下载锁,并在会话销毁及 `tabs.onRemoved` 时完整清理 `observeIdentityStreaks`、`hoverLatches` 与 `hoverBypassTabs`;补齐 `cooperate sync` 对 Canvas `[visual:screenshot]` `@eN` 引用的对齐,将 `renderDoubleLayer` 底部 `L2 page … occluded by L1` 遮挡汇总行纳入 `maxTokens` 预算核算,并将弹出状态标签 `[expanded]` / `[has-submenu]` 从 `vom.name` 迁至引号后 VOM 标签以消除自定义下拉重复输出 `[expanded]`。
- 消除 `guardedRunner.runBrowserMutation`(`action-guard.ts`)中重复的连续整标签页帧状态采集、跳过 `handleFullPageScreenshot` 中只读 `inspect` 探测步骤的变更门禁,并将协作帧 IPC 响应超时阈值(`FRAME_REPLY_TIMEOUT_MS` / `OWNER_REPLY_MS`)从 `1200ms` 放宽至 `3000ms`,防止高分屏(2x DPR)/ 软光栅冷启动与按需浮层挂载期间误报 `FrameUnresponsiveError`(#761)。
- 为 `bsk step`、`bsk plan`、MCP `bsk_interact`(搭配 `sync_after` 或 `until_visible`)及 `bsk_plan` 补齐 `--max-tokens`(MCP 为 `max_tokens`)参数并透传至后置 `tool.cooperate` 同步请求,解决此前硬编码 `None` 导致带预算会话在后续动作触发 `token_budget_changed` 并回退为整页无上限重采的问题(#755)。
Expand Down
59 changes: 50 additions & 9 deletions apps/extension/src/browser-driver/__tests__/chromium-cdp.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1721,16 +1721,20 @@ describe("document-bound references", () => {
await cdp.ensureAttached(4);
onEvent.fire({ tabId: 4 }, "Page.navigatedWithinDocument", { frameId: "root" });
expect(changed).not.toHaveBeenCalled();
for (const method of ["Page.frameNavigated", "Page.documentOpened", "DOM.documentUpdated"]) {
for (const method of ["Page.frameNavigated", "Page.documentOpened"]) {
onEvent.fire({ tabId: 4 }, method, { frame: { id: "root" } });
expect(changed).toHaveBeenLastCalledWith(4);
}
expect(changed).toHaveBeenCalledTimes(3);
expect(changed).toHaveBeenCalledTimes(2);
// In Chromium, same-process child iframe navigations/srcdoc mounts emit
// parameterless `DOM.documentUpdated` on the main target; that must not
// masquerade as a root document replacement.
onEvent.fire({ tabId: 4 }, "DOM.documentUpdated", {});
onEvent.fire({ tabId: 4 }, "Page.navigatedWithinDocument", { frameId: "child" });
onEvent.fire({ tabId: 4, sessionId: "child-session" }, "Page.navigatedWithinDocument", {
frameId: "root",
});
expect(changed).toHaveBeenCalledTimes(3);
expect(changed).toHaveBeenCalledTimes(2);
// Every route event revokes, including away and back before the next observation.
onEvent.fire({ tabId: 4 }, "Page.navigatedWithinDocument", {
frameId: "root",
Expand All @@ -1740,12 +1744,43 @@ describe("document-bound references", () => {
frameId: "root",
url: "https://site.test/",
});
expect(changed).toHaveBeenCalledTimes(5);
expect(changed).toHaveBeenCalledTimes(4);
await cdp.detach(4);
expect(changed).toHaveBeenCalledTimes(6);
expect(changed).toHaveBeenCalledTimes(5);
await cdp.ensureAttached(4);
onDetach.fire({ tabId: 4 }, "target_closed");
expect(changed).toHaveBeenCalledTimes(7);
expect(changed).toHaveBeenCalledTimes(6);
cdp.dispose();
});

it("routes root Page.navigatedWithinDocument to onRouteChanged when provided without firing onDocumentChanged", async () => {
const { api, onEvent } = fakeApi();
const docChanged = vi.fn();
const routeChanged = vi.fn();
const cdp = new ChromiumCdp(api, {
onDocumentChanged: docChanged,
onRouteChanged: routeChanged,
});
await cdp.ensureAttached(4);
onEvent.fire({ tabId: 4 }, "Page.frameNavigated", {
frame: { id: "root", loaderId: "loader-1", url: "https://site.test/" },
});
expect(docChanged).toHaveBeenCalledExactlyOnceWith(4);
expect(routeChanged).not.toHaveBeenCalled();
docChanged.mockClear();

onEvent.fire({ tabId: 4 }, "Page.navigatedWithinDocument", {
frameId: "child",
url: "https://site.test/child#2",
});
expect(routeChanged).not.toHaveBeenCalled();

onEvent.fire({ tabId: 4 }, "Page.navigatedWithinDocument", {
frameId: "root",
url: "https://site.test/?q=shoes#top",
});
expect(routeChanged).toHaveBeenCalledExactlyOnceWith(4, "https://site.test/?q=shoes#top");
expect(docChanged).not.toHaveBeenCalled();
cdp.dispose();
});

Expand Down Expand Up @@ -2941,6 +2976,7 @@ describe("idle soft-detach and immediate close/return detach", () => {
vi.useFakeTimers();
const { api, onEvent } = fakeApi();
let attachGeneration = 0;
let rootUrl = "https://host.test/";
let childADocElemBackendNodeId = 201;
let childBState: { present: boolean; loaderId: string; url: string } = {
present: true,
Expand Down Expand Up @@ -2984,7 +3020,7 @@ describe("idle soft-detach and immediate close/return detach", () => {
if (!target.sessionId) {
return {
frameTree: {
frame: { id: "root", loaderId: "root-loader-1", url: "https://host.test/" },
frame: { id: "root", loaderId: "root-loader-1", url: rootUrl },
childFrames: [
{ frame: { id: "child-a", parentId: "root", url: "https://oopif-a.test/pay" } },
...(childBState.present
Expand Down Expand Up @@ -3041,7 +3077,7 @@ describe("idle soft-detach and immediate close/return detach", () => {
return {
root: {
backendNodeId: 1,
documentURL: "https://host.test/",
documentURL: rootUrl,
children: [{ nodeType: 1, backendNodeId: 101 }],
},
};
Expand All @@ -3054,11 +3090,13 @@ describe("idle soft-detach and immediate close/return detach", () => {

const refs = new RefStore();
const docChanged = vi.fn((tabId: number) => refs.invalidateTab(tabId));
const routeChanged = vi.fn();
const childInvalidated = vi.fn((tabId: number, staleFrameIds: ReadonlySet<string>) => {
refs.invalidateChildFrames(tabId, staleFrameIds);
});
const cdp = new ChromiumCdp(api, {
onDocumentChanged: docChanged,
onRouteChanged: routeChanged,
onChildFramesInvalidated: childInvalidated,
idleDetachMs: DEFAULT_IDLE_DETACH_MS,
});
Expand Down Expand Up @@ -3118,9 +3156,11 @@ describe("idle soft-detach and immediate close/return detach", () => {
expect(childInvalidated).not.toHaveBeenCalled();
expect(refs.revision).toBe(initialRevision);

// Soft-detach #2 -> `child-a` gets `s3-a` (unchanged), `child-b` navigates (`child-b-loader-2`).
// Soft-detach #2 -> root updates `?q=shoes` in-place, `child-a` gets `s3-a` (unchanged),
// `child-b` navigates (`child-b-loader-2`). Both `onRouteChanged` and child reconciliation run!
await vi.advanceTimersByTimeAsync(DEFAULT_IDLE_DETACH_MS);
expect(cdp.isAttached(53)).toBe(false);
rootUrl = "https://host.test/?q=shoes";
childBState = {
present: true,
loaderId: "child-b-loader-2",
Expand All @@ -3136,6 +3176,7 @@ describe("idle soft-detach and immediate close/return detach", () => {
expect((await resolveGen3).routedSessionId).toBe("s3-a");

expect(docChanged).not.toHaveBeenCalled();
expect(routeChanged).toHaveBeenCalledExactlyOnceWith(53, "https://host.test/?q=shoes");
expect(childInvalidated).toHaveBeenCalledExactlyOnceWith(53, new Set(["child-b"]));
// Main ref `@e1` and unchanged OOPIF ref `@e2` remain valid; navigated OOPIF `@e3` is invalidated!
expect(refs.resolve("e1", { tabId: 53 })).toBe(10);
Expand Down
Loading
Loading