Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ determines which versions can connect.

- Fixed `tool.wheel` skipping compositor paint readiness (`waitForInputPaint`) under `ToolDispatcher` persistent background leases on unfocused or minimized windows by decoupling `ReadyInput.hidden` (`lacksForegroundPresentation`) from temporary `restoreFocus` and wiring `windowsApi` in `ToolDispatcher`; stopped `target`-policy `--selector` actions in `CooperationController` from polluting `session.refStore`, emitting `used_ref` (while preserving `used_selector` on `--selector` action receipts without polluting `tool.drag`'s `DragResult` or `tool.press`'s `PressResult`), or leaking hallucinated `@eN` references in `handleSelect` custom-dropdown error messages (`withPinnedSelectorTargets`); refreshed live frame activity inside `CooperationController.beginDomMutation()` under `target` policy before scripted DOM writes while preserving `state.baseline` (`observation_id`) when pre-write validation fails with `effect_state: "none"`; fixed `clearRecordingForSession` to use the caller/dispatcher's `deps` (`RECORD_CANCEL` + `bypassOverlay(tabId, false)`), required explicit `isAgentWindowId` in `returnBorrowedTab` so returning a borrowed user tab never routes into another session's Agent Window, replaced module-global download locking with instance-scoped `DownloadConcurrencyGate` (`session_busy`, `effect_state: "none"`, `dispatched: false`), and cleaned up `observeIdentityStreaks`, `hoverLatches`, and `hoverBypassTabs` on session teardown and `tabs.onRemoved`; enabled Canvas `[visual:screenshot]` `@eN` parity on `cooperate sync`, budgeted the `renderDoubleLayer` `L2 page … occluded by L1` footer line within `maxTokens`, and moved popup `[expanded]` / `[has-submenu]` tags out of `vom.name` to post-quote VOM tags so custom dropdowns no longer emit duplicate `[expanded]` suffixes.
- Eliminated redundant back-to-back frame collection sweeps in `guardedRunner.runBrowserMutation` (`action-guard.ts`), skipped mutation fencing on passive `inspect` probes in `handleFullPageScreenshot`, and raised the per-frame cooperation IPC reply deadline (`FRAME_REPLY_TIMEOUT_MS` / `OWNER_REPLY_MS`) from `1200ms` to `3000ms` so cold high-DPR / software-rasterized full-page captures and Tier-A overlay mounts do not spuriously fail with `FrameUnresponsiveError` (#761).
- Bounded Chrome debugger attachment waits to 10 seconds, returned a clear error when an attach does not answer, and fenced late attach completions so they are detached before a retry (#452).
- Added `--max-tokens` (`max_tokens` in MCP) to `bsk step`, `bsk plan`, `bsk_interact` (with `sync_after` or `until_visible`), and `bsk_plan`, forwarding it to the post-action `tool.cooperate` sync instead of hardcoding `None` so `--max-tokens` baselines stay reusable across `delta` and `unchanged` syncs without forcing an unbounded page re-capture (#755).
- Ignored internal `data-bsk-*` attributes and `<browser-skill-overlay>` (`[data-bsk-overlay]`) ShadowHost mutations in `cooperation-capture.ts` so active recording and sensitive-field markers no longer advance `FrameState.structure`, excluded extension ShadowRoot UI controls (`HelpRequestOverlay`, `RecordOverlay`, `BorrowConfirmationOverlay`) across `record-capture.ts`, `record-hover-surface.ts`, and `cooperation-capture.ts` so overlay interactions and hover surfaces are never mis-recorded as webpage steps, and re-evaluated tab control ownership after awaiting `ensureOverlayInjected` so mid-injection tab detachments never flash stale `control` state.
- VOM token estimation now handles unpaired UTF-16 surrogates without combining them with the following symbol or throwing a `RangeError`; valid emoji pairs retain their existing estimates.
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ CLI 和扩展可以分别发布,能否连接由协议兼容性决定。更新

- 修复 `ToolDispatcher` 持久后台执行租约下失焦或最小化窗口上的 `tool.wheel` 因耦合临时 `restoreFocus` 而跳过合成器渲染就绪检查(`waitForInputPaint`)的问题(将 `ReadyInput.hidden` 拆耦为 `lacksForegroundPresentation` 并在 `ToolDispatcher` 注入 `windowsApi`);修复 `CooperationController` 在 `target` 策略下执行 `--selector` 动作时污染 `session.refStore`、误输出 `used_ref`(同时保留 `--selector` 动作回执的 `used_selector` 且不污染 `tool.drag` 的 `DragResult` 与 `tool.press` 的 `PressResult`)或在 `handleSelect` 自定义下拉报错引导中泄漏伪造 `@eN` 引用的问题(改用请求级 `withPinnedSelectorTargets`);在 `target` 策略脚本写 DOM 前于 `CooperationController.beginDomMutation()` 内主动刷新实时帧活动,并在写入前校验以 `effect_state: "none"` 失败时保留 `state.baseline`(`observation_id`);修复 `clearRecordingForSession` 未使用调用方/派发器注入 `deps`(`RECORD_CANCEL` + `bypassOverlay(tabId, false)`)的问题,将 `returnBorrowedTab` 的 `isAgentWindowId` 设为必填契约以防归还借用标签页误落入其他会话的 Agent 窗口,以实例级 `DownloadConcurrencyGate`(`session_busy`、`effect_state: "none"`、`dispatched: false`)替换模块级全局下载锁,并在会话销毁及 `tabs.onRemoved` 时完整清理 `observeIdentityStreaks`、`hoverLatches` 与 `hoverBypassTabs`;补齐 `cooperate sync` 对 Canvas `[visual:screenshot]` `@eN` 引用的对齐,将 `renderDoubleLayer` 底部 `L2 page … occluded by L1` 遮挡汇总行纳入 `maxTokens` 预算核算,并将弹出状态标签 `[expanded]` / `[has-submenu]` 从 `vom.name` 迁至引号后 VOM 标签以消除自定义下拉重复输出 `[expanded]`。
- 消除 `guardedRunner.runBrowserMutation`(`action-guard.ts`)中重复的连续整标签页帧状态采集、跳过 `handleFullPageScreenshot` 中只读 `inspect` 探测步骤的变更门禁,并将协作帧 IPC 响应超时阈值(`FRAME_REPLY_TIMEOUT_MS` / `OWNER_REPLY_MS`)从 `1200ms` 放宽至 `3000ms`,防止高分屏(2x DPR)/ 软光栅冷启动与按需浮层挂载期间误报 `FrameUnresponsiveError`(#761)。
- 为 Chrome debugger attach 增加 10 秒上限;未响应时返回明确错误,并在允许重试前隔离并回滚迟到的 attach 完成(#452)。
- 为 `bsk step`、`bsk plan`、MCP `bsk_interact`(搭配 `sync_after` 或 `until_visible`)及 `bsk_plan` 补齐 `--max-tokens`(MCP 为 `max_tokens`)参数并透传至后置 `tool.cooperate` 同步请求,解决此前硬编码 `None` 导致带预算会话在后续动作触发 `token_budget_changed` 并回退为整页无上限重采的问题(#755)。
- 在 `cooperation-capture.ts` 中忽略扩展内部私有 `data-bsk-*` 属性与 `<browser-skill-overlay>`(`[data-bsk-overlay]`)ShadowHost 变更以消除录制/敏感字段标记对 `FrameState.structure` 的自污染,在 `record-capture.ts`、`record-hover-surface.ts` 与 `cooperation-capture.ts` 中统一过滤扩展自身 ShadowRoot 浮层控件(`HelpRequestOverlay`、`RecordOverlay`、`BorrowConfirmationOverlay`)防止浮层内点击/输入或悬停面被误录为网页操作,并在 `ensureOverlayInjected` 异步注入完成后二次校验标签页受控归属以杜绝注入途中断开控制导致的陈旧浮层闪现。
- VOM Token 估算现在能处理未配对的 UTF-16 代理字符,不会再误合并后续符号或抛出 `RangeError`;有效 emoji 代理对的估算保持不变。
Expand Down
41 changes: 41 additions & 0 deletions apps/extension/src/browser-driver/__tests__/chromium-cdp.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import {
type CdpDebuggee,
type CdpDebuggerApi,
ChromiumCdp,
DEBUGGER_ATTACH_FENCE_WAIT_MS,
DEBUGGER_ATTACH_TIMEOUT_MS,
DEFAULT_IDLE_DETACH_MS,
} from "../chromium-cdp";
import { CdpReadTimeoutError, READ_TIMEOUT_MS } from "../command-deadline";
Expand Down Expand Up @@ -1981,6 +1983,45 @@ it("detaches on failed policy release even when a passive reader remains", async
});

describe("bounded debugger cleanup", () => {
it("bounds a stuck attach and rolls back a late success before retrying", async () => {
vi.useFakeTimers();
try {
const { api } = fakeApi();
let finishAttach!: () => void;
vi.mocked(api.attach).mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
finishAttach = resolve;
}),
);
const cdp = new ChromiumCdp(api);

const first = cdp.ensureAttached(5);
const firstRejected = expect(first).rejects.toThrow(/debugger\.attach.*10000ms/i);
await vi.advanceTimersByTimeAsync(DEBUGGER_ATTACH_TIMEOUT_MS);
await firstRejected;
expect(cdp.isAttached(5)).toBe(false);

const retryWhilePending = cdp.ensureAttached(5);
const retryRejected = expect(retryWhilePending).rejects.toThrow(/still settling/);
await vi.advanceTimersByTimeAsync(DEBUGGER_ATTACH_FENCE_WAIT_MS);
await retryRejected;
expect(api.attach).toHaveBeenCalledTimes(1);

finishAttach();
await vi.advanceTimersByTimeAsync(0);
expect(api.detach).toHaveBeenCalledWith({ tabId: 5 });

await cdp.ensureAttached(5);
expect(api.attach).toHaveBeenCalledTimes(2);
expect(cdp.isAttached(5)).toBe(true);
await cdp.detachAll();
cdp.dispose();
} finally {
vi.useRealTimers();
}
});

it("fences a late attach through rollback, including Chrome's detach event", async () => {
vi.useFakeTimers();
try {
Expand Down
88 changes: 79 additions & 9 deletions apps/extension/src/browser-driver/chromium-cdp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ const MAX_NETWORK_FIELD_LENGTH = 4096;
const MAX_NETWORK_REQUEST_META = 1024;
const FRAME_DISCOVERY_TIMEOUT_MS = 1000;
const FRAME_DISCOVERY_QUIET_MS = 20;
export const DEBUGGER_ATTACH_TIMEOUT_MS = 10_000;
export const DEBUGGER_ATTACH_FENCE_WAIT_MS = 1_000;

interface ParsedDialogOpening {
type: JavaScriptDialogType;
Expand Down Expand Up @@ -319,6 +321,7 @@ export type TabAttachment =
version: number;
snapshot: SoftDetachSnapshot;
detachInFlight?: Promise<void>;
fencedAttach?: Promise<void>;
leases: Set<TabLease>;
eventListeners: Set<TabEventListener>;
history: TabHistoryState;
Expand Down Expand Up @@ -686,9 +689,19 @@ export class ChromiumCdp {
await entry.detachInFlight;
entry = this.tabs.get(tabId);
}
if (entry?.kind === "detached" && entry.fencedAttach) {
await entry.fencedAttach;
return;
if ((entry?.kind === "detached" || entry?.kind === "soft_detached") && entry.fencedAttach) {
const fencedAttach = entry.fencedAttach;
if (!(await cleanupWait(fencedAttach, DEBUGGER_ATTACH_FENCE_WAIT_MS))) {
throw new Error(
`A previous debugger attachment to tab ${tabId} is still settling. Wait for the browser to reconnect and retry.`,
);
}
entry = this.tabs.get(tabId);
if (entry?.kind === "attached") {
await this.verifyRootDocumentAfterDialog(tabId);
this.touchTabActivity(tabId);
return;
}
}
if (entry?.kind === "attaching") {
await entry.attachInFlight;
Expand Down Expand Up @@ -737,8 +750,53 @@ export class ChromiumCdp {
}
};

let attachTimedOut = false;
let lateAttachCleanupSettled = false;
let lateAttachCleanup: Promise<void> | undefined;
try {
await this.api.attach({ tabId }, CDP_PROTOCOL_VERSION);
const rawAttach = this.api.attach({ tabId }, CDP_PROTOCOL_VERSION);
lateAttachCleanup = rawAttach
.then(
async () => {
if (!attachTimedOut) return;
try {
await this.api.detach({ tabId });
} catch (detachErr) {
console.debug("[bsk cdp] late attach rollback failed", { tabId, detachErr });
}
},
() => {},
)
.then(() => {
lateAttachCleanupSettled = true;
const current = this.tabs.get(tabId);
if (
(current?.kind === "detached" || current?.kind === "soft_detached") &&
current.fencedAttach === lateAttachCleanup
) {
current.fencedAttach = undefined;
this.maybePruneDetachedEntry(tabId);
}
});

let attachTimer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
rawAttach,
new Promise<never>((_, reject) => {
attachTimer = setTimeout(() => {
attachTimedOut = true;
reject(
new Error(
`Chrome did not respond to debugger.attach within ${DEBUGGER_ATTACH_TIMEOUT_MS}ms.`,
),
);
}, DEBUGGER_ATTACH_TIMEOUT_MS);
}),
]);
} finally {
clearTimeout(attachTimer);
}
attachingEntry.rawAttached = true;
try {
check();
Expand Down Expand Up @@ -897,21 +955,33 @@ export class ChromiumCdp {
kind: "soft_detached",
version: attachingEntry.version,
snapshot: previousSoftDetach,
...(attachTimedOut && lateAttachCleanup && !lateAttachCleanupSettled
? { fencedAttach: lateAttachCleanup }
: {}),
leases: attachingEntry.leases,
eventListeners: attachingEntry.eventListeners,
history: attachingEntry.history,
}
: {
kind: "detached",
version: attachingEntry.version,
...(attachTimedOut && lateAttachCleanup && !lateAttachCleanupSettled
? { fencedAttach: lateAttachCleanup }
: {}),
leases: attachingEntry.leases,
eventListeners: attachingEntry.eventListeners,
history: attachingEntry.history,
},
);
this.maybePruneDetachedEntry(tabId);
} else if (latest?.kind === "detached" && latest.fencedAttach === attachInFlight) {
latest.fencedAttach = undefined;
} else if (
(latest?.kind === "detached" || latest?.kind === "soft_detached") &&
latest.fencedAttach === attachInFlight
) {
latest.fencedAttach =
attachTimedOut && lateAttachCleanup && !lateAttachCleanupSettled
? lateAttachCleanup
: undefined;
this.maybePruneDetachedEntry(tabId);
}
void attachInFlight.catch(() => {});
Expand Down Expand Up @@ -1786,7 +1856,7 @@ export class ChromiumCdp {
const existingFencedAttach =
entry.kind === "attaching"
? entry.attachInFlight
: entry.kind === "detached"
: entry.kind === "detached" || entry.kind === "soft_detached"
? entry.fencedAttach
: undefined;

Expand Down Expand Up @@ -1979,7 +2049,7 @@ export class ChromiumCdp {
fencedAttach:
entry.kind === "attaching"
? entry.attachInFlight
: entry.kind === "detached"
: entry.kind === "detached" || entry.kind === "soft_detached"
? entry.fencedAttach
: undefined,
leases: entry.leases,
Expand Down Expand Up @@ -3269,7 +3339,7 @@ export class ChromiumCdp {
fencedAttach:
entry.kind === "attaching"
? entry.attachInFlight
: entry.kind === "detached"
: entry.kind === "detached" || entry.kind === "soft_detached"
? entry.fencedAttach
: undefined,
leases: entry.leases,
Expand Down