Skip to content

refactor(extension,vom): seal browser-driver boundary, enforce RAII CdpRunner, and unify DOM predicates & VOM metadata - #753

Merged
NianJiuZst merged 3 commits into
Tencent:mainfrom
NianJiuZst:refactor/wave4c-extension-driver-boundary-and-vom-unification
Nov 17, 2026
Merged

NianJiuZst merged 3 commits into
Tencent:mainfrom
NianJiuZst:refactor/wave4c-extension-driver-boundary-and-vom-unification

Conversation

@NianJiuZst

Copy link
Copy Markdown
Collaborator

Summary

Completes Wave 4C across apps/extension, @browser-skill/vom, and crates/bsk-cli:

  1. Complete browser-driver/ Boundary Encapsulation:
    • Sealed CdpTarget, CdpFrame, CdpFrameGraph, sendToTarget, sendGuarded, sendAttached, and getFrameGraph strictly inside apps/extension/src/browser-driver/.
    • Migrated all remaining upper-layer modules (read, extract, waits, wheel, drag, action-guard, vom/facts, AccessibilityLease, debug/, recording-runtime, and geometry/frame-context) onto FrameDocumentHandle, FrameNodeHandle, FrameProcessBoundary, and TabEventSource (BskDriver.childTargetAttached / BskDriver.childTargetDetached).
  2. Non-Optional CdpRunner & Deterministic RAII V8 Handle Release:
    • Promoted all CdpRunner methods to a non-optional interface backed by delegateCdpRunner and createStubCdpRunner.
    • Enforced deterministic RAII withObjectGroup / withRemoteObject V8 handle cleanup across handleSelect, handleFill, handleFocus/handleBlur, action-checks, file-input-transaction, and evaluate(return_by_value: false) via cleanup.sendToFrame(..., { allowReattach: false }) so cleanup on throw/timeout/abort never resurrects detached/soft-detached tabs, resets the 75s Soft-Detach idle timer, or masks pre-dispatch errors.
  3. Unified DOM Predicates, --selector Resolution & Session Isolation:
    • Extracted shared DOM predicates (apps/extension/src/lib/dom-predicates.ts) and cross-frame / open Shadow DOM --selector resolution (apps/extension/src/tools/selector-resolution.ts) across interaction, CooperationController, human-loop, read, and extract.
    • Fixed tool.wheel compositor paint readiness (waitForInputPaint) under persistent background leases (lacksForegroundPresentation), isolated target-policy --selector refs (withPinnedSelectorTargets), required explicit isAgentWindowId on returnBorrowedTab, and replaced module-global download locking with instance-scoped DownloadConcurrencyGate.
  4. Structured VomLineMetadata & Cross-Language Contract Alignment:
    • Converged @browser-skill/vom, cooperation/text-delta.ts, and crates/bsk-cli/src/cli/recipe/vom.rs (crates/bsk-cli/tests/fixtures/vom_contract.json) onto structured VomLineMetadata (inspectVomLine, VOM_DERIVED_NAME_CAP), post-quote [expanded] / [has-submenu] tags, budgeted renderDoubleLayer occlusion footers, and Canvas [visual:screenshot] @eN parity on cooperate sync.
    • Synced bilingual [Unreleased] entries in CHANGELOG.md and CHANGELOG.zh-CN.md (rebased onto latest main including fix(vom): handle unpaired surrogates in token estimates #746).

Verification

  • Static & Contract Gates:
    • pnpm --filter @browser-skill/extension exec tsc --noEmit
    • pnpm ext:build
    • npx biome check . & pnpm lint
    • cargo clippy --workspace --all-targets --locked -- -D warnings
    • pnpm protocol:check
    • python3 scripts/check-changelog-sync.py
    • rg boundary encapsulation check across apps/extension/src outside browser-driver/ (0 hits)
  • Fast Subpackage & Targeted Tests:
    • pnpm vom:test (182/182 passed)
    • cargo test -p bsk --lib vom (15/15 passed) & cargo test -p bsk-protocol (182/182 passed)
    • python3 -m unittest evals/browser/skills/browserskill-release-test/scripts/test_release_audit.py (23/23 passed)
    • 66 rg-matched & modified @browser-skill/extension Vitest unit suites (~2,400+ tests, 0 failures)
    • Real-browser smoke test: BSK_TEST_CHROME + BSK_TEST_LOAD_CDP=1 node scripts/run-browser-tests.mjs apps/extension/src/tools/__tests__/click.browser.test.ts (3/3 passed)
    • Full workspace & browser regression suites deferred to PR CI per tiered testing policy.

…dpRunner, and unify DOM predicates & VOM metadata

Why:
- Upper extension layers (`read`, `extract`, `waits`, `wheel`, `drag`,
  `action-guard`, `vom/facts`, `AccessibilityLease`, `debug/`,
  `recording-runtime`, and `geometry/frame-context`) still reached into
  `browser-driver/frame-graph` internals (`CdpTarget`, `CdpFrame`,
  `CdpFrameGraph`, `sendToTarget`, `sendGuarded`, `sendAttached`,
  `getFrameGraph`) and transient OOPIF session routing.
- `CdpRunner` carried optional `?` methods for handwritten test stubs, while V8
  `objectGroup` / `objectId` handles in `handleSelect`, `handleFill`,
  `handleFocus`/`handleBlur`, `action-checks`, `file-input-transaction`, and
  `evaluate(return_by_value: false)` risked leaks on throw/abort or could
  resurrect soft-detached tabs / mask pre-dispatch cooperation errors during
  `finally` cleanup.
- `--selector` resolution, `[data-bsk-overlay]` exclusion, open Shadow DOM
  walks, sensitive `autocomplete` checks, interactive ARIA roles, and VOM line
  metadata (`text-delta` regex re-parsing vs `@browser-skill/vom`) had drifted
  across independent implementations.

What:
- Seal `CdpTarget`, `CdpFrame`, `CdpFrameGraph`, `sendToTarget`, `sendGuarded`,
  `sendAttached`, and `getFrameGraph` strictly inside
  `apps/extension/src/browser-driver/` behind `FrameDocumentHandle`,
  `FrameNodeHandle`, `FrameProcessBoundary`, and `TabEventSource`
  (`BskDriver.childTargetAttached` / `BskDriver.childTargetDetached`).
- Promote all `CdpRunner` methods to a non-optional contract backed by
  `delegateCdpRunner` and `createStubCdpRunner`, and enforce deterministic RAII
  `withObjectGroup` / `withRemoteObject` V8 handle release via
  `cleanup.sendToFrame(..., { allowReattach: false })` without reattaching
  detached/soft-detached tabs, resetting idle timers, or masking caller errors.
- Unify DOM predicates (`apps/extension/src/lib/dom-predicates.ts`) and
  cross-frame / open Shadow DOM `--selector` resolution
  (`apps/extension/src/tools/selector-resolution.ts`), fix background-lease
  `tool.wheel` paint readiness (`lacksForegroundPresentation`), isolate
  `target`-policy `--selector` refs via `withPinnedSelectorTargets`, require
  explicit `isAgentWindowId` on `returnBorrowedTab`, and replace module-global
  download state with instance-scoped `DownloadConcurrencyGate`.
- Converge `@browser-skill/vom`, `cooperation/text-delta.ts`, and
  `crates/bsk-cli/src/cli/recipe/vom.rs` (`vom_contract.json`) onto structured
  `VomLineMetadata` (`inspectVomLine`, `VOM_DERIVED_NAME_CAP`), post-quote
  `[expanded]` / `[has-submenu]` tags, budgeted `renderDoubleLayer` occlusion
  footers, and Canvas `[visual:screenshot]` `@eN` parity on `cooperate sync`.
- Sync bilingual `[Unreleased]` entries in `CHANGELOG.md` and
  `CHANGELOG.zh-CN.md`.
… cleanup, and rustfmt gaps

Why:
- `GeometryContext.endpointForFrame` returned frame-scoped `frame.handle`
  instead of `frameProcessBoundary(frame)`, splitting cache keys from
  process-scoped `layoutMetrics` queries and doubling `Page.getLayoutMetrics`
  IPC calls on multi-frame / OOPIF pages.
- `createStubCdpRunner` dropped `CdpDispatchGuard` (`signal`, `attachmentId`,
  `onDispatch`) when 3-argument `sendToFrame` overrides were supplied and
  closed over `stub.send` instead of `this.send` on spread clones, breaking
  `CooperationController` dispatch accounting, `pointerTransactionDeps`
  mouse-release tracking, and mid-read `resolveVisualRegionNow` overrides.
- `ChromiumCdp.sendToFrame` / `resolveValidatedFrameRoute` checked
  `ignoreFrameNavigation` instead of `isCleanup` before dispatching
  `Runtime.releaseObjectGroup` / `Runtime.releaseObject`, risking redundant
  `Page.getFrameTree` refreshes or `StaleFrameHandleError` after same-process
  frame navigation; `cdpRunnerForFrame` forwarded aborted `guard.signal` into
  `Runtime.releaseObjectGroup` on cancelled VOM captures; `waits`,
  `element-geometry` (`scrollNodeIntoView` / `descendantBoundingRect`),
  `visual-target`, `vom/document-identity`, and `viewport-screenshot` had
  remaining V8 handle release / `allowReattach: false` gaps; and `vom.rs` had
  an unformatted line failing `cargo fmt --check`.

What:
- Route `GeometryContext.endpointForFrame` via `frameProcessBoundary(frame)` so
  `Page.getLayoutMetrics` and owner caches share one key per renderer process.
- Update `createStubCdpRunner` to resolve `this` on spread clones and enforce
  `CdpDispatchGuard` (`signal`, `allowReattach`, `attachmentId`, `onDispatch`)
  on `<4`-argument `sendToFrame` overrides (including Vitest `vi.fn` spies).
- Harden `ChromiumCdp.sendToFrame` / `resolveValidatedFrameRoute`,
  `cdpRunnerForFrame`, `waits`, `element-geometry`, `scroll`, `wheel`,
  `pointer-transaction`, `file-input-transaction`, `file-drop-transaction`,
  `visual-target`, `vom/document-identity`, and `viewport-screenshot` so V8
  handle cleanup bypasses aborted signals, skips stale-loader rejections, and
  never reattaches detached tabs or masks caller outcomes.
- Format `crates/bsk-cli/src/cli/recipe/vom.rs` (`cargo fmt --all`), add
  regression tests across `chromium-cdp.test.ts`, `geometry-context.test.ts`,
  `form-state.test.ts`, `element-geometry.test.ts`, and `waits.test.ts`, and
  sync bilingual `CHANGELOG.md` / `CHANGELOG.zh-CN.md`.
@NianJiuZst
NianJiuZst force-pushed the refactor/wave4c-extension-driver-boundary-and-vom-unification branch from 6768297 to 60d2918 Compare November 17, 2026 10:55
…tor receipts cleanly, and pin happy-dom MO callbacks

Why:
1. `BSK_OVERLAY_SELECTOR` interpolated `[data-bsk-overlay],[${OVERLAY_HOST_MARKER_ATTR}]` where `OVERLAY_HOST_MARKER_ATTR` is already `"data-bsk-overlay"`, silently dropping legacy `[data-bsk-overlay-host]` exclusion from `READ_REDACTED_ANCESTOR_SELECTOR` (`tool.read`) and `extract` compared to pre-unification predicates, while `content/dom-primitives.ts` still used `OVERLAY_HOST_SELECTOR`.
2. Switching `target`-policy `--selector` actions to request-scoped `withPinnedSelectorTargets` avoided `used_ref` pollution, but `CooperationController.runAction` needed to preserve `used_selector` on `--selector` action receipts (`click`, `fill`, `hover`, `select`, `focus`, `blur`, `scroll_to`, `wheel`, `upload`, `download`) without injecting an undocumented `used_selector` field onto `tool.drag`'s `DragResult` (`params.source` / `params.target`) or `tool.press`'s `PressResult` (where `PressParams` accepts `params.selector` as an optional focus target while `PressResult` omits `used_selector`).
3. `evaluate.test.ts` still imported internal `type CdpDebuggee` from `@/browser-driver/chromium-cdp`.
4. `happy-dom` 15.x constructs `MutationObserverListener` with `new WeakRef((record) => this.report(record))` without retaining the closure, causing V8 young-gen scavenges across async turns to drop active `MutationObserver` callbacks in `unit-dom` tests (`record-frame-agent.test.ts`), while `WindowContextClassExtender` subclasses `MutationObserverImplementation` per `Window`.

What:
- Add `OVERLAY_HOST_LEGACY_ATTR = "data-bsk-overlay-host"` to `BSK_OVERLAY_SELECTOR` in `lib/dom-predicates.ts` and reuse it in `content/dom-primitives.ts` (`isExtensionOverlayElement` / `isExtensionOverlayNode`), with regression coverage in `tools/read/read.test.ts`.
- Populate `used_selector` on object action results in `CooperationController.runAction` when `params.selector` was supplied and `req.method !== "tool.press"` (including `tool.download` before its early return) without polluting `DragResult` or `PressResult`, covered in `cooperation/target-policy.test.ts` and real-browser `cooperation/live-text.browser.test.ts`.
- Derive the `sendCommand` target parameter type via `Parameters<CdpDebuggerApi["sendCommand"]>[0]` in `tools/__tests__/evaluate.test.ts`.
- Patch `happy-dom`'s shared `MutationObserverImplementation` prototype once per V8 isolate in `src/vitest-setup.ts` so both top-level and subframe `Window` `MutationObserver` instances pin active callbacks in a `WeakMap` keyed by each `mutationListener` record while still releasing upon `disconnect()`.
- Sync `CHANGELOG.md` and `CHANGELOG.zh-CN.md`.
@NianJiuZst
NianJiuZst force-pushed the refactor/wave4c-extension-driver-boundary-and-vom-unification branch from 60d2918 to 1e98f45 Compare November 17, 2026 11:24
@NianJiuZst
NianJiuZst merged commit 3d4c1bb into Tencent:main Nov 17, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant