Skip to content

fix: snapshot tool schemas and align verification guidance - #26

Open
TueJon wants to merge 4 commits into
mainfrom
chore/adapter-proof
Open

TueJon wants to merge 4 commits into
mainfrom
chore/adapter-proof

Conversation

@TueJon

@TueJon TueJon commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Workbench schema metadata currently shares nested objects with callers, so changing a supplied or enumerated schema silently changes later evidence. Optional outputSchema is also absent from types and contract refresh comparisons.

The simulation now snapshots both schemas through JSON, rejects serialization failures before registration, and returns fresh copies. Types and Workbench comparison/display/refresh support optional outputSchema, with compile, unit and read-only browser regressions. The existing TS/JS runtime wrappers already forward metadata and remain unchanged.

The reference updates cover existing framework tool ownership, optional native Playwright verification, cross-origin identity, current Site tools form/frame discovery, and output-only contract invalidation. They keep draft serialization distinct from measured Chrome scalar behavior. Runtime dependencies, defaults, distribution versions and mutation-journal behavior do not change.

Validation: npm run check passes all type/syntax checks and 83/83 tests; node tests/schema-browser.mjs proves output-only drift updates the observed contract UI. The local headed Chrome 156.0.8078.47 proof recorded ten read-only scalar cases, schema snapshot/error behavior and target-document policy denial. Native outputSchema propagation is unsupported in that browser; the about:blank document-replacement probe timed out and is not a pass. No WPT, account-availability or universal framework certification claim.

Source-only PR; no merge, release or deployment. Companion site-copy PR updates the public summaries. Approvals d-20261102-0740 and d-20261109-0644 cover the scoped reference edits.

November 16 bounded lifecycle proof

Chrome156/PW1.61.1 read-only loopback observations confirm callback-before-activation and policy-gated ancestor/sibling discovery. This extension corrects ambient-type comments only: activation is after callback invocation, discovery covers the same traversable, and opaque origin strings are not identity proofs. Runtime, guides and site PR24 stay unchanged.

Local npm run check: 83/83 plus type/syntax checks. Twenty native rows observed; four remain unavailable (Symbol-result execution and saved-context register/getTools/execute after navigation timed out). Five simulation serialization negatives reject. Detached-context calls reject InvalidStateError; opaque own-tool execution remains unsupported on Chrome156. These custom probes are not WPT conformance. Continuations proposal inspected only; no feature, browser upgrade, retry, merge or deploy.

@TueJon TueJon added documentation Improvements or additions to documentation webmcp WebMCP agent surface labels Nov 2, 2026
@TueJon TueJon self-assigned this Nov 2, 2026

@TueJon TueJon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — #26 @ afd3bd9 (round 1, first review)

Docs-only PR (+56/−0 across integrate.md, security.md, verify.md). No runtime, template, manifest, or workflow changes. CI check green at this head; ai-review is this task.

Product fit

In scope and warranted. Each addition guards a real failure mode of the skill's own workflow: duplicate tool ownership when a framework adapter already exposes a form action, adapter registration flags / fulfilled calls masking failures during VERIFY, and cross-origin tool selection confusing discovery with authorization. All four fit the skill's purpose (correct inventory → integrate → verify). No new product surface, no dependency, no default behavior change.

Correctness — claims checked against the repo

  • Angular Signal Forms ownership (integrate.md): https://angular.dev/ai/webmcp resolves (200). Guidance is "inventory before annotating, never add a second owner" — consistent with the skill's baseline-aware ground rule and the existing duplicate-name rollback behavior in heal.md/runtime.md.
  • Adapter observations (integrate.md): explicitly dated and versioned (2026-11-02) local observations, placed directly above the existing "not tested framework-version certifications" disclaimer — correctly hedged, no certification implied. Contrast with the vendored runtime is accurate: runtime.md confirms createToolScope disposal is AbortController registration teardown, and the runtime forwards execution options unchanged and converts bare null/undefined to structured errors, so the "adapter flag is not native evidence / preserve explicit structured results" advice matches shipped runtime semantics.
  • Native Playwright page.webmcp (verify.md): https://playwright.dev/docs/api/class-webmcp resolves (200), so the API exists as cited. The isError-without-throwing warning is consistent with the repo's own fail-closed result conventions, and the paragraph keeps the pinned standalone-harness default unchanged with an explicit do-not-upgrade-toolchain guard. No conflict with the existing Puppeteer paragraph.
  • Cross-origin gates (security.md): three-gate description (embedder delegation, provider exposedTo, consumer fromOrigins) matches the repo's own webmcp.d.ts (exposedTo on register, fromOrigins on getTools, RegisteredTool carrying origin + window), and the cited CG anchor section exists (spec page 200). "Discovery does not authorize execution" restates the skill's existing server-as-trust-boundary rule.

What I checked and found clean

  • No secrets, no code, no schema/manifest changes; package.json/skill.json/gemini-extension.json all remain 0.7.1 and CHANGELOG.md untouched — correct for a docs-only change under the repo's version-parity rule.
  • No frontmatter touched (the known argument-hint/tags validator rejection noted in the PR body is pre-existing and out of scope; base is equally affected, so not this PR's blocker per the scope-boundary test).
  • No dead code introduced; no docs invalidated (no env vars, runbooks, or examples changed); no release-steps/ trigger (docs-only, no rollout step).
  • No external effects: nothing sends, publishes, provisions, or mutates customer systems; no deploy blast radius (no workflow/runner changes).

Findings

None. No blocking, no caveats. Two untested areas (full router navigation, Angular hydration/declarative duplicate registration) are disclosed in the PR body itself, not hidden.

Verdict

ship_it — mergeable as-is.

@TueJon
TueJon marked this pull request as draft November 9, 2026 06:36
@TueJon TueJon changed the title docs: guard framework tool ownership and verification results fix: snapshot tool schemas and align verification guidance Nov 9, 2026
@TueJon TueJon added the bug Something isn't working label Nov 9, 2026
@TueJon
TueJon marked this pull request as ready for review November 9, 2026 08:50

@TueJon TueJon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — #26 @ 7e3a551 (round 2, follow-up)

Round 1 (afd3bd9) was docs-only and ship_it, with no open checklist. This round reviews only the afd3bd9..7e3a551 delta (code fix a4d8cd5 + docs alignment 7e3a551); no new checklist items were open, so only regressions introduced by these commits could block.

What the delta does

  • webmcp-workbench.js: registerTool now JSON-snapshots inputSchema/outputSchema at registration (throws TypeError before set/notify on non-serializable input, leaving the name reusable), and getTools returns fresh JSON.parse copies per enumeration. Output-aware status comparison, display, and refresh fingerprint.
  • webmcp.d.ts / webmcp.spec.ts: optional outputSchema on ModelContextTool / RegisteredTool; comments explicitly deny automatic output validation and note browser-dependent scalar serialization.
  • Tests: snapshot isolation, pre-registration rejection with name reuse, omission/toJSON handling, wrapper metadata passthrough, and a read-only Playwright refresh regression. tsconfig.templates.json covers the new compile-only fixture.
  • Docs: client.md drops the stale model/workspace matrix for generic eligibility checks; native-compatibility.md adds dated 2026-11-09 Chrome 156 scalar observations explicitly marked non-WPT; verify.md/reverify.md add output-drift comparison rules without permitting retries of application tools.

Checked and clean

  • Snapshot semantics correct: mutation-after-register and mutation-after-enumerate both verified by tests; rejection path performs no set and no notify (events assertion in test). function-typed schemas fall through to the not-serializable TypeError — safe.
  • optionalSchema preserves omission (undefined stays undefined, never coerced to empty schema); null/non-object inputs still reject at registration, consistent with the existing parseSchema display behavior for inputs.
  • No version/manifest changes, correct: prior runtime fixes also shipped without same-PR bumps (releases cut versions separately).
  • Exact-head CI: check SUCCESS at this head; ai-review is this task.
  • Links resolve (200): learn.chatgpt.com/docs/webmcp, webmcp PR #254. New doc claims are dated, hedged, and keep the standalone-harness default unchanged.
  • No secrets, no external effects, no deploy/workflow changes, no release-steps/ trigger. Scope-boundary test: residual annotations object sharing in the simulation predates this PR (base equally affected) — not introduced here.

Notes (non-blocking, no action required)

  • CHANGELOG.md [Unreleased] has no entry for the simulation behavior change (precedent 471dfca logged its fix in-PR). Release notes may consolidate this at release time.
  • Cited approvals d-20261102-0740 / d-20261109-0644 for the reference edits were not re-verified through the task API from this session; taken as stated in the PR body.

Verdict

ship_it — no blocking findings, no regressions from the fix commits.

@TueJon TueJon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — #26 @ 056207a (round 3, follow-up)

Round 2 (7e3a551) was ship_it with an empty checklist. Since then exactly one commit landed: 056207a "docs: clarify event and frame identity type comments" — skills/webmcpify/templates/webmcp.d.ts only, +10/−3, three comment blocks and no declaration, type or runtime change. No round-1/2 checklist item was open, so this round checks the delta for regressions and verifies the new factual claims; it does not reopen the earlier source audit.

What the delta claims, and what the spec says

  • ToolActivatedEvent — "Fired after callback invocation; cancellation here does not undo started work." Correct. CG PR #371 (Fire toolactivated event after invoking execute, merged 2026-11-09) moved the fire to the end of the invoke-execute-callback algorithm, i.e. after tool.execute is called and before its promise settles. Cancellation aborts the execution-time signal and rejects the caller, but the spec removes only the pending-execution entry — there is no rollback of work the callback already started, and the unregister note says an already-invoked callback is not cancelled.
  • getTools — "active documents in the same traversable, including permitted ancestors and siblings (CG clarification, 2026-11-13). Cross-origin policy, exposedTo and fromOrigins gates still apply." Correct. The algorithm enumerates toolRequestor's traversable navigable's active document's inclusive descendant navigables, so a nested caller also sees its ancestors and sibling frames. CG PR #382 (Describe the traversable scope of WebMCP getTools(), merged 2026-11-11) states exactly this ("an eligible parent or sibling document can also contribute tools"). The dated anchor is sound: the live Draft CG Report is versioned 13 November 2026 and contains the clarified traversal. The named gates match §6.3.4.1 (per-document tools Permissions Policy, provider exposedTo, consumer fromOrigins).
  • RegisteredTool.origin — browser-serialized origin, opaque origins may serialize as "null", equal strings are not an identity proof, keep the window identity. Correct and consistent with references/security.md (select on the complete origin + window identity). The draft defines origin as the serialized target document origin and executeTool rejects a parsed opaque origin with NotSupportedError; two distinct opaque origins both serialize to "null".

Checked and clean

  • Comment-only diff: no interface, type, runtime, template or fixture semantics change, so no compile or behavioral regression surface; tsconfig.templates.json / webmcp.spec.ts remain valid as reviewed in round 2.
  • Exact-head CI: check SUCCESS at 056207a (actions/runs/43817617919); ai-review is this task.
  • No contradictions with the PR's reference changes: client.md, security.md and discovery.md describe the same three cross-origin gates and never claim getTools is limited to the calling document.
  • Scope boundary: nothing here adds surface area, dependencies, workflows or release steps; no secrets, no external effects.

Notes (non-blocking, no action required)

  • The phrase "potentially trustworthy opaque origins" is easy to misread — an opaque origin is not itself potentially trustworthy; the practical case is an opaque origin arising in a secure context (e.g. a sandboxed frame) that still serializes as "null". The following sentences already carry the correct rule, so this is wording only; a future touch could say "opaque origins (e.g. sandboxed frames)".
  • Unchanged from round 2: CHANGELOG.md [Unreleased] still has no entry for the schema-snapshot behavior change; appropriate to consolidate at release time.

Verdict

ship_it — no blocking findings, no regressions; the three new type comments match the current CG draft and its dated clarifications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation webmcp WebMCP agent surface

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant