Repository navigation
fix(settle): compare post-action trees in one view across every route #4500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
a8995d3
7181fe9
3d1ae0e
a79152d
9a623dc
6e5ada8
5443831
3b6a9f7
937ae62
2e941c2
0149a53
776276e
1f15eb0
bba6172
5b2c4cd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,9 +5,16 @@ | |
| SettleObservation, | ||
| SettleParams, | ||
| } from '@agent-device/contracts/interaction'; | ||
| import { captureInteractionSnapshot } from './interaction-snapshot-capture.ts'; | ||
| import { summarizePostActionEvidence } from './post-action-surface.ts'; | ||
| import { surfaceScopedNodes } from '@agent-device/capture-kit/snapshot-baseline'; | ||
| import { isSameSnapshotPresentation } from '@agent-device/kernel/snapshot'; | ||
| import { | ||
| resolvePostActionSurfaceChange, | ||
| summarizePostActionEvidence, | ||
| } from './post-action-surface.ts'; | ||
| import { | ||
| postActionPresentation, | ||
| surfaceScopedNodes, | ||
| } from '@agent-device/capture-kit/snapshot-baseline'; | ||
| import { captureSelectorSnapshot } from './selector-read-shared.ts'; | ||
| import { settleAfterInteraction, settleEvidence } from './settle.ts'; | ||
|
|
||
| type ObservedResult<T extends object> = T & { | ||
|
|
@@ -81,9 +88,17 @@ | |
| ...params.settle, | ||
| resolved, | ||
| }); | ||
| const evidence = params.verify | ||
| ? settleEvidence(outcome.settledCapture, resolved.preAction) | ||
| : undefined; | ||
| // A surface transition is evidence in every view; node digests compare only within one view. | ||
| const baseline = | ||
| outcome.settledCapture && | ||
| (resolvePostActionSurfaceChange(resolved.preAction, outcome.settledCapture) || | ||
| isSameSnapshotPresentation( | ||
| resolved.preAction?.presentation, | ||
| outcome.settledCapture.presentation, | ||
| )) | ||
| ? resolved.preAction | ||
| : undefined; | ||
| const evidence = params.verify ? settleEvidence(outcome.settledCapture, baseline) : undefined; | ||
|
Check warning on line 101 in packages/command-runtime/src/commands/interaction/runtime/post-action-observation.ts
|
||
|
Comment on lines
+91
to
+101
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🆕 🟡 Minor · --settle --verify loses changedFromBefore when preAction and frame views differ
Suggested change Feed settleEvidence the baseline the settle loop actually diffed against (the postActionBaseline result on outcome) rather than gating resolved.preAction on its view; keep the surface-change escape hatch. Detailspersona: Useful? Reply Reply
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Partially valid at merged head 5b2c4cd: when selector preAction is interactive but settle substitutes a full ref-frame baseline, changedFromBefore is omitted. The claim that all evidence disappears is inaccurate: settleEvidence receives an undefined baseline and summarizePostActionEvidence still returns digest and node counts. Known surface transitions retain their verdict. Local revision ed9ffd13a4277e507e73770621e82c23f3662e8f addresses the underlying substitution by keeping fresh selector pre-action captures and the latest stored observation for scroll/back, then reproducing that baseline presentation. It was not pushed before merge. The user then requested cleanup, which is complete. No worktree or follow-up PR has been recreated. Leaving this finding open for follow-up instead of marking an unmerged fix as resolved. |
||
| return { settle: outcome.observation, ...(evidence ? { evidence } : {}) }; | ||
| } | ||
| if (!params.verify) return {}; | ||
|
|
@@ -92,8 +107,8 @@ | |
| } | ||
|
|
||
| /** | ||
| * Post-action side of `--verify` (#1047): one interactive-only capture through | ||
| * the same capture helper the resolution path already uses, digested and then | ||
| * Post-action side of `--verify` (#1047): one capture in the pre-action tree's presentation | ||
| * (interactive-only by default), so the comparison reads one view (#4486), digested and then | ||
| * discarded. The node tree itself is never attached to the result, only the | ||
| * cheap summary. A missing baseline yields no `changedFromBefore` claim — | ||
| * no baseline, no comparison. | ||
|
|
@@ -104,7 +119,11 @@ | |
| resolved: ResolvedInteractionTarget, | ||
| ): Promise<InteractionEvidence | undefined> { | ||
| try { | ||
| const capture = await captureInteractionSnapshot(runtime, options, true); | ||
| const capture = await captureSelectorSnapshot(runtime, options, { | ||
| updateSession: true, | ||
| includeRects: true, | ||
| presentation: postActionPresentation(resolved.preAction), | ||
| }); | ||
| return summarizePostActionEvidence(surfaceScopedNodes(capture.snapshot), resolved.preAction); | ||
| } catch { | ||
| return undefined; | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -117,12 +117,13 @@ test('press --settle attaches no diff across a sheet-to-app surface change and d | |
| assert.match(settle.hint ?? '', /different surfaces/); | ||
| }); | ||
|
|
||
| test('press --settle --verify reports one app-to-sheet surface change on both payloads', async () => { | ||
| const before = buttonSnapshot(); | ||
| const sheet = webSignInSheetSnapshot(WEB_SIGN_IN_SHEET_LABELS); | ||
| test('press --settle --verify reports a surface change even when the selector and frame views differ', async () => { | ||
| const before = { ...buttonSnapshot(), presentation: { interactiveOnly: true } }; | ||
| const frame = { ...before, presentation: {} }; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: This fixture does not actually exercise the claimed selector-versus-frame view mismatch: the first selector capture overwrites Prompt for AI agents |
||
| const sheet = { ...webSignInSheetSnapshot(WEB_SIGN_IN_SHEET_LABELS), presentation: {} }; | ||
| let captures = 0; | ||
| const device = createSettleDevice({ | ||
| stored: before, | ||
| stored: frame, | ||
| captureSnapshot: () => { | ||
| captures += 1; | ||
| return { snapshot: captures === 1 ? before : sheet }; | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.