test(browser): stage controlled Agent Task without workflow mutation - #288
test(browser): stage controlled Agent Task without workflow mutation#288seonghobae wants to merge 43 commits into
Conversation
Preserve the reviewed pinned-Chrome Agent Task runner, sandbox contract, post-condition evidence, cleanup checks, and scoped documentation on current protected main without carrying the leaf .github workflow mutation. Workflow activation remains owned by #212. Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head review on e051a3d06a613233781272ffdc0e564023ba52b0: this successor is directly based on protected main@87c4daa1830bac5a5228b6036752ad5633232085, 1 ahead / 0 behind with merge base exactly main, and contains the nine reviewed non-workflow #70 blobs only. No .github/** delta is present. CI 33931732869 and MV3 33931733072 completed skipped because this PR is intentionally Draft under #286; that is expected admission behavior, not GREEN. Security 33931732996, Semgrep 33931732911, and CodeQL 33931732899 remain queued. Product acceptance is still a hosted sandboxed pinned-Chromium execution proving semantic observation → native action → page-observed post-condition → cleanup; ACK-only success and --no-sandbox remain rejected. #212 owns the missing workflow activation/setup.
|
Exact-head local review on |
Remove the workflow-owned sandbox-helper assertion from the staged runner slice and document that hosted sandbox activation remains separately governed. Commit-Message-Assisted-by: Claude (via Claude Code) Signed-off-by: Seongho Bae <me@seonghobae.me>
|
정확한 head |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head security finding: the new tests/test_mv3_browser_sandbox_contract.py is intentionally RED on this generation. _run_agent_task_browser_pass is sandbox-preserving, but ordinary _run_browser_pass still contains "--no-sandbox". This reintroduces a security/evidence behavior already causally repaired in #43 by a45c83e4d8988fe89920ecb6a9eac469815f5b9b, whose minimal production change was removal of that one launch override. Do not import #43's broader runner/download delta and do not mutate .github/** from this scheduled product lane. Repair only the ordinary launch argument, preserve the new two-path regression, then require complete current-head Python GREEN plus #212-owned sandbox-helper workflow setup and fresh pinned-Chromium execution. Draft/skipped hosted jobs are not GREEN.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Current-head review: doctoring now records the Chromium primary-source security boundary and exact owner path, but this generation is intentionally still RED. tests/test_mv3_browser_sandbox_contract.py requires both real-browser launch paths to omit --no-sandbox; ordinary _run_browser_pass has not yet adopted the reviewed #43 one-line causal repair. Do not call this head GREEN. The safe next product mutation is the narrow runner-line removal only; #212 remains responsible for .github/** sandbox-helper setup and hosted sandboxed Chrome evidence.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head repair verification: the test-first sandbox contract is now source-level GREEN. Current ordinary MV3 Chrome options and Agent Task Chrome options both omit --no-sandbox. The two repair commits after doctoring have one net semantic line of delta versus 99fea898...: deletion of the ordinary launch override; the intermediate accidental diagnostic wording change was immediately restored by 0f434bc.... This adopts #43's causal invariant without importing its unrelated downloads/diagnostic tree. Keep Draft: protected #286 skips native CI/MV3 for Drafts, and #212 still owns the hosted chrome_sandbox setup needed for real pinned-Chromium GREEN. Skipped/absent hosted evidence is not acceptance.
seonghobae
added
maintenance
priority: medium
type: maintenance
labels
ChatGPT Codex Connector
|
Fresh RCA for the exact-head CodeQL failures: jobs Canonical owner is |
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head security RED on a71af97982b5e25cefc9f828501e610be636a875: the controlled Agent Task lane already treats page content as untrusted, but the ordinary MV3 evidence path still serializes page-derived values into CI exceptions. _exercise_real_click raises real click post-condition failed: {text!r}, and _wait_for_extension_evidence raises with observed={latest!r}. Those values originate from WebDriver/page observations and therefore may decide pass/fail but must not become diagnostic payload. Commit a71af979... adds behavioral regressions that inject a hostile marker and require stable closed diagnostics (real click post-condition failed, MV3 fixture did not converge) with no marker retention. This is intentionally test-first; current production is expected to fail both contracts. Minimal causal fix is to keep the observed values only in the local comparison state, emit no raw observed/page text on mismatch/timeout, and preserve the existing real-browser post-condition, 3-trial denominators, sandbox-enabled launch and cleanup semantics. This finding is separate from #148's ChromeDriver process/protocol diagnostic lane and does not authorize copying its active source or touching .github/**.
|
Writer lease RELEASED — exact |
|
PR-state lease ACTIVE — exact |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Current-head review on 2e1ca45266bfc64e17e6d40e0a4b5bb2aa587a2e: the causal post-condition ambiguity identified in review 5147216471 is minimally repaired in source commit 8b7aa28ecf7acb1e3f3b2dcadd4cb3cbf59ea01f. The runner now observes the controlled #task-result baseline (data-state=idle, rendered text idle) before clear/type/click, rejects any unexpected/pre-fired state with one closed diagnostic that does not echo page data, preserves the native action and existing URL/submitted/exact-echo checks, and propagates pre_action_baseline_verified: true into per-trial evidence. Compare 4d61c2f...8b7aa28 changes only scripts/ci/run_mv3_compatibility.py by 25 added lines; no workflow, trial denominator, sandbox flag, protocol-diagnostic owner, or post-condition criterion changed. docs/traceability/agent-task-action-transition-evidence.md pins the repair and the W3C WebDriver 2 Working Draft dated 2026-07-02. This is not approval or browser GREEN: current CI/MV3 are Draft-policy skipped and current security/CodeQL lanes are still pending at this read. Fresh sandbox-enabled pinned-Chromium 3-trial execution remains required through #212.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Current-head follow-up on ec131617a2d90657911c6b072982b079c4a79fc2: the original causal-evidence finding is now closed at the acceptance predicate, not merely at evidence emission. Test-first 42a9a129... requires _agent_task_surfaces_complete to reject a successful-looking post-condition record that lacks the pre-action witness; ed6af6eb... updates the existing successful-trial doubles with pre_action_baseline_verified: true; production commit e1dd5099... adds exactly one acceptance line requiring that witness in every successful trial. Compare ed6af6eb...e1dd5099 changes only scripts/ci/run_mv3_compatibility.py by +1 line. The browser pass still proves the controlled idle/idle baseline before native clear/type/click and preserves URL, submitted-state, exact-echo, cleanup, trial denominator, sandbox, workflow, and #148 diagnostic boundaries. Traceability ec131617... records the stronger gate and W3C WebDriver 2 2026-07-02 provenance. No approval/browser GREEN is claimed: CI/MV3 are Draft-policy skipped and current security lanes are pending at this read.
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head test-first RED on 07540f0cdb3178998d305382d9176cccaeabce57: the current runner observes the idle result before clear/type/click, but never re-observes that result after typing and immediately before the submit click. A fixture regression can therefore transition #task-result during input and leave the later click with an already-satisfied submitted/echo post-condition. tests/test_agent_task_action_transition_evidence_contract.py now requires a second closed baseline after "/value" and before "/click", plus a mandatory pre_click_baseline_verified witness in _agent_task_surfaces_complete. Minimal repair is to reuse the existing non-echoing idle validator on a fresh browser observation immediately before click, thread that boolean through trial evidence, and keep the existing semantic target, native click, submitted/exact-echo post-condition, URL stability, cleanup, 3-trial denominator, Draft state, and #212/#148 authority boundaries unchanged. This is source-semantic/test-first RED unless hosted checks execute it; Draft-policy skips are not RED execution evidence.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head follow-up on 81b7869dbfaa9927a7cea0c02698c8544d0ea056: the immediate pre-click causal-evidence RED from 07540f0cdb3178998d305382d9176cccaeabce57 is minimally repaired. Production 942e4c1a44119384d01ee4c7ec4168e6c5ab38b5 re-observes the existing #task-result after WebDriver typing and submit-target semantic verification, immediately before click, reuses the closed/non-echoing idle validator, carries pre_click_baseline_verified, and makes it mandatory in _agent_task_surfaces_complete. 3d3166ec... and efca7d69... align successful evidence doubles instead of weakening the new predicate. Traceability f82eb097... and TEST_STRATEGY 81b7869d... make the stronger sequence explicit: idle before input → idle immediately before click → native click → observed submitted/exact echo. No browser version, trial denominator, URL/post-condition, cleanup, extension-isolation semantics, sandbox/workflow, or #148 protocol-diagnostic authority changed. This is not approval or executable browser GREEN: exact CI/MV3 are Draft-policy skipped, and sandbox-enabled pinned-Chromium 3-trial evidence remains #212-dependent.
|
Writer lease RELEASED — exact |
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head test-first RED on 4306f38c45f65a7fba7ccd842ae42b2ed4646e29: the controlled Agent Task runner samples post_submit_url immediately after the native click, then accepts submitted plus the exact synthetic echo without another URL observation before returning success evidence. tests/test_agent_task_action_transition_evidence_contract.py now requires a browser URL read after the accepted post-condition/echo and before the success return, with a closed diagnostic if the outcome boundary has navigated. On the current production source, the new source-order contract fails because there is no "/url" read after if text != AGENT_TASK_INPUT_VALUE; the only post-click URL sample precedes _validate_agent_task_submitted_state. This is a source-semantic/test-first RED; Draft-policy hosted skips are not claimed as executed RED. Minimal repair is to re-read the browser URL after submitted-state/exact-echo verification, require equality with the initial controlled fixture URL, and only then emit url_unchanged success. Preserve the earlier immediate URL check, native action, both idle baselines, semantic target evidence, credential-safe diagnostics, cleanup, three-trial denominator, #148 protocol diagnostics and #212 workflow/sandbox authority.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head follow-up on 4af7b2211766d74579f7365f6973030d00eefe5f: the accepted-outcome URL timing RED from 4306f38c45f65a7fba7ccd842ae42b2ed4646e29 is minimally repaired without weakening the earlier action evidence. Production cac13ab79cd41ca55315622c3148126bee5643f8 retains the immediate post-click Get Current URL check, then verifies submitted plus the exact synthetic echo, performs a second Get Current URL observation, and only emits url_unchanged: true when both observations equal the original controlled fixture URL. The final mismatch is the closed diagnostic Agent Task URL changed before accepted outcome, so page-controlled URLs are not serialized. 3da94582... updates traceability, f46767e1... repairs an accidental CHANGELOG wording duplication from the documentation commit while preserving only the intended net bullet, and current 4af7b221... makes TEST_STRATEGY require URL stability through the accepted outcome. Compare from prior checkpoint 81b7869d... is 6 ordinary commits ahead / 0 behind and the net delta is limited to the action-transition test, runner, traceability, TEST_STRATEGY and one CHANGELOG bullet. This is not approval or executable browser GREEN: Draft admission may skip repository/browser gates, and #212 still owns current-generation sandbox activation before pinned-Chromium three-trial acceptance.
|
Writer lease RELEASED — exact |
1 participant
Why this split exists
PR #70 contains valid controlled Agent Task browser work and a leaf
.github/workflows/mv3-compatibility.ymlmutation. Protected-mainAGENTS.mdreserves workflow changes for the authorized workflow-owner path, so #288 retains browser/test/documentation work while #212 owns workflow/sandbox activation. Raw WebDriver/ChromeDriver protocol diagnostics remain #148 authority.Exact lineage
main@87c4daa1830bac5a5228b6036752ad5633232085;88abfe308def92ef88fde54f4200b56149e36854;46f90e941c03dd12b6f0a25a3503d404426f013a;e1dd50999cd3a52977179047e8d5d77a2e85aef9;942e4c1a44119384d01ee4c7ec4168e6c5ab38b5plus evidence-double/doc alignment through81b7869dbfaa9927a7cea0c02698c8544d0ea056;4306f38c45f65a7fba7ccd842ae42b2ed4646e29;cac13ab79cd41ca55315622c3148126bee5643f8;3da94582127900c18a98320768a861dd473047f4;f46767e1642dca1bb27b9bd15a2869131fa7d9a8;4af7b2211766d74579f7365f6973030d00eefe5f.Compare
81b7869d...4af7b221...is 6 ordinary commits ahead / 0 behind. Net files are limited totests/test_agent_task_action_transition_evidence_contract.py,scripts/ci/run_mv3_compatibility.py,docs/traceability/agent-task-action-transition-evidence.md,docs/TEST_STRATEGY.md, and one intendedCHANGELOG.mdbullet. No.github/**, workflow/ruleset/secrets, browser version, trial denominator, sandbox configuration, provider/model authority, base, release state, or #148 protocol-diagnostic authority changed.Buyer-visible browser evidence boundary
The controlled Agent Task fixture uses pinned Chrome for Testing / ChromeDriver, browser-computed textbox/button role and accessible name, native WebDriver clear/type/click, browser-observed page state, exact synthetic input echo, URL stability, session cleanup, and profile deletion. WebDriver command acknowledgement is not task success. Page-derived values remain untrusted observation and are not policy or instruction authority.
--disable-extensionsremains launch intent only (extensions_disabled_requested); it is not counted as verified extension isolation without independent browser observation.Causal action and URL-stability repair
The accepted action sequence already required two closed
idle/idleobservations of#task-result: once before clear/type and again after typing plus submit-target semantic verification, immediately before native click. Bothpre_action_baseline_verifiedandpre_click_baseline_verifiedremain mandatory in_agent_task_surfaces_complete.Fresh review found a separate timing gap in URL evidence. The runner sampled
post_submit_urlimmediately after click, then verifieddata-state=submittedand the exact synthetic echo, and returned success without another URL observation. A click handler could synchronously satisfy the accepted post-condition, schedule navigation, pass the earlier URL check, and still be accepted without proving same-document stability through the outcome actually accepted by OriginWeave.Test-first
4306f38c...requires another Get Current URL observation after submitted-state/exact-echo verification and before success evidence. Productioncac13ab7...retains the immediate post-click URL check, performs the second URL observation at the accepted-outcome boundary, and keepsurl_unchangedtrue only when both samples equal the initial controlled fixture URL. The final mismatch is the closed diagnosticAgent Task URL changed before accepted outcome; the observed URL is not serialized into CI diagnostics.The accepted sequence is therefore idle before input → native clear/type → idle immediately before click → native click → immediate URL check → page-observed submitted + exact synthetic echo → accepted-outcome URL check → cleanup.
Review
5147893561records the test-first RED on4306f38c...; exact-head COMMENT review5147973150records the minimal production/doc repair on4af7b221.... Neither is approval or executable browser GREEN.docs/traceability/agent-task-action-transition-evidence.md,docs/TEST_STRATEGY.md, andCHANGELOG.mdare code-current for this stronger boundary. Standards traceability remains W3C WebDriver Working Draft, 2 July 2026; the stronger causal/outcome requirement is an OriginWeave evidence invariant layered above WebDriver command completion.Exact-head verification state
For exact
4af7b2211766d74579f7365f6973030d00eefe5fat the latest read:34289499146: terminal skipped because this PR remains Draft;34289499040: terminal skipped for the same Draft admission policy;34289499049: terminal success;34289499110: terminal success;34289499053: in progress. Language detection job102272640880is success. JavaScript/TypeScript102273041824and Python102273041860both read the current-head dispatch verdict successfully and fail only atRelease runner or enforce current-head CodeQL verdict; Actions102273041924is still in the same verdict-read path at the latest read. This is the existing central dispatch/authenticated-terminal-verdict materialization failure class, not a discovered test(browser): stage controlled Agent Task without workflow mutation #288 source finding. Exact specimen is handed to canonical owner.github#712in comment5593207312.The new regression is test-first/source-semantic evidence, not claimed hosted RED. The exact source now contains the required second post-condition URL observation, but Draft-policy skips and code inspection are not executable repository/browser GREEN. Full Python repository contracts, rustfmt/workspace tests, strict Clippy, rustdoc, exact owned-production 100% coverage, terminal current-head governance verdicts, and sandbox-enabled pinned-Chromium execution remain required.
Remaining owner path
#212 owns the current-generation least-privilege
chrome_sandboxworkflow activation. Exact consumer handoff is comment5593193716. After authorized workflow repair, replay this exact successor lineage on Chrome/ChromeDriver150.0.7871.129for three independent trials and require both idle baselines, native action, submitted/exact-echo post-condition, both post-click URL observations, session cleanup, and profile deletion..github#712separately owns the CodeQL central dispatch/verdict materialization failure. Do not mutate leaf source or create no-op commits to manufacture a new check.Command ACK, launch-argument intent, mock output, cleanup alone, skipped jobs, or another head's browser result are insufficient. Keep Draft until exact consumer evidence and the active ruleset are satisfied. Do not restore
--no-sandbox, disable Ubuntu security restrictions runner-wide, reduce trials, copy #148 source, force-push, destructively rebase, self-approve, bypass, weaken workflows/rulesets/secrets, mark Ready, merge, tag, release, or close predecessor work before every valid delta is fully adopted by an authorized owner.