[earakely-scale]

What and why

Task designers can now change a website browser's state partway through a task with typed steps, without loading a whole state through load_artifact. For example, a step can sign a user in between agent turns, open a reference page in a background tab, or clear storage to simulate a logout. This is FD-4943 (M5.14). Its outcome and seven decisions are recorded there.

A new task_steps/browser/ package holds a BrowserStep base and one thin step per change. Each step calls one browser/v1 operation with the env's control token, as browser_connect_cdp already did.

Step	Fields	Calls
browser_set_cookies	cookies, in the shape cookies/set takes	cookies/set
browser_delete_cookies	name?, domain?, path? (decision 4)	cookies/delete
browser_set_local_storage	origin, items as a map (decision 3)	local_storage/set
browser_clear_local_storage	origin?	local_storage/clear
browser_open_tab	url, background? (decision 2)	tabs/open
browser_close_tab	url	tabs/list, then tabs/close
browser_activate_tab	url	tabs/list, then tabs/activate

Every step also takes env_id, and browser to name one when the env has several.

Finding the browser: among the deployed env's children, by the card that serves browser/v1. That covers chromium inside a computer and the browser alone (chromium-driver). Browser members of a multi env come with M8.14, and the context argument with M8.15.

Preflight (decision 1) refuses, before deploy, a step whose env has no browser, names a browser the env doesn't have, or doesn't name one when the env has several. It reads the stored env: a computer env's apps that declare browser: true. Malformed fields are config problems:

• a cookie that doesn't fit CookieToSet, reported without its value;
• an origin that isn't one;
• a URL that isn't absolute;
• background that isn't true or false.

At run time:

• Operation check (decision 6): each step first checks that the deployed browser's card lists its operation, and fails with "serves no browser/v1 tabs/open: rebuild it on a newer agentenv-framework-protocol" if not.
• Picking tabs (decision 5): tab steps pick the one open tab whose URL matches exactly, after normalizing both as the browser does: lowercase scheme and host, a host in another script as punycode (bücher.test as xn--bcher-kva.test), default port dropped, / for an empty path. No match, or several, fails the step and lists the open tabs' URLs.
• What's recorded: under context.metadata["browser_steps"][<step id>], browser_open_tab records the new tab's id, the tab steps record the tab they acted on, and browser_delete_cookies records how many cookies it deleted.

tabs/open {url?, background?} (decision 2, protocol): a background tab is created behind the active one, which stays active on screen and in tabs/list. On the headless browser alone, it's still held to the window's size before it loads. If Chrome hands the new tab focus anyway, agentenv brings the previously active tab back to the front. browser_open_tab sends background only when it's set: a browser built on an earlier protocol refuses an unknown parameter, so plain opens keep working on it, and only a background open there fails, naming background.

Preflight reads the env at the version the task pins (from Greptile's review). Task.preflight and a bundle's preflight give each step the env versions pinned by the deploy_env steps before it, the first for each env, through agent_env.task_step.preflight.pinned. The browser steps, load_artifact and snapshot_computer_env read their env at that version through stored_env, else at the latest; before, all three read the latest. TaskStep.deploys_env() says what a step deploys, and deploy_env returns its env_id and env_version.

Episodes: the seven new steps name no agent, so they're listed in task/episode/rules.py's UNAFFECTED, and a task with agents played from outside can run them.

browser_connect_cdp moves to task_steps/browser/connect_cdp.py as a BrowserStep subclass (decision 7). Its type name and behaviour are unchanged; it now shares the finder, and gains the preflight.

Not in this PR: read steps (graders read snapshot_env's sections); the context argument (M8.15); multi-env browser members (M8.14). apply_server_config is unchanged.

How it was tested

• ruff check: clean.
• Protocol unit tests: 1296 passed. test_website_browser.py adds background tabs. The tab opens behind the active one. If focus moves to it anyway, the previously active tab is brought back to the front, but a tab another call switched to meanwhile stays in front.
• Core unit tests:
  • test_browser_steps.py (42 tests, the HTTP boundary mocked):
    • each step's operation, body and control token;
    • tab steps matching the one tab at a URL as the browser shows it, and failing on none or several with the open tabs listed;
    • a card that lacks the operation refused before any call;
    • preflight on a desktop with chromium, the browser alone, two browsers, none, a website env and a missing env;
    • config problems, with no cookie value shown;
    • each step registered and round-tripping;
    • URL normalization.
  • test_preflight_pinned_versions.py: a task's preflight and a bundle's read a browser step's env at the latest version before any deploy, at the version the first deploy_env pins after it, and still at that one after a second deploy pins another. load_artifact and snapshot_computer_env read the pinned version too.
  • test_browser_connect_cdp.py passes unchanged except for its import path, and the entity-refs and episode-rules checks cover the new steps.
  • The areas touched (task_step, bundle, env, serialization): 5518 passed. The 2 test_disk_recipe failures fail on macOS only, and a gateway time-trigger test failed once under the parallel run and passes on its own.
• VM e2e, agent-environments-admin, amd64 and arm64: the chromium, browser computer, website browser, gateway CDP and computer app suites. amd64 passed all 32. arm64 passed 28 of 29, with the three echo-agent tests deselected, since their image needs qemu there. The one failure was GHCR dropping the connection while serving the base image's manifest, and it passed on rerun. The new browser_steps.check_browser_steps runs in a desktop and on the browser alone, each step built from a task's JSON through the registry and passing preflight on the real stored env:
  • browser_set_cookies and browser_set_local_storage sign in a tab that browser_open_tab opens in the background: it loads signed in, while the start tab stays active;
  • browser_activate_tab brings it to the front by its URL, and browser_close_tab closes it;
  • a close naming a URL no tab shows fails, listing the tab still open;
  • browser_delete_cookies records 1 deleted, and with browser_clear_local_storage the site has no cookies or storage left;
  • a tab opened at http://faß.localhost:8783/other shows as xn--fa-hia.localhost, as UTS 46 without transitional mapping spells it, and browser_close_tab closes it by the URL as written.
• A whole task through task.run (browser alone, amd64 on a VM; CI runs it on every push): the browser env is stored at a new id as version 1, and as version 2 with no browser declared. A task's deploy_env pins version 1, so its preflight passes, while the same task unpinned reports all five browser steps. Its cookie and tab steps then run on the version deployed, the open, activate and close steps record one tab, and the cookie delete records 1.
• CDP suites: browser_connect_cdp, now on the shared base, passes its e2e on amd64: it hands an echo agent its address with the env's tools hidden, which a reset ends until the step reruns. The tests of an agent keeping its address through restarts and of two CDP agents at once also pass, along with the Playwright, Puppeteer and gateway CDP tests. The echo-agent tests are deselected on arm64. The Browser Use CLI test isn't part of this suite.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; the previous Unicode tab-matching issue is fixed.

Summary

Adds seven browser task steps for cookies, local storage, and tabs, with shared browser lookup and preflight checks.

• Task steps can change browser state and manage tabs during a run.
• A browser can open a tab behind the one already in front.
• Task preflight checks an environment at the version its earlier deploy step pins.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Browser task step] --> B[Find deployed browser]
  B --> C[Check browser supports operation]
  C --> D[Call browser/v1 with control token]
  D --> E[Change cookies, storage, or tabs]
  E --> F[Record tab ID or deleted-cookie count when applicable]

Loading

_{Reviews (10) · Last reviewed commit: "fix(browser): tab URLs keep a host IDNA ..." · Reviewed by Greptile}