Skip to content

fix(web)!: scope headers and basicAuth to the app's origin - #1393

Merged
okwasniewski merged 4 commits into
mainfrom
krzysztof/fix/web-basicauth-origin-scope
Nov 12, 2026
Merged

okwasniewski merged 4 commits into
mainfrom
krzysztof/fix/web-basicauth-origin-scope

Conversation

@KrzysztofMoch

@KrzysztofMoch KrzysztofMoch commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

web({ basicAuth }) answered a 401 from any origin, and web({ headers }) went to every host on the app's site: another port on the same host, sibling subdomains, every host under a shared domain such as vercel.app. A third-party origin the app embeds could collect a preview password or a x-vercel-protection-bypass token. Fixes ENG-846.

What changed

  • Both now go to the app's origin only: the scheme, host, and port of app.url.
  • One new option, credentialOrigins, lists extra origins for both. Each entry must be an exact http:/https: origin, else INVALID_CONFIG at config load. That includes a * in the host and port 0, which would match nothing.
  • basicAuth becomes Playwright's per-origin httpCredentials array, one entry per allowed origin (packages/web/src/protected-app.ts).
  • WebKit is the exception. It never matches a scoped credential to an origin on a default port (443/80) or with an IPv6 host. When an allowed origin is one of those, WebKit gets one unscoped credential, as on main, and the docs say so.
  • The engine only knew the app's site, so the runner now passes EngineAppInfo.origin to prepare and init (packages/e2e/src/config/app.ts). It carries the run's real port when app.url uses port 0, and is left out until that port is assigned (e2e doctor).
  • @e2e-dev/web raises its e2e peer floor to >=0.23.1. On an older runner, init fails with INVALID_CONFIG instead of sending credentials nowhere. If chore: version packages #1590 releases 0.23.1 first, the floor moves to 0.23.2 when I rebase.
  • browser.route's route.continue() and route.fetch() add the configured headers under the same origin rule.
  • Breaking for @e2e-dev/web (minor, **Breaking:**). The changeset carries the migration note.
engine: web({
  headers: { 'x-vercel-protection-bypass': process.env.VERCEL_BYPASS! },
  basicAuth: { username: 'preview', password: secrets.get('previewPassword') },
  credentialOrigins: ['https://api.example.com'],
}),

Not changed: a header Playwright injects into a request stays on that request's redirects, to any origin. Playwright reapplies a routed request's headers to every redirect hop without routing the hop (crNetworkManager, _alreadyContinuedParams). This was already true on main. The docs now say so, and a test pins it. Closing it means fetching outside the browser with route.fetch({ maxRedirects: 0 }), which costs the HTTP cache and streaming, so it's out of scope here.

Verified

Ran it locally: yes

  • I built the CLI and ran a scratch project against a gate server. The app runs on one port and the same gate listens on a second port of the same host. Each page reports the header it received and whether basic auth passed.
Case main this branch branch, other port in credentialOrigins
App origin: header / basic auth on / Protected on / Protected on / Protected
Other port: header on (the bug) none on
Other port: basic auth Protected (the bug) Unauthorized Protected
  • The CLI refuses credentialOrigins: ['https://*.vercel.app'] and ['http://127.0.0.1:0'] at load with INVALID_CONFIG, naming the entry.
  • WebKit on macOS with Playwright 1.64.1, headless, against a basic-auth server. Chromium answers in every case.
WebKit scoped credential unscoped (this branch's fallback)
port 80 Unauthorized Protected
a free port Protected -
[::1] Unauthorized Protected
  • New integration tests cover the bug cases. protected-app.test.ts sends a request to another port on the same host and to a third-party site, for both headers and basicAuth, with and without the extra origin, plus redirects in both directions. routes.test.ts covers continue({ url }) and route.fetch() to another port on the same host and to a listed one. Both new route tests fail against main's engine source.
  • Earlier, the same tests ran headed on macOS (Chromium). Headed Chromium fails a navigation to an unanswered 401 with ERR_INVALID_AUTH_CREDENTIALS rather than rendering the page. Main does the same for any unanswered 401, but now more origins go unanswered.
  • pnpm check and pnpm test pass.

Opus 5.5 in Claude Code (T3 Code). Reviewed by Fable 5.1.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 88.3k tokens (94% cached) · $0.0034

All 33 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.4s
🟢 control inventory › check sets the checkbox and the radio 1 step 686ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 251ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 868ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.5s
🟢 control inventory › upload attaches the named project files 1 step 2.4s
🟢 control inventory › back returns from the details view to the inventory 1 step 298ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 374ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 383ms
🟢 act signs in with the benchmark account 1 step 383ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 10.9s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 5.6s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.6s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 3.7s
🟢 tests-agent/scenarios.e2e.ts · 21 passed 21 steps · 12 calls 1m 9s
🟢 act completes shadow-dom-form 1 step · 3 calls 3.5s
🟢 act completes lying-labels 1 step · 3 calls 4.3s
🟢 act completes iframe-form 1 step · 3 calls 4.2s
🟢 act completes infinite-scroll 1 step 3.5s
🟢 act completes virtualized-table 1 step 20.3s
🟢 act completes overlay-trap 1 step 896ms
🟢 act completes sticky-chrome 1 step 924ms
🟢 act completes async-states 1 step · 3 calls 6.6s
🟢 act completes debounced-search 1 step 2.4s
🟢 act completes hover-menu 1 step 522ms
🟢 act completes drag-and-drop 1 step 502ms
🟢 act completes kanban-board 1 step 680ms
🟢 act completes context-menu-trap 1 step 2.7s
🟢 act completes stale-dom 1 step 501ms
🟢 act completes otp-auto-advance 1 step 691ms
🟢 act completes date-picker 1 step 768ms
🟢 act completes rich-text-editor 1 step 10.3s
🟢 act completes playbook-cleanup 1 step 646ms
🟢 act completes promo-storefront 1 step 2.3s
🟢 act completes gift-card-purchase 1 step 2.5s
🟢 act filters the list and the URL carries the filter 1 step 248ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 368ms
🟢 act adds an address with a run-unique label 1 step 368ms

Tested by e2e 0.23.0 · 1m 28s · web · run artifacts

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 91 passed

All 91 tests in 41 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 1.1s
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 761ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 326ms
🟢 tests/async-states.e2e.ts · 2 passed 7.3s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.4s
🟢 async states › a wrong code is rejected after the toast has gone 4.9s
🟢 tests/canvas-only.e2e.ts · 2 passed 537ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 312ms
🟢 canvas only › a shape out of order paints the error and resets progress 225ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 612ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 370ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 242ms
🟢 tests/cart-totals.e2e.ts · 2 passed 656ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 371ms
🟢 cart totals › a quantity never drops below one 285ms
🟢 tests/checkout-review.e2e.ts · 1 passed 302ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 302ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 798ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 418ms
🟢 context menu trap › the decoy menu items refuse and keep the file 380ms
🟢 tests/control-inventory.e2e.ts · 8 passed 4.3s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 427ms
🟢 control inventory › check picks a radio the pick replaces with its summary 240ms
🟢 control inventory › a long press and a double click are told apart from a tap 1.1s
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 327ms
🟢 control inventory › setInputFiles attaches files named from the project root 194ms
🟢 control inventory › the details view is left through the browser history 246ms
🟢 control inventory › scrollIntoView brings the footnote into view 343ms
🟢 control inventory › every exercise done shows the success message 1.5s
🟢 tests/css-content-ui.e2e.ts · 2 passed 444ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 239ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 205ms
🟢 tests/date-picker.e2e.ts · 2 passed 814ms
🟢 date picker › navigates three months ahead and books the required date 459ms
🟢 date picker › confirming without the exact date shows the error 355ms
🟢 tests/debounced-search.e2e.ts · 2 passed 3.1s
🟢 debounced search › a decoy one character off is refused 1.6s
🟢 debounced search › picking the exact item adds it to the cart 1.5s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.2s
🟢 one submit opens one confirm, even when double-clicked 1.2s
🟢 tests/div-soup.e2e.ts · 2 passed 846ms
🟢 div soup › a wrong code is rejected 364ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 482ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 816ms
🟢 drag and drop › an out-of-order item resets the dropzone 413ms
🟢 drag and drop › dropping Banana then Cherry passes 403ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 337ms
🟢 filter deep link › filtering writes the query to the URL 183ms
🟢 filter deep link › opening a shared link restores the filter 154ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 789ms
🟢 gift card purchase › refuses to buy without an amount 226ms
🟢 gift card purchase › refuses a recipient without an at sign 270ms
🟢 gift card purchase › sends the gift card 293ms
🟢 tests/hover-menu.e2e.ts · 2 passed 622ms
🟢 hover menu › a decoy leaf is refused 280ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 342ms
🟢 tests/iframe-form.e2e.ts · 2 passed 991ms
🟢 iframe form › a wrong code is refused by the host page 521ms
🟢 iframe form › the code read from one frame applies through the other 470ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 722ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 171ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 272ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 279ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 3.0s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 3.0s
🟢 tests/kanban-board.e2e.ts · 3 passed 856ms
🟢 kanban board › pointer drags move both cards into the goal layout 415ms
🟢 kanban board › a swipe along a path drags a card as well 192ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 249ms
🟢 tests/login-form.e2e.ts · 4 passed 1.0s
🟢 login form › rejects a malformed email 236ms
🟢 login form › rejects a short password 279ms
🟢 login form › rejects an unknown account 223ms
🟢 login form › signs in and logs out 288ms
🟢 tests/lying-labels.e2e.ts · 4 passed 827ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 191ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 131ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 239ms
🟢 lying labels › visible text drives the payment through 266ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 657ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 239ms
🟢 native dialogs › dismissing the confirm cancels the order 196ms
🟢 native dialogs › a wrong gift code is rejected 222ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.3s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.3s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 6.8s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 448ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 578ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.8s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 2.0s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 2.0s
🟢 tests/order-history.e2e.ts · 2 passed 1.7s
🟢 order history › overview and settings render their panels 288ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.4s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.7s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 560ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 242ms
🟢 otp auto-advance › a wrong code clears every box and starts over 898ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 742ms
🟢 continues only after the banner and the overlay are gone 742ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 739ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 375ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 364ms
🟢 tests/price-sorting.e2e.ts · 3 passed 653ms
🟢 price sorting › opens in the featured order with sale prices struck through 184ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 225ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 244ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.3s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.3s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.5s
🟢 buys a deal once the promo chrome is out of the way 1.5s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 742ms
🟢 rich text editor › bolding exactly the required word passes the check 378ms
🟢 rich text editor › bold that spills into the previous word fails the check 364ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 910ms
🟢 saved addresses › adds an address and clears the form 310ms
🟢 saved addresses › ignores an add with an empty field 280ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 320ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 547ms
🟢 shadow DOM form › rejects a wrong access code 275ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 272ms
🟢 tests/smoke.e2e.ts · 1 passed 406ms
🟢 home lists the scenarios and opens one 406ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.2s
🟢 taps the moving target three times across rebuilds of the list 2.2s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 254ms
🟢 accepts the terms once the button clears the sticky footer 254ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 19.7s
🟢 scrolls the windowed table to the Golden Row and claims it 19.7s
🟢 tests/wishlist.e2e.ts · 2 passed 1.9s
🟢 wishlist › saving toggles the button and the tab badge 408ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.5s

Tested by e2e 0.23.0 · 1m 18s · web · run artifacts

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 12 replayed from cache · 56 model calls · 343.6k tokens (82% cached) · $0.0236

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 56 calls 8m 18s
🟢 act completes Login Form 1 step 34.5s
🟢 act completes Infinite Scroll List 1 step 35.8s
🟢 act completes Modal Flow 1 step 9.8s
🟢 act completes Bottom Tabs 1 step 7.2s
🟢 act completes Text Input Variations 1 step 13.2s
🟢 act completes Broken Accessibility 2 steps · 6 calls 23.9s
🟢 act completes Vision Only 2 steps · 6 calls 27.8s
🟢 act completes Gestures 1 step · 9 calls 38.6s
🟢 act completes Async States 1 step · 9 calls 31.4s
🟢 act completes Debounced Search 1 step · 1 call 27.7s
🟢 act completes Flattened Registration Form 2 steps · 1 call 30.0s
🟢 act completes Flattened Login 2 steps · 10 calls 51.8s
🟢 act completes Sticky Chrome Target 1 step 14.1s
🟢 act completes WebView Accessibility 1 step 12.7s
🟢 act completes Permission Prompt 1 step 11.5s
🟢 act completes Photo Picker 1 step · 2 calls 30.2s
🟢 act completes Product Catalog 1 step 10.3s
🟢 act completes Error Recovery 1 step 9.8s
🟢 act completes Choice Controls 1 step · 5 calls 36.6s
🟢 act completes Stripe PaymentSheet 1 step · 7 calls 29.9s
🟢 act completes Apple Pay 1 step 11.6s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.0 · 8m 19s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 2m 58s
🟢 control inventory › a label query resolves a labeled text field 13.2s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 17.0s
🟢 control inventory › a placeholder query and attribute reads find the field 5.3s
🟢 control inventory › a secure field takes a fill and denies value reads 8.3s
🟢 control inventory › press sends a character and Enter through the soft keyboard 10.0s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 10.0s
🟢 control inventory › toBeFocused follows the focus 8.7s
🟢 control inventory › a switch checks, unchecks, and reports its state 10.3s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 8.3s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 13.0s
🟢 control inventory › a radio group reports the checked option 9.1s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 7.4s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.8s
🟢 control inventory › a header answers its accessible name 3.9s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.0s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.5s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.0s
🟢 control inventory › pointer verbs land at the point asked for 9.3s
🟢 control inventory › an ambiguous locator fails at once 5.4s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 10.1s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 442ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 38s
🟢 device fixture › setAppearance flips the color scheme the app reads 16.2s
🟢 device fixture › setOrientation rotates the window the app measures 11.2s
🟢 device fixture › the clipboard reads back what was written 6.4s
🟢 device fixture › home leaves the app, openApp brings it back where it was 6.3s
🟢 device fixture › closeApp then openApp relaunches on the home list 9.4s
🟢 device fixture › closeApp leaves the next commands on the same device 11.0s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 6.5s
🟢 device fixture › device.back pops the scenario 5.6s
🟢 device fixture › unstable_raw reaches the test session through agent-device 5.1s
🟢 device fixture › setLocation and clearLocation change what the app reads 10.5s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 10.1s
🟢 tests/device.e2e.ts · 3 passed 1m 6s
🟢 sequential onboarding echoes the exact values 41.0s
🟢 onboarding rejects a malformed email before advancing 13.6s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.3s
🟢 tests/flows-lists.e2e.ts · 7 passed 4m 17s
🟢 infinite scroll reaches the target item 37.5s
🟢 product catalog adds exactly two of the right variant 10.5s
🟢 gestures: long-press, then swipe left 9.1s
🟢 sticky chrome: accept below the fold, then continue 12.5s
🟢 async states: load, pull to refresh, claim through the toast 11.8s
🟢 huge virtualized list reaches row 512 2m 44s
🟢 web view coupon form applies the code shown on the page 12.2s
🟢 tests/flows.e2e.ts · 7 passed 1m 9s
🟢 text inputs unlock the submit button 12.9s
🟢 filled text inputs report their values 10.4s
🟢 modal flow confirms through the native alert 10.0s
🟢 bottom tabs act inside the last tab 7.3s
🟢 error recovery retries, then confirms the delete 9.2s
🟢 choice controls place the exact order 7.4s
🟢 debounced search selects the target once results arrive 11.8s
🟢 tests/login-form.e2e.ts · 3 passed 49.4s
🟢 login form › rejects a malformed email 14.6s
🟢 login form › rejects an unknown account 14.2s
🟢 login form › signs in and logs out 20.7s
🟢 tests/smoke.e2e.ts · 1 passed 7.7s
🟢 home lists the scenarios and opens one 7.7s

Tested by e2e 0.23.0 · 7m 2s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 1 flaky, 19 passed

24 agent steps · 15 replayed from cache · 36 model calls · 330.6k tokens (79% cached) · $0.0250

⚠️ 1 flaky test passed on a retry

⚠️ act completes Async States
tests-agent/scenarios.e2e.ts:266

ASSERTION_FAILED at step 4 of 4: agent.act "Complete this scenario as the screen instructs: wait for the screen to load, scroll up on the reward area to refresh it, then claim the reward", after 44.2s and 17 model calls

agent.act failed: Waited for loading to finish and tried scrolling and pull-to-refresh gestures on the reward area, but it still shows “Pull down to refresh and reveal your reward.” No reward or claim control appeared, so I could not claim it.

  • Turn 14: observe({}) → [observe] Observed.
  • Turn 15: swipe({"from":{"x":120,"y":190},"to":{"x":120,"y":700}}) → [swipe] Swiped from (120, 190) to (120, 700).
  • Turn 16: observe({}) → [observe] Observed.
  • Screen: dev.e2e.benchmark

Evidence: screenshot, log

All 20 tests in 1 file
Test Agent Time
⚠️ tests-agent/scenarios.e2e.ts · 1 flaky, 19 passed 24 steps · 36 calls 6m 30s
🟢 act completes Login Form 1 step 26.4s
🟢 act completes Infinite Scroll List 1 step 26.1s
🟢 act completes Modal Flow 1 step 12.6s
🟢 act completes Bottom Tabs 1 step 8.6s
🟢 act completes Text Input Variations 1 step 10.5s
🟢 act completes Broken Accessibility 2 steps · 5 calls 16.0s
🟢 act completes Vision Only 2 steps · 1 call 6.9s
🟢 act completes Gestures 1 step · 7 calls 20.4s
⚠️ act completes Async States (1 failed attempt first) 1 step · 9 calls 21.8s
🟢 act completes Debounced Search 1 step 7.1s
🟢 act completes Huge Virtualized List 1 step 1m 57s
🟢 act completes Flattened Registration Form 2 steps · 1 call 11.4s
🟢 act completes Flattened Login 2 steps · 8 calls 27.4s
🟢 act completes Sticky Chrome Target 1 step 10.7s
🟢 act completes WebView Accessibility 1 step · 5 calls 18.1s
🟢 act completes Permission Prompt 1 step 8.3s
🟢 act completes Product Catalog 1 step 9.8s
🟢 act completes Error Recovery 1 step 10.0s
🟢 act completes Choice Controls 1 step 9.6s
🟢 act completes Sequential Onboarding 1 step 10.9s

Tested by e2e 0.23.0 · 7m 22s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android: 57 passed, 1 skipped

All 58 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 42s
🟢 control inventory › a label query resolves a labeled text field 23.4s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 12.3s
🟢 control inventory › a placeholder query and attribute reads find the field 5.5s
🟢 control inventory › a secure field takes a fill and denies value reads 6.4s
🟢 control inventory › press sends a character and Enter through the soft keyboard 7.5s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 8.0s
🟢 control inventory › toBeFocused follows the focus 7.8s
🟢 control inventory › a switch checks, unchecks, and reports its state 7.7s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 6.6s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 9.9s
🟢 control inventory › a tab role answers getByRole 3.6s
🟢 control inventory › a radio group reports the checked option 7.3s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.8s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.9s
🟢 control inventory › a header answers its accessible name 3.7s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.4s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.4s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.0s
🟢 control inventory › a double tap lands inside the 300 ms window 6.5s
🟢 control inventory › an ambiguous locator fails at once 4.9s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 6.9s
🟢 refusals › verbs a device never declares are refused before the engine 7ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.3s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 19s
🟢 device fixture › setOrientation rotates the window the app measures 4.9s
🟢 device fixture › home leaves the app, openApp brings it back where it was 5.0s
🟢 device fixture › foregroundApp sees the home screen after home() 4.1s
🟢 device fixture › closeApp then openApp relaunches on the home list 6.7s
🟢 device fixture › closeApp leaves the next commands on the same device 6.5s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 5.2s
🟢 device fixture › device.back pops the scenario 4.8s
🟢 device fixture › unstable_raw reaches the test session through agent-device 3.5s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 17.0s
🟢 device fixture › setLocation and clearLocation change what the app reads 21.7s
🟢 tests/device.e2e.ts · 5 passed 44.7s
🟢 permission prompt › allowing the system dialog enables the microphone 6.2s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 15.5s
🟢 sequential onboarding echoes the exact values 10.4s
🟢 onboarding rejects a malformed email before advancing 5.8s
🟢 dismissing the keyboard over choice controls leaves them as they were 6.8s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 16s
🟢 infinite scroll reaches the target item 29.3s
🟢 product catalog adds exactly two of the right variant 9.0s
🟢 gestures: long-press, then swipe left 8.0s
🟢 gestures: double-tap completes the flow 9.2s
🟢 sticky chrome: accept below the fold, then continue 11.1s
🟢 async states: load, pull to refresh, claim through the toast 12.0s
🟢 huge virtualized list reaches row 512 1m 57s
🟢 tests/flows.e2e.ts · 7 passed 52.9s
🟢 text inputs unlock the submit button 7.3s
🟢 filled text inputs report their values 6.1s
🟢 modal flow confirms through the native alert 10.0s
🟢 bottom tabs act inside the last tab 6.8s
🟢 error recovery retries, then confirms the delete 8.8s
🟢 choice controls place the exact order 5.7s
🟢 debounced search selects the target once results arrive 8.2s
🟢 tests/login-form.e2e.ts · 3 passed 25.2s
🟢 login form › rejects a malformed email 6.7s
🟢 login form › rejects an unknown account 6.7s
🟢 login form › signs in and logs out 11.8s
🟢 tests/smoke.e2e.ts · 1 passed 4.9s
🟢 home lists the scenarios and opens one 4.9s

Tested by e2e 0.23.0 · 9m 25s · android-emulator · run artifacts

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the suites, a can-fail pass against the merge base, and a real CLI repro. The leak fix is right. On the base build another port on the same host gets x-gate and answers basic auth. On this branch it gets neither, and listing it in headerOrigins / basicAuth.origins brings it back, on the real port for a port-0 app.url. The new tests fail on the base, so they pin the fix. engineAppInfo() is the only builder, so every path passes origin.

Holding it for one regression and a merge-order issue.

Blocker. WebKit basic auth breaks on 443/80 and IPv6 hosts. Details inline on protected-app.ts.

Merge order vs #1590. Inline on package.json.

Should-fix

  • Wildcard and port-0 origins pass validation and match nothing (inline).
  • SECURITY.md:79-81 still says the browser engine's headers reach the site and no other host. It isn't in the diff, so noting it here.
  • The headers JSDoc and docs/reference/web.mdx say other origins never get them, which contradicts the redirect behavior documented elsewhere (inline).
  • The routes.test.ts cross-origin case can't fail (inline).

Decision. headerOrigins sits beside headers while basicAuth.origins is nested. I get why, since an origins key inside the headers record would collide with a header name. Still, this breaking minor is the cheap moment to settle the shape, so worth a deliberate call.

Nits

  • docs/migrate/playwright.mdx: the extraHTTPHeaders row doesn't mention headerOrigins, and that's the row most likely to break a migrating suite.
  • docs/security.mdx says basicAuth goes on every request. It only answers a challenge.
  • e2e doctor passes origin with :0 for a port-0 app (doctor.ts:205 never assigns ports), against the new JSDoc that says "on the run's port".
  • "origin headers" in writing-tests.md and browser.ts reads like the HTTP Origin header. route-options.ts:29 still says "the site's".

Comment thread packages/web/src/protected-app.ts Resolved
Comment thread packages/web/src/protected-app.ts Resolved
Comment thread packages/web/package.json
},
"peerDependencies": {
"e2e": ">=0.22.1 <1"
"e2e": ">=0.23.1 <1"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1590 already proposes e2e 0.23.1 from the three patch changesets on main, without this PR's origin change. If it merges first, the runner side of this PR ships as 0.23.2, but web 0.16.0 still accepts 0.23.1. Headers and basicAuth users on 0.23.1 then hit "needs e2e 0.23.1 or later" in init while running 0.23.1. onlyUpdatePeerDependentsWhenOutOfRange won't move the range.

Either land this before #1590, or bump the floor, the guard message in surface.ts and the changeset to 0.23.2.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude-opus-5-5] responding on behalf of KrzysztofMoch

d875e67 takes the version out of the init guard message and the changeset. It now says to update e2e to the release that ships with this @e2e-dev/web. The floor stays >=0.23.1. If this lands first, 0.23.1 is the release that carries origin, and a 0.23.2 floor would leave @e2e-dev/web with no runner that satisfies it. If #1590 lands first, I'll raise the floor to 0.23.2 when I rebase. Leaving this open, since the merge order isn't mine to pick.

Comment thread packages/web/src/surface.ts Resolved
@KrzysztofMoch
KrzysztofMoch force-pushed the krzysztof/fix/web-basicauth-origin-scope branch from 5cccbf0 to 7049646 Compare November 6, 2026 10:31
@github-actions github-actions Bot added size:XL Up to 999 changed lines, not counting tests, fixtures, or the lockfile and removed Ready for Human Review size:L Up to 499 changed lines, not counting tests, fixtures, or the lockfile labels Nov 6, 2026
@KrzysztofMoch

Copy link
Copy Markdown
Member Author

[claude-opus-5-5] responding on behalf of KrzysztofMoch

The review-level items, all in d875e67 and 7049646:

  • Decision. Krzysztof chose one list: web({ headers, basicAuth, credentialOrigins }). It replaces headerOrigins and basicAuth.origins, covers both, and needs headers or basicAuth beside it. headers stays a plain record, so existing configs don't change shape.
  • SECURITY.md:79-81. It now says headers and basicAuth are scoped to the exact origin plus credentialOrigins. It also names the redirect and WebKit caveats.
  • Migration row. use.extraHTTPHeaders and use.httpCredentials both point to credentialOrigins.
  • docs/security.mdx. basicAuth now "answers a 401 from those origins", not "goes on every request".
  • e2e doctor :0. engineAppInfo now leaves origin out while a free port is unassigned. The EngineAppInfo.origin JSDoc and reference/engine.mdx say so, and app-ports.test.ts pins it. The web engine's doctor hook doesn't read origin, so nothing else changes.
  • "origin headers". Renamed configuredHeaders, configuredHeadersFor, and installConfiguredHeaders. writing-tests.md says "the configured headers", and route-options.ts:29 "the configured ones".

The PR body has the updated CLI table and the WebKit run.

headers went to every host on the app's site and basicAuth answered a 401 from any origin. Both now go to the app's exact origin, with headerOrigins and basicAuth.origins for extras. The runner passes the origin to engines as EngineAppInfo.origin.
…t ports

headerOrigins and basicAuth.origins become one credentialOrigins option
that covers both. WebKit never matches a scoped credential to an origin
on a default port or with an IPv6 host, so there basic auth goes
unscoped, as Playwright's default. A wildcard host or port 0 in the list
is INVALID_CONFIG, since it would match nothing. e2e doctor leaves the
origin out until a free port is assigned. Docs no longer claim headers
skip other origins: a redirect keeps them.
@KrzysztofMoch
KrzysztofMoch force-pushed the krzysztof/fix/web-basicauth-origin-scope branch from 7049646 to 4ad8d7b Compare November 10, 2026 15:37
@okwasniewski
okwasniewski merged commit 046bd08 into main Nov 12, 2026
32 checks passed
@okwasniewski
okwasniewski deleted the krzysztof/fix/web-basicauth-origin-scope branch November 12, 2026 08:49
Varshith-Kali added a commit to Varshith-Kali/e2e that referenced this pull request Nov 12, 2026
The rebase onto 046bd08 dropped the web package.json bump, since e2e-inc#1393 already raised web's e2e peer floor to >=0.23.1 on main. The changeset still listed @e2e-dev/web as patch and described a web floor raise, which would have shipped a wrong CHANGELOG line and an extra web release. Drop web from the frontmatter and the bullet; mobile, acp, and limrun are unchanged.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Varshith-Kali added a commit to Varshith-Kali/e2e that referenced this pull request Nov 15, 2026
The rebase onto 046bd08 dropped the web package.json bump, since e2e-inc#1393 already raised web's e2e peer floor to >=0.23.1 on main. The changeset still listed @e2e-dev/web as patch and described a web floor raise, which would have shipped a wrong CHANGELOG line and an extra web release. Drop web from the frontmatter and the bullet; mobile, acp, and limrun are unchanged.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Varshith-Kali added a commit to Varshith-Kali/e2e that referenced this pull request Nov 15, 2026
The rebase onto 046bd08 dropped the web package.json bump, since e2e-inc#1393 already raised web's e2e peer floor to >=0.23.1 on main. The changeset still listed @e2e-dev/web as patch and described a web floor raise, which would have shipped a wrong CHANGELOG line and an extra web release. Drop web from the frontmatter and the bullet; mobile, acp, and limrun are unchanged.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL Up to 999 changed lines, not counting tests, fixtures, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants