Skip to content

Upstream: Playwright's page tests in the testbed suite (#1505) #1060

Closed

Description

@kblok

Upstream commit: tester-army/e2e bdf769b (#1505), e2e-inc/e2e@bdf769b0 (8744 additions, 130 files: apps/testbed/app/playwright/** assets and the tests that use them, plus config). Per CLAUDE.md, port the tests and their testbed pages, as real-model or web tests where upstream uses a real browser. This is big: first list what the commit adds (assets, test files, it count), then port in slices (one PR per test file or group), each with Refs until the last one. Report which tests need features the port lacks (name each, and file an issue per missing feature).

Activity

  1. kblok commented on Nov 4, 2026

    @kblok
    MemberAuthor

    Plan for #1060 (upstream bdf769b, e2e-inc/e2e#1505)

    What the commit adds. 130 files. apps/testbed/app/playwright/ holds Playwright's test assets (about 64 KB in total; the biggest file is a 6 KB png; chromium-linux.zip is a 325-byte stub), a LICENSE (Apache-2.0), a NOTICE, and routes.mjs (dynamic routes: /content, /echo, /status, /slow, /never, /redirect, /download, /respond, /cors-echo, /bounce, /count, /options, /upload, /binary, /download-stall, /sse, and a /ws upgrade). It adds 32 test files (about 496 it, tagged playwright, written against the browser fixture and the e2e public API). Smaller changes: a patternLabel step label for waitForURL, route, unroute, waitForResponse, and a translated(call, ...) timeout message (with a unit test in fixture-runtime.test.ts).

    Size and licensing. No size problem and no big binary. The port is already Apache-2.0 and has a NOTICE. The assets need Playwright's LICENSE and NOTICE next to them. No decision needed. Plan: copy only the assets a slice uses, and add the license files in the first slice.

    How the port hosts these. The port's testbed is tests/E2E.Agent.Tests/Testbed.cs (an HttpListener on a random port, linked into E2E.NUnit.Tests). Browser tests live in tests/E2E.NUnit.Tests/<UpstreamFile>/ (NUnit E2ETest, Browser, Screen). Upstream's fixed PREFIX (127.0.0.1:4271) and CROSS (localhost:4271) become the testbed's own port. Upstream's fail: 'reason' (an expected failure) has no port equivalent: a small .NET-only test helper keeps such a test with its upstream assertions and passes only while the body still fails.

    Upstream test files (it count)

    Group Files it
    A. Only /content and a few assets check 13, wait-for-url 7, locator-list 1, selectors-text 2, selectors-get-by 19, selectors-role 15, locator-is-visible 10, locator-convenience 7, locator-misc-2 10, locator-query 19 103
    B. Input pages fill 39, select-option 17, set-input-files 18, filechooser 6, click 47, click-scroll 6, elementhandle-scroll-into-view 8, keyboard 50 191
    C. Frames locator-frame 17, selectors-frame 13, frame-goto 1, frame-hierarchy 3 34
    D. Network and routes route 40, request-continue 35, request-fulfill 16, request-fallback 6, wait-for-response 3, wait-for-navigation 9, network-idle 11 (need the routes.mjs routes and /ws) 120
    E. Dialog and download dialog 9, download 19 28
    F. Screenshots screenshot 20 (10 toHaveScreenshot goldens; check how the port stores goldens first) 20

    Missing features. Upstream already left out every test whose feature e2e lacks (each file header has a "Not ported" list, for example: no force/trial/position options, no route(..., { times }), no fallback({ headers | url | method | postData }), no download.saveAs, no fullPage/clip screenshots, no element handles or Frame objects, no waitForEvent). The port mirrors e2e, so those stay unported and are named in each PR. Gaps specific to the port are not known until each file is ported. I check each slice and file an issue per gap (searching first). To verify first: the error texts of check/uncheck (group A), the fail: semantics, toHaveScreenshot goldens (group F), and the patternLabel / timeout-label source change (its own small PR with its unit test).

    Slicing (one PR each, Refs #1060)

    1. Slice 1 (starting now): the /content route and asset serving in the testbed, Playwright LICENSE/NOTICE, the fail helper, and check (13), wait-for-url (7), locator-list (1), selectors-text (2). 23 tests.
    2. Rest of group A (one or two PRs: selectors; locator-*).
    3. patternLabel and the timeout label (src change plus its unit test).
    4. Group B in three PRs: fill + select-option + set-input-files + filechooser; click + click-scroll + elementhandle-scroll-into-view; keyboard.
    5. Group C.
    6. Group D in three PRs: route; request-continue + fallback + fulfill; wait-for-* + network-idle (with the routes.mjs routes and /ws).
    7. Group E, then group F.
  2. 12 remaining items

  3. kblok commented on Nov 5, 2026

    @kblok
    MemberAuthor

    All slices are merged. #1138 delivered the last one (upstream testbed edits from e2e-inc/e2e#1548). Known partials, as upstream: FrameHierarchy Partial_Should_send_framenavigated_when_navigating_on_anchor_URLs and Partial_Should_report_frame_from_inside_shadow_DOM need frame events and Frame objects, which the port does not have. The fixture-runtime test is tracked by #1120 (PR #1140).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions