From a5648688d6c0526fbdd5e8a69a64320f440a9b1f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dar=C3=ADo=20Kondratiuk?= Date: Wed, 4 Nov 2026 21:15:54 -0300 Subject: [PATCH 1/2] Drop the known-difference markers of three click tests: #1079, #1080 and #1081 are fixed (Refs #1060) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01Ls2e1yaZFgZ3FEJkYScfkK --- COMPATIBILITY.md | 2 +- tests/E2E.NUnit.Tests/Click/PageClickTests.cs | 32 +++++++------------ .../ElementhandleScrollIntoViewTests.cs | 8 ++--- 3 files changed, 15 insertions(+), 27 deletions(-) diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index ad311d45e..e8203cc78 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -162,7 +162,7 @@ Differences: - Upstream's docs site pages that have no counterpart in the port: the page objects page (`docs/page-objects.mdx`, moved to `/page-object-model` in `d30114a`), `e2e feedback` and the `docs/reference/cli.mdx` text about its opt-out variables (`4c1e2e8`), and `examples/` with its registry (`b3ae3a4`, `38b802d`, `a29edcb`, `b54646f`). The migration guides (`docs/migrate/*.mdx`, `8537abc`) are ported: see [Migration guides](#migration-guides). The docs site has .NET-only guides for upstream's five example apps of `caf51eb` (tester-army/e2e#1497: i18n, file upload, location, biometrics, deep links), under [Examples](docs-site/src/content/docs/examples/index.mdx). They are not ports: that commit only adds registry entries that link to separate repositories, so there was no guide text. Each page writes C# tests for the upstream app. The self-healing deep dive (`6fa99ea`) is ported as the [Self-healing tests](README.md#self-healing-tests) section of the guide, since the port has no docs pages for the cache, the trace page, or executors to link to. Known gap: only `e2e explore` prints a run summary, since it is the one command that feeds a run to `ListReporter` (see [List reporter](#list-reporter)): the `Cache 4 replayed · 1 handed off · 1 missed` row is `Format.CacheText` over `Format.StepsCacheTally` (`E2E.Cli`, upstream's `cacheText` and `stepsCacheTally` of `report/format.ts`), which an exploration, with its cache off, never fills, and a test run does not print ([#799](https://github.com/hardkoded/e2e-dotnet/issues/799)) - Parallel workers and serial suites - `test.fail` and the `fail` test option (tester-army/e2e#1539, `732ea8a`): a test expected to fail, whose product failure passes and whose pass fails with `UNEXPECTED_PASS`. It lives in upstream's test registry, scheduler, retry, and reporters (`settleExpectedFailure` runs on a finished attempt), and the port has none of them. NUnit and xUnit v3 own the verdict and the retries, and neither has an expected-failure outcome, so a port would need a new public API of its own -- Playwright's page tests in the testbed suite (tester-army/e2e#1505, `bdf769b`) are ported in slices (hardkoded/e2e-dotnet#1060), as the `playwright` category of `tests/E2E.NUnit.Tests`, against the playground's `/content` route and Playwright's test assets (Apache-2.0, `Testbed/playwright/`, with Playwright's `LICENSE` and `NOTICE`). A test upstream marks `{ fail: 'reason' }` keeps its assertions inside the test-only `Helpers.ExpectedToFailAsync`, which passes while the body fails and fails once it passes. Ported so far: `check`, `wait-for-url`, `locator-list`, `selectors-text`, `fill`, `select-option`, `set-input-files`, `filechooser`, `click`, `click-scroll`, and `elementhandle-scroll-into-view` (three known differences: hardkoded/e2e-dotnet#1079, #1080, #1081). Upstream's `select-option` test marked `fail` (a bare string selects by label only) passes in the port, which matches a value or a label, so it has no marker. A missing file reports .NET's `Could not find file`, where upstream shows Node's `ENOENT`. +- Playwright's page tests in the testbed suite (tester-army/e2e#1505, `bdf769b`) are ported in slices (hardkoded/e2e-dotnet#1060), as the `playwright` category of `tests/E2E.NUnit.Tests`, against the playground's `/content` route and Playwright's test assets (Apache-2.0, `Testbed/playwright/`, with Playwright's `LICENSE` and `NOTICE`). A test upstream marks `{ fail: 'reason' }` keeps its assertions inside the test-only `Helpers.ExpectedToFailAsync`, which passes while the body fails and fails once it passes. Ported so far: `check`, `wait-for-url`, `locator-list`, `selectors-text`, `fill`, `select-option`, `set-input-files`, `filechooser`, `click`, `click-scroll`, and `elementhandle-scroll-into-view`. Upstream's `select-option` test marked `fail` (a bare string selects by label only) passes in the port, which matches a value or a label, so it has no marker. A missing file reports .NET's `Could not find file`, where upstream shows Node's `ENOENT`. - Serial groups (upstream tester-army/e2e#1056, #1470): serial runs, group retries, and skipping the rest of a group after a failure belong to the test runner, which is NUnit or xUnit here, so the port keeps no serial groups. The schema has `serialAttempt.cutRetry` and the serial verdict is read from it, but nothing in the port writes a serial attempt: see [Report document](#report-document) - The out-of-process engine protocol (upstream tester-army/e2e#1232, [#873](https://github.com/hardkoded/e2e-dotnet/issues/873)): the port has no runner and no worker pool, so there is no engine process to talk to. An engine is written in .NET against `IEngine` - Diff-only observations for a custom executor's own model turns. The built-in agent has them (#1022): `#id` lines on the opening screen and on `observe`, and a diff after each action (see [MCP server](#mcp-server) for the presenter). `ExecutorObservation.Text`, which an executor reads from `Observe`, keeps the full `[ref=…]` snapshot, cut at the observation byte budget (see [Agents](#agents)), and a node is addressed by its ref there (`new ExecutorTarget("e12")`), so a custom executor has no presenter of its own. Upstream gives every executor the `#id` view. diff --git a/tests/E2E.NUnit.Tests/Click/PageClickTests.cs b/tests/E2E.NUnit.Tests/Click/PageClickTests.cs index 7f340513a..73edf187e 100644 --- a/tests/E2E.NUnit.Tests/Click/PageClickTests.cs +++ b/tests/E2E.NUnit.Tests/Click/PageClickTests.cs @@ -59,14 +59,10 @@ await Browser.GotoAsync(Content(""" [Test] public async Task Should_click_the_button_if_window_Node_is_removed() { - // Known difference (hardkoded/e2e-dotnet#1079): the page script reads window.Node. - await ExpectedToFailAsync("Node is not defined", async () => - { - await Browser.GotoAsync("/input/button.html"); - await Browser.EvaluateAsync("() => { delete window.Node; }"); - await Browser.Locator("button").TapAsync(); - Equal("Clicked", await ResultAsync()); - }); + await Browser.GotoAsync("/input/button.html"); + await Browser.EvaluateAsync("() => { delete window.Node; }"); + await Browser.Locator("button").TapAsync(); + Assert.That(await ResultAsync(), Is.EqualTo("Clicked")); } [Test] @@ -519,18 +515,14 @@ public async Task Should_click_the_button_when_window_innerWidth_is_corrupted() [Test] public async Task Should_click_zero_sized_input_by_label() { - // Known difference (hardkoded/e2e-dotnet#1080): the label is not actionable. - await ExpectedToFailAsync("not visible", async () => - { - await Browser.GotoAsync(Content(""" - - """)); - await Screen.GetByText("Click me").TapAsync(new ActionOptions { Timeout = TimeSpan.FromSeconds(3) }); - Equal(true, await Browser.EvaluateAsync("window.__clicked")); - }); + await Browser.GotoAsync(Content(""" + + """)); + await Screen.GetByText("Click me").TapAsync(); + Assert.That(await Browser.EvaluateAsync("window.__clicked"), Is.True); } [Test] diff --git a/tests/E2E.NUnit.Tests/ElementhandleScrollIntoView/ElementhandleScrollIntoViewTests.cs b/tests/E2E.NUnit.Tests/ElementhandleScrollIntoView/ElementhandleScrollIntoViewTests.cs index 20a95d6d7..d7fb78143 100644 --- a/tests/E2E.NUnit.Tests/ElementhandleScrollIntoView/ElementhandleScrollIntoViewTests.cs +++ b/tests/E2E.NUnit.Tests/ElementhandleScrollIntoView/ElementhandleScrollIntoViewTests.cs @@ -81,12 +81,8 @@ await Browser.GotoAsync(Content(""" [Test] public async Task Should_work_for_visibility_hidden_element() { - // Known difference (hardkoded/e2e-dotnet#1081): ScrollIntoViewAsync refuses a hidden node. - await ExpectedToFailAsync("is hidden", async () => - { - await Browser.GotoAsync(Content("
Hello
")); - await Browser.Locator("div").ScrollIntoViewAsync(new ActionOptions { Timeout = TimeSpan.FromSeconds(3) }); - }); + await Browser.GotoAsync(Content("
Hello
")); + await Browser.Locator("div").ScrollIntoViewAsync(); } [Test] From 5a35eeefe28744ab3e35116853863a15c8692866 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dar=C3=ADo=20Kondratiuk?= Date: Wed, 4 Nov 2026 21:19:20 -0300 Subject: [PATCH 2/2] Run the nested-frame role tests as normal tests (Refs #1060) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01Ls2e1yaZFgZ3FEJkYScfkK --- .../LocatorFrame/LocatorFrameTests.cs | 26 ++++++++----------- .../SelectorsFrame/SelectorsFrameTests.cs | 10 +++---- 2 files changed, 14 insertions(+), 22 deletions(-) diff --git a/tests/E2E.NUnit.Tests/LocatorFrame/LocatorFrameTests.cs b/tests/E2E.NUnit.Tests/LocatorFrame/LocatorFrameTests.cs index aeaed09b9..aeb324bf0 100644 --- a/tests/E2E.NUnit.Tests/LocatorFrame/LocatorFrameTests.cs +++ b/tests/E2E.NUnit.Tests/LocatorFrame/LocatorFrameTests.cs @@ -156,24 +156,20 @@ public async Task Click_should_survive_iframe_navigation() await navigated; } - // Known difference, hardkoded/e2e-dotnet#1066: a frame-scoped getByRole also matches the nested frame's button. [Test] public async Task GetBy_coverage_role_text_test_id_label_placeholder() { - await ExpectedToFailAsync("observed matched 2 nodes", async () => - { - await Browser.GotoAsync(Routed); - var button1 = Browser.FrameLocator("iframe").GetByRole("button"); - var button2 = Browser.FrameLocator("iframe").GetByText("Hello", exact: false); - var button3 = Browser.FrameLocator("iframe").GetByTestId("buttonId"); - await Expect.That(button1).ToHaveTextAsync("Hello iframe"); - await Expect.That(button2).ToHaveTextAsync("Hello iframe"); - await Expect.That(button3).ToHaveTextAsync("Hello iframe"); - var input1 = Browser.FrameLocator("iframe").GetByLabel("Name"); - await Expect.That(input1).ToHaveValueAsync(""); - var input2 = Browser.FrameLocator("iframe").GetByPlaceholder("Placeholder"); - await Expect.That(input2).ToHaveValueAsync(""); - }); + await Browser.GotoAsync(Routed); + var button1 = Browser.FrameLocator("iframe").GetByRole("button"); + var button2 = Browser.FrameLocator("iframe").GetByText("Hello", exact: false); + var button3 = Browser.FrameLocator("iframe").GetByTestId("buttonId"); + await Expect.That(button1).ToHaveTextAsync("Hello iframe"); + await Expect.That(button2).ToHaveTextAsync("Hello iframe"); + await Expect.That(button3).ToHaveTextAsync("Hello iframe"); + var input1 = Browser.FrameLocator("iframe").GetByLabel("Name"); + await Expect.That(input1).ToHaveValueAsync(""); + var input2 = Browser.FrameLocator("iframe").GetByPlaceholder("Placeholder"); + await Expect.That(input2).ToHaveValueAsync(""); } [Test] diff --git a/tests/E2E.NUnit.Tests/SelectorsFrame/SelectorsFrameTests.cs b/tests/E2E.NUnit.Tests/SelectorsFrame/SelectorsFrameTests.cs index 6e468c92f..94e5d06fa 100644 --- a/tests/E2E.NUnit.Tests/SelectorsFrame/SelectorsFrameTests.cs +++ b/tests/E2E.NUnit.Tests/SelectorsFrame/SelectorsFrameTests.cs @@ -85,16 +85,12 @@ public async Task DollarDollarEval_should_not_throw_for_missing_frame() Assert.That(await Browser.FrameLocator("iframe").Locator("canvas").AllTextContentsAsync(), Has.Count.EqualTo(0)); } - // Known difference, hardkoded/e2e-dotnet#1066: a frame-scoped getByRole also matches the nested frame's button. [Test] public async Task Should_work_for_DollarEval() { - await ExpectedToFailAsync("observed count 2", async () => - { - await Browser.GotoAsync(Routed); - await Expect.That(Browser.FrameLocator("iframe").GetByRole("button")).ToHaveCountAsync(1); - await Expect.That(Browser.FrameLocator("iframe").Locator("button")).ToHaveCountAsync(1); - }); + await Browser.GotoAsync(Routed); + await Expect.That(Browser.FrameLocator("iframe").GetByRole("button")).ToHaveCountAsync(1); + await Expect.That(Browser.FrameLocator("iframe").Locator("button")).ToHaveCountAsync(1); } [Test]