Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 3 additions & 0 deletions .csharpierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"printWidth": 120
}
5 changes: 5 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Commits that only reformat code. `git blame` skips them when run with
# `git config blame.ignoreRevsFile .git-blame-ignore-revs`. GitHub skips them already.

# Format codebase with CSharpier
eaacd7b0a690886a276bb48409e6a715ddff5e33
26 changes: 25 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,26 @@ permissions:
contents: read

jobs:
csharpier:
name: csharpier
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
10.0.x
11.0.x

- name: Restore tools
run: dotnet tool restore

- name: Check formatting
run: dotnet csharpier check .

build:
name: restore-build-test
runs-on: ubuntu-latest
Expand All @@ -28,8 +48,12 @@ jobs:
- name: Restore
run: dotnet restore E2E.slnx

# CSharpier owns whitespace and layout (see the csharpier job), so `dotnet format` checks only code style and
# analyzers. Its whitespace pass would undo CSharpier's layout.
- name: Format
run: dotnet format E2E.slnx --verify-no-changes --no-restore
run: |
dotnet format style E2E.slnx --verify-no-changes --no-restore
dotnet format analyzers E2E.slnx --verify-no-changes --no-restore

- name: Build
run: dotnet build E2E.slnx --no-restore -c Release
Expand Down
4 changes: 4 additions & 0 deletions .husky/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

dotnet husky run --group pre-push
11 changes: 11 additions & 0 deletions .husky/task-runner.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"$schema": "https://alirezanet.github.io/Husky.Net/schema.json",
"tasks": [
{
"name": "csharpier-check",
"group": "pre-push",
"command": "dotnet",
"args": ["csharpier", "check", "."]
}
]
}
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ This repo is a .NET port of [tester-army/e2e](https://github.com/tester-army/e2e

## Before you push

- CSharpier owns formatting. Run `dotnet csharpier format .` from the repo root before every commit/push. The Husky.Net pre-push hook runs `dotnet csharpier check .` and blocks unformatted pushes. Do not run `dotnet format whitespace` or a plain `dotnet format`, because their whitespace pass fights CSharpier. Use `dotnet format style` / `dotnet format analyzers` only.
- If the change can affect the agent, run every real-model test: `dotnet test E2E.slnx --filter "TestCategory=RealModel"`. This covers the agent, prompts, cache, replay, the web engine, and the page script. Doc-only and config-only changes skip this step.
- The tests use the GitHub Copilot login (`dotnet run --project src/E2E.Cli -- login github-copilot`). If you cannot log in, say so in your report. Do not skip the tests silently.
- Real-model tests can be flaky. Rerun a failed test once. If it fails twice, treat it as a real failure and report it.
Expand Down
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ dotnet test

Style is enforced at build time through `.editorconfig` and `Directory.Build.props` (`EnforceCodeStyleInBuild`, `TreatWarningsAsErrors`). C# files use the Apache file header.

[CSharpier](https://csharpier.com/) formats C# and the MSBuild files. It is a local tool in `dotnet-tools.json`, with its settings in `.csharpierrc.json`. Run `dotnet csharpier format .` before you commit. CI runs `dotnet csharpier check .` and fails on unformatted files. `dotnet format` in CI checks only code style and analyzers, because CSharpier owns the whitespace.

A [Husky.Net](https://alirezanet.github.io/Husky.Net/) `pre-push` hook runs `dotnet csharpier check .` and stops the push if a file is not formatted. The hook installs itself: restoring or building `src/E2E` runs `dotnet tool restore` and `dotnet husky install`. This step is skipped in CI (`CI=true`), outside a git checkout, and when `HUSKY=0` is set. To install the hook by hand, run `dotnet tool restore` and then `dotnet husky install`.

The commit that first formatted the codebase is listed in `.git-blame-ignore-revs`. Run `git config blame.ignoreRevsFile .git-blame-ignore-revs` once so `git blame` skips it. GitHub skips it already.

Do not commit API keys.

## Documentation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,22 @@ public async Task Drags_both_shapes_into_their_slots_and_saves()
Assert.That(await PixelAt(Browser, RectSlot), Is.EqualTo(White));
Assert.That(await PixelAt(Browser, CircleSlot), Is.EqualTo(White));

await Screen.SwipeAsync(new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, RectHome),
To = await OnCanvas(Browser, CanvasSize, RectSlot),
});
await Screen.SwipeAsync(
new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, RectHome),
To = await OnCanvas(Browser, CanvasSize, RectSlot),
}
);
await Expect.Poll(() => PixelAt(Browser, RectSlot)).ToMatchAsync(Is.EqualTo(Blue));

await Screen.SwipeAsync(new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, CircleHome),
To = await OnCanvas(Browser, CanvasSize, CircleSlot),
});
await Screen.SwipeAsync(
new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, CircleHome),
To = await OnCanvas(Browser, CanvasSize, CircleSlot),
}
);
await Expect.Poll(() => PixelAt(Browser, CircleSlot)).ToMatchAsync(Is.EqualTo(Orange));
await Expect.Poll(() => PixelAt(Browser, SaveButtonEdge)).ToMatchAsync(Is.EqualTo(Blue));

Expand All @@ -65,11 +69,13 @@ await Screen.SwipeAsync(new SwipePathOptions
[Test]
public async Task Bounces_a_shape_off_the_wrong_slot_and_refuses_to_save_early()
{
await Screen.SwipeAsync(new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, RectHome),
To = await OnCanvas(Browser, CanvasSize, CircleSlot),
});
await Screen.SwipeAsync(
new SwipePathOptions
{
From = await OnCanvas(Browser, CanvasSize, RectHome),
To = await OnCanvas(Browser, CanvasSize, CircleSlot),
}
);
await Expect.Poll(() => CountColor(Browser, BannerBand, Red)).ToMatchAsync(Is.GreaterThan(0));
Assert.That(await PixelAt(Browser, RectHome), Is.EqualTo(Blue));
Assert.That(await PixelAt(Browser, CircleSlot), Is.EqualTo(White));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,13 @@ public sealed class CheckoutReviewTests : BenchmarkTest
public async Task Shows_a_raw_greeting_token_and_an_undefined_delivery_date_planted_bug_yet_places_the_order()
{
await App.OpenAsync("/e/checkout-review");
await Expect.That(Screen.GetByTestId("checkout-greeting")).ToHaveTextAsync("Thanks, {{firstName}}! Review your order below.");
await Expect
.That(Screen.GetByTestId("checkout-greeting"))
.ToHaveTextAsync("Thanks, {{firstName}}! Review your order below.");
await Expect.That(Screen.GetByTestId("delivery-estimate")).ToHaveTextAsync("Delivery: Arrives by undefined");
await Expect.That(Screen.GetByTestId("order-lines").GetByRole("listitem")).ToContainTextAsync(["Gooseneck Kettle x1", "Single Origin Beans 1kg x2"]);
await Expect
.That(Screen.GetByTestId("order-lines").GetByRole("listitem"))
.ToContainTextAsync(["Gooseneck Kettle x1", "Single Origin Beans 1kg x2"]);
await Expect.That(Screen.GetByText("Ship to: 221B Baker Street, London")).ToBeVisibleAsync();

await Screen.GetByRole("button", "Place order - $123.00").TapAsync();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ public async Task The_decoy_menu_items_refuse_and_keep_the_file()
var more = Screen.GetByRole("button", "More actions for report.pdf");
await more.TapAsync();
await Screen.GetByRole("button", "Duplicate").TapAsync();
await Expect.That(Screen.GetByTestId("notice-message")).ToHaveTextAsync("Duplicating is disabled for sample files");
await Expect
.That(Screen.GetByTestId("notice-message"))
.ToHaveTextAsync("Duplicating is disabled for sample files");

await more.TapAsync();
await Screen.GetByRole("button", "Delete").TapAsync();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ public async Task The_file_menu_opens_on_right_click_only_and_Rename_renames_the
{
var row = Screen.GetByText("report.pdf");
await row.TapAsync();
await Expect.That(Screen.GetByLabel("File state")).ToHaveTextAsync("a click selects nothing; the menu opens on right-click");
await Expect
.That(Screen.GetByLabel("File state"))
.ToHaveTextAsync("a click selects nothing; the menu opens on right-click");
await row.SecondaryTapAsync();
await Expect.That(Screen.GetByLabel("File state")).ToHaveTextAsync("menu open for report.pdf");
await Screen.GetByRole("menuitem", "Rename").TapAsync();
Expand Down Expand Up @@ -96,7 +98,9 @@ public async Task Every_exercise_done_shows_the_success_message()
await Screen.GetByLabel("Agree to terms").CheckAsync();
await Screen.GetByRole("radio", "Medium").CheckAsync();
await Screen.GetByRole("radio", "Express").CheckAsync();
await Screen.GetByRole("button", "Hold me").LongPressAsync(new LongPressOptions { Duration = TimeSpan.FromMilliseconds(700) });
await Screen
.GetByRole("button", "Hold me")
.LongPressAsync(new LongPressOptions { Duration = TimeSpan.FromMilliseconds(700) });
await Screen.GetByRole("button", "Double-click me").DoubleTapAsync();
await Screen.GetByText("report.pdf").SecondaryTapAsync();
await Screen.GetByRole("menuitem", "Rename").TapAsync();
Expand Down
4 changes: 3 additions & 1 deletion apps/WebBenchmark.E2ETests/DatePicker/DatePickerTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ public async Task Navigates_three_months_ahead_and_books_the_required_date()
await Expect.That(dateInput).ToHaveValueAsync("2026-09-17");

await Screen.GetByRole("button", "Confirm booking").TapAsync();
await Expect.That(Screen.GetByTestId("success-message")).ToHaveTextAsync("Appointment booked for September 17, 2026");
await Expect
.That(Screen.GetByTestId("success-message"))
.ToHaveTextAsync("Appointment booked for September 17, 2026");
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ public async Task One_submit_opens_one_confirm_even_when_double_clicked()
try
{
await Screen.GetByRole("button", "Submit report").DoubleTapAsync();
await Expect.That(Screen.GetByTestId("success-message")).ToHaveTextAsync("Report submitted after deferred confirmation");
await Expect
.That(Screen.GetByTestId("success-message"))
.ToHaveTextAsync("Report submitted after deferred confirmation");
Assert.That(Volatile.Read(ref dialogs), Is.EqualTo(1));
}
finally
Expand Down
21 changes: 18 additions & 3 deletions apps/WebBenchmark.E2ETests/Fixtures/Canvas.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ public static class Canvas
/// <summary>Maps a point in canvas pixels to the viewport through the canvas's current box.</summary>
public static async Task<Point> OnCanvas(Browser browser, Size size, Point point)
{
var box = await browser.Locator("canvas").BoundingBoxAsync() ?? throw new InvalidOperationException("the canvas has no box");
var box = await browser.Locator("canvas").BoundingBoxAsync()
?? throw new InvalidOperationException("the canvas has no box");
return new Point(box.X + point.X * box.Width / size.Width, box.Y + point.Y * box.Height / size.Height);
}

Expand All @@ -35,7 +36,8 @@ public static async Task<int[]> PixelAt(Browser browser, Point point)
return Array.from(context.getImageData(at.x, at.y, 1, 1).data.slice(0, 3));
}
""";
return await browser.EvaluateAsync<int[]>(script, new { x = point.X, y = point.Y }) ?? throw new InvalidOperationException("no pixel");
return await browser.EvaluateAsync<int[]>(script, new { x = point.X, y = point.Y })
?? throw new InvalidOperationException("no pixel");
}

/// <summary>Counts the canvas pixels painted exactly <paramref name="color"/> inside a canvas-space band.</summary>
Expand All @@ -55,6 +57,19 @@ public static async Task<int> CountColor(Browser browser, Band band, int[] color
return count;
}
""";
return await browser.EvaluateAsync<int>(script, new { band = new { x = band.X, y = band.Y, width = band.Width, height = band.Height }, color });
return await browser.EvaluateAsync<int>(
script,
new
{
band = new
{
x = band.X,
y = band.Y,
width = band.Width,
height = band.Height,
},
color,
}
);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ public async Task Sends_the_gift_card()
await Expect.That(Screen.GetByTestId("total-amount")).ToHaveTextAsync("$25.00");
await Screen.GetByPlaceholder("friend@example.com").FillAsync("friend@example.com");
await Screen.GetByTestId("buy-button").TapAsync();
await Expect.That(Screen.GetByTestId("success-message")).ToHaveTextAsync("Gift card sent to friend@example.com");
await Expect
.That(Screen.GetByTestId("success-message"))
.ToHaveTextAsync("Gift card sent to friend@example.com");
await Expect.That(Screen.GetByText("A $25.00 gift card is on its way.")).ToBeVisibleAsync();
}
}
5 changes: 4 additions & 1 deletion apps/WebBenchmark.E2ETests/ImageOnlyUi/ImageOnlyUiTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ public async Task OpenTheImageOnlyUi()
public async Task The_icons_have_no_names_so_the_bare_svg_locator_is_ambiguous()
{
var error = await Support.FailureAsync(() => Browser.Locator("svg").TapAsync());
Assert.That(error, Is.InstanceOf<E2EException>().And.Property(nameof(E2EException.Code)).EqualTo("LOCATOR_AMBIGUOUS"));
Assert.That(
error,
Is.InstanceOf<E2EException>().And.Property(nameof(E2EException.Code)).EqualTo("LOCATOR_AMBIGUOUS")
);
}

[Test]
Expand Down
12 changes: 7 additions & 5 deletions apps/WebBenchmark.E2ETests/KanbanBoard/KanbanBoardTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@ public async Task Pointer_drags_move_both_cards_into_the_goal_layout()
[Test]
public async Task A_swipe_along_a_path_drags_a_card_as_well()
{
await Screen.SwipeAsync(new SwipePathOptions
{
From = await Support.CenterOfAsync(Screen.GetByTestId("card-fix-payment-bug")),
To = await Support.CenterOfAsync(Screen.GetByTestId("column-done")),
});
await Screen.SwipeAsync(
new SwipePathOptions
{
From = await Support.CenterOfAsync(Screen.GetByTestId("card-fix-payment-bug")),
To = await Support.CenterOfAsync(Screen.GetByTestId("column-done")),
}
);
await Expect.That(Screen.GetByTestId("column-done").GetByText("Fix payment bug")).ToBeVisibleAsync();
}

Expand Down
4 changes: 3 additions & 1 deletion apps/WebBenchmark.E2ETests/LoginForm/LoginFormTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ public async Task Rejects_a_short_password()
await Screen.GetByPlaceholder("Email").FillAsync(_account.Username);
await Screen.GetByPlaceholder("Password").FillAsync("short");
await Screen.GetByRole("button", "Log in").TapAsync();
await Expect.That(Screen.GetByTestId("error-message")).ToHaveTextAsync("Password must be at least 8 characters");
await Expect
.That(Screen.GetByTestId("error-message"))
.ToHaveTextAsync("Password must be at least 8 characters");
}

[Test]
Expand Down
8 changes: 6 additions & 2 deletions apps/WebBenchmark.E2ETests/LyingLabels/LyingLabelsTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,10 @@ public async Task The_button_whose_accessible_name_says_Pay_now_is_the_wrong_one
public async Task A_text_match_for_Pay_now_catches_the_decoy_phrase_too()
{
var error = await Support.FailureAsync(() => Screen.GetByText(PayNow).TapAsync());
Assert.That(error, Is.InstanceOf<E2EException>().And.Property(nameof(E2EException.Code)).EqualTo("LOCATOR_AMBIGUOUS"));
Assert.That(
error,
Is.InstanceOf<E2EException>().And.Property(nameof(E2EException.Code)).EqualTo("LOCATOR_AMBIGUOUS")
);
await Expect.That(Screen.GetByTestId("error-message")).ToBeHiddenAsync();
}

Expand All @@ -36,7 +39,8 @@ public async Task The_field_whose_accessible_name_says_Amount_is_the_wrong_one()
await ButtonReading("Pay now").TapAsync();
await Screen.GetByLabel("Amount").FillAsync("42.50");
await Screen.GetByText("Submit payment").TapAsync();
await Expect.That(Screen.GetByTestId("error-message"))
await Expect
.That(Screen.GetByTestId("error-message"))
.ToHaveTextAsync("Type 42.50 into the field visibly labeled Amount, leave Reference empty");
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ public async Task Answers_the_confirm_prompt_and_alert_chain()
{
await Screen.GetByRole("button", "Place order").TapAsync();
await Expect.That(Screen.GetByTestId("success-message")).ToHaveTextAsync("Order placed with code GIFT-7");
Assert.That(seen, Is.EqualTo(new[] { "Place this order?", "Enter your gift code", "Order placed. Thank you!" }));
Assert.That(
seen,
Is.EqualTo(new[] { "Place this order?", "Enter your gift code", "Order placed. Thank you!" })
);
}
finally
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ public async Task Subscribe_renders_no_feedback_and_logs_a_TypeError_planted_bug
await Expect.That(Screen.GetByTestId("success-message")).Not.ToBeVisibleAsync();
await Expect.That(Screen.GetByTestId("error-message")).Not.ToBeVisibleAsync();
await Expect.That(email).ToHaveValueAsync("maria.novak@example.com");
await Expect.Poll(async () => string.Join('\n', await consoleErrors())).ToMatchAsync(
Does.Contain("Cannot read properties of undefined (reading 'subscribe')"));
await Expect
.Poll(async () => string.Join('\n', await consoleErrors()))
.ToMatchAsync(Does.Contain("Cannot read properties of undefined (reading 'subscribe')"));
}

/// <summary>
Expand All @@ -32,7 +33,8 @@ await Expect.Poll(async () => string.Join('\n', await consoleErrors())).ToMatchA
/// </summary>
private async Task<Func<Task<string[]>>> CaptureConsoleErrorsAsync()
{
await Browser.EvaluateAsync<object>("""
await Browser.EvaluateAsync<object>(
"""
() => {
const errors = [];
window.e2eConsoleErrors = errors;
Expand All @@ -43,7 +45,8 @@ await Browser.EvaluateAsync<object>("""
};
return null;
}
""");
"""
);
return async () => await Browser.EvaluateAsync<string[]>("() => window.e2eConsoleErrors ?? []") ?? [];
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,9 @@ public async Task The_review_step_repeats_the_brief_and_finishes_with_the_PIN()
await CompleteProfileAsync();
await CompletePreferencesAsync();

await Expect.That(Screen.GetByText("Maria Novak · maria.novak@example.com · +48 601 222 333")).ToBeVisibleAsync();
await Expect
.That(Screen.GetByText("Maria Novak · maria.novak@example.com · +48 601 222 333"))
.ToBeVisibleAsync();
await Expect.That(Screen.GetByText("Krucza 12, 00-585 Warsaw, Poland")).ToBeVisibleAsync();
await Expect.That(Screen.GetByText("Born March 14, 1990 · Product designer · 7 years")).ToBeVisibleAsync();
await Expect.That(Screen.GetByText("Plan: Studio")).ToBeVisibleAsync();
Expand Down
4 changes: 3 additions & 1 deletion apps/WebBenchmark.E2ETests/OrderHistory/OrderHistoryTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ public async Task Overview_and_settings_render_their_panels()
await Expect.That(Screen.GetByTestId("overview-panel")).ToBeHiddenAsync();
await Expect.That(Screen.GetByTestId("settings-panel")).ToContainTextAsync("Email: alex@example.com");
await Screen.GetByRole("button", "Overview").TapAsync();
await Expect.That(Screen.GetByTestId("overview-panel")).ToContainTextAsync("You have 3 orders and 1 active subscription.");
await Expect
.That(Screen.GetByTestId("overview-panel"))
.ToContainTextAsync("You have 3 orders and 1 active subscription.");
}

[Test]
Expand Down
Loading