Skip to content

Run the hung-resolve contract test on a fake clock (#1006) - #1533

Merged
kblok merged 2 commits into
mainfrom
fix/1006-fake-clock-deadline-tests
Nov 10, 2026
Merged

kblok merged 2 commits into
mainfrom
fix/1006-fake-clock-deadline-tests

Conversation

@kblok

@kblok kblok commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Closes #1006

Part of #929. Most of #1006 already landed on main (#1049 moved the late-match resolve and late screenshot tests onto a fake clock, and the expect, wait-for, and mobile observation tests already use one). This PR moves the last real-clock deadline race in the LocatorEngine contract tests onto the fake clock.

What changed

  • tests/E2E.Tests/LocatorEngine/SlowLocate.cs: new test helper HangAsync(FakeTimeProvider clock, TimeSpan budget). It moves the clock by the whole budget and ends with OPERATION_TIMEOUT, so a hang costs no real time. Port-only test helper, no upstream symbol (test-only).
  • tests/E2E.Tests/LocatorEngine/LocatorEngineResolveRetryContractTests.cs: Keeps_a_resolve_that_hung_with_budget_to_spare_a_timeout_never_a_missing_node now runs on a SteppingTime clock, not a real 2 s deadline with real Task.Delay waits. Test name and assertions (ACTION_FAILED, two resolves) are unchanged.

Upstream parity

  • Upstream change: none ported. This is a repo-local flake fix. The reference is upstream packages/e2e/tests/unit/locator-engine.test.ts, test keeps a resolve that hung with budget to spare a timeout, never a missing node.
  • Deviation: upstream does not use fake timers for this test. It uses a real sleep(100) and a never-resolving promise against a real Deadline(2_000). The port keeps the test name and assertions and replaces only the timing mechanism with a fake clock, because the real-clock version was the flake. Reported here as a justified exception.
  • Counts: ported 1 (name and assertions), partial 0, missing 0, port-only 1 test helper (SlowLocate.HangAsync(FakeTimeProvider, ...)), reason: lets the test run without real waits.
  • Repo rules: CLAUDE.md parity rules concern src/ symbols and test layout. This change touches no src/ file and adds no test, so no symbol mapping or layout change applies.
  • Verdict: complete for names and assertions. Mechanism deviates from upstream as stated above.

Verification

  • dotnet format --verify-no-changes on the two files: clean (LF, matching .editorconfig).
  • dotnet test tests/E2E.Tests --filter "FullyQualifiedName~LocatorEngine|FullyQualifiedName~LocatorWaitFor|FullyQualifiedName~ExpectPoll": 106 passed, 0 failed.
  • Not run: the full RealModel suite. This change is test-only and does not touch the agent, prompts, cache, replay, or the page script, so CLAUDE.md does not require it.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Improved coverage for locator resolution when a locate operation exceeds its time budget, verifying that it reports a failure and stops after the expected number of attempts.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 39 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 16d693e and cb4ae49.

Files selected for processing: 2

  • src/E2E/Locator/LocatorEngine.cs
  • tests/E2E.Tests/SteppingTime.cs

No actionable comments were generated in the recent review.

Walkthrough

The hung-resolve contract test now uses a shared fake clock for its locate steps and screen. A new SlowLocate.HangAsync overload advances that clock by the full budget and returns a non-retryable OPERATION_TIMEOUT. The test still checks for ACTION_FAILED and exactly two locate calls.

Changes: Locator timeout test

Layer / File(s) Summary
Fake-clock hung-resolve contract
tests/E2E.Tests/LocatorEngine/SlowLocate.cs, tests/E2E.Tests/LocatorEngine/LocatorEngineResolveRetryContractTests.cs
SlowLocate adds a fake-clock overload that advances the full budget and returns a non-retryable timeout. The hung-resolve test uses the shared clock while awaiting resolve and retains its error and call-count assertions.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other · Severity of issue fixed: Low

Possibly related issues

  • Issue 929 tracks real-clock deadline failures across tests. This PR fixes the hung-resolve test listed in that tracker, but not its other failures.

Possibly related PRs

  • hardkoded/e2e-dotnet#893: Adds locator operation-budget behavior and the hung-resolve contract coverage exercised by this test.
  • hardkoded/e2e-dotnet#956: Converts another test in the same fixture to fake-clock timing and adds TimeProvider support to LocatingScreen.
  • hardkoded/e2e-dotnet#884: Maps locator OPERATION_TIMEOUT failures to ACTION_FAILED, which this test asserts.
Merge Risk: Minimal · previous review 16d69

The test removes the real-time deadline race without changing production behavior. No merge-blocking issue remains.

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

Failed Failed checks: 2 inconclusive

Check name Status Explanation Resolution
Linked Issues check Inconclusive For #1006, the diff moves the hung-resolve test to SteppingTime and adds SlowLocate.HangAsync(FakeTimeProvider, TimeSpan). It preserves the ACTION_FAILED and two-resolve assertions. The inspecte… Evidence of the replay, dispatch, readiness-probe, and other remaining #1006 test implementations at the reviewed head is needed to decide whether the full linked issue's coding requirements are satisfied.
Upstream Parity Inconclusive The PR changes one test and adds a test-only helper. The PR identifies the helper as .NET-only and gives a reason. COMPATIBILITY.md identifies LocatorEngine/ as a port of the resolve retry describ… Provide the upstream packages/e2e/tests/unit/locator-engine.test.ts source so the test, assertions, and describe layout can be compared.
Passed Passed checks: 3 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change: running the hung-resolve contract test with a fake clock.
Out of Scope Changes check Passed The only changes are a fake-clock test helper and its use in the LocatorEngine hung-resolve contract test. Both directly support #1006's goal of removing real-clock deadline races.
Full details: Linked Issues check

Explanation

For #1006, the diff moves the hung-resolve test to SteppingTime and adds SlowLocate.HangAsync(FakeTimeProvider, TimeSpan). It preserves the ACTION_FAILED and two-resolve assertions. The inspected resolve-retry and perform contract tests also use fake clocks for their listed deadline races. However, #1006 also covers replay, dispatch, readiness-probe, and other test areas. The available evidence does not establish whether those requirements are met at the reviewed head.

Resolution

Evidence of the replay, dispatch, readiness-probe, and other remaining #1006 test implementations at the reviewed head is needed to decide whether the full linked issue's coding requirements are satisfied.

Full details: Upstream Parity

Explanation

The PR changes one test and adds a test-only helper. The PR identifies the helper as .NET-only and gives a reason. COMPATIBILITY.md identifies LocatorEngine/ as a port of the resolve retry describe. The diff preserves the test name and assertions. Upstream source is unavailable, so the upstream test's assertions and layout could not be verified.

Resolution

Provide the upstream packages/e2e/tests/unit/locator-engine.test.ts source so the test, assertions, and describe layout can be compared.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Linked repositories: Public OSS repositories can only analyze public repositories installed in this organization. No linked repositories were analyzed; skipped tester-army/e2e.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between a8e3861 and 16d693e.

Files selected for processing: 2

  • tests/E2E.Tests/LocatorEngine/LocatorEngineResolveRetryContractTests.cs
  • tests/E2E.Tests/LocatorEngine/SlowLocate.cs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

… fake-clock pump (Refs #1006)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@kblok
kblok merged commit 0c64be0 into main Nov 10, 2026
4 checks passed
@kblok
kblok deleted the fix/1006-fake-clock-deadline-tests branch November 10, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky tests: real-clock deadlines in locator, expect, replay, screenshot and mobile locate tests

1 participant