feat: explore and test the app without a live Gateway

[shanselman]

Related: #1437. The separate product fix is #1438 (fix(chat): restore the tail when switching conversations), currently a draft.

What Problem This Solves

Developers need to test the populated desktop application without provisioning a real Gateway, pairing a machine, or paying for AI calls.

User Impact

Developers can launch the real app against a fictional Gateway, browse several conversations and settings pages, and run repeatable native UI smoke tests. Each run owns its profile, credentials, ports and processes, so installed pairings are not imported.

There is no change to normal installed-app behavior unless the explicit fixture mode is enabled.

Known limitation: the strict Release session-tail test can fail on existing chat behavior tracked in #1437. This PR preserves that failure rather than weakening the test or changing production scrolling.

Why This Change Was Made

This adds the missing middle tier between unit/component tests and full system E2E. The fixture speaks the normal Gateway WebSocket protocol; the app still uses its real connection manager, chat provider, history loader and renderer.

The same synthetic scenario supports interactive exploration and automation: five sessions, 240 mixed-height messages, an empty conversation, model/agent data and read-only Gateway configuration. MCP handles discovery, readiness and navigation; UI Automation operates the actual session picker and scrollbar.

The changes stay focused: test-only server/run infrastructure, narrow WSL/autostart/toast isolation guards, and passive fixture-only render metadata. No production scrolling workaround, provider replacement, new AI dependency, or Gateway installation.

CI boundary: existing unit-test jobs pick up the new tests in their projects. The real-app fixture smoke remains opt-in. This PR does not add a required desktop CI check, edit workflows, or change branch protection. Mandatory CI promotion is a separate decision after the known runtime failure and runner reliability are addressed.

Evidence

• 104 new test cases cover protocol parsing, parameter/error handling, isolation, concurrent apps, native session switching, delayed history, empty state and actual final-message visibility.
• Two simultaneous app instances were verified with independent profiles, Gateway endpoints, MCP ports/tokens and preferences. Stopping one left the other usable.
• The interactive launcher was exercised with a bounded lifetime, live winnode --list-tools and app.status; it exited successfully and removed its owned profile and process.
• The delayed-history test waits for metadata from the snapshot consumed by the actual composer render. A provider/MCP snapshot alone is not treated as UI proof.
• The harness already caught a regression: a Release repeat returned to messages 1-3 instead of message 240 after A -> B -> A. The strict assertion and failure evidence remain intact. Earlier successful runs do not replace that failed result; Chat can return to the first message instead of the tail when switching back to a populated session #1437 tracks the separate investigation.

Usage: docs/GATEWAY_FIXTURE_TESTING.md.

.\scripts\run-gateway-fixture.ps1 -AppPath '<built OpenClaw.Tray.WinUI.exe>'
.\scripts\test-gateway-fixture.ps1 -AppPath '<built OpenClaw.Tray.WinUI.exe>' -Screenshots

Change Type

• Bug fix
• Feature
• Refactor
• Docs or instructions
• Tests or validation
• Security hardening
• Chore or infrastructure

Scope

• Tray or WinUI UX
• Windows node capability
• Local MCP or winnode
• Gateway, connection, or pairing
• Setup or onboarding
• Permissions, privacy, or security
• Tests, CI, or docs

Required proof pools

• windows-winui-interactive: populated native chat, actual picker/scrolling/navigation, isolated startup and MCP proof. Windows x64 proof collected; the known Release scrolling failure is retained.
• windows-11-arm64: the launcher has an ARM64 path, but native ARM64 execution was not available on this host.

Validation

Windows x64. Required tests use an isolated settings directory and explicit repository root.

Command	Result
.\build.ps1	Passed, including documentation validation
dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restore	4,034 passed, 32 skipped
dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore	3,006 passed, no skips
dotnet test .\tests\OpenClaw.WinNode.Cli.Tests\OpenClaw.WinNode.Cli.Tests.csproj --no-restore	127 passed, no skips
Shared tests filtered to GatewayFixtureProtocolTests	29 passed, no skips
Tray Integration tests filtered to GatewayFixtureProfileTests and GatewayFixtureAppTests (--no-restore -r win-x64 -p:Platform=x64)	22 passed on committed publication head, no skips; real isolated apps and MCP, no UI navigation
.\scripts\test-gateway-fixture.ps1 -AppPath '<Debug app>' -Screenshots	26 passed, no skips
.\build.ps1 -Project WinUI -Configuration Release	Passed
.\scripts\test-gateway-fixture.ps1 -AppPath '<Release app>' -Configuration Release -Screenshots	Latest repeat: 25 passed, 1 failed natural-tail assertion; no skips
.\scripts\validate-docs.ps1 and git diff --check	Passed

Structured Codex reviews completed cleanly with no accepted/actionable findings:

python .\.agents\skills\autoreview\scripts\autoreview --mode commit --commit 7def9212 --engine codex
python .\.agents\skills\autoreview\scripts\autoreview --mode branch --base 7def9212 --engine codex

The implementation and complete test/documentation range were reviewed separately to respect the helper's per-diff size limit, without truncating or omitting files. The final frozen-tree runs both exited 0. Full build and required suites were rerun on publication head afc3e9f7e616bde264ed78490e6738d0c1bbbd44.

Real behavior proof

• Environment tested: real unpackaged Windows x64 app, .NET 10.0.12, Windows App SDK 2.4.0, native Reactor chat. Release uses DevBuild=false, System.Diagnostics.Debugger.IsSupported=false, and no attached debugger.
• PR head or commit tested: afc3e9f7e616bde264ed78490e6738d0c1bbbd44. Full build/required tests, 22 real app/profile/MCP checks and final source review ran on this committed tree. The native UI evidence was collected from the identical production source in implementation commit 7def9212; subsequent commits add tests/documentation and equivalent test-only cancellation argument formatting.
• Exact steps: start an isolated fixture; discover MCP tools; verify the connected operator and five sessions; use the real picker for A/B/A; check natural message-240 visibility before any manual scrolling; exercise two measured window widths, page navigation, an empty session and controlled delayed history.
• Observed result: Debug completed all 26 cases. Earlier Release runs completed all 26; the latest Release repeat completed 25 and retained one genuine failed assertion with a screenshot showing the selected long conversation at its beginning. Required unit suites pass. The product failure is not claimed fixed by this PR.
• Interactive proof: run-gateway-fixture.ps1 -DurationSeconds 90 exited 0; the generated profile and app process were both gone afterward. The same running app returned:

{
  "connectionStatus": "Connected",
  "overallState": "Ready",
  "operatorState": "Connected",
  "nodeState": "Disabled",
  "gatewayVersion": "fixture-1",
  "sessionCount": 5,
  "nodeCount": 0,
  "operatorScopes": ["operator.read"]
}

• Screenshot or artifact links verified: N/A for hosted images at creation; copied live MCP output is inline, and local captures/TRX reports are retained.
• Not verified / blocked: the intermittent native scrolling regression; native ARM64; real Gateway pairing, WSL/MXC and installer/MSIX behavior. Fixture evidence is not a substitute for full system E2E. CI promotion, canned streaming, and multiple saved Gateways within one app are not part of this first version.

Security Impact

• New permissions or capabilities? No. No new remote node command or app MCP command is added.
• Secrets or tokens handling changed? Yes, in test infrastructure. Each run generates its own fixture credential and uses the app's isolated MCP token. No installed settings, identity files or credentials are copied.
• New or changed network calls? Yes. A per-run authenticated numeric-loopback WebSocket server supplies synthetic data; existing local MCP drives the app. There is no live proxy or AI/provider fallback.
• Command or tool execution surface changed? Yes, developer tooling. The launcher starts only an explicitly selected built app and rejects binaries without the fixture-isolation capability marker. It does not expose shell execution through the fake Gateway.
• Data access scope changed? Yes, test-owned state. Runs create owned temporary profiles and separate setup/Local AI roots, and clean up only their own process trees/directories. Published artifacts exclude profile/registry/identity files.

The explicit fixture guards suppress WSL keepalive/stale cleanup, Windows autostart writes and toast registration. Auto-start changes through local MCP also fail explicitly before saving. Normal isolated runs without the fixture flag retain their existing behavior. The supported workflow is browsing and safe local preferences, not a general OS sandbox.

Compatibility and Migration

• Backward compatible? Yes. Existing MCP-only integration defaults and normal credential precedence are unchanged.
• Config or environment changes? Yes, opt-in developer controls. The launcher supplies OPENCLAW_GATEWAY_FIXTURE=1, isolated tray/setup roots, a dedicated MCP port, and safe fixture settings in the child process only. The app advertises OpenClaw.GatewayFixtureIsolationVersion=1 for preflight.
• Migration needed? No. No user settings or pairing migration and no installer changes.

Review Conversations

• I replied to or resolved every bot review conversation addressed by this PR.
• I left unresolved only conversations that still need maintainer judgment.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed September 17, 2026, 1:56 PM ET / 17:56 UTC (Revision 2).

ClawSweeper review

What this changes

Adds a disposable simulated Gateway, launch scripts, native application tests, and explicit isolation guards so developers can explore populated conversations without a live Gateway.

Merge readiness

✅ Ready for maintainer review

Keep open: this is a distinct, useful testing capability absent from current main, with convincing native-app evidence and no concrete introduced defect found. The retained scrolling failure belongs to the separately tracked product fix.

Priority: P2
Reviewed head: afc3e9f7e616bde264ed78490e6738d0c1bbbd44

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	A useful, well-supported harness with focused production guards and no actionable correctness finding.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (live_output): The fixture host drives the real Windows app through its Gateway client and native renderer, supported by inspected screenshots, inline live MCP output, and reported concurrent-instance and cleanup runs. Capture-source equivalence to the reviewed head was verified; the retained Release scrolling failure does not negate proof that this harness works.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (live_output): The fixture host drives the real Windows app through its Gateway client and native renderer, supported by inspected screenshots, inline live MCP output, and reported concurrent-instance and cleanup runs. Capture-source equivalence to the reviewed head was verified; the retained Release scrolling failure does not negate proof that this harness works.
Evidence reviewed	8 items	Pinned review identity and continuity: GitHub still reports the open, non-draft PR at afc3e9f against 3c43751. The previous completed review examined this same head and retained no findings; the subsequent contributor comment supplies native screenshots. Existing fixture does not provide this capability: The existing TrayAppFixture supplies MCP-only capability testing without populated Gateway histories. Main's WSL keepalive service also lacks the new fixture guard. The main tree search found no equivalent synthetic Gateway launcher; v2026.9.4 points to the supplied main revision. Explicit isolation and preserved ordinary behavior: Fixture mode requires an exact opt-in and valid absolute profile/setup roots. The launcher removes inherited OPENCLAW overrides, generates fresh state, disables sensitive capabilities, and rejects older app binaries. Host-policy tests exercise both fixture refusal and unchanged ordinary isolated-mode autostart behavior.
Findings	None	None.
Security	None	None.

How this fits together

The fixture supplies synthetic Gateway responses to the Windows app's existing connection, history, and rendering stack. Local MCP and Windows UI Automation drive the app and collect assertions and artifacts.

flowchart LR
  A[Explicit built app] --> B[Fixture launcher]
  B --> C[Disposable profile and credentials]
  B --> D[Loopback simulated Gateway]
  C --> E[Real Windows app]
  D --> E
  F[MCP and native UI automation] --> E
  E --> G[Assertions and captured artifacts]

Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Code growth	Production +224 net lines; tests and test infrastructure +3,022 net lines	Most growth implements the reusable test environment; production growth is justified by explicit isolation guards and passive render observation.
Native smoke results	Debug 26/26; latest Release 25/26	The retained Release failure demonstrates the strict assertion and remains separately tracked rather than being reported as passing.

Technical review

Best possible solution:

Keep synthetic data at the Gateway boundary, preserve strict native assertions, and maintain the opt-in harness independently of the scrolling repair.

Do we have a high-confidence way to reproduce the issue?

Not applicable to this feature PR. The inspected native screenshots and reported runs demonstrate the harness; the intermittent scrolling failure is separately tracked.

Is this the best way to solve the issue?

Yes. Supplying synthetic responses through the existing Gateway client exercises more production behavior than substituting a chat provider, while the explicit fixture guards preserve ordinary application behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 3c43751b2bac.

Labels

Label justifications:

• P2: This provides useful, bounded developer testing infrastructure without establishing an urgent installed-user regression.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The fixture host drives the real Windows app through its Gateway client and native renderer, supported by inspected screenshots, inline live MCP output, and reported concurrent-instance and cleanup runs. Capture-source equivalence to the reviewed head was verified; the retained Release scrolling failure does not negate proof that this harness works.
• proof: sufficient: Contributor real behavior proof is sufficient. The fixture host drives the real Windows app through its Gateway client and native renderer, supported by inspected screenshots, inline live MCP output, and reported concurrent-instance and cleanup runs. Capture-source equivalence to the reviewed head was verified; the retained Release scrolling failure does not negate proof that this harness works.

Evidence

What I checked:

• Pinned review identity and continuity: GitHub still reports the open, non-draft PR at afc3e9f against 3c43751. The previous completed review examined this same head and retained no findings; the subsequent contributor comment supplies native screenshots. (afc3e9f7e616)
• Existing fixture does not provide this capability: The existing TrayAppFixture supplies MCP-only capability testing without populated Gateway histories. Main's WSL keepalive service also lacks the new fixture guard. The main tree search found no equivalent synthetic Gateway launcher; v2026.9.4 points to the supplied main revision. (tests/OpenClaw.Tray.IntegrationTests/TrayAppFixture.cs:162, 3c43751b2bac)
• Explicit isolation and preserved ordinary behavior: Fixture mode requires an exact opt-in and valid absolute profile/setup roots. The launcher removes inherited OPENCLAW overrides, generates fresh state, disables sensitive capabilities, and rejects older app binaries. Host-policy tests exercise both fixture refusal and unchanged ordinary isolated-mode autostart behavior. (src/OpenClaw.Shared/GatewayFixtureIsolation.cs:28, afc3e9f7e616)
• Protocol and authority boundary: The test server binds numeric loopback, checks the per-run token before serving synthetic reads, rejects writes, and never forwards requests. Protocol tests exercise the real Gateway client, cross-run token rejection, out-of-order histories, and shutdown. The scenario identifies this repository's Gateway client and protocol snapshot as its contract. (tests/OpenClaw.TestSupport/Gateway/FixtureGatewayServer.cs:298, afc3e9f7e616)
• Native evidence inspected and matched to source: Both downloaded images from feat: explore and test the app without a live Gateway #1439 (comment) were inspected. One visibly shows the connected native app and final message 240; the other shows the selected long conversation at messages 1–3. Production, launcher, fixture-host, and shared fixture sources have no differences between the stated capture commit and reviewed head. (tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs:27, afc3e9f7e616)
• Reported validation and its limits: The captured body reports a passing full build, Shared 4,034 passed/32 skipped, Tray 3,006 passed, CLI 127 passed, 22 real app/profile checks, and Debug smoke 26 passed. Latest Release smoke remains 25 passed/1 failed. Inline live MCP output shows a connected operator, five sessions, and disabled node mode; bounded launcher cleanup was reported successful. Native ARM64 and full-system pairing/MXC/MSIX were explicitly unverified. This read-only review did not execute builds or tests. (afc3e9f7e616)

Likely related people:

• Natalie Aguinaldo: Raw commit bd9ce43 adds src/OpenClaw.Tray.WinUI/Services/AutoStartSettingsApplier.cs:18 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: bd9ce43b4d9c; files: src/OpenClaw.Tray.WinUI/Services/AutoStartSettingsApplier.cs)
• shanselman: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)

• reviewed 2026-09-17T17:49:56.032Z sha afc3e9f :: needs maintainer review before merge. :: none

[shanselman]

Native UI evidence from the fixture harness

Successful Debug example: the real app displays the actual final message of the 240-message synthetic conversation. This is a harness demonstration, not an "after" screenshot for the separate scrolling fix.

Known Release failure retained: after returning to the long conversation, the selected session is correct but the viewport is at messages 1-3. The harness failed its natural-tail assertion rather than repairing the scroll position or reporting success. This is tracked in #1437; the separate draft product fix is #1438 (fix(chat): restore the tail when switching conversations).

Both captures contain only the fictional scenario and the target app window. The PR description preserves the latest Release result: 25 passing cases and one failed assertion.