The full end-to-end suite for automatic approval, and notify only when the sign-in needs you

[popen2]

Supersedes #89 — its commit is the first of the four here, so merging this delivers everything and #89 can be closed. (If #89 goes in first, this shrinks to the three new commits on its own.)

Eight scenarios, all driving the real refresh() against a stubbed AWS SSO, reported by CI as their own check. Building them turned up a behavior gap, which is one of the commits.

Notify mode was interrupting you about work it was going to do without you

Notify mode is a promise not to put a login page in front of you unannounced, and it kept that promise by announcing every refresh: notification, wait for the hotkey, then the login window. With automatic approval that is backwards — most refreshes now ask you for nothing, so notify mode was interrupting you once a working day about work Frost was about to do silently.

The notification moves to the moment a refresh turns out to need a person. Frost works through the approval off screen; if the page asks for a password, a one-time code or a security key, notify mode says so and waits for the hotkey or a click before showing anything. A refresh your identity provider session covers passes in silence. With automatic approval off, nothing changes — a login page is still about to open, so it is still announced first.

Two details that go with it: the pending trigger is cancelled when the run ends (otherwise hasPendingAuth() keeps saying yes and swallows your next hotkey press), and the hand-over is once-only for every surface, so two things noticing the page needs you cannot raise two notifications.

The suite

Scenario	What must be true
Portal session live	Token collected, no window ever shown, no notification
AWS redirects to the IdP, session live there	Same — the hop happens and still nothing is shown
AWS redirects to the IdP, it wants a password	Window shown; after the check signs in on the page, the driver finishes the approval
The same in notify mode	Notification first, nothing shown until the trigger fires; then the window, then the token
The same in default-browser mode	openExternal gets the verification URL, no window shown
Automatic approval off	Window visible from the start, nothing driven
IdP page with an "Allow access" button	Never clicked — the label it wants, on a host that is not ours
AWS page with nothing recognisable	Nothing clicked, including a refusal wearing cli_login_button's id; window comes up

Bold rows are new. The IdP redirect is a real cross-origin 302 to another origin — and another renderer process — not a same-origin fake.

What makes it possible

Three interceptions, none of which asks the app to know it is under test:

• AWS_ENDPOINT_URL_SSO_OIDC / AWS_ENDPOINT_URL_SSO — an AWS SDK feature, so the device authorization, the polling and its AuthorizationPendingException are the real client speaking a real protocol. The token only becomes redeemable when the stub's approval page is actually fetched, so nothing passes without a real click.
• session.protocol.handle("https", …) — pages served at their real names, so the renderer gets https://d-1234567890.awsapps.com, a secure context, and a genuine redirect to https://idp.example.test. Served from localhost this would prove nothing; the host rule is the point.
• Notification.prototype.show and shell.openExternal are recorded, not performed — "what was the user told" and "where were they sent" are the assertions, and a CI runner has neither a notification daemon nor a browser. Notification is a non-configurable export, hence the prototype.

HOME and the electron-store move to a temp directory, so a run touches nothing of the developer's.

It fails when the feature breaks

Each mutation fails exactly one scenario, which is what makes the suite worth having:

Mutation	Fails
isApprovalHost → false	silent approval, federated session
isApprovalHost → true	identity provider (clicks its "Allow access")
isDenial → false	unrecognised page (clicks the id trap)
skip the notify branch of the hand-over	notify mode (window shown without asking)

In CI, under its own name

Both checks that boot the real app were steps inside the job called 🧹 Lint app — check:overlay already was, and this suite was appended beside it. A pull request's check list therefore said nothing about either: the only line it showed claimed the work was linting.

They are their own job now, 🧪 End-to-end checks, so the check list answers "did the end-to-end suite run, and did it pass" without opening a log. It costs a second npm ci and build, which run in parallel with lint.

CI timings match local ones to within a few hundred milliseconds, so the waits are not tuned so tightly that a slower runner would flake:

  ok   silent approval: token collected, nothing shown (1.3s)
  ok   federated session: the identity provider hop stays silent too (1.6s)
  ok   sign-in needed: window shown, approval finished afterwards (1.1s)
  ok   notify mode: notified first, shown only on the go-ahead (1.1s)
  ok   default browser: handed over there, no window shown (6.1s)
  ok   setting off: window shown from the start, nothing driven (3.1s)
  ok   identity provider: its buttons are never clicked (1.1s)
  ok   unrecognised page: nothing clicked, not even the id trap, window shown (13.1s)
auto-approve end-to-end check OK (8 scenarios)

Still not covered

A real desktop — no security key, no keychain, no dock — and the real AWS pages, so the id and label matching against what AWS serves today still wants a smoke test on a real machine.

🤖 Generated with Claude Code

https://claude.ai/code/session_018XK1mi3rSPLzxu7XUhyC6a