Repository navigation
Conversation
🟢 e2e web: 93 passedAll 93 tests in 42 files
Tested by e2e 0.23.1 · 1m 1s · web · run artifacts |
🟢 e2e web agent: 34 passed35 agent steps · 26 replayed from cache · 27 model calls · 127.6k tokens (88% cached) · $0.0069 All 34 tests in 5 files
Tested by e2e 0.23.1 · 1m 38s · web · run artifacts |
There was a problem hiding this comment.
All reported issues were addressed
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
View guided diff | Turn on auto-fix | Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 688 files
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Requires human review: Auto-approval blocked because this review re-detected 10 unresolved issues already reported by Cubic.
View guided diff | Turn on auto-fix | Re-trigger cubic
🟢 e2e android agent: 1 flaky, 19 passed24 agent steps · 15 replayed from cache · 40 model calls · 328.6k tokens (79% cached) · $0.0260
|
| Test | Agent | Time | |
|---|---|---|---|
| tests-agent/scenarios.e2e.ts · 1 flaky, 19 passed | 24 steps · 40 calls | 5m 55s | |
| 🟢 | act completes Login Form | 1 step | 24.3s |
| 🟢 | act completes Infinite Scroll List | 1 step | 20.1s |
| 🟢 | act completes Modal Flow | 1 step | 11.6s |
| 🟢 | act completes Bottom Tabs | 1 step | 7.7s |
| 🟢 | act completes Text Input Variations | 1 step | 7.1s |
| 🟢 | act completes Broken Accessibility | 2 steps · 5 calls | 13.8s |
| 🟢 | act completes Vision Only | 2 steps · 1 call | 6.6s |
| act completes Gestures (1 failed attempt first) | 1 step · 9 calls | 29.1s | |
| 🟢 | act completes Async States | 1 step · 13 calls | 33.8s |
| 🟢 | act completes Debounced Search | 1 step | 6.5s |
| 🟢 | act completes Huge Virtualized List | 1 step | 1m 49s |
| 🟢 | act completes Flattened Registration Form | 2 steps · 1 call | 8.3s |
| 🟢 | act completes Flattened Login | 2 steps · 6 calls | 15.1s |
| 🟢 | act completes Sticky Chrome Target | 1 step | 8.6s |
| 🟢 | act completes WebView Accessibility | 1 step · 5 calls | 13.6s |
| 🟢 | act completes Permission Prompt | 1 step | 6.1s |
| 🟢 | act completes Product Catalog | 1 step | 7.9s |
| 🟢 | act completes Error Recovery | 1 step | 8.7s |
| 🟢 | act completes Choice Controls | 1 step | 8.0s |
| 🟢 | act completes Sequential Onboarding | 1 step | 9.3s |
Tested by e2e 0.23.1 · 6m 36s · android-emulator · run artifacts
🟢 e2e ios agent: 21 passed, 1 skipped25 agent steps · 12 replayed from cache · 55 model calls · 342.4k tokens (82% cached) · $0.0227 All 22 tests in 1 file
Tested by e2e 0.23.1 · 8m 28s · ios-simulator · run artifacts |
There was a problem hiding this comment.
All reported issues were addressed across 21 files (changes from recent commits).
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Turn on auto-fix | Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 18 files (changes from recent commits).
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Turn on auto-fix | Re-trigger cubic
🟢 e2e android: 58 passed, 1 skippedAll 59 tests in 7 files
Tested by e2e 0.23.1 · 8m 34s · android-emulator · run artifacts |
🟢 e2e ios: 56 passed, 1 skippedAll 57 tests in 7 files
Tested by e2e 0.23.1 · 6m 39s · ios-simulator · run artifacts |
|
[claude-opus-5-5] responding on behalf of kdy1 Left for a human:
|
a6e56df to
ddc9313
Compare
|
[claude-opus-5-5] responding on behalf of kdy1 This PR is now layer 3 of a stack: #1736 → #1737 → #1734. It supersedes my earlier handoff comment here:
|
c34adb0 to
4241361
Compare
500cd14 to
8efa596
Compare
@e2e-dev/core becomes usable on its own. launch(engine) opens a core attempt on the standalone worker (nothing published to globalThis, no process secret registry, no trace); sessions save and restore session-state-1 state, AES-256-GCM encrypted under a caller's key, and take secrets from launch(). createAgent from @e2e-dev/core/agent adds the agent with a replay cache keyed by cacheKey and a guard the dispatcher checks before every action, replayed ones included. @e2e-dev/web/engine and @e2e-dev/mobile/engine export the factories without test, so the engines' e2e peer becomes optional. Device targets take app.linkBase, opened through session.openLink. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main changed AgentDeviceSurface.inputBudget to take the operation's signal and timeout instead of a number of milliseconds; the openLink hook added in this branch still passed operation.timeoutMs, which no longer typechecks after the rebase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8efa596 to
4dca8a7
Compare
…, sessions, and the agent cache
Guard
- Reads the screen the turn's earlier actions left before it names what an
action lands on, so a batched turn cannot walk past a stop rule.
- A `key` rule matches the key under any modifier, and typed text holding a
line break counts as Enter. A misspelled `key` or `actions` name is
INVALID_ARGUMENT instead of a rule that never matches.
- A stop that `check` decided is asked again when its recording replays.
Sessions
- A caught agent failure drops only the recordings it found unconfirmed, and
a close before a check evicts nothing.
- `launch({ timeout })` and `setDefaultTimeout` bound app opens and restarts.
- SIGHUP ends the program though the engine listens for it.
- Thrown errors are classified and redacted; a closed session says so.
- A fixture may not take the name of a session method.
- A recording's key no longer depends on the working directory.
- A session keeps its newest 500 step records.
Types
- `createAgent` takes a `Session`, and options held in a variable give an
agent whose `act` takes either form.
- The agent entry exports the executor, store, and agent config types.
- New reference page: launch options, host, session, saved state, createAgent, cache, guard, and the act result. - Guide: Node 22 form without `await using`, the state flow closes the session first, what secrets are redacted from, error handling, default timeouts, and the install line for the agent. - The checkout example no longer checks an error code a program never sees.
A `unique symbol` is unique to the declaration file that declares it. A project that updates `e2e` or an engine alone holds two copies of @e2e-dev/core, and `engine: web()` then failed to typecheck against the runner's `EngineHandle`, as did `expect(locator)` and `expect(browser)`. The runtime keys stay the registered symbols, which every copy shares.
Removes guard, ActResult.status and stoppedAt, and the recorded stop from the trace. The agent loop and trace format are back to main's. Kept for a follow-up.
Stack 3/3: session-half split → move into
@e2e-dev/core→ this PR.@e2e-dev/coredrives a browser or a native app from plain code. It is the engines, locators, agent, and replay cache, with no test runner, config file, or report.e2eis now built on it.Engines without
e2e@e2e-dev/web/engineand@e2e-dev/mobile/engineexport the factories and their types withouttest, so a program never loadse2e.e2epeer becomes optional, needed only by the main entry that extendstest.@e2e-dev/coregains its@e2e-dev/coreand@e2e-dev/core/agententries.@e2e-dev/core/agentalso exports whate2e/agentdoes.What
@e2e-dev/coreaddslaunch(engine, options)returns a host, andhost.newSession()returns an isolated session withscreen,app,platform, and the engine's fixture.EngineHandle<Fixtures>makeslaunch(web())typesession.browser.defineEngine<Fixtures>()requires a factory infixturesfor each fixture it names (besidesscreen,app,platform), returning that fixture, and a union of fixture sets needs one member's factories.globalThisand has no session deadline.setDefaultTimeout.launchandnewSessiontake asignal, and the host closes on Ctrl-C.session.saveState()returns an in-memorySessionState.state.export({ key }), encrypted with AES-256-GCM as a newsession-state-1document (schema and fixtures inpackages/e2e/schema).launch({ secrets })and get handles fromhost.secret(name). There is no process-wide registry.createAgent(session, options)in@e2e-dev/core/agent.cache, everyactnames its recording withcacheKey, enforced in types and at runtime.app.linkBaseplus a new optional engine hooksession.openLink(protocol methodsession/openLink) makeapp.open(path)open a link in a device app.@e2e-dev/mobileimplements the hook.Docs: a new
docs/core.mdx, plus updates to the engine, config, errors, security, and agent reference pages.Verified
Ran it locally: yes
@e2e-dev/corewithoute2e: a scratch project installed only the packed@e2e-dev/coreand@e2e-dev/web, andnode_modulesheld noe2e. It ran a script on real Chromium: a login page with ahost.secret()fill,saveState(),export({ key })andSessionState.import, then a new session restored from that state.Install smoke:
scripts/install-smoke.tsnow packs@e2e-dev/coretoo and fails on a second copy of it. It passes with pnpm 11 and pnpm 12, a single@e2e-dev/corecopy after both the fresh and the frozen-lockfile install.Built CLI: testbed default suite 588 passed. Web benchmark
tests/93 passed.Agentic benchmark (
CI=1 test:agent --strict-cache, no model key): the branch matches main, so no cache key moved. The 8 failures on both are steps that need a model call.Review: a fresh-context review (GPT-6.1 Sol) found three secret-handling gaps in session state, all fixed with regression tests that fail without the fix:
saveState()errors bypassed the sanitized error boundary;SESSION_INVALID.Fixes from cubic, each with a test that fails without it:
openLinkuses the input budget, so a timed-out open isACTION_MAY_HAVE_COMMITTED;linkBaserefuses an empty?or#and surrounding whitespace;presskeys);setDefaultTimeout;isRefusedLinkSchemeis shared from@e2e-dev/core/enginewith the mobile engine.defineEngine<Fixtures>typing (added after the split, maintainer's choice on the cubic thread):tests/types/sdk-types.tsasserts that a missing factory, a factory returning the wrong type, and a union with no member's factories fail to compile, and that narrowingappneeds none. Each assertion fails without the change.web()andmobile()compile unchanged.Since the split: this PR's code is the head reviewed and verified above, plus that typing change and the changesets split across the stack. A second fresh-context review (GPT-6.1 Sol) of the whole stack found the union bypass, fixed here.
Left for later
guard(stop,deny,check) andActResult.statuswere removed from this PR in85af095ab. They can come back as their own PR.@e2e-dev/core's tests still live inpackages/e2e/testsand import it through@e2e-dev/core/lib/*.@e2e-dev/kernel,eas,testmu, and others) still peer one2e, so they don't yet work with@e2e-dev/corealone.@e2e-dev/core'stypes.tsstill declares the test API types. They are type-only and add nothing at runtime.session-state-1copies session-1's 64 MiB cap without enforcing it on export, as session-1 does not; one rule for both formats would be its own change.🤖 Generated with Claude Code