Skip to content

feat(browser): save values in browser scenarios and send them in later steps - #7162

Closed
earakely-scale wants to merge 28 commits into
morluto:mainfrom
earakely-scale:feat/recorder-saved-values
Closed

earakely-scale wants to merge 28 commits into
morluto:mainfrom
earakely-scale:feat/recorder-saved-values

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

An agent that builds an environment from a prompt records the application with one browser scenario. Until now a
scenario could not carry a value from one step to a later one:

  • No ids from responses. After creating an object, the agent could not open, read, edit or delete it by the id the
    application assigned. Agents chose their own ids where the API allows it, or read lists back instead.
  • No derived auth headers. An API that takes Basic auth, or a header a frontend builds from a login token, could not
    be sent. Miniflux's agent stopped and asked for an API key.

This PR lets steps save values and later steps send them:

  1. A request step saves [{name, json_pointer, secret}] from its JSON response. A click or press step saves from
    the response to the request it triggers, with save_response: {method, path, save}, so a create made through the
    UI yields its id.
  2. {saved:name} sends a saved value in a request path (and its query), a goto URL, and a JSON body, where a string that
    is only the placeholder keeps the saved JSON type. Header values and Basic credential parts take
    {"source": "saved", "name"}.
  3. A header can send Basic credentials, {"source": "basic_credentials", "parts": [username, password]}, each part a
    literal, declared secret, saved value, cookie or local storage entry.
  4. The skill records writes the way the UI does, saving ids from the responses its clicks trigger.
  5. The write model serves UI-made creates as faithfully as API-made ones. Recording through the UI let Grafana generate
    folder uids, which exposed three gaps; this PR closes them (below).

Results, agents building seven applications from the same prompts, with main and with this branch:

  • Served writes go from 9 to 24, and write tasks from 4/20 to 8/20 and 9/20 with the two models. Miniflux goes from no
    environment to 3/3, and Vikunja from 0/5 to 3/5.
  • Rebuilding the branch's own recordings with the write-model fixes changes only Grafana's plans, whose tasks go from
    0/2 to 2/2.
  • The fresh Grafana and EspoCRM recordings scored below main's, for reasons in what was recorded (below).

Problem and expected behavior

  • A step should save a string or number from a JSON response, by name, and a later step should send it in a path, a
    URL, a JSON body or a header.
  • A UI step that submits a form should save from the response to the request it triggers.
  • A header should carry Basic credentials built from any value kind.
  • A scenario that uses none of this should record and hash as before, and earlier REA versions should read captures that
    use it.

Change and scope

  • Schema (browserScenarioValues.ts, browserScenario.ts):
    • save on request steps, save_response on click and press, the saved value source in headers and credential
      parts, basic_credentials with two parts, and {saved:name} (a parsed URL's percent-encoded form included).
    • Validation refuses a saved value no earlier step saves. Other braces, such as {id}, are sent as written.
  • Runtime (PlaywrightScenarioRequest.ts, PlaywrightScenarioActions.ts, BrowserScenarioSecrets.ts):
    • A saving request step reads its response text, up to 4,000,000 characters, in the page.
    • A UI step's save_response starts waiting before the step acts and fails the step unless a matching 2xx JSON
      response arrives before its timeout.
    • Saved values keep their JSON type and fill placeholders percent-encoded in paths and URLs.
    • Basic credentials are base64-encoded and redacted, as are parts read from the page. A value saved with
      secret: true is redacted from later records.
  • Evidence (BrowserScenarioEvidence.ts): an empty save is left out of the recorded step, so parameters and
    scenario_sha256 are unchanged for scenarios that save nothing.
  • Environment build: warns about saved values named like credentials that are not marked secret
    (credential-named-saved-values).
  • Skill (version 50):
    • Writes the way the UI does and saves ids instead of guessing them.
    • Falls back to request steps when the UI cannot reach an operation or coverage withholds the UI-shaped write.
    • Its recording example creates through the UI.
  • Write model (src/domain/analysisView):
    • A composed string that reads a field besides the generated value a template would rebuild it around wins over
      the template, so Grafana's /dashboards/f/{uid}/{slug(title)} is composed when the uid is generated.
    • Later creates check template fields against their answers, as they check composed ones, so coverage no longer
      serves a template a later create contradicts.
    • A slug learns how the application spells &: ampersand: "and" on a slug rule or composed part, learned from the
      call that shows it. Calls that slugged no & agree with either spelling.
    • Where one text could come from two sources (an id equal to the title's slug), the entities the type holds pick the
      one reading they share. When they cannot, a later create that tells the fields apart lends its composed rule to
      the first create, whose body contract is learned again with it.
  • Skill: also asks agents to delete what an earlier attempt created before recording again, since the environment
    holds a leftover as seeded data.
  • Docs and spec: the browser scenario contract's saved values section, the environment and web capture references,
    and docs/specs/agent-environments/scenario-saved-values/.

Not included:

  • general value transforms or script steps;
  • saved values in fill and select_option;
  • multi-factor sign-in;
  • lists sorted by a field an update changes, which REA keeps in place (pre-existing; seen in Grafana's fresh run).

Contract and boundary impact

  • Semantic owner and earliest changed stage: the browser scenario contract (src/domain/browserScenarioValues.ts)
    and its Playwright runtime.
  • Contribution scope fit and linked maintainer decision: within the scenario request steps of feat(environment): build agent environments from a prompt #6946. Part of [Design] Project captures into an environment manifest, serve it over streamable HTTP, export agent-environment bundles #4933.
  • CLI and MCP/tool-catalog contract:
    • capture_browser_scenario accepts save, save_response, the saved source, basic_credentials and
      {saved:name}.
    • The compact catalog grows by 3,664 bytes, from 1,038,975 on main 2b84efe to 1,042,639, under the 1 MiB frame
      outputSchemaProfile.test.ts measures. A first design with path_params and json_values fields failed that test.
    • save_response.path uses the request path's portable pattern, now shared as START_ORIGIN_PATH.
  • Provider, bridge, target-format, or platform compatibility: unchanged.
  • Evidence, artifact, provenance, or reconstruction contract:
    • Scenarios that save nothing record the same parameters and scenario_sha256.
    • Captures that save values parse on earlier REA versions.
    • environment-state views may carry ampersand on slug rules and parts; earlier REA versions refuse views that do,
      as the schemas are strict. Earlier views parse unchanged.
  • Target selection, authority, upstream behavior, and cleanup details:
    • A UI step with save_response waits for a response on the start origin.
    • Request steps stay same-origin, as before.
  • Generated metadata, package, or installation impact: the skill version goes from 49 to 50.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Agent-built environments. Claude Code (claude-sonnet-5-5) built each environment from the same prompt with main
ff6d96b (skill 49) and with this branch, on freshly seeded instances. Each bundle's write tasks ran with
fireworks_ai/gpt-oss-120b and fireworks_ai/glm-5p3-flash, each checking the result from the environment's state.

The branch's recordings, rebuilt before and after the write-model fixes (gpt-oss, GLM). Only Grafana's plans change:

Application Before After
Miniflux 3/3 3/3 3/3 3/3
Paperless 1/3 2/3 1/3 1/3
Vikunja 1/3 1/3 + 0/2 1/3 1/3 + 0/2
Linkding 0/2 0/2 0/2 0/2
Homebox 0/2 0/2 0/2 0/2
Grafana 0/2 0/2 2/2 2/2
EspoCRM 1/3 1/3 1/3 1/3

Paperless's plans are identical in both builds; its GLM difference is the eval model's.

Fresh recordings, main and the final branch:

Application Write tasks Main: served, tasks Branch: served, tasks
Miniflux 3 no bundle 4, 3/3 3/3
Paperless 3 3, 1/3 1/3; 6, 2/3 2/3 6, 1/3 2/3
Vikunja 3 + 2 1, 0/3 0/3 + 0/2 0/2 3, 1/3 1/3 + 2/2 2/2
Linkding 2 0, 0/2 0/2 3, 0/2 0/2
Homebox 2 1, 0/2 0/2 5, 0/2 0/2
Grafana 2 2, 2/2 2/2; 3, 2/2 2/2 1, 1/2 1/2
EspoCRM 3 2, 1/3 1/3 2, 0/3 0/3

"Served" counts the write tools the final capture serves; tasks are gpt-oss then GLM. Paperless and Grafana were
recorded twice on main.

  • Miniflux: main's agent stopped for an API key; with Basic credentials the branch serves category create, rename and
    delete, and every category task passed in all three branch recordings.
  • Saved values in use: every branch agent saved ids, with save_response on UI writes.
  • How the branch got here. The first branch recording showed that EspoCRM's agent copied the skill's request-step
    example and wrote through the API, which EspoCRM answers in a shape the write model withholds (0/3); the example now
    creates through the UI. The second showed Grafana's UI-made creates served with wrong addresses (0/2); the write-model
    fixes above close that.
  • Grafana, fresh: the create is served with the right address and passes. The rename is withheld: Grafana lists
    folders sorted by title, the recording renamed a folder to a title that sorts first, and REA keeps an updated entity
    where it was in a list. Main's REA reads this recording the same way, and would serve its create with the template
    that keeps the first folder's slug. Main's two recordings passed because their titles kept their order.
  • EspoCRM, fresh: the recording started with a contact an earlier attempt created and left, so the environment
    holds it as seeded data and the "nothing else changed" checks fail. The skill now asks agents to delete such objects
    before recording again; that was not measured.
  • Paperless: the two main runs differ as much as main and the branch do.
  • Linkding and Homebox serve more writes, but their create tasks still fail. Linkding's served create requires every
    member its recorded calls sent, so a task sending a URL and a title is refused.
  • Census: write plans for 109 captures (the 50 published captures, 29 earlier agent recordings and the 30 above),
    built by main and by this branch: 524 plans and 247 served on each. Three plans differ, all Grafana folder creates,
    whose url the branch composes from the uid and the title's slug.

Real Chrome, a local application with a cookie-session UI and an API that takes only Basic auth:

  • An 11-step scenario saved an id from a Save click and opened /items/{saved:item1}.
  • It sent Basic credentials from local storage and a secret, and saved a secret token.
  • It created a child with {"name": "Child of {saved:item1}", "ref": "{saved:item1}"}, which was sent as
    {"name":"Child of 101","ref":101}.
  • The password appears nowhere in the capture, and the token header is recorded redacted.

Tests added:

  • schema:
    • a saved value sent before it is saved, in a path, query, goto URL (percent-encoded too), JSON body, header or
      credential part;
    • literal braces sent as written;
    • credential parts from page values, an undeclared secret part, and a wrong part count;
    • save_response paths and an empty save;
  • runtime with a fake page:
    • ids sent in a path and a JSON body with their type, and a saved secret redacted;
    • a missing pointer and a non-JSON response;
    • Basic credentials from local storage and a cookie, encoded and redacted;
    • a goto filled percent-encoded;
    • save_response waiting before the click and matching method, origin and {*} segments, and failing without a
      matching or 2xx response;
  • Evidence: a scenario that saves nothing records and hashes as before; saved values and placeholders are recorded;
  • the credential-named-saved-values warning;
  • write model:
    • a generated uid and the title's slug compose a folder's address;
    • the & spelling is learned from a later create, a dropped & stays dropped and a create spelling it otherwise is
      withheld;
    • an id equal to the title's slug is told apart by a later create, or by held entities;
    • a template a later create answers otherwise is withheld;
    • a copied member a composition rebuilds is accepted with any value, and one a template would keep stays fixed.
      Each fails on main.
  • Review round (each fails on 812f9ff unless noted):
    • Basic credentials from one-letter secrets and a two-character secret save are redacted;
    • a literal Basic password is refused, an empty one accepted;
    • save_response ignores a request already running when the step began, matches My List against My%20List,
      refuses a response whose received size is over the cap without reading it, and stops listening when the step
      fails (passes on 812f9ff, which never listened);
    • a saving request step stops reading after the chunk that passes the cap;
    • a goto keeps a structured query value that spells a placeholder;
    • a slug field keeps a dropped & an accepted create showed, takes and from held tags, and learns no slug rule
      they contradict; a composed address takes and from held folders;
    • many fields sharing one text leave an address unread (passes on 812f9ff, which reached the same result with
      unbounded work).
  • Second review round (each fails on af3d562, the merge of main before it):
    • a saved integer past 2^53 fails the step instead of saving a rounded id;
    • a JSON string spelling %7Bsaved:item%7D needs no save and is sent as written;
    • a held slug neither & spelling gives keeps a create that slugs & from learning a slug rule, and leaves a create
      without & its plain rule.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Fifth review round at 9f5191ab0 (main ca3fd7ba merged), on macOS with Node v24.22.1:

  • 6f6c61613 fixes Greptile's three findings on 22a93fadc: a saved secret is redacted as a URL's query spells it once
    parsed (' as %27); a step's saved-value fills are counted before each is built against a 4,000,000-character budget
    (percent-encoded in a path or goto URL), so repeated references fail the step before it sends; and a create whose
    slug the held & spellings refuse accepts the slugged member only as recorded. Its six tests fail on 22a93fadc;
    9f5191ab0 makes one of them check the plan's body is present first. 3de21ef5d merges main ca3fd7ba.
  • typecheck, check:fast, format:check, knip, docs:check, mcp-catalog:check, verify:test-discovery and
    scan:todos: passed. domain + services + adapters: 18,758 passed, with the same two failures as main. Browser
    boundary tests and webArtifactReadOwnership: 1,223 passed, with the two sazWebCapture failures main has.
  • Census on main ca3fd7ba and this head: 524 plans and 246 served on each (main's anchorless-clock change withholds
    one Vikunja label update on both), with the same three Grafana creates differing.

Fourth review round at 22a93fadc (main dd7a2da5 merged), on macOS with Node v24.22.1:

  • The red CI on 81a275b13 came from main as merged for that run: 662ec2e45 left scripts/verify-apk-signature.mjs
    unformatted (main fixed it in 741f2de75), and webArtifactReadOwnership.test.ts, which passes on this branch,
    failed only in that merge. 723fe02c8 merges main dd7a2da5.
  • 22a93fadc decodes a saving response as fatal UTF-8 in both save paths, so a malformed byte fails the step instead of
    saving U+FFFD, and bounds a secret save's spellings from the value before building any (Codex follow-ups). Its four
    tests fail on 723fe02c8.
  • typecheck, check:fast, format:check, knip, docs:check, mcp-catalog:check, verify:test-discovery and
    scan:todos: passed. domain + services + adapters: 18,673 passed; the Inspector stat test and
    SetupRegistrationPrewarm's detached-descendant test fail, as they do on main dd7a2da5. Browser boundary tests and
    webArtifactReadOwnership: 1,223 passed; two sazWebCapture tests fail, as they do on main.
  • Census on main dd7a2da5 and this head: 524 plans and 247 served on each, with the same three Grafana creates
    differing.

Third review round at 69f38a78b (main a30ffd27 merged), on macOS with Node v24.22.1:

  • 7b2e78077 then checks each filled path segment after substitution, so a saved . that the text beside it turns into a
    dot segment fails the step (Greptile's follow-up); its tests fail on 69f38a78b, check:fast passes and src/browser/
    passes 1,043 tests.
  • a712f52e2 saves no number that is not finite (1e400), and d0d7c248b lets a later update refine an ambiguous slug
    spelling as creates do (Codex follow-ups); their tests fail on 7b2e78077. check:fast, docs:check and 2,852 browser,
    write-model and environment tests pass, and the 109-capture census stays at 524 plans and 247 served.
  • 22ac315c3 keeps an update's word spelling of & for a later name without one and withholds a later create that omits
    a composed or templated field; d39f386d2 caps the saved values a scenario keeps at 4,000,000 characters. Their tests
    fail on d0d7c248b; check:fast, docs:check and 2,859 browser, write-model and environment tests pass, and the census
    is unchanged at 524 plans and 247 served.
  • 059322048 reads a saved response only once its decoded size is bounded (gzip/deflate by DEFLATE's maximum expansion;
    br, zstd or an unknown size fail the step) and counts only the spellings a secret save adds; 81a275b13 saves no number
    JSON parsing changes (1e-400, 1.0000000000000001) and spells an update's & as held entities do. Their tests fail on
    the head each thread was reported on; check:fast, docs:check and 2,880 browser, write-model and environment tests
    pass, and the census is unchanged at 524 plans and 247 served.
  • Fixes the five Codex findings on 98f18cc5: saved-response reads bounded by the step's deadline and abort signal; the
    4,000,000-character cap kept, with a 12,000,000-byte bound before reading; Basic usernames with a colon refused;
    dot-only saved path segments refused; mixed held & spellings leave no slug rule. It also makes the saved-values
    example a whole scenario, as the product catalog test requires. Each new test fails on 98f18cc52.
  • tsc, check:fast, format:check, docs:check, mcp-catalog:check, verify:test-discovery, scan:todos and
    jscpd: passed. knip reports only isToolFamilyId, as main does. domain + services + adapters: 18,201
    passed, with the Inspector stat test failing as on main. Product catalog, output schema profile, MCP schema
    validation, tool schema validity, the environment build and serve CLI, served-writes MCP and analysis-view CLI
    boundaries: 8 files, 75 passed.
  • Census on main a30ffd27 and this head: 524 plans and 247 served on each, with the same three Grafana creates
    differing; none of this round's fixes changes a plan.

Earlier rounds:

Run on macOS with Node v24.22.1 at e42362c (main 2b84efe). Agent recordings ran on Ubuntu 24.04 VMs with snap
Chromium.

  • npm run typecheck, lint:built, format:check, metadata:check, verify:architecture-guards, knip:production,
    lint:typed, docs:check, mcp-catalog:check, verify:test-discovery, scan:todos and jscpd: passed. knip
    reports one unused export, isToolFamilyId in src/contracts/toolFamilies.ts, which this branch does not touch and
    main 2b84efe reports too.
  • npx vitest run --project domain --project services --project adapters: 18,261 passed and 1 failed, "stops Inspector
    target authorization after the current stat settles", which fails the same way on main in this environment.
  • npm run test:focused -- the browser scenario CLI and MCP boundaries, scenario input diagnostics, scenario
    cancellation and cleanup, the analysis-view CLI and MCP boundaries, the environment build, export and serve CLI
    boundaries, the environment serve, served-writes and bundle MCP boundaries, tool schema validity, MCP schema
    validation, and output schema constraints and profile: 17 files, 128 passed.
  • Census on main 2b84efe and this branch: 524 plans and 247 served on each, with the same three Grafana creates
    differing.
  • Scenarios that save nothing: identical parameters and scenario_sha256 on main and on this branch.
  • Main reads the real-Chrome capture: environment-build --dry-run gives the same manifest, state, coverage and file plan
    on both.
  • After the first review round, the real-Chrome scenario, with its save_response path written as /ui/{*}, completed
    all 12 steps with no password or raw token recorded.

Compatibility, safety, and release

  • Breaking changes or migration steps: none for callers.
  • Real Hopper/Ghidra, browser, or OS coverage: Google Chrome on macOS for the local check; snap Chromium on Ubuntu
    24.04 for the agent recordings.
  • Package or release metadata impact: skill version 50.
  • Security, privacy, process, or containment review:
    • Saved values are kept in memory for the scenario only.
    • A value saved with secret: true, Basic credentials, and parts read from the page are redacted from later records.
    • The response a secret was saved from may keep it, since the recorder reads bodies on their own schedule; the
      contract says so.
    • The build warns about credential-named saved values not marked secret.

Author readiness

Human owner: @earakely-scale

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

Part of #4933.

🤖 Generated with Claude Code

earakely-scale and others added 6 commits October 25, 2026 10:05
Request steps can save strings and numbers from their JSON responses and
send them in path placeholders, query, headers and the JSON body; a header
can send Basic credentials built from a username and password. The skill
reference teaches both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- click and press steps take save_response {method, path, save}: the
  recorder waits, before the step acts, for the 2xx JSON response to the
  request it triggers and saves values from it
- {saved:name} sends a saved value in a request path (and its query), a goto
  URL, and a request's JSON body, where a string that is only the
  placeholder keeps the saved JSON type; header values and Basic
  credentials parts take {source: "saved", name}
- Basic credentials take parts: [username, password], each a literal,
  secret, saved, cookie or local storage value, so the schema repeats the
  part union once; path_params and json_values are gone, which keeps the
  compact MCP catalog under its 1 MiB frame
- validation refuses a saved value no earlier step saves; other braces are
  sent as written
- Evidence records an empty save list as absent, so scenarios that save
  nothing record the same parameters and scenario_sha256 as before
- environment-build warns about saved values named like credentials that
  are not marked secret (credential-named-saved-values)
- the skill writes the way the UI does and falls back to request steps;
  skill version 50

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example created its object with a request step, and agents copied it
even where the UI writes the fields the server needs (EspoCRM). It now
fills a form and saves the id from the response the Save click triggers.
A request-step write coverage withholds is recorded through the UI
instead, and Basic credentials are no longer offered as a way around the
UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Outcome, plan, tasks and verification for saved values, save_response
and Basic credentials, with the agent-built comparison against main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5 Tier: plus

[Medium impact] The PR appears safe to merge; no new actionable issue was established.

Summary

Adds saved response values and Basic credentials to browser scenarios, then uses those values in later requests and navigation.

  • Browser scenarios save response values and reuse them in later steps.
  • Write plans rebuild addresses from the values each create sends.
  • Slug rules record when an ampersand becomes the word “and.”

earakely-scale documents two intentional limits: a response that supplies a saved secret may retain it, and earlier REA versions reject environment views containing the new ampersand field.

Review was source-only; no tests were run. Supersession status could not be checked because authenticated GitHub access was unavailable.

Diagram

sequenceDiagram
  participant Step as Scenario step
  participant Browser
  participant Values as Saved values
  participant Records as Capture
  Step->>Browser: Request or UI action
  Browser-->>Step: Matching response
  Step->>Step: Check size, UTF-8, JSON and selected values
  Step->>Values: Save typed values
  Values->>Records: Register secret spellings
  Step->>Values: Resolve later references within step budget
  Step->>Browser: Send filled request or navigation
  Browser->>Records: Record observations with redaction
Loading

Reviews (11) · Last reviewed commit: "test: assert the refused-slug plan has a..." · Reviewed by Greptile

Comment thread src/application/BrowserScenarioEvidence.ts
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/PlaywrightScenarioActions.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
earakely-scale and others added 4 commits October 25, 2026 16:03
…templates on later creates

Recording through the UI let Grafana generate folder uids, and three write
model gaps then served folders whose address kept the first folder's slug:

- a template around a generated value won over a composed string that also
  reads the title's slug; the composed string now wins when it reads a field
  besides the template's value
- later creates checked composed strings against their own answers, but not
  templates, so coverage served a template a later folder contradicted; a
  template is now checked the same way
- a slug dropped `&`, which Grafana's Go slugs spell as the word; a slug rule
  or part learns `ampersand: "and"` from the call that shows it, and calls
  that slugged no `&` agree with either spelling

An address whose uid equals its title's slug could come from either field.
The entities the type holds now pick the one reading they share, and when
they cannot, a later create that tells the fields apart lends its composed
rule to the first create, whose body contract is learned again with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o clear earlier attempts

The skill asks agents to delete what an earlier attempt created before
recording again, since the environment holds a leftover as seeded data
(EspoCRM's fresh run). The spec and verification record the write-model
fixes, the same recordings rebuilt before and after them, and a fresh
recording of every application.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/domain/analysisView/environmentStateWriteCreateCalls.ts Outdated
Comment thread src/domain/analysisView/environmentStateWriteComposed.ts Outdated
@earakely-scale
earakely-scale marked this pull request as ready for review October 26, 2026 02:32
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T21:47:22.245751Z 9f5191a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 812f9ff2fa

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/BrowserScenarioSecrets.ts Outdated
Comment thread src/browser/PlaywrightScenarioActions.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
- Basic credentials and values saved with `secret: true` are redacted however
  short, as declared secrets are; page values keep the 8-character floor
- a literal Basic password other than empty is refused, since Evidence
  records literals as written
- `save_response` takes only a request the step starts, so an earlier
  autosave cannot answer it, matches its path as the browser spells it, and
  stops watching requests when the step ends
- saving reads are bounded: a request step streams its answer up to
  4,000,000 characters; a UI step refuses a larger received size unread
- goto placeholders fill the destination URL before structured query values
  join it
- reading a composed string stops at the reading cap before comparing or
  allocating more
- a slug's `&` spelling follows every create the plan accepted and every held
  entity; direct slug rules and composed strings take it from held entities
  when the create slugs none, and a slug rule they contradict is not learned

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Contributor Author

212063f addresses all eight Greptile findings and the three Codex findings; each thread has a reply naming the fix.

Verified at 212063f:

  • npm run check, docs:check: passed.
  • npx vitest run --project domain --project services --project adapters: 17,882 passed; the 4 failures are the environment-specific ones that fail on main too.
  • 17 browser, environment and analysis-view boundary files: 122 passed.
  • Census of 109 recorded captures: the same 524 plans as before the review round.
  • Real-Chrome scenario with save_response on /ui/{*}: 12 of 12 steps, no password or raw token recorded.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 212063ff6c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/PlaywrightScenarioRequest.ts
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/domain/analysisView/environmentStateWriteExplain.ts Outdated
earakely-scale and others added 3 commits October 26, 2026 07:33
…alues

# Conflicts:
#	src/domain/analysisView/environmentStateWriteApply.ts
#	src/domain/analysisView/environmentStateWriteChanges.ts
#	src/domain/analysisView/environmentStateWriteComposed.ts
#	src/domain/analysisView/environmentStateWriteCreate.ts
#	src/domain/analysisView/environmentStateWriteExplain.ts
… written

- a saved integer past 2^53, which JSON parsing would round to another id,
  fails the step that saves it
- request paths and JSON strings read only `{saved:name}` as written; the
  percent-encoded spelling is read only in a goto URL, whose parsed form
  spells braces that way
- a held slug that neither `&` spelling gives counts as unknown: a create
  whose slug spells `&` learns no slug rule beside it, and no later create
  widens the spelling
- `save_response.path` takes the request path's portable pattern, shared as
  START_ORIGIN_PATH, so JSON Schema clients compiling it with the `v` flag
  read it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Contributor Author

Second review round: e42362c fixes the three Codex findings on 212063f (replies in each thread), after merging main 2b84efe. The merge resolved one conflict with the JSON-field ownership refactor; this branch's new field reads now use ownMember too. save_response.path now shares the request path's portable pattern (START_ORIGIN_PATH), which main's new pattern test requires.

Verified at e42362c on Node 24.22.1, with dependencies reinstalled for main's zod 4.6.6:

  • typecheck, lint, format, metadata, architecture guards, production knip, typed lint, docs, catalog, test discovery, todo scan and jscpd: passed. knip reports one unused export in src/contracts/toolFamilies.ts; this branch does not touch it and main 2b84efe reports it too.
  • npx vitest run --project domain --project services --project adapters: 18,261 passed; 1 failed, the Inspector cancellation test that fails on main here too.
  • 17 browser, environment and analysis-view boundary files: 128 passed.
  • Census of 109 recorded captures on main and this branch: 524 plans and 247 served on each; only the three Grafana folder creates differ.
  • Compact catalog: 1,038,975 bytes on main, 1,042,639 on this branch.
  • Real-Chrome scenario: 12 of 12 steps, the saved id sent as a JSON number, no password or raw token recorded.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 98f18cc52c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/domain/analysisView/environmentStateWriteExplain.ts Outdated
earakely-scale and others added 2 commits October 26, 2026 08:06
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cannot carry

- Each read of a saved response (the wait, sizes() and text()) races the
  step's deadline and abort signal, so a body still arriving when the
  step times out or is cancelled saves nothing, and the watcher detaches
  at once.
- A UI step keeps the 4,000,000-character cap and refuses before reading
  only a response received as more than 12,000,000 bytes, three per
  character; the decoded text is then held to the character cap.
- A Basic username with a colon fails the step before anything is sent.
- A saved `.` or `..` that fills a whole URL path segment fails the
  step, since the URL would drop it.
- Held entities that spell `&` both ways leave a create without `&` no
  slug rule.
- The saved-values example in the browser scenario contract is a whole
  scenario, as the catalog test reads every documented example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Contributor Author

Third review round: 69f38a78b fixes the five Codex findings on 98f18cc52, after merging main a30ffd27. Each thread has a reply, and all threads are resolved:

  • Saved-response reads race the step's deadline and abort signal.
  • The character cap gets a 12,000,000-byte bound before reading.
  • A Basic username with a colon fails the step.
  • A saved ./.. filling a whole path segment fails the step.
  • Mixed held & spellings leave no slug rule.

The saved-values example in the browser scenario contract is now a whole scenario, which fixes CI's productCatalog failure. Each new test fails on 98f18cc52.

Verified at 69f38a78b:

  • Static gates pass. knip's one unused export, isToolFamilyId, is reported by main too.
  • domain + services + adapters: 18,201 passed, plus the Inspector stat failure that main also shows locally.
  • 8 focused boundary files: 75 passed.
  • The 109-capture census is unchanged from main: 524 plans and 247 served, with the same three Grafana creates differing.

Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
…gment

Each filled path segment is checked after substitution, so a saved `.`
that a literal dot or `%2e` beside it turns into `..` fails the step in a
request path and a goto URL, before URL parsing drops it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b2e780776

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/PlaywrightScenarioRequest.ts
Comment thread src/domain/analysisView/environmentStateWriteChanges.ts
earakely-scale and others added 2 commits October 26, 2026 08:57
A saved number that is not finite, such as `1e400`, fails the step that
saves it, since a later step would send it as null in JSON and as
Infinity in a path or header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A later update whose slug spells `&` as the word refines a slug rule an
earlier update learned from a name without `&`, as creates do, unless an
accepted call or a held entity dropped it. Every earlier retained update
is rechecked under the refined rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/domain/analysisView/environmentStateWriteCalls.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0d7c248ba

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/domain/analysisView/environmentStateWriteCreate.ts Outdated
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/BrowserScenarioSecrets.ts
earakely-scale and others added 2 commits October 26, 2026 09:32
…ter creates that omit derived fields

- A later update whose slug slugs no `&` keeps the plan's `and`
  spelling; only a later rule that spells `&` as the word where the plan
  drops it is rechecked against earlier calls.
- A later create whose response leaves out a `composed` or `template`
  field the plan rebuilds is withheld as `unexplained-change`, since
  serving would add the field; an omitted reference still passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A scenario keeps at most 4,000,000 characters of saved values at once,
counting the current value of each name and the redaction spellings
secret saves keep, and a save that would exceed it fails its step before
anything is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/browser/BrowserScenarioSecrets.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d39f386d20

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/PlaywrightScenarioRequest.ts
Comment thread src/domain/analysisView/environmentStateWriteEffects.ts
earakely-scale and others added 2 commits October 26, 2026 11:15
…e spellings a secret save adds

- A UI step reads a saved response only once its decoded size is bounded
  before reading: the bytes received, or the declared length of one
  served from cache, times DEFLATE's 1,032 maximum expansion for gzip or
  deflate, must stay within 12,000,000 bytes. Another content coding
  (br, zstd) or an unknown size fails the step without reading.
- The saved-value cap counts the redaction spellings a secret save
  actually adds, its trimmed spelling included and an already held one
  not again, so re-saving the same secret does not grow the total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…held entities do

- A saved number whose JSON spelling names another decimal than the
  parsed value, such as `1e-400` (parsed to 0) or `1.0000000000000001`,
  fails the step that saves it, checked against the number's source
  text through the `JSON.parse` reviver.
- An update plan's slug rule takes the spelling of `&` that the entity's
  earlier value and the type's other entities show, as a create's does,
  and held entities that spell it both ways leave no slug rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 81a275b130

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/browser/BrowserScenarioSecrets.ts Outdated
earakely-scale and others added 2 commits October 26, 2026 12:40
…re building them

A saving response is decoded as fatal UTF-8 in both the request step and a UI
step's save_response, so a malformed byte fails the step instead of saving
U+FFFD in its place.

A secret save first counts a lower bound on its redaction spellings from the
value alone (it and its longer percent-encoded spelling), and refuses there
when that cannot fit, before any spelling is built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/browser/PlaywrightScenarioRequest.ts Outdated
Comment thread src/domain/analysisView/environmentStateWriteExplain.ts
Comment thread src/browser/PlaywrightScenarioRequest.ts
earakely-scale and others added 3 commits October 26, 2026 14:26
…lue fills, and pin a refused slug's source

A secret's redaction spellings include the one a URL's query parser gives
the percent-encoded value (`'` as `%27`), so a saved secret sent in a query
is redacted from captured URLs.

Saved values fill at most 4,000,000 characters into what one step sends,
counted before each fill is built (percent-encoded in a path or goto URL, as
text in a header or JSON body), so a value referenced many times fails the
step instead of multiplying.

A create whose slug the held entities' `&` spellings refuse keeps the
recorded slug, so it now accepts the member slugged only as recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f5191ab0f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

url: request.url(),
})
)
started.add(request);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Tie saved responses to the dispatched UI action

When Playwright is still waiting for locator actionability, a periodic or background request with the same method and path can begin after this listener is installed but before the click or press is dispatched; watch adds it and waitForResponse can settle on its response first. The step then saves an unrelated ID under the caller's name, so a later request can edit or delete the wrong object. Tie candidate request identity to the actual input dispatch, or reject ambiguous matches, rather than treating every post-listener request as triggered by the action.

AGENTS.md reference: AGENTS.md:L21-L21

Useful? React with 👍 / 👎.

);
const pass = await credentialPart(step, password!, `${label}.password`);
const encoded = Buffer.from(`${user}:${pass}`).toString("base64");
secrets.admitCredential(label, encoded);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Budget derived Basic credentials before retaining them

Fresh evidence beyond the aggregate saved-value fix is that one request can declare many Basic headers which share a large declared-secret username and use distinct short saved passwords, and every header is resolved before dispatch. This call permanently retains each distinct Base64 credential and recompiles all redaction patterns, while SavedFill charges only the short saved parts and MAX_RETAINED_SAVED_CHARACTERS does not charge these derived values, so a single admitted scenario can exhaust memory before fetch; preflight and account the encoded credential spellings in a retained-state budget.

AGENTS.md reference: AGENTS.md:L53-L54

Useful? React with 👍 / 👎.

if (match === null) return undefined;
const [, sign = "", whole = "", fraction = "", exponent = "0"] = match;
const digits = `${whole}${fraction}`.replace(/^0+/u, "");
if (digits === "") return "0";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve negative zero in saved numeric values

Fresh evidence after the numeric round-trip fixes is that a response number -0 reaches this branch with no significant digits, so decimalOf returns "0" and accepts it as equal to String(value). JSON.parse had preserved negative zero, but later path/header substitution and JSON.stringify emit positive 0, which changes the selected value for consumers that distinguish signed zero; reject -0 or retain its source representation.

AGENTS.md reference: AGENTS.md:L33-L33

Useful? React with 👍 / 👎.

}
let spellings: { readonly tree: unknown } | undefined;
for (const { name, json_pointer, secret } of entries) {
const value = pointerValue(parsed, json_pointer);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject array metadata in saved JSON pointers

When a response is {"ids":["a","b"]} and a save uses /ids/length, pointerValue skips the numeric-index branch and then Object.hasOwn reads the JavaScript array's own length, so this new save path stores 2 even though the JSON contains no member at that pointer. A later write can therefore send computed runtime metadata as if it were response evidence; array traversal should reject every non-index token.

AGENTS.md reference: AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

@morluto

morluto commented Oct 26, 2026

Copy link
Copy Markdown
Owner

Thank you for the substantial saved-values work and the detailed review follow-ups. #7267 builds on this proposal and credits its original work, but says it compared changes through an earlier head; this PR has since moved. Could the contributors coordinate on one review path and check the later fixes here against #7267 before consolidating? I am leaving both open while that comparison is made.

gensokyo-dev added a commit that referenced this pull request Oct 27, 2026
Save typed response fields and reuse generated IDs or authentication values across browser scenario requests, UI observations and navigation through the shared CLI/MCP workflow.

Preserve exact numbers, retained Evidence, explicit source/coverage limits and prospective page-value semantics. Bound response decoding, retained spellings and request serialization before allocation, and protect marked response values across source records, follow-up text and planned screenshots.

Integrates Edgar Arakelyan's #7162, including follow-ups through 9f5191a. Final local validation: 560 feature-branch tests and 561 current-main integration tests, static/build/docs checks and real PocketBase CLI/MCP privacy workflows. Whole-task comparison and verification limits are recorded in docs/verification/browser-saved-values-2026-10-26.md.

Co-authored-by: Edgar Arakelyan <edgar.arakelyan@scale.com>
@earakely-scale

Copy link
Copy Markdown
Contributor Author

Thanks — compared #7162 at 9f5191ab against main after #7267: all 55 of #7162's write-model tests pass on main, and its skill text is there. One behaviour #7267 left out: saving from a compressed or cached response (#7162 read gzip/deflate within a bound from the received size); Grafana gzips its folder creates, which recordings save ids from. I'll propose that separately. Closing this in favour of #7267.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants