Repository navigation
🤖 fix: start the perf servers with a per-server token instead of --no-auth - #7908
Conversation
Red: scripts/perf/serverToken.ts does not exist yet.
…-auth pageTiming, firstRunViewParity, generatePhoneChrome and run-server-bench started xum server with --no-auth on a loopback port that every local account can reach. Each now sets XUM_SERVER_AUTH_TOKEN in the server's env (never argv), navigates with ?token= (kept out of result files), or sends a bearer header.
…l base for the next edits)
…in argv - The skill opens the app with `agent-browser batch`: node builds the `[["open", url]]` JSON with the token and the process pipes it on stdin. Neither node nor agent-browser nor Chrome gets the token as an argument. The old `agent-browser open "<url with token>"` puts it in the client's command line. - The curl example gets `-s`. - The Security notes no longer say that the sandbox backend runs with `--no-auth` (stale since the backend takes a token).
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 602a49edf5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…UX_ name The perf scripts start built trees from before the MUX_ to XUM_ rename for --tree comparisons. Those trees read only MUX_SERVER_AUTH_TOKEN, so the XUM_ token was ignored and the browser's token was refused. serverTokenEnv writes both names through assignXumEnvironmentValue.
🟢 e2e stories (Haiku): 14 passed22 agent steps · 7 replayed from cache · 15 model calls · 47.1k tokens (23% cached) All 14 tests in 9 files
e2e 0.20.0 · 1m 13s · web · run artifacts 🟢 e2e Electron stories (Haiku): 3 passed3 agent steps · 3 model calls · 7.9k tokens (22% cached) All 3 tests in 3 files
e2e 0.20.0 · 18.6s · electron · run artifacts 🟢 e2e agent repros (Haiku): no tests selectedSpend: $0.00 of the $1.00 cap (proxy ledger) e2e 0.20.0 · 5.8s · 2 targets (web, phone) · run artifacts |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
…er skill agent-browser batch prints the URL it opened, token included. The recipe now keeps that output in a variable and prints nothing on success, and prints it with the token removed on failure. Also fixes the stale --no-auth comment in dev-server-sandbox.ts.
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Summary
Four perf scripts started
xum server --no-authon a loopback port, which every local account can reach. Each now starts the server with a per-server random token in its environment, the same waymake dev-serverdoes since #7904. The skill gets the argv-free browser open,-son the curl example, and a fix for a sentence that was stale.Background
make dev-serverstopped using--no-authin #7904, with the perf scripts named there as the same class of issue.scripts/perf/{pageTiming,firstRunViewParity,generatePhoneChrome}.tsandscripts/perf/workspace-scale/run-server-bench.tsstill passed--no-authfor their short-lived servers.run-server-benchruns inperf-profiles.ymlthroughmake perf-workspace-scale. The bug-bash container and the test harness (src/node/services/sessionLocks/backendKill.testHarness.ts) also start--no-authservers. They are not in this PR.Implementation
scripts/perf/serverToken.tshasnewServerToken()(32 random bytes, hex) andwithServerToken(url, token).serverTokenEnv:XUM_SERVER_AUTH_TOKENand the legacyMUX_SERVER_AUTH_TOKEN, written byassignXumEnvironmentValue) and drops--no-auth. The token is never in argv.?token=, which the app and the static views already read.pageTimingkeeps the URL in its per-run record without the token.firstRunViewParitynow matches its route by path, because the URL has a query.run-server-benchsends a bearer header..xum/skills/dev-server-sandbox/SKILL.md:[["open", url]]from node toagent-browser batchon stdin, so the token is in no command line.agent-browser batchprints the URL it opened, token included. The recipe keeps that output in a variable: it prints nothing when the open works, and prints it withtoken=<redacted>when it fails (the same approach ascmd_openinverify-xum.shon 🤖 feat: add verify-xum skill that drives the real Xum web UI #7895).-s.--no-auth. 🤖 fix: stop the dev-server sandbox from allowing every Host header #7902 added that sentence before 🤖 fix: start the make dev-server backend with an auth token #7904 changed it.Validation
Red first:
scripts/perf/serverToken.test.tsfailed because the module did not exist, then passed.Each script ran with auth on. A /proc probe watched every
xum serverthat a script started: no--no-authin argv, a 64-character token in the environment only, and an unauthenticatedPOST /api/config/getConfigreturned 401.generatePhoneChrome --check: passes ("phone chrome skeleton matches the live page").pageTiming --base <tree> --head <tree> --pages p1,p2with one tree for both arms: exit 0. The probe saw 64 servers, all returned 401, and 0 of the 64 per-run records contain a token.run-server-benchon a 50-workspace fixture (--repetitions 1 --launches 1): completed and wrote its results.firstRunViewParityexits 1 on this host with the same shell pixel diffs as the copy of the script on main (for example390-dark: shell: 490 pixels off by more than 16 at x,y,w,h 13,15,14,14in both). I did not find the cause. No workflow runs this script.Browser-open output (round 4 fix): I ran the skill's recipe as written against a sandbox with a token of
Xq+9/aZ=_proof&k#zand saved stdout and stderr:agent-browser, success: 0 bytes on stdout and stderr, 0 hits for the raw and the URL-encoded token. The page loaded authenticated (the first-run provider dialog, not "Authentication Required") and its URL had no token.agent-browserthat echoes the opened URL with the token, as the batch can: the old recipe printed it (1 hit for the encoded token). The new recipe printed nothing with exit 0, and with exit 1 it printedhttp://localhost:<port>/?token=<redacted>(0 hits).Argv-free browser open: sandbox token
Xq+9/aZ=_proof&k#z, so the raw and URL-encoded forms differ. A /proc scanner andstrace -f --seccomp-bpf -e trace=execvecovered the chain, including the agent-browser daemon and Chrome with its helper processes.agent-browser open "<url>") as a control: the execve of the agent-browser client carried the encoded token, and the scanner hit on it. Chrome's argv held neither the token nor any URL in either form.make static-check,make lint-eslint,make crap-checkandmake dry-checkpass.scripts/perf/,scripts/devServerAuth.test.tsandscripts/rlm-eval/give 284 tests: 282 passed on the first run here, and the twodesktopColdStarttests hit bun's 5 s default timeout on a coldtsxstart in a fresh checkout under load. They pass with a longer timeout and on a warm rerun.Risks
--treecomparisons can start trees from before the Xum rename (🤖 feat: transition product identity from Mux to Xum #3903), which read onlyMUX_SERVER_AUTH_TOKEN, so the scripts set both names. A unit test covers the two names. I did not run a real pre-rename build.Generated with
xum• Model:anthropic:claude-sonnet-5-5• Thinking:xhigh