test: guard user-action-events file-backend writes against leaking to real data/ - #5627
Conversation
… real data/ server/services/userActions.js writes through createPgFileFacade, which selects the file backend under NODE_ENV=test. Any suite that exercises an instrumented route (recordUserAction) without redirecting PATHS.data to a temp root therefore writes data/user-action-events.json into the developer's live data/ tree - the same bug class as atomantic#3683/ atomantic#3687. atomantic#5594 patched the three suites that tripped on it, but that is a per-suite fix: the next hook added to a route an untethered suite exercises silently re-opens the hole (epic atomantic#5593 phase 3/atomantic#5596 is explicitly about growing that allowlist). Add a structural guard directly on the write path (option 1 from the issue): right before the file-backend's atomicWrite, compare the live PATHS.data against the real repo data/ dir - computed independently via the same fileURLToPath/resolveInstallRoot technique lib/paths.js itself uses, so a suite's PATHS mock can't spoof the comparison too. A write attempted against the unredirected real path now throws a clear, actionable error instead of landing on disk. userActionsDataRootGuard.test.js proves the guard fires (and that nothing lands in the real data/ tree) by deliberately NOT mocking fileUtils.js - the one test in the suite meant to run unredirected. Verified the three previously-patched suites, every other known recordUserAction caller (settings.js, taskSchedule.js), and userActions.js's own suite all still pass with the guard active. Fixes atomantic#5605
|
Thanks for leaving the guard mechanism up to the implementer — went with option 1 since it protects the next route/hook automatically rather than needing another per-suite patch. Glad to switch to one of the other two approaches if you have a preference I'm not seeing. |
atomantic
left a comment
There was a problem hiding this comment.
Reviewed by /do:review (security-focused pass, external contributor) — 2 critical, 1 improvement.
The approach is right and the ticket is satisfied in spirit. Option 1 from #5605, correctly narrowed to the recordUserAction write chokepoint instead of blanket-wrapping atomicWrite — that scoping call is good, and it means #5596's growing hook allowlist stays covered for free. Deriving REAL_REPO_DATA_DIR independently of the mockable PATHS is the right technique; I verified it agrees with lib/paths.js for a plain install, a PORTOS_DATA_ROOT pin, and a CoS worktree checkout.
Verified against the branch (worktree, isolated):
- Full
cd server && npm test— 1809 files / 36801 tests pass, exit 0. Nodata/user-action-events.jsonafter the run. AC #2 ✔ - The guard does fire on an unredirected write. AC #1 ✔
- Security scan: diff is pure ASCII apart from em-dashes (no zero-width/bidi), no network calls, no
process.envreads, noeval/exec/child_process, no dependency or CI changes, production behavior unchanged (isTestRunner()-gated). Single commit, single author. Nothing injection-shaped in comments or prose.
Blocking (please address):
-
The new test deletes real user data (
userActionsDataRootGuard.test.js:22-26). TheafterEachrmSyncis unconditional, and this is the one test that deliberately runs withPATHS.dataunredirected — so it targets the live tree. On an install using the documentedMEMORY_BACKEND=fileescape hatch,npm testboth fails spuriously (line 36 asserts the file is absent) and then permanently removes the developer's ledger. Reproduced end to end. A guard against leaking intodata/must not itself destroydata/. -
The guard is bypassed by the dedupe short-circuit (
userActions.js:321-322). It sits afterloadFileEvents()and after thereturn null, so an un-redirected suite replaying an existing(type, dedupeKey)silently no-ops with no throw — reproduced. The real ledger has also already been read into the test process by that point. Hoisting the assertion to the top of the queued callback closes both.
Non-blocking: the REAL_REPO_DATA_DIR derivation is a second copy of paths.js's INSTALL_ROOT formula and fails open if that formula ever changes — worth collapsing into one exported helper in lib/dataRoot.js.
Coherence check: the PR description matches the diff; the test-plan claims hold up (I re-ran them). One AC gap worth noting: #5605 asks that the guard "names the offending suite" — the message names the fix but not the suite. Vitest's stack trace covers it in practice, so I'd call that satisfied, but a one-line mention would close it cleanly.
Thanks for taking this on — the structural instinct here is exactly right, and with the two fixes above it's a clear improvement over the per-suite patching in #5594.
Generated by /do:review
Addresses both blocking findings from the maintainer's review of atomantic#5627: 1. userActionsDataRootGuard.test.js's afterEach ran an unconditional rmSync on the repo's real user-action-events.json. On an install using the documented MEMORY_BACKEND=file escape hatch, that file can legitimately already be the developer's real ledger - the old `expect(existsSync(...)).toBe(false)` assertion failed spuriously against it, and the afterEach then deleted it regardless. Rewritten to snapshot present/absent + content before exercising the guard, assert the tree comes back unchanged (not merely absent), and only clean up a file this run's own guard regression actually created. 2. The guard in userActions.js's file-backend record() sat after the dedupe short-circuit's `return null`, so an un-redirected suite replaying an existing (type, dedupeKey) silently no-op'd past the guard with no throw - and by then loadFileEvents() had already read the real ledger into the test process regardless. Hoisted the assertion above both loadFileEvents() and the dedupe check. Also addresses the non-blocking note: REAL_REPO_DATA_DIR was a second, independent copy of paths.js's CODE_ROOT formula (fails open if it ever drifts). Extracted resolveCodeRootForModule() into dataRoot.js as the single source of truth; both paths.js and userActions.js now derive their root through it, with paths.js's own PATHS object unchanged in every value it produces. Verified: full `cd server && npm test` - 1793/1810 files pass, same 16 files/58 tests fail with or without this change (confirmed via git stash), all pre-existing Windows-only environment noise (symlink creation without Developer Mode privileges, EBUSY temp-file locks, one missing local Python module) - none touch paths.js/dataRoot.js/ userActions.js. Also re-ran the guard test and userActions.test.js in isolation, both green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed both blocking findings, plus the non-blocking suggestion:
Verified: full 🤖 Generated with Claude Code |
Addresses the maintainer's /do:review design question on atomantic#5625: routing the six server/services/* consumers through aiToolkit/index.js (the toolkit's composition root) pulled 24 modules including express and child_process, where the previous direct import to internal/ pulled 1 module and nothing else - three lines below a comment in aiProvider.js explaining why it deliberately avoids exactly that. Moves evaluateSecretEndpoint/assertSecretEndpoint from aiToolkit/internal/endpointGuard.js to aiToolkit/endpointGuard.js - a peer of aiToolkit/errorDetection.js, which already follows this exact "pure toolkit-root module, importable directly AND re-exported via the barrel" shape. index.js now does `export * from './endpointGuard.js'` instead of a named re-export, matching errorDetection.js's own pattern. All 8 consumers repointed at the new path: the 6 services from the original PR (aiProvider.js, askService.js, insightsService.js, localLlmPlayground.js, visionTest.js, voice/llm.js, plus visionTest.frameGuard.test.js's mock target), and 2 toolkit-internal callers the original PR's scope didn't touch (providers.js, runner.js) - both were still importing the pre-move internal/ path directly and would have broken had they been left pointed at a path that no longer exists. Also updated 3 doc-comment references to the old path (providers.js, aiToolkit/validation.js, lib/validation.js) for accuracy; left one reference alone (scripts/migrations/ 195-cerebras-provider.js) since it's a historical, already-applied migration's own comment, not live code. Verified: the 12 directly relevant test files pass (583 tests, plus 5 pre-existing intentional skips); a repo-wide grep confirms zero remaining references to the old path outside that one historical migration comment; lib/aiToolkit/runner.test.js's own EBUSY failures reproduce identically with or without this change (pre-existing Windows temp-file-lock flakiness, already confirmed unrelated on sibling PRs atomantic#5626/atomantic#5627 via git stash). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Addresses the maintainer's /do:review design question on atomantic#5625: routing the six server/services/* consumers through aiToolkit/index.js (the toolkit's composition root) pulled 24 modules including express and child_process, where the previous direct import to internal/ pulled 1 module and nothing else - three lines below a comment in aiProvider.js explaining why it deliberately avoids exactly that. Moves evaluateSecretEndpoint/assertSecretEndpoint from aiToolkit/internal/endpointGuard.js to aiToolkit/endpointGuard.js - a peer of aiToolkit/errorDetection.js, which already follows this exact "pure toolkit-root module, importable directly AND re-exported via the barrel" shape. index.js now does `export * from './endpointGuard.js'` instead of a named re-export, matching errorDetection.js's own pattern. All 8 consumers repointed at the new path: the 6 services from the original PR (aiProvider.js, askService.js, insightsService.js, localLlmPlayground.js, visionTest.js, voice/llm.js, plus visionTest.frameGuard.test.js's mock target), and 2 toolkit-internal callers the original PR's scope didn't touch (providers.js, runner.js) - both were still importing the pre-move internal/ path directly and would have broken had they been left pointed at a path that no longer exists. Also updated 3 doc-comment references to the old path (providers.js, aiToolkit/validation.js, lib/validation.js) for accuracy; left one reference alone (scripts/migrations/ 195-cerebras-provider.js) since it's a historical, already-applied migration's own comment, not live code. Verified: the 12 directly relevant test files pass (583 tests, plus 5 pre-existing intentional skips); a repo-wide grep confirms zero remaining references to the old path outside that one historical migration comment; lib/aiToolkit/runner.test.js's own EBUSY failures reproduce identically with or without this change (pre-existing Windows temp-file-lock flakiness, already confirmed unrelated on sibling PRs atomantic#5626/atomantic#5627 via git stash). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
atomantic
left a comment
There was a problem hiding this comment.
Re-reviewed after the fix commit — both blocking findings are resolved, and I verified each one independently rather than taking the description's word for it. Approving.
Verified against the branch (isolated worktree):
- Destructive cleanup — fixed. Seeded a real
data/user-action-events.json(theMEMORY_BACKEND=fileinstall shape that broke the previous version) and ran the guard test: it now passes and leaves the file byte-identical. The snapshot-then-compare rewrite plus the!existedBeforecleanup gate is exactly right, and thetry/finallyis a better shape than theafterEachI suggested. - Dedupe bypass — fixed. Reproduced the original hole's setup: seeded
{type:'cos.task.create', dedupeKey:'preexisting-key'}and replayed the same pair. Before the hoist this returnednullsilently; now it throws, andloadFileEvents()never reads the real ledger. - Bypass probe. Removed
assertTestDataRootRedirected()and re-ran — the guard test goes red with the leaked event visible in the diff. The test genuinely pins the contract rather than passing vacuously. - Shared helper (non-blocking) — addressed.
resolveCodeRootForModulecollapses the duplicated derivation, andpaths.js'sCODE_ROOTis expression-identical to before, so the fail-open drift risk is gone. - Full
cd server && npm test— nodata/user-action-events.jsonafter the run, and zero guard firings across all 36.5k tests (so no existing suite is falsely tripped). CI's Server tests job is green on this head. - Security pass: diff is pure ASCII apart from em-dashes (no zero-width/bidi), no
child_process/eval/network/process.envadditions, no dependency or workflow changes, production behavior unchanged (isTestRunner()-gated early return). The only destructive call is the now-correctly-gatedrmSync.
Ticket alignment (#5605): option 1, scoped to the recordUserAction chokepoint. AC #1 ✔ (bypass probe), AC #2 ✔ (clean tree after a full run), AC #3 — the message names the fix but not the offending suite; Vitest's stack covers that in practice, so I'm calling it satisfied.
Follow-ups I'll take on our side (all minor, none worth another round-trip here):
listis still unguarded — an un-redirected suite can read the live ledger throughlistUserActions. Read-only, so outside this ticket's AC.- If the guard ever regresses on a machine that already has the ledger, the test leaves the probe event appended (it snapshots
contentBeforebut doesn't restore it). lib/README.md'sdataRoot.jsrow doesn't mention the new export.
Thanks for the thorough turnaround on this — the structural fix is a real improvement over the per-suite patching in #5594, and it keeps #5596's growing hook allowlist covered for free.
#5627 landed the #5605 structural guard on the file backend's `record` path, so an un-redirected suite can no longer write user-action-events.json into the developer's live data/ tree. `list` was left unguarded, so such a suite could still READ the live ledger — machine-local operator records the privacy ADR keeps off the wire and out of untethered test processes. Also hardens the guard test's own cleanup: it snapshotted the pre-existing file's bytes but only ever deleted a file it had created, so a guard regression on a MEMORY_BACKEND=file install left the developer's real ledger with the probe event appended. It now restores the snapshot. The guard message takes the attempted operation as an argument so the read path doesn't report itself as a write.
Summary
server/services/userActions.jswrites throughcreatePgFileFacade, which selects the file backend underNODE_ENV=test. Any suite that exercises an instrumented route (recordUserAction) without redirectingPATHS.datato a temp root writesdata/user-action-events.jsoninto the developer's livedata/tree — the same bug class as unlockPass.test.js reads the developer's real data/ directory — 2 failures on any checkout with real series #3683/Audit the whole server suite for tests that read the checkout's real data/ directory #3687.atomicWrite, compares the livePATHS.dataagainst the real repodata/dir — computed independently via the samefileURLToPath/resolveInstallRoottechniquelib/paths.jsitself uses, so a suite'sPATHSmock can't spoof the comparison too. A write attempted against the unredirected real path now throws a clear, actionable error (naming the fix and pointing atlib/mockPathsDataRoot.js) instead of landing on disk.recordUserActionwrite chokepoint specifically (not a blanket wrap ofatomicWrite, which is used by dozens of unrelated stores repo-wide) — every route that ever attaches arecordUserActionhook funnels through this one function regardless of which suite exercises it, so growing the hook allowlist (Grow the user-action allowlist and detect leftover branches after idle agent runs #5596) stays covered automatically.Test plan
userActionsDataRootGuard.test.jsproves the guard fires — deliberately does NOT mockfileUtils.js(the one test meant to run unredirected), assertsrecordUserActionthrows before any write, and asserts no file lands in the realdata/treeuserActions.test.js(own suite, already correctly redirected) — 13/13 pass, guard stays silentrecordUserActioncaller + its test suite —routes/cos.test.js,routes/cosTaskRoutes.test.js,services/cos.test.js,services/cosAgentFeedback.test.js,routes/settings.test.js,services/settings.test.js,services/taskSchedule.test.js— 655/655 passdata/user-action-events.jsonexisted before or after the guard test runFixes #5605