Skip to content

test: enforce declared launches for daemon fixtures - #4432

Merged
thymikee merged 5 commits into
mainfrom
fix/daemon-fixture-launch-plan
Nov 12, 2026
Merged

thymikee merged 5 commits into
mainfrom
fix/daemon-fixture-launch-plan

Conversation

@thymikee

@thymikee thymikee commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow up on #4416 with declared daemon-fixture launch plans and teardown joins. Eighteen files; test infrastructure, fixture consumers and Vitest discovery. Production daemon behavior and gate thresholds are unchanged.

Review corrections cover module resolution, wrapped/typed launchers, namespace destructuring, bound plans, duplicate observer diagnostics and no-daemon teardown. Guard tests run directly in Vitest. The guard remains bounded syntactic enforcement; assignment/container aliases are outside its coverage. Design answer and receipt.

Validation

Exact head: 18fad61f628ce7744b672b880d94a8da5d3d6002; previous head: a2dc6558ade4598585d1dd6d8dc56ac6a4ccfb60.

  • Fresh frozen install/build and exact-head check:affected --run --head passed, receipt run-7tAruR: 42 runnable passes, 25 skips, no failed/unrun checks; clean unchanged checkout. Unit: 19,080 Vitest/76 Node passed, three/six skipped.
  • Guard inputs: before 14 failed/14 passed; corrected 28 passed. Observer: before 1 failed/5 passed; corrected 6 passed. Focused: 38 passed. Planted no-daemon teardown omission fails only with the correction.
  • Earlier run-Z5Avmu failed a release-publish timeout; unchanged-base unit passed. That receipt remains failed, cause unproven. No source or timeout changes preceded this clean rerun.
  • Previous hosted merge-ref CI and requested mutation shards passed. Ready-for-review CI/review remain required. No device-facing change. Cold self-review of the follow-up; no fresh independent source review.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff Turn on auto-fix

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.21 MB 6.21 MB 0 B
Package (unpacked) 6.21 MB 6.21 MB 0 B
Package (download) 1.88 MB 1.88 MB -1 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 21.9 ms 22.9 ms +1.0 ms
CLI --help 46.7 ms 48.4 ms +1.6 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@apex-sentinel-staging

apex-sentinel-staging Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Apex Sentinel

Verdict: approve

Since the last review: 0 new · 0 still open · 0 no longer raised

No findings.

1 finding below minor not shown (user_settings.min_severity).

1 new finding away from what changed since a2dc655 not shown. Comment @apex-sentinel-staging review for a full review.

Reviewed with this repository's own Sentinel configuration (e5c56032505c).

Feedback goes in the review comment threads, as a reply under the inline comment of a finding:

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Comment @apex-sentinel-staging review on this conversation to run a full review.

Earlier reviews

Review of a2dc6558 · 11.11.2026 12:46 UTC

Apex Sentinel

Verdict: approve

No findings.

Reviewed with this repository's own Sentinel configuration (a8774f3632eb).

@thymikee

Copy link
Copy Markdown
Member Author

At a2dc655 the runtime design is sound, but seven open threads on the AST guard still apply and need an answer before this is ready. The runtime launch planner and the teardown join make a spawn inside a plan impossible, which is the #4416 failure mode. The Mutants kernel-errors-2, -3 and -4 shards were still running when I looked, not failed. Those Mutants shards run the migrated fixture family, so green results there, including the baseline stage with no untracked child or fixture-teardown abort, are what must happen before merge. I did not run the planted-fault experiments, the guard self-tests or check:affected, so I take those claims as unverified.

Not blocking, and you can take or leave each: (1) The AST guard in daemon-fixture-launch-guard.ts is about 169 lines and will grow with every new way to spell a mock call. (2) The guard self-test lives in scripts/mutation/ but does not test the mutation harness, and daemon-fixture-launches.test.ts runs it through a node --test subprocess. It would fit better as a vitest test in scripts/tests/ listed in unit-core, and it would stop waking the mutation canary on every edit. (3) The violations list in registered-daemon-launch-plan.ts is scoped to the module and drained only by finishRegisteredDaemonFixture(s). A violation in a test that never calls finish is reported by a later test, or never. Asserting the plans once per test, in afterEach or when the next plan is installed, would close that gap.

Could the guard be much smaller? Keep the roughly 105-line planner and the teardown changes. Let the two shared fixture modules, packages/daemon-client/src/tests/registered-daemon-fixture.ts and src/tests/test-utils/registered-daemon-fixture.ts, own the vi.mock of '@agent-device/host-kit/command'. Implement runCmdDetachedMonitored there with the planner on an empty plan, so any launch refuses and latches. Export planLaunches(steps) to swap the active plan, and assert plans in afterEach. Tests then never call mockImplementation, and the guard becomes a short rule: no mockImplementation, mockReturnValue or spyOn on runCmdDetachedMonitored outside those two modules. That needs no alias or closure analysis. The first step is to move the per-file vi.mock declarations in the daemon-client tests and daemon-client-host.test.ts into the shared modules, using a setup file or a re-exported factory because vitest hoists vi.mock per test file.

Eight threads on the guard still apply, and a smaller guard would resolve most of them without patching each bypass. The seven that block: execFileSync stdout, duplicate observer error, relative import resolution, inline planner call, as-cast bypass, MockedFunction annotation, namespace destructuring bypass. The smaller one: no-daemon test drain. None of them is outdated.

@thymikee
thymikee marked this pull request as draft November 12, 2026 11:15
@thymikee

Copy link
Copy Markdown
Member Author

Pushed the review corrections at 18fad61f628ce7744b672b880d94a8da5d3d6002 and answered/resolved all eight threads.

  • All 14 original guard cases remain, now directly discovered by Vitest. Final inputs fail 14 cases on the old guard and pass 28/28 with the correction. The observer test detects the old duplicate error; all six planner tests pass. The focused four-file selection passes 38/38.
  • A planted extra launch in the CLI-only test passes with old teardown and fails at the corrected afterEach. No real child is spawned by that fault. The separate checkout was restored before validation.
  • The clean exact-head affected gate remains failed, receipt run-Z5Avmu, exactCommit:true, checkoutChanged:false: 19,079 unit passed, one release-publish fixture timed out at 5 seconds, three skipped. All preceding checks passed; later stages were not run. An unchanged-base full-unit comparison is running. Cause is unproven; the PR is draft. Earlier complexity failures remain recorded; the corrected guard passes without baseline changes.

On the smaller-design question: this round retains the existing fixture mock initialization and runtime planner ownership. The proposed centralized vi.mock setup would also change initialization and the consumer API across the fixture family; I have not implemented or claimed that redesign. I did move the guard tests into Vitest and make no-daemon teardown unconditional. This follow-up was cold self-reviewed; a fresh independent review is still a CI/reviewer obligation.

@thymikee

Copy link
Copy Markdown
Member Author

The unchanged-base comparison finished. At a2dc6558ade4598585d1dd6d8dc56ac6a4ccfb60, after frozen install/build, the same pnpm check:unit command passed 19,053 tests, three skipped; the Node suite passed 76, six skipped. No competing test run was active during this comparison.

The release-publish timeout at 18fad61 did not reproduce. That does not establish host contention, and the failed run-Z5Avmu receipt remains failed with cause unproven. All review corrections and focused regression proofs remain as recorded; the PR stays draft pending authoritative current-head validation. No timeout, test, baseline or gate was weakened.

@thymikee

Copy link
Copy Markdown
Member Author

Hosted CI for 18fad61 is green, including Coverage (19,704 passed, 70 skipped) and the requested kernel-errors mutation shards. All eight review threads remain resolved. Draft smoke lanes were skipped; the review check was neutral.

The jobs used GitHub merge checkout eace8d4eaa (PR head plus main 18dd00ea), so this is hosted integration evidence. It does not turn the exact-head local run-Z5Avmu timeout into a pass. The unchanged-base unit comparison passed and the local timeout cause remains unproven. I have updated the validation notes and kept the PR draft; no further source change follows from this check notification.

@thymikee

Copy link
Copy Markdown
Member Author

I found no defect in 18fad61, but I am holding approval until the evidence gaps below are closed. This follows my earlier review of a2dc655 (#4432 (comment)). The bypasses I reported there are now fixed: the Vitest-run guard suite, the single-violation planner path, .js and /index.ts import resolution, binding-following plan checks, wrapped-expression unwrapping, the Mock*< annotation match, destructured launchers, and the teardown finally that reaches assertPlans().

Hosted CI is green: 26/26 checks pass. They ran on the merge ref eace8d4 (this head plus main 18dd00e), not on the exact head. Hosted Coverage and the mutation shards that run the migrated fixture family passed. I did not run the guard self-tests, the planner tests or the planted-fault experiments, so the before/after failure counts are the author's claim. The author's local exact-head check:affected run (run-Z5Avmu) timed out in a release-publish fixture. That fixture does not overlap the files this PR touches, but the timeout is still unexplained. Please explain it, or show a clean re-run, before the PR leaves draft.

Not blocking, and you can take or leave it: the guard grew from 169 to 221 lines by adding one special case per reported bypass, and similar spellings still pass. I found these by reading the code and did not run them. An assignment alias (let l; l = vi.mocked(runCmdDetachedMonitored); l.mockImplementation(() => spawnRegisteredDaemonFixture())) passes, and so does a container alias (const o = { l: launch }; o.l.mockImplementation(...)). Bindings come only from VariableDeclarator, and resolves() matches a member access only when its property is named runCmdDetachedMonitored (scripts/lib/daemon-fixture-launch-guard.ts#L121). You can keep the guard as it is and stop chasing spellings.

Could the shared fixture modules own vi.mock('@agent-device/host-kit/command') instead? Each would export a factory that every test file's hoisted vi.mock calls, or a setup file would do it. runCmdDetachedMonitored would run the planner on an empty plan there, so an undeclared launch refuses and latches. planLaunches(steps) would swap the active plan, and a shared afterEach would assert it. The guard then becomes one rule: no mockImplementation, mockReturnValue or spyOn on runCmdDetachedMonitored outside those modules. That removes the alias, cast, binding and annotation analysis and most of the 172-line guard self-test. The first change would be to move the per-file vi.mock declarations in the daemon-client tests and daemon-client-host.test.ts into the shared modules, since Vitest hoists vi.mock per test file. Please either adopt that or say why you keep the AST guard.

The eight earlier review threads are fixed at this head, so you can resolve them: #4432 (comment), #4432 (comment), #4432 (comment), #4432 (comment), #4432 (comment), #4432 (comment), #4432 (comment) and #4432 (comment).

No conflicts. Before merge, please answer the simplicity question and resolve the run-Z5Avmu timeout.

@thymikee

thymikee commented Nov 12, 2026 •

Copy link
Copy Markdown
Member Author

I am retaining the bounded AST guard in this PR, and I am not adding assignment-alias or container-alias analysis. Those examples are valid limits of the guard; it is not a sound analysis of arbitrary JavaScript.

The runtime planner owns the describe/observe boundary and rejects a child launch from those callbacks. The fixture separately owns every child it launches and joins it during teardown, including a CLI-only test that creates no HTTP daemon. The AST rule catches accidental direct replacement of the monitored launcher in the existing fixture consumers; it does not establish those runtime guarantees by itself.

The shared hoisted mock factory is a reasonable smaller direction. It changes who initializes and resets the launcher across the existing consumers, including tests that inspect the mock's call history. I am keeping that API/hoisting migration separate from this correction of the reported bypasses and teardown gap. This PR retains the existing initialization, with no claim that the two additional spellings are rejected. All eight original threads are already resolved.

The requested clean rerun passed at the unchanged head after frozen install/build: pnpm check:affected --run --head 18fad61f628ce7744b672b880d94a8da5d3d6002. Receipt run-7tAruR: 42 runnable checks passed, 25 skipped, zero failed or unrun. Full unit: 19,080 Vitest passed/three skipped; Node: 76 passed/six skipped.

{
  "head": "18fad61f628ce7744b672b880d94a8da5d3d6002",
  "status": "passed",
  "exactCommit": true,
  "checkoutChanged": false,
  "passed": 42,
  "skipped": 25,
  "failed": 0,
  "not-run": 0
}

The earlier run-Z5Avmu receipt remains failed and its timeout cause remains unproven. This new clean run validates the exact PR head. No source, timeout, test, baseline or gate was changed. The PR is moving out of draft for review; this is not a merge or a ready-for-human label change.

@thymikee
thymikee marked this pull request as ready for review November 12, 2026 12:32

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 18 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="test/repo-guards/daemon-fixture-launches.test.ts">

<violation number="1" location="test/repo-guards/daemon-fixture-launches.test.ts:16">
P3: The file list includes tracked files that were deleted from the working tree but not staged, so `readFileSync` throws ENOENT and the test crashes. Skip paths that no longer exist on disk, or list with `-z` and check existence first.</violation>
</file>

<file name="packages/test-kit/src/registered-daemon-launch-plan.ts">

<violation number="1" location="packages/test-kit/src/registered-daemon-launch-plan.ts:20">
P2: A companion's `exitGate` is silently ignored: `start()` discards the companion's result and never applies the gate. The type still accepts it, so a caller can declare a gate that does nothing. Remove `exitGate` from the companion type.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@thymikee

Copy link
Copy Markdown
Member Author

Thanks, that answers the design question. Keeping the bounded guard here is reasonable: moving the launcher mock into shared hoisted factories changes how consumers initialize and reset it, including tests that read its call history, so it belongs in its own change. Please open a follow-up issue for that migration so the smaller design is not lost.

The clean check:affected rerun at 18fad61 (run-7tAruR, 42 passed, none failed) closes the earlier timeout question, and CI is green. No code findings remain, so this is ready for maintainer review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 12, 2026
@thymikee
thymikee merged commit 59c5246 into main Nov 12, 2026
32 checks passed
@thymikee
thymikee deleted the fix/daemon-fixture-launch-plan branch November 12, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant