Skip to content

feat(cli): add doctor, designate and status commands (Task 2.2, part 2) - #9

Merged
Andreas-Froyland merged 3 commits into
mainfrom
task-2.2b-cli
Sep 22, 2026
Merged

Andreas-Froyland merged 3 commits into
mainfrom
task-2.2b-cli

Conversation

@Andreas-Froyland

@Andreas-Froyland Andreas-Froyland commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Summary

Task 2.2 of the plan, part 2: the first slice of the CLI —
environment- and project-inspection commands that are fully testable without a candidate, a driver, or a real
scenario. run/resume (which need those) land in a following PR alongside the Tauri driver and the sample project.

Commands

node packages/qa/src/cli/main.ts doctor --project qa/project.json --profile windows [--json]
node packages/qa/src/cli/main.ts designate [--root <path>] [--json]
node packages/qa/src/cli/main.ts status [--root <path>] [--json]
  • doctor loads a qa/project.json, measures this machine against the named profile (inspectEnvironment), and
    reports whether it matches, alongside the raw capabilities and the display kind.
  • designate/status wrap Task 2.1's test-root and ledger primitives. --root defaults to .release-qa under the
    current directory (now gitignored) when not given.
  • Every command prints to stdout on success, stderr on failure; --json switches both to one line of JSON.

Exit codes

Matches the plan's taxonomy: 0 passed/ready, 2 this machine does not meet the requested profile, 3 everything
else that stopped the command (bad usage, an unreadable/invalid project file, an undeclared profile). status
reports what it finds (undesignated, dirty) as a fact, not a command failure — matching git status's own
convention. 1 (a scenario the candidate failed) is defined in the exit-code table but not yet reachable, since no
command runs a scenario yet.

args.ts

Never throws on bad input, matching this codebase's ParseResult-style rigor elsewhere: unknown or missing commands,
unknown flags, a flag given more than once, a flag with no value — including one whose "value" looks like another
flag (--project --profile w), which would otherwise silently swallow --profile as --project's value and leave
w as an unexplained extra argument — and extra positional arguments are all reported, never guessed past.

Proven as an executable, not just a module

test/cli/executable.test.ts spawns node packages/qa/src/cli/main.ts ... as a real child process (matching
test/no-build.test.ts's spirit for the package as a whole), exercising the malformed-args and unsupported-profile
cases the plan calls out specifically "through the CLI executable". --candidate verification and cancellation,
also mentioned there, apply to run and are deferred with it.

Scope note

The plan's Task 2.2 bullet describes the whole CLI (doctor, run, resume, the sample scenario, the Linux
preflight) as one task. Part 1 (#8, merged) delivered the no-build guarantee and the Linux display preflight; this
is part 2. Part 3 will add run/resume, candidate/artifact SHA-256 verification, the Tauri driver adapter, and the
examples/tauri-smoke sample, which is when the plan's "run the sample on Windows and Ubuntu/Xvfb" exit criterion is
actually met.

Testing

Test-first throughout (57 new tests). Mutation-checked with a throwaway script covering the trickiest branches in
each new file (duplicate/unknown/missing flags, a flag's value looking like another flag, JSON/schema failures,
unknown profile, mismatch reporting, swallowed errors, wrong exit codes); one survivor (the "value looks like
another flag" case wasn't distinctly tested) was closed with a new test, then reconfirmed killed.

  • Clean npm ci, npm run typecheck, npm test: 519 passed, 2 skipped (Windows 11, Node 24.13); no leaked
    processes, temp directories, or home markers
  • CI on ubuntu-24.04 and windows-2025 (a genuine Windows-only failure surfaced: on the CI account
    (runneradmin), the temp directory's 8.3 short name and its realpath-resolved long name differ, and
    checkTestRoot deliberately reports the resolved path; four tests comparing it against the pre-resolution path
    they created the directory with failed there specifically — fixed by resolving the expected path the same way)

🤖 Generated with Claude Code


Summary by cubic

Adds the first slice of the CLI — doctor, designate, and status — for environment and project inspection, fully testable without a candidate, driver, or scenario.

Commands

  • doctor measures this machine against a profile in qa/project.json and reports readiness, raw capabilities, and display kind.
  • designate and status wrap the Task 2.1 test-root and ledger primitives; --root defaults to .release-qa (now gitignored) under the current directory.
  • Every command prints to stdout on success and stderr on failure; --json switches both to one line of JSON, honored even when the invocation is malformed.
  • JSON error output always includes an issues array (empty when none), so consumers can key on it unconditionally.

Behavior

  • Argument parsing never throws: unknown commands and flags, duplicate flags (including --json given twice), missing values, and extra positionals are all reported, including a flag whose "value" looks like another flag.
  • Exit codes follow the plan: 0 ready, 2 machine does not meet the requested profile, 3 anything else that stopped the command; status reports undesignated or dirty roots as facts, like git status.
  • Commands never throw either: status converts even the test-root check failing (a narrow race) into a reported command error.
  • The CLI is proven to run as a real child process (node packages/qa/src/cli/main.ts ...), not just an imported module.
  • run/resume, candidate verification, and the Tauri driver land in the next PR with the sample project.

Written for commit 9555e44. Summary will update on new commits.

Review in cubic

Task 2.2 of the plan, the first slice of the CLI: environment- and project-inspection commands, testable end to end
without a candidate, a driver or a real scenario. `run`/`resume` (which need those) land in a following PR alongside
the Tauri driver and the sample project.

- `release-qa doctor --project <path> --profile <id> [--json]` measures this machine against a profile a project.json
  declares and reports whether it matches, alongside the raw capabilities and the display kind.
- `release-qa designate [--root <path>] [--json]` and `release-qa status [--root <path>] [--json]` wrap the existing
  test-root and ledger primitives from Task 2.1; `--root` defaults to `.release-qa` under the current directory
  (gitignored) when not given.
- `args.ts` never throws on bad input: unknown/missing commands, unknown flags, flags given more than once, a flag
  with no value (including one whose "value" looks like another flag, which would otherwise be silently swallowed),
  and extra positional arguments are all reported, not guessed past.
- Exit codes match the plan's taxonomy: 0 passed/ready, 2 this machine does not meet the requested profile, 3
  everything else that stopped the command (bad usage, an unreadable/invalid project file, an undeclared profile).
  `status` reports what it finds (undesignated, dirty) as a fact, not a command failure, matching `git status`.
  1 (a scenario the candidate failed) is defined but not yet reachable, since no command runs a scenario yet.
  `--candidate` verification and cancellation, both mentioned in the plan's test list, apply to `run` and are
  deferred with it.
- The CLI is proven to run as `node packages/qa/src/cli/main.ts ...` from a separate process (test/cli/executable.test.ts),
  not just imported into the test worker, exercising the malformed-args and unsupported-profile cases the plan calls
  out through the executable specifically.
- README documents the new commands, the default root, output modes and exit codes, and its Status section no longer
  says the tool is unbuilt.

Test-first throughout (57 new tests). Mutation-checked with a throwaway script covering the trickiest branches in
each new file (duplicate/unknown/missing flags, a flag's value looking like another flag, JSON/schema failures,
unknown profile, mismatch reporting, swallowed errors, wrong exit codes); one survivor (a flag's value looking like
another flag was not distinctly tested) was closed with a new test, then reconfirmed killed. Clean npm ci, typecheck,
and full suite (512 passed, 2 skipped); no leaked processes, temp directories, or home markers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 13 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/qa/src/cli/args.ts">

<violation number="1" location="packages/qa/src/cli/args.ts:51">
P2: When a recognized command is malformed with `--json`, parsing discards the JSON mode before `main` handles the error, so stderr is plain text instead of one-line JSON. Preserve `--json` in parse failures and serialize the error in the caller.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/qa/src/cli/args.ts Outdated
Comment thread packages/qa/src/cli/args.ts Outdated
if (!isCommandName(name)) return fail(`unknown command "${name}"; expected one of ${COMMAND_NAMES.join(', ')}`);

const flags = parseFlags(rest, SPECS[name]);
if (!flags.ok) return flags;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a recognized command is malformed with --json, parsing discards the JSON mode before main handles the error, so stderr is plain text instead of one-line JSON. Preserve --json in parse failures and serialize the error in the caller.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/qa/src/cli/args.ts, line 51:

<comment>When a recognized command is malformed with `--json`, parsing discards the JSON mode before `main` handles the error, so stderr is plain text instead of one-line JSON. Preserve `--json` in parse failures and serialize the error in the caller.</comment>

<file context>
@@ -0,0 +1,102 @@
+  if (!isCommandName(name)) return fail(`unknown command "${name}"; expected one of ${COMMAND_NAMES.join(', ')}`);
+
+  const flags = parseFlags(rest, SPECS[name]);
+  if (!flags.ok) return flags;
+
+  switch (name) {
</file context>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 24e0902. json is now decided independently of where parsing fails, and threaded through every failure path (including the top-level no-command/unknown-command cases), so a malformed invocation that also asked for --json still gets a JSON error on stderr instead of silently downgrading to plain text.

Comment thread README.md Outdated
Comment thread packages/qa/src/cli/environment-commands.ts Outdated
Comment thread packages/qa/test/cli/main.test.ts Outdated
Comment thread packages/qa/test/cli/environment-commands.test.ts Outdated
…n, catch checkTestRoot throwing

- Windows CI failure: on the runner's account (runneradmin), the temp directory's short (8.3) name and its realpath-
  resolved long name differ; checkTestRoot deliberately reports the resolved path, so four tests comparing it against
  the pre-resolution path they created the directory with failed there specifically. They now resolve their own
  expected path the same way before comparing.
- args.ts: a second --json is now rejected like every other duplicate flag, instead of silently accepted. --json is
  now decided by one scan of the whole invocation, independent of where it appears and of everything else wrong with
  it, so a malformed invocation that also asked for --json is still reported as JSON on stderr instead of silently
  downgrading to plain text.
- environment-commands.ts: runStatus now also catches checkTestRoot itself failing (it can throw, not just reject
  cleanly, if the root is removed between its own existence check and resolving the real path), closing the one gap
  in its "never throws" contract; test exercises it by mocking checkTestRoot rather than racing the real filesystem.
- Test fixture fix: the "machine matches profile" test hardcoded arch: 'x86_64', which fails on an arm64 host; now
  built from hostProfile() like the rest of the suite already does.
- Test quality: `expect(result.ok && result.report.X)` obscures the real error when result.ok is false (a confusing
  matcher error instead of the underlying message); replaced with an explicit ok check that throws the real error.
- README: the doctor example didn't run from a fresh checkout (qa/project.json doesn't exist yet); reworded so
  designate/status are shown as runnable now and doctor is clearly marked as needing a project-supplied file.

Mutation-checked the three behavioral fixes (duplicate --json, json preserved through a parse failure, runStatus
catching checkTestRoot) against a throwaway script; all three killed, files restored identical. Clean typecheck and
full suite: 518 passed, 2 skipped; no leaked processes, temp directories, or home markers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 existing issue remains and no new issues found across 8 files (changes from recent commits).

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Re-trigger cubic

Comment thread packages/qa/test/cli/environment-commands.test.ts Outdated
Comment thread packages/qa/src/cli/main.ts Outdated
…own the checkTestRoot mock in afterEach

- doctor's JSON error output always includes issues now (empty when there are none), rather than omitting the key
  for read/parse failures and only including it for schema failures: a consumer keying on its presence unconditionally
  no longer sees it vanish depending on which way loading failed.
- The checkTestRoot mock in the "root check itself failing" test is now torn down in an afterEach, not at the end of
  the test body, so a failed assertion cannot leave it registered and confusingly break a later test in the file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Andreas-Froyland
Andreas-Froyland merged commit 86bae90 into main Sep 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant