Skip to content

[fix] - Give the installer contract tests headroom and fail fast on prompts - #1727

Merged
cgfixit merged 2 commits into
mainfrom
claude/windows-installer-test-timeout
Oct 20, 2026
Merged

cgfixit merged 2 commits into
mainfrom
claude/windows-installer-test-timeout

Conversation

@cgfixit

@cgfixit cgfixit commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Proposed changes

tests/test_installer_python_contract.py::test_powershell_probes_exact_py312_launcher_and_rejects_313 hit its 60s subprocess.TimeoutExpired on the windows-latest leg of #1722 (run 39416581582). #1722 only touched utils/health.py. Main's Windows leg was green on the same tree, and the one re-run passed. The failure showed nothing useful, because TimeoutExpired discards the installer's partial output, so there was no way to see where it stalled.

Both installer runners, _run_macos_installer and _run_powershell_installer, now go through one helper, _run_installer:

Change Effect
_INSTALLER_TIMEOUT_SEC = 180 (was 60, in two places) headroom for Windows PowerShell 5.1 start-up and module autoload on a loaded runner
stdin=subprocess.DEVNULL anything that reads stdin gets EOF at once instead of waiting
-NonInteractive on powershell.exe a prompt (Read-Host etc.) errors immediately instead of hanging until the timeout
On TimeoutExpired, raise AssertionError(...) from exc carrying the last 4,000 chars of stdout and stderr the next timeout shows the last thing the installer printed

No assertion changed, and no test was skipped or loosened. I grepped Install-CyClaw.ps1 and CyClaw-SecretStore.ps1 and found no prompts in the install path, so -NonInteractive changes no behaviour these tests check.

The second commit (b685469) answers CodeQL py/mixed-returns on this PR. The except branch first ended in pytest.fail(), which never returns but which CodeQL reads as an implicit None. It now raises explicitly, with the same message.

Root cause, stated plainly: I cannot run Windows PowerShell here, so "slow PowerShell start-up on a loaded hosted runner" is an inference, not a measurement. Every fake command in these tests answers at once, which points at host start-up rather than the installer logic. If it times out again at 180s, the new failure message will show where it stopped.

Invariant / Governance Impact: none. Test-only change.

Types of changes

  • Refactor or tests (behavior-preserving, no new capability)

Scope: Infrastructure / CI only.

Benefits / why

Risks to monitor

  • A real regression that makes the installer slow but not hung would now pass for up to 180s instead of failing at 60s. The four PowerShell tests run one after another, so the worst case adds about 8 minutes to a failing Windows leg.
  • Signal: the Windows-leg duration for this file. An AssertionError: installer exceeded 180s ... means a real stall, and the stdout tail shows where it stopped.

Checklist

  • I have read docs/THREAT_MODEL.md, INVARIANTS.md and SECURITY.md (and any relevant Phase docs)
  • This change preserves all 6 security invariants and I6 module isolation (test-only)
  • Full sandbox validation has been run (per CLAUDE.md §5, left to CI)
  • No new external network dependencies or mandatory online LLM assumptions were introduced
  • Commit messages follow the title prefix convention above

Further comments

Run directly (Linux, Python 3.12):

  • ruff check and ruff format --check on the file: clean.
  • The file under pytest: the 4 bash-installer tests pass and the 4 PowerShell tests skip (no powershell.exe here).
  • _run_installer called directly:
    • with the timeout forced to 1s around echo started; sleep 5, it raised AssertionError with __cause__ set to TimeoutExpired and the message installer exceeded 1s | --- stdout --- | started;
    • read x || echo stdin-closed printed stdin-closed.

Only CI can run: the four PowerShell tests on windows-latest. That leg is the real test of this PR.

Suggested merge order of open PRs

Trial merges verified (git merge --no-commit --no-ff + abort, each on its own) against main @ cdcef76 on 2026-10-20.

#1726 · Blocked (draft)
Impact: .github/dependabot.yml only.
Action: Independent; either order. Clean.

#1727 (this PR) · Blocked (draft)
Impact: installer contract test timeout and diagnostics.
Action: Merge once windows-latest is green. Clean.

ELI5

A Windows test runs CyClaw's PowerShell installer against fake Python programs that all answer instantly, and it gave the whole thing 60 seconds. On a busy Windows CI machine PowerShell itself can be slow to start, and once it ran out of time on a PR that had nothing to do with installers. Worse, when it timed out it threw away everything the installer had printed, so nobody could see where it got stuck.

Now tests/test_installer_python_contract.py gives it 180 seconds. It also tells PowerShell never to wait for a person to type (-NonInteractive, plus closed input), so a prompt fails at once instead of hanging. A timeout now shows the installer's last output.

  • What it enhances: fewer false reds, and a readable message when one is real.
  • What it makes obsolete: two copy-pasted subprocess.run blocks.
  • What it retrieves: nothing.
  • What it flips into when pushed too far: a slow-but-working regression stays hidden longer before it fails.

I ran the Linux half and the helper itself. Only Windows CI can run the PowerShell half.

🤖 Generated with Claude Code

https://claude.ai/code/session_014FZ8JTF2v1eQd1L5CKQR8j

Last updated: 2026-10-20 09:45 ET

…rompts

test_powershell_probes_exact_py312_launcher_and_rejects_313 hit its 60s
subprocess timeout on #1722's windows-latest leg, a PR that never touched
the installers; the re-run passed. Every fake command in these tests
answers at once, so the time goes to Windows PowerShell start-up on a
loaded runner.

Both installer runners now share _run_installer: a 180s timeout, stdin
closed, -NonInteractive on powershell.exe so a prompt errors instead of
hanging, and a pytest.fail that shows the installer's partial output when
the timeout does fire.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FZ8JTF2v1eQd1L5CKQR8j
Comment thread tests/test_installer_python_contract.py Fixed
… explicit

CodeQL (py/mixed-returns) flagged _run_installer: the except branch ended
in pytest.fail(), which never returns but reads as an implicit None.
Raise AssertionError from the TimeoutExpired instead; the message and the
partial stdout/stderr are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FZ8JTF2v1eQd1L5CKQR8j
@cgfixit
cgfixit marked this pull request as ready for review October 20, 2026 14:05
@cgfixit
cgfixit merged commit 57ac5a4 into main Oct 20, 2026
64 checks passed
@cgfixit
cgfixit deleted the claude/windows-installer-test-timeout branch October 20, 2026 14:05
cgfixit pushed a commit that referenced this pull request Oct 20, 2026
…c removal and extras drop

Doc pass against main @ cdcef76 with the two open PRs (#1726, #1727) treated
as merged; the deterministic checkers (doc-sync, dep-guard, env drift, lock
contract, invariant-guard) were already green, so every change is prose.

- README: the agentic loop no longer appears in the ship-off list, the
  security-model row or "Current development"; the docs map stops crediting
  setup-guide.md with Docker/conda and gains locks/README.md.
- setup-guide.md: the self-install line is `pip install -e . -c
  constraints.txt` on Linux/Windows and `pip install -e . --no-deps` on
  macOS (the old `/tmp/constraints-macos.txt` no longer exists, and the
  install goes through the hashed locks, not requirements.txt); the lock
  contract covers 19 locks, not 25; "step 4b" is step 4; the stranded
  lock-freshness section sits above the footer; reconciled date 2026-10-20.
- locks/README.md: records that constraints.txt and requirements*.txt are
  hashed byte-for-byte into the lock records, and what a Dependabot pip PR
  is under the #1726 config.
- docs/DOCKER.md, macos/README.md, powershell/README.md, AGENTS.md,
  CLAUDE.md section 8: stale agentic-registry/agentic-writes wording, the
  verify-deps name, and the macOS note on the editable install.
- docs/audits/2026-10-20-dependency-currency-sweep.md: PyPI currency and
  advisory sweep over all 42 pins. No pin changed: the hashed locks
  regenerate only on arm64 macOS and the contract hashes the manifests, so
  bump candidates are listed for the owner. 0 new advisories.

Checks run directly: doc_sync.py, check_deps.py, check_env_drift.py,
check_lock_contract.py, check_invariants.py (all 0 failures); `pip install
--dry-run --no-deps -e .` in a 3.12 venv. CI only: everything else.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0196C68GFwq9cXGmpqDMDGJe
cgfixit added a commit that referenced this pull request Oct 20, 2026
…c removal and extras drop (#1731)

## Proposed changes
Documentation and dependency resync against `main` @ `b0683515` (after
#1726 and #1727 merged). The five deterministic checkers (doc-sync
D1-D13, dep-guard D1-D11, env drift E1-E7, `check_lock_contract.py`,
invariant-guard) were already green, so every change here is prose plus
one dated audit. **No pin, lock, manifest, or code line changes.**

- `README.md`: the removed agentic coding loop no longer appears in the
ship-off list, the security-model "Optional surfaces" row, or "Current
development"; the docs map stops crediting `setup-guide.md` with
Docker/conda and gains a `locks/README.md` row.
- `setup-guide.md`: the editable self-install is `pip install -e . -c
constraints.txt` on Linux/Windows and `pip install -e . --no-deps` on
macOS. The old text pointed at `/tmp/constraints-macos.txt` (exists
nowhere in the repo) and said the install went through
`requirements.txt` (it goes through the hashed locks). Lock count 25 ->
19 (six locks left with #1700). "step 4b" -> step 4 (no 4b exists). The
lock-freshness section was stranded after the footer and now sits above
it. Reconciled date 2026-10-20. Option C step 5 notes `GH_TOKEN` is
optional and unread by the runtime.
- `locks/README.md`: records that `constraints.txt`, `requirements*.txt`
and `scripts/ci-lock-inputs/` are hashed byte-for-byte into the lock
records (a comment edit alone fails the contract), and a Dependabot
section matching #1726.
- `docs/DOCKER.md`, `macos/README.md`, `powershell/README.md`: "agentic
registry" / "agentic writes" / "Agentic / registry" wording replaced
with what exists (model caches, fsconnect writes, connectors).
- `AGENTS.md`: `verify-dep` -> `verify-deps`. `CLAUDE.md` §8: one
trailing comment on the macOS editable install. Both files steer agent
sessions; the diff is those two lines, nothing copied from any external
source.
- `docs/audits/2026-10-20-dependency-currency-sweep.md` (new) and a
`docs/changelog.txt` row: PyPI currency + per-release advisory sweep
over all 42 pins. 0 new advisories (chromadb and nltk remain the two
accepted in `SECURITY.md`). Bump candidates are grouped for the owner;
the stale `constraints.txt` pydantic comment is flagged but left alone
because fixing it would re-hash the file.

**Invariant / Governance Impact:** none. No core file, graph edge,
config value, sanitizer pattern, auth path, or manifest changed.
`check_invariants.py`: 46 passed, 0 failed on the pushed head.

---

## Types of changes
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Security (auth, privacy, injection, egress, redaction, or secrets)
- [ ] Invariant / Governance refinement (use this for changes that
strengthen or evolve the 6 invariants, I6 isolation, or harness phases)
- [ ] Dependencies (added, removed, or bumped)
- [ ] CI / infrastructure (workflows, hooks, templates, or checkers)
- [ ] Refactor or tests (behavior-preserving, no new capability)
- [x] Documentation update
- [ ] Other (none of the boxes above fit)

Scope note: Docs + audits.

---

## Benefits / why
- An operator following `setup-guide.md` on macOS was told to pass a
constraints file that does not exist; the new line (`--no-deps`) was run
and resolves.
- README, Docker and installer docs stop describing a subsystem removed
two days ago (#1699/#1700), so a reader cannot go looking for the
agentic loop, registry, or `/ops/agentic`.
- `locks/README.md` now states the one rule that bites a would-be
contributor: the manifests are hashed, so a pin or comment edit needs
the Mac re-lock. Dependabot's role (#1726) is documented where the locks
live.
- The currency sweep gives the owner a reviewed bump list without
landing a red lock contract.

---

## Risks to monitor
- Prose only; the only executable claim is the macOS `pip install -e .
--no-deps` line. Signal if wrong: `cyclaw-*` shims missing after that
command on a Mac. Verified on Linux 3.12 only (dry-run, "Would install
cyclaw-1.9.0"); not run on macOS.
- `doc_sync.py` D9/D10 would catch a broken link or anchor from the
moved section; it reports 0 drift on the head.
- The sweep is a point-in-time PyPI read (2026-10-20); it is not a pin
change and goes stale like every dated audit.

---

## Checklist
- [x] I have read `docs/THREAT_MODEL.md`, `INVARIANTS.md` and
`SECURITY.md` (and any relevant Phase docs)
- [x] This change preserves all 6 security invariants and I6 module
isolation (explicit evidence or invariant matrix included for core
changes)
- [ ] Full sandbox validation has been run (docs-only; per CLAUDE.md §5
the suites are CI's to run)
- [x] No new external network dependencies or mandatory online LLM
assumptions were introduced without explicit justification + offline
fallback path
- [ ] For any agentic/fsconnect change: n/a
- [x] Relevant architecture docs or threat model notes have been updated
if core behavior or topology changed
- [x] Commit messages follow the title prefix convention above
- [ ] For large or complex changes: n/a

---

## Further comments
Checks run directly on the pushed head: `doc_sync.py` (0 drift),
`check_deps.py` (0/0), `check_env_drift.py` (0/0),
`check_lock_contract.py` (passed), `check_invariants.py` (46/0), a
lock-vs-constraints agreement pass over 148 shared entries, and `pip
install --dry-run --no-deps -e .` in a Python 3.12 venv. CI only: ruff,
the four test legs, coverage, pr-template-check.

---

## Suggested merge order of open PRs
_Trial merges verified against main @ b068351 on 2026-10-20: this
branch is rebased directly onto that SHA (fast-forward, nothing to
merge)._

**#1728** · Blocked
Impact: Dependabot Docker base 3.12 -> 3.15; `requires-python <3.13`, so
this cannot merge as proposed.
Action: Close or re-pin; independent of this PR.

**#1729** · Blocked
Impact: Dependabot pip minor+patch, 16 packages; red on the lock
contract until the Mac re-lock (see `locks/README.md` Dependabot section
in this PR).
Action: Owner re-locks on arm64 macOS, then merge; does not touch this
PR's files.

**this PR** · Safe
Impact: Docs and one audit only; no shared file with #1728/#1729.
Action: Merge any time; squash.

## ELI5
The code moved on (the agentic coding loop and its extras were deleted
last week, the lock set shrank, Dependabot got new rules) and the
instruction manuals did not. This PR is the manual catching up; it
changes no behavior.

- **What was broken or missing:** like a museum map still showing a wing
that was demolished. `README.md`, `docs/DOCKER.md` and the
macOS/PowerShell READMEs still named the agentic loop, its registry and
its writes. `setup-guide.md` told Mac users to pass `-c
/tmp/constraints-macos.txt` (a file nothing creates), said the install
went through `requirements.txt` (it goes through `locks/`), counted 25
locks (there are 19), referenced a "step 4b" that does not exist, and
had its lock-freshness section sitting after the signature line.
- **What changed:** those sentences now say what the tree does. The
macOS self-install is `pip install -e . --no-deps`, because
`constraints.txt` pins `torch==2.13.0+cpu` and no such wheel exists for
macOS. `locks/README.md` gains two facts: the manifests are hashed into
the lock records byte-for-byte, and a Dependabot pip PR is a shopping
list until someone re-locks on the Mac. A new dated audit lists every
pin against PyPI today with advisories checked per pinned release; two
advisories exist (chromadb, nltk) and both were already accepted in
`SECURITY.md`. Deliberately not done: no pin bumped, no
`constraints.txt` comment fixed, because either one turns
`check_lock_contract.py` red until the arm64 re-lock.
- **What could still go wrong:** the only runnable claim is the macOS
`--no-deps` line. It was dry-run on Linux 3.12 and resolved; it was not
run on a Mac. If it is wrong, `cyclaw-user` and friends stay "command
not found" after it. Everything else is words; `doc_sync.py` D9/D10
guard the links and anchors and report 0 drift.

Last updated: 2026-10-20 10:12 ET

---
_Generated by [Claude
Code](https://claude.ai/code/session_0196C68GFwq9cXGmpqDMDGJe)_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants