Skip to content

guard: revive the silently-skipped toolchain health check + fix sibling-import violations - #756

Open
matt82198 wants to merge 1 commit into
mainfrom
guard/fix-dead-toolchain-import
Open

matt82198 wants to merge 1 commit into
mainfrom
guard/fix-dead-toolchain-import

Conversation

@matt82198

Copy link
Copy Markdown
Owner

Two findings from the #746 lane, both in the same class: a check that silently never runs.

Finding 1 -- the toolchain health check was dead from the day it was written

tools/toolchain_health.py imported StateReadAPI from state_store.read_api. That
symbol has never existed -- the class is ReadAPI. So the import raised on every
invocation and fell into the except ImportError: ... = None fallback, and
check_heartbeat() returned early with "Heartbeat <name> check unavailable (StateAPI not loaded)". Both heartbeat checks have been permanently reported-unavailable rather than
evaluated for the entire life of the file.

Fixing the symbol exposed two more defects behind it:

  1. state_store was not importable. Run by file path (python tools/toolchain_health.py),
    sys.path[0] is tools/, so state_store is invisible regardless of the symbol name.
    The tool now puts the repo root on sys.path (the same idiom tracker_autoclose.py
    and gen_state_md.py use).
  2. The facade was constructed with the wrong arity. ReadAPI.__init__(self, state_dir)
    requires the state directory; the call site was StateReadAPI(). It now passes
    get_state_dir() (AESOP_STATE_ROOT, else ./state) -- the repo-wide convention.

With the import actually working, what does it report?

It reports real problems that were hidden. Not a regression -- a finding.

$ AESOP_STATE_ROOT=~/aesop/state python tools/toolchain_health.py --json
{
  "status": "FAIL",
  "findings": [
    { "type": "HEARTBEAT_STALE", "file": ".watchdog-heartbeat",
      "message": "Heartbeat watchdog stale or missing" },
    { "type": "HEARTBEAT_STALE", "file": ".monitor-heartbeat",
      "message": "Heartbeat monitor stale or missing" }
  ],
  "summary": "2 issue(s) found",
  "checks_performed": 7
}

All 5 binary checks (bash, git, python, node, curl) pass. Both heartbeat checks
fail, and the failures are genuine:

  • aesop/state/.watchdog-heartbeat contains epoch 1784681587 -- 2026-07-21, roughly
    12 days stale against a 300s threshold.
  • aesop/state/.monitor-heartbeat does not exist at all.

The live heartbeat is in a different state root (conductor3/state/.watchdog-heartbeat,
fresh), which is exactly the kind of split the tool exists to notice. Relative to aesop's
own AESOP_STATE_ROOT, the daemons look dead -- and until this PR nothing could say so.

These findings are deliberately NOT silenced. toolchain_health.py is not wired into
any CI workflow, the pre-push hook, package.json, or daemons/ (verified), so its
non-zero exit blocks nothing. Deciding whether the fix is "point aesop's state root at the
live heartbeats" or "the daemons should also beat into the repo state dir" is a follow-up
that belongs to whoever owns the daemon topology, not to this lane.

Finding 2 -- sibling_import_check was red on main

$ python tools/sibling_import_check.py --check     # BEFORE (origin/main)
Sibling import check: 3 violation(s) found
  tools/merge_queue.py:70: from merge_train import ...
  tools/merge_queue.py:71: from common import ...
  tools/merge_queue.py:72: from generated_paths import ...
  -> exit 1

$ python tools/sibling_import_check.py --check     # AFTER
Sibling import check: CLEAN
  -> exit 0

merge_queue.py was already guarded -- but with the conditional form:

if str(_TOOLS_DIR) not in sys.path:
    sys.path.insert(0, str(_TOOLS_DIR))

That guards identically at runtime, but SiblingImportAnalyzer.visit_Module only scans
top-level statements for a bare ast.Expr sys.path.insert(...) call, so a guard nested
inside an if is invisible to it. Switched to the repo's sanctioned unconditional idiom --
the same form as auto_merge.py:29 and tracker_autoclose.py:53. A module body executes
once, so an unconditional insert cannot accumulate duplicate entries.

The checker itself is untouched: not weakened, not exempted, no suppression comment.
The diff is one hunk in the import block only -- no REGENERATORS or logic changes, so it
does not collide with the concurrent guard/register-index-generated lane.

Tests (TDD -- both sets proven RED against the pre-fix code)

tests/test_toolchain_health.py::TestStateAPIImportIsLive (4 new tests):

  • every symbol imported from state_store.read_api must actually exist in that module
    (AST-parsed from the source, so any future rename is caught the same way)
  • the module global must be the real facade class, not the None fallback
  • run_checks must hand check_heartbeat a non-None state_api -- i.e. the check is
    live, not skipped
  • the facade exposes the check_heartbeat_fresh method the check calls

All 4 fail against the original file:

FAILED ::test_facade_exposes_the_method_the_check_calls
FAILED ::test_facade_is_bound_not_none
FAILED ::test_imported_state_store_symbols_exist
FAILED ::test_run_checks_passes_a_live_state_api_to_heartbeat
E  AssertionError: check_heartbeat received state_api=None: the check is skipped, not live
4 failed, 15 deselected

tests/test_sibling_import_check.py::TestRepoToolsTreeIsClean (2 new tests) -- the real
tools/ tree scans clean, and merge_queue.py's transport imports read as guarded. Both
fail against origin/main's merge_queue.py:

FAILED ::test_merge_queue_transport_imports_are_guarded
FAILED ::test_repo_tools_directory_has_no_unguarded_sibling_imports
2 failed, 11 deselected

A setUpModule pins AESOP_STATE_ROOT to a temp dir so building a real ReadAPI (which
mkdirs its state directory) can never create ./state in the caller's cwd -- test hygiene.

Verification

  • tools/sibling_import_check.py --check -- exit 0 (was 1)
  • tools/toolchain_health.py -- exit 1 with the two real heartbeat findings above (was a
    vacuous "unavailable" on a dead import)
  • Python suite: all 4 shards green, 1551 tests, OK (skipped=3)
  • Node suite (npm run test:node): green
  • Targeted: test_toolchain_health + test_sibling_import_check + test_merge_queue =
    172 passed (was 166 -- 6 new)
  • Gates green: secret_scan --staged, sibling_import_check, claudemd_lint,
    claudemd_sync_gate, encoding_lint, watcher_linter, spec_contract_validator,
    workflow_model_linter, import_cycle_check, verify_test_suite_count,
    claudemd_contract, metrics_gate, agent_prompt_hygiene
  • dispatch_lint, subprocess_guard, commit_lint, file_size_lint, docstring_check
    are non-zero on origin/main too (verified against a clean detached checkout) --
    pre-existing, untouched by this diff
  • Full pre-push hook battery passed on push

A third instance of the same class, reported not fixed

tools/import_resolution_check.py reports merge_queue.py's three bare sibling imports as
unresolvable -- identically before and after this PR (reproduced on a clean origin/main
checkout with merge_queue.py staged), since the diff neither adds nor removes an import.
It is wired into hooks/pre-push-policy.sh as fail-closed, but it reads
git diff --cached --name-only -- the staging index, which is empty at push time:

$ python tools/import_resolution_check.py    # post-commit, i.e. what the hook sees
No staged Python files found.
-> exit 0

So a fail-closed pre-push gate is a no-op on every normal push. That is the same failure
mode as findings 1 and 2 and deserves its own lane; fixing it here would mean changing a
gate's rules, which is outside this lane's ownership.

Generated with Claude Code

…ng-import violations

Two instances of the same class: a check that silently never runs.

1. tools/toolchain_health.py imported `StateReadAPI` from state_store.read_api,
   but the class there is `ReadAPI`. The import raised on every single run and
   fell into the `= None` fallback, so BOTH heartbeat checks reported
   "unavailable" instead of being evaluated -- dead since the day it was
   written. It also never put the repo root on sys.path, so `state_store` was
   invisible when run by file path, and it called the facade with no args when
   ReadAPI requires a state_dir. Fixed all three; the facade is now built with
   get_state_dir() (AESOP_STATE_ROOT, else ./state).

2. `python tools/sibling_import_check.py --check` was RED on main with 3
   violations, all in tools/merge_queue.py. Its guard used the conditional
   `if str(d) not in sys.path:` form, which guards identically at runtime but
   is invisible to the checker (it only recognizes a top-level
   sys.path.insert statement). Switched to the repo's sanctioned unconditional
   idiom -- same form as auto_merge.py and tracker_autoclose.py. The checker
   is unchanged: not weakened, not exempted.

Tests (TDD, both proven RED against the pre-fix code):
  - TestStateAPIImportIsLive: every symbol imported from state_store.read_api
    must exist in it, the module global must not be the None fallback, and
    run_checks must hand check_heartbeat a non-None state_api (the check is
    live, not skipped). 4 tests, all 4 fail on the old import.
  - TestRepoToolsTreeIsClean: the real tools/ tree scans clean, and
    merge_queue.py's transport imports read as guarded. 2 tests, both fail
    against origin/main's merge_queue.py.
  - setUpModule pins AESOP_STATE_ROOT to a temp dir so building a real ReadAPI
    cannot mkdir ./state in the caller's cwd.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@matt82198 matt82198 added the merge-queue Queued for the merge-queue advancer daemon label Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-queue Queued for the merge-queue advancer daemon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant