fix: make the soak fixture able to start the engine at all - #89
Conversation
The engine refuses to start under the soak fixture, and the refusal is correct.
Its boot-time safety liveness check reports
Dead safety/alarm channel pattern(s): 1 match NO channel on the plane their
consumer sees (F-1 silent safety kill).
- pattern='.*' source=safety.yaml critical_channels
`critical_channels` entries are EXACT canonical identities, never patterns:
_resolve_critical_bindings does `if channel_id not in storage_catalog.by_channel_id`.
The fixture declared the literal string ".*", meaning "everything", and no channel
has that identity, so the guard against a silently dead safety reference did
exactly what it exists to do.
The fixture's safety config is therefore derived from the roster the same call
just wrote, instead of being a static string that cannot know what channels
exist. The intent is unchanged -- every channel is critical -- and it can no
longer drift from the descriptors.
Measured on Ubuntu 22.04.5 by running the engine alone against a freshly
materialised isolated config, because the launcher discards engine stderr and
the reason was otherwise kept nowhere. Before: CONFIG ERROR at the safety
liveness check. The new test asserts every declared identity is on the roster,
that the set is the whole roster, and that no entry carries a wildcard; it fails
against master.
tests/scripts: 131 passed, 94 skipped.
…fication too Declaring the whole roster moved the refusal from one plane to the next rather than clearing it. A declared identity must ALSO be classified safety-critical by its own descriptor: the check requires safety_class is SAFETY_CRITICAL_INPUT and a role that is not SOURCE_READBACK. Measured: all sixteen descriptors of the passive fixture are observational, nine reference measurements and seven primary ones. There is no safety-critical input in this fixture, so the honest declaration is the empty one. It is derived rather than assumed. If a safety-critical descriptor is ever added to the passive instrument, it is declared automatically, and the fixture cannot quietly stop monitoring something it should monitor.
The test now asserts the rule rather than a number: every declared identity is on the roster, none carries a wildcard, and the set is exactly the descriptors classified safety_critical_input with a role other than source_readback. For this fixture that set is empty, and the test says why rather than hard-coding the emptiness, so adding a safety-critical descriptor later is a change the test follows instead of a change it blocks.
The fixture could not start the engine at all, and no edit to safety.yaml could change that. The engine refuses to start a SafetyManager with no critical channel, and refuses again when a declared channel is not classified safety-critical by its own descriptor. Every one of LS218_1's sixteen descriptors is observational, so the three candidate declarations each failed a different way: the literal ".*" is an identity nobody has; the whole roster is not safety-critical; the empty list is refused outright. LS218_2 is the same driver, lakeshore_218s, the same passive measurement authority, and the same reviewed cardinality of sixteen descriptors and sixteen bindings, and it carries two safety_critical_input channels. Nothing reviewed is loosened and no classification is fabricated. The fixture simply uses the tracked instrument that has what the startup path requires, which also means the soak now exercises the path production takes rather than one production never takes. Measured on Ubuntu 22.04.5 by running the engine alone against a freshly materialised isolated config, because the launcher discards engine stderr.
…rm it The fixture wrote a four-line static string, and the production loader refused it: it requires exactly cooldown, vacuum and landmarks, complete key sets in the first two, and the two canonical landmark channels with non-empty alias lists in the third. The engine stopped at PhysicalAlarmsConfigError: physical alarms document must contain exactly cooldown, vacuum, landmarks A short document cannot satisfy that, and inventing a long one would put a document the loader has never reviewed in front of it. The tracked document is taken instead and only the three arming flags are overridden, so the soak stays passive while the document stays the reviewed one. Worth noting what the loader asks for: landmarks must be exactly the two canonical channels that LS218_2 carries as safety-critical inputs. The startup path and the fixture's instrument now agree, which is the same finding from a second direction. tests/scripts: 132 passed, 94 skipped.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Review found a false statement presented as true in tracked code, and it was right. Two comments said every descriptor in this fixture is observational and the honest declaration is the empty one. That was true while the fixture used LS218_1 and stopped being true in the same change that switched it to LS218_2, which carries two safety_critical_input channels. The file's own header comment says so three hundred lines earlier, so the tree contradicted itself. Measured rather than reasoned: the materialiser writes critical_channels: [Т11, Т12]. The code and the test were already correct -- the test derives its expectation from the descriptors, so it would have failed on an empty declaration. Only the prose was wrong, and prose that a maintainer would act on is not a small thing: believing the soak monitors nothing safety-critical invites someone to force the list empty, which is exactly what the engine refuses to start on.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
These tests and the theme-pack tests were added to the same module on different branches, and the two branches then conflicted there every time they met. That contention is created by WHERE the tests were put, not by what they assert, and it blocked the combined measurement that shows what the soak does with both changes present. Splitting costs nothing. The new module states what it covers: the chain of refusals the engine raises before it will run -- an exact canonical identity in the critical-channel list, a descriptor that classifies that identity safety-critical, and a physical-alarms document the production loader accepts whole. tests/scripts: 135 passed, 94 skipped.
# Conflicts: # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The protected evidence gate refuses evidence from any run attempt after the first, so a cancelled round cannot be repaired by re-running it. This commit changes no content and exists only to produce the fresh run the gate requires.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…a literal Review found that switching the fixture to LS218_2 left the acceptance validator hard-coding LS218_1, so the soak would have run its full duration and then failed its own verdict step with "source fixture semantics or tree seal is invalid". Every manifest test stayed green because they all hand-build the payload, so the failure would have arrived as a red run with a green suite. The literal is gone from both sides. The validator reads the runner's constant through the lazy import this module already uses in three other places, and the test fixture reads it too, so there is no second place left to disagree. This is the same shape as the topology drift found on 86 one round earlier: the seal and the validator are two halves of one contract, and every test that hand-builds the payload is a test that cannot see them separate. tests/scripts: 138 passed, 94 skipped.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The red Linux job on this pull request is not this pull request's doing
Three facts place it outside this branch:
Why it fails at random on Linux, measuredThe window is Measured against the predictor directly, no GUI involved (
So on a loaded hosted runner — this suite took 17 minutes 53 seconds in that job — the tick arriving more than about half a minute late is enough, and that is a property of the machine's load, not of any branch. #90 is the fix for this, and it is worth saying exactly what it does and does not do: it widens the tolerance by one cadence, from about 30 seconds to about 60. It does not remove the knife edge; it moves it. That is still the right change, because the margin today is zero by construction. Nothing to do on this branch. It needs a fresh round once #92 unblocks the evidence gate. |
# Conflicts: # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # docs/CLAIM_CORRECTIONS.md # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
… by a near miss The reviewer found that the fixture's filter does not match the production gate it exists to satisfy, and that the test restated the same filter, so it could never detect the difference. Both are true and both were verified here against the production code rather than taken on trust. The engine builds its critical manifest from descriptors whose quantity is temperature and whose safety class is safety critical input, and refuses when the declared set is not exactly that set. It applies no role test. The fixture tested safety class and excluded the source readback role. That role test is a no operation: the descriptor validator refuses any source readback descriptor without the hazardous source readback class, so a channel can never be both source readback and safety critical input. And the quantity test, the one that decides, was missing. Measured on the tracked descriptors: both of the instrument's safety critical descriptors are temperature and primary measurement, so the two filters agree today and the role test changes nothing. The risk is drift. A safety critical descriptor of another quantity added later would be over declared, the engine's union check would fire, and the engine would refuse to start, while the fixture's own test still passed. The rule now lives in one named place, copied from production and citing it. The assertion states the engine's rule instead of the fixture's, and a second test exercises the case the first could never reach, with a synthetic descriptor set where a pressure channel and a raw sensor channel are both classified safety critical. Putting the old filter back makes that second test fail, which was confirmed by doing it and restoring the file.
The workflow-exact format check refused the previous commit for both files. The line length allows the joined form. The formatter was never run in mutating mode over the reviewed tree; its diff was read and applied by hand, which is what the registered guard requires.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 participant
The result first
With this branch, on the laboratory machine, the engine runs and stores data:
Thirty seconds of acquisition, 120 rows written, and a clean layered shutdown with every owner settled. Before this branch the engine did not start at all under the soak fixture.
Measured on Ubuntu 22.04.5, in a worktree cut from the native Linux clone, by running the engine alone against a freshly materialised isolated config — because the launcher discards engine stderr, so the reasons below were otherwise kept nowhere (
evidence/tools/engine_probe.sh).Three fixture defects, each hidden behind the previous one
1.
critical_channelsheld a pattern where identities are required._resolve_critical_bindingsdoesif channel_id not in storage_catalog.by_channel_id, so the literal string.*is a channel nobody has. The F-1 silent-safety-kill guard was doing exactly its job.2. No declaration could fix it, because the fixture had nothing safety-critical. Declaring the whole roster moved the refusal to the next plane — a declared identity must ALSO be classified
SAFETY_CRITICAL_INPUTby its own descriptor, with a role other thanSOURCE_READBACK. All sixteen ofLS218_1's descriptors are observational. And the empty list is refused outright: "refusing to start SafetyManager without critical channel monitoring".So the fixture is switched to
LS218_2: the same driverlakeshore_218s, the same passive measurement authority, the same reviewed cardinality of sixteen descriptors and sixteen bindings, and it carries twosafety_critical_inputchannels. Nothing reviewed is loosened and no classification is fabricated — the fixture simply uses the tracked instrument that has what the startup path requires. The declaration itself is now derived from the descriptors, so a safety-critical channel added later is declared automatically rather than silently unmonitored.3. The physical-alarms document was a four-line string where a complete one is required.
The loader wants exactly those three keys, complete key sets in the first two, and the two canonical landmark channels with non-empty alias lists in the third. The tracked document is taken instead and only the three arming flags are overridden, so the soak stays passive while the document stays the reviewed one.
A detail worth pinning: the loader requires landmarks to be exactly the two canonical channels that
LS218_2carries as safety-critical. The startup path and the fixture's instrument now agree, which is the same finding arriving from a second direction.What this does not claim
The short soak still fails at the four-role startup cut. That is now about the launcher and bridge orchestration, not about the engine — the engine phase is past. Each barrier here was only visible once the one in front of it was gone, so the next one is the next measurement, not a prediction.
It also does not claim anything about the lab's real configuration. This is the soak's own fixture, which had apparently never started the engine.
Evidence
tests/scripts: 132 passed, 94 skipped.ruff check src/ tests/clean; the workflow-exact format check clean over 702 changed files. The three new tests fail against master.Depends on #87 and #86 to reach this far; measured with all three merged locally and never pushed (
evidence/tools/combine_and_soak.sh).Written with assistance from Claude (Anthropic).