fix: the bridge belongs to this launcher even when a fork server made it - #91
Conversation
"reported bridge is not a direct launcher child" says a condition and withholds everything needed to act on it: which process, whose parent it reports, and what was expected instead. The only way to learn those numbers is to add them and run again, so the message costs a full measurement round every time it fires. It fired today. With the three soak fixes present the whole stack starts on the laboratory machine -- twelve of twelve fonts, the ZeroMQ bridge subprocess, the engine with readiness established on attempt two of ten, and the assistant -- and the run is then destroyed by this refusal with no way to see whether the parent is off by a fork, a re-exec, or a restart. All three refusals of this shape now carry the observed pid, the parent it reports, and the launcher pid expected. Nothing else changes: the same conditions refuse in the same places. tests/scripts: 136 passed, 94 skipped.
Python 3.14 makes forkserver the Linux default and the laboratory target is 3.14.6, so a multiprocessing child is forked from a fork-server process and is NOT a direct child of the launcher. The repository already documents the change in zmq_client.py; the soak runner still assumed the older topology and destroyed every run at the four-role cut. Measured on that machine, once the refusal was made to name its subject: bridge pid 821 reports parent 820, launcher is 793. The property this check exists to prove is that the bridge belongs to THIS launcher. A two-step chain proves exactly that, but only when the intermediate is BOTH a direct child of the launcher AND recognisably the fork server, so both halves are checked and neither is inferred from the other. Anything looser would accept a grandchild of an unrelated shape. The binder cannot look at processes, so the OBSERVER records its verdict and the binder reads it instead of re-deriving parentage from a pid that no longer decides the question. The field defaults to False, so a construction that forgets it refuses rather than passes, and the test asserts exactly that case alongside the fork-server child it now accepts. tests/scripts: 136 passed, 94 skipped.
The bare sentence sends the next turn to read the whole startup path. It has: the run gets further after every fix and stops here again, each time for a different reason the message did not say. Four things must be present and the message named none of them. Now it names the ones that are still absent, so the next measurement starts from the answer rather than from the search.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…it proved one Review found that my own change removed a defence-in-depth leg, and it was right. The binder used to re-verify parentage against the recorded launcher pid independently of the observer. Replacing that with a bare boolean left it unable to tell WHICH launcher the observer had proved, so its error message named a pid that had taken no part in the decision, and a second call site that passed different pids to the handshake parser and to the observer would have been accepted. Today the two coincide -- the single call site passes the same pid to both -- so this was latent rather than exploitable. Latent is not a reason to leave it: the next step after accepting a wrong bridge is signalling it, and SIGTERM to another experiment's process is data loss. The observation now carries the launcher pid it was verified against, zero meaning nothing was proved, and the binder re-checks that against the record. The new test case is a bridge proved against a DIFFERENT launcher, which a boolean could not tell apart from the accepted case. Also tightened, on the reviewer's note rather than as a finding: the fork-server is recognised by the exact module it runs, not by a substring any argument could carry. No reachable false positive existed either way, but a heuristic inside an identity check is still a heuristic inside an identity check. tests/scripts: 136 passed, 94 skipped.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…rvives The observer accepted the fork-server chain and the run still stopped, because classify_tree carries the same direct-child assumption and refuses twice over under forkserver: the bridge is no longer a direct child, and the fork server itself is a descendant whose argv matches no role, so it read as an unclassified process. The property the function proves is unchanged. Every live descendant is still accounted for and still has to belong to this launcher; the fork server is accounted for BY NAME rather than excused, so an unexpected descendant refuses exactly as before. The caller no longer swallows the classifier's reason. It caught ValueError and discarded the message, so the run reported only that roles were missing -- the one place the reason could have been read threw it away, and finding this took a source read instead of a log line. The refusal now carries it, and the unclassified-process message names the pid and the head of its argv. tests/scripts: 136 passed, 94 skipped.
…t names them Accepting only the fork server left the run refusing on unclassified descendant process: pid 801 argv ['python', '-B', '-s', '-c'] which is the resource tracker. multiprocessing starts two helper processes, not one, and both are launched with -c and name their module inside that code string. Both are now accounted for BY NAME. Excusing every -c child would have been the easy version and the wrong one: an unexpected descendant must still refuse, which is the whole point of classifying the tree. The refusal also stopped truncating its evidence. It printed the first four argv elements, and the module name that identifies these processes is the fifth, so the message showed exactly the part that carries no information. It now names the parent as well and keeps each argument up to a bound. tests/scripts: 136 passed, 94 skipped.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The run now produces endurance samplesWith this branch and #87 + #86 + #89 merged locally, the short soak on the laboratory machine gets past the four-role cut and samples for 185 seconds, 38 records, each carrying per-role resident memory, open descriptors and thread counts: {"elapsed_s": 185.0, "roles": {
"assistant": {"pid": 926, "rss_bytes": 86073344, "descriptors": 26, "threads": 7, "epoch": 0},
"bridge": {"pid": 852, "rss_bytes": 213196800, "descriptors": 29, "threads": 34, "epoch": 0},
"engine": {...}}}That is the evidence the laboratory week actually needs — memory, descriptors and threads over time — and it is the first time the run has produced any of it. It also injected a bridge fault and observed recovery: What this branch had to learn, in orderEach barrier was invisible until the one in front of it was gone, and each refusal had to be made to name its subject first.
The refusal that reported this had been truncating its own evidence to the first four argv elements, and the module name that identifies these processes is the fifth — so it printed exactly the part carrying no information. Where it stops nowThe bridge recovered; this is a different child. That lands on a decision already on the owner's list — a faulted engine is deliberately never replaced and stays in a permanent HOLD — so it is not being changed here. Evidence
Written with assistance from Claude (Anthropic). |
"faulted child did not recover within the reviewed ceiling" leaves the reader to guess between three roles that mean three different things. A bridge that cannot recover is a defect. An engine that does not is a deliberate permanent HOLD and an owner decision. An assistant is a third case again. The run reached this refusal today after the bridge had already been faulted and replaced successfully, so the bare message could not even say whether the earlier recovery counted. Naming the role is one interpolation and removes the guess.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The protected evidence gate refuses evidence from any run attempt after the first, so a cancelled round cannot be repaired by re-running it. This commit changes no content and exists only to produce the fresh run the gate requires.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The reviewer found three things at the previous head and all three were right. Measuring the platform made one of them larger than it looked. Measured on the laboratory interpreter, Python 3.14.6: a process forked BY the fork server inherits the fork server's command line EXACTLY. The fork server and the child it forks print byte-identical argv, because forking does not replace the command line. So the token multiprocessing.forkserver appears in the argv of the fork server AND of every child it makes, the bridge included. That makes the classifier's infrastructure set wrong in a way its own comment denied. It matched on the token alone, so every multiprocessing child was excused from role classification by name, and the promise that an unexpected descendant still refuses was false for exactly the descendants the check exists to catch. Only the parent separates them: the fork server and the resource tracker are direct children of the launcher, their children are not. The set now requires a direct child of the root, which is the same requirement the runner's own check makes, so the two implementations prove the same property instead of one proving a weaker one. Two comments claimed more than their code did, and both are corrected rather than defended. The runner said it matched the exact module and not a substring; it does match a substring, and it cannot do otherwise, because the token lives inside a -c code string with no argument equal to the module name. The check is sound because of the parent test above it, and the comment now says so. The observation field's comment implied that carrying the launcher pid restored an independent parentage re-check in the binder. It does not: the binder cannot look at processes, and the observer already refuses when it proves nothing, so for any observation the observer returns the comparison holds by construction. Its real value is fail-closed, against an observation built by hand or by a future second call site that verified nothing. The comment now says that. The relaxation was untested and now is not. The new tests live in a sibling module and are built from the measured topology, including the inherited command line, rather than from a shape invented to suit the check. Two of them fail when the direct-child requirement is removed from the production file, which was confirmed by removing it and restoring it, so they are load-bearing rather than decoration. An earlier draft of them passed for the wrong reason: the tree was missing the engine and the assistant, so three refusal tests raised a missing-role error and never reached the check they claimed to exercise. Every refusal now names the reason it must fire for.
…odule Seven hundred and three, measured at the merged staged index as a set difference in both directions rather than by arithmetic. Master is seven hundred and two, one path entered, and none left. The path that entered is the sibling module that tests the fork-server topology against the command lines the platform really produces.
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # docs/CLAIM_CORRECTIONS.md # docs/architecture-montana-important.svg # docs/current_candidate_metrics.md
|
@codex review Head is |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 participant
A platform assumption that stopped being true
The soak runner requires the ZeroMQ bridge to be a direct child of the launcher. On the laboratory machine it is not, and the run is destroyed at the four-role cut after the whole stack has already started.
Measured there, once the refusal was made to name its subject:
The repository already documents why, in
src/cryodaq/gui/zmq_client.py:511:forkserveris the Linux default from Python 3.14, and the laboratory runs 3.14.6. A multiprocessing child is forked from the fork-server process, so its parent is that server, not the launcher. The direct-child rule was true underfork, the previous default, and silently stopped being true.What the check is for, and what replaces it
The property it exists to prove is that the bridge belongs to this launcher. A two-step chain proves exactly that — but only when the intermediate is both a direct child of the launcher and recognisably the fork server. Both halves are checked and neither is inferred from the other; anything looser would accept a grandchild of an unrelated shape.
The binder cannot look at processes, so the observer records its verdict and the binder reads it instead of re-deriving parentage from a pid that no longer decides the question. The new field defaults to
False, so a construction that forgets it refuses rather than passes, and the test asserts that case alongside the fork-server child it now accepts.Three refusals that named a condition without its subject
reported bridge is not a direct launcher childgave no pid, no parent, no expectation — so the only way to learn them was to add them and run again. Two sibling refusals had the same shape, and so didsource stack did not reach the exact four-role startup cut, which named none of the four things it waits for. All four now say what they saw and what they expected.That is not tidying. Each of them cost a measurement round today.
Where the run gets to now
source stack did not reach the exact four-role startup cutafter ~1 sPeriodic PNG runtime authority established, ~28 s of running…four-role startup cut; still missing: rolesSo the bridge, the handshake and the guard are all satisfied and the remaining gap is the classification of the process tree. That is the next measurement, and it starts from the answer rather than from a search.
Evidence
tests/scripts: 136 passed, 94 skipped.ruff check src/ tests/clean; the workflow-exact format check clean over 702 changed files. Measured with #87, #86 and #89 merged locally and never pushed (evidence/tools/combine_and_soak.sh), because every push starts a full CI round.Written with assistance from Claude (Anthropic).