Skip to content

docs: record the measured laboratory-readiness position of 2026-08-18 - #84

Closed
test1card wants to merge 21 commits into
masterfrom
docs/lab-readiness-measured-2026-08-18
Closed

test1card wants to merge 21 commits into
masterfrom
docs/lab-readiness-measured-2026-08-18

Conversation

@test1card

Copy link
Copy Markdown
Owner

Records the measured laboratory-readiness position of 2026-08-18 in the roadmap,
under the live pre-lab milestone. Documentation only; no production code, no
test, no gate.

The roadmap said what remained to do. It did not say where the soak actually
stops
, which environment starts it, or which measurements had been taken
through a layer that changes the answer
. All three are now written down with
their dates, so the next reader starts from evidence rather than from argument.

What is recorded

The soak starts on Ubuntu 22.04. At fa52b35804, in a worktree cut from a
native Linux clone, it reaches the runner phase, writes seven evidence files,
and its exact-six integration gate PASSES. It then fails at one named barrier:
source stack did not reach the exact four-role startup cut.

The barrier has a cause, with lines. scripts/soak_mock_stack_runner.py:153
copies only _ISOLATED_TRACKED_CONFIG_FILES = ("channels.yaml",) into the
isolated config set, so config/themes/ never arrives; src/cryodaq/gui/theme.py
calls resolve_theme() at module import, and _theme_loader.py:26 names
warm_stone as the default. The pack exists in the repository and not in the
copied tree, so the launcher cannot import.

The environment recipe that works, and the one that does not. The
qualification section of docs/lab_verification_checklist.md is right that the
run must happen on a native Linux filesystem and never under /mnt/c. Its
runtime recipe is stale: it symlinks .venv at /root/cryodaq-soak-py313,
which is not readable on the laboratory image, whose interpreter is Python
3.14.6. That correction is deliberately NOT made here — the checklist is
already changed by the open long-soak pull request, and editing it in two places
would conflict for no benefit.

A measurement trap. A Windows-created worktree reached from WSL resolves no
git repository, so every test that shells out to git fails for a reason
unrelated to the code, while tests that never touch git pass normally. It
produced four wrong answers in one evening and two published claims had to be
corrected.

Four rungs, each a stopping point where the system stays coherent if work
stops there, and the note that sufficiency at every rung is the same object: a
sealed soak PASS at increasing duration.

Two owner facts of 2026-08-18. The laboratory computer is on an
uninterruptible power supply, so host death mid-profile is a schedule risk for
that machine rather than a data risk. And he travels this week to test on real
hardware — green software gates do not replace that test, and Keithley A8-0
on real 2604B firmware remains the heater-control gate that no soak can close.

Verification

tests/docs — 67 passed at this head, on Windows.

Every statement carries its date, and the section opens by telling the reader to
re-measure before acting on any of it.


Written with AI assistance (Claude), reviewed and pushed by the coordinator.

The roadmap said what remained to do. It did not say where the soak actually
stops, which environment starts it, or which measurements had been taken through
a layer that changes the answer. This records all three, dated, so the next
reader starts from evidence instead of from argument.

What is now written down:

- The soak STARTS on Ubuntu 22.04 and passes its exact-six integration gate. Its
  next barrier is named, with the file and line that cause it: the isolated
  config set copies one tracked file and never copies `config/themes/`, so the
  launcher cannot import a default theme pack that exists in the repository.
- The environment recipe that was measured to work, and the fact that the
  checklist's own runtime recipe is stale on the laboratory image — it points at
  a path that is not readable and at a Python that is not the one installed.
- A measurement trap that produced four wrong answers in one evening: a
  Windows-created worktree reached from WSL resolves no git repository, so every
  test that shells out to git fails for a reason unrelated to the code. Two
  published claims were corrected because of it.
- Four rungs to a laboratory-ready week, each one a stopping point where the
  system is still coherent if work stops there.
- Two owner facts of 2026-08-18: the laboratory computer is on an
  uninterruptible power supply, and he travels this week to test on real
  hardware. Green software gates do not replace that test, and the Keithley A8-0
  gate on real firmware remains the heater-control gate that no soak can close.

The checklist's stale runtime recipe is NOT corrected here. That file is already
changed by the open long-soak pull request, and correcting it in two places
would conflict for no benefit.
…the risk

deepseek-v4-pro caught an over-claim on its first review under the new reviewer
setting, and it was the class that must never pass: reporting a result the
evidence does not support.

The owner said the laboratory computer is on an uninterruptible power supply. I
wrote that host death mid-profile is therefore a schedule risk and not a data
risk. He did not say that, and it is not true. The supply removes mains power
loss and nothing else. A kernel panic, an out-of-memory kill, a storage or
filesystem failure, or a crash of the program itself still kills the host and
still drops whatever measurements are not yet persisted.

It also contradicted this same roadmap two paragraphs above, where A8c
host-death is still listed as a physical blocker. A reader could have taken the
supply as closing it and skipped the check.

The text now says which single cause is removed, states that host-death
persistence stays OPEN and that A8c remains a physical blocker, and names the
case the supply really does close: the mains outage that interrupted work on the
development machine, which has no such supply.
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 98f8c52fb98c2913fa2596675cd4e599a197055b. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@test1card

Copy link
Copy Markdown
Owner Author

Review verdict: CLEAN — deepseek/deepseek-v4-pro, bound to 98f8c52fb9

Receipt: evidence/reviews/REVIEW_deepseek-v4-pro_pr84_98f8c52fb9.md in the
workspace layer. This is the first review under the new reviewer setting: the
hosted reviewer is exhausted, glm-5.3 ran low, and deepseek-v4-pro reviews from
here on the owner's direction.

It earned the switch on its first run. The previous head of this pull
request was reviewed FINDINGS, and the finding was right.

The text had said that because the laboratory computer is on an uninterruptible
power supply, host death mid-profile is "a schedule risk for that machine, not a
data risk". The reviewer objected that a supply removes only mains power loss —
a kernel panic, an out-of-memory kill, a storage failure or a crash of the
program still kills the host and still drops whatever is not yet persisted — and
that the conclusion contradicted this roadmap's own irreducible milestone two
paragraphs above, where A8c host-death is still a physical blocker.

That was an over-claim of mine, not the owner's statement. He said the machine
has a supply; the rest was my inference and it does not hold.

The section now names the single cause the supply removes, states plainly that
host-death persistence stays OPEN and that A8c remains a physical blocker, and
names what the supply really does close: the mains outage that interrupted work
on the development machine, which has none.

tests/docs — 67 passed at this head.

…measured-2026-08-18

# Conflicts:
#	docs/architecture-montana-important.svg
#	docs/current_candidate_metrics.md
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 018d598e772bb034031b10542045c9be4945c0f5. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@test1card
test1card marked this pull request as ready for review August 18, 2026 23:52
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@test1card

Copy link
Copy Markdown
Owner Author

Master is RED on the target platform, and this branch is what fixes it

Measured on Ubuntu 22.04.5 LTS, Python 3.14.6, in clean worktrees cut from the
native Linux clone, each with the --system-site-packages environment the
runner requires.

head tests/scripts/test_soak_mock_stack_runner.py -k strict_exact_six
master 4e3fa39066 2 failed
this branch fa0fcb12be 2 passed

The failure master carries

_RunnerFoundationError: exact-six execution failed: exit code 4; captured stdout:
ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.15' not found
(required by .../miniforge3/envs/cryodaq-lab/lib/python3.14/lib-dynload/../.././libicui18n.so.78)

The child loads the system libstdc++ together with the conda
environment's
libicui18n, and the two do not match. Measured directly:

  • .../envs/cryodaq-lab/lib/libstdc++.so.6 -> libstdc++.so.6.0.35, and
    CXXABI_1.3.15 appears in it twice;
  • /lib/x86_64-linux-gnu/libstdc++.so.6 contains it zero times.

The library the child needs is present in the environment and is not the one it
loads.

Why this branch fixes it

Master computes the controlled LD_LIBRARY_PATH from sys.prefix. Under the
virtual environment the runner itself demands, that directory holds no shared
objects at all, so the child falls back to the system loader path and finds the
wrong libstdc++. This branch computes it from sys.base_prefix when the two
differ, which is the environment that actually holds the interpreter's libraries.

That change was made for a different reason — a stock virtual environment
produced an empty native-library closure — and it repairs this as well.

What it means for the queue

Every open pull request inherits this red node, including ones that touch no
code at all: #84 changes only ROADMAP.md and shows exactly the same failure.
Anyone chasing it inside another branch is chasing master's defect.

It is therefore not an acceptance to be written per branch. It is one fix, here.

@test1card

Copy link
Copy Markdown
Owner Author

Correction: the red node here is NOT the cause I named earlier

An earlier comment on this pull request said the red
test (ubuntu-latest, remaining) was master's system-versus-conda libstdc++
mismatch, which #82 fixes. That was measured on the laboratory host, and it is
not what fails here.
I checked the hosted job's log rather than assuming the
same wrapper meant the same cause.

What actually fails inside the exact-six run on the hosted runner:

_RunnerFoundationError: exact-six execution failed: exit code 1; captured stdout:
.....F.
FAILURES
____ test_killed_rendering_leader_promotes_then_authorizes_one_delivery ____

That is a leader-election and delivery-authorisation test, not a library
mismatch and not the native-library ownership refusal seen on an earlier head.
Three distinct causes have now appeared behind the same
_RunnerFoundationError: exact-six execution failed wrapper
, and they must not
be folded into one claim.

What still stands from the earlier comment

The libstdc++ measurement itself is unchanged and reproducible on the laboratory
host: at master the node fails there, at #82's head it passes, and the conda
environment carries the symbol the system library lacks. That remains a real
defect on the target platform and #82 remains its fix. It is simply not the
reason this job is red.

What is being done here

The node passes on the laboratory host at master — measured just now, 1 passed
— so the failure is specific to the hosted runner or intermittent there. The
failed job has been re-run: if it goes green, it was a flake on a shared runner
and the record should say so; if it fails again, it is environment-specific and
gets its own investigation rather than a guess.

Nothing about this pull request's own content is implicated. It changes one
markdown file.

…measured-2026-08-18

# Conflicts:
#	docs/architecture-montana-important.svg
#	docs/current_candidate_metrics.md
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 06b03606237d1bfdacfc37a282000f2f7585c1d6. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

…measured-2026-08-18

# Conflicts:
#	docs/architecture-montana-important.svg
#	docs/current_candidate_metrics.md
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 40035a9d1072f42c223ab54a3e1c7889efff9dbf. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

# Conflicts:
#	docs/architecture-montana-important.svg
#	docs/current_candidate_metrics.md
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is d963fdadc1e251ed45a614f14f617c7de463047a. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

The paragraph said the theme pack never arrives because the isolated config
copy lists only channels.yaml. That is true of the copy and it is not the
mechanism, and the difference decides where a reader looks next.

The pack is tracked and git archive carries it into the sealed snapshot, so
the tree has it. The child cannot see it because _source_environment points
CRYODAQ_ROOT at the isolated root, and get_config_dir is that root plus
config, which holds only the curated passive set.

Measured both ways on Ubuntu 22.04.5 with the workspace rootprobe tool: an
override at a tree without config reproduces the recorded traceback exactly,
and CRYODAQ_ROOT at the application tree with CRYODAQ_STATE_ROOT at the
isolated directory imports cleanly. The second arrangement is the split
paths.py already documents, so the fix is a configuration decision rather
than new code.

The refusal is left alone. A test pins it and a tree with no config has no
safety configuration either, so whether the program should stop over a colour
file is the owner's decision.
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is d44bfba867c6ad6694c011a65a05079414e52042. A lane closed the live review findings; the coordinator ran the landing gates on this tree -- byte-order-mark, encoding and parse checks on every changed file, a refusal on any tree that deletes more than it adds, ruff check and ruff format --check on the changed Python, the derived documentation pair regenerated to a fixed point, and the documentation gate green.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@test1card

Copy link
Copy Markdown
Owner Author

test (windows-latest, core) failed once at head d44bfba867 on tests/core/test_operator_snapshot_subprocess_ingress.py::test_real_spawn_two_sub_sockets_keep_readings_and_newest_snapshot_independent.

Two things say this is a hosted-runner timing flake rather than something this branch caused, and both are measurements rather than reasoning:

  1. This branch changes documentation only. The failing test spawns real subprocesses and two ZeroMQ subscriber sockets; nothing in the diff can reach it.
  2. Measured on this Windows host at the same head: 0 of 12 runs failed. That bounds the rate here. It does not prove the hosted runner is healthy, and it is a smaller sample than the hosted job's own history.

The failed job is being rerun rather than argued away. If it fails again the reading changes and the test needs its own work, because a required check that fails twice is not a flake, it is a defect with an unknown owner.

Recording it here so that a green rerun is not later read as "it passed", which is a different claim from "it failed once and passed on rerun at a measured rate of zero in twelve locally".

soak measurement added 2 commits August 20, 2026 15:22
# Conflicts:
#	docs/architecture-montana-important.svg
#	docs/current_candidate_metrics.md
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 432b1cc46c798b51ad06ad9497861edcfd562cc7. No content change in this push: master is taken in so the protected jobs get a merge reference and a FIRST run attempt, which the protected evidence gate requires and a re-run can never supply. Only the two generated documentation files conflicted; they were resolved by taking one side and regenerating to a fixed point, and the documentation gate is green at this head (68 passed). The pull request itself touches ROADMAP.md and nothing else outside the derived pair.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 432b1cc46c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ROADMAP.md
Comment on lines +318 to +321
`_source_environment` sets `CRYODAQ_ROOT` to the ISOLATED root,
`get_config_dir()` is `get_project_root() / "config"`, and that isolated
`config/` holds only the curated passive set (`_ISOLATED_TRACKED_CONFIG_FILES`
plus `_ISOLATED_STATIC_CONFIGS`), with no `themes/` among them.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Re-measure before naming the current soak barrier

At this commit, the stated cause has already been fixed: scripts/soak_mock_stack_runner.py declares _ISOLATED_TRACKED_CONFIG_DIRS = ("themes",) and _materialize_isolated_mock_config copies those packs into the isolated configuration. Keeping the old fa52b35804 diagnosis under “its next barrier” makes the forward roadmap describe completed work as the current blocker and can send the next investigation down the wrong path; re-run the soak at the reviewed head and record the barrier that remains.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +342 to +347
The qualification section of `docs/lab_verification_checklist.md` is right about
the important thing — the run must happen in a clone on a native Linux
filesystem, never under `/mnt/c` — and **its runtime recipe is stale**. It
symlinks `.venv` at `/root/cryodaq-soak-py313`. Measured 2026-08-18 on the
laboratory WSL image: that path is not readable, and the interpreter present is
`cryodaq-lab`, Python 3.14.6.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Repair the canonical qualification recipe in the same slice

On the measured laboratory WSL image, this says /root/cryodaq-soak-py313 is unreadable, but the authoritative turnkey procedure in docs/lab_verification_checklist.md:55-62 still creates .venv symlinks to exactly that path. An operator following the canonical checklist will therefore reach a dangling symlink and fail at .venv/bin/python even though this roadmap contains a different recipe; update and verify the checklist rather than leaving two incompatible procedures.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +387 to +388
Sufficiency at every rung is the same object: a sealed soak PASS at increasing
duration. That is why the soak is graded as an instrument rather than a feature.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not make the passive soak sufficient for these rungs

When these rungs are used as laboratory-readiness criteria, a sealed soak PASS cannot be their sufficient evidence: the canonical checklist defines this soak as one passive mock LS218 fixture with production alarms, interlocks, and physical alarms disabled, while the rungs require heater-control, real end-to-end instrument operation, and fault delivery to a human. Describe the soak as one required software gate and retain separate prescribed evidence for those requirements; otherwise even a 168-hour mock PASS appears to certify behavior it never exercises.

AGENTS.md reference: AGENTS.md:L23-L26

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +288 to +291
**The owner travels to the laboratory this week to test on real hardware. Green
software gates do not replace that test, and he has said so.** Read the
irreducible hardware milestone above with that in mind: **Keithley A8-0 on real
2604B firmware is the heater-control gate, and no soak substitutes for it.**

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the owner's travel schedule from the public roadmap

Because ROADMAP.md is the repository's public forward product plan, recording that the identifiable owner will travel to the laboratory “this week” publishes personal scheduling information and turns a durable plan into immediately stale campaign evidence. Retain only the product requirement that physical testing must occur, and keep the person's schedule in an appropriately private or campaign-local coordination channel.

AGENTS.md reference: AGENTS.md:L544-L549

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md
Comment on lines +379 to +385
2. **Trust the evidence** — the suite that certifies the system is itself
trustworthy: no test leaves a production module mocked for the tests after
it, no unavailable source root passes as a compile check, no vacuous
assertion. Plus a 12-hour PASS.
3. **Unattended week** — faults reach a human, configuration parsing has an
owner, the predictor baseline is honest. Plus a 72-hour PASS.
4. **The week** — a 168-hour PASS on a frozen SHA.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark the long-soak rungs unavailable until they can run

These stopping criteria cannot currently produce the evidence they require: scripts/soak_mock_stack.py exposes only short, 12h, and 72h, immediately returns exit code 3 for every profile other than short, and defines no 168-hour profile at all. Consequently the 12-hour and 72-hour PASSes are refused and the 168-hour PASS is not even selectable; describe these as open implementation/activation gates or add the corresponding runnable, sealable evidence contracts before presenting them as executable rungs.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md
Comment on lines +299 to +300
**So host-death persistence remains OPEN, and the A8c host-death gate above
remains a physical blocker.** Do not read the uninterruptible supply as closing

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Separate persistence loss from the A8c OFF gate

When the laboratory team works through the declared physical blockers, this introduces “host-death persistence” as an open gate next to A8c, but docs/lab_verification_checklist.md:294-304 defines A8c only as killing the host and observing whether the Keithley output turns OFF without another command; it contains no database or data-durability criterion. As written, there is no prescribed evidence that can close the newly named persistence gate, and an A8c result can be mistaken for covering it; define a separate persistence procedure and artifacts, or explicitly classify the data loss as an accepted residual while keeping A8c solely about verified OFF.

AGENTS.md reference: AGENTS.md:L37-L42

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +301 to +302
either. What it does close is the specific case that interrupted work on
2026-08-18: a mains outage on the development machine, which has no such supply.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not close the development-machine outage with a lab UPS

In the stated scenario, the interrupted soak ran on the development machine and this sentence explicitly says that machine has no UPS, so a UPS attached to the separate laboratory computer cannot close that specific mains-outage case. It only changes the risk for future runs performed on the protected lab machine, and even there requires bounded-runtime evidence; reword this as a machine-scoped mitigation rather than claiming the observed failure case is closed.

AGENTS.md reference: AGENTS.md:L23-L26

Useful? React with 👍 / 👎.

soak measurement added 2 commits August 20, 2026 16:53
…he soak from proving what it never ran

Seven review findings on the readiness record, and four of them say the same
thing in different places: the document states a position that has since moved.

THE NAMED BARRIER WAS ALREADY BUILT. The theme-pack cause is fixed in the
runner, so keeping it under "the next barrier" pointed the next investigation at
finished work. Measured on Ubuntu 22.04 at the heads then under review, the run
now recovers the engine fault -- ready in 6.749 s, with bridge data resumed --
and stops three steps further on: the periodic reporter flaps between ready and
degraded_runtime about once a second, active never leaves null, so no slot is
allocated, so no receipt can be sealed, so the assistant fault refuses. Its own
health names the cause: periodic_live_source_stopped, and beneath that
periodic_engine_unavailable. The old measurements are kept as dated history,
because they are still true of the commits they name.

THE CANONICAL RECIPE STAYED BROKEN. Recording a working recipe in the roadmap
while the checklist -- the procedure an operator actually follows -- still
symlinks the unreadable /root/cryodaq-soak-py313 leaves the operator at a
dangling symlink and a failure at .venv/bin/python. The checklist is repaired in
this same commit: the interpreter path is measured on the machine, one link to
the interpreter is enough, and what was measured on the laboratory image is
named.

THREE OF THE FOUR RUNGS CANNOT BE RUN. scripts/soak_mock_stack.py registers
exactly short, 12h and 72h; every profile other than short exits 3, the runner
refuses it a second time, and there is no 168-hour profile at all. The ladder now
says so, and says what has to be built before those rungs can be attempted.

AND THE SOAK DOES NOT PROVE HEATER CONTROL. It drives mock sources and commands
no heater, so a PASS of any duration certifies continuity and nothing about the
physical gate. Sufficiency is no longer one object at every rung.

The remaining three: the owner's travel schedule is out of the public plan, the
2026-08-18 outage is scoped to the development machine it actually struck, and
A8c is stated as an OFF gate that does not cover persistence.
@test1card
test1card marked this pull request as draft August 20, 2026 13:55
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is c10040c37256e1e20a91dce1a65102aebbf9c9dc. All seven findings of the previous round are closed in one commit.

  • Barrier re-measured. The fa52b35804 theme-pack diagnosis is kept only as dated history; the section now records what was measured on Ubuntu 22.04 at the reviewed heads — engine fault recovered in 6.749 s with bridge data resumed, then a stop three steps further on at periodic_engine_unavailable / periodic_live_source_stopped, active never leaving null, so no slot and no receipt.
  • Canonical recipe repaired in this same commit. docs/lab_verification_checklist.md no longer symlinks /root/cryodaq-soak-py313; the interpreter path is measured on the machine and one link to the interpreter is made.
  • Long-soak rungs marked unavailable. Measured at this commit: scripts/soak_mock_stack.py registers exactly short, 12h, 72h; non-short exits 3 and the runner refuses it again; no 168-hour profile exists.
  • The passive soak is no longer sufficient for the rungs. It commands no heater, so its PASS certifies continuity only; the physical A8-0 gate keeps its own evidence.
  • The owner's travel schedule is out of the public plan; the 2026-08-18 outage is scoped to the development machine it struck; A8c is stated as an OFF gate that does not cover persistence.

Gates run on this tree: the derived documentation pair regenerated to a fixed point and committed, and tests/docs green — 68 passed. The pull request is a draft until the review is clean, so the full check set is not called on a round that is expected to produce findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c10040c372

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ROADMAP.md Outdated
Comment on lines +461 to +463
heater control or about the instrument running end to end. Rung 1 asks for both
of those, and they come from the physical Keithley A8-0 gate, not from a soak of
any duration. What the soak alone certifies is continuity: that the program

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep heater-control evidence outside the OFF-grammar gate

At this head, the corrected text assigns both heater control and end-to-end instrument operation to A8-0, but docs/lab_verification_checklist.md:258-281 requires A8-0 to run while the source is already confirmed OFF and without applying power; it verifies only the nonce-bound OFF reply grammar. Rung 1 could therefore be declared complete without ever exercising heater control. Keep A8-0 open for its stated purpose and name a separate prescribed powered/end-to-end hardware procedure for these two claims.

AGENTS.md reference: AGENTS.md:L23-L26

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +394 to +399
**Fixing this document is not enough: the checklist is the CANONICAL procedure,
and an operator follows that.** While the two disagree, the operator meets a
dangling symlink and fails at `.venv/bin/python` no matter what this roadmap
says. Repairing `docs/lab_verification_checklist.md` belongs in the same slice
as this section; until it lands, treat the recipe below as the one that was
measured and the checklist as known-stale.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the obsolete warning about the canonical recipe

In the reviewed tree, docs/lab_verification_checklist.md:59-71 has already replaced the old /root/cryodaq-soak-py313 links, but this section still tells operators that the checklist has a dangling symlink, is known-stale, and should be disregarded in favor of the roadmap's different venv recipe. That leaves the repaired canonical procedure contradicted by the public plan; update this section to describe the checklist that actually landed.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +332 to +333
**Measured 2026-08-20 on Ubuntu 22.04, at the heads then under review, the run
gets much further and stops somewhere else entirely:**

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bind the barrier measurement to an immutable commit

When later changes move the soak behavior again, “at the heads then under review” does not identify which code produced the 6.749-second recovery and periodic-reporter failure, even though the section promotes that result as the current barrier. Record the exact commit or tree and command/evidence artifact, or label this result pending/historical, so reviewers can determine whether the measurement applies to the candidate they are qualifying.

AGENTS.md reference: AGENTS.md:L445-L450

Useful? React with 👍 / 👎.

@test1card test1card mentioned this pull request Aug 20, 2026
soak measurement added 2 commits August 20, 2026 19:13
Three review findings, and the first is the one that mattered.

A8-0 CANNOT PROVE HEATER CONTROL. The checklist says so in its own words: the
procedure runs with the source already confirmed OFF and with no power applied,
and it checks exactly one property -- that the instrument answers a nonce-bound
OFF command with the exact expected line and nothing else. I had assigned both
heater control and end-to-end operation to it, so rung 1 could have been declared
complete without a heater ever being driven. Heater control now cites the POWERED
procedures, A8b and A8d; A8-0 stays open for its own purpose.

AND END-TO-END OPERATION HAS NO PROCEDURE AT ALL. The A8 series covers the OFF
command, the watchdog, host death and terminal measurement. None of them is an
experiment run end to end on real hardware with acquisition, storage and export.
Naming that gap is more useful than assigning it to a gate that does not test it.

THE WARNING I WROTE WAS OBSOLETE BY THE TIME IT LANDED, because the same change
repaired the checklist it warned about. It told operators to disregard the
canonical procedure that had just been fixed.

AND THE BARRIER MEASUREMENT NAMES NO TREE. It was taken on a throwaway merge of
several open branches, so no tracked commit produced it and none can be cited --
which makes it directional rather than qualification of any candidate. It becomes
bindable when those branches merge and the soak runs at one candidate head.
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 2d4a360faf511ff6ef63dfb83ba40ba8bd2fdeac. All three findings are closed, and the first one was a real defect in the plan rather than in its wording.

  • A8-0 proves the OFF grammar and nothing else. The checklist says so in its own words — the procedure runs with the source already confirmed OFF and with no power applied, and it checks exactly one property. I had assigned both heater control and end-to-end operation to it, so rung 1 could have been declared complete without a heater ever being driven. Heater control now cites the powered procedures, A8b and A8d; A8-0 stays open for its own purpose.
  • End-to-end operation has no prescribed procedure at all, and the roadmap now says that instead of assigning it to a gate that does not test it. The A8 series covers the OFF command, the watchdog, host death and terminal measurement; none of them runs an experiment end to end on real hardware with acquisition, storage and export.
  • The obsolete warning is gone. It told operators to disregard the canonical checklist that the same change had just repaired.
  • The barrier measurement is labelled directional. It was taken on a throwaway merge of several open branches made to see whether they work together, so no tracked commit produced it and none can be cited. I would rather say that plainly than attach a SHA that does not describe the tree that ran. It becomes bindable once those branches merge and the soak runs at one candidate head.

tests/docs is green at 68 passed and the derived documentation pair is regenerated to a fixed point.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d4a360faf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ROADMAP.md Outdated
Comment on lines +290 to +291
with that in mind: **Keithley A8-0 on real 2604B firmware is the heater-control
gate, and no soak substitutes for it.** When that testing happens is scheduling,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stop naming A8-0 as the heater-control gate

Fresh evidence at head 2d4a360 is that this sentence still calls A8-0 the heater-control gate even though the corrected rung text below and docs/lab_verification_checklist.md:258-281 establish that A8-0 runs already confirmed OFF, applies no power, and verifies only reply grammar. Leaving this earlier summary intact gives operators a contradictory path for treating an unpowered grammar check as physical heater-control evidence; name the powered A8b/A8d procedures here as well.

AGENTS.md reference: AGENTS.md:L23-L26

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +393 to +395
a behaviour decision, not a mechanism one — **and the owner has since decided
it** (2026-08-20: *"файл цветов не должен останавливать"*), so that test now
states the opposite and the loader falls back to a built-in copy.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not claim the built-in theme fallback has landed

At the reviewed head, resolve_theme() still raises RuntimeError when the default pack is missing (src/cryodaq/gui/_theme_loader.py:162-180), and tests/gui/test_theme_loader.py:182-186 still explicitly expects that exception. This statement therefore reports an open-branch or intended behavior as current behavior, which can mislead investigation of startup failures; describe it as pending until the implementation and opposite test are actually in this candidate.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

Comment thread ROADMAP.md Outdated
Comment on lines +409 to +413
The qualification section of `docs/lab_verification_checklist.md` is right about
the important thing — the run must happen in a clone on a native Linux
filesystem, never under `/mnt/c` — and **its runtime recipe is stale**. It
symlinks `.venv` at `/root/cryodaq-soak-py313`. Measured 2026-08-18 on the
laboratory WSL image: that path is not readable, and the interpreter present is

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the stale-recipe warning that remains

Fresh evidence at head 2d4a360 is that the obsolete warning remains immediately after the new paragraph saying the canonical checklist was repaired: these lines still state in the present tense that the checklist uses /root/cryodaq-soak-py313, while docs/lab_verification_checklist.md:58-71 now resolves and links the measured cryodaq-lab interpreter instead. Operators are consequently given mutually exclusive descriptions of the canonical procedure; remove or clearly historicize this paragraph.

AGENTS.md reference: AGENTS.md:L37-L39

Useful? React with 👍 / 👎.

soak measurement added 2 commits August 20, 2026 21:22
Three review findings, and all three are the same failure: a correction was made
in one place and the sentence that said the opposite was left standing. A reader
then has two mutually exclusive statements and no way to tell which is current.

A8-0 WAS STILL CALLED THE HEATER-CONTROL GATE in the summary above the rungs,
even though the rung text below and the checklist both establish that it runs
with the source already confirmed OFF, applies no power, and verifies reply
grammar only. That gave an operator a path to treat an unpowered grammar check as
physical heater-control evidence. The powered procedures, A8b and A8d, are named
here as well now.

THE THEME FALLBACK WAS REPORTED AS LANDED. It is not. Verified in this candidate:
`src/cryodaq/gui/_theme_loader.py` still raises `RuntimeError` on a missing or
invalid default pack, and `test_missing_default_pack_raises` still requires that
exception, so a missing colour file still stops startup today. The owner's
decision that it should NOT is kept beside that, because both halves matter -- the
decision is made and the behaviour has not changed yet, and reading only the first
half sends an investigation of a startup failure down the wrong path.

AND THE OBSOLETE RECIPE WARNING WAS STILL THERE, in the present tense, directly
after the new paragraph saying the canonical checklist had been repaired.

This slice was drafted by a worker lane and then verified here rather than taken
on its word. Two things needed correcting: the lane wrote CRLF into an LF file,
which would have shown as a whole-file diff, and it removed the owner's decision
along with the stale claim instead of keeping it as pending.
@test1card

Copy link
Copy Markdown
Owner Author

@codex review

Head is 9aa05ea3c9938d9f98326bff1b3f71417f699fa1. All three findings are closed, and they were one failure repeated: a correction made in one place while the sentence saying the opposite was left standing.

  • A8-0 is no longer called the heater-control gate in the summary above the rungs. The powered procedures, A8b and A8d, are named there as well, so an operator has no path to treat an unpowered grammar check as physical heater-control evidence.
  • The theme fallback is described as pending, and verified here: _theme_loader.py still raises RuntimeError on a missing or invalid default pack, and test_missing_default_pack_raises still requires that exception. The owner's decision that it should NOT stop startup is kept beside that, because both halves matter — the decision is made and the behaviour has not changed yet.
  • The obsolete recipe warning is gone.

Two process notes, since they affect how you should weigh this head. This slice was drafted by a worker lane and then verified here rather than taken on its word, and two things needed correcting afterwards: the lane wrote CRLF into an LF file, which would have shown as a whole-file diff; and it removed the owner's decision along with the stale claim, instead of keeping it as pending.

tests/docs is green at 68 passed with the derived documentation pair regenerated.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9aa05ea3c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ROADMAP.md
Comment on lines +412 to +413
interpreter. **The recipe below is the same one and is kept as the record of what
was measured**, not as a competing procedure.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the roadmap recipe distinct from the checklist

When qualification is reproduced from this roadmap, the asserted equivalence is false: docs/lab_verification_checklist.md:66-71 resolves the installed conda interpreter and symlinks it directly into .venv/bin, whereas the recipe below creates a new venv with --system-site-packages. Those approaches produce different environment prefixes and isolation, so the public plan still presents two distinct qualification procedures while claiming canonical parity; make the commands equivalent or clearly label the roadmap recipe as separate historical evidence.

AGENTS.md reference: AGENTS.md:L37-L42

Useful? React with 👍 / 👎.

@test1card

Copy link
Copy Markdown
Owner Author

Closing by the approved laboratory-readiness disposition. This pull request is a dated readiness snapshot that was superseded by later measured laboratory runs. Merging the older position would publish stale status rather than improve the Ubuntu 22.04 week-long run.

@test1card test1card closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant