Skip to content

test(release): add standalone qualification and acceptance evidence - #49

Open
DivyamTalwar wants to merge 10 commits into
codex/jitmind-j09-reviewed-20260928from
codex/jitmind-j08-reviewed-20260928
Open

DivyamTalwar wants to merge 10 commits into
codex/jitmind-j09-reviewed-20260928from
codex/jitmind-j08-reviewed-20260928

Conversation

@DivyamTalwar

Copy link
Copy Markdown
Owner

Scope and stack

J-08, refs #38. Stacked on #48 after the reviewed storage, scope, parser, anchor, preflight, checkpoint and durable-history changes. This completes the planned PR series plus the J-09 follow-up; opening this PR is not permission to merge or a production-release certification.

Implementation

  • Add a fixed, offline qualification suite with bounded subprocesses/logs, candidate snapshots, strict artifact validation and exact pytest setup/call/teardown evidence.
  • Preserve all 32 handoff cases and 41 required stages, mapped to 100 stage-node associations / 94 unique identities: 93 pytest nodes plus the separately executed installed smoke.
  • Include real integration/fault fixtures for ENOSPC, SQLite capacity, process-exit migration, parser/binding/projection redelivery and primary-forget/open-obligation behavior.
  • Build a wheel from a clean captured source snapshot, install it into a fresh environment outside the checkout, and run python -I smoke tests without checkout PYTHONPATH, Node or optional provider packages. Socket connections are denied in the smoke.
  • Keep source-controlled case statuses as specifications (not_run); only the retained execution bundle assigns observed results. Unknown/failed/skipped/missing evidence cannot become a pass.
  • Preserve a distinct checks_only_complete result and a separate release_qualified result. The latter remains false until independent release requirements are reviewed.

Review corrections

The supplementary fixtures used evidence labels outside the existing fixed vocabulary and two labels for the same reused test. These were normalized with an explicit ledger; no node, parameter, assertion, stage, denominator or release blocker was removed. The missing-runtime smoke now checks the actual typed node_executable_unavailable contract, with negative controls for untyped or unrelated errors.

Actual assembled-candidate qualification

Candidate: a19e981edcb5d86ec38bde8f15e84f5d6bc2f6a3
Source snapshot SHA-256: cb2afd78303f341d2fc290224dbb1e3599a6d648ff5098fe37e49d61dade8f92
Built/installed wheel SHA-256: ac53a2fca3f4dc8d7e1e9aaffdca17056105232c9b09bbd61f654bb9c7a1af0b

python scripts/qualify_jitmind.py run \
  --output /outside-checkout/new-evidence-directory \
  --wheelhouse /reviewed/offline-wheels \
  --node-binary /reviewed/absolute/node \
  --overall-seconds 900 --command-seconds 180

Observed on macOS arm64, Python 3.12.12, provisioned Node 25.2.1 for checkout parser tests:

  • 879 passed, 0 failed, 0 skipped, 0 unknown in the fixed suite.
  • 32/32 cases, all 41 required stages passed.
  • Wheel build, isolated venv creation, offline core installation and all five installed-smoke checks passed.
  • Installed import came from lib/python3.12/site-packages/jitmind/__init__.py, not the checkout.
  • Actual installed durable-store diagnostics: SQLite 3.51.2, DELETE journal, FULL synchronous, foreign keys enabled.
  • checks_only_complete=true; release_qualified=false.

The earlier run without explicit Node correctly reported parser-stage failures rather than silently skipping them; its independent installed-wheel smoke passed. The correctly configured run above is the qualified checks bundle. Supervisor full tests, compilation, E9/F lint, pip check and whitespace checks also passed before committing. Counts include regression/helper cases, not 879 independent production scenarios or benchmarks.

Remaining release requirements

Independent signoff, full supported-platform/provider qualification, physical power-loss drills and meaningful donor comparison benchmarks are not supplied by these runs. Logical deletion is not erasure of every historical backup. Parser-language/call-resolution and POSIX/local-filesystem limitations remain explicit. No code from the other user products is required; no live provider calls, production credentials or automatic merge.

See docs/release-qualification.md, docs/engineering/qualification-review.md and the versioned acceptance map for commands, provenance, fault scopes and rollback. After prerequisite merges, retarget and rerun against the resulting main revision.

Copy link
Copy Markdown
Owner Author

Final reviewed JITMIND delivery — exact-head CI and installed-package evidence

Checked at 2026-09-28 23:34 UTC. This supersedes the earlier candidate counts in the PR description; those remain historical results for their stated revisions.

Complete issue / PR map

Ticket Issue PR Latest exact-head CI
J-00 — baseline #30 #39 Passed; baseline PR already merged
J-01 — legacy persistence #31 #41 Passed
J-02 — transactional ingest #32 #42 Passed
J-03 — scoped retrieval #33 #43 Passed
J-04 — bounded code context #34 #44 Passed
J-05 — versioned code anchors #35 #45 Passed
J-06 — open work / preflight #36 #46 Passed
J-07 — closed-range checkpoints #37 #47 Passed
J-09 — durable temporal history follow-up #40 #48 Passed
J-08 — final qualification #38 #49 Passed

All nine remaining implementation PRs are open and non-draft. The configured Ubuntu Python 3.10/3.11/3.12 CI matrix is green for each current head. No automatic PR merge was performed. JITMIND stays independent of RUN-PHANTOM and HELMRYTH.

Final published candidate

  • Git head: 48a2a72a4637298f46bce79d7bd045efcc30d58c
  • Source snapshot SHA-256: fcd30f291d1362f74b4601b25f19b54d57ee7c854c838a06bb5d0653780d7697
  • Built and installed wheel SHA-256: b14fea1b7b28c66ead63d0b494b20feda641ac844e814fcc6d978fc2c32e2a96
  • 882 passed, zero failed, skipped or unknown tests.
  • 32/32 acceptance cases and all 41 required stages passed. The map preserves the original nodes and now includes three explicit default-deadline boundary controls: 103 stage-node associations / 97 unique identities (96 pytest nodes plus installed smoke).
  • All five installed-smoke checks passed: isolated import origin, legacy memory/research, typed missing-adapter behavior, optional adapter import, and absence of optional dependencies.
  • The wheel was installed into a fresh environment outside the checkout. The smoke used python -I, no checkout PYTHONPATH, no Node/optional providers, and denied socket connections. Actual import originated in site-packages.
  • Actual product storage diagnostics: SQLite 3.51.2, DELETE journal, FULL synchronous, foreign keys enabled.
  • Read-only evidence verification returned exit 0 against the expected Git and snapshot hashes.

Current-main compatibility

The same candidate was composed with current main d9ba612e62e29672e1ce4583e0d5b79ea5d11110 in an isolated, unpublished verification branch, not by changing main or merging a PR.

  • Verification head: e0f644e6f9bc5bd1183600b9f49621b670db2374
  • Snapshot: 4a8ecf533b3670e42de596a0252f4fdd5ca2ef987dcc58f2083eda24cbe45870
  • Installed wheel: 8b9edda35cabe03bd54a225693051557aae806fbb50ba93b6948881c419a3a8e
  • 886 passed, zero failed/skipped/unknown; 32 cases and 41 stages passed. All installed-smoke and read-only verification checks passed.

These counts overlap; they must not be added together as independent scenarios.

Last CI corrections

Quota setup now uses one bounded transaction rather than hundreds of fsync commits. Successful RESERVED-lock read compatibility is tested separately from a shared-runner latency claim. Production deadline/authorization logic was not changed. Three new real-SQLite controls test before/at/after the default 150 ms cutoff; expiry must return no payload and create no receipt. The existing blocked-call and sleep-driven exhaustion controls remain.

The same eight deadline/independent-process cases were repeated 25 times with every repetition passing. This is stability evidence for those controls, not 200 independent tasks or a production latency benchmark. The test-oracle change is explicit in the docs rather than described as unchanged assertions.

Evidence and review handoff

Retained on the authorized implementation workspace:

  • qualification/final-candidate-48a2a72a/
  • qualification/final-current-main-e0f644e6/
  • reports/final-readonly-verification.json
  • reports/final-github-status.json
  • reports/resume-deadline-repeat-25.json

The qualification directories contain the captured sources, complete test collection/results, wheel, installed-smoke report, command logs and versioned verdict. They are local retained evidence, not an assertion that those bundles were uploaded as GitHub artifacts.

Review/merge sequence: #41 → #42 → #43 → #44 → #45 → #46 → #47 → #48 → #49. After prerequisites merge, retarget as appropriate and rerun against the resulting base. Existing unrelated proposals #27 and #29 were not merged or rewritten.

Both bundles report checks_only_complete=true and release_qualified=false. Independent release signoff, the full supported-platform matrix, live-provider qualification, physical power-loss drills and donor-comparison benchmarks remain explicit release requirements. Core Linux CI and local macOS evidence do not establish Windows support or measured superiority over donors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant