Skip to content

Core correctness recovery: identity, filesystem, registry and validation - #28

Draft
corbensorenson wants to merge 24 commits into
mainfrom
codex/correctness-recovery-roadmap
Draft

corbensorenson wants to merge 24 commits into
mainfrom
codex/correctness-recovery-roadmap

Conversation

@corbensorenson

@corbensorenson corbensorenson commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

The project review reproduced incorrect identities, filesystem capability escapes, corrupt registry acceptance, count-dependent no-op work, and incomplete process/resource ownership. This draft repairs their common boundaries under Core task R4.2.e and records the finite F01–F11 recovery roadmap in ROADMAP.md §3.29. All eleven acceptance rows and eight active P1 defects remain open; local observations do not authorize independent acceptance or release.

The resulting behavior is:

  • Canonical symbol namespaces and proper serialized terms are admitted before effect identity and store dispatch. Empty and zero-count repetition avoids count-dependent work. Package-verifier expectations use artifact-backed commit authority, and its independent Python checker is pinned. Fresh current-source observations reproduce improper-term collisions and compiled closures whose different integer captures share a hash. An additive bounded runtime-term v0.3 codec now separates all term tags and pair fields without printing, cloning, recursive traversal or integer byte buffers. Current store/value hashes and log versions stay unchanged until the complete capture and replay migration; F01 remains open.
  • Compiled module slots now read and update the same live named scope as the reference evaluator. An earlier closure observes a warm redefinition; right-hand sides read the preceding binding, and parent/forward-name semantics agree. Sparse lexical captures retain actual entries rather than allocating discarded depth holes. Live accounting charges their complete semantic capture frame; a retained 16 KiB string consumes 16,420 units in both tiers and rejects a 2,048-unit ceiling. GCKM5 readers reject inconsistent definition/slot names, duplicate names and literal-looking slot names while preserving valid writer bytes. The shared reader now independently checks complete definition inventories, named lexical/module/external resolution and closure body/IR structure. It validates every stored literal/quote/container field, sequential binder, curried parameter, primitive/seal identity, argument order and flattened application tick count without recompiling or trusting capture/forward plans. Every term field is serialized through the common checked canonical-domain printer, which rejects improper pairs and unadmitted raw symbols instead of emitting lossy bytes. Valid GCKM5 bytes remain unchanged. Source-authenticated caches and general optimizer translation certificates remain open.
  • Filesystem effects use held capability directories and document grants. Physical link traversal precedes parent components; supported rename replaces atomically without pre-delete. Document replacement owns bounded exclusive temporaries and cleanup. Package lock, VCS patch and WASI response consumers use held readers, and response limits cover the entire encoded body before decoding or field selection. Remaining manifest/module, GPK, pins, quarantine and external-process consumers still require migration.
  • Registry clients admit canonical identities, verify returned bytes across backends, reject every redirect, and bound streaming reads. Serving owns its listener and active connection; natural join waits, while stop/drop reaps the worker. Admission bounds bodies, framing, objects, sessions, reservations, deadlines and finish assembly. Slow bodies cannot renew deadlines. Actual CLI serving commits objects and supports finite completion.
  • Sync pull stages verified payloads in a request-owned file-backed overlay. Every root, ref head and closure batch passes transport, identity, budget and typed-commit admission before downloaded-object publication. Late corruption, missing objects and admission rejection preserve destination inventory. Staged duplicates preserve successful counters. Aggregate indexing retains the existing 50k allowance per selector; configured logical budgets apply before spool growth. Publication preflights integrity and charges completed writes. Later filesystem failure or ref conflict can retain verified writes; object/ref atomicity and arbitrary I/O rollback remain unproven.
  • Generated-state admission accounts for observed idle allocation, distinct live/requested growth and bounded metadata writes. Metadata reservations add frozen payload/temp/ancestor/journal costs. Type-admitted registry fields, policy digests, process identities, monotone build-reservation upgrades and denial preservation close reproduced admission defects. Reclamation re-reads physical free space and never treats deleted allocated bytes as guaranteed recovery. Generic build estimates and the free-space floor still need an operation-specific replacement.
  • Lifecycle and authority observers share bounded allocated-block accounting. Each inode is charged once within an observed forest; sparse logical length never substitutes for allocated blocks. Concurrent disappearance is a sampled race, while unknown metadata/backends fail explicitly. Directory descriptors hold child descent; lifecycle rejects links and aggregate observers count link entries without following destinations. Cancellation protects descriptor/iterator ownership transfer and closes resources. Root/deletion authority, policy quotas and reservations are preserved.
  • Nested telemetry and authority supervision own process groups, cancellation and rollback. Darwin telemetry uses an isolated unprivileged libproc helper with finite process/output/collection bounds and interruptible reads. Strict decoding rejects malformed, duplicate, contradictory and oversized inventory rows before committing a sample. Linux traversal checks stop. Darwin lifecycle birth identities use native information while preserving legacy digest bytes and retaining leases when identity is unknown.
  • Canonical replay teaching logs are recorded from unchanged programs/policies through a freshly built production CLI in isolated workspaces. Valid replay succeeds; the sole allow-to-deny mutation must fail at the decision/capability invariant. The complete candidate is validated before its explicit generated owner publishes both logs and the manifest. Historical runtime-identity/version compatibility remains open.
  • CI/full prerequisites declare the independent Python blake3 1.0.4 checker through an isolated read-only probe, bound to the reviewed four-wheel hash lock. Provisioning requires reviewed hashes, binary wheels and forced reinstallation. GB-8 admits that declaration and standard-library unittest/ctypes while retaining undeclared-module rejection and bounded compound-import controls. Core membership and SDK envelopes remain unchanged.
  • Borrowed contexts restore strict effects Clippy without suppressions or changed validation order, seals, budgets, policy or public/wire contracts. Refs custody controls reject decoy forwarding. Reviewed Windows feature-identity refreshes retain dependency membership, versions, edges and policy lists. Dependency locks and bootstrap/trust contracts are preserved.

Validation and its limits:

  • Fresh whole-pull negatives reproduce retained-prefix installation on the published pre-repair baseline in both execution tiers, one/four workers, multiple roots and nested closure batches. Repaired controls preserve names, bytes, native identities and strict error replay. Eight import-library tests, the 29-test sync suite, fourteen file-transport cases and final-source effects/registry all-target Clippy pass. Unchanged Rust coverage is reused by source identity.
  • Actual wasm32-wasip1 execution on pinned Wasmtime 36.0.9 passes six controls, including the fourteen file-transport cases in both tiers, within the retained 240-second runtime bound. Independent host snapshots preserve denied/outside fixture contents and identities. This is local actual-target behavior; Windows and cross-host release qualification remain unproven.
  • All eleven canonical valid/invalid pairs pass through the production CLI. Fourteen structural negatives and seven producer faults retain wrong-diagnostic, earlier-mismatch, missing/oversize/malformed log, wrong-version and accepted-mutation rejection. Teaching fixtures do not establish historical compatibility.
  • Telemetry passes 25 control groups, including nine helper, 26 native-contract, nine protocol, nine birth-identity and the existing cancellation controls. Native Darwin controls cover parent/child RSS, exited-helper cleanup and live/unknown-state rejection. Default, UTC and America/Chicago comparisons preserve legacy birth-identity bytes; no SDK qualification follows.
  • Allocated-block observation passes 16 control groups; cleanup passes all 59 groups; the authority supervisor passes 58 controls. These retain sparse/hardlink correctness, disappearance races, unknown-backend/permission rejection, directory replacement, finite cost, descriptor cleanup and actual acquisition-time cancellation. Reclaim-priority fixtures assert genuine allocation pressure in native and finite directory-overhead models. Engineering retains all 75 controls and its existing budgets.
  • One bounded offline CLI-build calibration completes six cases in 182.075 seconds and removes its private source/cache workspace. Cold sampled peaks are 827,838,464 bytes in the slim class and 2,018,762,752 bytes in the normal class. Source-replacement warm builds need up to 74,371,072 and 20,606,976 additional bytes despite almost stable final sizes. The byte-unchanged stages rewrite timestamps and do not prove untouched Cargo input state. Exact transient/COW maxima, full/strict/WASI/cross-host profiles and a universal floor replacement remain unproven. Failed calibration and candidate counterexamples remain retained.
  • The additive structural term codec matches 19 independent Python/BLAKE3 vectors and distinguishes 167 nested structural terms. Exact and one-short bounds, large scalar preflight, 20,000 nested pairs on a small host stack, wide-container traversal under constant frame ceilings, map ordering and historical canonical/value identities pass. The kernel suite passes 91 active tests with its existing ignored test retained; a subsequently added maximum-counter overflow control passes separately. Final all-target kernel Clippy denies warnings. Seven private semantic mutations and the kernel guard's nine controls reject their intended faults. No production runtime/log profile is activated by this component.
  • Fresh exact-source negatives reproduce the 22-versus-16,420 live-unit discrepancy, stale warm module bindings, three admitted malformed slot inventories and a 120,024-byte sparse slot allocation for one retained value. The published named-capture foundation stage passes 99 active tests with one existing ignored stress test, and warning-denied all-target Clippy passes. New controls cover exact/one-short live limits, both tiers plus blob round trips, finite/unlimited-step warm updates, nested sparse captures, sharing/shadowing, external scopes and retained-root/reclamation behavior. Seven private semantic mutations and the kernel guard's nine controls reject the intended faults. Source-decomposition and selfhost-boundary guards pass; existing source ceilings remain unchanged. The failed first allocation fixture and the preliminary report with subsequent source refinement remain retained separately from final-source observations.
  • Fresh published-source controls reproduce a name/depth contradiction that admits a retained 16 KiB capture under 2,048 live units at only 86 charged units (valid control rejects at 16,480), a stored-body/IR contradiction that hashes results 1 and 42 identically, and distinct improper-pair/nested-symbol values with identical serialized bytes. Final-source kernel verification passes 105 active tests plus one existing ignored stress test and warning-denied all-target Clippy. Eighteen body-field faults, root scope/inventory/tag faults, 104 source-form round trips and eight semantic mutations pass. Iterative exact-term equality agrees with 169 finite pairwise comparisons and traverses 20,000 nested pairs on a small stack; that observer excludes the inherited destructor boundary. TCB, decomposition and selfhost guards pass. Failed private fixtures and intermediate candidates remain retained; they do not replace final-source evidence.
  • The diagnostics verifier now enables its required structured-failure fixture feature and resolves the four CLI suites in one Cargo invocation, preserving separate production/parity entrypoints and including all 89 normal-driver library tests. Fresh grouped verification passes at 271,741 ms under the unchanged 300,000-ms runtime limit and 1,178,107,904 allocated owned bytes. The prior zero-test target and failed 313,481-ms candidate remain retained.
  • The grouped canonical candidate passes diagnostics at 81,214 ms but stops before promotion on inventory-group signal denial with a delayed leader exit observation. An actual native exited/no-live-group control with a forced nonblocking-poll schedule reproduces that rejection. Cleanup now admits one 0.5-second reap only after native confirmation of no live group members; live/unknown groups, other-host denial and timeout retain explicit failure. The full telemetry gate passes its original 25 groups plus six termination-schedule controls. Four semantic mutations are rejected, and the native exited-helper positive reaps status 0. Both failed canonical candidates remain retained.
  • The current final-source canonical V2 passes in 1,282.949 seconds: 24 generators, all 22 read-only checks and five owned promotions. Diagnostics runtime is 249,387 ms under the unchanged 300,000-ms limit (444,341 ms for the whole check, including build work). Sampled peak owned allocation is 1,710,743,552 bytes under the unchanged 8 GiB transaction budget; separate volume-wide decline is 4,701,958,144 bytes and is not attributed owned allocation. The complete 14-path diff preserves all 379 task states/dates/commands/owners, 124 audit classifications, gate budgets/predicates/input membership, dependency locks, trust material, current value/log profiles and replay fixtures. Three explicit validator modules receive new source ceilings without raising existing ones. Generated source line/hash updates are reviewed through their owners. Formatting and diff checks pass; changed-fast is inspected only as a dry run. This canonical transaction is the sole integration route for the final source revision. Reports and failed candidates are retained under .genesis/perf/f01-compiled-admission-*; none supplies independent acceptance.

Remaining work includes lossless runtime identity and versioned compatibility; all remaining filesystem consumers; total heap/RSS, physical allocation and I/O cancellation; measured build admission and shared-volume coordination; graceful CLI signals; object/ref/crash atomicity; documentation size and strict-golden work; supported-host/cross-host coverage; and retained independent acceptance. Hosted published baseline 1bae0dd6 remains red: documentation is 35,526,825 bytes above 33,554,432 bytes, standard CI rejects the unapproved io-lifetimes 2.0.4/3.0.1 duplicate family, strict-golden takes 608,374 ms against 480,000 ms, and the workspace job's pure run with a retired high-level capability allowlist unexpectedly exits 0 instead of 10 (11 other regression cases pass). Exact run/job metadata and four failed logs are retained under .genesis/perf/hosted-1bae0dd6-2026-10-03-*; their hashes bind retrieved observations without triggering another hosted campaign. The earlier f191d2f baseline remains retained. These facts were rechecked after a separate read-only review supplied the newer CI results and documentation results; that review does not supply independent recovery acceptance. The capability failure requires a fresh source-bound local negative before redesign. The local prerequisite guard rejects Xcode/SDK 27 and Apple Clang 21 against the preserved supported envelopes. None is resolved by weakening a gate or treating a local presence check as qualification. The PR remains draft; readiness remains not-ready with eight unresolved defects.

@corbensorenson corbensorenson changed the title Bind the correctness recovery roadmap and repair package-verify custody Bind correctness recovery, repair verification custody, and bound empty repetition Oct 2, 2026
@corbensorenson corbensorenson changed the title Bind correctness recovery, repair verification custody, and bound empty repetition Bind correctness recovery and enforce canonical admission before effect identity Oct 2, 2026
@corbensorenson corbensorenson changed the title Bind correctness recovery and enforce canonical admission before effect identity Core correctness recovery: admission, work bounds and independent checker repair Oct 2, 2026
@corbensorenson corbensorenson changed the title Core correctness recovery: admission, work bounds and independent checker repair Core correctness recovery: canonical admission, resource accounting and checker provisioning Oct 2, 2026
Validate object identities and bytes before backend dispatch or batch installation. Preserve store/package/sync error semantics; repair regular-file cache reclamation and generated active-defect visibility.

Fresh focused regression and mutation controls pass. One final canonical transaction passes 29 nodes and 27 read-only checks with budgets and claim/task states preserved. Recovery findings remain open pending complete independent acceptance.
@corbensorenson corbensorenson changed the title Core correctness recovery: canonical admission, resource accounting and checker provisioning Core correctness recovery: canonical admission and registry/resource boundaries Oct 2, 2026
@corbensorenson corbensorenson changed the title Core correctness recovery: canonical admission and registry/resource boundaries Core correctness recovery: rooted filesystem and document-write boundaries Oct 2, 2026
@corbensorenson corbensorenson changed the title Core correctness recovery: rooted filesystem and document-write boundaries Core correctness recovery: canonical identity, resource admission and filesystem boundaries Oct 2, 2026
@corbensorenson corbensorenson changed the title Core correctness recovery: canonical identity, resource admission and filesystem boundaries Core correctness recovery: identity, filesystem, registry and resource boundaries Oct 3, 2026
@corbensorenson corbensorenson changed the title Core correctness recovery: identity, filesystem, registry and resource boundaries Core correctness recovery: identity, filesystem, registry and validation Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant