Repository navigation
BS-73: decoding a semantic module from canonical JSON as source - #504
humuhumu33 wants to merge 19 commits into
Conversation
The decoding half of SemanticModule::parse (SPEC.md §17.12) as language-1.2 source in bootstrap/, ninth slice of the typed IR (#269): decodeSemanticModule of SemanticDecode reads a text and returns the module's specification and declarations, or the first of the parser's decoding rejections it meets (invalid semantic-module JSON under serde's rules, JSON that is not canonical, a member outside the closed schema with its path). The decoders are generated from the type description BS-71 and BS-72 write from, each recursing on a count of the document's values. The case runs every committed semantic module, example modules, the nesting bound, and mutations; 10 planted defects. Register row (502), SPEC §31 row and §17.23 prose, scenario, bdd part and lane, README count and range, CHANGELOG, VERIFICATION record, roots, the register-count mirrors; BS-72's enc_any made pub(super). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Two concrete source findings at9da8cdaa; neither is a newly executed runtime failure.
P2 — Preserve the oracle's rejection order. SemanticDecode.lex.tex:3951 starts with the strict canonical JsonRead reader and maps its rejection directly to DInvalid. That reader rejects whitespace and requires strictly increasing keys. The Rust oracle typed-decodes first and then compares canonical text, so valid but noncanonical text such as {"declarations":[],"spec": "lexlean/semantic-module/2"} reaches DNotCanonical before semantic validation. The current source instead rejects the space as DInvalid. Existing whitespace/reordering mutations already require this distinction. Repair the input-decoding/classification bridge without weakening the oracle or narrowing the promised domain.
P2 — Make the duplicate-member plant observable. semantic_decode.rs:405 disables checkKey's duplicate test, but JsonRead rejects duplicate/nonincreasing keys before checkKey is reachable. The selected case also constructs objects through serde_json::Value maps and has no explicit duplicate-field text. Consequently that plant cannot demonstrate the promised falsification on this path. After repairing the phase boundary, include a literal duplicate known field and plant the rule that actually owns its first rejection; reuse the existing case and qualified build rather than adding a separate gate.
The row/scenario/test/root/effect bindings are present, and validation after decoding is explicitly outside the claim. The stated top-of-stack, single-batch regeneration intent is useful; preserve that owner coordination and all applicable prior evidence. Current source preparation/case activity is not complete product or final-head acceptance, and #269 remains open. No build, runtime, log, writer, competing patch or CI action was performed by this review.
The standard library reader takes canonical text only, so a text out of canonical form read as invalid where the parser reports it not canonical. SemanticDecode now reads whitespace, members in any order, every escape including surrogate pairs, and integers without fractions or exponents; counts beyond the canonical range decode as invalid. Two plants and three non-canonical mutations cover the reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Source repair credited at701e6df3: readLoose now accepts whitespace, member ordering and escape differences before typed decoding/canonical comparison, addressing the strict-reader phase problem from5883129170. The new textual mutations and plants are useful; no runtime equivalence or regenerated-product acceptance is inferred from this readback.
The duplicate-member plant remains without a witness. Object mutations still go through serde_json::Value maps, while the new whitespace/escape/order strings retain unique known fields. Making checkKey reachable does not make disabling its duplicate guard observable on those inputs. Add a literal duplicate ordinary known-field string to the existing case and ensure that plant reaches this first rejection boundary; a duplicate kind field can be rejected earlier by kindOf and mask it again. Reuse the existing test/build and top-of-stack owner batch rather than a separate producer or new gate.
Current PR CI43343349855 failed, repro43343349798 passed; separate case run43343348434 is active and is not aggregated with PR CI. No failure cause is inferred from metadata, no logs or runtime were repeated, and no competing patch was created. Full final-tree qualification remains owed.
A repeated field, name, or tag in the mutated texts, so the plant that admits a repeated member is detected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
The remaining duplicate-member witness finding is structurally resolved at00f3c093. Literal spec/name/kind duplicate strings are now inserted as raw text, avoiding serde_json::Value map deduplication. The ordinary spec duplicate reaches closedKeys/checkKey; disabling that guard can proceed to canonical comparison and change DInvalid to DNotCanonical, so kindOf's separate duplicate-kind rejection no longer masks every witness. Register/scenario/spec/evidence wording now includes repeated-member mutations.
This five-file delta leaves the root implementation and generated products unchanged. It credits the source correction, not an independently executed plant or full serde-equivalence result. Keep the existing case run and top-of-stack batch as the qualification path; do not add a standalone producer or aggregate those runs with PR CI. The body still says10 plants while the source has12; update that summary with the next evidence checkpoint. Existing owner retains final qualification. No runtime, logs or writers were repeated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Bounded current-stack readback atc7b5278e: the seven advanced lanes #487–#504 carry the same five-path BS67 amendment, raising CERTIFICATE_BUDGET3145728→4194304 with claim/scenario mirrors and a documented3599711-byte maximum. This is a source acceptance-bound change, not generated-product adoption. The documented maximum is owner evidence; please link its preserved receipt and explicitly disposition the revised bound in source review. This readback does not accept a relaxed bound or request another measurement.
Current output closure is still incomplete: the exact root registry has85 roots against67 committed rust-std package directories (18 registered IR packages absent). The retained manifest records a1620-byte bootstrap config while current config is1849 bytes; its matching old term_text provenance remains scoped historical credit. No product path changed in this five-path delta. Finish run43351082071 atca8f and separate case43352267528 at910b both ended failure, separately from current PR CI43362168187. Their causes and successful partial-output disposition cannot be inferred from status.
Please provide the concrete failed-run/output disposition and smallest next repair in the existing canonical owner batch, retaining completed applicable outputs. Prior BS73 literal-witness/reader source fixes remain credited; broader BS67 native-corpus selection, Err(_) fallback versus absence-only wording, unsupported memory-cause wording and BS69 coverage findings are unchanged. Preserve original logs/measurements rather than repeating them; no automatic rerun, new producer, blanket regeneration or competing patch was initiated.
…code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Current stack preserves actual #419/b8b6 release history; the inspected IR implementation and cases are unchanged, so prior bounded source reviews remain applicable. The new bytes are not evidence that finish-ir outputs were adopted.
P2 — Restore the official claim projection after integration. Current CONFORMANCE.md (cfa9806b, byte-identical to release) contains none of BS65–73, while current model/ids.toml (b20c9ec0, rows2886–2937) contains all nine. This is a concrete R1 mismatch, independently of the cause of current CI/repro failures. Include the necessary official model projection in the existing canonical repair batch and review its emitted bytes; no second whole producer is needed to fix that projection.
The sampled bootstrap manifest is also inherited release data: semantic5d59/compiler06639 and a1598-byte config, versus the current1849-byte IR config. Retained term_text provenance still binds semantic64ebb/compiler635a; the earlier matching-manifest credit applies to its old inputs only. The exact85-root/67-package inventory still lacks18 registered IR packages. Finish43361968064 atd084 ended failure, separate from this PR's CI43372151285/repro43372151091 failures; no failure cause is inferred and no jobs are aggregated.
Existing owner should disposition the failed finish outputs, the retained source findings and the smallest necessary current product delta together. Preserve valid #419 and original IR evidence. No logs, writer, proof replay, competing patch or new execution was initiated.
|
Correction, 2026-11-13 10:10 UTC: I retract the earlier pipeline-status P2 in this comment. GitHub invokes this step with The actual new blocker is established by run43401399556 at a53a2ee. BS69 job131804838123 reports44813 rustc comparisons agreeing; BS70 job131804837640 reports4801. Each then fails the normative-host pinned-Lean prerequisite at support.rs2871, exits101, and does not complete the Lean-backed case. The log shows core_pattern=core and core limit0 in the BS70 step; no inference about older run0ac's failure cause is needed. Both jobs retained source-bound stack-bound artifacts:14026429747 (BS69) and14027955039 (BS70). Preserve and review those completed native measurements against the exact source/config/tool bindings. The smallest owner action is to restore the required pinned Lean4.32.1 availability for the remaining qualification and disposition the existing native outputs before choosing any necessary continuation. Do not weaken the normative-host assertion, claim full-case success, or blindly repeat the native measurements. Artifact metadata and native run counts alone are not product/final acceptance. This separate older preparation still does not resolve current #50493d's R1 projection/products or omitted cases. Existing owner retains the canonical repair batch. Two previously unread terminal logs were fetched once and retained; no local execution, producer, cancellation, rerun or competing patch was performed. |
Measured on GitHub runners: BS-66, 67, 68, 71, 72 from the regenerate/ r5-bs72-semantic-reasoning follow-up finish-ir.yml runs; BS-69, 70 from run 43401399556 with core dumps disabled (kernel.core_pattern=core, ulimit -c 0), after apport was found consuming memory on aborting stack-bisection probes. All seven cases agree with the Rust oracle on every run short of this step. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…code Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors the writer side added to bs69-semantic-terms. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A lexlean.lock records the source digests under the identity the compiler that built it used. Merging release across an identity shift (RB-24, then RegionCheck's PF-24) leaves the lock stale against the regenerating binary, which the gate's lock check (LLC0102) then catches mid-write, after the heavy recipes have already run. Build the lexlean binary once per writer job and relock bootstrap, stdlib, and compiler before the recipes run, so the capture step's `git add --all` carries the refreshed locks in the patch. Carried on this branch until release gets the same step; the stale ~/sh-ir-bin/lexlean copy used throughout this track's earlier checks is unreliable from here on. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
P2 — The new lock-refresh step breaks the regeneration parts' disjoint-path contract. regenerate.yml:74–80 runs the same three project lock commands in every matrix writer. Lines103–105 then capture each checkout's entire diff, including those shared lock changes. combine:134–137 applies all three patches in sequence with git apply --index. Whenever a refreshed lock changes, its old→new patch is therefore included repeatedly; after the first application, the next copy no longer matches the index preimage. Identical output bytes do not make repeated patch application valid. This is a source-level publication blocker, not a claim that the active run has already reached or failed combine.
Keep refreshed locks available to each writer, but give their published delta one explicit owner (or require identical shared bytes and include them only once). Preserve rejection of conflicting output ownership. Review and retain applicable outputs of the current run; this does not call for another proof run, wholesale producer replay or a competing implementation.
The new BS69 RegionCheck constructor/encoder/vector coverage is credited. Current official CONFORMANCE projection still omits BS65–73, and generated products remain incomplete for the composed IR inputs; include the necessary projection and output disposition in the existing canonical batch. Current regeneration43481945154 is active. New case-ir43480087650 failed, but no log/cause is inferred here. Earlier native measurements retain only their original input scope. The previous pipeline-errexit finding remains RETRACTED and is unrelated to this patch-ownership issue. Independent bounded receipt ir-stack-owner-delta-20261113T1850 records the changes; no checks, logs, producers or CI-control actions were repeated.
The stack's regeneration at this slice's tip (run 43481945154, r9) stopped bootstrap's production export at LLS8002: a part of its certificate is 12,928,150 bytes, beyond the 6,291,456 the project configured. Ilya approved 24 MiB. The project now configures max_file_bytes = 25165824, with room for the further slices of the typed IR. Which root produces the oversized part, and whether it has a structural fix as RB-24's validator twin did, is still open. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Bounded c6→e2e delta review: only SPEC, CHANGELOG, VERIFICATION and bootstrap/lexlean.toml change, raising the project certificate allowance from6MiB to24MiB. The owner attributes the preceding regeneration failure to an LLS8002 certificate of12,928,150 bytes; I have not fetched a new log or independently attributed that failure. The BS73 case's3MiB bound is unchanged, and the failing root is not established here, so this is not a finding that the case bound was exceeded.
Review5896118935's shared-lock publication P2 remains applicable: regenerate.yml is byte-identical01253d62, with all three writer parts capturing the same refreshed lock deltas and combine applying their patches sequentially. This delta does not fix the repeated-preimage problem. Disposition that single ownership defect, the current outputs and the official BS65–73 projection in the existing owner batch; no competing patch or repeat producer is requested. Regeneration43481945154 failed; replacement43498736513 is pending, not acceptance. Required final source/products/gates remain outstanding. The earlier pipeline-errexit finding stays retracted.
oneByte, and fifteen more hand-written helpers of the loose JSON reader (byteAt through readLoose), declared axioms their proofs never use, copied by habit from the stdlib reader this module was built from. Only the root, decodeSemanticModule, calls writeJson and needs them, matching every other module in this track. No semantic change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A reader built from an existing one inherits axioms it never needs: LLV7005 only catches an unused axiom when a root's own proof is examined, so the sixteen helpers copied from the stdlib reader this one models carried axioms silently until decodeSemanticModule itself was checked. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Bounded e2e→ea6→e7 review credits the targeted policy-declaration correction. ea6 removes17 helper axiom declarations without changing their terms; decodeSemanticModule retains [Quot.sound,propext]. Compiler/schema/exact-axiom checking is unchanged. A single saved production-job132138602522 log from43498736513 confirms LLV7005 for oneByte, trailing and utf8Of, each observed[]. That directly supports those three removals; it is not proof acceptance for all17 helpers or the root/export.
Please narrow the new VERIFICATION paragraph: it says the gate caught sixteen helpers, while this log names three and the source correction changes17. Distinguish observed failures from the larger source correction; preserve the original failure rather than recreating it. Saved log SHA2561b2faf485fd0b78156ed2ccecc08073c08c4af2e4c79494e190239d3470237ff will not be refetched.
The prior e2e oracle writer succeeded and records remain active; retain their exact input bindings and disposition applicability to the corrected source before repeating work. New regeneration43506256981 is queued on ea6, while e7 adds only verification prose. The shared-lock publication P2 remains unchanged in regenerate.yml: give shared deltas one publishing owner before combining. Current helper/root policies, necessary changed products, BS65–73 projection and final qualification still need acceptance. No observed combine failure, compiler weakening, completed native replay or reviewer-started producer is claimed.
…code CHANGELOG.md conflicted (union-resolved, both sides kept); SPEC.md and repository.rs auto-merged. The register-count mirrors need no change (505 -> 505). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every writer now relocks, but only production-export reports the lock paths it wrote, so combine no longer sees three identical captures of the same files. Byte-identical to #521's branch, so no conflict when it merges. This is the branch's own genuine case: nine slices shift the compiler-semantics ID, so every writer's relock step actually rewrites something, which the unfixed version would have handed to combine three times over. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
The shared-lock publication blocker is resolved at source: this head's regenerate.yml is byte-identical to #52159a33784, where all writers refresh working locks but only production-export publishes the three shared deltas. Records/oracles unstage exactly those paths; conflicting-output refusal remains. Actual6a02 history is also integrated through the IR merge chain. The complete16-path delta from e7 is automation/docs/repository-test/xtask work, without changing helper implementation, audit policy or generated products; prior scoped evidence remains applicable.
The small evidence correction in5897992043 remains: VERIFICATION still says the gate caught sixteen helpers, whereas the saved log names three and the source removes17 declarations. Correct that distinction without recreating the failure. Official BS65–73 projection, coherent current products and corrected helper/root policy qualification remain owed. Current regeneration43509954359 belongs to this9eaa head; review/disposition its outputs before any further producer choice. Earlier canceled/partial runs do not combine into acceptance. No logs, native comparisons, writers or gates were repeated.
r10 (run 43498736513) named exactly three loose-JSON-reader helpers whose proofs use no axioms: oneByte, trailing, utf8Of. The prior fix swept all seventeen helpers to no axioms instead of just those three, and r13 (run 43509954359) caught the opposite violation on the other fourteen (byteAt, digitAt, digitsUntil, hexAt, looseAdvance, looseEscape, looseInteger, looseKeyAt, looseStep, looseString, looseStringStep, looseValueAt, readLoose, unicodeEscape): their own proofs do use [Quot.sound, propext]. Restores the declaration on exactly those fourteen, relocks bootstrap, and corrects VERIFICATION.md's BS-73 paragraph, which wrongly said sixteen helpers carried copied axioms. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Bounded 9eaa→41a12 review credits the evidence-wording repair. The complete three-file delta restores14 helper axiom declarations without changing their terms, retains none for oneByte/trailing/utf8Of, and keeps decodeSemanticModule at [Quot.sound,propext]. VERIFICATION now distinguishes the three observed failures, the broader17-declaration sweep, and14 restorations. This supersedes the earlier broad17-removal source credit; it does not weaken exact-axiom checking. The new14-helper diagnosis is owner-attributed, not independently confirmed by another log or proof replay.
The shared-lock publication repair remains unchanged, and the earlier pipeline-status finding stays retracted. The lock delta changes only two top-level bindings. Existing current run43531029081 has oracle success with records/production active; preserve its outputs and complete helper/root policy checks, official BS65–73 projection and coherent adoption before final-head full qualification. The previous9eaa run's records/production failures are preserved, not combined with this run as acceptance. Current preadoption CI43531038752 failed generated freshness; repro passed.
No new source blocker was found in this delta. Correct the stale body’s ten-plants description when recording current evidence. Independent receipt: pr504-41a12-helper-policy-review/receipt.json. No logs, native comparisons, writer or gate repeated; existing owner retains implementation/qualification.
r14 (run 43531029081) stopped at LLS8002: lean exceeded the 300,000 ms
child_timeout_ms default checking the certificate of production root
LexLeanPreserve.Cf85a168f307e9378e5b0df3a87d82042.R62, observed at
300,004 ms. Per the user's approval ("Yes, 15 minutes"), the bootstrap
project alone now configures child_timeout_ms = 900000 (the
maintainer's decision); compiler and stdlib stay at the default.
Relocks bootstrap and records the decision in SPEC.md, CHANGELOG.md
and VERIFICATION.md's BS-73 section. R62's root name is still being
identified and is left as "to be identified" pending that.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
afflom
left a comment
There was a problem hiding this comment.
Bounded41a12→ee154 review: the complete five-path delta raises only bootstrap child_timeout_ms from300000 to900000, updates its project_config lock digest, and documents the change. Compiler identity, Lean pin, helper terms, exact axiom policies and shared-lock publication repair are unchanged. The existing three-none/fourteen-listed/root policy distinctions retain their source credit.
The commit attributes maintainer approval and an R62 certificate timeout at300004ms; this review has not independently fetched that log or established its authored-root mapping. The allowance is not a demonstrated structural fix or proof acceptance. Use the preserved failure evidence to name the actual root and disposition applicable completed outputs against the new configuration, retaining the structural question honestly. No blanket replay of native or axiom evidence is requested.
Old43531029081 has one oracle success, production-export failure and four cancellations; its sole published oracle artifact14090926727 remains nonexpired. Do not aggregate that run into acceptance. Current43543748916 is the owner's canonical continuation; coherent config-bound products, BS65–73 projection, helper/root acceptance and final full qualification remain owed. Update the stale ten-plants body when recording final evidence. Receipt pr504-ee154-timeout-delta/receipt.json; no logs, tests, producer or CI controls repeated.
|
Current ee154 regeneration43543748916 is terminal FAILURE: oracle writer succeeded, records and production-export failed, combine/counts/delivery skipped. Preserve the successful oracle artifact14095080808 (SHA256abebf9bf68f2fb172d2a125503e2b52005014c1c08b412780f0ece7d3baef4c4); this is not combined-product acceptance. One new production-export log was fetched and preserved (job132290037478, SHA2566fcaea2908ebbabd5551d99f36d6eff33932bad55d7dbec2a41f881e806cd0a2). It shows pinned Lean4.32.1 installed, all three locks current, then check-production-export --write running from02:28:29 until03:28:09, when the recipe receives signal15 and the runner reports shutdown. Capture/upload were skipped. This establishes interrupted execution; it does not establish another R62 child timeout, an OOM, or a particular administrative cause. The records log returned BlobNotFound on its single retrieval attempt; its cause remains unclassified. Smallest owner next step: establish the runner-shutdown/records termination cause and disposition retained outputs against exact source/config before choosing any necessary continuation. Preserve earlier native/R62/failed receipts and the current oracle result; no automatic retry, duplicate oracle writer, policy weakening, or timeout increase is requested. Diagnostic: pr504-ee154-terminal-diagnostic. |
BS-73, ninth slice of the typed IR (#269), stacked on #497 (BS-72).
decodeSemanticModuleofbootstrap/src/SemanticDecode.lex.texis the decoding half ofSemanticModule::parseas language-1.2 source. It returns the module's specification and declarations, or the first decoding rejection the parser reports: invalid semantic-module JSON under serde's rules (unknown, missing and duplicate members, tagged unit variants, the 128-level nesting limit), JSON that is not canonical, or a member outside the closed schema with its path.The decoders are generated from the type description that BS-71 and BS-72 write from. Each one recurses on a count of the document's values.
The case runs:
It also has 10 planted defects.
Draft: the stack is regenerated once at its top and lands as a batch.
🤖 Generated with Claude Code