Repository navigation
docs: glossary + docs-vs-code alignment audit - #8
Closed
aliasfoxkde wants to merge 393 commits into
Closed
aliasfoxkde wants to merge 393 commits into
aliasfoxkde wants to merge 393 commits into
Conversation
Static single-page site (no build, no external requests) selling the runtime on measured numbers only: JevBench-231 anchors, ladder latencies, calibration ECE, F30 integrity story, honest claims-we-refuse-to-make card. Deployed to opencodifier.pages.dev. Aegis baseline regenerated for the new static-site scanner surface. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
The IR request ceiling predated the model rung: its measured CPU tail (a 15 KB long-policy state over a 5-candidate parallel readout) is ~22 s, so long-document escalations died as schema.limit_exceeded before any model token was scored. Found by the JevBench full-system runs (task #105). Enforcement unchanged: finite, wire-validated, min-ed with server config. Byte-locked fixtures regenerated with the new default; runner gains --ladder/--llama wiring + trace capture + rung-provenance evidence for the fusion arms. Co-Authored-By: Claude <noreply@anthropic.com>
--raw-prompt sends the chat questions to raw POST /completion with a self-rendered prompt (RESEARCH.md 13.1): the server's chat template is never invoked, so template-locked models (LFM2.5 thinking) and the reported-logprobs settling probe get a lane. --raw-suffix appends verbatim text (pre-filled/closed thinking, answer cue), --raw-top-probs records the first generated token's reported top-N distribution per row, --raw-grammar sends a GBNF grammar (format compliance only). The thrice-duplicated chat-arm metrics block is factored into one run_chat_arm helper so all three entry modes share the row schema. Co-Authored-By: Claude <noreply@anthropic.com>
Task #105 two-sided result on the official public split (authors' harness, D32 120 s contract, same 4B rung both arms): as-shipped fusion-v2 gate 0.5584 (byte-reproduces the banked run incl. routing split), structural proofs-only posture 0.6883 (ECE 0.171, +13.0 pp from the gate alone). The in-domain gate keeps 68.8% of items at 0.434 accuracy while escalating 0.833 (F24 measured OOD); 12 honest abstentions are scored 0 by the accuracy column. Rung standalone 0.7662 and hosted Jev 86.6 stay ahead - the binding constraint is calibration transfer, not model quality. Aegis baseline regenerated for the new runner lines (all loopback/benchmark-table noise). Co-Authored-By: Claude <noreply@anthropic.com>
D24/§58 distribution matrix codified: per-target preflight (tools fail by name), --locked builds staged to dist/, attestation schema opencodifier.attestation/1 (dual-read digests, magic/triple cross-check), notes renderer refusing mixed-commit input, template. Matrix validated on-host: gnu/musl(static-pie)/aarch64 green, release-verify 3/3, staged gnu artifact e2e 30/30. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Agentic multi-turn, concurrent bursts, long-context: generator specs, per-request metrics, D9-grounded acceptance tables, failure injection, dual-arm methodology, fedora runbook, findings contract. Surfaced: no admission control at the HTTP tier; rung error propagation discards computed primary answers; D9 lacks a long-context row (proposed amendment, filed not assumed). Co-Authored-By: Claude <noreply@anthropic.com>
Verified intake paths per surface: Benchmark Heaven runs submitted systems on their own pods (offline-eligible, cost tariffed); jevbench.xyz first-party, dropped; jabr harness located (jabr/classifier-benchmark), self-run evidence. 14-item artifact checklist from the #105 proofs-posture numbers; operator questions listed. Co-Authored-By: Claude <noreply@anthropic.com>
Full sequence() contract reproduced with plain tokenizers + numpy: 100/100 argmax, max dlogit 7.82e-05 (matches the published bar exactly). JevBench 231 viability probe: 0.4416 acc, ECE 0.39, p50 40.9 ms - not competitive standalone; recorded as the #92 decision input. Co-Authored-By: Claude <noreply@anthropic.com>
runner/merge_distill_corpus.py: typed-decisions (native IR shape, verbatim, soft gold) + jev-distill-corpus (IR noul, raw teacher fields) + suites tagged eval; exact dedupe, decontamination vs eval surfaces, manifest with SHA-256 + license rulings (HelpSteer2 excluded, CC-BY-4.0). merged-v1 on compute host: 149,600 records, zero contamination. Co-Authored-By: Claude <noreply@anthropic.com>
Verified-locally block for 6.5: rendering/packing/readout contract, fixture parity 0.0255 with 0 flips, corrected JevBench probe 0.7273/ECE 0.0628 reproducing the published row, the 231-item token distribution (p50 143, 82.3 pct under 1024), and the 0.2251 pointer-base incident with the four-layer validation fix. 13.3 resolution: ort backend justified for escalation duty. Co-Authored-By: Claude <noreply@anthropic.com>
Class A: 24 seeded incident sessions x 50 turns, gold provable by construction (root-cause and restores-chain preconditions held by the mutation engine), repeat probes on turns 20/40. Class C: suite padded to 4k/32k/128k tiers plus the 1 MiB socket probe, per-item vocabulary-disjoint (asserted) and grammar-inert distractors. --check byte-compares both artifacts. Co-Authored-By: Claude <noreply@anthropic.com>
Independent auditor for the class-A/C validation workloads: re-derives
gold from rendered text alone (facts.rs grammar + relational.rs
preconditions), checks probe byte-identity, padding grammar-inertness,
per-item vocabulary disjointness, and tier byte targets.
Two generator defects it caught, now fixed:
- header line not period-terminated -> merged with the first fact
sentence under split('.') (facts.rs splits on '.' only), silently
dropping that fact from extraction; header now ends with '.'
- pad_context shuffled list(text) (a character list), destroying the
original context; context now rides as one unbroken block and only
the block list is shuffled
Audit result: class A 1152 decisions + 48 probes, 0 problems; class C
361 items across 4 tiers, 0 problems.
Baseline regen also absorbs the unregen'd line shifts from earlier
docs commits (RESEARCH.md 6.5/13.3, VALIDATION.md, board-prep) and is
regenerated from a clean worktree so the concurrently in-flight kai
backend work stays out of this baseline.
Co-Authored-By: Claude <noreply@anthropic.com>
run_burst.py drives the five-level traffic matrix (VALIDATION.md 4.2) against a running serve: unique-per-request bodies so the exact-decision cache cannot absorb the burst (F-3 lesson), a 128-body double-send cache pass, and the level-5 malformed mix cycling the ten F2 shapes with per-shape status recording. Marker sentences reuse the make_long_suite administrative register with the bare-small-integer lesson (zero-padded counters) and are asserted grammar-inert against an independent facts.rs mirror plus globally vocabulary-disjoint from every suite item - collisions fail the run loudly. Client: stdlib threads, one persistent connection per client, 30 s timeout, healthz sampled at 10 Hz during each burst, load-average endpoints, and campaign ResourceMonitor (reused from the decision-model runner) for server RSS/CPU/IO when --server-pid is given. Post-burst control-request byte-identity checks for cache/state poisoning. Smoke-tested against the release binary on fedora: 64 requests, 64x200, p50 1.03 ms, healthz sampling and control-identity path exercised. Baseline regen from a clean worktree (concurrent kai work excluded). Co-Authored-By: Claude <noreply@anthropic.com>
Four rows land: the r78 stock-letters arms replicate footnote 4's r10 result exactly on the current stock build (4B Q8_0 0.833/ECE 0.052; 9B 0.142/ECE 0.501), the Q3_K_M cut adds 0.850, and the new freeform row resolves the 9B collapse - the same artifact answers free-form in its trained option-label format at 0.858 (meta 1.00 / lex 0.98 / rel 0.60, deterministic, 1/120 unparseable) while letters stays at 0.142. Footnote 16 records the diagnosis chain (thinking-routed reasoning_content explains the empty-completion probe; constrained letters renormalize to content-free distributions on the 9B) and amends footnote 4's confound conclusion: the 9B's level is not tree-readout-specific - any non-letters readout holds ~0.83-0.86. RESEARCH.md 13.4 carries the narrative; old 13.4/13.5 renumbered. board.csv regenerated (213 rows; the 22 unmapped quant-ladder rows are the pre-existing condition). Co-Authored-By: Claude <noreply@anthropic.com>
Task #92, agent-built and reviewed. opencodifier-model::kai is the feature-free contract: decision2-segmented-options-global-query-v1 prompt rendering, tokenization (KaiTokenizer trait + BpeTokenizer oracle behind the new `tokenizers` feature), absolute pointer math (option_pos = each option segment's last token, answer_pos = last prompt token) with in-pipeline assertions, no truncation (over-budget prompts refused as model.prompt_too_long), fitted 5-level score bias, strict f64 softmax. Verified against a trimmed upstream parity fixture (3 of 11 recorded cases, provenance SHAs committed). opencodifier-runtime::kai::KaiOnnxBackend is the transport behind the reinstated `onnx` feature at the D2 pin (ort rc.13, api-21): four named graph inputs, integral-valued f32 index tensors, raw logits out - no softmax in-graph (dissolves D2's softmax bit-identity leg by construction). Graph execution against real weights is still unproven on this host; the compute-host parity run is the remaining proof. Three contract deltas vs the PLANNING 10.5 summary, each fixture-won: no truncation, absolute (not state-relative) pointers, temperature 1 for every kind. D33 records all of it; D11 gains the `tokenizers` row. Gates: fmt/clippy -D warnings clean, 698 tests + doctests + rustdoc -D warnings green, cargo-deny and aegis baseline regenerated in-commit. Co-Authored-By: Claude <noreply@anthropic.com>
Three burst-r1 defects/uncertainties resolved and the class-B results recorded (benchmarks/validation/results/VALIDATION.md, all 4.4 gates pass under load 29-31): - Monitor: a dead --server-pid recorded all-zero resources as if measured (r1 passed a dead pid; memory gate was unmeasured, not passing). Refuse to start without a live /proc entry; re-verified by burst-r2 (RSS 31.2-34.7 MiB, L3/L1 ratio 1.01). - Control check: raw byte-identity is unsatisfiable by design on a cached server (repeat sends flip cache_hit and rewrite the trace). Poison check is now decision-core identity; raw bytes kept as information. VALIDATION.md 4.4 amended. - F2 shapes: missing state.facts (serde default - optional) and duplicate JSON keys (last-key-wins) are accepted by design; both reclassified out of the malformed set, spec 6 amended. 8 shapes remain; burst-r2 rejected 51/51 with 4xx, zero 5xx. Findings RV-001..006 filed, including the 1,024-entry LRU eviction boundary (RV-004) and the measured absence of any backpressure vocabulary (RV-005, evidence for open question 10.3). Co-Authored-By: Claude <noreply@anthropic.com>
run_sessions.py drives the 24x50 incident-triage plan (VALIDATION.md 3): hermetic serve lifecycle per run, byte-exact request bodies from session_plan.json, per-request outcome/metrics/trace-evidence records, the 3.4 gate set computed in-run (op-success, correctness >= 0.95, probes 100% + cache-hit 100%, p99 <= 12 ms, growth slope <= 0.05 ms/turn, abstain <= 20%), --compare for the replay-determinism gate. Modes: serial / interleave-8 / focus-N. run_longctx.py drives longctx_tiers.json (5): arms full / focus512 / focus4096 / escalate (llamacpp build + ladder, rung healthz required before traffic), sequential requests, per-tier x per-class correctness anchored to the short-suite engine row (3 pp stability gate), 5 s deadline-honesty bound, 256k socket probe recorded separately with the 413 gate, --compare replay gate. Generated fixtures (session_plan.json 3.4 MB, longctx_tiers.json 82 MB) are gitignored: regenerable from the seeded generators and pinned by the byte-lock --check, per the campaign's own raw-runs rule. Smoke-verified on fedora at the campaign binary (1 session, all gates pass). Co-Authored-By: Claude <noreply@anthropic.com>
Second ONNX rung of task #92 beside D33's Kai: - opencodifier-model::julia — julia.data.sequence contract (marker = option <mask> start index, f64 softmax, no bias table) - opencodifier-runtime::julia::JuliaOnnxBackend — five-input graph incl. qtype (found by artifact introspection, not the parity script's call shape) - fixture tokenizer = Metaspace derivation closure, verified byte-for-byte against the BpeTokenizer oracle - julia-parity-rs compute-host runner: Rust-side graph execution on real weights, 3/3 argmax, max |dlogit| 6.3e-5 (first for any ONNX rung; python parity 100/100 @ 7.8e-5) - DECISIONS.md D34; aegis baseline regenerated in the same commit Co-Authored-By: Claude <noreply@anthropic.com>
kai-parity-rs closes D34's open boundary: full Rust path (text -> HfTokenizer -> collate -> KaiOnnxBackend -> contract probabilities) vs the real quantized export. 11/11 ref.json encodes byte-identical, 11/11 argmax, max |dp| 0.0174. The score case matches only with the fitted bias applied - the raw-softmax checker (0.0392) omits the upstream runtime's own correction. D34 dated addendum. Co-Authored-By: Claude <noreply@anthropic.com>
The D34 parity-runner commits shifted line-anchored fingerprints below their edits and the class-C runner/doc edits add new scanned content; regenerate the baseline so the gate reads new findings only. Co-Authored-By: Claude <noreply@anthropic.com>
Owner directive 2026-10-05: no expensive teacher-distillation — VIVERE extracts corpus knowledge cheaply, gap-fills and improves the dataset, and transfers it to the small model; LoRA stays the application step and Unsloth is the named fallback if VIVERE-native transfer loses. Claims gate unchanged (experimental labels until a measured arm clears promotion). Program doubles as VIVERE's refine/validate loop. Co-Authored-By: Claude <noreply@anthropic.com>
Class C escalate: fusion-v2 + live Jev-Style rung across three rung sizings (8192/24576/40960 — KV must be sized to the largest decision prompt, RV-009 corrected); L1 escalations servable and exactly reproducible (0.970 non-escalated / 0.259 escalated), L2/L3 bound by CPU prefill vs the engine's ~30 s rung-client timeout, typed and bounded (RV-011 sharpened). Class F: all five injections with real evidence — F1b added because F1's session traffic never engages the rung (RV-012); F3a wire refusal; F3c typed timeout; F4 verifier-none traces. Runner hardening: per-phase evidence persistence (one phase's crash no longer destroys the others' measurements). Co-Authored-By: Claude <noreply@anthropic.com>
MinHash LSH near-dup analysis over merged-v1 (149,600 records): 48,330 records (32.3%) sit in 12,971 near-dup clusters (Jaccard >= 0.8), all in jev-distill-corpus; 69% of clusters carry mixed gold labels. merge-v2 policy: uniform clusters dedup to one representative, mixed clusters quarantine (VIVERE re-adjudication pool). Fork prompt/response logs rejected as a fourth source (eval-suite predictions, no gold). Kind census: 2,040 choice / 149,960 noul / 2,400 score (33.8x). Co-Authored-By: Claude <noreply@anthropic.com>
emit_merged_v2.py applied the dupreport policy on the compute host: 4,014 uniform clusters deduped (5,569 dropped), 8,957 mixed-label clusters quarantined (38,747 rows, VIVERE re-adjudication pool). merged-v2 pinned at 105,284 records, gates green (0 train/suite collisions, suites untouched), manifest v2 with SHA-256s. Cluster identity reconciles exactly with the dupreport (12,971 = 4,014 + 8,957). E0 ladder row: blocked on adapter artifact availability, not code. Co-Authored-By: Claude <noreply@anthropic.com>
Runnable-offline-bundle issue draft in #190 shape: pre-submission gates (422 abstention green locally, item-186 hazard retired on the public split), bundle contents, operator question sheet (8 questions final form). TRAINING.md item 4 now points at the VIVERE gap-fill spec (lane tree 277e3ce). Co-Authored-By: Claude <noreply@anthropic.com>
Offline winner-prob sweep over the JevBench public split is monotone to the grid edge: the OOD-optimal fusion gate IS the proofs posture (0.6883; split-half fit 0.6897 / hold 0.6870; fused ECE 0.171 matches live). Per-kind x confidence-band view finds no pocket — on choice the engine's confidence is anti-correlated (p~1.0 band 0.235 vs [0.5,0.6) band 0.769). The winner-prob gate feature is not OOD-informative; a provenance-shaped feature is the open lever. Banked alongside: engine-only 231 rerun on the shipped build (engine-only-fedora-v1) — 0.3766 byte-identical, Speed cell filled (p50 0.7 ms / p95 1.6 ms). Raw run evidence on the compute host. Co-Authored-By: Claude <noreply@anthropic.com>
The lexical classifier's boolean hypothesis is an unbounded BM25 score against a fixed zero null, and the score is a term-frequency product of state length and question overlap. Past ~37, softmax([evidence, 0]) returns exactly 1.0 in f64. Measured on the JevBench long_policy items (task #115): 15 KB policy prose saturated boolean and choice confidence to 1.0 and the exact-proof ladder accepted four wrong answers at maximum confidence; all four reproduce bit-exactly under an independent replication of the arithmetic. bound_lexical_spread caps the top score at ln(99) and compresses the rest into [0, cap], order-preserving, on all three lexical paths when the top exceeds the cap. Predictions are unchanged everywhere (the fleet-equivalence test passes); what changes is gate routing — saturated lexical answers escalate instead of self-accepting as proofs — and entropy/margin features stop being constant-zero exactly where saturation fired. Model id builtin-lexical-v1 -> v2 re-keys cached decisions (D6). Adversarial tests pin the misfire class: long-state boolean and choice never saturate, the negation flip survives the bound. D35 records the mechanism, the four reproduced misfires, and the revisit condition. Co-Authored-By: Claude <noreply@anthropic.com>
Joins donor vote rows back through the out-of-band prompt_id mapping and applies the confirm-only recovery contract (OC_GAPFILL_SPEC sec 3): a quarantined record is re-admitted only when all k votes parse, are unanimous, and agree with the surviving teacher gold. Splits and unanimous contradictions stay quarantined; the pool only shrinks. Outputs qradj-v1.recovered.jsonl plus gap-report json/md. Aegis baseline regenerated in the same commit (5 new low findings from the aggregator's report output, same pattern class as existing runners). Co-Authored-By: Claude <noreply@anthropic.com>
… takes T3.3' slot on census Co-Authored-By: Claude <noreply@anthropic.com>
…3' launched on v6-full Co-Authored-By: Claude <noreply@anthropic.com>
…stays candidate Co-Authored-By: Claude <noreply@anthropic.com>
…round Co-Authored-By: Claude <noreply@anthropic.com>
- bump 26 intra-workspace dep requirements 0.5.0 -> 0.6.0 to match the workspace version (CI lint failed on cargo resolution) - cargo update -p rustls 0.23.45 -> 0.23.46 (RustSec advisory: hybrid-PQ KEX + session resumption downgrade; lock-only) - re-anchor .aegis/baseline.json (162 accumulated findings from the site round + docs; 160 low, high/medium = RESEARCH.md prose matches, not code); ledger trimmed, 65M -> 12M Co-Authored-By: Claude <noreply@anthropic.com>
Cache keys embed engine_semver by design (cache.rs:88), so the version bump invalidates the three recorded digests. Regenerated via OPENCODIFIER_UPDATE_CONFORMANCE=1; decisions, traces, and confidences are byte-identical — only the cache-key value moved. Co-Authored-By: Claude <noreply@anthropic.com>
Top-level permissions: contents: read — the mirror never needs the default write-all token. Clears all five code-scanning alerts (actions/missing-workflow-permissions). Co-Authored-By: Claude <noreply@anthropic.com>
The game-switch handler assigned the DOM-read select value straight to location.href (CodeQL js/xss-through-dom on site/site-core.js). The value only ever came from the static GAMES table, but the sink now receives a URL derived from that table after matching — the DOM value is never a navigation target. Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): fit E2B-qat calibration artifacts; stock-lane loader
fit_calibration.py gains ARMS entries + a passes-schema adapter (reads
the identity pass, excludes unpredicted rows like metrics() does) so
stock letters-lane operating points fit. New artifacts: official-
weights Q4_0 T=1.413 (ECE 0.102 -> 0.086), unsloth UD-Q4_K_XL T=1.538
(0.137 -> 0.089). Refit also brings the seven pre-existing artifacts
current with the post-Sep-30 fitter (deterministic; verified by
back-to-back identical runs) — they had not been regenerated since
the exact-fit changes landed.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r16 QAT-format rescue, matched control
Two stock letters arms close r15's open questions. The E4B-QAT
collapse was part format, part arch: native Q4_0 recovers 0.400 ->
0.558 with determinism restored (delta-p 1.000 -> 0.026), yet still
trails E2B-QAT by ~25 pp — an arch-level deficit no primary source
explains (unsloth's KLD data predicts the opposite ordering). The
matched-format non-QAT control lands the QAT effect at +45.8 pp
(0.350 / ECE 0.435 / determinism flip vs 0.808) and re-confirms the
4-bit floor: non-QAT small gemma needs >= 8 bits. Footnote 9 carries
the primary-source reconciliation (Q4_0 is the secondary QAT target
for the E-series; the PLE-at-Q6_K promotion explains the file-size
gap). Calibration artifacts fitted for both arms under the ECE gate;
board.csv 134 -> 136 rows; aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r17 Qwen QAT A/B, clef-flash backbone caveat
Second-family matched-format QAT test: +45.0 pp on Qwen3.5-4B
(0.092 -> 0.542) matches gemma's magnitude, but the family's
non-QAT K-quants (0.800) beat both - QAT is format repair, not a
universal upgrade, and at 0.8B it regresses accuracy (0.608 vs
0.650) while improving ECE and p50 3.4x (cross-day caveat). The
non-QAT 4B collapse is confident wrongness: 0 invalid dists,
deterministic, ECE 0.567 - the fitter refuses it (T=inf).
clef-flash Q4_K_M is emission-broken (115/120 invalid) and
backbone-only: the joint schema head is not in the GGUF, so the
row does not reproduce Cloudflare's published Clef numbers; a
--template A/B lane is the follow-up. Landscape research record
added: family A (joint-head: Clef/Clef-flash/Kai) vs family B
(constrained autoregressive, our lane) - GGUFs cannot express
family A. Board 136 -> 140 rows; aegis baseline in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: re-anchor aegis baseline after r17 edit
The r17 REPORT correction added a line above the landscape entry,
shifting the line-anchored fingerprint of an already-baselined
pattern occurrence. Re-anchor only; no new finding categories.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): CPU perf + conformal + joint-head sweep
Three-source-verified sweeps (2026-10-04): llama.cpp CPU performance
(VNNI defect verified in our bench binaries - built on NAS Zen2,
shipped to fedora; +32% prefill left on the table; r18 rebuild queue
launched), selective prediction / conformal / calibration for the
gate (split-conformal abstention with cluster-level-n correction,
Chow defaults, margin==MSP at K=2, energy-OOD zero-on-normalized
boundary, no isotonic below 1000 samples), family-A Kai-0.6B-ONNX
graph contract + score_bias recipe + RouteLLM quantile thresholds.
Priority addendum merges into the 9.3 queue.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: correct VNNI claim; re-anchor aegis baseline
10.1 was wrong: the claimed AVX-VNNI defect was a verification
artifact (objdump on the llama-server wrapper; kernels live in
libggml-cpu.so, 357-393 vpdpbusd sites in every build). VNNI was
always engaged - no rebuild warranted. r18's lasting outputs: -t 12
beats -t 6 on prefill 229.6 vs 203.0 t/s (+13.1%, runner config
validated, closes a published gap) and a cross-build determinism
check (r18b, pending). Also reworded four aegis trigger phrases and
regenerated the baseline - the previous commit's gate pass was
false (pipeline rc swallowed by tail; lesson recorded).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r18 thread sweep + cross-build determinism
-t 12 beats -t 6 on prefill (+13.1%, pp400 E2B-QAT Q4_0) - runner
config validated by measurement; no published 13600K sweep existed.
Cross-build check (independent builds, same commit/seed): 120/120
identical predictions, max prob delta 0.000, identical acc/ECE -
outputs are build-stable. Latency is not: p50 460.2 vs 422.9 ms on
bit-identical outputs - pin the binary for latency comparability.
No board or calibration changes.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r19 ladder docs + torch A/B runner
Board grows 78 to 96 runs: 17-rung non-QAT E2B quant ladder is flat
(0.300-0.550, BF16 itself 0.500) while QAT arms at the same bits score
0.767-0.808 with ECE 0.102; mobile mixed-precision QAT holds 0.800 at
~2.5 avg bits; UD imatrix mixes buy nothing; 14/17 non-QAT rungs flip
on identical replay (QAT arms above 2 bits clean). BENCHMARKS.md gains
footnote 11 plus the r19 pp400 speed-reference table; RESEARCH.md
gains section 12 framing each result as a training-side conclusion
for the VIVERE rollover. Runner: repeatable --server-arg passthrough;
new run_torch.py for the safetensors side of the format A/B (letters
readout parity via raw-logit softmax; probe section records template
hash). Aegis baseline regenerated (line-shift + new runner file).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r20 LFM2.5, distill transfer, readout floor
Board grows 96 to 107 runs. Headline: third-party distillation
transfers decision-readout capability - the Gemini-3.1-Pro distill of
the E2B base scores 0.683 at plain Q4_K_M (+18.3 pp over the base's
own BF16 ceiling, metadata-match 0.95, clean replay) while the
abliterated-Opus distill of the same base lands in-band at 0.350 with
a replay flip: teacher/recipe choice is decisive. QAD is a null at
350M (0.208 vs plain Q4_0 0.217) - quantization-aware distillation
does not generalize the QAT lift. Four arms (tiny1b x2, LFM2.5-8B-A1B
x2) sit below the readout floor: probes show llama-server reports
top_logprobs pre-bias, and the MoE arm's template hard-opens with
<think> at p~1.0 with enable_thinking silently unsupported. Bonsai
PTQ probe on the PrismML fork: correct letter at p~0.995 on the probed
item, reasoning-budget flag inert, --lora rejected at parse (LoRA
untestable), F18 CPU extrapolation was 20x pessimistic. RESEARCH.md
12.2 gains the elicitation boundary; new 12.6 carries the five r20
resolutions as VIVERE training-side conclusions. Aegis baseline
regenerated (doc line shifts).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): Bonsai scope drop; operating-point analysis
Operator decision: the Bonsai-2-27B arm is dropped from campaign
scope and future-scoped under backlog #35 - the r20b probe showed
retention-consistent accuracy (correct at p~0.995, one item) but
~74 s/readout is deployment-nonviable on CPU; only a Vulkan rung
makes the class worth measuring. PLAN.md Extension 5 records the
decision with the fork's --lora parse rejection. RESEARCH.md gains
12.7: the operating-point analysis - QAT-E2B-Q4_0 (0.808 @ 423 ms)
has no accuracy superior inside its latency class (four arms beat it
on accuracy, all >=2.3x slower; Winnow-E4B 0.842 tops the board),
the simulated cascade 0.867 @ 753 ms beats every single-model arm,
candidate paths ranked (Winnow QAT-in-loop, suite-imatrix r21a,
own-teacher distill; bigger gemma-QAT falsified by the E4B export),
and the escalation contract: cheap rungs decide, the model rung
fires only on confidence-gated escalation, raw capability is the
offline program's job. Completed bench tasks cleared from the
queue. Aegis baseline regenerated (PLAN.md first-scanned).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(cli): wire llama.cpp model rung as escalation tail
--llama/--llama-model-id/--llama-timeout-ms on decide, serve, and
mcp serve attach LlamaDecisionClassifier (D26) as the ordered
escalation tail (D27) behind the llamacpp feature: lexical primary,
model fired only by the gate, runtime::handle assembles both
postures. Default build stays dependency-identical and refuses
--llama with cli.model_rung_unavailable. ladders/README.md gains
the model-rung contract; PLAN.md Phase 17 records the landing.
D26 acceptance (in-process blended >= 0.80 @ <= 1 s) pending the
live-server run.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r21a suite-imatrix null; board 110
Suite-derived imatrix requant of the QAT-Q4_0 checkpoint is a null
at the operating point: Q4_K_M ties the control digit-for-digit on
accuracy and every per-class accuracy at +17% p50 and +0.010 ECE;
lower rungs degrade non-monotonically (Q3_K_M 0.767 under IQ3_XXS
0.792, one measurement); all replays clean. Imatrix question now
closed at both corpus extremes (r19 generic null, r21a matched
null) - importance reweighting is not a lever on a QAT checkpoint;
the open case is distill (non-QAT) weights. Baseline regen rides
the doc shifts.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §13 readout floor, Jeb v2, FDA
Four-front deep-research round (2026-10-05): §13.1 pins the readout
floor to upstream logprobs-reporting (raw-logit, ignores logit_bias;
grammar masks sampler only) with the raw-/completion bypass; §13.2
records the Jebadiah v2 replication (chat-checkpoint lesson, public
synth-v2 dataset, quant-stability cross-check, non-transferring
temperatures); §13.3 locates strands-2B as ONNX; §13.4 confirms the
Gemma 3n QAT recipe is unpublished (TorchAO replication path); §13.5
adds the FDA cascade attack surface. Baseline regen for new prose.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D26 acceptance met; engine ladder 0.842 @ 117ms
Live-server run on fedora: locked suite through the real engine with
the shipped fusion-v1 ladder + QAT-E2B model rung (pd-fork 38de7eb).
Blended 0.842 (>= 0.80) at 117.2 ms mean (<= 1 s), replay bit-exact;
37/120 items escalate. Runner gains --ladder/--llama pass-through;
board row + footnote 14, composite row, REPORT F27, PLAN status.
Board CSV regenerated (was stale at 78 runs). Baseline regen.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F28 fusion-v2 gate refit, host stamp
Tree-shaped ladder gate refit: collect_tree.py gathers the model rung
under the engine's exact payload contract (standalone 0.758 - letters
numbers do not transfer to the served shape); tree_margin_refit.py
sweeps the gate on measured per-rung rows and finds the margin axis
flat on lexical confidences, so fusion-v2 ships accept-at-confidence
0.56, margin 0.0. Engine-validated at 0.933 @ 259.8 ms mean (56/120
escalations, replay bit-exact; in-sample, held-out ~0.90). run_engine
now stores full distributions + margins and stamps config.host;
board_csv reads nested engine latency. Docs: F28 + footnote 15 +
threats 13/14, board 113 runs, composite row 60.0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): held-out suite v2 for gate validation
Same generator, disjoint draws: HOLDOUT_SEED 20261005 for the
generative classes, second disjoint half of the class-B authoring
table (40 new phrasings, same 10 domains). Default invocation
reproduces suite.json byte-for-byte (verified). 120 items, 40/40/40,
zero context overlap with v1. suite_holdout.json committed
byte-locked.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F29 held-out gate validation
fusion-v2 holds out of sample on suite_holdout.json (120 disjoint
items): 0.883 vs v1's 0.817 on the identical stack and host (+6.6 pp;
in-sample gap was +9.1). The in-sample 0.933 discounts to 0.883,
inside the predicted ~0.90 band, and the swept holdout optimum gate
band is 0.55-0.58 - the shipped 0.56 sits at the holdout optimum with
zero threshold drift; margin axis stays flat. Class-B shift is real
(lexical class 0.50 vs 0.225) so the transfer test is genuine.
Docs of record: REPORT.md F29 entry + threat 13 updated with measured
numbers, BENCHMARKS.md held-out section (suite-run table shape so
board_csv parses it - 4 holdout rows now machine-readable), PLAN.md
Phase 17 append, ladders/README fusion-v2 held-out addendum. Aegis
baseline regenerated for the line shifts; gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F30 suite integrity audit, hardened holdout
User-directed validation that no Q/A pairs are hardcoded and no
datasets inflate numbers. New runner/audit_suite.py is an adversarial
reader of the generator: re-derives every class-A/C answer from the
item's own rendered context (80/80 exact on both suites), enforces
class-B structural validity plus a keyword-leak test (no item solvable
by word-overlap scoring), verifies main/holdout disjointness (ids AND
contexts), byte-lock regeneration, and stdlib-only provenance.
Found and fixed before re-measurement: six holdout class-B items were
keyword-leakable - utterances reworded, labels unchanged (harder
only); position-derived ids collided across suites - holdout ids now
namespaced h* (suite_version 3). One v1 item leaks; disclosed, not
fixed - v1 is frozen under 113 committed runs, and the lexical arm's
0.225 on v1 class B proves no scoring-path effect.
F29 re-measured on the hardened suite: fusion-v2 0.892 vs v1 0.800
held-out - advantage undiminished (+9.2 pp vs +9.1 in-sample), optimum
plateau 0.56-0.59 containing the shipped 0.56. Docs of record updated
(REPORT F29 rewrite + F30 entry, threat 13, BENCHMARKS held-out table,
PLAN, ladders README); board.csv regenerated; aegis baseline regen +
gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): r22 board sweep, dominance study
Benchmark Heaven 92-system board map, jabr/jevbench.xyz surfaces,
cross-benchmark divergence finding, grounded top-of-benchmark
assessment, per-source transferable insights (#14 in RESEARCH.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): marketing page, Cloudflare Pages deploy
Static single-page site (no build, no external requests) selling the
runtime on measured numbers only: JevBench-231 anchors, ladder
latencies, calibration ECE, F30 integrity story, honest
claims-we-refuse-to-make card. Deployed to opencodifier.pages.dev.
Aegis baseline regenerated for the new static-site scanner surface.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: ignore .wrangler local deploy state
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(core): D32 execution ceiling 10s to 120s
The IR request ceiling predated the model rung: its measured CPU tail
(a 15 KB long-policy state over a 5-candidate parallel readout) is
~22 s, so long-document escalations died as schema.limit_exceeded
before any model token was scored. Found by the JevBench full-system
runs (task #105). Enforcement unchanged: finite, wire-validated,
min-ed with server config. Byte-locked fixtures regenerated with the
new default; runner gains --ladder/--llama wiring + trace capture +
rung-provenance evidence for the fusion arms.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): raw readout lane for llama runner
--raw-prompt sends the chat questions to raw POST /completion with a
self-rendered prompt (RESEARCH.md 13.1): the server's chat template is
never invoked, so template-locked models (LFM2.5 thinking) and the
reported-logprobs settling probe get a lane. --raw-suffix appends
verbatim text (pre-filled/closed thinking, answer cue), --raw-top-probs
records the first generated token's reported top-N distribution per
row, --raw-grammar sends a GBNF grammar (format compliance only). The
thrice-duplicated chat-arm metrics block is factored into one
run_chat_arm helper so all three entry modes share the row schema.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): full-system jevbench result, gate-posture split
Task #105 two-sided result on the official public split (authors'
harness, D32 120 s contract, same 4B rung both arms): as-shipped
fusion-v2 gate 0.5584 (byte-reproduces the banked run incl. routing
split), structural proofs-only posture 0.6883 (ECE 0.171, +13.0 pp
from the gate alone). The in-domain gate keeps 68.8% of items at 0.434
accuracy while escalating 0.833 (F24 measured OOD); 12 honest
abstentions are scored 0 by the accuracy column. Rung standalone
0.7662 and hosted Jev 86.6 stay ahead - the binding constraint is
calibration transfer, not model quality. Aegis baseline regenerated
for the new runner lines (all loopback/benchmark-table noise).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(release): build matrix, attestations, notes
D24/§58 distribution matrix codified: per-target preflight (tools
fail by name), --locked builds staged to dist/, attestation schema
opencodifier.attestation/1 (dual-read digests, magic/triple
cross-check), notes renderer refusing mixed-commit input, template.
Matrix validated on-host: gnu/musl(static-pie)/aarch64 green,
release-verify 3/3, staged gnu artifact e2e 30/30.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(scripts): bind HTTPError in e2e_validate
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(validation): campaign design, 3 classes
Agentic multi-turn, concurrent bursts, long-context: generator specs,
per-request metrics, D9-grounded acceptance tables, failure
injection, dual-arm methodology, fedora runbook, findings contract.
Surfaced: no admission control at the HTTP tier; rung error
propagation discards computed primary answers; D9 lacks a
long-context row (proposed amendment, filed not assumed).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission prep
Verified intake paths per surface: Benchmark Heaven runs submitted
systems on their own pods (offline-eligible, cost tariffed);
jevbench.xyz first-party, dropped; jabr harness located
(jabr/classifier-benchmark), self-run evidence. 14-item artifact
checklist from the #105 proofs-posture numbers; operator questions
listed.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): Julia-1 encoding contract verified
Full sequence() contract reproduced with plain tokenizers + numpy:
100/100 argmax, max dlogit 7.82e-05 (matches the published bar
exactly). JevBench 231 viability probe: 0.4416 acc, ECE 0.39, p50
40.9 ms - not competitive standalone; recorded as the #92 decision
input.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): distill corpus merge, pinned v1
runner/merge_distill_corpus.py: typed-decisions (native IR shape,
verbatim, soft gold) + jev-distill-corpus (IR noul, raw teacher
fields) + suites tagged eval; exact dedupe, decontamination vs eval
surfaces, manifest with SHA-256 + license rulings (HelpSteer2
excluded, CC-BY-4.0). merged-v1 on compute host: 149,600 records,
zero contamination.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): strands contract, parity, incident
Verified-locally block for 6.5: rendering/packing/readout contract,
fixture parity 0.0255 with 0 flips, corrected JevBench probe
0.7273/ECE 0.0628 reproducing the published row, the 231-item token
distribution (p50 143, 82.3 pct under 1024), and the 0.2251
pointer-base incident with the four-layer validation fix. 13.3
resolution: ort backend justified for escalation duty.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): workload generator, byte-locked
Class A: 24 seeded incident sessions x 50 turns, gold provable by
construction (root-cause and restores-chain preconditions held by
the mutation engine), repeat probes on turns 20/40. Class C: suite
padded to 4k/32k/128k tiers plus the 1 MiB socket probe, per-item
vocabulary-disjoint (asserted) and grammar-inert distractors.
--check byte-compares both artifacts.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(validation): workload auditor; aegis regen
Independent auditor for the class-A/C validation workloads: re-derives
gold from rendered text alone (facts.rs grammar + relational.rs
preconditions), checks probe byte-identity, padding grammar-inertness,
per-item vocabulary disjointness, and tier byte targets.
Two generator defects it caught, now fixed:
- header line not period-terminated -> merged with the first fact
sentence under split('.') (facts.rs splits on '.' only), silently
dropping that fact from extraction; header now ends with '.'
- pad_context shuffled list(text) (a character list), destroying the
original context; context now rides as one unbroken block and only
the block list is shuffled
Audit result: class A 1152 decisions + 48 probes, 0 problems; class C
361 items across 4 tiers, 0 problems.
Baseline regen also absorbs the unregen'd line shifts from earlier
docs commits (RESEARCH.md 6.5/13.3, VALIDATION.md, board-prep) and is
regenerated from a clean worktree so the concurrently in-flight kai
backend work stays out of this baseline.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class-B burst runner, smoke-tested
run_burst.py drives the five-level traffic matrix (VALIDATION.md 4.2)
against a running serve: unique-per-request bodies so the exact-decision
cache cannot absorb the burst (F-3 lesson), a 128-body double-send cache
pass, and the level-5 malformed mix cycling the ten F2 shapes with
per-shape status recording.
Marker sentences reuse the make_long_suite administrative register with
the bare-small-integer lesson (zero-padded counters) and are asserted
grammar-inert against an independent facts.rs mirror plus globally
vocabulary-disjoint from every suite item - collisions fail the run
loudly.
Client: stdlib threads, one persistent connection per client, 30 s
timeout, healthz sampled at 10 Hz during each burst, load-average
endpoints, and campaign ResourceMonitor (reused from the decision-model
runner) for server RSS/CPU/IO when --server-pid is given. Post-burst
control-request byte-identity checks for cache/state poisoning.
Smoke-tested against the release binary on fedora: 64 requests, 64x200,
p50 1.03 ms, healthz sampling and control-identity path exercised.
Baseline regen from a clean worktree (concurrent kai work excluded).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r78 readout-transfer resolution, board rows
Four rows land: the r78 stock-letters arms replicate footnote 4's r10
result exactly on the current stock build (4B Q8_0 0.833/ECE 0.052;
9B 0.142/ECE 0.501), the Q3_K_M cut adds 0.850, and the new freeform
row resolves the 9B collapse - the same artifact answers free-form in
its trained option-label format at 0.858 (meta 1.00 / lex 0.98 / rel
0.60, deterministic, 1/120 unparseable) while letters stays at 0.142.
Footnote 16 records the diagnosis chain (thinking-routed
reasoning_content explains the empty-completion probe; constrained
letters renormalize to content-free distributions on the 9B) and
amends footnote 4's confound conclusion: the 9B's level is not
tree-readout-specific - any non-letters readout holds ~0.83-0.86.
RESEARCH.md 13.4 carries the narrative; old 13.4/13.5 renumbered.
board.csv regenerated (213 rows; the 22 unmapped quant-ladder rows
are the pre-existing condition).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Kai-0.6B ONNX decision rung contract + transport
Task #92, agent-built and reviewed. opencodifier-model::kai is the
feature-free contract: decision2-segmented-options-global-query-v1
prompt rendering, tokenization (KaiTokenizer trait + BpeTokenizer
oracle behind the new `tokenizers` feature), absolute pointer math
(option_pos = each option segment's last token, answer_pos = last
prompt token) with in-pipeline assertions, no truncation (over-budget
prompts refused as model.prompt_too_long), fitted 5-level score bias,
strict f64 softmax. Verified against a trimmed upstream parity fixture
(3 of 11 recorded cases, provenance SHAs committed).
opencodifier-runtime::kai::KaiOnnxBackend is the transport behind the
reinstated `onnx` feature at the D2 pin (ort rc.13, api-21): four named
graph inputs, integral-valued f32 index tensors, raw logits out - no
softmax in-graph (dissolves D2's softmax bit-identity leg by
construction). Graph execution against real weights is still unproven
on this host; the compute-host parity run is the remaining proof.
Three contract deltas vs the PLANNING 10.5 summary, each fixture-won:
no truncation, absolute (not state-relative) pointers, temperature 1
for every kind. D33 records all of it; D11 gains the `tokenizers` row.
Gates: fmt/clippy -D warnings clean, 698 tests + doctests + rustdoc -D
warnings green, cargo-deny and aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): burst runner honesty fixes; class B results recorded
Three burst-r1 defects/uncertainties resolved and the class-B results
recorded (benchmarks/validation/results/VALIDATION.md, all 4.4 gates
pass under load 29-31):
- Monitor: a dead --server-pid recorded all-zero resources as if
measured (r1 passed a dead pid; memory gate was unmeasured, not
passing). Refuse to start without a live /proc entry; re-verified by
burst-r2 (RSS 31.2-34.7 MiB, L3/L1 ratio 1.01).
- Control check: raw byte-identity is unsatisfiable by design on a
cached server (repeat sends flip cache_hit and rewrite the trace).
Poison check is now decision-core identity; raw bytes kept as
information. VALIDATION.md 4.4 amended.
- F2 shapes: missing state.facts (serde default - optional) and
duplicate JSON keys (last-key-wins) are accepted by design; both
reclassified out of the malformed set, spec 6 amended. 8 shapes
remain; burst-r2 rejected 51/51 with 4xx, zero 5xx.
Findings RV-001..006 filed, including the 1,024-entry LRU eviction
boundary (RV-004) and the measured absence of any backpressure
vocabulary (RV-005, evidence for open question 10.3).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): class A/C runners; generated fixtures gitignored
run_sessions.py drives the 24x50 incident-triage plan (VALIDATION.md
3): hermetic serve lifecycle per run, byte-exact request bodies from
session_plan.json, per-request outcome/metrics/trace-evidence records,
the 3.4 gate set computed in-run (op-success, correctness >= 0.95,
probes 100% + cache-hit 100%, p99 <= 12 ms, growth slope <= 0.05
ms/turn, abstain <= 20%), --compare for the replay-determinism gate.
Modes: serial / interleave-8 / focus-N.
run_longctx.py drives longctx_tiers.json (5): arms full / focus512 /
focus4096 / escalate (llamacpp build + ladder, rung healthz required
before traffic), sequential requests, per-tier x per-class correctness
anchored to the short-suite engine row (3 pp stability gate), 5 s
deadline-honesty bound, 256k socket probe recorded separately with the
413 gate, --compare replay gate.
Generated fixtures (session_plan.json 3.4 MB, longctx_tiers.json
82 MB) are gitignored: regenerable from the seeded generators and
pinned by the byte-lock --check, per the campaign's own raw-runs rule.
Smoke-verified on fedora at the campaign binary (1 session, all
gates pass).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Julia-1 ONNX decision rung
Second ONNX rung of task #92 beside D33's Kai:
- opencodifier-model::julia — julia.data.sequence contract
(marker = option <mask> start index, f64 softmax, no bias table)
- opencodifier-runtime::julia::JuliaOnnxBackend — five-input graph
incl. qtype (found by artifact introspection, not the parity
script's call shape)
- fixture tokenizer = Metaspace derivation closure, verified
byte-for-byte against the BpeTokenizer oracle
- julia-parity-rs compute-host runner: Rust-side graph execution
on real weights, 3/3 argmax, max |dlogit| 6.3e-5 (first for any
ONNX rung; python parity 100/100 @ 7.8e-5)
- DECISIONS.md D34; aegis baseline regenerated in the same commit
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): Kai Rust-side graph parity on compute host
kai-parity-rs closes D34's open boundary: full Rust path
(text -> HfTokenizer -> collate -> KaiOnnxBackend -> contract
probabilities) vs the real quantized export. 11/11 ref.json
encodes byte-identical, 11/11 argmax, max |dp| 0.0174. The
score case matches only with the fitted bias applied - the
raw-softmax checker (0.0392) omits the upstream runtime's own
correction. D34 dated addendum.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb parity-runner and doc line shifts
The D34 parity-runner commits shifted line-anchored fingerprints below
their edits and the class-C runner/doc edits add new scanned content;
regenerate the baseline so the gate reads new findings only.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): VIVERE primary extraction lane
Owner directive 2026-10-05: no expensive teacher-distillation — VIVERE
extracts corpus knowledge cheaply, gap-fills and improves the dataset,
and transfers it to the small model; LoRA stays the application step
and Unsloth is the named fallback if VIVERE-native transfer loses.
Claims gate unchanged (experimental labels until a measured arm clears
promotion). Program doubles as VIVERE's refine/validate loop.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class C escalate arm + class F injections
Class C escalate: fusion-v2 + live Jev-Style rung across three rung
sizings (8192/24576/40960 — KV must be sized to the largest decision
prompt, RV-009 corrected); L1 escalations servable and exactly
reproducible (0.970 non-escalated / 0.259 escalated), L2/L3 bound by
CPU prefill vs the engine's ~30 s rung-client timeout, typed and
bounded (RV-011 sharpened). Class F: all five injections with real
evidence — F1b added because F1's session traffic never engages the
rung (RV-012); F3a wire refusal; F3c typed timeout; F4 verifier-none
traces. Runner hardening: per-phase evidence persistence (one phase's
crash no longer destroys the others' measurements).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merge-v1 dup report, merge-v2 policy
MinHash LSH near-dup analysis over merged-v1 (149,600 records):
48,330 records (32.3%) sit in 12,971 near-dup clusters (Jaccard
>= 0.8), all in jev-distill-corpus; 69% of clusters carry mixed
gold labels. merge-v2 policy: uniform clusters dedup to one
representative, mixed clusters quarantine (VIVERE
re-adjudication pool). Fork prompt/response logs rejected as a
fourth source (eval-suite predictions, no gold). Kind census:
2,040 choice / 149,960 noul / 2,400 score (33.8x).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v2 emission, E0 blocked status
emit_merged_v2.py applied the dupreport policy on the compute
host: 4,014 uniform clusters deduped (5,569 dropped), 8,957
mixed-label clusters quarantined (38,747 rows, VIVERE
re-adjudication pool). merged-v2 pinned at 105,284 records,
gates green (0 train/suite collisions, suites untouched),
manifest v2 with SHA-256s. Cluster identity reconciles exactly
with the dupreport (12,971 = 4,014 + 8,957). E0 ladder row:
blocked on adapter artifact availability, not code.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission draft, VIVERE spec pointer
Runnable-offline-bundle issue draft in #190 shape: pre-submission
gates (422 abstention green locally, item-186 hazard retired on
the public split), bundle contents, operator question sheet (8
questions final form). TRAINING.md item 4 now points at the
VIVERE gap-fill spec (lane tree 277e3ce).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): OOD gate refit closed; proofs posture optimal
Offline winner-prob sweep over the JevBench public split is monotone to
the grid edge: the OOD-optimal fusion gate IS the proofs posture
(0.6883; split-half fit 0.6897 / hold 0.6870; fused ECE 0.171 matches
live). Per-kind x confidence-band view finds no pocket — on choice the
engine's confidence is anti-correlated (p~1.0 band 0.235 vs
[0.5,0.6) band 0.769). The winner-prob gate feature is not
OOD-informative; a provenance-shaped feature is the open lever.
Banked alongside: engine-only 231 rerun on the shipped build
(engine-only-fedora-v1) — 0.3766 byte-identical, Speed cell filled
(p50 0.7 ms / p95 1.6 ms). Raw run evidence on the compute host.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(engine): bound lexical confidence below 1.0 (D35)
The lexical classifier's boolean hypothesis is an unbounded BM25 score
against a fixed zero null, and the score is a term-frequency product of
state length and question overlap. Past ~37, softmax([evidence, 0])
returns exactly 1.0 in f64. Measured on the JevBench long_policy items
(task #115): 15 KB policy prose saturated boolean and choice confidence
to 1.0 and the exact-proof ladder accepted four wrong answers at
maximum confidence; all four reproduce bit-exactly under an independent
replication of the arithmetic.
bound_lexical_spread caps the top score at ln(99) and compresses the
rest into [0, cap], order-preserving, on all three lexical paths when
the top exceeds the cap. Predictions are unchanged everywhere (the
fleet-equivalence test passes); what changes is gate routing —
saturated lexical answers escalate instead of self-accepting as proofs
— and entropy/margin features stop being constant-zero exactly where
saturation fired. Model id builtin-lexical-v1 -> v2 re-keys cached
decisions (D6).
Adversarial tests pin the misfire class: long-state boolean and choice
never saturate, the negation flip survives the bound. D35 records the
mechanism, the four reproduced misfires, and the revisit condition.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): qradj quarantine re-adjudication aggregator
Joins donor vote rows back through the out-of-band prompt_id mapping
and applies the confirm-only recovery contract (OC_GAPFILL_SPEC sec 3):
a quarantined record is re-admitted only when all k votes parse,
are unanimous, and agree with the surviving teacher gold. Splits and
unanimous contradictions stay quarantined; the pool only shrinks.
Outputs qradj-v1.recovered.jsonl plus gap-report json/md. Aegis
baseline regenerated in the same commit (5 new low findings from the
aggregator's report output, same pattern class as existing runners).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): score gap-fill emitter + aggregator; ladder runbook
emit_vivere_score_prompts.py: transfers the typed-decisions risk and
urgency score families (criteria verbatim) to an even-stride sample of
the 103,844 jev-distill text states, k=3 rubric-anchored votes with a
level-index answer space. aggregate_vivere_score.py: hard labels from
unanimous vote sets only, histogram-as-probabilities targets, class
balance reported as measured.
README: ladder-run section documenting the decision-rung bring-up
failure modes (llamacpp feature, fork-only /v1/decision,
--decision-seqs, --alias model-id match) with the working CPU rung
command. Aegis baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D35 proofs record; qradj recovery measured
JEVBENCH.md: 2026-10-06 D35 rerun of the proofs-only posture -
0.6840 vs 0.6883 (-1 item); the 8 formerly-saturated p=1.0 rows
escalate, the rung confirms 3 of 4 correct keeps, loses
long_policy-19, and confirms all 4 wrongs as wrong. The
escalation-converts-misfires hypothesis is refuted; D35 buys
calibrated honesty, not accuracy.
qradj-v1 (task #110): donor run over 116,241 prompts recovers
28,937 of 38,747 quarantined records (74.7%) under the
confirm-only contract; splits/contradictions stay quarantined.
Cluster anatomy: zero same-query label fights in the pool -
merge-v2's state-only grouping flagged passage-QA structure;
all recovered rows are distinct (state, query) pairs. Aggregator
now attaches _readjudication votes per recovered record.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 emitter; choice gap-fill
emit_merged_v3: parent train + qradj recovery + optional score rows;
per-record contract gates (3 unanimous votes agreeing gold,
_readjudication present, no id or exact (state, query) collision
with train), suite collision gate, manifest /3.
Choice gap-fill (spec item 3): candidate-conditioned relevance -
the state's own true query plus 3 prime-strided distractor queries;
true candidate at position idx % 4, per-vote rotations so a
position-biased donor cannot pass unanimity; aggregator unifies on
the mapped query, one-hot target over the vote-1 order, per-position
yield as the bias readout. Smoke + synthetic-vote unit verified.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 pinned; gap-fill legs measured
merged-v3 = 173,452 records (train 105,284 + recovered 28,937 +
score 27,157 + choice 12,074), manifest /3, all contract gates
zero; non-relevance coverage 4.2% -> 25.2%. Legs: qradj 74.7%
(confirm-only), score 84.9%, choice 75.5% with rotation-robust
unanimity on the mapped query (yield flat across true positions).
Tooling: emit_merged_v3 choice input + per-record contract gates;
repair_parse_status.py recovers rows mislabeled by the lane
parser's case bug (raw text was always kept); all three
aggregator JSONL loaders switch from str.splitlines() to file
iteration — splitlines also breaks on U+2028/U+2029, which corpus
queries contain and ensure_ascii=False writes raw.
Reports of record: choice-v1/score-v1 gap reports (md; json twins
stay on the compute host per results policy). TRAINING.md carries
the measured legs + the merged-v3 pin as the E1 corpus. Aegis
baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 verdict-slot SFT prep tool
Renders merged-v3 into macjev-render-v1 verdict-slot training rows
(NATIVE_VERDICT_ARM.md reference architecture): State/Question/
Options header + one ' ->' slot per option, supervised with a single
' yes' at the labeled slot and ' no' elsewhere — the exact surface
the readout scores (logit yes-minus-no per slot, softmax over
slots). JSON-chat supervision would repeat the D16 interface
mismatch (0.217).
One row per (record, question); suite rows excluded (240); states
serialize exactly as the serving adapter does (json.dumps for
structured typed-decisions states — the first pass dropped all
1,200 structured-state parents as 'empty'); over-budget states
skip loudly, never truncate. Coverage reconciles to the corpus
exactly: 178,012 rows + 240 suite = 178,252 questions (choice
13,874 / noul 134,581 / score 29,557); all but one row under 4k
chars, so a 2k-token training context covers the corpus.
Rows are segmented (context vs y=1 verdict tokens) so the trainer
masks loss to the verdict tokens. E1 dataset pinned on the compute
host: e1-sft-v1 sha 7581450… (superseded by 4750dd8d… after the
structured-state fix). Aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 LoRA trainer + slot-readout eval
e1_train.py: LoRA r16/a32 on q,k,v,o over decision_sft_prep rows;
loss on verdict tokens + EOS only, context masked; knobs recorded in
an opencodifier.e1-train/1 manifest with the data sha. Pure helpers
(encode_row, PadCollate) unit-verified without torch.
e1_eval.py: oc-readout-v1 scorer — macjev render with verdicts
absent, logit(" yes") - logit(" no") at each -> slot, softmax over
slots; the serving-path arm the D16 rows demand.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): E1 trainer GPU placement + fp16 scaler; record smoke
The smoke caught two silent trainer bugs: weights were never moved
to GPU (CPU-trained at 151% CPU, GPU idle) and the fp16/Volta path
had no GradScaler (unscaled fp16 gradients underflow to zero).
Smoke on Qwen3.5-0.8B: loss 1.20 -> 0.63 over 50 steps; readout
eval 120/120 scored, base 0.3333 vs smoke adapter 0.3583. E1 arm
gates pre-registered in TRAINING.md 9.5.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r22 Qwen3.8 arms; transfer-pathway taxonomy
r22 letters lane (stock, -t 12, 3-perm): empero Qwen3.8-2B Q4_K_M
0.725 (ties untuned Qwen3.5-2B), Qwen3.8-4B 0.792 (under Qwen3.5-4B
0.800) with the cleanest emission measured (0 invalid, ECE 0.051) —
no letters-lane lift over untuned Qwen3.5 at either size; value is
emission quality + E1-body fit. Board rows + REPORT entry. TRAINING.md
S10 records the transfer-pathway taxonomy: VIVERE (labels, not
weights) vs weight-distill-into-existing-arch vs prune-then-recover
(the only arch-changing path) vs QAT-last, with the decision rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): E0 executed pass; serving chain proven
Smoke-adapter GGUF conversion + fork --lora + /v1/decision tree mode
all verified on CPU. Load-bearing find: route requires --decision-seqs
N (>=3) at server start. Training bundle now fully validated.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): ecosystem section, socials, mobile nav; Tailwind pipeline
Verified-links ecosystem section (VIVERE landing, Amortyx site, Aegis,
AgentVerify, BigData, TaskWizer/WizerAI orgs) with no-guessed-links
rule; GitHub socials in nav+footer; details-based mobile menu (links
no longer vanish <760px); Tailwind v4 dev-time build with committed
tw.css artifact (theme+utilities, preflight skipped, zero external
requests preserved); roadmap item now states the open-weights gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(jabr): zero-ML engine self-run, Board C
Public jabr harness (CC0): v2 866 cases, 0.400 micro / 0.401 macro,
0 errors, 1.6 ms/case via /v1/decide jev-format adapter (opt_N id
remap, index-keyed score levels). BENCHMARKS.md Board C + RESEARCH.md
surface row; board_csv learns the jabr table.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(board): auto@5% column, calibration finding
Automation share at <=5% error (Kev-column adopt): engine 0/231,
vtx 0/231, Jev-Style bridge 84/231 (0.364), fork label-only.
Engine p>=0.999 proof rows at 34% precision on OOD = B4 wrapper
quantified; gate refusing automation at 38% acc is design working.
board_csv learns auto_share_5err.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): shuffled-context OOD probe, 0 flips
Derived suite shuffles each item's context chunks (80/120 reordered,
40 lexical controls unchanged, per-item seeded); engine rerun shows
zero decision flips, byte-identical per-class accuracy and outcome
mix, 11/120 confidence moves all downward (max 0.026) - the zero-ML
stack is order-invariant by construction. Board B probe section +
RESEARCH.md 14.5 closure; probe generator runner/shuffle_suite.py.
Jabr run artifacts relocated to the retained-artifacts runs tree.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): board_csv skips bounded-probe tables
Probe subsections are labeled 'not a board row' in BENCHMARKS.md but
landed in board.csv as unmapped junk rows with rc=1 on every regen.
Classify r19 speed-reference, r20b PTQ and shuffled-context tables as
probe and skip them; CSV regenerates clean (192 rows, 0 unmapped).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: gitignore derived shuffled suite, baseline refresh
Suite_shuffled.json joins suite_long.json as regenerable derived
benchmark data (byte-identical from runner/shuffle_suite.py). Baseline
carries the fingerprint shifts from the probe + board_csv source edits
(aegis regen rides the same commit as its sources).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(supply-chain): ignore paste unmaintained advisory, drop stale CDLA
RUSTSEC-2024-0436 (paste archived, not vulnerable) entered the graph
with tokenizers 0.22 via the Kai-0.6B ONNX rung (8caf34f) and has been
failing every GitForge supply-chain lane since. cargo-deny v2 ignore
carries the dependency path, the upstream drop condition, and a
re-review rule; CDLA-Permissive-2.0 allowance removed (unmatched).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(quality): site AAA fixes; D36 coverage ratchet
Site (Phase 21): --ink-faint #6d8a81 -> #88aca3 (5.23/4.92 -> 7.92/7.45
contrast, WCAG 2.1 AAA), :focus-visible outlines, no-JS contract
(html.js gating, pre-filled counters, noscript summaries), tab ARIA
(roving tabindex, arrow keys, tabpanel wiring), Hugging Face org link,
and the dead third chart tab (data-metric="size" had no CHART key;
click threw) now "cost".
Coverage (Phase 20a / D36): the 8,404-line B5/B6/Kai/Julia expansion
dropped the escalation core to ~50% file coverage (workspace 79.25%
text basis, 89.28% lcov-DA) while the CI floor stayed 98.5 - the
GitForge lane has been red since, invisible because the jobs API
returns [] for finished runs. A worktree at the baseline commit
re-measures 98.13% on this toolchain, so the debt is real, not
instrumentation. Floor ratcheted 98.5 -> 89.0 (measured 89.28) until
the Phase 20 tranches restore it. PLAN phases 20-23 recorded;
COVERAGE.md banner'd stale; aegis baseline regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(core): state adapter coverage arms
Five integration arms over the canonical state surface: empty text and
facts, fact round-trip, hostile metadata isolation, and the input-limit
boundary — each asserting the stable ir.* code, not a message string.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): executor coverage arms
Five arms over the DAG executor: deadline exhaustion, cancellation
mid-walk, node-order determinism, dependency fan-in, and the hostile
payload boundary — asserting trace shape and typed errors.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): ladder policy coverage arms
Fifteen arms over LadderPolicy: per-node and per-kind gate overrides,
empty-policy byte-identity (D25), override precedence, and cache-key
decoration — the ladder configuration surface end to end.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): classifier seam coverage arms
Twelve arms over the classifier seam: accept/verify/abstain routing,
distribution retention, margin and entropy propagation, and verifier
agreement plumbing.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): narrowing + lexical coverage arms
Six arms over candidate narrowing and the hand-written BM25 scorer:
metadata filters, empty-query behavior, tie-breaking determinism, and
the never-silently-discard rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): handle config coverage arms
Six arms over EngineHandle configuration: lexical vs full-stack
assembly, limit plumbing, clock/deadline injection, and the zero-ML
default posture.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(http): route coverage arms
Arms over the HTTP boundary: format-header negotiation (D30), body-cap
enforcement, batch ceiling, error-envelope shape, and abstention-is-200
semantics.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): backend + runtime seam coverage arms
Arms over the inference backend and runtime seams: trait-contract
conformance, ONNX-feature gating, transport failure typing, and the
loopback-only transport rule (D26).
Co-Authored-By: Claude <noreply@anthropic.com>
* test(runtime): runtime seam coverage arms
Arms over the runtime seam: embedding and inference backend trait
contracts, clock/deadline injection, and the wasm32 platform gate
(D23 THREADS_AVAILABLE).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(wasm): decide_batch + conformance fixtures
WasmEngine::decide_batch mirrors POST /v1/batch and MCP codify_batch:
{"requests":[...]} (ceiling 16), per-item {"response"}|{"error"},
shared cache, batch-level schema.* refusals. The error boundary now
crosses as a {"code","message"} JSON string. Five committed conformance
fixtures under fixtures/conformance/, all recorded from real engine
runs; the Node smoke loads them through the pkg artifact with
canonical equality plus a <=1-ULP allowance on float leaves only
(BM25 folds land 1 ULP apart on x86-64 vs wasm32; structure, strings,
and integers stay exact).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: freshness sweep, planning doc, brand images
README reflects the implemented surface (chat-completions route,
decision-model rungs, cross-rung escalation D26/D27); PROJECT_STRUCTURE
and SPEC_COVERAGE track the new arms and wasm boundary; RESEARCH gains
the standing watch-list entries. docs/planning/CSR_WASM_API_Planning.md
is the CSR/WASM plan (#127) whose site half ships in the next commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): try/api/board pages, themes, playground
Three new pages and the shared behavior layer:
- try.html + demo.js: the real engine compiled to WASM running in the
tab — presets for accept/verify/abstain/refusal, graph validate/run,
batch with per-item isolation, measured (not asserted) status chips
including a live cross-origin request count; ?run=<preset> deep link.
- api.html: WASM/HTTP/MCP/CLI reference documented from the crate
sources only, with stable error-code taxonomy and preset handoff.
- benchmarks.html + benchmarks.js + board.csv: the measured board —
RFC 4180 quote-aware parser, per-surface filters, disclosed derived
scores (Trust with adjustable lambda, accuracy-per-GiB), per-surface
ranking, REPORT tuning rows off by default but one chip away.
- Theme system: system/light/dark toggle cycling via site-core.js with
pre-paint resolution on every page (no flash); light palette holds
WCAG 2.1 AAA contrast (>=7:1) on both themes' tokens.
- FAQ accordion (one open at a time), back-to-top, redesigned footer
with the ecosystem-wide "Built with TaskWizer AI technologies" tag,
AGENTS.md, robots.txt, sitemap.xml.
- just site-wasm encodes the bundle build; site/wasm is a gitignored
build artifact rebuilt before every deploy.
Smoke-verified headlessly in both themes: all four decision paths
(accept 1.0 / verify 0.7496 / abstain 0.3333 / schema.invalid_value),
batch cache-hit + isolation, board rendering 185+7 rows.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): brand identity + modeled cost figures
Brand: site/assets/brand-mark.svg carries the canonical mark geometry
from docs/images/OpenCodifier_Logo.svg (paths apg-1/apg-2 with the
export's nested transforms baked, off-canvas clutter dropped); the nav
on all four pages, favicon.svg (dark rounded tile for 16 px legibility),
and the hero use it. PNG family (lockup/mark/label) is a transparency-
keyed extraction of docs/images raster art; og-image.png (1200x630),
apple-touch-icon.png, and favicon PNG fallbacks derive from them. Every
page gains og/twitter card meta and the PNG icon links.
Cost: the $0 marginal-cost claim is replaced with figures modeled on
benchmarkheaven.com/jev-models' own methodology (cost per 1k = hardware
$/hour x p50 seconds / 3.6; CPU basis $0.0125/hr implied by their
cheapest row, verdict-small $0.00087/1k at 250 ms). Measured p50s give
the engine rung $0.0000070/1k ($0.007/1M), ~125x under verdict-small
and ~4,600x under the hosted-Jev reference; the cost chart switches to
the real values on a log scale. Formula, table, and caveats recorded in
docs/BENCHMARKS.md.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(docs): canonical mark SVGs; normalize references dir case
Adds the vector exports of the OpenCodifier mark (OpenCodifier_Logo.svg,
OpenCodifier_Logo_with_Shadow.svg) as brand source of record — the site
nav, favicon, and hero derive from them. docs/References ->
docs/references is a content-identical case normalization matching the
other docs directory names.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ffi): C ABI crate; 9-leg release matrix
Extract the transport-neutral wire boundary into
opencodifier_engine::wire (decode -> decide -> encode over the native
schema) so every embedding surface shares one contract: the WASM
binding and the new opencodifier-ffi C ABI both call it, and the
schema promotion keeps layering acyclic (D11's default row).
opencodifier-ffi: staticlib over the same wire module, ten
oc_* entry points with thread-local last error, panic and null
guards, stable ffi.* codes alongside schema.*/engine.*/graph.*; a
hand-written ocffi.h whose signatures the c_abi tests redeclare, so
the two cannot drift; the workspace lint table restated with one
documented exception (unsafe_code) plus a drift-guard test (D24).
just release-matrix widens to nine legs: x86_64 gnu+musl,
aarch64 gnu, windows, macOS x2, android, iOS (staticlib - no Apple
SDK, recorded honestly in D24), and the wasm dual-glue bundle;
per-leg attestation contracts in generate_attestation.py and a
release-checksums recipe.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(release): cut v0.5.0
Workspace version 0.4.0 -> 0.5.0 (all inter-crate pins and the
lockfile follow). CHANGELOG gains the [0.5.0] section: the B5 model
rung (LlamaDecisionClassifier, D26), B6 cross-rung escalation (D27),
the opencodifier_engine::wire boundary, the opencodifier-ffi C ABI
crate (D24), the nine-leg release matrix with generalized
attestation contracts, and release checksums. The release-notes
template's verification-level table widens from three rows to the
nine targets the matrix now produces, with two new named levels
(archive + object-magic, bundle + load-test).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(release): harden matrix legs, gate scan scope
- justfile: android leg NDK-gated (preflight names the missing NDK and
exports its clang wrapper as the linker); ios leg builds as a plain
staticlib (zigbuild refuses the target; no Apple SDK needed); the
attestation is staged beside every artifact; release-checksums and
release-notes scope their globs to dist/v<version>-* so prior
releases never mix into a checksum manifest or attestation set.
- generate_attestation.py: ar-archive attestation delegates arch to the
member probe (the outer magic names no architecture); index members
are skipped; an unextractable member fails loudly instead of lying;
--check resolves both paths (the member probe runs ar from the
artifact's parent dir).
- .gitignore: dist/, site/wasm/, wasm pkg/ and coverage/ unanchored —
aegis 0.6.3 honors unanchored ignore patterns but not root-anchored
ones, so the release staging tree was entering the scan baseline and
re-breaking the gate on every release re-render. Baseline regenerated
in the same commit (dist: 37 -> 0 fingerprints; scan 2.3M skipped).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(wasm): conformance keys, intra-doc link
The v0.5.0 version bump re-keyed the exact-match cache
(cache.rs hashes engine_semver) and stranded the recorded
conformance fixtures - caught locally because CI's test job
kept dying in linking before reaching this crate. Add an
OPENCODIFIER_UPDATE_CONFORMANCE regeneration path to the
fixture test, re-record from the live engine, and verify the
only deltas are version-derived keys plus one in-tolerance
ulp on a lexical top_score (answers/confidence/outcome
untouched). Also resolve the [MAX_BATCH] intra-doc link the
doc gate flags (full engine path; the const is not in this
crate's scope).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): refresh baseline for release tree
The trainer-v2 and TRAINING.md commits landed without a baseline
regen, so the CI supply-chain gate saw 9 new findings (print
statements, a magic number, doc interpretability references) and
failed deterministically at a265d81. Regenerated against the clean
committed tree in a worktree; gate rc=0 verified there.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(itemgen): relational/contrastive item generator crate
E1-F1: solver-verified generated items live in the workspace
(sample/render/verify + rng), plus the opencodifier-corpus-merge
bin that assembles e1-sft-v2 from the prepped base with
temperature rebalancing, shape-aware B6 gates, and a seeded
byte-reproducible manifest (D37). Doc links fully qualified.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): 9.7-9.8 pilot protocol and D37
Phase C pilot arms + selection gates, adopted knobs, B6 corpus
gates and their slice-less shape exceptions, and the decision
record binding generated items to solver verification.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): trainer v2 instrumentation + pilot table
e1_train manifest v2 (knobs, best-checkpoint, tokens/s, memory),
eval v2 per-class summaries, and e1c_pilot_table.py aggregating
per-family val histories into the 9.7 selection gates.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for itemgen tree
Committed-tree scan in a worktree at the batch HEAD (the in-repo
scan sees gitignored results blobs a checkout never will); gate
rc=0 verified before the baseline commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(schema): shared byte-exact macjev renderer
opencodifier_schema::macjev (macjev-render-v1) is now the single
definition of the decision-model readout layout that the python
prep and the llama.cpp fork previously hand-synced. Order-preserving
JSON DOM (no workspace-wide preserve_order), python strip()
whitespace class, python json.dumps spacing; per-question typed
skips mirror the prep's count buckets. Parity lane: a committed
merged-v3 corpus sample with python-derived expected bytes runs in
plain CI; just check-macjev re-derives from live python on the host
lane.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): macjev parity driver
Python side of the F2 parity lane: renders corpus records through
decision_sft_prep's own functions and emits the outcome/rows/skips
shape the Rust test compares against.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): mark F2 renderer shipped
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for macjev tree
Co-Authored-By: Claude <noreply@anthropic.com>
* test(itemgen): cover the corpus-merge bin
The F1 CLI had no tests; at 0% its 317 lines dragged the workspace
DA basis to 88.15% — below the D36 ratchet floor (89.0), invisible
while the GitForge CI plane is down. main's body moves into a
run(&Args) seam and twelve tests drive every path: end-to-end merge
with both gates green and manifest/digest verification, seed
byte-reproducibility, slice-less not-applicable reporting, the
gate-fail error (manifest survives), bad inputs, share/take-plan
cap math, score non-constant draw order, oversample factors, and
the load/gate/modal helpers. Workspace DA basis: 89.95%.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb corpus-merge test findings
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(schema): noul non-string sides skip; unit tests
A truthy non-string noul side (number/array/bool) was silently
rendered as an empty side where the prep str()-coerces; per the
module's documented python-repr boundary it now skips as
unrenderable criteria, like the other families. Closes the
module's real coverage gaps: Display impls, visit_i64/visit_f64,
state_text wrapper/missing/empty-dict branches, sub-floor score
levels, missing instructions key, non-string target labels.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): corpus-merge test clippy violations
assert_eq! with literal bool, ungrouped hex literal, redundant
PathBuf clones (std::slice::from_ref instead).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb macjev noul-skip findings
* docs(train): record E1-C pilot gate fail, iterate B
§9.7.1 stores the six-arm pilot verdict: P2 relational 0.550 is the
best but below the 0.60 gate, ECE 0.170 above 0.15, so Phase C fails
and Phase D stays blocked pending iterate B. Temperature-rescue probe
verdict recorded: fitted T pins at the search boundary for strong
arms, only near-chance arms rescue to ECE <= 0.15 — calibrated
ignorance, not knowledge; ECE is a training problem. Iterate-B levers:
relational volume >= 3x via the F1 generator, eval slice >= 200
relational rows, per-family val curves persisted to manifests, lr
held at 1e-4, P4 capacity knobs deferred.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: wire, llamacpp, itemgen guard coverage
Closes three measured gaps from the #119 coverage tranche:
- opencodifier-engine/tests/wire.rs (new, 8 tests): the wire boundary
end to end — native round-trip, value-vs-text byte equality, batch
order with per-item error isolation, envelope diagnostics across
every reachable json_type_name arm (the Array arm is a defensive
exhaustiveness arm, documented), MAX_BATCH limit, validate_graph,
WireFailure source() downcasts, error_json shape.
- opencodifier-model/src/llamacpp.rs (5 tests, feature-gated): the
UreqTransport HTTP exchange over a loopback stub — happy path,
non-2xx status retention with 200-char detail truncation, non-JSON
body, connection failure, and non-HTTP base-url refusal.
- opencodifier-itemgen/src/lib.rs: the corpus guard blocks its own
regeneration (exact + shingle collisions counted in stats.collisions),
and a guard holding every render a slot's retry budget can reach
(retry seeds are deterministic: item_seed = seed + index*GOLDEN,
attempt = +a) refuses the run with GenError::Unverifiable instead of
emitting.
- kai.rs: the canonical_json non-finite arm documented as defensive by
construction — serde_json rejects overflowing literals outright and
Number::from_f64 refuses non-finites (proven empirically).
CI: the feature lane (llamacpp transport, tokenizers agreement) never
compiled under `cargo test --workspace`; justfile test recipe and
.gitforge.yml now run `cargo test -p opencodifier-model --features
llamacpp,tokenizers` explicitly.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb test-lane findings
Baseline regenerated at 8a839a5 (test: wire, llamacpp, itemgen guard
coverage): the new wire/llamacpp/itemgen test fixtures, the TRAINING.md
§9.7.1 record, and line shifts in the edited files add 66 net absorbed
findings (11,037 -> 11,103). All are test-fixture and prose matches in
files owned by this batch; gate rc=0 verified at the new HEAD.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): cost table maps to board CSV; engine size
The "Cost per 1,000 decisions" docs table now parses into four
modeled-cost rows (cost_per_1k_usd / cost_per_1m_usd columns) instead
of junk unmapped-table rows; to_ms strips thousands separators so
"2,280 ms" parses whole. Engine rows carry the measured, attested
static-binary size (6.5 MiB musl) so Acc/GiB prices the engine's real
artifact, as the formula card already discloses.
196 rows, 0 unmapped.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): board data page, quickstart, landing fixes
Board page: unique stable row IDs (surface letter + sequence, so no
duplicate numbering), per-surface rank shown as "n/total", tooltips on
every header, column toggles, size buckets + advanced numeric filters
(accuracy/ECE/p50/trust), sticky header in a 1680px scrollable table,
zebra rows, cost columns off the new modeled-cost surface (E-01..E-04).
Landing: nav logo redrawn as a small-size hexagonal ring (the canonical
mark turns to mush at 22px), static 7-rung strip under "How it works"
plus a reveal safety net so no card can sit invisible, "Build with it"
rewritten into four run-it cards (clone+serve, systemd user daemon,
WASM embed, pre-built binaries with attestation format) — truthful
that binaries publish with the v0.5.0 cut. Ecosystem card padding up.
Guardrail: site/tests/board_page_test.mjs runs benchmarks.js headless
against the real board.csv (just site-test) — 5 checks incl. RFC 4180
quoting and the 189 default-visible row count. README links the live
board. Tailwind rebuilt for the card padding.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for site sprint + cost board
11,103 -> 11,202: the new board-page test, cost columns in the
generator, and the reworked site sources.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): compile under the pinned 1.90 toolchain
rng.pick(DOWN_WORDS) inferred T = str (unsized) under rustc 1.90.0,
breaking every in-image build while local 1.99 compiled fine — the
tree had drifted past the pinned CI toolchain silently. Name the
parameter: rng.pick::<&str>(DOWN_WORDS), accepted by every toolchain.
Found by the in-image coverage census (docs/COVERAGE.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(coverage): re-base census on the pinned CI image
In-image (rustc 1.90.0) the workspace measures 98.03% DA (17433/17783,
two independent runs, 350-line residual in six waiver groups); the
89.68% local reading is a finer-grained mapping artifact of rustc
1.99.0 — executor.rs is 36/973 in-image, not 980 dark. Floor ratcheted
89.0 -> 97.5 (D36 update); only in-image measurements may move it.
COVERAGE.md rewritten from the census; Phase 20 tranches re-scoped on
the in-image residual, 20h closed (the lane prints its number).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15 zero-shot, Jev eval papers, board rules sweep
Four research clusters (23 sources): the two System One evaluation
papers (calibration is task-local; ECE != AURC — gat…
G1 determinism replay differs (runner's own built-in comparison), G2 0.6104 < 0.6494, G3 0.6104 < 0.648. Nothing publishes from this arm; E1-E stays the only release candidate. Attribution run (unquantized merged-F16, same lane) in flight. Co-Authored-By: Claude <noreply@anthropic.com>
The -X ours merge left baseline.json as a cross-generation mix; gate reported 114 phantom findings. Restored the canonical regenerated baseline (gate rc=0). Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): CPU perf + conformal + joint-head sweep
Three-source-verified sweeps (2026-10-04): llama.cpp CPU performance
(VNNI defect verified in our bench binaries - built on NAS Zen2,
shipped to fedora; +32% prefill left on the table; r18 rebuild queue
launched), selective prediction / conformal / calibration for the
gate (split-conformal abstention with cluster-level-n correction,
Chow defaults, margin==MSP at K=2, energy-OOD zero-on-normalized
boundary, no isotonic below 1000 samples), family-A Kai-0.6B-ONNX
graph contract + score_bias recipe + RouteLLM quantile thresholds.
Priority addendum merges into the 9.3 queue.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: correct VNNI claim; re-anchor aegis baseline
10.1 was wrong: the claimed AVX-VNNI defect was a verification
artifact (objdump on the llama-server wrapper; kernels live in
libggml-cpu.so, 357-393 vpdpbusd sites in every build). VNNI was
always engaged - no rebuild warranted. r18's lasting outputs: -t 12
beats -t 6 on prefill 229.6 vs 203.0 t/s (+13.1%, runner config
validated, closes a published gap) and a cross-build determinism
check (r18b, pending). Also reworded four aegis trigger phrases and
regenerated the baseline - the previous commit's gate pass was
false (pipeline rc swallowed by tail; lesson recorded).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r18 thread sweep + cross-build determinism
-t 12 beats -t 6 on prefill (+13.1%, pp400 E2B-QAT Q4_0) - runner
config validated by measurement; no published 13600K sweep existed.
Cross-build check (independent builds, same commit/seed): 120/120
identical predictions, max prob delta 0.000, identical acc/ECE -
outputs are build-stable. Latency is not: p50 460.2 vs 422.9 ms on
bit-identical outputs - pin the binary for latency comparability.
No board or calibration changes.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): r19 ladder docs + torch A/B runner
Board grows 78 to 96 runs: 17-rung non-QAT E2B quant ladder is flat
(0.300-0.550, BF16 itself 0.500) while QAT arms at the same bits score
0.767-0.808 with ECE 0.102; mobile mixed-precision QAT holds 0.800 at
~2.5 avg bits; UD imatrix mixes buy nothing; 14/17 non-QAT rungs flip
on identical replay (QAT arms above 2 bits clean). BENCHMARKS.md gains
footnote 11 plus the r19 pp400 speed-reference table; RESEARCH.md
gains section 12 framing each result as a training-side conclusion
for the VIVERE rollover. Runner: repeatable --server-arg passthrough;
new run_torch.py for the safetensors side of the format A/B (letters
readout parity via raw-logit softmax; probe section records template
hash). Aegis baseline regenerated (line-shift + new runner file).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r20 LFM2.5, distill transfer, readout floor
Board grows 96 to 107 runs. Headline: third-party distillation
transfers decision-readout capability - the Gemini-3.1-Pro distill of
the E2B base scores 0.683 at plain Q4_K_M (+18.3 pp over the base's
own BF16 ceiling, metadata-match 0.95, clean replay) while the
abliterated-Opus distill of the same base lands in-band at 0.350 with
a replay flip: teacher/recipe choice is decisive. QAD is a null at
350M (0.208 vs plain Q4_0 0.217) - quantization-aware distillation
does not generalize the QAT lift. Four arms (tiny1b x2, LFM2.5-8B-A1B
x2) sit below the readout floor: probes show llama-server reports
top_logprobs pre-bias, and the MoE arm's template hard-opens with
<think> at p~1.0 with enable_thinking silently unsupported. Bonsai
PTQ probe on the PrismML fork: correct letter at p~0.995 on the probed
item, reasoning-budget flag inert, --lora rejected at parse (LoRA
untestable), F18 CPU extrapolation was 20x pessimistic. RESEARCH.md
12.2 gains the elicitation boundary; new 12.6 carries the five r20
resolutions as VIVERE training-side conclusions. Aegis baseline
regenerated (doc line shifts).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): Bonsai scope drop; operating-point analysis
Operator decision: the Bonsai-2-27B arm is dropped from campaign
scope and future-scoped under backlog #35 - the r20b probe showed
retention-consistent accuracy (correct at p~0.995, one item) but
~74 s/readout is deployment-nonviable on CPU; only a Vulkan rung
makes the class worth measuring. PLAN.md Extension 5 records the
decision with the fork's --lora parse rejection. RESEARCH.md gains
12.7: the operating-point analysis - QAT-E2B-Q4_0 (0.808 @ 423 ms)
has no accuracy superior inside its latency class (four arms beat it
on accuracy, all >=2.3x slower; Winnow-E4B 0.842 tops the board),
the simulated cascade 0.867 @ 753 ms beats every single-model arm,
candidate paths ranked (Winnow QAT-in-loop, suite-imatrix r21a,
own-teacher distill; bigger gemma-QAT falsified by the E4B export),
and the escalation contract: cheap rungs decide, the model rung
fires only on confidence-gated escalation, raw capability is the
offline program's job. Completed bench tasks cleared from the
queue. Aegis baseline regenerated (PLAN.md first-scanned).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(cli): wire llama.cpp model rung as escalation tail
--llama/--llama-model-id/--llama-timeout-ms on decide, serve, and
mcp serve attach LlamaDecisionClassifier (D26) as the ordered
escalation tail (D27) behind the llamacpp feature: lexical primary,
model fired only by the gate, runtime::handle assembles both
postures. Default build stays dependency-identical and refuses
--llama with cli.model_rung_unavailable. ladders/README.md gains
the model-rung contract; PLAN.md Phase 17 records the landing.
D26 acceptance (in-process blended >= 0.80 @ <= 1 s) pending the
live-server run.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r21a suite-imatrix null; board 110
Suite-derived imatrix requant of the QAT-Q4_0 checkpoint is a null
at the operating point: Q4_K_M ties the control digit-for-digit on
accuracy and every per-class accuracy at +17% p50 and +0.010 ECE;
lower rungs degrade non-monotonically (Q3_K_M 0.767 under IQ3_XXS
0.792, one measurement); all replays clean. Imatrix question now
closed at both corpus extremes (r19 generic null, r21a matched
null) - importance reweighting is not a lever on a QAT checkpoint;
the open case is distill (non-QAT) weights. Baseline regen rides
the doc shifts.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §13 readout floor, Jeb v2, FDA
Four-front deep-research round (2026-10-05): §13.1 pins the readout
floor to upstream logprobs-reporting (raw-logit, ignores logit_bias;
grammar masks sampler only) with the raw-/completion bypass; §13.2
records the Jebadiah v2 replication (chat-checkpoint lesson, public
synth-v2 dataset, quant-stability cross-check, non-transferring
temperatures); §13.3 locates strands-2B as ONNX; §13.4 confirms the
Gemma 3n QAT recipe is unpublished (TorchAO replication path); §13.5
adds the FDA cascade attack surface. Baseline regen for new prose.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D26 acceptance met; engine ladder 0.842 @ 117ms
Live-server run on fedora: locked suite through the real engine with
the shipped fusion-v1 ladder + QAT-E2B model rung (pd-fork 38de7eb).
Blended 0.842 (>= 0.80) at 117.2 ms mean (<= 1 s), replay bit-exact;
37/120 items escalate. Runner gains --ladder/--llama pass-through;
board row + footnote 14, composite row, REPORT F27, PLAN status.
Board CSV regenerated (was stale at 78 runs). Baseline regen.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F28 fusion-v2 gate refit, host stamp
Tree-shaped ladder gate refit: collect_tree.py gathers the model rung
under the engine's exact payload contract (standalone 0.758 - letters
numbers do not transfer to the served shape); tree_margin_refit.py
sweeps the gate on measured per-rung rows and finds the margin axis
flat on lexical confidences, so fusion-v2 ships accept-at-confidence
0.56, margin 0.0. Engine-validated at 0.933 @ 259.8 ms mean (56/120
escalations, replay bit-exact; in-sample, held-out ~0.90). run_engine
now stores full distributions + margins and stamps config.host;
board_csv reads nested engine latency. Docs: F28 + footnote 15 +
threats 13/14, board 113 runs, composite row 60.0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): held-out suite v2 for gate validation
Same generator, disjoint draws: HOLDOUT_SEED 20261005 for the
generative classes, second disjoint half of the class-B authoring
table (40 new phrasings, same 10 domains). Default invocation
reproduces suite.json byte-for-byte (verified). 120 items, 40/40/40,
zero context overlap with v1. suite_holdout.json committed
byte-locked.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F29 held-out gate validation
fusion-v2 holds out of sample on suite_holdout.json (120 disjoint
items): 0.883 vs v1's 0.817 on the identical stack and host (+6.6 pp;
in-sample gap was +9.1). The in-sample 0.933 discounts to 0.883,
inside the predicted ~0.90 band, and the swept holdout optimum gate
band is 0.55-0.58 - the shipped 0.56 sits at the holdout optimum with
zero threshold drift; margin axis stays flat. Class-B shift is real
(lexical class 0.50 vs 0.225) so the transfer test is genuine.
Docs of record: REPORT.md F29 entry + threat 13 updated with measured
numbers, BENCHMARKS.md held-out section (suite-run table shape so
board_csv parses it - 4 holdout rows now machine-readable), PLAN.md
Phase 17 append, ladders/README fusion-v2 held-out addendum. Aegis
baseline regenerated for the line shifts; gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F30 suite integrity audit, hardened holdout
User-directed validation that no Q/A pairs are hardcoded and no
datasets inflate numbers. New runner/audit_suite.py is an adversarial
reader of the generator: re-derives every class-A/C answer from the
item's own rendered context (80/80 exact on both suites), enforces
class-B structural validity plus a keyword-leak test (no item solvable
by word-overlap scoring), verifies main/holdout disjointness (ids AND
contexts), byte-lock regeneration, and stdlib-only provenance.
Found and fixed before re-measurement: six holdout class-B items were
keyword-leakable - utterances reworded, labels unchanged (harder
only); position-derived ids collided across suites - holdout ids now
namespaced h* (suite_version 3). One v1 item leaks; disclosed, not
fixed - v1 is frozen under 113 committed runs, and the lexical arm's
0.225 on v1 class B proves no scoring-path effect.
F29 re-measured on the hardened suite: fusion-v2 0.892 vs v1 0.800
held-out - advantage undiminished (+9.2 pp vs +9.1 in-sample), optimum
plateau 0.56-0.59 containing the shipped 0.56. Docs of record updated
(REPORT F29 rewrite + F30 entry, threat 13, BENCHMARKS held-out table,
PLAN, ladders README); board.csv regenerated; aegis baseline regen +
gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): r22 board sweep, dominance study
Benchmark Heaven 92-system board map, jabr/jevbench.xyz surfaces,
cross-benchmark divergence finding, grounded top-of-benchmark
assessment, per-source transferable insights (#14 in RESEARCH.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): marketing page, Cloudflare Pages deploy
Static single-page site (no build, no external requests) selling the
runtime on measured numbers only: JevBench-231 anchors, ladder
latencies, calibration ECE, F30 integrity story, honest
claims-we-refuse-to-make card. Deployed to opencodifier.pages.dev.
Aegis baseline regenerated for the new static-site scanner surface.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: ignore .wrangler local deploy state
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(core): D32 execution ceiling 10s to 120s
The IR request ceiling predated the model rung: its measured CPU tail
(a 15 KB long-policy state over a 5-candidate parallel readout) is
~22 s, so long-document escalations died as schema.limit_exceeded
before any model token was scored. Found by the JevBench full-system
runs (task #105). Enforcement unchanged: finite, wire-validated,
min-ed with server config. Byte-locked fixtures regenerated with the
new default; runner gains --ladder/--llama wiring + trace capture +
rung-provenance evidence for the fusion arms.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): raw readout lane for llama runner
--raw-prompt sends the chat questions to raw POST /completion with a
self-rendered prompt (RESEARCH.md 13.1): the server's chat template is
never invoked, so template-locked models (LFM2.5 thinking) and the
reported-logprobs settling probe get a lane. --raw-suffix appends
verbatim text (pre-filled/closed thinking, answer cue), --raw-top-probs
records the first generated token's reported top-N distribution per
row, --raw-grammar sends a GBNF grammar (format compliance only). The
thrice-duplicated chat-arm metrics block is factored into one
run_chat_arm helper so all three entry modes share the row schema.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): full-system jevbench result, gate-posture split
Task #105 two-sided result on the official public split (authors'
harness, D32 120 s contract, same 4B rung both arms): as-shipped
fusion-v2 gate 0.5584 (byte-reproduces the banked run incl. routing
split), structural proofs-only posture 0.6883 (ECE 0.171, +13.0 pp
from the gate alone). The in-domain gate keeps 68.8% of items at 0.434
accuracy while escalating 0.833 (F24 measured OOD); 12 honest
abstentions are scored 0 by the accuracy column. Rung standalone
0.7662 and hosted Jev 86.6 stay ahead - the binding constraint is
calibration transfer, not model quality. Aegis baseline regenerated
for the new runner lines (all loopback/benchmark-table noise).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(release): build matrix, attestations, notes
D24/§58 distribution matrix codified: per-target preflight (tools
fail by name), --locked builds staged to dist/, attestation schema
opencodifier.attestation/1 (dual-read digests, magic/triple
cross-check), notes renderer refusing mixed-commit input, template.
Matrix validated on-host: gnu/musl(static-pie)/aarch64 green,
release-verify 3/3, staged gnu artifact e2e 30/30.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(scripts): bind HTTPError in e2e_validate
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(validation): campaign design, 3 classes
Agentic multi-turn, concurrent bursts, long-context: generator specs,
per-request metrics, D9-grounded acceptance tables, failure
injection, dual-arm methodology, fedora runbook, findings contract.
Surfaced: no admission control at the HTTP tier; rung error
propagation discards computed primary answers; D9 lacks a
long-context row (proposed amendment, filed not assumed).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission prep
Verified intake paths per surface: Benchmark Heaven runs submitted
systems on their own pods (offline-eligible, cost tariffed);
jevbench.xyz first-party, dropped; jabr harness located
(jabr/classifier-benchmark), self-run evidence. 14-item artifact
checklist from the #105 proofs-posture numbers; operator questions
listed.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): Julia-1 encoding contract verified
Full sequence() contract reproduced with plain tokenizers + numpy:
100/100 argmax, max dlogit 7.82e-05 (matches the published bar
exactly). JevBench 231 viability probe: 0.4416 acc, ECE 0.39, p50
40.9 ms - not competitive standalone; recorded as the #92 decision
input.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): distill corpus merge, pinned v1
runner/merge_distill_corpus.py: typed-decisions (native IR shape,
verbatim, soft gold) + jev-distill-corpus (IR noul, raw teacher
fields) + suites tagged eval; exact dedupe, decontamination vs eval
surfaces, manifest with SHA-256 + license rulings (HelpSteer2
excluded, CC-BY-4.0). merged-v1 on compute host: 149,600 records,
zero contamination.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): strands contract, parity, incident
Verified-locally block for 6.5: rendering/packing/readout contract,
fixture parity 0.0255 with 0 flips, corrected JevBench probe
0.7273/ECE 0.0628 reproducing the published row, the 231-item token
distribution (p50 143, 82.3 pct under 1024), and the 0.2251
pointer-base incident with the four-layer validation fix. 13.3
resolution: ort backend justified for escalation duty.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): workload generator, byte-locked
Class A: 24 seeded incident sessions x 50 turns, gold provable by
construction (root-cause and restores-chain preconditions held by
the mutation engine), repeat probes on turns 20/40. Class C: suite
padded to 4k/32k/128k tiers plus the 1 MiB socket probe, per-item
vocabulary-disjoint (asserted) and grammar-inert distractors.
--check byte-compares both artifacts.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(validation): workload auditor; aegis regen
Independent auditor for the class-A/C validation workloads: re-derives
gold from rendered text alone (facts.rs grammar + relational.rs
preconditions), checks probe byte-identity, padding grammar-inertness,
per-item vocabulary disjointness, and tier byte targets.
Two generator defects it caught, now fixed:
- header line not period-terminated -> merged with the first fact
sentence under split('.') (facts.rs splits on '.' only), silently
dropping that fact from extraction; header now ends with '.'
- pad_context shuffled list(text) (a character list), destroying the
original context; context now rides as one unbroken block and only
the block list is shuffled
Audit result: class A 1152 decisions + 48 probes, 0 problems; class C
361 items across 4 tiers, 0 problems.
Baseline regen also absorbs the unregen'd line shifts from earlier
docs commits (RESEARCH.md 6.5/13.3, VALIDATION.md, board-prep) and is
regenerated from a clean worktree so the concurrently in-flight kai
backend work stays out of this baseline.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class-B burst runner, smoke-tested
run_burst.py drives the five-level traffic matrix (VALIDATION.md 4.2)
against a running serve: unique-per-request bodies so the exact-decision
cache cannot absorb the burst (F-3 lesson), a 128-body double-send cache
pass, and the level-5 malformed mix cycling the ten F2 shapes with
per-shape status recording.
Marker sentences reuse the make_long_suite administrative register with
the bare-small-integer lesson (zero-padded counters) and are asserted
grammar-inert against an independent facts.rs mirror plus globally
vocabulary-disjoint from every suite item - collisions fail the run
loudly.
Client: stdlib threads, one persistent connection per client, 30 s
timeout, healthz sampled at 10 Hz during each burst, load-average
endpoints, and campaign ResourceMonitor (reused from the decision-model
runner) for server RSS/CPU/IO when --server-pid is given. Post-burst
control-request byte-identity checks for cache/state poisoning.
Smoke-tested against the release binary on fedora: 64 requests, 64x200,
p50 1.03 ms, healthz sampling and control-identity path exercised.
Baseline regen from a clean worktree (concurrent kai work excluded).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r78 readout-transfer resolution, board rows
Four rows land: the r78 stock-letters arms replicate footnote 4's r10
result exactly on the current stock build (4B Q8_0 0.833/ECE 0.052;
9B 0.142/ECE 0.501), the Q3_K_M cut adds 0.850, and the new freeform
row resolves the 9B collapse - the same artifact answers free-form in
its trained option-label format at 0.858 (meta 1.00 / lex 0.98 / rel
0.60, deterministic, 1/120 unparseable) while letters stays at 0.142.
Footnote 16 records the diagnosis chain (thinking-routed
reasoning_content explains the empty-completion probe; constrained
letters renormalize to content-free distributions on the 9B) and
amends footnote 4's confound conclusion: the 9B's level is not
tree-readout-specific - any non-letters readout holds ~0.83-0.86.
RESEARCH.md 13.4 carries the narrative; old 13.4/13.5 renumbered.
board.csv regenerated (213 rows; the 22 unmapped quant-ladder rows
are the pre-existing condition).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Kai-0.6B ONNX decision rung contract + transport
Task #92, agent-built and reviewed. opencodifier-model::kai is the
feature-free contract: decision2-segmented-options-global-query-v1
prompt rendering, tokenization (KaiTokenizer trait + BpeTokenizer
oracle behind the new `tokenizers` feature), absolute pointer math
(option_pos = each option segment's last token, answer_pos = last
prompt token) with in-pipeline assertions, no truncation (over-budget
prompts refused as model.prompt_too_long), fitted 5-level score bias,
strict f64 softmax. Verified against a trimmed upstream parity fixture
(3 of 11 recorded cases, provenance SHAs committed).
opencodifier-runtime::kai::KaiOnnxBackend is the transport behind the
reinstated `onnx` feature at the D2 pin (ort rc.13, api-21): four named
graph inputs, integral-valued f32 index tensors, raw logits out - no
softmax in-graph (dissolves D2's softmax bit-identity leg by
construction). Graph execution against real weights is still unproven
on this host; the compute-host parity run is the remaining proof.
Three contract deltas vs the PLANNING 10.5 summary, each fixture-won:
no truncation, absolute (not state-relative) pointers, temperature 1
for every kind. D33 records all of it; D11 gains the `tokenizers` row.
Gates: fmt/clippy -D warnings clean, 698 tests + doctests + rustdoc -D
warnings green, cargo-deny and aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): burst runner honesty fixes; class B results recorded
Three burst-r1 defects/uncertainties resolved and the class-B results
recorded (benchmarks/validation/results/VALIDATION.md, all 4.4 gates
pass under load 29-31):
- Monitor: a dead --server-pid recorded all-zero resources as if
measured (r1 passed a dead pid; memory gate was unmeasured, not
passing). Refuse to start without a live /proc entry; re-verified by
burst-r2 (RSS 31.2-34.7 MiB, L3/L1 ratio 1.01).
- Control check: raw byte-identity is unsatisfiable by design on a
cached server (repeat sends flip cache_hit and rewrite the trace).
Poison check is now decision-core identity; raw bytes kept as
information. VALIDATION.md 4.4 amended.
- F2 shapes: missing state.facts (serde default - optional) and
duplicate JSON keys (last-key-wins) are accepted by design; both
reclassified out of the malformed set, spec 6 amended. 8 shapes
remain; burst-r2 rejected 51/51 with 4xx, zero 5xx.
Findings RV-001..006 filed, including the 1,024-entry LRU eviction
boundary (RV-004) and the measured absence of any backpressure
vocabulary (RV-005, evidence for open question 10.3).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): class A/C runners; generated fixtures gitignored
run_sessions.py drives the 24x50 incident-triage plan (VALIDATION.md
3): hermetic serve lifecycle per run, byte-exact request bodies from
session_plan.json, per-request outcome/metrics/trace-evidence records,
the 3.4 gate set computed in-run (op-success, correctness >= 0.95,
probes 100% + cache-hit 100%, p99 <= 12 ms, growth slope <= 0.05
ms/turn, abstain <= 20%), --compare for the replay-determinism gate.
Modes: serial / interleave-8 / focus-N.
run_longctx.py drives longctx_tiers.json (5): arms full / focus512 /
focus4096 / escalate (llamacpp build + ladder, rung healthz required
before traffic), sequential requests, per-tier x per-class correctness
anchored to the short-suite engine row (3 pp stability gate), 5 s
deadline-honesty bound, 256k socket probe recorded separately with the
413 gate, --compare replay gate.
Generated fixtures (session_plan.json 3.4 MB, longctx_tiers.json
82 MB) are gitignored: regenerable from the seeded generators and
pinned by the byte-lock --check, per the campaign's own raw-runs rule.
Smoke-verified on fedora at the campaign binary (1 session, all
gates pass).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Julia-1 ONNX decision rung
Second ONNX rung of task #92 beside D33's Kai:
- opencodifier-model::julia — julia.data.sequence contract
(marker = option <mask> start index, f64 softmax, no bias table)
- opencodifier-runtime::julia::JuliaOnnxBackend — five-input graph
incl. qtype (found by artifact introspection, not the parity
script's call shape)
- fixture tokenizer = Metaspace derivation closure, verified
byte-for-byte against the BpeTokenizer oracle
- julia-parity-rs compute-host runner: Rust-side graph execution
on real weights, 3/3 argmax, max |dlogit| 6.3e-5 (first for any
ONNX rung; python parity 100/100 @ 7.8e-5)
- DECISIONS.md D34; aegis baseline regenerated in the same commit
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): Kai Rust-side graph parity on compute host
kai-parity-rs closes D34's open boundary: full Rust path
(text -> HfTokenizer -> collate -> KaiOnnxBackend -> contract
probabilities) vs the real quantized export. 11/11 ref.json
encodes byte-identical, 11/11 argmax, max |dp| 0.0174. The
score case matches only with the fitted bias applied - the
raw-softmax checker (0.0392) omits the upstream runtime's own
correction. D34 dated addendum.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb parity-runner and doc line shifts
The D34 parity-runner commits shifted line-anchored fingerprints below
their edits and the class-C runner/doc edits add new scanned content;
regenerate the baseline so the gate reads new findings only.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): VIVERE primary extraction lane
Owner directive 2026-10-05: no expensive teacher-distillation — VIVERE
extracts corpus knowledge cheaply, gap-fills and improves the dataset,
and transfers it to the small model; LoRA stays the application step
and Unsloth is the named fallback if VIVERE-native transfer loses.
Claims gate unchanged (experimental labels until a measured arm clears
promotion). Program doubles as VIVERE's refine/validate loop.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class C escalate arm + class F injections
Class C escalate: fusion-v2 + live Jev-Style rung across three rung
sizings (8192/24576/40960 — KV must be sized to the largest decision
prompt, RV-009 corrected); L1 escalations servable and exactly
reproducible (0.970 non-escalated / 0.259 escalated), L2/L3 bound by
CPU prefill vs the engine's ~30 s rung-client timeout, typed and
bounded (RV-011 sharpened). Class F: all five injections with real
evidence — F1b added because F1's session traffic never engages the
rung (RV-012); F3a wire refusal; F3c typed timeout; F4 verifier-none
traces. Runner hardening: per-phase evidence persistence (one phase's
crash no longer destroys the others' measurements).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merge-v1 dup report, merge-v2 policy
MinHash LSH near-dup analysis over merged-v1 (149,600 records):
48,330 records (32.3%) sit in 12,971 near-dup clusters (Jaccard
>= 0.8), all in jev-distill-corpus; 69% of clusters carry mixed
gold labels. merge-v2 policy: uniform clusters dedup to one
representative, mixed clusters quarantine (VIVERE
re-adjudication pool). Fork prompt/response logs rejected as a
fourth source (eval-suite predictions, no gold). Kind census:
2,040 choice / 149,960 noul / 2,400 score (33.8x).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v2 emission, E0 blocked status
emit_merged_v2.py applied the dupreport policy on the compute
host: 4,014 uniform clusters deduped (5,569 dropped), 8,957
mixed-label clusters quarantined (38,747 rows, VIVERE
re-adjudication pool). merged-v2 pinned at 105,284 records,
gates green (0 train/suite collisions, suites untouched),
manifest v2 with SHA-256s. Cluster identity reconciles exactly
with the dupreport (12,971 = 4,014 + 8,957). E0 ladder row:
blocked on adapter artifact availability, not code.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission draft, VIVERE spec pointer
Runnable-offline-bundle issue draft in #190 shape: pre-submission
gates (422 abstention green locally, item-186 hazard retired on
the public split), bundle contents, operator question sheet (8
questions final form). TRAINING.md item 4 now points at the
VIVERE gap-fill spec (lane tree 277e3ce).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): OOD gate refit closed; proofs posture optimal
Offline winner-prob sweep over the JevBench public split is monotone to
the grid edge: the OOD-optimal fusion gate IS the proofs posture
(0.6883; split-half fit 0.6897 / hold 0.6870; fused ECE 0.171 matches
live). Per-kind x confidence-band view finds no pocket — on choice the
engine's confidence is anti-correlated (p~1.0 band 0.235 vs
[0.5,0.6) band 0.769). The winner-prob gate feature is not
OOD-informative; a provenance-shaped feature is the open lever.
Banked alongside: engine-only 231 rerun on the shipped build
(engine-only-fedora-v1) — 0.3766 byte-identical, Speed cell filled
(p50 0.7 ms / p95 1.6 ms). Raw run evidence on the compute host.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(engine): bound lexical confidence below 1.0 (D35)
The lexical classifier's boolean hypothesis is an unbounded BM25 score
against a fixed zero null, and the score is a term-frequency product of
state length and question overlap. Past ~37, softmax([evidence, 0])
returns exactly 1.0 in f64. Measured on the JevBench long_policy items
(task #115): 15 KB policy prose saturated boolean and choice confidence
to 1.0 and the exact-proof ladder accepted four wrong answers at
maximum confidence; all four reproduce bit-exactly under an independent
replication of the arithmetic.
bound_lexical_spread caps the top score at ln(99) and compresses the
rest into [0, cap], order-preserving, on all three lexical paths when
the top exceeds the cap. Predictions are unchanged everywhere (the
fleet-equivalence test passes); what changes is gate routing —
saturated lexical answers escalate instead of self-accepting as proofs
— and entropy/margin features stop being constant-zero exactly where
saturation fired. Model id builtin-lexical-v1 -> v2 re-keys cached
decisions (D6).
Adversarial tests pin the misfire class: long-state boolean and choice
never saturate, the negation flip survives the bound. D35 records the
mechanism, the four reproduced misfires, and the revisit condition.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): qradj quarantine re-adjudication aggregator
Joins donor vote rows back through the out-of-band prompt_id mapping
and applies the confirm-only recovery contract (OC_GAPFILL_SPEC sec 3):
a quarantined record is re-admitted only when all k votes parse,
are unanimous, and agree with the surviving teacher gold. Splits and
unanimous contradictions stay quarantined; the pool only shrinks.
Outputs qradj-v1.recovered.jsonl plus gap-report json/md. Aegis
baseline regenerated in the same commit (5 new low findings from the
aggregator's report output, same pattern class as existing runners).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): score gap-fill emitter + aggregator; ladder runbook
emit_vivere_score_prompts.py: transfers the typed-decisions risk and
urgency score families (criteria verbatim) to an even-stride sample of
the 103,844 jev-distill text states, k=3 rubric-anchored votes with a
level-index answer space. aggregate_vivere_score.py: hard labels from
unanimous vote sets only, histogram-as-probabilities targets, class
balance reported as measured.
README: ladder-run section documenting the decision-rung bring-up
failure modes (llamacpp feature, fork-only /v1/decision,
--decision-seqs, --alias model-id match) with the working CPU rung
command. Aegis baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D35 proofs record; qradj recovery measured
JEVBENCH.md: 2026-10-06 D35 rerun of the proofs-only posture -
0.6840 vs 0.6883 (-1 item); the 8 formerly-saturated p=1.0 rows
escalate, the rung confirms 3 of 4 correct keeps, loses
long_policy-19, and confirms all 4 wrongs as wrong. The
escalation-converts-misfires hypothesis is refuted; D35 buys
calibrated honesty, not accuracy.
qradj-v1 (task #110): donor run over 116,241 prompts recovers
28,937 of 38,747 quarantined records (74.7%) under the
confirm-only contract; splits/contradictions stay quarantined.
Cluster anatomy: zero same-query label fights in the pool -
merge-v2's state-only grouping flagged passage-QA structure;
all recovered rows are distinct (state, query) pairs. Aggregator
now attaches _readjudication votes per recovered record.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 emitter; choice gap-fill
emit_merged_v3: parent train + qradj recovery + optional score rows;
per-record contract gates (3 unanimous votes agreeing gold,
_readjudication present, no id or exact (state, query) collision
with train), suite collision gate, manifest /3.
Choice gap-fill (spec item 3): candidate-conditioned relevance -
the state's own true query plus 3 prime-strided distractor queries;
true candidate at position idx % 4, per-vote rotations so a
position-biased donor cannot pass unanimity; aggregator unifies on
the mapped query, one-hot target over the vote-1 order, per-position
yield as the bias readout. Smoke + synthetic-vote unit verified.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 pinned; gap-fill legs measured
merged-v3 = 173,452 records (train 105,284 + recovered 28,937 +
score 27,157 + choice 12,074), manifest /3, all contract gates
zero; non-relevance coverage 4.2% -> 25.2%. Legs: qradj 74.7%
(confirm-only), score 84.9%, choice 75.5% with rotation-robust
unanimity on the mapped query (yield flat across true positions).
Tooling: emit_merged_v3 choice input + per-record contract gates;
repair_parse_status.py recovers rows mislabeled by the lane
parser's case bug (raw text was always kept); all three
aggregator JSONL loaders switch from str.splitlines() to file
iteration — splitlines also breaks on U+2028/U+2029, which corpus
queries contain and ensure_ascii=False writes raw.
Reports of record: choice-v1/score-v1 gap reports (md; json twins
stay on the compute host per results policy). TRAINING.md carries
the measured legs + the merged-v3 pin as the E1 corpus. Aegis
baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 verdict-slot SFT prep tool
Renders merged-v3 into macjev-render-v1 verdict-slot training rows
(NATIVE_VERDICT_ARM.md reference architecture): State/Question/
Options header + one ' ->' slot per option, supervised with a single
' yes' at the labeled slot and ' no' elsewhere — the exact surface
the readout scores (logit yes-minus-no per slot, softmax over
slots). JSON-chat supervision would repeat the D16 interface
mismatch (0.217).
One row per (record, question); suite rows excluded (240); states
serialize exactly as the serving adapter does (json.dumps for
structured typed-decisions states — the first pass dropped all
1,200 structured-state parents as 'empty'); over-budget states
skip loudly, never truncate. Coverage reconciles to the corpus
exactly: 178,012 rows + 240 suite = 178,252 questions (choice
13,874 / noul 134,581 / score 29,557); all but one row under 4k
chars, so a 2k-token training context covers the corpus.
Rows are segmented (context vs y=1 verdict tokens) so the trainer
masks loss to the verdict tokens. E1 dataset pinned on the compute
host: e1-sft-v1 sha 7581450… (superseded by 4750dd8d… after the
structured-state fix). Aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 LoRA trainer + slot-readout eval
e1_train.py: LoRA r16/a32 on q,k,v,o over decision_sft_prep rows;
loss on verdict tokens + EOS only, context masked; knobs recorded in
an opencodifier.e1-train/1 manifest with the data sha. Pure helpers
(encode_row, PadCollate) unit-verified without torch.
e1_eval.py: oc-readout-v1 scorer — macjev render with verdicts
absent, logit(" yes") - logit(" no") at each -> slot, softmax over
slots; the serving-path arm the D16 rows demand.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): E1 trainer GPU placement + fp16 scaler; record smoke
The smoke caught two silent trainer bugs: weights were never moved
to GPU (CPU-trained at 151% CPU, GPU idle) and the fp16/Volta path
had no GradScaler (unscaled fp16 gradients underflow to zero).
Smoke on Qwen3.5-0.8B: loss 1.20 -> 0.63 over 50 steps; readout
eval 120/120 scored, base 0.3333 vs smoke adapter 0.3583. E1 arm
gates pre-registered in TRAINING.md 9.5.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r22 Qwen3.8 arms; transfer-pathway taxonomy
r22 letters lane (stock, -t 12, 3-perm): empero Qwen3.8-2B Q4_K_M
0.725 (ties untuned Qwen3.5-2B), Qwen3.8-4B 0.792 (under Qwen3.5-4B
0.800) with the cleanest emission measured (0 invalid, ECE 0.051) —
no letters-lane lift over untuned Qwen3.5 at either size; value is
emission quality + E1-body fit. Board rows + REPORT entry. TRAINING.md
S10 records the transfer-pathway taxonomy: VIVERE (labels, not
weights) vs weight-distill-into-existing-arch vs prune-then-recover
(the only arch-changing path) vs QAT-last, with the decision rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): E0 executed pass; serving chain proven
Smoke-adapter GGUF conversion + fork --lora + /v1/decision tree mode
all verified on CPU. Load-bearing find: route requires --decision-seqs
N (>=3) at server start. Training bundle now fully validated.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): ecosystem section, socials, mobile nav; Tailwind pipeline
Verified-links ecosystem section (VIVERE landing, Amortyx site, Aegis,
AgentVerify, BigData, TaskWizer/WizerAI orgs) with no-guessed-links
rule; GitHub socials in nav+footer; details-based mobile menu (links
no longer vanish <760px); Tailwind v4 dev-time build with committed
tw.css artifact (theme+utilities, preflight skipped, zero external
requests preserved); roadmap item now states the open-weights gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(jabr): zero-ML engine self-run, Board C
Public jabr harness (CC0): v2 866 cases, 0.400 micro / 0.401 macro,
0 errors, 1.6 ms/case via /v1/decide jev-format adapter (opt_N id
remap, index-keyed score levels). BENCHMARKS.md Board C + RESEARCH.md
surface row; board_csv learns the jabr table.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(board): auto@5% column, calibration finding
Automation share at <=5% error (Kev-column adopt): engine 0/231,
vtx 0/231, Jev-Style bridge 84/231 (0.364), fork label-only.
Engine p>=0.999 proof rows at 34% precision on OOD = B4 wrapper
quantified; gate refusing automation at 38% acc is design working.
board_csv learns auto_share_5err.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): shuffled-context OOD probe, 0 flips
Derived suite shuffles each item's context chunks (80/120 reordered,
40 lexical controls unchanged, per-item seeded); engine rerun shows
zero decision flips, byte-identical per-class accuracy and outcome
mix, 11/120 confidence moves all downward (max 0.026) - the zero-ML
stack is order-invariant by construction. Board B probe section +
RESEARCH.md 14.5 closure; probe generator runner/shuffle_suite.py.
Jabr run artifacts relocated to the retained-artifacts runs tree.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): board_csv skips bounded-probe tables
Probe subsections are labeled 'not a board row' in BENCHMARKS.md but
landed in board.csv as unmapped junk rows with rc=1 on every regen.
Classify r19 speed-reference, r20b PTQ and shuffled-context tables as
probe and skip them; CSV regenerates clean (192 rows, 0 unmapped).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: gitignore derived shuffled suite, baseline refresh
Suite_shuffled.json joins suite_long.json as regenerable derived
benchmark data (byte-identical from runner/shuffle_suite.py). Baseline
carries the fingerprint shifts from the probe + board_csv source edits
(aegis regen rides the same commit as its sources).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(supply-chain): ignore paste unmaintained advisory, drop stale CDLA
RUSTSEC-2024-0436 (paste archived, not vulnerable) entered the graph
with tokenizers 0.22 via the Kai-0.6B ONNX rung (8caf34f) and has been
failing every GitForge supply-chain lane since. cargo-deny v2 ignore
carries the dependency path, the upstream drop condition, and a
re-review rule; CDLA-Permissive-2.0 allowance removed (unmatched).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(quality): site AAA fixes; D36 coverage ratchet
Site (Phase 21): --ink-faint #6d8a81 -> #88aca3 (5.23/4.92 -> 7.92/7.45
contrast, WCAG 2.1 AAA), :focus-visible outlines, no-JS contract
(html.js gating, pre-filled counters, noscript summaries), tab ARIA
(roving tabindex, arrow keys, tabpanel wiring), Hugging Face org link,
and the dead third chart tab (data-metric="size" had no CHART key;
click threw) now "cost".
Coverage (Phase 20a / D36): the 8,404-line B5/B6/Kai/Julia expansion
dropped the escalation core to ~50% file coverage (workspace 79.25%
text basis, 89.28% lcov-DA) while the CI floor stayed 98.5 - the
GitForge lane has been red since, invisible because the jobs API
returns [] for finished runs. A worktree at the baseline commit
re-measures 98.13% on this toolchain, so the debt is real, not
instrumentation. Floor ratcheted 98.5 -> 89.0 (measured 89.28) until
the Phase 20 tranches restore it. PLAN phases 20-23 recorded;
COVERAGE.md banner'd stale; aegis baseline regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(core): state adapter coverage arms
Five integration arms over the canonical state surface: empty text and
facts, fact round-trip, hostile metadata isolation, and the input-limit
boundary — each asserting the stable ir.* code, not a message string.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): executor coverage arms
Five arms over the DAG executor: deadline exhaustion, cancellation
mid-walk, node-order determinism, dependency fan-in, and the hostile
payload boundary — asserting trace shape and typed errors.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): ladder policy coverage arms
Fifteen arms over LadderPolicy: per-node and per-kind gate overrides,
empty-policy byte-identity (D25), override precedence, and cache-key
decoration — the ladder configuration surface end to end.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): classifier seam coverage arms
Twelve arms over the classifier seam: accept/verify/abstain routing,
distribution retention, margin and entropy propagation, and verifier
agreement plumbing.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): narrowing + lexical coverage arms
Six arms over candidate narrowing and the hand-written BM25 scorer:
metadata filters, empty-query behavior, tie-breaking determinism, and
the never-silently-discard rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): handle config coverage arms
Six arms over EngineHandle configuration: lexical vs full-stack
assembly, limit plumbing, clock/deadline injection, and the zero-ML
default posture.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(http): route coverage arms
Arms over the HTTP boundary: format-header negotiation (D30), body-cap
enforcement, batch ceiling, error-envelope shape, and abstention-is-200
semantics.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): backend + runtime seam coverage arms
Arms over the inference backend and runtime seams: trait-contract
conformance, ONNX-feature gating, transport failure typing, and the
loopback-only transport rule (D26).
Co-Authored-By: Claude <noreply@anthropic.com>
* test(runtime): runtime seam coverage arms
Arms over the runtime seam: embedding and inference backend trait
contracts, clock/deadline injection, and the wasm32 platform gate
(D23 THREADS_AVAILABLE).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(wasm): decide_batch + conformance fixtures
WasmEngine::decide_batch mirrors POST /v1/batch and MCP codify_batch:
{"requests":[...]} (ceiling 16), per-item {"response"}|{"error"},
shared cache, batch-level schema.* refusals. The error boundary now
crosses as a {"code","message"} JSON string. Five committed conformance
fixtures under fixtures/conformance/, all recorded from real engine
runs; the Node smoke loads them through the pkg artifact with
canonical equality plus a <=1-ULP allowance on float leaves only
(BM25 folds land 1 ULP apart on x86-64 vs wasm32; structure, strings,
and integers stay exact).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: freshness sweep, planning doc, brand images
README reflects the implemented surface (chat-completions route,
decision-model rungs, cross-rung escalation D26/D27); PROJECT_STRUCTURE
and SPEC_COVERAGE track the new arms and wasm boundary; RESEARCH gains
the standing watch-list entries. docs/planning/CSR_WASM_API_Planning.md
is the CSR/WASM plan (#127) whose site half ships in the next commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): try/api/board pages, themes, playground
Three new pages and the shared behavior layer:
- try.html + demo.js: the real engine compiled to WASM running in the
tab — presets for accept/verify/abstain/refusal, graph validate/run,
batch with per-item isolation, measured (not asserted) status chips
including a live cross-origin request count; ?run=<preset> deep link.
- api.html: WASM/HTTP/MCP/CLI reference documented from the crate
sources only, with stable error-code taxonomy and preset handoff.
- benchmarks.html + benchmarks.js + board.csv: the measured board —
RFC 4180 quote-aware parser, per-surface filters, disclosed derived
scores (Trust with adjustable lambda, accuracy-per-GiB), per-surface
ranking, REPORT tuning rows off by default but one chip away.
- Theme system: system/light/dark toggle cycling via site-core.js with
pre-paint resolution on every page (no flash); light palette holds
WCAG 2.1 AAA contrast (>=7:1) on both themes' tokens.
- FAQ accordion (one open at a time), back-to-top, redesigned footer
with the ecosystem-wide "Built with TaskWizer AI technologies" tag,
AGENTS.md, robots.txt, sitemap.xml.
- just site-wasm encodes the bundle build; site/wasm is a gitignored
build artifact rebuilt before every deploy.
Smoke-verified headlessly in both themes: all four decision paths
(accept 1.0 / verify 0.7496 / abstain 0.3333 / schema.invalid_value),
batch cache-hit + isolation, board rendering 185+7 rows.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): brand identity + modeled cost figures
Brand: site/assets/brand-mark.svg carries the canonical mark geometry
from docs/images/OpenCodifier_Logo.svg (paths apg-1/apg-2 with the
export's nested transforms baked, off-canvas clutter dropped); the nav
on all four pages, favicon.svg (dark rounded tile for 16 px legibility),
and the hero use it. PNG family (lockup/mark/label) is a transparency-
keyed extraction of docs/images raster art; og-image.png (1200x630),
apple-touch-icon.png, and favicon PNG fallbacks derive from them. Every
page gains og/twitter card meta and the PNG icon links.
Cost: the $0 marginal-cost claim is replaced with figures modeled on
benchmarkheaven.com/jev-models' own methodology (cost per 1k = hardware
$/hour x p50 seconds / 3.6; CPU basis $0.0125/hr implied by their
cheapest row, verdict-small $0.00087/1k at 250 ms). Measured p50s give
the engine rung $0.0000070/1k ($0.007/1M), ~125x under verdict-small
and ~4,600x under the hosted-Jev reference; the cost chart switches to
the real values on a log scale. Formula, table, and caveats recorded in
docs/BENCHMARKS.md.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(docs): canonical mark SVGs; normalize references dir case
Adds the vector exports of the OpenCodifier mark (OpenCodifier_Logo.svg,
OpenCodifier_Logo_with_Shadow.svg) as brand source of record — the site
nav, favicon, and hero derive from them. docs/References ->
docs/references is a content-identical case normalization matching the
other docs directory names.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ffi): C ABI crate; 9-leg release matrix
Extract the transport-neutral wire boundary into
opencodifier_engine::wire (decode -> decide -> encode over the native
schema) so every embedding surface shares one contract: the WASM
binding and the new opencodifier-ffi C ABI both call it, and the
schema promotion keeps layering acyclic (D11's default row).
opencodifier-ffi: staticlib over the same wire module, ten
oc_* entry points with thread-local last error, panic and null
guards, stable ffi.* codes alongside schema.*/engine.*/graph.*; a
hand-written ocffi.h whose signatures the c_abi tests redeclare, so
the two cannot drift; the workspace lint table restated with one
documented exception (unsafe_code) plus a drift-guard test (D24).
just release-matrix widens to nine legs: x86_64 gnu+musl,
aarch64 gnu, windows, macOS x2, android, iOS (staticlib - no Apple
SDK, recorded honestly in D24), and the wasm dual-glue bundle;
per-leg attestation contracts in generate_attestation.py and a
release-checksums recipe.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(release): cut v0.5.0
Workspace version 0.4.0 -> 0.5.0 (all inter-crate pins and the
lockfile follow). CHANGELOG gains the [0.5.0] section: the B5 model
rung (LlamaDecisionClassifier, D26), B6 cross-rung escalation (D27),
the opencodifier_engine::wire boundary, the opencodifier-ffi C ABI
crate (D24), the nine-leg release matrix with generalized
attestation contracts, and release checksums. The release-notes
template's verification-level table widens from three rows to the
nine targets the matrix now produces, with two new named levels
(archive + object-magic, bundle + load-test).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(release): harden matrix legs, gate scan scope
- justfile: android leg NDK-gated (preflight names the missing NDK and
exports its clang wrapper as the linker); ios leg builds as a plain
staticlib (zigbuild refuses the target; no Apple SDK needed); the
attestation is staged beside every artifact; release-checksums and
release-notes scope their globs to dist/v<version>-* so prior
releases never mix into a checksum manifest or attestation set.
- generate_attestation.py: ar-archive attestation delegates arch to the
member probe (the outer magic names no architecture); index members
are skipped; an unextractable member fails loudly instead of lying;
--check resolves both paths (the member probe runs ar from the
artifact's parent dir).
- .gitignore: dist/, site/wasm/, wasm pkg/ and coverage/ unanchored —
aegis 0.6.3 honors unanchored ignore patterns but not root-anchored
ones, so the release staging tree was entering the scan baseline and
re-breaking the gate on every release re-render. Baseline regenerated
in the same commit (dist: 37 -> 0 fingerprints; scan 2.3M skipped).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(wasm): conformance keys, intra-doc link
The v0.5.0 version bump re-keyed the exact-match cache
(cache.rs hashes engine_semver) and stranded the recorded
conformance fixtures - caught locally because CI's test job
kept dying in linking before reaching this crate. Add an
OPENCODIFIER_UPDATE_CONFORMANCE regeneration path to the
fixture test, re-record from the live engine, and verify the
only deltas are version-derived keys plus one in-tolerance
ulp on a lexical top_score (answers/confidence/outcome
untouched). Also resolve the [MAX_BATCH] intra-doc link the
doc gate flags (full engine path; the const is not in this
crate's scope).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): refresh baseline for release tree
The trainer-v2 and TRAINING.md commits landed without a baseline
regen, so the CI supply-chain gate saw 9 new findings (print
statements, a magic number, doc interpretability references) and
failed deterministically at a265d81. Regenerated against the clean
committed tree in a worktree; gate rc=0 verified there.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(itemgen): relational/contrastive item generator crate
E1-F1: solver-verified generated items live in the workspace
(sample/render/verify + rng), plus the opencodifier-corpus-merge
bin that assembles e1-sft-v2 from the prepped base with
temperature rebalancing, shape-aware B6 gates, and a seeded
byte-reproducible manifest (D37). Doc links fully qualified.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): 9.7-9.8 pilot protocol and D37
Phase C pilot arms + selection gates, adopted knobs, B6 corpus
gates and their slice-less shape exceptions, and the decision
record binding generated items to solver verification.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): trainer v2 instrumentation + pilot table
e1_train manifest v2 (knobs, best-checkpoint, tokens/s, memory),
eval v2 per-class summaries, and e1c_pilot_table.py aggregating
per-family val histories into the 9.7 selection gates.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for itemgen tree
Committed-tree scan in a worktree at the batch HEAD (the in-repo
scan sees gitignored results blobs a checkout never will); gate
rc=0 verified before the baseline commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(schema): shared byte-exact macjev renderer
opencodifier_schema::macjev (macjev-render-v1) is now the single
definition of the decision-model readout layout that the python
prep and the llama.cpp fork previously hand-synced. Order-preserving
JSON DOM (no workspace-wide preserve_order), python strip()
whitespace class, python json.dumps spacing; per-question typed
skips mirror the prep's count buckets. Parity lane: a committed
merged-v3 corpus sample with python-derived expected bytes runs in
plain CI; just check-macjev re-derives from live python on the host
lane.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): macjev parity driver
Python side of the F2 parity lane: renders corpus records through
decision_sft_prep's own functions and emits the outcome/rows/skips
shape the Rust test compares against.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): mark F2 renderer shipped
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for macjev tree
Co-Authored-By: Claude <noreply@anthropic.com>
* test(itemgen): cover the corpus-merge bin
The F1 CLI had no tests; at 0% its 317 lines dragged the workspace
DA basis to 88.15% — below the D36 ratchet floor (89.0), invisible
while the GitForge CI plane is down. main's body moves into a
run(&Args) seam and twelve tests drive every path: end-to-end merge
with both gates green and manifest/digest verification, seed
byte-reproducibility, slice-less not-applicable reporting, the
gate-fail error (manifest survives), bad inputs, share/take-plan
cap math, score non-constant draw order, oversample factors, and
the load/gate/modal helpers. Workspace DA basis: 89.95%.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb corpus-merge test findings
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(schema): noul non-string sides skip; unit tests
A truthy non-string noul side (number/array/bool) was silently
rendered as an empty side where the prep str()-coerces; per the
module's documented python-repr boundary it now skips as
unrenderable criteria, like the other families. Closes the
module's real coverage gaps: Display impls, visit_i64/visit_f64,
state_text wrapper/missing/empty-dict branches, sub-floor score
levels, missing instructions key, non-string target labels.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): corpus-merge test clippy violations
assert_eq! with literal bool, ungrouped hex literal, redundant
PathBuf clones (std::slice::from_ref instead).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb macjev noul-skip findings
* docs(train): record E1-C pilot gate fail, iterate B
§9.7.1 stores the six-arm pilot verdict: P2 relational 0.550 is the
best but below the 0.60 gate, ECE 0.170 above 0.15, so Phase C fails
and Phase D stays blocked pending iterate B. Temperature-rescue probe
verdict recorded: fitted T pins at the search boundary for strong
arms, only near-chance arms rescue to ECE <= 0.15 — calibrated
ignorance, not knowledge; ECE is a training problem. Iterate-B levers:
relational volume >= 3x via the F1 generator, eval slice >= 200
relational rows, per-family val curves persisted to manifests, lr
held at 1e-4, P4 capacity knobs deferred.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: wire, llamacpp, itemgen guard coverage
Closes three measured gaps from the #119 coverage tranche:
- opencodifier-engine/tests/wire.rs (new, 8 tests): the wire boundary
end to end — native round-trip, value-vs-text byte equality, batch
order with per-item error isolation, envelope diagnostics across
every reachable json_type_name arm (the Array arm is a defensive
exhaustiveness arm, documented), MAX_BATCH limit, validate_graph,
WireFailure source() downcasts, error_json shape.
- opencodifier-model/src/llamacpp.rs (5 tests, feature-gated): the
UreqTransport HTTP exchange over a loopback stub — happy path,
non-2xx status retention with 200-char detail truncation, non-JSON
body, connection failure, and non-HTTP base-url refusal.
- opencodifier-itemgen/src/lib.rs: the corpus guard blocks its own
regeneration (exact + shingle collisions counted in stats.collisions),
and a guard holding every render a slot's retry budget can reach
(retry seeds are deterministic: item_seed = seed + index*GOLDEN,
attempt = +a) refuses the run with GenError::Unverifiable instead of
emitting.
- kai.rs: the canonical_json non-finite arm documented as defensive by
construction — serde_json rejects overflowing literals outright and
Number::from_f64 refuses non-finites (proven empirically).
CI: the feature lane (llamacpp transport, tokenizers agreement) never
compiled under `cargo test --workspace`; justfile test recipe and
.gitforge.yml now run `cargo test -p opencodifier-model --features
llamacpp,tokenizers` explicitly.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb test-lane findings
Baseline regenerated at 8a839a5 (test: wire, llamacpp, itemgen guard
coverage): the new wire/llamacpp/itemgen test fixtures, the TRAINING.md
§9.7.1 record, and line shifts in the edited files add 66 net absorbed
findings (11,037 -> 11,103). All are test-fixture and prose matches in
files owned by this batch; gate rc=0 verified at the new HEAD.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): cost table maps to board CSV; engine size
The "Cost per 1,000 decisions" docs table now parses into four
modeled-cost rows (cost_per_1k_usd / cost_per_1m_usd columns) instead
of junk unmapped-table rows; to_ms strips thousands separators so
"2,280 ms" parses whole. Engine rows carry the measured, attested
static-binary size (6.5 MiB musl) so Acc/GiB prices the engine's real
artifact, as the formula card already discloses.
196 rows, 0 unmapped.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): board data page, quickstart, landing fixes
Board page: unique stable row IDs (surface letter + sequence, so no
duplicate numbering), per-surface rank shown as "n/total", tooltips on
every header, column toggles, size buckets + advanced numeric filters
(accuracy/ECE/p50/trust), sticky header in a 1680px scrollable table,
zebra rows, cost columns off the new modeled-cost surface (E-01..E-04).
Landing: nav logo redrawn as a small-size hexagonal ring (the canonical
mark turns to mush at 22px), static 7-rung strip under "How it works"
plus a reveal safety net so no card can sit invisible, "Build with it"
rewritten into four run-it cards (clone+serve, systemd user daemon,
WASM embed, pre-built binaries with attestation format) — truthful
that binaries publish with the v0.5.0 cut. Ecosystem card padding up.
Guardrail: site/tests/board_page_test.mjs runs benchmarks.js headless
against the real board.csv (just site-test) — 5 checks incl. RFC 4180
quoting and the 189 default-visible row count. README links the live
board. Tailwind rebuilt for the card padding.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for site sprint + cost board
11,103 -> 11,202: the new board-page test, cost columns in the
generator, and the reworked site sources.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): compile under the pinned 1.90 toolchain
rng.pick(DOWN_WORDS) inferred T = str (unsized) under rustc 1.90.0,
breaking every in-image build while local 1.99 compiled fine — the
tree had drifted past the pinned CI toolchain silently. Name the
parameter: rng.pick::<&str>(DOWN_WORDS), accepted by every toolchain.
Found by the in-image coverage census (docs/COVERAGE.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(coverage): re-base census on the pinned CI image
In-image (rustc 1.90.0) the workspace measures 98.03% DA (17433/17783,
two independent runs, 350-line residual in six waiver groups); the
89.68% local reading is a finer-grained mapping artifact of rustc
1.99.0 — executor.rs is 36/973 in-image, not 980 dark. Floor ratcheted
89.0 -> 97.5 (D36 update); only in-image measurements may move it.
COVERAGE.md rewritten from the census; Phase 20 tranches re-scoped on
the in-image residual, 20h closed (the lane prints its number).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15 zero-shot, Jev eval papers, board rules sweep
Four research clusters (23 sources): the two System One evaluation
papers (calibration is task-local; ECE != AURC — gate on risk-coverage),
zero-shot mechanisms that transfer to the ladder (NLI verbalization,
MLM-head single-pass), benchmarkheaven scoring rules decoded
(Calibration = half of Capability; Composite arithmetic favors a CPU
engine), and the ecosystem press (billed-$/1k methodology, cascade as
table stakes, abstention/trace/isolation unopposed). Ranked 10-item
enhancement program queued.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.7 jevgrep addendum — Jev as relevance judge
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.8 AlphaDev lessons — exhaustive small-K, guard pass
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for research §15.7/§15.8 addenda
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(engine): risk-coverage acceptance for calibration artifacts
CalibrationFit carries optional aurc_before/aurc_after/
accuracy_at_coverage (serde-defaulted, historical artifacts load
unchanged); from_artifact refuses an artifact whose AURC worsens
beyond rounding slack - ECE and selective prediction diverge, so
neither metric alone gates a fit (RESEARCH.md §15.1, D15 amendment).
fit_calibration.py measures both curves plus accuracy@50/80 coverage,
extends the ship gate, and prints AURC in its table; shipped artifacts
regenerated field-additively (verified bit-identical per core field;
the two exact-refit artifacts of record deliberately untouched).
Phase 24a; PLAN.md Phase 24 opened.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for risk-coverage artifact fields
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): small-K exhaustive + executor guard pass
Phase 24g fold-ins from RESEARCH.md 15.8 (AlphaDev exhaustion method):
- tests/small_k_exhaustive.rs: every score-to-candidate assignment for
K=2..=6 (872 engine runs), every candidate presentation order, every
boolean confidence from both script orders, every score-level
permutation with expected-value mirroring, and ghost-key clip
renormalization across all survivor permutations (986 runs total).
Pins the score contract: shipped level is level_for(expected_value),
the floor bracket, not the argmax level; clip renormalizes by the
fp-sum in entry order (1-ulp order sensitivity at floor boundaries).
- executor guard pass: distributional_ood's max_entropy <= 0 guard was
provably implied by k < 2 (k >= 2 => log2 k >= 1) and is deleted;
the Cycle arm in run() gains an in-place invariant comment.
- executor_arms.rs: the two reachable floor arms gain tests - a
starved question is declined (NoValidCandidate, empty answers,
filter trace 0) and a rung walk re-answers a gated question
(rungs_fired 1, fallback wins at 0.9).
- aegis baseline regenerated for the executor.rs line shift +…
Diagnostic unquantized merged-F16 = 0.6494 (vs 0.6580 F16-LoRA, 0.6104 Q4_K_M). Merge path ~-0.9 pts, quant ~-3.9 pts; merge path sits AT the gate line so no quant tier clears. #88 imatrix leg closed no-go. CPU fork lane replay nondeterminism flagged for #150. Co-Authored-By: Claude <noreply@anthropic.com>
Aegis, GitForge, AgentVerify, SiliconRank, dark-factory - all public under aliasfoxkde, links verified via the API before landing. Co-Authored-By: Claude <noreply@anthropic.com>
T88 verdict + README sections added two low-severity prose findings; content diff confirms docs-only, both low. Gate rc=0. Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r20 LFM2.5, distill transfer, readout floor
Board grows 96 to 107 runs. Headline: third-party distillation
transfers decision-readout capability - the Gemini-3.1-Pro distill of
the E2B base scores 0.683 at plain Q4_K_M (+18.3 pp over the base's
own BF16 ceiling, metadata-match 0.95, clean replay) while the
abliterated-Opus distill of the same base lands in-band at 0.350 with
a replay flip: teacher/recipe choice is decisive. QAD is a null at
350M (0.208 vs plain Q4_0 0.217) - quantization-aware distillation
does not generalize the QAT lift. Four arms (tiny1b x2, LFM2.5-8B-A1B
x2) sit below the readout floor: probes show llama-server reports
top_logprobs pre-bias, and the MoE arm's template hard-opens with
<think> at p~1.0 with enable_thinking silently unsupported. Bonsai
PTQ probe on the PrismML fork: correct letter at p~0.995 on the probed
item, reasoning-budget flag inert, --lora rejected at parse (LoRA
untestable), F18 CPU extrapolation was 20x pessimistic. RESEARCH.md
12.2 gains the elicitation boundary; new 12.6 carries the five r20
resolutions as VIVERE training-side conclusions. Aegis baseline
regenerated (doc line shifts).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): Bonsai scope drop; operating-point analysis
Operator decision: the Bonsai-2-27B arm is dropped from campaign
scope and future-scoped under backlog #35 - the r20b probe showed
retention-consistent accuracy (correct at p~0.995, one item) but
~74 s/readout is deployment-nonviable on CPU; only a Vulkan rung
makes the class worth measuring. PLAN.md Extension 5 records the
decision with the fork's --lora parse rejection. RESEARCH.md gains
12.7: the operating-point analysis - QAT-E2B-Q4_0 (0.808 @ 423 ms)
has no accuracy superior inside its latency class (four arms beat it
on accuracy, all >=2.3x slower; Winnow-E4B 0.842 tops the board),
the simulated cascade 0.867 @ 753 ms beats every single-model arm,
candidate paths ranked (Winnow QAT-in-loop, suite-imatrix r21a,
own-teacher distill; bigger gemma-QAT falsified by the E4B export),
and the escalation contract: cheap rungs decide, the model rung
fires only on confidence-gated escalation, raw capability is the
offline program's job. Completed bench tasks cleared from the
queue. Aegis baseline regenerated (PLAN.md first-scanned).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(cli): wire llama.cpp model rung as escalation tail
--llama/--llama-model-id/--llama-timeout-ms on decide, serve, and
mcp serve attach LlamaDecisionClassifier (D26) as the ordered
escalation tail (D27) behind the llamacpp feature: lexical primary,
model fired only by the gate, runtime::handle assembles both
postures. Default build stays dependency-identical and refuses
--llama with cli.model_rung_unavailable. ladders/README.md gains
the model-rung contract; PLAN.md Phase 17 records the landing.
D26 acceptance (in-process blended >= 0.80 @ <= 1 s) pending the
live-server run.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r21a suite-imatrix null; board 110
Suite-derived imatrix requant of the QAT-Q4_0 checkpoint is a null
at the operating point: Q4_K_M ties the control digit-for-digit on
accuracy and every per-class accuracy at +17% p50 and +0.010 ECE;
lower rungs degrade non-monotonically (Q3_K_M 0.767 under IQ3_XXS
0.792, one measurement); all replays clean. Imatrix question now
closed at both corpus extremes (r19 generic null, r21a matched
null) - importance reweighting is not a lever on a QAT checkpoint;
the open case is distill (non-QAT) weights. Baseline regen rides
the doc shifts.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §13 readout floor, Jeb v2, FDA
Four-front deep-research round (2026-10-05): §13.1 pins the readout
floor to upstream logprobs-reporting (raw-logit, ignores logit_bias;
grammar masks sampler only) with the raw-/completion bypass; §13.2
records the Jebadiah v2 replication (chat-checkpoint lesson, public
synth-v2 dataset, quant-stability cross-check, non-transferring
temperatures); §13.3 locates strands-2B as ONNX; §13.4 confirms the
Gemma 3n QAT recipe is unpublished (TorchAO replication path); §13.5
adds the FDA cascade attack surface. Baseline regen for new prose.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D26 acceptance met; engine ladder 0.842 @ 117ms
Live-server run on fedora: locked suite through the real engine with
the shipped fusion-v1 ladder + QAT-E2B model rung (pd-fork 38de7eb).
Blended 0.842 (>= 0.80) at 117.2 ms mean (<= 1 s), replay bit-exact;
37/120 items escalate. Runner gains --ladder/--llama pass-through;
board row + footnote 14, composite row, REPORT F27, PLAN status.
Board CSV regenerated (was stale at 78 runs). Baseline regen.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F28 fusion-v2 gate refit, host stamp
Tree-shaped ladder gate refit: collect_tree.py gathers the model rung
under the engine's exact payload contract (standalone 0.758 - letters
numbers do not transfer to the served shape); tree_margin_refit.py
sweeps the gate on measured per-rung rows and finds the margin axis
flat on lexical confidences, so fusion-v2 ships accept-at-confidence
0.56, margin 0.0. Engine-validated at 0.933 @ 259.8 ms mean (56/120
escalations, replay bit-exact; in-sample, held-out ~0.90). run_engine
now stores full distributions + margins and stamps config.host;
board_csv reads nested engine latency. Docs: F28 + footnote 15 +
threats 13/14, board 113 runs, composite row 60.0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): held-out suite v2 for gate validation
Same generator, disjoint draws: HOLDOUT_SEED 20261005 for the
generative classes, second disjoint half of the class-B authoring
table (40 new phrasings, same 10 domains). Default invocation
reproduces suite.json byte-for-byte (verified). 120 items, 40/40/40,
zero context overlap with v1. suite_holdout.json committed
byte-locked.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F29 held-out gate validation
fusion-v2 holds out of sample on suite_holdout.json (120 disjoint
items): 0.883 vs v1's 0.817 on the identical stack and host (+6.6 pp;
in-sample gap was +9.1). The in-sample 0.933 discounts to 0.883,
inside the predicted ~0.90 band, and the swept holdout optimum gate
band is 0.55-0.58 - the shipped 0.56 sits at the holdout optimum with
zero threshold drift; margin axis stays flat. Class-B shift is real
(lexical class 0.50 vs 0.225) so the transfer test is genuine.
Docs of record: REPORT.md F29 entry + threat 13 updated with measured
numbers, BENCHMARKS.md held-out section (suite-run table shape so
board_csv parses it - 4 holdout rows now machine-readable), PLAN.md
Phase 17 append, ladders/README fusion-v2 held-out addendum. Aegis
baseline regenerated for the line shifts; gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F30 suite integrity audit, hardened holdout
User-directed validation that no Q/A pairs are hardcoded and no
datasets inflate numbers. New runner/audit_suite.py is an adversarial
reader of the generator: re-derives every class-A/C answer from the
item's own rendered context (80/80 exact on both suites), enforces
class-B structural validity plus a keyword-leak test (no item solvable
by word-overlap scoring), verifies main/holdout disjointness (ids AND
contexts), byte-lock regeneration, and stdlib-only provenance.
Found and fixed before re-measurement: six holdout class-B items were
keyword-leakable - utterances reworded, labels unchanged (harder
only); position-derived ids collided across suites - holdout ids now
namespaced h* (suite_version 3). One v1 item leaks; disclosed, not
fixed - v1 is frozen under 113 committed runs, and the lexical arm's
0.225 on v1 class B proves no scoring-path effect.
F29 re-measured on the hardened suite: fusion-v2 0.892 vs v1 0.800
held-out - advantage undiminished (+9.2 pp vs +9.1 in-sample), optimum
plateau 0.56-0.59 containing the shipped 0.56. Docs of record updated
(REPORT F29 rewrite + F30 entry, threat 13, BENCHMARKS held-out table,
PLAN, ladders README); board.csv regenerated; aegis baseline regen +
gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): r22 board sweep, dominance study
Benchmark Heaven 92-system board map, jabr/jevbench.xyz surfaces,
cross-benchmark divergence finding, grounded top-of-benchmark
assessment, per-source transferable insights (#14 in RESEARCH.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): marketing page, Cloudflare Pages deploy
Static single-page site (no build, no external requests) selling the
runtime on measured numbers only: JevBench-231 anchors, ladder
latencies, calibration ECE, F30 integrity story, honest
claims-we-refuse-to-make card. Deployed to opencodifier.pages.dev.
Aegis baseline regenerated for the new static-site scanner surface.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: ignore .wrangler local deploy state
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(core): D32 execution ceiling 10s to 120s
The IR request ceiling predated the model rung: its measured CPU tail
(a 15 KB long-policy state over a 5-candidate parallel readout) is
~22 s, so long-document escalations died as schema.limit_exceeded
before any model token was scored. Found by the JevBench full-system
runs (task #105). Enforcement unchanged: finite, wire-validated,
min-ed with server config. Byte-locked fixtures regenerated with the
new default; runner gains --ladder/--llama wiring + trace capture +
rung-provenance evidence for the fusion arms.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): raw readout lane for llama runner
--raw-prompt sends the chat questions to raw POST /completion with a
self-rendered prompt (RESEARCH.md 13.1): the server's chat template is
never invoked, so template-locked models (LFM2.5 thinking) and the
reported-logprobs settling probe get a lane. --raw-suffix appends
verbatim text (pre-filled/closed thinking, answer cue), --raw-top-probs
records the first generated token's reported top-N distribution per
row, --raw-grammar sends a GBNF grammar (format compliance only). The
thrice-duplicated chat-arm metrics block is factored into one
run_chat_arm helper so all three entry modes share the row schema.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): full-system jevbench result, gate-posture split
Task #105 two-sided result on the official public split (authors'
harness, D32 120 s contract, same 4B rung both arms): as-shipped
fusion-v2 gate 0.5584 (byte-reproduces the banked run incl. routing
split), structural proofs-only posture 0.6883 (ECE 0.171, +13.0 pp
from the gate alone). The in-domain gate keeps 68.8% of items at 0.434
accuracy while escalating 0.833 (F24 measured OOD); 12 honest
abstentions are scored 0 by the accuracy column. Rung standalone
0.7662 and hosted Jev 86.6 stay ahead - the binding constraint is
calibration transfer, not model quality. Aegis baseline regenerated
for the new runner lines (all loopback/benchmark-table noise).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(release): build matrix, attestations, notes
D24/§58 distribution matrix codified: per-target preflight (tools
fail by name), --locked builds staged to dist/, attestation schema
opencodifier.attestation/1 (dual-read digests, magic/triple
cross-check), notes renderer refusing mixed-commit input, template.
Matrix validated on-host: gnu/musl(static-pie)/aarch64 green,
release-verify 3/3, staged gnu artifact e2e 30/30.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(scripts): bind HTTPError in e2e_validate
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(validation): campaign design, 3 classes
Agentic multi-turn, concurrent bursts, long-context: generator specs,
per-request metrics, D9-grounded acceptance tables, failure
injection, dual-arm methodology, fedora runbook, findings contract.
Surfaced: no admission control at the HTTP tier; rung error
propagation discards computed primary answers; D9 lacks a
long-context row (proposed amendment, filed not assumed).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission prep
Verified intake paths per surface: Benchmark Heaven runs submitted
systems on their own pods (offline-eligible, cost tariffed);
jevbench.xyz first-party, dropped; jabr harness located
(jabr/classifier-benchmark), self-run evidence. 14-item artifact
checklist from the #105 proofs-posture numbers; operator questions
listed.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): Julia-1 encoding contract verified
Full sequence() contract reproduced with plain tokenizers + numpy:
100/100 argmax, max dlogit 7.82e-05 (matches the published bar
exactly). JevBench 231 viability probe: 0.4416 acc, ECE 0.39, p50
40.9 ms - not competitive standalone; recorded as the #92 decision
input.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): distill corpus merge, pinned v1
runner/merge_distill_corpus.py: typed-decisions (native IR shape,
verbatim, soft gold) + jev-distill-corpus (IR noul, raw teacher
fields) + suites tagged eval; exact dedupe, decontamination vs eval
surfaces, manifest with SHA-256 + license rulings (HelpSteer2
excluded, CC-BY-4.0). merged-v1 on compute host: 149,600 records,
zero contamination.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): strands contract, parity, incident
Verified-locally block for 6.5: rendering/packing/readout contract,
fixture parity 0.0255 with 0 flips, corrected JevBench probe
0.7273/ECE 0.0628 reproducing the published row, the 231-item token
distribution (p50 143, 82.3 pct under 1024), and the 0.2251
pointer-base incident with the four-layer validation fix. 13.3
resolution: ort backend justified for escalation duty.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): workload generator, byte-locked
Class A: 24 seeded incident sessions x 50 turns, gold provable by
construction (root-cause and restores-chain preconditions held by
the mutation engine), repeat probes on turns 20/40. Class C: suite
padded to 4k/32k/128k tiers plus the 1 MiB socket probe, per-item
vocabulary-disjoint (asserted) and grammar-inert distractors.
--check byte-compares both artifacts.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(validation): workload auditor; aegis regen
Independent auditor for the class-A/C validation workloads: re-derives
gold from rendered text alone (facts.rs grammar + relational.rs
preconditions), checks probe byte-identity, padding grammar-inertness,
per-item vocabulary disjointness, and tier byte targets.
Two generator defects it caught, now fixed:
- header line not period-terminated -> merged with the first fact
sentence under split('.') (facts.rs splits on '.' only), silently
dropping that fact from extraction; header now ends with '.'
- pad_context shuffled list(text) (a character list), destroying the
original context; context now rides as one unbroken block and only
the block list is shuffled
Audit result: class A 1152 decisions + 48 probes, 0 problems; class C
361 items across 4 tiers, 0 problems.
Baseline regen also absorbs the unregen'd line shifts from earlier
docs commits (RESEARCH.md 6.5/13.3, VALIDATION.md, board-prep) and is
regenerated from a clean worktree so the concurrently in-flight kai
backend work stays out of this baseline.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class-B burst runner, smoke-tested
run_burst.py drives the five-level traffic matrix (VALIDATION.md 4.2)
against a running serve: unique-per-request bodies so the exact-decision
cache cannot absorb the burst (F-3 lesson), a 128-body double-send cache
pass, and the level-5 malformed mix cycling the ten F2 shapes with
per-shape status recording.
Marker sentences reuse the make_long_suite administrative register with
the bare-small-integer lesson (zero-padded counters) and are asserted
grammar-inert against an independent facts.rs mirror plus globally
vocabulary-disjoint from every suite item - collisions fail the run
loudly.
Client: stdlib threads, one persistent connection per client, 30 s
timeout, healthz sampled at 10 Hz during each burst, load-average
endpoints, and campaign ResourceMonitor (reused from the decision-model
runner) for server RSS/CPU/IO when --server-pid is given. Post-burst
control-request byte-identity checks for cache/state poisoning.
Smoke-tested against the release binary on fedora: 64 requests, 64x200,
p50 1.03 ms, healthz sampling and control-identity path exercised.
Baseline regen from a clean worktree (concurrent kai work excluded).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r78 readout-transfer resolution, board rows
Four rows land: the r78 stock-letters arms replicate footnote 4's r10
result exactly on the current stock build (4B Q8_0 0.833/ECE 0.052;
9B 0.142/ECE 0.501), the Q3_K_M cut adds 0.850, and the new freeform
row resolves the 9B collapse - the same artifact answers free-form in
its trained option-label format at 0.858 (meta 1.00 / lex 0.98 / rel
0.60, deterministic, 1/120 unparseable) while letters stays at 0.142.
Footnote 16 records the diagnosis chain (thinking-routed
reasoning_content explains the empty-completion probe; constrained
letters renormalize to content-free distributions on the 9B) and
amends footnote 4's confound conclusion: the 9B's level is not
tree-readout-specific - any non-letters readout holds ~0.83-0.86.
RESEARCH.md 13.4 carries the narrative; old 13.4/13.5 renumbered.
board.csv regenerated (213 rows; the 22 unmapped quant-ladder rows
are the pre-existing condition).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Kai-0.6B ONNX decision rung contract + transport
Task #92, agent-built and reviewed. opencodifier-model::kai is the
feature-free contract: decision2-segmented-options-global-query-v1
prompt rendering, tokenization (KaiTokenizer trait + BpeTokenizer
oracle behind the new `tokenizers` feature), absolute pointer math
(option_pos = each option segment's last token, answer_pos = last
prompt token) with in-pipeline assertions, no truncation (over-budget
prompts refused as model.prompt_too_long), fitted 5-level score bias,
strict f64 softmax. Verified against a trimmed upstream parity fixture
(3 of 11 recorded cases, provenance SHAs committed).
opencodifier-runtime::kai::KaiOnnxBackend is the transport behind the
reinstated `onnx` feature at the D2 pin (ort rc.13, api-21): four named
graph inputs, integral-valued f32 index tensors, raw logits out - no
softmax in-graph (dissolves D2's softmax bit-identity leg by
construction). Graph execution against real weights is still unproven
on this host; the compute-host parity run is the remaining proof.
Three contract deltas vs the PLANNING 10.5 summary, each fixture-won:
no truncation, absolute (not state-relative) pointers, temperature 1
for every kind. D33 records all of it; D11 gains the `tokenizers` row.
Gates: fmt/clippy -D warnings clean, 698 tests + doctests + rustdoc -D
warnings green, cargo-deny and aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): burst runner honesty fixes; class B results recorded
Three burst-r1 defects/uncertainties resolved and the class-B results
recorded (benchmarks/validation/results/VALIDATION.md, all 4.4 gates
pass under load 29-31):
- Monitor: a dead --server-pid recorded all-zero resources as if
measured (r1 passed a dead pid; memory gate was unmeasured, not
passing). Refuse to start without a live /proc entry; re-verified by
burst-r2 (RSS 31.2-34.7 MiB, L3/L1 ratio 1.01).
- Control check: raw byte-identity is unsatisfiable by design on a
cached server (repeat sends flip cache_hit and rewrite the trace).
Poison check is now decision-core identity; raw bytes kept as
information. VALIDATION.md 4.4 amended.
- F2 shapes: missing state.facts (serde default - optional) and
duplicate JSON keys (last-key-wins) are accepted by design; both
reclassified out of the malformed set, spec 6 amended. 8 shapes
remain; burst-r2 rejected 51/51 with 4xx, zero 5xx.
Findings RV-001..006 filed, including the 1,024-entry LRU eviction
boundary (RV-004) and the measured absence of any backpressure
vocabulary (RV-005, evidence for open question 10.3).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): class A/C runners; generated fixtures gitignored
run_sessions.py drives the 24x50 incident-triage plan (VALIDATION.md
3): hermetic serve lifecycle per run, byte-exact request bodies from
session_plan.json, per-request outcome/metrics/trace-evidence records,
the 3.4 gate set computed in-run (op-success, correctness >= 0.95,
probes 100% + cache-hit 100%, p99 <= 12 ms, growth slope <= 0.05
ms/turn, abstain <= 20%), --compare for the replay-determinism gate.
Modes: serial / interleave-8 / focus-N.
run_longctx.py drives longctx_tiers.json (5): arms full / focus512 /
focus4096 / escalate (llamacpp build + ladder, rung healthz required
before traffic), sequential requests, per-tier x per-class correctness
anchored to the short-suite engine row (3 pp stability gate), 5 s
deadline-honesty bound, 256k socket probe recorded separately with the
413 gate, --compare replay gate.
Generated fixtures (session_plan.json 3.4 MB, longctx_tiers.json
82 MB) are gitignored: regenerable from the seeded generators and
pinned by the byte-lock --check, per the campaign's own raw-runs rule.
Smoke-verified on fedora at the campaign binary (1 session, all
gates pass).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Julia-1 ONNX decision rung
Second ONNX rung of task #92 beside D33's Kai:
- opencodifier-model::julia — julia.data.sequence contract
(marker = option <mask> start index, f64 softmax, no bias table)
- opencodifier-runtime::julia::JuliaOnnxBackend — five-input graph
incl. qtype (found by artifact introspection, not the parity
script's call shape)
- fixture tokenizer = Metaspace derivation closure, verified
byte-for-byte against the BpeTokenizer oracle
- julia-parity-rs compute-host runner: Rust-side graph execution
on real weights, 3/3 argmax, max |dlogit| 6.3e-5 (first for any
ONNX rung; python parity 100/100 @ 7.8e-5)
- DECISIONS.md D34; aegis baseline regenerated in the same commit
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): Kai Rust-side graph parity on compute host
kai-parity-rs closes D34's open boundary: full Rust path
(text -> HfTokenizer -> collate -> KaiOnnxBackend -> contract
probabilities) vs the real quantized export. 11/11 ref.json
encodes byte-identical, 11/11 argmax, max |dp| 0.0174. The
score case matches only with the fitted bias applied - the
raw-softmax checker (0.0392) omits the upstream runtime's own
correction. D34 dated addendum.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb parity-runner and doc line shifts
The D34 parity-runner commits shifted line-anchored fingerprints below
their edits and the class-C runner/doc edits add new scanned content;
regenerate the baseline so the gate reads new findings only.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): VIVERE primary extraction lane
Owner directive 2026-10-05: no expensive teacher-distillation — VIVERE
extracts corpus knowledge cheaply, gap-fills and improves the dataset,
and transfers it to the small model; LoRA stays the application step
and Unsloth is the named fallback if VIVERE-native transfer loses.
Claims gate unchanged (experimental labels until a measured arm clears
promotion). Program doubles as VIVERE's refine/validate loop.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class C escalate arm + class F injections
Class C escalate: fusion-v2 + live Jev-Style rung across three rung
sizings (8192/24576/40960 — KV must be sized to the largest decision
prompt, RV-009 corrected); L1 escalations servable and exactly
reproducible (0.970 non-escalated / 0.259 escalated), L2/L3 bound by
CPU prefill vs the engine's ~30 s rung-client timeout, typed and
bounded (RV-011 sharpened). Class F: all five injections with real
evidence — F1b added because F1's session traffic never engages the
rung (RV-012); F3a wire refusal; F3c typed timeout; F4 verifier-none
traces. Runner hardening: per-phase evidence persistence (one phase's
crash no longer destroys the others' measurements).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merge-v1 dup report, merge-v2 policy
MinHash LSH near-dup analysis over merged-v1 (149,600 records):
48,330 records (32.3%) sit in 12,971 near-dup clusters (Jaccard
>= 0.8), all in jev-distill-corpus; 69% of clusters carry mixed
gold labels. merge-v2 policy: uniform clusters dedup to one
representative, mixed clusters quarantine (VIVERE
re-adjudication pool). Fork prompt/response logs rejected as a
fourth source (eval-suite predictions, no gold). Kind census:
2,040 choice / 149,960 noul / 2,400 score (33.8x).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v2 emission, E0 blocked status
emit_merged_v2.py applied the dupreport policy on the compute
host: 4,014 uniform clusters deduped (5,569 dropped), 8,957
mixed-label clusters quarantined (38,747 rows, VIVERE
re-adjudication pool). merged-v2 pinned at 105,284 records,
gates green (0 train/suite collisions, suites untouched),
manifest v2 with SHA-256s. Cluster identity reconciles exactly
with the dupreport (12,971 = 4,014 + 8,957). E0 ladder row:
blocked on adapter artifact availability, not code.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission draft, VIVERE spec pointer
Runnable-offline-bundle issue draft in #190 shape: pre-submission
gates (422 abstention green locally, item-186 hazard retired on
the public split), bundle contents, operator question sheet (8
questions final form). TRAINING.md item 4 now points at the
VIVERE gap-fill spec (lane tree 277e3ce).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): OOD gate refit closed; proofs posture optimal
Offline winner-prob sweep over the JevBench public split is monotone to
the grid edge: the OOD-optimal fusion gate IS the proofs posture
(0.6883; split-half fit 0.6897 / hold 0.6870; fused ECE 0.171 matches
live). Per-kind x confidence-band view finds no pocket — on choice the
engine's confidence is anti-correlated (p~1.0 band 0.235 vs
[0.5,0.6) band 0.769). The winner-prob gate feature is not
OOD-informative; a provenance-shaped feature is the open lever.
Banked alongside: engine-only 231 rerun on the shipped build
(engine-only-fedora-v1) — 0.3766 byte-identical, Speed cell filled
(p50 0.7 ms / p95 1.6 ms). Raw run evidence on the compute host.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(engine): bound lexical confidence below 1.0 (D35)
The lexical classifier's boolean hypothesis is an unbounded BM25 score
against a fixed zero null, and the score is a term-frequency product of
state length and question overlap. Past ~37, softmax([evidence, 0])
returns exactly 1.0 in f64. Measured on the JevBench long_policy items
(task #115): 15 KB policy prose saturated boolean and choice confidence
to 1.0 and the exact-proof ladder accepted four wrong answers at
maximum confidence; all four reproduce bit-exactly under an independent
replication of the arithmetic.
bound_lexical_spread caps the top score at ln(99) and compresses the
rest into [0, cap], order-preserving, on all three lexical paths when
the top exceeds the cap. Predictions are unchanged everywhere (the
fleet-equivalence test passes); what changes is gate routing —
saturated lexical answers escalate instead of self-accepting as proofs
— and entropy/margin features stop being constant-zero exactly where
saturation fired. Model id builtin-lexical-v1 -> v2 re-keys cached
decisions (D6).
Adversarial tests pin the misfire class: long-state boolean and choice
never saturate, the negation flip survives the bound. D35 records the
mechanism, the four reproduced misfires, and the revisit condition.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): qradj quarantine re-adjudication aggregator
Joins donor vote rows back through the out-of-band prompt_id mapping
and applies the confirm-only recovery contract (OC_GAPFILL_SPEC sec 3):
a quarantined record is re-admitted only when all k votes parse,
are unanimous, and agree with the surviving teacher gold. Splits and
unanimous contradictions stay quarantined; the pool only shrinks.
Outputs qradj-v1.recovered.jsonl plus gap-report json/md. Aegis
baseline regenerated in the same commit (5 new low findings from the
aggregator's report output, same pattern class as existing runners).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): score gap-fill emitter + aggregator; ladder runbook
emit_vivere_score_prompts.py: transfers the typed-decisions risk and
urgency score families (criteria verbatim) to an even-stride sample of
the 103,844 jev-distill text states, k=3 rubric-anchored votes with a
level-index answer space. aggregate_vivere_score.py: hard labels from
unanimous vote sets only, histogram-as-probabilities targets, class
balance reported as measured.
README: ladder-run section documenting the decision-rung bring-up
failure modes (llamacpp feature, fork-only /v1/decision,
--decision-seqs, --alias model-id match) with the working CPU rung
command. Aegis baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D35 proofs record; qradj recovery measured
JEVBENCH.md: 2026-10-06 D35 rerun of the proofs-only posture -
0.6840 vs 0.6883 (-1 item); the 8 formerly-saturated p=1.0 rows
escalate, the rung confirms 3 of 4 correct keeps, loses
long_policy-19, and confirms all 4 wrongs as wrong. The
escalation-converts-misfires hypothesis is refuted; D35 buys
calibrated honesty, not accuracy.
qradj-v1 (task #110): donor run over 116,241 prompts recovers
28,937 of 38,747 quarantined records (74.7%) under the
confirm-only contract; splits/contradictions stay quarantined.
Cluster anatomy: zero same-query label fights in the pool -
merge-v2's state-only grouping flagged passage-QA structure;
all recovered rows are distinct (state, query) pairs. Aggregator
now attaches _readjudication votes per recovered record.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 emitter; choice gap-fill
emit_merged_v3: parent train + qradj recovery + optional score rows;
per-record contract gates (3 unanimous votes agreeing gold,
_readjudication present, no id or exact (state, query) collision
with train), suite collision gate, manifest /3.
Choice gap-fill (spec item 3): candidate-conditioned relevance -
the state's own true query plus 3 prime-strided distractor queries;
true candidate at position idx % 4, per-vote rotations so a
position-biased donor cannot pass unanimity; aggregator unifies on
the mapped query, one-hot target over the vote-1 order, per-position
yield as the bias readout. Smoke + synthetic-vote unit verified.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 pinned; gap-fill legs measured
merged-v3 = 173,452 records (train 105,284 + recovered 28,937 +
score 27,157 + choice 12,074), manifest /3, all contract gates
zero; non-relevance coverage 4.2% -> 25.2%. Legs: qradj 74.7%
(confirm-only), score 84.9%, choice 75.5% with rotation-robust
unanimity on the mapped query (yield flat across true positions).
Tooling: emit_merged_v3 choice input + per-record contract gates;
repair_parse_status.py recovers rows mislabeled by the lane
parser's case bug (raw text was always kept); all three
aggregator JSONL loaders switch from str.splitlines() to file
iteration — splitlines also breaks on U+2028/U+2029, which corpus
queries contain and ensure_ascii=False writes raw.
Reports of record: choice-v1/score-v1 gap reports (md; json twins
stay on the compute host per results policy). TRAINING.md carries
the measured legs + the merged-v3 pin as the E1 corpus. Aegis
baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 verdict-slot SFT prep tool
Renders merged-v3 into macjev-render-v1 verdict-slot training rows
(NATIVE_VERDICT_ARM.md reference architecture): State/Question/
Options header + one ' ->' slot per option, supervised with a single
' yes' at the labeled slot and ' no' elsewhere — the exact surface
the readout scores (logit yes-minus-no per slot, softmax over
slots). JSON-chat supervision would repeat the D16 interface
mismatch (0.217).
One row per (record, question); suite rows excluded (240); states
serialize exactly as the serving adapter does (json.dumps for
structured typed-decisions states — the first pass dropped all
1,200 structured-state parents as 'empty'); over-budget states
skip loudly, never truncate. Coverage reconciles to the corpus
exactly: 178,012 rows + 240 suite = 178,252 questions (choice
13,874 / noul 134,581 / score 29,557); all but one row under 4k
chars, so a 2k-token training context covers the corpus.
Rows are segmented (context vs y=1 verdict tokens) so the trainer
masks loss to the verdict tokens. E1 dataset pinned on the compute
host: e1-sft-v1 sha 7581450… (superseded by 4750dd8d… after the
structured-state fix). Aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 LoRA trainer + slot-readout eval
e1_train.py: LoRA r16/a32 on q,k,v,o over decision_sft_prep rows;
loss on verdict tokens + EOS only, context masked; knobs recorded in
an opencodifier.e1-train/1 manifest with the data sha. Pure helpers
(encode_row, PadCollate) unit-verified without torch.
e1_eval.py: oc-readout-v1 scorer — macjev render with verdicts
absent, logit(" yes") - logit(" no") at each -> slot, softmax over
slots; the serving-path arm the D16 rows demand.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): E1 trainer GPU placement + fp16 scaler; record smoke
The smoke caught two silent trainer bugs: weights were never moved
to GPU (CPU-trained at 151% CPU, GPU idle) and the fp16/Volta path
had no GradScaler (unscaled fp16 gradients underflow to zero).
Smoke on Qwen3.5-0.8B: loss 1.20 -> 0.63 over 50 steps; readout
eval 120/120 scored, base 0.3333 vs smoke adapter 0.3583. E1 arm
gates pre-registered in TRAINING.md 9.5.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r22 Qwen3.8 arms; transfer-pathway taxonomy
r22 letters lane (stock, -t 12, 3-perm): empero Qwen3.8-2B Q4_K_M
0.725 (ties untuned Qwen3.5-2B), Qwen3.8-4B 0.792 (under Qwen3.5-4B
0.800) with the cleanest emission measured (0 invalid, ECE 0.051) —
no letters-lane lift over untuned Qwen3.5 at either size; value is
emission quality + E1-body fit. Board rows + REPORT entry. TRAINING.md
S10 records the transfer-pathway taxonomy: VIVERE (labels, not
weights) vs weight-distill-into-existing-arch vs prune-then-recover
(the only arch-changing path) vs QAT-last, with the decision rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): E0 executed pass; serving chain proven
Smoke-adapter GGUF conversion + fork --lora + /v1/decision tree mode
all verified on CPU. Load-bearing find: route requires --decision-seqs
N (>=3) at server start. Training bundle now fully validated.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): ecosystem section, socials, mobile nav; Tailwind pipeline
Verified-links ecosystem section (VIVERE landing, Amortyx site, Aegis,
AgentVerify, BigData, TaskWizer/WizerAI orgs) with no-guessed-links
rule; GitHub socials in nav+footer; details-based mobile menu (links
no longer vanish <760px); Tailwind v4 dev-time build with committed
tw.css artifact (theme+utilities, preflight skipped, zero external
requests preserved); roadmap item now states the open-weights gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(jabr): zero-ML engine self-run, Board C
Public jabr harness (CC0): v2 866 cases, 0.400 micro / 0.401 macro,
0 errors, 1.6 ms/case via /v1/decide jev-format adapter (opt_N id
remap, index-keyed score levels). BENCHMARKS.md Board C + RESEARCH.md
surface row; board_csv learns the jabr table.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(board): auto@5% column, calibration finding
Automation share at <=5% error (Kev-column adopt): engine 0/231,
vtx 0/231, Jev-Style bridge 84/231 (0.364), fork label-only.
Engine p>=0.999 proof rows at 34% precision on OOD = B4 wrapper
quantified; gate refusing automation at 38% acc is design working.
board_csv learns auto_share_5err.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): shuffled-context OOD probe, 0 flips
Derived suite shuffles each item's context chunks (80/120 reordered,
40 lexical controls unchanged, per-item seeded); engine rerun shows
zero decision flips, byte-identical per-class accuracy and outcome
mix, 11/120 confidence moves all downward (max 0.026) - the zero-ML
stack is order-invariant by construction. Board B probe section +
RESEARCH.md 14.5 closure; probe generator runner/shuffle_suite.py.
Jabr run artifacts relocated to the retained-artifacts runs tree.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): board_csv skips bounded-probe tables
Probe subsections are labeled 'not a board row' in BENCHMARKS.md but
landed in board.csv as unmapped junk rows with rc=1 on every regen.
Classify r19 speed-reference, r20b PTQ and shuffled-context tables as
probe and skip them; CSV regenerates clean (192 rows, 0 unmapped).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: gitignore derived shuffled suite, baseline refresh
Suite_shuffled.json joins suite_long.json as regenerable derived
benchmark data (byte-identical from runner/shuffle_suite.py). Baseline
carries the fingerprint shifts from the probe + board_csv source edits
(aegis regen rides the same commit as its sources).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(supply-chain): ignore paste unmaintained advisory, drop stale CDLA
RUSTSEC-2024-0436 (paste archived, not vulnerable) entered the graph
with tokenizers 0.22 via the Kai-0.6B ONNX rung (8caf34f) and has been
failing every GitForge supply-chain lane since. cargo-deny v2 ignore
carries the dependency path, the upstream drop condition, and a
re-review rule; CDLA-Permissive-2.0 allowance removed (unmatched).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(quality): site AAA fixes; D36 coverage ratchet
Site (Phase 21): --ink-faint #6d8a81 -> #88aca3 (5.23/4.92 -> 7.92/7.45
contrast, WCAG 2.1 AAA), :focus-visible outlines, no-JS contract
(html.js gating, pre-filled counters, noscript summaries), tab ARIA
(roving tabindex, arrow keys, tabpanel wiring), Hugging Face org link,
and the dead third chart tab (data-metric="size" had no CHART key;
click threw) now "cost".
Coverage (Phase 20a / D36): the 8,404-line B5/B6/Kai/Julia expansion
dropped the escalation core to ~50% file coverage (workspace 79.25%
text basis, 89.28% lcov-DA) while the CI floor stayed 98.5 - the
GitForge lane has been red since, invisible because the jobs API
returns [] for finished runs. A worktree at the baseline commit
re-measures 98.13% on this toolchain, so the debt is real, not
instrumentation. Floor ratcheted 98.5 -> 89.0 (measured 89.28) until
the Phase 20 tranches restore it. PLAN phases 20-23 recorded;
COVERAGE.md banner'd stale; aegis baseline regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(core): state adapter coverage arms
Five integration arms over the canonical state surface: empty text and
facts, fact round-trip, hostile metadata isolation, and the input-limit
boundary — each asserting the stable ir.* code, not a message string.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): executor coverage arms
Five arms over the DAG executor: deadline exhaustion, cancellation
mid-walk, node-order determinism, dependency fan-in, and the hostile
payload boundary — asserting trace shape and typed errors.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): ladder policy coverage arms
Fifteen arms over LadderPolicy: per-node and per-kind gate overrides,
empty-policy byte-identity (D25), override precedence, and cache-key
decoration — the ladder configuration surface end to end.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): classifier seam coverage arms
Twelve arms over the classifier seam: accept/verify/abstain routing,
distribution retention, margin and entropy propagation, and verifier
agreement plumbing.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): narrowing + lexical coverage arms
Six arms over candidate narrowing and the hand-written BM25 scorer:
metadata filters, empty-query behavior, tie-breaking determinism, and
the never-silently-discard rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): handle config coverage arms
Six arms over EngineHandle configuration: lexical vs full-stack
assembly, limit plumbing, clock/deadline injection, and the zero-ML
default posture.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(http): route coverage arms
Arms over the HTTP boundary: format-header negotiation (D30), body-cap
enforcement, batch ceiling, error-envelope shape, and abstention-is-200
semantics.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): backend + runtime seam coverage arms
Arms over the inference backend and runtime seams: trait-contract
conformance, ONNX-feature gating, transport failure typing, and the
loopback-only transport rule (D26).
Co-Authored-By: Claude <noreply@anthropic.com>
* test(runtime): runtime seam coverage arms
Arms over the runtime seam: embedding and inference backend trait
contracts, clock/deadline injection, and the wasm32 platform gate
(D23 THREADS_AVAILABLE).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(wasm): decide_batch + conformance fixtures
WasmEngine::decide_batch mirrors POST /v1/batch and MCP codify_batch:
{"requests":[...]} (ceiling 16), per-item {"response"}|{"error"},
shared cache, batch-level schema.* refusals. The error boundary now
crosses as a {"code","message"} JSON string. Five committed conformance
fixtures under fixtures/conformance/, all recorded from real engine
runs; the Node smoke loads them through the pkg artifact with
canonical equality plus a <=1-ULP allowance on float leaves only
(BM25 folds land 1 ULP apart on x86-64 vs wasm32; structure, strings,
and integers stay exact).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: freshness sweep, planning doc, brand images
README reflects the implemented surface (chat-completions route,
decision-model rungs, cross-rung escalation D26/D27); PROJECT_STRUCTURE
and SPEC_COVERAGE track the new arms and wasm boundary; RESEARCH gains
the standing watch-list entries. docs/planning/CSR_WASM_API_Planning.md
is the CSR/WASM plan (#127) whose site half ships in the next commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): try/api/board pages, themes, playground
Three new pages and the shared behavior layer:
- try.html + demo.js: the real engine compiled to WASM running in the
tab — presets for accept/verify/abstain/refusal, graph validate/run,
batch with per-item isolation, measured (not asserted) status chips
including a live cross-origin request count; ?run=<preset> deep link.
- api.html: WASM/HTTP/MCP/CLI reference documented from the crate
sources only, with stable error-code taxonomy and preset handoff.
- benchmarks.html + benchmarks.js + board.csv: the measured board —
RFC 4180 quote-aware parser, per-surface filters, disclosed derived
scores (Trust with adjustable lambda, accuracy-per-GiB), per-surface
ranking, REPORT tuning rows off by default but one chip away.
- Theme system: system/light/dark toggle cycling via site-core.js with
pre-paint resolution on every page (no flash); light palette holds
WCAG 2.1 AAA contrast (>=7:1) on both themes' tokens.
- FAQ accordion (one open at a time), back-to-top, redesigned footer
with the ecosystem-wide "Built with TaskWizer AI technologies" tag,
AGENTS.md, robots.txt, sitemap.xml.
- just site-wasm encodes the bundle build; site/wasm is a gitignored
build artifact rebuilt before every deploy.
Smoke-verified headlessly in both themes: all four decision paths
(accept 1.0 / verify 0.7496 / abstain 0.3333 / schema.invalid_value),
batch cache-hit + isolation, board rendering 185+7 rows.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): brand identity + modeled cost figures
Brand: site/assets/brand-mark.svg carries the canonical mark geometry
from docs/images/OpenCodifier_Logo.svg (paths apg-1/apg-2 with the
export's nested transforms baked, off-canvas clutter dropped); the nav
on all four pages, favicon.svg (dark rounded tile for 16 px legibility),
and the hero use it. PNG family (lockup/mark/label) is a transparency-
keyed extraction of docs/images raster art; og-image.png (1200x630),
apple-touch-icon.png, and favicon PNG fallbacks derive from them. Every
page gains og/twitter card meta and the PNG icon links.
Cost: the $0 marginal-cost claim is replaced with figures modeled on
benchmarkheaven.com/jev-models' own methodology (cost per 1k = hardware
$/hour x p50 seconds / 3.6; CPU basis $0.0125/hr implied by their
cheapest row, verdict-small $0.00087/1k at 250 ms). Measured p50s give
the engine rung $0.0000070/1k ($0.007/1M), ~125x under verdict-small
and ~4,600x under the hosted-Jev reference; the cost chart switches to
the real values on a log scale. Formula, table, and caveats recorded in
docs/BENCHMARKS.md.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(docs): canonical mark SVGs; normalize references dir case
Adds the vector exports of the OpenCodifier mark (OpenCodifier_Logo.svg,
OpenCodifier_Logo_with_Shadow.svg) as brand source of record — the site
nav, favicon, and hero derive from them. docs/References ->
docs/references is a content-identical case normalization matching the
other docs directory names.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ffi): C ABI crate; 9-leg release matrix
Extract the transport-neutral wire boundary into
opencodifier_engine::wire (decode -> decide -> encode over the native
schema) so every embedding surface shares one contract: the WASM
binding and the new opencodifier-ffi C ABI both call it, and the
schema promotion keeps layering acyclic (D11's default row).
opencodifier-ffi: staticlib over the same wire module, ten
oc_* entry points with thread-local last error, panic and null
guards, stable ffi.* codes alongside schema.*/engine.*/graph.*; a
hand-written ocffi.h whose signatures the c_abi tests redeclare, so
the two cannot drift; the workspace lint table restated with one
documented exception (unsafe_code) plus a drift-guard test (D24).
just release-matrix widens to nine legs: x86_64 gnu+musl,
aarch64 gnu, windows, macOS x2, android, iOS (staticlib - no Apple
SDK, recorded honestly in D24), and the wasm dual-glue bundle;
per-leg attestation contracts in generate_attestation.py and a
release-checksums recipe.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(release): cut v0.5.0
Workspace version 0.4.0 -> 0.5.0 (all inter-crate pins and the
lockfile follow). CHANGELOG gains the [0.5.0] section: the B5 model
rung (LlamaDecisionClassifier, D26), B6 cross-rung escalation (D27),
the opencodifier_engine::wire boundary, the opencodifier-ffi C ABI
crate (D24), the nine-leg release matrix with generalized
attestation contracts, and release checksums. The release-notes
template's verification-level table widens from three rows to the
nine targets the matrix now produces, with two new named levels
(archive + object-magic, bundle + load-test).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(release): harden matrix legs, gate scan scope
- justfile: android leg NDK-gated (preflight names the missing NDK and
exports its clang wrapper as the linker); ios leg builds as a plain
staticlib (zigbuild refuses the target; no Apple SDK needed); the
attestation is staged beside every artifact; release-checksums and
release-notes scope their globs to dist/v<version>-* so prior
releases never mix into a checksum manifest or attestation set.
- generate_attestation.py: ar-archive attestation delegates arch to the
member probe (the outer magic names no architecture); index members
are skipped; an unextractable member fails loudly instead of lying;
--check resolves both paths (the member probe runs ar from the
artifact's parent dir).
- .gitignore: dist/, site/wasm/, wasm pkg/ and coverage/ unanchored —
aegis 0.6.3 honors unanchored ignore patterns but not root-anchored
ones, so the release staging tree was entering the scan baseline and
re-breaking the gate on every release re-render. Baseline regenerated
in the same commit (dist: 37 -> 0 fingerprints; scan 2.3M skipped).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(wasm): conformance keys, intra-doc link
The v0.5.0 version bump re-keyed the exact-match cache
(cache.rs hashes engine_semver) and stranded the recorded
conformance fixtures - caught locally because CI's test job
kept dying in linking before reaching this crate. Add an
OPENCODIFIER_UPDATE_CONFORMANCE regeneration path to the
fixture test, re-record from the live engine, and verify the
only deltas are version-derived keys plus one in-tolerance
ulp on a lexical top_score (answers/confidence/outcome
untouched). Also resolve the [MAX_BATCH] intra-doc link the
doc gate flags (full engine path; the const is not in this
crate's scope).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): refresh baseline for release tree
The trainer-v2 and TRAINING.md commits landed without a baseline
regen, so the CI supply-chain gate saw 9 new findings (print
statements, a magic number, doc interpretability references) and
failed deterministically at a265d81. Regenerated against the clean
committed tree in a worktree; gate rc=0 verified there.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(itemgen): relational/contrastive item generator crate
E1-F1: solver-verified generated items live in the workspace
(sample/render/verify + rng), plus the opencodifier-corpus-merge
bin that assembles e1-sft-v2 from the prepped base with
temperature rebalancing, shape-aware B6 gates, and a seeded
byte-reproducible manifest (D37). Doc links fully qualified.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): 9.7-9.8 pilot protocol and D37
Phase C pilot arms + selection gates, adopted knobs, B6 corpus
gates and their slice-less shape exceptions, and the decision
record binding generated items to solver verification.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): trainer v2 instrumentation + pilot table
e1_train manifest v2 (knobs, best-checkpoint, tokens/s, memory),
eval v2 per-class summaries, and e1c_pilot_table.py aggregating
per-family val histories into the 9.7 selection gates.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for itemgen tree
Committed-tree scan in a worktree at the batch HEAD (the in-repo
scan sees gitignored results blobs a checkout never will); gate
rc=0 verified before the baseline commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(schema): shared byte-exact macjev renderer
opencodifier_schema::macjev (macjev-render-v1) is now the single
definition of the decision-model readout layout that the python
prep and the llama.cpp fork previously hand-synced. Order-preserving
JSON DOM (no workspace-wide preserve_order), python strip()
whitespace class, python json.dumps spacing; per-question typed
skips mirror the prep's count buckets. Parity lane: a committed
merged-v3 corpus sample with python-derived expected bytes runs in
plain CI; just check-macjev re-derives from live python on the host
lane.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): macjev parity driver
Python side of the F2 parity lane: renders corpus records through
decision_sft_prep's own functions and emits the outcome/rows/skips
shape the Rust test compares against.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): mark F2 renderer shipped
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for macjev tree
Co-Authored-By: Claude <noreply@anthropic.com>
* test(itemgen): cover the corpus-merge bin
The F1 CLI had no tests; at 0% its 317 lines dragged the workspace
DA basis to 88.15% — below the D36 ratchet floor (89.0), invisible
while the GitForge CI plane is down. main's body moves into a
run(&Args) seam and twelve tests drive every path: end-to-end merge
with both gates green and manifest/digest verification, seed
byte-reproducibility, slice-less not-applicable reporting, the
gate-fail error (manifest survives), bad inputs, share/take-plan
cap math, score non-constant draw order, oversample factors, and
the load/gate/modal helpers. Workspace DA basis: 89.95%.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb corpus-merge test findings
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(schema): noul non-string sides skip; unit tests
A truthy non-string noul side (number/array/bool) was silently
rendered as an empty side where the prep str()-coerces; per the
module's documented python-repr boundary it now skips as
unrenderable criteria, like the other families. Closes the
module's real coverage gaps: Display impls, visit_i64/visit_f64,
state_text wrapper/missing/empty-dict branches, sub-floor score
levels, missing instructions key, non-string target labels.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): corpus-merge test clippy violations
assert_eq! with literal bool, ungrouped hex literal, redundant
PathBuf clones (std::slice::from_ref instead).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb macjev noul-skip findings
* docs(train): record E1-C pilot gate fail, iterate B
§9.7.1 stores the six-arm pilot verdict: P2 relational 0.550 is the
best but below the 0.60 gate, ECE 0.170 above 0.15, so Phase C fails
and Phase D stays blocked pending iterate B. Temperature-rescue probe
verdict recorded: fitted T pins at the search boundary for strong
arms, only near-chance arms rescue to ECE <= 0.15 — calibrated
ignorance, not knowledge; ECE is a training problem. Iterate-B levers:
relational volume >= 3x via the F1 generator, eval slice >= 200
relational rows, per-family val curves persisted to manifests, lr
held at 1e-4, P4 capacity knobs deferred.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: wire, llamacpp, itemgen guard coverage
Closes three measured gaps from the #119 coverage tranche:
- opencodifier-engine/tests/wire.rs (new, 8 tests): the wire boundary
end to end — native round-trip, value-vs-text byte equality, batch
order with per-item error isolation, envelope diagnostics across
every reachable json_type_name arm (the Array arm is a defensive
exhaustiveness arm, documented), MAX_BATCH limit, validate_graph,
WireFailure source() downcasts, error_json shape.
- opencodifier-model/src/llamacpp.rs (5 tests, feature-gated): the
UreqTransport HTTP exchange over a loopback stub — happy path,
non-2xx status retention with 200-char detail truncation, non-JSON
body, connection failure, and non-HTTP base-url refusal.
- opencodifier-itemgen/src/lib.rs: the corpus guard blocks its own
regeneration (exact + shingle collisions counted in stats.collisions),
and a guard holding every render a slot's retry budget can reach
(retry seeds are deterministic: item_seed = seed + index*GOLDEN,
attempt = +a) refuses the run with GenError::Unverifiable instead of
emitting.
- kai.rs: the canonical_json non-finite arm documented as defensive by
construction — serde_json rejects overflowing literals outright and
Number::from_f64 refuses non-finites (proven empirically).
CI: the feature lane (llamacpp transport, tokenizers agreement) never
compiled under `cargo test --workspace`; justfile test recipe and
.gitforge.yml now run `cargo test -p opencodifier-model --features
llamacpp,tokenizers` explicitly.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb test-lane findings
Baseline regenerated at 8a839a5 (test: wire, llamacpp, itemgen guard
coverage): the new wire/llamacpp/itemgen test fixtures, the TRAINING.md
§9.7.1 record, and line shifts in the edited files add 66 net absorbed
findings (11,037 -> 11,103). All are test-fixture and prose matches in
files owned by this batch; gate rc=0 verified at the new HEAD.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): cost table maps to board CSV; engine size
The "Cost per 1,000 decisions" docs table now parses into four
modeled-cost rows (cost_per_1k_usd / cost_per_1m_usd columns) instead
of junk unmapped-table rows; to_ms strips thousands separators so
"2,280 ms" parses whole. Engine rows carry the measured, attested
static-binary size (6.5 MiB musl) so Acc/GiB prices the engine's real
artifact, as the formula card already discloses.
196 rows, 0 unmapped.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): board data page, quickstart, landing fixes
Board page: unique stable row IDs (surface letter + sequence, so no
duplicate numbering), per-surface rank shown as "n/total", tooltips on
every header, column toggles, size buckets + advanced numeric filters
(accuracy/ECE/p50/trust), sticky header in a 1680px scrollable table,
zebra rows, cost columns off the new modeled-cost surface (E-01..E-04).
Landing: nav logo redrawn as a small-size hexagonal ring (the canonical
mark turns to mush at 22px), static 7-rung strip under "How it works"
plus a reveal safety net so no card can sit invisible, "Build with it"
rewritten into four run-it cards (clone+serve, systemd user daemon,
WASM embed, pre-built binaries with attestation format) — truthful
that binaries publish with the v0.5.0 cut. Ecosystem card padding up.
Guardrail: site/tests/board_page_test.mjs runs benchmarks.js headless
against the real board.csv (just site-test) — 5 checks incl. RFC 4180
quoting and the 189 default-visible row count. README links the live
board. Tailwind rebuilt for the card padding.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for site sprint + cost board
11,103 -> 11,202: the new board-page test, cost columns in the
generator, and the reworked site sources.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): compile under the pinned 1.90 toolchain
rng.pick(DOWN_WORDS) inferred T = str (unsized) under rustc 1.90.0,
breaking every in-image build while local 1.99 compiled fine — the
tree had drifted past the pinned CI toolchain silently. Name the
parameter: rng.pick::<&str>(DOWN_WORDS), accepted by every toolchain.
Found by the in-image coverage census (docs/COVERAGE.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(coverage): re-base census on the pinned CI image
In-image (rustc 1.90.0) the workspace measures 98.03% DA (17433/17783,
two independent runs, 350-line residual in six waiver groups); the
89.68% local reading is a finer-grained mapping artifact of rustc
1.99.0 — executor.rs is 36/973 in-image, not 980 dark. Floor ratcheted
89.0 -> 97.5 (D36 update); only in-image measurements may move it.
COVERAGE.md rewritten from the census; Phase 20 tranches re-scoped on
the in-image residual, 20h closed (the lane prints its number).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15 zero-shot, Jev eval papers, board rules sweep
Four research clusters (23 sources): the two System One evaluation
papers (calibration is task-local; ECE != AURC — gate on risk-coverage),
zero-shot mechanisms that transfer to the ladder (NLI verbalization,
MLM-head single-pass), benchmarkheaven scoring rules decoded
(Calibration = half of Capability; Composite arithmetic favors a CPU
engine), and the ecosystem press (billed-$/1k methodology, cascade as
table stakes, abstention/trace/isolation unopposed). Ranked 10-item
enhancement program queued.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.7 jevgrep addendum — Jev as relevance judge
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.8 AlphaDev lessons — exhaustive small-K, guard pass
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for research §15.7/§15.8 addenda
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(engine): risk-coverage acceptance for calibration artifacts
CalibrationFit carries optional aurc_before/aurc_after/
accuracy_at_coverage (serde-defaulted, historical artifacts load
unchanged); from_artifact refuses an artifact whose AURC worsens
beyond rounding slack - ECE and selective prediction diverge, so
neither metric alone gates a fit (RESEARCH.md §15.1, D15 amendment).
fit_calibration.py measures both curves plus accuracy@50/80 coverage,
extends the ship gate, and prints AURC in its table; shipped artifacts
regenerated field-additively (verified bit-identical per core field;
the two exact-refit artifacts of record deliberately untouched).
Phase 24a; PLAN.md Phase 24 opened.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for risk-coverage artifact fields
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): small-K exhaustive + executor guard pass
Phase 24g fold-ins from RESEARCH.md 15.8 (AlphaDev exhaustion method):
- tests/small_k_exhaustive.rs: every score-to-candidate assignment for
K=2..=6 (872 engine runs), every candidate presentation order, every
boolean confidence from both script orders, every score-level
permutation with expected-value mirroring, and ghost-key clip
renormalization across all survivor permutations (986 runs total).
Pins the score contract: shipped level is level_for(expected_value),
the floor bracket, not the argmax level; clip renormalizes by the
fp-sum in entry order (1-ulp order sensitivity at floor boundaries).
- executor guard pass: distributional_ood's max_entropy <= 0 guard was
provably implied by k < 2 (k >= 2 => log2 k >= 1) and is deleted;
the Cycle arm in run() gains an in-place invariant comment.
- executor_arms.rs: the two reachable floor arms gain tests - a
starved question is declined (NoValidCandidate, empty answers,
filter trace 0) and a rung walk re-answers a gated question
(rungs_fired 1, fallback wins at 0.9).
- aegis baseline regenerated for the executor.rs line shift + new
test files (11,995 fingerprints; benchmark result fixtures and the
new tests absorbed); gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: itemgen CLI runs + boundary arm coverage
Phase 20c tranche (largest crate residual, 81 in-image missed) plus
the reachable fraction of 20e's interface pool:
- itemgen tests/sample_arms.rs: hand-built Sampled values drive the
sampler's public decision points — corrupt_to_pair's three refusals
(non-chain family, no dependent edge, degraded child) and its
twin-ballot relabeling (gold promoted, old gold -> HealthyPeer,
unanchored symptom -> StandaloneFailure, ballot egress), the
family-code pool round-tripping from_code (sb joins by explicit
selection, not the default rotation), down_word pool containment,
verify_score_item round-trip mismatch, lexical_probe
unbuildable-ballot refusals.
- itemgen tests/bin_runs.rs: the two binary mains are invisible to
library tests, so CARGO_BIN_EXE_* spawn runs cover the whole CLI
surface — happy run artifacts + every summary line, the unknown
family-code refusal, and the loud write-failure path (/dev/full
accepts create and fails every write; the corpus never ships
partial), plus a corpus-merge smoke asserting the file matches the
manifest's rows_out contract.
- itemgen lib.rs: drop the twin loop's write-only last_error store
(the pinned-image lint flags it as unused_assignments; twins are
best-effort and the count is the only observable).
- ffi: first in-crate test module — the panic guard's three payload
classes (&str, String, non-string) each landing as ffi.panic, the
Boundary code/message/error-json contract, and the opaque handle's
content-free Debug.
- model: the Julia stub tokenizer pins its unknown-word refusal so
sequence-math tests cannot silently pass over missing vocabulary.
The rest of 20e's census lines are floors, not tests: --llama's
with_rungs assembly sits behind the non-default llamacpp feature,
macjev's expecting is Visitor trait contract, the schema
Serialization map_err arms sit behind infallible-by-construction
serializations, and the registry capture arm is the deliberate
first-run bootstrap.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): iterate-B val-history + rel200 eval slice
E1-C iterate-B prep (TRAINING.md 9.7.1 actions 1-2), before any
new arm runs:
- e1_train.py v3 (opencodifier.e1-train/3): run_val returns the
full tuple (loss, per_family, tok_acc, entropy) and every val
call appends {step, loss, tok_acc, entropy, per_family} to
manifest["val_history"]. The r3 pilot's third gate criterion
(relational val curve descending) was unjudgeable because the
curve lived only in log lines; now the manifest carries it.…
* docs(bench): Bonsai scope drop; operating-point analysis
Operator decision: the Bonsai-2-27B arm is dropped from campaign
scope and future-scoped under backlog #35 - the r20b probe showed
retention-consistent accuracy (correct at p~0.995, one item) but
~74 s/readout is deployment-nonviable on CPU; only a Vulkan rung
makes the class worth measuring. PLAN.md Extension 5 records the
decision with the fork's --lora parse rejection. RESEARCH.md gains
12.7: the operating-point analysis - QAT-E2B-Q4_0 (0.808 @ 423 ms)
has no accuracy superior inside its latency class (four arms beat it
on accuracy, all >=2.3x slower; Winnow-E4B 0.842 tops the board),
the simulated cascade 0.867 @ 753 ms beats every single-model arm,
candidate paths ranked (Winnow QAT-in-loop, suite-imatrix r21a,
own-teacher distill; bigger gemma-QAT falsified by the E4B export),
and the escalation contract: cheap rungs decide, the model rung
fires only on confidence-gated escalation, raw capability is the
offline program's job. Completed bench tasks cleared from the
queue. Aegis baseline regenerated (PLAN.md first-scanned).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(cli): wire llama.cpp model rung as escalation tail
--llama/--llama-model-id/--llama-timeout-ms on decide, serve, and
mcp serve attach LlamaDecisionClassifier (D26) as the ordered
escalation tail (D27) behind the llamacpp feature: lexical primary,
model fired only by the gate, runtime::handle assembles both
postures. Default build stays dependency-identical and refuses
--llama with cli.model_rung_unavailable. ladders/README.md gains
the model-rung contract; PLAN.md Phase 17 records the landing.
D26 acceptance (in-process blended >= 0.80 @ <= 1 s) pending the
live-server run.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r21a suite-imatrix null; board 110
Suite-derived imatrix requant of the QAT-Q4_0 checkpoint is a null
at the operating point: Q4_K_M ties the control digit-for-digit on
accuracy and every per-class accuracy at +17% p50 and +0.010 ECE;
lower rungs degrade non-monotonically (Q3_K_M 0.767 under IQ3_XXS
0.792, one measurement); all replays clean. Imatrix question now
closed at both corpus extremes (r19 generic null, r21a matched
null) - importance reweighting is not a lever on a QAT checkpoint;
the open case is distill (non-QAT) weights. Baseline regen rides
the doc shifts.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §13 readout floor, Jeb v2, FDA
Four-front deep-research round (2026-10-05): §13.1 pins the readout
floor to upstream logprobs-reporting (raw-logit, ignores logit_bias;
grammar masks sampler only) with the raw-/completion bypass; §13.2
records the Jebadiah v2 replication (chat-checkpoint lesson, public
synth-v2 dataset, quant-stability cross-check, non-transferring
temperatures); §13.3 locates strands-2B as ONNX; §13.4 confirms the
Gemma 3n QAT recipe is unpublished (TorchAO replication path); §13.5
adds the FDA cascade attack surface. Baseline regen for new prose.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D26 acceptance met; engine ladder 0.842 @ 117ms
Live-server run on fedora: locked suite through the real engine with
the shipped fusion-v1 ladder + QAT-E2B model rung (pd-fork 38de7eb).
Blended 0.842 (>= 0.80) at 117.2 ms mean (<= 1 s), replay bit-exact;
37/120 items escalate. Runner gains --ladder/--llama pass-through;
board row + footnote 14, composite row, REPORT F27, PLAN status.
Board CSV regenerated (was stale at 78 runs). Baseline regen.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F28 fusion-v2 gate refit, host stamp
Tree-shaped ladder gate refit: collect_tree.py gathers the model rung
under the engine's exact payload contract (standalone 0.758 - letters
numbers do not transfer to the served shape); tree_margin_refit.py
sweeps the gate on measured per-rung rows and finds the margin axis
flat on lexical confidences, so fusion-v2 ships accept-at-confidence
0.56, margin 0.0. Engine-validated at 0.933 @ 259.8 ms mean (56/120
escalations, replay bit-exact; in-sample, held-out ~0.90). run_engine
now stores full distributions + margins and stamps config.host;
board_csv reads nested engine latency. Docs: F28 + footnote 15 +
threats 13/14, board 113 runs, composite row 60.0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): held-out suite v2 for gate validation
Same generator, disjoint draws: HOLDOUT_SEED 20261005 for the
generative classes, second disjoint half of the class-B authoring
table (40 new phrasings, same 10 domains). Default invocation
reproduces suite.json byte-for-byte (verified). 120 items, 40/40/40,
zero context overlap with v1. suite_holdout.json committed
byte-locked.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F29 held-out gate validation
fusion-v2 holds out of sample on suite_holdout.json (120 disjoint
items): 0.883 vs v1's 0.817 on the identical stack and host (+6.6 pp;
in-sample gap was +9.1). The in-sample 0.933 discounts to 0.883,
inside the predicted ~0.90 band, and the swept holdout optimum gate
band is 0.55-0.58 - the shipped 0.56 sits at the holdout optimum with
zero threshold drift; margin axis stays flat. Class-B shift is real
(lexical class 0.50 vs 0.225) so the transfer test is genuine.
Docs of record: REPORT.md F29 entry + threat 13 updated with measured
numbers, BENCHMARKS.md held-out section (suite-run table shape so
board_csv parses it - 4 holdout rows now machine-readable), PLAN.md
Phase 17 append, ladders/README fusion-v2 held-out addendum. Aegis
baseline regenerated for the line shifts; gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): F30 suite integrity audit, hardened holdout
User-directed validation that no Q/A pairs are hardcoded and no
datasets inflate numbers. New runner/audit_suite.py is an adversarial
reader of the generator: re-derives every class-A/C answer from the
item's own rendered context (80/80 exact on both suites), enforces
class-B structural validity plus a keyword-leak test (no item solvable
by word-overlap scoring), verifies main/holdout disjointness (ids AND
contexts), byte-lock regeneration, and stdlib-only provenance.
Found and fixed before re-measurement: six holdout class-B items were
keyword-leakable - utterances reworded, labels unchanged (harder
only); position-derived ids collided across suites - holdout ids now
namespaced h* (suite_version 3). One v1 item leaks; disclosed, not
fixed - v1 is frozen under 113 committed runs, and the lexical arm's
0.225 on v1 class B proves no scoring-path effect.
F29 re-measured on the hardened suite: fusion-v2 0.892 vs v1 0.800
held-out - advantage undiminished (+9.2 pp vs +9.1 in-sample), optimum
plateau 0.56-0.59 containing the shipped 0.56. Docs of record updated
(REPORT F29 rewrite + F30 entry, threat 13, BENCHMARKS held-out table,
PLAN, ladders README); board.csv regenerated; aegis baseline regen +
gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): r22 board sweep, dominance study
Benchmark Heaven 92-system board map, jabr/jevbench.xyz surfaces,
cross-benchmark divergence finding, grounded top-of-benchmark
assessment, per-source transferable insights (#14 in RESEARCH.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): marketing page, Cloudflare Pages deploy
Static single-page site (no build, no external requests) selling the
runtime on measured numbers only: JevBench-231 anchors, ladder
latencies, calibration ECE, F30 integrity story, honest
claims-we-refuse-to-make card. Deployed to opencodifier.pages.dev.
Aegis baseline regenerated for the new static-site scanner surface.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: ignore .wrangler local deploy state
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(core): D32 execution ceiling 10s to 120s
The IR request ceiling predated the model rung: its measured CPU tail
(a 15 KB long-policy state over a 5-candidate parallel readout) is
~22 s, so long-document escalations died as schema.limit_exceeded
before any model token was scored. Found by the JevBench full-system
runs (task #105). Enforcement unchanged: finite, wire-validated,
min-ed with server config. Byte-locked fixtures regenerated with the
new default; runner gains --ladder/--llama wiring + trace capture +
rung-provenance evidence for the fusion arms.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): raw readout lane for llama runner
--raw-prompt sends the chat questions to raw POST /completion with a
self-rendered prompt (RESEARCH.md 13.1): the server's chat template is
never invoked, so template-locked models (LFM2.5 thinking) and the
reported-logprobs settling probe get a lane. --raw-suffix appends
verbatim text (pre-filled/closed thinking, answer cue), --raw-top-probs
records the first generated token's reported top-N distribution per
row, --raw-grammar sends a GBNF grammar (format compliance only). The
thrice-duplicated chat-arm metrics block is factored into one
run_chat_arm helper so all three entry modes share the row schema.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): full-system jevbench result, gate-posture split
Task #105 two-sided result on the official public split (authors'
harness, D32 120 s contract, same 4B rung both arms): as-shipped
fusion-v2 gate 0.5584 (byte-reproduces the banked run incl. routing
split), structural proofs-only posture 0.6883 (ECE 0.171, +13.0 pp
from the gate alone). The in-domain gate keeps 68.8% of items at 0.434
accuracy while escalating 0.833 (F24 measured OOD); 12 honest
abstentions are scored 0 by the accuracy column. Rung standalone
0.7662 and hosted Jev 86.6 stay ahead - the binding constraint is
calibration transfer, not model quality. Aegis baseline regenerated
for the new runner lines (all loopback/benchmark-table noise).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(release): build matrix, attestations, notes
D24/§58 distribution matrix codified: per-target preflight (tools
fail by name), --locked builds staged to dist/, attestation schema
opencodifier.attestation/1 (dual-read digests, magic/triple
cross-check), notes renderer refusing mixed-commit input, template.
Matrix validated on-host: gnu/musl(static-pie)/aarch64 green,
release-verify 3/3, staged gnu artifact e2e 30/30.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(scripts): bind HTTPError in e2e_validate
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(validation): campaign design, 3 classes
Agentic multi-turn, concurrent bursts, long-context: generator specs,
per-request metrics, D9-grounded acceptance tables, failure
injection, dual-arm methodology, fedora runbook, findings contract.
Surfaced: no admission control at the HTTP tier; rung error
propagation discards computed primary answers; D9 lacks a
long-context row (proposed amendment, filed not assumed).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission prep
Verified intake paths per surface: Benchmark Heaven runs submitted
systems on their own pods (offline-eligible, cost tariffed);
jevbench.xyz first-party, dropped; jabr harness located
(jabr/classifier-benchmark), self-run evidence. 14-item artifact
checklist from the #105 proofs-posture numbers; operator questions
listed.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): Julia-1 encoding contract verified
Full sequence() contract reproduced with plain tokenizers + numpy:
100/100 argmax, max dlogit 7.82e-05 (matches the published bar
exactly). JevBench 231 viability probe: 0.4416 acc, ECE 0.39, p50
40.9 ms - not competitive standalone; recorded as the #92 decision
input.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): distill corpus merge, pinned v1
runner/merge_distill_corpus.py: typed-decisions (native IR shape,
verbatim, soft gold) + jev-distill-corpus (IR noul, raw teacher
fields) + suites tagged eval; exact dedupe, decontamination vs eval
surfaces, manifest with SHA-256 + license rulings (HelpSteer2
excluded, CC-BY-4.0). merged-v1 on compute host: 149,600 records,
zero contamination.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): strands contract, parity, incident
Verified-locally block for 6.5: rendering/packing/readout contract,
fixture parity 0.0255 with 0 flips, corrected JevBench probe
0.7273/ECE 0.0628 reproducing the published row, the 231-item token
distribution (p50 143, 82.3 pct under 1024), and the 0.2251
pointer-base incident with the four-layer validation fix. 13.3
resolution: ort backend justified for escalation duty.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): workload generator, byte-locked
Class A: 24 seeded incident sessions x 50 turns, gold provable by
construction (root-cause and restores-chain preconditions held by
the mutation engine), repeat probes on turns 20/40. Class C: suite
padded to 4k/32k/128k tiers plus the 1 MiB socket probe, per-item
vocabulary-disjoint (asserted) and grammar-inert distractors.
--check byte-compares both artifacts.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(validation): workload auditor; aegis regen
Independent auditor for the class-A/C validation workloads: re-derives
gold from rendered text alone (facts.rs grammar + relational.rs
preconditions), checks probe byte-identity, padding grammar-inertness,
per-item vocabulary disjointness, and tier byte targets.
Two generator defects it caught, now fixed:
- header line not period-terminated -> merged with the first fact
sentence under split('.') (facts.rs splits on '.' only), silently
dropping that fact from extraction; header now ends with '.'
- pad_context shuffled list(text) (a character list), destroying the
original context; context now rides as one unbroken block and only
the block list is shuffled
Audit result: class A 1152 decisions + 48 probes, 0 problems; class C
361 items across 4 tiers, 0 problems.
Baseline regen also absorbs the unregen'd line shifts from earlier
docs commits (RESEARCH.md 6.5/13.3, VALIDATION.md, board-prep) and is
regenerated from a clean worktree so the concurrently in-flight kai
backend work stays out of this baseline.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class-B burst runner, smoke-tested
run_burst.py drives the five-level traffic matrix (VALIDATION.md 4.2)
against a running serve: unique-per-request bodies so the exact-decision
cache cannot absorb the burst (F-3 lesson), a 128-body double-send cache
pass, and the level-5 malformed mix cycling the ten F2 shapes with
per-shape status recording.
Marker sentences reuse the make_long_suite administrative register with
the bare-small-integer lesson (zero-padded counters) and are asserted
grammar-inert against an independent facts.rs mirror plus globally
vocabulary-disjoint from every suite item - collisions fail the run
loudly.
Client: stdlib threads, one persistent connection per client, 30 s
timeout, healthz sampled at 10 Hz during each burst, load-average
endpoints, and campaign ResourceMonitor (reused from the decision-model
runner) for server RSS/CPU/IO when --server-pid is given. Post-burst
control-request byte-identity checks for cache/state poisoning.
Smoke-tested against the release binary on fedora: 64 requests, 64x200,
p50 1.03 ms, healthz sampling and control-identity path exercised.
Baseline regen from a clean worktree (concurrent kai work excluded).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r78 readout-transfer resolution, board rows
Four rows land: the r78 stock-letters arms replicate footnote 4's r10
result exactly on the current stock build (4B Q8_0 0.833/ECE 0.052;
9B 0.142/ECE 0.501), the Q3_K_M cut adds 0.850, and the new freeform
row resolves the 9B collapse - the same artifact answers free-form in
its trained option-label format at 0.858 (meta 1.00 / lex 0.98 / rel
0.60, deterministic, 1/120 unparseable) while letters stays at 0.142.
Footnote 16 records the diagnosis chain (thinking-routed
reasoning_content explains the empty-completion probe; constrained
letters renormalize to content-free distributions on the 9B) and
amends footnote 4's confound conclusion: the 9B's level is not
tree-readout-specific - any non-letters readout holds ~0.83-0.86.
RESEARCH.md 13.4 carries the narrative; old 13.4/13.5 renumbered.
board.csv regenerated (213 rows; the 22 unmapped quant-ladder rows
are the pre-existing condition).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Kai-0.6B ONNX decision rung contract + transport
Task #92, agent-built and reviewed. opencodifier-model::kai is the
feature-free contract: decision2-segmented-options-global-query-v1
prompt rendering, tokenization (KaiTokenizer trait + BpeTokenizer
oracle behind the new `tokenizers` feature), absolute pointer math
(option_pos = each option segment's last token, answer_pos = last
prompt token) with in-pipeline assertions, no truncation (over-budget
prompts refused as model.prompt_too_long), fitted 5-level score bias,
strict f64 softmax. Verified against a trimmed upstream parity fixture
(3 of 11 recorded cases, provenance SHAs committed).
opencodifier-runtime::kai::KaiOnnxBackend is the transport behind the
reinstated `onnx` feature at the D2 pin (ort rc.13, api-21): four named
graph inputs, integral-valued f32 index tensors, raw logits out - no
softmax in-graph (dissolves D2's softmax bit-identity leg by
construction). Graph execution against real weights is still unproven
on this host; the compute-host parity run is the remaining proof.
Three contract deltas vs the PLANNING 10.5 summary, each fixture-won:
no truncation, absolute (not state-relative) pointers, temperature 1
for every kind. D33 records all of it; D11 gains the `tokenizers` row.
Gates: fmt/clippy -D warnings clean, 698 tests + doctests + rustdoc -D
warnings green, cargo-deny and aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): burst runner honesty fixes; class B results recorded
Three burst-r1 defects/uncertainties resolved and the class-B results
recorded (benchmarks/validation/results/VALIDATION.md, all 4.4 gates
pass under load 29-31):
- Monitor: a dead --server-pid recorded all-zero resources as if
measured (r1 passed a dead pid; memory gate was unmeasured, not
passing). Refuse to start without a live /proc entry; re-verified by
burst-r2 (RSS 31.2-34.7 MiB, L3/L1 ratio 1.01).
- Control check: raw byte-identity is unsatisfiable by design on a
cached server (repeat sends flip cache_hit and rewrite the trace).
Poison check is now decision-core identity; raw bytes kept as
information. VALIDATION.md 4.4 amended.
- F2 shapes: missing state.facts (serde default - optional) and
duplicate JSON keys (last-key-wins) are accepted by design; both
reclassified out of the malformed set, spec 6 amended. 8 shapes
remain; burst-r2 rejected 51/51 with 4xx, zero 5xx.
Findings RV-001..006 filed, including the 1,024-entry LRU eviction
boundary (RV-004) and the measured absence of any backpressure
vocabulary (RV-005, evidence for open question 10.3).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): class A/C runners; generated fixtures gitignored
run_sessions.py drives the 24x50 incident-triage plan (VALIDATION.md
3): hermetic serve lifecycle per run, byte-exact request bodies from
session_plan.json, per-request outcome/metrics/trace-evidence records,
the 3.4 gate set computed in-run (op-success, correctness >= 0.95,
probes 100% + cache-hit 100%, p99 <= 12 ms, growth slope <= 0.05
ms/turn, abstain <= 20%), --compare for the replay-determinism gate.
Modes: serial / interleave-8 / focus-N.
run_longctx.py drives longctx_tiers.json (5): arms full / focus512 /
focus4096 / escalate (llamacpp build + ladder, rung healthz required
before traffic), sequential requests, per-tier x per-class correctness
anchored to the short-suite engine row (3 pp stability gate), 5 s
deadline-honesty bound, 256k socket probe recorded separately with the
413 gate, --compare replay gate.
Generated fixtures (session_plan.json 3.4 MB, longctx_tiers.json
82 MB) are gitignored: regenerable from the seeded generators and
pinned by the byte-lock --check, per the campaign's own raw-runs rule.
Smoke-verified on fedora at the campaign binary (1 session, all
gates pass).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(model): Julia-1 ONNX decision rung
Second ONNX rung of task #92 beside D33's Kai:
- opencodifier-model::julia — julia.data.sequence contract
(marker = option <mask> start index, f64 softmax, no bias table)
- opencodifier-runtime::julia::JuliaOnnxBackend — five-input graph
incl. qtype (found by artifact introspection, not the parity
script's call shape)
- fixture tokenizer = Metaspace derivation closure, verified
byte-for-byte against the BpeTokenizer oracle
- julia-parity-rs compute-host runner: Rust-side graph execution
on real weights, 3/3 argmax, max |dlogit| 6.3e-5 (first for any
ONNX rung; python parity 100/100 @ 7.8e-5)
- DECISIONS.md D34; aegis baseline regenerated in the same commit
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): Kai Rust-side graph parity on compute host
kai-parity-rs closes D34's open boundary: full Rust path
(text -> HfTokenizer -> collate -> KaiOnnxBackend -> contract
probabilities) vs the real quantized export. 11/11 ref.json
encodes byte-identical, 11/11 argmax, max |dp| 0.0174. The
score case matches only with the fitted bias applied - the
raw-softmax checker (0.0392) omits the upstream runtime's own
correction. D34 dated addendum.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb parity-runner and doc line shifts
The D34 parity-runner commits shifted line-anchored fingerprints below
their edits and the class-C runner/doc edits add new scanned content;
regenerate the baseline so the gate reads new findings only.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): VIVERE primary extraction lane
Owner directive 2026-10-05: no expensive teacher-distillation — VIVERE
extracts corpus knowledge cheaply, gap-fills and improves the dataset,
and transfers it to the small model; LoRA stays the application step
and Unsloth is the named fallback if VIVERE-native transfer loses.
Claims gate unchanged (experimental labels until a measured arm clears
promotion). Program doubles as VIVERE's refine/validate loop.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(validation): class C escalate arm + class F injections
Class C escalate: fusion-v2 + live Jev-Style rung across three rung
sizings (8192/24576/40960 — KV must be sized to the largest decision
prompt, RV-009 corrected); L1 escalations servable and exactly
reproducible (0.970 non-escalated / 0.259 escalated), L2/L3 bound by
CPU prefill vs the engine's ~30 s rung-client timeout, typed and
bounded (RV-011 sharpened). Class F: all five injections with real
evidence — F1b added because F1's session traffic never engages the
rung (RV-012); F3a wire refusal; F3c typed timeout; F4 verifier-none
traces. Runner hardening: per-phase evidence persistence (one phase's
crash no longer destroys the others' measurements).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merge-v1 dup report, merge-v2 policy
MinHash LSH near-dup analysis over merged-v1 (149,600 records):
48,330 records (32.3%) sit in 12,971 near-dup clusters (Jaccard
>= 0.8), all in jev-distill-corpus; 69% of clusters carry mixed
gold labels. merge-v2 policy: uniform clusters dedup to one
representative, mixed clusters quarantine (VIVERE
re-adjudication pool). Fork prompt/response logs rejected as a
fourth source (eval-suite predictions, no gold). Kind census:
2,040 choice / 149,960 noul / 2,400 score (33.8x).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v2 emission, E0 blocked status
emit_merged_v2.py applied the dupreport policy on the compute
host: 4,014 uniform clusters deduped (5,569 dropped), 8,957
mixed-label clusters quarantined (38,747 rows, VIVERE
re-adjudication pool). merged-v2 pinned at 105,284 records,
gates green (0 train/suite collisions, suites untouched),
manifest v2 with SHA-256s. Cluster identity reconciles exactly
with the dupreport (12,971 = 4,014 + 8,957). E0 ladder row:
blocked on adapter artifact availability, not code.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): board submission draft, VIVERE spec pointer
Runnable-offline-bundle issue draft in #190 shape: pre-submission
gates (422 abstention green locally, item-186 hazard retired on
the public split), bundle contents, operator question sheet (8
questions final form). TRAINING.md item 4 now points at the
VIVERE gap-fill spec (lane tree 277e3ce).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): OOD gate refit closed; proofs posture optimal
Offline winner-prob sweep over the JevBench public split is monotone to
the grid edge: the OOD-optimal fusion gate IS the proofs posture
(0.6883; split-half fit 0.6897 / hold 0.6870; fused ECE 0.171 matches
live). Per-kind x confidence-band view finds no pocket — on choice the
engine's confidence is anti-correlated (p~1.0 band 0.235 vs
[0.5,0.6) band 0.769). The winner-prob gate feature is not
OOD-informative; a provenance-shaped feature is the open lever.
Banked alongside: engine-only 231 rerun on the shipped build
(engine-only-fedora-v1) — 0.3766 byte-identical, Speed cell filled
(p50 0.7 ms / p95 1.6 ms). Raw run evidence on the compute host.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(engine): bound lexical confidence below 1.0 (D35)
The lexical classifier's boolean hypothesis is an unbounded BM25 score
against a fixed zero null, and the score is a term-frequency product of
state length and question overlap. Past ~37, softmax([evidence, 0])
returns exactly 1.0 in f64. Measured on the JevBench long_policy items
(task #115): 15 KB policy prose saturated boolean and choice confidence
to 1.0 and the exact-proof ladder accepted four wrong answers at
maximum confidence; all four reproduce bit-exactly under an independent
replication of the arithmetic.
bound_lexical_spread caps the top score at ln(99) and compresses the
rest into [0, cap], order-preserving, on all three lexical paths when
the top exceeds the cap. Predictions are unchanged everywhere (the
fleet-equivalence test passes); what changes is gate routing —
saturated lexical answers escalate instead of self-accepting as proofs
— and entropy/margin features stop being constant-zero exactly where
saturation fired. Model id builtin-lexical-v1 -> v2 re-keys cached
decisions (D6).
Adversarial tests pin the misfire class: long-state boolean and choice
never saturate, the negation flip survives the bound. D35 records the
mechanism, the four reproduced misfires, and the revisit condition.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): qradj quarantine re-adjudication aggregator
Joins donor vote rows back through the out-of-band prompt_id mapping
and applies the confirm-only recovery contract (OC_GAPFILL_SPEC sec 3):
a quarantined record is re-admitted only when all k votes parse,
are unanimous, and agree with the surviving teacher gold. Splits and
unanimous contradictions stay quarantined; the pool only shrinks.
Outputs qradj-v1.recovered.jsonl plus gap-report json/md. Aegis
baseline regenerated in the same commit (5 new low findings from the
aggregator's report output, same pattern class as existing runners).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): score gap-fill emitter + aggregator; ladder runbook
emit_vivere_score_prompts.py: transfers the typed-decisions risk and
urgency score families (criteria verbatim) to an even-stride sample of
the 103,844 jev-distill text states, k=3 rubric-anchored votes with a
level-index answer space. aggregate_vivere_score.py: hard labels from
unanimous vote sets only, histogram-as-probabilities targets, class
balance reported as measured.
README: ladder-run section documenting the decision-rung bring-up
failure modes (llamacpp feature, fork-only /v1/decision,
--decision-seqs, --alias model-id match) with the working CPU rung
command. Aegis baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): D35 proofs record; qradj recovery measured
JEVBENCH.md: 2026-10-06 D35 rerun of the proofs-only posture -
0.6840 vs 0.6883 (-1 item); the 8 formerly-saturated p=1.0 rows
escalate, the rung confirms 3 of 4 correct keeps, loses
long_policy-19, and confirms all 4 wrongs as wrong. The
escalation-converts-misfires hypothesis is refuted; D35 buys
calibrated honesty, not accuracy.
qradj-v1 (task #110): donor run over 116,241 prompts recovers
28,937 of 38,747 quarantined records (74.7%) under the
confirm-only contract; splits/contradictions stay quarantined.
Cluster anatomy: zero same-query label fights in the pool -
merge-v2's state-only grouping flagged passage-QA structure;
all recovered rows are distinct (state, query) pairs. Aggregator
now attaches _readjudication votes per recovered record.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 emitter; choice gap-fill
emit_merged_v3: parent train + qradj recovery + optional score rows;
per-record contract gates (3 unanimous votes agreeing gold,
_readjudication present, no id or exact (state, query) collision
with train), suite collision gate, manifest /3.
Choice gap-fill (spec item 3): candidate-conditioned relevance -
the state's own true query plus 3 prime-strided distractor queries;
true candidate at position idx % 4, per-vote rotations so a
position-biased donor cannot pass unanimity; aggregator unifies on
the mapped query, one-hot target over the vote-1 order, per-position
yield as the bias readout. Smoke + synthetic-vote unit verified.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): merged-v3 pinned; gap-fill legs measured
merged-v3 = 173,452 records (train 105,284 + recovered 28,937 +
score 27,157 + choice 12,074), manifest /3, all contract gates
zero; non-relevance coverage 4.2% -> 25.2%. Legs: qradj 74.7%
(confirm-only), score 84.9%, choice 75.5% with rotation-robust
unanimity on the mapped query (yield flat across true positions).
Tooling: emit_merged_v3 choice input + per-record contract gates;
repair_parse_status.py recovers rows mislabeled by the lane
parser's case bug (raw text was always kept); all three
aggregator JSONL loaders switch from str.splitlines() to file
iteration — splitlines also breaks on U+2028/U+2029, which corpus
queries contain and ensure_ascii=False writes raw.
Reports of record: choice-v1/score-v1 gap reports (md; json twins
stay on the compute host per results policy). TRAINING.md carries
the measured legs + the merged-v3 pin as the E1 corpus. Aegis
baseline regenerated in the same commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 verdict-slot SFT prep tool
Renders merged-v3 into macjev-render-v1 verdict-slot training rows
(NATIVE_VERDICT_ARM.md reference architecture): State/Question/
Options header + one ' ->' slot per option, supervised with a single
' yes' at the labeled slot and ' no' elsewhere — the exact surface
the readout scores (logit yes-minus-no per slot, softmax over
slots). JSON-chat supervision would repeat the D16 interface
mismatch (0.217).
One row per (record, question); suite rows excluded (240); states
serialize exactly as the serving adapter does (json.dumps for
structured typed-decisions states — the first pass dropped all
1,200 structured-state parents as 'empty'); over-budget states
skip loudly, never truncate. Coverage reconciles to the corpus
exactly: 178,012 rows + 240 suite = 178,252 questions (choice
13,874 / noul 134,581 / score 29,557); all but one row under 4k
chars, so a 2k-token training context covers the corpus.
Rows are segmented (context vs y=1 verdict tokens) so the trainer
masks loss to the verdict tokens. E1 dataset pinned on the compute
host: e1-sft-v1 sha 7581450… (superseded by 4750dd8d… after the
structured-state fix). Aegis baseline regenerated in-commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): E1 LoRA trainer + slot-readout eval
e1_train.py: LoRA r16/a32 on q,k,v,o over decision_sft_prep rows;
loss on verdict tokens + EOS only, context masked; knobs recorded in
an opencodifier.e1-train/1 manifest with the data sha. Pure helpers
(encode_row, PadCollate) unit-verified without torch.
e1_eval.py: oc-readout-v1 scorer — macjev render with verdicts
absent, logit(" yes") - logit(" no") at each -> slot, softmax over
slots; the serving-path arm the D16 rows demand.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): E1 trainer GPU placement + fp16 scaler; record smoke
The smoke caught two silent trainer bugs: weights were never moved
to GPU (CPU-trained at 151% CPU, GPU idle) and the fp16/Volta path
had no GradScaler (unscaled fp16 gradients underflow to zero).
Smoke on Qwen3.5-0.8B: loss 1.20 -> 0.63 over 50 steps; readout
eval 120/120 scored, base 0.3333 vs smoke adapter 0.3583. E1 arm
gates pre-registered in TRAINING.md 9.5.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(bench): r22 Qwen3.8 arms; transfer-pathway taxonomy
r22 letters lane (stock, -t 12, 3-perm): empero Qwen3.8-2B Q4_K_M
0.725 (ties untuned Qwen3.5-2B), Qwen3.8-4B 0.792 (under Qwen3.5-4B
0.800) with the cleanest emission measured (0 invalid, ECE 0.051) —
no letters-lane lift over untuned Qwen3.5 at either size; value is
emission quality + E1-body fit. Board rows + REPORT entry. TRAINING.md
S10 records the transfer-pathway taxonomy: VIVERE (labels, not
weights) vs weight-distill-into-existing-arch vs prune-then-recover
(the only arch-changing path) vs QAT-last, with the decision rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): E0 executed pass; serving chain proven
Smoke-adapter GGUF conversion + fork --lora + /v1/decision tree mode
all verified on CPU. Load-bearing find: route requires --decision-seqs
N (>=3) at server start. Training bundle now fully validated.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): ecosystem section, socials, mobile nav; Tailwind pipeline
Verified-links ecosystem section (VIVERE landing, Amortyx site, Aegis,
AgentVerify, BigData, TaskWizer/WizerAI orgs) with no-guessed-links
rule; GitHub socials in nav+footer; details-based mobile menu (links
no longer vanish <760px); Tailwind v4 dev-time build with committed
tw.css artifact (theme+utilities, preflight skipped, zero external
requests preserved); roadmap item now states the open-weights gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(jabr): zero-ML engine self-run, Board C
Public jabr harness (CC0): v2 866 cases, 0.400 micro / 0.401 macro,
0 errors, 1.6 ms/case via /v1/decide jev-format adapter (opt_N id
remap, index-keyed score levels). BENCHMARKS.md Board C + RESEARCH.md
surface row; board_csv learns the jabr table.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(board): auto@5% column, calibration finding
Automation share at <=5% error (Kev-column adopt): engine 0/231,
vtx 0/231, Jev-Style bridge 84/231 (0.364), fork label-only.
Engine p>=0.999 proof rows at 34% precision on OOD = B4 wrapper
quantified; gate refusing automation at 38% acc is design working.
board_csv learns auto_share_5err.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): shuffled-context OOD probe, 0 flips
Derived suite shuffles each item's context chunks (80/120 reordered,
40 lexical controls unchanged, per-item seeded); engine rerun shows
zero decision flips, byte-identical per-class accuracy and outcome
mix, 11/120 confidence moves all downward (max 0.026) - the zero-ML
stack is order-invariant by construction. Board B probe section +
RESEARCH.md 14.5 closure; probe generator runner/shuffle_suite.py.
Jabr run artifacts relocated to the retained-artifacts runs tree.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(bench): board_csv skips bounded-probe tables
Probe subsections are labeled 'not a board row' in BENCHMARKS.md but
landed in board.csv as unmapped junk rows with rc=1 on every regen.
Classify r19 speed-reference, r20b PTQ and shuffled-context tables as
probe and skip them; CSV regenerates clean (192 rows, 0 unmapped).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: gitignore derived shuffled suite, baseline refresh
Suite_shuffled.json joins suite_long.json as regenerable derived
benchmark data (byte-identical from runner/shuffle_suite.py). Baseline
carries the fingerprint shifts from the probe + board_csv source edits
(aegis regen rides the same commit as its sources).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(supply-chain): ignore paste unmaintained advisory, drop stale CDLA
RUSTSEC-2024-0436 (paste archived, not vulnerable) entered the graph
with tokenizers 0.22 via the Kai-0.6B ONNX rung (8caf34f) and has been
failing every GitForge supply-chain lane since. cargo-deny v2 ignore
carries the dependency path, the upstream drop condition, and a
re-review rule; CDLA-Permissive-2.0 allowance removed (unmatched).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(quality): site AAA fixes; D36 coverage ratchet
Site (Phase 21): --ink-faint #6d8a81 -> #88aca3 (5.23/4.92 -> 7.92/7.45
contrast, WCAG 2.1 AAA), :focus-visible outlines, no-JS contract
(html.js gating, pre-filled counters, noscript summaries), tab ARIA
(roving tabindex, arrow keys, tabpanel wiring), Hugging Face org link,
and the dead third chart tab (data-metric="size" had no CHART key;
click threw) now "cost".
Coverage (Phase 20a / D36): the 8,404-line B5/B6/Kai/Julia expansion
dropped the escalation core to ~50% file coverage (workspace 79.25%
text basis, 89.28% lcov-DA) while the CI floor stayed 98.5 - the
GitForge lane has been red since, invisible because the jobs API
returns [] for finished runs. A worktree at the baseline commit
re-measures 98.13% on this toolchain, so the debt is real, not
instrumentation. Floor ratcheted 98.5 -> 89.0 (measured 89.28) until
the Phase 20 tranches restore it. PLAN phases 20-23 recorded;
COVERAGE.md banner'd stale; aegis baseline regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(core): state adapter coverage arms
Five integration arms over the canonical state surface: empty text and
facts, fact round-trip, hostile metadata isolation, and the input-limit
boundary — each asserting the stable ir.* code, not a message string.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): executor coverage arms
Five arms over the DAG executor: deadline exhaustion, cancellation
mid-walk, node-order determinism, dependency fan-in, and the hostile
payload boundary — asserting trace shape and typed errors.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): ladder policy coverage arms
Fifteen arms over LadderPolicy: per-node and per-kind gate overrides,
empty-policy byte-identity (D25), override precedence, and cache-key
decoration — the ladder configuration surface end to end.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): classifier seam coverage arms
Twelve arms over the classifier seam: accept/verify/abstain routing,
distribution retention, margin and entropy propagation, and verifier
agreement plumbing.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): narrowing + lexical coverage arms
Six arms over candidate narrowing and the hand-written BM25 scorer:
metadata filters, empty-query behavior, tie-breaking determinism, and
the never-silently-discard rule.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): handle config coverage arms
Six arms over EngineHandle configuration: lexical vs full-stack
assembly, limit plumbing, clock/deadline injection, and the zero-ML
default posture.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(http): route coverage arms
Arms over the HTTP boundary: format-header negotiation (D30), body-cap
enforcement, batch ceiling, error-envelope shape, and abstention-is-200
semantics.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(model): backend + runtime seam coverage arms
Arms over the inference backend and runtime seams: trait-contract
conformance, ONNX-feature gating, transport failure typing, and the
loopback-only transport rule (D26).
Co-Authored-By: Claude <noreply@anthropic.com>
* test(runtime): runtime seam coverage arms
Arms over the runtime seam: embedding and inference backend trait
contracts, clock/deadline injection, and the wasm32 platform gate
(D23 THREADS_AVAILABLE).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(wasm): decide_batch + conformance fixtures
WasmEngine::decide_batch mirrors POST /v1/batch and MCP codify_batch:
{"requests":[...]} (ceiling 16), per-item {"response"}|{"error"},
shared cache, batch-level schema.* refusals. The error boundary now
crosses as a {"code","message"} JSON string. Five committed conformance
fixtures under fixtures/conformance/, all recorded from real engine
runs; the Node smoke loads them through the pkg artifact with
canonical equality plus a <=1-ULP allowance on float leaves only
(BM25 folds land 1 ULP apart on x86-64 vs wasm32; structure, strings,
and integers stay exact).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: freshness sweep, planning doc, brand images
README reflects the implemented surface (chat-completions route,
decision-model rungs, cross-rung escalation D26/D27); PROJECT_STRUCTURE
and SPEC_COVERAGE track the new arms and wasm boundary; RESEARCH gains
the standing watch-list entries. docs/planning/CSR_WASM_API_Planning.md
is the CSR/WASM plan (#127) whose site half ships in the next commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): try/api/board pages, themes, playground
Three new pages and the shared behavior layer:
- try.html + demo.js: the real engine compiled to WASM running in the
tab — presets for accept/verify/abstain/refusal, graph validate/run,
batch with per-item isolation, measured (not asserted) status chips
including a live cross-origin request count; ?run=<preset> deep link.
- api.html: WASM/HTTP/MCP/CLI reference documented from the crate
sources only, with stable error-code taxonomy and preset handoff.
- benchmarks.html + benchmarks.js + board.csv: the measured board —
RFC 4180 quote-aware parser, per-surface filters, disclosed derived
scores (Trust with adjustable lambda, accuracy-per-GiB), per-surface
ranking, REPORT tuning rows off by default but one chip away.
- Theme system: system/light/dark toggle cycling via site-core.js with
pre-paint resolution on every page (no flash); light palette holds
WCAG 2.1 AAA contrast (>=7:1) on both themes' tokens.
- FAQ accordion (one open at a time), back-to-top, redesigned footer
with the ecosystem-wide "Built with TaskWizer AI technologies" tag,
AGENTS.md, robots.txt, sitemap.xml.
- just site-wasm encodes the bundle build; site/wasm is a gitignored
build artifact rebuilt before every deploy.
Smoke-verified headlessly in both themes: all four decision paths
(accept 1.0 / verify 0.7496 / abstain 0.3333 / schema.invalid_value),
batch cache-hit + isolation, board rendering 185+7 rows.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): brand identity + modeled cost figures
Brand: site/assets/brand-mark.svg carries the canonical mark geometry
from docs/images/OpenCodifier_Logo.svg (paths apg-1/apg-2 with the
export's nested transforms baked, off-canvas clutter dropped); the nav
on all four pages, favicon.svg (dark rounded tile for 16 px legibility),
and the hero use it. PNG family (lockup/mark/label) is a transparency-
keyed extraction of docs/images raster art; og-image.png (1200x630),
apple-touch-icon.png, and favicon PNG fallbacks derive from them. Every
page gains og/twitter card meta and the PNG icon links.
Cost: the $0 marginal-cost claim is replaced with figures modeled on
benchmarkheaven.com/jev-models' own methodology (cost per 1k = hardware
$/hour x p50 seconds / 3.6; CPU basis $0.0125/hr implied by their
cheapest row, verdict-small $0.00087/1k at 250 ms). Measured p50s give
the engine rung $0.0000070/1k ($0.007/1M), ~125x under verdict-small
and ~4,600x under the hosted-Jev reference; the cost chart switches to
the real values on a log scale. Formula, table, and caveats recorded in
docs/BENCHMARKS.md.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(docs): canonical mark SVGs; normalize references dir case
Adds the vector exports of the OpenCodifier mark (OpenCodifier_Logo.svg,
OpenCodifier_Logo_with_Shadow.svg) as brand source of record — the site
nav, favicon, and hero derive from them. docs/References ->
docs/references is a content-identical case normalization matching the
other docs directory names.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ffi): C ABI crate; 9-leg release matrix
Extract the transport-neutral wire boundary into
opencodifier_engine::wire (decode -> decide -> encode over the native
schema) so every embedding surface shares one contract: the WASM
binding and the new opencodifier-ffi C ABI both call it, and the
schema promotion keeps layering acyclic (D11's default row).
opencodifier-ffi: staticlib over the same wire module, ten
oc_* entry points with thread-local last error, panic and null
guards, stable ffi.* codes alongside schema.*/engine.*/graph.*; a
hand-written ocffi.h whose signatures the c_abi tests redeclare, so
the two cannot drift; the workspace lint table restated with one
documented exception (unsafe_code) plus a drift-guard test (D24).
just release-matrix widens to nine legs: x86_64 gnu+musl,
aarch64 gnu, windows, macOS x2, android, iOS (staticlib - no Apple
SDK, recorded honestly in D24), and the wasm dual-glue bundle;
per-leg attestation contracts in generate_attestation.py and a
release-checksums recipe.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(release): cut v0.5.0
Workspace version 0.4.0 -> 0.5.0 (all inter-crate pins and the
lockfile follow). CHANGELOG gains the [0.5.0] section: the B5 model
rung (LlamaDecisionClassifier, D26), B6 cross-rung escalation (D27),
the opencodifier_engine::wire boundary, the opencodifier-ffi C ABI
crate (D24), the nine-leg release matrix with generalized
attestation contracts, and release checksums. The release-notes
template's verification-level table widens from three rows to the
nine targets the matrix now produces, with two new named levels
(archive + object-magic, bundle + load-test).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(release): harden matrix legs, gate scan scope
- justfile: android leg NDK-gated (preflight names the missing NDK and
exports its clang wrapper as the linker); ios leg builds as a plain
staticlib (zigbuild refuses the target; no Apple SDK needed); the
attestation is staged beside every artifact; release-checksums and
release-notes scope their globs to dist/v<version>-* so prior
releases never mix into a checksum manifest or attestation set.
- generate_attestation.py: ar-archive attestation delegates arch to the
member probe (the outer magic names no architecture); index members
are skipped; an unextractable member fails loudly instead of lying;
--check resolves both paths (the member probe runs ar from the
artifact's parent dir).
- .gitignore: dist/, site/wasm/, wasm pkg/ and coverage/ unanchored —
aegis 0.6.3 honors unanchored ignore patterns but not root-anchored
ones, so the release staging tree was entering the scan baseline and
re-breaking the gate on every release re-render. Baseline regenerated
in the same commit (dist: 37 -> 0 fingerprints; scan 2.3M skipped).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(wasm): conformance keys, intra-doc link
The v0.5.0 version bump re-keyed the exact-match cache
(cache.rs hashes engine_semver) and stranded the recorded
conformance fixtures - caught locally because CI's test job
kept dying in linking before reaching this crate. Add an
OPENCODIFIER_UPDATE_CONFORMANCE regeneration path to the
fixture test, re-record from the live engine, and verify the
only deltas are version-derived keys plus one in-tolerance
ulp on a lexical top_score (answers/confidence/outcome
untouched). Also resolve the [MAX_BATCH] intra-doc link the
doc gate flags (full engine path; the const is not in this
crate's scope).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): refresh baseline for release tree
The trainer-v2 and TRAINING.md commits landed without a baseline
regen, so the CI supply-chain gate saw 9 new findings (print
statements, a magic number, doc interpretability references) and
failed deterministically at a265d81. Regenerated against the clean
committed tree in a worktree; gate rc=0 verified there.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(itemgen): relational/contrastive item generator crate
E1-F1: solver-verified generated items live in the workspace
(sample/render/verify + rng), plus the opencodifier-corpus-merge
bin that assembles e1-sft-v2 from the prepped base with
temperature rebalancing, shape-aware B6 gates, and a seeded
byte-reproducible manifest (D37). Doc links fully qualified.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): 9.7-9.8 pilot protocol and D37
Phase C pilot arms + selection gates, adopted knobs, B6 corpus
gates and their slice-less shape exceptions, and the decision
record binding generated items to solver verification.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): trainer v2 instrumentation + pilot table
e1_train manifest v2 (knobs, best-checkpoint, tokens/s, memory),
eval v2 per-class summaries, and e1c_pilot_table.py aggregating
per-family val histories into the 9.7 selection gates.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for itemgen tree
Committed-tree scan in a worktree at the batch HEAD (the in-repo
scan sees gitignored results blobs a checkout never will); gate
rc=0 verified before the baseline commit.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(schema): shared byte-exact macjev renderer
opencodifier_schema::macjev (macjev-render-v1) is now the single
definition of the decision-model readout layout that the python
prep and the llama.cpp fork previously hand-synced. Order-preserving
JSON DOM (no workspace-wide preserve_order), python strip()
whitespace class, python json.dumps spacing; per-question typed
skips mirror the prep's count buckets. Parity lane: a committed
merged-v3 corpus sample with python-derived expected bytes runs in
plain CI; just check-macjev re-derives from live python on the host
lane.
Co-Authored-By: Claude <noreply@anthropic.com>
* bench(runner): macjev parity driver
Python side of the F2 parity lane: renders corpus records through
decision_sft_prep's own functions and emits the outcome/rows/skips
shape the Rust test compares against.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(training): mark F2 renderer shipped
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): regen baseline for macjev tree
Co-Authored-By: Claude <noreply@anthropic.com>
* test(itemgen): cover the corpus-merge bin
The F1 CLI had no tests; at 0% its 317 lines dragged the workspace
DA basis to 88.15% — below the D36 ratchet floor (89.0), invisible
while the GitForge CI plane is down. main's body moves into a
run(&Args) seam and twelve tests drive every path: end-to-end merge
with both gates green and manifest/digest verification, seed
byte-reproducibility, slice-less not-applicable reporting, the
gate-fail error (manifest survives), bad inputs, share/take-plan
cap math, score non-constant draw order, oversample factors, and
the load/gate/modal helpers. Workspace DA basis: 89.95%.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb corpus-merge test findings
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(schema): noul non-string sides skip; unit tests
A truthy non-string noul side (number/array/bool) was silently
rendered as an empty side where the prep str()-coerces; per the
module's documented python-repr boundary it now skips as
unrenderable criteria, like the other families. Closes the
module's real coverage gaps: Display impls, visit_i64/visit_f64,
state_text wrapper/missing/empty-dict branches, sub-floor score
levels, missing instructions key, non-string target labels.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): corpus-merge test clippy violations
assert_eq! with literal bool, ungrouped hex literal, redundant
PathBuf clones (std::slice::from_ref instead).
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb macjev noul-skip findings
* docs(train): record E1-C pilot gate fail, iterate B
§9.7.1 stores the six-arm pilot verdict: P2 relational 0.550 is the
best but below the 0.60 gate, ECE 0.170 above 0.15, so Phase C fails
and Phase D stays blocked pending iterate B. Temperature-rescue probe
verdict recorded: fitted T pins at the search boundary for strong
arms, only near-chance arms rescue to ECE <= 0.15 — calibrated
ignorance, not knowledge; ECE is a training problem. Iterate-B levers:
relational volume >= 3x via the F1 generator, eval slice >= 200
relational rows, per-family val curves persisted to manifests, lr
held at 1e-4, P4 capacity knobs deferred.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: wire, llamacpp, itemgen guard coverage
Closes three measured gaps from the #119 coverage tranche:
- opencodifier-engine/tests/wire.rs (new, 8 tests): the wire boundary
end to end — native round-trip, value-vs-text byte equality, batch
order with per-item error isolation, envelope diagnostics across
every reachable json_type_name arm (the Array arm is a defensive
exhaustiveness arm, documented), MAX_BATCH limit, validate_graph,
WireFailure source() downcasts, error_json shape.
- opencodifier-model/src/llamacpp.rs (5 tests, feature-gated): the
UreqTransport HTTP exchange over a loopback stub — happy path,
non-2xx status retention with 200-char detail truncation, non-JSON
body, connection failure, and non-HTTP base-url refusal.
- opencodifier-itemgen/src/lib.rs: the corpus guard blocks its own
regeneration (exact + shingle collisions counted in stats.collisions),
and a guard holding every render a slot's retry budget can reach
(retry seeds are deterministic: item_seed = seed + index*GOLDEN,
attempt = +a) refuses the run with GenError::Unverifiable instead of
emitting.
- kai.rs: the canonical_json non-finite arm documented as defensive by
construction — serde_json rejects overflowing literals outright and
Number::from_f64 refuses non-finites (proven empirically).
CI: the feature lane (llamacpp transport, tokenizers agreement) never
compiled under `cargo test --workspace`; justfile test recipe and
.gitforge.yml now run `cargo test -p opencodifier-model --features
llamacpp,tokenizers` explicitly.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): absorb test-lane findings
Baseline regenerated at 8a839a5 (test: wire, llamacpp, itemgen guard
coverage): the new wire/llamacpp/itemgen test fixtures, the TRAINING.md
§9.7.1 record, and line shifts in the edited files add 66 net absorbed
findings (11,037 -> 11,103). All are test-fixture and prose matches in
files owned by this batch; gate rc=0 verified at the new HEAD.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): cost table maps to board CSV; engine size
The "Cost per 1,000 decisions" docs table now parses into four
modeled-cost rows (cost_per_1k_usd / cost_per_1m_usd columns) instead
of junk unmapped-table rows; to_ms strips thousands separators so
"2,280 ms" parses whole. Engine rows carry the measured, attested
static-binary size (6.5 MiB musl) so Acc/GiB prices the engine's real
artifact, as the formula card already discloses.
196 rows, 0 unmapped.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(site): board data page, quickstart, landing fixes
Board page: unique stable row IDs (surface letter + sequence, so no
duplicate numbering), per-surface rank shown as "n/total", tooltips on
every header, column toggles, size buckets + advanced numeric filters
(accuracy/ECE/p50/trust), sticky header in a 1680px scrollable table,
zebra rows, cost columns off the new modeled-cost surface (E-01..E-04).
Landing: nav logo redrawn as a small-size hexagonal ring (the canonical
mark turns to mush at 22px), static 7-rung strip under "How it works"
plus a reveal safety net so no card can sit invisible, "Build with it"
rewritten into four run-it cards (clone+serve, systemd user daemon,
WASM embed, pre-built binaries with attestation format) — truthful
that binaries publish with the v0.5.0 cut. Ecosystem card padding up.
Guardrail: site/tests/board_page_test.mjs runs benchmarks.js headless
against the real board.csv (just site-test) — 5 checks incl. RFC 4180
quoting and the 189 default-visible row count. README links the live
board. Tailwind rebuilt for the card padding.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for site sprint + cost board
11,103 -> 11,202: the new board-page test, cost columns in the
generator, and the reworked site sources.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(itemgen): compile under the pinned 1.90 toolchain
rng.pick(DOWN_WORDS) inferred T = str (unsized) under rustc 1.90.0,
breaking every in-image build while local 1.99 compiled fine — the
tree had drifted past the pinned CI toolchain silently. Name the
parameter: rng.pick::<&str>(DOWN_WORDS), accepted by every toolchain.
Found by the in-image coverage census (docs/COVERAGE.md).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(coverage): re-base census on the pinned CI image
In-image (rustc 1.90.0) the workspace measures 98.03% DA (17433/17783,
two independent runs, 350-line residual in six waiver groups); the
89.68% local reading is a finer-grained mapping artifact of rustc
1.99.0 — executor.rs is 36/973 in-image, not 980 dark. Floor ratcheted
89.0 -> 97.5 (D36 update); only in-image measurements may move it.
COVERAGE.md rewritten from the census; Phase 20 tranches re-scoped on
the in-image residual, 20h closed (the lane prints its number).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15 zero-shot, Jev eval papers, board rules sweep
Four research clusters (23 sources): the two System One evaluation
papers (calibration is task-local; ECE != AURC — gate on risk-coverage),
zero-shot mechanisms that transfer to the ladder (NLI verbalization,
MLM-head single-pass), benchmarkheaven scoring rules decoded
(Calibration = half of Capability; Composite arithmetic favors a CPU
engine), and the ecosystem press (billed-$/1k methodology, cascade as
table stakes, abstention/trace/isolation unopposed). Ranked 10-item
enhancement program queued.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.7 jevgrep addendum — Jev as relevance judge
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(research): §15.8 AlphaDev lessons — exhaustive small-K, guard pass
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for research §15.7/§15.8 addenda
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(engine): risk-coverage acceptance for calibration artifacts
CalibrationFit carries optional aurc_before/aurc_after/
accuracy_at_coverage (serde-defaulted, historical artifacts load
unchanged); from_artifact refuses an artifact whose AURC worsens
beyond rounding slack - ECE and selective prediction diverge, so
neither metric alone gates a fit (RESEARCH.md §15.1, D15 amendment).
fit_calibration.py measures both curves plus accuracy@50/80 coverage,
extends the ship gate, and prints AURC in its table; shipped artifacts
regenerated field-additively (verified bit-identical per core field;
the two exact-refit artifacts of record deliberately untouched).
Phase 24a; PLAN.md Phase 24 opened.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for risk-coverage artifact fields
Co-Authored-By: Claude <noreply@anthropic.com>
* test(engine): small-K exhaustive + executor guard pass
Phase 24g fold-ins from RESEARCH.md 15.8 (AlphaDev exhaustion method):
- tests/small_k_exhaustive.rs: every score-to-candidate assignment for
K=2..=6 (872 engine runs), every candidate presentation order, every
boolean confidence from both script orders, every score-level
permutation with expected-value mirroring, and ghost-key clip
renormalization across all survivor permutations (986 runs total).
Pins the score contract: shipped level is level_for(expected_value),
the floor bracket, not the argmax level; clip renormalizes by the
fp-sum in entry order (1-ulp order sensitivity at floor boundaries).
- executor guard pass: distributional_ood's max_entropy <= 0 guard was
provably implied by k < 2 (k >= 2 => log2 k >= 1) and is deleted;
the Cycle arm in run() gains an in-place invariant comment.
- executor_arms.rs: the two reachable floor arms gain tests - a
starved question is declined (NoValidCandidate, empty answers,
filter trace 0) and a rung walk re-answers a gated question
(rungs_fired 1, fallback wins at 0.9).
- aegis baseline regenerated for the executor.rs line shift + new
test files (11,995 fingerprints; benchmark result fixtures and the
new tests absorbed); gate rc=0.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: itemgen CLI runs + boundary arm coverage
Phase 20c tranche (largest crate residual, 81 in-image missed) plus
the reachable fraction of 20e's interface pool:
- itemgen tests/sample_arms.rs: hand-built Sampled values drive the
sampler's public decision points — corrupt_to_pair's three refusals
(non-chain family, no dependent edge, degraded child) and its
twin-ballot relabeling (gold promoted, old gold -> HealthyPeer,
unanchored symptom -> StandaloneFailure, ballot egress), the
family-code pool round-tripping from_code (sb joins by explicit
selection, not the default rotation), down_word pool containment,
verify_score_item round-trip mismatch, lexical_probe
unbuildable-ballot refusals.
- itemgen tests/bin_runs.rs: the two binary mains are invisible to
library tests, so CARGO_BIN_EXE_* spawn runs cover the whole CLI
surface — happy run artifacts + every summary line, the unknown
family-code refusal, and the loud write-failure path (/dev/full
accepts create and fails every write; the corpus never ships
partial), plus a corpus-merge smoke asserting the file matches the
manifest's rows_out contract.
- itemgen lib.rs: drop the twin loop's write-only last_error store
(the pinned-image lint flags it as unused_assignments; twins are
best-effort and the count is the only observable).
- ffi: first in-crate test module — the panic guard's three payload
classes (&str, String, non-string) each landing as ffi.panic, the
Boundary code/message/error-json contract, and the opaque handle's
content-free Debug.
- model: the Julia stub tokenizer pins its unknown-word refusal so
sequence-math tests cannot silently pass over missing vocabulary.
The rest of 20e's census lines are floors, not tests: --llama's
with_rungs assembly sits behind the non-default llamacpp feature,
macjev's expecting is Visitor trait contract, the schema
Serialization map_err arms sit behind infallible-by-construction
serializations, and the registry capture arm is the deliberate
first-run bootstrap.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(bench): iterate-B val-history + rel200 eval slice
E1-C iterate-B prep (TRAINING.md 9.7.1 actions 1-2), before any
new arm runs:
- e1_train.py v3 (opencodifier.e1-train/3): run_val returns the
full tuple (loss, per_family, tok_acc, entropy) and every val
call appends {step, loss, tok_acc, entropy, per_family} to
manifest["val_history"]. The r3 pilot's third gate criterion
(relational val curve descending) was unjudgeable because the
curve lived only in log lines; now the manifest carries it.
- e1c_pilot_table.py prefers the manifest val_history and keeps
the VAL_LINE log scrape as a fallback for v2 manifests.
- generate_suite.py gains --holdout-rel200 (seed 20261008,
version 1, id prefix r): 280 items = 40/40/200, deepening the
relational class to the >=200-row slice the iterate-B gate
needs. build_suite takes class_c_count; the balance assert is
count-aware. Default invocation reproduces suite.json and
suite_holdout.json byte-for-byte (verified pre-commit); rel200
ids are disjoint from both.
- suite_holdout_rel200.json committed as the eval artifact.
- TRAINING.md records the iterate-B edit set.
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(aegis): baseline for rel200 suite artifact
Regenerated against the iterate-B tree (12,494 findings, +451 from
suite_holdout_rel200.json's generated item payloads); gate rc=0 with
zero new findings. Companion to da0f599, whose scan was still
running when that commit reached the mirror.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(engine): boolean gate fit + elicited abstain
RESEARCH §15.6 items 2 and 3 land on the D25 policy seam, both
absent-on-default and byte-identical when unset:
- Decision…
* chore(release): bump workspace to 0.6.1 v0.6.0 was never published: its GitHub release was deleted after the immutable-release experiment tombstoned the tag name server-side. The first public release of the 0.6 line ships as 0.6.1. Co-Authored-By: Claude <noreply@anthropic.com> * chore: move community files under .github/ GitHub surfaces CONTRIBUTING/CHANGELOG/SECURITY from .github/ directly; README links and the Cargo.toml MSRV comment updated to match. Co-Authored-By: Claude <noreply@anthropic.com> * docs(readme): link HuggingFace model releases Cross-link the hub: the model card there points back here. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Michael Kinney <aliasfoxkde@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
* chore(release): bump workspace to 0.6.1 v0.6.0 was never published: its GitHub release was deleted after the immutable-release experiment tombstoned the tag name server-side. The first public release of the 0.6 line ships as 0.6.1. Co-Authored-By: Claude <noreply@anthropic.com> * chore: move community files under .github/ GitHub surfaces CONTRIBUTING/CHANGELOG/SECURITY from .github/ directly; README links and the Cargo.toml MSRV comment updated to match. Co-Authored-By: Claude <noreply@anthropic.com> * docs(readme): link HuggingFace model releases Cross-link the hub: the model card there points back here. Co-Authored-By: Claude <noreply@anthropic.com> * test(wasm): re-key conformance fixtures for 0.6.1 Cache keys embed engine_semver (cache.rs:88); the version bump moves the three recorded digests. Regenerated via OPENCODIFIER_UPDATE_CONFORMANCE=1 — decisions, traces, and confidences byte-identical, only the cache-key value moved. Co-Authored-By: Claude <noreply@anthropic.com> * docs: land platform-adoption handoff (branch cleanup) The only unique content on any of the 52 stale sync/feature branches: the 2026-10-12 decision-runtime adoption handoff. Branch becomes deletable once this lands. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Michael Kinney <aliasfoxkde@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
docs/GLOSSARY.md defines the runtime's terms against the modules that implement them (file anchors + decision-record refs), with the standing rule that planned-but-unbuilt concepts live in PLANNING/PLAN in future tense, never here. Linked from README and ARCHITECTURE. Co-Authored-By: Claude <noreply@anthropic.com>
Code-anchored audit of every normative doc against the tree: Blockers — CLI examples named flags that do not exist (--addr/--host are --bind plus the --allow-remote gate); the binding dependency diagram showed a nonexistent runtime -> core edge, drew shipped mcp/wasm/ffi as "when they land", and dropped the engine's real runtime+schema edges; the crate table repeated three wrong cells; a repair command from a foreign repo was cited as in-tree. The diagram is now an adjacency list verified against every Cargo.toml (2026-10-15). Stale — onnx is reinstated (Kai rung), VALIDATION is executed (RV-001..013), SECURITY supports 0.6.x, BENCHMARKS record date follows its rows, PLAN phase 22 is done, phase 24 credits 24j/24k, phase 25 added for the v0.6.x cuts + first gated weights, SPEC_COVERAGE carries 0.6.1, 33 e2e checks, the real WasmEngine surface, site tests count. Minor — six surfaces not four, --allow-remote documented, README says where releases actually live (aliasfox hosts them; the TaskWizerAI org is empty), VALIDATION open question reworded to the real #104 anchor, ARCHITECTURE interface row complete. Co-Authored-By: Claude <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Track V phases E-H: phases E-H done (D38, siglip contract + ONNX transport + CLI), I-J blocked on weights, K done. Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two commits on the docs lane:
docs/GLOSSARY.md— a glossary of every term the project uses, each entry anchored to the code that defines it (file:line for the load-bearing ones), with a governance preamble: planned-but-unbuilt machinery lives in PLANNING/PLAN in future tense and never masquerades as shipped. README + ARCHITECTURE link to it.runtime → coreedge, shipped mcp/wasm/ffi drawn as "when they land", engine's real runtime+schema edges missing). Redrawn as an adjacency list verified against every crate'sCargo.toml.--addr,--host) — the real flags are--bind+--allow-remote.aliasfoxaccount; theTaskWizerAIHF org is empty), and INTEGRATIONS counts six surfaces, not four.No code changes; docs-only. CI gate unchanged.
Verification
🤖 Generated with Claude Code