Skip to content

docs: one-command reproduction, and what orchestrate check guarantees - #13

Closed
Immanuel-Fowler wants to merge 2 commits into
test/diagnostics-corpusfrom
docs/check-guarantees-and-reproduction
Closed

Immanuel-Fowler wants to merge 2 commits into
test/diagnostics-corpusfrom
docs/check-guarantees-and-reproduction

Conversation

@Immanuel-Fowler

Copy link
Copy Markdown
Owner

Stacked on #12 (test/diagnostics-corpus); merge that first. Last of the series.

What

  • One command regenerates every table. python3 benchmarks/run_all.py builds the release compiler and runs the boundary ladder, the tick cost, and the diagnostics coverage scripts in order, writing every CSV, JSON, and Markdown table under benchmarks/results/ and joining the three reports into results.md. --no-typescript, --skip <step>, --out DIR, and ORCHESTRATE=<path> are the knobs. Documented in benchmarks/README.md, which also gains the diagnostics-coverage section.
  • What orchestrate check guarantees, in one place. A paragraph in docs/language-reference.md (Chapter 5, before "Debugging Generated Code") states precisely what check does — parse, resolve modules, register every sidecar including a .wasm module's export table, typecheck the entry file — and what it does not: no codegen, no Cargo, library-only rules not enforced, module serverlet bodies not walked, foreign sources not checked. It is written to be quoted, so it is accurate rather than flattering.
  • LIMITATIONS-notes.md is complete for the series.

The series

PR Branch What
#7 feat/sandbox-arrays-structs arrays and structs across the sandbox; string leak; handler returns its state
#8 feat/sandbox-crash-and-grants grant call as linker imports; on_crash on the host
#9 feat/boundary-ladder-benchmark every rung; rate and concurrent-callers sweeps; CSV/JSON/MD
#10 feat/tick-cost-benchmark the tick cost as a script; two shared let codegen fixes
#11 test/determinism-trace 10,000-tick byte-identical trace; exclusions as documented
#12 test/diagnostics-corpus 84 cases, 76 rejected by check; sixteen typechecker fixes
#13 docs/check-guarantees-and-reproduction this one

Changelog entry under [Unreleased]. If a release is wanted, v0.15.0 fits: new features, nothing that worked changed behaviour. Not cut here.

🤖 Generated with Claude Code

Immanuel-Fowler and others added 2 commits September 20, 2026 23:38
benchmarks/run_all.py builds the release compiler and runs the boundary
ladder, the tick cost and the diagnostics coverage in order, writing
every table under benchmarks/results and joining the reports into
results.md; the benchmarks README documents it and gains the coverage
section. The language reference states in one paragraph what a check
does and finds and what it does not, written to be quoted, so accurate
rather than flattering.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
All three tables in the README now come from a single end-to-end run
of benchmarks/run_all.py, so their headers share one commit, one
timestamp and one machine.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant