Skip to content

Unified dev CLI: scenario TOML as the single artifact (run → trace → build → verify) #19

Description

@adilfaisal01

Deferred from the 2026-09-07 deployment-provenance session. The trace/compile path is currently "dirt road": only two hardcoded targets (gen_base.py, gen_mpc.py), no "does my component trace?" check, MPC requires knowing to run two scripts, verify is opt-in, and the scenario→compile link is a manual transcription.

Concept

The scenario TOML (tests/integration/scenarios/base_tracking.toml) already contains almost everything gen_base.py hardcodes — controller config, estimator config, input_limits, dt. Make it the single artifact the developer authors, and expose verbs over it via a shinro console script (precedent: shinro-mcp in pyproject.toml):

shinro check  <cfg.toml>          # factory .create() + infer_contract — "loads? constructs?"
shinro run    <scenario.toml>     # ScenarioFactory + run_scenario + assertions (behavior gate)
shinro trace  <cfg.toml>          # trace_node + op-coverage report via available_ops (trace gate)
shinro build  <scenario.toml>     # bake? → trace → compose → lower → zig → oracle → stamp
shinro verify <scenario.toml>     # verify_deployment against that target's record

Key design points

  • [deploy] section added to the scenario TOML: recipe (registered composition), plant_dims, optional solver = "emosqp" (presence triggers bake), out (per-target dir — kills the shared-path clobbering structurally).
  • Graph-recipe registry (@register_graph("closed_loop_tracking")): the compose wiring in gen_base.py/gen_mpc.py becomes parameterized code; the two scripts collapse into recipes + scenario files. Provenance config slot is then derived from the scenario file, not hand-written (removes the silent under-reporting hazard).
  • Oracle-in-build, two gates: gate A (pre-compile, numpy, seconds) interpret(graph) vs live components — trace fidelity; gate B (post-compile, ctypes) .so vs interpret(graph) bit-exact — lowering fidelity. Stamp only runs after both pass, so a record can't exist for a binary that doesn't reproduce the simulated math.
  • Build modes/targets: runtime/build.zig already supports -Doptimize and -Dtarget (cross-compile, no sysroot) and records them in the manifest. CLI surfaces --mode/--target. Two provenance gaps to close: (1) stamp_deployment.py must copy target/optimize/strip/zig version into the record (currently a Debug vs ReleaseFast build of identical configs differs only in the binary slot, unexplained); (2) per-mode output dirs (build/<name>/release-safe-aarch64/) or the current record clobbers itself. Cross-compiled builds can't run the ctypes oracle — record should note "oracle": "not_run (cross-compiled)" and reference the native build's master hash.

Verification points (four, not one)

Behavior (scenario tolerances) → equivalence (oracle bit-exactness) → identity (master hash) → drift (verify re-hash). Each layer's output anchors the next.

What exists vs new

Exists (just call it): factories, ScenarioFactory, trace_node, compose, interpret, lower_zig, stamp, verify, console-script pattern. New: src/shinro/cli.py (~100 lines argparse, stdlib), graph-recipe registry (refactor of the two gen scripts), [deploy] parsing + shapes probing, moving run_scenario out of tests/integration/helpers/ into src/shinro/simulation/.

Suggested phasing

  1. make zig-build-mpc target (Makefile-only) + verify wired into zig-build — closes the sharpest edges now.
  2. Standalone shinro trace script (~30 lines) — the highest-value single command, no build machinery needed.
  3. Full CLI + recipe registry + oracle-in-build when the trace/compile path gets real users.

Priority: P3.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions