diff --git a/docs/methodology/euroc-consistency-gate.mdx b/docs/methodology/euroc-consistency-gate.mdx index fb1a67c..8b97a0a 100644 --- a/docs/methodology/euroc-consistency-gate.mdx +++ b/docs/methodology/euroc-consistency-gate.mdx @@ -110,7 +110,7 @@ on grid complete: |---|---|---| | the median normalized NIS over the matched runs is ≈ 1 (the bench's noise model is true) | `scripts/euroc-consistency.sh` | holds — 1.03 for both backends at c6b87ec (1.65–4.3 with the real IMU and the raw ground truth) | | every sequence's matched runs have normalized NIS within the χ² band for their dof | `scripts/euroc-consistency.sh` | failing — 1.01–1.04 on the Machine Hall and V1_01, V2_01; 1.1–1.9 on V1_02, V1_03, V2_02, V2_03 for both backends: S2's sample-hold skew, an estimator defect (#616) | -| the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order | `tests/tools/gt_spline.cpp` “gt spline: S2 closure, the shipped S2 within what the camera resolves…” | failing — the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616 | +| the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order (bound 1.09e-4 rad rotation, 0.22 mm position at 2 m, over a 0.55 s clone window, from what **run_filter** resolves: the backend's update resolves a clone pose only to the track noise, 0.5 px at fx = 458.654 (1.09e-3 rad); propagation error must stay a tenth of that over the clone window so it never competes with the measurement, and the same angle at a 2 m landmark depth is 0.22 mm) | `tests/tools/gt_spline.cpp` “gt spline: S2 closure, the shipped S2 within what the camera resolves…” | failing — the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616 | | every backend runs with the same S6d gate line | `scripts/euroc-consistency.sh` | holds — since #608 both use the χ² quantile; before it the R-IEKF used 16 per dof | | each condition's result changes with the estimator, not with the gate's line | — | holds — each remaining failure rerun with the gate at 5 per dof: one divergence per backend vanishes (with a true noise model, the estimator's transient over-confidence made the 5% gate fatal), the others persist; condition 2's fast-sequence loss is #616's S2 skew. Conditions 1 and 2 still depend on the noise draw at three seeds (#613) | diff --git a/docs/methodology/openvins-msckf.mdx b/docs/methodology/openvins-msckf.mdx index 17fd124..c69d607 100644 --- a/docs/methodology/openvins-msckf.mdx +++ b/docs/methodology/openvins-msckf.mdx @@ -66,6 +66,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*. - *Pre:* a successful seed - *Post:* P₀ symmetric positive-definite, in the error convention every later node assumes - *Test:* `tests/sdk/msckf_backend.cpp` “the backend initializes attitude from a stationary window…” +- *Decision — the prior P₀:* isotropic σ = 0.1 on every core state over the tilt–accelerometer-bias correlation an IMU window implies (#609). **no reason recorded** - *Deviates from the paper:* the prior is an isotropic σ = 0.1 with no tilt-bias correlation, whatever the init method (#609) **propagate** (S2, IMU propagation) @@ -74,7 +75,12 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*. - *Post:* R ∈ SO(3); P symmetric PSD; Φ preserves the 4-D unobservable subspace - *Test:* `tests/sdk/msckf_stages.cpp` “S2_propagation and S3_augmentation…” - *Precision:* T2 (msckf/stage_types.hpp) — posit<32,2> passes the bench; P must be written in double on EuRoC (#532) +- *Decision — the IMU input over each interval:* the sample that ends it, held constant (#186) over the previous sample held, both samples, interpolated and integrated by RK4 (geneva2020). **no reason recorded** +- *Decision — the mean's integrator:* one Euler step, the specific force rotated with the attitude at the start of the interval over midpoint, RK4. **no reason recorded** +- *Decision — the transition Φ:* first order, I + A·dt over closed form, a higher-order series. #471: a higher-order Φ leaks more yaw in the body-frame error state +- *Decision — the discrete process noise Q_d:* diagonal, σ²·dt per state over G·Q·Gᵀ with the θ–v–p correlations (geneva2020). #212's S2 probe found the full Q_d did not change the over-confidence (docs/pipeline/stages/s2-imu-propagation.mdx) - *Deviates from the paper:* no first-estimates Jacobians on Φ (FEJ, #606) +- *Deviates from the paper:* the sample that ends each interval is held over it: a 2.5 ms IMU–camera skew, NIS 1.1–1.9 on the fast sequences (#616) **augment** (S3, State augmentation) @@ -96,6 +102,8 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*. - *Test:* `tests/sdk/msckf_camera_update.cpp` “S6d gate line: the chi-square quantile by default…” - *Test:* `tests/tools/stage_bench_s6.cpp` “S6 update: no information along the unobservable directions…” - *Precision:* T6, T6e (msckf/stage_types.hpp) — S6e writes P: double on EuRoC (#532); the carried square-root factor holds in posit32 +- *Decision — the S6d gate line:* the χ²₁₋α(dof) quantile, α = 0.05 over 5 per dof (before #481). #481: it admits 1 − α of consistent inliers at every dof; the per-dof line admitted most 2σ-biased short tracks +- *Decision — S6c compression:* none (the identity) over QR to n + 1 rows (geneva2020). one feature per update leaves 2m − 3 rows, fewer than the state dimension, so QR would not shrink it - *Deviates from the paper:* no first-estimates Jacobians on H (FEJ, #606); the yaw leak is S6 moving the mean (95–99%, #480) - *Deviates from the paper:* S6c compression is the identity (one feature per update leaves 2m − 3 rows) @@ -131,7 +139,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*. | **NavSeed** | data | R, v, p, b_g, b_a, the time, and the prior P₀ in the filter's error convention | `sdk/include/branes/sdk/msckf/stages/s1_initialization.hpp` `seed_from_imu` | ✓ gravity sign and magnitude; ✓ P₀ symmetric positive-definite; ✗ P₀ states what the init window does and does not determine (tilt-bias correlation) | | **FilterState** | state | nav (θ, p, v, b_g, b_a: 15), clones (θ, p: 6 each), optional camera↔IMU extrinsics (6 per camera), the joint covariance P, and the error-state convention every Jacobian assumes | `sdk/include/branes/sdk/msckf/state.hpp` `State` | ✓ P symmetric; ✓ P positive semidefinite; ✓ R and every clone rotation ∈ SO(3); ✗ the layout and the error convention agree with every node's Jacobians | | **Landmark** | data | feature position p_f (world), the triangulation's parallax and conditioning | `sdk/include/branes/sdk/msckf/stages/s5_triangulation.hpp` `Result` | ✓ positive depth in every observing camera; ✓ normal-matrix condition number reported | -| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows | +| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows (bound χ²₁₋α(dof) per update, α = 0.05, from what **msckf_update** resolves: the S6d gate rejects above it, so a consistent filter loses α of its inliers and no more) | - **ImageFrame** precision: integer pixels; no arithmetic-type choice - **NormalizedObs** precision: the narrowest edge that keeps sub-pixel accuracy; float holds (#532) @@ -176,7 +184,7 @@ on image (20 Hz): ## Gap report -What `pipeline_graph.py gaps` reports for this spec (22 items): +What `pipeline_graph.py gaps` reports for this spec (26 items): - edge 'ImageFrame': no validator for "t increasing and inside the IMU span" - edge 'ImageFrame': no validator for "the image matches the calibration's resolution" @@ -191,11 +199,15 @@ What `pipeline_graph.py gaps` reports for this spec (22 items): - node 'initialize': deviates from the paper: the dynamic path is excitation-limited on EuRoC and has open SfM defects (#254, #499–#501) - node 'seed_state': deviates from the paper: the prior is an isotropic σ = 0.1 with no tilt-bias correlation, whatever the init method (#609) - node 'propagate': deviates from the paper: no first-estimates Jacobians on Φ (FEJ, #606) +- node 'propagate': deviates from the paper: the sample that ends each interval is held over it: a 2.5 ms IMU–camera skew, NIS 1.1–1.9 on the fast sequences (#616) - node 'msckf_update': deviates from the paper: no first-estimates Jacobians on H (FEJ, #606); the yaw leak is S6 moving the mean (95–99%, #480) - node 'msckf_update': deviates from the paper: S6c compression is the identity (one feature per update leaves 2m − 3 rows) - node 'slam_update' (S7): not implemented - node 'online_calib' (S10): partially implemented - node 'online_calib': deviates from the paper: extrinsics only (VioConfig::estimate_extrinsics, off by default); no intrinsics; no time offset (#332) +- node 'seed_state': decision 'the prior P₀' has no recorded reason (chosen: isotropic σ = 0.1 on every core state) +- node 'propagate': decision 'the IMU input over each interval' has no recorded reason (chosen: the sample that ends it, held constant (#186)) +- node 'propagate': decision 'the mean's integrator' has no recorded reason (chosen: one Euler step, the specific force rotated with the attitude at the start of the interval) - system invariant 'nees': failing — the body-frame filter is over-confident on EuRoC (#212, #605) - system invariant 'gauge': failing — the yaw leak (#480); fixed structurally by FEJ (#606) or the R-IEKF (#608) - system invariant 'consistency_gate': failing — with one truth (#613, median NIS 1.03): fails 1 (2 of 66, MH_03 at 1 px), fails 2 (1.09× vs 0.97×; 0.84× vs 0.69×), fails 3 (yaw σ ratio 0.107: the yaw leak, #480) diff --git a/docs/methodology/overview.mdx b/docs/methodology/overview.mdx index e773eac..697335b 100644 --- a/docs/methodology/overview.mdx +++ b/docs/methodology/overview.mdx @@ -59,11 +59,13 @@ The decomposition was retrofitted to debug a running filter. What that cost: 1. **Extract the graph from the paper.** Write the spec: every node with the equations and section it comes from, its pre- and post-conditions and the failure modes the literature reports; every - edge with its contents, units, manifold and invariants; the schedule; the system invariants. A - paper that modifies an existing pipeline is a **delta spec** on it (below). + edge with its contents, units, manifold and invariants; the schedule; the system invariants. Every + choice the paper leaves open is recorded as a **decision** with its reason. A paper that modifies + an existing pipeline is a **delta spec** on it (below). 2. **Contract the edges.** Give each data structure a validator before implementing any node. The checks live in `sdk/include/branes/sdk/eval/invariants/`; the spec points at them, and an - invariant without one is reported as a gap. + invariant without one is reported as a gap. A tolerance is a **bound** derived from what the + consuming node must resolve, never set to pass what the implementation does today. 3. **Stub and bench the nodes.** Each node becomes an `sdk/` transformation header, a [stage bench](../debugging/stage-benches.mdx) with its oracle (finite-difference Jacobians, a synthetic closed form, a captured fixture) and a sensitivity sweep. The spec records the header, @@ -96,6 +98,12 @@ Before implementation starts, a spec review answers: - Does every edge invariant have a validator, or is the gap accepted for now? - Does every node have pre- and post-conditions, a bench and an oracle? - Is every deviation from the paper written down, with its issue? +- **For every option the paper offers, does the spec record which was chosen and why?** A choice + without a reason is a default, and the gap report lists it (#616: the IMU mean integrator, a + default since #186). +- **Is every tolerance derived from what the consuming node must resolve?** A threshold fitted to + the implementation's current residual blesses its defects (#613's first S2-closure test passed the + error #616 found). - Is every system invariant tied to a check, with its current status? ## Running the tool @@ -124,3 +132,4 @@ change to the graph. | [The right-invariant EKF](riekf-msckf.mdx) as a delta spec: written before #608's code, which then closed its gaps | done (#608) | | A timed pilot on a new paper | planned | | Per-node and per-edge precision fields | in the format; filled for S2, S6 and the state | +| Decisions with reasons, and bounds derived from the consumer | in the format and the checklist; the OpenVINS spec records S1b, S2 and S6's choices, three of them still undecided defaults (#609, #616) | diff --git a/docs/methodology/riekf-msckf.mdx b/docs/methodology/riekf-msckf.mdx index cc71d7d..db854c4 100644 --- a/docs/methodology/riekf-msckf.mdx +++ b/docs/methodology/riekf-msckf.mdx @@ -129,7 +129,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*. | **NavSeed** | data | R, v, p, b_g, b_a, the time, and the prior P₀ in the filter's error convention | `sdk/include/branes/sdk/msckf/stages/s1_initialization.hpp` `seed_from_imu` | ✓ gravity sign and magnitude; ✓ P₀ symmetric positive-definite; ✗ P₀ states what the init window does and does not determine (tilt-bias correlation) | | **FilterState** | state | nav on SE₂(3) with additive biases (θ, ν, ρ, b_g, b_a: 15), clones (θ, ρ: 6 each) in the same right-invariant error, the joint covariance P | `sdk/include/branes/sdk/msckf/invariant_propagator.hpp` `InvariantNavState` | ✓ P symmetric; ✓ P positive semidefinite; ✓ R and every clone rotation ∈ SO(3); ✗ every Jacobian and retraction uses R = Exp(δθ)·R̂, v = Exp(δθ)·v̂ + ν, p = Exp(δθ)·p̂ + ρ | | **Landmark** | data | feature position p_f (world), the triangulation's parallax and conditioning | `sdk/include/branes/sdk/msckf/stages/s5_triangulation.hpp` `Result` | ✓ positive depth in every observing camera; ✓ normal-matrix condition number reported | -| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows | +| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows (bound χ²₁₋α(dof) per update, α = 0.05, from what **msckf_update** resolves: the S6d gate rejects above it, so a consistent filter loses α of its inliers and no more) | - **ImageFrame** precision: integer pixels; no arithmetic-type choice - **NormalizedObs** precision: the narrowest edge that keeps sub-pixel accuracy; float holds (#532) diff --git a/docs/methodology/spec-format.mdx b/docs/methodology/spec-format.mdx index b40eb8c..51dbb83 100644 --- a/docs/methodology/spec-format.mdx +++ b/docs/methodology/spec-format.mdx @@ -33,7 +33,7 @@ is `docs/methodology/specs/openvins-msckf.yaml`. | `kind` | `data` (default); `input` (a sensor or a calibration: no node writes it); `state` (the shared mutable state); `output` (leaves the graph: no node reads it) | | `contents` | what it holds, with units | | `code`, `symbol` | the file and the type or function that implements it; the symbol must occur in the file | -| `invariants` | a list of `{text, check}`; `check` is a [reference](#references). An invariant without a check is a gap | +| `invariants` | a list of `{text, check, bound?}`; `check` is a [reference](#references), `bound` is [below](#bounds). An invariant without a check is a gap | | `precision` | free text: how narrow a type this structure tolerates | | `deviations` | how the implementation departs from the paper; each is a gap | @@ -52,6 +52,7 @@ is `docs/methodology/specs/openvins-msckf.yaml`. | `status` | `implemented`, `partial`, `planned` (specified ahead of the code: the schedule may call it, and it is reported as a gap) or `missing` (deliberately left out: the schedule must not call it) | | `optional` | true for a node the algorithm can run without | | `precision` | `{policy, note}`: the node's arithmetic-type policy and what was measured | +| `decisions` | the choices the paper leaves open, [below](#decisions) | | `deviations` | how the implementation departs from the paper; each is a gap | ## Schedule @@ -71,9 +72,46 @@ a `missing` node must not be. A spec written before its code calls its `planned` ## System invariants -`{id, text, check?, status?, note?}`, where `status` is `holds`, `failing` or `unknown` (the +`{id, text, check?, bound?, status?, note?}`, where `status` is `holds`, `failing` or `unknown` (the default). A failing invariant is reported as a gap together with its note. +## Decisions + +A paper often leaves a choice open, or names options: a zeroth-order hold or RK4 for the IMU +mean, a gate line, a prior. Each is recorded on the node that makes it: + +```yaml +decisions: + - question: the IMU input over each interval + chosen: the sample that ends it, held constant + alternatives: [the previous sample held, "both samples, interpolated and integrated by RK4"] + reason: "…" # what was measured or argued; omitted, the choice was a default +``` + +`question` and `chosen` are required. A decision **without a `reason` is reported as a gap**: the +choice was made by default rather than decided. The rule exists because of #616. The VIO mean +integrator was a one-sample Euler step from the first commit (#186). The canonical pipeline document +named RK4 as OpenVINS's alternative, but nothing recorded a decision. Five months later the choice +turned out to hold each IMU sample 2.5 ms off the camera: NIS 1.1–1.9 on the fast sequences, and +ATE 10–15× worse there than with both samples used. + +## Bounds + +A check's tolerance comes from what the node that **consumes** the result must resolve, never from +what the implementation currently achieves: + +```yaml +bound: + value: 1.1e-4 rad over a 0.55 s clone window + consumer: msckf_update # a node id: whose resolution the bound comes from + derivation: a tenth of the 0.5 px track noise the update resolves, at fx = 458.65 +``` + +`value` and `derivation` are required, and `consumer` must name a node in the spec. The rule +exists because of #613. Its first S2-closure test measured S2's first-order error, then set its +threshold to accept it, and so passed exactly the defect #616 later found. A bound derived from the +camera's resolution failed it 28×. + ## References A reference is `{file, symbol?, case?}`. The file must exist; a `symbol` must occur in its code as diff --git a/docs/methodology/specs/euroc-consistency-gate.yaml b/docs/methodology/specs/euroc-consistency-gate.yaml index 528c60b..00a97fc 100644 --- a/docs/methodology/specs/euroc-consistency-gate.yaml +++ b/docs/methodology/specs/euroc-consistency-gate.yaml @@ -260,6 +260,10 @@ system_invariants: - id: s2_closure text: "the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order" check: {file: tests/tools/gt_spline.cpp, case: "gt spline: S2 closure, the shipped S2 within what the camera resolves"} + bound: + value: "1.09e-4 rad rotation, 0.22 mm position at 2 m, over a 0.55 s clone window" + consumer: run_filter + derivation: "the backend's update resolves a clone pose only to the track noise, 0.5 px at fx = 458.654 (1.09e-3 rad); propagation error must stay a tenth of that over the clone window so it never competes with the measurement, and the same angle at a 2 m landmark depth is 0.22 mm" status: failing note: "the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616" - id: same_gate diff --git a/docs/methodology/specs/openvins-msckf.yaml b/docs/methodology/specs/openvins-msckf.yaml index 0afcb7d..9e8b01f 100644 --- a/docs/methodology/specs/openvins-msckf.yaml +++ b/docs/methodology/specs/openvins-msckf.yaml @@ -142,6 +142,10 @@ edges: invariants: - text: NIS ~ χ²(dof) over windows check: {file: sdk/include/branes/sdk/eval/invariants/stream_checks.hpp, symbol: check_chi2_window} + bound: + value: "χ²₁₋α(dof) per update, α = 0.05" + consumer: msckf_update + derivation: "the S6d gate rejects above it, so a consistent filter loses α of its inliers and no more" # ── The transformations ────────────────────────────────────────────────────── # Every node has the shape (state, inputs, params) → (state′, diagnostics): it @@ -228,6 +232,10 @@ nodes: status: implemented deviations: - the prior is an isotropic σ = 0.1 with no tilt-bias correlation, whatever the init method (#609) + decisions: + - question: the prior P₀ + chosen: "isotropic σ = 0.1 on every core state" + alternatives: ["the tilt–accelerometer-bias correlation an IMU window implies (#609)"] - id: propagate stage: S2 @@ -249,8 +257,24 @@ nodes: - {file: tests/sdk/msckf_stages.cpp, case: "S2_propagation and S3_augmentation"} status: implemented precision: {policy: "T2 (msckf/stage_types.hpp)", note: "posit<32,2> passes the bench; P must be written in double on EuRoC (#532)"} + decisions: + - question: the IMU input over each interval + chosen: "the sample that ends it, held constant (#186)" + alternatives: ["the previous sample held", "both samples, interpolated and integrated by RK4 (geneva2020)"] + - question: the mean's integrator + chosen: one Euler step, the specific force rotated with the attitude at the start of the interval + alternatives: [midpoint, RK4] + - question: the transition Φ + chosen: first order, I + A·dt + alternatives: ["closed form", "a higher-order series"] + reason: "#471: a higher-order Φ leaks more yaw in the body-frame error state" + - question: the discrete process noise Q_d + chosen: "diagonal, σ²·dt per state" + alternatives: ["G·Q·Gᵀ with the θ–v–p correlations (geneva2020)"] + reason: "#212's S2 probe found the full Q_d did not change the over-confidence (docs/pipeline/stages/s2-imu-propagation.mdx)" deviations: - "no first-estimates Jacobians on Φ (FEJ, #606)" + - "the sample that ends each interval is held over it: a 2.5 ms IMU–camera skew, NIS 1.1–1.9 on the fast sequences (#616)" - id: augment stage: S3 @@ -317,6 +341,15 @@ nodes: deviations: - "no first-estimates Jacobians on H (FEJ, #606); the yaw leak is S6 moving the mean (95–99%, #480)" - S6c compression is the identity (one feature per update leaves 2m − 3 rows) + decisions: + - question: the S6d gate line + chosen: "the χ²₁₋α(dof) quantile, α = 0.05" + alternatives: ["5 per dof (before #481)"] + reason: "#481: it admits 1 − α of consistent inliers at every dof; the per-dof line admitted most 2σ-biased short tracks" + - question: S6c compression + chosen: none (the identity) + alternatives: ["QR to n + 1 rows (geneva2020)"] + reason: "one feature per update leaves 2m − 3 rows, fewer than the state dimension, so QR would not shrink it" - id: marginalize stage: S9 diff --git a/tools/pipeline_graph/pipeline_graph.py b/tools/pipeline_graph/pipeline_graph.py index 4811b7e..c05bf60 100644 --- a/tools/pipeline_graph/pipeline_graph.py +++ b/tools/pipeline_graph/pipeline_graph.py @@ -47,7 +47,7 @@ EDGE_KINDS = {"data", "input", "state", "output"} DELTA_OPS = {"add", "replace", "remove"} -NODE_KEYS = {"id", "stage", "title", "does", "reads", "writes", "source", "pre", "post", "code", "bench", +NODE_KEYS = {"id", "stage", "title", "does", "reads", "writes", "source", "pre", "post", "code", "bench", "decisions", "tests", "status", "optional", "precision", "deviations", "op", "note"} EDGE_KEYS = {"id", "kind", "contents", "code", "symbol", "invariants", "precision", "deviations", "op", "note"} TOP_KEYS = {"schema", "name", "title", "summary", "sources", "edges", "nodes", "schedule", "system_invariants", @@ -297,6 +297,45 @@ def strings(node: object, where: str = ""): yield from strings(v, f"{where}[{i}]") +def stated(x: object) -> bool: + """A field is stated when it is present and not blank: 0 and false are values.""" + return x is not None and str(x).strip() != "" + + +def lint_decision(where: str, d: object, rep: Report) -> None: + """A design choice the paper leaves open, recorded with its reason (#610): + {question, chosen, alternatives, reason}. A missing reason is a gap: the + choice was made by default, not decided.""" + if not isinstance(d, dict) or not stated(d.get("question")) or not stated(d.get("chosen")): + rep.error(f"{where}: a decision needs a question and what was chosen") + return + if not isinstance(d.get("alternatives", []), list): + rep.error(f"{where}: decision '{d['question']}': alternatives must be a list") + for k in d: + if k not in {"question", "chosen", "alternatives", "reason"}: + rep.error(f"{where}: decision '{d['question']}': unknown key '{k}'") + if not d.get("reason"): + rep.gap(f"{where}: decision '{d['question']}' has no recorded reason (chosen: {d['chosen']})") + + +def lint_bound(where: str, inv: dict, nodes: dict, rep: Report) -> None: + """A check's tolerance, derived from what the node that consumes the result + must resolve, not from what the implementation achieves (#610): + {value, consumer, derivation}, consumer a node id.""" + b = inv.get("bound") + if b is None: + return + if not isinstance(b, dict) or not stated(b.get("value")) or not stated(b.get("derivation")): + rep.error(f"{where}: a bound needs a value and its derivation") + return + for k in b: + if k not in {"value", "consumer", "derivation"}: + rep.error(f"{where}: bound: unknown key '{k}'") + consumer = b.get("consumer") + if not isinstance(consumer, str) or consumer not in nodes: + rep.error(f"{where}: bound: consumer must name the node whose resolution it comes from") + + def lint(spec: dict, delta: dict | None, rep: Report) -> None: raw = delta if delta is not None else spec # YAML ends a plain scalar at " #": "(#474, #574)" silently becomes "(#474,". @@ -406,6 +445,17 @@ def lint(spec: dict, delta: dict | None, rep: Report) -> None: rep.gap(f"node '{nid}' ({n.get('stage', '')}): partially implemented") for d in n.get("deviations", []): rep.gap(f"node '{nid}': deviates from the paper: {d}") + for n in nodes.values(): # decisions are recorded for missing nodes too + if not isinstance(n.get("decisions", []), list): + rep.error(f"node '{n['id']}': decisions must be a list") + continue + for d in n.get("decisions", []): + lint_decision(f"node '{n['id']}'", d, rep) + for eid, e in edges.items(): + for inv in e.get("invariants", []): + lint_bound(f"edge '{eid}' invariant '{inv.get('text')}'", inv, nodes, rep) + for inv in spec.get("system_invariants", []): + lint_bound(f"system invariant '{inv.get('id')}'", inv, nodes, rep) # Every edge is produced (an input, or some node writes it) and consumed. for eid, e in edges.items(): @@ -517,6 +567,14 @@ def md_escape(s: object) -> str: return out +def bound_md(inv: dict) -> str: + b = inv.get("bound") + if not b: + return "" + return (f" (bound {md_escape(b['value'])}, from what **{md_escape(b.get('consumer', ''))}** resolves: " + f"{md_escape(b['derivation'])})") + + def ref_md(ref: dict | None) -> str: if not ref: return "—" @@ -578,7 +636,7 @@ def page_source(spec: dict, delta: dict | None, rep: Report) -> str: f"{ref_md(n.get('bench'))} |") out += ["", "### Contracts", ""] for n in spec.get("nodes", []): - if not (n.get("pre") or n.get("post") or n.get("deviations") or n.get("precision")): + if not (n.get("pre") or n.get("post") or n.get("deviations") or n.get("precision") or n.get("decisions")): continue out.append(f"**{n['id']}** ({n.get('stage', '')}, {n.get('title', '')})") out.append("") @@ -592,6 +650,11 @@ def page_source(spec: dict, delta: dict | None, rep: Report) -> str: p = n["precision"] out.append(f"- *Precision:* {md_escape(p.get('policy', ''))} — {md_escape(p.get('note', ''))}" if isinstance(p, dict) else f"- *Precision:* {md_escape(p)}") + for d in n.get("decisions", []): + alts = d.get("alternatives") or [] + over = f" over {', '.join(md_escape(a) for a in alts)}" if alts else "" + why = md_escape(d["reason"]) if d.get("reason") else "**no reason recorded**" + out.append(f"- *Decision — {md_escape(d['question'])}:* {md_escape(d['chosen'])}{over}. {why}") for d in n.get("deviations", []): out.append(f"- *Deviates from the paper:* {md_escape(d)}") out.append("") @@ -599,7 +662,8 @@ def page_source(spec: dict, delta: dict | None, rep: Report) -> str: "| Edge | Kind | Contents | Type | Invariants (✓ has a validator) |", "|---|---|---|---|---|"] for e in spec.get("edges", []): - invs = "; ".join(("✓ " if i.get("check") else "✗ ") + md_escape(i["text"]) for i in e.get("invariants", [])) + invs = "; ".join(("✓ " if i.get("check") else "✗ ") + md_escape(i["text"]) + bound_md(i) + for i in e.get("invariants", [])) typ = f"`{e['code']}`" + (f" `{e['symbol']}`" if e.get("symbol") else "") if e.get("code") else "—" out.append(f"| **{e['id']}** | {e.get('kind', 'data')} | {md_escape(e.get('contents', ''))} | {typ} | " f"{invs or '—'} |") @@ -634,7 +698,7 @@ def page_source(spec: dict, delta: dict | None, rep: Report) -> str: st = inv.get("status", "unknown") if inv.get("note"): st += f" — {md_escape(inv['note'])}" - out.append(f"| {md_escape(inv['text'])} | {ref_md(inv.get('check'))} | {st} |") + out.append(f"| {md_escape(inv['text'])}{bound_md(inv)} | {ref_md(inv.get('check'))} | {st} |") if delta is not None: out += ["", "## Changes from the base", ""] for section in ("edges", "nodes"): diff --git a/tools/pipeline_graph/test_pipeline_graph.py b/tools/pipeline_graph/test_pipeline_graph.py index 1b5b99d..a9892da 100644 --- a/tools/pipeline_graph/test_pipeline_graph.py +++ b/tools/pipeline_graph/test_pipeline_graph.py @@ -240,6 +240,55 @@ def test_an_unhashable_key_is_an_error_not_a_traceback(self) -> None: _, _, rep = self.analyse("? [schema]\n: x\n" + BASE) self.assertTrue(any("unhashable key" in e for e in rep.errors), rep.errors) + def test_a_decision_without_a_reason_is_a_gap(self) -> None: + text = edit(BASE, " does: propagate the state\n", + " does: propagate the state\n" + " decisions:\n" + " - {question: input over an interval, chosen: the end sample held,\n" + " alternatives: [both samples (RK4)]}\n" + " - {question: integrator, chosen: RK4, reason: the paper's choice}\n") + spec, delta, rep = self.analyse(text) + self.assertEqual(rep.errors, []) + self.assertIn("node 'predict': decision 'input over an interval' has no recorded reason " + "(chosen: the end sample held)", rep.gaps) + self.assertFalse(any("integrator" in g for g in rep.gaps)) + page = pg.page_source(spec, delta, rep) + self.assertIn("the end sample held over both samples (RK4). **no reason recorded**", page) + + bad = edit(BASE, " does: propagate the state\n", + " does: propagate the state\n decisions:\n - {question: integrator}\n") + _, _, rep = self.analyse(bad) + self.assertTrue(any("needs a question and what was chosen" in e for e in rep.errors), rep.errors) + + for chosen in ("0", "false"): + with self.subTest(chosen=chosen): + zero = edit(BASE, " does: propagate the state\n", + " does: propagate the state\n decisions:\n" + f" - {{question: substeps, chosen: {chosen}, reason: one is enough}}\n") + _, _, rep = self.analyse(zero) + self.assertEqual(rep.errors, []) + + def test_a_bound_needs_its_derivation_and_a_consumer(self) -> None: + good = BASE + ("system_invariants:\n" + " - id: closure\n" + " text: S2 closes on the truth\n" + " bound: {value: 1.1e-4 rad, consumer: correct, derivation: a tenth of 0.5 px}\n") + spec, delta, rep = self.analyse(good) + self.assertEqual(rep.errors, []) + self.assertIn("(bound 1.1e-4 rad, from what **correct** resolves: a tenth of 0.5 px)", + pg.page_source(spec, delta, rep)) + cases = [ + ("needs a value and its derivation", good.replace(", derivation: a tenth of 0.5 px", "")), + ("consumer must name the node", good.replace("consumer: correct", "consumer: nobody")), + ("consumer must name the node", good.replace("consumer: correct", "consumer: [correct]")), + ] + _, _, rep = self.analyse(good.replace("value: 1.1e-4 rad", "value: 0")) + self.assertEqual(rep.errors, []) + for want, text in cases: + with self.subTest(want, text=text): + _, _, rep = self.analyse(text) + self.assertTrue(any(want in e for e in rep.errors), rep.errors) + def test_the_figure_draws_an_in_place_update_as_one_arrow(self) -> None: spec, _, _ = self.analyse(BASE) dot = pg.dot_source(spec)