Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/methodology/euroc-consistency-gate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ on grid complete:
|---|---|---|
| the median normalized NIS over the matched runs is ≈ 1 (the bench's noise model is true) | `scripts/euroc-consistency.sh` | holds — 1.03 for both backends at c6b87ec (1.65–4.3 with the real IMU and the raw ground truth) |
| every sequence's matched runs have normalized NIS within the χ² band for their dof | `scripts/euroc-consistency.sh` | failing — 1.01–1.04 on the Machine Hall and V1_01, V2_01; 1.1–1.9 on V1_02, V1_03, V2_02, V2_03 for both backends: S2's sample-hold skew, an estimator defect (#616) |
| the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order | `tests/tools/gt_spline.cpp` “gt spline: S2 closure, the shipped S2 within what the camera resolves…” | failing — the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616 |
| the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order (bound 1.09e-4 rad rotation, 0.22 mm position at 2 m, over a 0.55 s clone window, from what **run_filter** resolves: the backend's update resolves a clone pose only to the track noise, 0.5 px at fx = 458.654 (1.09e-3 rad); propagation error must stay a tenth of that over the clone window so it never competes with the measurement, and the same angle at a 2 m landmark depth is 0.22 mm) | `tests/tools/gt_spline.cpp` “gt spline: S2 closure, the shipped S2 within what the camera resolves…” | failing — the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616 |
| every backend runs with the same S6d gate line | `scripts/euroc-consistency.sh` | holds — since #608 both use the χ² quantile; before it the R-IEKF used 16 per dof |
| each condition's result changes with the estimator, not with the gate's line | — | holds — each remaining failure rerun with the gate at 5 per dof: one divergence per backend vanishes (with a true noise model, the estimator's transient over-confidence made the 5% gate fatal), the others persist; condition 2's fast-sequence loss is #616's S2 skew. Conditions 1 and 2 still depend on the noise draw at three seeds (#613) |

Expand Down
16 changes: 14 additions & 2 deletions docs/methodology/openvins-msckf.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*.
- *Pre:* a successful seed
- *Post:* P₀ symmetric positive-definite, in the error convention every later node assumes
- *Test:* `tests/sdk/msckf_backend.cpp` “the backend initializes attitude from a stationary window…”
- *Decision — the prior P₀:* isotropic σ = 0.1 on every core state over the tilt–accelerometer-bias correlation an IMU window implies (#609). **no reason recorded**
- *Deviates from the paper:* the prior is an isotropic σ = 0.1 with no tilt-bias correlation, whatever the init method (#609)

**propagate** (S2, IMU propagation)
Expand All @@ -74,7 +75,12 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*.
- *Post:* R ∈ SO(3); P symmetric PSD; Φ preserves the 4-D unobservable subspace
- *Test:* `tests/sdk/msckf_stages.cpp` “S2_propagation and S3_augmentation…”
- *Precision:* T2 (msckf/stage_types.hpp) — posit<32,2> passes the bench; P must be written in double on EuRoC (#532)
- *Decision — the IMU input over each interval:* the sample that ends it, held constant (#186) over the previous sample held, both samples, interpolated and integrated by RK4 (geneva2020). **no reason recorded**
- *Decision — the mean's integrator:* one Euler step, the specific force rotated with the attitude at the start of the interval over midpoint, RK4. **no reason recorded**
- *Decision — the transition Φ:* first order, I + A·dt over closed form, a higher-order series. #471: a higher-order Φ leaks more yaw in the body-frame error state
- *Decision — the discrete process noise Q_d:* diagonal, σ²·dt per state over G·Q·Gᵀ with the θ–v–p correlations (geneva2020). #212's S2 probe found the full Q_d did not change the over-confidence (docs/pipeline/stages/s2-imu-propagation.mdx)
- *Deviates from the paper:* no first-estimates Jacobians on Φ (FEJ, #606)
- *Deviates from the paper:* the sample that ends each interval is held over it: a 2.5 ms IMU–camera skew, NIS 1.1–1.9 on the fast sequences (#616)

**augment** (S3, State augmentation)

Expand All @@ -96,6 +102,8 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*.
- *Test:* `tests/sdk/msckf_camera_update.cpp` “S6d gate line: the chi-square quantile by default…”
- *Test:* `tests/tools/stage_bench_s6.cpp` “S6 update: no information along the unobservable directions…”
- *Precision:* T6, T6e (msckf/stage_types.hpp) — S6e writes P: double on EuRoC (#532); the carried square-root factor holds in posit32
- *Decision — the S6d gate line:* the χ²₁₋α(dof) quantile, α = 0.05 over 5 per dof (before #481). #481: it admits 1 − α of consistent inliers at every dof; the per-dof line admitted most 2σ-biased short tracks
- *Decision — S6c compression:* none (the identity) over QR to n + 1 rows (geneva2020). one feature per update leaves 2m − 3 rows, fewer than the state dimension, so QR would not shrink it
- *Deviates from the paper:* no first-estimates Jacobians on H (FEJ, #606); the yaw leak is S6 moving the mean (95–99%, #480)
- *Deviates from the paper:* S6c compression is the identity (one feature per update leaves 2m − 3 rows)

Expand Down Expand Up @@ -131,7 +139,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*.
| **NavSeed** | data | R, v, p, b_g, b_a, the time, and the prior P₀ in the filter's error convention | `sdk/include/branes/sdk/msckf/stages/s1_initialization.hpp` `seed_from_imu` | ✓ gravity sign and magnitude; ✓ P₀ symmetric positive-definite; ✗ P₀ states what the init window does and does not determine (tilt-bias correlation) |
| **FilterState** | state | nav (θ, p, v, b_g, b_a: 15), clones (θ, p: 6 each), optional camera↔IMU extrinsics (6 per camera), the joint covariance P, and the error-state convention every Jacobian assumes | `sdk/include/branes/sdk/msckf/state.hpp` `State` | ✓ P symmetric; ✓ P positive semidefinite; ✓ R and every clone rotation ∈ SO(3); ✗ the layout and the error convention agree with every node's Jacobians |
| **Landmark** | data | feature position p_f (world), the triangulation's parallax and conditioning | `sdk/include/branes/sdk/msckf/stages/s5_triangulation.hpp` `Result` | ✓ positive depth in every observing camera; ✓ normal-matrix condition number reported |
| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows |
| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows (bound χ²₁₋α(dof) per update, α = 0.05, from what **msckf_update** resolves: the S6d gate rejects above it, so a consistent filter loses α of its inliers and no more) |

- **ImageFrame** precision: integer pixels; no arithmetic-type choice
- **NormalizedObs** precision: the narrowest edge that keeps sub-pixel accuracy; float holds (#532)
Expand Down Expand Up @@ -176,7 +184,7 @@ on image (20 Hz):

## Gap report

What `pipeline_graph.py gaps` reports for this spec (22 items):
What `pipeline_graph.py gaps` reports for this spec (26 items):

- edge 'ImageFrame': no validator for "t increasing and inside the IMU span"
- edge 'ImageFrame': no validator for "the image matches the calibration's resolution"
Expand All @@ -191,11 +199,15 @@ What `pipeline_graph.py gaps` reports for this spec (22 items):
- node 'initialize': deviates from the paper: the dynamic path is excitation-limited on EuRoC and has open SfM defects (#254, #499–#501)
- node 'seed_state': deviates from the paper: the prior is an isotropic σ = 0.1 with no tilt-bias correlation, whatever the init method (#609)
- node 'propagate': deviates from the paper: no first-estimates Jacobians on Φ (FEJ, #606)
- node 'propagate': deviates from the paper: the sample that ends each interval is held over it: a 2.5 ms IMU–camera skew, NIS 1.1–1.9 on the fast sequences (#616)
- node 'msckf_update': deviates from the paper: no first-estimates Jacobians on H (FEJ, #606); the yaw leak is S6 moving the mean (95–99%, #480)
- node 'msckf_update': deviates from the paper: S6c compression is the identity (one feature per update leaves 2m − 3 rows)
- node 'slam_update' (S7): not implemented
- node 'online_calib' (S10): partially implemented
- node 'online_calib': deviates from the paper: extrinsics only (VioConfig::estimate_extrinsics, off by default); no intrinsics; no time offset (#332)
- node 'seed_state': decision 'the prior P₀' has no recorded reason (chosen: isotropic σ = 0.1 on every core state)
- node 'propagate': decision 'the IMU input over each interval' has no recorded reason (chosen: the sample that ends it, held constant (#186))
- node 'propagate': decision 'the mean's integrator' has no recorded reason (chosen: one Euler step, the specific force rotated with the attitude at the start of the interval)
- system invariant 'nees': failing — the body-frame filter is over-confident on EuRoC (#212, #605)
- system invariant 'gauge': failing — the yaw leak (#480); fixed structurally by FEJ (#606) or the R-IEKF (#608)
- system invariant 'consistency_gate': failing — with one truth (#613, median NIS 1.03): fails 1 (2 of 66, MH_03 at 1 px), fails 2 (1.09× vs 0.97×; 0.84× vs 0.69×), fails 3 (yaw σ ratio 0.107: the yaw leak, #480)
Expand Down
15 changes: 12 additions & 3 deletions docs/methodology/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ The decomposition was retrofitted to debug a running filter. What that cost:

1. **Extract the graph from the paper.** Write the spec: every node with the equations and section
it comes from, its pre- and post-conditions and the failure modes the literature reports; every
edge with its contents, units, manifold and invariants; the schedule; the system invariants. A
paper that modifies an existing pipeline is a **delta spec** on it (below).
edge with its contents, units, manifold and invariants; the schedule; the system invariants. Every
choice the paper leaves open is recorded as a **decision** with its reason. A paper that modifies
an existing pipeline is a **delta spec** on it (below).
2. **Contract the edges.** Give each data structure a validator before implementing any node. The
checks live in `sdk/include/branes/sdk/eval/invariants/`; the spec points at them, and an
invariant without one is reported as a gap.
invariant without one is reported as a gap. A tolerance is a **bound** derived from what the
consuming node must resolve, never set to pass what the implementation does today.
3. **Stub and bench the nodes.** Each node becomes an `sdk/` transformation header, a
[stage bench](../debugging/stage-benches.mdx) with its oracle (finite-difference Jacobians, a
synthetic closed form, a captured fixture) and a sensitivity sweep. The spec records the header,
Expand Down Expand Up @@ -96,6 +98,12 @@ Before implementation starts, a spec review answers:
- Does every edge invariant have a validator, or is the gap accepted for now?
- Does every node have pre- and post-conditions, a bench and an oracle?
- Is every deviation from the paper written down, with its issue?
- **For every option the paper offers, does the spec record which was chosen and why?** A choice
without a reason is a default, and the gap report lists it (#616: the IMU mean integrator, a
default since #186).
- **Is every tolerance derived from what the consuming node must resolve?** A threshold fitted to
the implementation's current residual blesses its defects (#613's first S2-closure test passed the
error #616 found).
- Is every system invariant tied to a check, with its current status?

## Running the tool
Expand Down Expand Up @@ -124,3 +132,4 @@ change to the graph.
| [The right-invariant EKF](riekf-msckf.mdx) as a delta spec: written before #608's code, which then closed its gaps | done (#608) |
| A timed pilot on a new paper | planned |
| Per-node and per-edge precision fields | in the format; filled for S2, S6 and the state |
| Decisions with reasons, and bounds derived from the consumer | in the format and the checklist; the OpenVINS spec records S1b, S2 and S6's choices, three of them still undecided defaults (#609, #616) |
2 changes: 1 addition & 1 deletion docs/methodology/riekf-msckf.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Each has the shape *(state, inputs, params) → (state′, diagnostics)*.
| **NavSeed** | data | R, v, p, b_g, b_a, the time, and the prior P₀ in the filter's error convention | `sdk/include/branes/sdk/msckf/stages/s1_initialization.hpp` `seed_from_imu` | ✓ gravity sign and magnitude; ✓ P₀ symmetric positive-definite; ✗ P₀ states what the init window does and does not determine (tilt-bias correlation) |
| **FilterState** | state | nav on SE₂(3) with additive biases (θ, ν, ρ, b_g, b_a: 15), clones (θ, ρ: 6 each) in the same right-invariant error, the joint covariance P | `sdk/include/branes/sdk/msckf/invariant_propagator.hpp` `InvariantNavState` | ✓ P symmetric; ✓ P positive semidefinite; ✓ R and every clone rotation ∈ SO(3); ✗ every Jacobian and retraction uses R = Exp(δθ)·R̂, v = Exp(δθ)·v̂ + ν, p = Exp(δθ)·p̂ + ρ |
| **Landmark** | data | feature position p_f (world), the triangulation's parallax and conditioning | `sdk/include/branes/sdk/msckf/stages/s5_triangulation.hpp` `Result` | ✓ positive depth in every observing camera; ✓ normal-matrix condition number reported |
| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows |
| **UpdateDiag** | output | NIS, dof, the gate verdict, and where a rejected track stopped | `sdk/include/branes/sdk/msckf/stages/s6_msckf_update.hpp` `Result` | ✓ NIS ~ χ²(dof) over windows (bound χ²₁₋α(dof) per update, α = 0.05, from what **msckf_update** resolves: the S6d gate rejects above it, so a consistent filter loses α of its inliers and no more) |

- **ImageFrame** precision: integer pixels; no arithmetic-type choice
- **NormalizedObs** precision: the narrowest edge that keeps sub-pixel accuracy; float holds (#532)
Expand Down
42 changes: 40 additions & 2 deletions docs/methodology/spec-format.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ is `docs/methodology/specs/openvins-msckf.yaml`.
| `kind` | `data` (default); `input` (a sensor or a calibration: no node writes it); `state` (the shared mutable state); `output` (leaves the graph: no node reads it) |
| `contents` | what it holds, with units |
| `code`, `symbol` | the file and the type or function that implements it; the symbol must occur in the file |
| `invariants` | a list of `{text, check}`; `check` is a [reference](#references). An invariant without a check is a gap |
| `invariants` | a list of `{text, check, bound?}`; `check` is a [reference](#references), `bound` is [below](#bounds). An invariant without a check is a gap |
| `precision` | free text: how narrow a type this structure tolerates |
| `deviations` | how the implementation departs from the paper; each is a gap |

Expand All @@ -52,6 +52,7 @@ is `docs/methodology/specs/openvins-msckf.yaml`.
| `status` | `implemented`, `partial`, `planned` (specified ahead of the code: the schedule may call it, and it is reported as a gap) or `missing` (deliberately left out: the schedule must not call it) |
| `optional` | true for a node the algorithm can run without |
| `precision` | `{policy, note}`: the node's arithmetic-type policy and what was measured |
| `decisions` | the choices the paper leaves open, [below](#decisions) |
| `deviations` | how the implementation departs from the paper; each is a gap |

## Schedule
Expand All @@ -71,9 +72,46 @@ a `missing` node must not be. A spec written before its code calls its `planned`

## System invariants

`{id, text, check?, status?, note?}`, where `status` is `holds`, `failing` or `unknown` (the
`{id, text, check?, bound?, status?, note?}`, where `status` is `holds`, `failing` or `unknown` (the
default). A failing invariant is reported as a gap together with its note.

## Decisions

A paper often leaves a choice open, or names options: a zeroth-order hold or RK4 for the IMU
mean, a gate line, a prior. Each is recorded on the node that makes it:

```yaml
decisions:
- question: the IMU input over each interval
chosen: the sample that ends it, held constant
alternatives: [the previous sample held, "both samples, interpolated and integrated by RK4"]
reason: "…" # what was measured or argued; omitted, the choice was a default
```

`question` and `chosen` are required. A decision **without a `reason` is reported as a gap**: the
choice was made by default rather than decided. The rule exists because of #616. The VIO mean
integrator was a one-sample Euler step from the first commit (#186). The canonical pipeline document
named RK4 as OpenVINS's alternative, but nothing recorded a decision. Five months later the choice
turned out to hold each IMU sample 2.5 ms off the camera: NIS 1.1–1.9 on the fast sequences, and
ATE 10–15× worse there than with both samples used.

## Bounds

A check's tolerance comes from what the node that **consumes** the result must resolve, never from
what the implementation currently achieves:

```yaml
bound:
value: 1.1e-4 rad over a 0.55 s clone window
consumer: msckf_update # a node id: whose resolution the bound comes from
derivation: a tenth of the 0.5 px track noise the update resolves, at fx = 458.65
```

`value` and `derivation` are required, and `consumer` must name a node in the spec. The rule
exists because of #613. Its first S2-closure test measured S2's first-order error, then set its
threshold to accept it, and so passed exactly the defect #616 later found. A bound derived from the
camera's resolution failed it 28×.

## References

A reference is `{file, symbol?, case?}`. The file must exist; a `symbol` must occur in its code as
Expand Down
4 changes: 4 additions & 0 deletions docs/methodology/specs/euroc-consistency-gate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,10 @@ system_invariants:
- id: s2_closure
text: "the shipped S2, fed the synthesized IMU noise-free, closes on the truth within what the camera resolves (a tenth of 0.5 px over a 0.55 s clone window: 1.1e-4 rad, 0.22 mm at 2 m), at second order"
check: {file: tests/tools/gt_spline.cpp, case: "gt spline: S2 closure, the shipped S2 within what the camera resolves"}
bound:
value: "1.09e-4 rad rotation, 0.22 mm position at 2 m, over a 0.55 s clone window"
consumer: run_filter
derivation: "the backend's update resolves a clone pose only to the track noise, 0.5 px at fx = 458.654 (1.09e-3 rad); propagation error must stay a tenth of that over the clone window so it never competes with the measurement, and the same angle at a 2 m landmark depth is 0.22 mm"
status: failing
note: "the shipped S2 holds each interval's end sample: 3.1e-3 rad (28× the bound) at first order on a 1.3 rad/s roll/pitch flight; averaging the two samples meets the rotation bound at second order, the position still needs the integrator's fix (#616). The test is a [!shouldfail] tripwire until #616"
- id: same_gate
Expand Down
Loading
Loading