Skip to content

fix: close pre-pilot correctness and provenance invariants - #13

Merged
downwithbgp merged 14 commits into
mainfrom
session-57-pre-pilot-invariant-closure
Aug 5, 2026
Merged

downwithbgp merged 14 commits into
mainfrom
session-57-pre-pilot-invariant-closure

Conversation

@downwithbgp

Copy link
Copy Markdown
Owner

fix: close pre-pilot correctness and provenance invariants

Closes the demonstrated pre-pilot invariant defects from the Session 56 Wirthian design-recovery audit (F-1..F-6, F-8). No new architecture, no features, no evidence changes.

Findings addressed

F-1 — Continuity-gate bypass (characterization). A reachable assessment branch could bypass the continuity gate when no findings were supplied: derive_verdict returned the empty-finding fallback (e.g. NoObservableBgpImpact) before checking unknown continuity / session resets. Intended ordering: the eligibility/continuity gate executes before any result derivation from finding cardinality; an empty finding set cannot imply "no route-state change" without a gap-free UPDATE sequence. The gate now runs first; the empty-finding fallback is only reachable with established continuity. Historical-impact inspection: all 12 tracked report.json files were statically checked — zero have the unknown-continuity + empty-findings combination, so no tracked run was demonstrated affected and no historical result was changed or rederived (F-10: no tracked run demonstrated affected).

F-2/F-3 — Artifact resolution. Four resolvers existed (run page, workbench coverage with only 4 hardcoded case-study dirs, demo fallback with unvalidated root.join, orphan audit). Structurally confirmed: 6 of 118 demo-catalog artifact rows were invisible to the workbench resolver. All consumers now use the single resolve_artifact; is_safe_relative_path is the one containment primitive (rejects empty, absolute, parent traversal, drive-letter/UNC prefixes, backslash separators, Windows trailing-dot/space hazards); resolve_artifact adds canonical containment so a symlink escaping the root is not served. All 118 tracked artifact rows still resolve; Git-checkout and packaged-source resolution agree.

F-4 — Open-event cutoff readiness. A Ready plan for an open event could be stored without a reviewed cutoff (import path) and an open manifest with a non-empty declared end skipped every cutoff gate. Now: Manifest::validate rejects open manifests without analysis_end_utc; build_plan_record classifies open-without-cutoff as Blocked; queue and worker require the cutoff regardless of any declared end (legacy invalid plans cannot reach source access); import requires recorded cutoff provenance (sidecar cutoff_provenance or analyst note) and never treats source fetch time as the cutoff; the cutoff participates in the canonical plan hash.

F-5 — Standalone analyze scope boundary. Decision: project scope is a universal first-party execution policy (docs/DOMAIN.md already documented CLI-wide loading and re-check before source access). cmd_analyze now loads the policy (fail closed on invalid), applies analyze_scope_block (exact-normalized, same semantics as queue/worker) before any planning or source access, exits EXIT_ANALYSIS_BLOCKED, writes no outputs, and warns when the policy file is absent. No exclusions changed; scope remains policy, not auth/sandboxing.

F-6 — Observed result vs expectation assessment. Stored verdict strings carry mixed legacy semantics (e.g. ExpectedLossOfReachability) and were exposed verbatim in observed-result positions. Projection-layer correction (no migration, no historical rewrite): the run API exposes structured observed_result {kind,label} and expectation_assessment {kind,label}; unrecognized stored values project as a neutral label; the raw verdict/assessment fields remain as documented legacy fields; case-study runs expose the projected labels.

F-8 — Documentation CI. The CHANGELOG "Session 55" heading tripped the intentional session-narrative audit (CHANGELOG.md is normative; main CI was already red at 92f83d89). The entry was rewritten in product/change language (release-facing content preserved), the render regenerated, and a regression test added. scripts/audit-docs.sh is green.

Also

  • Status-boundary matrix pinned with negative tests (open event ≠ incomplete analysis, no-change ≠ continuity failure, insufficient visibility ≠ no-change, Ready ≠ in-scope, etc.).
  • Invariant register updated (6 invariants now enforced) and counts reconciled (the session-56 declared totals were arithmetically wrong); a deterministic parse test enforces the arithmetic.
  • Incremental Wirthian design checksum written for b2fde81..HEAD (docs/audits/2026-08-pre-pilot-invariant-closure.md); Session 56 audit gains a follow-up status table.
  • Evaluation kit verified unaffected: answer-key regeneration is deterministic and byte-identical to the tracked key.

Constraints honored

  • Canonical evidence unchanged · source snapshots unchanged · reviewed case-study interpretations unchanged · project-scope exclusions unchanged · network profiles unchanged · no schema/migration changes.
  • No analysis rerun; no source contacted (GRNOC, RouteViews, RIPE RIS, PeeringDB, RIR); no archive acquired; no worker executed against a real plan.
  • Zero external evaluation sessions; pilot registry unchanged.
  • No runtime files, databases, caches, or screenshots committed.

Not claimed: no historical verdict was proven wrong, no formal verification, no claim that all invalid states are impossible, no external validation.

Gates

cargo fmt --check ✓ · cargo test 1517 passed ✓ · cargo test --release ✓ · cargo test --doc ✓ · clippy -D warnings ✓ · rustdoc -D warnings ✓ · cargo deny licenses/bans ✓ · cargo package ✓ · scripts/audit-docs.sh ✓ · demo init/verify ✓ · evaluator bootstrap ✓ · answer-key generation + drift ✓ · evaluation pack ✓ · read-only DB hash check ✓ · project-scope validation/audit ✓ · artifact audit ✓ · internal-link audit ✓ · secret scan ✓ · tracked-runtime audit ✓ · clean-clone + packaged-source verification ✓

- Rewrite the changelog entry in product/change language; the 'Session 55'
  heading tripped the intentional session-narrative audit rule (CHANGELOG.md
  is a normative current document). Release-facing content is preserved.
- Regenerate the repository-truth audit render (audit-date rollover).

The documentation CI failure predated Session 57: the 'Session 55' heading
was introduced by commit 92f83d8 (Session 55 changelog entry) and main CI
for 92f83d8 was already red.
A reachable assessment branch could bypass the continuity gate when no
findings were supplied: derive_verdict returned the empty-finding
fallback verdict (e.g. NoObservableBgpImpact) before checking
any_unknown_continuity / session resets, so a run with UPDATE archive
gaps and zero transitions would overstate 'no route-state change'.

The gate now executes before result derivation from finding cardinality;
the empty-finding fallback is only reachable with established continuity.
Existing variants represent the correct state (InsufficientVisibility),
so no new result variant was introduced.

Characterization tests (continuity_failure_precedes_empty_finding_fallback,
empty_findings_do_not_imply_no_change_without_continuity,
continuity_gate_decision_table, etc.) initially demonstrated the bug and
now protect the intended ordering. No tracked historical result was
changed or rederived: static inspection of all 12 tracked report.json
files found zero runs with the unknown-continuity + empty-findings
combination (F-10: no tracked run demonstrated affected).
…, F-3)

F-2: the run-page archive-coverage lookup used a hand-rolled resolver
with only four hardcoded case-study dirs, which missed every row under
case-studies/indiana-gigapop-smithville-2026/out (structurally confirmed:
6 of 118 demo-catalog artifact rows were invisible to it). It now uses
artifact_path::resolve_artifact, the single shared resolver.

F-3: the demo resolver fell back to an unvalidated root.join(rel),
and publish.rs orphan reconciliation joined data/runs rows without
validation. Both now enforce the shared lexical containment primitive.

artifact_path.rs now provides:
- is_safe_relative_path: the single containment primitive (rejects
  empty, absolute, parent traversal, drive-letter/UNC prefixes, and
  backslash separators on every platform);
- resolve_artifact: lexical containment + canonical containment for
  existing candidates (a symlink escaping the root is not served);
  missing files remain distinct from invalid paths.

Document serving uses the same primitive plus its canonical check.
All 118 demo-catalog artifact rows still resolve; 13 resolver tests
(including symlink-escape, consumer-agreement, demo/workbench
equivalence, and Git/package equivalence) pass.
…F-4)

An open event could previously carry a Ready plan without an explicit
reviewed cutoff: the import path stored Ready without checking
analysis_end_utc, and an open manifest with a non-empty declared end
skipped every cutoff gate.

Now:
- Manifest::validate rejects open manifests without a non-empty
  analysis_end_utc (internally contradictory reviewed input).
- build_plan_record classifies open-without-cutoff as Blocked
  (MissingAnalysisEndForOpenTicket) as defense in depth.
- import_one rejects open manifests whose cutoff has no recorded
  provenance (snapshot sidecar cutoff_provenance or an analyst note
  naming the cutoff); the source fetch time is never an implicit cutoff.
- Queue-time and worker-time validation require the cutoff for open
  events regardless of any declared end, so a legacy invalid plan
  cannot reach source access.
- CanonicalPlan.analysis_end uses the reviewed cutoff for open events:
  the cutoff now participates in the plan hash (it is execution-
  relevant).

All tracked manifests already follow the convention (the only open
manifest carries analysis_end_utc and the sidecar cutoff_provenance);
no canonical evidence was changed.
The standalone 'inim analyze' CLI never loaded ProjectScope, so an
excluded subject could be analyzed through a first-party command
(contradicting docs/DOMAIN.md, which documents scope as loaded by the
CLI and rechecked before source access).

Decision: project scope is a universal first-party execution policy
(Outcome A). cmd_analyze now loads the reviewed policy (fail closed on
invalid policy), and analyze_scope_block rejects an excluded source
record, reviewed entity label, or reviewed origin ASN with the same
exact-normalized semantics as the queue/worker checks — before any
planning, broker discovery, archive acquisition, or MRT parsing, and
without writing outputs (a blocked analyze exits EXIT_ANALYSIS_BLOCKED).
Missing config still yields the documented empty all-Included policy.

DOMAIN.md now states the boundary explicitly. No exclusions changed;
scope remains policy, not authentication or sandboxing.
…ons (F-6)

Stored verdict strings carry mixed legacy semantics: some name
expectations (e.g. ExpectedLossOfReachability, LessImpactThanExpected)
and were previously exposed verbatim in the observed-result position of
the run API and web projections.

Projection-layer correction (no migration, no historical rewrite):
- present_run_verdict: an unrecognized stored value no longer appears in
  the observed-result slot; it projects as the neutral
  'Observed result not classified (legacy value)'.
- /api/v1/analyses/{run_id} now exposes structured observed_result
  {kind,label} and expectation_assessment {kind,label} derived from the
  stored verdict; the raw verdict/assessment fields remain for
  compatibility and are documented as legacy.
- /api/v1/case-studies/{slug} runs expose the structured projections
  alongside the legacy raw fields.
- docs/reference/API.md documents the two projections and the legacy
  status of the raw fields.

Tests cover: structured API fields, legacy verdict not overriding the
current projection, historical runs remaining readable, Smithville
insufficient-visibility separate from its assessment, route-change
separate from expectation consistency, and optical applicability not
projected as a route result. No tracked report or run row was changed.
Source lifecycle, plan readiness, project scope, applicability, job
status, run completion, evidence continuity, observed result, and
expectation assessment remain separate axes: open event is not an
incomplete analysis; a no-change result with established continuity is
not a continuity failure; insufficient visibility is not a no-change
result; plan readiness does not imply scope inclusion.
…unts (part 13)

TC-5, AR-1, AR-2, PR-6, SC-3, FR-1 are now enforced with the session-57
enforcement and tests recorded; the previously suppressed claims section
records that AR-1 and TC-5 were resolved (PV-2 remains narrowed: F-7 WAL
sidecars are out of implementation scope).

Counts reconciled: the register's table had 56 rows (the session-56
completion report's totals were wrong — 33/5/1 did not match the
register). Declared counts now match the parsed rows (53 enforced,
3 partially enforced, 0 assumed, 0 claimed, 0 unknown, 56 total), and a
deterministic parse test (tests/invariant_register_test.rs) enforces the
arithmetic going forward.
…up (parts 14-15)

- docs/audits/2026-08-pre-pilot-invariant-closure.md: Wirthian design
  checksum for b2fde81..HEAD covering data structures, semantic identity,
  invariants, algorithms, state transitions, effects, authority
  boundaries, information-loss boundaries, and complexity changes.
- docs/audits/2026-08-wirthian-design-recovery.md: follow-up status table
  for F-1..F-10 (resolved / accepted boundary / remains open).
- Evaluation kit verified unaffected: answer-key regeneration is
  deterministic and byte-identical to the tracked key.
… explicit scope boundary (part 18)

- is_safe_relative_path rejects components ending in '.' or a space:
  Windows path normalization rewrites them ( can become parent
  traversal on Win32), so the lexical-only probe in orphan audit could
  act as an outside-root existence oracle on Windows.
- cmd_analyze warns on stderr when config/project-scope.toml is absent
  under the current directory, so the standalone path never silently
  skips exclusions; DOMAIN.md documents the CWD loading rule.
- cargo fmt; restored the too_many_arguments allow on cmd_analyze;
  kept the pilot-projection test comment incident-neutral.
…ing, changelog entry

- API reference and invariants register corrected to match the actual
  case-study runs array shape (flat projected labels; structured
  {kind,label} only on the run-detail endpoint).
- Invariant register uses date-based attribution (no session narrative
  in the normative register).
- CHANGELOG gains a product-language entry for the pre-pilot invariant
  closure (no session narrative).
@downwithbgp
downwithbgp merged commit 198cbb7 into main Aug 5, 2026
8 checks passed
@downwithbgp
downwithbgp deleted the session-57-pre-pilot-invariant-closure branch August 5, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant