fix: close pre-pilot correctness and provenance invariants - #13
Merged
Merged
Conversation
- Rewrite the changelog entry in product/change language; the 'Session 55' heading tripped the intentional session-narrative audit rule (CHANGELOG.md is a normative current document). Release-facing content is preserved. - Regenerate the repository-truth audit render (audit-date rollover). The documentation CI failure predated Session 57: the 'Session 55' heading was introduced by commit 92f83d8 (Session 55 changelog entry) and main CI for 92f83d8 was already red.
A reachable assessment branch could bypass the continuity gate when no findings were supplied: derive_verdict returned the empty-finding fallback verdict (e.g. NoObservableBgpImpact) before checking any_unknown_continuity / session resets, so a run with UPDATE archive gaps and zero transitions would overstate 'no route-state change'. The gate now executes before result derivation from finding cardinality; the empty-finding fallback is only reachable with established continuity. Existing variants represent the correct state (InsufficientVisibility), so no new result variant was introduced. Characterization tests (continuity_failure_precedes_empty_finding_fallback, empty_findings_do_not_imply_no_change_without_continuity, continuity_gate_decision_table, etc.) initially demonstrated the bug and now protect the intended ordering. No tracked historical result was changed or rederived: static inspection of all 12 tracked report.json files found zero runs with the unknown-continuity + empty-findings combination (F-10: no tracked run demonstrated affected).
…, F-3) F-2: the run-page archive-coverage lookup used a hand-rolled resolver with only four hardcoded case-study dirs, which missed every row under case-studies/indiana-gigapop-smithville-2026/out (structurally confirmed: 6 of 118 demo-catalog artifact rows were invisible to it). It now uses artifact_path::resolve_artifact, the single shared resolver. F-3: the demo resolver fell back to an unvalidated root.join(rel), and publish.rs orphan reconciliation joined data/runs rows without validation. Both now enforce the shared lexical containment primitive. artifact_path.rs now provides: - is_safe_relative_path: the single containment primitive (rejects empty, absolute, parent traversal, drive-letter/UNC prefixes, and backslash separators on every platform); - resolve_artifact: lexical containment + canonical containment for existing candidates (a symlink escaping the root is not served); missing files remain distinct from invalid paths. Document serving uses the same primitive plus its canonical check. All 118 demo-catalog artifact rows still resolve; 13 resolver tests (including symlink-escape, consumer-agreement, demo/workbench equivalence, and Git/package equivalence) pass.
…F-4) An open event could previously carry a Ready plan without an explicit reviewed cutoff: the import path stored Ready without checking analysis_end_utc, and an open manifest with a non-empty declared end skipped every cutoff gate. Now: - Manifest::validate rejects open manifests without a non-empty analysis_end_utc (internally contradictory reviewed input). - build_plan_record classifies open-without-cutoff as Blocked (MissingAnalysisEndForOpenTicket) as defense in depth. - import_one rejects open manifests whose cutoff has no recorded provenance (snapshot sidecar cutoff_provenance or an analyst note naming the cutoff); the source fetch time is never an implicit cutoff. - Queue-time and worker-time validation require the cutoff for open events regardless of any declared end, so a legacy invalid plan cannot reach source access. - CanonicalPlan.analysis_end uses the reviewed cutoff for open events: the cutoff now participates in the plan hash (it is execution- relevant). All tracked manifests already follow the convention (the only open manifest carries analysis_end_utc and the sidecar cutoff_provenance); no canonical evidence was changed.
The standalone 'inim analyze' CLI never loaded ProjectScope, so an excluded subject could be analyzed through a first-party command (contradicting docs/DOMAIN.md, which documents scope as loaded by the CLI and rechecked before source access). Decision: project scope is a universal first-party execution policy (Outcome A). cmd_analyze now loads the reviewed policy (fail closed on invalid policy), and analyze_scope_block rejects an excluded source record, reviewed entity label, or reviewed origin ASN with the same exact-normalized semantics as the queue/worker checks — before any planning, broker discovery, archive acquisition, or MRT parsing, and without writing outputs (a blocked analyze exits EXIT_ANALYSIS_BLOCKED). Missing config still yields the documented empty all-Included policy. DOMAIN.md now states the boundary explicitly. No exclusions changed; scope remains policy, not authentication or sandboxing.
…ons (F-6)
Stored verdict strings carry mixed legacy semantics: some name
expectations (e.g. ExpectedLossOfReachability, LessImpactThanExpected)
and were previously exposed verbatim in the observed-result position of
the run API and web projections.
Projection-layer correction (no migration, no historical rewrite):
- present_run_verdict: an unrecognized stored value no longer appears in
the observed-result slot; it projects as the neutral
'Observed result not classified (legacy value)'.
- /api/v1/analyses/{run_id} now exposes structured observed_result
{kind,label} and expectation_assessment {kind,label} derived from the
stored verdict; the raw verdict/assessment fields remain for
compatibility and are documented as legacy.
- /api/v1/case-studies/{slug} runs expose the structured projections
alongside the legacy raw fields.
- docs/reference/API.md documents the two projections and the legacy
status of the raw fields.
Tests cover: structured API fields, legacy verdict not overriding the
current projection, historical runs remaining readable, Smithville
insufficient-visibility separate from its assessment, route-change
separate from expectation consistency, and optical applicability not
projected as a route result. No tracked report or run row was changed.
Source lifecycle, plan readiness, project scope, applicability, job status, run completion, evidence continuity, observed result, and expectation assessment remain separate axes: open event is not an incomplete analysis; a no-change result with established continuity is not a continuity failure; insufficient visibility is not a no-change result; plan readiness does not imply scope inclusion.
…unts (part 13) TC-5, AR-1, AR-2, PR-6, SC-3, FR-1 are now enforced with the session-57 enforcement and tests recorded; the previously suppressed claims section records that AR-1 and TC-5 were resolved (PV-2 remains narrowed: F-7 WAL sidecars are out of implementation scope). Counts reconciled: the register's table had 56 rows (the session-56 completion report's totals were wrong — 33/5/1 did not match the register). Declared counts now match the parsed rows (53 enforced, 3 partially enforced, 0 assumed, 0 claimed, 0 unknown, 56 total), and a deterministic parse test (tests/invariant_register_test.rs) enforces the arithmetic going forward.
…up (parts 14-15) - docs/audits/2026-08-pre-pilot-invariant-closure.md: Wirthian design checksum for b2fde81..HEAD covering data structures, semantic identity, invariants, algorithms, state transitions, effects, authority boundaries, information-loss boundaries, and complexity changes. - docs/audits/2026-08-wirthian-design-recovery.md: follow-up status table for F-1..F-10 (resolved / accepted boundary / remains open). - Evaluation kit verified unaffected: answer-key regeneration is deterministic and byte-identical to the tracked key.
… explicit scope boundary (part 18) - is_safe_relative_path rejects components ending in '.' or a space: Windows path normalization rewrites them ( can become parent traversal on Win32), so the lexical-only probe in orphan audit could act as an outside-root existence oracle on Windows. - cmd_analyze warns on stderr when config/project-scope.toml is absent under the current directory, so the standalone path never silently skips exclusions; DOMAIN.md documents the CWD loading rule. - cargo fmt; restored the too_many_arguments allow on cmd_analyze; kept the pilot-projection test comment incident-neutral.
…ing, changelog entry
- API reference and invariants register corrected to match the actual
case-study runs array shape (flat projected labels; structured
{kind,label} only on the run-detail endpoint).
- Invariant register uses date-based attribution (no session narrative
in the normative register).
- CHANGELOG gains a product-language entry for the pre-pilot invariant
closure (no session narrative).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix: close pre-pilot correctness and provenance invariants
Closes the demonstrated pre-pilot invariant defects from the Session 56 Wirthian design-recovery audit (F-1..F-6, F-8). No new architecture, no features, no evidence changes.
Findings addressed
F-1 — Continuity-gate bypass (characterization). A reachable assessment branch could bypass the continuity gate when no findings were supplied:
derive_verdictreturned the empty-finding fallback (e.g.NoObservableBgpImpact) before checking unknown continuity / session resets. Intended ordering: the eligibility/continuity gate executes before any result derivation from finding cardinality; an empty finding set cannot imply "no route-state change" without a gap-free UPDATE sequence. The gate now runs first; the empty-finding fallback is only reachable with established continuity. Historical-impact inspection: all 12 trackedreport.jsonfiles were statically checked — zero have the unknown-continuity + empty-findings combination, so no tracked run was demonstrated affected and no historical result was changed or rederived (F-10: no tracked run demonstrated affected).F-2/F-3 — Artifact resolution. Four resolvers existed (run page, workbench coverage with only 4 hardcoded case-study dirs, demo fallback with unvalidated
root.join, orphan audit). Structurally confirmed: 6 of 118 demo-catalog artifact rows were invisible to the workbench resolver. All consumers now use the singleresolve_artifact;is_safe_relative_pathis the one containment primitive (rejects empty, absolute, parent traversal, drive-letter/UNC prefixes, backslash separators, Windows trailing-dot/space hazards);resolve_artifactadds canonical containment so a symlink escaping the root is not served. All 118 tracked artifact rows still resolve; Git-checkout and packaged-source resolution agree.F-4 — Open-event cutoff readiness. A Ready plan for an open event could be stored without a reviewed cutoff (import path) and an open manifest with a non-empty declared end skipped every cutoff gate. Now:
Manifest::validaterejects open manifests withoutanalysis_end_utc;build_plan_recordclassifies open-without-cutoff as Blocked; queue and worker require the cutoff regardless of any declared end (legacy invalid plans cannot reach source access); import requires recorded cutoff provenance (sidecarcutoff_provenanceor analyst note) and never treats source fetch time as the cutoff; the cutoff participates in the canonical plan hash.F-5 — Standalone analyze scope boundary. Decision: project scope is a universal first-party execution policy (docs/DOMAIN.md already documented CLI-wide loading and re-check before source access).
cmd_analyzenow loads the policy (fail closed on invalid), appliesanalyze_scope_block(exact-normalized, same semantics as queue/worker) before any planning or source access, exitsEXIT_ANALYSIS_BLOCKED, writes no outputs, and warns when the policy file is absent. No exclusions changed; scope remains policy, not auth/sandboxing.F-6 — Observed result vs expectation assessment. Stored verdict strings carry mixed legacy semantics (e.g.
ExpectedLossOfReachability) and were exposed verbatim in observed-result positions. Projection-layer correction (no migration, no historical rewrite): the run API exposes structuredobserved_result {kind,label}andexpectation_assessment {kind,label}; unrecognized stored values project as a neutral label; the rawverdict/assessmentfields remain as documented legacy fields; case-study runs expose the projected labels.F-8 — Documentation CI. The CHANGELOG "Session 55" heading tripped the intentional session-narrative audit (CHANGELOG.md is normative; main CI was already red at
92f83d89). The entry was rewritten in product/change language (release-facing content preserved), the render regenerated, and a regression test added.scripts/audit-docs.shis green.Also
b2fde81..HEAD(docs/audits/2026-08-pre-pilot-invariant-closure.md); Session 56 audit gains a follow-up status table.Constraints honored
Not claimed: no historical verdict was proven wrong, no formal verification, no claim that all invalid states are impossible, no external validation.
Gates
cargo fmt --check✓ ·cargo test1517 passed ✓ ·cargo test --release✓ ·cargo test --doc✓ · clippy-D warnings✓ · rustdoc-D warnings✓ · cargo deny licenses/bans ✓ ·cargo package✓ ·scripts/audit-docs.sh✓ · demo init/verify ✓ · evaluator bootstrap ✓ · answer-key generation + drift ✓ · evaluation pack ✓ · read-only DB hash check ✓ · project-scope validation/audit ✓ · artifact audit ✓ · internal-link audit ✓ · secret scan ✓ · tracked-runtime audit ✓ · clean-clone + packaged-source verification ✓