Add severity-override floor + policy audit (M1, v0.17)

[pengfei-threemoonslab]

Summary

• Closes the biggest trust hole in the release gate: today a manifest can downgrade SHIP-POLICY-APPROVAL-MISSING (critical) to info silently. M1 enforces hard severity floors on 16 release-critical built-in checks, surfaces every applied override in a top-of-report policy_audit block, and rejects expired acknowledgements as manifest config errors.
• Bumps report_schema_version 0.16 → 0.17. Breaking for manifests that currently downgrade any of the floored checks below their new floor — failure mode is exit 2 with a routable error message, not silent.
• 12 files changed, +1273 / -20. Two new files: core/severity_overrides.py (331 LOC pure resolver), tests/test_severity_override_floor.py (507 LOC, 27 cases).

Design highlights

Hard floor. CheckMetadata.floor_severity declares the lowest severity an override is allowed to resolve to. Floors set on:

• critical → high (5 checks): SHIP-POLICY-APPROVAL-MISSING, SHIP-ACTION-{FINANCIAL-WRITE-CONTROL-MISSING, DESTRUCTIVE-ROLLBACK-MISSING, WILDCARD-SCOPE, EFFECT-ESCALATED, APPROVAL-REMOVED}
• high → medium (11 checks): SHIP-AUTH-* (4), SHIP-SCOPE-* (2), SHIP-INVENTORY-* (2), SHIP-POLICY-CONFIRMATION-MISSING, SHIP-SIDEFX-IDEMPOTENCY-MISSING

No acknowledgement bypasses the floor.

Tier-crossing acknowledgement. checks.acknowledge_overrides[] is required for any severity downgrade that crosses a tier boundary (critical / high / medium-low). Tier-crossing upgrades and same-tier downgrades never require ack.

Rich override shape. severity_overrides accepts both the legacy scalar form (SHIP-XYZ: medium) and {severity, reason, expires}. Reason flows into the audit row; expires gives reviewers a time-bounded override.

Expiry is hard. Expired ack → manifest config error (exit 2), no advisory-mode bypass.

Audit envelope. New report.policy_audit.severity_overrides_applied[] with {check_id, default_severity, applied_severity, manifest_path, reason, tier_crossed, direction, expires}. Required + non-nullable on the wire (mirrors v0.12 agent_summary pattern).

Architectural notes

• New module core/severity_overrides.py owns the validation policy as a pure function with explicit today= injection. Lets tests pin the date without monkey-patching.
• Legacy apply_severity_overrides(findings, dict[str, Severity]) signature unchanged — direct callers in tests/test_findings.py and tests/test_policy_packs.py keep working byte-for-byte.
• Resolver runs up front in cli/scan.py; the mutation pass below only ever sees a manifest that has passed policy validation. Failing fast = failing routable.
• AgentsShipgateManifest.severity_overrides() still returns the flat scalar projection; new severity_override_entries() returns the rich shape and acknowledge_overrides() returns the ack list.

Files touched

Layer	File	What changed
Domain	core/models.py	report_schema_version: "0.17", CheckMetadata.floor_severity + validator, new SeverityOverrideAuditEntry + PolicyAudit, ReadinessReport.policy_audit
Domain	core/severity_overrides.py (new)	Pure resolver with floor/tier/ack/expiry logic
Domain	core/findings.py	build_report(policy_audit=...) kwarg
Config	config/schema.py	SeverityOverrideEntry, OverrideAcknowledgement, scalar back-compat coercion, duplicate-ack rejection
Checks	checks/registry.py	floor_severity declared on 16 built-ins
CLI	cli/scan.py	resolve_severity_overrides wired before mutation; audit threaded into build_report
Render	report/markdown.py	New ## Policy Audit section between Release Decision and Summary
Render	ci/github_summary.py	One-liner: Policy audit: N override(s) · K downgrade · J tier-crossed
Schema	scripts/generate_schemas.py	Marks policy_audit required + non-nullable on the wire
Docs	STABILITY.md	New stable field + "Severity-override floor" section
Docs	CHANGELOG.md	M1 entry under Unreleased with breaking-change disclosure
Tests	tests/test_severity_override_floor.py (new)	27 cases

Test plan

• python scripts/generate_schemas.py — regenerate docs/report-schema.v0.17.json + refresh docs/checks.json + docs/manifest-v0.1.json. The generator already knows how to mark policy_audit required + non-nullable; this PR updates the generator but leaves schema file generation to the developer because the sandbox has no Python runtime.
• pytest tests/test_severity_override_floor.py -v — new test file (27 cases).
• pytest tests/test_findings.py tests/test_config.py tests/test_scan.py tests/test_policy_packs.py -v — regression on the legacy paths that bypass the resolver and call apply_severity_overrides directly with a scalar dict. Should pass byte-for-byte.
• ruff check src tests — lint.
• Manual smoke: scan samples/support_refund_agent and confirm report.policy_audit.severity_overrides_applied exists as an empty array (no overrides in that fixture).
• Manual breaking-change smoke: add checks.severity_overrides: SHIP-POLICY-APPROVAL-MISSING: info to a manifest, confirm exit 2 with the floor error message.
• Manual tier-crossing smoke: add SHIP-AUTH-MANIFEST-BROAD-SCOPE: medium without an ack, confirm exit 2 with the tier-boundary error message. Add the matching acknowledge_overrides entry, confirm scan succeeds and the audit row carries the reason.

Out of scope

• Schema file generation. The CHANGELOG explicitly notes the regen step. Doing it in this PR would require running Python in a non-pinned environment; the existing generator script already handles the regen correctly and the test plan calls it out.
• The other M-items from the v2 trust-hardening plan (M2 baseline integrity, M3 no-import invariant, M4 schema round-trip, M5 plugin validation, M6 --json-summary, M7 agent-mode unification, M8 truth-table doc). Each lands in its own PR; this one is the trust-spine pivot.

🤖 Generated with Claude Code

[pengfei-threemoonslab]

Pushed fix commit 5148e21 addressing all four review findings. Full pytest passes (1122 passed, 3 skipped) and ruff check . is clean.

P1.1 — schema artifacts. Ran scripts/generate_schemas.py to write docs/report-schema.v0.17.json and refresh docs/checks.json + docs/manifest-v0.1.json. Bumped v0.16 → v0.17 across every surface that referenced it: .well-known/agents-shipgate.json, README (3 callsites), AGENTS.md (3 callsites), docs/INDEX.md, docs/agent-contract-current.md (3 callsites), docs/examples.md, docs/autofix-policy.md, llms.txt (2 callsites), skills/agents-shipgate/SKILL.md. v0.16 moves to the frozen-reference list in each. Also regenerated llms-full.txt and the four samples/*/expected/report.json golden fixtures so they carry the new schema version.

P1.2 — policy-pack rule regression. Extended resolve_severity_overrides with extra_known_check_defaults: dict[str, Severity] | None. cli/scan.py now passes {resolved.rule.id: resolved.rule.severity for ... in policy_packs.rules}, so manifests overriding policy-pack rule IDs no longer fail as "unknown check_id". Policy-pack rules have floor=None by design (floors are a built-in trust contract). Updated the existing tests/test_policy_packs.py fixture — the exact high → medium silent-downgrade pattern M1 closes — to carry an acknowledge_overrides entry. Added 4 new test cases covering policy-pack rule ID acceptance, tier-crossing semantics, same-tier passthrough, and the ack path.

P2.3 — rich-form expiry was advisory. Added _enforce_override_expiry() parallel to _enforce_ack_expiry(). Same hard contract: exit 2 on/past the date, no advisory bypass. Three new test cases (expired, expires-today, expires-tomorrow).

P2.4 — broken test fixtures. The two flagged cases overrode SHIP-SCHEMA-MISSING-BOUNDS (default high) → medium and called it "same tier" — but high → medium IS tier-crossing (high tier → normal tier). Swapped to SHIP-DOC-MISSING-DESCRIPTION (medium) → low, which is genuinely same-tier. Ruff import-order auto-fixed.

Collateral fix. report/tool_surface_diff.py:418 iterated manifest.checks.severity_overrides values expecting scalars, but they're now SeverityOverrideEntry objects after the load-time validator coerces. Extracted entry.severity for the hash/summary so the diff stays stable for repos that didn't add reason/expires.

Schema-generator change worth a second look: I added a manifest-schema override in scripts/generate_schemas.py::write_manifest_schema so checks.severity_overrides accepts both the legacy scalar form and the rich SeverityOverrideEntry form. Without this the Pydantic-autogenerated schema only sees the post-coercion type and would reject every legacy scalar manifest — tests/test_config.py::test_manifest_examples_validate_against_generated_schema caught it.

[pengfei-threemoonslab]

Pushed 88551a2 (after force-push following rebase onto current main). Two findings addressed.

P1.1 — v0.17 collision with M8 merged to main. Rebased onto b4aef44 (M8 release_decision.contribution_rules[]). Chose to combine M1 + M8 under a single v0.17 wire contract rather than bump M1 to v0.18 — both are additive top-level fields under the same trust-hardening pass, so one bump is honest. Regenerated docs/report-schema.v0.17.json from the merged source (now contains both PolicyAudit defs and the M8 ContributionRule defs) and ran scripts/generate_schemas.py to refresh docs/checks.json + docs/manifest-v0.1.json. Resolved text-doc conflicts in README, AGENTS.md, SKILL.md, docs/INDEX.md, docs/examples.md to mention both v0.17 features wherever the original docs framed v0.17 around just one. Regenerated llms-full.txt and the four samples/*/expected/report.json golden fixtures so they reflect the combined schema.

P1.2 — action-surface policy severity bypass. Real and important. The resolver previously compared overrides against CheckMetadata.default_severity (catalog static), but SHIP-ACTION-POLICY-VIOLATION findings emit at the manifest-declared action_surface.policies[].severity per action_surface_diff.py:228. So a severity: critical policy with override high would compare against the catalog's high and silently bypass the critical → high tier gate.

Fix: extended extra_known_check_defaults semantics so it carries the effective default severity, not just outside-catalog defaults. For check IDs in the catalog, the resolver takes max(catalog default, supplied default) for tier-crossing and audit purposes. cli/scan.py aggregates the strongest declared severity across manifest.action_surface.policies[] and passes it as extra_known_check_defaults["SHIP-ACTION-POLICY-VIOLATION"]. The same dict still carries policy-pack rule defaults — both use cases are unified under "effective default per check ID, take the stronger".

Three new test cases in tests/test_severity_override_floor.py: tier-crossing rejection without ack, tier-crossing with ack carries default_severity: critical (the effective default) in the audit row, and the never-weaken invariant (a manifest-declared medium for an action policy doesn't de-escalate the catalog high).

The audit row now reports the effective default_severity (critical, in the bypass scenario), not the catalog static (high), so the policy_audit block can no longer disagree with the finding's evidence about what the user actually downgraded from.

STABILITY.md "Severity-override floor" gains a new "Dynamic-severity check classes" paragraph documenting the contract for SHIP-ACTION-POLICY-VIOLATION and policy-pack rules.

Final state after both fixes: 1150 passed, 3 skipped; ruff clean.