Skip to content

Add release_decision.contribution_rules[] audit (v0.17)#81

Merged
pengfei-threemoonslab merged 2 commits into
mainfrom
claude/heuristic-panini-4d4332
May 16, 2026
Merged

Add release_decision.contribution_rules[] audit (v0.17)#81
pengfei-threemoonslab merged 2 commits into
mainfrom
claude/heuristic-panini-4d4332

Conversation

@pengfei-threemoonslab
Copy link
Copy Markdown
Contributor

Summary

  • Adds release_decision.contribution_rules[] — a deterministic per-finding audit of how each finding contributed to the release decision. Exactly one row per report.findings entry, including suppressed. The (rule, category) pair documents which branch of the gate fired.
  • Bumps report_schema_version 0.160.17. Documents the existing v0.8 classification as the new "Release decision truth table" in STABILITY.md — 10 rows + prose explainers covering baseline asymmetry and the exit-code-vs-decision split.
  • No semantic change. decision, blockers[], review_items[], fail_policy.exit_code, and strict-mode exit are byte-identical to v0.16. The audit reflects existing behavior, it does not modify it.

What's in the audit

{
  "finding_id": "fp_f092940f62fbb012",
  "fingerprint": "fp_f092940f62fbb012",
  "check_id": "SHIP-POLICY-APPROVAL-MISSING",
  "category": "blocker",
  "rule": "policy_block_new",
  "rationale": "blocks_release=true and baseline_status=new; explicit policy blocker."
}

category enum: blocker | review_item | excluded
rule enum: policy_block_new | severity_block_new | policy_baseline_accepted | severity_baseline_accepted | review_required | sub_threshold | suppressed

Both enums inlined in docs/report-schema.v0.17.json so consumers can switch on them.

Why

Per the architecture review's M8 trust-hardening item: a reviewer reading the JSON should be able to predict the gate outcome for any finding without re-deriving the decision logic. The truth table in STABILITY.md is the contract; contribution_rules[] is the runtime witness that the gate followed it.

The audit is purely additive — defaults to [] for legacy reports loaded via explain-finding, so consumers never need an existence check. No new flags, no new exit codes, no behavior change.

Test plan

  • All 1099 tests pass (was 1086 before; +13 truth-table cases + 2 invariants + 1 schema lock = 16 new tests)
  • All 20 pre-existing test_release_decision.py tests still pass — no semantic regression
  • ruff check clean on changed files
  • python scripts/generate_schemas.py is idempotent (running twice produces no diff)
  • All 4 sample expected reports regenerated; len(contribution_rules) == len(findings) invariant verified across all samples
  • Schema lock test (test_v17_schema_requires_contribution_rules) freezes both enums and required keys
  • CI passes

Files

Layer Files
Domain src/agents_shipgate/core/models.py (new ContributionRule, ContributionRuleName, field on ReleaseDecision, schema bump)
Decision src/agents_shipgate/ci/release_decision.py (per-finding audit emission, helpers, comments documenting that the branching mirrors v0.16 exactly)
Schema scripts/generate_schemas.py (pin required keys), docs/report-schema.v0.17.json (regenerated)
Tests tests/test_release_decision.py (+16 tests), tests/test_reports.py (v0.17 lock test), tests/test_provenance_kind.py, tests/test_public_surface_contract.py (v0.16 → legacy pattern)
Contract STABILITY.md (truth table + stable field), CHANGELOG.md, .well-known/agents-shipgate.json
Public surface (0.16 → 0.17) README.md, AGENTS.md, llms.txt, llms-full.txt (regenerated), skills/agents-shipgate/SKILL.md, docs/{INDEX,examples,agent-contract-current,autofix-policy,baseline,report-reading-for-agents}.md
Samples samples/{support_refund_agent,simple_openai_api_agent,simple_langchain_agent,simple_crewai_agent}/expected/report.json

🤖 Generated with Claude Code

pengfei-threemoonslab and others added 2 commits May 15, 2026 22:26
Adds a deterministic per-finding audit of how each finding contributed
to the release decision. Exactly one row per report.findings entry,
including suppressed; the (rule, category) pair documents which branch
of the gate fired.

Bumps report_schema_version 0.16 -> 0.17. Documents the existing v0.8
classification as the new "Release decision truth table" in
STABILITY.md (10 rows + prose explainers covering baseline asymmetry
and exit-code-vs-decision split).

No semantic change: decision, blockers[], review_items[],
fail_policy.exit_code, and strict-mode exit are byte-identical to
v0.16. The audit reflects existing behavior, it does not modify it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P2 #1 — build_release_decision crashed on Finding(id=None) because
ContributionRule.finding_id is required-as-string. Direct/internal
callers (test fixtures, plugin checks emitting Findings before
assign_finding_ids, explain-finding rebuilding from a stripped report)
hit a Pydantic ValidationError. Fix the _rule helper to fall back
through finding.fingerprint to finding.check_id, both of which are
guaranteed-present strings on a valid Finding. Adds an explicit
regression test exercising both the fingerprint and check_id fallbacks.

P2 #2 — sample expected reports carried /private/tmp/shipgate_regen/
paths in generated_reports[] from the v0.17 regen. Restored to the
original repo convention (relative paths like expected/report.md) by
running scan from each sample dir with output_dir='expected'.

P3 #3 — added release_decision.contribution_rules[] to the canonical
agent surface: a one-line entry in docs/agent-contract-current.md
"Read these first for release gating", a new entry in
contract.MANUAL_REVIEW_SIGNALS so contract --json exposes it (with a
comment explaining why reviewers triaging review_items want the audit
row), and the regenerated llms-full.txt that aggregates the contract
doc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pengfei-threemoonslab pengfei-threemoonslab merged commit b4aef44 into main May 16, 2026
1 check passed
@pengfei-threemoonslab pengfei-threemoonslab deleted the claude/heuristic-panini-4d4332 branch May 16, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant