Skip to content

feat(scoring): impact-weighted urgency, calendar deadlines, work queue#2

Draft
IAwiz87 wants to merge 1 commit into
mainfrom
risk-scoring-gaps-1-7
Draft

feat(scoring): impact-weighted urgency, calendar deadlines, work queue#2
IAwiz87 wants to merge 1 commit into
mainfrom
risk-scoring-gaps-1-7

Conversation

@IAwiz87

@IAwiz87 IAwiz87 commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Summary

Closes gaps #1-#7 from the risk-scoring impact-weighted review, and reworks ROADMAP.md to fold in the resulting features. This is a draft PR for review — nothing here is merged yet.

Gaps 1-7: what changed and why

# Gap Fix
1 Urgency saturation bug migration_urgency_score is now round(0.8 × inherent_risk_score + 0.2 × deadline_pressure_score) instead of min(100, risk + deadline_weight). The old model made every critical-tier asset (risk ≥ 80) collapse to urgency = 100 regardless of deadline, since 80 + up to 20 always hit the ceiling. Two critical assets — one overdue, one 3 years out — now score 97 vs. 85 instead of tying at 100.
2 Deadlines went stale Deadlines can now come from a calendar-anchored regulatory_category (mapped to CNSA 2.0 dates: 2030 for signing/networking, 2033 for everything else) recomputed from time.now_ns() on every evaluation, instead of only a hand-entered migration_deadline_months that silently rots. Explicit migration_deadline_months always overrides the calendar value.
3 Overdue assets rejected as invalid Negative migration_deadline_months is now valid input (schema minimum: 0 removed), representing an already-missed deadline rather than an error. New valid_number() validator allows negatives; secrecy_lifetime_years still uses the non-negative-only validator.
4 Classification/impact double-counting New classification_impact_divergence() / needs_divergence_review flags cases where data_classification and impact disagree by 2+ tiers (e.g. public classification, mission-critical impact) — reported for human review, never changes the score.
5 No documented rationale New docs/SCORING_METHODOLOGY.md — ties every weight and bucket boundary back to NIST SP 800-30, CNSA 2.0, and NIST IR 8547, with worked examples.
6 No ranked work queue New migration_work_queue — a single fully-ordered list across all valid assets: risk desc → urgency desc → remediation effort as tiebreaker-only → asset_id as final tiebreaker. This is the previously-unscheduled "PQC Migration Priority Matrix generator."
7 Impact tier could be manually under-rated New dependent_asset_count field + impact_may_be_underrated() flags high fan-in assets (e.g. a signing key backing 40 certs marked "operational") — advisory only, never overrides the manual impact value.

A real bug found during the walkthrough

impact_may_be_underrated() and needs_divergence_review() were originally written as partial rules (undefined when false), then used directly as values inside scored_inventory's entry object. In Rego, an undefined value poisons the whole object literal — this was silently dropping entire asset entries whenever either flag was false, which is most of the time. Fixed by making both total functions with else := false. Caught by testing every combination, not by inspection.

Testing

  • opa test policies/57/57 passing (23 in risk_score_test.rego, up from the original 8; rest of the suite unchanged).
  • opa check --strict policies/ → clean.
  • opa fmt -w applied to both scoring files — formatting-only, re-confirmed 57/57 after.
  • check-jsonschema passes on all three example inventories, including existing ones (mixed-platform-inventory.json, scoring-ready-inventory.json), confirming backward compatibility.
  • New examples/inventory/calendar-anchored-inventory.json added and validated end-to-end; a real opa eval run against it (not mocked) showed ~42 months remaining for a software_firmware_signing asset, consistent with the 2030 CNSA 2.0 date.
  • Confirmed no other policy file references quantumforge.scoring — this is a self-contained, low-blast-radius change.

Schema changes (schemas/crypto-inventory.schema.json)

  • Removed minimum: 0 from migration_deadline_months.
  • Added optional regulatory_category (enum) and dependent_asset_count (integer) fields.
  • schema_version deliberately left at 1.0.0 — additive and backward-compatible.

Documentation updated

  • New: docs/SCORING_METHODOLOGY.md — full rationale doc (gap #5).
  • docs/GOVERNANCE.md — risk-model section rewritten to describe the blended urgency formula, calendar-anchored deadlines, migration_work_queue, and the two new advisory flags.
  • docs/INVENTORY.md — enrichment field list updated to include regulatory_category and dependent_asset_count.
  • docs/PQC_Readiness_Program_Build_Guide.md — added a note flagging that its original Phase 2 code snippet is historical and pointing to the current implementation, without rewriting the historical narrative itself.
  • README.md — test count in the Validation Status table updated from 42/42 to 57/57.

ROADMAP.md rework

  • Restored old-roadmap-style concrete engineering checklists (file paths, function names) under each Priority 1/2/3 item, on top of the new principle-based structure.
  • Marked the "migration work queue" / Migration Priority Matrix generator item as implemented, linking to migration_work_queue.
  • Restored the "Other Build Guide Items Not Yet Scheduled" section with the two items not already covered elsewhere: CBOM generation in CI and executive crypto-census briefing generator. Noted that "automated CMVP gap analysis" now lives under Priority 2's CMVP watchlist bullet, and the Migration Priority Matrix idea is now implemented rather than unscheduled.
  • Restored Priority 3's dropped "every new module should follow this pattern" checklist as its own subsection alongside the newer generic contribution rule.

Files touched

 README.md                                           |   2 +-
 ROADMAP.md                                          |  81 +++++--
 docs/GOVERNANCE.md                                  |  22 ++-
 docs/INVENTORY.md                                   |   2 +-
 docs/PQC_Readiness_Program_Build_Guide.md           |   3 +
 policies/scoring/risk_score.rego                    | 257 +++++++++++++++++--
 policies/scoring/risk_score_test.rego               | 189 ++++++++++++--
 schemas/crypto-inventory.schema.json                |  11 +-
 docs/SCORING_METHODOLOGY.md                         | new file
 examples/inventory/calendar-anchored-inventory.json | new file

🤖 Draft PR opened for review. Holding off on any further merge steps until reviewed.

Closes gaps #1-#7 from the risk-scoring impact-weighted review, and
reworks ROADMAP.md to fold in the resulting features.

Scoring changes (policies/scoring/risk_score.rego):
- migration_urgency_score is now round(0.8*inherent_risk_score +
  0.2*deadline_pressure_score) instead of min(100, risk + deadline_weight).
  The old additive-and-clamp model collapsed differentiation for every
  asset already at risk >= 80, since 80 + up to 20 always hit the 100
  ceiling. (gap #1)
- deadline_pressure_score is a new, independent 0-100 scale with its own
  7-bucket table, replacing deadline_weight's 3 buckets. (gap #1)
- Deadlines can now be calendar-anchored via an optional
  regulatory_category mapped to CNSA 2.0 dates and recomputed from
  time.now_ns() on every evaluation, instead of only a hand-entered
  migration_deadline_months that can go stale. Explicit
  migration_deadline_months always takes precedence. (gap #2)
- Negative migration_deadline_months is now valid input (an overdue
  asset), not rejected. New valid_number() validator allows negatives;
  schema's minimum:0 removed. (gap #3)
- classification_impact_divergence() / needs_divergence_review flags
  assets where data_classification and impact diverge by 2+ tiers, for
  human review only -- never changes the score. (gap #4)
- migration_work_queue: a single fully-ordered list across all valid
  assets (risk desc, urgency desc, remediation_effort as tiebreaker-only,
  asset_id as final tiebreaker). Implements the previously-unscheduled
  "PQC Migration Priority Matrix generator." (gap #6)
- dependent_asset_count + impact_may_be_underrated() advises reviewers
  when a high-fan-in asset's manually assigned impact tier looks too low
  -- advisory only, never overrides the manual value. (gap #7)
- Bug fix found during testing: impact_may_be_underrated() and
  needs_divergence_review were partial rules (undefined when false) used
  directly as values in scored_inventory's entry object -- undefined
  poisons Rego object literals, silently dropping entries. Fixed by
  making both total functions with `else := false`.

Docs (gap #5 + consistency updates):
- New docs/SCORING_METHODOLOGY.md: NIST SP 800-30 / CNSA 2.0 / NIST IR
  8547 rationale for every weight and bucket boundary, with worked
  examples.
- docs/GOVERNANCE.md: risk-model section rewritten for the new formula,
  calendar deadlines, migration_work_queue, and the two advisory flags.
- docs/INVENTORY.md: enrichment field list updated with
  regulatory_category and dependent_asset_count.
- docs/PQC_Readiness_Program_Build_Guide.md: added a note flagging the
  Phase 2 code snippet as historical, pointing to the current model.
- README.md: OPA test count in Validation Status updated 42/42 -> 57/57.

Schema (schemas/crypto-inventory.schema.json):
- Removed minimum:0 from migration_deadline_months.
- Added optional regulatory_category (enum) and dependent_asset_count
  (integer) fields. schema_version kept at 1.0.0 (additive, backward
  compatible -- verified against existing example files).

New example: examples/inventory/calendar-anchored-inventory.json,
validated against the updated schema and against opa eval.

Roadmap (ROADMAP.md):
- Merged the new principle-based structure with the old roadmap's
  concrete engineering checklists (file paths, function names) per
  Priority 1-3 item.
- Marked the migration-priority-matrix generator item as implemented,
  linking to migration_work_queue.
- Restored "Other Build Guide Items Not Yet Scheduled" with the two
  items not covered elsewhere: CBOM generation in CI, and executive
  crypto-census briefing generator.
- Restored Priority 3's per-module engineering checklist as its own
  subsection alongside the newer generic contribution rule.

Testing:
- opa test policies/ -> 57/57 passing (23 in risk_score_test.rego, up
  from 8 in the previous version of the file).
- opa check --strict policies/ -> clean.
- opa fmt -w applied to both scoring files.
- check-jsonschema passes on all three example inventories, confirming
  backward compatibility with pre-existing example files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant