Skip to content

Persist Shadow maturity evidence for reviewed improvements - #84

Draft
hongmaple0820 wants to merge 19 commits into
feat/failure-learning-review-lifecyclefrom
feat/persistent-shadow-evidence
Draft

hongmaple0820 wants to merge 19 commits into
feat/failure-learning-review-lifecyclefrom
feat/persistent-shadow-evidence

Conversation

@hongmaple0820

@hongmaple0820 hongmaple0820 commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Add a persistent RuleMaturity evidence store under .scale/evals/rule-maturity/.
  • Allow only accepted Failure Learning candidates to initialize non-blocking Shadow maturity records with rollback instructions.
  • Verify the original Failure Replay artifact still exists and parses with the expected failure ID before Shadow initialization; fail closed if provenance is missing.
  • Require shadow-hit to reference an existing regular evidence file inside the project directory (10 MiB maximum), hash it with SHA-256, reject paths escaping the project, and reject duplicate content by digest.
  • Enforce sha256:<64 lowercase hex> evidence IDs at the store API boundary and serialize writes with a fail-closed per-record lock to prevent concurrent lost updates.
  • Add store and CLI tests for lock contention, malformed digest rejection, missing source evidence, path escape rejection, digest identity and duplicate content; update English and Chinese documentation.

Safety boundary

This is a persistent evidence ledger, not an executable Shadow replay engine. shadow-hit verifies and registers an externally produced artifact; it does not run the rule or replay the task. shadow-evaluate reports eligibility but never changes the persisted shadow stage, creates hooks, or edits production policies. A passing result only identifies candidate-hook as the next review stage.

Validation

  • Initial CI attempts exposed two malformed imports; both were corrected in commits 90e8375 and d185583.
  • On d185583, Coverage and Published Package Gate (Ubuntu/macOS) passed; Source CI failed only at the supply-chain audit after lint/typecheck/build passed and the Windows guardrails corpus passed (113 tests).
  • The audit command is npm audit --audit-level=high --omit=dev. It reports 9 production-tree advisories: 7 high (picomatch GHSA-fc5q-39xc-vjpf) and 2 moderate (Hono / @hono/node-server). The report traces picomatch through both chokidar -> anymatch/readdirp and @typescript-eslint/typescript-estree -> tinyglobby/fdir. npm's suggested --force remediation would move chokidar 3 to 6 (breaking major); no fixed picomatch version is currently listed by the advisory. The audit is not suppressed and no force-upgrade is being applied without compatibility and security evidence.
  • Added fail-closed source-failure provenance validation, project-local file-backed SHA-256 evidence validation, and store-level digest-format enforcement. Fresh Source CI (44259119953), Coverage (44259119796) and Published Package Gate (44259119907) runs are pending for head 4cf20098cb4bdca3a3293289b532c2b24aea7c0c; do not treat this PR as validated until all required jobs finish successfully.

No local execution is claimed in this environment. This PR is stacked on #83 and should be reviewed together with its base; do not merge it independently into master.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant