Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/exp63-evidence.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: exp63-evidence
on:
pull_request:
push:
branches: [main]
jobs:
replay-and-real-package:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 10.9.0
- uses: actions/setup-node@v4
with:
node-version: '22'
cache: npm
cache-dependency-path: experiments/63-rule-overlay-coverage-consent/ci/package-lock.json
- name: Install portable experiment/core toolchain
# Older experiments retain machine-local file deps; do not rewrite unrelated packages.
run: |
npm ci --prefix experiments/63-rule-overlay-coverage-consent/ci --workspaces=false
ln -s experiments/63-rule-overlay-coverage-consent/ci/node_modules node_modules
mkdir -p node_modules/@swarmlab
ln -s "$GITHUB_WORKSPACE/core" node_modules/@swarmlab/core
- name: Build and type-check core plus new experiment
run: |
node_modules/.bin/tsc -b core experiments/63-rule-overlay-coverage-consent
node_modules/.bin/tsc -b --force core experiments/63-rule-overlay-coverage-consent
- name: Verify committed corpus and exact exp63 receipts
run: npm run verify:evidence && node --test experiments/63-rule-overlay-coverage-consent/verify-receipts.test.mjs && node experiments/63-rule-overlay-coverage-consent/verify-receipts.mjs
- name: Checkout exact evidence-backed Aegis runtime (not moving main)
uses: actions/checkout@v4
with:
repository: heybeaux/aegis
ref: bc879d6c7fd213b49ef355b64bd83196098ffa0e
path: .aegis-candidate
- name: Build real Aegis
working-directory: .aegis-candidate
run: pnpm install --frozen-lockfile && pnpm --filter @heybeaux/lattice-aegis build
- name: Rerun exact frozen exp63 and inspect metrics, not exit code
run: |
AEGIS_REPO="$GITHUB_WORKSPACE/.aegis-candidate" node experiments/63-rule-overlay-coverage-consent/run-and-verify.mjs > "$RUNNER_TEMP/exp63.log"
cat "$RUNNER_TEMP/exp63.log"
AEGIS_REPO="$GITHUB_WORKSPACE/.aegis-candidate" node --test experiments/63-rule-overlay-coverage-consent/verify-receipts.test.mjs
164 changes: 164 additions & 0 deletions CLAIMS.json
Original file line number Diff line number Diff line change
Expand Up @@ -6333,5 +6333,169 @@
"evidenceStatus": "verified",
"seedSetId": "async-evaluator-config-integrity-v1",
"notes": "Pre-registration b0378e0 BEFORE implementation/output; 21 frozen scenarios. Current-main baseline repeated with identical 63 arm scenario rows, same harness hash candidate; full 68-event replay equality. Two diagnostic traces had fixture-only I/O-accounting error; preserved locally but excluded from claims. Thresholds/scenarios unchanged. Exp57/RT47 remains prior unlanded work, NOT novelty tonight. No threshold tuned; holdout reserved unused. Authentic plain-data host, no post-return/ABA/getter/proxy/latency claims."
},
{
"id": "RT-53",
"claim": "Unconsented matcher/scope replacement silently removes effective cascade coverage even at unchanged or higher severity; real Aegis conservative overlay-consent enforcement preserves preceding enabled classifiers without weakening explicit host retunes.",
"experiment": "experiments/63-rule-overlay-coverage-consent",
"spec": "specs/69-rule-overlay-coverage-consent.md",
"runIds": [
"occ-mv7qlhb7",
"occ-mv7qlhdk",
"occ-mv7qn439"
],
"tracePaths": [
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl"
],
"scoreFields": [
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisUnsafeCoverageLossRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisCoverageRejectionRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisResolutionAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.pinnedBaseControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisInputOwnership",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.protectedBaseFixtureAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.explicitConsentAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisUnsafeCoverageLossRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisCoverageRejectionRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisResolutionAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.pinnedBaseControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisInputOwnership",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.protectedBaseFixtureAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.explicitConsentAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisUnsafeCoverageLossRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisCoverageRejectionRate",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisResolutionAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.pinnedBaseControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisControlAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisInputOwnership",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.protectedBaseFixtureAccuracy",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.explicitConsentAccuracy"
],
"expectedValues": {
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisUnsafeCoverageLossRate": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisCoverageRejectionRate": {
"equals": 0,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisResolutionAccuracy": {
"equals": 0.56,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.pinnedBaseControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisControlAccuracy": {
"equals": 0.9333333333333333,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.aegisInputOwnership": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.protectedBaseFixtureAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.jsonl#last.explicitConsentAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisUnsafeCoverageLossRate": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisCoverageRejectionRate": {
"equals": 0,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisResolutionAccuracy": {
"equals": 0.56,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.pinnedBaseControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisControlAccuracy": {
"equals": 0.9333333333333333,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.aegisInputOwnership": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.protectedBaseFixtureAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.jsonl#last.explicitConsentAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisUnsafeCoverageLossRate": {
"equals": 0,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisCoverageRejectionRate": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisResolutionAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.pinnedBaseControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.aegisInputOwnership": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.protectedBaseFixtureAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.jsonl#last.explicitConsentAccuracy": {
"equals": 1,
"tolerance": 0
}
},
"reproductionCommand": "npm run build && AEGIS_REPO=/Users/beauxwalton/projects/worktrees/aegis-2026-10-13-exp63 node experiments/63-rule-overlay-coverage-consent/dist/main.js",
"stackRecommendation": "Require per-overlay allowDowngrade:true for opaque matcher replacement or contracted enabled tool scope; do not infer regex language inclusion. Hosts own review/provenance.",
"stackReposAndPRs": [
{
"repo": "https://github.com/heybeaux/aegis",
"package": "@heybeaux/lattice-aegis",
"branch": "nightly/2026-10-13-exp63",
"commits": [
"bc879d6c7fd213b49ef355b64bd83196098ffa0e"
],
"prs": [
"https://github.com/heybeaux/aegis/pull/69",
"https://github.com/heybeaux/swarmlab/pull/52"
],
"dirtyState": "clean on admitted runtime result"
}
],
"evidenceStatus": "verified",
"notes": "Deterministic machine-owned frozen25 scenarios; 10 hazards baseline unsafe allow10/10, coverage rejection0/10; same-harness candidate25/25. 15 controls include conservative rejection of opaque broadening. No holdout tuning; reserved unused. Host explicit retunes require review after tightening. No predictive calibration, regex inclusion or post-merge alias claim. Candidate not remotely merged; CI tracked separately. Prior RT50/51/52 remain separate unmerged experiments, not silently certified.",
"artifactReceipts": [
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhb7.receipt.json",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qlhdk.receipt.json",
"experiments/63-rule-overlay-coverage-consent/runs/occ-mv7qn439.receipt.json"
]
}
]
20 changes: 20 additions & 0 deletions JOURNAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,3 +307,23 @@ Spec64 b0378e0; 21 frozen scenarios, seed async-evaluator-config-integrity-v1. R
Local regression:1087 Aegis tests, gate47/47; SwarmLab46 claims/96 traces/881 assertions; exp55/56 green regression only. Orphaned exp57 preserved, not landed or independently certified. RT47 rebase must reconcile trusted clock function capture with RT48 plain-policy snapshot; combined behavior unproven.

Linked PRs: https://github.com/heybeaux/aegis/pull/63 (Aegis), https://github.com/heybeaux/swarmlab/pull/46 (SwarmLab). Local green; CI status inspected in tonight's pipeline ledger before any merge.

## 2026-10-13 — Novel exp63; authority is about coverage, not just severity

## RT53 — Rule-overlay coverage consent (exp63 / Spec69)

Pre-registration `c8bc120` preceded implementation and observations; pre-observation configuration correction `060e6b8` pins high/critical deny for every arm. Real fetched-main Aegis `4be84ed4aab018bee1b9a77b6012b9b97c966d1c` (`occ-mv7qlhb7`, repeat `occ-mv7qlhdk`) allowed **10/10** unconsented matcher/scope losses: rejection **0/10**, resolution **14/25**. Preceding base fixture and pinned-host control **10/10**; explicit reviewed allowances **3/3**. Scenario rows and complete package manifests identical across baseline runs.

Aegis runtime `bc879d6c7fd213b49ef355b64bd83196098ffa0e` compares enabled classifier kind/pattern/target/effective flags and case-sensitive tool-set containment at mergeLayers. Opaque matcher replacements and tool contractions require consent on THIS overlay; warning retains entire preceding effective rule. Flag order/empty equivalence, set-equivalent/expanded tools, metadata, disabled-base re-enable, explicit retunes and insertion order preserved. **Same unchanged harness/seed/scenarios** `occ-mv7qn439`: unsafe allow **0/10**, coverage rejection **10/10**, exact resolution **25/25**, all secondary metrics **1**. Three full replay-verified37-event traces plus source/dist manifests and SHA-256 receipts committed. Focused tests **13 red/6 pass ->19 green**; no thresholds moved.

This tests cascade authority BEFORE compilation, distinct from RT48 in-flight config drift and RT52 declared/cache binding. No regex-language inclusion proof: even safe opaque broadening needs explicit review. No post-merge alias isolation, sandbox execution, learned predictor or production incident claim. Host owns trusted overlays, intentional consent and safe action execution. Prior exp60/61/62 candidates remain unmerged and separate. Explicit authority tightening needs migration review; local evidence green is not remote CI/merge confirmation. Stored old evidence is regression, not novel work.

Setup: default pnpm8 rejected lockfile9 before install; used already available corepack pnpm10.9.0 matching CI, no lockfile recreation. Both primary dirty checkouts untouched; dated origin/main worktrees. No .acr edits; Territory refresh not required.

RT53 linked candidates: [Aegis69](https://github.com/heybeaux/aegis/pull/69) / [SwarmLab52](https://github.com/heybeaux/swarmlab/pull/52). Local1144 tests + Aegis stored50/50; SwarmLab49 claims105 traces983 assertions; exact clean-worktree CI rehearsal green. Remote CI pending; NOT merged; migration review required. Stored gate range is50 cases throughRT53 with50–52 absent, not53/53 or certification of prior candidates.

### Same-experiment observable evidence-verifier audit

Independent verification reproduced false acceptance: bare `--latest` certified the existing committed green as fresh; missing secondary metrics and altered action/effective-rule rows were accepted when stored self-check booleans stayed green. This is evidence tooling, NOT a new nightly experiment or an Aegis policy defect. The original three traces and all thresholds remain unchanged.

Receipt verification now pins the frozen scenario/harness contract, requires the complete exact metric set, independently recomputes row field checks and all eight metrics, rejects duplicate/omitted or relabeled scenarios, and inspects every live source/dist hash. New `run-and-verify.mjs` requires exactly one newly produced receipt and checks its exact ID against prior IDs and invocation start; bare `--latest` fails closed. Fifteen verifier regressions cover stale-run acceptance, altered trace checks and live artifact mismatch. The exact unchanged experiment reran green in a clean real-Aegisbc879d6 worktree; audit trace remains non-admitted. Core/experiment build/typecheck, full evidence corpus, and all15 verifier tests green. Runtime still byte-identical to bc879d6; no thresholds tuned, no `.acr` changes, migration hold retained.
Loading
Loading