Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/exp66-evidence.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: exp66-evidence
on:
pull_request:
push:
branches: [main]
jobs:
replay-and-real-package:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 10.9.0
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Portable evidence core tools (legacy absolute file deps untouched)
run: |
npm ci --prefix experiments/64-merged-rule-ownership/ci --workspaces=false
ln -s experiments/64-merged-rule-ownership/ci/node_modules node_modules
mkdir -p node_modules/@swarmlab
ln -s "$GITHUB_WORKSPACE/core" node_modules/@swarmlab/core
node_modules/.bin/tsc -b core
node_modules/.bin/tsc -b --force core
for file in experiments/66-loader-probe-preemption/*.mjs; do node --check "$file"; done
npm run verify:evidence
node experiments/66-loader-probe-preemption/verify-receipts.mjs
- uses: actions/checkout@v4
with:
repository: heybeaux/aegis
ref: db899177bea17b15f814c5a7ccd7e605a01bb112
path: .aegis-exp66
- name: Build exact real Aegis package
working-directory: .aegis-exp66
run: pnpm install --frozen-lockfile && pnpm --filter @heybeaux/lattice-aegis build
- name: Frozen scenario design, actual source/dist, machine outcomes and tamper controls
run: |
START="$(node -e 'console.log(Date.now())')"
AEGIS_REPO="$GITHUB_WORKSPACE/.aegis-exp66" node experiments/66-loader-probe-preemption/main.mjs > "$RUNNER_TEMP/exp66.log"
cat "$RUNNER_TEMP/exp66.log"
node experiments/66-loader-probe-preemption/verify-receipts.mjs --run "$RUNNER_TEMP/exp66.log" --since "$START"
node experiments/66-loader-probe-preemption/test-verifier.mjs "$RUNNER_TEMP/exp66.log"
- uses: actions/upload-artifact@v4
if: always()
with:
name: exp66-runtime-evidence
path: experiments/66-loader-probe-preemption/runs/
133 changes: 133 additions & 0 deletions CLAIMS.json
Original file line number Diff line number Diff line change
Expand Up @@ -6545,5 +6545,138 @@
],
"evidenceStatus": "verified",
"notes": "Spec71 pre-registered before implementation/output; real public built API; frozen18 scenarios repeat-red then same-harness green,38-event full replay; content-addressed receipts with complete source/dist manifests. Direct loadPack not mergeLayers/compiled-cache tamper/in-flight config. No thresholds tuned, holdout unused. Local runtime result verified; PR/CI/merge not inferred. Static scores are historical regression only; live pinned-package CI added separately."
},
{
"id": "RT-56",
"claim": "Interrupting V8 deadlines reject pathological loader regex probes and preserve synchronous API recovery; post-test elapsed checks instead leave all five hazard children stuck past the watchdog.",
"experiment": "experiments/66-loader-probe-preemption",
"spec": "specs/72-loader-probe-preemption.md",
"runIds": [
"lpp-mvevxxcc-75248",
"lpp-mvevy3zp-75313",
"lpp-mvew4ef8-93479"
],
"tracePaths": [
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl"
],
"scoreFields": [
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisProbeWatchdogRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisHazardRejectionRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisResolutionAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisRecoveryRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.substringControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.infrastructureFailures",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisProbeWatchdogRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisHazardRejectionRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisResolutionAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisRecoveryRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.substringControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.infrastructureFailures",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisProbeWatchdogRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisHazardRejectionRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisResolutionAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisRecoveryRate",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.substringControlAccuracy",
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.infrastructureFailures"
],
"expectedValues": {
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisProbeWatchdogRate": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisHazardRejectionRate": {
"equals": 0,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisResolutionAccuracy": {
"equals": 0.631578947368421,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisControlAccuracy": {
"equals": 0.8571428571428571,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisRecoveryRate": {
"equals": 0.631578947368421,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.substringControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.infrastructureFailures": {
"equals": 0,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisProbeWatchdogRate": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisHazardRejectionRate": {
"equals": 0,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisResolutionAccuracy": {
"equals": 0.631578947368421,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisControlAccuracy": {
"equals": 0.8571428571428571,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisRecoveryRate": {
"equals": 0.631578947368421,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.substringControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.infrastructureFailures": {
"equals": 0,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisProbeWatchdogRate": {
"equals": 0,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisHazardRejectionRate": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisResolutionAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisRecoveryRate": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.substringControlAccuracy": {
"equals": 1,
"tolerance": 0
},
"experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.infrastructureFailures": {
"equals": 0,
"tolerance": 0
}
},
"reproductionCommand": "AEGIS_REPO=/path/to/built/aegis node experiments/66-loader-probe-preemption/main.mjs && node experiments/66-loader-probe-preemption/verify-receipts.mjs",
"stackRecommendation": "Aegis must interrupt its own native load-time probes; preserve RulePackError context and fail closed. Finite probes do not prove general regex safety or bounded runtime evaluate.",
"stackReposAndPRs": [
"heybeaux/aegis@db899177bea17b15f814c5a7ccd7e605a01bb112",
"https://github.com/heybeaux/aegis/pull/72"
],
"evidenceStatus": "verified",
"notes": "Spec632df6b before implementation and observation. Five hazards timed out twice,14 controls included2 real invalid-budget failures (0/-1),5substring controls green; no fixture defect found. Candidate19/19+5/5. No tuning, holdout unused. Stored catalogue regression not live proof; frozen harness + actual built package CI required. Automatic merge held: one safe-builtin25ms false positive in full regression; later green rerun does not resolve the risk."
}
]
42 changes: 42 additions & 0 deletions JOURNAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -431,3 +431,45 @@ Existing exp64 workflow [38862709134](https://github.com/heybeaux/swarmlab/actio
**SUCCESS** as regression only. Aegis release CI pending at this documentation
write; no merge inferred. Final head/checks/merge tracked in nightly ledger;
CodeRabbit SUCCESS alone is not review approval.

## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18)

Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and
output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls.
New failure boundary: the loader's own synchronous regex probe can block before
its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership,
compiled-cache binding or in-flight config drift. Original queue is already built.

Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`,
`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms,
**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls,
**5/5** literal substring controls. Repeated deterministic outcomes identical;
raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern
rejections, not fixture errors. Both red traces preserved, no thresholds moved.

Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive
pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack
refusal and RulePackError identities preserved. Invalid env values use default25ms.
Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog,
all5 hazards return exact typed errors and subsequent safe load works. Focused public
artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness,
trace hashes and machine oracles in receipts.json. Fresh live run independently
checked against actual files; verifier rejects15 tampers and accepts positive.

Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56),
evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax +
verify:evidence51claims111traces1022assertions. One intermediate release failed a
stale explicit catalogue count51→52 (corrected); another **falsely rejected safe
builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and
subsequent full release passed unchanged. This new wall-clock compatibility risk
is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic
merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned
real package, not merely stored scores. Remote CI unobserved at this write.

Scope: finite loader probes only. Native runtime evaluate still has no general
regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack
resource bound, production certification or tuned holdout claim. No .acr changes;
Territory untouched. Existing dirty primary checkouts untouched. Prior pending
experiments remain separate. Next new hypothesis: patterns absent from the finite
probe alphabet may load but block runtime evaluation; pre-register a separate
bounded evaluation watchdog/host-input contract before testing, not tonight.
42 changes: 42 additions & 0 deletions SYNTHESIS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1600,3 +1600,45 @@ Existing exp64 workflow [38862709134](https://github.com/heybeaux/swarmlab/actio
**SUCCESS** as regression only. Aegis release CI pending at this documentation
write; no merge inferred. Final head/checks/merge tracked in nightly ledger;
CodeRabbit SUCCESS alone is not review approval.

## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18)

Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and
output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls.
New failure boundary: the loader's own synchronous regex probe can block before
its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership,
compiled-cache binding or in-flight config drift. Original queue is already built.

Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`,
`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms,
**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls,
**5/5** literal substring controls. Repeated deterministic outcomes identical;
raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern
rejections, not fixture errors. Both red traces preserved, no thresholds moved.

Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive
pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack
refusal and RulePackError identities preserved. Invalid env values use default25ms.
Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog,
all5 hazards return exact typed errors and subsequent safe load works. Focused public
artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness,
trace hashes and machine oracles in receipts.json. Fresh live run independently
checked against actual files; verifier rejects15 tampers and accepts positive.

Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56),
evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax +
verify:evidence51claims111traces1022assertions. One intermediate release failed a
stale explicit catalogue count51→52 (corrected); another **falsely rejected safe
builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and
subsequent full release passed unchanged. This new wall-clock compatibility risk
is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic
merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned
real package, not merely stored scores. Remote CI unobserved at this write.

Scope: finite loader probes only. Native runtime evaluate still has no general
regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack
resource bound, production certification or tuned holdout claim. No .acr changes;
Territory untouched. Existing dirty primary checkouts untouched. Prior pending
experiments remain separate. Next new hypothesis: patterns absent from the finite
probe alphabet may load but block runtime evaluation; pre-register a separate
bounded evaluation watchdog/host-input contract before testing, not tonight.
42 changes: 42 additions & 0 deletions docs/EVIDENCE-LEDGER.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,3 +268,45 @@ copies of a fresh live log, plus a positive control and a CI freshness check, an
runs in CI. Frozen harness hash, thresholds, scenarios, receipts and traces are
unchanged. The audit rerun was not admitted as evidence. Limitation: run identity
is bound by filename because the frozen harness does not embed `runId` in the trace.

## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18)

Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and
output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls.
New failure boundary: the loader's own synchronous regex probe can block before
its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership,
compiled-cache binding or in-flight config drift. Original queue is already built.

Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`,
`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms,
**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls,
**5/5** literal substring controls. Repeated deterministic outcomes identical;
raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern
rejections, not fixture errors. Both red traces preserved, no thresholds moved.

Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive
pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack
refusal and RulePackError identities preserved. Invalid env values use default25ms.
Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog,
all5 hazards return exact typed errors and subsequent safe load works. Focused public
artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness,
trace hashes and machine oracles in receipts.json. Fresh live run independently
checked against actual files; verifier rejects15 tampers and accepts positive.

Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56),
evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax +
verify:evidence51claims111traces1022assertions. One intermediate release failed a
stale explicit catalogue count51→52 (corrected); another **falsely rejected safe
builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and
subsequent full release passed unchanged. This new wall-clock compatibility risk
is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic
merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned
real package, not merely stored scores. Remote CI unobserved at this write.

Scope: finite loader probes only. Native runtime evaluate still has no general
regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack
resource bound, production certification or tuned holdout claim. No .acr changes;
Territory untouched. Existing dirty primary checkouts untouched. Prior pending
experiments remain separate. Next new hypothesis: patterns absent from the finite
probe alphabet may load but block runtime evaluation; pre-register a separate
bounded evaluation watchdog/host-input contract before testing, not tonight.
Loading
Loading