From 632df6b424123cb000dd83f4f918c213b654a7f3 Mon Sep 17 00:00:00 2001 From: heybeaux Date: Sun, 18 Oct 2026 23:38:17 -0700 Subject: [PATCH 1/5] spec(exp66): pre-register bounded loader regex-probe preemption experiment --- .../66-loader-probe-preemption/README.md | 3 +++ specs/72-loader-probe-preemption.md | 25 +++++++++++++++++++ 2 files changed, 28 insertions(+) create mode 100644 experiments/66-loader-probe-preemption/README.md create mode 100644 specs/72-loader-probe-preemption.md diff --git a/experiments/66-loader-probe-preemption/README.md b/experiments/66-loader-probe-preemption/README.md new file mode 100644 index 0000000..e6a8694 --- /dev/null +++ b/experiments/66-loader-probe-preemption/README.md @@ -0,0 +1,3 @@ +# Exp66 — Loader regex-probe preemption + +Pre-registered [Spec72](../../specs/72-loader-probe-preemption.md) before implementation/baseline. New question: can the loader's own synchronous ReDoS check hang before it inspects its timer? Native Node ESM harness, real public Aegis artifact, child-process watchdog protecting the experiment only. Frozen19 scenarios,5 real substring controls, no LLM judge. Commands/thresholds/ownership/limitations in spec; traces and results will be recorded without rewriting red. diff --git a/specs/72-loader-probe-preemption.md b/specs/72-loader-probe-preemption.md new file mode 100644 index 0000000..d58f153 --- /dev/null +++ b/specs/72-loader-probe-preemption.md @@ -0,0 +1,25 @@ +# Spec 72 — Loader regex-probe preemption (exp66) + +## Pre-registration / novelty +Selected 2026-10-19 UTC before harness implementation or baseline output. Hypothesis: loadPack's synchronous native RegExp.test can monopolize the host before its elapsed-time check executes; its advertised deadline is observation, not preemption. Aegis owns rejecting pathological packs without hanging the agent. Real fetched origin/main Aegis f73ce63d5ab1f389fbda3465916bcba2afb4b117; SwarmLab 3f49985f9a734a5b9b642cbbc0015c09aad32fc5. Inspected all existing specs/experiment README questions, CLAIMS, synthesis, recent ledgers, source/tests/docs/.acr (713-file inventory in nightly artifacts). Spec24/25 and original queue already built. No existing claim measures bounded completion of the loader's own validation. RT54/55 are reference ownership; RT52 cache-binding; RT48 awaits; none preempts synchronous regex probes. Prior October16 stale lock had no ledger/worktree/output; not counted as built work. No old experiment rerun counted as novelty. + +## Ownership / required behaviors +Preserve public synchronous loadPack(pack):CompiledRule[], RulePackError and packId/ruleId; reject entire pack, never return partial compiled policy. Fail closed on exceeded/unavailable deadline, without executing arbitrary pattern as script. Preserve exact JS regex flags/features, substring bypass, ordering, duplicate/shape/syntax/flag errors, editable independently owned outputs and inputs. Timeout must interrupt native regex, not Promise.race or an elapsed check after blocking. A timed-out probe must not poison subsequent valid loads in same process. Environment budget is a positive finite millisecond number, rounded up to an integer; absent/invalid/nonpositive/out-of-range uses default25ms rather than disabling the guard or throwing unrelated RangeError. Host owns authentic policies and real tool execution. Finite probe corpus is NOT a proof of general regex safety or bounded runtime evaluation; no changed evaluation API, no full pack cumulative budget, no arbitrary getter/proxy/cycle/hostile JS code claim. + +## Frozen design / controls / machine oracle +Seed `loader-probe-preemption-v1`, reserved holdout `loader-probe-preemption-holdout-v1` unused unless tuning (no tuning planned). No LLM judge/random sampling. 19 fixed scenarios in `scenarios.mjs`: +- 5 hazards: nested-a `^(a+)+b$`; ambiguous-a `^(a|aa)+b$`; nested-slash `^(/+)+x$`; mixed safe prefix then nested-a; disabled nested-a. Each must return RulePackError with exact pack/rule binding and ReDoS/budget diagnostic BEFORE parent watchdog, never load any rules; then a safe subsequent load must work in that child. +- 14 controls: linear regex; lookbehind+backreference; i/m/s/u flags; substring literal resembling nested regex; empty pack; invalid g flag; duplicate id; invalid syntax; missing appliesTo; env budgets `0`, `NaN`, `Infinity`, `-1`, `1.5` on safe regex. Valid controls must load synchronously, preserve original RegExp source/flags, repeated matching semantics and source recipe, and allow later safe load. Error controls must preserve RulePackError classification and exact binding. +Arms: real public built Aegis loadPack; substring negative control using identical hazardous text as a substring (real Aegis, no substitute policy). The 5 substring negative-control runs must load normally. +Each fresh child imports real built package then announces ready (10s import limit). Parent sends start and starts fixed1500ms watchdog, kills only its own child on expiry, records watchdog red (not a library rejection). Deadline threshold intentionally broad vs25ms, not a microbenchmark. Import/IPC/crash/unexpected output is infrastructure-blocked, NOT policy red. All finite inputs fixed; whole experiment bounded to <2min. Execute baseline twice; classification/checks/metrics must repeat (raw elapsed diagnostic expected variable). Preserve stderr/stdout, child result, ready/start/end, exit/signal, full artifact manifest, exact SHA/dirty and harness/scenario hash. Ground truth is typed outcomes, binding and explicit checks, never exit0 alone. + +## Immutable thresholds +Primary aegisProbeWatchdogRate = killed hazard children /5 EQ0; aegisHazardRejectionRate = exact RulePackError+bound ReDoS/budget rejection plus recovery /5 EQ1; aegisResolutionAccuracy = correct real scenario rows /19 EQ1. Secondary aegisControlAccuracy =14 controls /14 EQ1; aegisRecoveryRate = all returned scenarios with successful later safe load /19 EQ1; substringControlAccuracy=5/5 EQ1; infrastructureFailures=0. Baseline red requires repeated hazard watchdog/rejection failure with all controls/substring arm passing; otherwise investigate fixture/infrastructure, no speculative patch. Baseline fully passing => honest-null and no Aegis patch. Do not move scenarios, thresholds or watchdog after observation. + +## Exact commands +Both Aegis candidate/baseline worktrees: `pnpm install --frozen-lockfile && pnpm --filter @heybeaux/lattice-aegis build` (pnpm10.9.0 via npx if installed version incompatible). Harness is native Node ESM, static checks `node --check experiments/66-loader-probe-preemption/{scenarios,child,main}.mjs` individually. No dependency installation needed for experiment. +Baseline twice (same design): +`AEGIS_REPO=/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline node experiments/66-loader-probe-preemption/main.mjs` +Post-fix rebuild committed candidate, unchanged design: +`AEGIS_REPO=/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66 node experiments/66-loader-probe-preemption/main.mjs` +Artifact/trace/result verification mandatory. After novel work only: `npm run verify:evidence`; Aegis focused loader tests then `pnpm run release:check` full build/typecheck/tests/evidence. CI must run frozen harness against pinned real package and inspect outcomes/manifests. Historical stored scores alone are regression, not live proof. .acr changes not planned => no Territory refresh. Linked PRs only merge after local/remote checks and clean mergeability; pending => unlock and leave PRs for morning. From d98b0cbe05562b05e12bdacd6f79d336ced7aea3 Mon Sep 17 00:00:00 2001 From: heybeaux Date: Sun, 18 Oct 2026 23:39:58 -0700 Subject: [PATCH 2/5] experiment(exp66): freeze real-package child-watchdog harness and machine oracle before baseline --- .../66-loader-probe-preemption/child.mjs | 15 +++++++++ .../66-loader-probe-preemption/main.mjs | 33 +++++++++++++++++++ .../66-loader-probe-preemption/oracle.mjs | 15 +++++++++ .../66-loader-probe-preemption/package.json | 1 + .../66-loader-probe-preemption/scenarios.mjs | 25 ++++++++++++++ 5 files changed, 89 insertions(+) create mode 100644 experiments/66-loader-probe-preemption/child.mjs create mode 100644 experiments/66-loader-probe-preemption/main.mjs create mode 100644 experiments/66-loader-probe-preemption/oracle.mjs create mode 100644 experiments/66-loader-probe-preemption/package.json create mode 100644 experiments/66-loader-probe-preemption/scenarios.mjs diff --git a/experiments/66-loader-probe-preemption/child.mjs b/experiments/66-loader-probe-preemption/child.mjs new file mode 100644 index 0000000..91e23c0 --- /dev/null +++ b/experiments/66-loader-probe-preemption/child.mjs @@ -0,0 +1,15 @@ +import {isDeepStrictEqual as same} from 'node:util'; +import {pathToFileURL} from 'node:url'; +const runtime=await import(pathToFileURL(process.env.AEGIS_DIST).href); +process.send({type:'ready',at:Date.now()}); +process.once('message',({scenario:s})=>{ + const source=structuredClone(s.pack);let response; + try{ + const compiled=runtime.loadPack(s.pack); + const semantics=compiled.every(c=>!c.regex||['DANGER','safe','DANGER\n','DANGER','DANGERN'].every(value=>c.regex.test(value)===new RegExp(c.rule.match.pattern,c.rule.match.flags).test(value))); + response={outcome:'loaded',synchronous:!compiled?.then,compiled:compiled.map(c=>({rule:c.rule,...(c.regex?{regex:{source:c.regex.source,flags:c.regex.flags}}:{})})),semantics}; + }catch(error){response={outcome:'error',error:{name:error.name,message:error.message,typed:error instanceof runtime.RulePackError,packId:error.packId,ruleId:error.ruleId}};} + response.sourceUnchanged=same(s.pack,source); + try{const p={packId:'recovery',version:'1',rules:[{id:'safe',category:'bash',severity:'critical',description:'safe',match:{kind:'regex',pattern:'^DANGER$',target:'command'},appliesTo:['Bash']}]};const c=runtime.loadPack(p);response.recovery=c.length===1&&c[0].regex.test('DANGER')&&!c[0].regex.test('safe');}catch{response.recovery=false;} + process.send({type:'result',at:Date.now(),response},()=>process.disconnect()); +}); diff --git a/experiments/66-loader-probe-preemption/main.mjs b/experiments/66-loader-probe-preemption/main.mjs new file mode 100644 index 0000000..f4d31b8 --- /dev/null +++ b/experiments/66-loader-probe-preemption/main.mjs @@ -0,0 +1,33 @@ +import {fork,execFileSync} from 'node:child_process'; +import {readFileSync,writeFileSync,mkdirSync,readdirSync,statSync} from 'node:fs'; +import {join,relative,resolve} from 'node:path'; +import {createHash} from 'node:crypto'; +import {isDeepStrictEqual as same} from 'node:util'; +import {experiment,seed,WATCHDOG_MS,IMPORT_MS,scenarios,negativeControls} from './scenarios.mjs'; +import {assess,metrics,expected,assertRows} from './oracle.mjs'; +const repo=process.env.AEGIS_REPO;if(!repo)throw Error('AEGIS_REPO required'); +const dist=resolve(repo,'packages/aegis/dist/index.js'); +export const hash=v=>`sha256:${createHash('sha256').update(v).digest('hex')}`; +const root=import.meta.dirname,runId=`lpp-${Date.now().toString(36)}-${process.pid}`,startedAt=Date.now(); +const walk=dir=>readdirSync(dir).sort().flatMap(n=>{const p=join(dir,n);return statSync(p).isDirectory()?walk(p):[p];}); +const manifest=[...walk(join(repo,'packages/aegis/src')),...walk(join(repo,'packages/aegis/dist')),join(repo,'packages/aegis/package.json')].sort().map(f=>({file:relative(repo,f),sha256:hash(readFileSync(f))})); +const harnessManifest=['scenarios.mjs','oracle.mjs','child.mjs','main.mjs'].map(file=>({file,sha256:hash(readFileSync(join(root,file)))})); +const aegis={sha:execFileSync('git',['-C',repo,'rev-parse','HEAD'],{encoding:'utf8'}).trim(),dirty:execFileSync('git',['-C',repo,'status','--porcelain'],{encoding:'utf8'}),repo,manifest,manifestSha256:hash(JSON.stringify(manifest))}; +const events=[],message=(from,topic,body)=>events.push({t:'message',ts:Date.now(),from,to:'moderator',topic,body}); +message('moderator','meta',{runId,experiment,seed,startedAt,watchdogMs:WATCHDOG_MS,importMs:IMPORT_MS,scenarios,negativeControls,harnessManifest,harnessSha256:hash(JSON.stringify(harnessManifest)),scenarioSha256:hash(JSON.stringify({scenarios,negativeControls})),aegis}); +async function run(s){return new Promise(resolveResult=>{ + const child=fork(join(root,'child.mjs'),[],{env:{...process.env,AEGIS_DIST:dist,AEGIS_REDOS_BUDGET_MS:s.budget??'25'},execArgv:[],silent:true}); + let stdout='',stderr='',status,readyAt,startAt,endAt,response,done=false,timer=setTimeout(()=>finish('import-timeout'),IMPORT_MS); + child.stdout.on('data',b=>stdout+=b);child.stderr.on('data',b=>stderr+=b); + function finish(next){if(status)return;status=next;endAt=Date.now();clearTimeout(timer);if(next!=='returned')child.kill('SIGKILL');} + child.on('message',m=>{if(m.type==='ready'&&!readyAt){readyAt=m.at;startAt=Date.now();clearTimeout(timer);timer=setTimeout(()=>finish('watchdog'),WATCHDOG_MS);child.send({scenario:s});}else if(m.type==='result'&&!status){response=m.response;finish('returned');timer=setTimeout(()=>{if(!done)child.kill('SIGKILL');},1000);}}); + child.on('error',e=>{stderr+=String(e);finish('spawn-error');}); + child.on('close',(exit,signal)=>{done=true;clearTimeout(timer);resolveResult({status:status??'child-exit',...(response?{response}:{}),readyAt:readyAt??null,startAt:startAt??null,endAt:endAt??Date.now(),exit,signal,stdout,stderr});}); +});} +const rows=[],controls=[]; +for(const s of scenarios){const row=assess(s,await run(s));rows.push(row);message('real-aegis','scenario',row);} +for(const s of negativeControls){const row=assess(s,await run(s));controls.push(row);message('substring-control','scenario',row);} +assertRows(rows,controls);const scores=metrics(rows,controls);events.push({t:'score',ts:Date.now(),scores}); +mkdirSync(join(root,'runs'),{recursive:true});const trace=join(root,'runs',`${runId}.jsonl`);writeFileSync(trace,events.map(e=>JSON.stringify(e)).join('\n')+'\n');const replay=readFileSync(trace,'utf8').trim().split('\n').map(JSON.parse);if(!same(replay,events))throw Error('full replay mismatch'); +const result={runId,trace,traceSha256:hash(readFileSync(trace)),startedAt,endedAt:Date.now(),aegis,harnessManifest,harnessSha256:hash(JSON.stringify(harnessManifest)),summary:scores,passed:same(scores,expected),failed:rows.filter(r=>!r.correct).map(r=>({id:r.id,checks:r.checks,status:r.result.status})),replayEvents:events.length}; +writeFileSync(join(root,'runs',`${runId}.result.json`),JSON.stringify(result,null,2)+'\n');console.log(JSON.stringify(result,null,2)); diff --git a/experiments/66-loader-probe-preemption/oracle.mjs b/experiments/66-loader-probe-preemption/oracle.mjs new file mode 100644 index 0000000..4e8c038 --- /dev/null +++ b/experiments/66-loader-probe-preemption/oracle.mjs @@ -0,0 +1,15 @@ +import {isDeepStrictEqual as same} from 'node:util'; +import {scenarios,negativeControls} from './scenarios.mjs'; +export function assess(s,result){ + const returned=result.status==='returned',r=result.response; + const checks={returned,classification:returned&&r?.outcome===s.expect,sourceUnchanged:returned&&r?.sourceUnchanged===true,recovery:returned&&r?.recovery===true}; + if(s.expect==='error')Object.assign(checks,{typed:returned&&r?.error?.typed===true,binding:returned&&r?.error?.packId===s.pack.packId&&r?.error?.ruleId===s.ruleId,diagnostic:returned&&r?.error?.message?.includes(s.fragment)===true}); + else Object.assign(checks,{compiled:returned&&same(r?.compiled,s.pack.rules.map(rule=>({rule,...(rule.match.kind==='regex'?{regex:{source:new RegExp(rule.match.pattern,rule.match.flags).source,flags:new RegExp(rule.match.pattern,rule.match.flags).flags}}:{})}))),semantics:returned&&r?.semantics===true,synchronous:returned&&r?.synchronous===true}); + return {id:s.id,kind:s.kind,result,checks,correct:Object.values(checks).every(Boolean)}; +} +export function metrics(rows,controls){ + const hazards=rows.filter(r=>r.kind==='hazard'),safe=rows.filter(r=>r.kind==='control'); + return {aegisProbeWatchdogRate:hazards.filter(r=>r.result.status==='watchdog').length/5,aegisHazardRejectionRate:hazards.filter(r=>r.correct).length/5,aegisResolutionAccuracy:rows.filter(r=>r.correct).length/19,aegisControlAccuracy:safe.filter(r=>r.correct).length/14,aegisRecoveryRate:rows.filter(r=>r.checks.recovery).length/19,substringControlAccuracy:controls.filter(r=>r.correct).length/5,infrastructureFailures:[...rows,...controls].filter(r=>!['returned','watchdog'].includes(r.result.status)).length}; +} +export const expected={aegisProbeWatchdogRate:0,aegisHazardRejectionRate:1,aegisResolutionAccuracy:1,aegisControlAccuracy:1,aegisRecoveryRate:1,substringControlAccuracy:1,infrastructureFailures:0}; +export function assertRows(rows,controls){if(!same(rows.map(r=>r.id),scenarios.map(s=>s.id))||!same(controls.map(r=>r.id),negativeControls.map(s=>s.id)))throw Error('scenario identity/order mismatch');for(const [list,defs] of [[rows,scenarios],[controls,negativeControls]])for(let i=0;i({id:'probe.guard',category:'bash',severity:'critical',description:'probe guard',match:{kind:'regex',pattern,target:'command'},appliesTo:['Bash'],...extra}); +const pack=(rules)=>({packId:'probe-pack',version:'1',rules}); +const hazard=(id,pattern,rules)=>({id,kind:'hazard',pack:pack(rules??[rule(pattern)]),expect:'error',fragment:'ReDoS',ruleId:'probe.guard'}); +const valid=(id,rules,extra={})=>({id,kind:'control',pack:pack(rules),expect:'loaded',...extra}); +const error=(id,rules,fragment,ruleId='probe.guard')=>({id,kind:'control',pack:pack(rules),expect:'error',fragment,ruleId}); +export const scenarios=[ + hazard('nested-a','^(a+)+b$'), + hazard('ambiguous-a','^(a|aa)+b$'), + hazard('nested-slash','^(/+)+x$'), + hazard('mixed-pack','^(a+)+b$',[rule('^DANGER$',{id:'safe.prefix'}),rule('^(a+)+b$')]), + hazard('disabled-hazard','^(a+)+b$',[rule('^(a+)+b$',{enabled:false})]), + valid('linear',[rule('^DANGER+$')]), + valid('lookbehind-backref',[rule('(?<=D)A(N)GER\\1')]), + valid('flags',[rule('^DANGER.$',{match:{kind:'regex',pattern:'^DANGER.$',flags:'imsu',target:'command'}})]), + valid('substring',[rule('^(a+)+b$',{match:{kind:'substring',pattern:'^(a+)+b$',target:'command'}})]), + valid('empty',[]), + error('invalid-g',[rule('DANGER',{match:{kind:'regex',pattern:'DANGER',flags:'g',target:'command'}})],"disallowed regex flag 'g'"), + error('duplicate',[rule('DANGER'),rule('DANGER')],'duplicate rule id'), + error('syntax',[rule('[')],'regex failed to compile'), + error('missing-scope',[rule('DANGER',{appliesTo:[]})],'must declare appliesTo'), + ...['0','NaN','Infinity','-1','1.5'].map(budget=>valid(`budget-${budget}`,[rule('^DANGER$')],{budget})), +]; +export const negativeControls=scenarios.filter(s=>s.kind==='hazard').map(s=>valid(`substring-${s.id}`,s.pack.rules.map(r=>({...r,match:{kind:'substring',pattern:r.match.pattern,target:'command'}})))); From 90ed53afd56d1ffa7dc9cbac3a018206e906c5ec Mon Sep 17 00:00:00 2001 From: heybeaux Date: Sun, 18 Oct 2026 23:51:52 -0700 Subject: [PATCH 3/5] experiment(exp66): preserve repeated loader watchdog red and real-package green; hold wall-clock regression --- .github/workflows/exp66-evidence.yml | 48 ++++ CLAIMS.json | 133 ++++++++++ JOURNAL.md | 42 +++ SYNTHESIS.md | 42 +++ docs/EVIDENCE-LEDGER.md | 42 +++ .../66-loader-probe-preemption/README.md | 44 ++++ .../66-loader-probe-preemption/receipts.json | 249 ++++++++++++++++++ .../runs/lpp-mvevxxcc-75248.jsonl | 26 ++ .../runs/lpp-mvevxxcc-75248.result.json | 186 +++++++++++++ .../runs/lpp-mvevy3zp-75313.jsonl | 26 ++ .../runs/lpp-mvevy3zp-75313.result.json | 186 +++++++++++++ .../runs/lpp-mvew4ef8-93479.jsonl | 26 ++ .../runs/lpp-mvew4ef8-93479.result.json | 94 +++++++ .../test-verifier.mjs | 31 +++ .../verify-receipts.mjs | 42 +++ 15 files changed, 1217 insertions(+) create mode 100644 .github/workflows/exp66-evidence.yml create mode 100644 experiments/66-loader-probe-preemption/receipts.json create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.result.json create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.result.json create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl create mode 100644 experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.result.json create mode 100644 experiments/66-loader-probe-preemption/test-verifier.mjs create mode 100644 experiments/66-loader-probe-preemption/verify-receipts.mjs diff --git a/.github/workflows/exp66-evidence.yml b/.github/workflows/exp66-evidence.yml new file mode 100644 index 0000000..02296c0 --- /dev/null +++ b/.github/workflows/exp66-evidence.yml @@ -0,0 +1,48 @@ +name: exp66-evidence +on: + pull_request: + push: + branches: [main] +jobs: + replay-and-real-package: + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v4 + - uses: pnpm/action-setup@v4 + with: + version: 10.9.0 + - uses: actions/setup-node@v4 + with: + node-version: '22' + - name: Portable evidence core tools (legacy absolute file deps untouched) + run: | + npm ci --prefix experiments/64-merged-rule-ownership/ci --workspaces=false + ln -s experiments/64-merged-rule-ownership/ci/node_modules node_modules + mkdir -p node_modules/@swarmlab + ln -s "$GITHUB_WORKSPACE/core" node_modules/@swarmlab/core + node_modules/.bin/tsc -b core + node_modules/.bin/tsc -b --force core + for file in experiments/66-loader-probe-preemption/*.mjs; do node --check "$file"; done + npm run verify:evidence + node experiments/66-loader-probe-preemption/verify-receipts.mjs + - uses: actions/checkout@v4 + with: + repository: heybeaux/aegis + ref: db899177bea17b15f814c5a7ccd7e605a01bb112 + path: .aegis-exp66 + - name: Build exact real Aegis package + working-directory: .aegis-exp66 + run: pnpm install --frozen-lockfile && pnpm --filter @heybeaux/lattice-aegis build + - name: Frozen scenario design, actual source/dist, machine outcomes and tamper controls + run: | + START="$(node -e 'console.log(Date.now())')" + AEGIS_REPO="$GITHUB_WORKSPACE/.aegis-exp66" node experiments/66-loader-probe-preemption/main.mjs > "$RUNNER_TEMP/exp66.log" + cat "$RUNNER_TEMP/exp66.log" + node experiments/66-loader-probe-preemption/verify-receipts.mjs --run "$RUNNER_TEMP/exp66.log" --since "$START" + node experiments/66-loader-probe-preemption/test-verifier.mjs "$RUNNER_TEMP/exp66.log" + - uses: actions/upload-artifact@v4 + if: always() + with: + name: exp66-runtime-evidence + path: experiments/66-loader-probe-preemption/runs/ diff --git a/CLAIMS.json b/CLAIMS.json index 38ffe00..3e0023b 100644 --- a/CLAIMS.json +++ b/CLAIMS.json @@ -6545,5 +6545,138 @@ ], "evidenceStatus": "verified", "notes": "Spec71 pre-registered before implementation/output; real public built API; frozen18 scenarios repeat-red then same-harness green,38-event full replay; content-addressed receipts with complete source/dist manifests. Direct loadPack not mergeLayers/compiled-cache tamper/in-flight config. No thresholds tuned, holdout unused. Local runtime result verified; PR/CI/merge not inferred. Static scores are historical regression only; live pinned-package CI added separately." + }, + { + "id": "RT-56", + "claim": "Interrupting V8 deadlines reject pathological loader regex probes and preserve synchronous API recovery; post-test elapsed checks instead leave all five hazard children stuck past the watchdog.", + "experiment": "experiments/66-loader-probe-preemption", + "spec": "specs/72-loader-probe-preemption.md", + "runIds": [ + "lpp-mvevxxcc-75248", + "lpp-mvevy3zp-75313", + "lpp-mvew4ef8-93479" + ], + "tracePaths": [ + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl" + ], + "scoreFields": [ + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisProbeWatchdogRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisHazardRejectionRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisResolutionAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisRecoveryRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.substringControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.infrastructureFailures", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisProbeWatchdogRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisHazardRejectionRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisResolutionAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisRecoveryRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.substringControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.infrastructureFailures", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisProbeWatchdogRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisHazardRejectionRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisResolutionAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisRecoveryRate", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.substringControlAccuracy", + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.infrastructureFailures" + ], + "expectedValues": { + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisProbeWatchdogRate": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisHazardRejectionRate": { + "equals": 0, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisResolutionAccuracy": { + "equals": 0.631578947368421, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisControlAccuracy": { + "equals": 0.8571428571428571, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.aegisRecoveryRate": { + "equals": 0.631578947368421, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.substringControlAccuracy": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl#last.infrastructureFailures": { + "equals": 0, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisProbeWatchdogRate": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisHazardRejectionRate": { + "equals": 0, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisResolutionAccuracy": { + "equals": 0.631578947368421, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisControlAccuracy": { + "equals": 0.8571428571428571, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.aegisRecoveryRate": { + "equals": 0.631578947368421, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.substringControlAccuracy": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl#last.infrastructureFailures": { + "equals": 0, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisProbeWatchdogRate": { + "equals": 0, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisHazardRejectionRate": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisResolutionAccuracy": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisControlAccuracy": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.aegisRecoveryRate": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.substringControlAccuracy": { + "equals": 1, + "tolerance": 0 + }, + "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl#last.infrastructureFailures": { + "equals": 0, + "tolerance": 0 + } + }, + "reproductionCommand": "AEGIS_REPO=/path/to/built/aegis node experiments/66-loader-probe-preemption/main.mjs && node experiments/66-loader-probe-preemption/verify-receipts.mjs", + "stackRecommendation": "Aegis must interrupt its own native load-time probes; preserve RulePackError context and fail closed. Finite probes do not prove general regex safety or bounded runtime evaluate.", + "stackReposAndPRs": [ + "heybeaux/aegis@db899177bea17b15f814c5a7ccd7e605a01bb112", + "https://github.com/heybeaux/aegis/pull/72" + ], + "evidenceStatus": "verified", + "notes": "Spec632df6b before implementation and observation. Five hazards timed out twice,14 controls included2 real invalid-budget failures (0/-1),5substring controls green; no fixture defect found. Candidate19/19+5/5. No tuning, holdout unused. Stored catalogue regression not live proof; frozen harness + actual built package CI required. Automatic merge held: one safe-builtin25ms false positive in full regression; later green rerun does not resolve the risk." } ] diff --git a/JOURNAL.md b/JOURNAL.md index 62be833..1aa4f19 100644 --- a/JOURNAL.md +++ b/JOURNAL.md @@ -431,3 +431,45 @@ Existing exp64 workflow [38862709134](https://github.com/heybeaux/swarmlab/actio **SUCCESS** as regression only. Aegis release CI pending at this documentation write; no merge inferred. Final head/checks/merge tracked in nightly ledger; CodeRabbit SUCCESS alone is not review approval. + +## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18) + +Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and +output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls. +New failure boundary: the loader's own synchronous regex probe can block before +its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership, +compiled-cache binding or in-flight config drift. Original queue is already built. + +Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`, +`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms, +**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls, +**5/5** literal substring controls. Repeated deterministic outcomes identical; +raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern +rejections, not fixture errors. Both red traces preserved, no thresholds moved. + +Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive +pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack +refusal and RulePackError identities preserved. Invalid env values use default25ms. +Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog, +all5 hazards return exact typed errors and subsequent safe load works. Focused public +artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness, +trace hashes and machine oracles in receipts.json. Fresh live run independently +checked against actual files; verifier rejects15 tampers and accepts positive. + +Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56), +evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax + +verify:evidence51claims111traces1022assertions. One intermediate release failed a +stale explicit catalogue count51→52 (corrected); another **falsely rejected safe +builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and +subsequent full release passed unchanged. This new wall-clock compatibility risk +is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic +merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned +real package, not merely stored scores. Remote CI unobserved at this write. + +Scope: finite loader probes only. Native runtime evaluate still has no general +regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack +resource bound, production certification or tuned holdout claim. No .acr changes; +Territory untouched. Existing dirty primary checkouts untouched. Prior pending +experiments remain separate. Next new hypothesis: patterns absent from the finite +probe alphabet may load but block runtime evaluation; pre-register a separate +bounded evaluation watchdog/host-input contract before testing, not tonight. diff --git a/SYNTHESIS.md b/SYNTHESIS.md index d79906f..f125957 100644 --- a/SYNTHESIS.md +++ b/SYNTHESIS.md @@ -1600,3 +1600,45 @@ Existing exp64 workflow [38862709134](https://github.com/heybeaux/swarmlab/actio **SUCCESS** as regression only. Aegis release CI pending at this documentation write; no merge inferred. Final head/checks/merge tracked in nightly ledger; CodeRabbit SUCCESS alone is not review approval. + +## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18) + +Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and +output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls. +New failure boundary: the loader's own synchronous regex probe can block before +its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership, +compiled-cache binding or in-flight config drift. Original queue is already built. + +Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`, +`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms, +**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls, +**5/5** literal substring controls. Repeated deterministic outcomes identical; +raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern +rejections, not fixture errors. Both red traces preserved, no thresholds moved. + +Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive +pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack +refusal and RulePackError identities preserved. Invalid env values use default25ms. +Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog, +all5 hazards return exact typed errors and subsequent safe load works. Focused public +artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness, +trace hashes and machine oracles in receipts.json. Fresh live run independently +checked against actual files; verifier rejects15 tampers and accepts positive. + +Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56), +evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax + +verify:evidence51claims111traces1022assertions. One intermediate release failed a +stale explicit catalogue count51→52 (corrected); another **falsely rejected safe +builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and +subsequent full release passed unchanged. This new wall-clock compatibility risk +is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic +merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned +real package, not merely stored scores. Remote CI unobserved at this write. + +Scope: finite loader probes only. Native runtime evaluate still has no general +regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack +resource bound, production certification or tuned holdout claim. No .acr changes; +Territory untouched. Existing dirty primary checkouts untouched. Prior pending +experiments remain separate. Next new hypothesis: patterns absent from the finite +probe alphabet may load but block runtime evaluation; pre-register a separate +bounded evaluation watchdog/host-input contract before testing, not tonight. diff --git a/docs/EVIDENCE-LEDGER.md b/docs/EVIDENCE-LEDGER.md index bd0532c..59da1fd 100644 --- a/docs/EVIDENCE-LEDGER.md +++ b/docs/EVIDENCE-LEDGER.md @@ -268,3 +268,45 @@ copies of a fresh live log, plus a positive control and a CI freshness check, an runs in CI. Frozen harness hash, thresholds, scenarios, receipts and traces are unchanged. The audit rerun was not admitted as evidence. Limitation: run identity is bound by filename because the frozen harness does not embed `runId` in the trace. + +## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18) + +Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and +output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls. +New failure boundary: the loader's own synchronous regex probe can block before +its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership, +compiled-cache binding or in-flight config drift. Original queue is already built. + +Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`, +`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms, +**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls, +**5/5** literal substring controls. Repeated deterministic outcomes identical; +raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern +rejections, not fixture errors. Both red traces preserved, no thresholds moved. + +Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive +pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack +refusal and RulePackError identities preserved. Invalid env values use default25ms. +Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog, +all5 hazards return exact typed errors and subsequent safe load works. Focused public +artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness, +trace hashes and machine oracles in receipts.json. Fresh live run independently +checked against actual files; verifier rejects15 tampers and accepts positive. + +Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56), +evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax + +verify:evidence51claims111traces1022assertions. One intermediate release failed a +stale explicit catalogue count51→52 (corrected); another **falsely rejected safe +builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and +subsequent full release passed unchanged. This new wall-clock compatibility risk +is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic +merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned +real package, not merely stored scores. Remote CI unobserved at this write. + +Scope: finite loader probes only. Native runtime evaluate still has no general +regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack +resource bound, production certification or tuned holdout claim. No .acr changes; +Territory untouched. Existing dirty primary checkouts untouched. Prior pending +experiments remain separate. Next new hypothesis: patterns absent from the finite +probe alphabet may load but block runtime evaluation; pre-register a separate +bounded evaluation watchdog/host-input contract before testing, not tonight. diff --git a/experiments/66-loader-probe-preemption/README.md b/experiments/66-loader-probe-preemption/README.md index e6a8694..a3ac170 100644 --- a/experiments/66-loader-probe-preemption/README.md +++ b/experiments/66-loader-probe-preemption/README.md @@ -1,3 +1,47 @@ # Exp66 — Loader regex-probe preemption Pre-registered [Spec72](../../specs/72-loader-probe-preemption.md) before implementation/baseline. New question: can the loader's own synchronous ReDoS check hang before it inspects its timer? Native Node ESM harness, real public Aegis artifact, child-process watchdog protecting the experiment only. Frozen19 scenarios,5 real substring controls, no LLM judge. Commands/thresholds/ownership/limitations in spec; traces and results will be recorded without rewriting red. + +## RT56 — Loader regex-probe preemption (exp66 / Spec72, 2026-10-18) + +Pre-registration `632df6b424123cb000dd83f4f918c213b654a7f3` preceded harness implementation and +output; harness `d98b0cbe05562b05e12bdacd6f79d336ced7aea3` froze19 scenarios and5 substring controls. +New failure boundary: the loader's own synchronous regex probe can block before +its elapsed-time check executes. Not receipt honesty/recall replay, rule ownership, +compiled-cache binding or in-flight config drift. Original queue is already built. + +Real origin/main Aegis `f73ce63d5ab1f389fbda3465916bcba2afb4b117`, +`lpp-mvevxxcc-75248` / `lpp-mvevy3zp-75313`: **5/5** hazard children watchdog-killed after1500ms, +**0/5** bound ReDoS rejections, **12/19** total correct, **12/14** controls, +**5/5** literal substring controls. Repeated deterministic outcomes identical; +raw timestamps differ. Invalid budgets0/-1 were genuine additional safe-pattern +rejections, not fixture errors. Both red traces preserved, no thresholds moved. + +Runtime `528438cc279623e5c57fa3c5b6f8375b3c7109fc` runs a fixed probe script with primitive +pattern/flags/text inside an interruptible V8 context; synchronous API, whole-pack +refusal and RulePackError identities preserved. Invalid env values use default25ms. +Frozen candidate `db899177bea17b15f814c5a7ccd7e605a01bb112`, `lpp-mvew4ef8-93479`: **19/19** + **5/5**, zero watchdog, +all5 hazards return exact typed errors and subsequent safe load works. Focused public +artifact tests8red10pass→18green. Complete source/dist manifests, frozen harness, +trace hashes and machine oracles in receipts.json. Fresh live run independently +checked against actual files; verifier rejects15 tampers and accepts positive. + +Regression-only: full build/typecheck/release **1185 tests** (628+190+22+86+203+56), +evidence52/52; SwarmLab core build/forced typecheck + native ESM syntax + +verify:evidence51claims111traces1022assertions. One intermediate release failed a +stale explicit catalogue count51→52 (corrected); another **falsely rejected safe +builtin secrets.openai-key.bash at25ms under parallel test load**. Focused hook and +subsequent full release passed unchanged. This new wall-clock compatibility risk +is unresolved; green rerun does NOT erase red. **Aegis PR72 held from automatic +merge**, even if CI passes. Linked SwarmLab CI runs frozen design against pinned +real package, not merely stored scores. Remote CI unobserved at this write. + +Scope: finite loader probes only. Native runtime evaluate still has no general +regex deadline. No general ReDoS safety proof, sandbox guarantee, arbitrary pack +resource bound, production certification or tuned holdout claim. No .acr changes; +Territory untouched. Existing dirty primary checkouts untouched. Prior pending +experiments remain separate. Next new hypothesis: patterns absent from the finite +probe alphabet may load but block runtime evaluation; pre-register a separate +bounded evaluation watchdog/host-input contract before testing, not tonight. + +Aegis: https://github.com/heybeaux/aegis/pull/72. SwarmLab paired PR recorded below once opened. diff --git a/experiments/66-loader-probe-preemption/receipts.json b/experiments/66-loader-probe-preemption/receipts.json new file mode 100644 index 0000000..b69712e --- /dev/null +++ b/experiments/66-loader-probe-preemption/receipts.json @@ -0,0 +1,249 @@ +{ + "experiment": "66-loader-probe-preemption", + "preRegistrationCommit": "632df6b424123cb000dd83f4f918c213b654a7f3", + "frozenHarnessCommit": "d98b0cbe05562b05e12bdacd6f79d336ced7aea3", + "harnessManifest": [ + { + "file": "scenarios.mjs", + "sha256": "sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497" + }, + { + "file": "oracle.mjs", + "sha256": "sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70" + }, + { + "file": "child.mjs", + "sha256": "sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431" + }, + { + "file": "main.mjs", + "sha256": "sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c" + } + ], + "harnessSha256": "sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e", + "candidateSha": "db899177bea17b15f814c5a7ccd7e605a01bb112", + "runs": [ + { + "role": "baseline", + "runId": "lpp-mvevxxcc-75248", + "trace": "experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl", + "traceSha256": "sha256:514137c8fde264ed62f0107bd5704f317e512d2dd4376477d7d523b2a38db116", + "metrics": { + "aegisProbeWatchdogRate": 1, + "aegisHazardRejectionRate": 0, + "aegisResolutionAccuracy": 0.631578947368421, + "aegisControlAccuracy": 0.8571428571428571, + "aegisRecoveryRate": 0.631578947368421, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "aegis": { + "sha": "f73ce63d5ab1f389fbda3465916bcba2afb4b117", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d" + }, + "startedAt": 1792392039948, + "endedAt": 1792392048522, + "replayEvents": 26 + }, + { + "role": "baseline-repeat", + "runId": "lpp-mvevy3zp-75313", + "trace": "experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl", + "traceSha256": "sha256:66b2f9a2f453f23abcd8d34ced50baa9cf8c0b3c12d5d6a9011e15190d003e69", + "metrics": { + "aegisProbeWatchdogRate": 1, + "aegisHazardRejectionRate": 0, + "aegisResolutionAccuracy": 0.631578947368421, + "aegisControlAccuracy": 0.8571428571428571, + "aegisRecoveryRate": 0.631578947368421, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "aegis": { + "sha": "f73ce63d5ab1f389fbda3465916bcba2afb4b117", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d" + }, + "startedAt": 1792392048565, + "endedAt": 1792392057191, + "replayEvents": 26 + }, + { + "role": "candidate", + "runId": "lpp-mvew4ef8-93479", + "trace": "experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl", + "traceSha256": "sha256:84bc7f16ed51a72ceb234365c81f28f3669eed943c0673bd0c9a4d565f283f4b", + "metrics": { + "aegisProbeWatchdogRate": 0, + "aegisHazardRejectionRate": 1, + "aegisResolutionAccuracy": 1, + "aegisControlAccuracy": 1, + "aegisRecoveryRate": 1, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "aegis": { + "sha": "db899177bea17b15f814c5a7ccd7e605a01bb112", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:baee170258cda623a9e1083f6674b0e933ed8300030ab87b99d7103e0043f6e9" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:51ede98e24cbb43f5fb1164b364a954293bf09432478c5032e451a4d6b8962d5" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:3baea4031f3916b2e38301a4d9317f0ff1f9dd0212582ea40605edd7cb6dc90e" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:605412aad0ce059e625c8c402e3a01c1e8eb8e46b04548d6cc657f9ebc4a0614" + }, + "startedAt": 1792392342020, + "endedAt": 1792392343279, + "replayEvents": 26 + } + ] +} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl b/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl new file mode 100644 index 0000000..ab3a224 --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl @@ -0,0 +1,26 @@ +{"t":"message","ts":1792392039995,"from":"moderator","to":"moderator","topic":"meta","body":{"runId":"lpp-mvevxxcc-75248","experiment":"66-loader-probe-preemption","seed":"loader-probe-preemption-v1","startedAt":1792392039948,"watchdogMs":1500,"importMs":10000,"scenarios":[{"id":"nested-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"ambiguous-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"nested-slash","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"mixed-pack","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"disabled-hazard","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"linear","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"lookbehind-backref","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"flags","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"empty","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[]},"expect":"loaded"},{"id":"invalid-g","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","flags":"g","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"disallowed regex flag 'g'","ruleId":"probe.guard"},{"id":"duplicate","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"duplicate rule id","ruleId":"probe.guard"},{"id":"syntax","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"[","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"regex failed to compile","ruleId":"probe.guard"},{"id":"missing-scope","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":[]}]},"expect":"error","fragment":"must declare appliesTo","ruleId":"probe.guard"},{"id":"budget-0","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"0"},{"id":"budget-NaN","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"NaN"},{"id":"budget-Infinity","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"Infinity"},{"id":"budget--1","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"-1"},{"id":"budget-1.5","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"1.5"}],"negativeControls":[{"id":"substring-nested-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-ambiguous-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-nested-slash","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-mixed-pack","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-disabled-hazard","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"loaded"}],"harnessManifest":[{"file":"scenarios.mjs","sha256":"sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497"},{"file":"oracle.mjs","sha256":"sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70"},{"file":"child.mjs","sha256":"sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431"},{"file":"main.mjs","sha256":"sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c"}],"harnessSha256":"sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e","scenarioSha256":"sha256:b51fb9ac84cd278fa875ddf4f16a6041da51958d87cf9f3a446769ef820e6d75","aegis":{"sha":"f73ce63d5ab1f389fbda3465916bcba2afb4b117","dirty":"","repo":"/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline","manifest":[{"file":"packages/aegis/dist/index.d.ts","sha256":"sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd"},{"file":"packages/aegis/dist/index.js","sha256":"sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec"},{"file":"packages/aegis/dist/index.js.map","sha256":"sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89"},{"file":"packages/aegis/package.json","sha256":"sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b"},{"file":"packages/aegis/src/eval/evaluate.ts","sha256":"sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d"},{"file":"packages/aegis/src/eval/preprocess.ts","sha256":"sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9"},{"file":"packages/aegis/src/eval/safe-command.ts","sha256":"sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a"},{"file":"packages/aegis/src/eval/source-policy-checkpoint.ts","sha256":"sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3"},{"file":"packages/aegis/src/index.ts","sha256":"sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129"},{"file":"packages/aegis/src/rules/loader.ts","sha256":"sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651"},{"file":"packages/aegis/src/rules/merge.ts","sha256":"sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e"},{"file":"packages/aegis/src/types.ts","sha256":"sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e"}],"manifestSha256":"sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d"}}} +{"t":"message","ts":1792392041537,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-a","kind":"hazard","result":{"status":"watchdog","readyAt":1792392040035,"startAt":1792392040035,"endAt":1792392041535,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392043082,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"ambiguous-a","kind":"hazard","result":{"status":"watchdog","readyAt":1792392041578,"startAt":1792392041578,"endAt":1792392043080,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392044627,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-slash","kind":"hazard","result":{"status":"watchdog","readyAt":1792392043122,"startAt":1792392043122,"endAt":1792392044624,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392046172,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"mixed-pack","kind":"hazard","result":{"status":"watchdog","readyAt":1792392044668,"startAt":1792392044668,"endAt":1792392046170,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392047716,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"disabled-hazard","kind":"hazard","result":{"status":"watchdog","readyAt":1792392046212,"startAt":1792392046212,"endAt":1792392047715,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392047759,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"linear","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER+$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392047756,"startAt":1792392047756,"endAt":1792392047757,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392047801,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"lookbehind-backref","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"(?<=D)A(N)GER\\1","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392047798,"startAt":1792392047798,"endAt":1792392047799,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392047843,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"flags","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER.$","flags":"imsu"}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392047841,"startAt":1792392047841,"endAt":1792392047841,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392047888,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"substring","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392047885,"startAt":1792392047885,"endAt":1792392047886,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392047931,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"empty","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392047929,"startAt":1792392047929,"endAt":1792392047929,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392047973,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"invalid-g","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] disallowed regex flag 'g' (allowed: i,m,s,u; never g)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392047971,"startAt":1792392047971,"endAt":1792392047972,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392048016,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"duplicate","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] duplicate rule id within pack","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392048013,"startAt":1792392048013,"endAt":1792392048014,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392048057,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"syntax","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] regex failed to compile: Invalid regular expression: /[/: Unterminated character class","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392048054,"startAt":1792392048054,"endAt":1792392048055,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392048098,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"missing-scope","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] rule must declare appliesTo (use [\"*\"] for any tool)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392048096,"startAt":1792392048096,"endAt":1792392048096,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392048140,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-0","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (0.0ms > 0ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":false},"readyAt":1792392048138,"startAt":1792392048138,"endAt":1792392048139,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":false,"sourceUnchanged":true,"recovery":false,"compiled":false,"semantics":false,"synchronous":false},"correct":false}} +{"t":"message","ts":1792392048183,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-NaN","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048181,"startAt":1792392048181,"endAt":1792392048182,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048225,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-Infinity","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048222,"startAt":1792392048222,"endAt":1792392048223,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048266,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget--1","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (0.0ms > -1ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":false},"readyAt":1792392048264,"startAt":1792392048264,"endAt":1792392048265,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":false,"sourceUnchanged":true,"recovery":false,"compiled":false,"semantics":false,"synchronous":false},"correct":false}} +{"t":"message","ts":1792392048307,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-1.5","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048305,"startAt":1792392048305,"endAt":1792392048306,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048349,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048347,"startAt":1792392048347,"endAt":1792392048348,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048395,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-ambiguous-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048392,"startAt":1792392048392,"endAt":1792392048393,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048438,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-slash","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048435,"startAt":1792392048435,"endAt":1792392048436,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048480,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-mixed-pack","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}},{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048477,"startAt":1792392048477,"endAt":1792392048478,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392048521,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-disabled-hazard","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392048519,"startAt":1792392048519,"endAt":1792392048520,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"score","ts":1792392048521,"scores":{"aegisProbeWatchdogRate":1,"aegisHazardRejectionRate":0,"aegisResolutionAccuracy":0.631578947368421,"aegisControlAccuracy":0.8571428571428571,"aegisRecoveryRate":0.631578947368421,"substringControlAccuracy":1,"infrastructureFailures":0}} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.result.json b/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.result.json new file mode 100644 index 0000000..360f634 --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.result.json @@ -0,0 +1,186 @@ +{ + "runId": "lpp-mvevxxcc-75248", + "trace": "/Users/beauxwalton/projects/worktrees/swarmlab-2026-10-18-exp66/experiments/66-loader-probe-preemption/runs/lpp-mvevxxcc-75248.jsonl", + "traceSha256": "sha256:514137c8fde264ed62f0107bd5704f317e512d2dd4376477d7d523b2a38db116", + "startedAt": 1792392039948, + "endedAt": 1792392048522, + "aegis": { + "sha": "f73ce63d5ab1f389fbda3465916bcba2afb4b117", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d" + }, + "harnessManifest": [ + { + "file": "scenarios.mjs", + "sha256": "sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497" + }, + { + "file": "oracle.mjs", + "sha256": "sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70" + }, + { + "file": "child.mjs", + "sha256": "sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431" + }, + { + "file": "main.mjs", + "sha256": "sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c" + } + ], + "harnessSha256": "sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e", + "summary": { + "aegisProbeWatchdogRate": 1, + "aegisHazardRejectionRate": 0, + "aegisResolutionAccuracy": 0.631578947368421, + "aegisControlAccuracy": 0.8571428571428571, + "aegisRecoveryRate": 0.631578947368421, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "passed": false, + "failed": [ + { + "id": "nested-a", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "ambiguous-a", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "nested-slash", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "mixed-pack", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "disabled-hazard", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "budget-0", + "checks": { + "returned": true, + "classification": false, + "sourceUnchanged": true, + "recovery": false, + "compiled": false, + "semantics": false, + "synchronous": false + }, + "status": "returned" + }, + { + "id": "budget--1", + "checks": { + "returned": true, + "classification": false, + "sourceUnchanged": true, + "recovery": false, + "compiled": false, + "semantics": false, + "synchronous": false + }, + "status": "returned" + } + ], + "replayEvents": 26 +} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl b/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl new file mode 100644 index 0000000..ac19d05 --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl @@ -0,0 +1,26 @@ +{"t":"message","ts":1792392048577,"from":"moderator","to":"moderator","topic":"meta","body":{"runId":"lpp-mvevy3zp-75313","experiment":"66-loader-probe-preemption","seed":"loader-probe-preemption-v1","startedAt":1792392048565,"watchdogMs":1500,"importMs":10000,"scenarios":[{"id":"nested-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"ambiguous-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"nested-slash","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"mixed-pack","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"disabled-hazard","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"linear","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"lookbehind-backref","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"flags","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"empty","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[]},"expect":"loaded"},{"id":"invalid-g","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","flags":"g","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"disallowed regex flag 'g'","ruleId":"probe.guard"},{"id":"duplicate","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"duplicate rule id","ruleId":"probe.guard"},{"id":"syntax","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"[","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"regex failed to compile","ruleId":"probe.guard"},{"id":"missing-scope","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":[]}]},"expect":"error","fragment":"must declare appliesTo","ruleId":"probe.guard"},{"id":"budget-0","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"0"},{"id":"budget-NaN","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"NaN"},{"id":"budget-Infinity","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"Infinity"},{"id":"budget--1","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"-1"},{"id":"budget-1.5","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"1.5"}],"negativeControls":[{"id":"substring-nested-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-ambiguous-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-nested-slash","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-mixed-pack","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-disabled-hazard","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"loaded"}],"harnessManifest":[{"file":"scenarios.mjs","sha256":"sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497"},{"file":"oracle.mjs","sha256":"sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70"},{"file":"child.mjs","sha256":"sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431"},{"file":"main.mjs","sha256":"sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c"}],"harnessSha256":"sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e","scenarioSha256":"sha256:b51fb9ac84cd278fa875ddf4f16a6041da51958d87cf9f3a446769ef820e6d75","aegis":{"sha":"f73ce63d5ab1f389fbda3465916bcba2afb4b117","dirty":"","repo":"/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline","manifest":[{"file":"packages/aegis/dist/index.d.ts","sha256":"sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd"},{"file":"packages/aegis/dist/index.js","sha256":"sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec"},{"file":"packages/aegis/dist/index.js.map","sha256":"sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89"},{"file":"packages/aegis/package.json","sha256":"sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b"},{"file":"packages/aegis/src/eval/evaluate.ts","sha256":"sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d"},{"file":"packages/aegis/src/eval/preprocess.ts","sha256":"sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9"},{"file":"packages/aegis/src/eval/safe-command.ts","sha256":"sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a"},{"file":"packages/aegis/src/eval/source-policy-checkpoint.ts","sha256":"sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3"},{"file":"packages/aegis/src/index.ts","sha256":"sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129"},{"file":"packages/aegis/src/rules/loader.ts","sha256":"sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651"},{"file":"packages/aegis/src/rules/merge.ts","sha256":"sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e"},{"file":"packages/aegis/src/types.ts","sha256":"sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e"}],"manifestSha256":"sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d"}}} +{"t":"message","ts":1792392050121,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-a","kind":"hazard","result":{"status":"watchdog","readyAt":1792392048617,"startAt":1792392048617,"endAt":1792392050119,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392051665,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"ambiguous-a","kind":"hazard","result":{"status":"watchdog","readyAt":1792392050163,"startAt":1792392050164,"endAt":1792392051664,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392053212,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-slash","kind":"hazard","result":{"status":"watchdog","readyAt":1792392051710,"startAt":1792392051710,"endAt":1792392053210,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392054758,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"mixed-pack","kind":"hazard","result":{"status":"watchdog","readyAt":1792392053256,"startAt":1792392053257,"endAt":1792392054757,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392056303,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"disabled-hazard","kind":"hazard","result":{"status":"watchdog","readyAt":1792392054799,"startAt":1792392054800,"endAt":1792392056302,"exit":null,"signal":"SIGKILL","stdout":"","stderr":""},"checks":{"returned":false,"classification":false,"sourceUnchanged":false,"recovery":false,"typed":false,"binding":false,"diagnostic":false},"correct":false}} +{"t":"message","ts":1792392056350,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"linear","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER+$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056346,"startAt":1792392056346,"endAt":1792392056347,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056403,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"lookbehind-backref","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"(?<=D)A(N)GER\\1","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056399,"startAt":1792392056400,"endAt":1792392056401,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056450,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"flags","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER.$","flags":"imsu"}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056447,"startAt":1792392056447,"endAt":1792392056448,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056496,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"substring","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056493,"startAt":1792392056493,"endAt":1792392056494,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056541,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"empty","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056538,"startAt":1792392056538,"endAt":1792392056539,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056588,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"invalid-g","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] disallowed regex flag 'g' (allowed: i,m,s,u; never g)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392056585,"startAt":1792392056585,"endAt":1792392056586,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392056635,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"duplicate","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] duplicate rule id within pack","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392056632,"startAt":1792392056632,"endAt":1792392056633,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392056681,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"syntax","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] regex failed to compile: Invalid regular expression: /[/: Unterminated character class","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392056678,"startAt":1792392056678,"endAt":1792392056679,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392056729,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"missing-scope","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] rule must declare appliesTo (use [\"*\"] for any tool)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392056726,"startAt":1792392056726,"endAt":1792392056727,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392056776,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-0","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (0.0ms > 0ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":false},"readyAt":1792392056773,"startAt":1792392056773,"endAt":1792392056774,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":false,"sourceUnchanged":true,"recovery":false,"compiled":false,"semantics":false,"synchronous":false},"correct":false}} +{"t":"message","ts":1792392056822,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-NaN","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056819,"startAt":1792392056820,"endAt":1792392056820,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056870,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-Infinity","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056867,"startAt":1792392056867,"endAt":1792392056868,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392056917,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget--1","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (0.0ms > -1ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":false},"readyAt":1792392056914,"startAt":1792392056915,"endAt":1792392056915,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":false,"sourceUnchanged":true,"recovery":false,"compiled":false,"semantics":false,"synchronous":false},"correct":false}} +{"t":"message","ts":1792392056963,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-1.5","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392056960,"startAt":1792392056960,"endAt":1792392056961,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392057008,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392057005,"startAt":1792392057005,"endAt":1792392057006,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392057052,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-ambiguous-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392057050,"startAt":1792392057050,"endAt":1792392057051,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392057099,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-slash","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392057096,"startAt":1792392057096,"endAt":1792392057097,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392057144,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-mixed-pack","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}},{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392057141,"startAt":1792392057141,"endAt":1792392057142,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392057190,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-disabled-hazard","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392057187,"startAt":1792392057187,"endAt":1792392057188,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"score","ts":1792392057190,"scores":{"aegisProbeWatchdogRate":1,"aegisHazardRejectionRate":0,"aegisResolutionAccuracy":0.631578947368421,"aegisControlAccuracy":0.8571428571428571,"aegisRecoveryRate":0.631578947368421,"substringControlAccuracy":1,"infrastructureFailures":0}} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.result.json b/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.result.json new file mode 100644 index 0000000..2e0a755 --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.result.json @@ -0,0 +1,186 @@ +{ + "runId": "lpp-mvevy3zp-75313", + "trace": "/Users/beauxwalton/projects/worktrees/swarmlab-2026-10-18-exp66/experiments/66-loader-probe-preemption/runs/lpp-mvevy3zp-75313.jsonl", + "traceSha256": "sha256:66b2f9a2f453f23abcd8d34ced50baa9cf8c0b3c12d5d6a9011e15190d003e69", + "startedAt": 1792392048565, + "endedAt": 1792392057191, + "aegis": { + "sha": "f73ce63d5ab1f389fbda3465916bcba2afb4b117", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66-baseline", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:5fbd5bb5a31d823c9a3a4c21e3d167db7e590eccb3ac5a4605526abe194cb1ec" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:d98673e07a5e9832012f07b7c4b14357d913faf0663ca7cd2a9b246a04841b89" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:8da48a548e77a343dca94beeddaea5d19926752ac9b72df8f6ad2c4b9bb3c651" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:4200ab154ac853dd1f273ae178287d2c5034ecd20c2e5cd595c9eb4ebf11180d" + }, + "harnessManifest": [ + { + "file": "scenarios.mjs", + "sha256": "sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497" + }, + { + "file": "oracle.mjs", + "sha256": "sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70" + }, + { + "file": "child.mjs", + "sha256": "sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431" + }, + { + "file": "main.mjs", + "sha256": "sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c" + } + ], + "harnessSha256": "sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e", + "summary": { + "aegisProbeWatchdogRate": 1, + "aegisHazardRejectionRate": 0, + "aegisResolutionAccuracy": 0.631578947368421, + "aegisControlAccuracy": 0.8571428571428571, + "aegisRecoveryRate": 0.631578947368421, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "passed": false, + "failed": [ + { + "id": "nested-a", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "ambiguous-a", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "nested-slash", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "mixed-pack", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "disabled-hazard", + "checks": { + "returned": false, + "classification": false, + "sourceUnchanged": false, + "recovery": false, + "typed": false, + "binding": false, + "diagnostic": false + }, + "status": "watchdog" + }, + { + "id": "budget-0", + "checks": { + "returned": true, + "classification": false, + "sourceUnchanged": true, + "recovery": false, + "compiled": false, + "semantics": false, + "synchronous": false + }, + "status": "returned" + }, + { + "id": "budget--1", + "checks": { + "returned": true, + "classification": false, + "sourceUnchanged": true, + "recovery": false, + "compiled": false, + "semantics": false, + "synchronous": false + }, + "status": "returned" + } + ], + "replayEvents": 26 +} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl b/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl new file mode 100644 index 0000000..5743554 --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl @@ -0,0 +1,26 @@ +{"t":"message","ts":1792392342035,"from":"moderator","to":"moderator","topic":"meta","body":{"runId":"lpp-mvew4ef8-93479","experiment":"66-loader-probe-preemption","seed":"loader-probe-preemption-v1","startedAt":1792392342020,"watchdogMs":1500,"importMs":10000,"scenarios":[{"id":"nested-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"ambiguous-a","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"nested-slash","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"mixed-pack","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"disabled-hazard","kind":"hazard","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"error","fragment":"ReDoS","ruleId":"probe.guard"},{"id":"linear","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"lookbehind-backref","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"flags","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"empty","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[]},"expect":"loaded"},{"id":"invalid-g","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","flags":"g","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"disallowed regex flag 'g'","ruleId":"probe.guard"},{"id":"duplicate","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"duplicate rule id","ruleId":"probe.guard"},{"id":"syntax","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"[","target":"command"},"appliesTo":["Bash"]}]},"expect":"error","fragment":"regex failed to compile","ruleId":"probe.guard"},{"id":"missing-scope","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"DANGER","target":"command"},"appliesTo":[]}]},"expect":"error","fragment":"must declare appliesTo","ruleId":"probe.guard"},{"id":"budget-0","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"0"},{"id":"budget-NaN","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"NaN"},{"id":"budget-Infinity","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"Infinity"},{"id":"budget--1","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"-1"},{"id":"budget-1.5","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded","budget":"1.5"}],"negativeControls":[{"id":"substring-nested-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-ambiguous-a","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-nested-slash","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-mixed-pack","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}]},"expect":"loaded"},{"id":"substring-disabled-hazard","kind":"control","pack":{"packId":"probe-pack","version":"1","rules":[{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}]},"expect":"loaded"}],"harnessManifest":[{"file":"scenarios.mjs","sha256":"sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497"},{"file":"oracle.mjs","sha256":"sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70"},{"file":"child.mjs","sha256":"sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431"},{"file":"main.mjs","sha256":"sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c"}],"harnessSha256":"sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e","scenarioSha256":"sha256:b51fb9ac84cd278fa875ddf4f16a6041da51958d87cf9f3a446769ef820e6d75","aegis":{"sha":"db899177bea17b15f814c5a7ccd7e605a01bb112","dirty":"","repo":"/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66","manifest":[{"file":"packages/aegis/dist/index.d.ts","sha256":"sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd"},{"file":"packages/aegis/dist/index.js","sha256":"sha256:baee170258cda623a9e1083f6674b0e933ed8300030ab87b99d7103e0043f6e9"},{"file":"packages/aegis/dist/index.js.map","sha256":"sha256:51ede98e24cbb43f5fb1164b364a954293bf09432478c5032e451a4d6b8962d5"},{"file":"packages/aegis/package.json","sha256":"sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b"},{"file":"packages/aegis/src/eval/evaluate.ts","sha256":"sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d"},{"file":"packages/aegis/src/eval/preprocess.ts","sha256":"sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9"},{"file":"packages/aegis/src/eval/safe-command.ts","sha256":"sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a"},{"file":"packages/aegis/src/eval/source-policy-checkpoint.ts","sha256":"sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3"},{"file":"packages/aegis/src/index.ts","sha256":"sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129"},{"file":"packages/aegis/src/rules/loader.ts","sha256":"sha256:3baea4031f3916b2e38301a4d9317f0ff1f9dd0212582ea40605edd7cb6dc90e"},{"file":"packages/aegis/src/rules/merge.ts","sha256":"sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e"},{"file":"packages/aegis/src/types.ts","sha256":"sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e"}],"manifestSha256":"sha256:605412aad0ce059e625c8c402e3a01c1e8eb8e46b04548d6cc657f9ebc4a0614"}}} +{"t":"message","ts":1792392342108,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-a","kind":"hazard","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (25ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342078,"startAt":1792392342078,"endAt":1792392342106,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342182,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"ambiguous-a","kind":"hazard","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (25ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342151,"startAt":1792392342151,"endAt":1792392342180,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342253,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"nested-slash","kind":"hazard","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (25ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342224,"startAt":1792392342224,"endAt":1792392342251,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342329,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"mixed-pack","kind":"hazard","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (25ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342298,"startAt":1792392342298,"endAt":1792392342327,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342401,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"disabled-hazard","kind":"hazard","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] pattern exceeded ReDoS budget (25ms) on a probe — likely catastrophic backtracking","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342372,"startAt":1792392342372,"endAt":1792392342399,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342446,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"linear","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER+$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER+$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342442,"startAt":1792392342442,"endAt":1792392342444,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342491,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"lookbehind-backref","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"(?<=D)A(N)GER\\1","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"(?<=D)A(N)GER\\1","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342487,"startAt":1792392342487,"endAt":1792392342489,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342535,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"flags","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER.$","flags":"imsu","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER.$","flags":"imsu"}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342532,"startAt":1792392342532,"endAt":1792392342534,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342582,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"substring","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342578,"startAt":1792392342578,"endAt":1792392342580,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342628,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"empty","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342625,"startAt":1792392342625,"endAt":1792392342626,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342675,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"invalid-g","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] disallowed regex flag 'g' (allowed: i,m,s,u; never g)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342672,"startAt":1792392342672,"endAt":1792392342673,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342724,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"duplicate","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] duplicate rule id within pack","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342720,"startAt":1792392342720,"endAt":1792392342722,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342769,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"syntax","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] regex failed to compile: Invalid regular expression: /[/: Unterminated character class","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342766,"startAt":1792392342766,"endAt":1792392342767,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342819,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"missing-scope","kind":"control","result":{"status":"returned","response":{"outcome":"error","error":{"name":"RulePackError","message":"[aegis:probe-pack/probe.guard] rule must declare appliesTo (use [\"*\"] for any tool)","typed":true,"packId":"probe-pack","ruleId":"probe.guard"},"sourceUnchanged":true,"recovery":true},"readyAt":1792392342815,"startAt":1792392342815,"endAt":1792392342817,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"typed":true,"binding":true,"diagnostic":true},"correct":true}} +{"t":"message","ts":1792392342866,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-0","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342862,"startAt":1792392342862,"endAt":1792392342864,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342912,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-NaN","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342908,"startAt":1792392342908,"endAt":1792392342910,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392342957,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-Infinity","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342953,"startAt":1792392342954,"endAt":1792392342956,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343003,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget--1","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392342999,"startAt":1792392343000,"endAt":1792392343001,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343051,"from":"real-aegis","to":"moderator","topic":"scenario","body":{"id":"budget-1.5","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"regex","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]},"regex":{"source":"^DANGER$","flags":""}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343047,"startAt":1792392343047,"endAt":1792392343049,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343096,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343093,"startAt":1792392343093,"endAt":1792392343094,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343141,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-ambiguous-a","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a|aa)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343138,"startAt":1792392343138,"endAt":1792392343140,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343187,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-nested-slash","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(/+)+x$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343183,"startAt":1792392343183,"endAt":1792392343185,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343232,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-mixed-pack","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"safe.prefix","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^DANGER$","target":"command"},"appliesTo":["Bash"]}},{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"]}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343229,"startAt":1792392343229,"endAt":1792392343230,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"message","ts":1792392343278,"from":"substring-control","to":"moderator","topic":"scenario","body":{"id":"substring-disabled-hazard","kind":"control","result":{"status":"returned","response":{"outcome":"loaded","synchronous":true,"compiled":[{"rule":{"id":"probe.guard","category":"bash","severity":"critical","description":"probe guard","match":{"kind":"substring","pattern":"^(a+)+b$","target":"command"},"appliesTo":["Bash"],"enabled":false}}],"semantics":true,"sourceUnchanged":true,"recovery":true},"readyAt":1792392343274,"startAt":1792392343274,"endAt":1792392343276,"exit":0,"signal":null,"stdout":"","stderr":""},"checks":{"returned":true,"classification":true,"sourceUnchanged":true,"recovery":true,"compiled":true,"semantics":true,"synchronous":true},"correct":true}} +{"t":"score","ts":1792392343278,"scores":{"aegisProbeWatchdogRate":0,"aegisHazardRejectionRate":1,"aegisResolutionAccuracy":1,"aegisControlAccuracy":1,"aegisRecoveryRate":1,"substringControlAccuracy":1,"infrastructureFailures":0}} diff --git a/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.result.json b/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.result.json new file mode 100644 index 0000000..3b4842c --- /dev/null +++ b/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.result.json @@ -0,0 +1,94 @@ +{ + "runId": "lpp-mvew4ef8-93479", + "trace": "/Users/beauxwalton/projects/worktrees/swarmlab-2026-10-18-exp66/experiments/66-loader-probe-preemption/runs/lpp-mvew4ef8-93479.jsonl", + "traceSha256": "sha256:84bc7f16ed51a72ceb234365c81f28f3669eed943c0673bd0c9a4d565f283f4b", + "startedAt": 1792392342020, + "endedAt": 1792392343279, + "aegis": { + "sha": "db899177bea17b15f814c5a7ccd7e605a01bb112", + "dirty": "", + "repo": "/Users/beauxwalton/projects/worktrees/aegis-2026-10-18-exp66", + "manifest": [ + { + "file": "packages/aegis/dist/index.d.ts", + "sha256": "sha256:c125251eed281a3e148f4795ac8e8fa7c7065b1e8e35e47a886b327fe30446bd" + }, + { + "file": "packages/aegis/dist/index.js", + "sha256": "sha256:baee170258cda623a9e1083f6674b0e933ed8300030ab87b99d7103e0043f6e9" + }, + { + "file": "packages/aegis/dist/index.js.map", + "sha256": "sha256:51ede98e24cbb43f5fb1164b364a954293bf09432478c5032e451a4d6b8962d5" + }, + { + "file": "packages/aegis/package.json", + "sha256": "sha256:d86e3de257d5712edb53461ffe68da6c5959c8099a493344fd8a36776d0c395b" + }, + { + "file": "packages/aegis/src/eval/evaluate.ts", + "sha256": "sha256:5eb4bebb46ec0dee67ddaada08b1a49d5bfbb814710489622fbde5c5be26b32d" + }, + { + "file": "packages/aegis/src/eval/preprocess.ts", + "sha256": "sha256:05b1c40d78d87538f5fe4a0892307a23af0447bc26a683c8beedc66e881952f9" + }, + { + "file": "packages/aegis/src/eval/safe-command.ts", + "sha256": "sha256:00236420fc95731348dd00331c4e0a5b7f55266b23604e300c7e44686e55bb9a" + }, + { + "file": "packages/aegis/src/eval/source-policy-checkpoint.ts", + "sha256": "sha256:e5507169549e2b558bd6624f48598d4c0d63cf288ec262b5eb36c6234dec68b3" + }, + { + "file": "packages/aegis/src/index.ts", + "sha256": "sha256:ee13318d75e32322eb9eed687e517df6a0dc9cf9d9394a5c85be486c99c45129" + }, + { + "file": "packages/aegis/src/rules/loader.ts", + "sha256": "sha256:3baea4031f3916b2e38301a4d9317f0ff1f9dd0212582ea40605edd7cb6dc90e" + }, + { + "file": "packages/aegis/src/rules/merge.ts", + "sha256": "sha256:15792d138874a8b4fac914412dd0a72134480eb770bf32404da29a28e6e1e99e" + }, + { + "file": "packages/aegis/src/types.ts", + "sha256": "sha256:ae2f91bbefd2320198b892d71f29bb0fcde737227cb0c0a25d06e6202da6dc3e" + } + ], + "manifestSha256": "sha256:605412aad0ce059e625c8c402e3a01c1e8eb8e46b04548d6cc657f9ebc4a0614" + }, + "harnessManifest": [ + { + "file": "scenarios.mjs", + "sha256": "sha256:9a8207c859d18dde75557d054afd298d8fa84e54b26ad91e8087a360dd013497" + }, + { + "file": "oracle.mjs", + "sha256": "sha256:27fa81ee719d92366a362d7081b27057969416bc445ebece1b62997808dbeb70" + }, + { + "file": "child.mjs", + "sha256": "sha256:8494340165640ad5999e7ab480c5c4ef7f5fcdbda02dc027d566fd7e39047431" + }, + { + "file": "main.mjs", + "sha256": "sha256:f191f1302024ac62b9e5d9059414c3e39287e968175530c953825d6402166a9c" + } + ], + "harnessSha256": "sha256:8fecf99f3254d2e152e5e87a4c496ef08f6e8b47446ecb748946f64ab8fd0c7e", + "summary": { + "aegisProbeWatchdogRate": 0, + "aegisHazardRejectionRate": 1, + "aegisResolutionAccuracy": 1, + "aegisControlAccuracy": 1, + "aegisRecoveryRate": 1, + "substringControlAccuracy": 1, + "infrastructureFailures": 0 + }, + "passed": true, + "failed": [], + "replayEvents": 26 +} diff --git a/experiments/66-loader-probe-preemption/test-verifier.mjs b/experiments/66-loader-probe-preemption/test-verifier.mjs new file mode 100644 index 0000000..a2429d7 --- /dev/null +++ b/experiments/66-loader-probe-preemption/test-verifier.mjs @@ -0,0 +1,31 @@ +import {readFileSync,writeFileSync,mkdtempSync,mkdirSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {execFileSync} from 'node:child_process'; +import {createHash} from 'node:crypto'; +const input=JSON.parse(readFileSync(process.argv[2],'utf8')),base=readFileSync(input.trace,'utf8').trim().split('\n').map(JSON.parse),since=String(input.startedAt),dir=mkdtempSync(join(tmpdir(),'exp66-verifier-')); +const hash=v=>`sha256:${createHash('sha256').update(v).digest('hex')}`; +function run(name,mutate,good=false){ + const events=structuredClone(base),result=structuredClone(input);mutate(events,result); + const folder=join(dir,name);mkdirSync(folder);result.trace=join(folder,result.runId+'.jsonl');writeFileSync(result.trace,events.map(JSON.stringify).join('\n')+'\n');result.traceSha256=hash(readFileSync(result.trace));const log=join(folder,'run.json');writeFileSync(log,JSON.stringify(result)); + let ok=true;try{execFileSync(process.execPath,[join(import.meta.dirname,'verify-receipts.mjs'),'--run',log,'--since',since],{stdio:'pipe'});}catch{ok=false;} + if(ok!==good)throw Error(`${name}: expected ${good?'accept':'reject'}, got ${ok}`);console.log(`${name}: ${ok?'accepted':'rejected'}`); +} +run('positive',()=>{},true); +const row=e=>e.find(x=>x.topic==='scenario').body; +run('missing-checks',e=>{row(e).checks={};row(e).correct=true;}); +run('changed-pattern',e=>{e[0].body.scenarios[0].pack.rules[0].match.pattern='SAFE';}); +run('changed-output',e=>{row(e).result.response.error.packId='wrong';}); +run('changed-diagnostic',e=>{row(e).result.response.error.message='ReDoS forged';}); +run('dropped-row',e=>e.splice(2,1)); +run('dropped-meta',e=>e.shift()); +run('stale-meta',e=>{e[0].ts=1;e[0].body.startedAt=1;}); +run('wrong-sha',e=>{e[0].body.aegis.sha='0'.repeat(40);}); +run('dirty-package',e=>{e[0].body.aegis.dirty='M loader.ts';}); +run('forged-score',e=>{e.at(-1).scores.aegisResolutionAccuracy=0;}); +run('forged-result',(_e,r)=>{r.summary={};}); +run('unknown-scenario',e=>{row(e).id='renamed';}); +run('missing-recovery',e=>{delete row(e).result.response.recovery;}); +run('wrong-runid',e=>{e[0].body.runId='other';}); +run('wrong-watchdog',e=>{e[0].body.watchdogMs=5000;}); +console.log('exp66 verifier: positive +15 tamper controls passed'); diff --git a/experiments/66-loader-probe-preemption/verify-receipts.mjs b/experiments/66-loader-probe-preemption/verify-receipts.mjs new file mode 100644 index 0000000..45769c9 --- /dev/null +++ b/experiments/66-loader-probe-preemption/verify-receipts.mjs @@ -0,0 +1,42 @@ +import {readFileSync,existsSync} from 'node:fs'; +import {resolve,join,basename} from 'node:path'; +import {createHash} from 'node:crypto'; +import {isDeepStrictEqual as same} from 'node:util'; +import {scenarios,negativeControls,experiment,seed,WATCHDOG_MS,IMPORT_MS} from './scenarios.mjs'; +import {assertRows,metrics,expected} from './oracle.mjs'; +const root=resolve(import.meta.dirname,'../..'),dir=import.meta.dirname; +const hash=v=>`sha256:${createHash('sha256').update(v).digest('hex')}`; +const check=(v,m)=>{if(!v)throw Error(m);}; +const receipts=JSON.parse(readFileSync(join(dir,'receipts.json'),'utf8')); +const normalized=rows=>rows.map(r=>({...r,result:Object.fromEntries(Object.entries(r.result).filter(([k])=>!['readyAt','startAt','endAt'].includes(k)))})); +const manifest=receipts.harnessManifest.map(f=>({file:f.file,sha256:hash(readFileSync(join(dir,f.file)))})); +check(same(manifest,receipts.harnessManifest)&&hash(JSON.stringify(manifest))===receipts.harnessSha256,'frozen harness manifest'); +let baselineRows,greenRows; +function inspect(trace,pin,live=false,since=0){ + const bytes=readFileSync(trace),events=bytes.toString().trim().split('\n').map(JSON.parse); + check(events.length===26,'exact26 events'); + const meta=events[0];check(meta.t==='message'&&meta.topic==='meta'&&meta.from==='moderator','metadata header'); + const m=meta.body;check(m.runId===basename(trace,'.jsonl')&&m.experiment===experiment&&m.seed===seed,'run identity'); + check(same(m.scenarios,scenarios)&&same(m.negativeControls,negativeControls)&&m.scenarioSha256===hash(JSON.stringify({scenarios,negativeControls})),'scenario identity'); + check(m.watchdogMs===WATCHDOG_MS&&m.importMs===IMPORT_MS&&same(m.harnessManifest,manifest)&&m.harnessSha256===receipts.harnessSha256,'design bound'); + check(m.aegis.dirty===''&&m.aegis.manifestSha256===hash(JSON.stringify(m.aegis.manifest)),'clean artifact manifest'); + const rows=events.filter(e=>e.t==='message'&&e.topic==='scenario'&&e.from==='real-aegis').map(e=>e.body),controls=events.filter(e=>e.t==='message'&&e.topic==='scenario'&&e.from==='substring-control').map(e=>e.body); + assertRows(rows,controls);const summary=metrics(rows,controls);check(same(events.at(-1).scores,summary),'recomputed score'); + for(const r of [...rows,...controls]){const x=r.result;check(x.readyAt!==null&&x.startAt!==null&&x.endAt>=x.startAt,'import complete/timing');if(x.status==='watchdog')check(x.endAt-x.startAt>=WATCHDOG_MS&&x.signal==='SIGKILL'&&!x.response,'real watchdog');if(x.status==='returned')check(x.exit===0&&x.signal===null&&x.stdout===''&&x.stderr==='','normal return/clean child');} + if(live){ + check(Number.isFinite(since)&&since>0&&m.startedAt>=since&&meta.ts>=since&&meta.ts<=Date.now()+5000,'live freshness'); + check(!receipts.runs.some(r=>r.runId===m.runId),'frozen run not live'); + const candidate=receipts.runs.find(r=>r.role==='candidate');check(m.aegis.sha===receipts.candidateSha&&same(m.aegis.manifest,candidate.aegis.manifest),'pinned real candidate'); + for(const f of m.aegis.manifest){check(!f.file.includes('..')&&!f.file.startsWith('/'),'safe manifest path');const actual=join(m.aegis.repo,f.file);check(existsSync(actual)&&hash(readFileSync(actual))===f.sha256,`actual artifact ${f.file}`);} + check(same(summary,expected),'live green');check(same(normalized([...rows,...controls]),greenRows),'full both-arm observable rows equal admitted green'); + }else{ + check(hash(bytes)===pin.traceSha256&&m.runId===pin.runId&&same(m.aegis,pin.aegis)&&same(summary,pin.metrics),'frozen receipt'); + if(pin.role==='baseline'){baselineRows=normalized([...rows,...controls]);check(summary.aegisProbeWatchdogRate===1&&summary.aegisHazardRejectionRate===0&&summary.aegisControlAccuracy===12/14&&summary.substringControlAccuracy===1,'admitted red');} + if(pin.role==='baseline-repeat')check(same(normalized([...rows,...controls]),baselineRows),'repeated exact deterministic observations'); + if(pin.role==='candidate'){check(same(summary,expected)&&m.aegis.sha===receipts.candidateSha,'admitted green');greenRows=normalized([...rows,...controls]);} + } + return {runId:m.runId,summary,events:events.length}; +} +for(const r of receipts.runs)inspect(join(root,r.trace),r); +const at=process.argv.indexOf('--run');if(at>=0){const result=JSON.parse(readFileSync(process.argv[at+1],'utf8'));const si=process.argv.indexOf('--since');check(si>=0,'--since required for live');const proof=inspect(result.trace,null,true,Number(process.argv[si+1]));check(result.runId===proof.runId&&result.replayEvents===proof.events&&result.traceSha256===hash(readFileSync(result.trace))&&same(result.summary,proof.summary)&&result.passed===true,'result log bound to trace');} +console.log('exp66 receipts verified: frozen repeated-red/green; exact design/observations, scores, package manifests'+(at>=0?' + fresh real artifact run':'')); From 327ad3d0d88abca0fdfc4898fc38f1b21065e27c Mon Sep 17 00:00:00 2001 From: heybeaux Date: Sun, 18 Oct 2026 23:52:23 -0700 Subject: [PATCH 4/5] docs(exp66): link paired PRs and observed Aegis CI without lifting merge hold --- experiments/66-loader-probe-preemption/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/experiments/66-loader-probe-preemption/README.md b/experiments/66-loader-probe-preemption/README.md index a3ac170..b38c392 100644 --- a/experiments/66-loader-probe-preemption/README.md +++ b/experiments/66-loader-probe-preemption/README.md @@ -45,3 +45,5 @@ probe alphabet may load but block runtime evaluation; pre-register a separate bounded evaluation watchdog/host-input contract before testing, not tonight. Aegis: https://github.com/heybeaux/aegis/pull/72. SwarmLab paired PR recorded below once opened. + +Paired SwarmLab PR: https://github.com/heybeaux/swarmlab/pull/56. Aegis release CI https://github.com/heybeaux/aegis/actions/runs/39212271270 observed SUCCESS at db899177; independent wall-clock compatibility merge hold remains. From b1bbd747e34f5b33e7110336ab43ed099903ebc5 Mon Sep 17 00:00:00 2001 From: heybeaux Date: Mon, 19 Oct 2026 00:03:21 -0700 Subject: [PATCH 5/5] test(exp66): reject deadline and result-log inconsistencies in live evidence --- .../66-loader-probe-preemption/README.md | 14 ++++++ .../test-verifier.mjs | 22 +++++++++- .../verify-receipts.mjs | 44 ++++++++++++++++--- 3 files changed, 74 insertions(+), 6 deletions(-) diff --git a/experiments/66-loader-probe-preemption/README.md b/experiments/66-loader-probe-preemption/README.md index b38c392..454e4e1 100644 --- a/experiments/66-loader-probe-preemption/README.md +++ b/experiments/66-loader-probe-preemption/README.md @@ -47,3 +47,17 @@ bounded evaluation watchdog/host-input contract before testing, not tonight. Aegis: https://github.com/heybeaux/aegis/pull/72. SwarmLab paired PR recorded below once opened. Paired SwarmLab PR: https://github.com/heybeaux/swarmlab/pull/56. Aegis release CI https://github.com/heybeaux/aegis/actions/runs/39212271270 observed SUCCESS at db899177; independent wall-clock compatibility merge hold remains. + + +### Same-experiment observable evidence audit + +An independent interface check found the live verifier accepted late returns, +missing/future/stale clocks, a retagged score event and contradictory result-log +metadata. These are **evidence-gate gaps**, not another nightly experiment or a +new Aegis runtime improvement. The verifier now enforces ordered run/event/child +timestamps, actual return strictly before the frozen1500ms watchdog, exact score/ +scenario envelopes and full result-log/trace metadata agreement. All existing +frozen harness/spec/scenarios/thresholds/receipts/traces are byte-identical. +Added18 tamper negatives to the existing15: **33 rejected + positive accepted**. +No threshold adjustment, whitelist or retry hides the safe-builtin timing failure. +**PR72/PR56 remain held, not merged**, despite local/remote green. diff --git a/experiments/66-loader-probe-preemption/test-verifier.mjs b/experiments/66-loader-probe-preemption/test-verifier.mjs index a2429d7..f0825b6 100644 --- a/experiments/66-loader-probe-preemption/test-verifier.mjs +++ b/experiments/66-loader-probe-preemption/test-verifier.mjs @@ -28,4 +28,24 @@ run('unknown-scenario',e=>{row(e).id='renamed';}); run('missing-recovery',e=>{delete row(e).result.response.recovery;}); run('wrong-runid',e=>{e[0].body.runId='other';}); run('wrong-watchdog',e=>{e[0].body.watchdogMs=5000;}); -console.log('exp66 verifier: positive +15 tamper controls passed'); +// Observable-interface audit: a deadline claim needs actual ordered timestamps, +// and the summary/log must describe the very trace whose outcomes were checked. +run('late-library-return',e=>{row(e).result.endAt=row(e).result.startAt+1501;}); +run('missing-ready',e=>{delete row(e).result.readyAt;}); +run('future-run-start',e=>{e[0].body.startedAt=Date.now()+86400000;}); +run('stale-row-time',e=>{const x=row(e).result;x.readyAt=1;x.startAt=1;x.endAt=2;}); +run('retagged-score-event',e=>{e.at(-1).t='not-a-score';}); +run('unbound-result-artifact',(_e,r)=>{r.aegis={sha:'FORGED'};}); +run('missing-result-timestamps',(_e,r)=>{delete r.startedAt;delete r.endedAt;}); +run('contradictory-failed-list',(_e,r)=>{r.failed=[{id:'nested-a',status:'watchdog'}];}); +run('missing-result-harness',(_e,r)=>{delete r.harnessManifest;}); +run('forged-result-harness',(_e,r)=>{r.harnessSha256='sha256:forged';}); +run('future-result-end',(_e,r)=>{r.endedAt=Date.now()+86400000;}); +run('reversed-row-times',e=>{const x=row(e).result;x.readyAt=x.startAt+10;}); +run('row-before-run',e=>{row(e).result.startAt=e[0].body.startedAt-1;}); +run('row-event-before-return',e=>{e.find(x=>x.topic==='scenario').ts=row(e).result.endAt-1;}); +run('invalid-row-clock',e=>{row(e).result.readyAt=null;}); +run('wrong-score-envelope',e=>{e.at(-1).from='fake';}); +run('wrong-scenario-envelope',e=>{e.find(x=>x.topic==='scenario').to='somewhere-else';}); +run('stale-score-time',e=>{e.at(-1).ts=1;}); +console.log('exp66 verifier: positive +33 tamper controls passed'); diff --git a/experiments/66-loader-probe-preemption/verify-receipts.mjs b/experiments/66-loader-probe-preemption/verify-receipts.mjs index 45769c9..8b682d4 100644 --- a/experiments/66-loader-probe-preemption/verify-receipts.mjs +++ b/experiments/66-loader-probe-preemption/verify-receipts.mjs @@ -11,18 +11,44 @@ const receipts=JSON.parse(readFileSync(join(dir,'receipts.json'),'utf8')); const normalized=rows=>rows.map(r=>({...r,result:Object.fromEntries(Object.entries(r.result).filter(([k])=>!['readyAt','startAt','endAt'].includes(k)))})); const manifest=receipts.harnessManifest.map(f=>({file:f.file,sha256:hash(readFileSync(join(dir,f.file)))})); check(same(manifest,receipts.harnessManifest)&&hash(JSON.stringify(manifest))===receipts.harnessSha256,'frozen harness manifest'); +const clock = (value, label) => { + check(Number.isSafeInteger(value) && value > 0 && value <= Date.now() + 5000, `valid ${label} clock`); +}; let baselineRows,greenRows; function inspect(trace,pin,live=false,since=0){ const bytes=readFileSync(trace),events=bytes.toString().trim().split('\n').map(JSON.parse); check(events.length===26,'exact26 events'); - const meta=events[0];check(meta.t==='message'&&meta.topic==='meta'&&meta.from==='moderator','metadata header'); + const meta=events[0];check(meta.t==='message'&&meta.topic==='meta'&&meta.from==='moderator'&&meta.to==='moderator','metadata header'); const m=meta.body;check(m.runId===basename(trace,'.jsonl')&&m.experiment===experiment&&m.seed===seed,'run identity'); check(same(m.scenarios,scenarios)&&same(m.negativeControls,negativeControls)&&m.scenarioSha256===hash(JSON.stringify({scenarios,negativeControls})),'scenario identity'); check(m.watchdogMs===WATCHDOG_MS&&m.importMs===IMPORT_MS&&same(m.harnessManifest,manifest)&&m.harnessSha256===receipts.harnessSha256,'design bound'); check(m.aegis.dirty===''&&m.aegis.manifestSha256===hash(JSON.stringify(m.aegis.manifest)),'clean artifact manifest'); + clock(m.startedAt,'run start'); + let previous = m.startedAt; + for(const event of events) { + clock(event.ts,'event');check(event.ts>=previous,'event time order');previous=event.ts; + } + check(events.at(-1).ts-m.startedAt<120_000,'bounded experiment duration'); + const defs=[...scenarios.map(s=>({from:'real-aegis',id:s.id})),...negativeControls.map(s=>({from:'substring-control',id:s.id}))]; + for(const [i,def] of defs.entries()) { + const event=events[i+1]; + check(same(Object.keys(event).sort(),['body','from','t','to','topic','ts']),`scenario envelope ${def.id}`); + check(event.t==='message'&&event.topic==='scenario'&&event.from===def.from&&event.to==='moderator'&&event.body.id===def.id,'ordered scenario envelope'); + } + const score=events.at(-1); + check(same(Object.keys(score).sort(),['scores','t','ts'])&&score.t==='score','score envelope'); const rows=events.filter(e=>e.t==='message'&&e.topic==='scenario'&&e.from==='real-aegis').map(e=>e.body),controls=events.filter(e=>e.t==='message'&&e.topic==='scenario'&&e.from==='substring-control').map(e=>e.body); assertRows(rows,controls);const summary=metrics(rows,controls);check(same(events.at(-1).scores,summary),'recomputed score'); - for(const r of [...rows,...controls]){const x=r.result;check(x.readyAt!==null&&x.startAt!==null&&x.endAt>=x.startAt,'import complete/timing');if(x.status==='watchdog')check(x.endAt-x.startAt>=WATCHDOG_MS&&x.signal==='SIGKILL'&&!x.response,'real watchdog');if(x.status==='returned')check(x.exit===0&&x.signal===null&&x.stdout===''&&x.stderr==='','normal return/clean child');} + for(const [i,r] of [...rows,...controls].entries()) { + const x=r.result; + for(const k of ['readyAt','startAt','endAt'])clock(x[k],k); + check(events[i].ts<=x.readyAt&&x.readyAt<=x.startAt&&x.startAt<=x.endAt&&x.endAt<=events[i+1].ts,'ordered child/run timing'); + if(x.status==='watchdog')check(x.endAt-x.startAt>=WATCHDOG_MS&&x.signal==='SIGKILL'&&!x.response,'real watchdog'); + if(x.status==='returned') { + check(x.endAt-x.startAt0&&m.startedAt>=since&&meta.ts>=since&&meta.ts<=Date.now()+5000,'live freshness'); check(!receipts.runs.some(r=>r.runId===m.runId),'frozen run not live'); @@ -30,13 +56,21 @@ function inspect(trace,pin,live=false,since=0){ for(const f of m.aegis.manifest){check(!f.file.includes('..')&&!f.file.startsWith('/'),'safe manifest path');const actual=join(m.aegis.repo,f.file);check(existsSync(actual)&&hash(readFileSync(actual))===f.sha256,`actual artifact ${f.file}`);} check(same(summary,expected),'live green');check(same(normalized([...rows,...controls]),greenRows),'full both-arm observable rows equal admitted green'); }else{ - check(hash(bytes)===pin.traceSha256&&m.runId===pin.runId&&same(m.aegis,pin.aegis)&&same(summary,pin.metrics),'frozen receipt'); + check(hash(bytes)===pin.traceSha256&&m.runId===pin.runId&&same(m.aegis,pin.aegis)&&same(summary,pin.metrics)&&m.startedAt===pin.startedAt,'frozen receipt');clock(pin.endedAt,'frozen end');check(pin.endedAt>=score.ts,'frozen run completion'); if(pin.role==='baseline'){baselineRows=normalized([...rows,...controls]);check(summary.aegisProbeWatchdogRate===1&&summary.aegisHazardRejectionRate===0&&summary.aegisControlAccuracy===12/14&&summary.substringControlAccuracy===1,'admitted red');} if(pin.role==='baseline-repeat')check(same(normalized([...rows,...controls]),baselineRows),'repeated exact deterministic observations'); if(pin.role==='candidate'){check(same(summary,expected)&&m.aegis.sha===receipts.candidateSha,'admitted green');greenRows=normalized([...rows,...controls]);} } - return {runId:m.runId,summary,events:events.length}; + return {runId:m.runId,summary,events:events.length,metadata:m,scoreAt:score.ts,failed:rows.filter(r=>!r.correct).map(r=>({id:r.id,checks:r.checks,status:r.result.status}))}; } for(const r of receipts.runs)inspect(join(root,r.trace),r); -const at=process.argv.indexOf('--run');if(at>=0){const result=JSON.parse(readFileSync(process.argv[at+1],'utf8'));const si=process.argv.indexOf('--since');check(si>=0,'--since required for live');const proof=inspect(result.trace,null,true,Number(process.argv[si+1]));check(result.runId===proof.runId&&result.replayEvents===proof.events&&result.traceSha256===hash(readFileSync(result.trace))&&same(result.summary,proof.summary)&&result.passed===true,'result log bound to trace');} +const args=process.argv.slice(2),at=args.indexOf('--run'); +if(at>=0) { + check(args.length===4&&args.filter(a=>a==='--run').length===1&&args.filter(a=>a==='--since').length===1,'exact --run/--since arguments'); + const si=args.indexOf('--since');check(si>=0&&args[at+1]&&!args[at+1].startsWith('--')&&args[si+1]&&!args[si+1].startsWith('--'),'--since required for live'); + const result=JSON.parse(readFileSync(args[at+1],'utf8')),proof=inspect(result.trace,null,true,Number(args[si+1])); + check(result.runId===proof.runId&&result.replayEvents===proof.events&&result.traceSha256===hash(readFileSync(result.trace))&&same(result.summary,proof.summary)&&result.passed===true,'result log bound to trace'); + check(result.startedAt===proof.metadata.startedAt&&same(result.aegis,proof.metadata.aegis)&&same(result.harnessManifest,proof.metadata.harnessManifest)&&result.harnessSha256===proof.metadata.harnessSha256&&same(result.failed,proof.failed),'result metadata bound to trace'); + clock(result.startedAt,'result start');clock(result.endedAt,'result end');check(result.endedAt>=proof.scoreAt,'result completion bound to trace'); +} else check(args.length===0,'unknown arguments'); console.log('exp66 receipts verified: frozen repeated-red/green; exact design/observations, scores, package manifests'+(at>=0?' + fresh real artifact run':''));