Repository navigation
Description
M2 · track:2-C · track:2-D · pipeline — target 7/31 (step 4) · depends on the Lean provider and dataset issues · blocks the cross-client audit issue · part of #86
Background
Run the property stage (01a → 01b → 01e, with the Lean provider where it applies) across the full in-scope EIP set, then measure coverage against the refined bug DB: for each historical bug, is there a generated property that would catch it? Gaps feed back into generation. The paper states it plainly — "as models improve, property generation becomes the binding constraint on recall" — so closing coverage gaps here is the highest-leverage work for 2-C. Keep the feedback loop bounded: full generation + a coverage report is the must-have; the iteration is the stretch and must not slip 8/31.
Tasks
- Assemble the EIP set (start from the Fusaka corpus referenced in
outputs/01a_STATE.json; confirm current scope), and run01a → 01b → 01eover all of it. - Build the coverage check: join generated
01eproperties to bug-DB rows oneip_or_spec+spec_clause/category; mark each historical bug as covered / not covered by ≥1 property. - Emit a coverage report (
outputs/coverage/+ a chart underbenchmarks/): counts plus the uncovered list with reasons. - (Stretch) For uncovered clauses, feed the gap back into generation (prompt or Lean) and re-measure; record before / after.
- Tests: the coverage join is correct on a small fixture with known covered and known uncovered rows.
Interfaces & contracts
- Coverage record:
{ "bug_id": "...", "covered": true, "matched_property_ids": [], "reason_if_uncovered": "" }.
Done when
01eproperties exist for the full set; the run is resumable.- The coverage report lists covered / uncovered historical bugs with reasons.
- (Stretch) at least one iteration shows a measurable coverage increase.
Review checklist
- The join key matches the dataset schema exactly — no silent misses from key drift.
- The uncovered list carries reasons, not just a count.
- The stretch loop is genuinely optional and cannot block the deliverable.
- Token / cost stays inside the per-phase budget (the harness enforces it; confirm no override).
Out of scope
Running the audit on clients. Reproduction.
Activity
grandchildrice commentedon Jul 19, 2026
歴史的脆弱性データセット(#89)がほぼ完成したので、こちらの cross-check 作業に着手可能です。
NyxFoundation/ethereum-vuln-datasetのdata/ethereum_vulns.parquet(2,225行)。スキーマはdata/manifest.json、閲覧用はdata/ethereum_vulns.preview.csv。spec_clause/categoryではなく、layer/label(プロトコル領域、設計はdocs/label_design.md)+stride/cwe_top25を使ってください。grandchildrice commentedon Jul 22, 2026
2026-07-22 M2 再構成のためクローズ。 3トラック(a=gasper / b=EL / c=vuln-dataset)× 4フェーズの新構成に作り直しました → トラッカー #86。
この issue の引き継ぎ先: トラック b/c(#144-#147)に分割。EL spec と vuln-dataset 起点の Lean→01e として再編。
作業内容・議論は新 issue とトラッカーに移行済みです。