Skip to content

Kurtosis reproduction of confirmed findings #92

Closed as not planned
Closed as not planned

Description

@grandchildrice

M2 · track:2-C · repro — target 8/22 (step 6) · depends on the cross-client audit (#91) and Lean provider (#88) issues · part of #86

Background

Each confirmed finding surviving the 04 review gates gets a Kurtosis E2E reproduction — the reproduction is the PoC, per the Milestone 1 convention. Implement the kurtosis verification backend behind the selectable post-04 VerificationBackend seam from #87, resolving to the external NyxFoundation/kurtosis-harness plugin (the E2E reproduction/verification base for Ethereum clients). Where a property already shipped a Kurtosis fixture from the Lean provider (#88), reuse it. Findings that don't reproduce are labeled as such; reproducible-and-unfixed ones are flagged for coordinated client disclosure.

Where Kurtosis runs — dedicated external plugin

The E2E machinery lives in NyxFoundation/kurtosis-harness, referenced by speca as a version-pinned, officially-certified plugin via the verification-backend boundary defined in #87 (the Kurtosis devnet machinery stays out of core speca). speca's kurtosis backend invokes the plugin across that seam, running strictly after the 04 gates on the filtered confirmed findings.

Tasks

  1. Implement the kurtosis VerificationBackend (the post-04 seam from Pluggable property providers and reproduction backends #87): dispatch to NyxFoundation/kurtosis-harness, bring up the devnet, run the fixture / assertion against the pinned client, capture verdict + logs. Pin the kurtosis-harness version from speca.
  2. For findings whose property carries a kurtosis_test (from Lean4 FV property provider: EIP spec → 01e properties + Kurtosis tests #88), run it directly; for the rest, generate the devnet scenario (reuse prompts/05_poc.md logic where it helps).
  3. Record one reproduction/verification result per finding (reproduced | not-reproduced | error, plus artifact + logs paths).
  4. Check fix status against the target's latest commit; flag reproduced && unfixed as disclosure-pending.
  5. Time-box it: the harness is heavy across a full client × finding matrix (Milestone 1 showed this). Batch the runs, start early, record cost.
  6. Tests: the backend returns the right verdict on one known-reproducible and one known-not fixture.

Interfaces & contracts

Done when

  • NyxFoundation/kurtosis-harness is referenced by speca as an official, version-pinned plugin and selected via the kurtosis verification backend.
  • Every confirmed finding from the cross-client audit (Cross-client audit run from the full property set #91) has a reproduction verdict + evidence.
  • Reproducible-and-unfixed findings are listed for disclosure.
  • The run fits the 8/22 → 8/31 window with buffer for the report (flag early if it won't).

Review checklist

Out of scope

Writing the cumulative report (#93). Public posting of unfixed findings — disclosure is a human call.

Activity

  1. added
    M2Milestone 2 grant work
    track:2-CDeliverable 2-C: cumulative bug & fix report
    reproReproduction / PoC
    on Jun 30, 2026
  2. grandchildrice commented on Jul 19, 2026

    @grandchildrice
    CollaboratorAuthor

    論文改訂(#109)との抱き合わせ依頼: 再現対象を選ぶとき、論文の「正解超え候補12件のうち外部検証が Level D 止まりの10件」を優先してほしい。ここで再現が通った分だけ、グラント成果物(2-C)と論文改訂の証拠が同時に強くなる。候補リストは #109 参照。

  3. grandchildrice commented on Jul 22, 2026

    @grandchildrice
    CollaboratorAuthor

    2026-07-22 M2 再構成のためクローズ。 3トラック(a=gasper / b=EL / c=vuln-dataset)× 4フェーズの新構成に作り直しました → トラッカー #86。

    この issue の引き継ぎ先: トラック別の kurtosis 再現 a-4 #151 / b-4 #152 / c-4 #153 に分割。

    作業内容・議論は新 issue とトラッカーに移行済みです。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

M2Milestone 2 grant workreproReproduction / PoCtrack:2-CDeliverable 2-C: cumulative bug & fix report

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions