feat: add miner-bench harness for local miner self-test - #17
Open
dataCenter430 wants to merge 1 commit into
Open
dataCenter430 wants to merge 1 commit into
dataCenter430 wants to merge 1 commit into
Conversation
Adds `miner_bench/`: a standalone harness that lets miners evaluate attack strategies against the same scoring rules the mainnet validator applies (faithful port of `verify_and_score`), without consuming validator resources or polluting on-chain history. - scoring.py: dependency-light port of `PerturbValidator.verify_and_score` with the same SSIM/PSNR/Linf/RMSE gates and score combination. - challenge.py: Pexels fetch + Challenge dataclass; mirrors the validator's challenge generation (prompt rotation, photo sampling, label-from-model). - strategy.py: `MinerStrategy` ABC + `PGDStrategy` reference baseline (textbook untargeted PGD, respects validator's effective max-delta cap). - runner.py: `run_bench()` + CLI with per-challenge + aggregate reporting, optional JSON dump for downstream dashboards. - scripts/bench_miner.py: thin CLI entrypoint. - README in package: install, CLI flags, library API, plug-in interface, calibration notes vs mainnet. Smoke-tested end-to-end against live Pexels with EfficientNetV2-M on GPU: 5/5 PGD challenges scored cleanly through the validator-parity scorer.
dataCenter430
force-pushed
the
feat/miner-bench-harness
branch
from
May 18, 2026 20:10
c9e4cab to
bc9940a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
miner_bench/: a standalone harness that lets miners evaluate attack strategies against the same scoring rules the mainnet validator applies (faithful port ofverify_and_score), without consuming validator resources or polluting on-chain history.PerturbValidator.verify_and_scorewith the same SSIM/PSNR/Linf/RMSE gates and score combination.MinerStrategyABC +PGDStrategyreference baseline (textbook untargeted PGD, respects validator's effective max-delta cap).run_bench()+ CLI with per-challenge + aggregate reporting, optional JSON dump for downstream dashboards.Smoke-tested end-to-end against live Pexels with EfficientNetV2-M on GPU: 5/5 PGD challenges scored cleanly through the validator-parity scorer.
Test logs:
Sample JSON dump entry (--out results.json)