-
Notifications
You must be signed in to change notification settings - Fork 2
Reproducible Experiments
Use the top-level orchestrator for dependency checks, graph preparation, mapping generation, benchmarks, cache simulation, and verification:
python3 scripts/graphbrew_experiment.py --helpLarge graphs, mappings, and result artifacts should live on an external data partition rather than in the repository.
| Path | Purpose |
|---|---|
| Rapid | dependency checks, parser failures, mapping bugs, and small candidate comparisons |
| Controlled | fixed graph/algorithm policy, repeated trials, verification, affinity, and scheduler checks |
Rapid example:
python3 scripts/graphbrew_experiment.py \
--full --quick --size small --trials 1 --skip-cacheInspect a broader plan without executing it:
python3 scripts/graphbrew_experiment.py \
--target-graphs 50 --size small --dry-runRepository-local outputs are intended for bounded development runs:
results/
├── data/
├── graphs/
├── logs/
└── mappings/
Use explicit graph and output roots for large campaigns. Do not commit graph corpora, mappings, raw timing matrices, or machine-specific logs.
Keep these components separate:
representation build
mapping generation
permutation validation
CSR relocation
kernel execution
executed work
verification state
mapping fingerprint
scheduler, nice value, and affinity
For repeated use, report:
mapping + reuse x kernel
Never infer mapping cost from a kernel run that only loads a pre-generated mapping.
The generic harness can run one phase at a time:
python3 scripts/graphbrew_experiment.py --phase reorder --size small
python3 scripts/graphbrew_experiment.py --phase benchmark --size small
python3 scripts/graphbrew_experiment.py --phase cache --size smallSpecialized campaign runners reuse the same shared download, build, mapping, verification, and result-store contracts. Their release instructions are maintained separately from this generic workflow.
Before accepting a comparison:
- confirm graph provenance and dimensions;
- validate every permutation;
- ensure each kernel uses the intended mapping fingerprint;
- bind source, iteration, and weighted-kernel parameters;
- verify answers or deterministic signatures;
- confirm trials, threads, affinity, and binary identity;
- record scheduler and nice state for timing; and
- preserve failed and timed-out attempts.