KINTSUGI — artifact for Does Graph Structure Earn Its Place in Microservice Root-Cause Analysis?

A controlled graph-vs-flat ablation on RCAEval. Every arm gets identical features, optimiser, learning rate, weight decay, validation split, early-stopping rule and scoring head. Three learned arms: gnn, gnn_no_neigh — the same encoder with W_neigh · (Â h) and its parameters removed and nothing else changed — and a conventional flat mlp. Only the first pair differs by exactly one term; gnn vs. mlp also differs 5.8× in parameters, because an MLP is not a GNN with message passing deleted. Both contrasts are reported and labelled. We run this on three tasks, two benchmarks and two topology sources.

The finding: no reliable graph-specific effect anywhere we tested it. Seventeen matched contrasts, signs in both directions, none surviving replication as a structure effect. In-distribution gnn − mlp is essentially zero (+0.0003 Avg@5, p = 0.844 at a genuine n = 6) — a first version of this analysis, run on resampled rather than disjoint folds, saw a significant capacity effect that did not survive checking the folds were actually independent (register entries 53 and 56). What separates methods is whether a method is learned at all versus a naive heuristic — and which of two opposite hard conditions you test under.

Paper (release)	paper/preprint.tex · PDF
Paper (Markdown draft)	PAPER.md
Results, generated from manifests	RESULTS.md
Figures, generated from manifests	figures/
Defect register (62 entries)	STATUS.md
Narrative log	PROGRESS.md
Pre-committed decision rule	PROJECT_PLAN.md §6
Journal-extension plan	JOURNAL_EXTENSION.md

---

Verify the paper without running anything

The evidence base — five seed manifests, 48 KB — is in the repository. Every number in the paper is checked against it:

python scripts/verify_paper.py --run output/real

Exit 0 means every number quoted in PAPER.md, in tables and in prose, is backed by a manifest. Read that as a drift detector, not a fraud detector: the accepted set is large enough that a value invented at random from the paper's own range is rejected only about half the time. What it does catch reliably — and what actually went wrong here — is a number that went stale when its experiment was re-run. The checker has also twice degraded into one that could not fail (register entries 29 and 32), so it has a self-test that tampers with a real number, confirms rejection, and prints the measured catch rate:

python scripts/verify_paper.py --run output/real --negative-control

That runs on a temporary copy and never touches the manuscript. It also runs on every pytest, so a change that breaks the checker breaks the suite.

Regenerate the tables and figures

Both are derived from the manifests, and both are byte-reproducible — running these on a clean checkout should leave git status empty:

python scripts/make_tables.py --run output/real --out RESULTS.md

python scripts/make_figures.py --run output/real

figures/figure_data.json records the manifest key behind every plotted value.

Run the tests

python -m pytest -q

142 tests, roughly 8–14 minutes — model training dominates.

---

Reproduce the experiments from scratch

Only needed if you want to regenerate the manifests rather than check them.

1. Get the data. RCAEval RE1 and RE2 come from Zenodo record 14590730 (MIT, ~4.9 GB unpacked). This prints what is present, what is missing, and the exact directory layout the loader expects — it reports, it does not download:

python scripts/download_data.py

2. Run an experiment. Each writes output/real/experiments/<ID>/seed_manifest.json:

python scripts/run_experiment.py E7 --output-dir output/real

ID	What it is	Paper
E7	Localization on RE1, stratified and cross-system	§5
E8	Localization on RE2, induced vs. real co-location topology, with blind-spot regimes	§6
E3	Recovery-action selection vs. baselines	§7
E4	Recovery-action graph-necessity ablations	§7

E4 is the long one — 7 conditions × 9 runs, roughly 3.7 hours. E3 and E4 both checkpoint per seed and per condition, so an interrupted run resumes rather than restarting (register entry 27; a spot VM was reclaimed mid-run during the real campaign and lost nothing).

E1, E2, E2b, E5 and E6 exist in the tree but are retired — two are structural zeros and three were superseded by the real-data pivot. RESULTS.md gives the reason for each.

---

Reading the numbers

Two things are easy to misread, so they are stated here as well as in the paper.

The leakage stamp in every manifest measures the recovery-action task, the one whose labels are ours. It travels with all manifests as a dataset-level record. Localization is scored against RCAEval's injected service — a label this codebase does not author and therefore cannot leak into. The generated tables say so above the localization results.

Significance is underpowered by construction in the localization studies. The independent unit for cross-system transfer is the held-out system, and RCAEval has three; a paired Wilcoxon test at n = 3 cannot go below p = 0.25 whatever the effect. The paper leads on effect sizes and per-fold values for that reason, and §4.6 explains what the design could and could not have resolved.

---

Requirements

Python 3.12, PyTorch, NumPy, SciPy, pandas, matplotlib. pip install -e .

License

• Code and scripts: MIT, see LICENSE.
• Paper text, figures, and documentation: CC BY 4.0, see LICENSE-CONTENT.md.

Citation

The preprint is archived on Zenodo: DOI 10.5281/zenodo.22832168. An arXiv version is planned; its identifier is added here after the announcement. Machine-readable metadata: CITATION.cff.

The dataset this study uses is RCAEval, Zenodo record 10.5281/zenodo.14590730 (MIT).

The released PDF (paper/preprint.pdf) has sha256 23d7d2644c9ef0a0d391547e73ca8c56c9cf605d01e86747bcb10d147615fd21.