Label-repair for skin-lesion classification under injected label noise, scored against a matched-budget random-repair control and a clean-label upper bound.
Solo research project. Manuscript only — not published, not peer-reviewed, not submitted.
ConvNeXt-Tiny, calibrated test metrics, mean ± std. HAM10000 arms use 5 seeds, anchors 3.
| Metric | Value | Evidence |
|---|---|---|
| HAM10000 accuracy — trusted repair | 0.8350 ± 0.0059 | internal_main/ablation_table.json |
| HAM10000 accuracy — no repair | 0.8321 ± 0.0071 | same |
| HAM10000 accuracy — random repair, matched budget | 0.8319 ± 0.0051 | same |
| ECE — trusted repair | 0.0445 ± 0.0090 | same |
| Clean-label upper bound (3 seeds) | 0.8521 ± 0.0055 | noisy_anchor/ablation_table.json |
| Generalized cross-entropy, no repair (3 seeds) | 0.6774 ± 0.0012, macro-F1 0.1224 | same |
| External ISIC 2019 accuracy — trusted repair | 0.5692 ± 0.0145, ECE 0.2000 ± 0.0253 | external_main/ablation_table.json |
| Repair − no repair, calibrated macro-F1 | +0.0007, 95% CI [−0.0231, +0.0217] | paired_significance.md |
Repair buys 0.3 accuracy points over doing nothing and loses to spending the same budget at random on AUROC and AURC. Every paired interval crosses zero. Under domain shift accuracy falls 0.835 → 0.569 and ECE rises 0.0445 → 0.2000 after temperature scaling — a gap far larger than any difference between the three arms.
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest -q # 31 passed in 6.7 s, CPU, no dataset
python scripts/run_experiment.py --config configs/quick_cifar100.yaml # ~80 s end-to-end pipeline smokeThe CIFAR-100 smoke run reaches ~1.5% accuracy on 100 classes — that is the correct outcome for 1 epoch on 512 images from scratch. It proves the pipeline runs; it is not a result. HAM10000 and ISIC 2019 are not redistributed here; scripts/prepare_ham10000.py, scripts/run_multiseed_experiment.py and scripts/run_external_validation.py consume local copies, and the full sweep needs a GPU.
- No arm wins. Trusted repair does not beat the matched-budget random control on the metrics this pipeline exists to improve (AUROC, AURC).
- 5 seeds: the exact sign-flip test cannot report p below 0.0625, and the anchors use 3. No comparison here is statistically significant.
deterministic: false— cuDNN determinism was not enforced, so a bit-for-bit rerun is not guaranteed even at fixed seed.- The label noise is injected (transition matrix, rate 0.2, realized flip rate 0.039–0.048). Real dermatology label noise is not symmetric, independent, or known.
- 14,185 near-duplicate HAM10000↔ISIC pairs at dHash threshold 4 were counted but never adjudicated, so external-validation leakage cannot be excluded.
- One backbone, one dataset pair. No SelectiveNet, DivideMix or SWAG comparison.
- ECE 0.20 on external data. This model must not be used to triage anything.
MIT — see LICENSE.