feat: pipeline hardening — reproducibility, loss SSOT, paired crop, test suite - #1
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Hardens the RGB→IR pipeline by making augmentation and training behavior reproducible, removing loss-selection drift via an architecture registry, adding a paired random crop augmentation, and introducing an initial unit test suite to lock in behavior.
Changes:
- Add deterministic global seeding + stateless, per-element-seeded augmentation (including paired RGB/IR random crop).
- Derive “advanced loss” selection from an architecture registry (SSOT) and centralize loss/crop settings.
- Introduce a pytest unit test suite covering core pipeline components and update tooling/docs accordingly.
Reviewed changes
Copilot reviewed 21 out of 25 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_visualization.py | Adds smoke tests ensuring plotting helpers return Matplotlib figures headlessly. |
| tests/unit/test_trainer.py | Tests architecture-based advanced-loss selection, unknown-arch handling, and checkpoint-missing behavior. |
| tests/unit/test_reproducibility.py | Verifies set_global_seed makes TF RNG and weight init reproducible. |
| tests/unit/test_models.py | Validates core model builders’ output shapes/ranges (excluding EfficientNet download). |
| tests/unit/test_metrics.py | Adds correctness tests for SSIM/PSNR metric wrappers and reset behavior. |
| tests/unit/test_losses.py | Adds tests for loss factories and basic invariants (zero on identical, positive otherwise). |
| tests/unit/test_inference_utils.py | Tests Gaussian blending helpers and new patch-size validation in overlap inference. |
| tests/unit/test_dataset.py | Tests pair loading, leakage-free grouped split, padding, cropping validation, and augmentation reproducibility. |
| tests/unit/test_config.py | Adds basic invariants for config ratios/weights/multiples and crop default. |
| tests/unit/test_augmentation.py | Tests stateless determinism, shape preservation, RGB-only photometric jitter, and paired crop alignment. |
| tests/conftest.py | Provides shared fixtures for tiny paired datasets and a minimal dummy model for fast tests. |
| scripts/trainer.py | Introduces uses_advanced_loss registry; compiles/loads models with loss derived from architecture + settings. |
| scripts/reproducibility.py | Adds set_global_seed() wrapper for unified seeding across TF/Python/NumPy (via TF utility). |
| scripts/losses.py | Aligns fft_loss docstring with implementation (normalization by image area). |
| scripts/inference_utils.py | Adds patch-size divisibility validation and minor formatting cleanup. |
| scripts/efficientnet_unet.py | Minor refactor for readability when unpacking encoder outputs. |
| scripts/dataset.py | Adds crop_size plumbing and deterministic stateless augmentation seeding via a per-element counter. |
| scripts/config.py | Centralizes advanced-loss weights, Laplacian levels, and crop size in Settings. |
| scripts/augmentation.py | Rewrites augmentation to be stateless/seeded; adds paired random crop and stateless photometric ops. |
| requirements.txt | Pins dependencies to specific versions (and removes unpinned entries). |
| README.md | Updates augmentation/testing/docs and project structure description. |
| pyproject.toml | Adjusts pytest pathing and adds coverage configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+54
to
+58
| shape = tf.shape(stacked) | ||
| max_y = tf.cast(shape[0] - crop_size + 1, tf.float32) | ||
| max_x = tf.cast(shape[1] - crop_size + 1, tf.float32) | ||
| ry = tf.random.stateless_uniform((), seed=seeds[4]) | ||
| rx = tf.random.stateless_uniform((), seed=seeds[5]) |
Comment on lines
+218
to
+220
| # Pair each element with a monotonic counter so the stateless | ||
| # augmentation seed is deterministic per sample (and varies across | ||
| # epochs), making the augmented stream reproducible run-to-run. |
Comment on lines
+47
to
+51
| CROP_SIZE : int or None | ||
| If set, training augmentation randomly crops each pair to a square | ||
| of this side length (must be a multiple of ``PATCH_MULTIPLE``). | ||
| ``None`` disables cropping. Applied only to the augmented (training) | ||
| split — never at evaluation or inference. |
Comment on lines
+10
to
+12
| def test_advanced_loss_weights_sum_to_one() -> None: | ||
| total = settings.ADV_LOSS_ALPHA + settings.ADV_LOSS_BETA + settings.ADV_LOSS_GAMMA | ||
| assert total == 1.0 |
Comment on lines
+15
to
+16
| def test_patch_multiple_is_power_of_two_divisible_by_pooling() -> None: | ||
| assert settings.PATCH_MULTIPLE % 16 == 0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Hardening of the RGB→IR pipeline following a project review: reproducibility, correctness fixes, a new augmentation, and the first test suite.
Changes by topic
Reproducibility & augmentation
scripts/reproducibility.pywithset_global_seed()(seeds Python/NumPy/TF fromsettings.SEED).stateless_uniform, compatible with the Metal backend). Active only whenaugment=True→ evaluation/inference stay full-image.Correctness fixes (review)
uses_advanced_loss): no more manualadvanced_lossflag that could drift between training and evaluation.Settings.predict_with_overlapnow validates thatpatch_sizeis a multiple of 16.fft_loss: docstring aligned with the actual behavior (normalization by image areaH·W).Tooling & tests
tests/unit/) mirroringscripts/: 75 tests, 89% coverage (including the per-artwork split anti-leakage test). EfficientNet is excluded from tests (it would download ImageNet weights).Notebooks & docs
Verification
ruff check+ruff format --checkclean onscripts/andtests/.pytest --cov=scripts→ 75 passed, 89%.Notes
CROP_SIZEdefaults toNone(crop disabled → training behavior unchanged).