-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathTODO
More file actions
114 lines (92 loc) · 6.35 KB
/
Copy pathTODO
File metadata and controls
114 lines (92 loc) · 6.35 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
# List of TODOs
## Planned changelog for next releases
- [x] v0.1.3
- [x] v0.2.X
- [x] Add prototype implementation of heads and backbones builders using registries
- [x] Fix unit tests
- [x] Add prototype implementation of dataset classes from paths
- [x] Complete 1st version of model profiler
- [x] Add prototype of model explainer for vector-to-vector regression using SHAP (basic explainer)
- [x] Add refactored version of augmentation pipeline (ImagesAugmentationHelper)
- [x] Refactor and improve ModelTrainingManager updating augmentation pipeline and adding batch accumulation
- [x] Improve and fix model evaluator
- [x] Implement new adapter type for basic engineered feature maps (image, brightness masks, pre-processing filters)
- [x] Fix issues with device selection query and some imports
- [x] Add support for automatic chaining of schedulers (todo debug learning rate update in Trainer?)
- [x] Update Trainer to save checkpoints in subfolder with mlflow experiment name
- [x] Upgrade translation augmentation to support translation of multiple (key)points
- [x] Implement custom rotation augmentation to support rotation of multiple (key)points, through warping --> Using kornia Augmentation Sequential
- [x] Update parametric template of CNN classes for different uses
- [x] Add semi-automatic function to run model inference on sample input and save to disk with resulting predictions
- [x] Implement Scaler adapter to store scaling factors for standalone input scaling (directly in forward)
- [x] Add prototype of automatic MC dropout wrapper for NNs with dropout layers
- [x] Review implementations and modify model parameters using register buffer/parameter where needed.
- [x] Improve PTAF trainer: reduce on plateau scheduler, better stats for evalExample
- [x] Implement prototype of labels containers from DatasetGeneratorLib and for general usage, with load/dump methods to/form yml and dict
- [x] Improve ModelTrainingManager printing and save method
- [x] Add automatic check of equivalence in model export helpers (onnx mainly)
- [x] Modify mlflow logging to unpack dict and dataclasses and save each key separately (better grouping)
- [x] Add automatic mixed precision support from torch
- [x] Add optional validation check in Augmentation helper to ensure all inputs are valid ones with remedy action
- [x] Improve handling of training errors (stop and kill run if nan or Inf)
- [x] Implement prototype usage of AMP for mixed precision training
- Minor versions and patches of v0.2.X
- [x] Major fixes in v0.2.1 (bugs introduced in v0.2.0)
- [x] Fix bugs in augmentation module (validation module)
- [x] Implement "binarize" augmentation for images (EDIT: soft-binarize in the final version)
- [x] Implement tailoring of RandomAffine to make it border crossing aware
- v0.3.X
- [x] Implement custom adaptive pooling layers for ONNx static export
- [x] Implement tests for *onnxability* of models provided by model_building module of PTAF
- v0.4.0
- [x] Improve CI setup
- [x] Add new datakeys and coverage
- [x] Improve backbone configuration
- v0.5.0
- [x] Migrate documentation from Read The Docs to GitHub Pages with MkDocs
- [ ] [MAJOR] Restructure and expand augmentations module to correctly handle images and 1D vectors jointly
- [ ] Implement 1d vector error models (from selected distributions)
- [ ] Change random apply and structure of augs module for images (split based on type)
- [ ] Review and unit test selection criteria implementation
- [ ] Extend augs-trainer integration to allow "discard" option
- v0.X.X
- [ ] Modify implementation of trainer to use static or cls methods instead of instance methods for increased flexibility
- [ ] Add capability to trainer/new class helper: transfer learning from checkpoint instead of replacing model!
- [ ] Fix all unit tests for up-to-date modules
- [ ] Implement export method (traced, onnx, model pth) in ModelTrainingManager
- [ ] Add abstract meta class for loss functions to enforce interface
- [ ] Review checkpoint resuming code (ensure the checkpoint is loaded correctly) and add evaluation before training (baseline score)
- [ ] Upgrade PTAF trainer and loss function interfaces to support image segmentation tasks
- [ ] Add batched bbox label generator using binary thresholded mask/image
- [ ] Modify DataLoaderIndex to accept datasets directly and a combination of datasets and dataloaders. If dataset is input, use default specifications for dataloader
- [ ] Implement new training mode: SWA_MODE
- [ ] Add pruning strategy in NORMAL MODE using "delta loss" over patience interval to check for pruning (like OPTUNA)
- [ ] Add configurable dataclasses from template yml files for training manager and extensible prototype (see pySR configuration in nav-frontend)
- [ ] Add pySR conveniency module
- [ ] Explore Hydra for configuration management from yml files
- [ ] Add prototype of SHAP for CNNs
- [x] Add prototype of model explainer for vector-to-vector regression using Captum (2 methods)
- [ ] Upgrade api.torch to save model "packages" with all info to run the model
- [ ] Improve docstrings in src code
- [ ] Implement prototype of K-fold cross-validation for training runs
- [ ] Extend ModelEvaluator to evaluate sklearn and xgb models
- [ ] All those above
- [ ] Add working argument parser for training scripts
- [ ] Add configurable dataclasses from template yml files
- [ ] Add prototype version of model hyperparams optim tuning with default params to search (scheduler, its parameters, batch size, num of epochs, what else?)
- [ ] Add prototype of model explainer for image to vector regression using Captum
- [ ] Rework ModelTrainingManager to avoid inheritance of configuration
## Modules development notes (to allocate)
### Library management and config
- [x] Review linters/code formatting tools and configure them properly to comply with standards
- [ ] Add type annotations for mypy
- [x] Automate documentation building process
- [x] Implement CI/CD to run tests
- [x] Configure project to use hatch build for automatic versioning linked to git tags
### Evaluation
- [ ]
### Optimization
- [ ]
### API module
- [ ] Add prototype implementation of mlflow api: MlflowHandler to replace manually written code in optimization module
- [ ] Add prototype implementation of telegram bot api for training push notifications