Make the background channel optional in the napari plugin#637
Conversation
|
@alessandrofelder I realize targetting two things at once is slightly messy but they were really intertwined so hence I put it this way. Happy to split your issue seperately if you that makes this revewing easier. |
d34a7e1 to
f320ab9
Compare
alessandrofelder
left a comment
There was a problem hiding this comment.
Thanks a lot for this @aymuos15
I've only skimmed the code changes (will come back to it later this week) and done some initial user-testing (largely all looks reasonable to me). Thought it might be useful to flag an initial specific comment before I have time to look more deeply - sorry if I misunderstood!
| ): | ||
| show_info( | ||
| "Running without a background image needs a single-channel " | ||
| "model. Enable 'Skip classification', or uncheck 'Use " |
There was a problem hiding this comment.
I think one should actually check "Use pre-trained weights" to use the default single channel model, no?
(If not, there is no easy way through the GUI to download and use the single-channel default model?)
(and therefore related logic needs to be adapted 🤔 )
There was a problem hiding this comment.
This was a brain fog moment of mine. I think I was just doing everything locally and sided with that. Sorry about that! Thank you very much for catching it! :D
There was a problem hiding this comment.
Have commited this.
| return | ||
|
|
||
| if ( | ||
| options["background_image"] is None |
There was a problem hiding this comment.
sorry - I am still unsure about this 🤔
Will this not evaluate to True and return too early if
- I don't have a background channel
- I want to classify (i.e. I don't want to skip classification)
- I have trained my own single-channel model and therefore do not want to use the pre-trained model?
I wonder whether we want more something like (pseudo-code):
n_channels = 2 if background_image else 1
model_has_correct_number_of channels = use_pre_trained_weights or model.channel_dimension == n_channels
if not model_has_correct_number_of_channels:
# sensible info message and returnThere was a problem hiding this comment.
Apologies once again, I think I confused myself here. I just wanna quickly make sure my understanding is right this time.
First, for context: How the default path works (pretrained ticked):
| Step | File | Action |
|---|---|---|
| 1 | detect.py:457 |
pretrained → trained_model = None (Path.home() discarded) |
| 2 | main.py:215 |
no background → resnet50_tv → resnet50_1ch |
| 3 | main.py:218 → prep.py:35 |
no weights given → download_models(...) |
| 4 | tools.py:get_model |
no custom model → build_model + load_weights(default) |
| 5 | classify.py:180 |
channel check |
This guard was set up because the tick-box use_pre_trained_weights is the real switch, but trained_model starts as Path.home() (a folder), not empty, and only becomes empty when the tick-box is on. So when the tick-box is off, there's no clean way to tell "I picked a model" from "I picked nothing", both look like a path. The guard just patches over that. I think this was what led to my choice above.
Maybe, we can actually just collapse to pretrained / custom / skip into one exclusive choice, so "nothing picked" is a checkable state and the guard isn't needed. Current vs proposed behaviour:
| Model choice | Background | Current | Proposed |
|---|---|---|---|
| Pretrained | yes | 2ch default, proceeds | same |
| Pretrained | no | 1ch default, proceeds | same |
| Custom file | yes | proceeds; core errors if ≠ 2ch | same |
| Custom file | no | guard blocks it | proceeds; core errors if ≠ 1ch |
| No model picked | yes | cryptic load_model(home) error |
blocked, "select a file" |
| No model picked | no | guard blocks (masks that error) | blocked, "select a file" |
| Skip classification | yes | detection only | same |
| Skip classification | no | detection only | same |
Since the logic you mentioned already exists there (classify.py:180-188), can we just follow the above and patch it there?
# detect_containers.py
class ModelSource(Enum):
PRETRAINED = "Pretrained default"
CUSTOM = "Custom model"
SKIP = "Skip classification"
# ClassificationInputs: one field replaces the two flags
model_source: ModelSource = ModelSource.PRETRAINED
# as_core_arguments(): translate the choice for the core
args["skip_classification"] = self.model_source is ModelSource.SKIP
if self.model_source is not ModelSource.CUSTOM:
args["trained_model"] = None # core downloads the default# detect.py: derive skip, guard only the "custom but no file" case
skip_classification = model_source is ModelSource.SKIP
if model_source is ModelSource.CUSTOM and not (
trained_model and Path(trained_model).is_file()
):
show_info("Select a trained model file.")
returnThere was a problem hiding this comment.
No worries at all, and thanks for dissecting this @aymuos15 - the fact that we use both a Path and a checkbox to encode one thing is confusing currently for sure! (See related 4-year issue I made that never went anywhere 😂 #328)
I agree with the "proposed" column in your table above, I think that's what the behaviour should be. My only minor note would be that the error message should also not the option to tick "pretrained"?
# detect.py: derive skip, guard only the "custom but no file" case
skip_classification = model_source is ModelSource.SKIP
if model_source is ModelSource.CUSTOM and not (
trained_model and Path(trained_model).is_file()
):
show_info("Select a trained model file or check the `Use pre-trained weights` to use the default model.")
returnThere was a problem hiding this comment.
My bad, Shouldve spotted that issue in advance haha. Thank you for pointing it out, I have added it to the todo list in our projct and will PR that after this.
In my latest commit, I have done what we discused in the table along with your proposed error message. Thank you!
When background_array is None and the default multi-channel model is still selected, switch to resnet50_1ch so the Python API runs without a dimension mismatch. An explicitly chosen model is left untouched. Closes brainglobe#636.
The detect widget rejected running with no background image when 'Use pre-trained weights' was checked, but that is the only way to reach the default single-channel model, which core auto-selects. Reject the uncovered case instead: no background, no pretrained weights, and no single-channel model chosen.
Replace the pre-trained checkbox and trained-model path pair with a single ModelSource choice (pretrained, custom, or skip), so a custom single-channel model runs without a background instead of being blocked, and an empty choice is a real, guarded state.
e3f227d to
ee72a0b
Compare
Follow-up to the core single-channel support (#619), extending it through the napari UI so users can run detection, curation, and training with only a signal channel. Also closes #636 by auto-selecting the single-channel model in the Python API.
What changed
detect.py): the background image is now optional. A missing background is passed to core asNone. Added a guard: running without a background requires a single-channel model, so the widget prompts the user to enable "Skip classification" or uncheck "Use pre-trained weights" and pick a single-channel model.curation.py): background layer is optional throughout. When absent, only the signal channel is written, and the training YAML marksbg_channel = -1(which core training already reads as signal-only). Signal/background selections reset cleanly toNonewhen deselected.classify.py): skips background normalization when there is no background array.main.py): whenbackground_arrayisNoneand the default multi-channel model is still selected, the API switches toresnet50_1ch, avoiding a dimension mismatch ([Feature] Choose our "resnet50_1ch" as the default ifbackgroundarray isNone#636). An explicitly chosen model is left untouched.DataInputs.background_arrayis nowOptional.Tests
bg_channel: -1YAML, verified consumable by core training.check_image_data_for_extractionaccepts signal-only; missing-signal message updated.DataInputs(background_array=None)passthrough and single-channel detection worker smoke test.resnet50_1ch; an explicitresnet50_allis preserved.