Model iterations - .pkl files not appearing in models directory

[NathanDoyleB612]

Hello!

So I've noticed a bit of an issue where after annotating over 30 new images no new .pkl files are appearing in the model directory.

for context (if it helps):

• I had issues with the metric's plot not loading and it was resolved by replacing a corrupted client annotation log.
• I'm not using the colab note book, I'm running the program through my GPU
• I believe my sync direcotry to the google drive is responsive
• I already have 81 .pkl files in that projects model directory

Any advice or guidance would be greatly appreciated!

thanks!

[Abe404]

Hello!

How big are your images?
how long did it take to annotate these 30 images?
How often did you get new models before that?
As you are using your own GPU, I'm curious why you are using google drive? perhaps everything could be local and not in drive?

Some of the following in this FAQ item might be helpful at this stage in your project.
https://github.com/Abe404/root_painter/blob/master/docs/FAQ.md#question---how-do-i-decide-when-the-model-is-good-enough

Figure3 at this paper also shows (a bit) how model creation slows down as a function of annotation duration.
https://nph.onlinelibrary.wiley.com/doi/full/10.1111/nph.71065

You should expect diminishing returns as time goes on. But if things have just 'stopped' then something is wrong.
I once inspected someone's project to try to figure this out and I looked at their annotations first, at one point they switched their foreground and background brush, so had started annotating roots in green and background in red. This caused a lot of label 'noise' and confusion in the training process.

I'm not implying you did this, but serious annotation errors are one type of thing that might have gone wrong.

It's also worth noting that there are limits in the architecture I currently use with RootPainter and with segmentation models in general (see https://www.biorxiv.org/content/10.64898/2026.02.18.706562v1.abstract). Annotating more should eventually improve things, but at some point the model may really struggle due to challenges with the images and limitations with the architecture and training process.

[NathanDoyleB612]

Hi!,

• my images are each just smaller than 1 mb.
• can't really definitively say probably a range of 1 - 7min/img depending on the root prevalence.
• from looking at the date modified times in the file directory it seems I was getting new models every 10-15 min. Some of the earlier ones were spaced even 2min apart which makes sense.
• I suppose I could have set it up locally as I am using my own GPU but at the time when following the tutorials it was a situation where "I didn't know what I didn't know" so I set it up in a google drive directory just in case (just pulling the files from the drive not using colab).

Generally the model is at a level which I'm happy with, and I suspect I'm probably at this "diminishing returns" interval. I did however create a new project based on the latest model from this project and new .pkl files appeared after 6 images.

thanks for putting things in a bit more context though!

[Abe404]

Hi

• I meant in pixels, like width and height, for the images that you actually annotate to train the model.
• 7 min is quite a lot, maybe your images are very big. In the original RootPainter paper we used images with width and height of 750x750 (approximately).
• 2-10-15min all sounds fine. Maybe if you annotate a few more minutes you will get one? :D I'm not sure I notice a reason for model saving to stop from what you have said yet. It's natural for it to slow down.
• When spending a long time annotating, it can feel like it makes sense to solve the errors you see, but when it comes to the actual analysis, it's possible they don't matter at all for the research question.
• Creating a new project, with just the new model will train on only the new annotations for the new project. So you will get new models that better fit those specific images/annotations and not necessarily the images/annotations from your previous project.

Kind regards,
Abraham

[NathanDoyleB612]

Hello!,

• I'm working with 1024 x 768 images.
• They are fairly, in real world dimensions they're about 10x9cm.
• I suppose I may be a little impatient haha.
• Fair point, sometimes one can't help try be a perfectionist.
• Ahh okay that makes sense!

Thanks for all this. I'll be sure to share my results in the "Show and tell" thread when I get there!

Keep well,
Nathan