backend/processing/segmenter: Improve segmentation speed

[ethanjli]

This is a tracking issue for our project to improve the throughput of the PlanktoScope's segmenter

Motivation

Currently, running the PlanktoScope segmenter on the PlanktoScope's Raspberry Pi computer is very slow (refer to these non-scientific performance benchmarking measurements). This means that it takes longer for the PlanktoScope segmenter to process each dataset than it does for the PlanktoScope to acquire that dataset - so real-time segmentation is impossible because the segmenter can't keep up with the rate of image acquisition, and when the PlanktoScope is highly utilized (e.g. running ~18 acquisitions per day on average), the PlanktoScope acquires data more rapidly than another Raspberry Pi running PlanktoScope software can process that data.

#322 and #323 already propose migrating the software to be able to run on the 64-bit version of the Raspberry Pi OS and to be able to run on the Raspberry Pi 5; each of those changes provides a dramatic improvement in segmenter speed (refer to the benchmarking measurements linked above), and combining the two changes (Raspberry Pi 5 with the 64-bit version of the OS) gets the segmenter performance almost at the level needed for real-time segmentation; so if we can further improve segmenter speed by 50-100%, that will give segmenter performance good safety margins for it to make sense for us to implement real-time segmentation on the PlanktoScope's Raspberry Pi.

Goals

• Make the segmenter fast enough that real-time segmentation would be possible on a Raspberry Pi 5 running the 64-bit version of the Raspberry Pi OS
• Make the segmenter fast enough on the Raspberry Pi 4 (running the 64-bit version of the Raspberry Pi OS) that users can get quick feedback on their samples, by being able to start seeing segmentation results quickly (with only a subjectively "reasonable" delay before seeing the first segmented object)

Steps

(these steps will be converted into tracking issues for stories)

• Rigorously identify the performance bottlenecks in the segmenter, with quantitative performance profiling (print statements for different segmenter stages could be adequate, if it's too hard to do something nicer) for both CPU steps and I/O steps
• Improve the speed of background calculation/subtraction? (we already suspect this is slow, but it would be good to have empirical evidence to check our intuition)
• Enable the segmenter to simultaneously process multiple datasets (e.g. by starting multiple segmentation workers, each of which can process a dataset)
• Enable the segmenter to simultaneously process multiple images in a dataset? (but this may require redesigning the segmenter algorithm, or limiting parallelism to be between background recalculation steps)
• ??? (TBD depending on what performance bottlenecks we identify)

Unresolved Questions

• Would it be more efficient to completely replace our current segmenter with a different algorithm? (e.g. an ML-based model)
• Would any performance improvements be worth the effort and complexity to make the segmenter take advantage of any connected TPU/GPU for computational efficiency?