Do not cache individual images

[Lauenburg]

Description
The data is depicted in two different views.
The primary view is composed of N tiles, where each tile shows the center slice of an instance overlaid with the corresponding segmentation mask.
The instance-specific view opens up when right-clicking one of the tiles and shows an enlarged version of the instance displayed in the tile. In addition, the user can now move a lever at the bottom of the newly opened window to scroll through all the slices of the selected instance and their associated segmentation masks. Both views load the request images as png files from a corresponding folder structure.

Desired Behaviour
Based on the back-end calculated bounding boxes, the front end should cut the images directly from the h5 volumes. The data is, either way, opened and processed in the backend. Saving the data and having the frontend reload it introduces (hopefully) unnecessary redundancy and processing time.

[Lauenburg]

Initially, the goal was to pass the data as NumPy arrays directly from the backend to the front end. However, this seems to be rather complicated.

I asked the author of Neuroglancer how he achieved this. For later reference here is his answer: google/neuroglancer#386 (comment)

[Lauenburg]

Checking out the library jsfive, I ran into the issue that the data is loaded as a single 1D array. Asking the author, he recommended to us h5wasm and mathjs.
usnistgov/jsfive#32

[Lauenburg]

Due to problems with the formats, while loading, I returned to using jsfive. I figured out that combined with numjs's reshape function it is simple to recreate the original shape. See: https://stackoverflow.com/a/72034334/11918697

[Lauenburg]

The image volume should be loaded to the frontend on page one, where the user provides the files and while the backend processes. However, a JS variable is document-dependent. This means that the data is lost as soon as we go from HTML page one to HTML page two. Saving the data in the session storage or local storage does not work since the data is larger than 10MB. A possible solution to this is using IndexedDB. But, while setting up and using IndexedDB is easy, with the wrapper https://dexie.org/, the loading from the DB is asynchronous - the same as for reading the h5 file. Accordingly, we do not win anything, i.e., loading the data to the frontend, saving it to the indexedDB, and reloading the data to the frontend is inefficient. Instead, it would make more sense on page two to directly read the data from the h5 file. The issue here is that we can not pass the file link provided on page one to page two due to security reasons (https://stackoverflow.com/questions/30901778/how-do-i-pass-input-file-data-between-pages). Furthermore, loading the data on page two is also far from optimal, as the user has just been waiting for page one to load.

[Lauenburg]

Idea: There is a "mother" HTML page that loads the content of the two pages described above on demand. Thus, the data can be processed on the mother page and is available for the loaded pages. See: https://stackoverflow.com/a/72051252/11918697

[Lauenburg]

The Jinja syntax will not work when loading the external templates via load since the loaded templates are first rendered and then included. Instead, both pages have to be included in a single file. To include two pages into one, I define two DIVs that span the whole page and manipulate their visibility accordingly.

[Lauenburg]

The issue with the above approach is that each Flask render_template call refreshes the page through which any js variables (loaded data and handles to loaded data) are lost. Accordingly, this approach would require a significant reconfiguration of the app that works against the Flask paradigm.

[Lauenburg]

Following the approach of storing each subvolume as an hdf5 file instead of saving each slice as an individual png.

├── 1/
│   └── image.h5
├── 2/
│   └── image.h5
├── 3/
│   └── image.h5
...
|
└── 176/
    └── image.h5

[Lauenburg]

The issue with the approach above is that the browser cannot access the user's file system due to security reasons. Instead, the user has to provide access to a file, e.g., via the HTML Tag.