Galiano Island

[tombh]

Tasks to compute Galiano Island's DSM as we've already done for the world, but, with the ability to render entire viewsheds instead of just longest lines.

DSM hillshade screenshots:

The raw data is 87,616x78,341 (~7 billion) elevation samples at 25cm resolution. ~14GB uncompressed. Computationally this is trivial. The hurdle is in fact the sheer scale of storing all the viewsheds, a naive approach would likely reach 100TB.

Reducing the resolution to 1m (~20,000x20,000) and using some basic storage optimisations could get us to around 2TB.

Core

Fixes

• Support custom scales, see Support variably scaled tiles #29 and Support variably scaled tiles CacheTVS#35
• Support u16 elevation data. Currently our .bt implementation and kernel only supports i16. The Galiano DSM is u16 with a max elevation of 33276cm. For now we can just hack it by offsetting into i16. feat: use gdal for loading and saving tiles CacheTVS#44
• Fix issues from first run, see Galiano run 2 TODOs #78

Storing viewsheds

• Store byte-optimised viewsheds during compute, see Storage engine CacheTVS#45
• Setup a tiny VM with a few TBs of disk, SQLite and a basic REST API. https://api.alltheviews.world/

Displaying viewsheds

• Setup https://galiano.alltheviews.world or similar.
• Generate the visibility heatmap. This is exactly the same as we did for the world.
• Reconstruct viewsheds in the webapp by querying the SQLite VM API which returns the binary viewshed data for a clicked point. This should be fast and hopefully feel almost instantaneous. Viewshed API server #73
• Some UI features that help do some basic exploration of coverage. Viewshed API server #73 Ideas:
  • Clicking a second point hides the previously rendered viewshed.
  • SHIFT+CLICK adds a new potentially overlapping viewshed without hiding the previously displayed viewshed.
  • Colour picker to change the fill colour of individual viewsheds.
  • Slider to change the cutoff range of the viewshed. This might help simulate different radio devices.

Optional Optimisations

Storage

If 1m resolution is good enough then these aren't necessary

• Use NODATA values or a simple polygon, even a basic rectangle would do, to cull all viewsheds outside the area of interest, namely the island itself.
• Instead of reducing the resolution of the DSM, only store the viewshed with the greatest surface visibility within a given radius, say within 1m, which would compare the 16 neighbouring viewsheds.
• Ideally we would store the entire viewshed dataset as a single binary file with a byte-offset index so we can host it on Cloudflare's R2 and query it with HTTP Range requests. Per TB this would be around 10 times cheaper than SQLite on a VM. But a VM for a 1m rendering of Galiano Island is just tens of dollars, so no need to worry at the moment.
• This is certainly not for now, but it's worth bearing in mind that neighbouring viewsheds are often very similar. This may well map to the concept of keyframes in video codecs, so we'd only occasionally store full viewsheds whilst most would just be stored as the diff against a "key-viewshed".

Computation

Seeing as computation is not our bottleneck, these are low priority

• Cull all lines of sight that fall entirely outside the area of interest. In the case of Galiano Island, whilst we are interested in all the points on the island, we are not interested in how far we can see out into the ocean.

[tombh]

First 1m visibility heatmap, full resolution: https://cdn.alltheviews.world/Galiano/output/1.0/total_surfaces.png

[tombh]

Viewshed viewer UI/UX demo with 50m resolution heatmap https://tombh-galiano-viewview.tom-364.workers.dev

[tombh]

The viewshed viewer demo now has a 1m resolution heatmap https://tombh-galiano-viewview.tom-364.workers.dev

The viewshed polygons themselves are still from the 50m resolution run.

[tombh]

We did a first run of Galiano at 1m, but there some issues, see #78