Prototype statistical outlier removal on the CHM's points, validate against filters.outlier

[amarcozzi]

Summary

Prototype statistical outlier removal on the points feeding the point-cloud CHM —
k-nearest-neighbour mean distance against a standard-deviation multiplier, the same rule
PDAL's filters.outlier implements — and validate it against filters.outlier the way
_pmf is validated against filters.pmf today.

Today the CHM has no point-domain noise handling at all. Everything it does about noise is
raster-domain and after the fact:

• chm_blocks.SURFACE_CLASSES = (0, 1, 2, 3, 4, 5) excludes ASPRS 7 and 18 at the read.
• chm_blocks.MAX_CANOPY_HEIGHT_M = 100.0 drops heights outside a fixed window.
• chm_point_cloud._remove_spikes nulls a finished cell that exceeds every 3x3 neighbour
  by SPIKE_THRESHOLD_M = 25.0.

Which clouds this is actually for

Not vendor-classified 3DEP. Those carry classes 7/18 and the read filter already drops
them before anything is rasterised.

The population that needs this is clouds nobody cleaned — user uploads, and the many 3DEP
acquisitions that classify only ground and unclassified, where a noise return sits in
class 0 or 1 alongside real vegetation and passes the read filter untouched. That is what
the 40-80 m returns behind SPIKE_THRESHOLD_M are, and it is the same population the
derived-ground (PMF) path exists to serve.

The prototype has to be measured against that population specifically. Measuring it on
vendor-classified data would show a filter with nothing to do.

Relationship to #485

#485 argues _remove_spikes cannot work because it decides on height alone, and rules out
simply disabling it:

the cell value is a maximum over its returns, so a single noise return still wins the
cell no matter how many points fall in it. Coarser cells do not average the spike away.

That is an argument for filtering before the maximum, which is what a point-domain
filter does — remove the return and the cell max is correct with no raster rule needed.
So this issue may be the fix for #485 rather than a feature beside it, and #485's proposed
spatial-extent replacement may not be needed at all.

Whichever is picked up first should be checked against the other. The two possible
outcomes are worth distinguishing up front:

• Outlier removal catches the returns _remove_spikes was written for -> delete
  _remove_spikes and close Point-cloud CHM spike filter deletes real canopy at coarse resolutions (height-only rule, cf. #428) #485.
• It does not -> keep both, and Point-cloud CHM spike filter deletes real canopy at coarse resolutions (height-only rule, cf. #428) #485's spatial-extent rewrite still stands.

The prototype should report which, not just an RMSE.

Validation

services/griddle/tests/local/test_chm_vs_pdal.py already does this job for PMF: PDAL
runs the filter, our code rasterises in both arms, so the comparison isolates the
algorithm from the gridding convention. Add an outlier case to it.

Note that the existing PMF comparison deliberately applies the same height window in both
arms so it compares ground surfaces "rather than of each one's outlier policy"
(test_chm_vs_pdal.py:610). An outlier case inverts that: the height window is what has
to differ, because removing it is the point.

Bar to clear, stated as questions the prototype answers:

1. Against filters.outlier on the same points, how many returns do the two disagree on?
2. On unclassified clouds, does it remove the returns _remove_spikes currently catches?
3. What does it cost? The CHM already reads every point in a block; this adds a kNN search
   over them. Implement SMRF ground detection (37% lower RMSE than our PMF) #491 rejected adopting PDAL partly on 9-13x slowdowns, so an in-repo
   prototype that is slower than the raster rule it replaces is not obviously a win.

Out of scope

Exposing any of this on the create request. #484 is where a control would surface if one
is wanted; this issue is the algorithm and the measurement.