Summary
Add SMRF (Simple Morphological Filter, Pingel et al. 2013) as a ground-detection
algorithm alongside the existing PMF, implemented in-repo the way _pmf implements
Zhang et al. 2003, and validated against PDAL's filters.smrf.
This is an accuracy change, not a refactor. Measured on 9M points of vendor-classified
3DEP at 1 m, scoring each derived ground surface against the vendor's own ASPRS class 2 —
the nearest thing to truth this data has:
| algorithm |
time |
RMSE |
p95 |
coverage |
ours (_pmf + snap) |
0.54 s |
0.056 |
0.020 |
92.7% |
pdal filters.pmf |
1.11 s |
0.057 |
0.020 |
92.8% |
pdal filters.smrf |
1.73 s |
0.035 |
0.020 |
94.0% |
pdal filters.csf |
6.49 s |
0.121 |
0.090 |
88.1% |
Two things follow. Our PMF is correct — it ties PDAL's own implementation at twice the
speed, so this is not a bug fix. And SMRF is 37% lower RMSE with 1.3 points more
coverage, which is a real quality gain we do not currently have.
Ground quality bounds everything downstream: canopy height is height above ground, so it
propagates into the CHM, the tree heights ITD derives from it, and the canopy fuels built
from those.
Why implement rather than depend on PDAL
Adopting PDAL for this was investigated and rejected on measurement:
- 9-13× slower end to end for the chunked CHM (
hag_nn), 25-37× with hag_delaunay.
- 709 MB / 181 conda packages, including a second GDAL beside the one rasterio already
bundles. There is no pip route — pdal on PyPI is an sdist needing libpdal present.
- Chunk + buffer + stitch never becomes exact: seam cells differing from an unchunked
run go 16.08% (no buffer) → 0.36% (15 m) → 0.03% (140 m), with a 0.39 m max still
present at 140 m. Our blocked implementation is bit-identical to unblocked by
construction, because its halo is on rasters rather than points.
SMRF is a published algorithm, so we can have the accuracy without any of that.
Validation
services/griddle/tests/local/test_chm_vs_pdal.py already does exactly this job for PMF —
PDAL filters, our code rasterises in both arms, so the comparison isolates the algorithm
from the gridding convention. Add an SMRF case to it.
Target: match filters.smrf as closely as _pmf matches filters.pmf today — median
0.00 m, p95 0.04 m, 98.3% of cells within 10 cm.
The tests are local-only (local marker, skipped without PDAL), so CI is unaffected.
Notes
Summary
Add SMRF (Simple Morphological Filter, Pingel et al. 2013) as a ground-detection
algorithm alongside the existing PMF, implemented in-repo the way
_pmfimplementsZhang et al. 2003, and validated against PDAL's
filters.smrf.This is an accuracy change, not a refactor. Measured on 9M points of vendor-classified
3DEP at 1 m, scoring each derived ground surface against the vendor's own ASPRS class 2 —
the nearest thing to truth this data has:
_pmf+ snap)filters.pmffilters.smrffilters.csfTwo things follow. Our PMF is correct — it ties PDAL's own implementation at twice the
speed, so this is not a bug fix. And SMRF is 37% lower RMSE with 1.3 points more
coverage, which is a real quality gain we do not currently have.
Ground quality bounds everything downstream: canopy height is height above ground, so it
propagates into the CHM, the tree heights ITD derives from it, and the canopy fuels built
from those.
Why implement rather than depend on PDAL
Adopting PDAL for this was investigated and rejected on measurement:
hag_nn), 25-37× withhag_delaunay.bundles. There is no pip route —
pdalon PyPI is an sdist needing libpdal present.run go 16.08% (no buffer) → 0.36% (15 m) → 0.03% (140 m), with a 0.39 m max still
present at 140 m. Our blocked implementation is bit-identical to unblocked by
construction, because its halo is on rasters rather than points.
SMRF is a published algorithm, so we can have the accuracy without any of that.
Validation
services/griddle/tests/local/test_chm_vs_pdal.pyalready does exactly this job for PMF —PDAL filters, our code rasterises in both arms, so the comparison isolates the algorithm
from the gridding convention. Add an SMRF case to it.
Target: match
filters.smrfas closely as_pmfmatchesfilters.pmftoday — median0.00 m, p95 0.04 m, 98.3% of cells within 10 cm.
The tests are local-only (
localmarker, skipped without PDAL), so CI is unaffected.Notes
algorithm itself, and Expose ground-source and ground-derivation controls on the point-cloud CHM request #484 is what would surface it.
filters.csfwas measured and is worse on this data — not worth pursuing.