Griddle processes one Cloud Task per invocation, so a multi-step LFPS job (submit → poll → poll → ... → download, up to 2 hours) needs to submit, then re-enqueue itself to check again later rather than blocking.
This splits into two files: the orchestration (submit/poll/timeout/download) lives in a new griddle/handlers/lfps.py, and griddle/handlers/landfire.py grows a way to fetch a raster from a local path instead of only a gs:// COG, so LFPS output can flow through the exact same alignment/extent_buffer_cells/nodata pipeline as everything else in that file, unchanged.
Tasks
- New
griddle/handlers/lfps.py. Given a grid's Firestore doc, either submit a new LFPS job or poll an existing one:
- No job yet →
submit_job(), write job_id/submitted_at onto the doc (lfps_job: {...}), re-enqueue a delayed Cloud Task for the same grid_id.
- In progress →
poll_status().
- Still running, under timeout → re-enqueue again.
- Over timeout → raise
ProcessingError (matching _validate_dem_has_data()'s pattern).
Failed → ProcessingError with LFPS's message.
Succeeded → download() and continue into the fetch path below.
*Timeout: 20 min. Not LFPS's own ~2hr ceiling — our jobs are small and consistently fast (12-30s), so anything still running that long is more likely stuck than genuinely slow, and we'd rather fail fast and let the user retry than poll a dead job for two hours.
- Re-enqueue delay/backoff: TBD, LFPS jobs typically finish in 12s–7min.
- Griddle processes one Cloud Task per invocation, so a step that isn't done yet can't just wait — it re-enqueues itself and stops. That "stop without marking the grid complete or failed" signal is a new
ProcessingDeferred exception (lib/errors.py), caught in main.py alongside the existing CancelledException/ProcessingError handling. The re-enqueue itself needs a synchronous Cloud-Tasks helper griddle doesn't have yet (api/tasks.py's is async, API-only) — new griddle/griddle/tasks.py.
_fetch_landfire_raster (in landfire.py) takes the raster location as a parameter instead of always building gs://{RASTERS_BUCKET}/LF{version}_{product}_CONUS.tif internally — RasterConnection opens a local path the same way it opens a gs:// one, so no separate fetch function or branch is needed there. fetch_fbfm40 gets a url= override so lfps.py can hand it an unzipped local file directly, reusing the existing remove_non_burnable/nodata/_to_dataset handling as-is.
- Wire the new path into
dispatch.py: source.product == "fbfm40" with an optional source.season field routes to lfps.py instead of the existing COG fetch; no season → unchanged behavior.
- Nodata consolidation is already shared (
_fetch_landfire_raster doesn't care where the path came from), so this falls out for free rather than needing separate handling.
Tests
- Unit tests for
lfps.py's orchestration, mocked lib.landfire_lfps client + mocked Firestore doc, no live calls:
- no job → submits;
- in progress/under timeout → re-enqueues;
- in progress/over timeout → fails, no re-enqueue;
- succeeded → downloads and continues;
- failed → fails with LFPS's message.
- Unit test for the unzip/locate-the-tif step in isolation (given zip bytes, finds the one GeoTIFF among the
.tfw/.aux.xml/VAT sidecar files; errors cleanly if none found).
- No new integration test needed for the fetch/align step itself — fetch_fbfm40's existing COG-path integration test already exercises that exact code, since the raster-fetch pipeline no longer branches on source at all once it has a path. The polling loop stays unit-test-only for the same reason as before: it doesn't resolve in a single
process_grid_request call, so a griddle_runner-style test would either assert nothing or race a live multi-minute LFPS job.
Griddle processes one Cloud Task per invocation, so a multi-step LFPS job (submit → poll → poll → ... → download, up to 2 hours) needs to submit, then re-enqueue itself to check again later rather than blocking.
This splits into two files: the orchestration (submit/poll/timeout/download) lives in a new
griddle/handlers/lfps.py, andgriddle/handlers/landfire.pygrows a way to fetch a raster from a local path instead of only ags://COG, so LFPS output can flow through the exact same alignment/extent_buffer_cells/nodata pipeline as everything else in that file, unchanged.Tasks
griddle/handlers/lfps.py. Given a grid's Firestore doc, either submit a new LFPS job or poll an existing one:submit_job(), writejob_id/submitted_atonto the doc (lfps_job: {...}), re-enqueue a delayed Cloud Task for the samegrid_id.poll_status().ProcessingError(matching_validate_dem_has_data()'s pattern).Failed→ProcessingErrorwith LFPS's message.Succeeded→download()and continue into the fetch path below.*Timeout: 20 min. Not LFPS's own ~2hr ceiling — our jobs are small and consistently fast (12-30s), so anything still running that long is more likely stuck than genuinely slow, and we'd rather fail fast and let the user retry than poll a dead job for two hours.
ProcessingDeferredexception (lib/errors.py), caught inmain.pyalongside the existingCancelledException/ProcessingErrorhandling. The re-enqueue itself needs a synchronous Cloud-Tasks helper griddle doesn't have yet (api/tasks.py's is async, API-only) — new griddle/griddle/tasks.py._fetch_landfire_raster(inlandfire.py) takes the raster location as a parameter instead of always buildinggs://{RASTERS_BUCKET}/LF{version}_{product}_CONUS.tifinternally —RasterConnectionopens a local path the same way it opens ags://one, so no separate fetch function or branch is needed there.fetch_fbfm40gets aurl=override solfps.pycan hand it an unzipped local file directly, reusing the existingremove_non_burnable/nodata/_to_datasethandling as-is.dispatch.py:source.product == "fbfm40"with an optionalsource.seasonfield routes tolfps.pyinstead of the existing COG fetch;no season→ unchanged behavior._fetch_landfire_rasterdoesn't care where the path came from), so this falls out for free rather than needing separate handling.Tests
lfps.py's orchestration, mockedlib.landfire_lfpsclient + mocked Firestore doc, no live calls:.tfw/.aux.xml/VATsidecar files; errors cleanly if none found).process_grid_requestcall, so agriddle_runner-style test would either assert nothing or race a live multi-minute LFPS job.