fix(spp_api_v2_gis): repair aborted-transaction fallbacks and geofence/incident scope actions

spp_api_v2_gis/README.rst

@@ -96,6 +96,11 @@ Scopes and Data Privacy
 | ``gis:geofence`` | Read + Write | Create and archive geofences (also |
 |                  |              | requires ``gis:read`` for listing) |
 +------------------+--------------+------------------------------------+
+| ``gis:incident`` | Reserved     | No endpoint checks it yet;         |
+|                  |              | selectable so incident-scoped      |
+|                  |              | clients can be provisioned ahead   |
+|                  |              | of the incidents API               |
++------------------+--------------+------------------------------------+
 
 **What data is exposed**
 
@@ -156,6 +161,24 @@ Dependencies
 Changelog
 =========
 
+19.0.2.0.1
+~~~~~~~~~~
+
+- fix: bind coordinate query parameters in the order the SQL expects
+- fix: run the coordinate statistics query inside a savepoint so the
+  area fallback stays reachable
+- fix: run the coordinate proximity query inside a savepoint so the area
+  fallback stays reachable
+- fix: run the area fallback queries inside savepoints so a failed
+  geometry cannot abort the transaction
+- fix: run each batch geometry and the batch summary inside savepoints
+  so one failed geometry cannot poison the rest of the batch
+- fix: propagate statistics failures instead of mislabelling them as
+  coordinate-query failures and retrying via the area fallback
+- fix: add ``geofence`` and ``incident`` scope actions so geofence
+  endpoints can be granted (``incident`` prepares for the incidents API
+  re-land)
+
 19.0.2.0.0
 ~~~~~~~~~~
 

spp_api_v2_gis/__manifest__.py

@@ -2,7 +2,7 @@
 {
     "name": "OpenSPP GIS API",
     "category": "OpenSPP/Integration",
-    "version": "19.0.2.0.0",
+    "version": "19.0.2.0.1",
     "sequence": 1,
     "author": "OpenSPP.org",
     "website": "https://github.com/OpenSPP/OpenSPP2",

spp_api_v2_gis/models/api_client_scope.py

@@ -16,3 +16,11 @@ class ApiClientScope(models.Model):
         ],
         ondelete={"gis": "cascade", "statistics": "cascade"},
     )
+
+    action = fields.Selection(
+        selection_add=[
+            ("geofence", "Geofence Management"),
+            ("incident", "Incident Management"),
+        ],
+        ondelete={"geofence": "cascade", "incident": "cascade"},
+    )

spp_api_v2_gis/readme/DESCRIPTION.md

@@ -48,6 +48,7 @@ Follows thin client architecture where QGIS displays data and OpenSPP performs a
 |-------|--------|-------------|
 | `gis:read` | Read-only | View collections, layers, statistics, export data |
 | `gis:geofence` | Read + Write | Create and archive geofences (also requires `gis:read` for listing) |
+| `gis:incident` | Reserved | No endpoint checks it yet; selectable so incident-scoped clients can be provisioned ahead of the incidents API |
 
 **What data is exposed**
 

spp_api_v2_gis/readme/HISTORY.md

@@ -1,3 +1,13 @@
+### 19.0.2.0.1
+
+- fix: bind coordinate query parameters in the order the SQL expects
+- fix: run the coordinate statistics query inside a savepoint so the area fallback stays reachable
+- fix: run the coordinate proximity query inside a savepoint so the area fallback stays reachable
+- fix: run the area fallback queries inside savepoints so a failed geometry cannot abort the transaction
+- fix: run each batch geometry and the batch summary inside savepoints so one failed geometry cannot poison the rest of the batch
+- fix: propagate statistics failures instead of mislabelling them as coordinate-query failures and retrying via the area fallback
+- fix: add `geofence` and `incident` scope actions so geofence endpoints can be granted (`incident` prepares for the incidents API re-land)
+
 ### 19.0.2.0.0
 
 - Initial migration to OpenSPP2

spp_api_v2_gis/services/spatial_query_service.py

@@ -53,11 +53,17 @@ def query_statistics_batch(self, geometries, filters=None, variables=None):
             geometry = item["geometry"]
 
             try:
-                result = self.query_statistics(
-                    geometry=geometry,
-                    filters=filters,
-                    variables=variables,
-                )
+                # Each iteration runs inside its own savepoint: the spatial
+                # legs guard themselves, but a statistics failure would
+                # otherwise abort the transaction for every later geometry
+                # and for the summary below.
+                # flush=False keeps unrelated pending ORM writes out of the rollback.
+                with self.env.cr.savepoint(flush=False):
+                    result = self.query_statistics(
+                        geometry=geometry,
+                        filters=filters,
+                        variables=variables,
+                    )
                 # Collect registrant IDs for deduplication in summary
                 registrant_ids = result.pop("registrant_ids", [])
                 all_registrant_ids.update(registrant_ids)
@@ -92,7 +98,15 @@ def query_statistics_batch(self, geometries, filters=None, variables=None):
         # Compute summary by aggregating unique registrants with metadata
         summary_stats_with_metadata = {"statistics": {}}
         if all_registrant_ids:
-            summary_stats_with_metadata = self._compute_statistics(list(all_registrant_ids), variables or [])
+            try:
+                # Same guard as the loop: a failure degrades to an empty
+                # summary instead of discarding the per-geometry results.
+                # flush=False keeps unrelated pending ORM writes out of the rollback.
+                with self.env.cr.savepoint(flush=False):
+                    summary_stats_with_metadata = self._compute_statistics(list(all_registrant_ids), variables or [])
+            except Exception as e:
+                _logger.warning("Batch summary statistics failed: %s", e)
+                summary_stats_with_metadata = {"statistics": {}}
 
         summary = {
             "total_count": len(all_registrant_ids),
@@ -133,25 +147,38 @@ def query_statistics(self, geometry, filters=None, variables=None):
         geometry_json = json.dumps(geometry)
 
         # Try coordinate-based query first (preferred method)
+        result = None
         try:
-            result = self._query_by_coordinates(geometry_json, filters)
-            if result["total_count"] > 0:
-                _logger.info(
-                    "Spatial query using coordinates: %s registrants found",
-                    result["total_count"],
-                )
-                # Compute statistics for the matched registrants with metadata
-                stats_with_metadata = self._compute_statistics(result["registrant_ids"], variables)
-                result.update(stats_with_metadata)
-                return result
+            # A failed statement aborts the whole transaction, which would make
+            # the area fallback below fail too. The savepoint contains it.
+            # flush=False keeps unrelated pending ORM writes out of the rollback.
+            with self.env.cr.savepoint(flush=False):
+                result = self._query_by_coordinates(geometry_json, filters)
         except Exception as e:
             _logger.warning(
                 "Coordinate-based query failed: %s, falling back to area-based query",
                 e,
             )
 
+        # Only the coordinate query itself is retried via the fallback; a
+        # statistics failure is not a spatial failure and must propagate.
+        if result is not None and result["total_count"] > 0:
+            _logger.info(
+                "Spatial query using coordinates: %s registrants found",
+                result["total_count"],
+            )
+            # Compute statistics for the matched registrants with metadata
+            stats_with_metadata = self._compute_statistics(result["registrant_ids"], variables)
+            result.update(stats_with_metadata)
+            return result
+
         # Fall back to area-based query
-        result = self._query_by_area(geometry_json, filters)
+        # A failed statement here would otherwise leave the transaction aborted
+        # for every subsequent query on this cursor, including later geometries
+        # in query_statistics_batch. The savepoint contains it.
+        # flush=False keeps unrelated pending ORM writes out of the rollback.
+        with self.env.cr.savepoint(flush=False):
+            result = self._query_by_area(geometry_json, filters)
         _logger.info(
             f"Spatial query using area fallback: {result['total_count']} registrants in {result['areas_matched']} areas"
         )
@@ -176,11 +203,11 @@ def _query_by_coordinates(self, geometry_json, filters):
         """
         # Build WHERE clause from filters
         where_clauses = ["p.is_registrant = true"]
-        params = [geometry_json]
+        filter_params = []
 
         if filters.get("is_group") is not None:
             where_clauses.append("p.is_group = %s")
-            params.append(filters["is_group"])
+            filter_params.append(filters["is_group"])
 
         if filters.get("disabled") is not None:
             if filters["disabled"]:
@@ -209,8 +236,8 @@ def _query_by_coordinates(self, geometry_json, filters):
               )
         """  # nosec B608 - SQL clauses built from hardcoded fragments, data uses %s params
 
-        # Add geometry parameter at the beginning
-        params = [geometry_json] + params[1:]
+        # Params ordered to match SQL: filter params (in where_clause) then geometry
+        params = filter_params + [geometry_json]
 
         self.env.cr.execute(query, params)
         registrant_ids = [row[0] for row in self.env.cr.fetchall()]
@@ -484,30 +511,42 @@ def query_proximity(self, reference_points, radius_km, relation="within", filter
         radius_meters = radius_km * 1000
 
         # Try coordinate-based query first
+        result = None
         try:
-            result = self._proximity_by_coordinates(reference_points, radius_meters, relation, filters)
-            if result["total_count"] > 0:
-                _logger.info(
-                    "Proximity query (%s, %.1f km) using coordinates: %s registrants found",
-                    relation,
-                    radius_km,
-                    result["total_count"],
-                )
-                registrant_ids = result["registrant_ids"]
-                stats_with_metadata = self._compute_statistics(registrant_ids, variables)
-                result.update(stats_with_metadata)
-                result["reference_points_count"] = len(reference_points)
-                result["radius_km"] = radius_km
-                result["relation"] = relation
-                return result
+            # A failed statement aborts the whole transaction, which would make
+            # the area fallback below fail too. The savepoint contains it.
+            # flush=False keeps unrelated pending ORM writes out of the rollback.
+            with self.env.cr.savepoint(flush=False):
+                result = self._proximity_by_coordinates(reference_points, radius_meters, relation, filters)
         except Exception as e:
             _logger.warning(
                 "Coordinate-based proximity query failed: %s, falling back to area-based",
                 e,
             )
 
+        # Only the proximity query itself is retried via the fallback; a
+        # statistics failure is not a spatial failure and must propagate.
+        if result is not None and result["total_count"] > 0:
+            _logger.info(
+                "Proximity query (%s, %.1f km) using coordinates: %s registrants found",
+                relation,
+                radius_km,
+                result["total_count"],
+            )
+            registrant_ids = result["registrant_ids"]
+            stats_with_metadata = self._compute_statistics(registrant_ids, variables)
+            result.update(stats_with_metadata)
+            result["reference_points_count"] = len(reference_points)
+            result["radius_km"] = radius_km
+            result["relation"] = relation
+            return result
+
         # Fall back to area-based query
-        result = self._proximity_by_area(reference_points, radius_meters, relation, filters)
+        # A failed statement here would otherwise leave the transaction aborted
+        # for every subsequent query on this cursor. The savepoint contains it.
+        # flush=False keeps unrelated pending ORM writes out of the rollback.
+        with self.env.cr.savepoint(flush=False):
+            result = self._proximity_by_area(reference_points, radius_meters, relation, filters)
         _logger.info(
             "Proximity query (%s, %.1f km) using area fallback: %s registrants in %s areas",
             relation,

spp_api_v2_gis/static/description/index.html

@@ -501,6 +501,13 @@ <h1>Scopes and Data Privacy</h1>
 <td>Create and archive geofences (also
 requires <tt class="docutils literal">gis:read</tt> for listing)</td>
 </tr>
+<tr><td><tt class="docutils literal">gis:incident</tt></td>
+<td>Reserved</td>
+<td>No endpoint checks it yet;
+selectable so incident-scoped
+clients can be provisioned ahead
+of the incidents API</td>
+</tr>
 </tbody>
 </table>
 <p><strong>What data is exposed</strong></p>
@@ -557,32 +564,52 @@ <h1>Dependencies</h1>
 <div class="contents local topic" id="contents">
 <ul class="simple">
 <li><a class="reference internal" href="#changelog" id="toc-entry-1">Changelog</a><ul>
-<li><a class="reference internal" href="#section-1" id="toc-entry-2">19.0.2.0.0</a></li>
+<li><a class="reference internal" href="#section-1" id="toc-entry-2">19.0.2.0.1</a></li>
+<li><a class="reference internal" href="#section-2" id="toc-entry-3">19.0.2.0.0</a></li>
 </ul>
 </li>
-<li><a class="reference internal" href="#bug-tracker" id="toc-entry-3">Bug Tracker</a></li>
-<li><a class="reference internal" href="#credits" id="toc-entry-4">Credits</a></li>
+<li><a class="reference internal" href="#bug-tracker" id="toc-entry-4">Bug Tracker</a></li>
+<li><a class="reference internal" href="#credits" id="toc-entry-5">Credits</a></li>
 </ul>
 </div>
 <div class="section" id="changelog">
 <h2><a class="toc-backref" href="#toc-entry-1">Changelog</a></h2>
 <div class="section" id="section-1">
-<h3><a class="toc-backref" href="#toc-entry-2">19.0.2.0.0</a></h3>
+<h3><a class="toc-backref" href="#toc-entry-2">19.0.2.0.1</a></h3>
+<ul class="simple">
+<li>fix: bind coordinate query parameters in the order the SQL expects</li>
+<li>fix: run the coordinate statistics query inside a savepoint so the
+area fallback stays reachable</li>
+<li>fix: run the coordinate proximity query inside a savepoint so the area
+fallback stays reachable</li>
+<li>fix: run the area fallback queries inside savepoints so a failed
+geometry cannot abort the transaction</li>
+<li>fix: run each batch geometry and the batch summary inside savepoints
+so one failed geometry cannot poison the rest of the batch</li>
+<li>fix: propagate statistics failures instead of mislabelling them as
+coordinate-query failures and retrying via the area fallback</li>
+<li>fix: add <tt class="docutils literal">geofence</tt> and <tt class="docutils literal">incident</tt> scope actions so geofence
+endpoints can be granted (<tt class="docutils literal">incident</tt> prepares for the incidents API
+re-land)</li>
+</ul>
+</div>
+<div class="section" id="section-2">
+<h3><a class="toc-backref" href="#toc-entry-3">19.0.2.0.0</a></h3>
 <ul class="simple">
 <li>Initial migration to OpenSPP2</li>
 </ul>
 </div>
 </div>
 <div class="section" id="bug-tracker">
-<h2><a class="toc-backref" href="#toc-entry-3">Bug Tracker</a></h2>
+<h2><a class="toc-backref" href="#toc-entry-4">Bug Tracker</a></h2>
 <p>Bugs are tracked on <a class="reference external" href="https://github.com/OpenSPP/OpenSPP2/issues">GitHub Issues</a>.
 In case of trouble, please check there if your issue has already been reported.
 If you spotted it first, help us to smash it by providing a detailed and welcomed
 <a class="reference external" href="https://github.com/OpenSPP/OpenSPP2/issues/new?body=module:%20spp_api_v2_gis%0Aversion:%2019.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**">feedback</a>.</p>
 <p>Do not contact contributors directly about support or help with technical issues.</p>
 </div>
 <div class="section" id="credits">
-<h2><a class="toc-backref" href="#toc-entry-4">Credits</a></h2>
+<h2><a class="toc-backref" href="#toc-entry-5">Credits</a></h2>
 </div>
 </div>
 <div class="section" id="authors">

spp_api_v2_gis/tests/__init__.py

@@ -1,11 +1,14 @@
 # Part of OpenSPP. See LICENSE file for full copyright and licensing details.
+from . import test_api_client_scope
 from . import test_catalog_service
 from . import test_export_service
 from . import test_geofence_model
 from . import test_layers_service
 from . import test_ogc_features
 from . import test_ogc_http
 from . import test_qml_template_service
+from . import test_spatial_query_coordinates
+from . import test_spatial_query_fallback
 from . import test_spatial_query_service
 from . import test_statistics_endpoint
 from . import test_batch_query