TileWarden

tilewarden inventories tiled map objects in Google Cloud Storage, writes per-level GeoPackage footprints, and provides a reviewed, generation-guarded cleanup workflow for tiles outside a keep extent.

The project targets XYZ-style Web Mercator tiles in EPSG:3857. inventory, viewer, and cleanup plan are read-only. cleanup apply is intentionally separate and deletes only exact objects from a reviewed manifest.

Install for development

conda create --name tilewarden python=3.13
conda activate tilewarden
pip install -e '.[dev]'

Run tests and linting with:

pytest
ruff format .
ruff check .

Authentication

tilewarden uses Application Default Credentials through google-cloud-storage. If credentials are not already configured, run:

gcloud auth application-default login

You can pass --project <project> when the storage client should be created for a specific Google Cloud project.

GCP Cost Awareness

Running tilewarden against a real Google Cloud Storage bucket can incur Google Cloud costs even though the tool is read-only. The CLI lists objects with the Cloud Storage API, and object listing requests are billed by Google Cloud as Cloud Storage operations, typically Class A operations. The exact cost depends on your bucket location, storage class, namespace type, free-tier eligibility, and current Google Cloud pricing.

Inventory does not download tile contents. It lists object names and metadata, including generation and byte size. For very large buckets, listing can require many paginated Class A requests, so use --prefix and --levels where practical. Cleanup apply issues one generation-guarded delete request per manifest object; deletion operations and any early-deletion charges are billed according to the bucket's current Google Cloud pricing and storage class.

Running the local test suite does not contact Google Cloud; tests use fakes and local temporary files.

Usage

tilewarden inventory <bucket-name> --output <directory> [--levels <levels>] [--prefix <prefix>] [--layout <layout>] [--project <project>] [--matrix-set webmercator] [--progress auto|always|never]
tilewarden viewer <geopackage> [--summary <summary-json>] [--port <port>] [--no-browser]
tilewarden cleanup plan <inventory.gpkg> --level <level> --extent <keep.geojson|keep.gpkg/layer> --output <directory> [--allow-delete-all] [--overwrite]
tilewarden cleanup apply <manifest.json> [--project <project>] [--progress auto|always|never] [--yes --confirm-bucket <bucket>]

The inventory command writes one GeoPackage footprint file named <bucket-name>-tile-footprints.gpkg, plus a JSON summary file named <bucket-name>-summary.json. Each discovered or selected level is stored as a separate GeoPackage layer named like tile_footprints_l05. The GeoPackage also contains a tile_extents layer with one dissolved multipart coverage feature per level.

While running in a terminal, the CLI shows progress bars for overall progress, object listing/parsing, and GeoPackage feature writing. Use --progress always to force progress output when stderr is redirected, or --progress never to disable it. After writing output, the CLI prints the bucket, prefix, layout, matrix set, output directory, skipped object count, excluded-by-level count, total tile count, generated file count, summary JSON path, and a per-level table with tile counts and source date ranges.

Examples

Honeycomb-style or layer-prefixed bucket objects such as Terrain/5/10/12:

tilewarden inventory my-tile-bucket --output ./inventory --prefix Terrain/ --layout auto

Unprefixed XYZ objects such as 5/10/12.png:

tilewarden inventory my-tile-bucket --output ./inventory --layout auto

Explicit row-before-column objects such as 5/12/10 where the path is z/y/x:

tilewarden inventory my-tile-bucket --output ./inventory --layout z/y/x

Level ranges and mixed lists:

tilewarden inventory my-tile-bucket --output ./inventory --levels 5-7
tilewarden inventory my-tile-bucket --output ./inventory --levels 5,7,10-12

Prefixed listing with an explicit project:

tilewarden inventory my-tile-bucket --output ./inventory --prefix Terrain/ --project my-gcp-project

Local Viewer

Open an existing inventory on an interactive map:

tilewarden viewer ./inventory/my-tile-bucket-tile-footprints.gpkg

The viewer binds only to 127.0.0.1, prints its local URL, and opens the default browser. Press Ctrl-C in the terminal to stop it. Use --no-browser to start it without opening a browser or --port to select a different local port. The adjacent <bucket-name>-summary.json is loaded automatically when present; --summary can select a different summary file.

Select an inventory level from the toolbar. At map zooms below level - 3, the map draws the dissolved coverage extent. At larger scales it queries only the individual tiles intersecting the current viewport. Views containing more than 5,000 tiles remain in extent mode until the map is zoomed in farther. Selecting a tile opens its stored metadata, a preview of the tile image, and a link to that object in Google Cloud Console.

Leaflet and the OpenStreetMap basemap are loaded over the internet. Inventory data and queries remain on the local machine. Tile previews are requested directly by the browser from the object's public storage.googleapis.com URL; the local server does not download objects or use Google credentials. Private or missing objects display an unavailable message instead of an image. GeoPackages created before the tile_extents layer was introduced can still return individual tile details, but they do not have the small-scale dissolved coverage display; regenerate the inventory to add it.

Layouts

Supported layouts are:

• auto: parse the final three path components, tolerate an extension on the final component, and interpret them as z/x/y.
• z/x/y: objects like 5/10/12.
• prefix/z/x/y: objects like Terrain/5/10/12.
• z/x/y.ext: objects like 5/10/12.png.
• prefix/z/x/y.ext: objects like Terrain/5/10/12.jpg.
• z/y/x: row-before-column objects like 5/12/10.
• prefix/z/y/x: prefixed row-before-column objects like Terrain/5/12/10.

WMTS defines TileMatrix, TileRow, and TileCol, but actual REST paths are service-specific. Use an explicit layout when auto would be ambiguous or when the object path stores row before column.

Every GeoPackage feature preserves the original blob_name, immutable GCS generation, and size_bytes so cleanup can target the exact reviewed object version. Features also include creation and last-modified timestamps when Cloud Storage returns them.

Matrix Set

Only --matrix-set webmercator is supported in this version. It assumes Google, ArcGIS Online, and XYZ Web Mercator semantics:

• EPSG:3857 output coordinates.
• Top-left origin.
• 2 ** z rows and columns at level z.
• Standard Web Mercator world extent.

This version is not correct for custom WMTS matrix sets, non-Web-Mercator tiles, bottom-left TMS row origins, unusual tile sizes, or custom scale sets.

Output Properties

Each feature includes:

• bucket
• prefix
• layout
• matrix_set
• level
• column
• row
• blob_name
• generation
• size_bytes
• date_created
• date_last_modified
• wkid: 3857

date_created comes from the GCS object's time_created metadata, and date_last_modified comes from the object's updated metadata. These fields are nullable when source metadata is unavailable. The JSON summary also includes overall and per-level min/max values for both dates. GeoPackage output includes a spatial index for each level layer.

The shared tile_extents layer includes:

• bucket
• prefix
• layout
• matrix_set
• level
• tile_count
• min_column and max_column
• min_row and max_row
• min_date_created and max_date_created
• min_date_last_modified and max_date_last_modified
• wkid: 3857

Each extent feature is a dissolved MultiPolygon in EPSG:3857 and has a spatial index. It is suitable for use in GIS software independently of the local viewer. The original individual tile layers and properties remain unchanged.

Cleanup Workflow

Create an inventory after upgrading so every tile includes its GCS generation, then supply a WGS84 GeoJSON or GeoPackage Polygon or MultiPolygon describing the area to keep:

tilewarden cleanup plan ./inventory/tiles-tile-footprints.gpkg \
	--level 18 \
	--extent ./keep.gpkg/keep_layer \
	--output ./cleanup

Any tile intersecting or touching the keep extent is retained. Only fully disjoint tiles become candidates. Planning is local and read-only; it writes a review GeoPackage and a JSON manifest containing exact blob names and generations. It refuses a plan that selects every tile unless --allow-delete-all is supplied, and it does not replace existing artifacts unless --overwrite is supplied.

For a GeoPackage, append the layer name to the file path with /, for example:

tilewarden cleanup plan ./inventory/tiles-tile-footprints.gpkg \
	--level 18 \
	--extent ./keep.gpkg/utah_keep \
	--output ./cleanup

A GeoPackage with one vector layer uses that layer automatically. A GeoPackage with multiple vector layers must include the layer suffix. The selected layer must contain only Polygon or MultiPolygon features and use WGS84 coordinates (EPSG:4326); all of its features are dissolved into one keep extent.

Review the candidates on the local map:

tilewarden viewer ./cleanup/tiles-cleanup-l18-plan.gpkg

The green geometry is the keep extent and red geometry marks deletion candidates. The viewer has no deletion endpoint or control.

After review, apply the exact manifest:

tilewarden cleanup apply ./cleanup/tiles-cleanup-l18-manifest.json

Interactive apply requires typing the bucket name and candidate count. Non-interactive use requires both --yes and an exact --confirm-bucket <bucket>. Every delete uses GCS if_generation_match: if an object was replaced after planning, its generation changes and the newer object is preserved. Objects created after inventory are not in the manifest and are untouched.

An audit JSONL file is flushed after every attempt and records deleted, already-missing, generation-mismatch, and failed outcomes. Generation mismatches and failures produce a nonzero exit code. Deletion has no guaranteed undo; recovery depends on bucket soft-delete, versioning, retention, and lifecycle configuration.

The applying identity needs permission to delete objects, normally storage.objects.delete, in addition to any permissions used for inventory. Use a narrowly scoped identity and review the bucket's retention and soft-delete settings before applying a plan.