ice

Iceberg table debugger and health analyzer. Deep inspection of Apache Iceberg tables without Spark or JVM.

Why ice?

Iceberg tables accumulate metadata debt silently -- small files from frequent appends, snapshot bloat from missing expiry policies, orphaned data files from failed compactions, partition skew that tanks query performance. The standard way to investigate any of this is to spin up a Spark session or query it through your warehouse, which is slow and heavyweight for what's fundamentally a metadata problem.

ice reads Iceberg metadata directly via PyIceberg and gives you git-style commands to inspect table history, diff snapshots, check health, find orphaned files, and trace schema evolution. It runs locally in seconds, outputs to your terminal or as JSON for piping into other tools, and works with any Iceberg REST catalog.

Install

Requires Python 3.11+.

# with uv (recommended)
uv pip install -e .

# or with pip
pip install -e .

This installs the ice CLI globally. Verify with:

ice --help

Configuration

ice needs to know how to connect to your Iceberg catalog. There are two ways to configure this.

Option 1: Config file (~/.icerc.yaml)

Create ~/.icerc.yaml with one or more catalog entries:

catalogs:
  default:
    type: rest
    uri: https://your-catalog-endpoint.com/api
    credential: ${ICEBERG_CREDENTIAL}
    warehouse: my-catalog          # required by most REST catalog servers

Environment variables in ${VAR} syntax are resolved automatically.

Use the --catalog / -c flag to select a catalog:

ice timeline my_db.my_table -c production

The default catalog is used when no flag is provided. You can also set the ICE_CATALOG environment variable.

Option 2: Environment variables

For single-catalog setups:

export ICEBERG_REST_URI=https://your-catalog-endpoint.com/api
export ICEBERG_CREDENTIAL=client_id:client_secret   # optional

Commands

All commands support --json for machine-readable output and --catalog / -c to select a catalog.

ice ls

List namespaces in your catalog, or tables within a namespace.

ice ls                        # list all namespaces
ice ls MY_DB                  # list tables in MY_DB
ice ls MY_DB --json           # JSON output

 Namespaces
┌────────────┬────────┐
│ Namespace  │ Tables │
├────────────┼────────┤
│ ICEBERG    │     12 │
│ STAGING    │      3 │
└────────────┴────────┘

Flag	Description
--json	JSON output

ice timeline

Git-log style history of a table's snapshots. Shows operation type, record/file counts per snapshot, and flags schema or partition changes.

ice timeline ICEBERG.MY_TABLE
ice timeline ICEBERG.MY_TABLE --limit 10

* 1406252174952524172 (2026-01-09 13:19:49) [append] +15841836926 records, +10380 files
* 387196757114825475 (2026-01-09 13:42:52) [append] +193073 records, +512 files
  parent: 1406252174952524172

Flag	Description
--limit, -n	Max snapshots to show
--json	JSON output

ice health

Runs automated health checks and reports findings with severity levels and actionable recommendations.

ice health ICEBERG.MY_TABLE

Checks performed:

Check	What it detects
File Count	Total files relative to ideal count based on observed median file size
Small Files	Files under 10 MB (read amplification)
Oversized Files	Files over 1 GB (memory pressure)
Snapshot Bloat	Too many retained snapshots (metadata overhead)
Delete Files	High delete-to-data file ratio (merge-on-read overhead)
Manifest Fragmentation	Too many small manifests (slow scan planning)
Partition Skew	Uneven partition sizes (identifies which partition)

Flag	Description
--json	JSON output

ice files

File-level analysis: size statistics, distribution histogram, format breakdown, and largest/smallest file lists.

ice files ICEBERG.MY_TABLE
ice files ICEBERG.MY_TABLE --snapshot 1406252174952524172
ice files ICEBERG.MY_TABLE --top 20

Flag	Description
--snapshot, -s	Analyze files from a specific snapshot
--top, -t	Number of largest/smallest files to show (default: 10)
--json	JSON output

ice diff

Compare two snapshots: files added/removed, row counts, size changes, schema diffs, and partition spec changes.

ice diff ICEBERG.MY_TABLE --from 1406252174952524172 --to 387196757114825475

Flag	Description
--from	Source snapshot ID (required)
--to	Target snapshot ID (required)
--json	JSON output

ice snap

Detailed snapshot view with branches, tags, and per-snapshot manifest statistics (added/existing/deleted file counts).

ice snap ICEBERG.MY_TABLE

Flag	Description
--json	JSON output

ice orphans

Finds data files on storage that aren't referenced by any snapshot. Optionally deletes them. Works with local filesystems, GCS, and S3.

ice orphans ICEBERG.MY_TABLE             # dry-run: list orphans
ice orphans ICEBERG.MY_TABLE --clean     # interactive: delete with confirmation

Flag	Description
--clean	Delete orphan files (prompts for confirmation)
--json	JSON output

ice schema

Schema evolution viewer. Shows the current schema, or a specific version, and diffs between versions (added, removed, and modified columns).

ice schema ICEBERG.MY_TABLE
ice schema ICEBERG.MY_TABLE --version 0

Flag	Description
--version, -v	Show a specific schema version
--json	JSON output

ice compact

Compaction advisor — analyzes write patterns, projects file accumulation, and recommends compaction strategy with ready-to-run SQL.

ice compact ICEBERG.MY_TABLE
ice compact ICEBERG.MY_TABLE --engine snowflake    # only Snowflake SQL
ice compact ICEBERG.MY_TABLE --engine spark         # only Spark SQL
ice compact ICEBERG.MY_TABLE --top 5                # top 5 partitions

The advisor requires at least 5 append snapshots before making file accumulation projections. With fewer appends, it reports current table state without extrapolating unreliable growth rates.

Target file size is inferred from the table's observed median file size (excluding small files under 10 MB) rather than a hardcoded value. This means the tool respects whatever size your engine (Snowflake, Spark, etc.) chose to write, and only flags drift from that baseline.

Flag	Description
--engine, -e	SQL dialect: spark, snowflake, or all (default)
--top, -t	Number of partitions to show (default: 10)
--days, -d	Projection window in days (default: 30)
--json	JSON output

ice info

Quick table overview: location, format version, partition spec, sort order, and table properties.

ice info ICEBERG.MY_TABLE

Table: ICEBERG.MY_TABLE
Location: gs://bucket/path/to/table
Format Version: 2

Current Schema: v0 (10 fields)
Partition Spec: [day(event_time), region]
Sort Order: [timestamp ASC NULLS FIRST]
Current Snapshot: 5510719240075213409

                Table Properties
┌────────────────────────────────────┬──────────┐
│ Property                           │ Value    │
├────────────────────────────────────┼──────────┤
│ write.parquet.compression-codec    │ zstd     │
│ write.target-file-size-bytes       │ 268435456│
└────────────────────────────────────┴──────────┘

Flag	Description
--json	JSON output

ice expire

Snapshot expiration analyzer. Without criteria, shows a summary of all snapshots. With criteria, shows what would be expired and provides ready-to-run commands.

ice expire ICEBERG.MY_TABLE                          # summary only
ice expire ICEBERG.MY_TABLE --older-than 30          # expire snapshots > 30 days old
ice expire ICEBERG.MY_TABLE --retain-last 10         # keep only last 10 snapshots
ice expire ICEBERG.MY_TABLE --older-than 14 --retain-last 5  # combine criteria

Without flags, shows snapshot distribution and prompts for criteria:

Snapshot Summary: ICEBERG.MY_TABLE

  Total Snapshots    42
  Oldest             2025-11-01 (94 days ago)
  Newest             2026-02-03 (0 days ago)

    By Operation
┌───────────┬───────┐
│ Operation │ Count │
├───────────┼───────┤
│ append    │    38 │
│ replace   │     4 │
└───────────┴───────┘

Specify criteria to see expiration plan:
  --older-than N   Expire snapshots older than N days
  --retain-last N  Always keep last N snapshots

With criteria, shows expiration plan with commands for PyIceberg, Spark, and Snowflake:

Flag	Description
--older-than, -o	Expire snapshots older than N days
--retain-last, -r	Always keep last N snapshots
--json	JSON output

Note on Snowflake: For Snowflake-managed Iceberg tables, snapshot expiration is automatic based on DATA_RETENTION_TIME_IN_DAYS. For tables using an external catalog (e.g., Polaris), you must use PyIceberg or Spark to expire snapshots — Snowflake won't delete them.

JSON output

Every command supports --json for piping into jq, scripts, or dashboards:

ice health ICEBERG.MY_TABLE --json | jq '.[] | select(.severity != "info")'
ice timeline ICEBERG.MY_TABLE --json | jq '.[0].summary'

Status

This project is a work in progress. Commands and output formats may change. So far it has only been tested against Snowflake Open Catalog (Polaris) but should work with any Iceberg REST catalog.

License

MIT