Skip to content

Don't use ctid scan for negative relpages (like during scanning a partitioned table).#435

Merged
staticlibs merged 2 commits intoduckdb:mainfrom
simi:graceful-partition-scan-fail
Apr 12, 2026
Merged

Don't use ctid scan for negative relpages (like during scanning a partitioned table).#435
staticlibs merged 2 commits intoduckdb:mainfrom
simi:graceful-partition-scan-fail

Conversation

@simi
Copy link
Copy Markdown
Contributor

@simi simi commented Apr 11, 2026

Scanning a partitioned table caused an infinite hang because PostgreSQL sets relpages=-1 for relkind='p' tables. Cast to idx_t (uint64) gives pages_approx = 2^64-1, so the scanner loops over ~18 quintillion pages.

Now throws NotImplementedException at bind time with a message that includes the relation kind and table name. Now fallback to 0 estimate as scanner already does in few other cases. Keep original relpages value (idx_t needs to be "downgraded" to int64_t to be able to hold original -1 value) in approx_num_pages and later during preparing scan, check for negative approx_num_pages and fallback to use_ctid_scan = false for now.


Initial step to provide support for partitioned tables, since those are broken as reported at #220.

I have plan to open set of smaller PRs to improve the situation ending with full support of partitioned tables. This is initial foundation - accepting the broken state and fallback to slower but working scan.

Example EXPLAIN (SELECT hangs):

lake D EXPLAIN (FORMAT JSON) SELECT * FROM pg.public.sales_revenues;

┌─────────────────────────────┐
│┌───────────────────────────┐│
││       Physical Plan       ││
│└───────────────────────────┘│
└─────────────────────────────┘
[
    {
        "name": "POSTGRES_SCAN",
        "children": [],
        "extra_info": {
            "Table": "sales_revenues",
            "Projections": [
                "id",
                "account_id",
                "sale_id",
                "report_id",
                "report_group_id",
                "organization_id",
                "contract_id",
                "configuration_id",
                "product_id",
                "product_asset_id",
                "asset_id",
                "base",
                "share",
                "royalty",
                "currency",
                "chain",
                "statement_id",
                "unit_amount"
            ],
            "Estimated Cardinality": "18446744073709551568"
        }
    }

pg_class returns

reporting_development=# SELECT relkind, relpages FROM pg_class WHERE relname = 'sales_revenues';
 relkind | relpages 
---------+----------
 p       |       -1

Scanning a partitioned table caused an infinite hang because PostgreSQL
sets relpages=-1 for relkind='p' tables. Cast to idx_t (uint64) gives
pages_approx = 2^64-1, so the scanner loops over ~18 quintillion pages.

Now throws NotImplementedException at bind time with a message that
includes the relation kind and table name.
Comment thread src/postgres_utils.cpp Outdated
Comment thread src/storage/postgres_table_set.cpp Outdated
Comment thread src/storage/postgres_table_set.cpp
@simi
Copy link
Copy Markdown
Contributor Author

simi commented Apr 11, 2026

I'll check on CI failures.

@staticlibs
Copy link
Copy Markdown
Collaborator

@simi

Just FYI, some details on local test runs: #431 (comment)

- fixes hanging partitioned table scan for now
@simi
Copy link
Copy Markdown
Contributor Author

simi commented Apr 12, 2026

@staticlibs thanks for the hint, by mistake I have run only new tests before submitting PR. Clearly this should not break ATTACH. I have reworked this batch to just fallback to use_ctid_scan = false for negative relpages (as partitioned table provides). I have added new commit to see the difference over original idea and updated the PR name and description. I can squash if needed. If welcomed, I'll continue working on better support of partitioned tables in other PRs.

@simi simi changed the title Fail with a clear error when scanning a partitioned table Don't use ctid scan for negative relpages (like during scanning a partitioned table). Apr 12, 2026
Comment thread src/include/storage/postgres_table_entry.hpp
Copy link
Copy Markdown
Collaborator

@staticlibs staticlibs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good to me.

Comment thread src/include/storage/postgres_table_entry.hpp
Comment thread src/storage/postgres_table_set.cpp
@staticlibs staticlibs merged commit b127e08 into duckdb:main Apr 12, 2026
15 checks passed
@simi simi deleted the graceful-partition-scan-fail branch April 12, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants