s3Cluster with use_hive_partitioning does not prune hive partitions (SIZES_OF_COLUMNS_DOESNT_MATCH / read amplification)

[CarlosFelipeOR]

Type of problem

Bug report - something's broken

Describe the situation

s3Cluster with use_hive_partitioning=1 does not prune hive partitions when a WHERE filter is applied on a partition column that exists only in the object path. The plain s3() function prunes correctly (reads 1 partition), but s3Cluster():

• Crashes with Code: 9 SIZES_OF_COLUMNS_DOESNT_MATCH when the partition column exists only in the path.
• Reads all partitions (no pruning, read amplification) when the partition column also exists in the file content, filtering only after reading.

Discovered via the /s3/minio/part 3/hive partitioning regression scenarios (hive_partitioning_cluster_*), whose timing assertion fails due to the read amplification.

This issue:

• Was failing on 25.8.22.20001.altinityantalya and is now also failing on 26.3.10.20001.altinityantalya
• Is NOT analyzer-dependent (reproduced with allow_experimental_analyzer 0 and 1)
• Is environment-independent (AWS m8g ARM and Hetzner ARM both read all partitions)

How to reproduce the behavior

Environment

• Version: 26.3.10.20001.altinityantalya (also 25.8.22.20001.altinityantalya)
• Cluster: 3 shards, S3/MinIO backend

Steps

1. Create 25 hive partitions where the CSV content has only d UInt64; the date column
   exists only in the path:

-- repeat for X = 01..25
INSERT INTO FUNCTION s3('http://minio:9000/bucket/date=2000-01-<X>/data.csv',
  'user','pass','CSVWithNames','d UInt64')
SELECT number FROM numbers(1000000) SETTINGS s3_truncate_on_insert=1;

2. s3() prunes correctly:

SELECT count(*) FROM s3('http://minio:9000/bucket/date=2000-01-*/data.csv',
  'user','pass','CSVWithNames','d UInt64')
WHERE date='2000-01-01' SETTINGS use_hive_partitioning=1;
-- => 1000000  (reads only the date=2000-01-01 object)

3. s3Cluster() without WHERE works:

SELECT count(*) FROM s3Cluster('cluster','http://minio:9000/bucket/date=2000-01-*/data.csv',
  'user','pass','CSVWithNames','d UInt64') SETTINGS use_hive_partitioning=1;
-- => 25000000

4. s3Cluster() with WHERE on the hive partition column crashes:

SELECT count(*) FROM s3Cluster('cluster','http://minio:9000/bucket/date=2000-01-*/data.csv',
  'user','pass','CSVWithNames','d UInt64')
WHERE date='2000-01-01' SETTINGS use_hive_partitioning=1;

Expected behavior

Returns 1000000, reading only the date=2000-01-01 partition (same as s3()).

Actual behavior

Code: 9. DB::Exception: Size of filter (25) doesn't match size of column (0): While executing Remote.

When the partition column also exists in the file content, no crash occurs, but the query reads all 25 partitions (25,000,000 rows) instead of pruning to 1,000,000.

[ianton-ru]

Full stack trace:

2026.06.01 16:52:36.607497 [ 91 ] {79c1f7c7-f9ce-4e31-9ff1-2767ca9155c3} <Error> executeQuery: Code: 9. DB::Exception: Size of filter (2) doesn't match size of column (0): While executing Remote. (SIZES_OF_COLUMNS_DOESNT_MATCH) (version 26.1.6.20001.altinityantalya (altinity build)) (from 172.19.0.1:59214) (query 1, line 1) (in query: SELECT count() FROM s3Cluster('ch-cluster', 'http://minio1:9001/root/hivebug/date=*/data.csv', 'minio', '[HIDDEN]', 'CSVWithNames', 'd UInt32') WHERE date = '2000-01-01' SETTINGS use_hive_partitioning = 1), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000165dab2a
1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000df55e0e
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000df55749
3. DB::Exception::Exception<unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, unsigned long&&, unsigned long&) @ 0x000000000f4dddb6
4. DB::ColumnVector<unsigned short>::filter(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) const @ 0x000000001be12127
5. DB::VirtualColumnUtils::filterBlockWithExpression(std::shared_ptr<DB::ExpressionActions> const&, DB::Block&) @ 0x000000001c4bc420
6. DB::VirtualColumnUtils::getFilterByPathAndFileIndexes(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::ExpressionActions> const&, DB::NamesAndTypesList const&, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const> const&) @ 0x000000001c4c08ae
7. void DB::VirtualColumnUtils::filterByPathOrFile<std::shared_ptr<DB::ObjectInfo>>(std::vector<std::shared_ptr<DB::ObjectInfo>, std::allocator<std::shared_ptr<DB::ObjectInfo>>>&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::ExpressionActions> const&, DB::NamesAndTypesList const&, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const> const&) @ 0x00000000199e14e6
8. DB::StorageObjectStorageSource::GlobIterator::next(unsigned long) @ 0x00000000199e041f
9. DB::StorageObjectStorageStableTaskDistributor::getMatchingFileFromIterator(unsigned long) @ 0x0000000019a35557
10. DB::StorageObjectStorageStableTaskDistributor::getNextTask(unsigned long) @ 0x0000000019a3353b
11. DB::TaskDistributor::operator()(unsigned long) const @ 0x00000000199987a4
12. DB::RemoteQueryExecutor::processPacket(DB::Packet) @ 0x000000001a3ae87c
13. DB::RemoteQueryExecutor::readAsync() @ 0x000000001a3b0612
14. DB::RemoteSource::tryGenerate() @ 0x000000001d5b03ba
15. DB::ISource::work() @ 0x000000001d17b981
16. DB::ExecutionThreadContext::executeTask() @ 0x000000001d198fd7
17. DB::PipelineExecutor::executeStepImpl(unsigned long, DB::IAcquiredSlot*, std::atomic<bool>*) @ 0x000000001d18ac44
18. void std::__function::__policy_func<void ()>::__call_func[abi:ne210105]<DB::PipelineExecutor::spawnThreads(std::shared_ptr<DB::IAcquiredSlot>)::$_0>(std::__function::__policy_storage const*) @ 0x000000001d18f083
19. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::worker() @ 0x000000001673666b
20. void std::__function::__policy_func<void ()>::__call_func[abi:ne210105]<ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*&&)::'lambda'()>(std::__function::__policy_storage const*) @ 0x000000001673d4a6
21. ThreadPoolImpl<std::thread>::ThreadFromThreadPool::worker() @ 0x000000001673398f
22. void* std::__thread_proxy[abi:ne210105]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x000000001673ad5a
23. ? @ 0x0000000000094ac3
24. __clone @ 0x0000000000125a84