The CPU executor introduced in #160 submits one file read per unique chunk in query order. It bypasses the read merging and round-robin ordering across shards that the CUDA executor already uses. Translated crops consequently issue many small reads concentrated on a few shards, even when every batch contains queries from all sixteen shards.
At revision 5126f0958c0be3922a0a00c49d127ba62ddbc6a9:
cpu_dispatch submits consecutive prepared-plan chunks, capped by decode_workers, with one store_read per chunk. Two groups can be active; each is reused after its reads, decoding, and assembly complete.- The shared planner deduplicates chunks within a batch but keeps first-encounter order.
coalesce_chunks merges adjacent/overlapping reads and then interleaves reads from distinct shard files. CUDA dispatch uses this; CPU dispatch does not.
A Turin CPU pilot used the same 16 GiB uint16 volume and translated 1x256x256 queries for Damacy and TensorStore 0.1.85. The volume had sixteen 1 GiB raw shards in a 1x4x4 grid, 1x128x256 chunks (64 KiB), and 16 KiB Blosc blocks with bitshuffle. Both backends used 32 decode/copy participants and 16 file readers. Damacy's benchmark snapshot also added Blosc-LZ4 support without changing read scheduling.
A separate 512-crop diagnostic, excluded from throughput timing, found:
| Measurement |
Damacy |
TensorStore |
| LZ4 file-read calls |
3,042 |
1,776 |
| LZ4 bytes returned by reads |
121.3 MiB |
121.1 MiB |
| LZ4 actual storage reads |
240.3 MiB |
128.0 MiB |
| Peak simultaneous file-read calls, both codecs |
16 |
16 |
| Peak distinct shards, LZ4 / Zstd |
9 / 10 |
14 / 14 |
The unique requested encoded byte ranges match exactly between backends. Both LZ4 traces cover 127.9 MiB after page rounding. Excess physical traffic therefore is not explained by different chunk selection; kernel readahead is a separate, unverified hypothesis. The traces do not establish how much of the throughput gap read ordering or merging causes. Neither backend demonstrated sixteen-way distinct-shard overlap in these diagnostics.
Requested change:
- Apply adjacent-read merging and round-robin ordering across shard files to the CPU executor, reusing the existing implementation where practical.
- Preserve one decode per distinct chunk, correct assembly of repeated/translated crops, configured decode worker counts, bounded memory and read sizes, and CPU-only operation.
- Add deterministic regression coverage proving that CPU reads merge and are distributed across shards, including bounded-reader behavior and unchanged outputs.
This issue concerns CPU read planning. Changing readahead policy or establishing a performance improvement needs separate measurement; translation also retains the geometric cost of intersecting more chunks.
Saved evidence: ~/tmp/2026-09-23-damacy-pr160-shard-grid/job-3804536/report/ and the four job-3804241/results/smooth4-{lz4,zstd}-c64-b16/concurrency/{damacy,tensorstore}/ diagnostics. The report retains the workload, hashes, repetitions, and limitations.
The CPU executor introduced in #160 submits one file read per unique chunk in query order. It bypasses the read merging and round-robin ordering across shards that the CUDA executor already uses. Translated crops consequently issue many small reads concentrated on a few shards, even when every batch contains queries from all sixteen shards.
At revision
5126f0958c0be3922a0a00c49d127ba62ddbc6a9:cpu_dispatchsubmits consecutive prepared-plan chunks, capped bydecode_workers, with onestore_readper chunk. Two groups can be active; each is reused after its reads, decoding, and assembly complete.coalesce_chunksmerges adjacent/overlapping reads and then interleaves reads from distinct shard files. CUDA dispatch uses this; CPU dispatch does not.A Turin CPU pilot used the same 16 GiB uint16 volume and translated
1x256x256queries for Damacy and TensorStore 0.1.85. The volume had sixteen 1 GiB raw shards in a1x4x4grid,1x128x256chunks (64 KiB), and 16 KiB Blosc blocks with bitshuffle. Both backends used 32 decode/copy participants and 16 file readers. Damacy's benchmark snapshot also added Blosc-LZ4 support without changing read scheduling.A separate 512-crop diagnostic, excluded from throughput timing, found:
The unique requested encoded byte ranges match exactly between backends. Both LZ4 traces cover 127.9 MiB after page rounding. Excess physical traffic therefore is not explained by different chunk selection; kernel readahead is a separate, unverified hypothesis. The traces do not establish how much of the throughput gap read ordering or merging causes. Neither backend demonstrated sixteen-way distinct-shard overlap in these diagnostics.
Requested change:
This issue concerns CPU read planning. Changing readahead policy or establishing a performance improvement needs separate measurement; translation also retains the geometric cost of intersecting more chunks.
Saved evidence:
~/tmp/2026-09-23-damacy-pr160-shard-grid/job-3804536/report/and the fourjob-3804241/results/smooth4-{lz4,zstd}-c64-b16/concurrency/{damacy,tensorstore}/diagnostics. The report retains the workload, hashes, repetitions, and limitations.