perf: choose pivots before sorting, and fuse phase 3 into phase 1 - #9

Closed

BenjaminDEMAILLE wants to merge 41 commits into

COMBINE-lab:mainfrom

BenjaminDEMAILLE:perf/fuse-phase1-phase3

Commits on Aug 11, 2026

test+build: validate the LCP array, and pin the release profile

BenjaminDEMAILLE
and
claude
committed
ccbcca6
feat: radix-seeded prefix doubling for the plain in-memory SA

BenjaminDEMAILLE
and
claude
committed
8174590
feat: `verify_sa`, an O(n) independent suffix-array check

BenjaminDEMAILLE
and
claude
committed
f11c459
perf: parallelise grouping and rank publication in the doubling path

BenjaminDEMAILLE
and
claude
committed
b4ecde6
ci: add a Rust workflow

BenjaminDEMAILLE
and
claude
committed
9f401e7
bench: add a reproducible chr21 harness

BenjaminDEMAILLE
and
claude
committed
c8683e6
docs: document the fast path, and correct two stale claims

BenjaminDEMAILLE
and
claude
committed
290c5af
docs: fix a broken intra-doc link in `build_ext_mem_for_filter`

BenjaminDEMAILLE
and
claude
committed
72db165
perf: replace the seed comparison sort with an MSD counting sort

BenjaminDEMAILLE
and
claude
committed
a7daf6d
feat: extend the doubling path to subsets and to in-memory sample sort

BenjaminDEMAILLE
and
claude
committed
7d79209
docs: refresh the fast-path numbers and coverage

BenjaminDEMAILLE
and
claude
committed
31c1bb4
perf: skip long periodic runs instead of scanning them

BenjaminDEMAILLE
and
claude
committed
2ec3064
docs: record the external-memory run-skipping results

BenjaminDEMAILLE
and
claude
committed
f07943b
perf: prefetch the next candidates' text in the merge loop

BenjaminDEMAILLE
and
claude
committed
d09bff7
perf: raise the auto-picked subarray target from 64Ki to 128Ki records

BenjaminDEMAILLE
and
claude
committed
6776439
perf: seed each phase-1 subarray with the packed key

BenjaminDEMAILLE
and
claude
committed
e9a1c93
docs: record the full external-memory speedup ladder

BenjaminDEMAILLE
and
claude
committed
4fc99de
perf: merge each cascade level's run pairs in parallel

BenjaminDEMAILLE
and
claude
committed
11d31f1
perf: pipeline phase 4's emit against the next chunk's merge

BenjaminDEMAILLE
and
claude
committed
b6300bf
docs: extend the external-memory ladder with the last two steps

BenjaminDEMAILLE
and
claude
committed
b1608c2
perf: re-sort run-free partitions by key instead of cascading merges

BenjaminDEMAILLE
and
claude
committed
c7f0cf1
docs: add the partition key re-sort to the ladder

BenjaminDEMAILLE
and
claude
committed
b1fd8fb
perf: rank the alphabet before packing keys

BenjaminDEMAILLE
and
claude
committed
a21b8c5
docs: record the alphabet-ranking result

BenjaminDEMAILLE
and
claude
committed
7c14607
perf: pack keys with a SWAR gather over a pre-ranked text

BenjaminDEMAILLE
and
claude
committed
1d9f3f2
docs: refresh in-memory numbers after the SWAR key packing

BenjaminDEMAILLE
and
claude
committed
f0af118
perf: drop the per-group allocation and prepass from doubling rounds

BenjaminDEMAILLE
and
claude
committed
a53ced9
docs: refresh in-memory numbers after the doubling-round cleanup

BenjaminDEMAILLE
and
claude
committed
3ef11f2

Commits on Aug 12, 2026

fix: require exact `u8` for packed keys, and stop taxing every LCP call

BenjaminDEMAILLE
and
claude
committed
7c780b0
feat: packed keys for segmented texts, including STAR's boundary order

BenjaminDEMAILLE
and
claude
committed
226e65c
bench: annotation-shaped splice-junction fixture, and run CI on every branch

BenjaminDEMAILLE
and
claude
committed
ca63c58
bench: measure segmented keys under a realistic splice-junction library

BenjaminDEMAILLE
and
claude
committed
470b55c
bench: correct the segmented-key memory figure, and sweep the partition target

BenjaminDEMAILLE
and
claude
committed
55ba9f1
feat: make the dense-subset full-SA path an explicit byte budget

BenjaminDEMAILLE
and
claude
committed
40a2593
feat: seed the fallback merge with the LCP the key sort already proved

BenjaminDEMAILLE
and
claude
committed
b3064ce
feat: `lcp_array`, deriving the LCP array from the suffix array in O(n)

BenjaminDEMAILLE
and
claude
committed
919e9ba
docs: how to measure the external-memory path without fooling yourself

BenjaminDEMAILLE
and
claude
committed
0682e5e
perf: gallop from the previous split in phase 3's pivot search

BenjaminDEMAILLE
and
claude
committed
007b3a6
bench: break down the seed sort's passes, and a negative result on bucket count

BenjaminDEMAILLE
and
claude
committed
5034129
docs: locate the external-memory path's remaining time in phase 3

BenjaminDEMAILLE
and
claude
committed
d88ae30
perf: choose pivots before sorting, and fuse phase 3 into phase 1

BenjaminDEMAILLE
and
claude
committed
2d90c29