feat: Benchmarking infrastructure, perf optimisations, and GC/latency investigation

[benmandrew]

Overview

This branch does two things: it builds a Landmark-based benchmarking framework that makes the heap vs zero-alloc stack allocation paths measurable, and uses that framework to drive a sequence of performance optimisations. A final investigation pass then goes deeper on GC behaviour and latency distribution across all 40 QR versions.

Cumulative speedup from the original baseline: 15.06G → 3.74G cycles (4.03×) across a 120 000-call benchmark (3 inputs × 4 ECLs × 10 000 iterations).

---

Benchmarking infrastructure

Landmark profiling (bench/bench.ml)

A bench/ executable comparing generate_qr (heap path, Qr.make each call) against generate_qr_stack (zero-alloc path, pre-allocated Qr.t in an Arena). Each pipeline phase is wrapped with its own Landmark node so the profiler emits a full call-graph breakdown across encode → split_into_blocks → interleave_blocks → place_pattern_modules → place_format_info → place_data_and_apply_mask.

Because generate_qr_stack is [@zero_alloc], its sub-functions cannot share Landmark IDs with the heap path. A generate_qr_stack_bench variant carries its own stack/*-prefixed Landmark set for instrumented profiling, leaving the original generate_qr_stack as the clean zero-alloc reference.

A third benchmark path calls generate_qr_stack directly with only one outer Landmark enter/exit, isolating Landmark overhead from allocation overhead:

Path	Cycles	Minor bytes/call	Major bytes/call
generate_qr (heap)	3.72G	2,275	6.1
generate_qr_stack (landmark-wrapped)	3.63G	197	0.03
generate_qr_stack (direct)	3.45G	35	0

The 197 bytes/call on the Landmark-wrapped stack path is entirely the six inner enter/exit pairs. The direct path is genuinely zero-alloc.

---

Performance optimisations

1. Precompute reserved-cell bitmap

Added a reserved : bytes field to Qr.t, populated once by mark_reserved. place_data and apply_mask_pattern previously called is_reserved on every cell, which walked alignment coordinate lists. They now do an O(1) bitmap lookup. Also fixes a latent bug where apply_mask_pattern passed hardcoded version=1 to is_reserved, silently skipping alignment-pattern exclusion for QR codes version 2+.

• place_data: 7.15G → 2.45G cycles (2.9×)
• apply_mask_pattern: 3.65G → 1.04G cycles (3.5×)

2. Rewrite mark_reserved to stamp regions

The old implementation iterated all width² cells testing each via is_reserved → is_in_alignment_pattern (O(alignment_coords²) per cell). The rewrite stamps known regions directly — three 9×9 corner rectangles, two timing strips, each alignment 5×5 square — all O(reserved_cells). is_in_alignment_pattern and is_reserved are removed entirely. Also replaces bit_pos/8 and bit_pos%8 in place_data with running refs (removing integer division from the inner loop) and (x+y) % 2 with (x+y) land 1 in apply_mask_pattern.

• place_pattern_modules: 6.68G → 761M cycles (8.8×)

3. Combine place_data + apply_mask into one pass; optimise Reed-Solomon

Both functions performed an identical full-matrix zigzag scan. The combined place_data_and_apply_mask does a single pass applying mask pattern 0 at write time, and bypasses the set_module bounds checks since the scan guarantees valid coordinates.

Reed-Solomon: polynomial_mult(generator[j], coef) was recomputing log_table[coef] on every inner-loop iteration. Caching it once eliminates that redundancy. log_table[generator[j]] is now precomputed into a generator_log_polynomials table at startup, removing one lookup per inner step and the polynomial_mult call overhead entirely.

• place_data_and_apply_mask: 3.40G → 1.54G cycles (2.2×)
• split_into_blocks (RS): 1.51G → 716M cycles (2.1×)

4. Replace assoc-list lookups with flat array indexing in Config

capacity_table and ec_table were association lists built by prepending, so version 1 sat at position 159. Both are now flat int arrays indexed by (version - 1) * 4 + ecl_idx — O(1) per lookup. get_ec_info reconstructs its record inside exclave_ (stack-allocated, zero heap cost). find_version no longer allocates in its loop.

• encode: 913M → 223M cycles (4.1×)
• Cumulative from baseline: 15.06G → 3.74G (4.03×)

---

GC and latency investigation

Major-heap spill threshold (bench/bench.ml — GC stat section)

Runs 1 000 iterations per version for inputs spanning v1–v21, capturing Gc.stat deltas for minor_gc, major_gc, minor_words, promoted_words, and major_direct.

OCaml's Max_young_wosize is 256 words (2 048 bytes). Qr.t carries two Bytes.make buffers of width² bytes:

Version	Per-buffer size	GC path
v6	1 681 B (210 words)	minor heap
v10	3 249 B (406 words)	direct to major heap

At v21, the heap path triggers 5× more major GC cycles per 1 000 calls than the stack path. The stack path shows constant minor_gc=6, major_gc=3 at every version.

Mean runtime vs QR version (bench/bench_versions.ml)

Scans increasing alphanumeric string lengths to find the minimum input forcing each version 1–40 under ECL L. Uses an adaptive iteration count targeting 300 ms of wall time (min 300, max 100 000). Output: CSV with version, input_chars, heap_ns, stack_ns, heap_iters, stack_iters.

Finding: mean latency is essentially identical for both paths at every version. The heap overhead from Qr.make is 1–5% at small versions and disappears into noise above v15 as Reed-Solomon and O(width²) matrix operations dominate.

Per-call latency distribution (bench/bench_dist.ml + bench/time_ns.c)

time_ns.c wraps clock_gettime(CLOCK_MONOTONIC) as a [@@noalloc] OCaml external returning an unboxed int (nanoseconds). [@@noalloc] prevents GC safe-point insertion around the call, so no collection can slip between the timer read and the function under test.

bench_dist.ml collects 5 000 individual call latencies per path per version and outputs p50/p90/p95/p99/p99.9/max as CSV.

Tail latency diverges sharply at v8–v25 — the range where Qr.t spills to the major heap and per-call time is still short enough for GC pauses to dominate the tail:

Version	heap p50	heap p99.9	stack p50	stack p99.9	ratio
v8	32 µs	211 µs	32 µs	83 µs	2.5×
v9	40 µs	287 µs	40 µs	104 µs	2.8×
v10	38 µs	265 µs	38 µs	98 µs	2.7×
v15	76 µs	321 µs	75 µs	181 µs	1.8×
v40	461 µs	956 µs	457 µs	914 µs	~1.0×

Above v30, per-call time (~270–460 µs) is long enough that OS scheduler jitter dominates both paths equally. Remaining spikes on the stack path at p99.9/max are pure OS preemption — the stack path does zero allocation inside the timed call.

---

Running the benchmarks

# Detailed Landmark profile (heap vs stack, sub-function breakdown, GC stats)
dune exec bench/bench.exe

# Mean runtime vs version — outputs CSV to stdout
taskset -c 7 dune exec bench/bench_versions.exe > versions.csv

# Latency distribution (p50–max) vs version — outputs CSV to stdout
taskset -c 7 dune exec bench/bench_dist.exe > dist.csv

taskset -c N pins the process to a single core to eliminate CPU-migration jitter. For cleaner tail-latency numbers, combine with sudo chrt -f 99 or configure isolcpus=N nohz_full=N in the kernel boot parameters.