Repository navigation
v0.4.0 — deterministic best-of-K multi-start + parallel feature - #1
Merged
Merged
Conversation
Profiling (examples/bench.rs, per-phase timing) shows the separation search is 97-99% of wall-clock on any nest with an unplaced tail (pentagon: 3330ms of 3340ms); construction/improve are noise. The dominant cost is per-sample collision-query + pole-pair proxy compute across a well-tuned iteration budget -- NOT allocation, key lookups, or loss sums. This reframes the perf work: micro-opts on those give ~0; the real levers are multi-start + safe parallelism (best-of-K at ~1x wall-clock). - examples/bench.rs: per-probe best-of-N wall-clock harness (Instant under a scoped allow -- a benchmark, never a placement input; criterion is avoided because its dep tree would breach deny.toml's exact license allowlist). - examples/seedsweep.rs: empirical best-of-K probe (public API only) -- the evidence for multi-start (bricks 91.9% -> 95.5% over 16 seeds). - sep/evaluator.rs (A1): reuse one BasicHazardCollector + terms Vec across the ~hundreds of evaluate_sample calls per move instead of allocating per sample. Byte-identical (the canonical terms sort makes the summed loss independent of the reused buffer's contents/order); the golden is unchanged. Measured ~0 speedup -- kept for reduced allocation pressure under the coming parallelism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A single greedy construction lands in a seed-dependent basin; sweeping K decorrelated seeds and keeping the densest result lifts packing on heterogeneous parts at no determinism cost. Measured end-to-end: 13x7 bricks 101 -> 105 placed (91.9% -> 95.5%) at K=8. This is the density lever docs/02 section 11 missed -- it tested depth (more budget = same basin), never breadth (K basins). - optimizer/lib.rs: nest_multistart + nest_multistart_per_item (additive; existing signatures untouched). Each start runs the unmodified, golden-stable nest_per_item at seed.wrapping_add(k); keep-best maximises total placed AREA (deterministic fixed-order sum compared via total_cmp, earliest-k on tie) -- no float sum drives the decision's ordering. n_starts=1 == nest() bit-for-bit. - ironnest façade re-exports both; py wheel nest() gains an optional restarts=1 kwarg (backward-compatible -- omitting it is byte-identical to today). - golden: a "# multistart-mixed" K=3 case appended to the cross-platform golden (purely additive -- every existing case stays byte-for-byte identical). - tests: restarts=1 == nest() byte-identity, determinism, never-worse-than-1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Best-of-K multi-start costs ~K× wall-clock sequentially (bricks K=8 = 22.6s). The `parallel` feature runs the K independent constructions on std::thread::scope threads, cutting that to ~3.6× faster (K=8 = 6.2s on 10 cores) with BYTE-IDENTICAL output. Why it stays deterministic: each start is the unmodified, golden-stable nest_per_item (own PCG64 stream, own Layout, zero shared mutable state); results are collected in k order via ordered join handles; the keep-best reduction runs afterwards in that fixed k order, so thread completion order is irrelevant and no float sum ever crosses a thread. This is the fixed-reduction-order carve-out the determinism rules allow — the ONLY sanctioned threading, and only on the multi-start meta-loop, never inside a placement search. - optimizer: `[features] parallel`; run_starts has sequential + scoped-thread impls (cfg-split). Off by default; std-only, no new dependency. - ironnest forwards the feature; the wheel (crates/py) turns it ON (the GIL is already released around the solve), so restarts>1 is threaded for consumers. - tests: parallel_multistart_equals_independent_sequential_best (gated on the feature) asserts the threaded reduction == an independent sequential best-of-K. - ci.yml: a new cross-platform step runs the golden + equivalence test under --features parallel on macOS-arm64/Win-x64/linux — the standing proof that parallel == sequential, byte-for-byte, on every shipped target. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…xamples)
The determinism-auditor caught the clippy gate red on this branch (the RTK proxy
summary had masked it):
- lib.rs nest_multistart doc: backtick `SplitMix64` (clippy::doc_markdown).
- examples/{bench,seedsweep}.rs: `&[item.clone()]` -> std::slice::from_ref(&item)
(clippy::redundant_clone equivalent). A doc-comment failure on the library was
short-circuiting clippy before it reached the examples, hiding these.
Verified GREEN via raw output (rtk proxy) in BOTH feature configs:
cargo clippy --workspace --exclude ironnest-py --all-targets [--features parallel] -- -D warnings
plus cargo fmt --all --check. No behavior change; determinism audit verdict was
SAFE on the underlying A1/B1/B2 work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docs/02 §11.1: the round-2 negative-results log. Records (a) the profiling finding that separation is 97-99% of wall-clock, so the A-stack micro-opts are ~0; (b) multi-start as the *breadth* lever §11 round-1 missed (bricks 91.9% -> 95.5%, byte-deterministic); (c) two more measured dead ends — B3 structured anchor seeding (null) and C2 ILS ruin-and-recreate (pentagon cap holds, +1 brick, dominated by multi-start). Concludes the pentagon-class gap is still architectural but the heterogeneous-parts gap was partly a breadth gap. - CLAUDE.md: document nest_multistart's `parallel` feature as the ONE sanctioned exception to "no threads on a placement-deciding path" — it already satisfies the existing "if parallel, fix the reduction order" clause; this names the instance so it is not later misread as a violation and ripped out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New (all additive — existing API + the cross-platform golden are byte-unchanged): - nest_multistart / nest_multistart_per_item: best-of-K multi-start that keeps the densest of K decorrelated-seed runs. Lifts heterogeneous packing (13x7 bricks 91.9% -> 95.5% at K=8) with zero determinism cost (integer/total_cmp argmax over placed area). n_starts=1 is byte-identical to nest(). - `parallel` cargo feature: runs the K starts on std::thread::scope threads (3.6x faster, byte-identical — golden re-runs under --features parallel on all 3 CI platforms). The wheel enables it; nest() gains an optional restarts=1 kwarg. - A1: byte-identical separator scratch-buffer reuse. Determinism-audited SAFE. Density/perf round-2 findings (incl. the measured-out A-stack micro-opts, B3, and C2 dead ends) recorded in docs/02 §11.1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds deterministic best-of-K multi-start — the density lever
docs/02 §11missed — made affordable by safe parallelism. Fully additive: existing API and the cross-platform golden are byte-unchanged.nest_multistart/nest_multistart_per_item(Rust) + an optional backward-compatiblerestarts=1kwarg on the wheel'snest(). Runs K decorrelated-seed constructions and keeps the densest.n_starts=1is byte-identical tonest().parallelcargo feature — runs the K starts onstd::thread::scopethreads: 3.6× faster best-of-K (K=8: 22.6s → 6.2s on 10 cores), byte-identical output. The wheel ships it ON.Why it's deterministic (the prime directive holds)
Each start is the unmodified, golden-stable
nest_per_item(own PCG64 stream, ownLayout, zero shared state). The keep-best reduction is an integer/total_cmpargmax over placed area, earliest-k on tie — no float sum on the decision path and no float reduction crosses a thread (results join in fixed k order, argmax runs after all joins). This satisfies CLAUDE.md's existing "if parallel, fix the reduction order" clause; the rule now names this as the one sanctioned threading instance.Proof: the cross-platform golden gained a K=3 multistart case (existing 84 lines byte-for-byte unchanged), and CI now re-runs the golden under
--features parallelon macOS-arm64 / Win-x64 / linux — it must match the same snapshot. A localparallel_multistart_equals_independent_sequential_besttest asserts threaded == independent sequential best-of-K. Determinism-audited SAFE.What was measured and rejected (so it's not re-tried)
Profiling reframed the perf work: separation is 97–99% of wall-clock, so the A-stack micro-opts (collector reuse, hazkey cache, incremental loss) are ~0 — A1 kept as hygiene, A2/A4 dropped. The density prototypes were each measured and rejected: B3 structured anchor seeding (null — the BL-slide already captures contacts) and C2 ILS ruin-and-recreate (pentagon cap holds, +1 brick, dominated by multi-start). All recorded in
docs/02 §11.1.Verification
cargo test --workspace(60) and--features parallel(47) green · golden byte-identical sequential and parallelcargo clippy -D warningsgreen in both feature configs ·cargo fmt --checkclean · py wheel compilesPost-merge
Tag
v0.4.0onmainafter merge to trigger the PyPI trusted-publisher build (the version bump is in this PR; the tag is intentionally not on the feature branch).🤖 Generated with Claude Code