fix(search): full-depth default layers and NaN-safe candidates#2
Open
smanaton wants to merge 3 commits into
Open
fix(search): full-depth default layers and NaN-safe candidates#2smanaton wants to merge 3 commits into
smanaton wants to merge 3 commits into
Conversation
…dates Replace the fixed 8B-era default --layers [1,2,3,4,34] with an even spread derived from architecture layer_count (up to 12 points). Without this, larger models (e.g. 64-layer 27B) only search ~8% of depth and appear unpatchable. Also: hard-fail non-finite baselines; treat non-finite candidate gaps as rejected flips instead of aborting the whole search; measure_probes returns NaN gaps as data for callers to handle.
Copilot stopped reviewing on behalf of
smanaton due to an error
July 19, 2026 09:31
Fake backends prove NaN baseline hard-fails before search iterates, while NaN-after-flip candidates are rejected and reverted without aborting the run.
…ates - SearchConfig default leaves search_layers empty; run_search fills them via default_search_layers(backend.layer_count()) so library users are not stuck on an 8B-sized (36-layer) default. - measure_probes is strict again (eval-safe). Search candidates use measure_probes_allowing_non_finite so a NaN flip is rejected, not fatal. - Doc comment documents small-model inclusion of layer 0; unit test for resolve_search_layers.
smanaton
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Search defaults and robustness for multi-scale models (especially 64-layer 27B-class GGUFs).
Default layers
--layers [1,2,3,4,34]with an even full-depth spread fromlayer_count(up to 12 points).SearchConfig::default()leavessearch_layersempty;run_searchresolves them from the live backend so library callers are not stuck on a hardcoded 36-layer default.--layersto use the computed default; help text updated.NaN handling
measure_probesis strict (eval-safe baselines/reports).measure_probes_allowing_non_finiteso a non-finite gap rejects that flip and continues, instead of aborting the whole search or poisoning eval transitions.Tests
resolve_search_layers.Motivation
On a 64-layer model the old absolute default never sampled past layer 34 (~8% of depth), which made search look effect-size limited when it was a coverage bug. With full-depth layers we fixed structured-output probes end-to-end (Bonsai 8B/27B lab runs).
Test plan
cargo test --lib search/cargo test --test search_engine(Windows, CUDA + mainline llama.cpp).--layers: checkpoint recorded full-depth layers (max 63).so_json_false_literalsearch + promote.Notes
Fork staging: smanaton#2
Related: Windows builds typically need the wwama MSVC/CUDA PR.