Skip to content

feat(survey): evaluate LHS points in greedy-nearest order#428

Closed
vsbuffalo wants to merge 1 commit into
mainfrom
feature/survey-greedy-order
Closed

feat(survey): evaluate LHS points in greedy-nearest order#428
vsbuffalo wants to merge 1 commit into
mainfrom
feature/survey-greedy-order

Conversation

@vsbuffalo

Copy link
Copy Markdown
Owner

camdl survey now evaluates its LHS points in a greedy-nearest order — seed a coarse spread, then each round evaluate the unevaluated points closest (in transform-normalized parameter space) to the current best-loglik point, a batch at a time so every core stays busy.

Why

The progress bar's best-loglik metric climbs fast and plateaus early when the box actually holds a basin — an at-a-glance read on whether the survey bounds are placed well — and the good region is the part that fills in first. Easier to watch, and quick to notice a mis-placed box. (No early-stop — deliberately. Every point is still evaluated; a survey's job is the global picture, and best-first + early-stop could bias toward the first basin and miss a distant better mode.)

Byte-identical output

This reorders when points run, never which:

  • every point is still evaluated;
  • each point's result is keyed on (seed, point_id) (derive_point_seed / mix_cell_seed), not a shared RNG stream, so it's independent of evaluation order;
  • the final sort by (loglik desc, point_id asc) is order-independent.

So landscape.tsv is byte-identical to the old index-order sweep — no golden / expected-output / CAS-identity impact. The greedy distance metric reuses EstimatedParam::to_transformed (log/logit-aware) and only orders evaluation; it never touches a loglik.

Test

survey_greedy_order_invariant.rs pins it from outside: the same survey run at --parallel 1 vs --parallel 4 walks two different greedy orders and must produce byte-identical landscape.tsv. Goes red if a future change makes a point's result order-dependent.

Green locally: the new invariant test, plus survey_roundtrip and fit_survey_denominator (no regression).

Replace the flat index-order `par_iter` sweep with a batched greedy
schedule: seed a coarse spread of points, then each round evaluate the
unevaluated points nearest — in transform-normalized parameter space — to
the current best-loglik point, a batch at a time so every core stays busy.

The payoff is a live signal. The progress bar's best-loglik metric now
climbs fast and plateaus early when the box holds a basin, which is an
at-a-glance read on whether the survey bounds are placed well, and the
good region is the part that fills in first — easier to watch, and quick
to notice a mis-placed box.

This reorders WHEN points run, never WHICH: every point is still
evaluated (no early stop), each point's result is keyed on
(seed, point_id) via `derive_point_seed` / `mix_cell_seed` rather than a
shared RNG stream, and the final sort by (loglik desc, point_id asc) is
order-independent. So `landscape.tsv` is byte-identical to the old sweep
— no golden, expected-output, or CAS-identity impact. The distance metric
reuses `EstimatedParam::to_transformed`, so points on log/logit params are
compared on the same scale the sampler and landscape geometry live on; it
only orders evaluation and never touches a loglik.

`survey_greedy_order_invariant.rs` pins the invariant from outside: the
same survey at `--parallel 1` vs `--parallel 4` walks two different greedy
orders and must produce byte-identical `landscape.tsv`. If a future change
makes a point's result order-dependent, it goes red.
@vsbuffalo

Copy link
Copy Markdown
Owner Author

Closing — wrong target. This applied greedy ordering to survey, but the intent was the 2D profile likelihood grid. Redoing against profile.rs.

@vsbuffalo vsbuffalo closed this Jul 11, 2026
@vsbuffalo
vsbuffalo deleted the feature/survey-greedy-order branch July 11, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant