Skip to content

Opt-in bounds_handling = BoundsProjection() for Newton-type solvers - #1354

Open
ParadaCarleton wants to merge 5 commits into
SciML:masterfrom
ParadaCarleton:bounds-projection
Open

ParadaCarleton wants to merge 5 commits into
SciML:masterfrom
ParadaCarleton:bounds-projection

Conversation

@ParadaCarleton

Copy link
Copy Markdown

Stacked on #1353 (its three commits appear here until it merges; the change of this PR is the last two commits).

Summary

NewtonRaphson, TrustRegion/TrustRegionDogleg, Broyden, Klement and LimitedMemoryBroyden take bounds through the logit change of variables from #814, which reaches a root on a bound only in the limit and stalls or returns Inf when the iterate saturates (#1142, #1147, #1152). #1254 asks for bounds in the original coordinates, with the transform kept as a fallback. This adds the smallest step in that direction as an extensible choice:

abstract type AbstractBoundsHandling end
struct BoundsTransform  <: AbstractBoundsHandling end  # change of variables (default, current behavior)
struct BoundsProjection <: AbstractBoundsHandling end  # clamp every candidate iterate into [lb, ub]

NewtonRaphson(; bounds_handling = BoundsProjection())

Further methods (active set, step truncation, semismooth) become new subtypes with their own allowsbounds trait instead of new keywords.

 every candidate iterate (initial guess, line-search trial, trust-region trial, accepted step)
+  project_iterate!!(u, lb, ub)          # one shared function in NonlinearSolveBase
   evaluate residual / Jacobian at the clamped point
 trust region ratio
-  predicted reduction of the unclamped step
+  predicted reduction of the clamped step actually taken
 quasi-Newton secant pair
+  the clamped step; a step clamped to nothing skips the update (and forces a rebuild
+  from an already-updated Jacobian)

With BoundsProjection() the algorithm reports allowsbounds, so the transform is not applied. A line search runs on a wrapper problem that clamps, so its merit function never sees a point outside the box; user jac/jvp/vjp are not forwarded to that wrapper.

Evidence

Tests: lib/NonlinearSolveFirstOrder/test/misc_tests__item11.jl (NewtonRaphson, with BackTracking, with jacobian_reuse, TrustRegion, Dogleg) and lib/NonlinearSolveQuasiNewton/test/core_tests__item15.jl (Broyden, with BackTracking, true Jacobian, Klement, LimitedMemoryBroyden). The residual throws if it is ever called outside the box. Each checks: allowsbounds is false by default; a root exactly on the upper bound is returned exactly; in-place/out-of-place; an infeasible initial guess is clamped (the problem's u0 untouched); scalar state and bounds; static arrays (first order); one-sided bounds and a fixed coordinate; reinit!(...; reuse_jacobian = true) across parameter values; a Newton step overshooting a bound is clamped onto it and accepted without shrinking the trust radius. FirstOrder Core and NativeBounds, QuasiNewton Core and Base Core pass.

Limits: a projected Newton method, not an active-set method, so at a corner where the Newton step points out of the box it stalls. A trust region inside a quasi-Newton method is rejected with an ArgumentError. Nonlinear least squares is untouched (TrustRegionReflective/Dogbox exist).

Merge Danger

Door: two-way

Blast Radius: opt-in

The default BoundsTransform() keeps today's path; project_bounds never shipped. No change to SciMLBase.allowedkeywords.

A matrix-free Jacobian operator is rebound to the current iterate on every
step, so JacobianReuse switched itself off for it. What is expensive there is
the preconditioner: an iterative linear solver rebuilt its `precs` whenever
`A` was rebound, even when the caller asked to reuse.

An explicit JacobianReuse policy now stays active for a matrix-free operator.
Steps remain exact Newton steps (the operator is bound to the current
iterate); the preconditioner built for the last refreshed Jacobian is kept
until the policy refreshes. The size-based default still leaves matrix-free
solves untouched.

`reuse_A_if_factorization = true` now keeps the preconditioner of an
iterative linear solver (via LinearSolve's `reinit!(...; reuse_precs = true)`)
in addition to a factorization.
…oss solves

reinit! reset the Jacobian reuse policy, so a sequence of related solves
through one cache (continuation correctors) refactorized and rebuilt the
preconditioner at the start of every solve; only the manual
step!(cache; recompute_jacobian = false) interface kept a Jacobian.

With reuse_jacobian = true, the first-order cache keeps its Jacobian, the
linear solver's factorization and preconditioner, and the policy's age. The
residual reference is set to the new initial residual, so the first step's
contraction decides a refresh as usual, and the stale-Jacobian retry still
applies. Default reinit! behavior is unchanged.
…e logit transform for NewtonRaphson, TrustRegion and quasi-Newton solvers
…hare project_iterate!! in NonlinearSolveBase; avoid per-step allocation in the clamped quasi-Newton step; test the clamped trust-region ratio

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant