Rewrite HybridPSO L-BFGS with custom Strong Wolfe line search

[AdityaPandeyCN]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Replaced SimpleNonlinearSolve dependency with custom implementation:

• Strong Wolfe line search (Nocedal & Wright Algorithm 3.5)
• Zoom phase (Nocedal & Wright Algorithm 3.6)
• L-BFGS two-loop recursion (Nocedal & Wright Algorithm 7.4)
• Cubic interpolation for step length selection

[AdityaPandeyCN]

@ChrisRackauckas Can you have a look?

[utkarsh530]

I think we can make this more modular here. First, https://github.com/SciML/LineSearch.jl exists, so any line-search implementation should go there. I believe those line searches are GPU compatible. If not, we can have our own LS here. But the core (L)-BFGS algorithm should be simple, and I am not sure why a complete custom implementation will be better. NonlinearSolve.jl seems to have been a more modular solution, unless it wasn't performant enough.

[AdityaPandeyCN]

I was assuming this is the Linesearch repo https://github.com/JuliaNLSolvers/LineSearches.jl whole time which is not gpu-compatible and implemented it from scratch here :(

[utkarsh530]

What's the status? Is it ready to go?

[AdityaPandeyCN]

Hello, it will be ready after this merges (SciML/NonlinearSolve.jl#948)

[AdityaPandeyCN]

@utkarsh530 Please take a look.