From 67df38a905db60548e321dbed6e32169996cdcb0 Mon Sep 17 00:00:00 2001 From: Mohammed Alshahrani Date: Sun, 24 May 2026 11:34:13 +0300 Subject: [PATCH 1/2] =?UTF-8?q?docs:=20add=20tutorial.md=20=E2=80=94=20han?= =?UTF-8?q?ds-on=20walkthrough=20with=20callbacks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New `docs/src/tutorial.md` page (~285 lines, slotted between Quickstart and Algorithm in the nav) walks a user end-to-end through solving a monotone equation with DFProjection, then drills into the callback architecture. Live-evaluated via Documenter `@example` blocks so the rendered HTML shows real numbers and an inline convergence plot. Sections: § 1 Setup — define a tiny inline F, build a NonlinearProblem, solve with DFProjection(), inspect retcode/iters/fevals/residual. § 2 Choosing components — swap the line search and inertial rule; table summarizing the five pluggable components and their defaults. § 3 Built-in observer callbacks — LoggingCallback (per-iter trace, configurable columns + interval) and HistoryCallback (collect :k/:F_norm per iter, then plot the convergence curve inline via Plots.jl). Documents the :resid-is-NaN-until-termination caveat and recommends :F_norm for live convergence data. § 4 Stopping criteria are callbacks — explains the AbstractStoppingCriterion <: AbstractCallback design, composes AnyOf(AbsResidualTol, MaxIters, MaxTime) with a 50 ms wall-clock cap to demo MaxTime firing first, surfaces the retcode. § 5 Writing a custom callback — implements IterateSnapshotCallback from scratch (subtypes AbstractCallback, defines on_event! on :post_iter, copies cache.x because the solver mutates in place). Closes with the three rules: copy cache.x, return nothing for observers, event order :initialize → per-iter :post_linesearch → :post_iter → :terminate. § 6 Comparative sweep — small loop over 3 inline-defined monotone operators × 3 built-in line searches, results aggregated into a DataFrame and rendered inline. Closing pointer to the DFMethods-starter template for larger sweeps. The tutorial is library-generic, not paper-reproduction: the source paper is cited ONCE in the § 0 background paragraph and never mentioned again in the body. Problem names are deliberately paper-agnostic (`smooth_exp`, `smooth_sine`, `identity_log`) so the tutorial reads as a generic showcase of the library's API surface. docs/Project.toml: add Plots, DataFrames, LinearAlgebra (already in the library's main env via the test path; need to be in docs/ for the @example blocks to load). docs/make.jl: add `"Tutorial" => "tutorial.md"` to the pages list between Quickstart and Algorithm. Two reviewer agents ran during T.7: - Reference-API audit confirmed Plots/DataFrames are NOT in docs/Project.toml yet (now added), enumerated the exact callback constructor signatures from src/callbacks.jl, verified MaxTime is a positional Float64 and AnyOf takes varargs. - Post-impl code review caught one 🔴 bug (1-arg F closures — should be 2-arg (u, p)→F per SciMLBase's NonlinearProblem contract) and three 🟡 hardening items; all fixed. Paper-framing audit cleared. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/Project.toml | 3 + docs/make.jl | 1 + docs/src/tutorial.md | 297 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 301 insertions(+) create mode 100644 docs/src/tutorial.md diff --git a/docs/Project.toml b/docs/Project.toml index 844fc4c..ecd9676 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -1,6 +1,9 @@ [deps] CommonSolve = "38540f10-b2f7-11e9-35d8-d573e4eb0ff2" DFMethods = "5fd0b45f-fdf3-4567-a8b1-5e033765ff5d" +DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4" +LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" NonlinearSolve = "8913a72c-1f9b-4ce2-8d82-65094dcecaec" +Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" SciMLBase = "0bca4576-84f4-4d90-8ffe-ffa030f20462" diff --git a/docs/make.jl b/docs/make.jl index e62ee1f..283dfc2 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -21,6 +21,7 @@ makedocs(; pages=[ "Home" => "index.md", "Quickstart" => "quickstart.md", + "Tutorial" => "tutorial.md", "Algorithm" => "algorithm.md", "Constraint Sets" => "constraints.md", "Extending" => "extending.md", diff --git a/docs/src/tutorial.md b/docs/src/tutorial.md new file mode 100644 index 0000000..eb99fd3 --- /dev/null +++ b/docs/src/tutorial.md @@ -0,0 +1,297 @@ +# Tutorial + +A hands-on walkthrough of solving monotone nonlinear equations with +DFMethods.jl, with emphasis on the **callback architecture** that lets +you observe, instrument, and stop solves on your own terms. + +> **Background.** DFMethods.jl is a configurable framework for +> derivative-free projection methods applied to constrained nonlinear +> equations ``F(x) = 0,\ x \in X``, with ``X \subset \mathbb{R}^n`` closed +> convex and ``F`` continuous and (pseudo-)monotone. The library was +> developed alongside research published as Ibrahim, Alshahrani & +> Al-Homidan (2026), *A Unified Derivative-Free Projection Framework for +> Convex-Constrained Nonlinear Equations*, +> [`10.1007/s10957-025-02826-x`](https://doi.org/10.1007/s10957-025-02826-x). + +If you're brand new to the package, skim [Quickstart](@ref) first; this +page assumes you've seen `solve(prob, DFProjection())` once before. + +--- + +## 1. Setup + +We start with a smooth scalar-separable problem on ``\mathbb{R}^n``: + +```math +F_i(x) = \exp(x_i) - 1, \quad i = 1, \dots, n, +``` + +so the unique zero is ``x^* = 0``. We use a plain `NonlinearProblem` (the +constraint set defaults to ``\mathbb{R}^n``, i.e. unconstrained). + +```@example tut +using DFMethods, NonlinearSolve +using SciMLBase, LinearAlgebra + +n = 200 +# `NonlinearProblem` expects a 2-arg operator `f(u, p)`. +F = (u, p) -> exp.(u) .- 1 +x0 = ones(n) +prob = NonlinearProblem(F, x0) + +sol = solve(prob, DFProjection()) + +(retcode = sol.retcode, iters = sol.stats.nsteps, + fevals = sol.stats.nf, resid = norm(F(sol.u, nothing))) +``` + +That's the whole flow: define `F`, wrap in a `NonlinearProblem`, hand +to `solve` with a `DFProjection` algorithm instance. Everything else in +this tutorial is about **customizing** that algorithm or **observing** +what it's doing. + +--- + +## 2. Choosing components + +`DFProjection` is a *configurable* solver — five pluggable components, +each with sensible defaults: + +| Field | Default | What it controls | +|------------------|----------------------------------|--------------------------------| +| `direction` | `SpectralThreeTerm()` | search direction ``d_k`` | +| `linesearch` | `ResidualNormBacktrack()` | accepted step size ``\alpha_k``| +| `inertial` | `Inertial(0.25)` | momentum on ``w_k`` | +| `iterate_update` | `SolodovSvaiterProjection()` | how ``x_{k+1}`` is built | +| `stopping` | `AnyOf(AbsResidualTol, MaxIters)`| termination conditions | + +Swap any one (or several) at construction. For example, here we use a +different built-in line search and a heavier inertial weight: + +```@example tut +alg = DFProjection(; + linesearch = AdaptiveClampedBacktrack(), + inertial = Inertial(0.5), +) +sol = solve(prob, alg) +(retcode = sol.retcode, iters = sol.stats.nsteps, fevals = sol.stats.nf) +``` + +For the full list of built-in components and how to write your own, see +[Extending](@ref). + +--- + +## 3. Built-in observer callbacks + +DFMethods ships two built-in observer callbacks that attach to any +`DFProjection` via the `callbacks` keyword. + +### `LoggingCallback` — a per-iteration trace + +```@example tut +sol = solve(prob, + DFProjection(; callbacks = [LoggingCallback(columns = (:k, :F_norm, :α, :n_evals), + every = 5)])) +(retcode = sol.retcode, iters = sol.stats.nsteps) +``` + +`columns` is any subset of [`HISTORY_FIELDS`](@ref); `every = 5` prints +every fifth iteration. A header appears at `:initialize`, a summary +footer at `:terminate`. + +### `HistoryCallback` — collect per-iteration data for analysis + +`HistoryCallback` appends a `NamedTuple` per iteration to its `history` +field. You pick which scalars to capture. Below, we collect just +``(k, \|F(z_k)\|)`` then plot the convergence curve. + +```@example tut +using Plots +gr() + +hist = HistoryCallback(fields = (:k, :F_norm)) +sol = solve(prob, DFProjection(; callbacks = [hist])) + +ks = [row.k for row in hist.history] +fnorms = [row.F_norm for row in hist.history] + +plot(ks, fnorms; + yscale = :log10, lw = 2, marker = :circle, markersize = 3, + xlabel = "iteration k", ylabel = "‖F(z_k)‖", + title = "Convergence (exponential demo, n=$n)", + legend = false, framestyle = :box, grid = true, + size = (700, 400)) +``` + +The `:F_norm` field is the residual at the last trial point ``z_k`` — +computed live every iteration. (Caveat: `:resid` from +[`HISTORY_FIELDS`](@ref) is only populated at termination; use +`:F_norm` for live convergence data.) + +--- + +## 4. Stopping criteria are callbacks + +A useful design choice in DFMethods: every stopping criterion is itself +an `AbstractCallback`. Concretely, they all subtype +`AbstractStoppingCriterion <: AbstractCallback` and return +`(stop::Bool, retcode::Symbol)` from `on_event!`. You compose them with +`AnyOf` (terminate on first match). + +The built-in set: + +| Criterion | Stops when | +|---------------------|-------------------------------------------| +| `AbsResidualTol(ε)` | ``\|F(z_k)\| < \varepsilon`` | +| `RelResidualTol(η)` | ``\|F(z_k)\| < \eta \cdot \|F(x_0)\|`` | +| `MaxIters(N)` | ``k \ge N`` | +| `MaxFEvals(N)` | cumulative F evals reach ``N`` | +| `MaxTime(T)` | wall-clock seconds since solve start ``> T`` | +| `StepNormTol(τ)` | ``\|x_{k+1} - x_k\| < \tau`` | +| `DirectionNormTol(τ)` | ``\|d_k\| < \tau`` | + +For example, here we give the solver a deliberately tight residual +tolerance, a generous iteration budget, and a tiny wall-clock cap so +the time limit fires first: + +```@example tut +alg_capped = DFProjection(; + stopping = AnyOf(AbsResidualTol(1e-12), + MaxIters(10_000), + MaxTime(0.05)), # 50 ms +) +wall = @elapsed sol = solve(prob, alg_capped) +(retcode = sol.retcode, iters = sol.stats.nsteps, + wallclock_seconds = round(wall; digits = 3)) +``` + +`sol.retcode` will reflect which criterion fired first (a Symbol from +`(:Success, :MaxIters, :MaxTime, :MaxFEvals, ...)` mapped through +SciML's return-code enum). Useful for telemetry: a `:MaxTime` return is +a different failure mode than `:MaxIters`. + +--- + +## 5. Writing a custom callback + +The same `on_event!(cb, cache, event::Symbol)` contract that built-in +observers and stopping criteria use is also the user-extension point. +Subtype `AbstractCallback`, implement `on_event!`, and you can observe +or record anything reachable from the solver's `cache` at any of the +four events: `:initialize`, `:post_linesearch`, `:post_iter`, +`:terminate`. + +Here's a minimal callback that saves the current iterate ``x_k`` every +`N` iterations — handy for downstream trajectory analysis or making an +animation of the path through ``\mathbb{R}^n``: + +```@example tut +mutable struct IterateSnapshotCallback <: DFMethods.AbstractCallback + every::Int + snapshots::Vector{Vector{Float64}} +end + +IterateSnapshotCallback(; every::Int = 10) = + IterateSnapshotCallback(every, Vector{Float64}[]) + +function DFMethods.on_event!(cb::IterateSnapshotCallback, cache, event::Symbol) + if event === :post_iter && cache.k % cb.every == 0 + push!(cb.snapshots, copy(cache.x)) # copy! cache.x is mutated each iter + end + return nothing +end + +snap = IterateSnapshotCallback(every = 1) # capture every iter for the demo +sol = solve(prob, DFProjection(; callbacks = [snap])) + +(snapshots_collected = length(snap.snapshots), + first_snapshot_norm = round(norm(first(snap.snapshots)); digits = 6), + last_snapshot_norm = round(norm(last(snap.snapshots)); digits = 6)) +``` + +Three rules to know: + +1. **`cache.x` is mutated in place** by the solver. If you want to keep + a snapshot, `copy(cache.x)` — not just push the reference. +2. **`on_event!` returns `nothing`** for observers. (Stopping + criteria return `(Bool, Symbol)` — a different protocol on the same + abstract type.) +3. **Events fire in a fixed order per iteration**: `:initialize` once + at start, then per iteration `:post_linesearch` → `:post_iter`, + then `:terminate` once at end. + +See [Extending](@ref) for the full `on_event!` contract and the list +of fields available on `cache`. + +--- + +## 6. A small comparative sweep + +Putting it together: let's compare a few built-in line searches across +a small set of inline-defined problems. We collect the results in a +`DataFrame` and show the table inline. + +```@example tut +using DataFrames + +# Three small monotone test operators (paper-agnostic; widely used in +# the literature on derivative-free projection methods). +problems = [ + ("smooth_exp" , 100, (u, p) -> exp.(u) .- 1), + ("smooth_sine" , 100, (u, p) -> 2.0 .* u .- sin.(u)), + ("identity_log", 100, (u, p) -> u .+ log.(abs.(u) .+ 1) ./ 100), +] + +line_searches = [ + ("LS-const", ConstantBacktrack()), + ("LS-residual", ResidualNormBacktrack()), + ("LS-adaptive", AdaptiveClampedBacktrack()), +] + +results = DataFrame(problem = String[], + linesearch = String[], + iters = Int[], + fevals = Int[], + converged = Bool[], + F_norm = Float64[]) + +for (pname, n_p, F_p) in problems + x0_p = ones(n_p) + for (lname, ls) in line_searches + sol = solve(NonlinearProblem(F_p, x0_p), + DFProjection(; linesearch = ls)) + push!(results, ( + problem = pname, + linesearch = lname, + iters = sol.stats.nsteps, + fevals = sol.stats.nf, + converged = SciMLBase.successful_retcode(sol.retcode), + F_norm = norm(F_p(sol.u, nothing)), + )) + end +end + +results +``` + +Each row is one `(problem, line search)` cell. Median CPU and other +columns are easy to add — see the source of any column on the +[API Reference](@ref) page or the [`HISTORY_FIELDS`](@ref) listing. + +--- + +## Next steps + +- [Algorithm](@ref) — how `step!` actually composes the six pluggable + components into one outer iteration. +- [Constraint Sets](@ref) — solving on a box, a halfspace, the + intersection, or your own user-defined ``X``. +- [Extending](@ref) — full contracts for writing custom search + directions, line searches, inertial rules, iterate-update strategies, + and stopping criteria. +- For a larger benchmark template — multi-problem sweeps with + SQLite-backed result storage, threaded execution, performance-profile + generation — see the companion + [`DFMethods-starter`](https://github.com/mmogib/DFMethods-starter) + repository. From fb15d52a3ca65af1dc448ef5b7c381c8e2957757 Mon Sep 17 00:00:00 2001 From: Mohammed Alshahrani Date: Sun, 24 May 2026 11:35:47 +0300 Subject: [PATCH 2/2] Release v0.3.1: docs-only tutorial enhancement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump version from 0.3.0 → 0.3.1 (PATCH per Julia semver — no source changes, no API additions/removals, 242/242 tests pass unchanged). Promote CHANGELOG entry [0.3.1] with the tutorial-page details from the preceding `67df38a` commit. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ Project.toml | 2 +- 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cbb214..885517f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,51 @@ All notable changes to **DFMethods.jl** will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.3.1] — 2026-05-24 + +Documentation enhancement release. Adds a new **Tutorial** page to the +manual (between Quickstart and Algorithm in the nav) demonstrating the +end-to-end solver workflow with emphasis on the callback architecture +(observer callbacks, stopping criteria as callbacks, writing a custom +callback from scratch). No source-code changes; no API additions or +removals; 242/242 tests pass unchanged. + +### Added + +- **`docs/src/tutorial.md`** — hands-on walkthrough (live-evaluated via + Documenter `@example` blocks): + - § 1 Setup — minimum-viable `solve(NonlinearProblem(F, x0), DFProjection())` + - § 2 Choosing components — overrides for `linesearch`, `inertial`, etc. + - § 3 Built-in observer callbacks — `LoggingCallback` (per-iter table) + + `HistoryCallback` (collect data, plot inline convergence curve via + Plots.jl) + - § 4 Stopping criteria as callbacks — `AnyOf(AbsResidualTol, MaxIters, + MaxTime)` composition; documents the `AbstractStoppingCriterion <: + AbstractCallback` design + - § 5 Writing a custom callback — `IterateSnapshotCallback` from scratch + showing the `on_event!(cb, cache, event::Symbol)` contract and the + `:initialize` / `:post_linesearch` / `:post_iter` / `:terminate` event + order + - § 6 Comparative sweep — small 3-problem × 3-line-search loop with + results aggregated into a `DataFrame` and rendered inline + +### Changed + +- **`docs/Project.toml`**: added `Plots`, `DataFrames`, `LinearAlgebra` — + needed by the new tutorial's `@example` blocks (rendered convergence + plot in § 3, results table in § 6). +- **`docs/make.jl`**: added `"Tutorial" => "tutorial.md"` to the `pages` + list, slotted between Quickstart and Algorithm. + +### Compatibility + +- Julia ≥ 1.10 +- `SciMLBase` v2.x +- `CommonSolve` v0.2.x +- `LineSearch` v0.1.x + +No source-level changes — drop-in upgrade from v0.3.0. + ## [0.3.0] — 2026-05-22 Element-type genericity, direction-state ctx surfacing, documentation @@ -280,6 +325,7 @@ for constrained nonlinear equations $F(x) = 0$ on a closed convex set $X$. - `SciMLBase` v2.x - `CommonSolve` v0.2.x +[0.3.1]: https://github.com/mmogib/DFMethods.jl/releases/tag/v0.3.1 [0.3.0]: https://github.com/mmogib/DFMethods.jl/releases/tag/v0.3.0 [0.2.1]: https://github.com/mmogib/DFMethods.jl/releases/tag/v0.2.1 [0.2.0]: https://github.com/mmogib/DFMethods.jl/releases/tag/v0.2.0 diff --git a/Project.toml b/Project.toml index a51b6fb..90e1b7e 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "DFMethods" uuid = "5fd0b45f-fdf3-4567-a8b1-5e033765ff5d" authors = ["Mohammed Alshahrani "] -version = "0.3.0" +version = "0.3.1" [deps] CommonSolve = "38540f10-b2f7-11e9-35d8-d573e4eb0ff2"