fix(anan): compensate the real ANAN-G2 DDC0 edge droop, in-app calibrated

[tropo1234]

Summary

The ANAN-G2's DDC0 CIC/half-band decimation chain has a measurable
amplitude droop toward the edges of the passband — real, bench-measured
attenuation, not a display artifact. This PR adds host-side compensation
for it, calibrated per-radio rather than assumed from a fixed model:

• AnanDroopCorrection — the correction table shape/application
  (kDroopCorrectionZero as the identity/no-op table), applied inside
  AnanRxDsp::processIqBlock(), looked up per rate via
  droopTableForRate().
• AnanDroopCalibrator — the in-app sweep engine that measures the
  actual droop on the connected radio (a live instance owned by
  RadioModel, wired through AnanBackend::connectRadio()'s
  loadTables() call).
• hostDroopCalibration — the persisted settings scope
  (AnanBackend's apply path), and a droopcal automation-bridge verb
  so the sweep can be driven/scripted.
• A "Droop Correction" tab in RadioSetupDialog — the operator-facing
  UI to run the sweep and see the result.
• A cosmetic fade at the true DDC0 edge instead of chasing the last dB
  of gain there, since fully flattening the extreme edge would mean
  amplifying a region with very little real signal-to-noise margin left.

Part of #4970.

Constitution principle honored

• Principle IV — clean-room: the correction is derived from
  bench-measured sweep data on real hardware (this project's own
  AnanDroopCalibrator sweep), not ported from another project's
  calibration table.
• Principle V — hostDroopCalibration is persisted as its own
  single nested-JSON settings object (per-radio, via
  RadioSettingsScope), not flat keys.
• Principle VI — calibration is receive-side gain compensation only;
  nothing here keys or drives the transmitter.

Test plan

• Local build passes (cmake --build build -j8)
• Bench-verified on a real ANAN-G2 as originally submitted — droop
  sweep run, correction table loaded and applied, edge response visibly
  flattened on the panadapter without the earlier over-gained edge
  artifact
• G2 re-run pending for the review fixes. Blockers 1 and 6 change
  what the sweep measures — the correction is now bypassed for its
  duration, and a failed rate change no longer reports as landed — so
  the original bench run does not cover them. Two things to confirm
  specifically: a second sweep reproduces the first one's table
  rather than collapsing to near-zeros, and Apply after a partial
  sweep leaves previously calibrated rates intact across a reconnect.
• Unit tests pass, run standalone from fresh binaries built on this
  branch (now rebased onto main, no longer stacked):
  anan_droop_correction_test, anan_droop_calibrator_test,
  anan_rxdsp_handedness_test, anan_backend_test,
  radio_capability_gating_test, connection_panel_size_test — all
  report all checks passed, exit 0
• New regression coverage: anan_rxdsp_handedness_test Group 7 — a
  bypassed and a cleared table each return the emitted frame to the raw
  FFT across the whole frame, tails included, which is what catches
  a zero-table "bypass" leaving the edge fade running — plus four
  saveTables cases in anan_droop_calibrator_test against a real
  settings store: a partial save merges, a re-measured rate overwrites
  only itself, a newer schema is refused without writing anything, and
  empty/invalid inputs report why
• Full ctest: 2 failures out of 334, both confirmed pre-existing by
  checking out 202d876e and rebuilding — hl2_state_restore_test
  (3 CW-passband checks) and phone_tx_filter_numeric_entry_test
  (aborts inside Qt's own testlib/qasciikey.cpp). Neither is
  reachable from this branch.

Checklist

• Commits are signed
• hostDroopCalibration is a single nested-JSON AppSettings object
  (Principle V), not flat keys
• Code is clean-room
• Documentation updated — 02-working-plan.md's Step 2b entries
• N/A — no security-sensitive change

[ten9876]

Issue fit

Part of #4970. The engineering core is genuinely good: a real, bench-measured DDC0 droop, corrected per-radio from an in-app sweep rather than an assumed model; the capability field's own comment ("what varies is whether the client has measured it, not whether the radio droops") is exactly the right framing; Principle V persistence via RadioSettingsScope; three socket-free tests with a regression row pinning a real bug the author found and fixed (kLog10Floor). The blockers are all in the sweep's lifecycle, not the math.

Also: the body's stacking note appears stale — anan-pr1-rx-skeleton looks like #5143, which is already main's tip commit, and this PR's 8 commits sit cleanly on it with green CI. Worth refreshing the body (and rebasing if anything drifted) so reviewers stop treating this as blocked.

Scope

All 28 files map to the feature, with one exception flagged for a maintainer call rather than waved through: RadioCertification::Phase::Spectrum and the pan bandwidth bridge verb (~130 lines of new public automation surface) exist to serve tools/anan_droop_calibration.py — which is not in the tree, and which AnanDroopCalibrator.h's own comment says was superseded by the in-app engine. Six comments across four files still point readers at the missing file. Three independent analysis passes converged: the in-app calibrator needs neither (it drives setPanBandwidth and taps panFeedSpectrumReady directly). New public verbs with zero in-tree consumers outlive the fix — recommend dropping both from this PR, or landing them separately with a real consumer.

Blockers

1. Re-running the sweep destroys a good calibration (inline). The sweep taps panFeedSpectrumReady, which carries bins after applyDroopCorrectionDb() and applyEdgeFade() — and nothing zeroes or bypasses the live tables for the sweep's duration, while connectRadio() re-seeds them every session. Second sweep → flat measured curve → near-zero tables → Apply replaces and persists zeros → the droop returns, silently. Worse at the edges: the synthetic 12 dB fade is baked into the new table as if it were hardware. "The same action twice" is the first thing an operator will do after wondering if the first sweep took. Fix: push kDroopCorrectionZero for each rate at sweep start (restore on abort), or tap a pre-correction feed.

2. The persist path violates the feature-document rules three ways (inline). A partial Apply builds the doc from only this sweep's rates and setFeature() replaces the whole document — wiping previously calibrated rates from disk while the DSP keeps them live, so the radio is correct until the next connect and then regresses with no indication (the header explicitly advertises partial sweeps as safe). The write result is discarded — the exact #4621 "worst failure shape" AGENTS.md names, while the UI reports "live and saved" (the PR's own cited precedent, Hl2FreqCal, checks and warns). And the handler never reads the existing row via featureExact() nor refuses a newer schema_version, both required by the same section. And a fourth facet: applyResult() reports success unconditionally — a radio dropped between sweep and Apply makes invokeBackendExtension a documented no-op, yet the dialog prints "Applied — the measured correction is now live and saved" and the bridge returns ok:true; the no-serial branch likewise warns and then reports success. Merge into the existing document, check the write, honor the schema guard, and thread the actual outcome back to the UI/bridge.

3. The Sampling phase has no timeout (inline). Only WaitingForRateLanded is bounded; Sampling breaks out every poll tick while m_haveLatestFrame is false, forever. A stalled feed (hidden/paused panadapter, a frame size ≠ 1024, a quiet network drop) leaves isRunning() latched — so start() and Apply are permanently no-ops — with the radio parked at the sweep's rate instead of m_originalRateKsps. Bound it like the rate-wait, and restore the original rate on the failure path.

4. Correction tables leak across radios (inline). m_dsp is constructed once and survives disconnect/reconnect; the connect-time seed only inserts, and no clear/erase path exists. Connect calibrated G2 #1, disconnect, connect G2 #2 in the same session: #2's panadapter renders with #1's per-bin corrections (plus the edge fade applied on top), with no UI indication — the Droop tab shows measuredTables(), which is empty. Clear the DSP tables on disconnect (or key the seed by serial and clear on mismatch).

5. The settings-search filter bypasses the capability gate (RadioSetupDialog.cpp:888; added after the cross-file trace completed). The per-keystroke search filter re-hides rows itself and exempts the capability-gated rows by name — the new Droop row is not in that exemption list (the write-only m_droopCalibrationPageIndex was the tell). Typing anything (or typing and clearing) un-hides "Droop Correction" on a Flex/HL2/Icom, and Start Sweep is family-agnostic: it drives setPanBandwidth through six zoom levels on a radio the correction can never apply to. Add the droop row to the gate expression.

6. "Rate landed" is inferred from a signal that lies on the failure path (AnanBackend.cpp:829 vs AnanDroopCalibrator.cpp:228). finishRateChange(ok=false)'s own comment says the old channel keeps running at the OLD rate, yet it emits pan state carrying the requested rate — and the calibrator's only landed test is that bandwidth matches the target. A failed 768 k channel build means eight frames of 384 k data are measured into the 768 k table and persisted: one rate's droop curve applied to another rate's spectrum — the exact cross-rate corruption the handedness test's Group 6 exists to prevent, reintroduced through the measurement side. Land detection needs the DSP's actual configured rate, not the requested pan state.

Nits (non-blocking, condensed)

• Layering (EB3, rule text quotable): RadioModel.h now includes a vendor header and holds AnanDroopCalibrator by value — every GUI TU compiles ANAN headers, every non-ANAN session constructs the timers/buffers, and the second family with droop (HL2's chain is architecturally similar, as the caps-map row itself says) can't reuse the seam. A forward-declared unique_ptr constructed when the capability is true keeps the shared-observer property; the backend already speaks invokeExtension for the apply half.
• Captures aren't gated on new frames — the 8-sample median can be eight copies of one frame (spectrum FPS ≤ 3) or contain previous-rate frames still in flight, defeating the documented robustness; clear m_haveLatestFrame on capture and stamp frames per rate.
• droopcal start returns ok:true, running:false when start() refused (no panadapter) — the refusal is invisible to automation; the #5263 loud-drop shape.
• The edge fade rewrites emitted dBm bins, not pixels — every data consumer (markers, the new radiocert spectrum capture that this PR documents as calibration-grade, the calibrator itself) receives fabricated values; and the renderer taper now disagrees between GPU (0.09) and software (0.05) paths on the same radio, with the software comment pointing at a comment that calls its value superseded. Pick one owner for the cosmetic fade.
• radiocert spectrum also inherits run()'s epilogue (unkey + re-tune slice 0) though it is capture-only, and defaults frequencyMhz to 14.200 and silently retunes slice 0 there when the argument is omitted — the adjacent phase parse in the same PR deliberately fails closed on omitted arguments; this stage should too (the comment above it even warns about exactly this).
• The DDC0 rate set is now spelled in five places (backend caps, nearestDdc0RateKsps, calibrator, DSP validity loop, ConnectionPanel); one shared constant prevents the silently-rejected-table failure when a rate is added.
• Table marshalling round-trips four representations with two hand-copied validators (loadTables vs the apply handler) — a symmetric saveTables() beside loadTables() (the Hl2FreqCal shape) leaves one codec. The median helper is the tree's seventh copy and the only sort-based one; the per-frame cos window and QMap lookup in the DSP hot path cache trivially; m_droopCalibrationPageIndex is write-only; the new settings-page widgets lack accessible names (a11y rules for src/gui/); the dB→linear→dB median round trip is a documented no-op that exists only to keep kLog10Floor — and that floor caused the bug the regression test pins.

What was verified vs read

• Verified by me: both sweep-lifecycle blockers (tap point, absent bypass, setFeature whole-doc replace, unchecked write result), the taper's capability gating — which refuted two angles' claim that the 0.09 change affects all backends (m_edgeTaperEnabled requires hasDdcPanEdgeRolloff, ANAN-only) — and the stale-stacking evidence (#5143 at main's tip).
• From the automated pass (all 8 angles; the cross-file tracer reported after initial posting and its two blocker-tier findings were added to this review as items 5 and 6), verified before adoption; the Sampling-timeout blocker was confirmed from the quoted phase logic.
• Not run: no hardware (the author's live G2 sweep covers the happy path; every blocker is a lifecycle argument reproducible from the code), no bridge session, and the three new tests were read, not executed — CI ran them green.

[ten9876]

Blocker 1 — this tap is downstream of the correction it is trying to measure. panFeedSpectrumReady carries bins after applyDroopCorrectionDb() and applyEdgeFade(), nothing zeroes or bypasses the live tables for the sweep, and connectRadio() re-seeds them every session — so the second sweep an operator ever runs measures a flat curve, computes ~zero tables, and Apply persists them: the good calibration is silently destroyed and the droop returns (with the synthetic 12 dB edge fade baked into the new table as if it were hardware). Push kDroopCorrectionZero per rate at sweep start and restore on abort, or tap a pre-correction feed.

[ten9876]

Blocker 2 — the persist path fails the feature-document rules four ways. (1) doc holds only this sweep's rates and setFeature() replaces the whole document — a partial Apply (which stop()'s own comment blesses as "a safe, real partial improvement") wipes the other rates from disk while the live DSP keeps them: correct until restart, then silent regression. (2) The write result is discarded — AGENTS.md's #4621 rule verbatim ("a mutation that silently doesn't persist while the UI repaints from the store is the worst failure shape… Log loudly at minimum"), and the precedent this PR cites, Hl2FreqCal, checks and warns. (3) No featureExact() read, no newer-schema_version refusal — both required by the same section. (4) applyResult() reports success unconditionally: a radio dropped before Apply makes the extension a documented no-op while the dialog prints "live and saved" and the bridge returns ok:true.

[ten9876]

Blocker 3 — Sampling has no timeout. Only WaitingForRateLanded is bounded; this branch breaks out every 200 ms tick forever if the feed stalls (hidden/paused panadapter, frame size ≠ 1024, quiet network drop). isRunning() stays latched — start() and Apply become permanent no-ops — and the radio is left parked at the sweep rate instead of m_originalRateKsps. Bound it like the rate-wait and restore the original rate on the failure path. Related, same loop: captures aren't gated on a new frame (m_haveLatestFrame is never cleared on capture), so the 8-sample median can be eight copies of one frame at low spectrum FPS, or contain the previous rate's frames.

[ten9876]

Blocker 4 — tables outlive the radio they were measured on. m_dsp is constructed once and survives disconnect/reconnect; this seed only inserts, and no clear/erase path exists. Calibrated G2 #1 → disconnect → G2 #2 in the same session: #2 renders with #1's per-bin corrections plus the edge fade on top, and the Droop tab shows nothing (it displays measuredTables(), which is empty). Clear the DSP's tables on disconnect, or key the seed by serial and clear on mismatch.

[tropo1234]

Thanks — this was a genuinely useful review. All six blockers are fixed, the scope objection is actioned, and the stale stacking note is gone. Four commits on top of the original eight. One item is still outstanding — a G2 re-run, detailed at the end of Tests.

Blockers

1 — the sweep measured its own output. Fixed, but not the way the review suggested, and the difference matters. Pushing kDroopCorrectionZero for each rate would have left the bug half-alive: setDroopCorrectionTable() stores a copy, so droopTableForRate() would stop returning the kDroopCorrectionZero object, and processIqBlock()'s &droopTable != &kDroopCorrectionZero is an identity test — the synthetic 12 dB edge fade would have stayed on and been measured as hardware droop, which is precisely the failure the blocker describes. Instead there's now AnanRxDsp::setDroopCorrectionBypassed(), routed through a droop.bypass extension verb (the calibrator has no handle on m_dsp, which lives on another thread). Armed in start() before the tap is connected, lifted in finishSweep() — so completion, stop() and abortSweep() all restore it. It is also non-destructive: the tables are hidden, never overwritten, so an abort or a crash mid-sweep cannot lose a calibration.

2 — the persist path. All four facets. The write half moved into AnanDroopCalibrator::saveTables(), symmetric with loadTables() — which also answers the "two hand-copied validators" nit, since the float↔JSON codec and its validity rules now live in one place. It reads the row with featureExact(), refuses a stored schema_version newer than this build understands, merges rather than replacing, checks the setFeature() result, and returns the reason when it fails instead of returning void. AnanBackend::persistDroopTables() keeps only the empty-radio_id guard, which is the one judgement saveTables() can't make for itself.

The fourth facet — reporting success unconditionally — is fixed end to end. applyResult() now correlates a real requestId against the backend's extensionResult/extensionError rather than firing and forgetting. That reply is synchronous (the handler completes locally, no device round trip), so connecting before the invoke and reading after it is deterministic rather than a race. A radio dropped between the sweep and Apply now reports the failure instead of printing "live and saved". The dialog also keeps the specific reason instead of overwriting it with generic finished() text — an aborted sweep had that same bug — and droopcal apply returns the failure to automation.

3 — no Sampling timeout. Bounded like the rate wait. Also fixed the related capture gating you flagged: m_haveLatestFrame is cleared on every capture, so the 8-sample median can't be eight copies of one frame at a low spectrum FPS, and anything in hand is dropped when Sampling begins, since previous-rate frames are still in flight then.

4 — tables outliving the radio. AnanRxDsp::clearDroopCorrectionTables(), called on disconnect and again before each connect's seed. The bypass flag is cleared on disconnect too — a radio that vanishes mid-sweep can't leave correction suppressed for whatever connects next.

5 — the search filter bypassed the capability gate. The Droop row is now in the gated expression alongside apdRow and calRow. This also gives m_droopCalibrationPageIndex its first read, so that nit goes with it.

6 — "rate landed" inferred from a signal that lies. Fixed at the source rather than in the calibrator. beginRateChange() captures the live rate before overwriting the pending fields, and finishRateChange(ok=false) restores it before emitPanState(). Zoom math and everything else reading pan bandwidth were wrong on that path too; they just failed less visibly.

Scope

Dropped, as recommended. RadioCertification::Phase::Spectrum, stageSpectrum() and the pan bandwidth verb are gone — RadioCertification.cpp/.h are now byte-identical to main and drop out of this PR entirely. tools/anan_droop_calibration.py was an offline prototype that was never in this tree and that the in-app engine superseded before either landed, so all six comments pointing at it are gone or reworded — including one that justified AnanDroopCalibrator's design by pointing at stageSpectrum(). tools/gen_bridge_docs.py re-run; the verb count is unchanged at 72, since droopcal is the one net-new verb and it has a real consumer.

You were right about the stacking note — the branch had already been rebased onto #5143. The body is updated.

PR is now 26 files / +1712 −29, down from 28 / +1799 −32.

Tests

anan_rxdsp_handedness_test grows Group 7: a bypassed table and a cleared table each emit bins equal to the raw FFT across the whole frame, tails included. The tails are the point — they're what would catch the zero-table trap, since the edge fade survives that. Plus a negative control (armed really does differ from raw) and a check that lifting the bypass restores the identical frame.

anan_droop_calibrator_test grows four saveTables cases against a real settings store in a temporary home: a partial save merges, a re-measured rate overwrites only itself, a newer schema is refused without writing anything, and empty/invalid inputs report why.

Full ctest: 2 failures out of 334, both verified pre-existing by checking out 202d876e and rebuilding — hl2_state_restore_test (3 CW-passband checks) and phone_tx_filter_numeric_entry_test (aborts inside Qt's own testlib/qasciikey.cpp). Neither is reachable from this branch.

Hardware: re-run pending, and I'd rather flag it than let it pass. The original sweep was bench-confirmed on a real G2, but that was before these fixes, and two of them change what the sweep actually measures — blocker 1 bypasses the correction for the sweep's duration, and blocker 6 changes when a rate counts as landed. The unit tests pin the mechanisms (Group 7 proves a bypassed table returns the whole frame to the raw FFT), but they cannot prove the bypass arrives before the first capture, since it crosses to the DSP thread on a queued connection. Only the radio can show that.

Three things I'm running on the bench, with the store backed up first:

1. A second sweep must reproduce the first. Sweep, Apply, then sweep again without disconnecting, and compare the per-rate min–max corrections. Before the fix the second sweep collapsed to near-zero — that is the regression this PR exists to kill, and it is the one measurement that can prove the bypass lands in time.
2. A partial Apply must merge. Sweep two or three rates, Stop, Apply, restart, then read the persisted document back and confirm all six rates survive with the untouched ones byte-identical.
3. A dropped radio must report failure, not "live and saved" — disconnect between the sweep and Apply.

I'll post the numbers as a follow-up. Happy to add to that list if there's something else you'd want to see from the bench.

The Test plan checkbox for this is deliberately left unchecked until then.

Nits — taken

The saveTables() codec, the write-only page index, droopcal start reporting its refusal, and the new-frame capture gating are all in, as described above. Two more in a fourth commit:

• The DDC0 rate set is now anan::kDdc0RatesKsps in P2Protocol.h, read by all five sites you listed — plus a sixth the review didn't catch: the hardcoded 48'000.0 / 1'536'000.0 in the pan-bandwidth-limits emit, whose own comment already claimed it matched sampleRatesHz's endpoints.
• Accessible names on the Droop tab, though deliberately not uniformly. startStopBtn toggles its text between "Start Sweep" and "Stop", and statusLbl carries the live status including the failure reasons this PR just made honest — a fixed accessibleName on either would replace the text a screen reader should be announcing. Those two get descriptions; the unlabelled progress bar and the ambiguous Apply/Discard buttons get names.

Nits — not taken, with reasons

• RadioModel layering. The one I'd most like your steer on. Making the calibrator a lazily-constructed unique_ptr changes droopCalibrator() from a reference to a pointer, and RadioSetupDialog builds the Droop page once per process, capturing &cal into a dozen lambdas at build time — those captures become invalid, so every one has to re-look-up instead. That's a real restructure of core radio UX, and the maintainers have asked for shared radio-model seams to be broken out for conflict review. I'd rather do it as its own PR; say the word if you'd prefer it here.
• The edge fade's owner (dBm bins vs pixels, GPU 0.09 vs software 0.05). A design decision, not a cleanup — it changes what every data consumer sees and wants bench confirmation.
• Deduplicating the median helper. Six call sites outside ANAN, for zero behaviour change — a conflict magnet against a queue of concurrent PRs.
• Caching the DSP hot path. No measurement says it matters, and a caching bug in the audio/spectrum path is the hardest kind to find.
• The dB→linear→dB round trip. Left on purpose: it's a documented no-op that keeps kLog10Floor in play, and that floor is what caused the bug the regression test pins.

Ready for another look.