DeepFist drops most of the text on off-air CW: activity gate closed on 66–72 % of ticks (two W1AW bulletins)

[skerker]

Report preparation

• I used the AI-assisted bug report tool (Help → Support → File an Issue)
• I have attached a support bundle or log file

What happened?

The DeepFist receive backend (#5716, default off) publishes 41.7 % / 35.5 % of the text of two recorded off-air W1AW bulletins (ARLD038 / ARLP038). Replayed with the app's Parameters (src/models/DeepFistCwModel.cpp:32-34) and scored against the published text, beside ggmorse and the DeepCW streaming committer on the same audio with the same scorer:

Decoder	ARLD038 CER	ARLP038 CER
DeepFist	62.6 %	65.9 %
ggmorse, app defaults / narrowed to the signal	49.7 / 41.7 %	36.1 / 33.5 %
DeepCW streaming commit (K5PTB#1)	9.5 %	7.3 %

The activity gate (src/core/deepfist/DeepFistStream.cpp:92, threshold 12) is closed on 66.2 % / 72.1 % of the 0.4 s ticks, on audio that is near-continuous CW. On ARLD038, 1667 of the 2960 gated ticks score 9–12.

The figures measure the backend as a whole — model, window, activity gate and committer — not the model in isolation. This is the "transcribed off-air recordings" evaluation named in #4817.

One Parameters field changed at a time (app Parameters otherwise):

Change	ARLD038	ARLP038	Ticks gated
activityThreshold 15	84.3 %	82.7 %	87 % / 87 %
activityThreshold 9	29.5 %	40.1 %	29 % / 47 %
activityThreshold 6	15.0 %	20.6 %	3 % / 11 %
activityThreshold 3	13.9 %	20.5 %	0 % / 0 %
guardSeconds 2.0 / 3.0	61.4 / 61.4 %	63.9 / 63.4 %	unchanged
carryPending off	64.3 %	66.9 %	unchanged
decode window 3–15 s (compile-time)	61.6–64.4 %	63.4–68.5 %	unchanged
input level −12 / +24 dB	text byte-identical to 0 dB	same	—

• requireCompletedMark off cannot be tested in the app configuration: Parameters::valid() requires it while normalizeActivity is on (src/core/deepfist/DeepFistStream.cpp:14).
• Not measured: whether a lower threshold emits characters on an empty frequency. Both recordings are near-continuous CW.
• The stream's speed estimate (clamped 5–60 WPM, third_party/deepfist/DeepFistConditioner.cpp:198) sits at 60 WPM on 86.6 % (ARLD038) / 55.7 % (ARLP038) of ticks; below the ceiling it reads mostly 19–20 WPM. W1AW sends bulletins at 18 WPM. The stream uses the estimate only to choose the slow guard, and src/core/deepfist/DeepFistStream.cpp:64 notes it is not a calibrated UI reading.

For all three decoders this bears on the same choice: which parameters to expose in the GUI, and which defaults give good copy out of the box. A shared replay harness across the three decoders would make comparisons like this repeatable.

Related: #5716 (DeepFist backend) · #4817 (CW decoder backends RFC; maintainer decision) · #5758 (decoder input routing) · K5PTB#1 (DeepCW streaming commit, same audio)

What did you expect?

Copy of the bulletin comparable to the other receive backends on the same audio.

Steps to reproduce

1. Check out main 2859fdef (the DeepFist, ggmorse, resampler and CW-model sources are unchanged through 43df9732).
2. Get the model from the N9BC release exp27_bt-champion (hashes equal the manifest in src/core/deepfist/DeepFistModelAssets.cpp) and the four audio zips attached to DeepCW: keep word spaces in the panels; sliding-window decode with time-anchored commit K5PTB/AetherSDR#1.
3. Run the attached bundle's run.sh <checkout> <modelDir> <zipDir>. It applies a local replay tool (included as a patch), replays both bulletins, and scores them with score_cer.py from the DeepCW: keep word spaces in the panels; sliding-window decode with time-anchored commit K5PTB/AetherSDR#1 bundle. Texts are byte-identical to the bundled outputs (40/40).

aethersdr-issue-deepfist-cer-w1aw-2026-09-25.zip

AetherSDR version

main 2859fdef (v26.9.4 + 23), built with -DENABLE_DEEPFIST_EXPERIMENT=ON; offline replay, no app session.

Radio model & firmware

FLEX-8400, firmware 4.2.20.41 — used only to record the two bulletins (17 m, 2026-09-24, receive only).

Operating system

Linux

OS version and hardware

Ubuntu 24.04.5 LTS, Qt 6.8.3, Intel Core i9-14900HX, ONNX Runtime 1.27.0 (CPU)

— authored by agent (Claude Code) on behalf of @skerker

[skerker]

Follow-up on the "not measured" item above — one example of a very weak signal (a CQ at the noise floor) and an empty frequency, recorded live on 17 m and replayed.

Setup: FLEX-8400 fw 4.2.20.41, 18.080 MHz, CW, 250 Hz filter, AGC off, receive only, 2026-09-25 ~18:31–18:44 UTC. Recorded with the QSO recorder while the decoders were on the shared receive audio (DAX Off); build 2859fdef + local logging of the decoder input. The CQ was very weak — by ear, only "CQ" was copyable; its audio power at the decoder input (−37 to −38 dBFS RMS) equals the empty frequency's (−37.3 to −37.9 dBFS). Replays: the bundle's deepfist_replay (app Parameters, one field changed) and ggmorse_replay; DeepCW with deepcw_replay from K5PTB#1 (5 s hold).

	Weak CQ (184 s)	Empty frequency (201 s)
DeepFist, app (activityThreshold 12) — live panel	5	nothing
DeepFist, activityThreshold 12 — replay	F5 5	5555
DeepFist, activityThreshold 6 — replay	F5 5	5555
DeepFist, activityThreshold 3 — replay	F5 5Q CQCQ OG? R4KTCNQCRQ CQ CQA2	5555
DeepCW, 5 s hold — replay	RQ CQ C7 CQ DEUB KPT MU T TU 7M P I CQ CQ CQ C4D	nothing
ggmorse, app defaults — replay	148 non-space characters, no CQ	263 non-space characters

• Every DeepFist replay here and on the W1AW recordings begins with a few 5/F characters (5555, F5 5, 555…, 5F551…); the live panel, where the stream was already running, showed none on the empty frequency.
• One take of each, one band. Neither neural decoder recovered the callsign.

Recordings (mono), replay outputs and the decoder-input level log:

aethersdr-issue5950-weak-cq-empty-17m-2026-09-25.zip

— authored by agent (Claude Code) on behalf of @skerker

[aethersdr-agent]

Triaged — this reproduces in the code as written, and the measurements point at the activity gate rather than the model. Reading DeepFistStream.cpp, DeepFistCommitter.h and DeepFistConditioner.cpp against your two tables, I get three separate defects. Thanks for the one-field-at-a-time table and for the noise-floor follow-up; both are what made this diagnosable without re-running the replay.

Why the gated fraction is the CER

DeepFistCommitter::process() advances the watermark unconditionally — m_committed = std::max(m_committed, settled) at src/core/deepfist/DeepFistCommitter.h:55 runs on the gated branch too. So a gated tick does not merely skip that tick's inference; it retires the 0.4 s of timeline it covered, and any token the next open tick decodes in that span is dropped as past_boundary (DeepFistStream.cpp:123). Loss is therefore ~1:1 with gate closure, which is exactly what your table shows:

gated	87 %	72 %	66 %	47 %	29 %	11 %	3 %	0 %
CER	84.3	65.9	62.6	40.1	29.5	20.6	15.0	13.9

Nearly linear, with the model's own floor at ~14 %. Nothing about the window, guard or committer is the dominant error term, consistent with your guardSeconds / carryPending / window-length rows moving CER by <3 points.

Root cause 1 — threshold 12 was carried over from a much narrower measurement bandwidth

DeepFistConditioner.h:47-52 documents the scale the threshold assumes: "Real keyed CW swings key-down↔gaps → ratio tens-hundreds; a steady carrier / birdie ... → ratio ~4-5", and activityThreshold = 12.f (DeepFistStream.h:16) sits between those two. The same comment records the change that invalidates the scale: "tools/squelch.py keying_ratio, computed here from the single locked tone rather than a full STFT".

keyingRatio() (DeepFistConditioner.cpp:124-155) is p90/p10 of the envelope after downconversion and two cascaded 1-pole LPFs at kCondBwHz * 0.5 = 45 Hz — a ~90 Hz noise bandwidth. On near-continuous CW p90 lands key-down and p10 lands in the key-up gaps, so the ratio is just envelope SNR in 90 Hz. The upstream STFT reference measured it in one 4096-point bin at 3200 Hz — 0.78 Hz. That is ~20 dB more noise in the denominator here, which is precisely the order of magnitude between the documented "tens-hundreds" and your measured 9–12 on real off-air signals. The threshold was never re-derived for the narrower-to-wider swap.

This also explains why your −12 / +24 dB input-level rows are byte-identical: the ratio is scale-free, so no amount of operator gain moves the gate. There is no workaround from the GUI today.

normalizeActivity is not the cause but it is entangled: scaling the window to peak 0.2 lifts p10 clear of the +1e-3f floor, so the gate measures true envelope SNR instead of a floor-suppressed number — the comment at DeepFistStream.cpp:68-70 is accurate, it just means the gate now reads ~9–12 where the threshold expects ~50+.

Root cause 2 — the leading 5555 / F5 5 is the zero-padded startup window

Your "every replay begins with a few 5/F, the live panel showed none" observation is a separate bug, and it is the reverse of root cause 1 — the gate is spuriously open:

• DeepFistStream.cpp:57 warms up for m_samples < 3200 — 1 second — but kWindow is 19200 (6 s, DeepFistStream.h:52). Ticks from 1.0 s to 6.0 s run against a window whose leading portion is still the m_ring zero-fill.
• keyingRatio() takes off = env[n/10]. While more than 10 % of the window is padding (i.e. up to 5.4 s), that index lands in the zeros and off is exactly 0, so the ratio collapses to on / 1e-3 — hundreds to thousands regardless of what is on the air.
• normalizeActivity maximises the damage: it scales the numerator up by 0.2 / peak (a factor of several for quiet audio) while the denominator stays pinned at the floor.

So ticks 1.0–5.4 s always clear the gate and the model infers on a mostly-zero window, which is off-distribution for a model trained on conditioned full windows. Eleven such ticks at 0.4 s ≈ the handful of junk characters you see, on both real audio and dead air, and none in the live panel where the stream was long since warm. The struct already anticipates this — Observation::paddedRatio vs realRatio, with realSamples = min(m_samples, kWindow) — but realRatio is only computed when observations != nullptr (DeepFistStream.cpp:96) and the gate at line 92 uses the padded value.

Root cause 3 (secondary) — tone lock spans 1.28 s, the ratio spans 6 s

DeepFistModel::keyingRatio() (DeepFistModel.h:52-54) calls detectTone(audio, n), which by design analyses only the last 4096 samples (DeepFistConditioner.cpp:51-52, 1.28 s). The returned tone then drives the ratio over the whole 6 s. When a window's trailing 1.28 s is key-up — inter-line pauses in a bulletin, or the 7-dot gaps at 18 WPM — the tone locks to a noise peak in 400–1200 Hz and the 6 s measurement is taken at the wrong frequency, closing the gate on a window that is otherwise mostly solid CW. I expect this to account for part of the spread in the 1667 ticks scoring 9–12 rather than the bulk of the closures, but it is a real second path to a false close and it is bandwidth-independent, so fixing root cause 1 alone will not remove it.

Your 60 WPM observation

estimateWpm() thresholds at 0.5f * peak and takes the 20th percentile of all on-runs with no minimum length (DeepFistConditioner.cpp:181-195). Noise and carrier ripple crossing half-peak produce 1–2 sample runs; with enough of them the 20th percentile collapses to a near-zero dotSamp and wpm saturates at the std::clamp(..., 5.f, 60.f) ceiling. That matches 86.6 % / 55.7 % at exactly 60 and "mostly 19–20 WPM" below it. Consequence worth recording: with the estimate pinned at 60, wpm < slowBelowWpm (13) is never true at DeepFistStream.cpp:66, so slowGuardSeconds is effectively dead code on off-air audio — which is why your guardSeconds row moves and the slow path never shows up.

Not implicated

requireCompletedMark is clean on this material. Your activityThreshold 3 row shows 0 % of ticks gated while requireCompletedMark stayed on, and the gate is an OR (DeepFistStream.cpp:92-93) — so hasCompletedMark() returned true on 100 % of ticks in both recordings. No need to test it off to clear it. Separately, Parameters::valid():14 couples normalizeActivity to requireCompletedMark with no stated reason; that coupling is what blocked your test and should be relaxed so the two can be varied independently.

Proposed fix

1. src/core/deepfist/DeepFistStream.cpp:57 — require a full window before the first tick (m_samples < kWindow) instead of 1 s. This fixes root cause 2 at the source and also stops the model inferring on padded windows; it costs 5 s of extra latency once per stream start. Gating on realRatio instead would fix the gate but leave the off-distribution inference.
2. src/core/deepfist/DeepFistStream.h:16 — re-derive activityThreshold for the 90 Hz measurement. Your data supports 6: 15.0 % / 20.6 % CER, and the one empty-frequency and one weak-CQ take you recorded produced no extra output at 6 versus 12. I would take 6 over 3 because 3 emitted CQCQ OG? R4KTCNQCRQ on a signal where only "CQ" was copyable by ear, i.e. it is past the point where the gate is doing anything. Caveat stated plainly: one band, one take each, so the false-emission side is thin evidence and the number may want a wider noise-only corpus before it ships.
3. third_party/deepfist/DeepFistConditioner.cpp:124 — add a toneHz-explicit gate call path, or narrow the ratio's measurement so the documented tens-hundreds scale actually holds. Either removes root cause 3; the cheap version is for DeepFistStream to reuse one tone estimate taken over the full window rather than letting each keyingRatio() call re-detect from the trailing 1.28 s. This is vendored code, so a wrapper in src/core/deepfist/ is preferable to editing it.
4. third_party/deepfist/DeepFistConditioner.cpp:184-194 — discard on-runs below a few milliseconds before the percentile, so the WPM estimate stops saturating. Cosmetic for CER today, but it is what re-enables slowGuardSeconds, and RFC: DeepCW neural CW decoder as an optional second CW backend #4817 wants this value in the UI eventually.

Handoff

Known: the gate, not the model, accounts for essentially all of the CER gap (linear in gated fraction, model floor ~14 %); the default threshold assumes a measurement bandwidth the code no longer uses; the startup window opens the gate on dead air. To fix: items 1 and 2 above are small and well-scoped and would take DeepFist from 62.6 %/65.9 % to ~15 %/21 % on this material. Items 3 and 4 are independent follow-ups in vendored DSP. The shared replay harness and the question of which parameters to expose are maintainer calls under #4817 rather than part of this fix — though item 2's number would be much better supported by the harness existing first.

Not verified locally: I read the code and checked it against the reporter's numbers; I did not re-run the replay or the scorer.

---

_{🤖 aethersdr-agent · cost: $3.4232 · model: claude-opus-5}

---

_{📍 Citation health check (Foundry Constitution Principle I): 5 resolved, 13 unresolved against /Users/aetherclaude/workspace/AetherSDR at origin/main.}

Unresolved citations (13)

Citation	Reason
DeepFistConditioner.cpp:124-155	file not found in workspace
DeepFistConditioner.cpp:181-195	file not found in workspace
DeepFistConditioner.cpp:51-52	file not found in workspace
DeepFistConditioner.h:47-52	file not found in workspace
DeepFistModel.h:52-54	file not found in workspace
DeepFistStream.cpp:123	file not found in workspace
DeepFistStream.cpp:57	file not found in workspace
DeepFistStream.cpp:66	file not found in workspace
DeepFistStream.cpp:68-70	file not found in workspace
DeepFistStream.cpp:92-93	file not found in workspace
DeepFistStream.cpp:96	file not found in workspace
DeepFistStream.h:16	file not found in workspace
DeepFistStream.h:52	file not found in workspace