You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ingestor: watchdog force-reconnect logs a misleading Connect() error while paho is in its initial ConnectRetry loop #102
To the watchdog, connecting looks the same as reconnecting (IsConnected=true, IsConnectionOpen=false). In reconnecting, Connect() is a genuine no-op.
The line is rate-limited by the force-reconnect throttle to about 1 per minute per source. But it appears exactly during the outages when operators read the logs, and it reads like a real failure.
The code comment in cmd/ingestor/main.go (around line 600) says Connect() alone is "a safe no-op per paho when a retry is already under way". That is true for reconnecting but not for connecting.
Suggested fix
Recognise the "status can only transition to connecting from disconnected" error (or check paho's connection status before calling Connect()), and log it at debug level or as an informational "retry already in progress" line. Genuine Connect() failures should still be logged as errors.
Correct the code comment.
Add a test with a real paho client in connecting state. The in-test broker from cmd/ingestor/mqtt_force_reconnect_paho_test.go can be reused. The test should assert that no error-level line is emitted, while a genuine failure still is.
Found in the independent review of #29 (SHOULD-FIX, non-blocking).
While paho is in its initial ConnectRetry loop, the watchdog's force-reconnect now logs an info line that keeps paho's error, instead of a misleading Connect() failed.
Genuine failures are still logged as failures.
The comments explain that IsConnected() in disconnecting is a strong hint, not a guarantee, because paho's willReconnect flag is sticky.
Activity
reviewfix(ingestor): keep paho's error in the retry-pending line; pin…
Merge pull request #210 from dborup/codex/issue-102-103-mqtt-watchdog
dborup commentedon Oct 4, 2026
Fixed by #210 (merged as
a0086bdd).Connect() failed.IsConnected()indisconnectingis a strong hint, not a guarantee, because paho'swillReconnectflag is sticky.