Skip to content

txgovernor: a failed send leaves the dedup key recorded, silently suppressing retries for 30s #581

Description

@kotfu

Found while investigating #564. This is a separate defect, not the cause of that issue.

Summary

The TX governor records a frame's dedup key at enqueue time, but the frame can still fail to transmit afterwards — either the AX.25 encode fails or the downstream Sender returns an error. In both cases the dedup entry stays recorded, so an identical retry within the 30-second window is silently suppressed even though nothing ever reached the radio.

Where

  • pkg/txgovernor/governor.go:327 — g.dedup.Record(key, struct{}{}) in Submit.
  • pkg/txgovernor/governor.go:469-473 — top.frame.Encode() fails, the frame is dropped, and processOne returns.
  • pkg/txgovernor/governor.go:489-490 — g.cfg.Sender(tf) returns an error, the frame is lost, and processOne returns.

Notably, this exact hazard was already identified and handled for the queue-full case. Submit does the capacity check before recording, with a comment explaining why:

Capacity check before recording dedup: if we reject the frame, we must not poison the dedup map, or the caller's retry within the window would be silently suppressed with zero visibility. This is why the governor uses Has+Record rather than the atomic Seen.

The same reasoning applies to the encode-failure and send-failure paths, which were missed.

Reproduction

With a Sender that always returns an error, submit a frame, wait for the send attempt to fail, then submit the identical frame again:

sender attempts=1 stats: enqueued=1 sent=1 deduped=1

The retry never reaches the sender. It was suppressed as a duplicate of a frame that was never transmitted.

Impact

Worst during exactly the conditions where retrying matters most: a modem hiccup, a bridge restart, or a transient IPC failure. The operator retries, sees nothing happen, and gets no feedback — the suppression is logged at Debug and the default level is Info, so there is no log line either. They have to wait out the full 30-second window before a retry can get through.

Possible direction

Either move the Record to the successful-send path in processOne, or have the failure paths remove the key they poisoned. Moving the record is cleaner but changes the semantics slightly: identical frames submitted while one is still queued would no longer dedup against each other, which may or may not be wanted.

Related, both found in the same processOne read during the #564 investigation: #580, #582, #583.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions