Skip to content

bound the storage queue writer channel #159

Description

@fabracht

The storage queue writer channel is unbounded, so the write-behind path has no memory ceiling during a burst.

  • crates/mqtt5/src/broker/storage/client_queue.rs:21pub type QueueWriter = mpsc::UnboundedSender<QueueOp>;
  • crates/mqtt5/src/broker/storage/file_backend.rs:224mpsc::unbounded_channel()

Every queue push enqueues a QueueOp::Write holding an Arc<QueuedMessage>. The per-client queue itself is bounded (enforce_limits_dir sheds on both count and bytes), but the ops in flight to the writer are not: if the writer task cannot drain as fast as messages arrive, the channel grows until it does. The effective ceiling is "however fast the disk keeps up" rather than anything configured.

Suggested change

Make the channel bounded with an explicit overflow policy — backpressure on the producer, or coalesce/drop with a counter — so the write-behind path has a predictable ceiling that relates to the configured queue limits.

Not a correctness defect: under the floods measured so far the writer kept up well enough that memory plateaued rather than growing without bound. This is about making the bound explicit instead of incidental.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions