Skip to content

feat(engine): preallocated command queue; zero-allocation send through a ready gate (#76) - #197

Merged
joaquinbejar merged 1 commit into
mainfrom
issue-76-command-queue
Oct 20, 2026
Merged

joaquinbejar merged 1 commit into
mainfrom
issue-76-command-queue

Conversation

@joaquinbejar

Copy link
Copy Markdown
Owner

Summary

Closes #76. Its acceptance criterion, zero heap allocations per steady-state send with no store, was not met: even a pooled message sent through a gate that decides at once cost about one block per 128 sends, because the tokio::sync::mpsc command channel grows its slot list. The channel is replaced by a ring allocated once, so that path now allocates nothing.

Changes

  • New private ironfix-engine/src/command_queue.rs: CommandSender / CommandReceiver over a VecDeque allocated with the session's outbound capacity at session start and never grown.
    • The ring sits behind a std::sync::Mutex held only to push or pop one command, never across an .await, as the governance list allows for short, synchronous sections.
    • tokio::sync::Notify wakes the other side; its waiters live in the waiting futures, so waiting allocates nothing.
  • It keeps the channel's semantics, each with a unit test:
    • FIFO order;
    • a full queue makes the send wait;
    • recv returns None once every handle is dropped and the ring is empty, and the last drop wakes a waiting receiver;
    • close refuses sends, waiting ones included, and still drains;
    • a dropped receiver refuses sends;
    • a cancelled send queues nothing;
    • the send future is Send.
  • Connection and the reactor use it. Nothing else changes: same capacity knob (with_outbound_capacity), same backpressure, same close behaviour.
  • The allocation test's ready-gate window now asserts at most 4 non-recurring blocks instead of the channel's per-send rate. README and ADR-0008 updated.

Measured

From ironfix-engine/tests/allocations.rs with dhat on macOS, block counts, not time:

Window Before After
pooled sends through a ready gate, no store 8 for 1024 0 for 1024
pooled sends, default to_app_checked 1032 1024 (the async_trait to_app future)

The full profile was recorded in #153 and is unchanged apart from the queue's blocks: TLS, FileStore, observer and Reject path, each attributed to a component in the test's table. MemoryStore stays at 5 blocks per send, as documented in ADR-0008.

Public / contract impact

None: the queue is private, and Connection's API is unchanged.

Testing

cargo test --workspace --all-features: 2655 passed, 0 failed. fmt, clippy -D warnings, doc and check-spanish are clean.

Closes #76

@joaquinbejar joaquinbejar added enhancement New feature or request session ironfix-session / store / transport / engine performance Hot-path optimization labels Oct 20, 2026
@joaquinbejar joaquinbejar self-assigned this Oct 20, 2026
@joaquinbejar
joaquinbejar merged commit 30608d6 into main Oct 20, 2026
25 of 35 checks passed
@joaquinbejar
joaquinbejar deleted the issue-76-command-queue branch October 20, 2026 16:59

@joaquinbejar joaquinbejar left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES. Two independent controls confirm sender starvation and pooled buffer retention after runtime shutdown. The allocation gate reproduces zero blocks for 1024 ready gate sends, and normal close, cancellation and backpressure checks pass. Submitted as COMMENT because GitHub prevents the author account from requesting changes on its own pull request.

// The guard lives in this block only, so the future holds no lock
// across the wait below (and stays `Send`).
let pushed = {
let mut ring = self.shared.ring();

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Warning: A fresh sender can take this slot before an older notified sender resumes. With capacity 1, the probe completes 1000 fresh sends while the older send is repolled each time and stays pending. Tokio reserves the freed slot for the older waiter. Under sustained producers this can indefinitely delay a waiting send or Logout. Preserve FIFO capacity reservation, including cancellation handoff, rather than relying on Notify ordering.

}

impl<T> Drop for CommandReceiver<T> {
fn drop(&mut self) {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Warning: Closing here leaves queued commands in Shared while Connection handles retain its Arc. Dropping the runtime with two pooled sends queued leaves their buffers unavailable even though the surviving connections report closed. The identical engine probe returns both buffers on PR196. Drain queued commands when the receiver is destroyed and release their payloads outside the queue mutex.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.55224% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
ironfix-engine/src/command_queue.rs 89.06% 7 Missing ⚠️
Files with missing lines Coverage Δ
ironfix-engine/src/connection.rs 97.64% <ø> (ø)
ironfix-engine/src/reactor.rs 77.62% <100.00%> (-0.05%) ⬇️
ironfix-engine/src/command_queue.rs 89.06% <89.06%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request performance Hot-path optimization session ironfix-session / store / transport / engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an allocation-free outbound encode path (engine)

2 participants