fix: avoid binary-copy overhead when reassembling gRPC message bodies - #576
Open
asweet-confluent wants to merge 2 commits into
Open
Conversation
asweet-confluent
force-pushed
the
claude/gun-adapter-performance-4596eb
branch
from
August 24, 2026 22:54
0d9192a to
8707308
Compare
recv_body (gun client adapter) and read_full_body (cowboy server handler) rebuilt the accumulated binary on every incoming chunk via `<<acc::binary, data::binary>>`. BEAM optimizes single-reference binary appends in place, so this isn't unbounded quadratic growth, but it still measured consistently several times slower in wall-clock time than an iolist accumulator across realistic body sizes. Accumulate chunks as a reversed list and flatten once with IO.iodata_to_binary/1 instead, which is both faster and doesn't depend on that runtime heuristic. Adds a regression test covering the request body reassembled across multiple separate :cowboy_req.read_body calls, synchronized by tracing read_full_body's recursion.
asweet-confluent
force-pushed
the
claude/gun-adapter-performance-4596eb
branch
from
August 24, 2026 22:59
8707308 to
ce4ff60
Compare
asweet-confluent
marked this pull request as ready for review
August 24, 2026 23:54
mix format --check-formatted was never run locally; this file had lines that violated the formatter's line-length rules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
recv_body(gun client adapter) andread_full_body(cowboy server handler) bothreassembled incoming message bodies by rebuilding the accumulated binary on every
chunk:
BEAM
optimizes single-reference binary appends in place
rather than always copying, so this isn't unbounded quadratic growth. Even so, it
measured consistently several times slower than an iolist accumulator across realistic
body sizes, with peak memory modestly above the final size (benchmark details below).
The iolist approach is both faster and doesn't depend on a runtime heuristic that other
code nearby (message sends, ETS inserts, pattern matches on the accumulator) could
invalidate.
Fix
Both functions now accumulate chunks as a list and flatten once via
IO.iodata_to_binary/1, givingO(n)time and allocation independent of theappend-in-place heuristic.
Testing
Added a regression test for the server-side recursive
:morepath inread_full_body/5. It previously had zero coverage, since every other existing testsends a body small enough to complete in a single
:cowboy_req.read_bodycall. Thenew test forces the body across multiple separate reads, pacing each send by tracing
read_full_body's recursion so the multi-chunk path is provably exercised on everyrun rather than assumed.
Benchmark methodology and results
Benchmarked on both Elixir 1.15.4/OTP 26 and Elixir 1.20.3/OTP 28, timing a body built
from 16KB chunks (HTTP/2's default max frame size, and what this project's gun/cowboy
stack actually negotiates). Median of 10 runs each, isolated in a fresh process per run:
These are medians; individual runs varied more (up to ~17x in one-off checks), since
wall-clock time is far more sensitive to scheduler noise than allocation size is.
Peak binary memory during a 128 MB reassembly: 137.6 MB for OLD vs 128.0 MB for NEW,
about 8% overhead — a full retained duplicate would show closer to 2x, so this is
modest slack, not a second copy sitting around. That figure was reproducible
bit-for-bit across 5 repeat runs.
Most real gRPC messages fit in a single HTTP/2 frame, so also checked single-chunk
bodies separately. No regression — NEW is faster there too, by a growing margin:
For a single-chunk body,
IO.iodata_to_binary/1returns the original chunk unchangedinstead of copying it (confirmed with
:erts_debug.same/2). OLD always pays a realcopy here, since appending onto
<<>>is a first-time append and can't skip it.Chunk size (independent of total body size) also shifts the ratio: it narrows to ~3x
with very small (1KB) chunks, since NEW's list overhead scales with chunk count, and
levels off around 7-8x from 16KB up through 1MB chunks. At the extreme where the whole
body is one chunk, NEW hits the zero-copy case above and the gap widens to 1000x+.
Oddly, OLD also gets fast at exactly 2 equal-sized chunks, likely a lucky alignment
with BEAM's binary growth doubling, briefly narrowing the gap to ~1x. NEW was never
slower than OLD in any configuration tested.
Scripts: https://gist.github.com/asweet-confluent/2c1c4fefba69da34639f62f12fb0204e
This PR was made with AI assistance.