Skip to content

fix: avoid binary-copy overhead when reassembling gRPC message bodies - #576

Open
asweet-confluent wants to merge 2 commits into
elixir-grpc:masterfrom
asweet-confluent:claude/gun-adapter-performance-4596eb
Open

fix: avoid binary-copy overhead when reassembling gRPC message bodies#576
asweet-confluent wants to merge 2 commits into
elixir-grpc:masterfrom
asweet-confluent:claude/gun-adapter-performance-4596eb

Conversation

@asweet-confluent

Copy link
Copy Markdown

Problem

recv_body (gun client adapter) and read_full_body (cowboy server handler) both
reassembled incoming message bodies by rebuilding the accumulated binary on every
chunk:

recv_body(stream_payload, <<acc::binary, data::binary>>, opts)

BEAM
optimizes single-reference binary appends in place
rather than always copying, so this isn't unbounded quadratic growth. Even so, it
measured consistently several times slower than an iolist accumulator across realistic
body sizes, with peak memory modestly above the final size (benchmark details below).
The iolist approach is both faster and doesn't depend on a runtime heuristic that other
code nearby (message sends, ETS inserts, pattern matches on the accumulator) could
invalidate.

Fix

Both functions now accumulate chunks as a list and flatten once via
IO.iodata_to_binary/1, giving O(n) time and allocation independent of the
append-in-place heuristic.

Testing

Added a regression test for the server-side recursive :more path in
read_full_body/5. It previously had zero coverage, since every other existing test
sends a body small enough to complete in a single :cowboy_req.read_body call. The
new test forces the body across multiple separate reads, pacing each send by tracing
read_full_body's recursion so the multi-chunk path is provably exercised on every
run rather than assumed.

Benchmark methodology and results

Benchmarked on both Elixir 1.15.4/OTP 26 and Elixir 1.20.3/OTP 28, timing a body built
from 16KB chunks (HTTP/2's default max frame size, and what this project's gun/cowboy
stack actually negotiates). Median of 10 runs each, isolated in a fresh process per run:

Body size OLD NEW ratio
4 MB 2.5 ms 0.2 ms 12.5x
8 MB 4.7 ms 0.5 ms 9.4x
16 MB 9.0 ms 1.3 ms 6.9x
32 MB 17.9 ms 2.4 ms 7.5x
64 MB 34.9 ms 4.9 ms 7.1x
128 MB 66.8 ms 9.5 ms 7.0x

These are medians; individual runs varied more (up to ~17x in one-off checks), since
wall-clock time is far more sensitive to scheduler noise than allocation size is.

Peak binary memory during a 128 MB reassembly: 137.6 MB for OLD vs 128.0 MB for NEW,
about 8% overhead — a full retained duplicate would show closer to 2x, so this is
modest slack, not a second copy sitting around. That figure was reproducible
bit-for-bit across 5 repeat runs.

Most real gRPC messages fit in a single HTTP/2 frame, so also checked single-chunk
bodies separately. No regression — NEW is faster there too, by a growing margin:

Payload size OLD NEW ratio
64 B 0.065 µs 0.017 µs 3.8x
256 B 0.067 µs 0.017 µs 3.9x
1 KB 0.073 µs 0.017 µs 4.3x
4 KB 0.108 µs 0.017 µs 6.4x
16 KB 0.279 µs 0.017 µs 16.4x

For a single-chunk body, IO.iodata_to_binary/1 returns the original chunk unchanged
instead of copying it (confirmed with :erts_debug.same/2). OLD always pays a real
copy here, since appending onto <<>> is a first-time append and can't skip it.

Chunk size (independent of total body size) also shifts the ratio: it narrows to ~3x
with very small (1KB) chunks, since NEW's list overhead scales with chunk count, and
levels off around 7-8x from 16KB up through 1MB chunks. At the extreme where the whole
body is one chunk, NEW hits the zero-copy case above and the gap widens to 1000x+.
Oddly, OLD also gets fast at exactly 2 equal-sized chunks, likely a lucky alignment
with BEAM's binary growth doubling, briefly narrowing the gap to ~1x. NEW was never
slower than OLD in any configuration tested.

Scripts: https://gist.github.com/asweet-confluent/2c1c4fefba69da34639f62f12fb0204e

This PR was made with AI assistance.

@asweet-confluent
asweet-confluent force-pushed the claude/gun-adapter-performance-4596eb branch from 0d9192a to 8707308 Compare August 24, 2026 22:54
recv_body (gun client adapter) and read_full_body (cowboy server handler)
rebuilt the accumulated binary on every incoming chunk via
`<<acc::binary, data::binary>>`. BEAM optimizes single-reference binary
appends in place, so this isn't unbounded quadratic growth, but it still
measured consistently several times slower in wall-clock time than an
iolist accumulator across realistic body sizes. Accumulate chunks as a
reversed list and flatten once with IO.iodata_to_binary/1 instead, which
is both faster and doesn't depend on that runtime heuristic.

Adds a regression test covering the request body reassembled across
multiple separate :cowboy_req.read_body calls, synchronized by tracing
read_full_body's recursion.
@asweet-confluent
asweet-confluent force-pushed the claude/gun-adapter-performance-4596eb branch from 8707308 to ce4ff60 Compare August 24, 2026 22:59
@asweet-confluent
asweet-confluent marked this pull request as ready for review August 24, 2026 23:54
mix format --check-formatted was never run locally; this file had
lines that violated the formatter's line-length rules.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant