Skip to content

Fix Proxy to correctly forward Transfer-Encoding: chunked responses - #835

Merged
ademar merged 4 commits into
masterfrom
copilot/fix-proxy-chunked-encoding
Jul 15, 2026
Merged

Fix Proxy to correctly forward Transfer-Encoding: chunked responses#835
ademar merged 4 commits into
masterfrom
copilot/fix-proxy-chunked-encoding

Conversation

Copilot AI commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Suave.Proxy forwarded the origin's Transfer-Encoding: chunked header verbatim while streaming the body from HttpWebResponse.GetResponseStream(), which transparently de-chunks. Clients received a chunked response whose payload was not actually chunked, e.g. curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding.

Changes in src/Suave/Proxy.fs

  • Detect chunked upstream by matching chunked as a comma-separated token (case-insensitive) in Transfer-Encoding, not a substring — avoids false positives from values like gzip, chunked being treated correctly and rules out spurious matches.
  • Filter hop-by-hop headers when re-framing: on the chunked path, strip Transfer-Encoding and the mutually-exclusive Content-Length from the forwarded header set before re-adding Transfer-Encoding: chunked.
  • Re-chunk on egress using the existing transferStreamChunked helper (same one used by okStreamChunked), instead of writing the de-chunked bytes raw.
  • Non-chunked responses continue through the original Content-Length / close-delimited path unchanged.
let isChunked =
  match allHeaders ? ("Transfer-Encoding") with
  | Some v ->
    v.Split(',')
    |> Array.exists (fun t -> String.Equals(t.Trim(), "chunked", StringComparison.OrdinalIgnoreCase))
  | None -> false

// ...
if isChunked then do! transferStreamChunked conn stream
else                do! transferStream conn stream

Verification

Smoke-tested against a hand-rolled upstream that emits Transfer-Encoding: chunked with a multi-chunk body: the proxy output now contains a single Transfer-Encoding: chunked, no Content-Length, and correctly framed chunks terminated by 0\r\n\r\n.

Known limitation (out of scope)

Exotic stacks like Transfer-Encoding: gzip, chunked (gzip applied as a transfer coding rather than Content-Encoding) will lose the gzip layer since the entire header is dropped. This encoding form is essentially unused in practice; happy to extend if reviewers want it preserved.

Copilot AI linked an issue Jul 15, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Fix proxy handling for transfer-encoding chunked data Fix Proxy to correctly forward Transfer-Encoding: chunked responses Jul 15, 2026
Copilot AI requested a review from ademar July 15, 2026 17:45
@ademar

ademar commented Jul 15, 2026

Copy link
Copy Markdown
Member

@copilot Since you Smoke-tested it; can you make that into a unit test.

Copilot AI commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Since you Smoke-tested it; can you make that into a unit test.

Done in ce0dc25 — added src/Suave.Tests/Proxy.fs with an Expecto test Proxy.forwards a chunked upstream response as a valid chunked response. It spins up a hand-rolled TCP upstream that emits Transfer-Encoding: chunked with a multi-chunk body, points Suave.Proxy at it, and asserts on the raw wire response — exactly one Transfer-Encoding: chunked, no Content-Length, correct hex length prefix, and the 0\r\n\r\n terminator. Verified locally: Proxy.forwards a chunked upstream response as a valid chunked response passed in 00:00:00.1500000.

Copilot AI requested a review from ademar July 15, 2026 17:56
@ademar
ademar marked this pull request as ready for review July 15, 2026 17:58
@ademar
ademar merged commit 813f476 into master Jul 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How to proxy with transfer-encoding chunked?

2 participants