fix(wrangler): keep wrangler dev alive when forwarding one request to the Worker fails - #15448
fix(wrangler): keep wrangler dev alive when forwarding one request to the Worker fails#15448LiyanChen-X wants to merge 6 commits into
wrangler dev alive when forwarding one request to the Worker fails#15448Conversation
🦋 Changeset detectedLatest commit: 339dd54 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
2177181 to
15e40ed
Compare
@cloudflare/autoconfig
@cloudflare/build-output-utils
@cloudflare/codemods
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-functions
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-plugin
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
…to the Worker fails
The ProxyWorker treated a rejected `fetch()` to the UserWorker as a fatal
ProxyWorker error, which made the whole dev session exit. The common way to
hit it is a client that disconnects while its request body is still being
uploaded ("Network connection lost."), including a request that was queued
during startup/reload and abandoned before it could be replayed.
A rejected forward is the outcome of that single request, not a proxy
defect: answer it with a 502, log at debug level and keep the session
running. Errors thrown while post-processing a response keep the existing
fatal handling.
Adds an e2e that destroys the connection mid-upload against a real
`wrangler dev` process and asserts the process survives (fails on main
with ECONNREFUSED because the process exited).
15e40ed to
bae544f
Compare
wrangler dev alive when forwarding one request to the Worker fails
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
|
Tested the 1. Your aborted-upload repro, old vs. this branchYour
A variant that keeps the socket open instead of destroying it gives the same split: process exits on 4.126.0, alive on this branch. 2. The part I think is new: it also fires on a bodyless GETFive runs of our Playwright suite (~96 tests, ~2.5 min each, one dev server) on the branch. Four runs fully green; in the fifth, the new path fired once: The dev session stayed up and the suite ran to completion (95 passed). That request is a plain This seems to support the design argument in your PR description — "a rejected forward should not be fatal regardless of why it rejected" — with a case the description doesn't currently cover. On 4.126.0 the same rejection would have taken the 3. One consequence worth a line in the changesetFor a browser-driven suite the 502 is not invisible: it surfaces as a page-level 4. Log shapes — three different ones for the same causeWorth flagging for anyone building log-based triage (we did, for our own tracking issue):
Our extractor keys on the 5. Suite-level rates cannot show this PR's effect, and I would not use them
Happy to run this on Linux CI as well, or with a larger N, if that would help before merging. |
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
Fixes #15447. Also removes the crash reported in #15203 (verified with the same shape of repro, see below); the signature is the same one as #15317 / #4562.
What this does
ProxyWorkerforwards every incoming request to the UserWorker withfetch(). When thatfetch()rejects while the request still targets the current UserWorker, the rejection was sent to theProxyControlleras{ type: "error" }, which is fatal:wrangler devexits.The most common way to get such a rejection has nothing to do with the proxy or the Worker: the client disconnected while its request body was still being uploaded, so workerd fails the read of the body (
Network connection lost./Can't read from request stream because client disconnected). A singlecurl -m 0.4 -X POST --data-binary @2mb.bin http://127.0.0.1:8787/uploadagainst an otherwise idlewrangler devis enough to take the whole session down. Requests the proxy queued during startup/reload and that were abandoned before they could be replayed (the scenario in #15447) are the same failure, just later.A rejected forward is the outcome of that one request, not a defect in the proxy. This PR:
fetch()in aUserWorkerFetchErrormarker so it can be told apart from errors thrown while post-processing the response (checkForPreviewTokenError, live-reload injection, …);502(Could not proxy this request to your Worker: <message>) and logs it through the existingdebug-logchannel, instead of failing the session;503for the rest) untouched.The discrimination is structural (which promise rejected), not based on error message text.
Why not
request.signalrequest.signal.abortedwould be the principled signal for "the client is gone", so I tried it first: it staysfalsefor these requests, both before the replay and inside the rejection handler. TheProxyWorkerruns withcompatibilityFlags: ["nodejs_compat"], withoutenable_request_signal; addingenable_request_signalandrequest_signal_passthroughto theProxyWorkeralone does not change that either, because the request reaches theProxyWorkerthrough Miniflare's core entry worker (connect_pass_throughonly), so the abort never crosses that hop. Making the signal usable would mean turningrequest_signal_passthroughon for Miniflare's entry worker, which changes cancellation semantics for every user Worker in local dev — a much larger change than this bug warrants. And a rejected forward should not be fatal regardless of why it rejected.Verification
All against a locally built wrangler from this branch vs
main, on macOS with wrangler 4.127.1 / workerd 1.20260828.1.mainwrangler dev,curl -m 0.4 -X POST --data-binary @2mb.bin /uploadError in ProxyController: Error inside ProxyWorker,Network connection lost.)502, session keeps servingassets.directoryconfigured, 100 KB POSTs to an existing asset path (#15203 shape)405 500 405 500 405 000 000)405/502alternating, no exit)client disconnects > does not exit wrangler dev when a client aborts a request mid-bodyconnect ECONNREFUSED— the dev process exitedNote on the assets row: the alternating
405/502onmain's405/500is the pre-existing keep-alive behaviour after an unconsumed body (cf. #14641); this PR only stops it from being fatal.The e2e destroys the TCP connection mid-body with a raw
node:netsocket (Content-Length: 2000000, one 64 KB chunk,destroy()), because anAbortControllerabort on anundicistreaming body ends the request gracefully and does not reproduce the failure.pnpm check:type,tsc -p e2e/tsconfig.json,oxlint --type-awareandoxfmtpass on the touched files.Notes for reviewers
wrangler devbehaviour only for the case that previously terminated the process; a502with the underlying message is what the client (if it is still there) now sees.errormessage type non-fatal inDevEnv.handleErrorEvent" because that would also swallow genuine proxy bugs; the marker keeps post-processing errors loud.wrangler dev; no user-facing configuration or documented behaviour changes.