Repository navigation
feat: fail only the request when a handler hits a runtime error - #1399
Merged
Merged
Conversation
# Conflicts: # self-host/bootstrap/ir_driver.ir
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A runtime error inside an http handler (std.net.http, std.web, the http/2 server) now fails that request with a 500, and one inside a grpc method fails the call with INTERNAL. The server keeps serving. Everywhere else the process still exits with the same message and status 1. docs/failure_isolation.md has the design, the alternatives and the semantics.
The handler call runs behind a catch frame on the task's own stack (
cranelift/runtime/src/failure.rs): a short naked routine for x86_64 and aarch64 saves the callee-saved registers, and a runtime error that finds a frame jumps back to it with its message. The runtime error sites in scope (index, key, value on none, division, substring split, and anypith_runtime_errorcode other than 3 to 5, so integer overflow is covered when it lands) now go throughfailure::failand own nothing that needs a drop when they jump. Pith frames that are skipped leak what they held and their defers do not run; the bound is in the doc. Mutexes locked inside the call are recorded and unlocked on failure. The new builtin isisolate_call(fn() -> Int) -> String. A nested coroutine per call was measured too: about 8 ns per call against under 1 ns for the catch frame, and it would need every block site to forward parks through the inner coroutine.The http/1.1 answer is a 500 with
Connection: close; http/2 answers a 500 on the stream (a streaming handler gets RST_STREAM INTERNAL_ERROR); grpc answers status 13 with the messageinternal error. stderr getspith runtime error in GET /path: <message>.Tests:
tests/cases/test_web_handler_failure.pith(each error kind over a real socket, a second request after each, pipelining behind a failure, and 20 failures while holding a Mutex),tests/cases/test_grpc_method_failure.pith(unary and streaming, then a good call on the same connection), three abort goldens showing that errors outside a handler still exit 1 with the same message (main after a caught failure, a spawned task, the os-thread backend), andtests/leaks/leak_isolated_failure.pith, which holds the machinery itself flat.Benchmark:
bench/http_sweep.sh, pith arm only, server on cpus 0-7,16-23 and wrk on 8-15,24-31, 10 s runs, four rounds with the arm order rotated each round. Other builds were running on the machine (load average 20 to 33), so single runs vary by 3% to 5%. Medians in req/s:The first version carried the response through a threadlocal, which marks it shared and made its counts atomic; that cost 4% to 7% at 256 and 1024 connections and is fixed in the last commit.
Not covered in phase 1: errors in tasks a handler spawns, stack overflow, allocation failure, streamed body producers and connection takers (they run after the handler returns), and semaphores or channels a failed handler was using.