Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
either header — or with a non-numeric `Retry-After` — keeps the default
exponential backoff.

- **Network errors become retryable and wrapped**: a failure at the network
level — connection refused, DNS failure, connection/read/write timeout,
reset connection, truncated stream or TLS handshake failure — no longer
escapes `Typesafe::Client#evaluate` (or `Typesafe::Jev.evaluate`) raw. It is
wrapped in a new retryable `Typesafe::ConnectionError < APIError` with the
original exception preserved as `#cause`, and follows the exact same retry
policy as the 429/529/5xx errors: replayed with the default
exponential backoff, then raised once the budget is exhausted. A brief
outage is therefore absorbed by the automatic retries.

## [1.0.0] - 2026-09-20

### Added
Expand Down
95 changes: 77 additions & 18 deletions lib/typesafe/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

require "json"
require "net/http"
require "openssl"

require_relative "question"

Expand All @@ -21,7 +22,8 @@ class Client
DEFAULT_MODEL = "jev-latest"
API_ENDPOINT = "https://api.typesafe.ai/v1/systemone"

# Default retry policy for retryable errors (429, 529, 5xx): at most
# Default retry policy for retryable failures — the 429/529/5xx status
# errors and the +ConnectionError+ wrapping network errors: at most
# +RETRIES+ retries (so a total of +RETRIES + 1+ attempts), with an
# exponential backoff between attempts starting at +BASE_DELAY+ seconds,
# doubled on each retry and capped at +MAX_DELAY+, with jitter.
Expand All @@ -30,6 +32,33 @@ class Client
MAX_DELAY = 8.0
private_constant :RETRIES, :BASE_DELAY, :MAX_DELAY

# Network-level exception families wrapped into a +ConnectionError+ before
# any HTTP response: socket and DNS failures, connection/read/write
# timeouts, refused or reset connections, truncated streams and TLS
# handshake failures. Exhaustive per-version lists are an implementation
# choice (settled in the ticket): classes that only exist on some Ruby
# versions (Resolv::ResolvError on >= 3.3 for DNS, and the write-timeout
# classes where they do not inherit from Timeout::Error) are added when
# defined.
NETWORK_ERRORS = [
SocketError, # generic socket failures ; DNS on Ruby <= 3.2
Errno::ECONNREFUSED, # connection refused
Errno::ECONNRESET, # connection reset
Errno::EHOSTUNREACH, # host unreachable
Errno::ENETUNREACH, # network unreachable
Errno::EPIPE, # write on a truncated stream
Errno::ETIMEDOUT, # system timeout exceeded
EOFError, # stream truncated while reading
Timeout::Error, # Net::OpenTimeout / ReadTimeout / WriteTimeout
Net::HTTPBadResponse, # unreadable HTTP framing
OpenSSL::SSL::SSLError # TLS handshake failure
]
NETWORK_ERRORS << Net::WriteTimeout if defined?(Net::WriteTimeout)
NETWORK_ERRORS << IO::TimeoutError if defined?(IO::TimeoutError)
NETWORK_ERRORS << Resolv::ResolvError if defined?(Resolv::ResolvError)
NETWORK_ERRORS.freeze
private_constant :NETWORK_ERRORS

# @return [String] the API key sent as +Authorization: Bearer <key>+.
attr_reader :api_key

Expand Down Expand Up @@ -64,18 +93,24 @@ def initialize(api_key: nil, model: nil)
# String/Symbol keys to Question values, +model+ is invalid, or the
# response body is not a valid response shape.
# @raise [JSON::ParserError] if the response body is not valid JSON.
# @raise [Typesafe::ConnectionError] if the request fails at the network
# level before any HTTP response (connection refused, DNS failure,
# connection/read/write timeout, reset connection or truncated stream);
# the original exception is available via +cause+. Retried like the
# retryable status errors below, and raised once the budget is
# exhausted.
# @raise [Typesafe::APIError] (or a subclass) on any non-2xx HTTP response:
# {Typesafe::BadRequestError}, {Typesafe::AuthenticationError},
# {Typesafe::PermissionDeniedError}, {Typesafe::NotFoundError},
# {Typesafe::UnprocessableEntityError}, {Typesafe::RateLimitError},
# {Typesafe::OverloadedError} and {Typesafe::ServerError}.
#
# Retryable errors (429, 529 and 5xx) are retried automatically up to
# +RETRIES+ times; the wait honors the server's +Retry-After+ /
# +Retry-After-Ms+ header on a 429, otherwise an exponential backoff
# applies. When the budget is exhausted, the last retryable error is
# raised. Non-retryable errors (400, 401, 403, 404, 422) raise
# immediately without any retry.
# Retryable errors — 429, 529, 5xx and network failures — are retried
# automatically up to +RETRIES+ times; the wait honors the server's
# +Retry-After+ / +Retry-After-Ms+ header on a 429, otherwise an
# exponential backoff applies. When the budget is exhausted, the last
# retryable error is raised. Non-retryable errors (400, 401, 403, 404,
# 422) raise immediately without any retry.
def evaluate(state:, questions:, model: nil)
model = model.nil? ? self.model : freeze_string(model, "model must be a non-empty String")
questions = validate_questions!(questions)
Expand All @@ -92,28 +127,45 @@ def evaluate(state:, questions:, model: nil)

private

# POSTs +body+ and retries retryable error responses up to +RETRIES+
# times. The same +body+ is replayed verbatim on every attempt: an
# evaluation is stateless, so the replay is safe. Raises the error as
# soon as it is not retryable, or once the retry budget is exhausted.
# POSTs +body+ and retries retryable failures up to +RETRIES+ times.
# The same +body+ is replayed verbatim on every attempt: an evaluation is
# stateless, so the replay is safe. Each attempt either returns the HTTP
# response, or an +APIError+ — the status error built from a non-2xx
# response, or the +ConnectionError+ wrapping a network-level failure. A
# failure that is not retryable raises immediately; once the budget is
# exhausted, the last retryable error is raised. Network errors follow
# exactly the same policy as the retryable statuses (429, 529, 5xx).
def request_with_retries(body)
retries = 0
loop do
response = post(body)
return response if response.is_a?(Net::HTTPSuccess)
outcome = attempt(body)
return outcome if outcome.is_a?(Net::HTTPSuccess)

error = Errors.from_response(status: response.code.to_i, headers: response, body: response.body)
raise error unless error.retryable?
raise error if retries >= RETRIES
raise outcome unless outcome.retryable?
raise outcome if retries >= RETRIES

retries += 1
Kernel.sleep(delay_before_retry(error, retries))
Kernel.sleep(delay_before_retry(outcome, retries))
end
end

# One HTTP attempt: posts +body+ and returns the response on success, or
# the matching +APIError+ otherwise — the status error built from the
# non-2xx response, or the +ConnectionError+ a network-level failure
# raised inside +post+. Never raises; the retry loop decides.
def attempt(body)
response = post(body)
return response if response.is_a?(Net::HTTPSuccess)

Errors.from_response(status: response.code.to_i, headers: response, body: response.body)
rescue ConnectionError => error
error
end

# Delay before retry +retry_number+. When the server imposes the pace on a
# rate limit (429), its +Retry-After+ / +Retry-After-Ms+ header is honored
# exactly; otherwise the delay is the exponential backoff: the base delay
# exactly; otherwise — including for +ConnectionError+, which carries no
# HTTP headers — the delay is the exponential backoff: the base delay
# doubled on each retry, capped at +MAX_DELAY+ seconds, with equal jitter
# so parallel clients do not align (uniform between half the nominal delay
# and the nominal delay). A non-numeric or non-positive +Retry-After+ is
Expand All @@ -138,6 +190,13 @@ def post(body)
use_ssl: true,
open_timeout: 5, write_timeout: 10, read_timeout: 30
) { |http| http.request(request) }
rescue *NETWORK_ERRORS => error
# No HTTP response was involved: the failure is a network error, so it
# is wrapped — instead of escaping raw — and marked retryable. Raising
# from within this rescue keeps the original exception as +cause+.
raise ConnectionError.new(
message: "The TypeSafe API could not be reached: #{error.message}"
), cause: error
end

def validate_questions!(questions)
Expand Down
30 changes: 30 additions & 0 deletions lib/typesafe/errors.rb
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,36 @@ class OverloadedError < APIError; end
# The server failed to process the request (5xx, excluding 529).
class ServerError < APIError; end

# A network-level failure before any HTTP response: connection refused, DNS
# failure, connection/read/write timeout, reset connection or truncated
# stream. Wraps the underlying network exception — available via +cause+ —
# so that no raw +Net::HTTP+ / socket / timeout exception escapes the
# client. Marked retryable: a brief outage is absorbed by the automatic
# retry loop, and the error is raised only once the budget is exhausted.
#
# begin
# client.evaluate(state:, questions:)
# rescue Typesafe::ConnectionError => e
# warn "API unreachable: #{e.cause}"
# end
#
# Unlike status errors, there is no HTTP status code: +status+ is nil.
class ConnectionError < APIError
# @param message [String] the rendered message.
# @raise The wrapped network exception is attached as +cause+ when the
# error is raised from within the rescue that catches it.
def initialize(message:)
super(status: nil, message: message)
end

# A network failure may succeed when replayed after a delay: the outage is
# likely transient, so the error is retryable by construction.
# @return [true]
def retryable?
true
end
end

# Builds error instances from raw HTTP responses.
module Errors
module_function
Expand Down
87 changes: 87 additions & 0 deletions spec/typesafe/client_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,93 @@
expect(sleeps[0]).to be_between(0.25, 0.5)
end
end

context "network errors" do
# Network-level failures are wrapped in Typesafe::ConnectionError and
# follow the retry policy of the retryable HTTP errors, so the kernel
# sleep is stubbed and delays recorded like in the "retries" context.
let(:sleeps) { [] }

before do
allow(Kernel).to receive(:sleep) { |delay| sleeps << delay }
end

# Representative network-failure families. Resolv::ResolvError exists
# only on Ruby >= 3.3: on 3.1/3.2 a DNS failure raises SocketError, so
# the spec keeps the list version-portable.
network_errors = [
Errno::ECONNREFUSED, # connexion refusée
Errno::ECONNRESET, # connexion réinitialisée
Net::ReadTimeout, # timeout de lecture
EOFError # flux coupé
]
network_errors << Resolv::ResolvError if defined?(Resolv::ResolvError)

network_errors.each do |network_error|
it "retries a #{network_error} and succeeds on the next attempt" do
stub = stub_request(:post, endpoint)
.to_raise(network_error).then
.to_return(status: 200, body: JSON.generate(example_response))

response = client.evaluate(state: state, questions: questions)

expect(response).to eq(Typesafe::Response.from_h(example_response))
expect(stub).to have_been_requested.times(2)
expect(sleeps.length).to eq(1)
end
end

it "retries a read timeout (to_timeout) and succeeds on the next attempt" do
stub = stub_request(:post, endpoint)
.to_timeout.then
.to_return(status: 200, body: JSON.generate(example_response))

response = client.evaluate(state: state, questions: questions)

expect(response).to eq(Typesafe::Response.from_h(example_response))
expect(stub).to have_been_requested.times(2)
end

it "raises a retryable Typesafe::ConnectionError with the original exception as cause once the budget is exhausted" do
stub = stub_request(:post, endpoint).to_raise(Errno::ECONNREFUSED)

expect { client.evaluate(state: state, questions: questions) }
.to raise_error(Typesafe::ConnectionError) { |error|
expect(error).to be_retryable
expect(error.status).to be_nil
expect(error.cause).to be_a(Errno::ECONNREFUSED)
}

expect(stub).to have_been_requested.times(3)
expect(sleeps.length).to eq(2)
end

raw_network_errors = [
Errno::ECONNREFUSED, Errno::ECONNRESET, Net::ReadTimeout, Timeout::Error,
SocketError, EOFError
]
raw_network_errors << Resolv::ResolvError if defined?(Resolv::ResolvError)

raw_network_errors.each do |network_error|
it "never lets a raw #{network_error} escape from #evaluate" do
stub_request(:post, endpoint).to_raise(network_error)

expect { client.evaluate(state: state, questions: questions) }
.to raise_error(Typesafe::ConnectionError)
end
end

it "applies to network errors exactly the delay policy of retryable HTTP errors" do
stub_request(:post, endpoint).to_raise(Errno::ECONNREFUSED)

expect { client.evaluate(state: state, questions: questions) }
.to raise_error(Typesafe::ConnectionError)

expect(sleeps.length).to eq(2)
expect(sleeps[0]).to be_between(0.25, 0.5)
expect(sleeps[1]).to be_between(0.5, 1.0)
end
end
end

private
Expand Down
12 changes: 11 additions & 1 deletion spec/typesafe/errors_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -224,12 +224,22 @@
end
end

describe Typesafe::ConnectionError do
it "is a retryable Typesafe::APIError without any HTTP status" do
error = Typesafe::ConnectionError.new(message: "The TypeSafe API could not be reached: boom")

expect(Typesafe::ConnectionError).to be < Typesafe::APIError
expect(error).to be_retryable
expect(error.status).to be_nil
end
end

describe Typesafe::Error do
it "is the root of all gem errors" do
[Typesafe::APIError, Typesafe::BadRequestError, Typesafe::AuthenticationError,
Typesafe::PermissionDeniedError, Typesafe::NotFoundError,
Typesafe::UnprocessableEntityError, Typesafe::RateLimitError,
Typesafe::OverloadedError, Typesafe::ServerError].each do |klass|
Typesafe::OverloadedError, Typesafe::ServerError, Typesafe::ConnectionError].each do |klass|
expect(klass).to be < Typesafe::Error
end
end
Expand Down
14 changes: 14 additions & 0 deletions spec/typesafe/jev_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,20 @@
expect(client.evaluate(state: state, questions: questions))
.to eq(Typesafe::Response.from_h(example_response))
end

it "inherits the network error wrapping: a refused connection raises Typesafe::ConnectionError" do
client = described_class.new(api_key: api_key)
allow(Kernel).to receive(:sleep)

stub_request(:post, endpoint)
.with(
headers: { "Authorization" => "Bearer #{api_key}" },
body: { "state" => state, "model" => "jev-latest", "questions" => { "is_urgent" => { "type" => "noul", "instructions" => "Does this convey urgency?" } } }
).to_raise(Errno::ECONNREFUSED)

expect { client.evaluate(state: state, questions: questions) }
.to raise_error(Typesafe::ConnectionError)
end
end

describe ".evaluate one-shot" do
Expand Down
Loading