Skip to content

Bound a single attempt with attempt_timeout - #5

Merged
azeemshaik025 merged 4 commits into
mainfrom
attempt-timeout
Aug 16, 2026
Merged

Bound a single attempt with attempt_timeout#5
azeemshaik025 merged 4 commits into
mainfrom
attempt-timeout

Conversation

@azeemshaik025

Copy link
Copy Markdown
Owner

Bound a single attempt with attempt_timeout

max_elapsed lives inside should_retry_after, which is only reached when an attempt returns.
An operation that hangs never returns, so the budget never applied. A dead TCP peer, a lost
response, a service that accepts your connection and then goes quiet: all produce a future that is
simply never ready, and the retry sat there forever.

That was not a missing feature so much as a broken promise — README.md said "give up after ~30s
total" and the crate did not do it. Measured before the fix: a 500 ms budget still running at 3
seconds, stopped only by an external tokio::time::timeout. After: gives up in 310 ms with
reason=max_elapsed.

retry(fetch)
    .attempt_timeout(Duration::from_secs(5), || MyError::Timeout)
    .max_elapsed(Duration::from_secs(30))
    .await?

The per-attempt bound is what makes the total budget enforceable at all: a timed-out attempt
returns control to the loop, so max_elapsed is finally consulted.

Why on_timeout is a closure

A timed-out attempt has no E, because the operation never returned one. Three ways out, and only
one is cheap for people who never use this:

option cost
RetryError::error() -> Option<&E> every caller unwraps an Option mid-match, forever, for a case most never hit
E: From<SomeTimeoutType> a bound on the whole builder — rules out E = String and anyhow::Error, exactly what ADR006 fought to keep
on_timeout: impl Fn() -> E one extra argument

Verified from outside the crate that the third stays a one-liner for every common error type —
io::Error is \|\| ErrorKind::TimedOut.into(). The builder is order-independent in both
directions, and the one-arg mistake a tokio::time::timeout user would make produces
error[E0061]: this method takes 2 arguments, not a generics wall.

Full reasoning, including why the deadline runs on the injected Clock and why there is no
blocking equivalent: ADR007.

Breaking

Retry and RetryFuture gain one type parameter, for the handler. retry(..) and every builder
method are unchanged, so this only affects code that names those types — storing one in a struct
field, say. A defaulted Q = NoTimeout was tried and does not work: NoTimeout cannot implement
Fn() -> E, so the no-timeout case would need a second, overlapping IntoFuture impl.

Also in this branch

  • The docs stopped lying. lib.rs claimed "timeout and circuit breaking are planned" — timeout
    ships here, and the circuit breaker was built and deliberately not shipped. The "resilience
    toolkit" tagline and a one-item ## Tools list also still sold a toolkit the roadmap had already
    decided against.
  • docs/ROADMAP.md lands on main. It was written on the circuit-breaker branch and stranded
    there, so the file recording what we refuse had never actually been on main.
  • Two unreachable expect()s removed from the poll loop. It reached the timeout handler through
    an Option it had just proved was Some. Matching the deadline and its handler as a pair makes
    the arm that needs both the only one that can run.
  • examples/retry.rs gains the case that motivated the feature: a call that never returns.

Verification

15 gates across all three feature configs (--all-features, default, blocking-only): tests,
clippy -D warnings, rustdoc -D warnings, and MSRV 1.85. 65 unit + 14 doctests. Both examples
run.

The README promised "give up after ~30s total" and we did not deliver it.
max_elapsed is only consulted between attempts, inside should_retry_after,
which is reached only when an attempt returns. An operation that hangs --
a dead TCP peer, a lost response -- never returns, so the budget never got
a chance to apply. Measured before the fix: a 500ms budget still running
at 3 seconds, killed only by an external tokio::time::timeout.

attempt_timeout(duration, on_timeout) arms a deadline alongside the
in-flight future. When it fires the future is dropped, which in Rust is
real cancellation, and the attempt becomes a failure that feeds the normal
backoff and the normal .when predicate. on_timeout supplies the error,
because the operation never produced one; mapping it into the caller's own
type keeps everything downstream working on one type rather than forcing
RetryError::error() to become an Option.

The deadline runs on the injected Clock, so unlike tokio::time::timeout it
is testable: the tests assert the exact sleep sequence on a mock clock and
finish instantly.

This also makes max_elapsed mean something. A timed-out attempt returns
control to the loop, so the budget is finally consulted -- the end-to-end
check now reports reason=max_elapsed where before it hung.

One thing the work exposed: the mock clock advanced virtual time when a
sleep was created rather than when it was polled. That was invisible while
retry only ever built a sleep it intended to await. Arming a deadline a
fast operation never polls made it visible, and the mock now matches real
timer semantics.

Breaking only for code that names Retry or RetryFuture, which gain a type
parameter for the handler.
ADR007 covers the four decisions the last commit made without writing them
down: why the error comes from a caller-supplied closure rather than making
RetryError::error() an Option (don't tax every user for a case most never
hit), why there is no blocking equivalent (interrupting a blocking closure
needs another thread, which forces Send + 'static, and two tests exist to
forbid that), why Retry gains a type parameter and why a defaulted NoTimeout
does not work, and the mock-clock lesson.

docs/ROADMAP.md was written on the circuit-breaker branch and stranded
there, so the file recording what we refuse has never been on main. It
lands updated: attempt_timeout shipped, standalone timeout refused with the
reason, and the circuit breaker recorded as built, tested and not shipped,
with the download numbers and the "does it answer 'a call failed, now
what'" test that ruled it out.

It also states what mettle is now, which is narrower than it started:
retry solved completely, not a toolkit of shallow tools.
The docs claimed "timeout and circuit breaking are planned". Timeout shipped in
the previous commit, and the circuit breaker was built and deliberately not
shipped, so both halves were false on the docs.rs landing page. The tagline and
a one-item "Tools" list also still sold a toolkit the roadmap had already
decided against.

The poll loop reached the timeout handler through an Option it had just proved
was Some, and funnelled the error through a second Option for the same reason.
Matching the deadline and its handler as a pair makes the arm that needs both
the only one that can run, so neither expect() is needed.

examples/retry.rs gains the case that motivated the feature: a call that never
returns, which before hung forever and now gives up in 310ms with
reason=max_elapsed. The blocking example says why it has no equivalent.
cargo-semver-checks runs on the pull request, not at release time, so a
breaking change left at the published version fails CI regardless of merit. It
flagged exactly what ADR007 predicted: Retry 4 -> 5 and RetryFuture 6 -> 7
required generic parameters, "semver requires new major version".

0.x makes that a minor bump, so 0.4.0 -> 0.5.0, with the CHANGELOG heading
dated and an Upgrading section for the only people affected: those who name
Retry or RetryFuture rather than using them inline. This follows ae3a5d2, where
the last breaking change carried its own bump for the same reason.

CONTRIBUTING said to run the semver check locally without saying how, and its
feature-matrix list was missing the default and async-only configurations that
CI actually runs. Both are now spelled out, along with the rule that a breaking
change bumps the version in its own branch.
@azeemshaik025
azeemshaik025 merged commit 35c2d33 into main Aug 16, 2026
14 checks passed
@azeemshaik025
azeemshaik025 deleted the attempt-timeout branch August 16, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant