Skip to content

Remote /health wait: count it against the command timeout and cap it #4567

Open

Description

@thymikee

Purpose

#4541 made the remote /health check before the first send retry until the command's own timeout when the probe times out or gets a 5xx. That wait does not count against the command's timeout, and the send then gets its full timeout too. As a result:

  • A dead remote daemon that still answers (for example a gateway returning 502, or dropped TLS) makes every command wait up to its full timeout before failing: 90 s by default.
  • A command can take up to about twice its timeout.

commands.md documents this bound today, and the merged code reports details.healthRecoveryBudgetMs.

Required behavior

  • The /health wait counts against the command's own timeout. Create one deadline before the first probe, and give the send only what is left.
  • If too little time is left to send and finish, refuse with remote_daemon_unavailable and dispatched: no instead of sending a command that can't complete. This keeps the ambiguity Deadline-aware recovery for read-only commands (adb offline, runner busy, remote health) #4522 forbids out: the client must never time out after the command reached the device.
  • Cap the health wait at a few seconds by default, so a dead host fails fast while a restarting daemon still gets a short grace period.
  • A refused connection still fails at once, as it does now.

Completion

  • Tests: a dead host that answers 502 fails within the cap; a daemon that comes back inside the cap gets the command with the remaining time; a command whose budget ran out during recovery is refused with dispatched: no and never sent.
  • commands.md drops the "up to about twice the command timeout" bound.

Follow-up to #4541 (part of #4522).

Activity

  1. apex-sentinel-staging commented on Nov 15, 2026

    @apex-sentinel-staging

    Apex Sentinel · Issue review

    Suggested label: feature

    The issue requests changing remote /health recovery behavior: count the health wait against the command timeout, cap it, and refuse sending when the remaining budget is too small. It includes concrete required behavior, completion tests, and documentation updates, so a maintainer can evaluate and implement it as an enhancement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions