[janosmurai]

Type of change

• New feature
• Bug fix
• Breaking change
• Refactoring or maintenance
• Documentation update

Related issue

Closes SHD-1094 (router retry).

Description

With llm-d v1.0, the router (EPP) can answer a request itself, for example with 503 while no model server pod is ready. It then sets x-llm-d-request-dropped-reason. A reason starting with rejected- means no model server saw the request, so the platform provider now retries it: up to 3 attempts, with a backoff starting at 500 ms. This applies to chat, completion and embedding requests. evicted requests already used GPU time and are not retried. Any other error is returned as before.

Validation

• cargo fmt --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo build --verbose --all-features
• cargo test --verbose --all-features
• Relevant Docker, service-integration, or documentation checks (no integration test exercises the router rejection path)

Additional validation details:

• Unit tests cover rejection detection, backoff and the attempt limit.

Deployment and compatibility

None. Responses without the header behave as before, so the change is safe with the pre-v1.0 serving stack.

Documentation

None needed: internal provider behavior.

Checklist

• I added or updated tests for changed behavior, or explained why tests are not needed.
• I updated user-facing and API documentation where necessary.
• My commits follow the Conventional Commits specification.
• Every commit includes a Signed-off-by trailer (git commit -s).

AI assistance:

• Tool: Claude Code
• Model: Claude Opus 5.5
• Used for: implementation, tests and this description
• Human verification: reviewed the change, ran the test suite and deployed the image to the dev cluster