Skip to content

fix(runner): retry sandbox teardown failures - #370

Open
aliasfoxkde wants to merge 1 commit into
mainfrom
fix/runner-container-teardown-20261105
Open

aliasfoxkde wants to merge 1 commit into
mainfrom
fix/runner-container-teardown-20261105

Conversation

@aliasfoxkde

Copy link
Copy Markdown
Owner

Summary

  • Keep failed sandbox destruction handles in a bounded cleanup queue instead of dropping them after one failed or timed-out attempt.
  • Retry exact-container teardown with capped exponential backoff, at most two concurrent reapers, and idempotent handling for already-removed containers.
  • Route timeout, cancellation, shutdown, workspace release, and pool release through the same cleanup path.
  • Replace the timeout integration test's manual cleanup with an assertion that the job container disappears automatically; add equivalent cancellation coverage.

Validation

  • cargo test --offline -p gitforge-runner --lib — 127 passed, 2 ignored.
  • Fedora Podman integration: test_real_docker_job_timeout_reaps_sandbox — passed.
  • Fedora Podman integration: test_real_docker_job_cancel_reaps_sandbox — passed.
  • cargo test -p gitforge-sandbox --lib — 94 passed, 3 ignored.
  • Strict clippy for gitforge-runner and gitforge-sandbox — passed.
  • cargo fmt --all -- --check and git diff --check — passed.

Refs #277. This PR addresses in-process teardown retries. Persistent cleanup recovery after abrupt runner-process death and pressure-aware job admission remain open follow-up work in #277.

@coderabbitai

coderabbitai Bot commented Nov 5, 2026

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 36a152ec-c29d-4dad-87dc-e049334931bd
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

🤖 AI Code Review Results

AI review skipped: provider 'anthropic' is not configured; deterministic gates remain authoritative.

✅ No critical issues found.

AI Code Review powered by GitForge

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant