Skip to content

fix(swe-smith): stop command descendants on timeout - #698

Open
Zhengyang Liu (liuzhengyang699) wants to merge 2 commits into
microsoft:mainfrom
liuzhengyang699:fix/swe-smith-command-timeout
Open

Zhengyang Liu (liuzhengyang699) wants to merge 2 commits into
microsoft:mainfrom
liuzhengyang699:fix/swe-smith-command-timeout

Conversation

@liuzhengyang699

Copy link
Copy Markdown

Fixes #697.

A timed-out shell action can leave its child commands running. _run() returns 124 after killing bash, but a surviving Python process can still modify /testbed during the next turn. The same helper runs pytest evaluation, so test workers can survive its timeout too.

Start each command in a new POSIX session and kill its process group on timeout. Reap the shell and close the captured pipes before returning the existing timeout message and exit code. Partial timeout output remains discarded without decoding, so arbitrary command bytes cannot turn a timeout into a startup error. Normal stdout/stderr and return-code handling are preserved.

The regressions launch real child and grandchild processes, including a case where the shell exits first but descendants retain its output pipes. After timeout, releasing those processes must not allow any late file writes. The two process-tree cases fail on unchanged main. Additional tests cover normal output, startup failure, an exit race, and non-UTF-8 timeout output. The docs describe command/evaluation timeout behavior in the Linux repository containers.

Validation

  • uv run --locked --no-sync pytest -q tests/examples/test_swe_smith_command_timeout.py — 6 passed.
  • uv run --locked --no-sync pytest -v --durations=20 tests — 247 passed, 4 skipped.
  • uv run --locked --no-sync pyright — 0 errors, 0 warnings.
  • uv run --locked --no-sync pre-commit run --all-files --show-diff-on-failure — passed; the new test file was also checked explicitly.
  • uv run --locked --no-sync ruff check ., uv run --locked --no-sync ruff format --check ., and uv run --locked --no-sync python scripts/check_headers.py — passed.
  • uv run --locked --no-sync mkdocs build --strict, uv build --no-sources, and git diff --cached --check — passed.

The environment was installed with uv sync --frozen --no-default-groups --extra dev --group dev --group verl-cpu --group docs.

This was found during a follow-up audit of the coding example with AI assistance. AI assistance was also used for the fix, tests, and review. Validation used CPU subprocess tests; no SWE-smith dataset rollout or GPU training was run.

@liuzhengyang699
Zhengyang Liu (liuzhengyang699) marked this pull request as ready for review November 7, 2026 06:32
Copilot AI balanced review requested due to automatic review settings November 7, 2026 06:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the changes that terminate timed-out command process groups to prevent lingering repository modifications.

No open issues 🎉

What changed in this PR

Addresses #697 by stopping timed-out SWE-smith command process groups, preventing child processes from continuing to modify the repository during later actions or evaluation.

Changes:

  • Starts commands in separate POSIX sessions and kills their process groups on timeout, preserving exit code 124 and discarding partial output.
  • Adds regression tests for child and grandchild cleanup, inherited output pipes, and existing output/error behavior.
  • Documents command and evaluation timeout behavior in Linux containers.
File Description
tests/​examples/​test_swe_smith_command_timeout.py Adds process-tree timeout regressions and output/error handling tests.
examples/​swe_smith/​agents/​smith_agent.py Terminates process groups on timeout, reaps the shell, and closes captured pipes.
docs/​75-example-coding-agent.md Documents timeout settings and process-group termination.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SWE-smith command timeouts leave child processes running

2 participants