Repository navigation
fix(swe-smith): stop command descendants on timeout - #698
Open
Zhengyang Liu (liuzhengyang699) wants to merge 2 commits into
Open
Zhengyang Liu (liuzhengyang699) wants to merge 2 commits into
Zhengyang Liu (liuzhengyang699) wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the changes that terminate timed-out command process groups to prevent lingering repository modifications.
No open issues 🎉
What changed in this PR
Addresses #697 by stopping timed-out SWE-smith command process groups, preventing child processes from continuing to modify the repository during later actions or evaluation.
Changes:
- Starts commands in separate POSIX sessions and kills their process groups on timeout, preserving exit code 124 and discarding partial output.
- Adds regression tests for child and grandchild cleanup, inherited output pipes, and existing output/error behavior.
- Documents command and evaluation timeout behavior in Linux containers.
| File | Description |
|---|---|
tests/examples/test_swe_smith_command_timeout.py |
Adds process-tree timeout regressions and output/error handling tests. |
examples/swe_smith/agents/smith_agent.py |
Terminates process groups on timeout, reaps the shell, and closes captured pipes. |
docs/75-example-coding-agent.md |
Documents timeout settings and process-group termination. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #697.
A timed-out shell action can leave its child commands running.
_run()returns 124 after killingbash, but a surviving Python process can still modify/testbedduring the next turn. The same helper runs pytest evaluation, so test workers can survive its timeout too.Start each command in a new POSIX session and kill its process group on timeout. Reap the shell and close the captured pipes before returning the existing timeout message and exit code. Partial timeout output remains discarded without decoding, so arbitrary command bytes cannot turn a timeout into a startup error. Normal stdout/stderr and return-code handling are preserved.
The regressions launch real child and grandchild processes, including a case where the shell exits first but descendants retain its output pipes. After timeout, releasing those processes must not allow any late file writes. The two process-tree cases fail on unchanged main. Additional tests cover normal output, startup failure, an exit race, and non-UTF-8 timeout output. The docs describe command/evaluation timeout behavior in the Linux repository containers.
Validation
uv run --locked --no-sync pytest -q tests/examples/test_swe_smith_command_timeout.py— 6 passed.uv run --locked --no-sync pytest -v --durations=20 tests— 247 passed, 4 skipped.uv run --locked --no-sync pyright— 0 errors, 0 warnings.uv run --locked --no-sync pre-commit run --all-files --show-diff-on-failure— passed; the new test file was also checked explicitly.uv run --locked --no-sync ruff check .,uv run --locked --no-sync ruff format --check ., anduv run --locked --no-sync python scripts/check_headers.py— passed.uv run --locked --no-sync mkdocs build --strict,uv build --no-sources, andgit diff --cached --check— passed.The environment was installed with
uv sync --frozen --no-default-groups --extra dev --group dev --group verl-cpu --group docs.This was found during a follow-up audit of the coding example with AI assistance. AI assistance was also used for the fix, tests, and review. Validation used CPU subprocess tests; no SWE-smith dataset rollout or GPU training was run.