Skip to content

feat(resilience): Robust Cross-Platform Process-Tree Orphan Sweeper #16

Description

@sdageltc

Summary & Motivation

Worker adapters frequently spawn external tool processes, compilers, test runners (pytest, npm test, cargo build), or language servers. If a contract times out or is aborted via Ctrl+C / SIGINT, child and grandchild processes can detach and become runaway orphans consuming 100% CPU in the background.

While orchestrator/verifier.py contains basic process termination, we need a unified, resilient Cross-Platform Process-Tree Orphan Sweeper across all worker adapters.


Proposed Solution

  1. Unified Process Tracker (orchestrator/process_guard.py):
    • Windows: Uses Job Objects (CreateJobObject, AssignProcessToJobObject) with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. When the parent handle closes, Windows OS kernel automatically terminates all descendants atomically.
    • POSIX (macOS / Linux): Uses process group leadership (os.setpgrp() / preexec_fn=os.setsid) paired with os.killpg(pgid, signal.SIGKILL).
  2. Signal Trap & Cleanup Registry:
    • Maintains an in-memory process registry.
    • Registers atexit and signal.signal handlers (SIGINT, SIGTERM, SIGHUP) to execute immediate sweep before exit.

Acceptance Criteria

  • Grandchild subprocesses spawned by workers or test runners are guaranteed terminated on timeout.
  • Pressing Ctrl+C during a running loop terminates all spawned subprocesses immediately with 0 orphan processes left behind.
  • Tested on macOS (Darwin), Linux, and Windows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureCore control loop and system architectureenhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions