fix: require Cursor terminal success and bounded cleanup - #17
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
🔵 Needs a closer look
It changes low-level process supervision/termination semantics (including platform-specific behavior) where subtle edge cases can have operational impact despite good test coverage.
Pull request overview
Tightens the Cursor Agent provider’s grading contract by requiring a successful result event and a clean harness exit, while ensuring helper processes are cleaned up deterministically via a small supervisor process that owns the POSIX process group through cleanup.
Changes:
- Fork a new
cursor-processsupervisor and require: terminal status observed, exit code0, and explicit cleanup confirmation before returning graded output. - Add extensive Darwin/real-process-style fixtures validating timeouts, stdin failures, supervisor/parent death, helper cleanup, and bounded pipe draining.
- Update packaging, consumer proof, and docs to ensure the supervisor ships beside the provider and is exercised in the packed artifact.
File summaries
| File | Description |
|---|---|
| vite.config.ts | Adds src/cursor-process.ts to the pack entry list so it is emitted into dist/ alongside the provider. |
| src/cursor-provider.ts | Replaces direct harness spawning with a forked supervisor and stricter success/cleanup gating before grading output. |
| src/cursor-process.ts | Introduces the supervisor responsible for owning/killing the process group (POSIX) and direct-child cleanup (Windows). |
| test/cursor-provider.test.ts | Adds process-level fixtures covering exit/status/result invariants, timeouts, cleanup behavior, and supervisor watchdog scenarios. |
| test/consumer.test.ts | Extends packed-consumer proof to assert cursor-process.js is shipped and the packaged provider+supervisor can execute. |
| docs/usage.md | Documents the stricter grading contract and the separate bounded cleanup/pipe-drain deadline behavior. |
| AGENTS.md | Updates packaging invariants to include the new dist/cursor-process.js adjacency requirement and proof expectations. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Problem
Cursor could return a successful result and then exit 7, yet still produce graded output. Timeout also returned after killing only the immediate CLI, leaving helpers alive.
Solution
Require harness exit zero, a successful result event, and completed cleanup before returning output. A small Node supervisor retains ownership of the macOS/Linux process group until it terminates remaining helpers. Execution and pipe draining have separate deadlines, and failures retain their cause and recorded skill/token evidence.
Helpers that deliberately leave the group are outside termination scope; retained pipes produce a bounded error. Windows keeps direct-child cleanup. The supervisor ships beside the provider without adding regular dependencies.
Proof
Darwin real-process fixtures cover failed/signal exits, missing/error results, stdin failure, timeout, helper cleanup after harness exit, parent death, supervisor death, watchdog expiry, and escaped helpers holding pipes. The exit-7 regression failed against the previous provider and passes with this change:
pnpm run verify:fullpassed uncached: 63 source tests and the installed-consumer test. The consumer installs with--ignore-scripts, confirms promptfoo is absent, and executes the packaged provider/supervisor using a fake binary.vp checkand independent Astra medium review passed. No live evaluation or Cursor authentication was used. Linux proof runs in PR CI before merge.