Skip to content

fix: make automation timeout tests deterministic under CI load - #20

Merged
xhwSkhizein merged 1 commit into
mainfrom
fix/automation-timeout-test-flake
Jul 24, 2026
Merged

fix: make automation timeout tests deterministic under CI load#20
xhwSkhizein merged 1 commit into
mainfrom
fix/automation-timeout-test-flake

Conversation

@xhwSkhizein

@xhwSkhizein xhwSkhizein commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • PR fix: chunk oversized extension WebSocket responses #19 merged before the CI flake fix landed; this follow-up ports that commit onto main.
  • Make automation timeout unit tests pin the task-execution remaining budget instead of racing a 50ms whole-run wall clock that can expire during pre-task setup on GitHub runners.

Test plan

  • uv run pytest tests/unit/test_automation_service.py -q
  • Confirm CI no longer fails test_automation_service_marks_run_failed_on_timeout

Made with Cursor

Summary by CodeRabbit

  • Tests
    • Improved automation timeout test coverage by standardizing slow-task setup and simulated time limits.
    • Updated scenarios to more reliably validate task execution behavior under constrained remaining time.
    • Reduced duplicated test logic, making timeout-related test cases clearer and more consistent.

Pin task-execution remaining budget instead of relying on a 50ms
whole-run wall clock that flaked when pre-task setup burned the quota.

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: dff79479-587e-4564-872d-4cd764a40426

📥 Commits

Reviewing files that changed from the base of the PR and between 18aa4e1 and d5800c0.

📒 Files selected for processing (1)
  • tests/unit/test_automation_service.py

📝 Walkthrough

Walkthrough

The automation service timeout tests now centralize slow-task creation and stage-specific timeout simulation. Both scenarios use a 30-second overall timeout while forcing a tight task_execution budget.

Changes

Automation timeout test refactoring

Layer / File(s) Summary
Centralize timeout simulation helpers
tests/unit/test_automation_service.py
Adds shared helpers for creating slow task files and patching the task-execution timeout budget.
Apply helpers to timeout scenarios
tests/unit/test_automation_service.py
Updates timeout failure and non-retry tests to use the helpers and a larger overall timeout.
Estimated code review effort: 2 (Simple) ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description has summary and test plan, but omits required template sections like Related issue, Type of Change, Checklist, and Architectural Boundaries. Add the missing template sections, including related issue, change type, checklist items, architectural boundary confirmation, and screenshots/output if applicable.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: making automation timeout tests deterministic under CI load.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/automation-timeout-test-flake

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@xhwSkhizein
xhwSkhizein merged commit e483438 into main Jul 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant