Skip to content

fix(cloud-agent): handle stream churn - #57079

Merged
tatoalo merged 3 commits into
masterfrom
chore/cloud-agent/handle-stream-churn
Apr 30, 2026
Merged

tatoalo merged 3 commits into
masterfrom
chore/cloud-agent/handle-stream-churn

Conversation

@tatoalo

@tatoalo tatoalo commented Apr 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

cloud task stream relays could mark the Redis stream complete when the sandbox /events SSE connection closed normally, even if the task run was still non-terminal.
ph-code then saw the stream end, fetched an in_progress run state, and reconnected, which can produce repeated stream churn.

Changes

  • mark the Redis task-run stream complete after a normal sandbox stream close when the task run is already terminal
  • reconnect on normal sandbox stream closure while the run is still non-terminal, and keep the existing retry budget behavior for exhausted reconnect attempts

@greptile-apps

greptile-apps Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
products/tasks/backend/temporal/process_task/activities/tests/test_relay_sandbox_events.py:337-364
**Missing unit tests for `_is_keepalive_event`**

`_is_keepalive_event` is a new exported helper analogous to `_is_session_update` and `_is_end_of_turn`, both of which have dedicated parameterized test classes. There are no direct unit tests for `_is_keepalive_event`, leaving the new reset-skipping logic (`if not _is_keepalive_event(event_data): reconnect_count = 0`) covered only implicitly by the integration test. A small parameterized suite (e.g. `{"type": "keepalive"} → True`, `{"type": "notification"} → False`, `{} → False`) would make the contract explicit and consistent with the rest of the file.

### Issue 2 of 2
products/tasks/backend/temporal/process_task/activities/tests/test_relay_sandbox_events.py:337-364
**`_mark_complete_if_run_is_terminal` `DoesNotExist` branch is untested**

The existing `_mark_error_unless_run_is_terminal` has a dedicated test for the `DoesNotExist` case (see `test_missing_run_marks_stream_error_on_relay_error`). The new `_mark_complete_if_run_is_terminal` has the analogous branch — `mark_error("Task run not found")` + `return True` — but no test covering it. If `TaskRunModel.DoesNotExist` is raised during a normal stream closure, the relay currently stops (returns `True`) and marks the stream errored, which may be surprising. A test would pin that behaviour and catch future regressions.

Reviews (1): Last reviewed commit: "fix(cloud-agent): handle stream churn" | Re-trigger Greptile

@tatoalo tatoalo self-assigned this Apr 30, 2026
@tatoalo
tatoalo requested a review from a team April 30, 2026 09:16
Comment thread products/tasks/backend/temporal/process_task/activities/relay_sandbox_events.py Outdated
@tatoalo
tatoalo enabled auto-merge (squash) April 30, 2026 09:47
@tatoalo
tatoalo merged commit 90cfa32 into master Apr 30, 2026
214 checks passed
@tatoalo
tatoalo deleted the chore/cloud-agent/handle-stream-churn branch April 30, 2026 10:12
@deployment-status-posthog

deployment-status-posthog Bot commented Apr 30, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-04-30 10:32 UTC Run
prod-us ✅ Deployed 2026-04-30 10:45 UTC Run
prod-eu ✅ Deployed 2026-04-30 10:50 UTC Run

sakce pushed a commit that referenced this pull request Apr 30, 2026
sakce pushed a commit that referenced this pull request May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants