fix(orchestrator): don't report a declined task as a started task - #1216
Open
Gilbert09 wants to merge 1 commit into
Open
fix(orchestrator): don't report a declined task as a started task#1216Gilbert09 wants to merge 1 commit into
Gilbert09 wants to merge 1 commit into
Conversation
The offer for a runner-seeded task is answered at seed time, but the answer was applied inside `runTask` — which the executor calls only after it marks the task running and fires `orchestrator task started`. A declined task therefore reported a start no agent ever ran, and then a skip, so the declines landed on both sides of every rate measured against starts. Apply the answers just before the drain instead. Everything upstream is untouched: the planner still sees the task in the graph, the deferral pass and the sink-closure check run against the same queue, and the skip still carries the reason that caused it. Only the drain sees the task already terminal, so it never starts it. Generated-By: PostHog Desktop Task-Id: 39495583-4ca3-4878-a7f1-c925bd1941ff
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The seeded warehouse task is offered, not imposed: the notice is answered
seconds into the run, and the work is deferred to the end of the queue. The
answer was applied inside
runTask— but the executor marks a task running,and fires
orchestrator task started, before it hands the task over.So a task the user declined reported a start no agent ever ran, followed by a
skip. Declines then landed on both sides of every rate measured against
starts: in the denominator as a task that began, and in the numerator as a
task that was skipped. In the current window that is a double-digit number of
runs, and it is why "started" exceeds "offer accepted" in the funnel — the
headline skip rate for this step is inflated by construction, on the one step
whose most common outcome is a decline.
Why: the warehouse-task funnel's skip rate is the metric being used to
judge this flow, and it cannot be read while declines are counted as starts.
Changes
Apply the seed-time answers to the queue just before the drain, in a small
exported helper, instead of on the far side of the executor's start.
Everything upstream is untouched: the planner still sees the task in the
graph, the deferral pass and the sink-closure check run against the same
queue, and a declined task is still skipped rather than dropped, still
carrying the skip reason and the handoff prose the report reads. Only the
drain sees it already terminal, so it never starts it. The queue panel now
shows the step as skipped from the moment the user skips it, rather than
leaving it queued until the drain reaches it.
Test plan
New unit tests around the helper: a declined task reaches
skippedwith nostarttransition, the reason distinguishes a decline from an unanswerednotice, the handoff keeps the label and wording the report depends on, an
accepted task is left pending for the drain, and an answer for a task no
longer in the queue is ignored.
pnpm build && pnpm test && pnpm lintall green (163 files, 2642 tests).Created with PostHog Desktop