Skip to content

🐛 bug(relay): #7886 note collection races omp's end-of-turn advisor delivery — verdict fast path finalizes ~1 s before the notes render, so they are dropped again (utah#218) #7907

Closed

Description

@Danathar

What happened

Same contributor and build as the #7886 verification (omp container mode, Sonnet 5 medium + Opus 5 xhigh advisor, relay from the :latest image carrying #7886, hub 37522cf).

projectbluefin/utah#169 → PR projectbluefin/utah#218. After the one #7759 follow-up the agent printed its second HIVE_VERDICT: complete. Two ⟦concern⟧ notes were on the pane directly under that verdict — one of them saying "Skip the rebase. Rewriting ffeca63 solely to change 'via omp' → 'via GitHub Copilot' needs a …" — and the relay finalized without recording either of them. No recording them (#7879) line, no PR comment, nothing in the summary. The same relay had done exactly that 12 minutes earlier for chairlift#187 (3 notes recorded and posted).

Why

omp queues advisor notes generated mid-turn and appends them to the transcript at the end of the agent's turn, in the same instant as the agent's final message. The session JSONL for this task shows it exactly:

21:52:36.313Z  assistant  (tool call)
21:52:36.660Z  toolResult main false https://github.com/projectbluefin/utah/pull/218
21:52:40.438Z  assistant  "PR #218 confirmed open against `main` … HIVE_VERDICT: complete — …"
21:52:40.438Z  custom_message  <advisory severity="concern"> Correction to my last note: index.md has no per-skill date …
21:52:40.438Z  custom_message  <advisory severity="concern"> Bound this: …
21:52:40.439Z  custom_message  <advisory severity="concern"> Skip the rebase. Rewriting ffeca63 solely to change "via omp" → "via GitHub Copilot" …
21:52:40.439Z  custom_message  <advisory severity="concern"> Disregard my …

The relay's verdict watch (VERDICT_WATCH_INTERVAL_MS = 5000, bin/contributor-relay.js:280) saw the verdict line while the final message was still streaming and ran the finalizing tick immediately:

2026-09-20T21:52:39  Task …utah-169…: HIVE_VERDICT: complete is on the pane — running the progress tick now … (#7841)
2026-09-20T21:52:40  Detected PR for …: https://github.com/projectbluefin/utah/pull/218
2026-09-20T21:52:40  Task …utah-169… completed — signal=verdict (HIVE_VERDICT: complete)

verdictWatchTick() (bin/contributor-relay.js:4431-4449) calls progressTick() the moment detectCompletionVerdict() matches, and the #7886 collection (postVerdictNoteBlocks(paneScanLines, …) at :5237-5243) runs on the pane captured in that tick. The notes rendered ~1 s later, under the verdict, on a pane nobody looked at again. My own pane sampler at 21:52:42Z shows both ⟦concern⟧ blocks under the verdict.

Whether a task's late notes get recorded therefore depends on the phase of a 5 s timer. chairlift#67/#187 landed on the good side of it; utah#169/#218 did not. (Verdicts printed inside TASK_GRACE_PERIOD_MS (:3899, 180 s) are unaffected — the tick refuses to judge until the grace ends, which is why the first verdicts of short tasks always picked up their notes tonight; that is an accident of the grace period, not a settle.)

There is a second, longer window this leaves open: the advisor's review of the final turn itself (Opus xhigh took 30–100 s per review tonight; in this task its last two outputs, 21:52:38Z and 21:52:43Z, happened to be "Silent"). A note from that review is generated after finalization and has nowhere to go at all. That one is harder — bounding a wait on advisor activity — and may not be worth it; the end-of-turn delivery race is cheap to close and is the one that actually lost notes.

What it cost

The dropped "Skip the rebase" note was about the agent rewriting an already-pushed commit solely to change its attribution trailer from via omp to via GitHub Copilot (utah's AGENTS.md prescribes the Copilot wording). The agent had already done it by the time the note was delivered, and the reviewer of #218 will never see the advisor's objection.

Suggested fix

On the finalizing tick, if the verdict line was first seen less than one watch interval ago, re-capture once before collecting: i.e. verdictWatchTick() records verdictFirstSeenAt, and the collection at :5237 uses a fresh captureTmuxLines() taken at least ~2 s after that (or simply defers finalization to the next 5 s watch tick when Date.now() - verdictFirstSeenAt < VERDICT_WATCH_INTERVAL_MS). Bounded at one extra interval, no new turns, and it makes the "record notes under the final verdict" guarantee independent of timer phase.

Test shape: pane that shows the verdict alone on capture N and the verdict + ⓘ Advisor 1 note block on capture N+1 → the notes must be in unaddressedNotes.

Relay bin/contributor-relay.js at v4 dc698c8 for line numbers; behaviour observed on the :latest container image (relay carrying #7886) at 2026-09-20T21:52Z.

Activity

  1. Danathar commented on Sep 20, 2026

    @Danathar
    ContributorAuthor

    Second occurrence, next task, same contributor: projectbluefin/chairlift#146 → PR chairlift#188.

    21:56:55.435Z  assistant       "PR #188 open, base main … HIVE_VERDICT: complete — …"
    21:56:55.436Z  custom_message  <advisory severity="nit">     Don't repeat the issue's "Renovate already tracks this repository" claim in the PR body — there is no renovate.json …
    21:56:55.436Z  custom_message  <advisory severity="nit">     File tools don't inherit your shell's cd …
    21:56:55.436Z  custom_message  <advisory severity="concern"> Amend before pushing: AGENTS.md:448-452 requires an AI-authored commit to carry both trailers …
    21:56:55.436Z  custom_message  <advisory severity="concern"> CONTRIBUTING.md:85 makes `make ci` required before opening a PR … You ran only -run TestWorkflowActionsUseImmutableCommitSHAs …
    

    Relay: 21:56:55 HIVE_VERDICT: complete is on the pane — running the progress tick now → 21:56:55 completed — signal=verdict. Four notes delivered 1 ms after the final message; relay had already finalized. Nothing recorded, nothing on the PR, and — the part #7907's title understates — the #7759 follow-up did not fire either: this was the task's first verdict, two ⟦concern⟧ notes were under it within a millisecond, and the agent never got the one turn #7759 promises, because the relay's capture predates the delivery. The make ci concern was actionable (a required gate skipped before opening the PR).

    So the race defeats both halves of the post-verdict design, not just the recording half. Tonight's tally on verdicts printed after the 180 s grace: caught 2 (utah#169 first verdict, chairlift#67/#187 second), missed 2 (utah#218, chairlift#188). Task assigned 21:52:5xZ, so grace ended ~21:55:5x, ~1 min before the verdict — consistent with the phase explanation: verdicts inside the grace period are always safe, verdicts after it are a coin flip on the 5 s watch phase versus omp's end-of-turn delivery.

    Same suggested fix — one re-capture ≥ 2 s after the verdict line is first seen, before both the #7759 decision and the #7886 collection.

  2. github-actions commented on Sep 20, 2026

    @github-actions
    Contributor

    @Danathar this issue was closed by @clubanderson as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions