Repository navigation
π fix(dashboard): Let hive decide preview says why it failed instead of "hive unavailable" - #16782
Conversation
β¦of "hive unavailable"
The preview answered every failure as 502 β a PR merged since the row
loaded, an intermittent GitHub 500 on check-runs β and the hosted
Ingress intercepts 502/503 and swaps the body for its generic outage
page, so the owner read "hive unavailable" for a hive that was up.
- retry transient GitHub failures twice with backoff before giving up
- answer not-open as 409 and unrecoverable reads as 424, never 502/503,
with {reason, retryable, attempts, elapsed_ms, hive, hint}
- log a Warn with the real error on every failed plan
- the dialog names the hive, explains the failure (merged since the row
loaded / GitHub failed after N attempts / no GitHub client / gateway
502), hides Retry when it cannot help, and says Let hive do that
stays off until a preview loads
Refs #16776
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Looks correct to me. I read the handler, the classifier and the helpers it leans on at the PR head rather than the diff alone:
api_repo_item_decide_delegated_plan.go:371returnsgatherDelegatedInput's error unwrapped, soerrors.As(err, ¬Open)indelegatedPlanFailureForreaches the*delegatedItemNotOpenErrorbuilt atapi_repo_item_decide_delegated.go:637β 409, matchingTestDelegatedPlanNotOpenIsConflictNotBadGateway.pkg/github/github_retryable.go:23-45(IsRetryableError) useserrors.Is/As, so the%w-wrapped check-runs 500 is classified retryable and the loop inplanDelegatedDecisionWithRetrystops atdelegatedPlanRetries(2 extra attempts, 750msβ1.5s). On ctx cancellationdelegatedPlanRetrySleepreturnsctx.Err()and the loop exits with the original error β no spin, no lost cause.- The non-502/503 choice is justified by what the tree says:
api_auth.go:312andapi_contribute.go:1190already document the hosted Ingress swallowing those statuses, and the newdelegatedPlanFailure.statusis only ever 409 or 424. tracker-actions.jsstageTrackerDelegatePlanFailuretreats a 502β504 with noreason(gateway HTML βr.json()fails β{}) asgateway, andstageTrackerDelegatePlanRetryablewithholds Retry fornot-open/config, consistent withRetryable:falseon the Go side.
The body's four root-cause claims each correspond to a concrete change in the diff; no gap between narrative and code.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Read the Go and JS changes against the tree at What I verified
One info-level note (not blocking) The comment in |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Problem
On the hosted dashboard, Let hive decide on a PR row opened the preview and immediately showed Could not preview #16776 (hive unavailable); Retry failed the same way. Seen on #16736 earlier too, several times a day.
Root cause (verified live on
hosted-available-oke-11-placeholder-r05x)handleRepoItemDecideDelegatedPlananswered every planning failure as 502 (src/pkg/dashboard/api_repo_item_decide_delegated_plan.go,jsonError(w, err.Error(), http.StatusBadGateway)), including non-transport ones:hivecommons/hive#16736 is not open(merged after the row loaded) andreading checks for β¦#16776: GET β¦/check-runs?per_page=100: 500 [](api.github.com's intermittent 500 on a 70-check head β reproduced 1-in-3 withgh api; the spoke's logs show the same 500s on check-runs and commit status for that head at the time).hive-api-xhrIngress hascustom-http-errors: "401,502,503"(src/pkg/hub/saas_provision.go), so ingress-nginx intercepted the hive's 502 and replaced its JSON body withsrc/deploy/k8s/error-pages.yaml's{"ok":false,"error":"hive unavailable",β¦}. The ingress access log shows upstream502with ~4 s upstream time on each click β the hive answered; the gateway discarded its answer. Same trap as π bug: lack of SSO looks like crashed applicationΒ #16652's SSO 503.Calling the endpoint on the pod directly (port-forward + proxy proof) returned
200for #16776 and502 {"error":"hivecommons/hive#16736 is not open"}for #16736 β the hive was up throughout.Fix
github.IsRetryableError: 5xx, 429, rate limit, timeout) twice with 750 ms β 1.5 s backoff before giving up.{ok:false, error, reason, retryable, attempts, elapsed_ms, hive, hint}.Tests
TestDelegatedPlanRetriesTransientGitHubFailureβ one check-runs 500 is retried to a 200; past the budget β 424reason: github,attempts: 3, retryable.TestDelegatedPlanNotOpenIsConflictNotBadGatewayβ closed item β 409reason: not-open.TestDelegatedPlanFailureStatusesAreNeverInterceptedβ no classification yields 502/503.TestStageTrackerDelegatePreviewSkeletonPrefetchβ the gateway / not-open / github / network renderings and the Retry rule.Refs #16776.