[quality] make pkg/dashboard copilot SDK-helper probe tests hermetic - #7535
Merged
Merged
Conversation
![hive-quality[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
TestProbeCopilotModelsSDK_AbsentHelperYieldsSentinel and
TestProbeCopilotModelsSDK_HelperNotInstalled exec'd whatever lived at the
production helper path (/usr/local/bin/copilot-models.mjs). They handled two
states — helper absent (assert sentinel) or helper works (skip) — but not the
third: helper installed without stored copilot auth, the normal state on live
agent hosts, where the helper exits 1 ('Not authenticated'), the sentinel
check fatals, and the entire pkg/dashboard suite goes red.
Fix: copilotSDKHelperPath becomes a var with an in-package test seam
(setCopilotSDKHelperPathForTest, mirroring knowledge.SetBaseDirForTest).
Both tests repoint it at a nonexistent temp path so the absence assertion
holds on every host, and a new TestProbeCopilotModelsSDK_FailingHelperIsNotAbsent
covers the present-but-failing exec path (#7365's signature) with a stub
script, hermetically. No production behavior change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: hive-quality[bot] <hive-quality[bot]@users.noreply.github.com>
Contributor
Author
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will automatically remove the |
kubestellar-prow
Bot
added
the
dco-signoff: yes
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
kubestellar-prow
Bot
added
the
size/M
kubestellar-hive
Bot
added
quality
testing
agent/quality
Contributor
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
clubanderson
added a commit
that referenced
this pull request
Sep 18, 2026
Carries v4 e74d45e..201c69c (v4.54.2, v4.55.0, v4.55.1) into v5, including #7510 (cmd/hive domain extraction), #7532 (manager.go split), #7527/#7528 (review routing + breadth-first slots), #7523 (token-access audit backport of v5 #6303), #7515 (branch-protection facts on blocked PR pills), #7535, #7539, #7520. Resolution notes: - cmd/hive, pkg/agent: declaration-level 3-way merge against base ff7630e; v5 seams (spokeStatePath, SetHubPushedDashboardURL, applyModeUnscheduledAlert, mutation boundary, runDuplicateSweepIfDue) preserved. F24 tombstone intact. - pkg/github/client.go: v5 fetchPRs signature (RepoPRBreakdown) + v4 BaseRef/prBaseRef; added Client.requiredStatusCheckContexts shim so v4's protection_facts.go compiles against v5, where the sweep lives in pkg/github/automerge. - pkg/github/token_access_audit{,_test}.go: v4 side (its backport of v5's own #6303, now using testutil.Eventually). - pkg/github/automerge/f3_trusted_merger_source_test.go: v4's f3ReadPackage path adjusted one level deeper for v5's subpackage. - pkg/review/prompts.go: v4 routing instruction + mentionableAuthor added; v5's #6212 deletion of dead BuildPerspectivePrompts / BuildSequentialPrompt kept. - pkg/agent tests: v4's normalizeModelName cases rewritten against v5's normalizeModelNameForBackend (v5 removed the wrapper in #5795). - docs/knowledge-curator.md: v5 promotion-only body + v4's knowledge.git_sources section (code exists on v5). roadmap.md: v4 rows. api-reference.md: v5 table, citations re-resolved (--fix). - .github/workflows/v2-ci.yml: v5 UID-isolation step + v4's fuller audit-log step comment. Refs #7463 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Andy Anderson <andy@clubanderson.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Improvement
Makes the copilot SDK-helper probe tests in
pkg/dashboardhermetic.Claimed ground (exactly):
src/pkg/dashboard/cli_models.go(test seam only —copilotSDKHelperPathconst→var +setCopilotSDKHelperPathForTest, mirroring theknowledge.SetBaseDirForTestconvention),src/pkg/dashboard/cli_models_sdk_severity_test.go,src/pkg/dashboard/cli_models_sdk_test.go, plus achangelog.dentry. Disjoint from open PRs #7523 (pkg/dashboard/api.go), #7510, #7527, #7520.What was broken:
TestProbeCopilotModelsSDK_AbsentHelperYieldsSentinelandTestProbeCopilotModelsSDK_HelperNotInstalledexec'd whatever lived at the production path/usr/local/bin/copilot-models.mjs. They handled helper-absent (assert sentinel) and helper-works (skip), but not the third state — helper installed without stored copilot auth, the normal state on live agent hosts — where the helper exits 1 (Not authenticated), the sentinel check fatals, and the wholepkg/dashboardsuite goes red.HelperNotInstalledalso ran a real ~3s network probe when the helper existed.Change:
TestProbeCopilotModelsSDK_FailingHelperIsNotAbsentcovers the previously CI-untestable exec branch — helper present but exiting nonzero (the 🐛 dashboard model list disagrees with the CLI — Copilot SDK discovery always fails (missing platform package), silently falling back to the chat-completions catalog #7365 log signature) must NOT matcherrCopilotSDKHelperAbsentand must fold stderr into the error — via a stub.mjs, skipping only ifnodeis absent.Verified: full
pkg/dashboardsuite green (was red on this host),go vetclean, on v4 @ e74d45e.Related Issue
Closes #7534
Filed by quality agent (hold-gated mode). Human review required.
— hive: agent=quality backend=copilot model=claude-fable-5