Finding
TestProbeCopilotModelsSDK_AbsentHelperYieldsSentinel (src/pkg/dashboard/cli_models_sdk_severity_test.go) and TestProbeCopilotModelsSDK_HelperNotInstalled (src/pkg/dashboard/cli_models_sdk_test.go) exec whatever is installed at the hardcoded production path copilotSDKHelperPath = /usr/local/bin/copilot-models.mjs (src/pkg/dashboard/cli_models.go).
They account for two states — helper absent (assert the errCopilotSDKHelperAbsent sentinel) or helper present and working (skip) — but not the third: helper installed without stored copilot auth, which is the normal state on live agent hosts. There the helper exits 1 with Not authenticated, the sentinel assertion fatals, and the entire pkg/dashboard suite goes red:
--- FAIL: TestProbeCopilotModelsSDK_AbsentHelperYieldsSentinel (1.56s)
cli_models_sdk_severity_test.go:46: helper absence did not yield the sentinel:
sdk helper: exit status 1 (stderr: copilot-models: Request models.list failed
with message: Not authenticated. Please authenticate first.)
Reproduced on v4 @ e74d45e. Same class of environmental coupling as the pkg/config K8s-mode hermeticity issue: tests whose verdict depends on host state outside the repo. TestProbeCopilotModelsSDK_HelperNotInstalled additionally runs a real ~3s network-dependent probe when the helper exists.
Recommendation
Give the helper path an in-package test seam (var + setCopilotSDKHelperPathForTest, mirroring the knowledge.SetBaseDirForTest convention), point both tests at a nonexistent temp path, and cover the present-but-failing exec path (the #7365 signature) hermetically with a stub script.
Fix implemented in the linked hold-gated PR (branch quality/test-sdk-helper-hermetic); files claimed: src/pkg/dashboard/cli_models.go (seam only), cli_models_sdk_severity_test.go, cli_models_sdk_test.go. Disjoint from open PRs #7523 (touches pkg/dashboard/api.go) and #7510/#7527/#7520.
Priority
- Impact: medium (blocks trustworthy
go test ./... on agent hosts; masks real dashboard regressions)
- Effort: low
Filed by quality agent (hold-gated mode)
🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 0568735
— hive: agent=quality backend=copilot model=claude-fable-5
Finding
TestProbeCopilotModelsSDK_AbsentHelperYieldsSentinel(src/pkg/dashboard/cli_models_sdk_severity_test.go) andTestProbeCopilotModelsSDK_HelperNotInstalled(src/pkg/dashboard/cli_models_sdk_test.go) exec whatever is installed at the hardcoded production pathcopilotSDKHelperPath = /usr/local/bin/copilot-models.mjs(src/pkg/dashboard/cli_models.go).They account for two states — helper absent (assert the
errCopilotSDKHelperAbsentsentinel) or helper present and working (skip) — but not the third: helper installed without stored copilot auth, which is the normal state on live agent hosts. There the helper exits 1 withNot authenticated, the sentinel assertion fatals, and the entirepkg/dashboardsuite goes red:Reproduced on v4 @ e74d45e. Same class of environmental coupling as the pkg/config K8s-mode hermeticity issue: tests whose verdict depends on host state outside the repo.
TestProbeCopilotModelsSDK_HelperNotInstalledadditionally runs a real ~3s network-dependent probe when the helper exists.Recommendation
Give the helper path an in-package test seam (var +
setCopilotSDKHelperPathForTest, mirroring theknowledge.SetBaseDirForTestconvention), point both tests at a nonexistent temp path, and cover the present-but-failing exec path (the #7365 signature) hermetically with a stub script.Fix implemented in the linked hold-gated PR (branch
quality/test-sdk-helper-hermetic); files claimed:src/pkg/dashboard/cli_models.go(seam only),cli_models_sdk_severity_test.go,cli_models_sdk_test.go. Disjoint from open PRs #7523 (touchespkg/dashboard/api.go) and #7510/#7527/#7520.Priority
go test ./...on agent hosts; masks real dashboard regressions)Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-available-oke-11-placeholder-r05x| SHA:0568735— hive: agent=quality backend=copilot model=claude-fable-5