Skip to content

[scanner] ci: gosec (insecure patterns) repeatedly loses its self-hosted runner (6+ runs, no code cause) #16040

Closed

Description

@hivecommons-hive

Summary

The gosec (insecure patterns) job in the Go Security Analysis workflow has failed at least six times on 2026-10-31 with the same annotation and no code cause:

The self-hosted runner lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.

In every case the job logs are BlobNotFound (the runner died before uploading), the sibling jobs in the same workflow run (govulncheck, golangci-lint, action pins resolve, NOTICE matches the module graph) completed green, and the same PR head passed gosec on a later or earlier attempt where one exists.

Evidence

Ref Run Job
v5 @ de6127e — runner lost (bead gh-v5-gosec)
#16014 attempt 1 41198755355 124530335184
#16014 attempt 2 41200489328 124538397043
#16007 41197691718 124527124413
#16031 41200277080 —
#16027 41201935213 124542607459

Meanwhile v5 runs 41199008359 → 41202519218 (six consecutive) are green, so this is not a code regression on the default branch — it is the gosec job specifically losing its self-hosted runner, repeatedly.

Why this needs a signal

  • gosec (insecure patterns) is a required status check, so a runner loss blocks the merge of an otherwise-green PR (🌱 chore(rfcnotify): add tests for card.go helper functions #16014 is stuck on it right now with every other check passing).
  • Agents cannot gh run rerun (out of tier), so a PR hit by this has no path back to green other than a new push.
  • Only gosec is affected among the four jobs in the workflow, which points at that job's resource profile (gosec on the full ./... tree is memory-heavy) rather than at the runner pool in general.

Asks

  1. Capture the runner-side evidence (OOM / eviction / network) for one of the job ids above so the cause is known rather than guessed.
  2. Either give the gosec job a retry/rerun-failed-jobs guard for the "runner lost communication" annotation, or move it to a runner class with more memory, or scope gosec per package shard like the tests.
  3. Until then, treat gosec (insecure patterns) failures carrying this annotation as infra, not as a code verdict.

Filed from the scanner lane during CI repair of #16014.

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Activity

  1. added
    kind/testCategorizes issue or PR as related to testing.
    agent/scannerCreated or modified by the scanner agent
    ciApproved by a Hive merger/owner for auto-merge on green CI
    needs-signalHive needs a better CI or test signal before continuing
    on Oct 31, 2026
  2. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Oct 31, 2026
  3. hivecommons-hive commented on Oct 31, 2026

    @hivecommons-hive
    ContributorAuthor

    🔒 Claimed by @Karibhou (contributor on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 30m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  4. hivecommons-hive commented on Oct 31, 2026

    @hivecommons-hive
    ContributorAuthor

    🔓 Claim by @Karibhou released (lease revoked).

  5. added 2 commits that reference this issue on Oct 31, 2026
    f300596
    20eb629
  6. added a commit that references this issue on Nov 2, 2026
    1e83f1f
  7. github-actions commented on Nov 2, 2026

    @github-actions
    Contributor

    @hivecommons-hive[bot] this issue was closed by @kubestellar-prow[bot] as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/scannerCreated or modified by the scanner agentciApproved by a Hive merger/owner for auto-merge on green CIhelp wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.kind/testCategorizes issue or PR as related to testing.needs-signalHive needs a better CI or test signal before continuing

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions