Skip to content

[codebase] scripts/codecov-pr-comment.sh hard-fails when CODECOV_TOKEN missing #164

Description

@BillyOutlast

Problem

scripts/codecov-pr-comment.sh (invoked from .github/workflows/ci.yml:190-196) calls the Codecov GraphQL API to fetch coverage deltas and post them as a PR comment. The script requires CODECOV_TOKEN (a repo secret) and fails hard with FATAL: CODECOV_TOKEN is not set (exit 1) when the secret is empty or unset.

This was silently broken before: the previous fail_ci_if_error: false setting on the codecov upload step hid upload failures but did NOT cover the comment script. After the CI hardening in #163 un-rubber-stamped the workflow, this surfaced as a Test + Coverage job failure on PR #163 (run 30300294689, log line 1656).

Evidence

2026-07-27T19:56:16.2046938Z FATAL: CODECOV_TOKEN is not set
2026-07-27T19:56:16.2046938Z Generate a token at https://app.codecov.io/gh/BillyOutlast/drop/settings/access
2026-07-27T19:56:16.2058846Z ##[error]Process completed with exit code 1.

Context

A CODECOV_TOKEN was added to the repo on 2026-07-27 during the hardening work, so the immediate failure should now resolve. However, the underlying fragility remains:

  • The script is hard-coupled to a single secret
  • A new contributor who forgets to configure CODECOV_TOKEN will see a hard failure that looks like a CI bug
  • The script is also not gated on pull_request events specifically — it runs on push too, where there's no PR to comment on

Goal

Out of scope for the CI hardening workstream (#163). This issue exists to track follow-up cleanup so the codebase itself is reliable independent of CI config.

Suggested Approach

  1. Make the comment script tolerant of missing CODECOV_TOKEN — exit 0 with a warning log instead of fatal.
  2. Gate invocation on github.event_name == 'pull_request' AND a non-empty token (already done in fix(ci): harden CI against rubber-stamping (PRs to develop, continue-on-error, codecov, root-config paths) #163, but consider also gating in the script itself for defense-in-depth).
  3. Investigate whether the script is still needed — Codecov's GitHub Action v5 already posts a PR comment by default; this custom script may be redundant.
  4. If kept, document the script's purpose in its header and add a smoke test.

Acceptance Criteria

  • scripts/codecov-pr-comment.sh exits 0 when CODECOV_TOKEN is unset (with a warning log)
  • Job no longer fails the workflow when secret is missing
  • Decision documented: keep or remove the script
  • If kept, README/header documents how to obtain and configure the token

Provenance

Discovered during the PR #155 adversarial CI audit (umbrella #157). The un-rubber-stamping changes in PR #163 caused the failure to surface for the first time.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions