Skip to content

verify-action-build: require a lock file for every dependency manifest#770

Open
potiuk wants to merge 2 commits intoapache:mainfrom
potiuk:verify-action-build-require-lock-files
Open

verify-action-build: require a lock file for every dependency manifest#770
potiuk wants to merge 2 commits intoapache:mainfrom
potiuk:verify-action-build-require-lock-files

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented Apr 24, 2026

Summary

Every action type — node, python, deno, dart, ruby, go, rust — now fails verification if a dependency manifest (package.json, pyproject.toml, go.mod, …) is found without a matching lock file. Without a lock file, a rebuild of the action pulls whatever transitive versions happen to be latest at build time, and any comparison we do against the committed dist/ becomes noise.

The new analyze_lock_files check runs for every action type, including JS actions — which is the primary case where a missing lock file breaks our rebuild-and-compare workflow. Previously analyze_dependency_pinning ran only in the composite/docker branch and only warned on unpinned deps; nothing required a lock file at all.

Why now

This came out of the #761 (mozilla-actions/sccache-action 0.0.9 → 0.0.10) investigation, which exposed that we have no affirmative check that actions we approve pin their transitive deps. sccache-action happens to ship a lock file (the drift there is a separate toolchain issue, addressed upstream at Mozilla-Actions/sccache-action#252), but the class of "action with no lock file" is wider and equally bad for reproducibility.

Manifest → lock file mappings

Ecosystem Manifest Acceptable lock files
Node package.json package-lock.json / yarn.lock / pnpm-lock.yaml / bun.lock / bun.lockb
Python pyproject.toml uv.lock / poetry.lock / pdm.lock / requirements.txt
Python Pipfile Pipfile.lock
Deno deno.json(c) deno.lock
Dart pubspec.yaml pubspec.lock
Ruby Gemfile Gemfile.lock
Go go.mod go.sum
Rust Cargo.toml Cargo.lock

Heuristics to avoid false positives

  • pyproject.toml with only tool config (ruff / black / mypy / ...) and no [project.dependencies] or [tool.<x>.dependencies] section is skipped.
  • go.mod with no require directives (no third-party deps) is skipped.
  • Rust library crates ([lib] without [[bin]]) skip the Cargo.lock requirement per Cargo convention; binary crates and workspaces do not.
  • Sub-path (monorepo sub-action) manifests fall back to repo-root manifests before flagging missing.
  • Pure composite actions with no manifests at all pass with no check output.

Output example

A new Lock File Presence section appears in the verification output, with one line per detected manifest:

Lock File Presence
  ✓ node: action/package.json → action/package-lock.json
  ⊘ python: pyproject.toml declares no dependencies
  ✗ go: go.mod has no matching lock file (expected one of: go.sum)

And a new Lock file presence row shows in the verification summary table (pass/fail).

Behaviour

  • Pass: every detected manifest has a matching lock file, or all detected manifests were skipped by a heuristic.
  • Fail: at least one manifest is present without a matching lock file. Folded into overall_passed alongside the JS-build and binary-download checks, so the RESULT panel turns red with a specific message.

Test plan

  • 31 new unit tests in test_security.py::TestAnalyzeLockFiles covering each ecosystem, the library / no-deps / no-require heuristics, sub-path handling, and multi-ecosystem aggregation. All 170 tests in the suite pass.
  • Smoke test against a known-good action (e.g. actions/checkout) — should pass.
  • Smoke test against an action without a lock file — should fail with the new message.

Generated-by: Claude Opus 4.7 (1M context)

Every action type — node, python, deno, dart, ruby, go, rust — now fails
verification if a dependency manifest (package.json, pyproject.toml, go.mod,
...) is found without a matching lock file. Without a lock file, a rebuild
of the action pulls whatever transitive versions are latest at build time,
which makes the dist/ we verify non-reproducible.

The new analyze_lock_files check runs for every action type (not just
composite/docker, where analyze_dependency_pinning already runs), since JS
actions are the primary case where a missing lock file breaks our rebuild
comparison.

Recognised manifest-to-lock mappings:
  package.json   -> package-lock.json / yarn.lock / pnpm-lock.yaml / bun.lock
  pyproject.toml -> uv.lock / poetry.lock / pdm.lock / requirements.txt
  Pipfile        -> Pipfile.lock
  deno.json(c)   -> deno.lock
  pubspec.yaml   -> pubspec.lock
  Gemfile        -> Gemfile.lock
  go.mod         -> go.sum
  Cargo.toml     -> Cargo.lock

Heuristics to avoid false positives:
  - pyproject.toml with only tool config (ruff/black/mypy) is skipped.
  - go.mod with no require directives is skipped.
  - Rust library crates ([lib] without [[bin]]) skip the Cargo.lock
    requirement per Cargo convention; binary crates and workspaces don't.
  - Sub-path manifests fall back to repo-root manifests when absent.
Some upstream projects are libraries or CLI tools that also ship a GitHub
Action wrapper — e.g. pypa/cibuildwheel (Python library on PyPI),
dart-lang/setup-dart (Dart package on pub.dev). These repos legitimately
don't commit a lock file because doing so would over-constrain their
library consumers. Hard-failing on those would block otherwise-valid
Dependabot bumps.

Add lock_file_exemptions.yml at the repo root listing per-(org/repo)
per-ecosystem exemptions. analyze_lock_files consults this file and
reports an exempted manifest as a dim skipped entry (⊘) instead of a
red failure. Exemptions are scoped per-ecosystem, so an action exempted
for one ecosystem is still checked for others it declares (e.g.
dart-lang/setup-dart's node side must still have package-lock.json).

Preseeded entries:
  pypa/cibuildwheel → python   (library on PyPI)
  dart-lang/setup-dart → dart  (Dart library convention)

Lookups lowercase org/repo so case mismatches don't silently miss.
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 24, 2026

Smoke-test results on previously-approved actions

Ran the new check end-to-end against 5 representative actions drawn from actions.yml / recent Dependabot PRs.

Action Type Lock-file check JS build Notes
actions/checkout@v5 (08c6903c) node24 ✓ pass (package.jsonpackage-lock.json) ✓ pass Baseline — canonical well-maintained action
mozilla-actions/sccache-action@v0.0.10 (9e7fa8a1) node24 ✓ pass ✗ fail (pre-existing toolchain drift, tracked in #761 / Mozilla-Actions/sccache-action#252) New check is orthogonal to the JS-build failure — both surfaced correctly
Kesin11/actions-timeline@v2.2.5 (54d513e0) node20 (Deno) ✓ pass (deno.jsoncdeno.lock) ✓ pass Confirms Deno ecosystem detection
pypa/cibuildwheel@v3.3.1 (298ed2fb) composite first fail, then ⊘ exempted ⊘ n/a pyproject.toml declares runtime deps but no lock file committed — cibuildwheel is a PyPI library shipped as an action wrapper
dart-lang/setup-dart@v1.7.1 (e51d8e57) node20 + Dart first fail, then ⊘ exempted ✓ pass pubspec.yaml without pubspec.lock — Dart library convention. Node side (package.jsonpackage-lock.json) still checked and passes

One more I tried didn't complete:

  • slackapi/slack-github-action@v3.0.2 (03ea5433) — crashed on build_in_docker before reaching the lock-file check. Pre-existing orphan-tag edge case unrelated to this PR; the v0.0.10 source-detached handling from verify-action-build: handle source-detached orphan release tags #768 should cover most of this class but this specific tag still needs investigation.

Follow-up in this PR

The two pyproject.toml / pubspec.yaml findings are real — those projects deliberately don't commit lock files per their ecosystem's library convention, and hard-failing Dependabot bumps on them would be unhelpful. Added a per-ecosystem opt-out mechanism (option 3 from earlier discussion):

  • New lock_file_exemptions.yml at the repo root — YAML-style per-(org/repo) per-ecosystem allowlist.
  • Preseeded with the two cases above (pypa/cibuildwheelpython, dart-lang/setup-dartdart).
  • Exemptions are per-ecosystem: dart-lang/setup-dart still has its node side checked and its package-lock.json is required. Nothing blanket-skipped.
  • Exempted manifests are rendered as ⊘ skipped entries in the Lock File Presence panel with the reason, so it's clear they were intentionally allowed rather than silently passed.

Pushed as a second commit (0327f65). Full test suite now 177 passing (38 in TestAnalyzeLockFiles).

Generated-by: Claude Opus 4.7 (1M context)

Comment thread lock_file_exemptions.yml
@@ -0,0 +1,36 @@
# lock_file_exemptions.yml
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RAT Check complains about the missing license header here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants