From f1a7a98f83b079c7a6b5bf471299c6bd04c0483f Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Tue, 1 Sep 2026 17:55:41 -0400 Subject: [PATCH] fix(ci): guard corpus-pin audits against an empty LANGUAGE_CRUCIBLE_REF GitHub withholds repository variables -- like secrets -- from pull_request runs raised from a fork, so `vars.LANGUAGE_CRUCIBLE_REF` expands to an empty string there and the five corpus-backed workflows ran `git clone --branch ""`, failing with a bare `fatal: Remote branch not found in upstream origin` before any engine code was exercised. #2628 is the first fork PR this repo has seen in a long while, which is why this went unnoticed: it arrived with four red audits that had nothing to do with the contributor's change. Guard the clone instead of falling back to the corpus's `main`: these pins exist to make the audits deterministic, and silently auditing against a different corpus snapshot would trade a confusing failure for an authoritative-looking wrong answer. The guard fails with an explicit `::error::` explaining the fork limitation and pointing at CONTRIBUTING.md. Also document what a fork contributor should expect, and add the `rosetta-audit` cross-language gate (#2557) to the baselines section -- it shipped without a CONTRIBUTING entry, so the cross-repo re-baseline protocol it depends on was undiscoverable from here. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/golden-crucible.yml | 7 +++- .../workflows/tree-sitter-accuracy-audit.yml | 8 ++++- .../tree-sitter-accuracy-history.yml | 8 ++++- .github/workflows/tri-comparison-audit.yml | 8 ++++- .github/workflows/tri-comparison-history.yml | 8 ++++- CONTRIBUTING.md | 36 +++++++++++++++++++ 6 files changed, 70 insertions(+), 5 deletions(-) diff --git a/.github/workflows/golden-crucible.yml b/.github/workflows/golden-crucible.yml index 158308e48..4c065583c 100644 --- a/.github/workflows/golden-crucible.yml +++ b/.github/workflows/golden-crucible.yml @@ -51,7 +51,12 @@ jobs: - name: Fetch Golden Test Repo (Language Crucible) run: | echo "Cloning Language Crucible repository (pinned via LANGUAGE_CRUCIBLE_REF)..." - git clone --branch "${{ vars.LANGUAGE_CRUCIBLE_REF }}" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible + ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}" + if [ -z "$ref" ]; then + echo "::error title=Corpus pin unavailable::LANGUAGE_CRUCIBLE_REF resolved to an empty value, so this corpus-backed audit cannot pin its corpus and did not run. GitHub withholds repository variables (like secrets) from pull_request runs raised from a fork, so this is expected on a fork PR and is NOT a problem with the contributor's changes -- a maintainer re-runs these audits from a branch in this repo before merge. See CONTRIBUTING.md, 'Checks that cannot run on fork PRs'." + exit 1 + fi + git clone --branch "$ref" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible - name: Run Golden Crucible Test (${{ matrix.mode }}) run: | diff --git a/.github/workflows/tree-sitter-accuracy-audit.yml b/.github/workflows/tree-sitter-accuracy-audit.yml index f2722c56f..6490caaff 100644 --- a/.github/workflows/tree-sitter-accuracy-audit.yml +++ b/.github/workflows/tree-sitter-accuracy-audit.yml @@ -52,7 +52,13 @@ jobs: # Same pin golden-crucible.yml uses -- this tool reuses that corpus rather than # a fresh clone (see tests/tools/tree_sitter_accuracy_audit.py's own CORPUS section). - name: Fetch Language Crucible corpus (pinned via LANGUAGE_CRUCIBLE_REF) - run: git clone --branch "${{ vars.LANGUAGE_CRUCIBLE_REF }}" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible + run: | + ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}" + if [ -z "$ref" ]; then + echo "::error title=Corpus pin unavailable::LANGUAGE_CRUCIBLE_REF resolved to an empty value, so this corpus-backed audit cannot pin its corpus and did not run. GitHub withholds repository variables (like secrets) from pull_request runs raised from a fork, so this is expected on a fork PR and is NOT a problem with the contributor's changes -- a maintainer re-runs these audits from a branch in this repo before merge. See CONTRIBUTING.md, 'Checks that cannot run on fork PRs'." + exit 1 + fi + git clone --branch "$ref" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible - name: Run baseline-gated regression check across every baselined language run: | diff --git a/.github/workflows/tree-sitter-accuracy-history.yml b/.github/workflows/tree-sitter-accuracy-history.yml index 429be57ab..4c145d8a5 100644 --- a/.github/workflows/tree-sitter-accuracy-history.yml +++ b/.github/workflows/tree-sitter-accuracy-history.yml @@ -75,7 +75,13 @@ jobs: pip install tree-sitter-language-pack - name: Fetch Language Crucible corpus (pinned via LANGUAGE_CRUCIBLE_REF) - run: git clone --branch "${{ vars.LANGUAGE_CRUCIBLE_REF }}" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible + run: | + ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}" + if [ -z "$ref" ]; then + echo "::error title=Corpus pin unavailable::LANGUAGE_CRUCIBLE_REF resolved to an empty value, so this corpus-backed audit cannot pin its corpus and did not run. GitHub withholds repository variables (like secrets) from pull_request runs raised from a fork, so this is expected on a fork PR and is NOT a problem with the contributor's changes -- a maintainer re-runs these audits from a branch in this repo before merge. See CONTRIBUTING.md, 'Checks that cannot run on fork PRs'." + exit 1 + fi + git clone --branch "$ref" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible - name: Append accuracy history + regenerate summary table + chart run: | diff --git a/.github/workflows/tri-comparison-audit.yml b/.github/workflows/tri-comparison-audit.yml index df025c2c1..34890d890 100644 --- a/.github/workflows/tri-comparison-audit.yml +++ b/.github/workflows/tri-comparison-audit.yml @@ -77,7 +77,13 @@ jobs: # Same pin golden-crucible.yml / tree-sitter-accuracy-audit.yml use -- this tool reuses # that corpus rather than a fresh clone. - name: Fetch Language Crucible corpus (pinned via LANGUAGE_CRUCIBLE_REF) - run: git clone --branch "${{ vars.LANGUAGE_CRUCIBLE_REF }}" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible + run: | + ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}" + if [ -z "$ref" ]; then + echo "::error title=Corpus pin unavailable::LANGUAGE_CRUCIBLE_REF resolved to an empty value, so this corpus-backed audit cannot pin its corpus and did not run. GitHub withholds repository variables (like secrets) from pull_request runs raised from a fork, so this is expected on a fork PR and is NOT a problem with the contributor's changes -- a maintainer re-runs these audits from a branch in this repo before merge. See CONTRIBUTING.md, 'Checks that cannot run on fork PRs'." + exit 1 + fi + git clone --branch "$ref" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible - name: Run baseline-gated precision regression check across every baselined language run: | diff --git a/.github/workflows/tri-comparison-history.yml b/.github/workflows/tri-comparison-history.yml index c83d8efb2..1fa1b333a 100644 --- a/.github/workflows/tri-comparison-history.yml +++ b/.github/workflows/tri-comparison-history.yml @@ -86,7 +86,13 @@ jobs: fi - name: Fetch Language Crucible corpus (pinned via LANGUAGE_CRUCIBLE_REF) - run: git clone --branch "${{ vars.LANGUAGE_CRUCIBLE_REF }}" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible + run: | + ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}" + if [ -z "$ref" ]; then + echo "::error title=Corpus pin unavailable::LANGUAGE_CRUCIBLE_REF resolved to an empty value, so this corpus-backed audit cannot pin its corpus and did not run. GitHub withholds repository variables (like secrets) from pull_request runs raised from a fork, so this is expected on a fork PR and is NOT a problem with the contributor's changes -- a maintainer re-runs these audits from a branch in this repo before merge. See CONTRIBUTING.md, 'Checks that cannot run on fork PRs'." + exit 1 + fi + git clone --branch "$ref" --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible - name: Regenerate chart + ledger + points-of-interest report run: | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 40c6a5fa9..10cf62486 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -78,10 +78,46 @@ When you modify GitGalaxy's core engine, several CI workflows will rigorously te - Run `python tests/tools/tree_sitter_accuracy_audit.py --ci --all`. - If your regex fix legitimately drops false positives causing ground truth drift, regenerate the baseline: `python tests/tools/tree_sitter_accuracy_audit.py --regenerate --lang ` +4. **Cross-Language Consistency (`rosetta-audit`, the [keyword-rosetta](https://github.com/squid-protocol/keyword-rosetta) corpus)** + This runs the control corpus's verifier across all 46 language folders against your engine build, + so a change that shifts corpus-observed counts fails **in the PR that caused it**. Unlike the three + baselines above, the expected values live in *another repository* — you cannot re-bless them here. + - Run one language locally: `GITGALAXY_PATH= python tools/verify_language.py ` from a keyword-rosetta checkout. + - If the drift is an unintentional regression: fix the engine change. **Do not touch the pin.** + - If the drift is an intentional, corpus-visible improvement: it needs a companion re-baseline PR + in keyword-rosetta *before* this one can merge. Follow + [`docs/self_scan/BUMPING_THE_ROSETTA_PIN.md`](docs/self_scan/BUMPING_THE_ROSETTA_PIN.md) — in short, + set that repo's committed `ENGINE_REF` file to `pull//head` so its gates run green + against your unmerged branch, then a maintainer restores it and bumps `KEYWORD_ROSETTA_REF` here. + - Bumping the pin needs repo admin, so **ask a maintainer** rather than trying to do it yourself. + Bumping a pin is never a way to make a red check go away. + If your PR touches any baseline fixtures, **explain why in the PR description** (e.g. "improved the Rust parser, now correctly detects async trait bounds"). A CI check flags any PR that modifies these files so it's never invisible in a large diff. --- +## 🍴 Checks that cannot run on fork PRs + +If you opened your PR from a fork, expect these four checks to fail immediately, before your code +is ever exercised: + +- `crucible-audit (full-precision)` and `crucible-audit (zero-dependency)` +- `tree-sitter-accuracy-audit` +- `tri-comparison-audit` + +**This is not a problem with your changes.** These audits clone the +[language-crucible](https://github.com/squid-protocol/language-crucible) corpus at a ref pinned in +the `LANGUAGE_CRUCIBLE_REF` Actions variable, and GitHub withholds repository variables — like +secrets — from `pull_request` runs raised from a fork. The pin resolves to an empty string and the +clone fails. A maintainer re-runs these audits from a branch in this repo before merging, so there +is nothing for you to do. + +`rosetta-audit` is unaffected (it falls back to the corpus's `main`), and every other check — +`full-suite`, the `smoke-test` matrix, `ruff-audit`, `mypy-audit`, `ast-accuracy-audit` — runs +normally on a fork PR and is worth taking seriously. + +--- + ## 🐛 Reporting Discrepancies If the GalaxyScope CLI misidentifies a repository's structure or fails to parse a specific file phenotype, please use our **[Parsing Discrepancy Form](https://github.com/squid-protocol/gitgalaxy/issues/new/choose)**.