From c9bde78a65b8324a0f3b87596981660a313e2a42 Mon Sep 17 00:00:00 2001 From: "Jorge O. Castro" Date: Thu, 9 Apr 2026 03:49:12 -0400 Subject: [PATCH 1/2] docs(skills): add Caveats column to subcommand table, expand internal/ layout SO-40: The SKILL.md subcommand table was missing a Caveats column per AC3. Changes: - Add 4th column 'Caveats' to all 19 subcommand table rows with: - GITHUB_TOKEN requirements (or explicit 'No GITHUB_TOKEN needed') - Anonymous OCI pull note for fetch-supply-chain (authn.Anonymous) - packages:read scope requirement for fetch-testhub - Fallback behaviour for cold-start CI - Per-image repo lists for fetch-builds-* variants - Weekly-update cadence and cache-skip behaviour for fetch-countme - Expand internal/ directory tree to list all actual packages: brewfile, builds, contributors, ghcli, quay, scorecard, supplychain (previously only listed countme/github/history/metrics/osanalytics/tap/tapanalytics/testhub) - Add fetch-contributors step to CI walkthrough (was previously missing) - Correct step numbers in CI walkthrough (was off by one after contributors step) --- skills/SKILL.md | 75 ++++++++++++++++++++++++++----------------------- 1 file changed, 40 insertions(+), 35 deletions(-) diff --git a/skills/SKILL.md b/skills/SKILL.md index 7fbac6f..5627bd2 100644 --- a/skills/SKILL.md +++ b/skills/SKILL.md @@ -43,11 +43,17 @@ bootc-ecosystem/ ├── stats-go/ # Go data-collection backend │ ├── cmd/stats/ # CLI entry point (main.go) — subcommand dispatch │ └── internal/ +│ ├── brewfile/ # Brewfile fetcher (parses GitHub-hosted Brewfiles) +│ ├── builds/ # GitHub Actions build run collector (per-image) +│ ├── contributors/ # GitHub commits, PRs, issues, discussions APIs │ ├── countme/ # Universal Blue countme badge + CSV fetcher -│ ├── github/ # GitHub REST API client (traffic data) +│ ├── ghcli/ # gh CLI wrapper (reads GITHUB_TOKEN automatically) │ ├── history/ # Snapshot persistence (.sync-cache/) │ ├── metrics/ # Pure stats computation (no I/O) │ ├── osanalytics/ # OS-breakdown analytics +│ ├── quay/ # Quay.io public API client (no auth required) +│ ├── scorecard/ # OpenSSF Scorecard public API client (no auth required) +│ ├── supplychain/ # OCI image supply-chain inspection (anonymous pull) │ ├── tap/ # Homebrew tap scraping │ ├── tapanalytics/ # Per-tap aggregation │ └── testhub/ # projectbluefin/testhub container package stats @@ -75,31 +81,29 @@ bootc-ecosystem/ ## stats-go Subcommands -The `stats` binary dispatches on `os.Args[1]`: - -| Subcommand | Writes | Source | -|---|---|---| -| `stats fetch-homebrew` | `src/data/stats.json` + `.sync-cache/history.json` | GitHub API (tap traffic) | -| `stats fetch-brewfile-taps` | `src/data/brewfile-stats.json` | Parses system Brewfiles in bluefin-common and bluefin repos | -| `stats fetch-testhub` | `src/data/testhub.json` + `.sync-cache/testhub-history.json` | GitHub Packages + Actions API (projectbluefin/testhub) | -| `stats fetch-countme` | `src/data/countme.json` + `.sync-cache/countme-history.json` | Fedora countme CSV (data-analysis.fedoraproject.org) | -| `stats fetch-releases` | `src/data/releases.json` | GitHub Releases API (Bluefin, Aurora, Bazzite, uCore) | -| `stats fetch-contributors` | `src/data/contributors.json` + `.sync-cache/contributor-history.json` | GitHub commits, PRs, issues, discussions APIs | -| `stats fetch-scorecard` | `src/data/scorecard.json` | OpenSSF Scorecard API (tracked repos) | -| `stats fetch-supply-chain` | `src/data/supply-chain.json` | GitHub workflow files (cosign, SBOM, Sigstore detection) | -| `stats fetch-builds-bluefin` | `src/data/builds-bluefin.json` + `.sync-cache/builds-bluefin-history.json` | GitHub Actions API | -| `stats fetch-builds-aurora` | `src/data/builds-aurora.json` + `.sync-cache/builds-aurora-history.json` | GitHub Actions API | -| `stats fetch-builds-bazzite` | `src/data/builds-bazzite.json` + `.sync-cache/builds-bazzite-history.json` | GitHub Actions API | -| `stats fetch-builds-universal-blue` | `src/data/builds-universal-blue.json` + `.sync-cache/builds-universal-blue-history.json` | GitHub Actions API | -| `stats fetch-builds-ucore` | `src/data/builds-ucore.json` + `.sync-cache/builds-ucore-history.json` | GitHub Actions API | -| `stats fetch-builds-zirconium` | `src/data/builds-zirconium.json` + `.sync-cache/builds-zirconium-history.json` | GitHub Actions API | -| `stats fetch-builds-bootcrew` | `src/data/builds-bootcrew.json` + `.sync-cache/builds-bootcrew-history.json` | GitHub Actions API | -| `stats fetch-builds-blue-build` | `src/data/builds-blue-build.json` + `.sync-cache/builds-blue-build-history.json` | GitHub Actions API | -| `stats fetch-quay-fedora` | `src/data/quay-fedora.json` | Quay.io API (fedora base images) | -| `stats fetch-quay-centos` | `src/data/quay-centos.json` | Quay.io API (centos base images) | -| `stats fetch-quay-almalinux` | `src/data/quay-almalinux.json` | Quay.io API (almalinux base images) | - -No-arg default = `fetch-homebrew` (backward compat for `just sync`). +The `stats` binary dispatches on `os.Args[1]`. No-arg default = `fetch-homebrew` (backward compat for `just sync`). + +| Subcommand | Writes | Source | Caveats | +|---|---|---|---| +| `stats fetch-homebrew` | `src/data/stats.json` + `.sync-cache/history.json` + `.sync-cache/stats-latest.json` | GitHub API (tap traffic, Homebrew analytics) | Requires `GITHUB_TOKEN`; traffic API needs push access — falls back to cached values in CI | +| `stats fetch-brewfile-taps` | `src/data/brewfile-stats.json` | Parses GitHub-hosted Brewfiles (bluefin-common, bazzite) + Homebrew analytics | Requires `GITHUB_TOKEN`; skips ublue-os taps (already tracked by fetch-homebrew) | +| `stats fetch-testhub` | `src/data/testhub.json` + `.sync-cache/testhub-history.json` | GitHub Packages API + Actions API (projectbluefin/testhub) | Requires `GITHUB_TOKEN` with `packages: read` scope; falls back to committed testhub.json on cold start | +| `stats fetch-countme` | `src/data/countme.json` + `.sync-cache/countme-history.json` | Fedora countme CSV (data-analysis.fedoraproject.org) | No GITHUB_TOKEN needed; CSV updates weekly (Sundays); skips re-fetch when current week already cached | +| `stats fetch-releases` | `src/data/releases.json` | GitHub Releases API (Bluefin, Aurora, Bazzite, uCore) | Requires `GITHUB_TOKEN`; tracks 4 repos: `ublue-os/{bluefin,aurora,bazzite,ucore}` | +| `stats fetch-contributors` | `src/data/contributors.json` + `.sync-cache/contributors-history.json` + `.sync-cache/contributor-profiles.json` | GitHub commits, PRs, issues, discussions, participation, punch card APIs | Requires `GITHUB_TOKEN`; fetches 365d data and filters in-memory; profiles are cached to avoid repeated API calls | +| `stats fetch-scorecard` | `src/data/scorecard.json` | OpenSSF Scorecard public API (api.securityscorecards.dev) | No GITHUB_TOKEN needed; some repos not yet indexed → marked as `indexed: false` in output | +| `stats fetch-supply-chain` | `src/data/supply-chain.json` | OCI image manifests + GitHub workflow YAML inspection | OCI metadata pulled **anonymously** (`authn.Anonymous`) — no registry credentials needed; detects cosign, SBOM, Sigstore | +| `stats fetch-builds-bluefin` | `src/data/builds-bluefin.json` + `.sync-cache/builds-bluefin-history.json` | GitHub Actions API (`ublue-os/bluefin` + related repos) | Requires `GITHUB_TOKEN`; 14-day lookback, 30 runs/workflow max | +| `stats fetch-builds-aurora` | `src/data/builds-aurora.json` + `.sync-cache/builds-aurora-history.json` | GitHub Actions API (`ublue-os/aurora`, `get-aurora-dev/{common,iso}`) | Requires `GITHUB_TOKEN`; excludes `get-aurora-dev/aurora-test` to avoid mirrored-run duplication | +| `stats fetch-builds-bazzite` | `src/data/builds-bazzite.json` + `.sync-cache/builds-bazzite-history.json` | GitHub Actions API (`ublue-os/bazzite`) | Requires `GITHUB_TOKEN` | +| `stats fetch-builds-universal-blue` | `src/data/builds-universal-blue.json` + `.sync-cache/builds-universal-blue-history.json` | GitHub Actions API (`ublue-os/main` + related repos) | Requires `GITHUB_TOKEN` | +| `stats fetch-builds-ucore` | `src/data/builds-ucore.json` + `.sync-cache/builds-ucore-history.json` | GitHub Actions API (`ublue-os/ucore`) | Requires `GITHUB_TOKEN` | +| `stats fetch-builds-zirconium` | `src/data/builds-zirconium.json` + `.sync-cache/builds-zirconium-history.json` | GitHub Actions API (`zirconium-dev/zirconium`) | Requires `GITHUB_TOKEN` | +| `stats fetch-builds-bootcrew` | `src/data/builds-bootcrew.json` + `.sync-cache/builds-bootcrew-history.json` | GitHub Actions API (`bootcrew/mono`) | Requires `GITHUB_TOKEN` | +| `stats fetch-builds-blue-build` | `src/data/builds-blue-build.json` + `.sync-cache/builds-blue-build-history.json` | GitHub Actions API (BlueBuild repos) | Requires `GITHUB_TOKEN` | +| `stats fetch-quay-fedora` | `src/data/quay-fedora.json` | Quay.io public API (fedora base images) | No GITHUB_TOKEN needed; Quay.io API is public for tracked repos | +| `stats fetch-quay-centos` | `src/data/quay-centos.json` | Quay.io public API (centos base images) | No GITHUB_TOKEN needed | +| `stats fetch-quay-almalinux` | `src/data/quay-almalinux.json` | Quay.io public API (almalinux base images) | No GITHUB_TOKEN needed | --- @@ -193,15 +197,16 @@ Runs daily at 06:00 UTC: 4. **`stats fetch-testhub`** — writes `testhub.json`; `continue-on-error: true` 5. **`stats fetch-countme`** — writes `countme.json`; `continue-on-error: true` 6. **`stats fetch-releases`** — writes `releases.json`; `continue-on-error: true` -7. **`stats fetch-builds-*`** — one step per image (bluefin/aurora/bazzite/universal-blue/ucore/zirconium/bootcrew/blue-build); `continue-on-error: true` -8. **`stats fetch-quay-*`** — fedora/centos/almalinux; `continue-on-error: true` -9. **`stats fetch-scorecard`** — writes `scorecard.json`; `continue-on-error: true` -10. **`stats fetch-supply-chain`** — writes `supply-chain.json`; `continue-on-error: true` -11. **Build Astro site** — `npm run build` -12. **Verify charts have data** — fails if `class="chart-empty"` appears in output pages -13. **Verify summary KPIs** — asserts `summary.total_packages > 0` -14. **Run Playwright E2E chart tests** — `npm run test:e2e` -15. **Deploy to GitHub Pages** +7. **`stats fetch-contributors`** — writes `contributors.json`; `continue-on-error: true` +8. **`stats fetch-builds-*`** — one step per image (bluefin/aurora/bazzite/universal-blue/ucore/zirconium/bootcrew/blue-build); `continue-on-error: true` +9. **`stats fetch-quay-*`** — fedora/centos/almalinux; `continue-on-error: true` +10. **`stats fetch-scorecard`** — writes `scorecard.json`; `continue-on-error: true` +11. **`stats fetch-supply-chain`** — writes `supply-chain.json`; `continue-on-error: true` +12. **Build Astro site** — `npm run build` +13. **Verify charts have data** — fails if `class="chart-empty"` appears in output pages +14. **Verify summary KPIs** — asserts `summary.total_packages > 0` +15. **Run Playwright E2E chart tests** — `npm run test:e2e` +16. **Deploy to GitHub Pages** --- From 8b42e593623062172d72e7722313c15c2d31cda9 Mon Sep 17 00:00:00 2001 From: "Jorge O. Castro" Date: Thu, 9 Apr 2026 03:58:13 -0400 Subject: [PATCH 2/2] fix(skills): restore github/ package to tree, correct CI step order in daily-build.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two factual errors found by QA review (SO-76): 1. github/ package was removed from internal/ tree when ghcli/ was added. Both packages exist and are in active use: - internal/ghcli/ wraps the gh CLI binary (exec.go) - internal/github/ is a REST API client (client.go, graphql.go) Restore github/ alongside ghcli/ in the layout tree. 2. fetch-contributors was incorrectly placed as step 7 in daily-build.yml walkthrough. It does NOT run in daily-build.yml — it runs in the separate .github/workflows/contributor-sync.yml workflow. Remove from daily-build.yml list; add clarifying note. Also correct fetch-releases position (step 6 → step 10, matching actual workflow order after fetch-supply-chain). Co-authored-by: docs-writer --- skills/SKILL.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/skills/SKILL.md b/skills/SKILL.md index 5627bd2..08c7b5a 100644 --- a/skills/SKILL.md +++ b/skills/SKILL.md @@ -48,6 +48,7 @@ bootc-ecosystem/ │ ├── contributors/ # GitHub commits, PRs, issues, discussions APIs │ ├── countme/ # Universal Blue countme badge + CSV fetcher │ ├── ghcli/ # gh CLI wrapper (reads GITHUB_TOKEN automatically) +│ ├── github/ # GitHub REST API client (traffic, packages, Actions data) │ ├── history/ # Snapshot persistence (.sync-cache/) │ ├── metrics/ # Pure stats computation (no I/O) │ ├── osanalytics/ # OS-breakdown analytics @@ -196,17 +197,18 @@ Runs daily at 06:00 UTC: 3. **`stats fetch-brewfile-taps`** — writes `brewfile-stats.json`; `continue-on-error: true` 4. **`stats fetch-testhub`** — writes `testhub.json`; `continue-on-error: true` 5. **`stats fetch-countme`** — writes `countme.json`; `continue-on-error: true` -6. **`stats fetch-releases`** — writes `releases.json`; `continue-on-error: true` -7. **`stats fetch-contributors`** — writes `contributors.json`; `continue-on-error: true` -8. **`stats fetch-builds-*`** — one step per image (bluefin/aurora/bazzite/universal-blue/ucore/zirconium/bootcrew/blue-build); `continue-on-error: true` -9. **`stats fetch-quay-*`** — fedora/centos/almalinux; `continue-on-error: true` -10. **`stats fetch-scorecard`** — writes `scorecard.json`; `continue-on-error: true` -11. **`stats fetch-supply-chain`** — writes `supply-chain.json`; `continue-on-error: true` -12. **Build Astro site** — `npm run build` -13. **Verify charts have data** — fails if `class="chart-empty"` appears in output pages -14. **Verify summary KPIs** — asserts `summary.total_packages > 0` -15. **Run Playwright E2E chart tests** — `npm run test:e2e` -16. **Deploy to GitHub Pages** +6. **`stats fetch-builds-*`** — one step per image (bluefin/aurora/bazzite/universal-blue/ucore/zirconium/bootcrew/blue-build); `continue-on-error: true` +7. **`stats fetch-quay-*`** — fedora/centos/almalinux; `continue-on-error: true` +8. **`stats fetch-scorecard`** — writes `scorecard.json`; `continue-on-error: true` +9. **`stats fetch-supply-chain`** — writes `supply-chain.json`; `continue-on-error: true` +10. **`stats fetch-releases`** — writes `releases.json`; `continue-on-error: true` +11. **Build Astro site** — `npm run build` +12. **Verify charts have data** — fails if `class="chart-empty"` appears in output pages +13. **Verify summary KPIs** — asserts `summary.total_packages > 0` +14. **Run Playwright E2E chart tests** — `npm run test:e2e` +15. **Deploy to GitHub Pages** + +> **Note:** `fetch-contributors` does NOT run in `daily-build.yml`. It runs in the separate `.github/workflows/contributor-sync.yml` workflow. ---