|
| 1 | +# Rollout playbook: shipping a v2 of a shared tool across a fleet of consumer repos |
| 2 | + |
| 3 | +Captures the pattern used for the 2026-04-17 → 2026-04-18 rollout of |
| 4 | +`smartwatermelon/github-workflows` v2.0.0 → v2.0.1 across 27 consumer |
| 5 | +repos. Written as a reusable playbook for future tool rollouts |
| 6 | +(e.g., `ralph-burndown`). |
| 7 | + |
| 8 | +## Preconditions |
| 9 | + |
| 10 | +- **One source repo** ships the tool (reusable workflow, library, CLI, config package). |
| 11 | +- **N consumer repos** reference it by version pin (`@v1`, `@v1.2.0`, git SHA). |
| 12 | +- You own both sides of the fence. |
| 13 | +- A CI gate exists in consumers that will execute when the pin changes. Know how it reacts to self-modification of the caller file — it's usually a silent failure mode and you'll discover it via bellwether. Build this knowledge into your plan. |
| 14 | + |
| 15 | +## Phases (in order) |
| 16 | + |
| 17 | +### Phase 0 — Harden the source repo first |
| 18 | + |
| 19 | +Before rolling to anyone: |
| 20 | + |
| 21 | +- Fix all bugs currently producing noise in consumer CI. (We shipped #39 first — a one-line grep fix — before any new feature work. It was the bypass-mechanism that Phase 1 would need.) |
| 22 | +- Ship new features / behavior changes as independent PRs, each with local + CI review passing. Squash-merge, one atomic PR per logical change. |
| 23 | +- Dogfood: ensure the source repo's own PRs exercise the tool against itself (`self-review.yml` pattern). This catches issues before consumers do. |
| 24 | +- When behavior changes significantly, tag **both** a specific semver (`v2.0.0`) and a floating major-version tag (`v2`). Ship a GitHub Release with full migration notes. |
| 25 | +- Backport critical fixes to the prior major (`v1.x.4`) for consumers who won't migrate immediately. |
| 26 | + |
| 27 | +### Phase 1 — Bellwether migration via PR flow |
| 28 | + |
| 29 | +Pick **3 actively-developed consumer repos** plus any org-wide template repo. Migrate them via the normal PR flow. Purpose: |
| 30 | + |
| 31 | +- Catch first-time friction the source-repo dogfood missed (for us: the `claude-code-action` workflow-validation-skip behavior). |
| 32 | +- Validate that local + CI review still passes on realistic diffs. |
| 33 | +- Build evidence to show the fleet the migration is safe. |
| 34 | + |
| 35 | +Timebox: 15-30 min per PR including manual observation. Plan on ~2 hours total. |
| 36 | + |
| 37 | +### Phase 2 — Admin direct-push batch |
| 38 | + |
| 39 | +Once Phase 1 confirms clean: |
| 40 | + |
| 41 | +- Use the GitHub REST Contents API (`PUT /repos/{owner}/{repo}/contents/{path}`) with `branch: main` to push changes directly. No branch, no PR, no CI cycle on the migration commit itself. |
| 42 | +- Script it. Expect to fix one or two things (for us: BSD sed `\b` didn't work, used `perl` instead; one repo had a non-standard filename). |
| 43 | +- Protocol 1 ("never commit to main") is deliberately bypassed here — admin authorization is explicit and documented in the plan. |
| 44 | +- Verify idempotency: script should detect `[SKIP]` when the target is already at the intended version. |
| 45 | + |
| 46 | +Timebox: 5-10 min for a batch of 20-30 repos. |
| 47 | + |
| 48 | +### Phase 3 — Optional cleanup (renames, filename normalization) |
| 49 | + |
| 50 | +If the fleet has accumulated naming inconsistency (we had legacy `claude-code-review.yml` vs current `claude-blocking-review.yml`), clean it up now. Contents API doesn't support rename, so use: `PUT /new/path` + `DELETE /old/path` = two commits per repo. |
| 51 | + |
| 52 | +### Phase 4 — Lock to specific versions + enable Dependabot |
| 53 | + |
| 54 | +The final state. Three sub-steps: |
| 55 | + |
| 56 | +- **4a — Ship any fix needed to make Dependabot PRs pass cleanly.** Dependabot auto-modifies caller workflow files, which triggers self-modification security skips in most review tooling. Source repo needs to handle this gracefully (skip, not fail). For us: `v2.0.1` shipped with a `.github/workflows/*.yml` auto-skip branch. |
| 57 | +- **4b — Bump every consumer to the specific semver** (`@v2` → `@v2.0.1`). Admin direct-push again. |
| 58 | +- **4c — Add `.github/dependabot.yml`** to each consumer. Fresh file if absent; skip-with-warning if present (don't touch existing YAML programmatically). Minimal config: |
| 59 | + |
| 60 | +```yaml |
| 61 | +version: 2 |
| 62 | +updates: |
| 63 | + - package-ecosystem: "github-actions" |
| 64 | + directory: "/" |
| 65 | + schedule: |
| 66 | + interval: "weekly" |
| 67 | + commit-message: |
| 68 | + prefix: "chore" |
| 69 | +``` |
| 70 | +
|
| 71 | +After this, new versions of the source repo auto-PR themselves to consumers within a week. No more manual batch migrations. |
| 72 | +
|
| 73 | +## Pitfalls encountered |
| 74 | +
|
| 75 | +| Pitfall | How we found it | How we handled it | |
| 76 | +|---|---|---| |
| 77 | +| BSD sed `\b` is a no-op | Phase 2 first run reported "sed made no change" on all 23 repos | Switched to `perl -pe` for word-boundary regex | |
| 78 | +| Bypass marker grep mismatch | Docs said `[skip-claude-review: reason]` but code matched `[skip-claude-review]` | Fixed in Phase 0 (PR #39) before anything else | |
| 79 | +| `gh api` in wrapper scripts doesn't forward `--repo` | Phase 1 merges failed from wrong CWD | `cd` into each clone before calling `gh pr merge` | |
| 80 | +| `claude-code-action` refuses to run when caller workflow changes | Phase 1 bellwethers failed with "exceeded turn limit" after 30s | Phase 4a ships a skip for the specific failure mode | |
| 81 | +| Unanchored `VERDICT: X` grep false-matches review prose | PR #47 self-review's text quoted the grep pattern | Anchor to line start/end in the Check verdict step | |
| 82 | +| Content-adjacent config files in "doc-only" allowlist | Local reviewer flagged CODEOWNERS, dependabot.yml | Explicit NON-DOC exclusion for security-adjacent meta files | |
| 83 | + |
| 84 | +## Timings observed |
| 85 | + |
| 86 | +| Phase | Operations | Wall clock | |
| 87 | +|---|---|---| |
| 88 | +| Phase 0 (source hardening) | 7 PRs shipped | 4-6 hours spread over session | |
| 89 | +| Phase 1 (4 bellwether PRs) | 4 PR/merge cycles | ~45 min including bypass-marker diagnosis | |
| 90 | +| Phase 2 (23 repo batch) | 23 admin direct-pushes | 30 seconds | |
| 91 | +| Phase 3 (4 renames) | 8 API calls (create + delete per repo) | 15 seconds | |
| 92 | +| Phase 4 (27 pins + 27 dependabot.yml) | 54 admin direct-pushes | 45 seconds | |
| 93 | + |
| 94 | +The PR-flow phases dominate. The admin batch phases are nearly free. |
| 95 | + |
| 96 | +## Key decisions to make upfront (not mid-rollout) |
| 97 | + |
| 98 | +- **Semver bump size** — major (v2) if behavior changes visibly, minor (v1.3) if purely additive. We chose major because the narrow-prompt behavior shift was user-visible. |
| 99 | +- **Keep the prior major maintained** — yes/no. We shipped `v1.2.4` as a terminal backport for the grep fix. |
| 100 | +- **Floating tag vs. specific tag + Dependabot** — floating is simpler short-term, Dependabot + specific is cleaner long-term. We chose Dependabot + specific once everything was migrated. |
| 101 | +- **Who reviews the source repo's own workflow-file edits after the skip lands** — local reviewers are the primary safety net. CI review will self-skip on workflow-file changes. Accept this before shipping v2.0.1. |
| 102 | + |
| 103 | +## Not to reuse blindly |
| 104 | + |
| 105 | +- The specific bypass marker `[skip-claude-review: reason]` is github-workflows specific. |
| 106 | +- The `claude-code-action` workflow-validation security skip is an upstream action's behavior; other tools won't have this exact issue but may have analogous self-modification concerns. |
| 107 | +- File-path allowlists (`*.md`, `docs/`, image extensions, etc.) are context-specific. `ralph-burndown` will have its own set of "doc-only" equivalents. |
| 108 | + |
| 109 | +## Adapting for ralph-burndown (or any other tool) |
| 110 | + |
| 111 | +The structural pattern transfers. To port: |
| 112 | + |
| 113 | +1. **Identify the source repo.** What ships the tool? What's its current version contract? |
| 114 | +2. **Enumerate consumers.** `gh search code "uses: OWNER/REPO"` is a good start. Store the inventory in a comment or `.claude-review-ignore`-like file. |
| 115 | +3. **Identify the self-modification failure mode.** If the tool runs on PR events that touch its caller, confirm how it behaves. Often: skip with non-zero exit (needs Phase 4a fix). Occasionally: silent success (no fix needed). |
| 116 | +4. **Write the source-side fix.** What's the equivalent of the `.github/workflows/*.yml` skip? Same shape: detect the condition, short-circuit with a clear verdict/output, exit 0. |
| 117 | +5. **Pick bellwethers.** Three active repos + any template. Same rationale. |
| 118 | +6. **Run the Phases.** Mechanical once the adaptations above are in place. |
| 119 | + |
| 120 | +The time-spent profile will look similar: most time in Phase 0 (source work) and Phase 1 (bellwether diagnosis); Phases 2–4 are scripted batch operations. |
0 commit comments