Skip to content

Commit bdb27c0

Browse files
twistedmelonmanClaude Code Bot
andauthored
docs: add v2 rollout playbook for future tool rollouts (#58)
Captures the pattern used for the 2026-04-17→2026-04-18 v2.0.0→v2.0.1 rollout across 27 consumer repos. Structured as four phases (harden source, bellwether PRs, admin batch, lock+dependabot) with pitfalls, timings, and adaptation guidance for future tools (e.g., ralph-burndown). Placed under docs/plans/ next to the other planning documents. No code impact; doc-only change. Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
1 parent a2566ef commit bdb27c0

1 file changed

Lines changed: 120 additions & 0 deletions

File tree

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Rollout playbook: shipping a v2 of a shared tool across a fleet of consumer repos
2+
3+
Captures the pattern used for the 2026-04-17 → 2026-04-18 rollout of
4+
`smartwatermelon/github-workflows` v2.0.0 → v2.0.1 across 27 consumer
5+
repos. Written as a reusable playbook for future tool rollouts
6+
(e.g., `ralph-burndown`).
7+
8+
## Preconditions
9+
10+
- **One source repo** ships the tool (reusable workflow, library, CLI, config package).
11+
- **N consumer repos** reference it by version pin (`@v1`, `@v1.2.0`, git SHA).
12+
- You own both sides of the fence.
13+
- A CI gate exists in consumers that will execute when the pin changes. Know how it reacts to self-modification of the caller file — it's usually a silent failure mode and you'll discover it via bellwether. Build this knowledge into your plan.
14+
15+
## Phases (in order)
16+
17+
### Phase 0 — Harden the source repo first
18+
19+
Before rolling to anyone:
20+
21+
- Fix all bugs currently producing noise in consumer CI. (We shipped #39 first — a one-line grep fix — before any new feature work. It was the bypass-mechanism that Phase 1 would need.)
22+
- Ship new features / behavior changes as independent PRs, each with local + CI review passing. Squash-merge, one atomic PR per logical change.
23+
- Dogfood: ensure the source repo's own PRs exercise the tool against itself (`self-review.yml` pattern). This catches issues before consumers do.
24+
- When behavior changes significantly, tag **both** a specific semver (`v2.0.0`) and a floating major-version tag (`v2`). Ship a GitHub Release with full migration notes.
25+
- Backport critical fixes to the prior major (`v1.x.4`) for consumers who won't migrate immediately.
26+
27+
### Phase 1 — Bellwether migration via PR flow
28+
29+
Pick **3 actively-developed consumer repos** plus any org-wide template repo. Migrate them via the normal PR flow. Purpose:
30+
31+
- Catch first-time friction the source-repo dogfood missed (for us: the `claude-code-action` workflow-validation-skip behavior).
32+
- Validate that local + CI review still passes on realistic diffs.
33+
- Build evidence to show the fleet the migration is safe.
34+
35+
Timebox: 15-30 min per PR including manual observation. Plan on ~2 hours total.
36+
37+
### Phase 2 — Admin direct-push batch
38+
39+
Once Phase 1 confirms clean:
40+
41+
- Use the GitHub REST Contents API (`PUT /repos/{owner}/{repo}/contents/{path}`) with `branch: main` to push changes directly. No branch, no PR, no CI cycle on the migration commit itself.
42+
- Script it. Expect to fix one or two things (for us: BSD sed `\b` didn't work, used `perl` instead; one repo had a non-standard filename).
43+
- Protocol 1 ("never commit to main") is deliberately bypassed here — admin authorization is explicit and documented in the plan.
44+
- Verify idempotency: script should detect `[SKIP]` when the target is already at the intended version.
45+
46+
Timebox: 5-10 min for a batch of 20-30 repos.
47+
48+
### Phase 3 — Optional cleanup (renames, filename normalization)
49+
50+
If the fleet has accumulated naming inconsistency (we had legacy `claude-code-review.yml` vs current `claude-blocking-review.yml`), clean it up now. Contents API doesn't support rename, so use: `PUT /new/path` + `DELETE /old/path` = two commits per repo.
51+
52+
### Phase 4 — Lock to specific versions + enable Dependabot
53+
54+
The final state. Three sub-steps:
55+
56+
- **4a — Ship any fix needed to make Dependabot PRs pass cleanly.** Dependabot auto-modifies caller workflow files, which triggers self-modification security skips in most review tooling. Source repo needs to handle this gracefully (skip, not fail). For us: `v2.0.1` shipped with a `.github/workflows/*.yml` auto-skip branch.
57+
- **4b — Bump every consumer to the specific semver** (`@v2``@v2.0.1`). Admin direct-push again.
58+
- **4c — Add `.github/dependabot.yml`** to each consumer. Fresh file if absent; skip-with-warning if present (don't touch existing YAML programmatically). Minimal config:
59+
60+
```yaml
61+
version: 2
62+
updates:
63+
- package-ecosystem: "github-actions"
64+
directory: "/"
65+
schedule:
66+
interval: "weekly"
67+
commit-message:
68+
prefix: "chore"
69+
```
70+
71+
After this, new versions of the source repo auto-PR themselves to consumers within a week. No more manual batch migrations.
72+
73+
## Pitfalls encountered
74+
75+
| Pitfall | How we found it | How we handled it |
76+
|---|---|---|
77+
| BSD sed `\b` is a no-op | Phase 2 first run reported "sed made no change" on all 23 repos | Switched to `perl -pe` for word-boundary regex |
78+
| Bypass marker grep mismatch | Docs said `[skip-claude-review: reason]` but code matched `[skip-claude-review]` | Fixed in Phase 0 (PR #39) before anything else |
79+
| `gh api` in wrapper scripts doesn't forward `--repo` | Phase 1 merges failed from wrong CWD | `cd` into each clone before calling `gh pr merge` |
80+
| `claude-code-action` refuses to run when caller workflow changes | Phase 1 bellwethers failed with "exceeded turn limit" after 30s | Phase 4a ships a skip for the specific failure mode |
81+
| Unanchored `VERDICT: X` grep false-matches review prose | PR #47 self-review's text quoted the grep pattern | Anchor to line start/end in the Check verdict step |
82+
| Content-adjacent config files in "doc-only" allowlist | Local reviewer flagged CODEOWNERS, dependabot.yml | Explicit NON-DOC exclusion for security-adjacent meta files |
83+
84+
## Timings observed
85+
86+
| Phase | Operations | Wall clock |
87+
|---|---|---|
88+
| Phase 0 (source hardening) | 7 PRs shipped | 4-6 hours spread over session |
89+
| Phase 1 (4 bellwether PRs) | 4 PR/merge cycles | ~45 min including bypass-marker diagnosis |
90+
| Phase 2 (23 repo batch) | 23 admin direct-pushes | 30 seconds |
91+
| Phase 3 (4 renames) | 8 API calls (create + delete per repo) | 15 seconds |
92+
| Phase 4 (27 pins + 27 dependabot.yml) | 54 admin direct-pushes | 45 seconds |
93+
94+
The PR-flow phases dominate. The admin batch phases are nearly free.
95+
96+
## Key decisions to make upfront (not mid-rollout)
97+
98+
- **Semver bump size** — major (v2) if behavior changes visibly, minor (v1.3) if purely additive. We chose major because the narrow-prompt behavior shift was user-visible.
99+
- **Keep the prior major maintained** — yes/no. We shipped `v1.2.4` as a terminal backport for the grep fix.
100+
- **Floating tag vs. specific tag + Dependabot** — floating is simpler short-term, Dependabot + specific is cleaner long-term. We chose Dependabot + specific once everything was migrated.
101+
- **Who reviews the source repo's own workflow-file edits after the skip lands** — local reviewers are the primary safety net. CI review will self-skip on workflow-file changes. Accept this before shipping v2.0.1.
102+
103+
## Not to reuse blindly
104+
105+
- The specific bypass marker `[skip-claude-review: reason]` is github-workflows specific.
106+
- The `claude-code-action` workflow-validation security skip is an upstream action's behavior; other tools won't have this exact issue but may have analogous self-modification concerns.
107+
- File-path allowlists (`*.md`, `docs/`, image extensions, etc.) are context-specific. `ralph-burndown` will have its own set of "doc-only" equivalents.
108+
109+
## Adapting for ralph-burndown (or any other tool)
110+
111+
The structural pattern transfers. To port:
112+
113+
1. **Identify the source repo.** What ships the tool? What's its current version contract?
114+
2. **Enumerate consumers.** `gh search code "uses: OWNER/REPO"` is a good start. Store the inventory in a comment or `.claude-review-ignore`-like file.
115+
3. **Identify the self-modification failure mode.** If the tool runs on PR events that touch its caller, confirm how it behaves. Often: skip with non-zero exit (needs Phase 4a fix). Occasionally: silent success (no fix needed).
116+
4. **Write the source-side fix.** What's the equivalent of the `.github/workflows/*.yml` skip? Same shape: detect the condition, short-circuit with a clear verdict/output, exit 0.
117+
5. **Pick bellwethers.** Three active repos + any template. Same rationale.
118+
6. **Run the Phases.** Mechanical once the adaptations above are in place.
119+
120+
The time-spent profile will look similar: most time in Phase 0 (source work) and Phase 1 (bellwether diagnosis); Phases 2–4 are scripted batch operations.

0 commit comments

Comments
 (0)