Skip to content

fix(wizard-ci): fix the known-broken CI legs and log links - #4326

Merged
edwinyjlim merged 1 commit into
mainfrom
posthog/ci-easy-fixes
Oct 5, 2026
Merged

edwinyjlim merged 1 commit into
mainfrom
posthog/ci-easy-fixes

Conversation

@gewenyu99

Copy link
Copy Markdown
Contributor

Problem

Wizard CI runs on release PRs fail the same way every time, from causes that are cheap to fix.

Changes

  • Logs links. "Get job ID" read only the first page of jobs (30 of 51) and matched by substring. Many rows had an empty logs link, and next-stripe linked to next-stripe-declined. The step now pages through all jobs and matches wizard-ci (<app>) exactly.
  • Signed commit 504s. createCommitOnBranch now retries 5xx up to 3 times. After each 5xx it waits, then checks the branch. If GitHub already wrote the commit, it uses that commit and does not resend, because a resend would fail on expectedHeadOid.
  • Stripe fixture. The data-warehouse-source skill asks for a restricted rk_live_ key, so the agent refused sk_test_… and created no Stripe source. The fixture now has a restricted-key shape.
  • Seeded legs. posthog-integration has no askAnswers, so the Stripe questions fell to the sentinel. The leg now passes the warehouse program's rules through E2E_ANSWERS_FILE. "report written" is skipped when the program declares no report file.
  • Checks now runs api-commits.test.ts. It was never run in CI before.

Left out on purpose. These are real regressions or need a decision:

  • multi-source-next expects Postgres and Supabase as two sources, and wizard [CI] (48ac55f) nuxt/movies-nuxt-4 #1300 folds them into one.
  • zero-source gets no abort. The driver goes past the no-sources screen.
  • Submodule apps (audit/*, flutter/localsend, migrate/triplex, self-driving/turbo-monorepo) can never open a PR, because a gitlink can't be committed. migrate also needs --product, and self-driving refuses CI mode.
  • The weekly scheduled run fails on the bot key's scopes (external_data_source:read, feature-flag read). That is a secrets change.

Test results

  • pnpm test:api-commits: 6/6 pass. The landed-commit test fails with the branch check removed.
  • pnpm test:warehouse-checks 98/98, pnpm test:mcp-stub 47/47, tsc --noEmit clean.
  • New job lookup against run 36541121714: node-weather and posthog-demo-3000 went from empty to 109317185146 and 109317185224, and next-stripe now gets its own 109317184957.

Sources

🤖 Generated with Claude Code

- Get job ID: page through every job and match the name exactly. A full
  run has 51 jobs, past the API's first page of 30, so many rows got an
  empty logs link, and next-stripe matched next-stripe-declined.
- Retry createCommitOnBranch on 5xx. After a 504, check the branch first:
  if GitHub already wrote the commit, use it instead of resending.
- Use a restricted-key shape for the Stripe fixture. The warehouse skill
  makes the agent refuse an sk_ key, so no Stripe source was created.
- Seeded legs: pass the warehouse program's askAnswers through
  E2E_ANSWERS_FILE, and skip "report written" when the program declares
  no report file.
- Run the api-commits tests in Checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 marked this pull request as ready for review October 5, 2026 09:53
@gewenyu99
gewenyu99 requested a review from a team as a code owner October 5, 2026 09:53
@edwinyjlim
edwinyjlim merged commit 2be6bc6 into main Oct 5, 2026
11 checks passed
@edwinyjlim
edwinyjlim deleted the posthog/ci-easy-fixes branch October 5, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants