Skip to content

ci: add a separate 0.22 nightly channel - #4545

Open
thymikee wants to merge 7 commits into
mainfrom
chore/nightly-022-release-config
Open

thymikee wants to merge 7 commits into
mainfrom
chore/nightly-022-release-config

Conversation

@thymikee

@thymikee thymikee commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Summary

Add a separate 0.22 nightly at 01:53 UTC through the existing main-owned release workflow. Preserve main's 01:23 nightly, stable/latest behavior, permissions and private packages. Twelve files, 703 gross lines. Source prerequisite #4546 is integrated; exact 0.22 push CI passed.

The maintenance channel keeps exact source-SHA/green-push-CI admission, its own version suffix, dist-tag, daily deduplication and concurrency. Main retains its ahead-of-stable guard. Documentation describes the stamping entrypoint and source-owned helpers precisely.

Validation

Published head 48bad16. Direct pnpm 11.17.0 frozen install/build passed. One intended RED: 9 failed/40 passed; identical GREEN: 49 passed, same environment/two workers/fixed child clock. This is runtime-boundary regression evidence, not an independent reproduction of the old clock race.

One exact affected gate passed: 42 local checks/25 skips, exit 0, exactCommit=true, checkoutChanged=false, matching clean fingerprints, default origin/main and merge base d78be6c. Parent verified 147 evidence files and original/copy receipt/archive hashes. Full validation evidence.

Independent source/prose/test-design/security rechecks found NO FINDINGS at this head; prior low findings and documentation nit are fixed. CodeQL at this exact head failed with four HIGH cache-poisoning alerts, matching the previously reviewed annotations. Independent cache-model-gap assessments do not waive this failed check. Activation remains blocked pending security disposition; other hosted lanes remain unconfirmed here. Size passed: +44 B installed/unpacked, +16 B download; no performance clearance is claimed. No npm publication, tags, dispatch or external policy changes occurred.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.28 MB 6.28 MB +44 B
Package (unpacked) 6.28 MB 6.28 MB +44 B
Package (download) 1.90 MB 1.90 MB +16 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 19.7 ms 21.0 ms +1.3 ms
CLI --help 41.0 ms 40.8 ms -0.2 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/release-version.mjs Resolved
Comment thread scripts/release-plan.mjs Resolved
@apex-sentinel-staging

apex-sentinel-staging Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Apex Sentinel · ✅ Approve

Since the last review: 1 new · 0 still open · 1 no longer raised

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
scripts/release-plan.mjs:365 🆕 PR dry-run now builds the PR head commit instead of the merge ref

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command What it does
@apex-sentinel-staging note <context> Adds context; Sentinel re-checks the cited code
@apex-sentinel-staging pivot <why> Says the finding is wrong; Sentinel re-checks the cited code
@apex-sentinel-staging like Marks the finding as useful
@apex-sentinel-staging dislike <why> Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

Earlier reviews

Review of ae654e8b · 14.11.2026 13:01 UTC

Apex Sentinel · 🟠 Request changes

Since the last review: 0 new · 1 still open · 0 no longer raised

1 finding(s): 1 major.

Required fixes

  • 0.22 nightlies stop the moment main's stable release passes 0.22.0

🟠 Major (1)

Where Finding
scripts/release-plan.mjs:52 0.22 nightlies stop the moment main's stable release passes 0.22.0

1 finding below minor hidden (user_settings.min_severity).

1 new finding away from what changed since 3ea1e40 not shown. Comment @apex-sentinel-staging review for a full review.

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

Review of 3ea1e405 · 14.11.2026 12:41 UTC

Apex Sentinel · ✅ Approve

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
scripts/release-plan.mjs:156 assertAheadOfStable rejects a 0.22 nightly while main holds that release

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 3ea1e40. The verdict and the full list are in the summary comment.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 finding moved to a new line on ae654e8. The verdict and the full list are in the summary comment.

Comment thread scripts/release-plan.mjs Resolved
@thymikee

Copy link
Copy Markdown
Member Author

I found two problems at ae654e8 that need a fix before this can merge.

The 0.22 nightly is published by npm publish --provenance inside a run whose ref is refs/heads/main and whose SHA is main's HEAD (release.yml#L168, release-publish.mjs#L68). npm builds the provenance statement from that runner context, but the tarballs were built from the 0.22 commit. So each nightly-0.22 package would carry a signed attestation that names a main commit whose sources and version (0.21.x-dev) are not the published bytes. Users who trace a package to source through npm provenance would land on the wrong commit; only the nightly-0.22/v<version> tag records the real source. I did not inspect a published attestation, since no 0.22 nightly exists yet. This follows from npm's documented use of the GitHub runner context. The rule the code must satisfy: the provenance source commit names the commit the bytes were built from. Either publish the 0.22 channel from a run whose ref and SHA are the 0.22 commit, which needs npm-publish and trusted-publisher access for 0.22, or accept the limitation. To accept it, say in the CONTRIBUTING activation section and in the tag-nightly step summary that nightly-0.22 provenance names main's workflow commit and that the tag names the source, and get maintainer sign-off before activation.

assertAheadOfStable still applies to the 0.22 source (release-plan.mjs#L56). Once stable reaches 0.22.0 or later, scheduled 0.22 nightlies fail and manual ones throw. The fix in e9d4741 is not pushed at this head. Please push it, keeping the guard for main only and keeping the 0.22.X-dev and exact push-CI checks.

Not blocking, and you can take or leave these: (1) cache-mode: none is enforced by GitHub's cache service, but the build runs on a Blacksmith macOS runner whose docs describe a colocated cache and do not mention cache-mode, so code from the planned 0.22 checkout may still write entries that main restores; run the build on a GitHub-hosted macOS runner, or show that an actions/cache save under cache-mode none is refused on blacksmith-6vcpu-macos-26, then dismiss the CodeQL alerts with that justification (release.yml#L83); (2) the source and tag rule is checked in both sourceBranchForRun and planRelease, so keep it in planRelease and let sourceBranchForRun only map the event to a source (release-plan.mjs#L24); (3) commits/0.22 accepts any ref name, so a tag named 0.22 could shadow the branch; resolve refs/heads/0.22 explicitly (release-plan.mjs#L294); (4) the test titled "the release build has no cache token access and restores no dependency cache" only asserts the YAML input cache-store: 'false', so retitle it to what it checks (release-workflow.test.ts#L50); (5) NIGHTLY accepts both suffixes under either tag prefix, which is hardening only because just the workflow writes these tags (release-version.mjs#L13).

Would one exported NIGHTLY_CHANNELS table in scripts/release-version.mjs be simpler? Each row would hold cron, dist tag, version suffix, tag prefix and dev-version pattern, for example main and 0.22. Then nightlyVersion, latestNightlyTag, NIGHTLY, the remote tag patterns, sourceBranchForRun and planNightly would read from it, validation would live once in planRelease, and the workflow test would assert that the crons, choice options and concurrency key in release.yml match the table. Adding or retiring a maintenance line would become a one-row change instead of edits at about 14 sites. The provenance decision above comes first: if 0.22 must publish from its own ref, the table would carry the ref and the run-on-main design changes shape.

CodeQL reports 4 high alerts, all on lines this PR adds or makes reachable: release.yml lines 107, 113 and 119, and setup-node-pnpm/action.yml line 73. They run after the new checkout of the planned commit and stay open until dismissed with a justification, since CodeQL does not model cache-mode. The other 31 checks pass, including the Release dry-run Plan and Build. There are no conflicts. I did not rerun the local suites, I did not review e9d4741, and I did not read prerequisite PR #4546 beyond its file list. I also could not see classic branch protection or org rulesets with the access I have, so I did not confirm that main and 0.22 have the same writers.

On the open threads: the one on the stable-version guard still applies and is the second problem above, and the one on tag suffix acceptance still applies as hardening (note 5). The thread on the daily cron firing twice does not apply: each cron fires once per UTC day, the unchanged-SHA skip covers a repeat, and a second same-day publish needs a manual workflow_dispatch, which CONTRIBUTING documents as publish now. Please resolve it.

Before merge, push e9d4741 and settle the provenance stance for nightly-0.22. Then either show that cache-mode: none is enforced on the Blacksmith macOS build runner or move the build to a GitHub-hosted runner, so the CodeQL alerts can be closed with evidence.

@thymikee

Copy link
Copy Markdown
Member Author

Verified local evidence for unpublished candidate 443d483, based on published ae654e8:

  • Fresh direct pnpm 11.17.0 frozen install/build: exit 0. The prior shell-hop setup failure remains invalid evidence; it ran no tests.
  • Exactly one RED with candidate tests and only release-plan.mjs + release-version.mjs replaced by AE654 bytes: 9 failed, 40 passed, exit 1. Failures cover the stable-version boundary, foreign channel suffixes, and selected npm-channel daily guard. The 0.22 subprocess case reaches the old stable blocker before its dedupe assertion; no isolated subprocess dedupe proof is claimed for that case.
  • Exactly one GREEN with original candidate bytes restored: 49 passed, exit 0. Identical argv/environment, two workers. The cron/concurrency guard passes in both phases; the old workflow already had that behavior.
  • Exactly once: pnpm check:affected --run --head 443d483. Raw exit 0; 42 passed / 25 skipped, 67 selected, FAIL-OPEN retained; exactCommit=true, checkoutChanged=false, clean matching fingerprint 25233167b22a00807f04098580a0ea870e63870d8e806cb42fab6e3109f5d456.
  • Gate UTC 2026-11-14 14:49:06.684–14:59:06.646, 599.957 seconds. No printed lock-wait messages; actual acquisition wait unmeasured. Default base/merge base remained main d78be6c.
  • Original/copied receipt bytes match SHA256 901339ddc9eed39ec08f2d13754f32d61d874f8677ccc88844c2f627c7a17e0f. All 141 evidence files verified in the directory and immutable archive; archive SHA256 53515d8f26cadc9425acea990baefaaf541001ac1504fe13120aa7d92bdb32a7.

Independent source/security delta recheck is still running; no new-head hosted/security or activation clearance follows from these local results. Current published AE654 CodeQL failure remains red. #4546 is now merged into 0.22 as 0081f61; successful ci.yml push CI at that exact SHA remains a separate activation requirement. No tags, publication, dispatch or external policy changes occurred.

@thymikee

Copy link
Copy Markdown
Member Author

Exact-head evidence for 48bad16:

  • Direct pnpm 11.17.0 frozen install/build: both exit 0.
  • One intended RED using the old ae654 planner/version-owner bytes with unchanged deterministic-clock candidate tests: 9 failed/40 passed, exit 1. Restore candidate; identical argv/environment, two workers: GREEN 49 passed, exit 0. This tests the runtime boundary fixes; it does not independently reproduce the old midnight test race.
  • One default-base affected invocation: exit 0, 42 local checks passed/25 skipped, FAIL-OPEN full selection, exactCommit=true, checkoutChanged=false. origin/main and merge base d78be6c. Receipt UTC 15:39:15.396–15:49:08.269 on 2026-11-14. Matching clean fingerprints 51880fc452d8948ed2b206680d9a728ee799718e68ba4949e210bf5736a7751f.
  • Parent independently verified all 147 SHA256 manifest entries in both evidence directory and immutable archive, original/copy receipt bytes, candidate runtime/test Git blobs, equal RED/GREEN argv/environment and final clean logical index. Receipt SHA256 a62ca3207ad431553eb544615c36f85fa72a13c156cdbf4924ff66afe90e193d; archive SHA256 5b0c0de74c55d837dc4824a1c800d26c9a58006fe184dc621b8b4ec217ebad15. No printed lock wait; actual wait unmeasured.
  • Independent Apex source/prose/test-design/security recheck found NO FINDINGS at this exact candidate, verifying the two low findings and documentation nit fixed. Planner/version runtime bytes match reviewed 443; follow-up is four files +18/-6.

Source prerequisite #4546 is integrated and exact resulting 0.22 push CI succeeded: https://github.com/callstack/agent-device/actions/runs/43607376172

New-head hosted/security admission remains pending. The four HIGH CodeQL failures at ae654 remain recorded; independent model-gap assessments do not suppress or waive them. No activation, npm publication, tag, dispatch or external policy change occurred.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 48bad16. The verdict and the full list are in the summary comment.

Comment thread scripts/release-plan.mjs Resolved
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. At 48bad16 the other reviewers' inline threads are addressed, but the nightly-0.22 provenance problem from the earlier review is still open, and so are the CodeQL alerts.

A nightly-0.22 run executes the publish job with GITHUB_REF=refs/heads/main and GITHUB_SHA set to main's workflow commit. release-publish.mjs runs npm publish --provenance, and npm builds the signed attestation from that runner context. The tarballs were built from the 0.22 commit (release.yml:95). So each nightly-0.22 package would carry a signed statement that names a main commit, not the commit the bytes came from, and anyone tracing a package through npm provenance lands on the wrong source. CONTRIBUTING.md:199-200 says merging activates the 01:53 schedule, so merge is when this ships. The rule the code must satisfy is that the provenance source commit names the commit the bytes were built from, or the gap is documented and signed off. Either publish nightly-0.22 from a run whose ref and SHA are the 0.22 commit (this needs npm-publish environment and trusted-publisher access for 0.22), or state in the CONTRIBUTING activation section and in the tag-nightly summary that the attestation names main's workflow commit and the nightly-0.22/v<version> tag names the source, with maintainer sign-off before merge.

CodeQL still fails with four HIGH cache-poisoning alerts, at release.yml lines 107, 113 and 119 and .github/actions/setup-node-pnpm/action.yml:73. They look related to this PR: the build job checks out the planned commit at release.yml:91-95 and then runs local actions and builds from the 0.22 source. This head only renames a step, so runs-on and cache-mode: none are unchanged and the alerts still apply. I did not capture a CodeQL excerpt for this head, and I did not verify how the Blacksmith macOS runner treats cache-mode: none. Please show that cache-mode: none blocks cache saves on that runner, or move the build to a GitHub-hosted runner, then dismiss the alerts with that justification.

Not blocking, and fine to take or leave: in a scheduled run, readRun waits for settled CI before the npm-day skip, which only needs latestNightly and today's date. For 0.22 with no push run yet, that means up to 40 x 30 s of polling before the skip, so reading latestNightly first and returning early would avoid it.

The PR now adds four more per-channel special cases: the nightlyDistTag ternary, the matchNightlyVersion suffix ternary, sourceBranch === 'main' around assertAheadOfStable, and the main-only latest read. Would one exported NIGHTLY_CHANNELS table in scripts/release-version.mjs, keyed by source (main, 0.22), be simpler? Each row would hold cron, distTag, versionSuffix, tagPrefix, devVersionPattern, guardsAgainstStable and allowsVersionOnlyCiFallback. nightlyVersion, matchNightlyVersion, latestNightlyTag, nightlyDistTag, the ls-remote patterns, sourceBranchForRun, readPublishedVersions, ciConclusion and planNightly would read the row, and release-workflow.test.ts would assert that the crons, choice options and concurrency key match the table. Adding or retiring a maintenance line would then be a one-row change. This was asked in the earlier review and is still unanswered. It depends on the provenance decision: if 0.22 must publish from its own ref, each row also carries the publishing ref.

The local suites and the RED/GREEN runs were not rerun here, and the 0.22 push CI run was not opened. I judged the planner changes from the diff against ae654e8.

For the other threads: matchNightlyVersion channel suffix, npm-day skip and assertAheadOfStable on main only are fixed at this head, so you can resolve them. The merge-commit build thread is benign: on pull_request GITHUB_SHA is the merge commit, so it builds the merge result, which is expected.

Before merge, please settle the nightly-0.22 provenance question and close the four CodeQL alerts with evidence.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant