fix(release): make registry publishing re-runnable and independent (#4)

The v0.1.2 release published five of six npm platform packages, tripped npm's
spam heuristic on the sixth, and took PyPI down with it. Three separate faults,
all in how the publish steps were wired rather than in the packages themselves.

npm publishing moves into packaging/npm/publish.sh
- npm's spam detection refused the sixth similarly-named package in a row with
  `403 Package name triggered spam detection`. It is rate-shaped, not permanent,
  so publishes are paced and that specific failure is retried with backoff.
- npm refuses to republish an existing version, so a naive retry of a partial
  release fails on everything that already succeeded and never reaches what
  did not. Versions already on the registry are skipped, which makes re-running
  the release job the correct recovery.
- A conflict raised between the check and the publish is treated as success
  rather than an error; it means a concurrent run got there first.
- Platform-before-entry ordering is unchanged and now enforced in one place.

PyPI no longer depends on npm succeeding
- They are independent registries. `if: always()` on the upload, and
  `--skip-existing` so a re-run after a partial upload is safe rather than a
  guaranteed failure on the versions that already landed.
- Also fixes a repeat of an earlier mistake: `secrets` is not an available
  context in a step-level `if`, so the guard reads the mapped env var.

Verified the skip and retry logic against the live registry: it identifies
exactly the two packages v0.1.2 left unpublished, classifies spam and 429 as
retryable, publish-conflict as already-done, and everything else as fatal.

Author: adhikjoshi

.github/workflows/release.yml

@@ -82,15 +82,7 @@ jobs:
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
         if: ${{ env.NODE_AUTH_TOKEN != '' }}
-        run: |
-          set -euo pipefail
-          # Platform packages first: the entry package depends on them, and an
-          # entry published against versions that do not exist yet is an install
-          # that fails for everyone until the next step lands.
-          for pkg in dist/npm/modelslab-cli-*; do
-            npm publish "$pkg" --access public --provenance
-          done
-          npm publish dist/npm/modelslab-cli --access public --provenance
+        run: bash packaging/npm/publish.sh dist/npm
 
       - uses: actions/setup-python@v5
         with:
@@ -99,13 +91,20 @@ jobs:
       - name: Build PyPI wheels
         run: python3 packaging/pypi/build.py "${GITHUB_REF_NAME}" artifacts dist/pypi
 
+      # `if: always()` because npm and PyPI are independent registries and a
+      # failure at one is not a reason to skip the other. v0.1.2 published five
+      # npm packages, tripped npm's spam heuristic on the sixth, and PyPI never
+      # ran at all — one registry's flakiness took the whole release with it.
       - name: Publish to PyPI
         env:
           TWINE_USERNAME: __token__
           TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
-        if: ${{ env.TWINE_PASSWORD != '' }}
+        # `secrets` is not an available context in a step-level `if` — hence env.
+        if: ${{ always() && env.TWINE_PASSWORD != '' }}
         run: |
           set -euo pipefail
           python3 -m pip install --quiet twine
           python3 -m twine check dist/pypi/*.whl
-          python3 -m twine upload dist/pypi/*.whl
+          # --skip-existing so a re-run after a partial failure is safe; PyPI
+          # rejects a repeated version outright and would fail the retry.
+          python3 -m twine upload --skip-existing dist/pypi/*.whl

packaging/README.md

@@ -34,9 +34,20 @@ rejected deliberately. It needs network at install time and produces a silently
 broken install under `npm ci --ignore-scripts`, which many CI and agent sandboxes
 set. Six small packages buy an install that cannot half-work.
 
-**Publish platform packages before the entry package.** The entry package pins
-exact versions of all six; publishing it first leaves a window where every
-install fails.
+Publishing goes through `packaging/npm/publish.sh`, which exists because two
+things bit the v0.1.2 release:
+
+- **Order.** The entry package pins exact versions of all six platform packages,
+  so publishing it first leaves a window where every install fails. Platform
+  packages go first.
+- **npm's spam heuristic.** Six similarly-named packages published back to back
+  tripped it on the sixth with `403 Package name triggered spam detection`. It is
+  rate-shaped rather than permanent, so the script paces publishes and retries
+  that specific failure with backoff.
+- **Re-running.** npm refuses to republish an existing version, so a naive retry
+  of a half-finished release fails on the packages that succeeded and never
+  reaches the ones that did not. The script skips versions already on the
+  registry, which makes a re-run the correct recovery for a partial publish.
 
 ## PyPI — `packaging/pypi/build.py`
 
@@ -67,7 +78,11 @@ Two things that are easy to get wrong and are covered by CI:
 
 `.github/workflows/release.yml` runs both builders on a tag and publishes if the
 corresponding token is configured. Missing tokens skip that registry rather than
-failing the release.
+failing the release, and the PyPI step runs even when npm fails — they are
+independent registries, and in v0.1.2 an npm failure meant PyPI never ran at all.
+
+Re-running the release job is the supported recovery for a partial publish: both
+publishers skip what is already on their registry.
 
 | Secret | Registry |
 | --- | --- |

packaging/npm/publish.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+#
+# Publishes the built npm packages, safe to re-run.
+#
+# Two things the naive `for pkg in *; do npm publish; done` gets wrong, both of
+# which bit the v0.1.2 release:
+#
+#   1. npm's spam heuristic. Publishing six similarly-named packages back to back
+#      tripped it on the sixth with a 403 "Package name triggered spam
+#      detection". It is rate-shaped, not permanent, so a paced retry clears it.
+#
+#   2. Re-running after a partial failure. npm refuses to republish a version
+#      that already exists, so a retry of a half-finished release fails on the
+#      packages that DID succeed and never reaches the ones that did not.
+#
+# Ordering still matters: platform packages before the entry package, which pins
+# exact versions of all of them. Publishing the entry first leaves a window where
+# every install fails.
+set -euo pipefail
+
+DIST="${1:?usage: publish.sh <dist/npm dir>}"
+ENTRY="modelslab-cli"
+MAX_ATTEMPTS=5
+
+already_published() {
+    local name="$1" version="$2"
+    npm view "${name}@${version}" version >/dev/null 2>&1
+}
+
+publish_one() {
+    local dir="$1"
+    local name version attempt delay output
+    name=$(node -p "require('${dir}/package.json').name")
+    version=$(node -p "require('${dir}/package.json').version")
+
+    if already_published "$name" "$version"; then
+        echo "skip    ${name}@${version} (already on the registry)"
+        return 0
+    fi
+
+    for attempt in $(seq 1 "$MAX_ATTEMPTS"); do
+        if output=$(npm publish "$dir" --access public --provenance 2>&1); then
+            echo "$output"
+            echo "publish ${name}@${version}"
+            return 0
+        fi
+
+        echo "$output"
+
+        # A version that appeared between the check and the publish is a success
+        # for our purposes — most likely a concurrent or retried run.
+        if grep -qi "cannot publish over\|EPUBLISHCONFLICT" <<<"$output"; then
+            echo "skip    ${name}@${version} (published concurrently)"
+            return 0
+        fi
+
+        if ! grep -qi "spam detection\|429\|rate.limit" <<<"$output"; then
+            echo "fatal   ${name}@${version}: not a retryable error" >&2
+            return 1
+        fi
+
+        delay=$((attempt * 30))
+        echo "retry   ${name}@${version} in ${delay}s (attempt ${attempt}/${MAX_ATTEMPTS}, spam/rate heuristic)" >&2
+        sleep "$delay"
+    done
+
+    echo "fatal   ${name}@${version}: still refused after ${MAX_ATTEMPTS} attempts" >&2
+    return 1
+}
+
+for dir in "${DIST}"/${ENTRY}-*; do
+    [ -d "$dir" ] || continue
+    publish_one "$dir"
+    # Pace the platform packages. Publishing them as fast as the API allows is
+    # what looks like spam in the first place.
+    sleep 10
+done
+
+publish_one "${DIST}/${ENTRY}"
+
+echo "npm publish complete"