Skip to content

Add GitHub App front end - #124

Merged
larsoner merged 7 commits into
scientific-python:masterfrom
larsoner:app-prototype
Jul 28, 2026
Merged

Add GitHub App front end#124
larsoner merged 7 commits into
scientific-python:masterfrom
larsoner:app-prototype

Conversation

@larsoner

@larsoner larsoner commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

A working prototype of the GitHub App idea from #27, so the conversation with the scientific-python infra folks can be about a real thing with real numbers.

What it is

  • src/core.js — the resolution logic, extracted unchanged. Runtime-neutral (only global fetch, which is why dropping node-fetch in MAINT: Drop node-fetch and eslintrc, fix package metadata #118 mattered).
  • src/config.js — one set of option names and defaults for both front ends.
  • index.js — the action, now a thin wrapper. Its 22 tests pass untouched, which is the evidence the refactor is faithful.
  • worker/index.js — the App: a Cloudflare Worker handling status webhooks (~180 lines: signature check, JWT, installation token, config read, post status, plus per-isolate caching).

A repo using the App has no workflow, so it produces no workflow runs at all.

Config

.github/circleci-artifacts.yml is the old with: block, dedented, minus repo-token:

artifact-path: 0/doc/index.html
circleci-jobs: build_docs
job-title: Check the rendered docs here!

Read from the default branch, never the event's ref — otherwise a forked PR could point domain: at a host it controls and have the App post a trusted-looking green link to it. This preserves a property on: status gives you today by accident.

Expected traffic (scikit-learn, MNE-Python, SciPy)

Measured over the last 7 days. Pipelines from the CircleCI API; statuses-per-commit from /commits/{sha}/statuses over a 20-commit sample per repo (12 recent PR heads + 8 default-branch commits).

CircleCI pipelines/wk statuses/commit webhook deliveries/wk
scikit-learn 206 14.6 ~3,000
MNE-Python 199 14.5 ~2,900
SciPy 160 14.2 ~2,300
combined ~8,200/wk = ~1,170/day = ~49/hr

Against Cloudflare Workers' free tier (100,000 requests/day) that is 1.2% of the daily allowance for all three projects together. Twenty repos of that size would be ~8%.

Not every delivery costs the same. Roughly 80% exit after an HMAC check and a string comparison — no subrequests. Of scikit-learn's 14.6 statuses/commit, 11.2 are CircleCI's but only ~2.8 are the watched job; the rest are codecov, other CircleCI jobs, and the redirector's own statuses.

Effect of the per-isolate cache

The installation token (50 min) and repo config (10 min) are memoized in a Map, so an unwatched CircleCI job no longer costs two API calls:

subrequests/wk before after
scikit-learn 5,770 2,360
MNE-Python 4,380 2,560
SciPy 4,560 1,420
combined ~14,700 ~6,300 (-57%)

Those "after" figures assume warm isolates; Cloudflare recycles them, so reality sits between the two columns — a cold isolate simply fetches again, which is why nothing here is correctness-critical. Failures are evicted rather than cached, and storing the promise means concurrent events for one repo share a single request.

The free plan's 10 ms CPU limit is the real constraint, not the request count; the only meaningful CPU is one RSA-2048 signature, now amortized across ~50 minutes of events per installation.

Verification

41 tests, 100% line/branch/function coverage across all four files, under the existing CI floor.

Beyond the unit tests, I ran the Worker against the live CircleCI API with only GitHub faked, using real job 361 from this repo:

worker response: 200 posted success: https://output.circle-artifacts.com/output/job/982be583-.../root_artifact.md
status it would post: {
  state: 'success',
  target_url: 'https://output.circle-artifacts.com/output/job/982be583-.../root_artifact.md',
  description: 'Link to 0/test_artifacts/root_artifact.md',
  context: 'Docs preview'
}

That URL returns 200, and it is the same URL the action produces for the same job.

The JWT test is also real rather than mocked: it generates an RSA key, signs through Web Crypto exactly as the Worker does, and verifies the signature with node:crypto — i.e. GitHub would accept it. The cache tests drive an injected clock across both TTLs and assert that a failed mint is retried rather than served from cache.

What is not here

  • No deployment. No Cloudflare account, no App registered, no wrangler deploy workflow, no API token in this repo's secrets. Merging changes nothing operationally.
  • No private CircleCI projects (would need server-side token storage) and no url output (no workflow step to consume it). The action remains the answer for both.
  • The GitHub App private key must be converted to PKCS#8 (openssl pkcs8 -topk8 …) since Web Crypto will not import GitHub's PKCS#1 download. Noted at the top of worker/index.js.

Suggested next step

If infra says yes: register the App, wrangler secret put three times, and install it on this repo only, alongside the existing action under a different job-title. Every commit then gives a side-by-side comparison of the two paths at zero risk to anyone else.

🤖 Generated with Claude Code

larsoner and others added 4 commits July 28, 2026 14:09
Extract the resolution logic into src/core.js and the option handling into
src/config.js, then add a second front end: worker/index.js, a Cloudflare
Worker that serves a GitHub App reacting to status webhooks directly. A repo
using the App has no workflow, so it gets no workflow runs at all, which is
the complaint in scientific-pythongh-27.

The action keeps working exactly as before; index.js is now a thin wrapper
over the same core, and its 22 tests pass unchanged.

Config for the App lives in .github/circleci-artifacts.yml, read from the
default branch so a forked PR cannot redirect the link it posts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cloudflare reuses an isolate across many requests, so memoizing the
installation token (50 min) and the repo config (10 min) in a plain Map
removes most of the GitHub API traffic without needing KV. Failures are
evicted rather than cached, and storing the promise means concurrent
events for one repo share a single request.

Cuts subrequests for scikit-learn + MNE-Python + SciPy from ~14,700/wk
to ~6,300/wk when isolates are warm; a cold isolate simply fetches again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Web Crypto rejects a zero-length HMAC key, so a missing or empty secret
threw and surfaced as a 500 instead of rejecting the delivery. Found by a
real webhook delivery while wiring up the app: GitHub reported 500 where
it should have been a 401.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every status posted is itself a status event that comes back around
(scientific-pythongh-27), so the 'Waiting for CircleCI ...' status doubles the traffic a
repo generates. The action gains a post-pending option, defaulting to
true so existing setups are unchanged; the app hard-codes it off, since
the final status says everything the pending one did.

When off, the pending event returns before the CircleCI call, so it costs
no API traffic either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CircleCI delivered the same build_docs status twice during live testing on
LABSN/expyfun, and the app posted an identical status for each. GitHub only
shows the latest per context, so it is invisible in the UI, but it doubles
both the posts and the status events they generate (scientific-pythongh-27).

Keyed on repo, sha, context, state and the resolved URL, so a re-run that
produces different artifacts still posts. Best-effort: two simultaneous
duplicates can both miss, and a cold isolate forgets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@larsoner

Copy link
Copy Markdown
Collaborator Author

Now validated on live infrastructure

This is no longer only a prototype on paper: it has been running against real repositories for the last hour, on a personal Cloudflare Workers free-tier deployment plus the (dormant since 2019) circleci-artifacts-redirector GitHub App, whose permissions — statuses: write, contents: read, subscribed to status — turned out to be exactly what this needs.

Test subject: LABSN/expyfun, which swapped its workflow for .github/circleci-artifacts.yml via a literal git mv + dedent, and LABSN/expyfun#583 as a fork PR.

Property How it was shown
Whole chain works posted success on real commits: webhook → HMAC verify → JWT → installation token → config read → CircleCI → status posted
Artifact URL correct Identical to what the action posts, verified 200
Fork PRs work Build on pull/583, link posted to the PR head
No api-token needed Fork build ran with forks-receive-secret-env-vars: false; unauthenticated CircleCI call succeeded
Config comes from the default branch Fork branch said job-title: SHOULD NOT APPEAR; zero statuses contained it — main's title was used
No pending status Pending events produced no status on two separate builds
Cancelled builds An auto-cancelled build posted failure / No artifacts found, correct under #57

The config-source result is the one I most wanted: a contributor cannot redirect where the app's link points, demonstrated rather than asserted.

Three changes the live run produced

  • c5498bc — a missing WEBHOOK_SECRET threw (Web Crypto rejects a zero-length HMAC key) and surfaced as a 500. Found because a real delivery from a repo still carrying the 2019 installation hit it. Now fails closed with 401.
  • 629f26cpost-pending. Every status posted is itself a status event, so the "Waiting for CircleCI ..." status doubles a repo's traffic. The action gains the option (default true, so nothing changes for existing users); the app hard-codes it off. When off, pending events return before the CircleCI call, so they cost no API traffic either.
  • cba19cc — CircleCI delivered the same build_docs status twice and the app posted twice. Invisible in the UI (GitHub shows only the latest per context) but it doubles the events generated. Now deduplicated on repo/sha/context/state/URL for 5 minutes, so a re-run with different artifacts still posts.

Each was mutation-tested: reverting the fix individually makes exactly the intended test fail.

State

48 tests, 100% line/branch/function coverage across all four files, under the existing CI floor.

Still not deployed for scientific-python — the running instance is a personal Cloudflare account and a personally-owned App. Both are transferable: GitHub App ownership transfers preserve installations, and the Worker is stateless, so moving it is wrangler deploy plus three secrets plus one webhook URL change.

Traffic estimate in the description above is unchanged: ~8,200 deliveries/week for scikit-learn + MNE-Python + SciPy combined, about 1.2% of the Workers free tier. The two changes above cut what the app itself contributes to that.

Working notes for agents: the shared-core layout, the commands, and the
gotchas that cost real debugging time (the Circle-Token 401, the status
semantics from scientific-pythongh-57, why exact artifact-path matching was declined, why
the app must read config from the default branch, and the fork-is-a-
CircleCI-project trap).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@larsoner larsoner changed the title ENH: Prototype a GitHub App front end (gh-27) Add GitHub App front end Jul 28, 2026
Migrating means git mv-ing the old workflow, which is called
circle_artifacts.yml in SciPy and circle-artifacts.yml in MNE-Python, so
requiring one exact name makes the migration silently no-op.

Find the config by listing .github/ and matching
circle(ci)?[-_]artifacts.ya?ml, preferring the documented spelling when a
repo has several. Costs one extra API call when a config exists, which the
10 minute cache absorbs; a repo with no config now costs one call rather
than one per candidate name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@larsoner
larsoner merged commit 3faa720 into scientific-python:master Jul 28, 2026
6 checks passed
@larsoner
larsoner deleted the app-prototype branch July 28, 2026 20:27
This was referenced Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant