Skip to content

Docs: add reusable remote docs preview workflow - #12

Open
Blargian wants to merge 8 commits into
ClickHouse:mainfrom
Blargian:codex/remote-docs-preview
Open

Blargian wants to merge 8 commits into
ClickHouse:mainfrom
Blargian:codex/remote-docs-preview

Conversation

@Blargian

@Blargian Blargian commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

Adds a reusable remote documentation preview workflow. A constrained pull_request_target caller pins that pull request's exact head SHA and directly creates a Vercel deployment of trusted Nimbus main in the connect-preview Custom Environment.

A maintainer can get a docs preview by adding the docs-preview label - this is done both to save unnecessary preview builds and to ensure there's a gate on what code gets run. JS can run inside of MDX so there is a security concern there:

  • This workflow triggers a Vercel build, passing the repo and PR ref
  • The docs site uses a Vercel Connect app with a short-lived token to pull the docs from that PR
  • Docs build inside the Vercel environment
  • The workflow polls for the deployment status and updates a comment with the public preview link

I have taken care to separate out MDX processing from the part of the build process which fetches the docs content from remote sources. The label acts as gate keeping mainly for community PRs.

The Actions workflow never checks out or executes pull-request content and never passes a GitHub credential into Vercel. During the build, Vercel Connect exchanges the deployment OIDC identity for a short-lived contents:read token scoped to the selected source repository. The workflow waits for completion, cancels superseded deployments, and posts a sticky preview link on the source pull request.

Callers receive VERCEL_TOKEN, VERCEL_ORG_ID, and VERCEL_PROJECT_ID through organization Actions secrets. A copy-paste caller and configuration documentation are included. Each caller can use the native paths filter to select which changes are eligible, for example docs/**. A new PR commit requires a new label event, so approval remains bound to the SHA selected when the label was added.

Registering a remote source

Before installing the caller, add an entry to remotes.json in ClickHouse/mintlify-docs-dev. Set name to the value passed as remote_name, repo to the source repository, path to its documentation directory, and mount to its destination relative to /docs. Add assets mappings and private: true when required. Do not pin a branch in the registry: production reads the source's default branch, while an approved preview supplies the exact pull-request SHA.

@Blargian Blargian changed the title Add reusable remote docs preview dispatcher Add reusable remote docs preview workflow Sep 12, 2026
@Blargian

Copy link
Copy Markdown
Member Author

Disclaimer:

This works for a custom Astro build of the docs (https://clickhouse-docs.vercel.app/docs). It's not actually triggering a Mintlify deploy. The two sites are visually and functionally identical though so we can use this for now as a work around for Mintlify's short-comings.

DOCS_REMOTE_NAME: ${{ inputs.remote_name }}
DOCS_REMOTE_REF: ${{ steps.request.outputs.head_sha }}
DOCS_REMOTE_REPOSITORY: ${{ github.repository }}
DOCS_REMOTE_SOURCE_REPOSITORY: ${{ steps.request.outputs.head_repository }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this work with community forks? Not sure if the process would be able to work with forks where the vercel app is not installed 🤔

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it would work as is. The suggestion to use refs/pull/N/head makes sense to me - pushed a change.

It would work like this:

  1. Maintainer adds the docs-preview label. This is done as a first gate, as it might be a PR to fix a typo or something not requiring a preview.
  2. Workflow triggers a preview build in Vercel. Using Vercel Connect app, the docs site from ClickHouse/ClickHouse is fetched (currently what is in development at https://github.com/ClickHouse/mintlify-docs-dev), it also fetches the markdown files from dbt-clickhouse branch using refs/pull/N/head. We use a dedicated Vercel environment containing only the necessary Vercel Connect secret. The Connect app mints short lived OIDC tokens and we separate out the Git fetch process from the build process. Tokens are cleaned before any .mdx is built.
  3. Workflow polls for the preview link and posts it as a comment

@Blargian

Blargian commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

Blocking

  1. Approval doesn't bind to the labeled SHA. .github/workflows/remote-docs-preview.yml:92 re-resolves the head SHA from the live PR at run time instead of using github.event.pull_request.head.sha. A contributor can push malicious MDX between the maintainer adding the label and the job reaching that step, and the preview builds the unapproved revision. Read the SHA from the event payload and fail if the PR head has moved.

  2. Fork PRs can't work as written. .github/workflows/remote-docs-preview.yml:124 passes the fork's full name as the source repository. Vercel Connect can't mint a token for a repo it isn't installed on, so the primary community-PR use case fails at clone time. Fetch refs/pull/N/head from the base repo and treat the fork name as metadata only.

  3. Cancellation cleanup is effectively dead. At .github/workflows/remote-docs-preview.yml:349, sleep 10 runs in the foreground, so bash defers the trap until the sleep ends. Confirmed empirically: SIGINT at 1s, trap fired at 6s. GitHub kills the tree 10s after SIGINT, so the Vercel cancel PATCH rarely runs and the deployment keeps burning a build slot. Use sleep 10 & wait $! and ignore TERM inside the cleanup body.

  4. Polling budget exceeds the job timeout. 360 iterations of 10s plus latency at .github/workflows/remote-docs-preview.yml:328 outlasts timeout-minutes: 60, so the handled timeout branch is unreachable and slow builds end in a raw GitHub kill with no cleanup.

Should fix

  1. Superseded-cancel failures abort the new preview. .github/workflows/remote-docs-preview.yml:222 runs under set -e. One wedged deployment that Vercel refuses to cancel permanently blocks previews for that PR. Downgrade to a warning. Conversely, the mapfile < <(jq ...) at line 210 swallows jq errors that should fail loudly.

  2. Preview URL is published before it's validated. The output is written at line 325 but the regex guard runs at line 362, so consumers with if: always() see the unvalidated value.

  3. Sticky comment marker isn't namespaced. .github/workflows/remote-docs-preview.yml:377 uses one constant marker and omits the SHA, so two callers in one repo overwrite each other and stale previews look current.

  4. Docs site identity is hardcoded in three places. Lines 55, 284-286, and 94 each name the repo or branch independently. The PR discussion already notes the target is temporary, so a partial update would pin one repo's SHA and deploy another's.

Minor

  1. contents: read at line 40 is unused in a pull_request_target job. Drop it.
  2. Three paginated list calls per state at line 184 could be one call with state=QUEUED,INITIALIZING,BUILDING.
  3. The sticky-comment upsert at line 380 duplicates the one in claude-pr-triage.yml lines 303-335 with a looser contains matcher.
  4. examples/caller-remote-docs-preview.yml:32 ships a real-looking remote_name: clickhouse-private instead of a placeholder.
  5. The security rationale is written out in five places, including .github/zizmor.yml:17. Findings 1 and 2 already show the prose and code disagreeing. Keep it once in the README with pointers elsewhere.
# Status Assessment
1 Resolved Uses the event SHA and rejects runs if the live PR head moved.
2 Resolved Central PR #363 is merged: Vercel fetches the base repository’s refs/pull/<N>/head and verifies its SHA.
3 Resolved Poll sleep is interruptible; cleanup ignores follow-up termination signals while cancelling.
4 Resolved Polling stops at 40 minutes, which is also the maximum Vercel deployment time.
5 Resolved Invalid deployment-list JSON fails; superseded cancellation failures warn and continue.
6 Resolved URL validation happens before the workflow output is written.
7 Resolved Sticky comment marker includes remote name and SHA; the visible text identifies the revision.
8 Resolved Central docs repository/ref live in one job-level configuration.
9 Resolved Unused contents: read was removed.
10 Deferred Still makes one Vercel list request per active state. Minor efficiency issue.
11 Intentionally partial Lookup is anchored with startswith; duplicated upsert logic remains per the narrow scope.
12 Resolved Example uses example-docs.
13 Resolved in code/docs README contains the detailed rationale; workflow/example/zizmor point to it. The PR description still needs its permanent-label wording updated.

Comment thread README.md
### `remote-docs-preview.yml` - Request a remote documentation preview

Starts a scoped documentation preview for a pull request in a repository
registered by `ClickHouse/mintlify-docs-dev/remotes.json`. A maintainer enables

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to ClickHouse/ClickHouse/docs, testing in this repo for now

@Blargian Blargian changed the title Add reusable remote docs preview workflow Docs: add reusable remote docs preview workflow Oct 8, 2026
@Blargian
Blargian requested a review from koletzilla October 9, 2026 11:53
@Blargian

Copy link
Copy Markdown
Member Author

Second review feedback addressed in 679beea

Area Status Result
Approved revision binding Addressed Uses the event SHA and verifies it still matches the live PR head.
Fork PR support Addressed Central docs fetches refs/pull/<N>/head from the base repo; fork identity is metadata only.
Cancellation on interruption Addressed Poll sleep is interruptible and cleanup protects its cancellation request from TERM.
Polling / time budget Addressed A Vercel build gets up to 40 minutes—the platform maximum—while the job preserves time for queueing and cleanup.
Superseded deployment failures Addressed Failure to list/cancel an old deployment becomes a warning; malformed Vercel responses still fail loudly.
Preview URL publication Addressed The URL is only exposed after Vercel reports READY.
Comment identity and updates Addressed Marker is remote-specific, SHA is shown in the comment body, and the query uses a safe jq argument.
Comment-upsert duplication Open cleanup The behavior is correct, but the upsert logic remains duplicated with the triage workflow.
Central docs identity Addressed Repository/ref are defined once as trusted workflow constants and derived fields use them.
Make docs repository caller-configurable Not adopted Intentionally kept as a trusted constant; this is not a security issue.
Permissions and example hygiene Addressed Removed unused contents: read; example no longer suggests a real private remote.
Vercel request timeouts Addressed All Vercel calls have connection and total-time limits.
Isolation between different remotes Addressed Both concurrency and supersession matching include remote_name.
Terminal deployment cleanup Addressed Terminal Vercel failures are marked complete before exit, preventing redundant cancellation.
Trigger behavior Addressed Unrelated labels no longer cancel previews, reopened labeled PRs work, and README accurately describes the path/event conjunction.
Security rationale duplication Addressed The detailed rationale is in the README; workflow/config files point to it.
Vercel build.env support Pending live verification The central guard fails closed if connect-preview lacks remote scope, but the first adopter must prove the variables reach a real deployment end-to-end.

@koletzilla

Copy link
Copy Markdown

LGTM! @alex-clickhouse @joe-clickhouse I see you're mainly doing changes in this repo. Is there something else you may miss from this PR? 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants