Skip to content

fix(scripts): bound prebuild sync fetches with a timeout - #395

Merged
hivecommons-hive[bot] merged 3 commits into
mainfrom
scanner/fix-394
Oct 20, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
mainfrom
scanner/fix-394

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Closes #394

Adds scripts/fetch-with-timeout.ts (AbortController-based, default 30s, override with DOCS_SYNC_FETCH_TIMEOUT_MS) and uses it at the four bare fetch() call sites:

  • scripts/sync-hive-docs.ts (fetchText)
  • scripts/sync-sibling-docs.ts (fetchFile, fetchBinary)
  • scripts/update-meeting-recordings.ts (feed fetch; timeout lands in the existing warn-and-keep-committed-data catch)

Includes unit tests, a changelog fragment, and a CONTRIBUTING.md note.

Tests were not run locally per hive policy; CI is the verdict.

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Add fetchWithTimeout (AbortController, default 30s, overridable via DOCS_SYNC_FETCH_TIMEOUT_MS) and use it for the sync-hive-docs, sync-sibling-docs and update-meeting-recordings fetches so a stalled remote fails fast instead of hanging until the job timeout.

Closes #394

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@netlify

netlify Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for hivecommons-docs ready!

Name Link
🔨 Latest commit 13ad83b
🔍 Latest deploy log https://app.netlify.com/projects/hivecommons-docs/deploys/6ad70c3e7776a34e2993f210
😎 Deploy Preview https://deploy-preview-395--hivecommons-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed dco-signoff: yes stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 20, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Triage of the two red checks on head a757aab — both are PR-local (main @ a239ee5 is green on both Run unit tests and TypeScript & ESLint):

  1. Run unit tests — scripts/update-meeting-recordings.inprocess.test.ts:118 asserts fetch was called with [url] only, but fetchWithTimeout now passes { signal: AbortSignal } as the second argument. Update the expectation to toHaveBeenCalledWith(url, expect.objectContaining({ signal: expect.any(AbortSignal) })) (or use mock.calls[0][0]).
  2. TypeScript & ESLint — Prettier check fails on the new scripts/fetch-with-timeout.ts ("Code style issues found"). Run npx prettier --write scripts/fetch-with-timeout.ts.

Not pushing to this branch (scanner-lane branch). Job logs: unit tests, lint.


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wrapper itself reads correct to me — scripts/fetch-with-timeout.ts sets timedOut before controller.abort(), clears the timer in finally, and only rewraps the error when the timer fired, so real network errors surface unchanged (and the new unit tests cover both paths). The three call sites are straight substitutions with no behaviour change beyond the deadline.

Both red checks are caused by this diff, though, and both are one-line fixes:

  1. Run unit tests — scripts/update-meeting-recordings.inprocess.test.ts:118 is a pre-existing test that asserts expect(fetch).toHaveBeenCalledWith(FEED_URL). fetchWithTimeout now calls fetch(url, { signal }) (scripts/fetch-with-timeout.ts:32), so the assertion fails with an extra { signal: AbortSignal } argument. The test needs updating to something like toHaveBeenCalledWith(FEED_URL, expect.objectContaining({ signal: expect.any(AbortSignal) })).
  2. TypeScript & ESLint — npm run format:check fails on scripts/fetch-with-timeout.ts only ("Code style issues found … Run Prettier with --write"). ESLint itself reports 0 errors. npx prettier --write scripts/fetch-with-timeout.ts resolves it.

No other concerns: the env override is parsed defensively (parseTimeoutMs rejects 0, negatives, non-integers), the CONTRIBUTING note and changelog fragment match the behaviour, and the fragment filename follows the fixed- convention.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the new helper, the three call sites, and package.json's prebuild chain; ran vitest run scripts/fetch-with-timeout.test.ts on this head (6/6 pass). The helper is correct for what it covers, but it does not fully deliver the stated goal ("a stalled remote fails fast instead of hanging until the job timeout").

Finding (medium): the deadline covers headers only — body reads are still unbounded.

  • scripts/fetch-with-timeout.ts:31 — return await fetchImpl(url, {...init, signal}) resolves as soon as response headers arrive; the body is a stream that has not been read yet.
  • scripts/fetch-with-timeout.ts:42 — finally { clearTimeout(timer) } then disarms the abort before anyone touches the body.
  • Every caller reads the body after the helper returns, outside the deadline: scripts/sync-hive-docs.ts:227 (response.text()), scripts/sync-sibling-docs.ts:447 (res.text()) and :453 (res.arrayBuffer()), scripts/update-meeting-recordings.ts:130 (response.text()).

Consequence: a host that answers 200 and then stalls mid-body (the classic half-open/stalled-CDN case) still hangs prebuild until the CI/Netlify job timeout, exactly the failure this PR sets out to remove. Connect/TLS/header stalls are fixed.

Cheap fix: have the helper own the body read — e.g. fetchTextWithTimeout / fetchBufferWithTimeout that call .text() / .arrayBuffer() before clearTimeout, or return the response while keeping the timer armed and exposing a done() for callers. The existing test with the hanging fake would need a body-stall counterpart.

Not a defect, just noting: DEFAULT_SYNC_FETCH_TIMEOUT_MS is evaluated once at module load (fetch-with-timeout.ts:17), so the env var must be set before the script starts — fine for prebuild, and CONTRIBUTING.md says as much.

Everything else looks right to me: parseTimeoutMs rejects 0, negatives, floats and non-digits with a test for each; the timeout error names the URL and keeps the original as cause; non-timeout errors pass through; changelog fragment name fixed-sync-fetch-timeout.md fits the <category>-<slug> convention. I have not approved or requested changes — a maintainer should decide whether to land the header-only bound now and follow up, or extend it to the body first.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness — one in-scope gap (reviewed head a757aab).

The deadline covers only time-to-headers; a body stall still hangs the build. scripts/fetch-with-timeout.ts:41-42 clears the timer in finally as soon as fetchImpl resolves (i.e. when response headers arrive). Every call site then reads the body outside the helper — scripts/sync-hive-docs.ts:227 (response.text()), scripts/sync-sibling-docs.ts:447 / :453 (res.text(), res.arrayBuffer()), scripts/update-meeting-recordings.ts:130 — with no signal attached any more. A remote that returns 200 and then stalls mid-body (the "accepts the connection but never completes the response" case in #394) still blocks until the 15-min job timeout, and DOCS_SYNC_OPTIONAL=1 still never engages. Reproduced in a scratch vitest: a fake fetch resolving new Response(new ReadableStream({start(){}})) passes fetchWithTimeout(..., 200) and res.text() is still pending 1.5s later.

Concrete fix: keep the deadline alive through body consumption — e.g. have the helper return the consumed payload (fetchTextWithTimeout / fetchBufferWithTimeout, moving clearTimeout after await res.text()/arrayBuffer()), or build the signal with AbortSignal.timeout(ms) (Node 22 in CI) so the body stream errors on expiry, and wrap the read in the same timed out after Ns fetching <url> mapping. Add a test with a never-closing body stream alongside the existing never-resolving one (scripts/fetch-with-timeout.test.ts:38-47).

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review (head a757aab)

Timeout only bounds time-to-headers; a stalled body still hangs the build (scripts/fetch-with-timeout.ts:30-43, in-scope)

fetchWithTimeout resolves as soon as fetchImpl resolves (headers received) and clearTimeout(timer) runs in finally (L42). The body is then read by the callers outside any deadline: response.text() in fetchText (scripts/sync-hive-docs.ts:221-226), res.text() / res.arrayBuffer() in fetchFile/fetchBinary (scripts/sync-sibling-docs.ts:447,453), and response.text() in scripts/update-meeting-recordings.ts:130. A remote that returns 200 and then drips or never finishes the body — the exact "accepts the connection but never completes the response" case in #394 — still blocks the step until the 15-min job / Netlify timeout, with no URL-naming error. The changelog fragment (changelog.d/fixed-sync-fetch-timeout.md) and PR body claim the hang is fixed; that is only true for the connect/headers phase.

Concrete fix: keep the deadline across body consumption — e.g. have the helper read the body inside the try (export fetchTextWithTimeout(url) / fetchBytesWithTimeout(url) returning { ok, status, statusText, body }, calling clearTimeout only after text()/arrayBuffer() settles), and switch the four call sites to those. undici cancels an in-flight body read when the signal aborts, so the existing timedOut → "timed out after Ns fetching " path then covers both phases. Add a test with a fake Response whose body stream never ends.

No other findings from this perspective: the env override is validated to a positive safe integer (fetch-with-timeout.ts:10-15), no new URLs, permissions, or secrets are introduced, and the default-parameter fetchImpl = fetch still resolves the globalThis.fetch stub used by scripts/sync-hive-docs.main.test.ts:39-41.

— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intent alignment (read head a757aab): the diff matches #394's contract — one shared scripts/fetch-with-timeout.ts helper, applied at exactly the four call sites the issue lists, non-2xx semantics preserved. Two in-scope CI breaks need fixing before this satisfies the issue:

  1. Existing test broken by the new call signature — scripts/update-meeting-recordings.inprocess.test.ts:118 asserts expect(fetch).toHaveBeenCalledWith(FEED_URL), but fetchWithTimeout (scripts/fetch-with-timeout.ts:31) now calls fetchImpl(url, { ...init, signal }), so the stub receives two args and the "merges feed entries…" case fails in the Run unit tests job. Fix: change that assertion to toHaveBeenCalledWith(FEED_URL, expect.objectContaining({ signal: expect.any(AbortSignal) })).
  2. Prettier gate — npm run format:check flags scripts/fetch-with-timeout.ts (the multi-line new Error(..., { cause: error }) at lines 34-37). Run prettier --write scripts/fetch-with-timeout.ts.

Advisory (in-scope, non-blocking, matches the design #394 proposed): the timer is cleared in finally (fetch-with-timeout.ts:42) as soon as headers arrive, so a remote that returns headers and then stalls the body still hangs res.text() / res.arrayBuffer() (sync-sibling-docs.ts:447,453, sync-hive-docs.ts:221+) unbounded. #394 explicitly scoped this change to a per-request AbortController wrap, so I'm not blocking on it; worth a follow-up if the "never completes the response" case matters in practice.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

…abort signal

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: b142c94

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed dco-signoff: yes stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Oct 20, 2026
Address review findings on #395:

- Read the response body inside the helper's deadline so a remote that
  returns headers and then stalls mid-body is bounded too. The body read is
  raced against the abort deadline, so it holds even for streams that
  ignore the signal. Replace fetchWithTimeout with fetchTextWithTimeout /
  fetchBytesWithTimeout returning { ok, status, statusText, body } and
  update the sync-hive-docs, sync-sibling-docs and update-meeting-recordings
  call sites. Add never-closing-body tests for both helpers.
- Update update-meeting-recordings.inprocess.test.ts to expect the
  AbortSignal init argument.
- Import SyncEnv from run-sync-step instead of re-declaring it.
- Run prettier on the helper; wrap the CONTRIBUTING.md line to match.

Hive-Fix-Attempt: 1/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: ci-maintainer <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at 13ad83b; this looks correct to me.

What I verified:

  • All four call sites named in #394 are covered, and grep -rn "fetch(" scripts --include=*.ts (excluding tests and the helper) now returns nothing — no bare fetch() left in the prebuild path.
  • scripts/fetch-with-timeout.ts:28-57 — the deadline is raced against both the fetch() and the body read(), so a remote that sends headers and then stalls mid-body (the "never completes the response" case from the issue) is bounded too, and the body is cancelled on timeout. expire is assigned synchronously inside the executor, so it is always set before the timer can fire; clearTimeout in finally means the orphaned deadline promise never rejects after a normal completion.
  • Non-timeout errors are rethrown unchanged (timedOut guard), so sync-sibling-docs.ts:447-456 keeps its existing "null on non-2xx, throw on transport error" contract, and update-meeting-recordings.ts:127-135 lands timeouts in the existing warn-and-keep-committed-data catch as the body states.
  • AbortSignal.reason and Error cause need Node ≥17.2 / ≥16.9; package.json engines is >=22.19, .nvmrc is 22, and every workflow plus netlify.toml pin Node 22.
  • parseTimeoutMs rejects 0, negatives, floats and non-numeric input back to 30000, so a bad DOCS_SYNC_FETCH_TIMEOUT_MS can't disable the bound. CONTRIBUTING and the changelog fragment document the knob.

No findings. The only non-green check is tide (pending), which is the merge gate, not a failure.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed (security) — no blocking findings at head 13ad83b.

Advisory (low): scripts/fetch-with-timeout.ts:16 accepts any safe integer for DOCS_SYNC_FETCH_TIMEOUT_MS, but Node's setTimeout (line 44) overflows above 2147483647 ms and fires after ~1 ms with a TimeoutOverflowWarning, so an oversized value makes every sync fetch time out immediately instead of extending the deadline. Clamp to 2 ** 31 - 1 (or fall back to the default) in parseTimeoutMs.

— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at 13ad83b and the three call sites. Looks correct to me.

What I verified:

  • scripts/fetch-with-timeout.ts:34-72 races the deadline against both the fetch() promise and the body read, so the "headers arrive, body stalls" case from #394 is bounded even when the stream ignores the abort signal; the catch at :60-66 only rewraps when timedOut is set, so genuine network errors surface unchanged.
  • grep fetch( over scripts/*.ts (non-test) shows no bare call sites remain after this change — all four named in the body are covered.
  • scripts/update-meeting-recordings.ts:127-135 — the timeout error lands in the existing warn-and-keep-committed-data catch, as the body states.
  • Node 22 (package.json engines, all workflows) supports Error cause and AbortSignal.reason.

Ran locally: npx vitest run → 112 files / 1152 tests pass; npx tsc --noEmit clean.

Note for a maintainer, not a defect: DEFAULT_SYNC_FETCH_TIMEOUT_MS (fetch-with-timeout.ts:19) reads the env once at import, which is fine for one-shot scripts but means a caller cannot change it after import without passing timeoutMs explicitly.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Hive filed review findings that do not block this PR as backlog items:

Cap: 10 backlog item(s) per PR per day.

@hivecommons-hive
hivecommons-hive Bot merged commit 3a30e2e into main Oct 20, 2026
12 of 13 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-394 branch October 20, 2026 06:55
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author
Evidence bundle `#39513ad83b` sha256:820e709c097950745b14cc182856ae4c0ee8856a993a750c9113c127a8556ad2 — download from the dashboard or `hivectl review evidence #395`.

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[scanner] prebuild sync scripts fetch() without any timeout — a stalled remote hangs the build until job timeout

0 participants