Fetch Logpush dataset pages from middlecache

[soheiokamoto]

Summary

Fetch validated Logpush dataset pages from middlecache during docs builds while retaining checked-in pages as a safe fallback.

• Bound and verify archive downloads before replacing pages.
• Sync only generated scopes that have existing Cloudflare Docs destination directories.
• Preserve checked-in pages when the archive is unavailable or invalid.
• Refresh the archive on every development and build run.
• Require a separate manual changelog when a dataset change needs a customer-facing announcement.

Screenshots (optional)

Not applicable.

Documentation checklist

• The change adheres to the documentation style guide.

[cloudflare-docs-bot]

Review

⚠️ 1 warning, 💡 1 suggestion found in commit 2cd1fa7.

👉 Fix in your agent 👈

Fix the following review findings in PR #33248 (https://github.com/cloudflare/cloudflare-docs/pull/33248).

Before making changes, review each finding and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g. the fix requires disproportionate effort for minimal benefit,
  or the finding is factually incorrect), explain why
- If you need clarification before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order

After triaging, follow this order:
1. Post a comment on this PR for any findings you are skipping, with the finding ID and your reasoning.
2. Then commit the fixes for the legitimate findings.

The comment must come before the commit — the bot reads PR comments when a new
push triggers a review, so skip comments posted after the push will be missed.

---

## Code Review

### Warnings (1)

#### CR-8688c6617e16 · Stale-scope handling asymmetry
- **File:** `bin/fetch-logpush-datasets.ts` line 118
- **Issue:** The deletion pass (lines 118-124) only prunes stale `.md` pages inside scopes that exist in the fetched archive. A dataset scope that exists in `DATASETS_DIR` but is absent from the archive is never touched and no warning is emitted, so pages for a dataset removed from the archive silently stay published. The reverse case (scope in archive without a destination dir) is explicitly logged and skipped, so the asymmetry appears unintentional.
- **Fix:** After preparing scopes, iterate the destination scope directories and warn (or fail in non-soft mode) when a destination scope is not covered by the archive, so removed datasets don't leave stale published pages.

### Suggestions (1)

#### CR-00bd743d4e20 · Staging files inside content tree
- **File:** `bin/fetch-logpush-datasets.ts` line 91
- **Issue:** Staged pages are written as `.logpush-<uuid>.tmp` directly inside `src/content/docs/...` (the destination dir). If the process is killed (SIGKILL/OOM) during the prepare phase, the `finally` cleanup never runs and these `.tmp` files remain in `src/content`, which violates the repo's allowed-file-types rule and breaks CI until the script runs again to clean them up (lines 78-82 only run on the next invocation).
- **Fix:** Consider staging under `.tmp/` (same filesystem as the repo, so the final rename stays atomic) and cleaning stale staging files there, so a killed run can't leave non-content files in `src/content/`.

Code Review

This code review is in beta and may not always be helpful — use your judgment.

Warnings (1)

File

Issue

bin/fetch-logpush-datasets.ts line 118

Stale-scope handling asymmetry — The deletion pass (lines 118-124) only prunes stale .md pages inside scopes that exist in the fetched archive. A dataset scope that exists in DATASETS_DIR but is absent from the archive is never touched and no warning is emitted, so pages for a dataset removed from the archive silently stay published. The reverse case (scope in archive without a destination dir) is explicitly logged and skipped, so the asymmetry appears unintentional. Fix: After preparing scopes, iterate the destination scope directories and warn (or fail in non-soft mode) when a destination scope is not covered by the archive, so removed datasets don't leave stale published pages.

Suggestions (1)

File

Issue

bin/fetch-logpush-datasets.ts line 91

Staging files inside content tree — Staged pages are written as .logpush-<uuid>.tmp directly inside src/content/docs/... (the destination dir). If the process is killed (SIGKILL/OOM) during the prepare phase, the finally cleanup never runs and these .tmp files remain in src/content, which violates the repo's allowed-file-types rule and breaks CI until the script runs again to clean them up (lines 78-82 only run on the next invocation). Fix: Consider staging under .tmp/ (same filesystem as the repo, so the final rename stays atomic) and cleaning stale staging files there, so a killed run can't leave non-content files in src/content/.

Conventions

_{No convention issues found.}

Style Guide Review

_{No style-guide issues found.}

---

Acknowledged by author (7)

Review	File	Issue	Note
Code	package.json line 43	Fetch failure silently tolerated in build path	The author says the production-build soft fallback is intentional so archive availability does not block unrelated documentation builds.
Code	package.json line 15	Force re-fetch inconsistent with sibling fetchers	The author says fresh archive downloads in both development and build hooks are intended (they plan to force a fresh download in both).
Code	bin/fetch-logpush-datasets.ts line 115	No rollback on partial mutation failure	The author says failures after mutation begins are fatal so a mixed working tree cannot be deployed, and full rollback would not improve publish safety.
Code	bin/fetch-logpush-datasets.ts line 87	Empty scope aborts entire sync	The author says the archive contains file entries only and therefore cannot emit an empty scope directory.
Code	package.json line 43	Build mutates tracked content	The author says these Markdown files are generated pages intentionally replaced during builds, and the tracked copies provide the fetch-failure fallback.
Code	package.json line 43	Inconsistent --soft flag in fetch:assets	The author says the production-build soft fallback is intentional so archive availability does not block unrelated documentation builds.
Conventions	PR	Scope accuracy	The author says the only functional AGENTS.md update registers the new loader and the remaining diff aligns existing tree comments, so no change is needed.

Commands

Only codeowners can run commands. Post a comment with the command to trigger it.

Command	Description
/review	Runs a review now. Incremental if a prior review exists, full if not.
/full-review	Re-reviews the entire PR diff from scratch, ignoring incremental history. Useful after a rebase, when you want a fresh review, or if the bot gets out of sync and reports issues that no longer exist.
/ignore-review-limit	Permanently lifts the 2-review automatic limit for this PR. Future pushes will trigger reviews as normal.
/disable-auto-review	Stops automatic reviews from triggering on future pushes to this PR. Codeowners can still run /review or /full-review manually.
/rebase	Rebases the PR branch against production. On conflict, attempts to resolve automatically using AI. Stops with an explanation if confidence is not high enough.

[github-actions]

This pull request requires reviews from CODEOWNERS as it changes files that match the following patterns:

Pattern	Owners
*	@cloudflare/product-owners
*.ts	@cloudflare/content-engineering, @kodster28
package.json	@cloudflare/content-engineering

[cloudflare-workers-and-pages]

🚀 Deploying Preview to Cloudflare 🚀

Preview URL: https://sohei-ds-17240-fetch-logpush-docs.previews.developers.cloudflare.com (commit 2cd1fa7)

This URL reflects your latest Preview deployment

Preview Deployments by commit

Status	Deployment URL	Commit	Updated (UTC)	See this deployment's details
Build: Success ✅ Deployment: Success ✅ View logs ↗	https://43b00578.previews.developers.cloudflare.com	2cd1fa7	2026-09-05T02:37:02.805Z	Visit the dashboard ↗
Build: Success ✅ Deployment: Success ✅ View logs ↗	https://c389b978.previews.developers.cloudflare.com	b02f2bd	2026-09-05T00:35:43.298Z	Visit the dashboard ↗
Build: Success ✅ Deployment: Success ✅ View logs ↗	https://6e7b0d16.previews.developers.cloudflare.com	84de696	2026-09-05T00:20:04.297Z	Visit the dashboard ↗

[github-actions]

Preview URL: https://2cd1fa7d.preview.developers.cloudflare.com
Preview Branch URL: https://sohei-ds-17240-fetch-logpush-docs.preview.developers.cloudflare.com

[soheiokamoto]

Review triage before the next push:

• CR-773782c2b6a8: No further change. Failures after mutation begins are fatal, so a mixed working tree cannot be deployed. Full rollback would not improve publish safety.
• CR-cff72d2e658a: No change. These Markdown files are generated pages intentionally replaced during builds; tracked copies provide the fetch-failure fallback.
• CR-9358915e6c5d: No change. The production-build soft fallback is intentional so archive availability does not block unrelated documentation builds.
• CR-8486ea85020a: No change. The archive contains file entries only and therefore cannot emit an empty scope directory.
• CV-c3eb79385248: No change. The only functional AGENTS.md update registers the new loader; the remaining diff aligns existing tree comments.
• CR-c970353aceac: Will fix by forcing a fresh archive download in both development and build hooks.