fix(import): bound archive expansion before inflation

[Patrick-Erichsen]

GitHub repository imports currently inflate the entire ZIP before checking entry-count and expanded-size limits. This applies those limits in fflate's pre-inflate filter, while preserving selective imports: an unrelated file over 10 MiB is skipped and a valid small skill still imports.

Refs #3678. Follows the hardening approach proposed in #3679 and resolves its skip-to-rejection compatibility regression. That contributor branch is unchanged.

The filter counts all entries toward the 7,500-entry limit, counts only retained files toward 80 MiB, and skips directories, invalid paths, Mac junk and oversized files before allocation. Actual-byte checks remain after extraction. The existing import spec records this contract.

Before/after proof

Ran the same three regressions against main 91aecdc22c531f0612e3819038655ca9ca3d7da9, PR #3679 head 786ce7d7dd3e19d4fe64aad4466f3fcae8a7485d, and this fix. These execute the production archive helper with real ZIP data and fflate on macOS/Bun 1.4.0.

Scenario	Main before	PR #3679	This fix
Valid skill beside an oversized unselected entry	Attempts to inflate skipped payload	Rejects the whole repository, including the valid skill	Retains the skill without inflating the oversized payload
7,501st entry	Inflates before checking the count	Rejects before inflation	Rejects before inflation
Retained bytes exceed 80 MiB by one byte	Inflates before checking the total	Rejects before inflation	Rejects before inflation

The oversized-file case first verifies normal valid-archive behavior. Each pre-inflate regression then corrupts only the DEFLATE payload of the entry that must be skipped/rejected, keeping the ZIP headers intact. Attempting inflation raises invalid block type, so a post-inflate limit cannot pass the test. Exactly 7,500 files and exactly 80 MiB are also verified as accepted.

bunx vitest run convex/githubImport.test.ts -t 'skips oversized|rejects excessive'
main:      3 failed (invalid block type)
PR #3679:  1 failed, 2 passed (Repo archive contains a file that is too large)
this fix:  3 passed

Validation

• Focused import suite: 16 tests passed.
• bun run ci:static: passed.
• bun run ci:unit: 6,698 passed, 3 skipped; coverage gate passed.
• bun run ci:types-build: passed.
• bunx tsc -p convex/tsconfig.json --noEmit: passed.
• Structured autoreview: clean, no accepted/actionable findings.

No production deploy was performed. This does not claim to fix #3700 or #3666: their nested picker paths use GitHub tree/blob fetching and bypass ZIP extraction, and their production exceptions remain uncorrelated.

Refreshed landing validation — September 16

Rebased onto main 3ab0037e1b as 2342f443d622839e0f3b160550c4bab0273561f1. Git range-diff confirms the archive patch is unchanged.

The earlier hosted moderation, inspector-version, and publish-new-version failures coincided with one-second Convex timeouts across unrelated queries. Inspected browser artifacts show the apparent missing-file symptom was an application error boundary after skills:listVersions timed out. All three same browser scenarios pass on the refreshed branch on the first local attempt.

Refreshed validation: ci:static, full ci:unit (6,917 tests), and explicit root/schema/CLI/Convex TypeScript checks pass. Existing paired ZIP regression proof and clean autoreview apply to the unchanged patch. Fresh hosted CI is being checked before merge.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
clawhub	Ready	Preview	Sep 16, 2026 4:16am UTC

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed September 16, 2026, 12:18 AM ET / 04:18 UTC (Revision 2).

ClawSweeper review

What this changes

Checks GitHub archive limits before decompression, preserves skipping of oversized files, and adds boundary regressions and an import safety specification.

Merge readiness

✅ Ready for maintainer review

No blocking findings. The hardening remains necessary on current main and in v0.23.3, and this patch preserves selective import compatibility while addressing the reported expansion defect.

Priority: P2
Reviewed head: 2342f443d622839e0f3b160550c4bab0273561f1

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	A focused, compatible repair with meaningful boundary regressions and no blocking findings.
Proof confidence	🌊 off-meta tidepool	Not applicable: The collaborator-authored PR is exempt from the external-contributor proof gate, and no authority change triggers additional proof. Its reported macOS/Bun regressions exercise the actual archive helper with real fflate data; they are focused test evidence, not a deployed Convex demonstration.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: The collaborator-authored PR is exempt from the external-contributor proof gate, and no authority change triggers additional proof. Its reported macOS/Bun regressions exercise the actual archive helper with real fflate data; they are focused test evidence, not a deployed Convex demonstration.
Evidence reviewed	9 items	Verified introduced scope: The verified test merge has the pinned main and PR head as its two parents. Its delta contains only the import implementation, tests, and specification: 84 additions and 3 deletions. Package and lockfile differences are base-branch work. Current main still checks after inflation: Current main invokes unzipSync without a filter, then checks entry count and retained bytes and skips oversized files. The central defect remains present. Latest release remains affected: The v0.23.3 implementation also inflates before applying the archive limits; this change is not already supplied by that release.
Findings	None	None.
Security	None	None.

How this fits together

ClawHub’s GitHub importer downloads repository archives to discover and publish selected skill files. This change bounds archive extraction before those files reach preview and publishing.

flowchart TD
  A[Owned GitHub repository] --> B[Bounded ZIP download]
  B --> C[Check entry headers]
  C --> D[Skip excluded files]
  C --> E[Reject exceeded budgets]
  C --> F[Inflate retained files]
  F --> G[Check actual bytes and preview skills]

Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch size	+84/-3 across 3 files	The patch is confined to archive extraction, its regression coverage, and its specification.
Production and test growth	production +14 net; tests +59; specification +8	Production growth implements pre-inflation limits, with tests covering all three enforced boundaries.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #3678
Summary: This PR is a candidate fix for the archive-expansion tracker; the reported nested-folder preview failures remain separate.

Members:

• canonical: GitHub import unzips the archive before size limits #3678 - Tracks the exact pre-inflation resource-limit defect addressed here.
• same_root_cause: fix(import): cap GitHub zip expansion before inflate #3679 - Proposes the same hardening but changes oversized-file skipping into whole-archive rejection.
• adjacent_distinct: GitHub import review returns Server Error Called by client #3666 - Reports a nested-folder review failure on the tree/blob path, with no established archive-expansion connection.
• adjacent_distinct: GitHub import preview fails for all repos — previewGitHubImportCandidate returns Server Error #3700 - Reports production preview exceptions and upload failures that this patch does not establish or resolve.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Technical review

Best possible solution:

Enforce header budgets in the existing archive helper while preserving selective skipping and retaining the actual-byte checks.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: current main inflates every archive entry before enforcing its expansion limits, and the supplied corrupted-payload regressions distinguish that failure. This review did not execute tests.

Is this the best way to solve the issue?

Yes—the filter is the narrowest effective location. Reusing Skill Sync’s whole-archive rejection policy would violate selective import behavior, while post-extraction checks alone cannot prevent allocation.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 56550718d163.

Labels

Label changes:

• add P2: This repairs authenticated import resource limits without evidence of an active service-wide outage.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.

Label justifications:

• P2: This repairs authenticated import resource limits without evidence of an active service-wide outage.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.

Evidence

What I checked:

• Verified introduced scope: The verified test merge has the pinned main and PR head as its two parents. Its delta contains only the import implementation, tests, and specification: 84 additions and 3 deletions. Package and lockfile differences are base-branch work. (ec56ca8db87d)
• Current main still checks after inflation: Current main invokes unzipSync without a filter, then checks entry count and retained bytes and skips oversized files. The central defect remains present. (convex/githubImport.ts:894, 56550718d163)
• Latest release remains affected: The v0.23.3 implementation also inflates before applying the archive limits; this change is not already supplied by that release. (convex/githubImport.ts:894, 87ca030c30f3)
• Production boundary and dependency signal: The importer directly imports fflate, pinned to 0.8.3. Root-repository imports and archive discovery use the changed helper; scoped-folder imports instead fetch tree/blob data. The compressed download limit remains 25 MiB. (convex/githubImport.ts:696, 2342f443d622)
• Dependency confirms pre-allocation filter ordering: Upstream package metadata verifies repository ownership, and the v0.8.3 tag resolves to this commit. unzipSync evaluates the filter before copying stored data or allocating the DEFLATE output buffer. (src/index.ts:3842, dcb3714a6c25)
• Focused regression coverage: Three added tests cover oversized-file skipping, the 7,500-entry boundary, and the 80 MiB retained-byte boundary. Corrupting the excluded entry’s DEFLATE payload distinguishes rejection before inflation from checks afterward. The full supplied PR body reports before/after results on macOS/Bun and refreshed static, unit, and TypeScript validation; these commands were not rerun during this read-only review. (convex/githubImport.test.ts:106, 2342f443d622)

Likely related people:

• Patrick-Erichsen: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
• vyctorbrzezowski: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)

• reviewed 2026-09-15T16:46:23.445Z sha 521fcbd :: needs maintainer review before merge. :: none