Skip to content

fix: keep completed skill installs when backup cleanup fails - #3853

Merged
steipete merged 1 commit into
mainfrom
work/oss-s6-clawhub-install
Sep 30, 2026
Merged

steipete merged 1 commit into
mainfrom
work/oss-s6-clawhub-install

Conversation

@steipete

Copy link
Copy Markdown
Contributor

A completed clawhub install --force or update could restore the old skill merely because deleting its backup failed with EBUSY or EPERM. Move backup cleanup outside the successful replacement's rollback boundary. Earlier swap failures still roll back; a completed replacement keeps its new files and matching lockfile even if the old backup remains.

Completes and supersedes #3734 on a maintainer-owned branch after its changelog conflicted with current main. Preserves @SebTardif's fix and regression coverage, the built-CLI filesystem regression, CLI documentation, and Unreleased credit. The contributor's branch is unchanged.

Fixes #3733.

The before/after artifact test runs the real built Node CLI against a synthetic loopback registry, extracts a ZIP into real directories, and injects only backup-removal EBUSY in the CLI child. Before the fix, that test failed with exit 1 while the other 21 artifact tests passed. Afterward, all 22 pass: new content remains, the lockfile records version 1.0.0, and the previous backup is retained. This is filesystem fault-injection proof, not a Windows antivirus test.

Validation passed through Crabbox on Blacksmith Testbox tbx_01m3re6te4c7mneqajsnq7007m (backend run): bun run ci:static, bun run ci:unit (7,250 passed, 3 skipped), bun run ci:packages (461 CLI source tests and 22 artifact tests), bun run ci:types-build, and bun run ci:e2e-http. Independent Codex autoreview is scoped-clean through P2.

AI-assisted completion; the code, regression and runtime proof have been reviewed. No release or project version bump.

Keep backup deletion outside the swap rollback boundary. Verify the built CLI with real filesystem operations and an injected EBUSY, preserving the replacement and matching lockfile.

Co-authored-by: Sebastien Tardif <sebtardif@ncf.ca>
@vercel

vercel Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
clawhub Ready Ready Preview Sep 30, 2026 6:16am UTC

Request Review

@clawsweeper

clawsweeper Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper

clawsweeper Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge. Reviewed September 30, 2026, 2:18 AM ET / 06:18 UTC.

ClawSweeper review

What this changes

The ClawHub CLI keeps a completed skill install or update when old-backup deletion fails, with CLI documentation, a changelog entry, and unit and built-CLI regressions.

Regression provenance

Possible regression — suspected (reviewed change). No predecessor PR is attributed.

Merge readiness

⛔ Blocked before merge - 5 items remain

Current main still rolls back a completed skill replacement when backup deletion fails. This PR fixes that boundary, but a retained backup is discoverable by clawhub sync --all and could be published as a separate skill, so the branch needs a focused repair before merge.

Priority: P2
Reviewed head: f01e56b8e3ba71fae8c779af4237c8c0d72d9d42

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The focused swap repair has meaningful built-CLI coverage, but the retained-backup sync path and missing invariant note require changes before merge.
Proof confidence 🐚 platinum hermit (4/6) Not applicable: The changed CLI replacement owner is exercised by a built Node CLI test using a loopback registry, real directories, and injected backup-removal EBUSY; the PR reports exit 0, new content, matching lockfile version, and retained backup, and the exact-head packages check passed. The author has repository admin permission, so the ordinary contributor proof gate does not apply. The linked Testbox run was cancelled and its logs were unavailable here; no stored-data format changes.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: The changed CLI replacement owner is exercised by a built Node CLI test using a loopback registry, real directories, and injected backup-removal EBUSY; the PR reports exit 0, new content, matching lockfile version, and retained backup, and the exact-head packages check passed. The author has repository admin permission, so the ordinary contributor proof gate does not apply. The linked Testbox run was cancelled and its logs were unavailable here; no stored-data format changes.
Evidence reviewed 10 items Current main still has the failure: Backup removal remains inside the replacement try block on current main; its catch removes the new target and restores the old backup when removal throws.
Proposed swap boundary: The introduced patch moves backup removal after the rollback try/catch and swallows its failure after the replacement completes.
Built-CLI fault scenario: The artifact test runs the built Node CLI against a loopback registry, injects backup-removal EBUSY, and checks that the new file and lockfile version remain while the old backup is retained.
Findings 2 actionable findings [P1] Exclude retained backups from skill discovery
[P3] Record the new rollback boundary in specs
Security Needs attention Old skill files can be published as a backup slug: After cleanup EBUSY, the backup still has SKILL.md. Sync discovers it as a new skill and --all can upload its old files to the configured registry without a separate selection.

How this fits together

The ClawHub CLI downloads a skill into a work directory, replaces an installed copy, and records its origin and version in a lockfile. The sync command scans that same skills directory and can publish folders it recognizes as skills.

flowchart LR
A[Registry skill ZIP] --> B[CLI install or update]
B --> C[Staged replacement and backup]
C --> D{Replacement complete?}
D -->|No| E[Restore old skill]
D -->|Yes| F[Installed skill and lockfile]
D -->|Cleanup fails| G[Retained old backup]
G --> H[Sync discovery and publish]
Loading

Before merge

  • Exclude retained backups from skill discovery (P1) - A failed cleanup now leaves .demo.backup-* beside the installed skill. findSkillFolders accepts that directory's SKILL.md and gives it a distinct slug, so clawhub sync --all can pass the old files to publish as another skill. Filter installer backups from discovery and cover this fault path.
  • Record the new rollback boundary in specs (P3) - The repository's AGENTS.md requires changed subsystem invariants in specs/. The completed-swap and best-effort-cleanup boundary is currently documented only in public CLI docs and the PR; add a focused spec note so future changes preserve it.
  • Resolve security concern: Old skill files can be published as a backup slug - After cleanup EBUSY, the backup still has SKILL.md. Sync discovers it as a new skill and --all can upload its old files to the configured registry without a separate selection.
  • Resolve merge risk (P1) - If backup removal fails, a later clawhub sync --all can discover the retained old skill as a new slug and upload its files to the selected registry.
  • Complete next step (P2) - Exclude installer backups from sync discovery, add a retained-backup sync regression, and record the rollback invariant in specs before merge.

Findings

  • [P1] Exclude retained backups from skill discovery — packages/clawhub/src/cli/commands/skills.ts:1860
  • [P3] Record the new rollback boundary in specs — docs/cli.md:156
  • [medium] Old skill files can be published as a backup slug — packages/clawhub/src/cli/commands/skills.ts:1860
Agent review details

Security

Needs attention: The retained backup can become an unintended authenticated sync upload; the diff adds no dependency or CI supply-chain change.

Review metrics

Metric Value Why it matters
PR size +100/-3 across 5 files Most of the introduced lines are regression coverage for a small runtime change.
Runtime versus tests runtime +3/-2; tests +94/-1 The replacement logic changes narrowly and is exercised by unit and built-CLI tests.

Root-cause cluster

Relationship: canonical
Canonical: #3853
Summary: This PR is the clean current candidate for the reported backup-cleanup defect; the earlier open repair carries substantially the same fix but is currently dirty against main.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Exclude installer backups (recommended)
    Filter retained installer backups from sync discovery and add a regression showing sync --all ignores the old copy while finding the installed skill.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Exclude installer-created backup directories from skill discovery, add a regression proving sync --all ignores a backup retained after injected EBUSY, and preserve normal installed-skill discovery.

Technical review

Best possible solution:

Keep the successful replacement and exclude installer-created backup directories from skill discovery; record the rollback boundary in a focused spec note.

Do we have a high-confidence way to reproduce the issue?

Yes. On current main, make backup deletion fail after both renames: the catch removes the replacement and restores the old tree; the PR's built-CLI test encodes that fault path.

Is this the best way to solve the issue?

Not yet. Moving cleanup outside rollback is the right repair for install and update, but retained backups must also be excluded from sync discovery; retrying deletion alone would not protect that path.

Full review comments:

  • [P1] Exclude retained backups from skill discovery — packages/clawhub/src/cli/commands/skills.ts:1860
    A failed cleanup now leaves .demo.backup-* beside the installed skill. findSkillFolders accepts that directory's SKILL.md and gives it a distinct slug, so clawhub sync --all can pass the old files to publish as another skill. Filter installer backups from discovery and cover this fault path.
    Confidence: 0.94
  • [P3] Record the new rollback boundary in specs — docs/cli.md:156
    The repository's AGENTS.md requires changed subsystem invariants in specs/. The completed-swap and best-effort-cleanup boundary is currently documented only in public CLI docs and the PR; add a focused spec note so future changes preserve it.
    Confidence: 0.88

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6520846333a8.

Labels

Label changes:

  • add P2: The fix addresses a limited CLI fault path, while retained backups need a bounded sync-discovery repair.
  • add merge-risk: 🚨 security-boundary: A cleanup failure can leave old skill files eligible for authenticated sync --all upload under an unintended slug.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.

Label justifications:

  • P2: The fix addresses a limited CLI fault path, while retained backups need a bounded sync-discovery repair.
  • merge-risk: 🚨 security-boundary: A cleanup failure can leave old skill files eligible for authenticated sync --all upload under an unintended slug.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.

Evidence

Security concerns:

  • [medium] Old skill files can be published as a backup slug — packages/clawhub/src/cli/commands/skills.ts:1860
    After cleanup EBUSY, the backup still has SKILL.md. Sync discovers it as a new skill and --all can upload its old files to the configured registry without a separate selection.
    Confidence: 0.9

Acceptance criteria:

  • [P1] bun run --cwd packages/clawhub test:src.
  • [P1] bun run --cwd packages/clawhub test:artifact.
  • [P1] bun run ci:static.
  • [P1] bun run ci:unit.
  • [P1] bun run ci:packages.

What I checked:

Likely related people:

  • vincentkoc: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Patrick Erichsen: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Goutam Adwant: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add a regression proving sync --all ignores a backup retained after cleanup EBUSY.
  • Record the completed-swap rollback boundary in a focused spec note.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit d26329b into main Sep 30, 2026
28 of 29 checks passed
@steipete
steipete deleted the work/oss-s6-clawhub-install branch September 30, 2026 06:18
@steipete

Copy link
Copy Markdown
Contributor Author

Landed in d26329b, preserving Sebastien Tardif's Co-authored-by credit and Unreleased thanks. #3733 is closed and #3734 is closed as superseded; the contributor branch remains intact.

Before/after proof ran the built CLI with real ZIP/filesystem operations against a synthetic loopback registry and child-only EBUSY injection. The new artifact regression failed before the fix (21 others passed); afterward all 22 artifact tests pass and verify the new content, matching lockfile, and retained old backup. The static, coverage (7,250 passed / 3 skipped), packages (461 CLI source tests), types/build, and HTTP gates passed on Blacksmith Testbox tbx_01m3re6te4c7mneqajsnq7007m through Crabbox. The lease was stopped after proof; independent review was scoped-clean through P2.

Exact PR-head CI and post-merge main CI passed on their first attempts. No CLI release was published.

This branch was successfully deployed

1 active deployment
Preview – clawhub — f01e56b8 Deployed Sep 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skill --force install rolls back after leftover backup delete fails

1 participant