Replace s3fs/rsync with rclone for R2 persistence #240
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andreasjansson
force-pushed
the
rclone
branch
from
a9e92d5 to
2483f1c
Compare
Replace the FUSE mount (s3fs) + rsync approach with rclone for direct S3 API access. This eliminates the root cause of slow syncs (200-400s) and the cascading bugs caused by s3fs latency. Key changes: - Dockerfile: install rclone instead of rsync - Restore: startup script uses rclone copy from R2 (fast, parallel) - Backup: uses rclone sync (not copy) to propagate deletions to R2 - Background sync loop runs IN the container, watches for file changes every 30s and uploads via rclone sync - Manual sync: uses sandbox.exec() which returns ExecResult directly, eliminating all startProcess/getStatus/polling workarounds - Remove cron trigger: no more Worker-side scheduled handler that could exceed time limits and reset the Durable Object - Remove FUSE mount: no mountBucket, no isR2Mounted, no R2_MOUNT_PATH - Pass R2 credentials to container env for rclone config - Rclone restore commands tolerate transient R2 errors (prevent set -e from killing the startup script on flaky API responses) - E2e tests verify data reaches R2 (rclone ls) and test the full backup-delete-restart-restore cycle via gateway restart - Fix lint: use deviceId as React key instead of array index fallback
andreasjansson
force-pushed
the
rclone
branch
from
2483f1c to
95307a3
Compare
E2E Test Recording (telegram)✅ Tests passed
|
E2E Test Recording (discord)✅ Tests passed
|
E2E Test Recording (base)✅ Tests passed
|
E2E Test Recording (workers-ai)✅ Tests passed
|
scott-edwards
added a commit
to scott-edwards/alfred
that referenced
this pull request
Feb 11, 2026
Upstream PRs merged: - cloudflare#235: Fix R2 persistence (waitForProcess, exitCode, restore race) - cloudflare#240: Replace s3fs/rsync with rclone (removes cron trigger) Conflict resolution: - Keep our GATEWAY_REQUEST_TIMEOUT_MS and CONTAINER_FETCH_TIMEOUT_MS - Drop CRON_TIMEOUT_MS and R2_MOUNT_PATH (no longer needed) - Remove scheduled handler (sync now runs inside container) - Keep sleepAfter: '4h' fix for keepAlive death spiral Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace the FUSE mount (s3fs) + rsync approach with rclone for direct S3 API access. This eliminates the root cause of slow syncs (200-400s) and the cascading bugs caused by s3fs latency.
Key changes: