feat(backup): scheduled cloud backups to the user's own Dropbox (phase 1) - #134
Merged
Conversation
…e 1) Once a month (or quarter) the cron builds each aircraft's .zip with the Phase-0 streaming builder and pushes it to storage the USER owns. Dropbox only; Google Drive slots in behind the same BackupProvider interface later. - 0049: tokens as AES-256-GCM ciphertext in the `private` schema behind SECURITY DEFINER functions (0047's pattern — a column-level revoke does not hold in Supabase). backup_schedule/backup_run are public + RLS'd so the owner can see whether it actually ran; backup_run.error is therefore redacted of anything token-shaped before it's stored. - Dropbox adapter: token_access_type=offline (refresh tokens never expire), App-folder access, files.content.write only, 8 MiB chunked upload session well under the 150 MB per-call cap. Timeouts everywhere, no token logged. - /api/cron/backup: its own route and Scheduler job (daily is at its budget). Lease-claimed, per-destination try/catch, 240 s deadline, day_of_month hashed from user_id over 1..28 so backups spread and February exists. Size guard sums blob bytes first and records skipped_too_large above 400 MB (env-tunable); the byte total is logged on every run. - Profile: connect / cadence / disconnect (disconnect DELETES the tokens), last run + result + size. Two consecutive failures → Resend email. - Absent DROPBOX_CLIENT_ID/SECRET is graceful: the sweep logs one line and skips, the card says it isn't configured. - Tests: node:test for the schedule maths (incl. Feb + the spread), the size guard, redaction and the chunking; E2E behind E2E_STUB_DROPBOX with a stub that refuses what production refuses (bad/expired token, wrong offset, closed session, malformed path, oversized call); backup tables added to the RLS isolation suite. Claude-Session: https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R
…ackup=error A missing migration and a broken Dropbox look identical from the redirect alone. The token never reaches the log. Claude-Session: https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R
Owner
Author
CI:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 of
docs/plan-cloud-backups.md: the scheduled backup actually ships to acloud account the user owns. Dropbox only — Google Drive slots in behind the
same
BackupProviderinterface in phase 2, Box is dropped. Cadence isoff / monthly / quarterly, monthly being the ceiling.0049_cloud_backups.sqlby hand to BOTH Supabase projects before mergingNothing here works until
supabase/migrations/0049_cloud_backups.sqlis appliedto the PROD project and the TEST project. Do not assume it has been applied.
The
e2echeck will fail onbackup.spec.tsand on the two newrls-isolation.spec.tscases until the TEST project has it — that failure mode is"missing tables/RPCs", nothing else.
What's in it
Migration 0049
private.backup_destination— the OAuth tokens, AES-256-GCM ciphertext, in theprivateschema behindSECURITY DEFINERfunctions granted toservice_role,exactly as 0047 did for MyFlightBook. A column-level
revokedoes not holdin Supabase (0039's note on why 0038 was cosmetic), so relocation is the only
thing that works. RLS scopes rows, not columns.
public.backup_schedule/public.backup_run— RLS'd, read-only for theowner. There is deliberately no write policy: every write goes through a
definer function, so nobody can set themselves
next_run_at = now()and make usship a full archive nightly.
my_backup_destinations()returns{ provider, account_label, connected, folder_path, frequency, next/last run, last status/bytes/error }and neverciphertext — that Profile-page leak is the exact bug 0047 existed to fix.
backup_run.erroris browser-readable by design, so the writer runsredactSecrets()over it first — a provider 401 body quotes the token back.log_change(): these rows are user-scoped, the trigger needs anon-null
aircraft_id(schedule's is nullable = "all aircraft"), and the iOSclient has no use for backup history. The 0045 lesson is written into the
migration header for whoever changes that: trigger list and backfill.
Dropbox adapter (
lib/backup/providers/dropbox.ts)token_access_type=offline(refresh tokens never expire), App folder access,scope
files.content.writeonly — noaccount_info.read, so the account labelis the account-id tail rather than an email we'd need a bigger scope to see.
upload_session/start → append_v2 → finish, 8 MiB chunks against Dropbox's150 MB per-call cap. Timeouts on every call, no token ever logged.
OAuth connect/revoke —
/api/backup/dropbox/authorize|callback, mirroring theMFB routes:
statein an httpOnly cookie and verified, redirect URI pinned topublicOrigin()rather than a request header. Disconnect deletes the tokenrow; it does not flag it.
/api/cron/backup— its own route and its own Cloud Scheduler job, not afourth pass on
api/cron/daily(already at its time budget). Same auth: POST-only,Bearer CRON_SECRET,timingSafeEqual. Oldest-due first, 240 s deadline, the restleft for tomorrow; lease-claimed so a retry can't double-upload; per-destination
try/catchlikerunSync/runReminders.sha256(user_id) % 28 + 1, so the fleet's backupsspread over the month instead of all landing on the 1st, and the date exists in
February.
blobSize()inlib/storage.ts—pagehas no size column); over the ceiling (400 MB,BACKUP_MAX_BYTES) the run is recordedskipped_too_largeand the user is told.The measured byte total is logged on every run so phase-4 is decided on data.
MyTailLog/<TAIL>/<date>-<TAIL>.zip.UI + notification — Profile card: connect / cadence / disconnect, plus last
run, result and size. Two consecutive failures → email via the existing Resend
path.
/helpupdated (standing project rule).Testing
test/backup-schedule.test.ts(26 cases): the day-of-month spread and its 1–28bound, monthly/quarterly
next_run_atincluding landing on 28 Feb and theyear boundary, the size guard and its env override, redaction of a realistic
Dropbox 401 body, and the chunking arithmetic.
e2e/backup.spec.tsbehindE2E_STUB_DROPBOX, set only inplaywright.config.ts'swebServer.env. The stub refuses what productionrefuses — bad/expired token (401), wrong offset (409
incorrect_offset),closed/unknown session, malformed path, oversized call. That is the ADS-B
lesson: a stub more permissive than reality is a blindfold.
backup_schedule/backup_runadded toe2e/rls-isolation.spec.ts, includingthat the schedule is not writable from the browser at all.
Local:
typecheck,lint,test(394),buildall green.Needs you before this can go live
files.content.write) with theredirect URI
https://mytaillog.com/api/backup/dropbox/callback, then createthe Secret Manager entries with exactly the IDs
DROPBOX_CLIENT_IDandDROPBOX_CLIENT_SECRET— thesecret:key must match character for character(the fix(deploy): point OPENSKY_* at the Secret Manager IDs that actually exist #131 lesson). Until they exist the sweep logs one line and skips and the
Profile card says it isn't configured.
https://mytaillog.com/api/cron/backupwithAuthorization: Bearer $CRON_SECRET,in the small hours and at a different minute from the daily job.
https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R