Skip to content

feat(backup): scheduled cloud backups to the user's own Dropbox (phase 1) - #134

Merged
iiamit merged 2 commits into
mainfrom
feat/cloud-backups-phase1
Aug 2, 2026
Merged

feat(backup): scheduled cloud backups to the user's own Dropbox (phase 1)#134
iiamit merged 2 commits into
mainfrom
feat/cloud-backups-phase1

Conversation

@iiamit

@iiamit iiamit commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Phase 1 of docs/plan-cloud-backups.md: the scheduled backup actually ships to a
cloud account the user owns. Dropbox only — Google Drive slots in behind the
same BackupProvider interface in phase 2, Box is dropped. Cadence is
off / monthly / quarterly, monthly being the ceiling.

⚠️ Apply migration 0049_cloud_backups.sql by hand to BOTH Supabase projects before merging

Nothing here works until supabase/migrations/0049_cloud_backups.sql is applied
to the PROD project and the TEST project.
Do not assume it has been applied.
The e2e check will fail on backup.spec.ts and on the two new
rls-isolation.spec.ts cases until the TEST project has it — that failure mode is
"missing tables/RPCs", nothing else.

What's in it

Migration 0049

  • private.backup_destination — the OAuth tokens, AES-256-GCM ciphertext, in the
    private schema behind SECURITY DEFINER functions granted to service_role,
    exactly as 0047 did for MyFlightBook. A column-level revoke does not hold
    in Supabase (0039's note on why 0038 was cosmetic), so relocation is the only
    thing that works. RLS scopes rows, not columns.
  • public.backup_schedule / public.backup_run — RLS'd, read-only for the
    owner. There is deliberately no write policy: every write goes through a
    definer function, so nobody can set themselves next_run_at = now() and make us
    ship a full archive nightly.
  • my_backup_destinations() returns { provider, account_label, connected, folder_path, frequency, next/last run, last status/bytes/error } and never
    ciphertext
    — that Profile-page leak is the exact bug 0047 existed to fix.
  • backup_run.error is browser-readable by design, so the writer runs
    redactSecrets() over it first — a provider 401 body quotes the token back.
  • Not added to log_change(): these rows are user-scoped, the trigger needs a
    non-null aircraft_id (schedule's is nullable = "all aircraft"), and the iOS
    client has no use for backup history. The 0045 lesson is written into the
    migration header for whoever changes that: trigger list and backfill.

Dropbox adapter (lib/backup/providers/dropbox.ts)

  • token_access_type=offline (refresh tokens never expire), App folder access,
    scope files.content.write only — no account_info.read, so the account label
    is the account-id tail rather than an email we'd need a bigger scope to see.
  • upload_session/start → append_v2 → finish, 8 MiB chunks against Dropbox's
    150 MB per-call cap. Timeouts on every call, no token ever logged.
  • Transport is injected, so the E2E stub exercises the production uploader.

OAuth connect/revoke/api/backup/dropbox/authorize|callback, mirroring the
MFB routes: state in an httpOnly cookie and verified, redirect URI pinned to
publicOrigin() rather than a request header. Disconnect deletes the token
row; it does not flag it.

/api/cron/backup — its own route and its own Cloud Scheduler job, not a
fourth pass on api/cron/daily (already at its time budget). Same auth: POST-only,
Bearer CRON_SECRET, timingSafeEqual. Oldest-due first, 240 s deadline, the rest
left for tomorrow; lease-claimed so a retry can't double-upload; per-destination
try/catch like runSync/runReminders.

  • Day-of-month spread: sha256(user_id) % 28 + 1, so the fleet's backups
    spread over the month instead of all landing on the 1st, and the date exists in
    February.
  • Size guard: sum the aircraft's blob bytes first (new blobSize() in
    lib/storage.tspage has no size column); over the ceiling (400 MB,
    BACKUP_MAX_BYTES) the run is recorded skipped_too_large and the user is told.
    The measured byte total is logged on every run so phase-4 is decided on data.
  • Never deletes anything: dated files only, MyTailLog/<TAIL>/<date>-<TAIL>.zip.

UI + notification — Profile card: connect / cadence / disconnect, plus last
run, result and size. Two consecutive failures → email via the existing Resend
path. /help updated (standing project rule).

Testing

  • test/backup-schedule.test.ts (26 cases): the day-of-month spread and its 1–28
    bound, monthly/quarterly next_run_at including landing on 28 Feb and the
    year boundary, the size guard and its env override, redaction of a realistic
    Dropbox 401 body, and the chunking arithmetic.
  • e2e/backup.spec.ts behind E2E_STUB_DROPBOX, set only in
    playwright.config.ts's webServer.env. The stub refuses what production
    refuses
    — bad/expired token (401), wrong offset (409 incorrect_offset),
    closed/unknown session, malformed path, oversized call. That is the ADS-B
    lesson: a stub more permissive than reality is a blindfold.
  • backup_schedule / backup_run added to e2e/rls-isolation.spec.ts, including
    that the schedule is not writable from the browser at all.

Local: typecheck, lint, test (394), build all green.

Needs you before this can go live

  1. Apply 0049 to prod and test. Nothing works without it.
  2. Create a Dropbox app (App folder access, files.content.write) with the
    redirect URI https://mytaillog.com/api/backup/dropbox/callback, then create
    the Secret Manager entries with exactly the IDs DROPBOX_CLIENT_ID and
    DROPBOX_CLIENT_SECRET — the secret: key must match character for character
    (the fix(deploy): point OPENSKY_* at the Secret Manager IDs that actually exist #131 lesson). Until they exist the sweep logs one line and skips and the
    Profile card says it isn't configured.
  3. Create the Cloud Scheduler job: daily POST to
    https://mytaillog.com/api/cron/backup with Authorization: Bearer $CRON_SECRET,
    in the small hours and at a different minute from the daily job.

https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R

iiamit added 2 commits August 2, 2026 17:46
…e 1)

Once a month (or quarter) the cron builds each aircraft's .zip with the
Phase-0 streaming builder and pushes it to storage the USER owns. Dropbox
only; Google Drive slots in behind the same BackupProvider interface later.

- 0049: tokens as AES-256-GCM ciphertext in the `private` schema behind
  SECURITY DEFINER functions (0047's pattern — a column-level revoke does
  not hold in Supabase). backup_schedule/backup_run are public + RLS'd so
  the owner can see whether it actually ran; backup_run.error is therefore
  redacted of anything token-shaped before it's stored.
- Dropbox adapter: token_access_type=offline (refresh tokens never expire),
  App-folder access, files.content.write only, 8 MiB chunked upload session
  well under the 150 MB per-call cap. Timeouts everywhere, no token logged.
- /api/cron/backup: its own route and Scheduler job (daily is at its budget).
  Lease-claimed, per-destination try/catch, 240 s deadline, day_of_month
  hashed from user_id over 1..28 so backups spread and February exists.
  Size guard sums blob bytes first and records skipped_too_large above
  400 MB (env-tunable); the byte total is logged on every run.
- Profile: connect / cadence / disconnect (disconnect DELETES the tokens),
  last run + result + size. Two consecutive failures → Resend email.
- Absent DROPBOX_CLIENT_ID/SECRET is graceful: the sweep logs one line and
  skips, the card says it isn't configured.
- Tests: node:test for the schedule maths (incl. Feb + the spread), the
  size guard, redaction and the chunking; E2E behind E2E_STUB_DROPBOX with
  a stub that refuses what production refuses (bad/expired token, wrong
  offset, closed session, malformed path, oversized call); backup tables
  added to the RLS isolation suite.

Claude-Session: https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R
…ackup=error

A missing migration and a broken Dropbox look identical from the redirect
alone. The token never reaches the log.

Claude-Session: https://claude.ai/code/session_01XBNGwWrPih2Xgu6MVrcd6R
@iiamit

iiamit commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

CI: checksemgrepe2e ❌ — and the e2e failure is only the unapplied migration

First run: 60 passed, 2 failed, both with the same root cause:

Could not find the table 'public.backup_schedule' in the schema cache   (PGRST205)
  • rls-isolation.spec.ts › cannot read or steer another user's backup schedule / runs (0049) — fails on the seed insert, table doesn't exist.
  • backup.spec.ts › backups: connect Dropbox … — the callback redirects ?backup=error because upsert_backup_destination doesn't exist. Everything before it works: the state cookie was set and verified, the stub bounced through the consent redirect, the code was exchanged.

(sync-pull.spec.ts was reported flaky — it passed on retry and is untouched by this PR.)

Both go green once supabase/migrations/0049_cloud_backups.sql is applied to the TEST project (and it must go to PROD too before merge). I have not papered over either failure — no skips, no conditional assertions: if 0049 is applied and they still fail, that is a real bug and should block.

Also pushed ec0ae8c: the connect callback now logs why it failed (redacted) instead of only redirecting ?backup=error — a missing migration and a broken Dropbox looked identical from the redirect alone.

@iiamit
iiamit merged commit 4aa7bc4 into main Aug 2, 2026
6 of 8 checks passed
@iiamit
iiamit deleted the feat/cloud-backups-phase1 branch August 2, 2026 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant