E2 — Upload & Encoding Pipeline#113
Conversation
Adds GET /api/videos/upload/:uploadId/status so the client can re-derive which chunks the server already has after a disconnect, plus server-side idempotency: re-sending an already-accepted chunk acks without paying for a redundant R2 multipart part. Frontend now retries each chunk with exponential backoff and persists uploadId in localStorage so a refresh mid-upload picks up where it left off.
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (7)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c55551a4c0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| const priorBytes = Object.values(uploadedPartsMap).reduce((sum, part) => sum + part.size, 0); | ||
| if (priorBytes + rawFile.size > MAX_VIDEO_BYTES) { |
There was a problem hiding this comment.
When the final chunk is retried and uploadedPartsMap[partKey] already exists, priorBytes already includes that chunk, but this guard still adds rawFile.size again. For uploads near MAX_VIDEO_BYTES, a lost final response followed by retry can now fail with a false file_too_large 400, which breaks the new idempotent final-chunk resend path. Apply this size check only when uploading a new part (or subtract the existing part size).
Useful? React with 👍 / 👎.
| } else { | ||
| try { localStorage.removeItem(persistKey); } catch { /* private mode */ } |
There was a problem hiding this comment.
The resume bootstrap deletes persistKey for every fetchResumeState miss, including transient network/5xx failures where the server upload session may still exist. In the flaky-connection scenario this feature targets, a refresh during temporary outage will discard the uploadId and force a full restart instead of resuming. Only clear the key for definitive stale cases (for example, 404/chunk-count mismatch), not transport or transient server errors.
Useful? React with 👍 / 👎.
Lets a creator pause/disconnect mid-upload and pick up where they left off
without re-sending bytes the server already has. Hardens chunk-0 retry so a
swallowed 202 doesn't leak a parallel multipart in R2.
Backend:
- GET /api/videos/upload/:uploadId/status — uploadedChunks + chunkCount
- DELETE /api/videos/upload/:uploadId — abort R2 multipart + clear KV
- chunk-0 retry now reuses the existing R2 multipart instead of creating
a duplicate when the same uploadId comes back
- Session state extracted to src/workers/upload-session.ts (read/abort/
delete helpers + zod-validated meta with optional fileName/fileSize
fingerprint)
Frontend:
- src/frontend/lib/upload-resume.ts — pure, testable upload pipeline
with exponential-backoff retry on 5xx/408/429/network errors,
online/offline awareness, AbortController support, and localStorage
persistence (24h TTL aligned with KV)
- Upload.tsx wires the new module: resume banner when a saved session
matches the picked file's name/size/lastModified fingerprint, status
chip ("Retrying…", "Waiting for connection…"), and a Cancel button
that aborts the multipart server-side
Tests: 60+ new unit tests covering resume helpers, status/cancel routes,
chunk retry classification, backoff jitter, and offline-wait sequencing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
Closes ALO-121.
Hardens the chunked upload pipeline so a 1GB upload over a flaky connection completes and plays back as HLS.
Summary
Resume on disconnect
GET /api/videos/upload/:uploadId/statusreturns{ chunkCount, uploadedChunks[], fileName, fileSize, title, description }so the client can re-derive server-known chunks after a disconnect or page refresh. KV key is user-scoped, so cross-user reads 404.DELETE /api/videos/upload/:uploadIdaborts the R2 multipart and clears KV state on user-initiated cancel — keeps R2 storage clean instead of waiting on lifecycle policy GC.localStoragepersistence keyed by file fingerprint (name + size + lastModified) survives page reloads. TTL aligned with the server's 24h KV TTL.Idempotent chunk retry
totalBytesfrom the parts map so completion stays consistent under retries.Frontend retry + offline awareness
navigator.onLinechecked before each attempt; uploader pauses onofflineand resumes ononline.AbortController+ Cancel button — callsDELETE /api/videos/upload/:uploadIdso the server-side multipart is torn down too./uploadwhen a stalelocalStorageentry matches the picked file's fingerprint.Refactors
src/workers/upload-session.ts— KV read/abort/delete helpers + zod-validated session schema. The previous inline KV manipulation invideos.tsis now a singlereadUploadSession()call.src/frontend/lib/upload-resume.ts— pure, testable upload pipeline (retry classification, backoff, fingerprinting, localStorage I/O) so the React component is mostly UI.Test plan
npm test— 538 pass (60+ new unit tests acrossupload-session.test.ts,upload-resume.test.ts, and the new endpoint tests invideos.test.ts)npm run lint— clean (incl. AI Gateway guard)npm run type-check— cleannpm run build— clean🤖 Generated with Claude Code