Correct rotated thumbnails for videos uploaded as a Telegram document#408
Conversation
Telegram's document.thumb for a video sent as a document is a raw frame grab that ignores the container's moov/trak/tkhd display-rotation matrix, so a portrait-recorded video (landscape pixel data + rotation matrix) got a sideways preview image. Read the video track's tkhd matrix from the already-downloaded video buffer, compute the clockwise rotation (0/90/180/270), and apply it to the thumbnail with sharp before uploading. tkhd is a standard ISO-BMFF box, so this also corrects rotation on plain MP4 uploads, not just Apple's MOV. Falls back to the unmodified thumbnail when no rotation is detected. Mux-hosted playback video is untouched — Mux already respects the rotation matrix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
document.thumbfor a video sent as a document is a raw frame grab that ignores the container'smoov/trak/tkhddisplay-rotation matrix — a portrait-recorded video (landscape pixel data + rotation matrix, e.g. 1920x1080 + 90°) got a sideways/squished preview image. Confirmed against a real iPhone MOV: video track matrixa=0,b=1,c=-1,d=0→ 90°, matching the actual video's portrait content vs. the mis-oriented thumbreadTkhdMatrix.ts+readMp4VideoRotationDegrees.ts(src/lib/media/) to read the video track's display matrix and compute the clockwise correction angle (atan2(b, a), normalized to 0/90/180/270)rotateImageBuffer.ts, a thinsharp().rotate()wrapperuploadVideoAttachment.tsnow computes rotation from the already-downloaded video buffer and applies it to the Telegram-provided thumbnail before uploading to Supabase; unmodified when no rotation is detected (identity matrix or no video track found) — same fail-safe fallback policy as the existing EXIF/QuickTime capture-date pathtkhdis a standard ISO-BMFF box (not Apple-specific), so this also corrects rotation on plain MP4 uploads (e.g. Android), unlike the capture-date feature which only works for Apple'scom.apple.quicktime.*metadataStacked on #407 (base branch is that PR's branch, not
main) since this branch was cut from its tip.Test plan
pnpm vitest run src/lib/media src/lib/telegram/chat— 326 tests passing, including new unit tests for the matrix parser (synthetic buffers matching real iPhone MOV byte values), rotation-degree calculation, and the realsharprotation wrapperreadMp4VideoRotationDegreesagainst the actual real-world MOV file bytes directly (not just synthetic fixtures) — returned 90°, matching the tkhd matrix read manuallytsc --noEmitclean on touched files🤖 Generated with Claude Code