Skip to content

media: decode HEIC photos with libheif so grid images aren't cropped - #272

Open
iFixRobots wants to merge 3 commits into
mainfrom
iFixRobots/heic-full-grid
Open

iFixRobots wants to merge 3 commits into
mainfrom
iFixRobots/heic-full-grid

Conversation

@iFixRobots

@iFixRobots iFixRobots commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Photos sent from an iPhone show up on LINE as a blurry, zoomed-in corner of the picture.

iPhone HEICs are grids of 512x512 tiles. convertHEICToJPEG ran ffmpeg -i x.heic -frames:v 1 … mjpeg, and ffmpeg (7.1.5 and 8.0.1) exposes the grid only as a "Tile Grid" stream group and encodes the first tile stream. -map 0:g:0 doesn't change that. The bridge log shows the cropped size:

{"megabridge":"line","thumb_width":896,"thumb_height":1024,"message":"Uploaded preview thumbnail"}

The conversion now uses libheif's heif-dec, which assembles the grid. If a file holds several top-level images, heif-dec numbers its outputs, and the first one is used. heif-dec also copies the photo's EXIF and XMP (location, device model), which ffmpeg didn't, so every APP1 segment is stripped from the JPEG. The color profile stays. The Dockerfile adds libheif-tools. The megabridge runtime image needs libheif-examples before this rolls out there (beeper/megabridge#88), or HEIC sends will fail.

Before After
3992x2992 iPhone HEIC (48 tiles) 512x512 JPEG 3992x2992 JPEG
Single-image HEIC 1280x854 1280x854

Before / after

convertHEICToJPEG at main vs this branch, same file:

HEIC before and after

Not yet checked with a live send to LINE.

View in Indent
Tag @indent to continue the conversation here.

@indent

indent Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Issues

1 potential issue found:

  • Malformed EXIF in the source HEIC still makes the conversion fail: libheif's JPEG encoder returns an error ('Invalid EXIF data (offset too large)'), heif-dec exits non-zero and the send fails, where ffmpeg used to convert the file. All APP1 data is now stripped anyway, so metadata problems shouldn't block the conversion.
    Found by Indent Review Agent

↑ Select any checkbox above to have Indent auto-fix the issue

Review agents

Select any unchecked box below to run or rerun that agent.

Found issues (1)
  • Indent Review Agent · APP1 stripping looks right now; HEIC sends still fail without heif-dec, and malformed EXIF still br…
Full results

Indent Review Agent

  • Summary: APP1 stripping looks right now; HEIC sends still fail without heif-dec, and malformed EXIF still br…
  • Last ran on commit: 473a989c
  • Latest result
    {
      "summary": "APP1 stripping looks right now; HEIC sends still fail without heif-dec, and malformed EXIF still breaks conversion.",
      "findings": [
        {
          "fix": "If heif-dec is missing (`exec.LookPath(\"heif-dec\")` fails, or cmd.Run returns exec.ErrNotFound), fall back to the old ffmpeg pipeline and log a warning.",
          "line": 64,
          "path": "pkg/connector/media.go",
          "issue": "Still open. convertHEICToJPEG only runs `heif-dec`. Any runtime with ffmpeg but without libheif-tools (self-hosted build.sh binaries, bbctl setups, or megabridge before beeper/megabridge#88 lands) gets `executable file not found`. send_message.go:199 turns that into a send error, so every HEIC image send fails. Before this PR, non-grid HEICs converted fine, and grid HEICs sent a cropped image.",
          "title": "HEIC sends fail outright wherever heif-dec is missing; no ffmpeg fallback",
          "severity": "bug"
        },
        {
          "fix": "Retry with `--skip-exif-offset`, or fall back to ffmpeg when heif-dec fails.",
          "line": 64,
          "path": "pkg/connector/media.go",
          "issue": "Still open. If the source HEIC's EXIF offset header is invalid, libheif's JPEG encoder returns false ('Invalid EXIF data (offset too large)'). heif-dec then exits non-zero, convertHEICToJPEG errors, and the send fails, where ffmpeg used to convert the same file. stripJPEGMetadata now drops all APP1 data anyway, so the conversion fails over metadata that would be thrown away.",
          "title": "Malformed EXIF in the source HEIC still makes the conversion fail",
          "severity": "nit"
        }
      ]
    }

Comment thread pkg/connector/media.go
// iPhone photos are HEIF grids of tiles. The ffmpeg CLI only decodes the
// first tile, while libheif assembles the full image.
var stderr bytes.Buffer
cmd := exec.CommandContext(ctx, "heif-dec", "--quiet", inputPath, outputPath)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Bug · Converted JPEG now leaks source EXIF/GPS metadata

The old ffmpeg mjpeg output had no EXIF. libheif's JPEG encoder copies the HEIC's EXIF block into the output, resetting Orientation but keeping GPS coordinates and device model, so photos sent to LINE now include location metadata. A malformed EXIF offset also makes heif-dec exit non-zero, which fails the conversion where ffmpeg used to succeed. Strip the EXIF segment from the result, or retry without metadata on failure.

Found by Indent Review Agent

@iFixRobots
iFixRobots requested a review from highesttt October 23, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant