Skip to content

Resize web image attachments before forwarding them to model providers #72

Description

@ashwin-pc

Summary

Browser-uploaded image attachments are forwarded to session.prompt() at their original dimensions. This bypasses pi's normal image processing path, which automatically resizes images used through read and @file to a 2000×2000 maximum.

This can leave a session in a permanently failing state once it accumulates enough images for a provider's stricter many-image limits to apply. At that point, even a text-only follow-up fails because the oversized historical attachment is resent as part of the conversation context.

Reproduction

  1. Start a session with an image-capable Anthropic model.
  2. Upload one or more screenshots with a dimension greater than 2000px through the web composer.
  3. Continue until the request contains enough historical images to trigger the provider's many-image validation rules.
  4. Send another message, including a text-only message.

The provider rejects the request before generation with an error similar to:

Validation error: messages.N.content.M.image.source.base64.data:
At least one of the image dimensions exceed max allowed size for
many-image requests: 2000 pixels

Expected behavior

Web attachments should be normalized before their first model request so that no embedded image exceeds the provider-compatible maximum dimension. A previously accepted attachment should not make all future turns fail when the session later crosses a provider image-count threshold.

Current behavior

The browser reads the original file as a data URL, and the server persists and forwards the same base64 payload directly. The automatic resizing used by pi's other image ingestion paths is therefore not applied to web-composer attachments.

Suggested fix

  • Process uploaded images server-side before adding them to session.prompt().
  • Apply the same default maximum dimension as pi's image processor (2000×2000), preserving aspect ratio.
  • Optionally keep the full-resolution upload on disk while embedding only the normalized copy in model history.
  • Validate the decoded MIME type and payload size at the same boundary.
  • Add a regression test with a synthetic image larger than 2000px and assert that the image passed to the session is resized.
  • Consider a recovery mechanism for existing sessions that already contain oversized embedded images.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions