feat(mms): gateway-side attachment conversion and composer improvements - #103
Open
zhangsmallchi wants to merge 14 commits into
Open
zhangsmallchi wants to merge 14 commits into
zhangsmallchi wants to merge 14 commits into
Conversation
Owner
|
感谢你在 #103 里集中修复并增强 MMS 功能。为了降低这次发版的回归风险，我们计划先从当前 PR 中拆出已经比较明确的纯修复部分，单独提交一个 repair-only PR：
为使上述提交能够各自独立检出和测试，我们还会从 纯修复合并后，希望你可以把 #103 rebase 成只保留增强部分。我们建议增强部分在下一个版本继续优化和验证，当前待办包括：
这是为了尽快、安全地发布修复，同时给增强部分留出充分的真机验证时间。请看看这个拆分方案是否合适；如有不同意见，请在 3 天内（9 月 25 日前）回复。若无异议，我们会在期限后推送并提交 repair-only PR。 Thank you for collecting the MMS fixes and enhancements in #103. To reduce regression risk for this release, we plan to first extract the well-scoped fixes into a separate repair-only PR:
To keep every extracted commit independently checkoutable and testable, we will also extract only the attachment naming helpers and MIME mapping from After the repair-only PR is merged, we would appreciate it if you could rebase #103 so that it contains only the enhancement work. We suggest optimizing and validating those enhancements for the next release. The current follow-up list is:
This lets us ship the fixes quickly and safely while leaving enough time for real-device validation of the enhancements. Please let us know whether this split works for you. If you disagree, please reply within three days (by September 25). Otherwise, we will push and open the repair-only PR after that deadline. |
Contributor
Author
|
The split works for us — no objection, please go ahead. Once the repair-only PR is merged we Seven of the eight follow-ups are now on the branch (
On iPhone HEVC There is a separate, smaller problem next to it that we would rather you decided on as well. Those files are common — "Most Compatible" on iOS, older devices, a lot of cameras — so this 中文拆分方案我们没有异议，请按计划推进。repair-only PR 合并后，我们会把 #103 rebase 到 八条待办中的七条已经在分支上（
关于 iPhone 的 HEVC 旁边还有一个更小的问题，我们同样希望由你们决定。今天 含 H.264 和 AAC 的 iPhone 这类文件很常见——iOS 的"兼容性最佳"、老一些的设备、很多相机——于是被拒绝的是本来就能发的内容。 |
Owner
|
感谢确认拆分方案并完成后续修改。纯修复部分已经通过 #111 合并，并随 v1.11.0 发布。 下一步请把 #103 rebase 到最新的 你新增的七项修改总体上覆盖了此前提出的问题，不过
关于视频格式：
这个增强 PR 计划放到 v1.11.0 之后的版本评估。完成 rebase 和上传限制调整后请通知我们，我们会重新审查。 Thank you for confirming the split and implementing the follow-up changes. The repair-only portion has already been merged through #111 and released in v1.11.0. Please now rebase #103 onto the latest The seven new changes generally address the issues previously identified. However, the upload-limit implementation in
Regarding video formats:
This enhancement PR will be evaluated for a release after v1.11.0. Please notify us after the rebase and upload-limit revision are complete, and we will review it again. |
…lly is
The gateway accepted any image/*, audio/* or video/* upload on the strength of
the MIME type the client declared, so a HEVC video, a WebP picture or a file
merely named .jpg went to the MMSC as is; the WebUI's picker and drop filter
followed a different list again.
- mms_media.FORMATS lists each type with a policy -- send, convert (a client
re-encodes it first; the gateway refuses it unconverted), receive (stored
when it arrives, never sent) -- and whether a browser can preview it. The
sendable set is small on purpose: JPEG/GIF/PNG, AMR/AMR-WB/MP3/AAC, H.263/
MPEG-4/H.264 in 3GP or MP4, text, vCard 2.1/3.0, vCalendar/iCalendar.
- check_attachment() reads the content: magic numbers, the codecs an MP4/3GP
file declares (HEVC, AV1 and encrypted tracks are refused with a hint), a
card's VERSION. The content decides the type within a kind of media; a file
whose content is another kind than declared is refused as mislabelled.
Audio/video playing time (MP4 mvhd, AMR frames) feeds the SMIL slide length.
- Composing and storing both go through check_attachments(), so a client
calling the API directly gets the same checks as the WebUI. Names are made
safe by display_name(): byte-limited, extension kept, paths and control
characters removed.
- The MMS settings response carries the table ("formats") and every stored
part says whether it can be previewed, for the WebUI and future clients such
as a SIP user agent submitting MMS through the gateway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ine's limit Pictures were only resized in the browser, so a client calling the API -- and later a SIP user agent -- sent camera originals as they were, and formats most phones cannot show (HEIC, WebP, AVIF) could not be sent at all. - mms_convert holds converters by kind of media. The image converter (Pillow, with pillow-heif for HEIC/HEIF) decodes any supported picture, applies its EXIF orientation, flattens transparency onto white and writes baseline JPEG without metadata, searching the largest size (1600 px down) and then the highest quality that fits a byte budget. A JPEG, PNG or GIF that already fits and is no larger than 1600 px keeps its pixels, but loses its EXIF, XMP, IPTC or PNG text -- where a photo was taken travels in its EXIF -- and one whose EXIF rotates it is re-encoded upright instead. An animated GIF is never re-encoded. A video converter can register under "video" with the same interface later; nothing else has to change for it. - mms.fit_attachments() plans the message: sound, video and animated GIFs count as they are, the rest of the limit after them, the text and the packaging is shared between the pictures (a small one keeps its size), each is fitted from its original, and the packaged result is measured and the shares tightened if needed. It reports each attachment's size before and after. prepare_outgoing() is what sending uses; a "convert" format is taken whenever a converter exists for it, and the capability table's new "attachable" flag says so to clients. - Uploads may now be up to 25 MB each, since the gateway shrinks them. - Pillow 12.3.0 and pillow-heif 1.7.0 have wheels for every supported platform (amd64/arm64, glibc/musl, Python 3.12/3.13); licenses listed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
To show the user what an attachment will cost as soon as it is added, the
client uploads it then, and the gateway keeps the original until the message
is sent or the attachment removed. Each re-plan -- another picture added, one
removed -- starts again from the original rather than from an earlier,
smaller encoding. Once sent, only what was sent is stored with the message.
- POST /api/instances/{id}/mms/attachments upload one file (checked
at once; refused formats
never get staged)
- POST /api/instances/{id}/mms/attachments/fit {ids, text, subject, to}:
each size before/after
and the packaged total
- GET /api/instances/{id}/mms/attachments/{aid}/preview the fitted picture
- DELETE /api/instances/{id}/mms/attachments/{aid}
- POST /api/instances/{id}/mms/send takes attachment_ids besides files, fits
everything again with the final text and recipients, and removes the
uploads it sent.
Uploads live under <data>/mms-staging/<line>/, at most 20 per line, and are
swept a day after their last use (the MMS worker's sweep now runs hourly).
The full local backup leaves them out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ill be sent
Picture compression now happens on the gateway, so the browser's own is gone
(mmsImage.js). Instead:
- Each attachment is uploaded as soon as it is added (picker, paste or drop);
a file the gateway refuses is dropped at once with its reason.
- After every change the composer asks the gateway to fit the message and
shows each attachment's size before and after ("3.2 MB -> 180 KB"), its
converted thumbnail, and the packaged total against the line's limit, with
the gateway's reason when it does not fit. Edits to the text, subject or
recipient refit after a pause. Send waits for uploads and fits to settle and
sends the staged attachment ids; removing an attachment or leaving the line
deletes its upload.
- The picker's accept list, paste and drag-drop follow the gateway's
capability table (mmsFormats.js): what it marks attachable -- including the
pictures it converts (HEIC, WebP, BMP, AVIF) -- matched by type, alias or,
when the browser gives no type, extension. An older gateway without the
table gets a built-in copy of the send-as-is formats.
- A received or sent part is shown inline only when the gateway marks it
previewable and, for sound and video, this browser can play the type; if it
still fails to load it becomes the download card, which says the preview is
not available.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A line's size limit applies to each MMS, so pictures sent together share it
and each comes out smaller. The sender can now choose to send every
attachment as its own MMS, each fitted to the whole limit.
- mms.plan_messages() plans either mode from the originals: one message whose
attachments share the limit, or one message per attachment with the text
and subject on the first. Its summary reports each message's packaged size
and whether it fits, besides the per-attachment sizes.
- The fit endpoint and POST .../mms/send take "split"; sending stores one
message per attachment and submits them one after another, so they arrive
in order. The response lists them all ("message" stays the first).
- The same attachment fits differently in the two modes, so a staged upload
keeps its last few fitted versions under a digest of their content; each fit
returns that token per attachment and the preview endpoint serves exactly
that version, never the other mode's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With two or more attachments the composer offers "One MMS" (the default) and "One MMS per attachment". Switching re-plans every attachment on the gateway from its original; the thumbnails come from the fitted version that plan produced (its preview token) and sending waits for the new plan. The total line reads "N MMS, each up to <limit>" when split, and the choice resets after sending. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…imit and backups TROUBLESHOOTING's MMS section lists the send / convert / receive-only formats and common refusals, explains how the gateway converts and shrinks pictures while a message is composed (and what it keeps afterwards), that the size limit is per MMS and covers the packaged message, and the choice between one MMS and one per attachment; it names the encapsulation version and SMIL layout targeted (without claiming conformance), and explains where attachments live and how to restore them from a pre-upgrade backup. CHANGELOG records the changes and the two new Python packages. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An upload cut off part way through still parses as ISO boxes, so the movie header is present but empty. Reading its version byte then raised IndexError, which reaches the client as a 500 instead of the reason the file cannot be sent. Each field is now read only once the box is long enough to hold it. A header that stops short has no duration, which is what a file without one already gave the caller, and the file itself is refused by the checks that follow -- it has no track. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Setting Image.MAX_IMAGE_PIXELS reads as a limit but is not one: Pillow only warns between that number and twice it, and raises above. A file claiming 96 megapixels therefore passed a 64-megapixel setting and was decoded in full -- about 380 MB of pixels for a few hundred bytes of header. The size is now read from the header, after opening and before anything is decoded, and a picture over MAX_PIXELS is refused with what it claims to be. Pillow's own guard stays as a backstop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fitting and sending both took the list of staged attachment ids as given. Every id is read into memory to be fitted, and nothing said an id could not appear twice, so a list naming one 25 MB upload a thousand times asked the gateway for gigabytes -- from a request of a few kilobytes. Both endpoints now refuse a list longer than the number of uploads a line is allowed to hold at once, which is already the most a composer can name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A line could hold MAX_PER_LINE uploads, and nothing said how large they were allowed to be: twenty files of the largest size the API accepts are half a gigabyte, per line, on an appliance whose disk is often an SD card. The gateway as a whole had no ceiling at all. An upload is now refused unless the line, and the gateway, have room for it (64 MB and 256 MB). An abandoned draft is swept before the room is declared full, so a gateway left at the ceiling yesterday still takes an attachment today. The whole upload is written while the budget is held, so what the next one measures is what is really there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
``max_part_size`` was given the upload limit, which reads as a 25 MB ceiling on an attachment and is neither. starlette applies it only to the fields it keeps in memory, so the effect was to raise the ceiling on the text and recipients from 1 MB to 25 MB; a file part is streamed to a spooled temporary file with no limit at all and was measured only after it had been written. Twenty files in one request could therefore fill the disk before a single check ran. The fields get a limit of their own (256 KB, more than an MMS can carry as text), the attachment endpoint takes one file, and the body is counted as it is read: the ASGI receive callable is wrapped, and parsing stops at the first chunk that takes the request past its limit -- one upload's worth for an attachment, what the staging area lets a line hold for sending. A declared Content-Length is only an early refusal when it is already too large. It is not required, since a chunked or HTTP/2 upload through a reverse proxy may carry none, and it is not trusted, since nothing obliges the body to match it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncies The control plane starts without Pillow and pillow-heif on purpose: MMS picture conversion is then simply not done. That is the right behaviour for a process that is already running and the wrong outcome for a reload, which would restart into a gateway that converts nothing and says so nowhere until somebody sends a photo. setup_venv now says what it means instead of leaving it to `set -e`, and proves the dependencies import before the control plane is restarted -- installed is not the same as usable for a wheel carrying native libraries. Either failure stops the reload with the reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THIRD_PARTY_LICENSES.md has the entry; NOTICE, which is the file that says what a distributed build obliges the distributor to offer, did not mention it at all. The wheel bundles libheif and libde265 (LGPL-3.0) and x265 (GPL-2.0-or-later), so a host install carrying it is in the same position as the engine image's Asterisk and pjproject. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
zhangsmallchi
force-pushed
the
feat/mms-improvements
branch
from
September 23, 2026 12:23
dee1e61 to
db5bb2e
Compare
zhangsmallchi
changed the title
14 tasks
Contributor
Author
|
Rebased onto
中文已 rebase 到
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
developafter v1.11.0. The repair-only part of this work was merged in #111; this PR now carries only the enhancements and the follow-ups from review.Summary
Moves MMS picture conversion from the browser to the gateway, so every client -- the WebUI, the API, later a SIP user agent -- gets the same checks and the same result. The WebUI uploads each attachment as it is added, shows its size before and after, and lets the sender send several attachments as one MMS or one MMS each.
Changes
formats; files are checked by content (HEVC video, vCard 4.0 and mislabelled files are refused with the reason). Builds on the naming helpers from fix(mms): preserve attachment integrity and complete backups #111 instead of adding its own./mms/attachmentsendpoints stage uploads (originals kept while composing, 24 h TTL) and fit them; the sent message keeps only what was sent. Staging is bounded by count and by disk (64 MB per line, 256 MB per gateway), and/fitandattachment_idsare capped.receive) and cut off at the endpoint's limit, whether or not aContent-Lengthis present; fields (256 KB, 40), files per request and total size are all bounded.install.sh reloadstops if the new packages cannot be installed or imported, before anything restarts.Upgrade impact
install.sh reload; an offline host needs them available first, and a reload without them stops with the reason..movstays receive-only; H.264/AAC.movsupport is left for a separate PR under the conditions discussed in review.Safety and privacy impact
tools/check-subscriber-identifiers.sh --commitsclean).Testing
SEND FAILmid-upload, unrelated to conversion and addressed in a separate PR).中文
基于 v1.11.0 之后的
develop。这项工作中的纯修复部分已通过 #111 合并；本 PR 现在只包含增强功能和评审后续修改。概要
把彩信图片转换从浏览器移到网关：网页端、API 以及今后的 SIP 客户端都走同一套检查，得到同样的结果。网页端在添加附件时即上传，显示压缩前后的大小，多个附件可选择合并为一条或每个单独发送。
改动
formats返回；按文件内容检查（HEVC 视频、vCard 4.0、内容与类型不符的文件会被拒绝并说明原因）。直接使用 fix(mms): preserve attachment integrity and complete backups #111 的命名辅助函数，不再自带一份。/mms/attachments接口暂存上传（编写期间保留原图，24 小时后清理）并计算压缩结果；发送后只保存实际发出的版本。暂存区按数量和磁盘占用双重限制（每线路 64 MB、整机 256 MB），/fit和attachment_ids的数量设上限。receive），超过接口上限即中断，不论是否带Content-Length；字段（256 KB、40 个）、单请求文件数和总大小都有上限。install.sh reload在重启任何服务之前中止。升级影响
install.sh reload从预编译 wheel 安装（amd64/arm64、glibc/musl）；离线主机需要事先准备，缺少时 reload 会中止并说明原因。.mov仍为仅接收；H.264/AAC.mov的支持按评审中讨论的条件另开 PR。安全与隐私影响
tools/check-subscriber-identifiers.sh --commits通过）。测试
SEND FAIL，与转换无关，另开 PR 处理）。🤖 Generated with Claude Code