Skip to content

dedup: survivor selection prefers older partial copy over attachment-complete duplicate #461

Description

@wesm

Follow-up from #457 / #459 (review discussion: #459 (comment)).

Scenario

Since #459, import-emlx imports *.partial.emlx files (complete body, attachments not cached by Apple Mail). If a partial is imported and the same message is later re-imported as a fully-downloaded .emlx (Apple Mail replaces the partial on disk once attachments download), the two copies have different raw MIME hashes, so both are stored. msgvault deduplicate groups the pair by RFC822 Message-ID (same source), but survivor selection picks the wrong copy.

Problem

isBetter in internal/dedup/dedup.go compares, in order: source priority → HasRawMIME → label count → older ArchivedAt → lower ID. For a partial-then-full pair both copies have raw MIME and (typically) the same labels, so the older ArchivedAt tiebreak keeps the earlier-imported, attachment-less partial and prunes the attachment-complete copy — hiding its attachments along with it.

Suggested fix

Prefer the more complete copy before the ArchivedAt tiebreak: larger raw size and/or more attachment rows. This may require adding raw size / attachment count to DuplicateMessage in the dedup query.

Needs its own tests (partial-first-then-full within one source, and cross-mailbox partial/full copies) and a check against the dedup spec in docs/internal/accounts-identities-collections-dedup/spec.md for interactions with the Message-ID/content-hash pass survivor rules.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions