Fix Reply-To parsing and RUF dashboards (release 10.0.3)

[seanthegeek]

Stacked on #783 (base will retarget to master once #783 merges). Release 10.0.3.

Follow-up to #783. The mailsuite 2.2.1 bump fixed the upstream phantom empty-address junk, but parsedmarc-side bugs remained that left Reply-To and From empty/wrong in the failure (RUF) dashboards. This PR fixes them across all four dashboards and makes address rendering uniform.

1. Reply-To/Delivered-To always dropped (parser)

parse_email() looked up mailparser's underscored reply_to / delivered_to keys, but mail_json names those headers reply-to / delivered-to. The lookup always missed, so parsed_sample["reply_to"] was always [] regardless of the message. Now reads the hyphenated keys and stores under the underscored name consumers expect, dropping the raw key so the body carries a single representation (matching to/cc/bcc). Independent of the mail-parser version — 4.2.1 still names the key reply-to.

This one fix flows everywhere parsed_sample is consumed: JSON/CSV output, the Elasticsearch/OpenSearch nested sample.reply_to, and the Postgres dmarc_failure_sample_address table (address_type='reply_to').

2. Uniform address rendering (works on historical data)

Every displayed address (From, To, Reply-To) now renders the same way on every dashboard: Display Name <addr>, or the bare address when there is no display name. The format is assembled at query time from display_name / address — fields that already exist on previously-indexed reports — so the panels render historical data, not only reports stored after upgrading. No new stored field is introduced.

One unavoidable exception: a report's Reply-To only appears for reports parsed by 10.0.3+. Earlier versions discarded it at parse time (§1), so it is absent from older stored reports; only re-parsing the originals recovers it.

3. Dashboards

• Splunk: the email-samples panel renamed parsed_sample.headers.from{}{} / ...reply-to{}{} — mis-cased and array-of-array shaped, so the columns were empty. Now builds from and reply_to with an eval that coalesces display_name <address> → bare address. (A multi-address Reply-To falls back to addresses-only — a Splunk multi-value-rendering limitation, not data loss.)
• OpenSearch: the reply_to column aggregated sample.headers.in-reply-to.keyword — the In-Reply-To threading Message-ID, not Reply-To. Repointed to sample.headers.reply-to.keyword, field added to the dmarc_f* index pattern. The ES/OS failure writer now flattens Reply-To into sample.headers["reply-to"], mirroring the existing From/To flattening.
• Grafana (Elasticsearch): the Failure Samples panel already read sample.headers.reply-to.keyword, but that field held the raw [[name, addr]] array until the writer flattening above. The existing ReplyTo column now renders a clean string — no dashboard change required.
• Grafana (PostgreSQL): the Failure Reports panel surfaced neither the header From nor Reply-To (only envelope Mail From / Rcpt To). Added a From column (from sample_from) and a Reply To column (aggregated from dmarc_failure_sample_address), both in the same Name <addr> format.

Tests

• parse_email() with single + multiple Reply-To → all captured; Delivered-To → captured.
• ES + OS failure writers: assert on the document handed to .save() — sample.headers["reply-to"] is the flattened string and nested sample.reply_to carries the address.
• Postgres: a parsed Reply-To is INSERTed with address_type='reply_to'.

Full suite: 633 passed. ruff check / ruff format --check clean.

Not verified against a live instance

The Splunk / OpenSearch / Grafana dashboard edits were verified structurally (field paths, parsed visState/index-pattern, valid SQL, XML re-parses, SPL un-escapes), not rendered against live backends. Re-import the dashboards (or refresh the dmarc_f* index pattern) to pick up the new OpenSearch field.

🤖 Generated with Claude Code