Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/added-18746-mail-reply-smtp-sender.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **Hub: the sender behind the confirmation reply for mail sent to a hive, off by default ([#18746](https://github.com/hivecommons/hive/issues/18746)).** When the DKIM key file, `From`, selector and the new `HIVE_HUB_MAIL_REPLY_HELO` name are all set, the hub now delivers the "received" and "waiting for approval" replies itself: one plain-text message signed with the key (RSA of at least 2048 bits or Ed25519), a null envelope sender (`MAIL FROM:<>`), one connection to the best-preference MX host of the sender's domain with required STARTTLS, timeouts on every step and no retry (any failure is final and the ledger keeps the claim). The ledger limits are applied before anything is sent; a key that cannot be loaded sends nothing and uses up nothing; a header or address with a control character, a non-ASCII address or a MX host in a private range is refused before any connection; the log holds only ids, a reason class and a server's numeric code. A hub without the key file builds no sender and opens no connection. Optionally the hub hands the signed message to a fixed-IP relay (smarthost) instead of the recipient's MX with `HIVE_HUB_MAIL_REPLY_RELAY` (`host` or `host:port`, default port 587; unset keeps direct delivery) and, if the relay needs SMTP AUTH, `HIVE_HUB_MAIL_REPLY_RELAY_AUTH_FILE` (user name and password read from a mounted file, sent only after STARTTLS, never logged): STARTTLS stays required with no fallback to direct delivery, the envelope sender stays null and the relay must not re-sign. The owner still has to publish the DKIM and SPF records, confirm outbound TCP 25 and run the two-provider `dmarc=pass` test before turning it on. See [Replies to the sender](src/docs/meetings.md#replies-to-the-sender-off-by-default).
5 changes: 4 additions & 1 deletion src/docs/env-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,12 @@ behavior. [Maintenance limitations](#keeping-this-reference-current) still apply
| `HIVE_HUB_MAIL_MAX_BODY_BYTES` | No | `50331648` | Largest delivery `POST /api/mail/inbound` accepts from a `hive-mx` that sends routing headers (48 MiB, this route only; [#18564](https://github.com/hivecommons/hive/issues/18564)). Without routing headers the cap stays 8 MiB. |
| `HIVE_HUB_MAIL_CHUNK_BYTES` | No | `786432` | Piece size when the hub relays a large mail-in delivery to the spoke (768 KiB, under the 1 MiB ingress default); clamped to 64 KiB to 8 MiB. |
| `HIVE_HUB_MAIL_SPOOL_DIR` | No | system temp directory | Where the hub keeps a delivery on disk while it relays it. |
| `HIVE_HUB_MAIL_REPLY_DKIM_KEY_FILE` | No | none (replies off) | Path of the mounted DKIM private key the hub signs confirmation replies with ([#18746](https://github.com/hivecommons/hive/issues/18746)). Replies are off while it is unset, missing or empty; the hub only checks that the file exists and never reads or logs it. The sender that uses it is not part of this release, so setting it alone sends nothing. See [Replies to the sender](meetings.md#replies-to-the-sender-off-by-default). |
| `HIVE_HUB_MAIL_REPLY_DKIM_KEY_FILE` | No | none (replies off) | Path of the mounted DKIM private key the hub signs confirmation replies with ([#18746](https://github.com/hivecommons/hive/issues/18746)): a PEM file, PKCS#8 or PKCS#1, RSA of at least 2048 bits (2048 recommended) or Ed25519. Replies are off while it is unset, missing or empty. The hub reads it only to sign a reply, checks on every delivery that it can be loaded (an unusable key sends nothing and uses up nothing), and never logs it. See [Replies to the sender](meetings.md#replies-to-the-sender-off-by-default). |
| `HIVE_HUB_MAIL_REPLY_FROM` | With the key | none | The one fixed `From` address replies are signed as, for example `hive@hive.<project domain>`. Its domain must carry the DKIM record. Mail from this address or its domain is never answered. |
| `HIVE_HUB_MAIL_REPLY_DKIM_SELECTOR` | With the key | none | DKIM selector of the key: the record is `<selector>._domainkey.<From domain>`. |
| `HIVE_HUB_MAIL_REPLY_HELO` | With the key | none | Fully qualified host name the hub introduces itself with when it delivers a reply (SMTP `EHLO`). It must be the name that carries the `v=spf1 a -all` record and that the sending IP's reverse DNS returns. Without it (or with a name that is not a fully qualified host) no reply is sent. |
| `HIVE_HUB_MAIL_REPLY_RELAY` | No | none (direct MX delivery) | Optional smarthost for confirmation replies, `<host>` or `<host>:<port>` (port `587` when omitted), so the mail leaves from one stable IP with reverse DNS and SPF. The hub still DKIM-signs the message, uses the null envelope sender and requires STARTTLS with a certificate that verifies for this host name; it never falls back to direct delivery. A value that is not a host name with an optional port builds no sender. See [Replies to the sender](meetings.md#replies-to-the-sender-off-by-default). |
| `HIVE_HUB_MAIL_REPLY_RELAY_AUTH_FILE` | No | none (no AUTH) | Path of a mounted file with the relay's SMTP AUTH credentials: user name on the first line, password on the second (at most 1 KiB). Needs `HIVE_HUB_MAIL_REPLY_RELAY`. Sent as AUTH PLAIN after STARTTLS only; read for each send, never logged. Credentials that cannot be read send nothing and use up nothing. |
| `HIVE_HUB_MAIL_REPLY_LEDGER_FILE` | No | `/data/saas/mail-reply-ledger.jsonl` | Reply ledger: ids, salted hashes and counts only, kept 8 days (no address, subject, Message-ID or body). It makes a retry or a restart unable to produce a second reply. |
| `HIVE_HUB_MAIL_REPLY_SENDER_PER_HOUR` | No | `10` | Most replies to one sender address in an hour. Past a limit the reply is dropped and counted; the mail is unaffected. |
| `HIVE_HUB_MAIL_REPLY_SENDER_PER_DAY` | No | `30` | Most replies to one sender address in 24 hours. |
Expand Down
6 changes: 4 additions & 2 deletions src/docs/inbox-voice-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -563,8 +563,10 @@ A hive never sends mail. What a sender hears back comes from two places:
Why no confirmation mail: an automatic reply to a `From` address that was
not authenticated goes to whoever the forger named, which is how a mail
server ends up on blocklists. A reply only to authenticated senders is
designed in [#18746](https://github.com/hivecommons/hive/issues/18746) and
is not built.
designed in [#18746](https://github.com/hivecommons/hive/issues/18746). It
is built but off until the project's DNS records exist and an operator turns
it on ([Replies to the sender](meetings.md#replies-to-the-sender-off-by-default)),
so today no confirmation mail is sent.

The entries below are the reasons, by the code the Recent mail list and a
problem report use. The limits are the defaults.
Expand Down
90 changes: 80 additions & 10 deletions src/docs/meetings.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ What `hive-mx` does with a message:
- **Sender authentication:** after DATA it checks SPF, DKIM and DMARC and records whether SPF or DKIM passed for a domain aligned with the `From` address (`adkim`/`aspf` honoured). Only an aligned sender can ever be trusted by a hive; the receiver itself decides nothing.
- **Calendar invites:** only `text/calendar` and `application/ics` parts are read. It keeps UID, SEQUENCE, METHOD (REQUEST or CANCEL), RECURRENCE-ID, the summary (200 characters), organizer and attendees, start and end converted to UTC (Outlook's Windows time zone names such as "Eastern Standard Time" are mapped to IANA zones), RRULE and EXDATE, and the join details: the link from `X-GOOGLE-CONFERENCE`, `X-MICROSOFT-SKYPETEAMSMEETINGURL`, `LOCATION` or a link to `meet.google.com`, `teams.microsoft.com`, `teams.live.com` or `*.zoom.us` found in the description, the platform, and the meeting id and passcode where the platform uses one (Zoom `pwd`, the Teams "Meeting ID" and "Passcode" lines). Nothing else from the description is kept.
- **Mail-in files:** a message with no invite may carry Markdown (`.md`), office document (`.pdf`, `.docx`, `.pptx`, `.xlsx`), audio (`.m4a`, `.mp3`, `.wav`, `.ogg`, `.webm`), video (`.mp4`, `.mov`, `.mkv`) and image (`.png`, `.jpg`, `.gif`, `.webp`) attachments. The type is decided by the file's content, not its name, and must agree with the extension; a Word, PowerPoint or Excel file must be a readable container of that kind with no macro project. At most 10 files, 25 MB per file and 30 MB in all ([#18564](https://github.com/hivecommons/hive/issues/18564), [#18699](https://github.com/hivecommons/hive/issues/18699)). The subject becomes the title. Nothing in the message is executed or followed, and a document is stored, never opened.
- **Refusals say why:** a message that leaves nothing to deliver is refused after DATA with a reply the sender's provider returns in its bounce, naming each attachment's extension and what was wrong with it, for example `550 5.6.0 nothing to deliver: a .key attachment is not an accepted type. Accepted types: md, pdf, docx, pptx, xlsx, ...` or `552 5.3.4 nothing to deliver: a .mp4 attachment is over the 25 MB per-file limit (30 MB of attachments per message)`. An attachment left out of a message that is otherwise delivered is listed in the delivery (`files.skipped`: extension and reason) so the hive can show it. Every refusal that is about the message or the address ends with three more lines: how to try again, a link to the reason's entry in [Mailing your hive: what the replies mean](inbox-voice-sources.md#mailing-your-hive-what-the-replies-mean), and where to report a problem ([#18744](https://github.com/hivecommons/hive/issues/18744)). The lines are the same for every sender. This bounce is the only message a sender gets: neither the receiver nor a hive sends mail, so nothing is ever sent to an address that may be forged (a reply to authenticated senders is designed in [#18746](https://github.com/hivecommons/hive/issues/18746)).
- **Refusals say why:** a message that leaves nothing to deliver is refused after DATA with a reply the sender's provider returns in its bounce, naming each attachment's extension and what was wrong with it, for example `550 5.6.0 nothing to deliver: a .key attachment is not an accepted type. Accepted types: md, pdf, docx, pptx, xlsx, ...` or `552 5.3.4 nothing to deliver: a .mp4 attachment is over the 25 MB per-file limit (30 MB of attachments per message)`. An attachment left out of a message that is otherwise delivered is listed in the delivery (`files.skipped`: extension and reason) so the hive can show it. Every refusal that is about the message or the address ends with three more lines: how to try again, a link to the reason's entry in [Mailing your hive: what the replies mean](inbox-voice-sources.md#mailing-your-hive-what-the-replies-mean), and where to report a problem ([#18744](https://github.com/hivecommons/hive/issues/18744)). The lines are the same for every sender. This bounce is the only message a sender gets unless the hub's confirmation reply is turned on, and the receiver never sends mail: nothing is ever sent to an address that may be forged, and a confirmation reply goes only to a sender whose mail passed DMARC alignment ([Replies to the sender](#replies-to-the-sender-off-by-default), [#18746](https://github.com/hivecommons/hive/issues/18746)).
- **Text body:** the message's body becomes a Markdown note. `text/plain` is kept as sent; an HTML-only body is reduced to its text, headings, lists and link targets, with no scripts, styles, images or other remote content. A message with no attachment becomes one Markdown item (`message.md`); a message with attachments keeps its body as the note on each of its items. The same allowlist, DMARC and approval rules apply as to attachments, and the body is untrusted data, never instructions. At most 256 KiB of it is kept.
- **Proof mails:** a message whose subject carries a proof code (`hive-proof-` and 12 letters or digits) is reported as a proof of the sender address.

Expand Down Expand Up @@ -305,15 +305,16 @@ nothing for the sender.
[#18746](https://github.com/hivecommons/hive/issues/18746) designs a short
reply from the hive to whoever mailed it ("received", or "waiting for the
owner's approval"). Hive has never sent mail, and `hive-mx` still does not:
the reply is composed and decided by the hub (option A of the design: a
hub-side sender with DKIM, one fixed `From`, the hive's own address as
`Reply-To`). This release ships the decision, the ledger and the two
templates; the SMTP and DKIM sender is a later step. With no sender built
in, a hub sends nothing whatever it is configured with.
the reply is composed, decided and sent by the hub (option A of the design:
a hub-side sender with DKIM, one fixed `From`, the hive's own address as
`Reply-To`).

It is off unless `HIVE_HUB_MAIL_REPLY_DKIM_KEY_FILE` names a file that
exists and is not empty (the variables are in
[env-vars.md](env-vars.md)). Removing the file stops replies at once.
exists and is not empty, and the `From` address, DKIM selector and HELO name
are set (the variables are in [env-vars.md](env-vars.md)). Removing the key
file stops replies at once; a key file that cannot be loaded sends nothing
and does not use up the mail's one reply. A hub with none of this set builds
no sender and opens no connection.

What decides a reply, all of it enforced in the hub:

Expand Down Expand Up @@ -352,6 +353,73 @@ outcome and salted hashes of the message id, sender, sender domain and hive:
no address, subject or body. The hub logs a sent reply with the delivery id,
the hive id and the outcome only, never a key or any part of a mail.

#### How a reply is sent

The sender (`src/pkg/hub/mail_reply_smtp.go`) runs only after the decision
and the ledger above allowed the reply, so the limits are applied before
anything is sent.

- **Plain text, null envelope sender.** One `text/plain` message, 7-bit,
with the headers of the design, and `MAIL FROM:<>`. A failed reply
therefore never bounces into a hive.
- **DKIM.** The message is signed (relaxed/relaxed, `rsa-sha256` or
`ed25519-sha256`) as the `From` domain with the configured selector, over
`From`, `To`, `Subject`, `Date`, `Message-ID`, `Reply-To`, `In-Reply-To`,
`References`, `Auto-Submitted`, `X-Auto-Response-Suppress`, `Precedence`
and the MIME headers. The signature expires after 72 hours.
- **Direct delivery to one MX host.** The hub looks up the sender domain's
MX records and uses only the best-preference host: one connection, one
message, no second host, no retry, no redirect. A domain with no MX record
or a null MX gets no reply. The hub connects to a public IPv4 address of
that host (IPv6 only if it has no IPv4 address), on port 25. Addresses in
private, loopback, link-local, shared (100.64.0.0/10), documentation
(IPv6) and similar ranges are refused, because the sender domain is
chosen by whoever sent the mail.
- **STARTTLS is required**, with a certificate that verifies for the MX host
name (TLS 1.2 or later). A server without it, or with a certificate that
does not verify, gets nothing.
- **Optional relay (smarthost).** With `HIVE_HUB_MAIL_REPLY_RELAY` set to
`<relay host>` or `<relay host>:<port>` (port 587 when none is given), the
one connection goes to that host instead of the sender's MX, so the mail
leaves from one stable IP that carries the reverse DNS and SPF records. It
is off while the variable is unset, and then everything above is unchanged.
Nothing else changes with a relay: the message is DKIM-signed by the hub
before the hand-off (the relay must not re-sign it), the envelope sender is
still `MAIL FROM:<>`, STARTTLS is still required with a certificate that
verifies for the relay's host name (a relay without it, or whose handshake
fails, gets nothing, and the hub never falls back to direct delivery), and
any failure is still final. The relay is chosen by the operator, so unlike
an MX host it may have a private address. If the relay needs SMTP AUTH, set
`HIVE_HUB_MAIL_REPLY_RELAY_AUTH_FILE` to a mounted file with the user name
on the first line and the password on the second; AUTH PLAIN is sent only
after STARTTLS, the credentials are read from the file for each send and
never come from an inline setting, and they are never logged. A relay
setting that is not a host name with an optional port, or a credentials
file named without a relay, builds no sender (the log names the variable,
not its value); credentials that cannot be read send nothing and use up
nothing, like an unusable key. The hub has no YAML block for this sender,
so the relay is configured through these environment variables only. A
relay runbook is the owner's side of this option.
- **Bounded.** 10 seconds for each DNS lookup, 15 for the connection, 45
for the whole SMTP conversation, 90 for the whole reply. At most 8 replies
are in flight; past that a reply is dropped and counted.
- **Any failure is final.** A refused recipient, a DNS or TLS failure, a
timeout or a server error ends the reply: it is not retried and the
ledger keeps the claim, so the same message is never answered twice.
- **Refused before sending.** A header value with a control character or a
line break, an address with a display name or outside ASCII, a
`Message-ID` that is not a plain token, a body that is not 7-bit text, or
a `From` other than the configured one is refused before any DNS lookup
or connection. The sender's subject only reaches the `Subject` header,
after the template removed control and invisible characters, and is sent
as an RFC 2047 encoded word when it has letters outside ASCII.
- **Nothing private in the log.** The hub logs a failed reply with the
delivery id, the hive id, a reason class (for example `mx_lookup`,
`no_mx`, `timeout`, `starttls`, `recipient_refused`, `connect`,
`signing_key_unusable`, and with a relay `relay_address_lookup`,
`relay_auth_unavailable`, `relay_auth`, `relay_credentials_unusable`) and, when the server refused, its numeric code.
Never a key, an address, a domain, a subject, a body or a server's text.

#### DNS records and network the hosted hub needs

For option A with one fixed `From` (shape 1), `<project domain>` being the
Expand All @@ -361,15 +429,17 @@ project's organizational domain and `<selector>` the DKIM selector:
|---|---|---|
| `<selector>._domainkey.hive.<project domain>` | TXT | `v=DKIM1; k=rsa; p=<base64 public key>` (2048-bit; the private key stays in the hub's Secret) |
| `hive.<project domain>` | TXT | `v=spf1 ip4:<sending IP> -all` (recommended: DKIM alone satisfies DMARC, since the envelope sender is null) |
| `<sending host name>` | TXT | `v=spf1 a -all` (recommended: covers the HELO identity used with a null envelope sender) |
| `<sending host name>` | TXT | `v=spf1 a -all` (recommended: covers the HELO identity used with a null envelope sender); this is the name to set in `HIVE_HUB_MAIL_REPLY_HELO` |
| reverse DNS of `<sending IP>` | PTR | `<sending host name>`, set at the cloud provider, not in the zone |

No existing record changes: the organizational DMARC policy (`p=reject`,
strict alignment) already covers `hive.<project domain>`, and the DKIM
signing domain equals the `From` domain, so it aligns. The hub's network
also needs outbound TCP 25 (many clouds block it by default); if it cannot
be opened the fallback is a transactional mail provider (option C of the
design). Before turning replies on for everyone, send to a test mailbox at
design), or a relay with a fixed IP (the optional relay above), in which
case the records name the relay's IP and host name and the hub itself needs
no outbound port 25. Before turning replies on for everyone, send to a test mailbox at
two large providers and check `Authentication-Results` shows `dmarc=pass`.

### Who can invite Betty
Expand Down
2 changes: 1 addition & 1 deletion src/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ require (
github.com/emersion/go-ical v0.0.0-20261112170022-46c5b53a1723
github.com/emersion/go-message v0.18.2
github.com/emersion/go-msgauth v0.7.0
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6
github.com/emersion/go-smtp v0.26.0
github.com/fsnotify/fsnotify v1.10.1
github.com/golang-jwt/jwt/v5 v5.3.1
Expand Down Expand Up @@ -43,7 +44,6 @@ require (
require (
github.com/ai-openshell/sdk-go v0.1.8 // indirect
github.com/antithesishq/antithesis-sdk-go v0.9.0-default-no-op // indirect
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.4 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
Expand Down
Loading
Loading