Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

listdist (V port)

A self-hosted mailing list manager: JSON-file member records, a Postfix-piped CLI for processing inbound mail, and a small admin web app (usafa, or the minimal reference app simple) for managing members and composing sends.

This is a V port of an earlier C program. Where the two differ, it's called out explicitly in the relevant section below and in the code comments.

Contents

Requirements & building

  • The V compiler (built from source is fine; no version-pinning is assumed beyond what's in v.mod, if present).
  • A mail transfer agent that can pipe inbound mail to a program (Postfix is assumed throughout this doc; anything with an equivalent pipe/alias mechanism works).
  • Optionally, hypermail on $PATH (or pointed at via HYPERMAIL) if you want browsable HTML archives, not just the raw mbox file.

Two app targets share one engine module: usafa (the full member-management web app) and simple (a minimal, unauthenticated reference app — see the warning in that section below). Which one you get is a compile-time choice:

cd bar/
v -d usafa  -o 79mailV main.v      # the real app
v -d simple -o 79mailV_simple main.v   # the reference app - do not deploy as-is

Naming the binary after your config file matters: if you run it without -c, it looks for <binary-name>.conf next to itself.

Run the test suite before deploying any change:

v test engine
v test apps/usafa

Running it

The same binary does two very different jobs depending on how it's invoked.

Finding the config file. -c behaves like most tools' -c/-f flags: an explicit value is taken seriously, not silently second-guessed.

  • -c foo.conf — already unambiguous, used exactly as given.
  • -c foo — tries a literal file named foo first; only if that doesn't exist does it fall back to foo.conf. So a bare name works whether you have an extensionless file or a conventional .conf one.
  • No -c at all — falls back to <this-executable's-name>.conf next to the binary (so 79mailV looks for 79mailV.conf), matching the naming convention above.

As the web server — no subcommand, just start it:

./79mailV -c /path/to/79mailV.conf

Binds to 127.0.0.1:8081 by default and stays running. Put a real reverse proxy (nginx/Caddy) in front for TLS — the app does not terminate TLS itself. See Security notes for why this matters beyond the obvious.

As a one-shot mail processor — this is what your MTA actually invokes, once per inbound message, reading the message from stdin and exiting:

# the real entry point for production - Postfix pipes each message to this
./79mailV -c 79mailV.conf dispatch

# force-post to a specific group, bypassing recipient-address routing
# (manual/testing use - see the caveat in "How a message is processed")
./79mailV -c 79mailV.conf post --group COFFEE --from you@example.com < msg.eml

# dry run: show the routing/moderation decision, send nothing
./79mailV -c 79mailV.conf test --group COFFEE --from you@example.com < msg.eml

# scan (and optionally fix) user record JSON files - see below
./79mailV -c 79mailV.conf cleanup [--fix]

# send one diagnostic message and exit, to sanity-check SMTP/sendmail config
./79mailV -c 79mailV.conf --mailtest --to you@example.com [--via smtp|sendmail]

A typical Postfix setup pipes the list address(es) to dispatch via /etc/aliases or a transport_maps entry, e.g.:

usafa79: "|/opt/listdist/79mailV -c /opt/listdist/79mailV.conf dispatch"

Use an absolute path for -c. A relative path resolves against the process's current working directory at the moment it's invoked — for a web server you started yourself that's whatever directory you were in, but for a Postfix-piped process it's Postfix's own working directory, not yours. This is exactly the same class of surprise as SPOOLDIR defaulting to . (see below) — an absolute -c sidesteps it entirely.

The config file

A single JSON file, conventionally named <something>.conf (see Running it above for exactly how a name given via -c is resolved), loaded once at startup for the web server, and once per invocation for the CLI. Every key is optional; a sensible default is defined for what's not set. Booleans accept a real JSON boolean, or (for compatibility with hand-edited configs) a string like "true"/"yes"/"on".

Core identity

Key Default Meaning
LISTPREFIX '' The local-part prefix all list addresses share, e.g. usafa79usafa79@…, usafa79-coffee@….
DOMAINNAME '' The domain your list addresses live on.
USERDIR USERS Directory of member record JSON files. Relative paths resolve against the config file's own directory.
PROGNAME Listdist Display name used in page titles etc.
SPOOLDIR . (!) Where held (moderated) messages are stored. Change this to an absolute path. The default of . means "whatever directory the process happened to be started in" — for a Postfix-piped process, that's Postfix's directory, not yours.

Admin & access

Key Default Meaning
ADMINPASS '' Password for the built-in master-admin login (email left blank or admin). Anyone with this password gets full admin access, including bulk export/delete of every member's data — treat it like the most important secret in this file.
LISTADMIN '' Address(es) (comma-separated) treated as list admin when posting by email. Note this is separate from a member record's own "Admin": true flag, which only grants admin status for the web login of that specific member — the two aren't automatically the same address. If you want a person to post without moderation and have "Admin": true on the web, both need to line up.
SUBJPASS '' A password that, if it's the first token of a message's Subject, grants admin treatment for that one post — an escape hatch that doesn't depend on the From address matching anything.
KEY '' Secret used to derive every per-record token (tracking beacon, account validate/authorize/set-password links). Treat this with the same care as ADMINPASS. Never logged in full even in debug output.
NOLOGIN false Disable the web login entirely.
LOCKED 0/off Controls what happens on signup. false/unset → open signup, active immediately. "check"/"verify"/"validate" → email-verify only. true/"yes"/"on" → email-verify and admin authorization required before the account is active. "nonew" → no new signups at all.
HASHPASS true Whether newly-set passwords are bcrypt-hashed (true, the safe default) or stored as literal plaintext (false). See Security notes before turning this off.

Mail sending

See Sending mail for how these interact.

Key Default Meaning
SENDMAIL (or USESENDMAIL) false If true, send by shelling out to a local sendmail-compatible binary. Takes priority over SMTP settings below — if both are configured, SMTP is simply never used.
SENDMAILPATH /usr/sbin/sendmail Path to that binary.
SMTPSERVER '' SMTP relay host (only used when SENDMAIL is false).
MAILPORT 25 SMTP port.
SMTPUSER / SMTPPASS '' SMTP auth, if the relay needs it.
SYSFROM <prefix>-bounces@<domain> The envelope sender used for most mail. Derived automatically if unset.
ENVFROM BOUNCES Envelope-from strategy. BOUNCES → use SYSFROM as-is. SRS → rewrite via Sender Rewriting Scheme (see caveat below).
MULTISLEEP 10 Seconds to pause between batched SMTP transactions.
SINGLESLEEP 15 Seconds to pause between single-recipient transactions (see TRACK and SINGLESENDDOMAINS).
SINGLESENDDOMAINS '' Domains (space/comma-separated, or "ALL") that must always be sent one-recipient-per-transaction, independent of tracking.
MAXMESSAGE 0 (no cap) Truncates the outgoing, distributed copy of a message body to this many bytes. Does not cap what's read from stdin, held for moderation, or archived — those always get the full original regardless of this setting.

ENVFROM: "SRS" caveat: the forward direction (rewriting an outgoing envelope sender into an SRS0=... address) works, but nothing in this port currently reverses it — there's no code path that decodes a returned SRS address back to the original sender. Don't rely on SRS round-tripping yet.

Moderation

Key Default Meaning
MODERATED LIST OPEN → anyone may post without moderation. LIST → members may post freely, non-members are held. ALLeveryone except admins is held, every time. DROP → posts from unrecognized senders are silently dropped rather than held.

See Moderation for the full hold/release/discard flow.

Tracking, web send, archiving

Key Default Meaning
TRACK false Enables per-recipient open tracking for web-composed sends. See Tracking — this does not, by itself, slow down ordinary list posts.
WEBSEND false Enables the /websend compose-and-send page for admins. (TRACK alone also enables it.)
ARCHIVE 0/off 0 → off. 1/"yes"/"split" → one archive per group. 2/"combined"/anything starting "comb" → a single combined archive. See Archiving.

Rewriting, privacy, DKIM/DMARC

Key Default Meaning
HIDEFROM false See Hiding the sender.
DROPDKIM false See DKIM.
REWRITEDOMAINS '' See DMARC / REWRITEDOMAINS.
REWRITEMSGID false Rewrite Message-ID on distributed mail rather than passing the original through.

Display / member-facing

Key Default Meaning
TAGLINE "An easy to use mailing list manager" Shown under the site title.
FOOTER "Powered by listdist; written by Geoff" First line of the footer appended to every distributed message.
SITEURL (falls back to http(s)://www.<domain>) Base URL used in emailed links (account emails, unsubscribe footer) and the tracking beacon. If your site is on https://, set this explicitly — among other things, it's what makes the session cookie's Secure flag turn on automatically (see Security notes).
SORTFIELD LastName Default member-list sort column.
LISTUSERS 1 (members only) Who can view the member list: 0/open → anyone, 1/list → members, 2/admin → admins only.
BACKGROUNDCOLOR, BACKGROUNDIMAGE, FAVICON, DECEASEDCOLOR '' Cosmetic.
PICTURES 0 Number of picture-upload slots per member (0, 1, or 2). Also inferred from the presence of PICTURENAME1/PICTURENAME2 if you don't set it explicitly.
PICTURENAME1 / PICTURENAME2 (or PICNAME1/PICNAME2) PICTURE 1 / PICTURE 2 Labels for the picture slots.
GENLIST false Historical flag; the actual export/delete tool always lives on the /users page for an admin (see below) regardless of this setting.

Member fields & groups

Key Meaning
FIELDS The set of member-record fields your signup/edit form exposes, comma/colon/newline-separated, e.g. "lastname:firstname", "squadron(G)". A field suffixed (G) is also usable as an ad hoc post-target group name (any member whose value in that field matches the group name given to post/test/a list address is a recipient) — this is why posting to, say, usafa79-CS21@… works even though CS21 isn't in GROUPS below.
GROUPS The list of named groups members can belong to, e.g. ["COFFEE", "TEST(A)"]. A (A) suffix means only an admin can add/remove a member's membership in that group via the web form (a member's own edit can't touch it, and an admin-set membership survives even if the member's own form submission omits it). An (L) suffix is parsed and shown on the config report page but isn't currently enforced anywhere.

User record files

Each member is one JSON file in USERDIR, named <anything>.json — the filename (minus .json) is the record's internal id, used in URLs, tokens, and the moderation/archive machinery. Keys are matched case-insensitively where it matters, but the samples below use the casing the app itself writes.

{
    "LastName": "Mulligan",
    "FirstName": "Geoff",
    "Email": {
        "Primary": ["geoff@mulligan.com", "ON"],
        "Squadron": true,
        "Off": ["someoldaddress@example.com"],
        "Spouse": "jess@mulligan.com"
    },
    "BDate": "1958-05-15",
    "Deceased": false,
    "Squadron": "CS21",
    "Address": { "City": "Colorado", "State": "CO", "Zip": "80920" },
    "Admin": true,
    "Groups": ["COFFEE", "COS", "WDC"],
    "Valid": "VALID",
    "Password": "$2b$10$....................................................",
    "LastUpdate": "Sat Aug  2 22:39:15 2025"
}
  • Email.Primary is either a bare address string, or a [address, status] pair/array. status (case-insensitive) is one of ON/YES/TRUE/VALID (active — the default if omitted) or OFF/NO/FALSE/0 (present on file but not currently receiving mail). If the first element isn't a real address at all (e.g. just ["ON"] with nothing else), the record simply has no working primary address — the cleanup tool flags this.
  • Email.Squadron can be a real address, or a bare true, meaning "use whatever this member's squadron's designated address is" rather than a personal one.
  • Email.Off is a list of addresses this member has opted out on — never contacted even for whole-list/force-all sends.
  • Password is either a bcrypt hash ($2a$/$2b$/$2y$ prefix) or, for records carried over from before this existed, plaintext — which is transparently upgraded to bcrypt the next time that member logs in (unless HASHPASS: false, in which case it's left as whatever format it's already in).
  • Valid drives the signup lifecycle: VALID (active), a needs-verification/needs-authorization state while LOCKED is on, or INVALID.
  • Groups is a plain list of group names the member belongs to — both configured GROUPS entries and, informally, squadron-style values from a (G)-marked field (see FIELDS above).
  • A field can be marked (G) in FIELDS (e.g. Squadron) to make its value usable as a group name for posting purposes, independent of the Groups list.

How a message is processed

For a normal inbound post (dispatch, or Postfix piping to it):

  1. Route by envelope recipient (X-Envelope-To/X-Original-To/ Delivered-To/To/Cc, in that preference order — not just a literal To: match, since many MTA setups only expose the real recipient via one of the other headers). An address like usafa79-coffee@… targets the COFFEE group; a bare usafa79@… targets the whole list.
  2. Special addresses are recognized first: -bounces@, -noreply@, -unsubscribe@/-unsubscribe-<group>@, -groups@, -release@, -discard@. Anything else is a normal post.
  3. Decide: sender vs. MODERATED mode, admin status (LISTADMIN match, SUBJPASS, or a logged-in admin's web session), and whether the target group even exists — an unrecognized group name (not configured, and not any member's value in a (G)-marked field) is reported as an error rather than silently resolving to "no recipients."
  4. If held for moderation, the raw message is written to SPOOLDIR and an admin notice is emailed (see Moderation); otherwise:
  5. Rewrite: subject tag, From/Reply-To per HIDEFROM and DMARC policy, DKIM demotion, a synthesized Date if the original had none, List-* headers, and the unsubscribe footer.
  6. Distribute: batched (up to 30 recipients per SMTP/sendmail transaction) unless tracking or SINGLESENDDOMAINS forces one-at-a-time.
  7. Archive, if ARCHIVE is on.

post --group NAME (as opposed to plain dispatch/post) is a manual/testing shortcut that forces direct distribution to a named group, bypassing normal recipient-address routing. If the moderation decision for that forced post comes back "moderate," it currently just reports that and exits — it does not enqueue the message the way the real dispatch path does. Use dispatch (or plain post with no --group) for anything that should actually go through the moderation queue.

Sending mail: SMTP vs sendmail

Set exactly one of these up, since SENDMAIL: true takes priority unconditionally — if it's on, SMTPSERVER/MAILPORT/SMTPUSER/SMTPPASS are parsed but never actually used.

Local sendmail binary (typical if you're already running Postfix on the same box, which ships a sendmail-compatible wrapper):

{ "SENDMAIL": true, "SENDMAILPATH": "/usr/sbin/sendmail" }

Direct SMTP relay:

{ "SENDMAIL": false, "SMTPSERVER": "smtp.example.com", "MAILPORT": 587,
  "SMTPUSER": "...", "SMTPPASS": "..." }

Sanity-check whichever one you pick without touching real list traffic:

./79mailV -c mylist.conf --mailtest --to you@example.com --via sendmail
./79mailV -c mylist.conf --mailtest --to you@example.com --via smtp

A failed send (bad path, wrong credentials, relay down) reports a normal error either way — it does not take down the process, whether that's a one-shot CLI invocation or, more importantly, the always-running web server.

Moderation

With MODERATED: "ALL", every non-admin post goes through this. With "LIST" (the default), only posts from people who aren't members do.

Holding: the raw message is written to SPOOLDIR/0MOD~<id>, and an email goes to LISTADMIN with the original subject wrapped in [<tag> - MODERATED <id>], and Reply-To set so a plain reply routes to the release address.

Acting on a hold, two ways:

  • Web: log in as admin and open /moderation — every pending message is listed (from, subject, group, how long it's been waiting, size) with Release/Discard buttons right there. The admin nav shows a live count, e.g. Moderation (3).
  • Email: reply to the notice to release it (its subject already carries the marker the release address needs), or send to the discard address to drop it without sending. This still works exactly as before — the web page is an additional way in, not a replacement.

Releasing re-derives the target group the same way the original message was routed (via the envelope-recipient headers, not just a literal To: match), so a message held for a specific group is released to that same group, not the whole list.

Tracking

TRACK: true enables per-recipient open tracking, but only for messages that actually carry a tracking placeholder — which currently means admin /websend compose-and-send messages, not ordinary list posts. A plain post relayed through dispatch/post has nothing to personalize per recipient, so it batches normally regardless of TRACK.

For a message that does carry the placeholder, each recipient gets their own SMTP/sendmail transaction (so a personalized 1×1 beacon can be inserted), throttled by SINGLESLEEP between each — for a large list this trades send-speed for per-recipient open data, so budget for it.

Archiving

ARCHIVE: 1 (per-group) or 2 (combined) appends every distributed message to an mbox file next to your config file (archive, or archive-<group> in per-group mode), and hands the same content to hypermail (if installed) to maintain a browsable HTML archive alongside it. The mbox write always happens and is the durable record; hypermail is best-effort — if it's missing or fails, the mail is still archived and still sent, just without an updated HTML view that round.

Two things worth knowing before you turn this on:

  • The archive files land in your config file's own directory. If that directory is anywhere under something your webserver serves as static files, the raw mbox (everyone's real addresses, unredacted) could become reachable without going through this app's login at all. Keep it outside any web root.
  • The in-app /archive viewer only serves the .html pages hypermail generates and only rewrites internal href= links to stay in-app — any images, attachments, or a separate stylesheet hypermail's templates emit will 404. Message text and threading work; anything non-HTML in there currently doesn't.

Hiding the sender

HIDEFROM: true makes the visible From/Reply-To on distributed mail always resolve to the list address, never the original sender's — this takes priority over anything else (including a sender's own explicit Reply-To), so it can't be quietly bypassed by a crafted inbound message. With it off (the default), the rewritten From still shows the list address (so replies work correctly as list mail) but a human-readable "on behalf of" note names the real sender, and Reply-To defaults to that real sender unless they set their own.

DKIM

Rewriting a message (new headers, footer appended, subject tag changed) invalidates any inbound DKIM signature — it would fail verification on the way back out, which is worse than not having one. So an inbound DKIM-Signature header is always demoted to X-DKIM-Signature (renamed so nothing mistakes it for a currently-valid signature) unless DROPDKIM: true, in which case it's removed entirely rather than kept even in demoted form.

This app doesn't sign outgoing mail with your own list domain's DKIM key — that's a job for your MTA (Postfix + OpenDKIM or similar), not this program.

DMARC / REWRITEDOMAINS

Whether the visible From gets munged to the list address (rather than showing the original sender's address directly) can depend on the sender's domain's own DMARC policy, controlled by REWRITEDOMAINS (space/comma separated tokens, case-insensitive):

  • ALL — always munge, for every sender, no DNS lookup at all. Simplest and safest choice if you don't want to think about this further.
  • An explicit domain (e.g. "gmail.com") — always munge senders from that domain specifically.
  • MAYBEYES — look up the sender domain's DMARC record; munge if it enforces (p=reject/p=quarantine); if the DNS lookup itself fails, munge anyway (fail closed).
  • MAYBENO — same lookup, but on DNS failure, don't munge (fail open).

The DNS/DMARC lookup is a small hand-rolled UDP client (no external resolver library dependency). It's solid for the common case but has two known rough edges if you ever rely on MAYBEYES/MAYBENO rather than ALL/an explicit list: it doesn't validate that a DNS response's transaction ID matches its query, and a truncated/partial UDP response is treated the same as "no DMARC record" rather than "lookup failed" (the less-safe direction for a security-relevant check). Neither matters if you're using ALL, which never reaches this code path at all.

Security notes

A few things worth knowing that don't fit neatly under a feature heading:

  • KEY and ADMINPASS are your two most important secrets. KEY derives every account-action token (validate/authorize/set-password) and the tracking-beacon token — these are intentionally different tokens for the same record (a leaked tracking-pixel URL, which is far more exposed than a one-time password-reset email, can't be replayed as a password-reset link), but both still derive from KEY, so protect it like a master password. Debug logging never writes KEY's value, only whether one is set and how long it is.
  • DEBUGFILE, if enabled, is created owner-read/write-only (0600). If your web server process and your Postfix-piped CLI process run as different Unix users and both need to append to the same debug log, you'll need to chmod it yourself (e.g. 0660) and put both users in a shared group — this app won't widen the permissions for you.
  • Session cookies get the Secure flag automatically, but only once SITEURL is an https:// address — while you're serving over plain HTTP, the cookie is deliberately not marked Secure (a browser would otherwise silently refuse to send it, breaking login entirely). Until you're behind TLS, treat session cookies as visible to anything on the network path.
  • No login rate-limiting exists yet on either the master admin login or member logins. Don't rely on ADMINPASS alone as your only defense — make it long and random.
  • A record write race: two genuinely simultaneous writes to the same member record (e.g. an admin and the member editing it in the same instant) aren't merged or flagged — whichever finishes last simply wins, silently. Low-probability at typical list sizes, but worth knowing it's not currently guarded against.

The JSON cleanup tool

Scans every record in USERDIR and reports (or, with --fix, corrects):

  • Duplicate JSON keys at any nesting level — invisible once parsed (a standard decoder just keeps whichever occurrence came last), so this reads the raw file text directly to find them, independently per object scope (a key repeated inside Address is a duplicate; the same name also appearing once at the top level is not).
  • A placeholder Email.Primary — holds no real address at all (already inert at runtime either way; this just makes the file honest about it).
  • Duplicate Groups entries (case-insensitive), de-duplicated in place.
  • A missing LastName/FirstName — reported only, never guessed at.
  • Files that don't even parse as JSON.
./79mailV -c mylist.conf cleanup          # dry run, changes nothing
./79mailV -c mylist.conf cleanup --fix    # applies the fixes above

Fixed files are rewritten through the same atomic write path (temp file, then rename) every other save in the app uses, so a partially-written file is never left behind even if the process is interrupted mid-run.

Member export & bulk delete (web)

On /users, an admin sees a checkbox per row, a "select all" checkbox, and shift-click range-select (sort the list the way you want first, then shift-click through the resulting order). Once anything's checked, an Export/Delete toolbar appears:

  • Export streams a CSV of exactly the selected members (name, every email address, squadron, address, phone, groups, deceased/valid status) as a download.
  • Delete removes the selected records outright, with a confirmation prompt first. If the admin's own account is among the selection, their session ends the same way the single-record delete already does.

Both are admin-only server-side regardless of what a client sends, not just hidden in the UI for non-admins.

Testing

v test engine          # the bulk of the logic - config, users, post/moderate/
                        # rewrite/distribute, tokens, tracking, archiving, ...
v test apps/usafa       # the web app's own auth/form logic

Every fix and feature in this codebase shipped with tests covering the specific behavior it changed — when modifying something, look for the existing test file with a matching name first (e.g. rewrite_test.v for rewrite.v) rather than starting from scratch.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages