Mailing lists and mail forwarders vs. DMARC

DMARC alignment and policies DMARC introduced an authentication system intended to detect and deter domain name impersonation in mail message From header fields. DMARC says that the address in a message's From header is "aligned" if the message also has a valid DKIM signature with the same domain, or the message's RFC5321.MailFrom (which we'll call the MailFrom) is in the same domain and it passes SPF checks. (This is somewhat oversimplified but close enough for our purposes. See for all the details.) Each domain can publish a DMARC record with a policy advising recipients what to do if the receive a message with the domain in the From header that is unaligned. The policy options are "none" for do nothing special, "quarantine" to mark the message as dubious typically by putting it in a spam folder, or "reject" to reject the message in the SMTP session. At this point most large mail systems follow senders' DMARC policy advice most or all of the time. While this definitely deters a lot of phishing attempts, it also makes a lot of long standing mail practices fail.

Forwarding failures Forwarding is a standard feature of Internet mail. One can send a message to one address that is then remailed to a different address. Common use cases are that someone moved from one organization to another and the mail follows her for a while, a school or professional organization offers a mail address that won't change even if someone changes mail providers, or (as at the IETF) a role address forwards to the people currently handling the role. DMARC makes forwarding fail in two ways. If a message is only aligned using SPF, remailing will make subsequent SPF checks fail since the message is now sent from the IP address of the forwarder, not the original sender. The remailer can fix the SPF failure by replacing the MailFrom with one in its own domain, but since that domain is different from the From header domain, there's no DMARC alignment. DKIM was designed to survive forwarding since it does not depend on the path that the message takes. Nonetheless, forwarded DKIM signed messages sometimes fail because the forwarding system modifies the message in ways that break DKIM signatures. Some mail systems tidy up message headers by rewrapping them or adjusting spacing, which will make DKIM validation fail if the signature uses the default "simple" algorithm but still succeed if it uses the optional "relaxed" one. (This makes debugging intermittent DMARC failures a challenge.) Sometimes forwarding systems add a tag to the end of the message which will also break the DKIM signature. When a forwarded message fails DMARC alignment, it might disappear into the recipient's spam folder if the sender policy is "quarantine", or reject at the end of the SMTP session it if the policy is "reject". The rejection report might bounce back to the forwarder, or to the original sender depending on what tne MailFrom is. Adding to the confusion, some mail systems such as Gmail look with disfavor on unauthenticated messages, independent of DMARC. This means that a forwarded message without a valid DKIM signature and that fails SPF due to forwarding is likely to get misfiled as spam.

Mailing list failures When mail transits a mailing list using a list manager such as Mailman or Sympa, messages suffer all of the DMARC problems of forwarded mail, and many more. Lists invariably replace the MailFrom with the list manager's address, so the list gets any bounce messages and can use them to prune dead addresses. This means any messages that only used SPF validation invariably fail DMARC alignment. Lists usually change the contents of the message, adding tags to subject lines, and headers and footers to message bodies. Some lists make even larger edits, reordering or removing MIME parts, or flattening HTML to text. All of these changes invalidate DKIM signatures. When a message from a mailing list fails DMARC alignment, all of the problems of forwarded messages are possible, along with a worse one, getting bounced off a list. Mailing list software collects bounce messages, and after some number of bounces (adjustable by heuristics and configuration), a subscriber is removed from the list. This can happen to any subscriber using a mail system that follows DMARC p=reject policy, regardless of what policy their own system publishes; any message from a domain with a p=reject policy provokes this problem.

Workarounds Painful experience has shown that telling mail system operators about the problems caused by their overly strict DMARC policies rarely helps. ("It must be your problem, it works for everyone else.") Forwarders and particularly mailing lists have come up with a variety of workarounds to try and get the mail delivered, with more or less severe costs to usability.

Irreversible From rewrites The most common mailing list workaround is to replace the address in the From header with the list's address. The MailFrom is in the same domain, which provides SPF validated DMARC alignment, and lists often apply their own DKIM signatures which provides DKIM validated alignment. This makes it harder to tell who originally sent the message. Some lists put the author's name or address in the From header comment, but some don't. On lists that don't, it is often impossible to tell who sent a message unless they happen to put their name in the message body. Some lists put the author's address in the Reply-To header, which makes it possible to respond to the author but also makes the default response private rather than to the list, which is rarely what list users want. For forwarded SPF validated message, mail systems can replace the MailFrom with the forwarder's address. This provides valid SPF but if the final delivery fails, there is no way for the original sender to find out. In the frequent case where the forwarder is an address that nobody looks at, it means that the bounces are unlikely to be noticed or acted on.

Reversible From rewrites When the local mail system allows, a less bad approach is to rewrite the From address into an address in the local domain and add a DKIM signature from the rewritten domain in a way that makes it possible to recover the original address. For example, my original implementation would rewrite steve@aol.com to steve@aol.com.dmarc.fail. The IETF's mail system would rewrite it to steve=40aol.com@dmarc.ietf.org. LISTSERV uses an opaque hash, something like 18a1298287d8@listserv.com. In each case, the mail system sets up a temporary forward so mail to the rewritten address is mailed back to the original author. This largely preserves the regular list semantics at the cost of the rewritten addresses ending up in users' address books, and the same forwarding problems when the responses are forwarded back. For this approach to be workable, the mailing list operator has to have enough control over the underlying mail system to manage the rewritten addresses and forwards. If as is often the case the list is on a shared server with a mail system run by someone else, this approach isn't practical. For mail forwarding, the SPF analog is known as Sender Rewriting Scheme (SRS) which would rewrite the address to SRS0=HHH=TT=aol.com=steve@forwarder.com where HHH is a validation hash and TT is a timestamp. SRS was proposed around 2003 but has never been widely used.

Message wrapping A different approach is to wrap the original message in an outer message from the list or forwarder. The original message might be wrapped as a single message/rfc822 body part, or as message/rfc822 inside multipart/digest, a one entry message digest. The outer message has the list's address in the From header, DKIM signature, and MailFrom, so it sails through DMARC alignment. The problem is what happens when recipients try to read the message. Some desktop mail programs deal with wrapped or digest messages fairly well, presenting the internal messages as real messages, some are so-so, attachments you can click on to see the internal messages, and some badly, without showing the internal message as separate from the wrapper, with no way to reply to the internal message short of cutting and pasting the subject and addresses by hand. Web mail consistently handles wrapped messages badly. I have never seen a web mail system that allowed responses to wrapped or digest messages. (Mailing list users are all too familiar with this problem, with replies typically starting "Re: foo list digest, Vol 42, No 17".)

List unmunging There have been some attempts to allow mailing lists to describe the modifications they make to messages on the way through, such as . The idea is that recipient systems can undo the changes and revalidate the original message to check for DMARC alignment. This may or may not be practical this is for several reasons. One is that lists make a lot of different kinds of changes, so it may not he possible to describe enough of them to be useful. More important, this depends on the recipient deciding when the modified message is "too different" to accept. In an extrme case, a malicious list might completely replace the original body with spam, or use HTML coding tricks to make the original body invisible and just show the added spam.

ARC ARC is intended to allow forwarding mail system to include the authentication history of a message. Each forwarding system adds a signed group of ARC headers which includes the authentication results of the message as it arrived at that system. This lets a recipient system look at the ARC headers on a mailing list message, see if it was DMARC aligned when it arrived at the list host, and treat the message as aligned even if it no longer is. The main limitation of ARC is that there is no technical bar to signing fake authentication results, so one can only use ARC headers from trustworthy forwarders. While it's not hard for large systems to use existing reputation data to decide who is trustworthy, it's a problem for small systems. ARC was designed in 2019 and many mail systems including Gmail and Outlook add ARC headers. Some mail systems such as Microsoft's allow administrators to configure lists of acceptable ARC forwarders, but none to my knowledge do so by default. It remains unclear what the practical barriers to adoption, beyond the forwarder reputation, are.