Tolerating Mailing-List Modifications

Introduction Mailing-lists have long complicated email authentication. They break SPF authentication due to forwarding and break DKIM authentication due to message modification that used to identify the mailing-list. This specification provides methods to restore authentication even in the presence of forwarding and message modification. Being able to restore authentication is particularly important as senders may specify a sender-defined receiver email handling policy that may prevent delivery of the message as defined in DMARC . Moreover malicious content is currently attributed to all parties in the mail flow. This specification permits a receiver to attribute the email content to its author be it the sender or the mailing-list. The approach in this document is to be highly opinionated about only supporting common case mutations to lessen the implementation burden upon mailing-lists and receivers. This also seeks to eliminate any burden for messages that don't go through mailing-lists. At origination, it is sufficient to sign a message with a DKIM signature as typically done already. It does ask those mailing-lists that wish to use this specification to characterize the modifications it performs at each forwarding step. The goal is to permit receivers to reverse the modification so that the message hash can be recovered and the prior signature verified at each forwarding step, be it from DKIM or ARC. This allows the receiver to determine which forwarder or the originating sender contributed which content, in the received message. This specification uses the ARC framework to describe each forwarder, and then record the mutation performed at each forwarding step. Consequently this attribution enables more precise reputation systems and UI features. The supported modifications are: 1) adding a description string to the Subject header, 2) rewriting the "From" header, 3) rewriting the original DKIM-Signature and 4) appending a footer to the message body. This document specifies the characterization of the modifications and how to apply the modifications. The validation results in this specification are orthogonal to the results in draft-chuang-replay-resistant-arc. In addition to better supporting DMARC in the presence of mailing-list modifications, this specification enables attribution of malicious content back to the author. However this specification is vulnerable to replay much like DKIM and ARC. draft-chuang-replay-resistant-arc validation is tolerant of header and message body modifications but unable to provide attribution. It also detects and prevents replay. A receiver may want to combine the results of these two validations to create a strong authentication result that provides greater certainty that a message was sent by some originating sender through a mailing-list.

Terminology and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

Algorithm The specification in this document is divided into procedures computed at the mailing-list forwarders and at the receiver and is based on the procedures in ARC . The steps at the forwarder are further divided at the forwarder into the inbound validation procedures and outbound forwarding procedures. The receiver only performs the inbound validation procedures. The inbound validation procedures are variable length corresponding to the depth of the ARC sets. At each ARC set corresponding to some earlier forwarder, the receiver computes any reversing procedure described later for any described mailing-list mutations to obtain the message hash of the message as seen at the inbound of the mailing-list. This hash comes from an ARC-Message-Signature if the sender is a forwarder and a DKIM-Signature (or possibly ARC-message-signature) if the sender is the originator. That signature MUST verify correctly, otherwise this is considered a signature verification failure by this specification. Interpretation of that failure is described later.

Forwarder Characterization Conformant forwarders characterize message mutations to enable receivers to interpret mutations and potentially verification failures. A description is added as a tag-value to the ARC-Message-Signature. The mutator description tag is "m" and has the following values:

mailinglist:: Describes a forwarder that distributes messages to multiple recipients and any modification conforms to the specification in this document. Consequently signature validation failures are described as "fail".
gateway: Optional mutating forwarder that works on behalf of the originating sender or final receiver. The receiver is presumed to have an established trust model with the gateway outside of this document and, and validation failure result is interpreted by the receiver. The receiver with such an arrangement MUST NOT use this description to detect the existence of such a relationship. However a gateway forwarder may provide a description for human operators to interpret gateway's actions.
generic: Optional self-description for a generic mutator. This document imposes no judgment on signature validation failures but the description may help a human operator interpret the actions of the forwarder.
none: Conformant forwarders that do not mutate the message declares their participation and lack of mutation with none value.

For example: Receivers use this mutator description tag to interpret the forwarder's action and whether they followed the guidelines in this specification. If the forwarder promised that they did, then it makes it possible to enforce a validation failure that protects that forwarder.

Header Rewriting Characterization A mailing-list forwarder that rewrites a Subject, From or DKIM-Signature headers can store the prior values by substituting with a modified header name i.e. prefixing "Prior-" to the original name. Moreover the header name retains the original capitalization, its place with respect to other headers, and is not ambiguous if there are multiple headers of the same name. First the bottoms-up line count of all headers is computed starting from the 0th position. Then all headers to be rewritten are collected, ordered by position to be prepended to the headers list and given a bottoms-up line count position. This is an offset from "Prior-" header to it's rewritten counterpart. It is assumed that the prior header has the following format: : ]]> The conserved header name is prefixed with a "Prior-". Then an ARC instance number tag "i" i.e. "=<#>," and line count tag "l" i.e. "l=<bottoms-up line count offset>" are prepended to the header value and separated by a semicolon and one whitespace. The position of this "i" and "l" is important because they may otherwise be ambiguous with the original header's tag-values e.g DKIM-Signature. Thus the conserved header is: : i=<#>; l=; ]]> For example: This stores the prior header in a format that SHOULD be ignored by subsequent forwarders, and if not then will cause a verification failure. The forwarder then rewrites or deletes the header as before by prepending the rewritten header to the message or skipping prepending for deletion. To protect the integrity of these headers, these headers are hashed separately from ARC-Message-Signature "h" header list. The forwarder generates the header hash by hashing the "Prior-" headers (and any "Content-footer" header described below) found in bottoms up order as found in the headers. This specification calls for the rewritten header hash to be explicitly added to the ARC-Message-Signature with a tag "hh". Conformant forwarders even without mailing-lists mutations MUST report header hash explicitly to better differentiate and tolerate body modifications when verifying headers. To reverse the rewriting or deletion of headers, all "Prior-" headers are collected at the given ARC instance number. The prior header name is extracted from the "Prior-" header, and the original value extracted and restored. The rewritten header is found by taking the "Prior-" header line count position and adding the offset in the "l" tag's value. Rewritten headers are deleted in top down order.

Message Footer Characterization Mailing-lists may choose to add a message footer to the body of the message to notify the recipient of the mailing-list distribution action and conformant mailing-list will describe this mutation to the receiver. One method is to append the footer text to all content-type text in the top level of MIME part "multipart/alternative". Another method is to append a new successor MIME part for a "multipart/mixed" part, and that MIME part contains the footer as a content-type text. If the message is not mime formatted, the entire body is treated as text and the mailing-list may append a text footer at the bottom of the message. Conformant mailing-list forwarders characterize these changes with a new "Content-footer" header. It contains a list of semicolon and whitespace separated tag-values using the formatting specifications in DKIM. The first of these is an ARC set instance number describing which forwarder added the header. When the mailing-list appends a text part, the receiver characterizes it with the beginning and ending byte offset of the MIME part or message-body if MIME unformatted. If the mailing-list adds a text MIME part as footer, it describes the mime part with its purpose tag-value. The Content-footer header is added to the MIME part for MIME formatted messages, and to the message headers for unformatted messages. To protect the integrity of the header, in the latter case, the Content-footer is hashed along with "Prior-" headers in the same bottoms up order. In the former case, Content-footer headers added to the message-body are protected by the message body hash. The following are the Content-footer header tag-values:

i: Instance tag. Must be the first tag-value. The value contains the instance number
b: Footer beginning octet offset from the start of the text MIME part or the message-body if no MIME unformatted. MUST always appear with a footer ending offset.
e: Footer ending octet offset from the start of the text MIME part or the message-body if no mime unformatted. MUST always appear with a footer beginning offset.
m: Mime-part purpose description. If the added MIME part contains the text footer, then the value is "footer". If the added MIME part is "multipart/mixed" and meant to support adding the text footer, then the value is "mixed". If the text footer is appended, then this tag-value is not added.

For example: To reverse the footer addition, the receiver looks at the headers to see if the message is MIME formatted. If not, the receiver looks for a "Content-footer" in the headers. If so, the receiver looks for "Content-footer" in the MIME tree content headers. For each "Content-footer" found header at the current ARC set instance, it reverses the footer mutation. For offset values designated by beginning and ending offset tags, it removes the text from that MIME part or the message-body designated. For a MIME part text footer, it deletes it along with any supporting "multipart/mixed" MIME part. Then the receiver deletes the Content-footer header from the updated message if not done so already.

Validation The receiver verifies prior ARC sets per the procedure described in ARC . In addition, the receiver validates the ARC sets starting from the largest instance number found to the smallest. First the receiver verifies the given instance ARC-Message-Signature or DKIM-Signature as appropriate. Then the receiver computes the rewritten header hash taking the header hash computed by ARC-Message-Signature at the given instance number or DKIM-Signature. Then the hash is taken for "Prior-" headers and "Content-footer" header if found in the headers at the current ARC set instance number and all prior ones. This is verified against the header hash value associated with the tag "hh", reporting signature failure if it mismatches. Next the receiver determines whether it needs to reverse any header or footer mutations at that ARC set instance by looking for the ARC-Message-Signature mutation tag "m=". For values of "mailinglist", it attempts to reverse mutations keeping the resulting message so that further validations are possible. The receiver attempts to provide header reversing procedures given in "Header Rewriting" section and body reversing procedures given in "Message Footer" section. For values "gateway" the receiver MAY apply local policy to interpret subsequent validation failures. For all other mutation tag "m" values, it assumes no mutations are present or outside the scope of mailing-list modifications. In addition, the originating sender's DKIM-signature or ARC-Message-Signature MUST successfully verify. If so and all prior signatures verify, then the result is a "pass". Verification failures are subject to interpretation in "Footer Characterization" section, and potentially indicate a "fail". The result of this procedure is written in Authentication-Result and ARC-Authentication-Result with a method named "reverse" as the REVERSE result. Informationally, if the receiver implements draft-chuang-replay-resistant-arc, this specification suggests modifying draft-chuang-replay-resistant-arc PATH results to take into account the REVERSE result. At each ARC set instance where PATH recursively combines the local DARA (or SeRCi) results, if REVERSE reports "fail" then the PATH result reports "fail". Because REVERSE _combined with DARA _represents a higher bar of verification than DARA alone, the receiver applies local policy when interpreting the PATH result.

Mailing-List Modifications When a message body or Subject header is modified by a forwarder, the sender's DKIM signature will no longer validate. To mitigate this forwarders MAY elect to replace the DKIM signature with their own with a new message hash that takes into account the modifications. Because the signature is not DMARC aligned, senders also MAY rewrite the From header to take ownership of the message. The following creates a specification making the message body or Subject header modification, replacing the DKIM signature and applying characterization headers.

Subject Header Modification The mailing-list may want to communicate to the recipient that a message went through a mailing-list by modifying the Subject header to append the name of the mailing-list. Typically the name is put between brackets e.g. "[name]" and prepended to the Subject header content. This specification supports arbitrary text changes by saving the earlier inbound version of the Subject header's content in the "Prior-Subject" header. The change in Subject text will break the DKIM-Signature so mailing-lists MAY rewrite DKIM-Signature to update the message header hash and resign the signature. Similarly they MAY update the From header to DMARC align the DKIM-Signature "d=" domain with the From header domain. Typically the From address is the set to the mailing-list address to further identify the mailing-list. This specification supports saving the earlier inbound version of the DKIM-Signature and From headers in the "Prior-DKIM-Signature" and "Prior-From" headers respectively. For example the original message looks like: A mailing-list rewrite Subject, From and DKIM-Signature headers and saves the original content in the "Prior-" headers

Body Modification The mailing-list may want to communicate to the recipient that a message went through a mailing-list by adding a text footer describing the mailing-list. Typically this is done by appending that text description at the bottom of message body text. This specification supports three different footer organizations, depending on the MIME structure, content type and content transport encoding of the MIME parts of the message. In particular the organization depends if its MIME parts can be concatenated, meaning whether a text footer can be appended and removed without having to re-encode the original content to preserve the original content. For example appending new content to "base64", and "binary" will likely introduce artifacts between the original message and the interpretation footer. MIME parts that can be concatenated are defined as having one of the following type and subtype of Content-type:

text/plain
text/html

and one of the following mechanisms for Content-transfer-encoding:

7bit
8bit
quoted-printable

Similarly, non-text MIME parts, and complex multipart MIME parts don't lend themselves to appending text. One exception to this is "multipart/alternative" as will be described shortly. To handle these scenarios, this specification calls for adding a new text MIME part footer that can handle any encoding or any mime structure but requires altering the MIME tree. The following procedure describes how the mailing-lists MAY choose from one of those formats to add a footer. Here a mailing-list should evaluate in the same order as these three sections, following the steps in each section. If any footer addition is successful, then the footer algorithm stops.

Unstructured or Text Message Body The simplest structure applies when the message lacks MIME structure or the top level MIME part can be concatenated. Here the forwarder appends a text footer with the same content type and content transfer encoding as the message-body. When a message lacks MIME struct, the default message content-type is "text/plain" (section 5.2) and default content transfer encoding is "7bit" (section 6.1). Next a "Content-footer" header is prepended describing the start and ending message body octet offsets. Reversing this transform is a straightforward deletion of the footer at the offsets given. For example given an original message of: A text footer can be appended as follows:

Multipart/Alternative When the top level MIME part is Content-type "multipart/alternative", the forwarder checks if any of the immediate children MIME parts can be concatenated. If so, it attempts to append a text footer with the same content type and content transfer encoding as the children MIME part. Next it prepends a Content-footer header in the child MIME part header description with the start and end octet offsets from the beginning of the part. If a footer can be added to a child MIME part, then this is considered success and the algorithm can halt. The reversing algorithm is to look for the top level MIME part with Content-type "multipart/alternative". If so then look in the immediate child MIME parts and delete the text at the offsets given by the Content-footer. For example given an original message of: A text footer can be appended as follows:

MIME Part Footer The following procedure adds a MIME "multipart/mixed" at the top level and has as its children the original top level MIME part as the first child and the footer MIME text part as the second child. This procedure will always succeed no matter the MIME structure or MIME content transfer encoding, however this modifies the MIME tree. The top level MIME "multipart/mixed" is denoted by adding a "Content-footer" header purpose description with "mixed" value and the footer MIME part denoted by adding a "Content-footer" header purpose of "footer" value. To reverse the mutation, the original top-level mime part is moved back to the actual top-level and delete the "multipart/mixed" and the footer MIME parts. For example given an original message of: A "multipart/mixed" and text footer MIME parts can be appended as follows:

Examples These examples are informational.

Originator ⇒ First Mailing-List ⇒ Second Mailing-List ⇒Receiver This message is sent through two mailing-lists to some receiver. The originating sender creates and signs a message as follows: The first mailing-list adds a Subject header prefix and message-body footer, and denotes this using the procedures and headers specified in this document. It denotes that it performed mailing-list mutations in the ARC-Message-Signature. The second mailing-list adds a Subject header prefix and message-body footer, and denotes this using the procedures and headers specified in this document. It denotes that it performed mailing-list mutations in the ARC-Message-Signature. The receiver sees the above message on inbound delivery, and attempts to verify the ARC message signature at the i=2. Upon success, the receiver notices that there were mailing-list mutations at ARC set i=2. It applies the REVERSE validation algorithm to reverse the mutations from the second mailing-list. After applying the reverse procedure, the reversed message looks like: Again the receiver attempts to verify the ARC message signature now at the i=1. Upon success, the receiver notices that there were mailing-list mutations at ARC set i=1. It applies the REVERSE validation algorithm to reverse the mutations from the first mailing-list, obtaining the original message. The reversed message looks like: The resulting reversed headers and message body DKIM-Signature verifies, and the REVERSE passing result is published in the ARC-Authentication-Result: This is the district mailing-list.

Security Considerations Care must be used if the reversed transformed message is used for authentication. The reversed transformed message is vulnerable to replay attacks and "hides" the potentially spammy contribution from the mailing-lists.

IANA Considerations There are no requests at this time.