D. J. Bernstein
Internet mail
Internet mail infrastructure

Moving a message closer to a remote recipient

For each domain, there are some computers---usually just one---with detailed instructions from the domain's owner on how to deliver messages to various addresses in that domain.

All other computers follow the general procedure described here. Each host has a distance from the domain; the strategy is to transfer responsibility to a host whose distance is smaller.

Distance

Intuition: Distances are published in the domain's MX records in DNS. For example, the MX record "7 openbsd.cs.colorado.edu" for openbsd.org says that openbsd.cs.colorado.edu is at distance 7 from openbsd.org.

Question: What if openbsd.cs.colorado.edu has several A records pointing to different hosts? What if the same host has several IP addresses showing up in different MX records?

Answer: ``Distance from H to D'' is defined as the smallest preference p such that H is touched by a preference-p MX record for D, i.e., H has an IP address listed as an A record of a preference-p MX record for D. If none of D's MX records touch H, the distance is 100000.

Exception: If D does not have any MX records, ``distance from H to D'' means 0 if H is touched by D, 100000 otherwise. This is equivalent to D having itself as a preference-0 MX record.

Finding all the closer hosts

Given a domain D, a client uses the following procedure to build a list of IP addresses of all the hosts that are closer to D than the client host.

Look up the MX records for D. If there are no MX records, use a default MX record of ``0 D.'' (Note that a temporary DNS failure looking up MX records does not necessarily mean ``no MX records''; the client has to wait and try again later.)

Run through the MX preferences in increasing order. For each preference p, shuffle the preference-p MX records into random order, and then look up all the A records of all the preference-p MX records. If any of those preference-p IP addresses match any of the client host's addresses, stop. Otherwise add each of the preference-p IP addresses to the list.

For example, here are the MX records for ohio-state.edu, and their A records:

     ohio-state.edu. IN MX  9 mx3.osu.edu.
     ohio-state.edu. IN MX  9 mx4.osu.edu.
     ohio-state.edu. IN MX 30 ds2.osu.edu.
     ohio-state.edu. IN MX  9 mx1.osu.edu.
     ohio-state.edu. IN MX  9 mx2.osu.edu.
     mx1.osu.edu. IN A 164.107.4.5
     mx2.osu.edu. IN A 164.107.4.6
     mx3.osu.edu. IN A 164.107.4.7
     mx4.osu.edu. IN A 128.146.225.199
     ds2.osu.edu. IN A 128.146.225.198
The client's list consists of the four distance-9 hosts in a random order, and then the distance-30 host: e.g., 164.107.4.6, 128.146.225.199, 164.107.4.7, 164.107.4.5, 128.146.225.198. If, however, the client is ds2.osu.edu (128.146.225.198), then its list stops after the distance-9 hosts; the list is limited to hosts whose distance is smaller than the client's distance. If the client is one of the distance-9 hosts, its list is empty.

Transferring responsibility to a closer host

Here is what a client does when it has responsibility to deliver a message to a remote recipient. It builds a list of IP addresses as above. For each address, it attempts to
  1. make an SMTP connection to port 25 of that address and
  2. transfer responsibility through SMTP. Note that there are alternatives to SMTP.
If an SMTP server accepts responsibility, the client has succeeded. If an SMTP server permanently rejects responsibility, the client gives up and sends a bounce message.

The client can stop at any point in the list after the first address. (For example, most mailers will stop if an SMTP connection is established but then fails temporarily. Lotus Notes reportedly stops after two MX records in all cases.) This means that, if all of the lowest-distance hosts are persistently unreachable, there is no guarantee that mail will be delivered, even if there are reachable higher-distance hosts in the MX list.

Note that, if the client is a lowest-distance host, its list of addresses will be empty. This indicates a local configuration error; all of the lowest-distance hosts should have been given delivery instructions for this domain. The client bounces the message immediately.

The behavior of mail exchangers

When an server accepts a message, it places a Received field at the top of the message. It does not touch any Received fields that might already be in the message.

Several clients may attempt to send messages to the same SMTP server at the same time. The SMTP server ignores or rejects TCP connection attempts if it is already handling as many simultaneous TCP connections as it can. Busy servers have to handle hundreds of simultaneous connections.

When an SMTP server pauses to think about a request, it is chewing up memory on the client host that could be used for other tasks. High-quality servers respond to every request quickly.

Servers generally do not reject messages with malformed headers. Many servers do not inspect messages at all, except to count Received fields.

Dumb clients

Dumb clients are client hosts that do not accept mail from the network. Dumb clients are defined to be distance 999999 from any domain. A dumb client can transfer mail to any host willing to accept it, without regard for MX records.

Typically a dumb client is configured to send all outgoing mail through a nearby ``smart host.'' It is not safe to select a random host on the network as a smart host; many hosts do not allow relaying from unauthorized clients.

Sometimes, by bilateral agreement between the dumb client and the smart host, the client uses a protocol other than SMTP to transfer mail. For example, QMTP is much faster than SMTP.

Sites not connected to the Internet often set up two or more layers of dumb clients. At a typical UUCP site, for example, most of the computers are configured to send all mail to a dialup host, which is configured to send remote mail through UUCP to an Internet smart host. This does not obey the general strategy of decreasing distance; misconfiguration can lead to loops.

Historical notes

The use of MX records was explained in RFC 974, published in 1986.

RFC 974 implicitly assumed that each Internet host is touched by only one name. Then a host can look for its name, rather than its IP addresses, in MX lists. In practice, however, multihomed and multinamed hosts are common, and mailers that do not check IP addresses (e.g., sendmail) sometimes get into loops when they fail to recognize themselves in MX lists.