Practical Exim - Configuration Book
===================================

Dan Shearer

1. Introduction
===============

Exim is a Mail Transfer Agent; a free tool for delivering electronic
mail on the Internet. Internet electronic mail is nothing more than
transferring text files from the sender to the recipient. The facility
has existed for nearly thirty years!  Sadly, the simplicity of the
principle is not reflected in the task that mail-handling programs
have. These programs, or Internet mail gateways as they are called,
are necessarily complicated.

New tools like Exim are needed because the task to be done is changing
constantly. Internet email management is definitely not a solved
problem. The World Wide Web has been wildly successful, yet email
continues to be the most used and most important Internet facility and
in some way is part of most new Internet technologies.

The Exim suite of commandline and graphical mail tools is one of the
more advanced mail delivery solutions available today at any price.
Exim's popularity speaks for itself. ~~TODO supply useage stats when
survey tool ready~~

1.1. Why Read This Book?
------------------------

If you want to solve a real-world mail transport problem then this
book is for you. You will find all the information required to build
both simple and complex gateways, with many examples. The finer points
of advanced Exim use are covered, particularly regular expressions and
embedded Perl. Users with an existing mail infrastructure that Exim
needs to cooperate with will find advice here, as will those who use
more exotic LDAP, DNS or even NIS database solutions.

This book is also aimed at anyone interested in Exim. Beginning
postmasters should concentrate on the earlier chapters first while if
you are already a mail administrator you can start with the specifics
of the Exim delivery model. A key market for this book are frustrated
postmasters bruised by encounters with sendmail or horrified by the
few commercial equivalents.

1.2. Exim's Guiding Principles
------------------------------

Exim is designed around some fundamental technical assumptions. These
have lead to some distinguishing design goals, developed and enhanced
since its inception.

General Purpose

     Other MTAs explicitly set out to be the most secure, fastest,
     smallest or best upgrade to sendmail available. Some of these
     MTAs have achieved significant success in their niche, but they
     emphasise their speciality at the expense of other functionality
     and usability. Most of the time users find Exim meets all their
     needs with a minimum of effort.  This is not to say that Exim is
     either slow or insecure, just that the design addresses first of
     all the most common mail delivery scenarios. Exim is
     forward-looking, without undue emphasis on delivering to a local
     Unix mailbox. Exim is ideally suited to be your first MTA.

Compatible

     Exim tries to be compatible with as many things as possible,
     sometimes making pragmatic decisions in order to help mail
     administrators and above all deliver mail correctly. Exim is
     compatible with the default configuration of every major and many
     minor variants of Unix; copes with mail passed from many kinds of
     buggy MTAs; has Perl-compatible syntax; is compatible with the
     sendmail commandline interface and can access all common Unix
     mailbox formats.

Extensible

     Exim is exceptionally flexible. A number of predefined components
     that deal with messages, databases and Internet protocols can be
     assembled to suit your particular needs.  If more advanced logic
     is required then these site-specific definitions can have strings
     of symbols embedded in them. These strings are subject to
     expansion and interpretation as regular expressions. Failing all
     else, there is an option to include the entire Perl language.
     Anything that can be written in Perl can be used to handle
     messages, and there is nothing that cannot be written in Perl! A
     mail filtering language is also provided with Exim which fits the
     gap between string expansion and external Perl routines.

Apart from all this, Exim has some features not commonly found in
MTAs. For example, Exim optionally supports:

o    the next generation of TCP/IP, IPv6

o    built-quotas independent of any operating system

o    customisable error and warning messages, with string-expansion
     facilities available at runtime in the messages

1.3. Mail Transport Solutions Compared
--------------------------------------

On the Internet debates over MTAs sometimes last for years and such
discussions have no place in this book. Benchmarking between the
various MTAs is very rare, which makes performance comparisons
difficult. The remarks here are mostly confined to readily-verifiable
facts, with some consensus opinions where the answers seem obvious. It
is interesting that nearly every MTA seems to have a combative and
vociferous group of supporters who are also technical experts, except
perhaps zmailer and Exim. The qmail and sendmail forums tend to be
particularly partisan. The philosophy of this book is that you should
use whatever tool works well for you.

Sendmail

     Sendmail is the most popular MTA and is reckoned by many
     authorities to deliver upwards of 65% of all Internet email. That
     works out to be billions of messages every day, so sendmail is
     one of the main reasons that the Internet is useful to so many
     millions of people. This deserves great respect.

     Like a centennarian sendmail's faults are chiefly due to age, but
     like communism it is a long way from dead yet. If the principal
     author of sendmail (Eric Allman) had foreseen the success of his
     program perhaps he would have designed it differently. Sendmail
     has an extraordinarily obscure configuration file, a poor history
     of security breaches and a design centred around Unix in the
     early 1980s. Add to this sendmail's renowned inefficiency and it
     might be hard to see why sendmail is still used at all. However
     sendmail:

     o    is installed by default on many freeware and commercial Unix
          operating systems, and for simple situations the default
          settings work with little or no modification

     o    has a large following of systems administrators who have
          battled and now understand - or think they understand - how
          to configure and run sendmail. There are even some who like
          it

     o    is entrenched by years of scripting and programming in the
          infrastructure of many organisations

     o    is a well-known name

Exim was inspired largely by the author's work with the smail 3
sourcecode, which was itself provoked by the many problems of
sendmail. Exim therefore embodies quite a history of improvements on
sendmail.

qmail

The primary design goal for qmail is to replace sendmail and give more
security and performance in the process. It is rare to find a qmail
user without some sendmail experience. qmail has appeal to those
highly technical administrators and end-users who:

     o    want a better sendmail. qmail is aimed directly at providing
          advanced mail features and good performance to existing
          sendmail users

     o    don't mind restructuring their Unix management practices to
          cope with a brand-new local delivery format, a plethora of
          mail programs (four long-running daemons and many others
          besides) and a large number of configuration files

     o    want to use the qmail-specific mailing list manager, or
          other qmail tools

Many people find qmail a rather radical solution. In comparison with
Exim it could hardly be regarded as a general-purpose MTA. The
principal author, Dan Bernstein, is a widely known and prolific
freeware author. The Constant Database Package written by Dan and used
in qmail can also be used by Exim.

Postfix

Postfix is similarly written by a prolific author, this time Wietse
Venema. Postfix has some features of qmail (especially being composed
of many small programs and having an extreme view of security) but
without the flexibility of either qmail or Exim. The intention is that
over time Postfix will get these features. Postfix uses the
Perl-Compatible Regular Expression library that was developed as part
of Exim.

Others

Following are brief comments on other well-known MTAs.

smail 3 - Free, aimed at replacing sendmail. Source of some of the
ideas in Exim. Maintainence lapsed for a while but has recently picked
up again.

zmailer - specialises in minimising server load by employing advanced
queue-management techniques. Specifically aimed at the largest and
most complicated installations.

Microsoft Exchange - negligible usage on the Internet, and included in
this list solely because many people have heard of the product.
Exchange is allegedly quite popular within large organisations but
suffers from the same principle problem as other non-free MTAs. At the
time of writing even Microsoft themselves have never implemented one
of their own products as an Internet gateway.

1.4. Things Exim Wasn't Designed To Do
--------------------------------------

There are some things that Exim was never intended to do. These
include:

o    Store Mail. Exim delivers mail and has no ability to store or
     retrieve mail. An MTA should not be used for this purpose
     although commercial advertising sometimes suggests the opposite.

o    Communicate using non-Internet mail client mail protocols. The
     Microsoft family of mail clients have maximum functionality when
     communicating with a Microsoft Exchange server using file-sharing
     protocols. This is completely outside Internet standards and, as
     users are finding, has many drawbacks. These clients can also use
     Internet protocols and can therefore use Exim when configured to
     do so. The freeware Windows NT Server clone Samba can with
     difficulty be configured to act as an Exchange server in this
     sense.

o    Communicate using encrypted Internet protocols. There are ways of
     transporting the SMTP protocol securely, but this happens without
     Exim's knowledge (~~TODO Refs~~). There are many techniques for
     securely encoding the contents of a message while still leaving
     it compliant with RFC822. However like other MTAs Exim does not
     attempt to defend against traffic pattern analysis or other
     advanced snooping techniques.

o    Run on Microsoft Windows platforms. The port could theoretically
     be done (using free products such as Cygwin) but has not happened
     yet.  There are many Posix-specific assumptions in the Exim
     source code and Windows NT as shipped has very weak Posix
     support.

There are also some tasks that many people use Exim for which might be
better addressed in another way:

o    Handling UUCP bang-path addresses such as fred!site1!site2. There
     are several Exim configurations available that do a reasonable
     job of UUCP addresses but these are not nearly as robust as
     purpose-built UUCP relay agents.  UUCP use has declined
     enormously and very few Internet gateways really need the
     facility.

o    Storing mail for dial-in computers until they connect to collect
     it. Some use Exim this way but an IMAP or POP server in
     conjunction with client programs such as fetchmail are probably
     more suitable.

o    Accepting external mail on dial-in computers. Many people use it
     this way and it certainly works, but again fetchmail is
     well-suited to this job.

o    Per-destination rewriting for a message with multiple recipients.
     This was an unforseen requirement that many users have since
     needed. Solving it neatly will require considerable work.

o    Be anything more than a simple mailing list handler

1.5. Exim and Free Software
---------------------------

All MTAs widely used on the Internet have their sourcecode freely
available. Free software (sometimes called Open Source software) is
one of the prime reasons for the success of the Internet. ~~TODO Check
with survey results~~

There are various different mechanisms for declaring a piece of
software free, most of which involve a license. Exim is copyright
under the terms of the GNU Public License (GPL). In brief, this
permits any use of the source code including any commercial activity,
provided that any derivative works are made freely available. The
concepts behind free software and this license are well explained by
the FSF and OSS web sites. The term public domain applied to software
means that no restrictions are placed on its use, modification or
redistribution. GPL'd software is not in the public domain because
there is a very definite restriction on hiding changes to sourcecode.

For those with the inclination, free software gives the opportunity to
peruse the source code of fine programs, learn from them and
contribute back. In the case of Exim the principal author has
painstakingly documented the logic in comments in the sourcecode,
making the audience as wide as possible. Very few programs have such
verbose comments in them, and while some very experienced programmers
might find them irritating the rest of us can learn a lot from what
Philip has written.

The GPL legally enforces the spontaneous culture of sharing ideas that
created the Internet in the first place. Respect it.

1.6. Conventions Used in this Book
----------------------------------

~TODO. To be filled in. Who really knows yet?

emphasised text

filename

Products such as Exim

 Commands you type
 Responses from the server

1.7. Running the Examples in this Book
--------------------------------------

There are very few specific requirements for running Exim apart from
those covered in Installing and Running Exim. A summary of what is
needed is:

o    A Unix or Unix-like operating system supported by Exim. At the
     time of writing there were over 30! Free operating systems such
     as Linux and FreeBSD which run on many kinds of hardware
     (including cheap Intel-compatible PCs) are popular Exim
     platforms. Debian GNU/Linux has Exim preinstalled as the default
     MTA.

o    Access either to tools to compile and build Exim (available on
     nearly all Unix platforms) or precompiled binaries for your
     particular operating system. You will of course need to be root

o    root access, although very little work requires it

1.8. Acknowledgements
---------------------

This book has had a long gestation, and a most peculiar
metamorphosis...

!sendmail

Viola

Smail 3

spec.txt

2. About Internet Mail Systems
==============================

2.1. The Internet Mail System
-----------------------------

821+822+IP == internet mail.

2.2. Alternative What Is SMTP :-)
---------------------------------

RFC2476

SMTP was defined as a message *transfer* protocol, that is, a means to
route (if needed) and deliver finished (complete) messages. Message
Transfer Agents (MTAs) are not supposed to alter the message text,
except to add 'Received', 'Return-Path', and other header fields as
required by [SMTP-MTA].

However, SMTP is now also widely used as a message *submission*
protocol, that is, a means for message user agents (MUAs) to introduce
new messages into the MTA routing network.  The process which accepts
message submissions from MUAs is termed a Message Submission Agent
(MSA).

Messages being submitted are in some cases finished (complete)
messages, and in other cases are unfinished (incomplete) in some
aspect or other.  Unfinished messages need to be completed to ensure
they conform to [MESSAGE-FORMAT], and later requirements.  For
example, the message may lack a proper 'Date' header field, and
domains might not be fully qualified.  In some cases, the MUA may be
unable to generate finished messages (for example, it might not know
its time zone).  Even when submitted messages are complete, local site
policy may dictate that the message text be examined or modified in
some way.  Such completions or modifications have been shown to cause
harm when performed by downstream MTAs -- that is, MTAs after the
first-hop submission MTA -- and are in general considered to be
outside the province of standardized MTA functionality.

Separating messages into submissions and transfers allows developers
and network administrators to more easily:

o    Implement security policies and guard against unauthorized mail
     relaying or injection of unsolicited bulk mail

o    Implement authenticated submission, including off-site submission
     by authorized users such as travelers

o    Separate the relevant software code differences, thereby making
     each code base more straightforward and allowing for different
     programs for relay and submission

o    Detect configuration problems with a site's mail clients

o    Provide a basis for adding enhanced submission services in the
     future

2.3. What is SMTP?
------------------

RFC821/draft-ietf-drums-smtpupd

The objective of the Simple Mail Transfer Protocol (SMTP) is to
transfer mail reliably and efficiently.

SMTP is independent of the particular transmission subsystem and
requires only a reliable ordered data stream channel.

The SMTP design can be pictured as:

               +----------+                +----------+
   +------+    |          |                |          |
   | User |<-->|          |      SMTP      |          |
   +------+    |  Sender- |Commands/Replies| Receiver-|
   +------+    |   SMTP   |<-------------->|    SMTP  |    +------+
   | File |<-->|          |    and Mail    |          |<-->| File |
   |System|    |          |                |          |    |System|
   +------+    +----------+                +----------+    +------+
                SMTP client                SMTP server

When an SMTP client has a message to transmit, it establishes a
two-way transmission channel to an SMTP server. The role of an SMTP
client is to transfer mail messages to one or more SMTP servers, or
report its failure to do so.

An SMTP server may be either the ultimate destination or an
intermediate "relay" (that is, it may assume the role of an SMTP
client after receiving the message) or "gateway" (that is, it may
transport the message further using some protocol other than SMTP). 
SMTP commands are generated by the SMTP client and sent to the SMTP
server.  SMTP replies are sent from the SMTP server to the SMTP client
in response to the commands.

Once the transmission channel is established and initial handshaking
completed, the SMTP client normally initiates a mail transaction. Such
a transaction consists of a series of commands to specify the
originator and destination of the mail and transmission of the message
content (including any headers or other structure) itself. When the
same message is sent to multiple recipients, this protocol encourages
the transmission of only one copy of the data for all recipients at
the same destination (or intermediate relay) host.

The server responds to each command with a reply; replies may indicate
that the command was accepted, that additional commands are expected,
or that a temporary or permanent error condition exists. Commands
specifying the sender or recipients may include server-permitted SMTP
service extension requests as discussed in section 2.2.  The dialog is
purposely lock-step, one-at-a-time, although this can be modified by
mutually-agreed extension requests such as in [RFC-Pipeline].

Once a given mail message has been transmitted, the client may either
request that the connection be shut down or may initiate other mail
transactions. In addition, an SMTP client may use a connection to an
SMTP server for ancillary services such as verification of email
addresses or retrieval of mailing list subscriber addresses.

As suggested above, this protocol provides mechanisms for the
transmission of mail.  This transmission normally occurs directly from
the sending user's host to the receiving user's host when the two
hosts are connected to the same transport service.  When they are not
connected to the same transport service, transmission occurs via one
or more relay SMTP servers. An intermediate host that acts as either
an SMTP relay or as a gateway into some other transmission environment
is usually selected through the use of the domain name service (DNS)
Mail eXchanger mechanism.

2.4. RFC 822 Envelopes
----------------------

2.5. Internet Mail Routing
--------------------------

RFC1711; RFC974

3. Installing and Running Exim
==============================

Exim can be run on dozens of different types of computers and
operating systems, all of them Unix or Unix-like. Some systems come
with Exim binaries pre-installed, notably Debian GNU/Linux, but unless
you have one of these you will need to build and install Exim
yourself.

3.1. Basic Building
-------------------

3.1.1. Obtaining and Unpacking the Distribution
-----------------------------------------------

3.1.2. Running the Configuration Script
---------------------------------------

(This can also be done by hand, of course.)

3.2. Optional and Advanced Installation
---------------------------------------

3.2.1. External Packages: DBM, IPv6, tcpwrappers
------------------------------------------------

3.2.2. Building for Multiple Architectures
------------------------------------------

3.3. Installing Exim
--------------------

Various make commands

3.3.1. Users and Groups
-----------------------

3.3.2. Invoking Exim
--------------------

3.3.3. Exim Directories
-----------------------

3.4. Testing Exim
-----------------

4. The Exim Commandline
=======================

There are xx instances where Exim is called from the commandline:

o    1

o    2

o    3

o    4

Sendmail compatibility. Extra features.

4.1. Local Commands
-------------------

4.2. Daemon Commands
--------------------

4.3. Testing and Debugging Commands
-----------------------------------

4.4. Message Commands
---------------------

4.5. Miscellaneous
------------------

5. The Exim Utilities
=====================

Exim provides a lot of information about how it is running, what it is
doing and what should be done in the future. A number of handy
utilities come with the package which use this information. They are
useful whether you suspect something is wrong or you just like to know
how Exim is running. These utilities are a mixture of programs and
perl scripts, with the scripts being examples of how the programs can
be used to build custom analysis tools. ~TODO: CHECK STATEMENT~

The possible data sources for the utilities are:

o    spool files

o    hints database

o    other Exim databases of addresses or other items

o    log files

o    operating system process and load information

o    operating system network interface and protocol-level information

o    ~TODO: OTHER~

Most of these are used in the default utilities but there are many
other useful things that could be done with this information.

5.1. eximon - Graphical Viewing and Control
-------------------------------------------

5.2. Querying Exim
------------------

5.2.1. exiwhat
--------------

5.2.2. exigrep
--------------

5.2.3. exiqsumm
---------------

5.2.4. exinext
--------------

5.3. Database Tools
-------------------

5.3.1. exim_dbmbuild
--------------------

5.3.2. exim_dumpdb
------------------

5.3.3. exim_tidydb
------------------

5.3.4. exim_fixdb
-----------------

5.4. exicyclog
--------------

5.5. Constructing New Exim Tools
--------------------------------

6. Understanding Exim Configuration
===================================

Exim views the world as a stream of attempts to connect to its SMTP
port. Exim looks at every IP address that is trying to connect, and if
it is an allowable host then the connection is made and one or more
messages are accepted. Provided no errors occurred during the
acceptance phase the message is then placed in a holding area pending
delivery to either a local or remote destination. It is up to you, the
Exim administrator, to specify the decision logic that applies to a
message from acceptance through to delivery.

The configuration file allows you define your choice in customised
processing facilities. Although the sample configuration file in the
Exim distribution needs little modification to function for many sites
an administrator needs to understand its workings to be able to solve
problems. The concepts and terms introduced in this section are the
key to understanding Exim, and build on those in About Internet Mail
Systems.

6.1. "Hello World", Exim-style
------------------------------

Like all complex Unix server programs, Exim configuration is entirely
done from a text configuration file. There are some things you can
specify on the commandline and at compile time, but Exim cannot be
used without a configuration file.

You can usually call your configuration file anything you like (see
Installing and Running Exim) and in this book we have called it
/etc/exim.conf.  Special circumstances may dictate particular names
and conventions, and this is covered in Advanced Configuration
Examples.

The smallest useful exim.conf looks something like the following. For
the moment just enter it in (or download it from
http://www.guesswho.com/book/exim/examples.)

Minimalist configuration file

     #Main (global) configuration settings
     # There are no main settings in this example, but the following
     # 'end' must still appear
     
     end
     
     #Transport configuration
     local_delivery:
       driver = appendfile
       file = /var/spool/mail/${local_part}
     # You may need to alter the previous line depending on your system
     end
     
     #Director configuration
     localuser:
       driver = localuser
       transport = local_delivery
     
     end

Make sure the permissions are set so that only root can write to the
file, otherwise Exim will complain and refuse to start up. After
running exim -bd as root you should now have a system that is capable
of receiving mail on port 25 and delivering it to local unix users by
appending to the user's spool file in /var/spool/mail/username, where
username is replaced by the name of the recipient. This last piece of
magic is done by means of an Exim variable called $local_part. Exim
variables are discussed a little later. There are dozens of different
ways to receive mail but it is always best to start off simply adding
it to the end of a Unix inbox, where you can easily see what is going
on.

Incoming messages should be readable by any normal Unix mail client,
from pine to xmh. Be sure to address the mail to a user listed in the
/etc/passwd file and to fully qualify the machine name. If you have
made an error in the configuration file try using the command exim -bd
-d4, which should display the nature of the error and the line number
if applicable.

Exim is listening for mail on port 25, and a good debugging technique
is to connect to the port manually and pretend to be mail software
attempting to send a message. The SMTP protocol is very simple and
lends itself to manual operation.

A test run like this:

Testing the Exim configuration

     [dan@meg exim]$ telnet meg 25
     Trying 192.168.1.50...
     Connected to meg.shearer.org.
     Escape character is '^]'.
     220 meg.shearer.org ESMTP Exim 3.02 #2 Mon, 21 Jun 1999 18:15:09 +0930
     MAIL From:dan@nowhere.com.au
     250 <dan@nowhere.com.au> is syntactically correct
     RCPT To:dan@meg.shearer.org
     250 <dan@meg.shearer.org> is syntactically correct
     DATA
     354 Enter message, ending with "." on a line by itself
     Test
     .
     250 OK id=10vzi4-0002HI-00
     QUIT
     221 meg.shearer.org closing connection
     Connection closed by foreign host.
     [dan@meg exim]$

should result in the user dan on the machine meg receiving a new mail
message from dan@nowhere.com.au. A receive-only mailsystem is hardly
useful, but this example illustrates how easy it is to get started
with Exim. Despite this, there is a good deal more happening than
meets the eye! The defaults for Exim almost guarantee legal
RFC-compliant behaviour, something that many other MTAs do not. Don't
be fooled by Exim's apparent simplicity.

Before going on to turn this example into a real-world example we need
to examine in detail how the /etc/exim.conf file was put together.

6.2. Parts of the Configuration File
------------------------------------

There are six parts to the configuration file, and they must appear in
the correct order, separated by the keyword end. In most installations
usually at least five parts will be required. The first part is for
general or global settings. The last two parts of the configuration
file define rules for retrying failed messages and altering message
headers. This last set of rules (for rewriting headers) is the only
part sometimes omitted.

Parts two, three and four are the crucial ones, concerned with
defining the flow of messages through Exim. They contain short chunks
of logic called transports, directors and routers respectively
according to which section they are in. The definitions (or, as we
say, the transports, directors and routers) that you write give a list
of parameters that apply to a particular driver. A driver is a code
routine within Exim that has specific knowedge about a message-related
task, for example, writing to a local inbox. Exactly which drivers
exist depends on the choices made when building the Exim binary, see
Installing and Running Exim.

The following is quite detailed even though a great deal has been left
out. You need to be clear about the terms introduced since they are
the basis of the Exim model of email processing.  This is perhaps the
most difficult thing for a new Exim user to grasp, so be prepared to
revisit the next few pages several times!

o    Main configuration settings and global options

          As at version 3.02 there are currently 179 values that can
          be set in the main section which have global effects, for
          example local_domains = fooblatz.org will cause Exim to
          regard any address ending in fooblatz.org as being local
          under all circumstances. All of the values have defaults,
          which is why our first example was valid without any
          settings (local_domains defaults to the domain of the
          computer Exim is running on.) You can view all these values
          by running exim -bP at the commandline. Macros can also be
          defined in this section for replacing text throughout the
          rest of the configuration file. The end statement still
          needed to be present because main is mandatory, even if it
          has no contents.

o    Transports and their Drivers, for final message handling

          A transport defines a method of definitively handling a
          message, perhaps by contacting another computer or maybe
          copying a file. Conceptually, once Exim passes a message to
          a transport it can be forgotten, unless the transport
          reports an error. Usually a transport is for delivering a
          message somewhere - from a local user's mail box to an SMTP
          server the other side of the world. Other possibilities
          include generating an automatic reply, or even deleting the
          message altogether. A driver usually has settable options
          specific to itself such as file =
          /var/spool/mail/{$local_part} in the example. Omitting this
          section very rarely makes sense, since without it mail
          cannot be delivered anywhere.

o    Directors and their Drivers, for examining local messages

          A director handles messages where the domain of the
          destination address is one of those listed in the
          local_domains main configuration setting. The address is
          passed along the directors in the order they appear in the
          configuration file until one of them reports that it can
          handle the local part of the address. The director will
          usually take one of two actions:

          o    determine that a particular transport needs to be
               called, such as the user-defined transport
               local_delivery in the example, which uses the built-in
               localuser driver.

          o    generate one or more new addresses. No assumptions are
               made about any new addresses so each must be completely
               and independently processed

The localuser driver knows how to test whether a local part of an
address is recognised by the operating system, for example it might
appear in the /etc/password file on the local machine or some other
operating system-dependent place. In general directors tend to
interact closely with the local machine, often invoking programs
external to Exim (but local to the machine!)  or writing user-specific
files. The Directors section is required if you need any handling
ability for local domains. (An instance where no Directors are
required is when Exim is being used exclusively as a gateway between
remote domains.)

o    Routers and their Drivers, for examining remote messages

          A router handles messages with addresses where the
          destination domain is not listed in local_domains. As for
          directors, messages are passed to each defined router in
          turn until there is a match reported. The matching router
          then makes decisions about the message and decides what
          transport it needs to be sent to. In the example there were
          no routers defined, so Exim will not accept a message for
          processing if the To: address doesn't match the local
          machine's domain. This section would never normally be
          omitted, although it can be as we have seen in the example.

o    Retry Rules, for retrying failed deliveries

          A transport always attempts to deliver a message
          immediately.  If the delivery attempt fails for any reason
          Exim takes various actions including noting the time that
          the delivery should be tried again. The retry rules specify
          the way in which retry times should increase with an
          increasing number of failures. Since many errors are very
          short-term it often makes sense to try frequently at first,
          and then at longer and longer intervals until eventually the
          message is deleted and an error message returned to the
          sender. There are no default retry rules, so in the example
          Exim reports an error and deletes the message after the
          first failure - a ridiculous thing to do on the Internet.
          For this reason you should always have a retry section.

o    Address Rewriting Rules, for changing the contents of messages

          If rewriting rules exist, every single incoming message is
          checked for matching addresses. When there is a match, the
          headers in the message are changed according to the rules.
          This facility is typically used on mail originating from
          your area of jurisdiction but not for mail originating
          elsewhere. In principle rewriting is equivalent to opening
          an envelope and tampering with the contents, and so these
          rules should be used with very great care! However a common
          legitimate use is to make all email from a particular
          network conform to one addressing scheme, for example
          dan123@test.net.work and dan21@other.net.work could both be
          changed so that people outside net.work only ever see
          Dan.Shearer@net.work. It is a very good idea for beginners
          not implement rewriting rules on an important system since
          dreadful things can happen, including delivering mail to the
          wrong person. In the example there were no rewriting rules
          so no action was taken.

6.3. A Real-Life Example
------------------------

Knowing how the configuration file works we can now design a useful
setup. This is sufficient for a permanently connected Internet host to
send, receive and process mail to and from anywhere else on the
Internet in an RFC-compliant way. It has some extra features too,
including:

o    consulting lists of known rogue (spam) sites before accepting an
     SMTP connection

o    logging the fully qualified domain name of machines initiating
     connections

o    refusing to deliver mail to the root account by writing local
     files (this is a standard place to start when probing for Unix
     security holes.)

o    provision of a forwarding facility so users can redirect their
     own mail without consulting the Exim administrator

o    retrying of all failed deliveries for up to 4 days at
     sensibly-spaced intervals

All this with just 18 more executed lines in the file!

A useful configuration file

     # Main settings
     
     host_lookup = 0.0.0.0/0
     rbl_domains = rbl.maps.vix.com:dul.maps.vix.com:relays.orbs.org
     never_users = root
     
     end
     
     # Transports
     
     local_delivery:
       driver = appendfile
       file = /var/spool/mail/${local_part}
     
     remote_smtp:
       driver = smtp
     
     end
     
     # Directors
     
     local_message:
       driver = localuser
       transport = local_delivery
     
     local_forward:
       driver = forwardfile
       file = .forward
       check_ancestor
     
     end
     
     # Routers
     
     host_exist:
       driver = lookuphost
       transport = remote_smtp
     
     literal:
       driver = ipliteral
       transport = remote_smtp
     
     end
     
     # Retry Rules
     #
     # Domain               Error       Retries
     
     *                      *           F,2h,15m; G,16h,1h,1.5; F,4d,8h
     
     end
     
     

We will assume that this file is implemented on a computer called
antenne in the network bovver.test. Its fully qualified domain name
(FQDN) is therefore antenne.bovver.test, ('test' is currently a
non-existent top-level domain.) Let's follow the logical path through
this file as as messages are handled from acceptance to delivery.

The acceptance phase for a message differs depending on whether the
message is coming from a remote system (ie on the SMTP port) or the
local system (ie accepted by Exim on the commandline.)

Note: It is crucial to understand the difference between local and
remote messages. If you don't, refer to About Internet Mail Systems
now.

Let's look at a greatly simplified walk-through of how this
configuration file works.

6.3.1. Accepting a Remote Message
---------------------------------

A remote user called claude on the system pole.bovver.test sends a
message to the user jeanluc on antenne. The MTA on pole contacts port
25 on antenne. Before responding, Exim performs some checks:

o    Is this IP address in those recorded as bad in the Realtime
     Blocking Database? (this check is turned on with rbl_domains})

o    Has the DNS name corresponding to this IP address been explicitly
     forbidden by the Exim administrator? (this check is specified by
     host_lookup, where 0.0.0.0 means "all hosts" - see Options,
     Strings, Addresses and Expressions for what these numbers mean.)

There are other checks that can be configured. Provided that none of
these checks fail, Exim responds to pole and accepts the sender's
address via the SMTP MAIL command.

By default {CMD:MAIL}} will accept any address at all, so long as it
appears to be a properly-formed address, eg it has just one '@' symbol
and no strange characters. So jeremie@1.2.3.4, marc@spam and
sylvain@mars.mars.mars.mars are all valid senders. There is some
degree of checking that can be enabled but in general RFC-compliant
SMTP email is very weak on verifying senders. This is what allows
relatively easy forging of sending addresses used for nuisance email,
but it is also the key to the Internet's flexibility. The theoretical
alternative would be to introduce some kind of scheme to fully
authenticate every user who sends mail. This is utterly impossible
given the scale of the Internet and the range of software in use, so
the only sensible option is to accept any sender address that is both
syntactically valid and which passed some simple tests.

As soon as a valid sender has been given Exim accepts the destination
address with the SMTP RCPT command. The domain of the destination
address is checked to see if it is in the global option local_domains
(which is the default of antenne.bovver.test) or relay_domains
(defaulted to empty.)  If not, the message "relaying prohibited by
administrator" is sent. Any address with anything other than
@antenne.bovver.test on the right hand side of the local part will
therefore be rejected.

Once a message has a known sender and one or more acceptable
recipients, Exim accepts the data that forms the message. This usually
starts with a series of RFC822-compliant headers documenting the
previous history of the message, for example its passage through
pole's MTA, the date of original transmission and so on. Following
this is the body of the message to the user jeanluc. Exim adds some
headers to indicate when the message was received and then creates
spool files in the input directory (see Appendix: Exim File Formats
and Directories.) After logging that this has happened the acceptance
is complete and a delivery process is started for the new message.

Note: It is most important to realise that a delivery attempt is
always run as soon as a message is received. It is a feature of Exim
that only messages that have had a delivery failure are placed in a
holding area for later delivery.

6.3.2. Accepting a Local Message
--------------------------------

Imagine a local user (brigitte) on antenne uses the pine mail program
to send a message to another local user (jeanluc). The default for
pine is to do all deliveries via sendmail. Exim should always be
callable by using the name 'sendmail' (see Installing and Running
Exim) and just like sendmail, exim accepts email on standard input if
there are no commandline parameters. After a few basic checks Exim
accepts the message.

The global options host_lookup and rbl_domains are ignored since there
is no sending SMTP server with a commandline Exim delivery.  Exim
initiates a delivery process immediately for the message without any
checking and finishes the the acceptance with a log file entry. If no
brigitte existed Exim would attempt to return an error message to the
sender.

6.3.3. Delivery for Local Messages
----------------------------------

Messages remain in the spool directory until they have been completely
delivered to an address, the maximum retry time has been exceeded or
they are deleted by an administrator. A delivery attempt is
immediately run for every newly added message.

The delivery process first examines the destination address on the
message. If it is determined to be local (that is, the domain matches
an entry in local_domains) then the first defined director is
consulted. By consulted we mean that the address is presented to the
driver mentioned in the first director along with all relevant
parameters, and the driver responds to say whether the message matched
its capabilities or not.

The first director is called local_message and uses the localuser
driver. This driver has been programmed to perform lookups of local
users (usually by means of /etc/passwd) against the destination
address of the message and return true if there is a match. The
address jeanluc matches, and so local_message is allowed to continue
its job. No more directors will be consulted for this message. The
next line in local_message instructs Exim to put the message on the
queue of the local_delivery transport and begin processing the next
message.

There are many ways that a local delivery can be done, but the typical
method on small (and many large) Unix systems is to append the message
to a file owned by the user. This is called the user's inbox, and when
Exim writes to this file it also talks to the comsat daemon to trigger
the ubiquitous "you have new mail" notification. The local_delivery
transport here does its job by invoking the appendfile driver
specifying the required file with an Exim variable, as in the previous
example.

There are several tricky aspects to the idea of a local delivery
writing to a file in this manner, but for now we will assume that it
works without incident.

6.3.4. Delivery for Remote Messages
-----------------------------------

This time brigitte sends a message to claude@pole.bovver.test. The
message is received locally and put in the spool area. A queue-runner
process picks up the message and starts to run through the
configuration file's logic.

When Exim sees that an address is not local (in the example, anything
other than the machine antenne.bovver.test is remote) it consults the
routers rather than the directors. Each router is visited in turn in
the order they appear in until the driver referenced in one of them
returns success. The result of a router completing successfully is
that control is transferred to another router, a director or a
transport.

In this case the host_exist router is called, and the lookuphost
driver does a DNS lookup to see if the bovver.test domain has an IP
address registered as a preferred recipient of email. (The exact
mechanism is called an MX-record, see Advanced SMTP and DNS Issues.)
In this case does pole is registered, so the router continues. The
next line causes the message to be transferred to the remote_smtp
transport's queue, which opens an SMTP connection to pole and delivers
the message.

The literal driver exists for those sites that have an IP address but
not a DNS entry. With the lack of IPv4 address space these days the
reverse is far more likely, however, there are some sites still which
can only be accessed as frank@111.222.333.444. This is particularly
true in Eastern Europe. Besides this reason, the RFCs stipulate that
IP literal addresses be handled.

6.4. More on appendfile Local Deliveries
----------------------------------------

For Exim, mail delivery consists of either transporting mail between
MTAs, or transporting mail to some kind of endpoint. Common endpoints
are mailstores (such as an IMAP server, see Integrated Mail Systems)
or files. Not very long ago nearly all Internet mail ended up in files
on a Unix system, and this is still a vital part of mail delivery. The
appendfile transport takes care of writing messages to local files.

The difference between the appendfile transport and all others ~~TODO
even pipe?~~ is that writing to a local file is potentially very
risky. The Exim spool files are in a known place with known
privileges, but the files addressed by the appendfile transport are
often owned by unpriveliged users. They may have been changed to
symbolic links, named pipes or any other number of things. The reason
this can be root dangerous is because Exim runs as root. The full
implications are covered in Security - It's Not What You Think and
Transport, Director and Router Drivers but in essence if a user has
privilege to change a file that root writes to, a sufficiently clever
user can gain root privileges.

Accordingly, appendfile is never run as root (in fact no local
transports is run as root as a matter of Exim policy.) After this a
painstaking series of checks is done to ensure that the directory and
the path pointing to it exists, that it is an ordinary file, that its
owner and group correspond to the owner and group associated with the
message being delivered and many other things.

~~TODO A little bit more explaining how apppendfile doesn't
necessarily fit in with the New World Order.

6.5. More Rules for Transports, Drivers and Routers
---------------------------------------------------

~~~NOT DONE, JUST NOTES HERE

It can help to think of director and router names as questions. Their
names can be any legal alphanumeric string, but in the examples in
this book we have chosen names that pose the questions which are
answered. In the example if the question is "known_locally?" the
answer is given by the localuser driver. If the question is
"host_exist?" then the answer is given by the lookuphost driver. And
so on.

Drivers can do anything to a message, from just steering it through
the logic path to completely changing the addresses, causing it to be
resubmitted to the delivery process.

Drivers and codeblocks can have the same name, but this is confusing
and is a deprecated practice. Driver names are not case-sensitive but
codeblocks (possibly??) are.

A transport cannot call a router or director. A director can call
transports or directors. A router can call routers or directors or
transports but rarely a director. (Totally made up these last three.
Must test.)

Exim notices what a router does to a message. If the message has been
altered then it is resubmitted to the

The transports part defines transports for both local and remote
deliveries.

It is most important to understand that no T/D/R entry concerning a
driver defines that driver in any way. The definition of the driver is
buried in the Exim binary: the configuration file just supplies values
to modify its behaviour. So when a message is run past the localuser
driver, for example, there is no reference in the file to the fact
that the director does a lookup of the destination against a list of
local users.

Only one cb can be referred to by any one driver. A driver can refer
to a previously defined driver instead of a cb.

7. Options, Strings, Addresses and Expressions
==============================================

This chapter gives a precise definition of the valid ways of writing
Exim statements. It is important to learn how to construct your own
statements as opposed to just copying examples because email
processing needs vary so widely.

At the most general level the six parts of the Exim configuration file
contain statements in just three forms:

o    options in the main, transport, driver or router sections

o    retry rules

o    rewriting rules

In every section there are places where the following constructs can
appear:

o    expandable address specifiers

o    expandable strings

o    regular expressions

In many cases they can all be mixed together. Since these three
constructions are the basic building blocks of a configuration file
they are discussed first.

7.1. Address Specifiers
-----------------------

This book (and the Exim documentation) contains many references to
addresses and parts of addresses. An address consists of:

According to RFC822 an address looks like this:

7.2. Strings
------------

7.3. Regular Expressions
------------------------

7.4. Option Format
------------------

The Transport, Router and Director sections have identical syntax.
Every ...

7.5. Retry Rules
----------------

~TODO: PHILIPS NICE PICTURE~

7.6. Rewriting Rules
--------------------

8. Transport, Director and Router Drivers
=========================================

Exim comes supplied with a large range of drivers. It should rarely be
necessary to develop a new driver (if you think you do, see Appendix:
Hacking Exim for an example.) The pipe driver ...

This section doesn't contain all the options for the various drivers.
Refer to Appendix: Complete Guide to Key Values for a cross-references
to find the values applicable to a particular driver or situation.

8.1. Transport Drivers
----------------------

8.2. Director Drivers
---------------------

8.3. Router Drivers
-------------------

9. Rewriting Headers
====================

10. The Exim Filtering Language
===============================

11. Wedded bliss: Exim and Perl
===============================

11.1. What Does Embedded Perl Mean?
-----------------------------------

(Refer to Installing and Running Exim, don't repeat anything)

11.2. Example Code
------------------

11.3. Security
--------------

11.4. Current Limitations
-------------------------

12. Security - It's Not What You Think
======================================

Security is one of those things..

Meaning in terms of integrity as well as against bad people

Security also means avoiding invalid results for whatever reason

12.1. Unix MTA Security Issues
------------------------------

Writing to local files and (e)uid issues

12.2. General Code Issues
-------------------------

Overruns, memory overwriting

Failures in the middle of doing something

12.3. Networking Issues
-----------------------

Host OS's job really Source routing IP options

12.4. Mail Issues
-----------------

Forging, spamming, bombing, infiltrating

13. Advanced Configuration Examples
===================================

14. Advanced SMTP and DNS Issues
================================

The Simple Mail Transfer Protocol is indeed simple, so simple that it
has outlived hundreds of other more complicated replacements.

14.1. EHLO Extensions to SMTP
-----------------------------

RFC1651

Examples. Turning on and off in Exim. ~TODO: SUBHEADINGS FOR SOME
SAMPLE EXTENSIONS~

14.2. SMTP Synchronisation
--------------------------

RFC1047

General advice. What Exim does to minimise probs.

14.3. Spam, Spam and Antispam
-----------------------------

RFC2505

14.4. Batch SMTP
----------------

15. Integrated Mail Systems
===========================

15.1. Mailbox Management
------------------------

Exim is a mail transport agent, so there always needs to be somewhere
for the mail to be delivered to. In most large organisations defining
and managing mailboxes is frequently a knotty problem. In Universities
it is often worse! Ways of addressing the problem are:

o    a monolithic centralised system of mailboxes

o    a central cluster of mailboxes, with a central delivery point

o    passing all mail out to individual machines

o    another alternative

o    and another

~TODO: mention forwarding and other unavoidable complications.~

This only caters for inboxes. What about stored email, often gigabytes
per user?

IMAP. RFC1203 And the main others.

Implementations. www.imap.org. Cyrus, commercial. UW-vs-Cyrus comments
(esp. mailbox types mail UW look better.) Database formats. Issues.

15.2. LDAP Integration
----------------------

Nature of LDAP: very fast, read-only. (Compare the CDB, but that isn't
general.) Very easy to interface to eg for update via web browsers.

Traditional uses:

o    Lists of addresses

o    Aliases

o    Granular permissions

o    Integration with contacts database, internal directories etc

o    Authentication (integration with many other systems)

Others :

o    A

o    B

o    C

15.3. Other MTAs
----------------

Within an organisation Exim often has to talk to other MTAs, some of
them extremely limited. Exim can work with them as peers, or act as a
buffer for less capable or reliable systems.

15.4. MUAs
----------

Gateway vs local access for independent MUAs. MUAs tend to be much
sloppier. No security guarantee - quite the reverse!

15.5. MIME Attachments
----------------------

MIME attachments are ...

MTA interaction with not for the purists

Mutliple message optimisation --> big potential savings large
attachments ~TODO: Understand this more and refer to other chapter~

15.5.1. Blocking, Filtering and Checking Attachments
----------------------------------------------------

MUAs typically vulnerable (refer sloppy above) MTAs often asked to
shoulder the burden Size concerns

15.5.2. Future of Attachment Handling
-------------------------------------

Things that would be nice: caching, checksumming ietf draft reference,
etc.

16. Exim Troubleshooting Guide
==============================

This troubleshooting guide is divided into two sections. The first
concerns problems that stop Exim working and the second is about
making Exim do what you want.

16.1. Specific Installation Problems
------------------------------------

16.2. Common Bounce Error Messages
----------------------------------

16.3. Errors in Expansion, Rewriting and Regular Expressions
------------------------------------------------------------

16.4. Directing and Routing Configuration Errors
------------------------------------------------

16.5. DNS, SMTP, Database and Other Problems
--------------------------------------------

17. How Do I...?
================

17.1. First Item
----------------

17.2. Second Item
-----------------

18. Appendix: Glossary of Terms
===============================

SMTP terms: ESMTP, BSMTP, LMTP

ESMTP see SMTP

BSMTP see SMTP

LMTP see SMTP

Delivery define completely delivered

Exim delivery terms: Transport Director Router

Transport see Exim delivery terms

Director see Exim delievry terms

Router (in the context of email) see Exim delivery terms

Local message

Remote message

Local delivery

Remote delivery

X.400 terms: MTA/MS/MUA

MTA - see X.400 terms MS - ditto MUA - ditto

LDAP

ph - see LDAP

IMAP terms: IMAP, IMSP, ACAP

ACAP - see IMAP

IMSP - see IMAP

Local user

Remote user

DNS terms: MX, reverse-lookup

MX see DNS terms

Reverse-lookup see DNS terms

RBL

local_part

domain

19. Appendix: Complete Guide to Key Values
==========================================

~TODO: Should this be in globals.c order or in logical groupings?
Maybe in globals order but with a good cross-reference somewhere,
maybe even in this chapter?~

20. Appendix: Exim File Formats and Directories
===============================================

21. Appendix: Hacking Exim
==========================

Most of the time Exim provides every facility that a mailadministrator
could want. The ability to call arbitrary perl code creates a very
powerful argument for not extending Exim.  ...

o    don't do it unless you absolutely have to

o    if you do, contribute back

21.1. A Sample Driver
---------------------

21.2. Extended Character Sets in Illegal Places
-----------------------------------------------

(warning - non-RFC but useful anyway)

22. Appendix: Cited Request For Comments Documents
==================================================

The following IETF Request for Comments (RFC) documents are referred
to in the text. Many of these standards, proposed standards and
informative commentries are cross-referenced to other RFCs in their
headers. ~~~TODO: Ref to RFC repositries~~~

822 2505 1203

