Exim Processes
The are four main process types in Exim
| Daemon process | listens for incoming SMTP connections and starts a reception process for each one. The daemon may also start queue runner processes. There is normally only one daemon process |
| Reception process | A non-SMTP reception process accepts one incoming message on its standard input, and stores the message on Exim's spool disk. An SMTP reception process can accept several messages from the same source, which can be another local process or TCP/IP connection |
| Queue Runner process | scans the pool of waiting messages, and starts a delivery process for each one in turn, waiting for each process to finish before starting the next one |
| Delivery process | performs one delivery attempt on a single message. This is where all the decisions about routing and retrying are taken. If the retry time for an address has not been reached, a delivery process skips that address. |
Most of Exims process are shorted lived, they perform one task and then exit. Exim process do interact with each other by referring to shared files, their contents affect the way the process operate, they fall into one of three categories
You can find the spooled directory by running the following command
| display spool directory | # exim -bP spool_directory |
Each message is held in two files with a unique identifier followed by -D and -H
Generally both the files are held in the same directory, if you use large volumes then the spool directory has many files, this can become a problem as the queue runner process has to scan this area, you can split the directory up in to many smaller directories (62 subdirectories) each containing a number of messages (the 6th character in the message ID determines the directory location - in the example above it would be f), this allows the queue runner process to search through small volume of messages
| splitting the spool directory | split_spool_directory = true |
During the delivery of a message, a third file man be created with a -J suffix, this is used to record each recipient address as soon as it is delivered. If the message has to be retained for subsequent delivery attempt at the end of a delivery run, the contents of the -J file are merged into the -H file and the -J file is deleted. There are two reasons for this journal file
There is a message log file which contains log entries that record the progress of the messages delivery. When Exim is working on a message it locks the -D file to prevent any other Exim process from trying to deliver the same message.
| turn off error messages regarding locking | skip_delivery = false |
Exim collects data about previous delivery problems, in order to change its behavior to changing circumstances, it remembers hosts that had problems connecting to before, so that it does not try them to often. It is not a critical operations that hints are written to the database but helps to improve performance.
The hints data is kept in a number of files in the db subdirectory of Exim's spool directory, the data is held in indexed DBM files. There are several different kinds of database
| retry database | holds information about temporary delivery failures, it records the type of error, the time of the first failure, the time of the last delivery attempt and the earliest time it is reasonable to try again |
| wait-<transport name> database | contains lists of messages that are awaiting delivery to specific hosts, after having failed at there first attempt. The data is keyed by hostname and IP address and consists of a list of message ID's |
| ratelimit database | used to hold data that is necessary for implementing the ACL ratelimit condition |
| misc database | this that two uses, firstly it it used to ensure that a client host cannot cause more than one queue runner process to run at once by issuing multiple ETRNS, secondly it is used to ensure that only one delivery process at a time connects to a remote host when using the serialize_hosts option. |
Depending on the type of DBM library the files have different suffixes, every delivery process consults the retry hint database, and after any SMTP deliveries , the wait-<transport> hints are checked.
Hints files are cleared out from time to time, but you may want to manually clear out the db, exim_tidydb is a utility that clears down the db, you should run this once a week.
| exim_tidydb utility | exim_tidydb /var/spool/exim retry ## clear down the retry hints database exim_tidydb /var/spool/exim wait-remote_smtp exim_tidydb /var/spool/exim callout exim_tidydb /var/spool/exim ratelimit exim_tidydb /var/spool/exim misc |
Exim does not have a central control process, but processes do interact with each other in various ways. The upper part of the figure deals with message reception and the bottom deals with delivery. The solid lines indicate data flows, the dashed ones show where one process creates another without passing any message data.
The daemon process perform two tasks
| starting a daemon process | # exim -bd -q15m Note: -bd = listen for incoming SMTP |
When the daemon starts it create a pid file, the location of the file is determined by the option pid_file_path, if no location is specified then the file will be created in the spool directory using the name exim-daemon.pid.
By default Exim will listen on all available interfaces, you can specify particular interfaces using the local_interfaces option. The standard port number of 25 is used, but this can be changes using the option daemon_smtp_ports or using the -oX option on the command line.
Two non-SMTP ports are in common use by MTA's
You can use the following to setup different interfaces and port numbers, this overrides the option daemon_smtp_ports.
| Using the same interface but a different port number | daemon_smtp_ports = 26 |
| Mixture of different interfaces and ports | local_interfaces = <; 192.168.0.1.25; 192.168.0.1.26; 192.168.0.2.25 Note: the first two are the same interface but different port numbers, but if you are using the same interface then you should use the option daemon_smtp_ports. |
when receiving SMTP connections, a number of checks are made which can be influenced by a number of options, these performance options can help out when systems are heavily loaded.
| smtp_accept_max | if the number of SMTP connections exceeds this limit then the connection is rejected. The default value is 20. |
| smtp_accept_max_per_host | If the number of SMTP connections with the same IP address exceeds this limit then the connection is rejected. The is no default limit. |
| smtp_accept_queue | If the SMTP connections exceeds its value, you can instead put them on the queue, they will remain there until picked up by a queue runner process, this means they are delivered serially instead of parallel, this keeps the load of the server down |
| smtp_accept_reserve smtp_reserve_hosts |
Out of the smtp_accept_max a number of connections are reserved and used by the hosts in smtp_reserver_hosts, this makes sure that your reserved hosts do get connections during load periods. |
| smtp_load_reserve | If the system load is above this one-minute load average (use uptime), then connections are only accepted from the hosts that are defined in smtp_reserve_hosts. |
The operating system maintains a queue of waiting connections, the length of this queue is specified by smtp_connect_backlog, if the option is exceeded then connections are rejected at the TCP/IP level, the default is normally about 20.
The reception processes are started by the daemon process, there are a number of options that affect the queue and thus reception processes
| smtp_accept_queue_per_connection | which if exceed no automatic delivery processes are started for the remainder, they are just put on the queue for the next queue runner to deal with. |
| queue_only | incoming messages are placed on the queue without an automatic delivery process being sent, deliveries then occur only via queue runner processes, or manual intervention |
| queue_only_load | When the system load is greater than the value set, incoming messages are queued without immediate delivery |
| queue_only_file | names an existing file, no immediate delivery takes place, this is used for dial-ups. |
The job of a queue runner process is to start delivery processes for all messages that are in the spool area, in other words a queue runner process works its way through the queue send messages one at a time until the queue is empty, by default it will skip frozen messages. We mentioned that split_spool_directory can be used to subdirectory the spool directory, this allows the queue runner process to only search through a shorter number of files in each directory. Remember the queue runner process does not perform the delivery, all it does is to arrange for delivery. There are two options to control the queue runner processes
| set starting interval | # exim -q15m ## start a queue runner process every 15 minutes # exim -q5m ## start a queue runner process every 5 minutes |
| Set the number of delivery processes | queue_run_max = 10 ## set the maximum queue runner process which in affects set the delivery ## processes |
Delivery processes are the most complex, all decisions about message routing and when to retry after temporary errors are taken by delivery processes. Each delivery process handles one delivery attempt for one message only, although there may be many recipients for the message and therefore many delivery actions.
You can manually delivery a message using the -M and supplying the message ID to the exim command.
| Manually send a message using exim | # exim -M 11w04d-00054F-03 Note: the -M option will also try and send frozen messages |
Before trying to deliver the message the system filter is run if one is configured, the system filter may add or remove header lines, modify its list of recipients, or cause a message to be frozen or all its recipients addresses to be failed.
After system filtering it determines what must be done for every recipient by running the routers for each address in turn. Once the routing is done, a delivery process run the appropriate transports in order to carry out the actual deliveries. Several parallel process can be configured to run using the remote_max_parallel option.
If any of the deliveries failed a bounce message is created and sent to the envelope sender address, it contains all the addresses that failed in the delivery run, if there is no senders address then the bounce message is not created and the failing message is left on the queue and is frozen, so that no further delivery is made.
There are a number of options that can be supplied to the exim command, these affect the message handling process
| -bd | Daemon process, listening for SMTP |
| -bs from inetd | SMTP reception process |
| -bs (not inetd) | Local SMTP reception process |
| -bS | Local batched SMTP reception process |
| -M | Forced delivery process for specific message |
| -q | Single queue runner process, starts delivery process |
| -q <time> | Daemon process, starts queue runner periodically (example: -q15m) |
| -bm, -t or none | Local reception process |