mpop 1.0.5

Table of Contents


Next: , Up: (dir)

mpop

This manual was last updated November 6, 2006 for version 1.0.5 of mpop.

Copyright (C) 2005, 2006 Martin Lambers

This program, including this manual, is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program, including this manual, is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.


Next: , Previous: Top, Up: Top

1 Introduction

mpop is a POP3 client.

In its default mode of operation, it retrieves mails from one or more POP3 mailboxes, optionally does some filtering, and delivers them through a mail delivery agent (MDA) or to maildir folders or mbox files. Mails that were successfully delivered before will not be retrieved a second time, even if errors occur or mpop is terminated in the middle of a session.

The best way to start is probably to have a look at the Examples section. See Examples.

In addition to the mail retrieval mode, mpop can be used in server information mode. In this mode, mpop prints as much information as it can get about a given POP3 server (greeting, supported features, login delay, maximum mail size, ...).

Normally, a configuration file contains information about which POP3 server to use and how to use it, but almost all settings can also be configured on the command line.

POP3 server information is organized in accounts. Each account describes one POP3 server: host name, authentication settings, TLS settings, and so on. Each configuration file can define multiple accounts.

Supported features include:


Next: , Previous: Introduction, Up: Top

2 Configuration file

If it exists and is readable, a user configuration file will be loaded (~/.mpoprc by default). This file must have no more permissions than user read/write. Configuration file settings can be changed by command line options.

A configuration file is a simple text file. Empty lines and comment lines (whose first non-blank character is '#') are ignored. Every other line must contain a command and may contain an argument to that command. The argument may be enclosed in double quotes (").

If the first character of a filename is the tilde (~), this tilde will be replaced by $HOME.

If a command accepts the argument `on', it also accepts an empty argument and treats that as if it was `on'.

Commands form groups. Each group starts with the `account' command and defines the settings for one POP3 server.

See Examples.

2.1 General commands

`defaults'
Set defaults. The following configuration commands will set default values for all following account definitions.
`account name [ : account[,...] ]'
Start a new account definition with the given name. The current default values are filled in (see defaults).
If a colon and a list of previously defined accounts is given after the account name, the new account, with the filled in default values, will inherit all settings from the accounts in the list.
`host hostname'
The POP3 server to retrieve mails from. The argument may be a host name or a network address. Every account definition must contain this command.
`port number'
The port that the POP3 server listens on. The default is 110, unless TLS without STARTTLS is used, in which case it is 995.
`timeout (off|seconds)'
Set or unset a network timeout, in seconds. The argument `off' means that no timeout will be set, which means that the operating system default will be used.
`pipelining (on|off)'
Enable or disable POP3 pipelining for obsolete servers. Modern servers can tell mpop if they support pipelining, so this command is usually ignored. See Pipelining.

2.2 Authentication commands

See Authentication.

`auth [(on|method)]'
This command chooses the POP3 authentication method. With the argument `on', mpop will choose the best one available for you (see below). This is the default. Accepted methods are `user', `apop', `plain', `cram-md5', `digest-md5', `gssapi', `external', `login', and `ntlm'. See Authentication.

`user [username]'
Set your user name for POP3 authentication. An empty argument unsets the user name.
`password [secret]'
Set your password for POP3 authentication. An empty argument unsets the password. If no password is set but one is needed during authentication, mpop will try to find it in ~/.netrc, and if that fails, will prompt you for it. See Authentication.
`ntlmdomain [ntlmdomain]'
Set a domain for the `ntlm' authentication method. The default is to use no domain (equivalent to an empty argument), but some servers seem to require one, even if it is an arbitrary string.

2.3 TLS commands

See Transport Layer Security.

`tls [(on|off)]'
This command enables or disables TLS/SSL encrypted connections to the POP3 server. Not every server supports TLS, and many that do require the `tls_starttls off' command. It is recommended to also use the `tls_trust_file' command. See Transport Layer Security.
`tls_starttls [(on|off)]'
This command enables or disables the use of the STARTTLS POP3 command to start TLS encryption. It is enabled by default. See Transport Layer Security.
`tls_trust_file [file]'
This command activates strict server certificate verification. The given file must contain one or more certificates of trusted Certification Authorities (CAs) in PEM format. An empty argument disables this feature. See Transport Layer Security.
`tls_key_file [file]'
This command (together with the `tls_cert_file') command enables mpop to send a client certificate to the POP3 server if requested. The file must contain the private key of a certificate in PEM format. An empty argument disables this feature. See Transport Layer Security.
`tls_cert_file [file]'
This command (together with the `tls_key_file' command) enables mpop to send a client certificate to the POP3 server if requested. The file must contain a certificate in PEM format. An empty argument disables this feature. See Transport Layer Security.
`tls_certcheck [(on|off)]'
This command enables or disables sanity checks for the server certificate. These checks are enabled by default, but can cause difficulties in rare cases. See Transport Layer Security.
`tls_force_sslv3 [(on|off)]'
Force TLS/SSL version SSLv3. This might be needed to use SSL with some old and broken servers. Do not use this unless you have to. See Transport Layer Security.

2.4 Commands specific to mail retrieval mode

See Mail retrieval mode.

`delivery method method_arguments...'
How to deliver messages received from this account. If the delivery method needs to parse the mail headers for an envelope from address (the mda method if the command contains %F, and the mbox method), then it needs to create a temporary file to store the mail headers (but not the body) in. See $TMPDIR in Environment / Files.
`uidls_file filename'
The file to store UIDLs in. These are needed to identify new messages. %U in the filename will be replaced by the username of the current account. %H in the filename will be replaced by the hostname of the current account. If the filename contains directories that do not exist, mpop will create them. mpop locks this file for exclusive access when accessing the associated POP3 account.
The default value is ~/.mpop_uidls/%U_at_%H. You can also use a single UIDLS file for multiple accounts, but then you cannot poll more than one of these accounts at the same time.
`only_new [(on|off)]'
By default, mpop processes only new messages (new messages are those that were not already successfully retrieved in an earlier session). If this option is turned off, mpop will process all messages.
`keep [(on|off)]'
Keep all mails on the POP3 server, never delete them. The default behavior is to delete mails that have been successfully delivered or filtered by kill filters.
`killsize (off|size)'
Mails larger than the given size will be deleted, not downloaded (unless the keep command is used, in which case they will just be skipped). The size argument must be zero or greater. If it is followed by a 'k' or an 'm', the size is measured in kilobytes/megabytes instead of bytes. Note that some POP3 servers report slightly incorrect sizes for mails. See Filtering.
`skipsize (off|size)'
Mails larger than the given size will be skipped (not downloaded). The size argument must be zero or greater. If it is followed by a 'k' or an 'm', the size is measured in kilobytes/megabytes instead of bytes. Note that some POP3 servers report slightly incorrect sizes for mails. See Filtering.
`filter [COMMAND]'
Set a filter which will decide whether to retrieve, skip, or delete each mail by investigating the mail's headers. The POP3 server must support the POP3 TOP command for this to work; see Server information mode. An empty argument disables filtering.
All occurences of %F in the command will be replaced with the envelope from address of the current message (or MAILER-DAEMON if none is found). Note that this address is guaranteed to contain only letters a-z and A-Z, digits 0-9, and any of .@_-+/, even though that is only a subset of what is theoretically allowed in a mail address. Other characters, including those interpreted by the shell, are replaced with _. Nevertheless, you should put %F into single quotes: '%F'.
All occurences of %S in the command will be replaced with the size of the current mail as reported by the POP3 server.
The mail headers (plus the blank line separating the headers from the body) will be piped to the command. Based on the return code, mpop decides what to do with the mail: Return codes greater than or equal to 3 mean that an error occurred. The sysexits.h error codes may be used to give information about the kind of the error, but this is not necessary. See Filtering.


Next: , Previous: Configuration file, Up: Top

3 Invocation

3.1 Synopsis

3.2 Exit code

The standard exit codes from sysexits.h are used.

3.3 Environment / Files

`~/.mpoprc'
The default user configuration file.
`~/.mpop_uidls'
Default directory to store UIDLs files in.
`~/.netrc'
The .netrc file contains login information. If a password is not found in the configuration file, msmtp will search it in .netrc before prompting the user for it. The syntax of .netrc is described in the netrc(5) or ftp(1) manual page.
`$USER, $LOGNAME'
These variables override the user's login name. $LOGNAME is only used if $USER is unset. The user's login name is used for Received headers.
`$TMPDIR'
Directory to create temporary files in. If this is unset, a system specific default directory is used.

3.4 Options

Options override configuration file settings. The following options are accepted:

3.4.1 General options

`--version'
Print version information. This includes information about the library used for TLS/SSL support (if any), the library used for authentication, and the authentication mechanisms supported by this library.
`--help'
Print help.
`-P'
`--pretend'
Print the configuration settings that would be used, but do not take further action. An asterisk ('*') will be printed instead of the password.
`-d'
`--debug'
Print lots of debugging information, including the whole conversation with the POP3 server. Be careful with this option: the (potentially dangerous) output will not be sanitized, and your password may get printed in an easily decodable format!
This option implies `--quiet', because the debugging output would otherwise interfere with the normal output.

3.4.2 Changing the mode of operation

`-S'
`--serverinfo'
Print information about the POP3 server and exit. This includes information about supported features (authentication methods, TOP command, ...), about parameters (time for which mails will not be deleted, minimum time between logins, ...), and about the TLS certificate (if TLS is active). See Server information mode.

3.4.3 Configuration options

Most options in this category correspond to a configuration file command. Please refer to Configuration file for detailed information.

`-C filename'
`--file=filename'
Use the given file instead of ~/.mpoprc as the configuration file.
`--host=hostname'
Use this POP3 server with settings from the command line; do not use any configuration file data. This option disables loading of the configuration file. You cannot use both this option and account names on the command line.
`--port=number'
Set the port number to connect to. See port.
`--timeout=(off|seconds)'
Set or unset a network timeout, in seconds. See timeout.
`--pipelining=(on|off)'
Enable or disable POP3 pipelining for obsolete servers. See pipelining.
`--auth[=(on|method)]'
Set the authentication method to automatic (with "on") or manually choose an authentication method. See auth.
`--user=[username]'
Set or unset the user name for authentication. See user.
`--tls[=(on|off)]'
Enable or disable TLS. See tls.
`--tls-starttls[=(on|off)]'
Enable or disable STARTTLS for TLS encryption. See tls_starttls.
`--tls-trust-file=[file]'
Set or unset a trust file for TLS encryption. See tls_trust_file.
`--tls-key-file=[file]'
Set or unset a key file for TLS encryption. See tls_key_file.
`--tls-cert-file=[file]'
Set or unset a cert file for TLS encryption. See tls_cert_file.
`--tls-certcheck[=(on|off)]'
Enable or disable server certificate checks for TLS encryption. See tls_certcheck.
`--tls-force-sslv3=(on|off)]'
Force TLS/SSL version SSLv3. See tls_force_sslv3.

3.4.4 Options specific to mail retrieval mode

`-q'
`--quiet'
Do not print progress information.
`-a'
`--all-accounts'
Query all accounts in the configuration file.
`-A'
`--auth-only'
Authenticate only; do not retrieve mail. Useful for SMTP-after-POP.
`-s'
`--status-only'
Print number and size of mails in each account only; do not retrieve mail.
`-n'
`--only-new[=(on|off)]'
Process only new messages. See only_new.
`-k'
`--keep[=(on|off)]'
Do not delete mails from POP3 servers, regardless of other options or settings. See keep.
`--killsize=(off|size)'
Set or unset kill size. See killsize.
`--skipsize=(off|size)'
Set or unset skip size. See skipsize.
`--filter=[command]'
Set a filter which will decide whether to retrieve, skip, or delete each mail by investigating the mail's headers. See filter.
`--delivery=method,method_arguments...'
How to deliver messages received from this account. See delivery. Note that a comma is used instead of a blank to separate the method from its arguments.
`--uidls-file=filename'
File to store UIDLs in. See uidls_file.


Next: , Previous: Invocation, Up: Top

4 POP3 features


Next: , Up: POP3 features

4.1 Transport Layer Security

Quoting from RFC2246 - the TLS 1.0 protocol specification:
"The TLS protocol provides communications privacy over the Internet. The protocol allows client/server applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery."

POP3 servers can use TLS in one of two modes:

mpop can switch between these modes with the `tls_starttls' command (see tls_starttls) command or the `--tls-starttls' option (see –tls-starttls).

When TLS is started, the server sends a certificate to identify itself. This certificate contains information about the certificate owner, the certificate issuer, and the activation and expiration times of the certificate. This information can be displayed in server information mode. See Server information mode.

Some sanity checks are done with the server certificate. These include:

Sometimes one of these checks fail. mpop will abort the connection in this case. If the user still wants to use this POP3 server with TLS, the sanity checks can be switched off with `tls_certcheck' or `--tls-certcheck' (see tls_certcheck, –tls-certcheck).

Note that the POP3 server cannot be fully trusted just because the certificate passes the sanity checks. To verify that the user can trust the POP3 server, it is necessary to use a (list of) certificates of Certification Authorities (CAs) that are trusted. If mpop can verify that the server certificate was issued by one of these CAs, then the POP3 server is trusted. A file containing CA certificates can be set with `tls_trust_file' or `--tls-trust-file' (see tls_trust_file, –tls-trust-file).

If the server requests it, the client can send a certificate, too. This allows the server to verify the identity of the client. See the EXTERNAL mechanism in Authentication. The `tls_key_file'/`tls_cert_file' commands or the `--tls-key-file'/`--tls-cert-file' options can be used to set a client certificate. See tls_key_file/–tls-key-file, tls_cert_file/–tls-cert-file. Note that GnuTLS will only send a client certificate if it matches one of the CAs advertised by the server. If you set a client certificate but it is not send to the server, probably does not match any CA that the server trusts.


Next: , Previous: Transport Layer Security, Up: POP3 features

4.2 Authentication

POP3 servers require a client to authenticate itself before it is allowed to retrieve mail.

Multiple authentication methods exist. Most POP3 servers support only some of them. Some methods send authentication data in plain text (or nearly plain text) to the server. These methods should only be used when TLS is active to prevent others from stealing the password. See Transport Layer Security.

mpop supports the following authentication methods:

It depends on the underlying authentication library and its version whether a particular method is supported or not. Use --version to find out which methods are supported by your version of mpop.

Authentication data can be set with the `user' and `password' commands or with the `--user' option. See user, password, –user. If no password is set but one is needed during authentication, mpop will try to find it in ~/.netrc, and if that fails, mpop will prompt you for it.

The authentication method can be chosen with the `auth' command or `--auth' option, but it is usually sufficient to just use the `on' argument to let mpop choose the method itself. See auth, –auth.

If mpop chooses the method itself, it will not choose a method that sends plain text authentication data when TLS is not active. This means that only APOP, CRAM-MD5, DIGEST-MD5, GSSAPI, and NTLM are available when TLS is inactive. USER, PLAIN and LOGIN are only available when TLS is active. If you really want to send clear text authentication data, you have to force mpop to do that by setting the authentication method to USER, PLAIN or LOGIN while TLS is off.


Next: , Previous: Authentication, Up: POP3 features

4.3 Pipelining

A POP3 client that sends multiple POP3 commands at once to a POP3 server before starting to read the server's answers is using POP3 pipelining. Since the client does not have to wait for the server's answer before sending the next command, and the server does not have to wait for the next command from the client, pipelining can speed up a POP3 session substantially.

Pipelining in mpop works by sending up to `PIPELINE_MAX' commands to the server, then begin to read its answers, and refill the command pipeline when the number of unanswered commands drops to `PIPELINE_MIN'. `PIPELINE_MIN' and `PIPELINE_MAX' are compile time constants.

mpop can enable or disable POP3 pipelining automatically if the server supports the CAPA command; see Server information mode.

If your server does not support the CAPA command, but you still want to use pipelining, you have to enable it manually with the `pipelining' command or `--pipelining' option. See pipelining, –pipelining.


Previous: Pipelining, Up: POP3 features

4.4 Defective POP3 servers

Some POP3 servers still do not support the UIDL command. In this case, mpop cannot recognize messages that were already successfully retrieved, and will treat all messages as new. Use the `--serverinfo' option to find out if a server supports the UIDL command.

Some POP3 servers count end-of-line characters as two bytes (CRLF) instead of one (LF), so that the size of a mail as reported by the POP3 server is slightly larger than the actual size. This has the following consequences: The size filters are not accurate. Do not rely on exact size filtering. The progress output may display inaccurate (slightly too low) percentage values for the first mail retrieved from a POP3 server. mpop will detect this after the first mail has been read and will display corrected values for subsequent mails.


Next: , Previous: POP3 features, Up: Top

5 Mail retrieval mode

In this mode, mpop retrieves mail from one or more POP3 servers. It delivers each of them using the method that was given with the `delivery' command or `--delivery' option. See delivery, –delivery.

While retrieving the mail, mpop displays approximate progress information, which can be turned off with the `--quiet' option; see –quiet.

If the delivery succeeded, the mail is deleted from the POP3 server by default. The `keep' command and `--keep' option can prevent the deletion of mails; see keep, –keep.

Important note: Some POP3 servers will delete mails without any user interaction. See EXPIRE in Server information mode. mpop can do nothing about that.

If you don't want to download certain mails, but skip them or delete them directly, you can do filtering based on the mail headers. See Filtering.

If you just want to know if you have new mails (and how many), use the `--status-only' option. See –status-only.

If you just want to authenticate to the POP3 server, but don't want to look at your mails, use the `--auth-only' option. See –auth-only. This can be useful for sending mail through SMTP servers that require SMTP-after-POP (aka POP-before-SMTP).

Before mpop delivers a mail, it prepends a Received header to it. This is necessary if the delivery method transmits the mail to an SMTP server, for example. mpop does not change the contents of the mail in any other way.


Next: , Previous: Filtering, Up: Top

6 Server information mode

In server information mode, mpop prints as much information about the POP3 server as it can get and then exits.

The POP3 features that can be detected are:

If TLS is activated for server information mode, the following information will be printed about the POP3 server's TLS certificate (if available):


Next: , Previous: Mail retrieval mode, Up: Top

7 Filtering

There are three filtering commands available. They will be executed in the following order:

  1. `killsize'
  2. `skipsize'
  3. `filter'

If a filtering command applies to a mail, the remaining filters will not be executed.

The POP3 server must support the POP3 TOP command (Server information mode) for filtering with a filter command: It is used to read the mail headers (plus the blank line separating the header from the body) and pipe them to the filter command.

Note that, if the filter decides that the mail should be retrieved, the complete mail has to be downloaded, including the headers, so the headers will be downloaded twice. This is because there's no way in POP3 to download just the mail body. Sometimes this overhead surpasses the savings of the filtering.

The filter command looks at the mail headers and signals with its exit code what mpop should do with the mail:

Return codes greater than or equal to 3 mean that an error occurred. The sysexits.h error codes may be used to give information about the kind of the error, but this is optional.

Since the filter command will be passed to a shell, you can use all shell command constructs in addition to just calling a script or program. This allows flexible filter constructs. See Filtering with SpamAssassin.

Some POP3 servers count end-of-line characters as two bytes (CRLF) instead of one (LF), so that the size of a mail as reported by the POP3 server is slightly larger than the actual size. The filters use the size values reported by the POP3 server since they cannot know the actual size in advance. Thus you cannot rely on exact size filtering.


Next: , Previous: Server information mode, Up: Top

8 Examples


Next: , Up: Examples

8.1 A configuration file

     #
     # Default values for all accounts.
     #
     
     defaults
     # Activate TLS.
     tls on
     # Use the POP3-over-TLS variant instead of the STARTTLS variant.
     tls_starttls off
     # Use the procmail mail delivery agent.
     delivery mda "/usr/bin/procmail -f '%F' -d $USER"
     
     # For Sendmail:
     #delivery mda "/usr/sbin/sendmail -oi -oem -f '%F' -- $USER"
     # For msmtp (delivery via SMTP):
     #delivery mda "/usr/bin/msmtp --host=localhost --from='%F' -- $USER"
     # Delivery to a maildir folder:
     #delivery maildir ~/Mail/incoming
     # Delivery to a MBOX mail folder:
     #delivery mbox ~/Mail/new
     
     
     #
     # Two pop3 mailboxes at the provider.
     #
     
     account provider1
     host mx.provider.example
     user john_smith
     password secret
     
     # Copy the settings from the previous account, and only override the
     # settings that differ.
     account provider2 : provider1
     user joey
     password secret2
     
     
     #
     # A freemail service.
     #
     
     account freemail
     host pop.freemail.example
     user 1238476
     password pass
     
     # The service runs SpamAssassin, so test each mail for the "X-Spam-Status: Yes"
     # header, and delete all mails with this header before downloading them.
     filter   if [ "`grep "^X-Spam-Status: Yes"`" ]; then exit 1; else exit 0; fi
     
     
     #
     # Set a default account (optional).
     #
     
     account default : provider1


Previous: A configuration file, Up: Examples

8.2 Filtering with SpamAssassin

Use the following to delete all mails that SpamAssassin classifies as spam:

     filter "/path/to/spamc -c > /dev/null"

Since no message body is passed to SpamAssassin, you should disable all body-specific tests in the SpamAssassin configuration file; for example set use_bayes 0.

If your mail provider runs SpamAssassin for you, you just have to check for the result. The following script can do that when used as an mpop filter:

     #!/bin/sh
     if [ "`grep "^X-Spam-Status: Yes"`" ]; then
         exit 1  # kill this message
     else
         exit 0  # proceed normally
     fi

Since the filter command is passed to a shell, all shell constructs are usable, so you can also use this directly:

     filter if [ "`grep "^X-Spam-Status: Yes"`" ]; then exit 1; else exit 0; fi


Previous: Examples, Up: Top

9 Development

The homepage of mpop is http://mpop.sourceforge.net/; the SourceForge project page is at http://sourceforge.net/projects/mpop/.

The mailing lists mpop-users can be accessed from the project page.

Please send any questions, suggestions, and bug reports either to the mailing list or to Martin Lambers (marlam@marlam.de, OpenPGP key: http://www.marlam.de/key.txt). If you send a bug report, please include the output of mpop --version.