File:  [NetBSD Developer Wiki] / wikisrc / guide / mail.mdwn
Revision 1.4: download - view: text, annotated - select for diffs
Thu Mar 21 11:11:21 2013 UTC (13 months ago) by jdf
Branches: MAIN
CVS tags: HEAD
Add TOCs to all chapters.

**Contents**

[[!toc levels=3]]

# Mail and news

This chapter explains how to set up NetBSD to use mail and news. Only a simple
but very common setup is described: the configuration of a host connected to the
Internet with a modem through a provider. You can think of this chapter as the
continuation of
[[Setting up TCP/IP on NetBSD in practice|guide/net-practice]], assuming a
similar network configuration. Even this *simple* setup proves to be difficult
if you don't know where to start or if you've only read introductory or
technical documentation. A general description of mail and news
configuration is beyond the scope of this guide; please read a good Unix
Administration book (some very good ones are listed on the NetBSD site).

This chapter also briefly describes the configuration (but not the usage) of two
popular applications, mutt for mail and tin for news. The usage is not described
because they are easy to use and well documented. Obviously, both mutt and tin
are not mandatory choices: many other similar applications exist but I think
that they are a good starting point because they are widely used, simple, work
well and don't use too much disk space and memory. Both are console mode
programs; if you prefer graphics applications there are also many choices for X.

In short, the programs required for the configuration described in this chapter
are:

 * postfix
 * fetchmail
 * mutt
 * tin

Of these, only postfix is installed with the base system; you can install the
other programs from the NetBSD package collection, pkgsrc.

*Note*: Since NetBSD 4.0, postfix is the default MTA (Mail Transport Agent) and
sendmail was removed. Also, because sendmail is widely popular and several
programs like fetchmail are designed to be used with it, postfix includes a
command line wrapper that accepts sendmail's commands line syntax but works with
postfix. See
[sendmail(1)](http://netbsd.gw.com/cgi-bin/man-cgi?sendmail+1+NetBSD-5.0.1+i386)
for more details.

Before continuing, remember that none of the programs presented in this chapter
is mandatory: there are other applications performing similar tasks and many
users prefer them. You'll find different opinions reading the mailing lists. You
can also use different strategies for sending and receiving mail: the one
explained here is only a starting point; once you understand how it works you'll
probably want to modify it to suit your needs or to adopt a different method
altogether.

At the opposite extreme of the example presented here, there is the usage of an
application like Mozilla, which does everything and frees you from the need of
configuring many components: with Mozilla you can browse the Internet, send and
receive mail and read news. Besides, the setup is very simple. There is a price
to pay, though: Mozilla is a "closed" program that will not cooperate easily
with other standard Unix utilities.

Another possibility is to use emacs to read mail and news. Emacs needs no
introduction to Unix users but, in case you don't know, it is an extensible
editor (although calling emacs an editor is somewhat reductive) which becomes a
complete work environment, and can be used to read mail, news and to perform
many operations. For many people emacs is the only environment that they need
and they use it for all their work. The configuration of emacs for mail and news
is described in the emacs manual.

In the rest of this chapter we will deal with a host connected to the Internet
through a PPP connection via serial modem to a provider.

 * the local host's name is `ape` and the internal network is `insetti.net`,
   which means that the FQDN (Fully Qualified Domain Name) is `ape.insetti.net`.
 * the user's login name on ape is `carlo`.
 * the provider's name is BigNet.
 * the provider's mail server is `mail.bignet.it`.
 * the provider's news server is `news.bignet.it`.
 * the user's (`carlo`) account at the provider is `alan` with the password
   `pZY9o`.

First some basic terminology:

 * *MUA (mail user agent)* -- a program to read and write mail. For example:
   mutt, elm and pine but also the simple mail application installed with the
   base system.

 * *MTA (mail transfer agent)* -- a program that transfers mail between two host
   but also locally (on the same host). The MTA decides the path that the mail
   will follow to get to the destination. On other BSD systems (but not only)
   the standard MTA is sendmail, other examples are qmail, exim and Microsoft
   Exchange.

 * *MDA (mail delivery agent)* -- a program, usually used by the MTA, that
   delivers the mail; for example, it physically puts the messages in the
   recipient's mailbox. For example, postfix uses one or more MDAs to deliver
   mail, and procmail is another well-known MDA.

The following figure depicts the mail system that we want to set up. Between the
local network (or the single host) and the provider there is a modem PPP
connection. The *bubbles* with the thick border (postfix, fetchmail, mutt) are
the programs launched manually by the user; the remaining bubbles are the
programs that are launched automatically. The circled numbers refer to the
logical steps of the mail cycle:

![Structure of the mail system](/guide/images/mail1.gif)

 1. In step (1) mail is downloaded from the provider's POP server using
    fetchmail, which hands messages off to postfix's sendmail wrapper to put the
    messages in the user's mailbox.

 2. In step (2) the user launches mutt (or another MUA) to read mail, reply and
    write new messages.

 3. In step (3) the user *sends* the mail from within mutt. Messages are
    accumulated in the spool area.

 4. In step (4) the user calls postfix's sendmail wrapper to transfer the
    messages to the provider's SMTP server, that will deliver them to the final
    destination (possibly through other mail servers). The provider's SMTP server
    acts as a *relay* for our mail.

The connection with the provider must be up only during steps (1) and (4); for
the remaining steps it is not needed.

## postfix

When an MTA must deliver a local message, it is delivered directly. If the
message is intended for a different domain, the MTA must find out the address of
the mail server for that domain. Postfix uses the DNS service (described in
[[The Domain Name System|guide/dns]]) to find a mail exchanger handling mail for
the given domain, and delivers the message to that mail server then.

Postfix is controlled by a set of configuration files and databases, of which
`/etc/postfix/main.cf` and `/etc/postfix/master.cf` are the most important.

*Note*: Prior to version 1.5 of NetBSD, the mail configuration files were in
`/etc` instead of `/etc/mail`. Since NetBSD 4.0, the `/etc/mail` directory is
only used to store the local aliases and the corresponding
[postmap(1)](http://netbsd.gw.com/cgi-bin/man-cgi?postmap+1+NetBSD-5.0.1+i386)
database.

The first problem to be solved is that the local network we are dealing with is
an internal network, i.e. not directly accessible from the Internet. This means
that the names used internally have no meaning on the Internet; in short,
`ape.insetti.net` cannot be reached by an external host: no one will be able to
reply to a mail sent with this return address (many mail systems will even
reject the message as spam prevention as it comes from an unknown host). The
true address, the one visible from everybody, is assigned by the provider and,
therefore, it is necessary to convert the local address `carlo@ape.insetti.net`
to the real address `alan@bignet.it`. Postfix, if correctly configured, will
take care of this when it transfers the messages.

You'll probably also want to configure postfix in order to send the e-mails to
the provider's mail server, using it as a *relay*. In the configuration
described in this chapter, postfix does not directly contact the recipient's
mail server (as previously described) but relays all its mail to the provider's
mail server.

*Note*: The provider's mail server acts as a *relay*, which means that it
delivers mail which is not destined to its own domain, to another mail server.
It acts as an intermediary between two servers.

Since the connection with the provider is not always active, it is not necessary
to start postfix as a daemon in `/etc/rc.conf`: you can disable it with the line
`postfix=NO`. As a consequence it will be necessary to launch postfix manually
when you want to transfer mail to the provider. Local mail is delivered
correctly even if postfix is not active as a daemon.

Let's start configuring postfix.

### Configuration of generic mapping

This type of configuration uses a new file `/etc/postfix/generic` which contains
the hostname mapping used by postfix to rewrite the internal hostnames.

The first step is therefore to write the mapping file:

    carlo@ape.insetti.net           alan@bignet.it
    root@ape.insetti.net            alan@bignet.it
    news@ape.insetti.net            alan@bignet.it

These entries will map the mail sent from the users given on the left side into
the globally valid email addresses given on the right, making it appear as if
the mail was really sent from that address.

For the sake of efficiency, `generic` must be transformed into a binary file
with the following command:

    # postmap /etc/postfix/generic

Now it's time to create the prototype configuration file which we'll use to
create the postfix configuration file.

    # vi /etc/postfix/main.cf

For the sake of simplicity, we'll only show the variables you need change:

    relayhost = mail.bignet.it
    smtp_generic_maps = hash:/etc/postfix/generic

This configuration tells postfix to rewrite the addresses of type
`ape.insetti.net` using the real names found in the `/etc/postfix/generic` file.
It also says that mail should be sent to the `mail.bignet.it` server. The
meaning of the options is described in detail in
[postconf(5)](http://netbsd.gw.com/cgi-bin/man-cgi?postconf+5+NetBSD-5.0.1+i386).

The last step is to reload the configuration. You can do that easily with:

    # /etc/rc.d/postfix reload
    postfix/postfix-script: refreshing the Postfix mail system

Now everything is ready to start sending mail.

### Testing the configuration

Postfix is finally configured and ready to work, but before sending real mail it
is better to do some simple tests. First let's try sending a local e-mail with
the following command (postfix's sendmail wrapper):

    $ sendmail carlo
    Subject: test
    
    Hello world
    .

Please follow exactly the example above: leave a blank line after Subject: and
end the message with a line containing only one dot. Now you should be able to
read the message with your mail client and verify that the From: field has been
correctly rewritten.

    From: alan@bignet.it

### Using an alternative MTA

Starting from version 1.4 of NetBSD sendmail is not called directly:

    $ ls -l /usr/sbin/sendmail
    lrwxr-xr-x  1 root  wheel  21 Nov  1 01:14 /usr/sbin/sendmail@ -> /usr/sbin/mailwrapper

The purpose of mailwrapper is to allow the usage of an alternative MTA instead
of postfix (for example, sendmail). If you plan to use a different mailer I
suggest that you read the
[mailwrapper(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mailwrapper+8+NetBSD-5.0.1+i386)
and the
[mailer.conf(5)](http://netbsd.gw.com/cgi-bin/man-cgi?mailer.conf+5+NetBSD-5.0.1+i386)
manpages, which are very clear.

## fetchmail

If someone sends me mail, it is received and stored by the provider, and not
automatically transferred to the local hosts; therefore it is necessary to
download it. Fetchmail is a very popular program that downloads mail from a
remote mail server (using e.g. the Post Office Protocol, POP) and forwards it to
the local system for delivery (usually using postfix's sendmail wrapper). It is
powerful yet easy to use and configure: after installation, the file
`~/.fetchmailrc` must be created and the program is ready to run
(`~/.fetchmailrc` contains a password so appropriate permissions on the file are
required).

This is an example `.fetchmailrc`:

    poll mail.bignet.it
    protocol POP3
    username alan there with password pZY9o is carlo here
    flush
    mda "/usr/sbin/sendmail -oem %T"

The last line (`mda ...`) is used only if postfix is not active as daemon on the
system. Please note that the POP-mail server indicated in this file
(mail.bignet.it) is only used to retrieve mails, and that it is not necessary
the same as the mail relay used by postfix to send out mails.

After setting up the `.fetchmailrc` file, the following command can be used to
download and deliver mail to the local system:

    $ fetchmail

The messages can now be read with mutt.

## Reading and writing mail with mutt

Mutt is one of the most popular mail programs: it is *lightweight*, easy to use
and has lots of features. The man page mutt is very bare bones; the real
documentation is in `/usr/pkg/share/doc/mutt/`, in particular `manual.txt`.

Mutt's configuration is defined by the `~/.muttrc` file. The easiest way to
create it is to copy mutt's example muttrc file (usually
`/usr/pkg/share/examples/mutt/sample.muttrc`) to the home directory and modify
it. The following example shows how to achieve some results:

 * Save a copy of sent mail.
 * Define a directory and two files for incoming and outgoing mail saved by mutt
   (in this example the directory is `~/Mail` and the files are `incoming` and
   `outgoing`).
 * Define some colors.
 * Define an alias.

    set copy=yes
    set edit_headers
    set folder="~/Mail"
    unset force_name
    set mbox="~/Mail/incoming"
    set record="~/Mail/outgoing"
    unset save_name
    
    bind pager <up> previous-page
    bind pager <down> next-page
    
    color normal white black
    color hdrdefault blue black
    color indicator white blue
    color markers red black
    color quoted cyan black
    color status white blue
    color error red white
    color underline yellow black
    
    mono quoted standout
    mono hdrdefault underline
    mono indicator underline
    mono status bold
    
    alias pippo Pippo Verdi <pippo.verdi@pluto.net>

To start mutt:

    $ mutt

Please note that mutt supports color, but this depends on the terminal settings.
Under X you can use "xterm-color", for example:

    $ env TERM=xterm-color mutt

## Strategy for receiving mail

This section describes a simple method for receiving and reading mail. The
connection to the provider is activated only for the time required to download
the messages; mail is then read offline.

 1. Activate the connection to the provider.
 2. Run `fetchmail`.
 3. Deactivate the connection.
 4. Read mail with mutt.

## Strategy for sending mail

When mail has been written and *sent* with mutt, the messages must be
transferred to the provider with postfix. Mail is sent from mutt with the `y`
command, but this does not really send it; the messages are enqueued in the
spool area; if postfix is not active as a daemon it is necessary to start it
manually or the messages will remain on the hard disk. The necessary steps are:

 1. Write mail with mutt, send it and exit mutt. You can check if and what
    messages are in the postfix mail queue using the
    [mailq(1)](http://netbsd.gw.com/cgi-bin/man-cgi?mailq+1+NetBSD-5.0.1+i386)
    program.
 2. Activate the connection with the provider.
 3. If your provider requires you to do "SMTP-after-POP", i.e. it first wants to
    make sure to know who you are before you are allowed to send mail (and no
    spam), you need to run `fetchmail` again first.
 4. Write the command `/usr/sbin/postfix flush` to transfer the queued
    messages to the provider.
 5. Deactivate the connection when the queue is empty.

## Advanced mail tools

When you start using mail, you won't probably have very sophisticated
requirements and the already described standard configuration will satisfy all
your needs. But for many users the number of daily messages will increase with
time and a more rational organization of the mail storage will become necessary,
for example subdividing mail in different mail boxes organized by topic. If, for
example, you subscribe to a mailing list, you will likely receive many
messages every day and you will want to keep them separate from the rest of
your mail. You will soon find that you are spending too much time every day
repeating the same manual operations to organize your mail boxes.

Why repeat the same operations manually when you can have a program perform them
automatically for you? There are numerous tools that you can add to your mail
system to increase its flexibility and automatically process your messages.
Amongst the most known and used there are:

 * *procmail*, an advanced mail delivery agent and general purpose mail filter
   for local mail, which automatically processes incoming mail using user
   defined rulesets. It integrates smoothly with sendmail/postfix.
 * *spamassassin* or *spamprobe*, to help fight spam.
 * *metamail*, a tool to process attachments.
 * *formail*, a mail formatter.

In the remaining part of this section a sample configuration for procmail will
be presented for a very common case: delivering automatically to a user defined
mailbox all the messages coming from a mailing list. The configuration of
postfix will be modified in order to call procmail directly (procmail will be
the *local mailer* used by sendmail). and a custom configuration file for
procmail will be created.

First, procmail must be installed using the package system (`mail/procmail`),
`pkg_add` or `pkgin`.

Next, the configuration of postfix must be changed, in order to use procmail as
local mailer:

    mailbox_command = /usr/pkg/bin/procmail

The line defines the path of the procmail program (you can see where procmail is
installed with the command `which procmail`).

The last step is the creation of the procmail configuration file, containing the
recipes for mail delivery.

Let's say that, for example, you subscribed to a mailing list on roses whose
address is `roses@flowers.org` and that every message from the list contains the
following line in the header:

    Delivered-To: roses@flowers.org

Assuming you want to automatically sort all mails going over that list into the
local mail folder `roses_list`, the procmail configuration file (`.procmailrc`)
looks like this:

    PATH=/bin:/usr/bin:/usr/pkg/bin
    MAILDIR=$HOME/Mail
    LOGFILE=$MAILDIR/from
    
    :0
    * ^Delivered-To: roses@flowers.org
    roses_list

The previous file contains only one rule, beginning with the line containing
`:0`. The following line identifies all messages containing the string
`Delivered-To: roses@flowers.org` and the last line says that the selected
messages must go to the `roses_list` mailbox (which you should have created in
$MAILDIR). The remaining messages will be delivered to the default mailbox. Note
that $MAILDIR is the same directory that you have configured with mutt:

    set folder="~/Mail"

Of course the mailing list is only an example; procmail is a very versatile tool
which can be used to filter mail based on many criteria. As usual, refer to the
man pages for more details:
[procmail(1)](http://netbsd.gw.com/cgi-bin/man-cgi?procmail+1+NetBSD-5.0.1+i386),
[procmailrc(5)](http://netbsd.gw.com/cgi-bin/man-cgi?procmailrc+5+NetBSD-5.0.1+i386),
and
[procmailex(5)](http://netbsd.gw.com/cgi-bin/man-cgi?procmailex+5+NetBSD-5.0.1+i386)
(this last one contains many examples of configuration files).

## News with tin

The word *news* indicates the set of messages posted to the USENET newsgroups, a
service available on the Internet. Each newsgroup contains articles related to a
specific topic. Reading a newsgroup is different than reading a mailing list:
when you subscribe to a mailing list you receive the articles by mail and you
read them with a standard mail program like mutt, which you use also to send
replies. News, instead, are read directly from a news server with a dedicated
program called *newsreader* like, for example, tin. With tin you can subscribe
to the newsgroups that you're interested in and follow the *threads*. A thread
is a sequence of articles which all derive from an article that we could call
*original*. In short, a message is sent to the group, someone answers, other
people answer to those who answered in the first place and so on, creating a
tree like structure of messages and replies: without a newsreader it is
impossible to understand the correct sequence of messages.

After the installation of tin (from the package collection as usual) the only
thing left to do is to write the name of the NNTP server in the file
`/usr/pkg/etc/nntp/server`, which you may need to create first. For example:

    news.bignet.it

Once this has been done, the program can be started with the command `tin`. On
the screen something similar to the following example will be displayed:

    $ tin
    Connecting to news.bignet.it...
    news.bignet.it InterNetNews NNRP server INN 1.7.2 08-Dec-1997 ready (posting ok).
    Reading groups from active file...
    Checking for new groups...
    Reading attributes file...
    Reading newsgroups file...
    Creating newsrc file...
    Autosubscribing groups...
    Reading newsrc file...

Be patient when you connect for the first time, because tin downloads an immense
list of newsgroups to which you can subscribe and this takes several minutes.
When the download is finished, the program's main screen is displayed; usually
no groups are displayed; to see the list of groups press `y`. To subscribe to a
group, move on the group's name and press `y`.

Once that you have subscribed to some newsgroups you can start tin more quickly
with the command `tin -Q`. The search for new groups is disabled (`-q`), only
active groups are searched (`-n`) and newsgroup description are not loaded
(`-d`): it will not be possible to use the `y` (yank) command in tin. When tin
is started with this option it can't tell if a newsgroup is moderated or not.

Note that if you are connecting from an internal network (like in our example),
when you send ("post") a message the address at the beginning of the message
will be wrong (because it is the internal address). To solve the problem, use
the option `mail_address` in the tin configuration file (`~/.tin/tinrc`) or set
the `REPLYTO` environment variable.


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb