Annotation of wikisrc/guide/mail.mdwn, revision 1.4

1.4     ! jdf         1: **Contents**
        !             2: 
        !             3: [[!toc levels=3]]
        !             4: 
1.1       jdf         5: # Mail and news
1.3       jdf         7: This chapter explains how to set up NetBSD to use mail and news. Only a simple
                      8: but very common setup is described: the configuration of a host connected to the
                      9: Internet with a modem through a provider. You can think of this chapter as the
1.1       jdf        10: continuation of
1.3       jdf        11: [[Setting up TCP/IP on NetBSD in practice|guide/net-practice]], assuming a
                     12: similar network configuration. Even this *simple* setup proves to be difficult
                     13: if you don't know where to start or if you've only read introductory or
                     14: technical documentation. A general description of mail and news
                     15: configuration is beyond the scope of this guide; please read a good Unix
1.1       jdf        16: Administration book (some very good ones are listed on the NetBSD site).
1.3       jdf        18: This chapter also briefly describes the configuration (but not the usage) of two
                     19: popular applications, mutt for mail and tin for news. The usage is not described
                     20: because they are easy to use and well documented. Obviously, both mutt and tin
                     21: are not mandatory choices: many other similar applications exist but I think
                     22: that they are a good starting point because they are widely used, simple, work
                     23: well and don't use too much disk space and memory. Both are console mode
1.1       jdf        24: programs; if you prefer graphics applications there are also many choices for X.
1.3       jdf        26: In short, the programs required for the configuration described in this chapter
1.1       jdf        27: are:
                     29:  * postfix
                     30:  * fetchmail
                     31:  * mutt
                     32:  * tin
1.3       jdf        34: Of these, only postfix is installed with the base system; you can install the
1.1       jdf        35: other programs from the NetBSD package collection, pkgsrc.
1.3       jdf        37: *Note*: Since NetBSD 4.0, postfix is the default MTA (Mail Transport Agent) and
                     38: sendmail was removed. Also, because sendmail is widely popular and several
                     39: programs like fetchmail are designed to be used with it, postfix includes a
                     40: command line wrapper that accepts sendmail's commands line syntax but works with
                     41: postfix. See
                     42: [sendmail(1)](
1.1       jdf        43: for more details.
1.3       jdf        45: Before continuing, remember that none of the programs presented in this chapter
                     46: is mandatory: there are other applications performing similar tasks and many
                     47: users prefer them. You'll find different opinions reading the mailing lists. You
                     48: can also use different strategies for sending and receiving mail: the one
                     49: explained here is only a starting point; once you understand how it works you'll
                     50: probably want to modify it to suit your needs or to adopt a different method
1.1       jdf        51: altogether.
1.3       jdf        53: At the opposite extreme of the example presented here, there is the usage of an
                     54: application like Mozilla, which does everything and frees you from the need of
                     55: configuring many components: with Mozilla you can browse the Internet, send and
                     56: receive mail and read news. Besides, the setup is very simple. There is a price
                     57: to pay, though: Mozilla is a "closed" program that will not cooperate easily
1.1       jdf        58: with other standard Unix utilities.
1.3       jdf        60: Another possibility is to use emacs to read mail and news. Emacs needs no
                     61: introduction to Unix users but, in case you don't know, it is an extensible
                     62: editor (although calling emacs an editor is somewhat reductive) which becomes a
                     63: complete work environment, and can be used to read mail, news and to perform
                     64: many operations. For many people emacs is the only environment that they need
                     65: and they use it for all their work. The configuration of emacs for mail and news
1.1       jdf        66: is described in the emacs manual.
1.3       jdf        68: In the rest of this chapter we will deal with a host connected to the Internet
1.1       jdf        69: through a PPP connection via serial modem to a provider.
1.3       jdf        71:  * the local host's name is `ape` and the internal network is ``,
1.1       jdf        72:    which means that the FQDN (Fully Qualified Domain Name) is ``.
                     73:  * the user's login name on ape is `carlo`.
                     74:  * the provider's name is BigNet.
                     75:  * the provider's mail server is ``.
                     76:  * the provider's news server is ``.
1.3       jdf        77:  * the user's (`carlo`) account at the provider is `alan` with the password
1.1       jdf        78:    `pZY9o`.
                     80: First some basic terminology:
1.3       jdf        82:  * *MUA (mail user agent)* -- a program to read and write mail. For example:
                     83:    mutt, elm and pine but also the simple mail application installed with the
1.1       jdf        84:    base system.
1.3       jdf        86:  * *MTA (mail transfer agent)* -- a program that transfers mail between two host
                     87:    but also locally (on the same host). The MTA decides the path that the mail
                     88:    will follow to get to the destination. On other BSD systems (but not only)
                     89:    the standard MTA is sendmail, other examples are qmail, exim and Microsoft
1.1       jdf        90:    Exchange.
1.3       jdf        92:  * *MDA (mail delivery agent)* -- a program, usually used by the MTA, that
                     93:    delivers the mail; for example, it physically puts the messages in the
                     94:    recipient's mailbox. For example, postfix uses one or more MDAs to deliver
1.1       jdf        95:    mail, and procmail is another well-known MDA.
1.3       jdf        97: The following figure depicts the mail system that we want to set up. Between the
                     98: local network (or the single host) and the provider there is a modem PPP
                     99: connection. The *bubbles* with the thick border (postfix, fetchmail, mutt) are
                    100: the programs launched manually by the user; the remaining bubbles are the
                    101: programs that are launched automatically. The circled numbers refer to the
1.1       jdf       102: logical steps of the mail cycle:
                    104: ![Structure of the mail system](/guide/images/mail1.gif)
1.3       jdf       106:  1. In step (1) mail is downloaded from the provider's POP server using
                    107:     fetchmail, which hands messages off to postfix's sendmail wrapper to put the
1.1       jdf       108:     messages in the user's mailbox.
1.3       jdf       110:  2. In step (2) the user launches mutt (or another MUA) to read mail, reply and
1.1       jdf       111:     write new messages.
1.3       jdf       113:  3. In step (3) the user *sends* the mail from within mutt. Messages are
1.1       jdf       114:     accumulated in the spool area.
1.3       jdf       116:  4. In step (4) the user calls postfix's sendmail wrapper to transfer the
                    117:     messages to the provider's SMTP server, that will deliver them to the final
                    118:     destination (possibly through other mail servers). The provider's SMTP server
1.1       jdf       119:     acts as a *relay* for our mail.
1.3       jdf       121: The connection with the provider must be up only during steps (1) and (4); for
1.1       jdf       122: the remaining steps it is not needed.
                    124: ## postfix
1.3       jdf       126: When an MTA must deliver a local message, it is delivered directly. If the
                    127: message is intended for a different domain, the MTA must find out the address of
1.1       jdf       128: the mail server for that domain. Postfix uses the DNS service (described in
1.3       jdf       129: [[The Domain Name System|guide/dns]]) to find a mail exchanger handling mail for
1.1       jdf       130: the given domain, and delivers the message to that mail server then.
1.3       jdf       132: Postfix is controlled by a set of configuration files and databases, of which
1.1       jdf       133: `/etc/postfix/` and `/etc/postfix/` are the most important.
1.3       jdf       135: *Note*: Prior to version 1.5 of NetBSD, the mail configuration files were in
                    136: `/etc` instead of `/etc/mail`. Since NetBSD 4.0, the `/etc/mail` directory is
                    137: only used to store the local aliases and the corresponding
                    138: [postmap(1)](
1.1       jdf       139: database.
1.3       jdf       141: The first problem to be solved is that the local network we are dealing with is
                    142: an internal network, i.e. not directly accessible from the Internet. This means
                    143: that the names used internally have no meaning on the Internet; in short,
                    144: `` cannot be reached by an external host: no one will be able to
                    145: reply to a mail sent with this return address (many mail systems will even
                    146: reject the message as spam prevention as it comes from an unknown host). The
                    147: true address, the one visible from everybody, is assigned by the provider and,
                    148: therefore, it is necessary to convert the local address ``
                    149: to the real address ``. Postfix, if correctly configured, will
1.1       jdf       150: take care of this when it transfers the messages.
1.3       jdf       152: You'll probably also want to configure postfix in order to send the e-mails to
                    153: the provider's mail server, using it as a *relay*. In the configuration
                    154: described in this chapter, postfix does not directly contact the recipient's
                    155: mail server (as previously described) but relays all its mail to the provider's
1.1       jdf       156: mail server.
1.3       jdf       158: *Note*: The provider's mail server acts as a *relay*, which means that it
                    159: delivers mail which is not destined to its own domain, to another mail server.
1.1       jdf       160: It acts as an intermediary between two servers.
1.3       jdf       162: Since the connection with the provider is not always active, it is not necessary
                    163: to start postfix as a daemon in `/etc/rc.conf`: you can disable it with the line
                    164: `postfix=NO`. As a consequence it will be necessary to launch postfix manually
                    165: when you want to transfer mail to the provider. Local mail is delivered
1.1       jdf       166: correctly even if postfix is not active as a daemon.
                    168: Let's start configuring postfix.
                    170: ### Configuration of generic mapping
1.3       jdf       172: This type of configuration uses a new file `/etc/postfix/generic` which contains
1.1       jdf       173: the hostname mapping used by postfix to rewrite the internal hostnames.
                    175: The first step is therefore to write the mapping file:
1.3       jdf       181: These entries will map the mail sent from the users given on the left side into
                    182: the globally valid email addresses given on the right, making it appear as if
1.1       jdf       183: the mail was really sent from that address.
1.3       jdf       185: For the sake of efficiency, `generic` must be transformed into a binary file
1.1       jdf       186: with the following command:
                    188:     # postmap /etc/postfix/generic
1.3       jdf       190: Now it's time to create the prototype configuration file which we'll use to
1.1       jdf       191: create the postfix configuration file.
                    193:     # vi /etc/postfix/
                    195: For the sake of simplicity, we'll only show the variables you need change:
                    197:     relayhost =
                    198:     smtp_generic_maps = hash:/etc/postfix/generic
1.3       jdf       200: This configuration tells postfix to rewrite the addresses of type
                    201: `` using the real names found in the `/etc/postfix/generic` file.
                    202: It also says that mail should be sent to the `` server. The
                    203: meaning of the options is described in detail in
1.1       jdf       204: [postconf(5)](
                    206: The last step is to reload the configuration. You can do that easily with:
                    208:     # /etc/rc.d/postfix reload
                    209:     postfix/postfix-script: refreshing the Postfix mail system
                    211: Now everything is ready to start sending mail.
                    213: ### Testing the configuration
1.3       jdf       215: Postfix is finally configured and ready to work, but before sending real mail it
                    216: is better to do some simple tests. First let's try sending a local e-mail with
1.1       jdf       217: the following command (postfix's sendmail wrapper):
                    219:     $ sendmail carlo
                    220:     Subject: test
                    222:     Hello world
                    223:     .
1.3       jdf       225: Please follow exactly the example above: leave a blank line after Subject: and
                    226: end the message with a line containing only one dot. Now you should be able to
                    227: read the message with your mail client and verify that the From: field has been
1.1       jdf       228: correctly rewritten.
                    230:     From:
                    232: ### Using an alternative MTA
                    234: Starting from version 1.4 of NetBSD sendmail is not called directly:
                    236:     $ ls -l /usr/sbin/sendmail
                    237:     lrwxr-xr-x  1 root  wheel  21 Nov  1 01:14 /usr/sbin/sendmail@ -> /usr/sbin/mailwrapper
1.3       jdf       239: The purpose of mailwrapper is to allow the usage of an alternative MTA instead
                    240: of postfix (for example, sendmail). If you plan to use a different mailer I
                    241: suggest that you read the
                    242: [mailwrapper(8)](
                    243: and the
                    244: [mailer.conf(5)](
1.1       jdf       245: manpages, which are very clear.
                    247: ## fetchmail
1.3       jdf       249: If someone sends me mail, it is received and stored by the provider, and not
                    250: automatically transferred to the local hosts; therefore it is necessary to
                    251: download it. Fetchmail is a very popular program that downloads mail from a
                    252: remote mail server (using e.g. the Post Office Protocol, POP) and forwards it to
                    253: the local system for delivery (usually using postfix's sendmail wrapper). It is
                    254: powerful yet easy to use and configure: after installation, the file
                    255: `~/.fetchmailrc` must be created and the program is ready to run
                    256: (`~/.fetchmailrc` contains a password so appropriate permissions on the file are
1.1       jdf       257: required).
                    259: This is an example `.fetchmailrc`:
                    261:     poll
                    262:     protocol POP3
                    263:     username alan there with password pZY9o is carlo here
                    264:     flush
                    265:     mda "/usr/sbin/sendmail -oem %T"
1.3       jdf       267: The last line (`mda ...`) is used only if postfix is not active as daemon on the
                    268: system. Please note that the POP-mail server indicated in this file
                    269: ( is only used to retrieve mails, and that it is not necessary
1.1       jdf       270: the same as the mail relay used by postfix to send out mails.
1.3       jdf       272: After setting up the `.fetchmailrc` file, the following command can be used to
1.1       jdf       273: download and deliver mail to the local system:
                    275:     $ fetchmail
                    277: The messages can now be read with mutt.
                    279: ## Reading and writing mail with mutt
1.3       jdf       281: Mutt is one of the most popular mail programs: it is *lightweight*, easy to use
                    282: and has lots of features. The man page mutt is very bare bones; the real
1.1       jdf       283: documentation is in `/usr/pkg/share/doc/mutt/`, in particular `manual.txt`.
1.3       jdf       285: Mutt's configuration is defined by the `~/.muttrc` file. The easiest way to
                    286: create it is to copy mutt's example muttrc file (usually
                    287: `/usr/pkg/share/examples/mutt/sample.muttrc`) to the home directory and modify
1.1       jdf       288: it. The following example shows how to achieve some results:
                    290:  * Save a copy of sent mail.
1.3       jdf       291:  * Define a directory and two files for incoming and outgoing mail saved by mutt
                    292:    (in this example the directory is `~/Mail` and the files are `incoming` and
1.1       jdf       293:    `outgoing`).
                    294:  * Define some colors.
                    295:  * Define an alias.
                    297:     set copy=yes
                    298:     set edit_headers
                    299:     set folder="~/Mail"
                    300:     unset force_name
                    301:     set mbox="~/Mail/incoming"
                    302:     set record="~/Mail/outgoing"
                    303:     unset save_name
                    305:     bind pager <up> previous-page
                    306:     bind pager <down> next-page
                    308:     color normal white black
                    309:     color hdrdefault blue black
                    310:     color indicator white blue
                    311:     color markers red black
                    312:     color quoted cyan black
                    313:     color status white blue
                    314:     color error red white
                    315:     color underline yellow black
                    317:     mono quoted standout
                    318:     mono hdrdefault underline
                    319:     mono indicator underline
                    320:     mono status bold
                    322:     alias pippo Pippo Verdi <>
                    324: To start mutt:
                    326:     $ mutt
1.3       jdf       328: Please note that mutt supports color, but this depends on the terminal settings.
1.1       jdf       329: Under X you can use "xterm-color", for example:
                    331:     $ env TERM=xterm-color mutt
                    333: ## Strategy for receiving mail
1.3       jdf       335: This section describes a simple method for receiving and reading mail. The
                    336: connection to the provider is activated only for the time required to download
1.1       jdf       337: the messages; mail is then read offline.
                    339:  1. Activate the connection to the provider.
1.2       jdf       340:  2. Run `fetchmail`.
1.1       jdf       341:  3. Deactivate the connection.
                    342:  4. Read mail with mutt.
                    344: ## Strategy for sending mail
1.3       jdf       346: When mail has been written and *sent* with mutt, the messages must be
                    347: transferred to the provider with postfix. Mail is sent from mutt with the `y`
                    348: command, but this does not really send it; the messages are enqueued in the
                    349: spool area; if postfix is not active as a daemon it is necessary to start it
1.1       jdf       350: manually or the messages will remain on the hard disk. The necessary steps are:
1.3       jdf       352:  1. Write mail with mutt, send it and exit mutt. You can check if and what
                    353:     messages are in the postfix mail queue using the
                    354:     [mailq(1)](
1.1       jdf       355:     program.
                    356:  2. Activate the connection with the provider.
1.3       jdf       357:  3. If your provider requires you to do "SMTP-after-POP", i.e. it first wants to
                    358:     make sure to know who you are before you are allowed to send mail (and no
1.1       jdf       359:     spam), you need to run `fetchmail` again first.
1.3       jdf       360:  4. Write the command `/usr/sbin/postfix flush` to transfer the queued
1.1       jdf       361:     messages to the provider.
                    362:  5. Deactivate the connection when the queue is empty.
                    364: ## Advanced mail tools
1.3       jdf       366: When you start using mail, you won't probably have very sophisticated
                    367: requirements and the already described standard configuration will satisfy all
                    368: your needs. But for many users the number of daily messages will increase with
                    369: time and a more rational organization of the mail storage will become necessary,
                    370: for example subdividing mail in different mail boxes organized by topic. If, for
                    371: example, you subscribe to a mailing list, you will likely receive many
                    372: messages every day and you will want to keep them separate from the rest of
                    373: your mail. You will soon find that you are spending too much time every day
1.1       jdf       374: repeating the same manual operations to organize your mail boxes.
1.3       jdf       376: Why repeat the same operations manually when you can have a program perform them
                    377: automatically for you? There are numerous tools that you can add to your mail
                    378: system to increase its flexibility and automatically process your messages.
1.1       jdf       379: Amongst the most known and used there are:
1.3       jdf       381:  * *procmail*, an advanced mail delivery agent and general purpose mail filter
                    382:    for local mail, which automatically processes incoming mail using user
1.1       jdf       383:    defined rulesets. It integrates smoothly with sendmail/postfix.
                    384:  * *spamassassin* or *spamprobe*, to help fight spam.
                    385:  * *metamail*, a tool to process attachments.
                    386:  * *formail*, a mail formatter.
1.3       jdf       388: In the remaining part of this section a sample configuration for procmail will
                    389: be presented for a very common case: delivering automatically to a user defined
                    390: mailbox all the messages coming from a mailing list. The configuration of
                    391: postfix will be modified in order to call procmail directly (procmail will be
                    392: the *local mailer* used by sendmail). and a custom configuration file for
1.1       jdf       393: procmail will be created.
1.3       jdf       395: First, procmail must be installed using the package system (`mail/procmail`),
1.1       jdf       396: `pkg_add` or `pkgin`.
1.3       jdf       398: Next, the configuration of postfix must be changed, in order to use procmail as
1.1       jdf       399: local mailer:
                    401:     mailbox_command = /usr/pkg/bin/procmail
1.3       jdf       403: The line defines the path of the procmail program (you can see where procmail is
1.1       jdf       404: installed with the command `which procmail`).
1.3       jdf       406: The last step is the creation of the procmail configuration file, containing the
1.1       jdf       407: recipes for mail delivery.
1.3       jdf       409: Let's say that, for example, you subscribed to a mailing list on roses whose
                    410: address is `` and that every message from the list contains the
1.1       jdf       411: following line in the header:
                    413:     Delivered-To:
1.3       jdf       415: Assuming you want to automatically sort all mails going over that list into the
                    416: local mail folder `roses_list`, the procmail configuration file (`.procmailrc`)
1.1       jdf       417: looks like this:
                    419:     PATH=/bin:/usr/bin:/usr/pkg/bin
                    420:     MAILDIR=$HOME/Mail
                    421:     LOGFILE=$MAILDIR/from
                    423:     :0
                    424:     * ^Delivered-To:
                    425:     roses_list
1.3       jdf       427: The previous file contains only one rule, beginning with the line containing
                    428: `:0`. The following line identifies all messages containing the string
                    429: `Delivered-To:` and the last line says that the selected
                    430: messages must go to the `roses_list` mailbox (which you should have created in
                    431: $MAILDIR). The remaining messages will be delivered to the default mailbox. Note
1.1       jdf       432: that $MAILDIR is the same directory that you have configured with mutt:
                    434:     set folder="~/Mail"
1.3       jdf       436: Of course the mailing list is only an example; procmail is a very versatile tool
                    437: which can be used to filter mail based on many criteria. As usual, refer to the
                    438: man pages for more details:
                    439: [procmail(1)](,
                    440: [procmailrc(5)](,
                    441: and
                    442: [procmailex(5)](
1.1       jdf       443: (this last one contains many examples of configuration files).
                    445: ## News with tin
1.3       jdf       447: The word *news* indicates the set of messages posted to the USENET newsgroups, a
                    448: service available on the Internet. Each newsgroup contains articles related to a
                    449: specific topic. Reading a newsgroup is different than reading a mailing list:
                    450: when you subscribe to a mailing list you receive the articles by mail and you
                    451: read them with a standard mail program like mutt, which you use also to send
                    452: replies. News, instead, are read directly from a news server with a dedicated
                    453: program called *newsreader* like, for example, tin. With tin you can subscribe
                    454: to the newsgroups that you're interested in and follow the *threads*. A thread
                    455: is a sequence of articles which all derive from an article that we could call
                    456: *original*. In short, a message is sent to the group, someone answers, other
                    457: people answer to those who answered in the first place and so on, creating a
                    458: tree like structure of messages and replies: without a newsreader it is
1.1       jdf       459: impossible to understand the correct sequence of messages.
1.3       jdf       461: After the installation of tin (from the package collection as usual) the only
                    462: thing left to do is to write the name of the NNTP server in the file
1.1       jdf       463: `/usr/pkg/etc/nntp/server`, which you may need to create first. For example:
1.3       jdf       467: Once this has been done, the program can be started with the command `tin`. On
1.1       jdf       468: the screen something similar to the following example will be displayed:
                    470:     $ tin
                    471:     Connecting to
                    472: InterNetNews NNRP server INN 1.7.2 08-Dec-1997 ready (posting ok).
                    473:     Reading groups from active file...
                    474:     Checking for new groups...
                    475:     Reading attributes file...
                    476:     Reading newsgroups file...
                    477:     Creating newsrc file...
                    478:     Autosubscribing groups...
                    479:     Reading newsrc file...
1.3       jdf       481: Be patient when you connect for the first time, because tin downloads an immense
                    482: list of newsgroups to which you can subscribe and this takes several minutes.
                    483: When the download is finished, the program's main screen is displayed; usually
1.1       jdf       484: no groups are displayed; to see the list of groups press `y`. To subscribe to a
                    485: group, move on the group's name and press `y`.
1.3       jdf       487: Once that you have subscribed to some newsgroups you can start tin more quickly
                    488: with the command `tin -Q`. The search for new groups is disabled (`-q`), only
                    489: active groups are searched (`-n`) and newsgroup description are not loaded
                    490: (`-d`): it will not be possible to use the `y` (yank) command in tin. When tin
1.1       jdf       491: is started with this option it can't tell if a newsgroup is moderated or not.
1.3       jdf       493: Note that if you are connecting from an internal network (like in our example),
                    494: when you send ("post") a message the address at the beginning of the message
                    495: will be wrong (because it is the internal address). To solve the problem, use
                    496: the option `mail_address` in the tin configuration file (`~/.tin/tinrc`) or set
1.1       jdf       497: the `REPLYTO` environment variable.

CVSweb for NetBSD wikisrc <> software: FreeBSD-CVSweb