File:  [NetBSD Developer Wiki] / wikisrc / guide / mail.mdwn
Revision 1.2: download - view: text, annotated - select for diffs
Sun Mar 10 10:11:00 2013 UTC (7 years, 6 months ago) by jdf
Branches: MAIN
CVS tags: HEAD
Cosmetics.

    1: # Mail and news
    2: 
    3: This chapter explains how to set up NetBSD to use mail and news. Only a simple 
    4: but very common setup is described: the configuration of a host connected to the 
    5: Internet with a modem through a provider. You can think of this chapter as the 
    6: continuation of
    7: [[Setting up TCP/IP on NetBSD in practice|guide/net-practice]], assuming a 
    8: similar network configuration. Even this *simple* setup proves to be difficult 
    9: if you don't know where to start or if you've only read introductory or 
   10: technical documentation. A general description of mail and news 
   11: configuration is beyond the scope of this guide; please read a good Unix 
   12: Administration book (some very good ones are listed on the NetBSD site).
   13: 
   14: This chapter also briefly describes the configuration (but not the usage) of two 
   15: popular applications, mutt for mail and tin for news. The usage is not described 
   16: because they are easy to use and well documented. Obviously, both mutt and tin 
   17: are not mandatory choices: many other similar applications exist but I think 
   18: that they are a good starting point because they are widely used, simple, work 
   19: well and don't use too much disk space and memory. Both are console mode 
   20: programs; if you prefer graphics applications there are also many choices for X.
   21: 
   22: In short, the programs required for the configuration described in this chapter 
   23: are:
   24: 
   25:  * postfix
   26:  * fetchmail
   27:  * mutt
   28:  * tin
   29: 
   30: Of these, only postfix is installed with the base system; you can install the 
   31: other programs from the NetBSD package collection, pkgsrc.
   32: 
   33: *Note*: Since NetBSD 4.0, postfix is the default MTA (Mail Transport Agent) and 
   34: sendmail was removed. Also, because sendmail is widely popular and several 
   35: programs like fetchmail are designed to be used with it, postfix includes a 
   36: command line wrapper that accepts sendmail's commands line syntax but works with 
   37: postfix. See 
   38: [sendmail(1)](http://netbsd.gw.com/cgi-bin/man-cgi?sendmail+1+NetBSD-5.0.1+i386) 
   39: for more details.
   40: 
   41: Before continuing, remember that none of the programs presented in this chapter 
   42: is mandatory: there are other applications performing similar tasks and many 
   43: users prefer them. You'll find different opinions reading the mailing lists. You 
   44: can also use different strategies for sending and receiving mail: the one 
   45: explained here is only a starting point; once you understand how it works you'll 
   46: probably want to modify it to suit your needs or to adopt a different method 
   47: altogether.
   48: 
   49: At the opposite extreme of the example presented here, there is the usage of an 
   50: application like Mozilla, which does everything and frees you from the need of 
   51: configuring many components: with Mozilla you can browse the Internet, send and 
   52: receive mail and read news. Besides, the setup is very simple. There is a price 
   53: to pay, though: Mozilla is a "closed" program that will not cooperate easily 
   54: with other standard Unix utilities.
   55: 
   56: Another possibility is to use emacs to read mail and news. Emacs needs no 
   57: introduction to Unix users but, in case you don't know, it is an extensible 
   58: editor (although calling emacs an editor is somewhat reductive) which becomes a 
   59: complete work environment, and can be used to read mail, news and to perform 
   60: many operations. For many people emacs is the only environment that they need 
   61: and they use it for all their work. The configuration of emacs for mail and news 
   62: is described in the emacs manual.
   63: 
   64: In the rest of this chapter we will deal with a host connected to the Internet 
   65: through a PPP connection via serial modem to a provider.
   66: 
   67:  * the local host's name is `ape` and the internal network is `insetti.net`, 
   68:    which means that the FQDN (Fully Qualified Domain Name) is `ape.insetti.net`.
   69:  * the user's login name on ape is `carlo`.
   70:  * the provider's name is BigNet.
   71:  * the provider's mail server is `mail.bignet.it`.
   72:  * the provider's news server is `news.bignet.it`.
   73:  * the user's (`carlo`) account at the provider is `alan` with the password 
   74:    `pZY9o`.
   75: 
   76: First some basic terminology:
   77: 
   78:  * *MUA (mail user agent)* -- a program to read and write mail. For example: 
   79:    mutt, elm and pine but also the simple mail application installed with the 
   80:    base system.
   81: 
   82:  * *MTA (mail transfer agent)* -- a program that transfers mail between two host 
   83:    but also locally (on the same host). The MTA decides the path that the mail 
   84:    will follow to get to the destination. On other BSD systems (but not only) 
   85:    the standard MTA is sendmail, other examples are qmail, exim and Microsoft 
   86:    Exchange.
   87: 
   88:  * *MDA (mail delivery agent)* -- a program, usually used by the MTA, that 
   89:    delivers the mail; for example, it physically puts the messages in the 
   90:    recipient's mailbox. For example, postfix uses one or more MDAs to deliver 
   91:    mail, and procmail is another well-known MDA.
   92: 
   93: The following figure depicts the mail system that we want to set up. Between the 
   94: local network (or the single host) and the provider there is a modem PPP 
   95: connection. The *bubbles* with the thick border (postfix, fetchmail, mutt) are 
   96: the programs launched manually by the user; the remaining bubbles are the 
   97: programs that are launched automatically. The circled numbers refer to the 
   98: logical steps of the mail cycle:
   99: 
  100: ![Structure of the mail system](/guide/images/mail1.gif)
  101: 
  102:  1. In step (1) mail is downloaded from the provider's POP server using 
  103:     fetchmail, which hands messages off to postfix's sendmail wrapper to put the 
  104:     messages in the user's mailbox.
  105: 
  106:  2. In step (2) the user launches mutt (or another MUA) to read mail, reply and 
  107:     write new messages.
  108: 
  109:  3. In step (3) the user *sends* the mail from within mutt. Messages are 
  110:     accumulated in the spool area.
  111: 
  112:  4. In step (4) the user calls postfix's sendmail wrapper to transfer the 
  113:     messages to the provider's SMTP server, that will deliver them to the final 
  114:     destination (possibly through other mail servers). The provider's SMTP server 
  115:     acts as a *relay* for our mail.
  116: 
  117: The connection with the provider must be up only during steps (1) and (4); for 
  118: the remaining steps it is not needed.
  119: 
  120: ## postfix
  121: 
  122: When an MTA must deliver a local message, it is delivered directly. If the 
  123: message is intended for a different domain, the MTA must find out the address of 
  124: the mail server for that domain. Postfix uses the DNS service (described in
  125: [[The Domain Name System|guide/dns]]) to find a mail exchanger handling mail for 
  126: the given domain, and delivers the message to that mail server then.
  127: 
  128: Postfix is controlled by a set of configuration files and databases, of which 
  129: `/etc/postfix/main.cf` and `/etc/postfix/master.cf` are the most important.
  130: 
  131: *Note*: Prior to version 1.5 of NetBSD, the mail configuration files were in 
  132: `/etc` instead of `/etc/mail`. Since NetBSD 4.0, the `/etc/mail` directory is 
  133: only used to store the local aliases and the corresponding 
  134: [postmap(1)](http://netbsd.gw.com/cgi-bin/man-cgi?postmap+1+NetBSD-5.0.1+i386) 
  135: database.
  136: 
  137: The first problem to be solved is that the local network we are dealing with is 
  138: an internal network, i.e. not directly accessible from the Internet. This means 
  139: that the names used internally have no meaning on the Internet; in short, 
  140: `ape.insetti.net` cannot be reached by an external host: no one will be able to 
  141: reply to a mail sent with this return address (many mail systems will even 
  142: reject the message as spam prevention as it comes from an unknown host). The 
  143: true address, the one visible from everybody, is assigned by the provider and, 
  144: therefore, it is necessary to convert the local address `carlo@ape.insetti.net` 
  145: to the real address `alan@bignet.it`. Postfix, if correctly configured, will 
  146: take care of this when it transfers the messages.
  147: 
  148: You'll probably also want to configure postfix in order to send the e-mails to 
  149: the provider's mail server, using it as a *relay*. In the configuration 
  150: described in this chapter, postfix does not directly contact the recipient's 
  151: mail server (as previously described) but relays all its mail to the provider's 
  152: mail server.
  153: 
  154: *Note*: The provider's mail server acts as a *relay*, which means that it 
  155: delivers mail which is not destined to its own domain, to another mail server. 
  156: It acts as an intermediary between two servers.
  157: 
  158: Since the connection with the provider is not always active, it is not necessary 
  159: to start postfix as a daemon in `/etc/rc.conf`: you can disable it with the line 
  160: `postfix=NO`. As a consequence it will be necessary to launch postfix manually 
  161: when you want to transfer mail to the provider. Local mail is delivered 
  162: correctly even if postfix is not active as a daemon.
  163: 
  164: Let's start configuring postfix.
  165: 
  166: ### Configuration of generic mapping
  167: 
  168: This type of configuration uses a new file `/etc/postfix/generic` which contains 
  169: the hostname mapping used by postfix to rewrite the internal hostnames.
  170: 
  171: The first step is therefore to write the mapping file:
  172: 
  173:     carlo@ape.insetti.net           alan@bignet.it
  174:     root@ape.insetti.net            alan@bignet.it
  175:     news@ape.insetti.net            alan@bignet.it
  176: 
  177: These entries will map the mail sent from the users given on the left side into 
  178: the globally valid email addresses given on the right, making it appear as if 
  179: the mail was really sent from that address.
  180: 
  181: For the sake of efficiency, `generic` must be transformed into a binary file 
  182: with the following command:
  183: 
  184:     # postmap /etc/postfix/generic
  185: 
  186: Now it's time to create the prototype configuration file which we'll use to 
  187: create the postfix configuration file.
  188: 
  189:     # vi /etc/postfix/main.cf
  190: 
  191: For the sake of simplicity, we'll only show the variables you need change:
  192: 
  193:     relayhost = mail.bignet.it
  194:     smtp_generic_maps = hash:/etc/postfix/generic
  195: 
  196: This configuration tells postfix to rewrite the addresses of type 
  197: `ape.insetti.net` using the real names found in the `/etc/postfix/generic` file. 
  198: It also says that mail should be sent to the `mail.bignet.it` server. The 
  199: meaning of the options is described in detail in 
  200: [postconf(5)](http://netbsd.gw.com/cgi-bin/man-cgi?postconf+5+NetBSD-5.0.1+i386).
  201: 
  202: The last step is to reload the configuration. You can do that easily with:
  203: 
  204:     # /etc/rc.d/postfix reload
  205:     postfix/postfix-script: refreshing the Postfix mail system
  206: 
  207: Now everything is ready to start sending mail.
  208: 
  209: ### Testing the configuration
  210: 
  211: Postfix is finally configured and ready to work, but before sending real mail it 
  212: is better to do some simple tests. First let's try sending a local e-mail with 
  213: the following command (postfix's sendmail wrapper):
  214: 
  215:     $ sendmail carlo
  216:     Subject: test
  217:     
  218:     Hello world
  219:     .
  220: 
  221: Please follow exactly the example above: leave a blank line after Subject: and 
  222: end the message with a line containing only one dot. Now you should be able to 
  223: read the message with your mail client and verify that the From: field has been 
  224: correctly rewritten.
  225: 
  226:     From: alan@bignet.it
  227: 
  228: ### Using an alternative MTA
  229: 
  230: Starting from version 1.4 of NetBSD sendmail is not called directly:
  231: 
  232:     $ ls -l /usr/sbin/sendmail
  233:     lrwxr-xr-x  1 root  wheel  21 Nov  1 01:14 /usr/sbin/sendmail@ -> /usr/sbin/mailwrapper
  234: 
  235: The purpose of mailwrapper is to allow the usage of an alternative MTA instead 
  236: of postfix (for example, sendmail). If you plan to use a different mailer I 
  237: suggest that you read the 
  238: [mailwrapper(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mailwrapper+8+NetBSD-5.0.1+i386) 
  239: and the 
  240: [mailer.conf(5)](http://netbsd.gw.com/cgi-bin/man-cgi?mailer.conf+5+NetBSD-5.0.1+i386) 
  241: manpages, which are very clear.
  242: 
  243: ## fetchmail
  244: 
  245: If someone sends me mail, it is received and stored by the provider, and not 
  246: automatically transferred to the local hosts; therefore it is necessary to 
  247: download it. Fetchmail is a very popular program that downloads mail from a 
  248: remote mail server (using e.g. the Post Office Protocol, POP) and forwards it to 
  249: the local system for delivery (usually using postfix's sendmail wrapper). It is 
  250: powerful yet easy to use and configure: after installation, the file 
  251: `~/.fetchmailrc` must be created and the program is ready to run 
  252: (`~/.fetchmailrc` contains a password so appropriate permissions on the file are 
  253: required).
  254: 
  255: This is an example `.fetchmailrc`:
  256: 
  257:     poll mail.bignet.it
  258:     protocol POP3
  259:     username alan there with password pZY9o is carlo here
  260:     flush
  261:     mda "/usr/sbin/sendmail -oem %T"
  262: 
  263: The last line (`mda ...`) is used only if postfix is not active as daemon on the 
  264: system. Please note that the POP-mail server indicated in this file 
  265: (mail.bignet.it) is only used to retrieve mails, and that it is not necessary 
  266: the same as the mail relay used by postfix to send out mails.
  267: 
  268: After setting up the `.fetchmailrc` file, the following command can be used to 
  269: download and deliver mail to the local system:
  270: 
  271:     $ fetchmail
  272: 
  273: The messages can now be read with mutt.
  274: 
  275: ## Reading and writing mail with mutt
  276: 
  277: Mutt is one of the most popular mail programs: it is *lightweight*, easy to use 
  278: and has lots of features. The man page mutt is very bare bones; the real 
  279: documentation is in `/usr/pkg/share/doc/mutt/`, in particular `manual.txt`.
  280: 
  281: Mutt's configuration is defined by the `~/.muttrc` file. The easiest way to 
  282: create it is to copy mutt's example muttrc file (usually 
  283: `/usr/pkg/share/examples/mutt/sample.muttrc`) to the home directory and modify 
  284: it. The following example shows how to achieve some results:
  285: 
  286:  * Save a copy of sent mail.
  287:  * Define a directory and two files for incoming and outgoing mail saved by mutt 
  288:    (in this example the directory is `~/Mail` and the files are `incoming` and 
  289:    `outgoing`).
  290:  * Define some colors.
  291:  * Define an alias.
  292: 
  293:     set copy=yes
  294:     set edit_headers
  295:     set folder="~/Mail"
  296:     unset force_name
  297:     set mbox="~/Mail/incoming"
  298:     set record="~/Mail/outgoing"
  299:     unset save_name
  300:     
  301:     bind pager <up> previous-page
  302:     bind pager <down> next-page
  303:     
  304:     color normal white black
  305:     color hdrdefault blue black
  306:     color indicator white blue
  307:     color markers red black
  308:     color quoted cyan black
  309:     color status white blue
  310:     color error red white
  311:     color underline yellow black
  312:     
  313:     mono quoted standout
  314:     mono hdrdefault underline
  315:     mono indicator underline
  316:     mono status bold
  317:     
  318:     alias pippo Pippo Verdi <pippo.verdi@pluto.net>
  319: 
  320: To start mutt:
  321: 
  322:     $ mutt
  323: 
  324: Please note that mutt supports color, but this depends on the terminal settings. 
  325: Under X you can use "xterm-color", for example:
  326: 
  327:     $ env TERM=xterm-color mutt
  328: 
  329: ## Strategy for receiving mail
  330: 
  331: This section describes a simple method for receiving and reading mail. The 
  332: connection to the provider is activated only for the time required to download 
  333: the messages; mail is then read offline.
  334: 
  335:  1. Activate the connection to the provider.
  336:  2. Run `fetchmail`.
  337:  3. Deactivate the connection.
  338:  4. Read mail with mutt.
  339: 
  340: ## Strategy for sending mail
  341: 
  342: When mail has been written and *sent* with mutt, the messages must be 
  343: transferred to the provider with postfix. Mail is sent from mutt with the `y` 
  344: command, but this does not really send it; the messages are enqueued in the 
  345: spool area; if postfix is not active as a daemon it is necessary to start it 
  346: manually or the messages will remain on the hard disk. The necessary steps are:
  347: 
  348:  1. Write mail with mutt, send it and exit mutt. You can check if and what 
  349:     messages are in the postfix mail queue using the 
  350:     [mailq(1)](http://netbsd.gw.com/cgi-bin/man-cgi?mailq+1+NetBSD-5.0.1+i386) 
  351:     program.
  352:  2. Activate the connection with the provider.
  353:  3. If your provider requires you to do "SMTP-after-POP", i.e. it first wants to 
  354:     make sure to know who you are before you are allowed to send mail (and no 
  355:     spam), you need to run `fetchmail` again first.
  356:  4. Write the command `/usr/sbin/postfix flush` to transfer the queued 
  357:     messages to the provider.
  358:  5. Deactivate the connection when the queue is empty.
  359: 
  360: ## Advanced mail tools
  361: 
  362: When you start using mail, you won't probably have very sophisticated 
  363: requirements and the already described standard configuration will satisfy all 
  364: your needs. But for many users the number of daily messages will increase with 
  365: time and a more rational organization of the mail storage will become necessary, 
  366: for example subdividing mail in different mail boxes organized by topic. If, for 
  367: example, you subscribe to a mailing list, you will likely receive many 
  368: messages every day and you will want to keep them separate from the rest of 
  369: your mail. You will soon find that you are spending too much time every day 
  370: repeating the same manual operations to organize your mail boxes.
  371: 
  372: Why repeat the same operations manually when you can have a program perform them 
  373: automatically for you? There are numerous tools that you can add to your mail 
  374: system to increase its flexibility and automatically process your messages. 
  375: Amongst the most known and used there are:
  376: 
  377:  * *procmail*, an advanced mail delivery agent and general purpose mail filter 
  378:    for local mail, which automatically processes incoming mail using user 
  379:    defined rulesets. It integrates smoothly with sendmail/postfix.
  380:  * *spamassassin* or *spamprobe*, to help fight spam.
  381:  * *metamail*, a tool to process attachments.
  382:  * *formail*, a mail formatter.
  383: 
  384: In the remaining part of this section a sample configuration for procmail will 
  385: be presented for a very common case: delivering automatically to a user defined 
  386: mailbox all the messages coming from a mailing list. The configuration of 
  387: postfix will be modified in order to call procmail directly (procmail will be 
  388: the *local mailer* used by sendmail). and a custom configuration file for 
  389: procmail will be created.
  390: 
  391: First, procmail must be installed using the package system (`mail/procmail`), 
  392: `pkg_add` or `pkgin`.
  393: 
  394: Next, the configuration of postfix must be changed, in order to use procmail as 
  395: local mailer:
  396: 
  397:     mailbox_command = /usr/pkg/bin/procmail
  398: 
  399: The line defines the path of the procmail program (you can see where procmail is 
  400: installed with the command `which procmail`).
  401: 
  402: The last step is the creation of the procmail configuration file, containing the 
  403: recipes for mail delivery.
  404: 
  405: Let's say that, for example, you subscribed to a mailing list on roses whose 
  406: address is `roses@flowers.org` and that every message from the list contains the 
  407: following line in the header:
  408: 
  409:     Delivered-To: roses@flowers.org
  410: 
  411: Assuming you want to automatically sort all mails going over that list into the 
  412: local mail folder `roses_list`, the procmail configuration file (`.procmailrc`) 
  413: looks like this:
  414: 
  415:     PATH=/bin:/usr/bin:/usr/pkg/bin
  416:     MAILDIR=$HOME/Mail
  417:     LOGFILE=$MAILDIR/from
  418:     
  419:     :0
  420:     * ^Delivered-To: roses@flowers.org
  421:     roses_list
  422: 
  423: The previous file contains only one rule, beginning with the line containing 
  424: `:0`. The following line identifies all messages containing the string 
  425: `Delivered-To: roses@flowers.org` and the last line says that the selected 
  426: messages must go to the `roses_list` mailbox (which you should have created in 
  427: $MAILDIR). The remaining messages will be delivered to the default mailbox. Note 
  428: that $MAILDIR is the same directory that you have configured with mutt:
  429: 
  430:     set folder="~/Mail"
  431: 
  432: Of course the mailing list is only an example; procmail is a very versatile tool 
  433: which can be used to filter mail based on many criteria. As usual, refer to the 
  434: man pages for more details: 
  435: [procmail(1)](http://netbsd.gw.com/cgi-bin/man-cgi?procmail+1+NetBSD-5.0.1+i386), 
  436: [procmailrc(5)](http://netbsd.gw.com/cgi-bin/man-cgi?procmailrc+5+NetBSD-5.0.1+i386), 
  437: and 
  438: [procmailex(5)](http://netbsd.gw.com/cgi-bin/man-cgi?procmailex+5+NetBSD-5.0.1+i386) 
  439: (this last one contains many examples of configuration files).
  440: 
  441: ## News with tin
  442: 
  443: The word *news* indicates the set of messages posted to the USENET newsgroups, a 
  444: service available on the Internet. Each newsgroup contains articles related to a 
  445: specific topic. Reading a newsgroup is different than reading a mailing list: 
  446: when you subscribe to a mailing list you receive the articles by mail and you 
  447: read them with a standard mail program like mutt, which you use also to send 
  448: replies. News, instead, are read directly from a news server with a dedicated 
  449: program called *newsreader* like, for example, tin. With tin you can subscribe 
  450: to the newsgroups that you're interested in and follow the *threads*. A thread 
  451: is a sequence of articles which all derive from an article that we could call 
  452: *original*. In short, a message is sent to the group, someone answers, other 
  453: people answer to those who answered in the first place and so on, creating a 
  454: tree like structure of messages and replies: without a newsreader it is 
  455: impossible to understand the correct sequence of messages.
  456: 
  457: After the installation of tin (from the package collection as usual) the only 
  458: thing left to do is to write the name of the NNTP server in the file 
  459: `/usr/pkg/etc/nntp/server`, which you may need to create first. For example:
  460: 
  461:     news.bignet.it
  462: 
  463: Once this has been done, the program can be started with the command `tin`. On 
  464: the screen something similar to the following example will be displayed:
  465: 
  466:     $ tin
  467:     Connecting to news.bignet.it...
  468:     news.bignet.it InterNetNews NNRP server INN 1.7.2 08-Dec-1997 ready (posting ok).
  469:     Reading groups from active file...
  470:     Checking for new groups...
  471:     Reading attributes file...
  472:     Reading newsgroups file...
  473:     Creating newsrc file...
  474:     Autosubscribing groups...
  475:     Reading newsrc file...
  476: 
  477: Be patient when you connect for the first time, because tin downloads an immense 
  478: list of newsgroups to which you can subscribe and this takes several minutes. 
  479: When the download is finished, the program's main screen is displayed; usually 
  480: no groups are displayed; to see the list of groups press `y`. To subscribe to a
  481: group, move on the group's name and press `y`.
  482: 
  483: Once that you have subscribed to some newsgroups you can start tin more quickly 
  484: with the command `tin -Q`. The search for new groups is disabled (`-q`), only 
  485: active groups are searched (`-n`) and newsgroup description are not loaded 
  486: (`-d`): it will not be possible to use the `y` (yank) command in tin. When tin 
  487: is started with this option it can't tell if a newsgroup is moderated or not.
  488: 
  489: Note that if you are connecting from an internal network (like in our example), 
  490: when you send ("post") a message the address at the beginning of the message 
  491: will be wrong (because it is the internal address). To solve the problem, use 
  492: the option `mail_address` in the tin configuration file (`~/.tin/tinrc`) or set 
  493: the `REPLYTO` environment variable.
  494: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb