Setting up a secure SMTP server with AUTH and TLS enabled in Sendmail

While postfix is the basesystem's SMTP server, it is still possible to use the venerable Sendmail as your mail server of choice. Securing a sendmail SMTP gateway in order to use it from anywhere using your system's credentials is an easy task, here is how to achieve it.

Enabling Sendmail as the system's SMTP server

First thing is to disable postfix as the system's SMTP server. This action is controlled by the postfix parameter in /etc/rc.conf:

postfix=NO

We will then Install sendmail from pkgsrc with SASL for the authentication mechanism and TLS as the secure transport layer:

$ grep sendmail /etc/mk.conf
PKG_OPTIONS.sendmail=   tls sasl
ACCEPTABLE_LICENSES+=   sendmail-license

AUTH with SASL

Enabling SASL will build security/cyrus-sasl, but this package build failed with the following on my NetBSD 5.0.2 box:

db_ndbm.c:95: warning: passing argument 3 of 'utils->getcallback' from incompatible pointer type

So we will specify that cyrus-sasl should use berkeley as its database type:

$ grep SASL /home/bulk/etc/mk.conf
SASL_DBTYPE=            berkeley

We can now install sendmail with TLS and SASL support the classic way:

$ cd /usr/pkgsrc/mail/sendmail && sudo make install clean

cyrus-sasl package does now include any authentication plugin, it's up to us to pick one that will suit our needs. As we want to authenticate over system's login/password, we will use cy2-login:

$ cd /usr/pkgsrc/security/cy2-login && sudo make install

In order to use this method, we will have to install the saslauthd package. Saslauthd is in charge of plaintext authentications on behalf of the SASL library.

$ cd /usr/pkgsrc/security/cyrus-saslauthd && sudo make install clean

Of course, we want this daemon to start at every boot of this mail server:

# cp /usr/pkg/share/examples/rc.d/saslauthd /etc/rc.d
# echo "saslauthd=YES" >> /etc/rc.conf
# /etc/rc.d/saslauthd start

Now we have to inform the SASL library that it should use saslauthd whenever sendmail asks for an authentication:

# echo "pwcheck_method:saslauthd" > /usr/pkg/lib/sasl2/Sendmail.conf

Setting up the secure transport layer

As everything is in place for authentication, we will now prepare the TLS prerequisites. Instead of generating a self-signed certificate, I use to rely on CACert, "a community driven, Certificate Authority that issues certificates to the public at large for free." (from CACert.org).

In order to generate the certificate signing request (CSR), you can use the CSRGenerator script from CACert, which is really handy.

Once you have generated your server's private key with CSRGenerator and received your server certificate from CACert, simply copy them to /etc/mail/certs, along with CACert root certificate. Make sure your private key has strict permissions, sendmail will refuse to start if it is readable by everyone.

Configuring sendmail

It is now time to write our sendmail configuration. Create a mc file corresponding to your needs in /usr/pkg/share/sendmail/cf, for example:

# cat > /usr/pkg/share/sendmail/cf/korriban.mc << EOF
divert(0)dnl
VERSIONID(`Mustafar')
OSTYPE(bsd4.4)dnl
DOMAIN(generic)dnl

FEATURE(access_db, `hash -T<TMPF> /etc/mail/access')
FEATURE(blacklist_recipients)
FEATURE(mailertable, `hash -o /etc/mail/mailertable')
FEATURE(virtusertable, `hash -o /etc/mail/virtusertable')
FEATURE(genericstable, `hash -o /etc/mail/genericstable')
FEATURE(local_procmail)

dnl ### I use procmail as my MDA
define(`PROCMAIL_MAILER_PATH',`/usr/pkg/bin/procmail')
dnl ### and dspam as my antispam
define(`LOCAL_MAILER_PATH', `/usr/pkg/bin/dspam')
define(`LOCAL_MAILER_ARGS', `dspam -t -Y -a $h "--deliver=innocent" --user $u -d %u')

define(`confMAX_MESSAGE_SIZE', 5000000)

dnl ### here begins the secure SMTP gateway parameters
dnl ###
dnl ### enable SMTP AUTH with LOGIN mechanism
define(`confAUTH_MECHANISMS', `LOGIN')dnl
TRUST_AUTH_MECH(`LOGIN')dnl
dnl ### enable STARTTLS
define(`confCACERT_PATH',`/etc/mail/certs/')dnl
define(`confCACERT', `/etc/mail/certs/cacert.crt')
define(`confSERVER_CERT',`/etc/mail/certs/korriban_server.pem')dnl
define(`confSERVER_KEY',`/etc/mail/certs/korriban_privatekey.pem')dnl
dnl ### end of secure SMTP gateway parameters

MAILER(local)dnl
MAILER(smtp)dnl
MAILER(procmail)
EOF

Once your configuration is ready, build and install it using the following:

# make install-cf CF=korriban
rm -f korriban.cf
m4 ../m4/cf.m4 korriban.mc > korriban.cf || ( rm -f korriban.cf && exit 1 )
echo "### korriban.mc ###" >>korriban.cf
sed -e 's/^/# /' korriban.mc >>korriban.cf
chmod 444 korriban.cf
/usr/bin/install -c -o root -g wheel -m 0444 korriban.cf /etc/mail/sendmail.cf
/usr/bin/install -c -o root -g wheel -m 0444 korriban.cf /etc/mail/submit.cf

Now that sendmail is configured, fire it up by invoking:

# /etc/rc.d/sendmail start

And test that the features we've added are working:

# sendmail -d0.1 -bv root | grep SASL
    SASLv2 SCANF SOCKETMAP STARTTLS TCPWRAPPERS USERDB XDEBUG
$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 korriban.imil.net ESMTP Sendmail 8.14.5/8.14.5; Sat, 12 Nov 2011 16:43:40 +0100 (CET)
ehlo localhost
250-korriban.imil.net Hello localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE 5000000
250-DSN
250-ETRN
250-AUTH LOGIN
250-STARTTLS
250-DELIVERBY
250 HELP

There you go! now configure your MUA so it always tries TLS for sending mail, using the LOGIN authentication method.