Configuring qmail

Defining alias and control files

echo 3 > /var/qmail/control/spfbehavior
echo "| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox" > /var/qmail/control/defaultdelivery
echo 20 > /var/qmail/control/concurrencyincoming
echo postmaster@yourdomain.net > /var/qmail/control/bouncefrom
echo 20000000 > /var/qmail/control/databytes
echo yourdomain.net > /var/qmail/control/doublebouncehost
echo postmaster > /var/qmail/control/doublebounceto
echo 272800 > /var/qmail/control/queuelifetime
echo 30000000 > /var/qmail/control/softlimit
echo 100 > /var/qmail/control/maxrcpt

cd /usr/local/src/netqmail-1.06
./config-fast yourdomain.net

When you run ./config-fast it will automatically populate these files: defaultdomain, locals, me, plusdomain, rcpthosts.

  • defaultdomain when you have many domains on the same server (defined later in the virtualhost file) this is the default domain
  • locals domains that we deliver locally (qmail-send via qmail-lspawn program). Other domains are spawned by qmail-rspawn and delivered to other MTAs.
  • me the name of the server. This is the domain name that appers in the from field when you receive system messages, for instance
  • plusdomain domain substituted for trailing "+"
  • rcpthosts Domains that we accept mail for. Later you will see how simscan/chkuser reject incoming emails for non existing recipients.
  • spfbehavior concerns the spf patch.
  • softlimit sets soft resource limits for qmail-smtpd
  • databytes is the max number of bytes in message (0=no limit)
  • doublebounceto is the account which will receive double-bounce messages. If you’re using my combined patch, you can erase the first line of /var/qmail/control/doublebounceto to delete these unwanted messages before they’re injected into the local queue.

defaultdelivery is the default .qmail file. It tells qmail how to deliver the email. In this case Maildir is our choice. In case you didn't understand yet how delivery is done, please read at this point the relaying chapter of Life with qmail and expecially the README.vdelivermail that comes with vpopmail, which explains how the .qmail files are used.

You can find an exhaustive presentation of all control configuration file on Life with qmail book http://www.lifewithqmail.org/lwq.html#configuration

Setup the primary administrator's email address. This address will receive mail for root, postmaster, and mailer-daemon.  Replace "postmaster@yourdomain.net" with the administrator email address (postmaster):

cd /var/qmail/alias
echo "postmaster@yourdomain.net" > .qmail-postmaster
ln -s .qmail-postmaster .qmail-mailer-daemon
ln -s .qmail-postmaster .qmail-root
chmod 644 .qmail*

Setup the log dirs

The log dirs belong to qmaill.nofiles user and should not be accessible by other users

mkdir -p /var/log/qmail

cd /var/log/qmail
chown -R qmaill.nofiles .
chgrp root .
chmod -R og-wrx .
chmod g+rx .

Defining supervise scripts

References: tcpserver page

Download the startup scripts from here and untar

cd /var/qmail 
wget http://notes.sagredo.eu/sites/notes.sagredo.eu/files/qmail/supervise.tar.gz 
tar xzf supervise.tar.gz
rm supervise.tar.gz
chown -R root.root rc supervise

You can see the rc excutable, which is the qmail-start script, and the supervise folder:

-supervise
| 
|----qmail-smtpd/
|    |
|    |-----run
|    |-----log/
|          |
|          |---run
|
|----qmail-submission/
|    |
|    |-----run
|    |-----log/
|          |
|          |---run
|
|----qmail-send/
|    |
|    |-----run
|    |-----log/
|          |
|          |---run
|
|----vpopmaild/
|    |
|    |-----run
|    |-----log/
           |
           |---run

When you create symbolic links to a supervise directory in the /service dir, the run command will be executed at boot time when /command/svcscanboot is launched

cd /service
ln -s /var/qmail/supervise/qmail-smtpd
ln -s /var/qmail/supervise/qmail-send
ln -s /var/qmail/supervise/vpopmaild

And if you’re going to build an SMTP relay, you may want to run a separate SMTP instance for authentication on port 587:

ln -s /var/qmail/supervise/qmail-submission

File qmail/rc

#!/bin/sh

# Using stdout for logging
# Using control/defaultdelivery from qmail-local to deliver messages by default

# DKIM signign
#exec env - PATH="/var/qmail/bin:$PATH" \
#QMAILREMOTE=/var/qmail/bin/spawn-filter  \
#FILTERARGS=/var/qmail/bin/dk-filter \
#qmail-start "`cat /var/qmail/control/defaultdelivery`"

exec env - PATH="/var/qmail/bin:$PATH" \
qmail-start "`cat /var/qmail/control/defaultdelivery`"

File qmail/supervise/qmail-smtpd/run

#!/bin/sh

QMAILDUID=`id -u vpopmail`
NOFILESGID=`id -g vpopmail`
MAXSMTPD=`cat /var/qmail/control/concurrencyincoming`
SOFTLIMIT=`cat /var/qmail/control/softlimit`

# This enables greetdelay for qmail-smtpd. Put 0 if you decide to delay rblsmtpd instead.
export SMTPD_GREETDELAY=20
export DROP_PRE_GREET=1

# This enables greetdelay for rblsmtpd
#export GREETDELAY=15
# This disables rblsmtpd reject
#export RBLSMTPD=""

# This enables chkuser
export CHKUSER_START=ALWAYS

# DKIM - SURBL configuration
# DKIMQUEUE and SURBLQUEUE are front-ends of qmail-queue
#export SURBL=1                               # Comment out to enable SURBL filtering
#export QMAILQUEUE=/var/qmail/bin/surblqueue  # executes surblfilter
#export SURBLQUEUE=/var/qmail/bin/qmail-dkim  # executes qmail-dkim afer sublfilter
#export DKIMQUEUE=/var/qmail/bin/simscan      # simscan is executed after qmail-dkim
# DKIM verification. Use carefully 
#export DKIMVERIFY="FGHKLMNOQRTVW"
# This is to avoid verification of outgoing messages
#export RELAYCLIENT_NODKIMVERIFY=1

# This turns off TLS on port 25
export DISABLETLS="1"

# This enables simscan debug
#export SIMSCAN_DEBUG=2

exec /usr/local/bin/softlimit -m "$SOFTLIMIT" \
    /usr/local/bin/tcpserver -v -H -R -l 0 \
    -x /home/vpopmail/etc/tcp.smtp.cdb -c "$MAXSMTPD" \
    -u "$QMAILDUID" -g "$NOFILESGID" 0 25 \
    /var/qmail/bin/qmail-smtpd 2>&1

Note that the standard smtp (port 25) does not allow the authentication.

You have to adjust the resource limit (softlimit in bytes). Each system is different, and has different requirements. Life with qmail suggests just 2MB. You have to experiment the correct value increasing by steps of 1MB, especially once you have loaded spamassassin, clamAV and simscan (the mail scanner).

We'll cover GREETDELAY, RBL and DKIM later.

File qmail/supervise/qmail-smtpd/log/run

#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/smtpd

# comment out the following to save the "qlogenvelope" lines also in a separate file
# you must have the file /usr/local/bin/archive_qmail_qlog executable and working!
#LOGUSER="qmaill"
#LOGDIR="/var/log/qmail/smtpd"
#LOGDIRQLOG="/var/log/qmail/smtpd/qlog"
#n5 s10000000 '-*' '+* qlog*' !/usr/local/bin/archive_qmail_qlog $LOGDIRQLOG

If you like, you can have the qlogenvelope lines saved in a separate file as well:

@400000005855db3028811e24 qlogenvelope: result=accepted code=250 reason=rcptto detail=chkuser helo=smtp.senderdomain.com mailfrom=sender@senderdomain.com rcptto=user@rcptdomain.com relay=no rcpthosts=yes size= authuser= authtype= encrypted= sslverified=no localip=10.0.0.4 localport=25 remoteip=83.103.72.231 remoteport=43618 remotehost= qp= pid=11928
@400000005855db322a892324 qlogreceived: result=accepted code=250 reason=queueaccept detail= helo=smtp.senderdomain.com mailfrom=sender@senderdomain.com rcptto=user@rcptdomain.com relay=no rcpthosts= size=2689 authuser= authtype= encrypted= sslverified=no localip=10.0.0.4 localport=25 remoteip=83.103.72.231 remoteport=43618 remotehost= qp=11934 pid=11928

In this case create the archive_qmail_qlog

cat > /usr/local/bin/archive_qmail_qlog << __EOF__
#!/bin/sh
tai64nlocal >> /var/log/qmail/qmail-smtpd.log
__EOF__

and assign the +x flag

chmod +x /usr/local/bin/archive_qmail_qlog

File qmail/supervise/qmail-send/run

#!/bin/sh
exec /var/qmail/rc

File qmail/supervise/qmail-send/log/run

#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/send

File qmail/supervise/qmail-submission/run

This service makes the MTA also act as an outgoing relay, but the user must authenticate (with TLS encryption).

#!/bin/sh

QMAILDUID=`id -u vpopmail`
NOFILESGID=`id -g vpopmail`
MAXSMTPD=`cat /var/qmail/control/concurrencyincoming`
SOFTLIMIT=`cat /var/qmail/control/softlimit`

# You MUST export this, otherwise you'd get a 30 sec timeout
# "!" if you want the submission feature (auth required)
export SMTPAUTH="!"

# This enables greetdelay for qmail-smtpd.
# comment out if using fail2ban on port 587
export SMTPD_GREETDELAY=5
export DROP_PRE_GREET=1

# This enables chkuser
export CHKUSER_START=ALWAYS

# This enables simscan debug
#export SIMSCAN_DEBUG=2

exec /usr/local/bin/softlimit -m "$SOFTLIMIT" \
    /usr/local/bin/tcpserver -v -H -R -l 0 \
    -x /home/vpopmail/etc/tcp.submission.cdb -c "$MAXSMTPD" \
    -u "$QMAILDUID" -g "$NOFILESGID" 0 587 \
    /var/qmail/bin/qmail-smtpd \
    /home/vpopmail/bin/vchkpw /bin/true 2>&1

Note the use of vchkpw in conjunction with qmail-smtpd to ensure authentication. The connection requires TLS enabled. This is the reason why we opened a separate secure connection on port 587 to allow remote clients to use our MTA as a relay.

The variable SMTPAUTH is related to the auth patch. You are invited to take a look to the README.auth file for further details.

File qmail/supervise/qmail-submission/log/run

#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/submission

File qmail/supervise/vpopmaild/run

#!/bin/sh
QMAILDUID=`id -u root`
NOFILESGID=`id -g root`

exec /usr/local/bin/softlimit -m 25000000 \
    /usr/local/bin/tcpserver -v -H -R -l 0 \
    -u "$QMAILDUID" -g "$NOFILESGID" 0 89 \
    /home/vpopmail/bin/vpopmaild 2>&1

vpopmaild is important when connecting to vpopmail via webmail to change the password, for instance.

File qmail/supervise/vpopmaild/log/run

#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t /var/log/qmail/vpopmaild

cronjobs

To backup the log files from the qmail services we'll use the convert-multilog script from John Simpson (thanks to the author and to Marc for the suggestion) who describes the script function as follows:

convert-multilog is a script which searches "/service/*/log/main" for any "@4*" files (the automatic cut-off files generated by multilog), converts their timestamps from tai64n to human-readable format, and writes them to /var/log/{service}.{date}. Once the lines from a given "@4*" file have been converted, the file is deleted.

The log files are saved separated by date, for example

-rw-r--r-- 1 root   root   259558 Aug 24 12:21 qmail-smtpd.2014-08-20
-rw-r--r-- 1 root   root   806917 Aug 24 12:21 qmail-smtpd.2014-08-21
-rw-r--r-- 1 root   root  1523116 Aug 24 12:21 qmail-smtpd.2014-08-22
-rw-r--r-- 1 root   root   364022 Aug 24 12:21 qmail-smtpd.2014-08-23

Inside each file the date is now human-readable

2014-08-23 00:31:49.503947500 tcpserver: status: 1/20

This is very useful when you have to do quick searches.

Install like this:

mkdir -p /var/log/qmail/backup
cd /usr/local/bin
wget http://notes.sagredo.eu/sites/notes.sagredo.eu/files/qmail/convert-multilog
chmod +x convert-multilog

cd /service/qmail-send/log/
ln -s /var/log/qmail/send main
cd /service/qmail-smtpd/log/
ln -s /var/log/qmail/smtpd main
cd /service/qmail-submission/log/
ln -s /var/log/qmail/submission main
cd /service/vpopmaild/log/
ln -s /var/log/qmail/vpopmaild main

Now set up a cronjob once a day (crontab -e):

59 2 * * * /usr/local/bin/convert-multilog 1> /dev/null

qmailctl script

As usual we will put the script in /usr/local/bin and give it the +x flag.

cd /usr/local/bin
wget http://notes.sagredo.eu/sites/notes.sagredo.eu/files/qmail/qmailctl
chmod +x qmailctl

The startup script below does the following:

  • Starts/stops the services
  • Calls tcprules to reload tcp.smtp.cdb and tcp.submission.cdb
  • Shows the status of the services and the queue

Note that it starts and stops vpopmaild also, and starts both normal SMTP on port 25, and the submission service on port 587, where SMTP authentication is required to perform outgoing relay for remote users. If you decide to disable the submission service, delete qmail-submission from the svclist variable on the 4th line below. In any event, be sure to review the service list to make sure it reflects the services you want to provide.

#!/bin/sh

# Put here the services you want to manage
svclist="qmail-send qmail-smtpd qmail-submission vpopmaild"

PATH=/var/qmail/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin
export PATH

QMAILDUID=`id -u qmaild`
NOFILESGID=`id -g qmaild`

case "$1" in
  start)
    echo "Starting qmail"

    for svc in $svclist ; do
    	if svok /service/$svc ; then
     		svc -u /service/$svc
     	else
     		echo $svc service not running
     	fi
    done 

    if [ -d /var/lock/subsys ]; then
      touch /var/lock/subsys/qmail
    fi
    ;;
  stop)
    echo "Stopping qmail..."
    for svc in $svclist ; do
      echo " $svc"
      svc -d /service/$svc
    done
    if [ -f /var/lock/subsys/qmail ]; then
    rm /var/lock/subsys/qmail
    fi
    ;;
  stat)
    for svc in $svclist ; do
      svstat /service/$svc
      svstat /service/$svc/log
    done
    qmail-qstat
    ;; 
  doqueue|alrm|flush)
    echo "Sending ALRM signal to qmail-send."
    svc -a /service/qmail-send
    ;;
  queue)
    qmail-qstat
    qmail-qread
    ;;
  reload|hup)
    echo "Sending HUP signal to qmail-send."
    svc -h /service/qmail-send
    ;;
  pause)
    for svc in $svclist ; do
      echo "Pausing $svc"
      svc -p /service/$svc
    done
    ;; 
  cont)
	  for svc in $svclist ; do
	    echo "Continuing $svc"
	    svc -c /service/$svc
	  done
    ;; 
  restart)
    echo "Restarting qmail:"
    for svc in $svclist ; do
      if [ "$svc" != "qmail-send" ] ; then
        echo "* Stopping $svc."
        svc -d /service/$svc
      fi
    done
    echo "* Sending qmail-send SIGTERM and restarting."
    svc -t /service/qmail-send
    for svc in $svclist ; do
      if [ "$svc" != "qmail-send" ] ; then
        echo "* Restarting $svc."
        svc -u /service/$svc
      fi
    done
    ;;
  cdb)
    if ! grep '\#define POP_AUTH_OPEN_RELAY 1' ~vpopmail/include/config.h >/dev/null; then
      (cd ~vpopmail/etc ; cat tcp.smtp | tcprules tcp.smtp.cdb tcp.smtp.tmp)
      echo "Updated tcp.smtp.cdb."
      (cd ~vpopmail/etc ; cat tcp.submission | tcprules tcp.submission.cdb tcp.submission.tmp)
      echo "Updated tcp.submission.cdb."
    else
      ~vpopmail/bin/clearopensmtp
      echo "Ran clearopensmtp."
    fi
    ;;
  clear)
    echo "Clearing readproctitle service errors with ................."
    svc -o /service/clear
    ;;
  kill)
    echo "First stopping services ... "
    for svc in $svclist ; do
    	if svok /service/$svc ; then
            svc -d /service/$svc
            svc -d /service/$svc/log
    	fi
    done
    echo "Now sending processes the kill signal ... "
        killall -g svscanboot
    echo "done"
    ;;
  boot)
    echo "Starting qmail"
    /command/svscanboot &
    ;;
  reboot)
    echo "First stopping services ... "
    for svc in $svclist ; do
        if svok /service/$svc ; then
            svc -d /service/$svc
            svc -d /service/$svc/log
        fi
    done
    echo "Now sending processes the kill signal ... "
        killall -g svscanboot
    echo "done"
    echo "Starting qmail"
    /command/svscanboot &
    ;;
  help)
    cat <<HELP
    stop -- stops mail service (smtp connections refused, nothing goes out)
   start -- starts mail service (smtp connection accepted, mail can go out)
   pause -- temporarily stops mail service (connections accepted, nothing leaves)
    cont -- continues paused mail service
    stat -- displays status of mail service
     cdb -- rebuild the tcpserver cdb file for smtp
 restart -- stops and restarts smtp, sends qmail-send a TERM & restarts it
 doqueue -- sends qmail-send ALRM, scheduling queued messages for delivery
  reload -- sends qmail-send HUP, rereading locals and virtualdomains
   queue -- shows status of queue
    alrm -- same as doqueue
   flush -- same as doqueue
     hup -- same as reload
   clear -- clears the readproctitle service errors with .....................
    kill -- svc -d processes in svclist, then do 'killall -g svscanboot'
    boot -- Boots qmail and all services in /service running /command/svscanboot
  reboot -- kill & boot commands in sequence
HELP
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|doqueue|flush|reload|stat|pause|cont|cdb|queue|clear|kill|boot|reboot|help}"
    exit 1
    ;;
esac

exit 0 

Usage

# qmailctl help

    stop -- stops mail service (smtp connections refused, nothing goes out)
   start -- starts mail service (smtp connection accepted, mail can go out)
   pause -- temporarily stops mail service (connections accepted, nothing leaves)
    cont -- continues paused mail service
    stat -- displays status of mail service
     cdb -- rebuild the tcpserver cdb file for smtp
 restart -- stops and restarts smtp, sends qmail-send a TERM & restarts it
 doqueue -- sends qmail-send ALRM, scheduling queued messages for delivery
  reload -- sends qmail-send HUP, rereading locals and virtualdomains
   queue -- shows status of queue
    alrm -- same as doqueue
   flush -- same as doqueue
     hup -- same as reload
   clear -- clears the readproctitle service errors with .....................
    kill -- svc -d processes in svclist, then do 'killall -g svscanboot'
    boot -- Boots qmail and all services in /service running /command/svscanboot
  reboot -- kill & boot commands in sequence

qmailctl can be used to kill all qmail processes and to reboot the server. I use this option inside the rc.6 of my Slackware virtual server to avoid errors messages when stopping or rebooting the guest. You can easily call the clear service as well.

svtools

This is a nice collection of tools to manage daemontools' services that you may want to consider. mlcat is one of those; it can cat a service's log with human readable dates with a short command like:

mlcat qmail-smtpd

I slightly modified that script here, just to use it without the need of the "qmail-" prefix:

mlcat smtpd

Allowing selected clients to send outgoing messages

Create /home/vpopmail/etc/tcp.smtp and /home/vpopmail/etc/tcp.submission (the latter one in case you want to enable the submission service).

Enable outgoing relay in this way:

10.0.0.:allow,RELAYCLIENT=""
127.:allow,RELAYCLIENT=""
1.2.3.4:deny
:allow

Localhost and 10.0.0.0 subnet can use our MTA as an outgoing relay, 1.2.3.4 is denied, the other IPs can only send messages to the domains listed inside /var/qmail/control/rcpthosts.

tcp.smtp

0.0.0.0:allow,RELAYCLIENT=""
10.0.0.:allow,RELAYCLIENT=""
127.:allow,RELAYCLIENT=""
:allow,CHKUSER_WRONGRCPTLIMIT="3"

chkuser will ban clients' IP after 3 consecutive failures.

tcp.submission

:allow,CHKUSER_WRONGRCPTLIMIT="3"

Updating cdb files

qmailctl can invoke tcprules to create the cdb file in this way:

# qmailctl cdb
Updated tcp.smtp.cdb.
Updated tcp.submission.cdb.

You must run this command every time you modify tcp.smtp or tcp.submission.

Configuring the standard SMTP service on 25 in tcp.smtp ensures that only localhost and authorized IPs can use the SMTP service as an outgoing relay. We will accept inbound messages from outside as long as the recipient domain is included in the file /var/qmail/control/rcpthosts. When someone sends a message to a domain name not listed in rcpthosts, qmail will respond with “Sorry, that domain isn’t in my list of allowed rcpthosts (#5.7.1)”.

When you enable SMTP authentication on port 587, remote users who successfully authenticate will be allowed to send messages using our MTA.

Improved qmail-send log

I modified extra.h in the source code (see the patch here) to record the Message-ID in the qmail-send log as explained here towards the bottom of the page. Therefore an alias ~alias/.qmail-log will be automatically added as well to store the awk command with the regex which retrieves the Message-ID.

Be aware that you must have a valid MX record for your FQDN (look at /var/qmail/control/me), otherwise you will get an error like this:

Sorry, No mailbox here #5.1.1

The qmail-send log now appears as follows. Note that a copy of each mail is apparently sent to the address specified in the alias above.

2014-11-05 12:00:47.930384500 status: local 1/10 remote 1/20
2014-11-05 12:00:47.952694500 delivery 11: success: Received:_(qmail_17359_invoked_by_uid_89);_5_Nov_2014_12:00:47_+0100/Received:_(qmail_17359_invoked_by_uid_89);_5_Nov_2014_12:00:47_+0100/Received:_from_unknown_(HELO_mx.test.net)_(1.2.3.4)/Received:_from_unknown_(HELO_mx.test.net)_(1.2.3.4)/__by_0_with_ESMTPS_(DHE-RSA-AES256-GCM-SHA384_encrypted);_5_Nov_2014_12:00:47_+0100/Received:_(qmail_17349_invoked_by_uid_89);_5_Nov_2014_12:00:47_+0100/Received:_(qmail_17349_invoked_by_uid_89);_5_Nov_2014_12:00:47_+0100/Received:_from_unknown_(HELO_mail-wg0-f47.google.com)_(74.125.82.47)/Received:_from_unknown_(HELO_mail-wg0-f47.google.com)_(74.125.82.47)/__by_0_with_ESMTPS_(RC4-SHA_encrypted);_5_Nov_2014_12:00:46_+0100/Received:_by_mail-wg0-f47.google.com_with_SMTP_id_a1so597995wgh.6/Received:_by_mail-wg0-f47.google.com_with_SMTP_id_a1so597995wgh.6/Received:_by_mail-wg0-f47.google.com_with_SMTP_id_a1so597995wgh.6/________for_<info@test.net>;_Wed,_05_Nov_2014_03:00:48_-0800_(PST)/X-Received:_by_10.180.23.98_with_SMTP_id_l2mr4797959wif.51.1415185247978;_Wed,/X-Received:_by_10.180.23.98_with_SMTP_id_l2mr4797959wif.51.1415185247978;_Wed,/Received:_by_10.27.203.139_with_HTTP;_Wed,_5_Nov_2014_03:00:47_-0800_(PST)/Received:_by_10.27.203.139_with_HTTP;_Wed,_5_Nov_2014_03:00:47_-0800_(PST)/Date:_Wed,_5_Nov_2014_12:00:47_+0100/Message-ID:_<CAD=Xf-WdCFwED9DiMqRj=bUR5RsRA9mPah1OXgA-tB1ffk-3sw@mail.gmail.com>/Message-ID:_<CAD=Xf-WdCFwED9DiMqRj=bUR5RsRA9mPah1OXgA-tB1ffk-3sw@mail.gmail.com>/Subject:_dasda/From:_xxx_<someone@@gmail.com>/From:_xxx_<someone@gmail.com>/To:_info@test.net/---/did_0+0+2/
2014-11-05 12:00:47.952726500 status: local 0/10 remote 1/20
2014-11-05 12:00:48.326103500 delivery 12: success: 1.2.3.4_accepted_message./Remote_host_said:_250_ok_1415185248_qp_17366/

Comments

Hi I have a problem with Gmail & roberto-netqmail-1.06.patch-2011.07.27 Problem concerns Forcetls patch: The history of the disease: tcprules:

root
# make + Rebuilding tcp.qmail-smtp from tcp.qmail-smtp tcprules tcp.qmail-smtp.cdb tcp.qmail-smtp.cdb.tmp < tcp.qmail-smtp root
# chmod 644 * root
# egrep -v '^$|#' /etc/tcprules.d/tcp.qmail-smtp 127.0.0.1:allow,RELAYCLIENT="" 

Qmail run script (Submission):

#!/bin/sh 
SERVICE=submission 
QMAILDUID='201' 
NOFILESGID='200' 
MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` 
SOFTLIMIT=`cat /var/qmail/control/softlimit` 
export FORCETLS="1" 
export CRAM_MD5="1" 
export GREETDELAY="0" 
export RBLSMTPD_GREETDELAY="1" 
# This enables chkuser 
export CHKUSER_START=ALWAYS 
export CHKUSER_WRONGRCPTLIMIT="3" 
# DKIM configuration 
export QMAILQUEUE="/var/qmail/bin/qmail-dkim" 
export DKIMKEY="/etc/domainkey/wampir7.pl/default" 
export DKIMQUEUE="/var/qmail/bin/qmail-scanner-queue" 
export DKIMSIGNOPTIONS="-b 1 -l -q -z 2" 
export DKIMVERIFY="FGHIKLMNORTUVW" 
exec /usr/bin/softlimit -m "$SOFTLIMIT" \ 
/usr/bin/tcpserver -v -H -R -l 0 \ 
-x /etc/tcprules.d/tcp.qmail-smtp.cdb -c "$MAXSMTPD" \ 
-u "$QMAILDUID" -g "$NOFILESGID" 0 587 \ 
/usr/bin/spamdyke -f /etc/spamdyke/spamdyke.conf \ 
/var/qmail/bin/qmail-smtpd \ 
/var/vpopmail/bin/vchkpw /bin/true 2>&1;

The server is working properly:

qmaild 1314 0.0 0.0 1736 244 ? S 20:48 0:00 /usr/bin/tcpserver -v -H -R -l 0 -x /etc/tcprules.d/tcp.qmail-smtp.cdb -c 20 -u 201 -g 200 0 587 /usr/bin/spamdyke -f /etc/spamdyke/spamdyke.conf /var/qmail/bin/qmail-smtpd /var/vpopmail/bin/vchkpw /bin/true 

Result (swaks):

swaks -t postmaster@example.com -f test@example.com -p587 --tls -s example.com -au test@example.com -ap q 

=== Trying example.com:587...
=== Connected to example.com. 
<- 220 example.com ESMTP 
-> EHLO example.com 
<- 250-example.com 
<- 250-STARTTLS
<- 250-PIPELINING 
<- 250-8BITMIME 
<- 250-SIZE 67108864 
<- 250 AUTH LOGIN PLAIN CRAM-MD5 
-> STARTTLS 
<- 220 Proceed. 
=== TLS started w/ cipher AES256-SHA 
=== TLS peer subject DN="/C=IT/ST=PL/L=Silesia/O=Test/OU=IMAP server/CN=example.com/emailAddress=postmaster@example.com" 
~> EHLO example.com 
<~ 250-example.com 
<~ 250-PIPELINING
<~ 250-8BITMIME 
<~ 250-SIZE 67108864 
<~ 250 AUTH LOGIN PLAIN CRAM-MD5 
~> AUTH CRAM-MD5 
<~* 538 auth not available without TLS (#5.3.3) 
~> AUTH LOGIN <~* 538 auth not available without TLS (#5.3.3) 
~> AUTH PLAIN AHRlc3RAZXhhbXBsZS5jb20AcQ== 
<~* 538 auth not available without TLS (#5.3.3) 
*** No authentication type succeeded 
~> QUIT 
<~ 221 example.com 
=== Connection closed with remote host. 

If I disable FORCETLS (export FORCETLS="1"), login works fine. My OS: Gentoo Hardened with grsec / Pax

cc-v 
Using built-in specs. 
Target: i686-pc-linux-gnu 
Configured with: / var/tmp/portage/sys-devel/gcc-4.4.5/work/gcc-4.4.5/configure - prefix = / usr - bindir = / usr/i686-pc-linux-gnu / gcc-bin/4.4.5 - includedir = / usr/lib/gcc/i686-pc-linux-gnu/4.4.5/include - datadir = / usr/share/gcc-data/i686-pc-linux- gnu/4.4.5 - mandir = / usr/share/gcc-data/i686-pc-linux-gnu/4.4.5/man - infodir = / usr/share/gcc-data/i686-pc-linux- gnu/4.4.5/info - with-Gxx-include-dir = / usr/lib/gcc/i686-pc-linux-gnu/4.4.5/include/g + +-v4 - host = i686-pc-linux -gnu - build = i686-pc-linux-gnu - disable-AltiVec - disable-fixed-point - without-ppl - without-cloog - enable-nls - without-included-gettext - with -system-zlib - disable-werror - enable-secureplt - disable-multilib - enable-libmudflap - disable-libssp - enable-esp - enable-libgomp - with-python-dir = / share / gcc-data/i686-pc-linux-gnu/4.4.5/python - enable-checking = release - disable-libgcj - with-arch = i686 - enable-languages ​​= c, c + +, fortran - enable -shared - enable-threads = posix - enable-__cxa_atexit - enable-CLocale = gnu - with-bugurl = http://bugs.gentoo.org/ - with-pkgversion = 'Gentoo Hardened 4.4.5 p1 .3, pie-0.4.5 ' 
Thread model: posix 
gcc version 4.4.5 (Gentoo Hardened 4.4.5 P1.3, pie-0.4.5)

Fortunately, for the moment I test the server in his home, but soon I'm going to run it on a server at work (~ 300 users)

Can I count on any suggestions to solve the problem?

Cheers ;)

I haven't tested my patch in conjunction with spamdyke. I would try to do a test disabling it.

Indeed, he was guilty Spamdyke.
How Spamdyke compiled with the flag -tls, the problem disappeared.

Thank You

Cheers
:)

Ciao Roberto, grazie per le info che hai messo, pero' nn mi mette la posta su new sotto vpopmail. i logs mostrano che arriva pero' nn si vede, inoltre nn mi fa connettere su 587 o 25 per mandare la posta suggerimenti?

Grazie

Ciao Remo,

stai seguendo la mia guida passo passo? che LDA stai usando?

per quanto riguarda l'invio da remoto, come certamente sai è necessario accoppiare qmail-smtpd a vchkpw nel tuo run file, vedi se vuoi gli esempi nella pagina sulla configurazione, ma sono dei run file molto standard, come vedi. Stai usando la mia patch, con qmail-auth di E.Hoffmann?

Se vuoi posta i log, se preferisci mandameli in privato (tsasto contact su in alto)

Roberto

riguardo al problema dell'invio da remoto, la qmail-auth path richiede che vanga esportata  la variabile SMTPAUTH nel run file. Se viene lasciata vuota è possibile autenticarsi con LOGIN o PLAIN

I appreciate that you are keeping your documentation referencing /var/qmail for the most part.  The supervise/vpopmaild/run script has a reference to "/usr/local/qmail/..." instead of "/var/qmail/.."  while the rest of your scripts reference "/var/qmail".  Up to you if you want to fix it, of course.  I caught it because I don't use /usr/loca/qmail and it was causing an error in readproctitle.

Thanks for the great work!

Thanks, Aaron. Actually the variable MAXSMTPD was not used by tcprules and I have corrected the vpopmaild/run script

Hi,

First off, thanks for your excellent notes / turial. I am building a new server 10 years after the last time I did it. It's so wonderful to have these step-by-step instructions, and all the patches consolidated.

I am curious about the 5 second greet delay in the submission service. Even that low number causes IT stress when I test tghe outgoing mail using Thunderbird.

Since the submission service requires STARTTLS and authentication, spammers are stopped anyway, aren't they?

Regards

/ Otto Dandenell, Sweden

Hi Otto, thanks for your comment.

From time to time I get heavy attacks on port 587 due to spammers who tries to guess users' passwords. Last time it was 2 days with 1 attempt avery few seconds. I see them in /var/log/maillog

I think that 5 seconds is a little stress for users, but a verrrry big loss of cpu time for spammers, if you multiply 5s times the number of attempts spanned in a couple of days :-)

anyway it's much better to have a firewall filter like fail2ban and avoid to use that GREETDELAY (http://notes.sagredo.eu/node/170). Maybe it's better to comment out that line on the run script and advise to use it just in case fail2ban is not active.

And time ago I read that a lot of spammers drop the connection if a GREETDELAY of just a couple of seconds is set, but comments are welcome on the purpose

best regards

Roberto

Roberto,

Thanks for clearing that up. I'll have a look at fail2ban.

Regards

/ Otto

 

I have ipv4 & ipv6 enabled on an Debian wheezy 64 machine and tcpserver binds the services only to ipv6 ports. I have to use the option -4 in the run files to use it with ipv4. But then ipv6 ist not possible anymore. In your standard run files the Option 0 is used which should bind tcpserver to any available IP address on any interface according to the tcpserver doc.  Do you have a clue wyh tcpserver binds the service ports only to ipv6?

Are you using the new e.h.'s tcpserver program http://www.fehcom.de/ipnet/ucspi-tcp6/tcpserver.html? In that case I would try to test the -6 option.

hi roberto, thanks for answering so fast. i'm using the package ucspi-tcp6-1.00. I have tried  the -6 option and the result is the same as without the option (binding only to ipv6). For now i will use the -4 option so that i can use the installation with ip4. When i have more time i will take a closer look. Thank you for this tutorial. It  has helped me much.
 

Hello,

as an alternative for rotating the log files from the qmail services i use the convert-multilog script from John Simpson, who describes the script function as follows:

convert-multilog is a script which searches "/service/*/log/main" for any "@4*" files (the automatic cut-off files generated by multilog), converts their timestamps from tai64n to human-readable format, and writes them to /var/log/{service}.{date}. Once the lines from a given "@4*" file have been converted, the file is deleted.

To make the script work with this tutorial, you have to create the following links:

cd /service/qmail-send/log/
ln -s /var/log/qmail/send main
cd /service/qmail-smtpd/log/
ln -s /var/log/qmail/smtpd main
cd /service/qmail-submission/log/
ln -s /var/log/qmail/submission main
cd /service/vpopmaild/log/
ln -s /var/log/qmail/vpopmaild main

Then you have to create a cronjob:

59 23 * * * /usr/local/bin/convert-multilog

I like this one because i have the log files inside my /var/log/ dir and the logfiles have normal timestamps.

Cheers

This is very interesting. I think I'll use it as soon as possible. Thank you

I modified this page with your suggestion :)

Hi Roberto once again thanks for this guide, it's the best Linux guide i've ever seen, so perfect and thourough!

But I have a problem, when I send mail from gmail to a valid vpopmail user at my qmail server, the mail is delivered fine, but the external sender additionally gets a postmaster bounce email from my qmail server about a log@[myserver.net] delivery failure, saying that user doesn't exist like this:

Hi. This is the qmail-send program at [myserver.net].
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<log@[myserver.net]>:
Sorry, no mailbox here by that name. (#5.1.1)

Also, my logs don't seem to have the extra timestamps and message-ID's that yours do, so obviously it's something wrong with logging but I can't figure out what/where the problem is!

+I am also using the netqmail-1.06-exttodo-channels patch: (info, patch file) - I had to patch the failing hunks myself, but it seems to work, hope thats not causing the logging problem), in the log below that's what the suppl[N] queues are for.

You can see what it's doing in the qmail-send log below, it seems to be generating a log@ email for every send, hence the double bounce at the end.

In the below log:
- myserver.net is my qmail server I am setting up (the one I followed your guide on):
- user@gmail.com is the user who sent the mail to my vpopmail user, and who recieves the postmaster bounce email above (about log@myserver.net failing).
- test@myserver.net is the valid vpopmail user who receives the mail from the gmail user ok.

/var/log/qmail/send/current:

@400000005601a2a50551e67c status: local 0/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 supl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a4953a8e805c new msg 4854452
@400000005601a4953a8e882c info msg 4854452: bytes 7316 from <user@gmail.com> qp 1728 uid 89
@400000005601a4953a8e8c14 starting delivery 1: msg 4854452 to local myserver.net-log@myserver.net
@400000005601a4953a8e8ffc status: local 1/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a4953a8f0914 starting delivery 2: msg 4854452 to local myserver.net-test@myserver.net
@400000005601a4953a8f10e4 status: local 2/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a49600472cb4 delivery 2: success: did_0+0+1/
@400000005601a49600473484 status: local 1/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a49601a8faec delivery 1: failure: Sorry,_no_mailbox_here_by_that_name._(#5.1.1)/
@400000005601a49601abdd34 status: local 0/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a49605b3a9d4 bounce msg 4854452 qp 1737
@400000005601a49605b3adbc end msg 4854452
@400000005601a4960c5ce4e4 new msg 4854781
@400000005601a4960c5ce8cc info msg 4854781: bytes 7878 from <> qp 1737 uid 1007
@400000005601a4960c5cecb4 starting delivery 3: msg 4854781 to local myserver.net-log@myserver.net
@400000005601a4960c5cf09c status: local 1/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a4960c5d42a4 starting delivery 4: msg 4854781 to remote user@gmail.com
@400000005601a4960c5d468c status: local 1/20 remote 1/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a4960f48f0e4 delivery 3: failure: Sorry,_no_mailbox_here_by_that_name._(#5.1.1)/
@400000005601a4960f49cba4 status: local 0/20 remote 1/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a496351e32ac delivery 4: success: 173.194.192.26_accepted_message./Remote_host_said:_250_2.0.0_OK_1442948236_t17si3813976ioi.28_-_gsmtp/
@400000005601a496351eceec status: local 0/20 remote 0/100 suppl0 0/20 suppl1 0/20 suppl2 0/20 suppl3 0/20 suppl4 0/20 suppl5 0/20 suppl6 0/20 suppl7 0/20 suppl8 0/20 suppl9 0/20 suppl10 0/20 suppl11 0/20 suppl12 0/20 suppl13 0/20 suppl14 0/20 suppl15 0/20 suppl16 0/20 suppl17 0/20 suppl18 0/20 suppl19 0/20
@400000005601a496352051a4 double bounce: discarding bounce/4854781
@400000005601a4963520b734 end msg 4854781

Thanks!

Hi George, I can't garantee that the patch you applayed on top of my package will not interfere in some way..

Anyway, as mentioned above in the "Improved qmail-send log" paragraph, you have an email to log@yourdomain just to record the Message-ID in the qmail-send log. You may want to revert that qmail-queue-extra patch or double check your configuration, in particular the content of the file .qmail-log

Thanks for getting back to me Roberto it's really appreciated!
I did realise it was something to do with the logging, but now I realise how it actually works, it's smart, you send an internal email to the log@ which is the command to log! So I can tell now, something is thinking log@ doesn't exist and therefore:

1. It's sending a bounce
2. The enhanced logging is not working.

So I just have to figure out why it's deciding log@ doesn't exist when actually it does according to qmail alias config.

Thanks a million, you're work has really helped us!

I was seeing the same as above. Fixed it by editing the control/locals file. You have to use the local host name, e.g. mx.example.tld rather than example.tld The 'hint' for me was in looking at .qmail-postmaster - this essentially causes postmaster@mx.example.tld to forward to postmaster@example.tld - the domain I configured with ./vadddomain

Hello,

I set everithing like you wrote in this tutorial and everithing works well, but I have a problem with receieving error message from postmaster when I send email to unknown user.

Instead from postmaster@example.com I get  error message from  "postmaster@mail.example.com"@mail.example.com

Do you know how can I fix that to get message from postfix@domain.net instead of postmaster@fqdn_hostname? 

Thank you,

Al

on qmail the hostname of the bounce sender is from the control/me file

Hi Roberto,

I now realized that I should get system messages from postmaster@FQDN instead postmaster@HOSTNAME(e.g. postmaster@mail.example.com instead postmaster@example.com) just like you wrote in this tutorial.

Also, I changed /control/bouncefrom file and put only "postamster" instead "postmaster@example.com", so now I receive mails from postmaster@mail.example.com instead  "postmaster@mail.example.com"@mail.example.com just like I wanted

Thank you for your support

One minor typo I noticed was that your supervise script has the pop server running on port 89 (the vpopmail uid) rather than the usual port 110. Probably this wasn't what you had in mind. Thanks for the great guide.

uh.. I don't remember to have published any note about qmail-pop3d... can you provide a link to the page please?

I'm referring to qmail/supervise/vpopmaild/run from your tarball and listed just up the page a bit from here. That last 89 should probably be a 110 instead. Looks like the vpopmail UID got typoed in place of the intended port number.

exec /usr/local/bin/softlimit -m 25000000 \
     /usr/local/bin/tcpserver -v -H -R -l 0 \
     -u "$QMAILDUID" -g "$NOFILESGID" 0 89 \     # You probably want a 110 here instead of an 89
     /home/vpopmail/bin/vpopmaild 2>&1

Not so sure about running vpopmaild as root either. I think you want to be doing that as vpopmail so as to access the maildirs it owns.

#!/bin/sh

QMAILDUID=`id -u root` # wouldn't this be better off as "-u vpopmail"

NOFILESGID=`id -g root` # and "-g vchkpw"?

No it is not the pop3 service (pop3 is served by dovecot in my configuration). vpopmaild is a daemon that you can use to connect and talk with vpopmail. Dovecot and roundcube can use it to change the passwords, for instance