qmail + vpopmail + Dovecot | Roberto's qmail notes

September 15, 2016 by Roberto Puzzanghera 47 comments

Quoting D. J. Bernstein definition

qmail is a secure, reliable, efficient, simple message transfer agent. It is designed for typical Internet-connected UNIX hosts

References

You can find in this page a comprehensible introduction on how a mail server works. The "qmail newbie's guide to relaying" (local copy) by Chris Johnson is very clear as well. It’s very suitable reading material for someone who’s just getting started.

Disclaimer

The aim of this short guide is NOT to teach you how a mail server works, even though by the time you’re finished reading it you will hopefully have a working e-mail server. These notes just serve as a reminder of the main steps to follow in order to build a quick installation of qmail and related software. I published them because of the lack of any up-to-date documentation concerning the qmail “distributions” I was familiar with, hoping that these notes could be useful to others out there. And I created this guide partly just because I enjoy doing this kind of thing.
Therefore, to learn in depth how a mail server works, you are invited to read carefully at least the references I will mention in each page.

Secondly, I am NOT responsible for what you do with your server :) Use my guide at your own risk.

Finally, comments, criticisms and suggestions are always welcome! :-p

Which distro?

These notes have been written without a specific Linux distribution in mind. I tested them on my Slackware virtual mail servers and several times on Debian and Ubuntu, and a number of guys out there can confirm that it works with other common distributions.

Is this a toaster?

According to the DJB's definition of a toaster, the answer would be yes. I personally consider a toaster something a la Bill Shupp or qmailtoaster, which comes with the packages included. Since I prefer to let you check for the latest versions of everything yourself, strictly speaking this shouldn’t properly be considered a toaster. I would simply call this site “Roberto’s qmail notes” instead. At any rate, I’ve included a paragraph about qmail toasters here just to satisfy the search engines -as most people come here actually looking for a toaster :)).

Before we start...

As I am not  a native english speaker, I will gladly accept every hint to improve the understanding of this guide.

Edit: I would like to address a big thank to Dave Martin, who revised my english in the qmail section of this guide.

Licence

Creative Commons License

Roberto's qmail notes is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Comments

Log size of every email sent ?

Hi
I'm looking for the way to "control" the email traffic by size not by quantity
Knows someone if there is some patch for this ?

By the way, I want to say "Thanks !!!", one more time, because I'm updating servers and I was looking for what was changed in certbot, and I found it here again !

Thanks again for all the share knowledge

Reply |

Log size of every email sent ?

Hi Pablo, thanks for the appreciation.

Do you mean a filter based on a per mail size or a filter which sums the size of msg in given period of time? Regarding the second option I don't know 

Reply |

Log size of every email sent ?

Hi !

the 2nd option !
I just find the way
Qmailanalog !
The program process de qmail-send log and make a nre file with the information arranged in better way, and has a lot scripts to generate different reports

Reply |

Log size of every email sent ?

Great! It is a program that I wanted to play with soon or later

Reply |

Log size of every email sent ?

Here is a script I made to process the qmail-send logs

#!/usr/local/bin/bash
#
#
# This script is for use on the log for the day before (yesterday :D)
#
# I use multilog, so I have to check if the log
# of the day is in more than one file
#

send_log_dir = "/var/log/qmail/send"

archivos=`find -s ${send_log_dir}/* -type f -mtime 1 ! -size 0c`
cat $archivos sin_procesar > send_log

# Fecha desde
desde=`date -v-1d -v0H -v0M -v0S +%s`

# For test only, using a day you want to test
# just remove # and change the date
#
#desde=`date -j -f "%Y%m%d%H%M%S" "20230902000000" +%s`

# Fecha hasta
hasta=`date -v0H -v0M -v0S -v-1S +%s`

#
# For test only, using a day you want to test
# just remove # and change the date
#
#hasta=`date -j -f "%Y%m%d%H%M%S" "20230902235959" +%s`

# The multilog use tai64 for date/time, and qmailanalog use seconds since 1970
# so I have to parse the log using tai64nfrac and filtering the result using awk
cat send_log | tai64nfrac | awk -v desde=$desde -v hasta=$hasta '{ if ($1 >= desde && $1 <= hasta) print $0 }' > log_$desde

# Here is where qmailanalog do "the magic" :D
cat log_$desde | /var/qmail/qmailanalog/bin/matchup >procesado_$desde 5>sin_procesar

#
# Then you can use the script in BIN dir, the ones starting with Z or X
cat procesado_$desde | /var/qmail/qmailanalog/bin/zoverall

Reply |

Log size of every email sent ?

Thanks for sharing 

Reply |

arm64

Hi all. Has anyone of you sucessfully installed on an arm64? I am planning to install this on a Raspberry PI 4.

Please share your experience.

Thank you

Reply |

arm64

Hi, thanks for your post. I've never played with arm but I'm curious. Do you run slackware on it? Let us know if you do it!

Reply |

Thank you! for all the documentation, patches and support

I just wanted to drop a quick THANK YOU for the great work you do with keeping all the valueable qmail information online.

I have studied the qmail source code back in '99 and learned a lot about C programming back then. IMHO, qmail is still one of the best MTA when it comes to send out tons of mail quickly.

 Your website has helped me tremendously through the past years, so again – thanks! I know it takes a lot of effort to keep up such a project.

Reply |

Upgrade documentation

Hello Roberto,

my qmail server has been working perfectly for the past year and I'm now planning to patch and upgrade O.S. and the entire qmail stack; I usually keep just the roundcube part constantly updated.

We don't have a upgrade complete set of instructions, right ? Any particular issues to take care of, besides:

- [backup tasks]

- qmailctl stop

- configure && make && make install

- [same per dovecot]

- qmailctl start

Thank you !

Maurizio

Reply |

Upgrade documentation

Ciao Maurizio,

generally the upgrade depends on your installed version, but sometime you can find tips of what you have to do when upgrading.

Concerning qmail, it depends on your current version of the big patch. If it's not too old just patch, stop, compile, start the server.

Concerning dovecot, if you are upgrading from 2.3.x to the latest version you can do the same. If moving from 2.2 to 2.3 version, take a look at the "Moving to 2.3 branch" paragraph.

Concerning spamassassin you have to recompile and restart. There are new configurations of razor, pyzor, spamcop that you may want to upgrade as well.

Roundcube is very easy to upgrade from every version. Look at the top of the page to find out how.

Clamav: recompile and install on top of the previous one

simscan: patch recompile and install on top of the previous version.

Reply |

Upgrade documentation

Thank you Roberto !

Looks reasonable to me. My qmail patch is about "one year old", I will have to double check everything before proceeding.

Do we have a simple way of backupping the current qmail installation (I'm referring to binary files), besides saving a copy of /usr/bin/ - qmail/control - etc. ?

(I'm snapshotting my virtual server just in case....)

I think that these simple steps you are reporting could become a standalone page, they are very useful and I think a lot of other sysadmins have to deal with upgrading issues, especially in the present unsecure world :-)

Thank you, ciao and Happy Christmas !

Maurizio

Reply |

Upgrade documentation

If your installed patch is one year old you can just recompile and restart.

I don't have suggestions to quickly backup the server. I'm used to daily rsync all configs and messages and also clone my virtual server before migrating or doing very big updates

Reply |

Upgrade documentation

Thank you Roberto, same procedure for me :-)

Reply |

Load Balance SMTP

Has anyone set up any load balances for Qmail SMTP?

I am currently using HAProxy, but I am having difficulty maintaining Remote IP on connections when sending to the Qmail server.

Reply |

rspamd

Hi,

Anyone here has a working setup with Qmail and rspamd? (is this possible at all?)

Reply |

rspamd

Hi,

It is certainly possible, either through running rspamc in the desired .qmail files or through hackery :)

Personally I've hacked together an ugly patch that will run rspamc through simscan, which works and lets me enable it for entire domains, but it's an ugly patch at the moment ("works for me") that could use quite a bit of polish.

That said, if you want to try it without warranty and on your own risk, you can grab it here: http://notes.benv.junerules.com/wp-content/uploads/2019/04/simscan-1.4.0.rspamc.patch

Improvements welcome ;)

Regards,

Wouter

Reply |

rspamd

Hello Wouter,

Sounds promising, might give it a try quick response also :)

cdr

Reply |

thanks

Great write up, I've used your notes a number of times and it works great.

Reply |

log@serverhost

Hi,

I had built a new toaster with the latest patch. It was about 2 days old. Everything was working fine and today it happened that every email sent, there a a bounced message. The bounced message  complaints that it tried to deliver the message to log@serverhost.

The intended recipient still maanged to get the email.

Bounced mail below :-
Hi. This is the qmail-send program at domain.com.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.
<log@domain.com>:
Sorry. Although I'm listed as a best-preference MX or A for that host, it isn't in my control/locals file, so I don't treat it as local. (#5.4.6)

I decided to create log@domain.com and tried to see what is being recieved, it happened to be the exact email sent out.

Any idea what went wrong?

thanks

nic

Reply |

qmail-extra was modified..

The mails sent to log@domain.com are needed just to improve the qmail-send log; read the qmail-extra patch on the purpose.

But domain.com must have a valid MX and has to be included in you locals file. Try to see what

qmail-showctl |grep local 

says

Reply |

Thanks Roberto.Since it is

Thanks Roberto.

Since it is needed, i will just keep the account log@domain.com and put up a crontab to delete mails nightly.

Reply |

no, this is not how it is

no, this is not how it is programmed to work. There's no actual email to that account. Check the contents of the file /var/qmail/alias/.qmail-log and compare with what I reported in the qmail-extra patch documentation

Reply |

no, this is not how it is

Hi Roberto

I have the same problem as Nic. My clients always receive bounced emails saying that they cannot send email to mslog@domain.com. Because I installed qmail from complied binary, I do not have an access to extra.h. How should I stop this logging? In my qmail log, it contains many "starting delivery xxxxx: msg xxxx to remote msglog@domain.com".

Thank you

Sam.

Reply |

no, this is not how it is

So your qmail has nothing to do with my patch and the present guide is not suitable for your installation. For this reason if you follow my instructions you will get an unpredictable behaviour

Reply |

Hi.

Hi.

Sorry for the late reply.

My .qmail-log as follows

| awk '/^$/ { exit } /^[mM][eE][sS][sS][aA][gG][eE]/ { print } /^[rR][eE][cC][eE][iI][vV][eE][dD]:/ { print; } /^[fF][rR][oO][mM]:/ { print } /^[tT][oO]:/ { print } /^[sS][uU][bB][jJ][eE][cC][tT]:/ { print } /^[xX]-[mM][aA][iI][lL][eE][rR]/ { print } /[hH][eE][lL][oO]/{ print } /^[rR][eE][pP][lL][yY]-[tT][oO]/{ print } /^[rR][eE][tT][uU][rR][nN]-[pP][aA][tT][hH]/{ print } /^[cC][cC]:/{ print } /^[dD][eE][lL][iI][vV][eE][rR][eE][dD]-[tT][oO]/{ print } /^[dD][aA][tT][eE]:/{ print } / by /{ print } / id /{ print } /<.*>/{ print }'
| echo "---"

thanks
nic

Reply |

it seems correct... double

it seems correct... double check everything

Reply |

it seems correct... double

its like this:

if /var/qmail/control/locals - does NOT include hostname.yourdomainname.com

then qmail is trying to send an e-mail to remote log@hostname.yourdomainname.com

if u put hostname.yourdomainname.com in /var/qmail/control/locals

then all goes ok.

Reply |

it seems correct... double

Thank you for pointing out this. I thought that having hostname.yourdomainname.com in /var/qmail/control/locals is a good practice regardless of the fact that you have that patch installed. This is the reason why I didn't even mentioned that.

I've just added a note about this in the "Configuring qmail" page

Reply |

I don't know if this might be

I don't know if this might be useful, but since I'm using a local account named "log@domain.com", and I cannot rename it, in order to avoid the duplicate email problem I had to revert this part of the patch:

-#define QUEUE_EXTRA ""
-#define QUEUE_EXTRALEN 0
+#define QUEUE_EXTRA "Tlog\0"
+#define QUEUE_EXTRALEN 5

So my extra.h looks like this:

#ifndef EXTRA_H
#define EXTRA_H

#define QUEUE_EXTRA ""
#define QUEUE_EXTRALEN 0

#endif

Regards,

miz

Reply |

Qmail log@

FIrst of all, 

congratulations for your work and tutorial.

we follow and mounted a new qmail install.

we repair that the new server trys to send a "copy" of every message to log@domain.com

can you tell us where you put this conf, cause we don't found any reference to this.

Best regards and congratulations again.

Matheus

Reply |

I added a clarification about

I added a clarification about this functionality at the bottom of the configuration page

Reply |

Hi, I didn't write a

Hi,

I didn't write a related page yet, just a note here http://notes.sagredo.eu/node/82#queue-extra. Actually it is a way to improve the qmail-send log.

Is the functionality working fine for you?

Reply |

Qmail and Mirroring

Hi

This documentation is the best for qmail, I'm working with qmail a year without problems, but it feels like I have to advance a bit.

Is there any possibility to setup a mirror qmail server? i mean have two servers working and when one is down the other keep working with all the services. Imap, Pop3, SMTP, etc.

Thanks in advance.

Reply |

uh... I really don't know how

uh... I really don't know how exactly, but I guess that a lot of work should be done to dinamically change the IP in the name server in the event that a program like ping (?) returns a certain response. And also to sync back and restore everything automatically.

Reply |

Thanks!

Thanks for the qmail info! Helped a ton when I had to update a old server recently and didn't want to switch away from qmail.

Saved me tons of time!

Reply |

thanks

Roberto, you did a great job! thanks a lot!

to be noted:

- your vpopmail guide is not --large-site=y, so your vpopmail mysql table only had 1 single 'vpopmail' table. i cannot find dovecot support --large-site=y configuration for sql auth, i am backward to use vpopmail auth instead to make it work atm.

regards,

GW

Reply |

I think you mean

I think you mean --enable-many-domains (http://www.inter7.com/vpopmail/FAQ.txt FAQ #19). It can be a choice, of course, in case of many domains.

So, if I understand well, this option is not dovecot compliant? If you use the SQL backend it should be sufficient to adjust the sql query accordingly. I can't get the vpopmail driver working anymore...

Reply |

regarding --enable-many-domains option

YES indeed, dovecot compliences issue, and another issue to be added, there is corrupted dovecot.index.chache each time new message arrived! Actually i can get running dovecot 2.2.5 with vpopmail 5.4.33 with --enable-many-domains option using vpopmail driver BUT another issues came up:

- LDA permission to user socket connection denied

- Dovecot Indexing causes dovecot.index.chache corrupted

- LDA issue causes Managesieve/Sieve did not work.

Reply |

I've never tried this

I've never tried this configuration, but if I were in you I would try to switch to the mysql driver.

Let us know...

Reply |

SQL driver migration

i can do migration from vpopmail to sql properly but imap-auth scheme support is only LOGIN and PLAIN, it is failed when i 'm using DIGEST-MD5 and CRAM-MD5. i will try to solve this later but the LDS Indexing issue still remind eventhough dovecot always fix it after checking it.

Reply |

You can get rid of DIGEST-MD5

You can get rid of DIGEST-MD5 and CRAM-MD5 enabling imaps and pop3s...

Reply |

Indeed, but LDA Indexing

Indeed, but LDA Indexing corrupted still annoying while courier works properly...stuck with corrupted dovecot.index.cache...

Reply |

qmail support site url changed

please note the URL has been moved to

www.fehcom.de/qmail/qmail.html

 Thank you for putting these notes together.

David.

Reply |

Thanks for an EXCELLENT resource

Roberto,

We migrated to our new mail server yesterday (built from the ground up using your notes as our guide).  It’s working great!  SpamAssassin is working!  ClamAV is working! SPF is working! DKIM is working!  RoundCube rocks!  In short—everything works! 

Up til now, we had been limping along with a partially broken, spam-laden mail server.  Worse yet, I didn’t know what steps to take to fix it—seemed like any fixes I implemented ended up breaking something else.

Thanks so much for notes.sagredo.eu—it’s been a Godsend!

-Dave Martin

Reply |

Nice!

Hello,

Congratulations for this how to, i think this is one of the most complete on web. Updated tools and perfect to Slackware, my favorite distro.

Reply |

Nice howto

Hello,

I just convert the mailboxes configuration with myself/scripts/bash etc.. nvm..  It's done.  this issue is close.
2 days ago, I switch my company domain, to the new mail server, and it works great! Good spam, virus protecion. Nice looking webmail, and fresh packages.  Bling-bling ;))

I'am very verrrryy glad that You help and assist me at time of the configuration of your toaster. Thank you! good job, nice howto ;-)

Reply |