Installing Dovecot and sieve on a vpopmail + qmail server

June 20, 2021 Roberto Puzzanghera69 comments

Changelog

  • June 20, 2021
    -15-mailboxes.conf: added Junk.TeachSpam and Junk.TeachNotSpam mailboxes to store messages for the learning and reporting systems (more info here)
  • March 17, 2021
    - 90-quota.conf: quota definition adjusted in this way quota = maildir:: as the GETQUOTA command was not returning the quota (thanks a.key)
  • March 2, 2021
    - "one table per domain" support added (--disable-many-domains)
    - added domains limits support to password_query (you have to compile vpopmail --enable-mysql-limits)
  • January 29, 2021
    - auth-sql.conf.ext now uses the userdb's prefetch driver in order to perform one single query when doing the auth
    - dovecot-sql.conf.ext has been modified to allow authentication both with real and alias domains, provided that you patched vpopmail accordingly. More info in this page.
    - the iterate_query in the sql driver now extracts the "user" field (was "username") as required by the docs.

Overview

Dovecot is an open source IMAP and POP3 email server for Linux/UNIX-like systems, written with security primarily in mind. Dovecot is an excellent choice for both small and large installations. It's fast, simple to set up, requires no special administration and it uses very little memory.

Dovecot is simple to configure, the project is alive and the documentation is exhaustive. I found Dovecot so friendly that I decided to switch off qmail-pop3d and use Dovecot not only as an IMAP but also as a POP3 server. I will show how to configure a secure connection as well.

Setting stats' service priviledges

February 24, 2020 update: today I stumbled upon this error in my qmail-send log (yes qmail-send, not dovecot.log)

@400000005e5303ff0fe76364 delivery 407: success: lda(postmaster@mydomain.xy):_Error:_net_connect_unix(/usr/local/dovecot/var/run/dovecot/stats-writer)_failed:_Permission_denied/did_0+0+1/

Fortunately the messages were delivered successfully to the Maildirs despite of this problem.

I realized that I had to setup the stats service when upgrading in the past days. Unfortunately the configuration example shipped by dovecot doesn't even mention how to setup this service, anyway I quickly found the solution on google. It is sufficient to start the service as vpopmail adding these lines to 10-master.conf

service stats {
  unix_listener stats-reader {
    user = vpopmail
    group = vchkpw
    mode = 0660
  }
  unix_listener stats-writer {
    user = vpopmail
    group = vchkpw
    mode = 0660
  }
}

Restoring the maildirs (maildir++ patch bug fix)

It was pointed out (tx MG) that the maildir++ patch was broken because of a wrong calculated maildir size, and this is causing troubles especially with the new dovecot-2.3. Now that patch was repaired but the maildirs' size remains broken. This can be quickly (actually this may take a long time if done with all (-A) users) restored as follows:

doveadm fetch -A text all > /dev/null

If you don't want to take the trouble to restore your users' maildirs, you can always choose to allow the broken size adding this parameter to 10-mail.conf:

maildir_broken_filename_sizes = yes

Moving to 2.3 branch

On december 2017 dovecot-2.3.0 was released. As far as my previous 2.2 configuration is concerned, there are just a few changes:

  • the submission service can be disabled (just rename 20-submission.conf);
  • if you were using the maildir++ patch for netqmail embedded in my combined patch, you have to add "maildir_broken_filename_sizes = yes" in your 10-mail.conf (tx MG). This parameter is not needed if you restored the maildirs as explained above;
  • changes in 10-ssl.conf: you have to setup a SSL DH parameter and replace some other obsolete parameter; take a look below for more info;
  • changes in 15-lda.conf: the sendmail program was not working anymore here when called by managesieve. So we can disable it and setup a submission_host (not to be confused with the submission service above);
  • some change in both 20-managesieve.conf and 90-sieve.conf: it's better to rebuild them from scratch following the next page.

Installing

cd /usr/local/src
wget https://www.dovecot.org/releases/2.3/dovecot-2.3.15.tar.gz
tar xzf dovecot-2.3.15.tar.gz
chown -R root.root dovecot-2.3.15
cd dovecot-2.3.15

./configure \
        --prefix=/usr/local/dovecot \
        --with-sql \
        --with-mysql \
        --with-docs \
        --with-ssl \
        --without-shadow \
        --without-pam \
        --without-ldap \
        --without-pgsql \
        --without-sqlite

Install prefix . : /usr/local/dovecot 
File offsets ... : 64bit 
I/O polling .... : epoll 
I/O notifys .... : inotify 
SSL ............ : yes (OpenSSL) 
GSSAPI ......... : no 
passdbs ........ : static passwd passwd-file checkpassword sql 
CFLAGS ......... : -std=gnu99 -g -O2 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=keep -mindirect-branch=keep -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2   
                 : -shadow -pam -bsdauth -ldap 
userdbs ........ : static prefetch passwd passwd-file checkpassword sql 
                 : -ldap 
SQL drivers .... : mysql 
                 : -pgsql -sqlite -cassandra 
Full text search : squat 
                 : -lucene -solr

In case you are compiling in a systemd environment you may need to add "--with-systemd systemdsystemunitdir" option to your configure (thanks to Bob Greco), for example:

--with-systemd \
systemdsystemunitdir=/lib/systemd/system

Check if SSL was detected and if the sql/mysql authentication backend is supported. Compile:

make

If the old dovecot is running, stop it and delete the symbolic link before installing:

dovecotctl stop
rm /usr/local/dovecot
make install

cd /usr/local
mv dovecot dovecot-2.3.15
ln -s dovecot-2.3.15 dovecot
This installs dovecot in /usr/local/dovecot.

Configuring

Create the dovecot user/group, the log folder and the directory where to store runtime data.

groupadd dovecot
useradd -g dovecot dovecot
useradd -g dovecot dovenull

mkdir -p /var/log/dovecot 
touch /var/log/dovecot/dovecot.log
chgrp vchkpw /var/log/dovecot/dovecot.log
chmod 660 /var/log/dovecot/dovecot.log

The config files must be copied from the share/doc to etc/dovecot folder

cd /usr/local/dovecot/etc/dovecot
cp -rp /usr/local/dovecot/share/doc/dovecot/example-config/* .

Adjust the MANPATH variable (add this line to your /etc/profile as well):

export MANPATH=$MANPATH:/usr/local/dovecot/share/man

Download my dovecot's config files

  • Download from here

If you don't want to have the trouble to modify your conf files and want to follow the directives of this guide you can download my config files and adjust them to your needs.

cd /usr/local/dovecot
mv etc etc-bak
wget https://notes.sagredo.eu/files/qmail/dovecot.conf.tar.gz
tar xzf dovecot.conf.tar.gz
mv dovecot-etc etc
cd etc/dovecot
chown -R root.root .

dovecot.conf

protocols = imap pop3
# delete IP6 addresses here
listen = *
dict {
  #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
  #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext
}
!include conf.d/*.conf
!include_try local.conf

Note that all conf.d/*.conf files will be included, so rename what is not needed.

10-auth.conf

# if the roundcube connection is from the local net this is secure provided that the 143 port is firewalled for the outnet
# put yes if you have roundcube on localhost
disable_plaintext_auth = no
# To append @domain to username in plaintext logins
auth_default_realm = defaultdomain.net
auth_mechanisms = plain login # the sql drivers requires that the pwd is transmitted with no encryption
# !include auth-system.conf.ext # now commented
# !include auth-vpopmail.conf.ext # commented
!include auth-sql.conf.ext

Starting from version 2.3.13 the vpopmail driver was removed. If you want to stick with the vpopmail driver you have to install dovecot-2.3.11.3 and modify the configuration as suggested here.

In my opinion sql is preferable because the vpopmail driver misses the iteration feature, which is needed to expunge old junk and trash mails, a task that will be not easy to achieve without it.

auth-sql.conf.ext

  • More info here (user and domain limits examples)
passdb {
  driver = sql
  # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
  args = /usr/local/dovecot/etc/dovecot/dovecot-sql.conf.ext
}

# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# <doc/wiki/UserDatabase.Prefetch.txt>
userdb {
  driver = prefetch
}

# This is for LDA. It will do a passdb query in any case.
# But using the prefetch driver will speed up our normal imap operations and
# this is more important.
userdb {
  driver = static
  args = uid=89 gid=89 home=/home/vpopmail/domains/%d/%n
}

# This is needed for the iteration feature. It needs a userdb sql query.
# The order of the declared drivers is important. Leave this at the end, otherwise
# it will be used also for the login instead of the prefetch, which is faster.
userdb {
  driver = sql
  args = /usr/local/dovecot/etc/dovecot/dovecot-sql.conf.ext
}

dovecot-sql.conf.ext

NB: the password_query requires that you installed the aliasdomains dbtable as explained here. If you don't need to store aliasdomains on mysql, use this file instead. If you don't know what to do proceed as described below.

# This file is opened as root, so it should be owned by root and mode 0600.
#
# Database driver: mysql, pgsql, sqlite
driver = mysql

# Database connection string. This is driver-specific setting.
#
connect = host=[MySQL IP] dbname=vpopmail user=vpopmail password=[PASSWORD]

# Default password scheme.
#
default_pass_scheme = MD5-CRYPT

# passdb query to retrieve the password. It can return fields:
#   password - The user's password. This field must be returned.
#   user - user@domain from the database. Needed with case-insensitive lookups.
#   username and domain - An alternative way to represent the "user" field.
#
# The "user" field is often necessary with case-insensitive lookups to avoid
# e.g. "name" and "nAme" logins creating two different mail directories. If
# your user and domain names are in separate fields, you can return "username"
# and "domain" fields instead of "user".
#
# The query can also return other fields which have a special meaning, see
# http://wiki2.dovecot.org/PasswordDatabase/ExtraFields
#
# Commonly used available substitutions (see http://wiki2.dovecot.org/Variables
# for full list):
#   %u = entire user@domain
#   %n = user part of user@domain
#   %d = domain part of user@domain
#
# Note that these can be used only as input to SQL query. If the query outputs
# any of these substitutions, they're not touched. Otherwise it would be
# difficult to have eg. usernames containing '%' characters.
#
# Example:
#   password_query = SELECT userid AS user, pw AS password \
#     FROM users WHERE userid = '%u' AND active = 'Y'
#
# (Thanks to Arturo Blanco for his hints concerning vpopmail limits
#  and to Alexandre Fonceca for quota_rule addition)
#
# NB: This query requires that the sql-aliasdomains patch has been installed
#     and that you have the aliasdomains dbtable
#     More info here https://notes.sagredo.eu/en/qmail-notes-185/dovecot-vpopmail-auth-driver-removal-migrating-to-the-sql-driver-241.html
password_query = \
	SELECT \
		CONCAT(vpopmail.pw_name, '@', vpopmail.pw_domain) AS user, \
  		vpopmail.pw_passwd AS password, \
                vpopmail.pw_dir AS userdb_home, \
                89 AS userdb_uid, \
                89 AS userdb_gid, \
                CONCAT('*:bytes=', REPLACE(SUBSTRING_INDEX(vpopmail.pw_shell, 'S', 1), 'NOQUOTA', '0')) AS userdb_quota_rule \
	FROM vpopmail \
		LEFT JOIN aliasdomains ON aliasdomains.alias='%d' \
                LEFT JOIN limits ON limits.domain='%d' \
	WHERE \
		vpopmail.pw_name='%n' \
		AND \
		(vpopmail.pw_domain='%d' OR vpopmail.pw_domain=aliasdomains.domain) \
                AND \
                ('%a'!='995' OR !(vpopmail.pw_gid & 2)) \
                AND \
                ('%r'!='<WEBMAIL-IP>' OR !(vpopmail.pw_gid & 4)) \
                AND \
                ('%r'='<WEBMAIL-IP>' OR '%a'!='993' OR !(vpopmail.pw_gid & 8)) \
                AND \ 
                ('%r'!='<WEBMAIL-IP>' OR COALESCE(disable_webmail,0)!=1) \ 
                AND \ 
                ('%r'='<WEBMAIL-IP>' OR COALESCE(disable_imap,0)!=1)

# <WEBMAIL-IP> is the IP of your webmail 
# I'm assuming that the imap connection is only on port 993 and the pop3 connection is on port 995. 
# Adjust to your needs 
# 
# logically this means: 
# 
# ************************** USER LIMITS via vpopmail.pw_gid filed 
# SELECT user 
# WHEN POP is not disabled for that user connecting on port 995 (995 is the pop3s port allowed from remote in my configuration) 
# AND WHEN webmail access is not disabled for that user when connecting from <WEBMAIL-IP> 
# AND WHEN IMAP is not disabled for that user connecting on port 993 (993 is the imap port allowed from remote 
# in my configuration) unless his remote ip the one belonging to the webmail 
# 
# ************************* DOMAIN LIMITS via limits table 
# AND WHEN WEBMAIL access for the domain is not disabled 
# AND WHEN IMAP access for the domain is not disabled when not connecting from <WEBMAIL-IP>

# Query to get a list of all usernames.
iterate_query = SELECT CONCAT(pw_name,'@',pw_domain) AS user FROM vpopmail

Since this file stores the database password and it's open only by root, you must secure the access priviledges:

chmod go-wrx ../dovecot-sql.conf.ext

"One table per domain" query

The above solution has to be enhanced when you store your accounts in one table for each domain (--disable-many-domains), because you have to retrieve the real domain in case of aliases, guess the dbtable and then execute the password_query. This task can be accomplished by "stored procedures" (thanks to Pablo Murillo for sharing his sql example).

I have created a procedure named "dovecot_password_query_disable_many_domains" which does the sql stuff and can be called as follows in your password_query:

password_query = CALL dovecot_password_query_disable_many_domains('%n','%d','127.0.0.1','%r','%a')

To install the procedure you have to download and execute the code above as follows:

wget https://notes.sagredo.eu/files/qmail/patches/vpopmail/dovecot-pwd-query_disable-many-domains.txt
mysql < dovecot-pwd-query_disable-many-domains.txt -u root -p

10-director.conf

If you don't want to use the Director service rename the conf file, so that it will not be loaded:

mv 10-director.conf 10-director.conf.disabled

10-logging.conf

Set your log destination. I prefer to have a dedicated logfile for dovecot (default is syslog)

log_path = /var/log/dovecot/dovecot.log
plugin {
}

Logrotate

Save the below scriptlet as /etc/logrotate.d/dovecot:

cat > /etc/logrotate.d/dovecot << __EOF__
/var/log/dovecot/*.log {
missingok
notifempty
delaycompress
sharedscripts
postrotate
/usr/local/dovecot/bin/doveadm log reopen
endscript
}
__EOF__

10-mail.conf

We have to tell dovecot which is the mailbox location. It will look for the Maildir in the home directory (%h):

mail_location = maildir:%h/Maildir
namespace inbox {
  inbox = yes
}

Set to 89 (vpopmail userid number) the mail_uid and mail_gid. The same for the valid first/last id as you want to manage only vpopmail users:

mail_uid = 89
mail_gid = 89

mail_privileged_group = 89
mail_access_groups = 89

first_valid_uid = 89
last_valid_uid = 89

first_valid_gid = 89
last_valid_gid = 89

Create the run folder:

mkdir -p /usr/local/dovecot/var/run/dovecot

Adjust the socket and the plugin folder:

auth_socket_path = /usr/local/dovecot/var/run/dovecot/auth-userdb
mail_plugin_dir = /usr/local/dovecot/lib/dovecot
mail_plugins = $mail_plugins quota

Finally, this has to be enabled because of the troubles caused by a broken maildir++ patch (now repaired, tx MG)

##
## Maildir-specific settings
##

# If enabled, Dovecot doesn't use the S=<size> in the Maildir filenames for
# getting the mail's physical size, except when recalculating Maildir++ quota.
# This can be useful in systems where a lot of the Maildir filenames have a
# broken size. The performance hit for enabling this is very small.
maildir_broken_filename_sizes = yes

10-master.conf

Read carefully this before continuing.

#default_process_limit = 100
#default_client_limit = 1000

# Default VSZ (virtual memory size) limit for service processes. This is mainly
# intended to catch and kill processes that leak memory before they eat up
# everything.
#default_vsz_limit = 256M

# Login user is internally used by login processes. This is the most untrusted
# user in Dovecot system. It shouldn't have access to anything at all.
default_login_user = vpopmail

# Internal user is used by unprivileged processes. It should be separate from
# login user, so that login processes can't disturb other processes.
default_internal_user = dovecot

service imap-login {
  inet_listener imap {
    #port = 143
  }
  inet_listener imaps {
    #port = 993
    #ssl = yes
  }

  # Number of connections to handle before starting a new process. Typically
  # the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
  # is faster. <doc/wiki/LoginProcess.txt>
  #service_count = 1

  # Number of processes to always keep waiting for more connections.
  #process_min_avail = 0

  # If you set service_count=0, you probably need to grow this.
  #vsz_limit = $default_vsz_limit
}

service pop3-login {
  inet_listener pop3 {
    #port = 110
  }
  inet_listener pop3s {
    #port = 995
    #ssl = yes
  }
}

service lmtp {
  unix_listener lmtp {
    #mode = 0666
  }

  # Create inet listener only if you can't use the above UNIX socket
  #inet_listener lmtp {
    # Avoid making LMTP visible for the entire internet
    #address =
    #port = 
  #}
}

service imap {
  # Most of the memory goes to mmap()ing files. You may need to increase this
  # limit if you have huge mailboxes.
  #vsz_limit = $default_vsz_limit

  # Max. number of IMAP processes (connections)
  #process_limit = 1024
}

service pop3 {
  # Max. number of POP3 processes (connections)
  #process_limit = 1024
}

service auth {
  # auth_socket_path points to this userdb socket by default. It's typically
  # used by dovecot-lda, doveadm, possibly imap process, etc. Users that have
  # full permissions to this socket are able to get a list of all usernames and
  # get the results of everyone's userdb lookups.
  #
  # The default 0666 mode allows anyone to connect to the socket, but the
  # userdb lookups will succeed only if the userdb returns an "uid" field that
  # matches the caller process's UID. Also if caller's uid or gid matches the
  # socket's uid or gid the lookup succeeds. Anything else causes a failure.
  #
  # To give the caller full permissions to lookup all users, set the mode to
  # something else than 0666 and Dovecot lets the kernel enforce the
  # permissions (e.g. 0777 allows everyone full permissions).
  unix_listener auth-userdb {
    mode = 0600
    user = vpopmail
    group = vchkpw
  }

  # Postfix smtp-auth
  #unix_listener /var/spool/postfix/private/auth {
  #  mode = 0666
  #}

  # Auth process is run as this user.
  user = $default_internal_user
}

service auth-worker {
  # Auth worker process is run as root by default, so that it can access
  # /etc/shadow. If this isn't necessary, the user should be changed to
  # $default_internal_user.
  user = $default_internal_user
}

service dict {
  # If dict proxy is used, mail processes should have access to its socket.
  # For example: mode=0660, group=vmail and global mail_access_groups=vmail
  unix_listener dict {
    #mode = 0600
    #user = 
    #group = 
  }
}

# The following lines concerning stats service's configuration are not part of the example configuration file shipped by Dovecot.
# I'm adding them becasue in v.2.3.9.3 I found a "Permission denied" error for /var/run/dovecot/stats-writer in my qmail-send log
service stats {
    unix_listener stats-reader {
    user = vpopmail
    group = vchkpw
    mode = 0660
  }
  unix_listener stats-writer {
    user = vpopmail
    group = vchkpw
    mode = 0660
  }
}

10-ssl.conf

Of course we want SSL support. First of all let's create a self-signed SSL certificate. Dovecot includes a script to build self-signed SSL certificates using OpenSSL; you can find it in doc/mkcert.sh:

cd /usr/local/src/dovecot/doc

mkcert.sh will create the SSL certificate. Before running mkcert.sh you have to customize the cert editing dovecot-openssl.cnf (in the same folder):

> nano dovecot-openssl.cnf

[ req ]
default_bits = 4096
encrypt_key = yes
distinguished_name = req_dn
x509_extensions = cert_type
prompt = no

[ req_dn ]
# country (2 letter code)
C=IT

# State or Province Name (full name)
ST=Italy

# Locality Name (eg. city)
L=MyLocality

# Organization (eg. company)
O=My Organization Name

# Organizational Unit Name (eg. section)
OU=IMAP server

# Common Name (*.example.com is also possible)
CN=mymailserver.net

# E-mail contact
emailAddress=postmaster@mymailserver.net

[ cert_type ]
nsCertType = server

Now we are ready to create the SSL certificate:

> sh mkcert.sh

and the files /etc/ssl/private/dovecot.pem and /etc/ssl/certs/dovecot.pem have been created. Remember their locations because you have to insert them in 10-ssl.conf:

ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem

Remember also to switch off SSLv3 because of security reasons:

ssl_min_protocol = TLSv1

Now generate new SSL DH parameters from command line with

openssl dhparam -out /usr/local/dovecot/etc/dh.pem 4096

and adjust the ssl_dh parameter accordingly:

ssl_dh = </usr/local/dovecot/etc/dh.pem

Installing a valid certificate

When your configuration is finished and you have your server working, it will be the case to install a valid certificate, following the directives from the page "Installing a valid SSL certificate". Once you obtain the certificate, you just have to adjust these two lines:

ssl_cert = </etc/letsencrypt/live/smtp.yourdomain.tld/fullchain.pem
ssl_key = </etc/letsencrypt/live/smtp.yourdomain.tld/privkey.pem

15-lda.conf

As I already said dovecot will also act as an LDA because we want to manage sieve rules and filter our incoming emails.

postmaster_address = postmaster@yourdomain.net
hostname = mail.yourdomain.net

submission_host = 0.0.0.0:25

# the next two to auto create folders
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes

protocol lda {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins quota # if you want to use sieve put sieve here
}

15-mailboxes.conf

The autoexpunge setting assures the expunge of Trash and Junk folders after a certain number of days.

##
## Mailbox definitions
##

# NOTE: Assumes "namespace inbox" has been defined in 10-mail.conf.
namespace inbox {

  #mailbox name {
    # auto=create will automatically create this mailbox.
    # auto=subscribe will both create and subscribe to the mailbox.
    #auto = no

    # Space separated list of IMAP SPECIAL-USE attributes as specified by
    # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash
    #special_use =
  #}

  # These mailboxes are widely used and could perhaps be created automatically:
  mailbox Drafts {
    special_use = \Drafts
    auto = subscribe
  }
  mailbox Junk {
    special_use = \Junk
    auto = subscribe
    autoexpunge = 30d
  } 
  mailbox "Junk.TeachSpam" { 
    auto = subscribe 
    autoexpunge = 5d 
  } 
  mailbox "Junk.TeachNotSpam" { 
    auto = subscribe 
    autoexpunge = 30d 
  }
  mailbox Trash {
    special_use = \Trash
    auto = subscribe
    autoexpunge = 60d
  }

  # For \Sent mailboxes there are two widely used names. We'll mark both of
  # them as \Sent. User typically deletes one of them if duplicates are created.
  mailbox Sent {
    special_use = \Sent
    auto = subscribe
  }
#  mailbox "Sent Messages" {
#    special_use = \Sent
#  }

  # If you have a virtual "All messages" mailbox:
  #mailbox virtual/All {
  #  special_use = \All
  #}

  # If you have a virtual "Flagged" mailbox:
  #mailbox virtual/Flagged {
  #  special_use = \Flagged
  #}
}

20-imap.conf

protocol imap {
  mail_plugins = $mail_plugins imap_quota
}

20-lmtp.conf

I don't want the support for proxying to other LMTP/SMTP servers, so I disable lmtp:

mv 20-lmtp.conf 20-lmtp.conf.disabled

20-pop3.conf

protocol pop3 {
  mail_plugins = $mail_plugins quota
}

20-submission.conf

If the submission service is not needed we can disable its configuration file:

mv 20-submission.conf 20-submission.conf.disabled

90-acl.conf

If no acl is needed you can disable it:

mv 90-acl.conf 90-acl.conf.disabled

90-plugin.conf

##
## Plugin settings
##

# All wanted plugins must be listed in mail_plugins setting before any of the
# settings take effect. See <doc/wiki/Plugins.txt> for list of plugins and
# their configuration. Note that %variable expansion is done for all values.

plugin {
#setting_name = value
}

90-quota.conf

Enable maildir++ in this way

quota = maildir::

Adjusting PATH and MANPATH

You may want to modify your environment variable as follows inside your /etc/profile:

export PATH=$PATH:/usr/local/dovecot/bin
export MANPATH=$MANPATH:/usr/local/dovecot/share/man

Comments

QUOTA backend defined incorrectly

Incorrect Maildir++ backend definition in the documentation.

Spent ages on trying to find out why QUOTA is not working on a recent version of Dovecot installation (2.3.13).

Don't use this quota definition for Maildir++ quota backend:

plugin { 
quota = maildir:User quota
}

This will break email clients!!! Read the documentation: https://wiki.dovecot.org/Quota/Configuration#Quota_Configuration

The BAD BAD BAD format above is supposed to fix Apple mail apparently. But it breaks everything else!!! Don't even know if it actually helps Apple mail...

The proper way of defining it is like this:

plugin { 
quota = maildir::
}

As per the documentation the format is: 

quota = [:[:]]

In essence the Name/quota root name part doesn't really mean anything it should remain empty. Maildir++ doesn't need any backend args hence it's empty as well.

The following formats will work as well:

plugin { 
quota = maildir
}
plugin { 
quota = maildir:
}

Just don't use the "quota = maildir:User quota:quota" literally (copy/paste)! It won't work.

Testing

Basically - there are two tools that you can use to check what dovecot thinks about your quota:

- doveadm quota -u

- Login over telnet IMAP and issue: GETQUOTA ""

About the tools doveadm quota

For whatever reason doveadm quota -u missreports the values. In my case it shows values 1000 times lower. This means that instead of showing 50G it shows 50M. This one liner shows it best:

# doveadm quota get -u  |grep "STORAGE" | awk '{print $3}' |numfmt --to=iec
50M

# vuserinfo  |grep "quota:" | awk '{print $2}' |sed 's/S//g'| numfmt --to=iec
50G

I don't know why it does it yet. I don't know why it doesn't show % of usage either when the manual for the command does show it. Maybe usage % is only shown when running against an interrative query (-A) option. Will check that later. It's useful to also run it with -D for debbuging but that's for another time.

IMAP session over telnet or netcat (optional SSL)

You can test IMAP using telnet (if you allow non encrypted logins to your server or you can use a recent version of nc (netcat) eg. nmap-ncat which supports the --ssl switch. 

You can login like this:

$ nc -v --ssl imap.example.com 993 
Ncat: Version 7.80 ( https://nmap.org/ncat )
Ncat: SSL connection to X.X.X.X:993.
Ncat: SHA-1 fingerprint: CF81 7B80 4D08 D4B8 3338 D1E6 986E 222F 26E3 E740
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN] Welcome to X IMAP!
a LOGIN
a OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE QUOTA] Logged in
a GETQUOTA ""
* QUOTA "" (STORAGE 4806876 52428800)
a OK Getquota completed (0.001 + 0.000 secs).
Ncat: 70 bytes sent, 699 bytes received in 38.94 seconds.

There are 2 important things up there:

1. the capabilities after login show QUOTA

2. result to GETQUOTA "" command returned: * QUOTA "" (STORAGE 4806876 52428800)

Great! That's the desired result!

Reply | Permalink

QUOTA backend defined incorrectly

Thank you for the contribution, as it contains several things that I didn't know.

The maildir settings that I use are reported at the page that you linked above and also in the example confg file that dovecot installs in share/doc/dovecot/example-config/, so I'm not sure that this is an incorrect definition.

Your suggested tests both via imap and via doveadm succeded here, but in the imap test I had to provide GETQUOTAROOT instead of GETQUOTA. Do you think this is normal?

EDIT: using your suggested settings I can see my quota via IMAP with the GETQUOTA command as well.

Reply | Permalink

Fix instruccionts

The commnet:

ln -s dovecot-2.3.9.2 dovecot

should be:

ln -s dovecot-2.3.9.3 dovecot

Reply | Permalink

Fix instructions

Thank you. Corrected

Reply | Permalink

Link broken

Hello Roberto,

Thanks for your work, it help me, to using CentOS 8.1 with qmail, vpopmail, dovecot, etc

When I finished, I will post more comments with my distribution.

In this page, I found a link broken: https://notes.sagredo.eu/files/qmail/dovecot.conf.tar.gz

I found the correct one: https://notes.sagredo.eu/files/qmail/dovecot.tar.gz

This is a cosmetic error, i know.

Thanks again !

Marco Varanda - from Brazil - São Paulo

Reply | Permalink

Link broken

Thank you. Corrected

Reply | Permalink

Quota Issues with SQL

I had some quota issues when using auth-sql.conf.ext instead of auth-vpopmail.conf.ext.

I was able to fix it by changing dovecot-sql.conf.ext with quota checking next to user_query:

user_query = \
   SELECT pw_dir AS home, \
   89 AS uid, \
   89 AS gid, \
   concat ('*: bytes =', trim (TRAILING 'S' FROM pw_shell)) AS quota_rule \
   FROM vpopmail \
   WHERE pw_name = '% n' AND pw_domain = '% d'

Regards,

Reply | Permalink

Quota Issues with SQL

In my MariaDB 10.0 server I had to avoid that blank space after TRIM, otherwise that will throw the query into an error, so

CONCAT('*: bytes =', TRIM(TRAILING 'S' FROM pw_shell)) AS quota_rule

In addition, once the query was executed, dovecot complains in this way:

Error: Failed to initialize quota: Invalid quota root quota: Invalid rule *: bytes =NOQUOTA: Unknown rule limit name: bytes

Then I managed to get rid of this error avoiding the blank spaces in the dovecot's quota_rule as well, therefore the query becomes

CONCAT('*:bytes=', TRIM(TRAILING 'S' FROM pw_shell)) AS quota_rule

Unfortunately I still get this error in case of users with NOQUOTA, I think because dovecot expects 0 as a code for unlimited quota.

Error: Failed to initialize quota: Invalid quota root quota: Invalid rule *:bytes=NOQUOTA: Invalid bytes limit: NOQUOTA

Instead your query works fine here for users with a limited quota.

Any hint on how to adjust the query in order to eventually replace NOQUOTA with 0?

Reply | Permalink

Quota Issues with SQL

CONCAT('*:bytes=', REPLACE(TRIM(TRAILING 'S' FROM pw_shell), 'NOQUOTA', '0')) AS quota_rule

this seems to work now...

Reply | Permalink

Quota Issues with SQL

indeed!

worked perfectly with noquota users too.

I use mariadb 10.1.41, it doesnt had the problems with blank spaces in TRIM()

Regards,

Reply | Permalink

Quota Issues with SQL

I tried the TRIM and TRAILING in user_query as mentioned, but it doesn't work for me.  The query cannot remove the string start from position of "S" to the end and cause error.  I did rework the query to:

CONCAT('*:bytes=', REPLACE(SUBSTRING_INDEX(pw_shell, 'S', 1), 'NOQUOTA', '0')) AS quota_rule \

That's work without error and extract the quota value correctly.  I am using MariaDB version 10.3.11.  Hope can share this to help others have issues.

Reply | Permalink

Quota Issues with SQL

This seems to work fine with mariadb-10.0. I assume that there are problems with v.10.3. I'm going to accept your solution and change my documentation

Thank you

Reply | Permalink

10-ssl.conf needs key of at least 4096 instead of 1024

Hi,

When creating the key (in part 10-ssl.cnf) you have to increase the limit of 1024 to 4096 otherwise ssl throws an error.

nano dovecot-openssl.cnf:

default_bits = 1024 has to be changed to:

default_bits = 4096

Reply | Permalink

10-ssl.conf needs key of at least 4096 instead of 1024

thank you

Reply | Permalink

Dovecot version 2.3.1 seems not to work with libc-2.28,

Hi,

after compiling dovecot version 2.3.1 I got the following in my log's:

Jul 31 11:25:31 imap kernel: [10964.853497] auth[804]: segfault at ffffffffd3aff200 ip 00007f60d3420a4a sp 00007ffe3de1ae68 error 5 in libc-2.28.so[7f60d33b2000+148000]
Jul 31 19:53:05 imap kernel: [41417.391192] auth[1131]: segfault at a3d200 ip 00007f050035ea4a sp 00007ffd3ddadec8 error 4 in libc-2.28.so[7f05002f0000+148000]
Jul 31 19:54:10 imap kernel: [41482.622418] auth[1134]: segfault at 4553b200 ip 00007f6744e5ca4a sp 00007ffcaff80cf8 error 4 in libc-2.28.so[7f6744dee000+148000]
Jul 31 19:55:43 imap kernel: [41575.954630] auth[1137]: segfault at ffffffffe3567200 ip 00007f6de2e88a4a sp 00007ffe4908c128 error 5 in libc-2.28.so[7f6de2e1a000+148000]
Jul 31 20:25:55 imap kernel: [43387.112967] auth[1223]: segfault at 7823c200 ip 00007fcd77b5da4a sp 00007ffcb28cc498 error 4 in libc-2.28.so[7fcd77aef000+148000]
Jul 31 20:59:17 imap kernel: [ 97.389263] auth[697]: segfault at ffffffffcccc8200 ip 00007feecc5e9a4a sp 00007ffc8aa3d938 error 5 in libc-2.28.so[7feecc57b000+148000]
Jul 31 21:05:29 imap kernel: [ 469.004309] auth[702]: segfault at 66b7b200 ip 00007f0c6649ca4a sp 00007ffde78e7df8 error 4 in libc-2.28.so[7f0c6642e000+148000]

After compiling latest Dovecot (version 2.3.7.1) everything worked.

PS this is on Debian Buster.

Reply | Permalink

cannot start up dovecot

Having this error. dovecot: Can't open log file /var/log/dovecot/dovecot.log: Read-only file system

Reply | Permalink

cannot start up dovecot

are you using my setup? in this case double check the priviledges in 10-master.conf

Reply | Permalink

cannot start up dovecot

Hi Roberto,

Replaced your 10-master.conf with the default and now another error.

10-ssl.conf line 6: Unknown setting: service { ssl

Line 6 in 10-ssl.conf is

ssl = yes

I replaced again with your 10-ssl.conf and still the same

thanks
nic

Reply | Permalink

cannot start up dovecot

If I grep the dovecot conf dir, there's no "service { ssl" line at all...

can you post the content of doveconf -n on pastebin?

Reply | Permalink

Error:_net_connect_unix(/usr/local/dovecot/var/run/dovecot/stats-writer)_failed:_Permission_denied

If you see errors like this at /var/log/qmail/send/current:

Error:_net_connect_unix(/usr/local/dovecot/var/run/dovecot/stats-writer)_failed:_Permission_denied

You need to edit /usr/local/dovecot/etc/dovecot/dovecot.conf and add:

service stats {
unix_listener stats-reader {
user = vpopmail
group = vchkpw
mode = 0660
}

unix_listener stats-writer {
user = vpopmail
group = vchkpw
mode = 0660
}

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

Hi there Roberto,

My system is CentOS-7 64bit, and runs as an LXC container under Proxmox on a dedicated server I rent. The domain for MX is mail.MYDOMAIN.TLD and the reverse lookup is working fine.

I've been pulling my hair out since yesterday with being able to log in from my mail client to Dovecot. I have downloaded and used your default config files for etc and modified them according to my TLD. But everytime I try to log in, the dovecot.log reports:

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

(i substituted my real IPs with hashes for privacy).

The user=<> is blank? Do you think I have an issue with vpopmail? I am able to add/remove domains and users using the vQAdmin or just by prompt without any problems, and the SQL database is updating accordingly.

The service is running as follows:

[root@######conf.d]# ps axfu
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.2 124976 4912 ? Ss 11:17 0:00 /sbin/init
root 37 0.0 0.2 36844 5652 ? Ss 11:17 0:00 /usr/lib/systemd/systemd-journald
root 48 0.0 0.1 24216 2768 ? Ss 11:17 0:00 /usr/lib/systemd/systemd-logind
dbus 50 0.0 0.1 24408 2820 ? Ss 11:17 0:00 /bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
root 52 0.0 0.2 212132 4944 ? Ssl 11:17 0:00 /usr/sbin/rsyslogd -n
root 54 0.0 0.0 6468 1688 pts/0 Ss+ 11:17 0:00 /sbin/agetty --noclear --keep-baud pts/0 115200 38400 9600 vt220
root 55 0.0 0.0 6468 1572 lxc/console Ss+ 11:17 0:00 /sbin/agetty --noclear --keep-baud console 115200 38400 9600 vt220
root 56 0.0 0.1 22656 2740 ? Ss 11:17 0:00 /usr/sbin/crond -n
root 60 0.0 0.0 6468 1564 pts/1 Ss+ 11:17 0:00 /sbin/agetty --noclear --keep-baud pts/1 115200 38400 9600 vt220
root 62 0.0 0.1 9524 2176 ? Ss 11:17 0:00 /bin/sh /usr/local/bin/svscanboot /dev/ttyS0
root 68 0.0 0.0 4360 1180 ? S 11:17 0:00 \_ svscan /var/service
root 70 0.0 0.0 4188 600 ? S 11:17 0:00 | \_ supervise vpopmaild
root 1268 0.0 0.0 4232 752 ? S 12:27 0:00 | | \_ /usr/local/bin/tcpserver -v -H -R -l 0 -u 0 -g 0 0 89 /home/vpopmail/bin/vpopmaild
root 71 0.0 0.0 4188 656 ? S 11:17 0:00 | \_ supervise log
qmaill 79 0.0 0.0 4200 612 ? S 11:17 0:00 | | \_ /usr/local/bin/multilog t /var/log/qmail/vpopmaild
root 72 0.0 0.0 4188 664 ? S 11:17 0:00 | \_ supervise qmail-send
qmails 1260 0.0 0.0 10752 1288 ? S 12:27 0:00 | | \_ qmail-send
root 1270 0.0 0.0 4364 624 ? S 12:27 0:00 | | \_ qmail-lspawn | /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox
qmailr 1272 0.0 0.0 4360 712 ? S 12:27 0:00 | | \_ qmail-rspawn
qmailq 1273 0.0 0.0 4320 1104 ? S 12:27 0:00 | | \_ qmail-clean
qmails 1274 0.0 0.0 4344 1092 ? S 12:27 0:00 | | \_ qmail-todo
qmailq 1275 0.0 0.0 4320 1108 ? S 12:27 0:00 | | \_ qmail-clean
root 73 0.0 0.0 4188 668 ? S 11:17 0:00 | \_ supervise log
qmaill 82 0.0 0.0 4200 704 ? S 11:17 0:00 | | \_ /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/send
root 74 0.0 0.0 4188 704 ? S 11:17 0:00 | \_ supervise qmail-smtpd
vpopmail 1261 0.0 0.0 4232 692 ? S 12:27 0:00 | | \_ /usr/local/bin/tcpserver -v -H -R -l 0 -x /home/vpopmail/etc/tcp.smtp.cdb -c 20 -u 89 -g 89 0 25 /var/qmail/bin/qmail-smtpd
root 75 0.0 0.0 4188 620 ? S 11:17 0:00 | \_ supervise log
qmaill 87 0.0 0.0 4200 712 ? S 11:17 0:00 | | \_ /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/smtpd
root 76 0.0 0.0 4188 668 ? S 11:17 0:00 | \_ supervise qmail-submission
vpopmail 1264 0.0 0.0 4232 676 ? S 12:27 0:00 | | \_ /usr/local/bin/tcpserver -v -H -R -l 0 -x /home/vpopmail/etc/tcp.submission.cdb -c 20 -u 89 -g 89 0 587 /var/qmail/bin/qmail-smtpd /home/vpopmail/bin/vchkpw /bi
root 77 0.0 0.0 4188 712 ? S 11:17 0:00 | \_ supervise log
qmaill 90 0.0 0.0 4200 676 ? S 11:17 0:00 | \_ /usr/local/bin/multilog t s16000000 n200 /var/log/qmail/submission
root 69 0.0 0.0 4176 684 ? S 11:17 0:00 \_ readproctitle service errors: ..........................................................................................................................................
root 404 0.0 1.1 466252 24132 ? Ss 11:17 0:00 /usr/sbin/httpd -DFOREGROUND
apache 532 0.0 1.1 572184 24100 ? S 11:17 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 534 0.0 0.7 466384 15020 ? S 11:17 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 535 0.0 1.4 577624 29572 ? S 11:17 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 537 0.0 1.3 577368 29336 ? S 11:17 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 538 0.0 0.7 466384 15024 ? S 11:17 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 620 0.0 0.6 466252 14040 ? S 11:27 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 883 0.0 0.7 466384 15012 ? S 11:50 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 884 0.0 0.7 466384 15036 ? S 11:50 0:00 \_ /usr/sbin/httpd -DFOREGROUND
apache 885 0.0 0.7 466384 15004 ? S 11:50 0:00 \_ /usr/sbin/httpd -DFOREGROUND
root 405 0.0 0.3 106008 7380 ? Ss 11:17 0:00 /usr/sbin/sshd -D
root 591 0.0 0.4 141656 9400 ? Ss 11:21 0:00 \_ sshd: root@pts/2
root 593 0.0 0.1 11924 3084 pts/2 Ss 11:21 0:00 \_ -bash
root 1308 0.0 0.1 47588 3408 pts/2 R+ 12:47 0:00 \_ ps axfu
root 407 0.0 0.1 176016 2120 ? Sl 11:17 0:00 /home/vpopmail/bin/vusaged
mysql 449 0.0 0.1 9528 2524 ? Ss 11:17 0:00 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
mysql 554 0.0 4.3 1038868 90552 ? Sl 11:17 0:02 \_ /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/lib/mysql/mangve.info.err --pid-file=mangve.info.pid
root 1091 0.0 0.1 12156 2472 ? Ss 12:12 0:00 /usr/local/dovecot/sbin/dovecot
dovecot 1092 0.0 0.1 9960 2576 ? S 12:12 0:00 \_ dovecot/anvil
root 1093 0.0 0.1 10092 2596 ? S 12:12 0:00 \_ dovecot/log
root 1094 0.0 0.2 20272 4364 ? S 12:12 0:00 \_ dovecot/config
dovecot 1289 0.0 0.3 142648 7660 ? S 12:27 0:00 \_ dovecot/auth

I've created the required pem files also.

Any help with where I should be looking to debug this would be greatly appreciated.

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

Hi Nick, did you compile vpopmail with --enable-auth-module=mysql? Did you pass the vpopmail test from command line? If yes, please post the logs and an imap telnet session as explained in the "testing dovecot" page

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

Hi again,

Just to add: As for vpopmail, i tested SMTP  on 25 using swaks and it worked fine.

And then vpop_auth, by telnet to 89:

[...] 

However!! SMTP on 587 seems to hang!

[root@mangve qmail-submission]# telnet 176.31.140.146 587
Trying 176.31.140.146...
Connected to 176.31.140.146.
Escape character is '^]'.
220 mail.mangve.info ESMTP

(then nothing forever!)

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

check also the GREETDELAY value in your run file...

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

this is a separate issue, I suggest to che check the logs. Please post a comment on the qmail/configuration page if you like

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

Hi Roberto,

Many thanks for your prompt and kind help.

Yes, I compiled vpopmail with the mysql authentification module, and have tested it localy from prompt. Please see below my sucess with it (only locally):

root@mangve dovecot]# telnet 176.31.140.146 143
Trying 176.31.140.146...
Connected to 176.31.140.146.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login nick@mangve.info PASSWORDHIDDEN
a OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE LITERAL+ NOTIFY QUOTA] Logged in
a select inbox
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1519135948] UIDs valid
* OK [UIDNEXT 1] Predicted next UID
a OK [READ-WRITE] Select completed (0.001 + 0.000 secs).
a list "" *
* LIST (\HasNoChildren) "." INBOX
a OK List completed (0.001 + 0.000 secs).
a logout
* BYE Logging out
a OK Logout completed (0.001 + 0.000 secs).
Connection closed by foreign host.

Below is my dovecot.log output from the above session:

Feb 20 14:26:07 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=PBxol6Vl8sSwH4yS lip=176.31.140.146 rip=176.31.140.146 lport=143 rport=50418 resp=AG5pY2tAbWFuZ3ZlLmluZm8AcjNuYWlzc2FuY2U= (previous base64 data may contain sensitive data)
Feb 20 14:26:07 auth-worker(20163): Debug: Loading modules from directory: /usr/local/dovecot/lib/dovecot/auth
Feb 20 14:26:07 auth-worker(20163): Debug: Module loaded: /usr/local/dovecot/lib/dovecot/auth/lib20_auth_var_expand_crypt.so
Feb 20 14:26:07 auth-worker(20163): Debug: sql(nick@mangve.info,176.31.140.146,): query: SELECT CONCAT(pw_name, '@', 'mangve.info') AS user, pw_passwd AS password, pw_dir as userdb_home, 89 AS userdb_uid, 89 AS userdb_gid FROM `vpopmail` WHERE pw_name = 'nick' AND pw_domain = 'mangve.info'
Feb 20 14:26:07 auth: Debug: client passdb out: OK 1 user=nick@mangve.info
Feb 20 14:26:07 auth: Debug: master in: REQUEST 2776629249 20160 1 f66d64a38fe65c15d3866c47df933597 session_pid=20166 request_auth_token
Feb 20 14:26:07 auth-worker(20163): Debug: sql(nick@mangve.info,176.31.140.146,): SELECT pw_dir AS home, 89 AS uid, 89 AS gid FROM vpopmail WHERE pw_name = 'nick' AND pw_domain = 'mangve.info' AND ('143'!='995' or !(pw_gid & 2)) AND ('176.31.140.146'!='176.31.140.145' or !(pw_gid & 4)) AND ('176.31.140.146'='176.31.140.145' or '143'!='993' or !(pw_gid & 8))
Feb 20 14:26:07 auth: Debug: master userdb out: USER 2776629249 nick@mangve.info home=/home/vpopmail/domains/mangve.info/nick uid=89 gid=89 auth_token=25f7d7d36ca630b8471a9c9b2e2f7a33fbf20cab
Feb 20 14:26:07 imap-login: Info: Login: user=<nick@mangve.info>, method=PLAIN, rip=176.31.140.146, lip=176.31.140.146, mpid=20166, secured, session=
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Loading modules from directory: /usr/local/dovecot/lib/dovecot
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Module loaded: /usr/local/dovecot/lib/dovecot/lib10_quota_plugin.so
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Module loaded: /usr/local/dovecot/lib/dovecot/lib11_imap_quota_plugin.so
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/mangve.info/nick
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Quota root: name=User quota backend=maildir args=
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Quota grace: root=User quota bytes=0 (10%)
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/vpopmail/domains/mangve.info/nick/Maildir
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: maildir++: root=/home/vpopmail/domains/mangve.info/nick/Maildir, index=, indexpvt=, control=, inbox=/home/vpopmail/domains/mangve.info/nick/Maildir, alt=
Feb 20 14:26:07 imap(nick@mangve.info)<20166>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping
Feb 20 14:26:25 imap(nick@mangve.info)<20166>: Debug: INBOX: Mailbox opened because: SELECT
Feb 20 14:26:27 auth: Debug: auth client connected (pid=20169)
Feb 20 14:26:36 pop3-login: Info: Disconnected (no auth attempts in 9 secs): user=<>, rip=192.241.213.179, lip=176.31.140.145, session=
Feb 20 14:26:42 imap(nick@mangve.info)<20166>: Info: Logged out in=37 out=794 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

So dovecot/vpopmail is fine. Check out your client's configuration

Reply | Permalink

imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=#.#.#.#, lip=#.#.#.#, session=

I see that qmail seems to be issue with login:

[root@mangve qmail-submission]# telnet 176.31.140.146 587
Trying 176.31.140.146...
Connected to 176.31.140.146.
Escape character is '^]'.
220 mail.mangve.info ESMTP
EHLO test
250-mail.mangve.info
250-STARTTLS
250-PIPELINING
250-8BITMIME
250 SIZE 20000000
AUTH PLAIN
538 auth not available without TLS (#5.3.3)
STARTTLS
220 ready for tls
AUTH PLAIN
454 TLS connection failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol (#4.3.0)
Connection closed by foreign host.

Reply | Permalink

Dovecot vpopmail userdb and passdb without SQL backend.

Some fixes required in your config files.

I've installed dovecot and added just auth-vpopmail.conf.ext in 10-auth.conf

But authentication was broken. (always unknown user)

After some investigation i found that auth-worker works with "default_internal_user" (which is dovecot) with no access to vpopmail domains.

So i suugest to add dovecot to "vchkpw" group or swithch auth-worker to vpopmail user...

Thanks.

Reply | Permalink

Dovecot vpopmail userdb and passdb without SQL backend.

My configuration concerns the sql auth driver, so these fixes are not needed. Of course some adjustments would  be needed if using the vpopmail driver

Reply | Permalink

Dovecot vpopmail userdb and passdb without SQL backend.

Sorry, I forget it. You must change 'service auth' user too, if you need vpopmail driver.

from:

service auth {
  unix_listener auth-userdb {
    mode = 0600
    user = vpopmail
    group = vchkpw
  }
  user = $default_internal_user
}

to:

service auth {
  unix_listener auth-userdb {
    mode = 0600
    user = vpopmail
    group = vchkpw
  }
  user = vpopmail
}

Reply | Permalink

Dovecot vpopmail userdb and passdb without SQL backend.

If you would like to use the vpopmail driver, you need to czhange auth-worker service in 10-master.conf

from:

service auth-worker {
   user = $default_internal_user
}

to:

service auth-worker {
  user = vpopmail
  unix_listener auth-worker {
    user = vpopmail
  }
}

Reply | Permalink

openssl dhparam -out /usr/local/dovecot/etc/dh.pem

Requires the bits specifying: 1024

i.e.: # openssl dhparam -out /usr/local/dovecot/etc/dh.pem 1024

Reply | Permalink

yes, thank you

Reply | Permalink

maildir++ patch and dovecot

The maildir++ patch to calculate the size for quota enforcement is actually broken. This causes issues with newer versions of Dovecot, it will complain about S= being wrong and either bail or try to fix it one at a time. You can fix this by setting this in 10-mail.conf:

maildir_broken_filename_sizes = yes

There's a post about it here: http://dovecot.2317879.n4.nabble.com/Maildir-filename-has-wrong-S-value-td22996.html#a23000 and a patch to fix it in the FreeBSD port of qmail (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168870)  - perhaps the same patch could be pulled into your huge patch. 

Reply | Permalink

maildir++ patch and dovecot

It's worth to notice what it's suggested in the thread that you linked above... the maildirs can be restored in this way (it can take a long time if done for all (-A) users):

doveadm fetch -A text all > /dev/null

In this case the additional option maildir_broken_filename_sizes = yes is not needed

Reply | Permalink

maildir++ patch and dovecot

I have just updated my big patch and corrected the dovecot config (2.3!) accordingly. Thanks again for the hint

Reply | Permalink

maildir++ patch and dovecot

Thank you. I'll check it out

Reply | Permalink

dovecot configure

Dear Roberto

Thank you for this thorough qmail guide. Its the best available...

I am building a new server 10 years after the last time I did it. It's so wonderful to have these step-by-step instructions. It really saves the day.

Concerning the dovecot configuration here are some minor remarks:

Probably the ssl_dh_parameters_length  in 10-ssl.conf should become 2048 to comply with current security recommendations.

Additionaly if someone wants to enable the dovecot service in a systemd environment (eg Debian jessie) the proposed way is

to configure the dovecot sources --with-systemdsystemunitdir=/lib/systemd/system

Thanks again  and good night from Greece.

Reply | Permalink

dovecot configure

configure: WARNING: unrecognized options: --with-systemdsystemunitdir

Should it just be  "systemdsystemunitdir=/lib/systemd/system" ?

Reply | Permalink

dovecot configure

Yes, you are right. Corrected

Reply | Permalink

Thanks Bob, I just added your

Thanks Bob, I've just added your remarks to this page

Reply | Permalink

Just gone in production !

Hi everybody,

just to let you know after a few month of testing I have successfully migrated my old qmailrocks / fedora 6 istallation on a new,

virtualized Centos 6.5 - 64 bit qmail+vpopmail+roundcube+all the other goodies here described.

Everything works fine ! a big thank you to Roberto for this great guide !

BTW, I'm using dovecot 2.2.13 and it works perfectly.

The only problem experienced is related to some POP3 clients re-downloading all the email, even if I have used the courier-dovecot-migrate.pl script; mostly some MAC+thunderbird.

Cheers up !

Reply | Permalink

Table 'vpopmail.vpopmail' doesn't exist

Hi,

Has anyone tried with dovecot 2.2.4/2.2.6?
I am having a mysql issue

Oct 27 04:16:46 auth-worker(464): Warning: mysql: Query failed, retrying: Table 'vpopmail.vpopmail' doesn't exist
Oct 27 04:16:46 auth-worker(464): Error: sql(user@test.com,127.0.0.1): Password query failed: Table 'vpopmail.vpopmail' doesn't exist

I had tested pop3 login without any issue. Should i downgrade to 2.2.2?

Thanks
nic

Reply | Permalink

I'm testing version 2.2.6,

I'm testing version 2.2.6, telnetting port 143, sql driver, and the auth works fine

Reply | Permalink

Hi,I had tried 2.2.6 and

Hi,

I had tried 2.2.6 and still having the same error.
My backend is mysql-5.5.34.

ls /var/lib/mysql/vpopmail/
total 64
-rw-rw---- 1 mysql mysql 8836 Oct 27 05:38 subdomain_domain_com.frm
-rw-rw---- 1 mysql mysql   65 Oct 27 02:35 db.opt
-rw-rw---- 1 mysql mysql 9248 Oct 27 03:48 dir_control.frm
-rw-rw---- 1 mysql mysql 8674 Oct 27 03:49 lastauth.frm
-rw-rw---- 1 mysql mysql 8836 Oct 27 03:49 test_com.frm
-rw-rw---- 1 mysql mysql 8832 Oct 27 05:00 vlog.frm

Any idea?

regards
 nic

Reply | Permalink

I correct myself. It appears

I correct myself. It appears that you configured vpopmail to use separate tables for each domain, so you have to modify the auth query accordingly, since it is looking for a common table 'vpopmail'

Reply | Permalink

Hi,Understand nowMany

Hi,

Understand now

Many thanks

nic

Reply | Permalink

Ok, please post the query

Ok, please post the query that should be used in case of many domains if you can

Reply | Permalink

Sorry for the late reply.

Sorry for the late reply.
I had setup more toasters and also tested with different versions of dovecot up to 2.2.23.
No such error occur anymore. So i am posting my configuration for anyone who is interested to have an individual table for each domain.

Thank you

auth-sql.conf.ext
passdb {
driver = sql
# Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
args = /usr/local/dovecot/etc/dovecot/dovecot-sql.conf.ext
}

userdb {
driver = sql
args = /usr/local/dovecot/etc/dovecot/dovecot-sql.conf.ext
}

dovecot-sql.conf.ext
driver = mysql

connect = host=localhost dbname=vpopmail user=user password=password
#(please replace with your mysql username and password)

# passdb query to retrieve the password. It can return fields:
password_query = SELECT CONCAT(pw_name, '@', '%d') AS user, \
pw_passwd AS password, \
pw_dir as userdb_home, \
89 AS userdb_uid, \
89 AS userdb_gid \
FROM `vpopmail` \
WHERE pw_name = '%n' AND pw_domain = '%d'

# userdb query to retrieve the user information. It can return fields:
user_query = \
SELECT pw_dir AS home, \
89 AS uid, \
89 AS gid \
FROM vpopmail \
WHERE pw_name = '%n' AND pw_domain = '%d'

# Query to get a list of all usernames.
iterate_query = SELECT CONCAT(pw_name,'@',pw_domain) AS username FROM `vpopmail
}

Reply | Permalink

Nic, you really miss the

Nic, you really miss the vpopmail table. I have:

-rw-rw---- 1 mysql mysql      65 Nov  8  2012 db.opt
-rw-rw---- 1 mysql mysql    3509 Jul  2 13:15 dir_control.MYD
-rw-rw---- 1 mysql mysql    2048 Jul  2 16:03 dir_control.MYI
-rw-rw---- 1 mysql mysql    9248 Dec 12  2012 dir_control.frm
-rw-rw---- 1 mysql mysql    4185 Oct 27 08:22 lastauth.MYD
-rw-rw---- 1 mysql mysql    2048 Oct 27 08:22 lastauth.MYI
-rw-rw---- 1 mysql mysql    8674 Dec 12  2012 lastauth.frm
-rw-rw---- 1 mysql mysql       0 Dec 12  2012 valias.MYD
-rw-rw---- 1 mysql mysql    1024 Dec 12  2012 valias.MYI
-rw-rw---- 1 mysql mysql    8640 Dec 12  2012 valias.frm
-rw-rw---- 1 mysql mysql 8183044 Oct  1 14:43 vlog.MYD
-rw-rw---- 1 mysql mysql 3813376 Oct  1 16:08 vlog.MYI
-rw-rw---- 1 mysql mysql    8832 Dec 12  2012 vlog.frm
-rw-rw---- 1 mysql mysql    8970 Dec 12  2012 vpopmail.frm

create again the domain and check if it is created.

Reply | Permalink

Hi,I had deleted the existing

Hi,

I had deleted the existing domains and re-added them again.
vpopmail.frm is still no where to be found.

I had checked my other production servers, they are working fine
and yet there isn't any vpopmail.frm also.

Pretty weird. Google has no answer too.

regards
nic

Reply | Permalink

as said above, the problem

as said above, the problem should be the configuration of vpopmail with

--enable-many-domains     Creates a table for each virtual domain instead of storing all users in a single table. Only valid for MySQL and PostgreSQL

Reply | Permalink

Hi Roberto,I had configured

Hi Roberto,

I had configured vpopmail with --disable-many-domains

I had figured out if i use

!include auth-vpopmail.conf.ext

but not

!include auth-sql.conf.ext

I have no problem. Because the latest guide instructs to use auth-sql, Dovecot looks for vpopmail.frm instead.

I don't know by doing is this correct. Please correct me.

Many thanks again

regards
nic

Reply | Permalink

yes if you use the vpopmail

yes if you use the vpopmail driver you don't have to adjust the sql query. I switched to the mysql/sql drivers since the vpomail one was not working for me, but it's interesting to know that you have it working

Reply | Permalink

I tested with mariadb 5.5,

I tested with mariadb 5.5, that is pretty much the same as mysql 5.5.

But.. are you sure that the table was actually created?

Reply | Permalink

Then it must be me. Let me

Then it must be me. Let me try re-installing it again

Thanks

nic

Reply | Permalink

Dovecot-2.2.0 and Vpopmail

Hi Roberto
Thanks for your excellent guide.

I had no problems upgrading from dovecot 2.1.16 to 2.2.0
Instead of using the vpopmail auth driver, I am set up to use the sql driver. I appreciate this might not be what you are trying to achieve, but at least it will allow you to have vpopmail work with dovecot 2.2.0

My relevant configuration:

in /etc/dovecot/conf.d/10-auth.conf:

!include auth-sql.conf.ext

in /etc/dovecot/conf.d/auth-sql.conf.ext:
passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = prefetch
}

in /etc/dovecot/dovecot-sql.conf.ext:
driver = mysql
default_pass_scheme = MD5-CRYPT
password_query = SELECT CONCAT(pw_name, '@', '%d') AS user, \
  pw_passwd AS password, \
  pw_dir as userdb_home, \
  89 AS userdb_uid, \
  89 AS userdb_gid \
  FROM `vpopmail` \
  WHERE pw_name = '%n' AND pw_domain = '%d'

I haven't spent the time looking to see what has happened to the vpopmail auth driver.

Best wishes,
Michael Dick

Reply | Permalink

Re: Dovecot-2.2.0 and Vpopmail

Hi Michael, thanks for your contribution.

of course the sql drivers works fine now as worked in the past. Anyway this is not the first time that the vpopmail breaks, so your comment is appreciated. Soon or later I should consider to expand this page with the sql driver method.

Reply | Permalink

qmail & dovecot

Please be patient with me.....am a newbie...I've been trying to install qmail on my F16 64 bit home machine for the past 3 weeks but haven't managed to get it running yet but I'm very hopeful despite the many many frustrations. I wanted to know whether i install dovecot after the successful installation of qmail or do i start my dovecot installation 'independent' of the qmail installation. Thanks for the great piece by....

Reply | Permalink

my answer is yes

Despite of the fact that you can have a dovecot successfull installation, if you don't have qmail or another MTA installed it doesn't make much sense having an imap server without the smtp service in order to transfer your emails around the world; at least you should have your MTA in another machine. So yes, you should install qmail firstly.

You may want to take a look at this article which shortly explains how all things work together: http://wiki.dovecot.org/MailServerOverview

Reply | Permalink

Thanks for this great HOWTO!

Thanks for this great HOWTO! Just tested it and it's ok!

Reply | Permalink

some more features about quota

to makes IMAP quota working with maildir++, you should change these settings :

20-imap.conf

You should replace quota with imap_quota

90-quota.conf

You should uncomment this line:

quota = maildir:User quota

Reply | Permalink

I tested it, and it's ok.

I tested it, and it's ok. Thanks for the contribution!

Reply | Permalink

doveadm

First of all thanks for this HOWTO!

anyway i'm having a problem with the "doveadm -A" listing "ErrorUser listing returned failure doveadm",

"doveadm mailbox list -u user@domain" is working just fine.

did i miss something?

Reply | Permalink

doveadm & vpopmail

If you are using vpopmail as a backend, I don't have an answer. It seems like the driver is not programmed to do the iteration..

Reply | Permalink

doveadm expunge for vpopmail

I have found a way....this is how is working for me:
(this is just for "Trash" ...same for "Junk" ...just change mailbox param)
 
#!/bin/bash
# MySQL details

HOST="localhost";
USER="vpopmail";
PWD="vpopmailpasswod";

# Output sql to a file that we want to run
echo "USE vpopmail; select concat(pw_name,'@',pw_domain) as username from vpopmail;" > /tmp/query.sql;

# Run the query and get the results
results=`mysql -h $HOST -u $USER -p$PWD -N < /tmp/query.sql`;

# Loop through each row
for row in $results
do
/usr/local/dovecot/bin/doveadm expunge mailbox Trash -u $row savedbefore 30d
done

Reply | Permalink

Excellent solution!

Can you believe it! I was facing the same problem with doveadm expunge and found on google your excellent solution inside your comment of my own blog! Thank you..

Reply | Permalink

networking

The exact same thing happened to me once, isn`t it great how networking works!

Reply | Permalink