Sieve interpreter & Dovecot ManageSieve

October 4, 2014 Roberto Puzzanghera20 comments

The Pigeonhole project provides Sieve support as a plugin for Dovecot's Local Delivery Agent (LDA) and also for its LMTP service. The plugin implements a Sieve interpreter, which filters incoming messages using a script specified in the Sieve language. The Sieve script is provided by the user and, using that Sieve script, the user can customize how incoming messages are handled. Messages can be delivered to specific folders, forwarded, rejected, discarded, etc.

Dovecot Managesieve Server is a service used to manage a user's Sieve script collection.

NB: the location of the global sieve script is now /usr/local/dovecot/etc/sieve/

What follows was tested with the v. 0.4.3 of pigeonhole. It's always better to install the latest version of the program.

cd /usr/local/src
wget http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.3.tar.gz
chown -R root.root dovecot-2.2-pigeonhole-0.4.3
cd dovecot-2.2-pigeonhole-0.4.3

# the program has to find the dovecot-config file in/usr/local/dovecot/lib/dovecot/
./configure \
        --prefix=/usr/local/dovecot-pigeonhole \
        --with-dovecot=/usr/local/dovecot/lib/dovecot/
make
make install

cd /usr/local
mv /usr/local/dovecot-pigeonhole /usr/local/dovecot-2.2-pigeonhole-0.4.3
ln -s /usr/local/dovecot-2.2-pigeonhole-0.4.3 /usr/local/dovecot-pigeonhole

Configuration

Copy the default config files in the actual config directory:

cd /usr/local/dovecot/etc/dovecot/conf.d
cp -p ../../../share/doc/dovecot/example-config/conf.d/20-managesieve.conf .
cp -p ../../../share/doc/dovecot/example-config/conf.d/90-sieve.conf .

In this way the next time you will run dovecot the two config files will be loaded.

Now enable (if not done yet) the plugin inside 15-lda.conf

protocol lda {
  mail_plugins = $mail_plugins sieve
}

and adjust the file conf.d/20-managesieve.conf to your needs. This is the file which works for me; you can findit in my tarball:

##
## ManageSieve specific settings
##

# Uncomment to enable managesieve protocol:
protocols = $protocols sieve

# Service definitions

service managesieve-login {
  inet_listener sieve {
    port = 4190
  }

  #inet_listener sieve_deprecated {
  #  port = 2000
  #}

  # Number of connections to handle before starting a new process. Typically
  # the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
  # is faster. <doc/wiki/LoginProcess.txt>
  service_count = 1

  # Number of processes to always keep waiting for more connections.
  #process_min_avail = 0

  # If you set service_count=0, you probably need to grow this.
  vsz_limit = 64M
}

service managesieve {
  # Max. number of ManageSieve processes (connections)
  #process_limit = 1024
}

# Service configuration

protocol sieve {
  # Maximum ManageSieve command line length in bytes. ManageSieve usually does
  # not involve overly long command lines, so this setting will not normally
  # need adjustment
  #managesieve_max_line_length = 65536

  # Maximum number of ManageSieve connections allowed for a user from each IP
  # address.
  # NOTE: The username is compared case-sensitively.
  #mail_max_userip_connections = 10

  # Space separated list of plugins to load (none known to be useful so far).
  # Do NOT try to load IMAP plugins here.
  #mail_plugins =

  # MANAGESIEVE logout format string:
  #  %i - total number of bytes read from client
  #  %o - total number of bytes sent to client
  #managesieve_logout_format = bytes=%i/%o

  # To fool ManageSieve clients that are focused on CMU's timesieved you can
  # specify the IMPLEMENTATION capability that Dovecot reports to clients.
  # For example: 'Cyrus timsieved v2.2.13'
  #managesieve_implementation_string = Dovecot Pigeonhole

  # Explicitly specify the SIEVE and NOTIFY capability reported by the server
  # before login. If left unassigned these will be reported dynamically
  # according to what the Sieve interpreter supports by default (after login
  # this may differ depending on the user).
  #managesieve_sieve_capability =
  #managesieve_notify_capability =

  # The maximum number of compile errors that are returned to the client upon
  # script upload or script verification.
  #managesieve_max_compile_errors = 5

  # Refer to 90-sieve.conf for script quota configuration and configuration of
  # Sieve execution limits.
}

Now adjust the file  conf.d/90-sieve.conf. This works for me (you have this file in your config directory if you have downloaded my tarball):

##
## Settings for the Sieve interpreter
##

# Do not forget to enable the Sieve plugin in 15-lda.conf and 20-lmtp.conf
# by adding it to the respective mail_plugins= settings.

plugin {
  # The path to the user's main active script. If ManageSieve is used, this the
  # location of the symbolic link controlled by ManageSieve.
  sieve=~/.sieve/dovecot.sieve

  # The default Sieve script when the user has none. This is a path to a global
  # sieve script file, which gets executed ONLY if user's private Sieve script
  # doesn't exist. Be sure to pre-compile this script manually using the sievec
  # command line tool.
  # --> See sieve_before fore executing scripts before the user's personal
  #     script.
  #sieve_default = /var/lib/dovecot/sieve/default.sieve

  # Directory for :personal include scripts for the include extension. This
  # is also where the ManageSieve service stores the user's scripts.
  sieve_dir = ~/.sieve

  # Directory for :global include scripts for the include extension.
  #sieve_global_dir =

  # Path to a script file or a directory containing script files that need to be
  # executed before the user's script. If the path points to a directory, all
  # the Sieve scripts contained therein (with the proper .sieve extension) are
  # executed. The order of execution within a directory is determined by the
  # file names, using a normal 8bit per-character comparison. Multiple script
  # file or directory paths can be specified by appending an increasing number.
  #sieve_before =
  #sieve_before2 =
  #sieve_before3 = (etc...)
  sieve_before = /usr/local/dovecot/etc/dovecot/sieve/

  # Identical to sieve_before, only the specified scripts are executed after the
  # user's script (only when keep is still in effect!). Multiple script file or
  # directory paths can be specified by appending an increasing number.
  #sieve_after =
  #sieve_after2 =
  #sieve_after2 = (etc...)

  # Which Sieve language extensions are available to users. By default, all
  # supported extensions are available, except for deprecated extensions or
  # those that are still under development. Some system administrators may want
  # to disable certain Sieve extensions or enable those that are not available
  # by default. This setting can use '+' and '-' to specify differences relative
  # to the default. For example `sieve_extensions = +imapflags' will enable the
        # deprecated imapflags extension in addition to all extensions were already
  # enabled by default.
  #sieve_extensions = +notify +imapflags
  sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric

  # Which Sieve language extensions are ONLY available in global scripts. This
  # can be used to restrict the use of certain Sieve extensions to administrator
  # control, for instance when these extensions can cause security concerns.
  # This setting has higher precedence than the `sieve_extensions' setting
  # (above), meaning that the extensions enabled with this setting are never
  # available to the user's personal script no matter what is specified for the
  # `sieve_extensions' setting. The syntax of this setting is similar to the
  # `sieve_extensions' setting, with the difference that extensions are
  # enabled or disabled for exclusive use in global scripts. Currently, no
  # extensions are marked as such by default.
  #sieve_global_extensions =

  # The Pigeonhole Sieve interpreter can have plugins of its own. Using this
  # setting, the used plugins can be specified. Check the Dovecot wiki
  # (wiki2.dovecot.org) or the pigeonhole website
  # (http://pigeonhole.dovecot.org) for available plugins.
        # The sieve_extprograms plugin is included in this release.
  #sieve_plugins =

  # The separator that is expected between the :user and :detail
  # address parts introduced by the subaddress extension. This may
  # also be a sequence of characters (e.g. '--'). The current
  # implementation looks for the separator from the left of the
  # localpart and uses the first one encountered. The :user part is
  # left of the separator and the :detail part is right. This setting
  # is also used by Dovecot's LMTP service.
  #recipient_delimiter = +

  # The maximum size of a Sieve script. The compiler will refuse to compile any
  # script larger than this limit. If set to 0, no limit on the script size is
  # enforced.
  #sieve_max_script_size = 1M

  # The maximum number of actions that can be performed during a single script
  # execution. If set to 0, no limit on the total number of actions is enforced.
  #sieve_max_actions = 32

  # The maximum number of redirect actions that can be performed during a single
  # script execution. If set to 0, no redirect actions are allowed.
  #sieve_max_redirects = 4

  # The maximum number of personal Sieve scripts a single user can have. If set
  # to 0, no limit on the number of scripts is enforced.
  # (Currently only relevant for ManageSieve)
  #sieve_quota_max_scripts = 0

  # The maximum amount of disk storage a single user's scripts may occupy. If
  # set to 0, no limit on the used amount of disk storage is enforced.
  # (Currently only relevant for ManageSieve)
  #sieve_quota_max_storage = 0
}

Now restart dovecot

dovecotctl restart

Adjusting dot-qmail files to enable Dovecot LDA and Sieve

Read this for more information on how dot-qmail files work.

The point is that if you want to use Sieve rules, vdelivermail will no longer deliver your mail but this will be a task to be accomplished by Dovecot LDA. In other words in each domain where you want to enable sieve you must modify the .qmail-default as follows:

|/var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER

Of course you can decide to limit sieve rules to certain users, in that case it is sufficient to adjust the .qmail-user file in the domain folder or the .qmail file in the user home dir.

If you decide to enable sieve by default you can always adjust /var/qmail/control/defaultdelivery provided that you clean .qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea.
Infact, adjusting defaultdelivery and cleaning .qmail-default has the risk to throw qmailadmin into confusion when listing the mailing-lists, because a non existent "default" mailing-list seems to appear in some case. Comments about this would be appreciated.

Setting up an anti spam sieve rule

Info: http://tools.ietf.org/html/rfc5235 - http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples

If you decided to let simscan pass through the spam with a score below spam_hits (qmail/control/simcontrol file), you may want to store them in the Junk folder. In this case Managesieve server will execute a script before processing the user's script so that all spam messages will be discarded or moved into junk and all the other rules ignored.

First of all create a folder where to store your global scripts and assign the write priviledges to the vpopmail user:

cd /usr/local/dovecot/etc/dovecot
mkdir sieve
chown -R vpopmail.vchkpw sieve

Modify conf.d/90-sieve.conf to enable the required sieve extensions and load the script that you want to execute before:

sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric
sieve_before = /usr/local/dovecot/etc/dovecot/sieve/

Now create the script /usr/local/dovecot/etc/dovecot/sieve/move-spam.sieve (.sieve extension otherwise it will be ignored):

require ["fileinto"];
if anyof (header :contains "X-Spam-Flag" "YES")
{
 fileinto "Junk";
}
/* Other messages get filed into INBOX */

Every time you modify the global files you have to pre-compile them using the sievec program (more info here):

su vpopmail
cd /usr/local/dovecot/etc/dovecot/sieve
/usr/local/dovecot-pigeonhole/bin/sievec .

Testing managesieve

First of all try to connect to the 4190 port via telnet. This is what you are going to see if the server is working:

> telnet 0 4190

Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
"IMPLEMENTATION" "Dovecot Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN CRAM-MD5"
"STARTTLS"
"VERSION" "1.0"
OK "Dovecot ready."

If you create a sieve rule by your mail client (Mozilla Thunderbird provides a special add-on here https://addons.mozilla.org/en-US/thunderbird/addon/2548/) or via webmail (read the next note concerning Roundcube webmail), this is what you should see in the log simply setting a redirect filter:

Oct 22 00:03:13 lda(test@yourdomain.net): Info: sieve: msgid=<c3445037f979a8cb793df1f858b7a4f9@somedomain.com>: forwarded to <someone@somewhere.net>

Comments

New versions of dovecot

Hello Roberto,

just to let you and all of the users that the procedure works also for version 2.2.9 of dovecot.

I'm testing version 2.2.10 which has been just released.

thank you again !

Reply | Permalink

Good to know, mz! Have you

Good to know, mz! Have you already tested pigeonhole-0.4.2? I think it will work as well.

btw, I think I will stick with the 2.2.2 version unless I have to install a new server before the next major release. It is very time consuming to update the dovecot.conf.tar.gz file with all the configuration files.

In case you are doing your configuration files from scratch, having copied them from the share/doc/dovecot/example-config/ dir, I hope you will be so kind to send me a targz. Send me it in private if you like at "roberto dot puzzanghera at sagredo dot eu" so that I can make it available for all users.

Reply | Permalink

yes, I'm using

yes, I'm using pigeonhole-0.4.2 and everything seems working, even if I still haven't my new server in production yet :-)

I made my config files starting from the examples and then customizing them step by step following your guide.

I'm using plain vpopmail authentication (not mysql) so configs are a bit different.

Didn't find differences between version 2.2.2 and 2.2.9; I just compiled and installed version 2.2.10 because there are a few interesting updates and bugfix  on various quota stuff which I need:

http://dovecot.org/list/dovecot-news/2013-December/000268.html

+ imap: Implemented SETQUOTA command for admin user when quota_set is configured. See http://master.wiki2.dovecot.org/Quota/Configuration + quota: Support "*" and "?" wildcards in mailbox names in quota_rules - quota-status: quota_grace was ignored

Will definitely send you my files when I'm confident that everything is working correcty, but now I have to go buy the last presents, so Merry Christmas to all of you out there ! :-)

Reply | Permalink

Error in the config file

Hello Roberto,
thank you for your GREAT tutorial!

Just to point out a little mistake in the conf file you are pasting here:

#protocols = $protocols sieve

should be

protocols = $protocols sieve

as your zipped tar reports correctly.

Thank you !
mz

Reply | Permalink

corrected. thank you

corrected. thank you

Reply | Permalink

Dovecot-lda and vpopmail quotas

Hi roberto Congrats for your great  job I have to admit your blog is  the reason why i decided to use this implementation for mail server

I have tried to follow your instructions as closer as as i  could. Everything works fine but  ... What i noticed is when i use dovecot-lda for delicery the sieve rule works fine but the per user quotas i have set through vpopmail are ignored Any ideas ??

this is my .qmail-default on the domain level

| /usr/bin/spamc | /var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER

Reply | Permalink

no idea at the moment..

no idea at the moment.. at least I can confirm that my config works fine here

Reply | Permalink

defaultdelivery

Thank you for your site.
Everything works fine, but there was a problem with the /var/qmail/control/defaultdelivery

If line |/ var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER in place .qmail, sieve-script works, if it is placed in .qmail-default, too fine.

 According to the description has changed .qmail-default, ie delete the first line, second line #. Added to defaultdelivery command preline, mail delivery stops working. But in the process, I see that defaultdelivery works:

qmail-lspawn |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER

I would be grateful for any idea.

Thank you.

Reply | Permalink

Hi Nik

Hi Nik, I would try to clean up the .qmail-default file so that it is completely blank

Reply | Permalink

defauldelivery

Hello Roberto!

Thank you very much for your reply.

Sorry for the delay in response.

So. I cleaned .qmail-default.

# cat/var/qmail/control/defauldelivery
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
# sudo ps -ax | grep deliver
qmail-lspawn |/var/qmail/bin/preline -f/usr/local/libexec/dovecot/deliver -d $EXT@$USER

But mail delivery stopped working.

In qmail logs the message:

delivery 6: deferral: Uh-oh: _first_line_of_.qmail_file_is_blank._ (# 4.2.1) /

The blog reported: "If you decide to enable sieve by default you can always adjust / var / qmail / control / defaultdelivery provided that you clean. Qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea. "

Of course, I can use. Qmail-default, but:

1. Want to understand why this is not working (either you control the situation, or the situation control you)

2. I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

Perhaps there is a solution based on the deliver a bounce message?

Thank you again.

I hope for your help.

P.S. Very uncomfortable system of confirmation messages.The picture is very promiscuous.

Reply | Permalink

Re: defaultdelivery

Sorry for the delay in response.

So. I cleaned .qmail-default.

# cat/var/qmail/control/defauldelivery
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
# sudo ps -ax | grep deliver
qmail-lspawn |/var/qmail/bin/preline -f/usr/local/libexec/dovecot/deliver -d $EXT@$USER

But mail delivery stopped working.

In qmail logs the message:

delivery 6: deferral: Uh-oh: _first_line_of_.qmail_file_is_blank._ (# 4.2.1) /

The blog reported: "If you decide to enable sieve by default you can always adjust / var / qmail / control / defaultdelivery provided that you clean. Qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea. "

I can confirm that cleaning .qmail-default works fine here. I wrote that it could not be a good idea just because at the time I wrote this note, as I said, qmailadmin showed a maling-list named "default" in that case. I'm doing some test right now but the issue is not shown anymore.

Of course, I can use. Qmail-default, but:

1. Want to understand why this is not working (either you control the situation, or the situation control you
)

I don't think that this guide would be available to the public if I am not experienced on this topic. That said, vpopmail, qmailadmin and so on are not programmed to make dovecot-lda or whatelse to manage the delivery, but patches and tests are needed. So feel free to contribute posting yours

2. I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

I suppose that you can do it by means of a global sieve rules.

P.S. Very uncomfortable system of confirmation messages.The picture is very promiscuous.

I know, but I have tens of bots trying to break the captcha filter at the same time. If I relax the filter I would have to connect at least once an hour to check/moderate spam messages..

Reply | Permalink

this is not true

I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

I suppose that you can do it by means of a global sieve rules.

sorry, this is not true, because at the time of dovecot-lda delivery chkuser has already bounced the message..

Reply | Permalink

.qmail-default

Hello, Roberto.

Apparently I was wrong to describe the situation.

The file contains two lines:

|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
|/usr/local/vpopmail/bin/vdelivermail '' delete

In this case, the message is delivered to a mailbox twice:

  1. dovecot (/usr/local/libexec/dovecot/deliver)
  2. vpopmail (/usr/local/vpopmail/bin/vdelivermail)

Now, about chkuser.

?hkuser closes the session when a message is received for a nonexistent mailbox (ie step rcpt to). I believe that it is wrong, because spammers can understand that this mailbox does not exist. Therefore, I accept all the messages and then vdelivermail deletes messages to nonexistent mailboxes.

Please correct me if I'm wrong.

Therefore, the question is by what means to remove messages to non-existent mailboxes?

Again, that using dovecot vpopmail in the file, it works. But the messages are duplicated.

Thank you for your help!

Reply | Permalink

Hi Nik, of course you get a

Hi Nik, of course you get a double relay in the situation described.

I've never tried to make chkuser delete messages for non existent users, but I suppose that it can do it. I would try to play with the CHKUSER_SPECIFIC_BOUNCING variable in the chkuser_settings.h and recompile. The documentation on the purpose is not so clear to me, but it seems like chkuser is going to look for a specific file in the domain dir to decide if the bouncing has to be done.

Let me know if you manage to avoid the replay.

Reply | Permalink

Sieve interpreter AND catch all in .qmail-default

hi roberto,

i wonder how i can make a catch all account available per domain but maintain the the functionality of thie in the .qmail-default per domain:

|/var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER

the above works great untill i activate a catch all account, the i have this instead:

| /home/vpopmail/bin/vdelivermail '' catchalluser@test.com

can i combine the two somehow or even better, can i alter the catchall function of qmailadmin to use a different line in .qmail-default including to have both enabled ?

thanks
Jan

Reply | Permalink

in case you are patching

in case you are patching qmail with chkuser, I would try to turn on this chkuser's option

CHKUSER_SPECIFIC_BOUNCING

and see if it does the trick. But you have to recompile qmail. See details here http://www.interazioni.it/opensource/chkuser/documentation/chkuser_settings.html

Let me know if it works!

Reply | Permalink

i will have another go on

i will have another go on monday, was at it all day today for over 12hrs now. i am getting 'blind' and start making mistakes. i let you know beginning next week how i get on.

thanks and have a nice sunday
Jan

Reply | Permalink

permission problems with 'sievec'

I am getting errors about directory permission and i am unsure how i can resolve this and stay secure. here the error i get after i have followed this

su vpopmail
cd /usr/local/dovecot/etc/dovecot/sieve
/usr/local/dovecot-2-0-pigeonhole/bin/sievec .

This is what i get:

$ /usr/local/dovecot-2-0-pigeonhole/bin/sievec .
sievec(vpopmail): Error: sieve: .: failed to stat sieve script: stat(.) in directory /root failed: Permission denied (euid=89(vpopmail) egid=89(vchkpw) missing +x perm: /root, dir owned by 0:0 mode=0700)
.: error: failed to open sieve script: internal error occurred: refer to server log for more information. [2011-11-26 16:19:38].
sievec(vpopmail): Error: failed to compile sieve script '.'
$

Any help would be very much appreciated

Thanks
Jan

Reply | Permalink

Re: permission problems

Hi Jan, it seems like if the sievec file is not executable. In this case, simply try to:

chmod +x /usr/local/dovecot-2-0-pigeonhole/bin/sievec

and recompile

Reply | Permalink

I see what i have

I see what i have missed:

chown -R vpopmail.vchkpw sieve

Thanks for the help

Reply | Permalink