Sieve interpreter & Dovecot ManageSieve

January 10, 2018 Roberto Puzzanghera31 comments

The Pigeonhole project provides Sieve support as a plugin for Dovecot's Local Delivery Agent (LDA) and also for its LMTP service. The plugin implements a Sieve interpreter, which filters incoming messages using a script specified in the Sieve language. The Sieve script is provided by the user and, using that Sieve script, the user can customize how incoming messages are handled. Messages can be delivered to specific folders, forwarded, rejected, discarded, etc.

Dovecot Managesieve Server is a service used to manage a user's Sieve script collection.

NB: the location of the global sieve script is now /usr/local/dovecot/etc/sieve/

What follows was tested with the v. 0.5.0.1 of pigeonhole. It's always better to install the latest version of the program.

cd /usr/local/src
wget https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.0.1.tar.gz
tar xzf dovecot-2.3-pigeonhole-0.5.0.1.tar.gz
chown -R root.root dovecot-2.3-pigeonhole-0.5.0.1
cd dovecot-2.3-pigeonhole-0.5.0.1

# the program has to find the dovecot-config file in /usr/local/dovecot/lib/dovecot/
./configure \
        --prefix=/usr/local/dovecot-pigeonhole \
        --with-dovecot=/usr/local/dovecot/lib/dovecot/
make
make install

cd /usr/local
mv /usr/local/dovecot-pigeonhole /usr/local/dovecot-2.3-pigeonhole-0.5.0.1
ln -s /usr/local/dovecot-2.3-pigeonhole-0.5.0.1 /usr/local/dovecot-pigeonhole

Configuration

Copy the default config files in the actual config directory:

cd /usr/local/dovecot/etc/dovecot/conf.d
cp -p ../../../share/doc/dovecot/example-config/conf.d/20-managesieve.conf .
cp -p ../../../share/doc/dovecot/example-config/conf.d/90-sieve.conf .

In this way the next time you will run dovecot the two config files will be loaded.

Now enable (if not done yet) the plugin inside 15-lda.conf

protocol lda {
  mail_plugins = $mail_plugins sieve
}

and adjust the file conf.d/20-managesieve.conf to your needs. This is the file which works for me; you can findit in my tarball:

##
## ManageSieve specific settings
##

# Uncomment to enable managesieve protocol:
protocols = $protocols sieve

# Service definitions

service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}

service managesieve {
}

# Service configuration

protocol sieve {
}

Now adjust the file  conf.d/90-sieve.conf. This works for me (you have this file in your config directory if you have downloaded my tarball):

##
## Settings for the Sieve interpreter
##

# Do not forget to enable the Sieve plugin in 15-lda.conf and 20-lmtp.conf
# by adding it to the respective mail_plugins= settings.

plugin {
 sieve = file:~/.sieve;active=~/.sieve/dovecot.sieve
 sieve_before = /usr/local/dovecot/etc/dovecot/sieve/
 sieve_extensions = +notify +imapflags +spamtest
}

Now restart dovecot

dovecotctl restart

Adjusting dot-qmail files to enable Dovecot LDA and Sieve

Read this for more information on how dot-qmail files work.

The point is that if you want to use Sieve rules, vdelivermail will no longer deliver your mail but this will be a task to be accomplished by Dovecot LDA. In other words in each domain where you want to enable sieve you must modify the .qmail-default as follows:

|/var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER

Of course you can decide to limit sieve rules to certain users, in that case it is sufficient to adjust the .qmail-user file in the domain folder or the .qmail file in the user home dir.

If you decide to enable sieve by default you can always adjust /var/qmail/control/defaultdelivery provided that you clean .qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea.
Infact, adjusting defaultdelivery and cleaning .qmail-default has the risk to throw qmailadmin into confusion when listing the mailing-lists, because a non existent "default" mailing-list seems to appear in some case. Comments about this would be appreciated.

Setting up an anti spam sieve rule

Info: http://tools.ietf.org/html/rfc5235 - http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples

If you decided to let simscan pass through the spam with a score below spam_hits (qmail/control/simcontrol file), you may want to store them in the Junk folder. In this case Managesieve server will execute a script before processing the user's script so that all spam messages will be discarded or moved into junk and all the other rules ignored.

First of all create a folder where to store your global scripts and assign the write priviledges to the vpopmail user:

cd /usr/local/dovecot/etc/dovecot
mkdir sieve
chown -R vpopmail.vchkpw sieve

Modify conf.d/90-sieve.conf to enable the required sieve extensions and load the script that you want to execute before:

sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric
sieve_before = /usr/local/dovecot/etc/dovecot/sieve/

Now create the script /usr/local/dovecot/etc/dovecot/sieve/move-spam.sieve (.sieve extension otherwise it will be ignored):

require ["fileinto"];
if anyof (header :contains "X-Spam-Flag" "YES")
{
 fileinto "Junk";
}
/* Other messages get filed into INBOX */

Every time you modify the global files you have to pre-compile them using the sievec program (more info here):

su vpopmail
cd /usr/local/dovecot/etc/dovecot/sieve
/usr/local/dovecot-pigeonhole/bin/sievec .

Testing managesieve

First of all try to connect to the 4190 port via telnet. This is what you are going to see if the server is working:

> telnet 0 4190

Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
"IMPLEMENTATION" "Dovecot Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN CRAM-MD5"
"STARTTLS"
"VERSION" "1.0"
OK "Dovecot ready."

If you create a sieve rule by your mail client (Mozilla Thunderbird provides a special add-on here https://addons.mozilla.org/en-US/thunderbird/addon/2548/) or via webmail (read the next note concerning Roundcube webmail), this is what you should see in the log simply setting a redirect filter:

Oct 22 00:03:13 lda(test@yourdomain.net): Info: sieve: msgid=<c3445037f979a8cb793df1f858b7a4f9@somedomain.com>: forwarded to <someone@somewhere.net>

Comments

POP3 user

Hi,

Just wondering, what happen to the spams if the user is on POP3 and Junk folder is not available?

Thanks
nic

Reply | Permalink

Hi Roberto,

My pop3 user do  not have a Junk folder. So spams scoring 6.0 - 9.4 will be discard?

Reply | Permalink

Re: POP3 user

Do you mean a junk folder in their server's maildir? I haven't verified what happens in that case

Reply | Permalink

Hi,

Yes. Correct,

POP3 has only

cur
new
tmp

Thanks

Reply | Permalink

Autocreate

The folders should be autocreated, look at lda and plug-in config

Reply | Permalink

Autocreate

Hello,

I am using your config files and i had downloaded again to compare. They are exactly the same.

I noticed that if i setup the account as pop3, .Junk will not be created. If i setup with IMAP or login  via roundcube, .Junk will be created.

Is it correct? If so, those users who only create their account on pop3 and did not login to roundcube, .Junk will not be created, what will happen to the spam email scoring 6.0 - 9.4 goes to?

Many thanks
nic

Reply | Permalink

Autocreate and dovecot-lda

Nic, according to the documentation inside 15-lda.conf file, a nonexistent mailbox is automatically created before saving an e-mail into it.

Reply | Permalink

Autocreate and dovecot-lda

I did a test and it works well. Steps to reproduce

-create a new account

-login via webmail and set a minimum spamscore =1 using your spam/userprefs settings

-manually delete the .Junk folder from shell

-send from the outnet a message to that account. To get a spamscore > 1 you can simply start the body with "Dear friend". It is important to send the test messages from the outnet, otherwise spamassassin won't run

Reply | Permalink

Autocreate

Yes, I think this is correct.. that folder is created as soon as user enters the mailbox via imap. What happens to the spam? Unfortunately I don't know, tests would be needed... and I wolud like to investigate if it's possible to force the autocreation with pop3 as well, when I find some time..

Edit: dovecot-ldais going to autocreate the missing folder before saving the message. Look at 15-lda.conf

Reply | Permalink

Autocreate

Hello,

Google leads me to these sites.
https://sys4.de/de/blog/2013/02/11/dovecot-virtual-setup-mit-globaler-sieve-spamfilter-regel-fur-pop3-nutzer/ 

https://wiki.dovecot.org/Plugins/Virtual

I will try them out on a test server. Hopefully i can get it.

Thanks
nic

Reply | Permalink

POP3 and junk folder

In my configuration, if the score is beyond the allowed threeshold (9.5 or so) the messages will be deleted and not shown to the user, if the score is in the "gray zone" (say 6.0 to 9.5) it is saved in the junk folder inside the Maildir, but the user will never retrieve these messages via POP3. This is because the junk folder is in the server, but the POP3 user just downloads the new messages stored in the Inbox

Reply | Permalink

New versions of dovecot

Hello Roberto,

just to let you and all of the users that the procedure works also for version 2.2.9 of dovecot.

I'm testing version 2.2.10 which has been just released.

thank you again !

Reply | Permalink

Good to know, mz! Have you

Good to know, mz! Have you already tested pigeonhole-0.4.2? I think it will work as well.

btw, I think I will stick with the 2.2.2 version unless I have to install a new server before the next major release. It is very time consuming to update the dovecot.conf.tar.gz file with all the configuration files.

In case you are doing your configuration files from scratch, having copied them from the share/doc/dovecot/example-config/ dir, I hope you will be so kind to send me a targz. Send me it in private if you like at "roberto dot puzzanghera at sagredo dot eu" so that I can make it available for all users.

Reply | Permalink

yes, I'm using

yes, I'm using pigeonhole-0.4.2 and everything seems working, even if I still haven't my new server in production yet :-)

I made my config files starting from the examples and then customizing them step by step following your guide.

I'm using plain vpopmail authentication (not mysql) so configs are a bit different.

Didn't find differences between version 2.2.2 and 2.2.9; I just compiled and installed version 2.2.10 because there are a few interesting updates and bugfix  on various quota stuff which I need:

http://dovecot.org/list/dovecot-news/2013-December/000268.html

+ imap: Implemented SETQUOTA command for admin user when quota_set is configured. See http://master.wiki2.dovecot.org/Quota/Configuration + quota: Support "*" and "?" wildcards in mailbox names in quota_rules - quota-status: quota_grace was ignored

Will definitely send you my files when I'm confident that everything is working correcty, but now I have to go buy the last presents, so Merry Christmas to all of you out there ! :-)

Reply | Permalink

Error in the config file

Hello Roberto,
thank you for your GREAT tutorial!

Just to point out a little mistake in the conf file you are pasting here:

#protocols = $protocols sieve

should be

protocols = $protocols sieve

as your zipped tar reports correctly.

Thank you !
mz

Reply | Permalink

corrected. thank you

corrected. thank you

Reply | Permalink

Dovecot-lda and vpopmail quotas

Hi roberto Congrats for your great  job I have to admit your blog is  the reason why i decided to use this implementation for mail server

I have tried to follow your instructions as closer as as i  could. Everything works fine but  ... What i noticed is when i use dovecot-lda for delicery the sieve rule works fine but the per user quotas i have set through vpopmail are ignored Any ideas ??

this is my .qmail-default on the domain level

| /usr/bin/spamc | /var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER

Reply | Permalink

no idea at the moment..

no idea at the moment.. at least I can confirm that my config works fine here

Reply | Permalink

defaultdelivery

Thank you for your site.
Everything works fine, but there was a problem with the /var/qmail/control/defaultdelivery

If line |/ var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER in place .qmail, sieve-script works, if it is placed in .qmail-default, too fine.

 According to the description has changed .qmail-default, ie delete the first line, second line #. Added to defaultdelivery command preline, mail delivery stops working. But in the process, I see that defaultdelivery works:

qmail-lspawn |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER

I would be grateful for any idea.

Thank you.

Reply | Permalink

Hi Nik

Hi Nik, I would try to clean up the .qmail-default file so that it is completely blank

Reply | Permalink

defauldelivery

Hello Roberto!

Thank you very much for your reply.

Sorry for the delay in response.

So. I cleaned .qmail-default.

# cat/var/qmail/control/defauldelivery
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
# sudo ps -ax | grep deliver
qmail-lspawn |/var/qmail/bin/preline -f/usr/local/libexec/dovecot/deliver -d $EXT@$USER

But mail delivery stopped working.

In qmail logs the message:

delivery 6: deferral: Uh-oh: _first_line_of_.qmail_file_is_blank._ (# 4.2.1) /

The blog reported: "If you decide to enable sieve by default you can always adjust / var / qmail / control / defaultdelivery provided that you clean. Qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea. "

Of course, I can use. Qmail-default, but:

1. Want to understand why this is not working (either you control the situation, or the situation control you)

2. I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

Perhaps there is a solution based on the deliver a bounce message?

Thank you again.

I hope for your help.

P.S. Very uncomfortable system of confirmation messages.The picture is very promiscuous.

Reply | Permalink

Re: defaultdelivery

Sorry for the delay in response.

So. I cleaned .qmail-default.

# cat/var/qmail/control/defauldelivery
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
# sudo ps -ax | grep deliver
qmail-lspawn |/var/qmail/bin/preline -f/usr/local/libexec/dovecot/deliver -d $EXT@$USER

But mail delivery stopped working.

In qmail logs the message:

delivery 6: deferral: Uh-oh: _first_line_of_.qmail_file_is_blank._ (# 4.2.1) /

The blog reported: "If you decide to enable sieve by default you can always adjust / var / qmail / control / defaultdelivery provided that you clean. Qmail-default of newly created domain (just remove the first line, never erase that file if you don't want vpopmail to stop working), even though this is not a good idea. "

I can confirm that cleaning .qmail-default works fine here. I wrote that it could not be a good idea just because at the time I wrote this note, as I said, qmailadmin showed a maling-list named "default" in that case. I'm doing some test right now but the issue is not shown anymore.

Of course, I can use. Qmail-default, but:

1. Want to understand why this is not working (either you control the situation, or the situation control you
)

I don't think that this guide would be available to the public if I am not experienced on this topic. That said, vpopmail, qmailadmin and so on are not programmed to make dovecot-lda or whatelse to manage the delivery, but patches and tests are needed. So feel free to contribute posting yours

2. I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

I suppose that you can do it by means of a global sieve rules.

P.S. Very uncomfortable system of confirmation messages.The picture is very promiscuous.

I know, but I have tens of bots trying to break the captcha filter at the same time. If I relax the filter I would have to connect at least once an hour to check/moderate spam messages..

Reply | Permalink

this is not true

I also need to use bounced messages:

|/usr/local/vpopmail/bin/vdelivermail'' delete

If I use in .qmail-default, these two lines, the messages duplicated, i.e. email goes through 'vdelivermail' and through 'deliver'.

I suppose that you can do it by means of a global sieve rules.

sorry, this is not true, because at the time of dovecot-lda delivery chkuser has already bounced the message..

Reply | Permalink

.qmail-default

Hello, Roberto.

Apparently I was wrong to describe the situation.

The file contains two lines:

|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
|/usr/local/vpopmail/bin/vdelivermail '' delete

In this case, the message is delivered to a mailbox twice:

  1. dovecot (/usr/local/libexec/dovecot/deliver)
  2. vpopmail (/usr/local/vpopmail/bin/vdelivermail)

Now, about chkuser.

?hkuser closes the session when a message is received for a nonexistent mailbox (ie step rcpt to). I believe that it is wrong, because spammers can understand that this mailbox does not exist. Therefore, I accept all the messages and then vdelivermail deletes messages to nonexistent mailboxes.

Please correct me if I'm wrong.

Therefore, the question is by what means to remove messages to non-existent mailboxes?

Again, that using dovecot vpopmail in the file, it works. But the messages are duplicated.

Thank you for your help!

Reply | Permalink

Hi Nik, of course you get a

Hi Nik, of course you get a double relay in the situation described.

I've never tried to make chkuser delete messages for non existent users, but I suppose that it can do it. I would try to play with the CHKUSER_SPECIFIC_BOUNCING variable in the chkuser_settings.h and recompile. The documentation on the purpose is not so clear to me, but it seems like chkuser is going to look for a specific file in the domain dir to decide if the bouncing has to be done.

Let me know if you manage to avoid the replay.

Reply | Permalink

Sieve interpreter AND catch all in .qmail-default

hi roberto,

i wonder how i can make a catch all account available per domain but maintain the the functionality of thie in the .qmail-default per domain:

|/var/qmail/bin/preline -f /usr/local/dovecot/libexec/dovecot/deliver -d $EXT@$USER

the above works great untill i activate a catch all account, the i have this instead:

| /home/vpopmail/bin/vdelivermail '' catchalluser@test.com

can i combine the two somehow or even better, can i alter the catchall function of qmailadmin to use a different line in .qmail-default including to have both enabled ?

thanks
Jan

Reply | Permalink

in case you are patching

in case you are patching qmail with chkuser, I would try to turn on this chkuser's option

CHKUSER_SPECIFIC_BOUNCING

and see if it does the trick. But you have to recompile qmail. See details here http://www.interazioni.it/opensource/chkuser/documentation/chkuser_settings.html

Let me know if it works!

Reply | Permalink

i will have another go on

i will have another go on monday, was at it all day today for over 12hrs now. i am getting 'blind' and start making mistakes. i let you know beginning next week how i get on.

thanks and have a nice sunday
Jan

Reply | Permalink

permission problems with 'sievec'

I am getting errors about directory permission and i am unsure how i can resolve this and stay secure. here the error i get after i have followed this

su vpopmail
cd /usr/local/dovecot/etc/dovecot/sieve
/usr/local/dovecot-2-0-pigeonhole/bin/sievec .

This is what i get:

$ /usr/local/dovecot-2-0-pigeonhole/bin/sievec .
sievec(vpopmail): Error: sieve: .: failed to stat sieve script: stat(.) in directory /root failed: Permission denied (euid=89(vpopmail) egid=89(vchkpw) missing +x perm: /root, dir owned by 0:0 mode=0700)
.: error: failed to open sieve script: internal error occurred: refer to server log for more information. [2011-11-26 16:19:38].
sievec(vpopmail): Error: failed to compile sieve script '.'
$

Any help would be very much appreciated

Thanks
Jan

Reply | Permalink

Re: permission problems

Hi Jan, it seems like if the sievec file is not executable. In this case, simply try to:

chmod +x /usr/local/dovecot-2-0-pigeonhole/bin/sievec

and recompile

Reply | Permalink

I see what i have

I see what i have missed:

chown -R vpopmail.vchkpw sieve

Thanks for the help

Reply | Permalink

Recent comments
Recent posts

RSS feeds