Expunging expired Junk and Trash emails with dovecot

Hi Roberto,

when i execute /usr/local/dovecot/etc/dovecot expire.sh manually in the output there is below warning:

mysql: [Warning] Using a password on the command line interface can be insecure.

I created small patch for it:

cat > /usr/local/src/dovecot_expire.patch <<__EOF__
--- /usr/local/dovecot/etc/dovecot_expire.sh.old 2024-10-04 11:48:15.436321335 +0000
+++ /usr/local/dovecot/etc/dovecot_expire.sh 2024-10-04 11:49:36.847198893 +0000
@@ -24,7 +24,7 @@
echo "USE vpopmail; select concat(pw_name,'@',pw_domain) as username from vpopmail;" > /tmp/query.sql;

# Run the query and get the results (adjust the path to mysql)
-results=`$MYSQL -h $HOST -u $USER -p$PWD -N < /tmp/query.sql`;
+results=`export MYSQL_PWD=$PWD; $MYSQL -h $HOST -u $USER -N < /tmp/query.sql`;

# Loop through each row
for row in $results
__EOF__

For all with already enabled dovecot_expire.sh cron can apply the patch:

patch /usr/local/dovecot/etc/dovecot_expire.sh < /usr/local/src/dovecot_expire.patch

I don't think that the script is unsecure. It is not readable and it doesn't hold any pwd... that warning can be safely ignored

wget https://notes.sagredo.eu/files/qmail/dovecot_expunge/dovecot_expunge-cb - ERROR 404

downloading dovecot_expunge-cb then moving dovecot_expire ??? - anyway it's not there

# Config and executables path  
VPOPMAIL_MYSQL_CONFIG="/home/vpopmail/etc/vpopmail.mysql"

ADD MYSQL=$(which mysql) - else script fails

Thank you. Corrected

https://doc.dovecot.org/configuration_manual/namespace/#mailbox-settings says about autoexpunge:

mailbox_list_index = yes is highly recommended when using this setting, as it avoids actually opening the mailbox to see if anything needs to be expunged.

mail_always_cache_fields = date.save is also recommended when using this setting with sdbox or Maildir, as it avoids using stat() to find out the mail’s saved-timestamp. With mdbox and obox formats this isn’t necessary, since the saved-timestamp is always available.

I see that mail_always_cache_fields has been added in v2.3.17 but I can't find it in any file of the example-config folder. I think putting it in 10-mail.conf will be fine

Hi Roberto,

I created a script to expunge mailboxes to remove emails in before specified days per domain/user/folder which defined in a conf file.  Schedule to run the script daily and it shall clean up the defined mailboxes.  I run it on CentOS 8 almost a year and save me a lot of time to house keep some mailboxes in use of notification.  Also, it could be applied as mail retention control for all or defined mailboxes.  Hope this share is helpful.

Here is the script (/usr/local/dovecot/bin/dovecot-expunge):
Change the variables (DOVEADM, CONF, LOG) in the script to desired values for your environment if necessary.

#!/bin/sh

DOVEADM="/usr/local/dovecot/bin/doveadm"
CONF="/usr/local/dovecot/etc/dovecot/dovecot-expunge.conf"
LOG="/var/log/dovecot/dovecot-expunge.log"

if [ ! -e "$CONF" ]; then
   echo $(date +"%Y%m%d %T") \
   "aborted with $CONF not found" >> $LOG
   exit
fi

while read line; do
         # Skip commented and blank lines:
         if [[ ${line:0:1} = "#" ]] || [ -z "$line" ]; then
            continue
         fi

         # Put fields to variables per line:
         DOMAIN=$(echo $line|cut -d"|" -f1)
         USER=$(echo $line|cut -d"|" -f2)
         DAY=$(echo $line|cut -d"|" -f3)
         FOLDER=$(echo $line|cut -d"|" -f4)

         # List mailbox folders per user:
         $DOVEADM mailbox list -u "$USER@$DOMAIN" | \
         while read list; do
                  if [[ "$USER" = "*" ]]; then
                     u=$(echo $list|cut -d" " -f1)
                     f=$(echo $list|cut -d" " -f2)
                  else
                     u="$USER@$DOMAIN"
                     f=$list
                  fi

                  if [[ "$FOLDER" = "*" ]]; then
                     $DOVEADM expunge -u $u mailbox $f savedbefore "$DAY""d"
                     echo $(date +"%Y%m%d %T") \
                     "expunged $u:$f:$DAY days before" >> $LOG
                  else
                     if [ -z "${FOLDER##*$f*}" ]; then
                        $DOVEADM expunge -u $u mailbox $f savedbefore "$DAY""d"
                        echo $(date +"%Y%m%d %T") \
                        "expunged $u:$f:$DAY days before" >> $LOG
                     fi
                  fi
         done
done < $CONF

exit

Here is the conf file and info (/usr/local/dovecot/etc/dovecot/dovecot-expunge.conf):

# Entry format: (email_domain)|(email_user)|(days_before)|(mail_folders)
#
# (email_domain) is the domain name of email address, can be "*" for all.
# (email_user) is the user name of email address, can be "*" for all.
# (days_before) is the number of days to remove in before of.
# (mail_folders) is the folders to expunge, can be "*" for all,
# or combination in any of "Drafts,Junk,Trash,Sent,INBOX".
#
# All blank and commented (begin with "#") lines are skipped.
#
# =============================================
#
# Example: example.com|postmaster|30|*
#
# This example will expunge postmaster@example.com,
# with all mail folders in before of 30 days.
#
# ---------------------------------------------
#
# Example: example.com|*|15|Trash
#
# This example will expunge all users of example.com,
# with mail folder Trash only in before of 15 days.
#
# ---------------------------------------------
#
# Example: example.com|*|60|Drafts,Trash
#
# This example will expunge all users of example.com,
# with mail folders Drafts and Trash in before of 60 days.
#
# ---------------------------------------------
#
# Example: *|*|90|Trash
#
# This example will expunge all users per domain,
# with mail folder Trash in before of 90 days.
#
# Using "*" for domain is advised to be general purpose,
# Such that the days before should be the largest in compare with other settings.
#
# =============================================

Here is an example of cron job:

# Daily dovecot expunge:
1 0 * * * root /usr/local/dovecot/bin/dovecot-expunge > /dev/null 2>&1

Hi, very usefull, indeed. How do I set 2 or 3 or more users but not one specific or all, as using space as username's separation doesn't seems to work? Thanks

Here's a patch to your script to allow spaces in the names of mailboxes and a fix to script type (it should be a bash script, otherwise, it  won't work on some systems):

--- /orig/dovecot-expunge 2021-01-17 11:23:55.845365502 +0200
+++ new/dovecot-expunge 2021-01-17 11:25:21.060576680 +0200
@@ -1,8 +1,9 @@
-#!/bin/sh
+#!/bin/bash

@@ -27,19 +28,19 @@
while read list; do
if [[ "$USER" = "*" ]]; then
u=$(echo $list|cut -d" " -f1)
- f=$(echo $list|cut -d" " -f2)
+ f=$(echo $list|sed 's/[^ ]* //')
else
u="$USER@$DOMAIN"
f=$list
fi

if [[ "$FOLDER" = "*" ]]; then
- $DOVEADM expunge -u $u mailbox $f savedbefore "$DAY""d"
+ $DOVEADM expunge -u $u mailbox "$f" savedbefore "$DAY""d"
echo $(date +"%Y%m%d %T") \
"expunged $u:$f:$DAY days before" >> $LOG
else
if [ -z "${FOLDER##*$f*}" ]; then
- $DOVEADM expunge -u $u mailbox $f savedbefore "$DAY""d"
+ $DOVEADM expunge -u $u mailbox "$f" savedbefore "$DAY""d"
echo $(date +"%Y%m%d %T") \
"expunged $u:$f:$DAY days before" >> $LOG
fi

It seems the <...> are removed, the followings are replaced by (...).  Hope this time can display correctly.

# Entry format: (email_domain)|(email_user)|(days_before)|(mail_folders)
#
# (email_domain) is the domain name of email address, can be "*" for all.
# (email_user) is the user name of email address, can be "*" for all.
# (days_before) is the number of days to remove in before of.
# (mail_folders) is the folders to expunge, can be "*" for all,
# or combination in any of "Drafts,Junk,Trash,Sent,INBOX".

Thank you, Tony. Very appreciated.

I think this can be useful expecially to clean mailboxes like postmaster which are not accessed very often. I'm going to link this post later

Hi, what about other folders?

When you mark a message as deleted in INBOX folder?

I think you should go through all user folders :)

Best,

Rafal.

Rafal, what do you mean by "mark a message as deleted in INBOX folder"?

Hi Roberto,

I wasn't clear before, I am sorry. Please look at this example:

1) A new message (Seen Flag)

root@mail J:0 S:1 Maildir# tree {cur,new,tmp}
cur
??? 1417989074.M205847P32064.XXX,S=2582,W=2640:2,S
new
tmp

0 directories, 1 file

2) The message is still in INBOX folder but has got a "Trashed" flag

root@mail J:0 S:1 Maildir# tree {cur,new,tmp}
cur
??? 1417989074.M205847P32064.XXX,S=2582,W=2640:2,ST
new
tmp

0 directories, 1 file

3) I tried to expunge deleted messages from Trash folder

root@mail J:0 S:1 # doveadm expunge -u XXX mailbox Trash savedbefore 1s deleted

4) The message is still in INBOX folder (still uses storage)

root@mail J:0 S:1 Maildir# tree {cur,new,tmp}
cur
??? 1417989074.M205847P32064.XXX,S=2582,W=2640:2,ST
new
tmp

0 directories, 1 file

5) I deleted all messages with "Trashed" flag from INBOX folder and..

root@mail J:0 S:1 Maildir# doveadm expunge -u XXX mailbox INBOX savedbefore 1s deleted

6) The message has been removed from the hard-drive

root@mail J:0 S:1 Maildir# tree {cur,new,tmp}
cur
new
tmp

0 directories, 0 files

If a mail reader doesn't support "Expunge" every folder on exit, you will have a lot of deleted emails that use your storage and they are still in dovecot's index file which affects performance...

Your script is good but you should consider to "clean" users folders from deleted messages which have not been moved to Trash or Junk folders (like savedbefore 60d).

I would also change this line:

$DOVEADM -v expunge mailbox Trash -u $row savedbefore 90d

to:

$DOVEADM -v expunge mailbox Trash -u "$row" savedbefore 90d

or

$DOVEADM -v expunge mailbox Trash -u "${row}" savedbefore 90d

If somehow in user's name is a spacebar that would "crash" the script... and it is vulnerable to inject user's code (eg. read other users emails).

I hope you won't consider my message as something bad or negative. These are just my conclusions. You can always delete my comment :)

Best,

Rafal.

how do you get that Trashed tag? I expect that as soon as you delete the message by your client it is moved to .Trash

PS: I've never missed to reply/publish a comment, when it contains interesting things to learn. And I think this is the case, so thanks to you :)

Hi Roberto,

I have been busy recently...

Try for example in Thunderbird to delete message by pressing Shift + Delete when a message is highlighted. You can do the same through telnet and other mail clients.

I hope it helps.

Best,

Rafal.

Ok, I see. But I can't find a way to iterate over all mailboxes...

any help?

There is a simple solution when using vpopmail:

40 4 12 * * for user in $(/home/vpopmail/bin/vpopbull -nV); do /usr/local/bin/doveadm -v expunge -u $user \( mailbox Trash OR mailbox Junk \) SAVEDBEFORE 8w ; done