Razor2, Pyzor, Spamcop and DCC setup

July 14, 2021 Roberto Puzzanghera 0 comments

This page concerns the setup of several filtering networks which help spamassassin to decide if a given message is spam or not. Enabling them, together with the bayesian learning system, drastically improves the spamassassin efficiency in doing this.

Spamassassin User Preferences via SQL

July 12, 2021 Roberto Puzzanghera 20 comments

Info: http://spamassassin.apache.org/dist/sql/README - http://wiki.apache.org/spamassassin/UsingSQL

SpamAssassin can now load users' score files from an SQL database.  The concept here is to have a web application (PHP/perl/ASP/etc.) that will allow users to be able to update their local preferences on how SpamAssassin will filter their e-mail. The most common use for a system like this would be for users to be able to update the white/black list of addresses without the need for them to update their $HOME/.spamassassin/user_prefs file.

You can skip this page if you want to manage only global options via /etc/mail/spamassassin.

Be aware that user rules will be easily managed by means of the "sauprefs" plugin of Rouncube webmail.

Changelog

• Jul 12, 2021
  -bug fix: the "preference" varchar length in the database "userprefs" table was increased to 50 (was 30) to create space for long labels such as  "bayes_auto_learn_threshold_spam", which resulted truncated before the modification.

read more

Setting up a script for the Spamassassin's learning and reporting systems

June 20, 2021 Roberto Puzzanghera 0 comments

• Download the shell script

Now that we have the spam filters in place we have to train our bayesian system and report our spam to Razor, Pyzor and Spamcop.

The obvious thing that comes in mind at this point could be to call sa_learn and spamassassin --report in cascade when clicking in the Roundcube webmail's "Mark as Junk"  button (look  at the cmd_learn and multi_driver drivers of the markasjunk plugin), but this option has a couple of downsides:

• the learning process, the resulting journal syncing and the connection to several filtering networks takes up to 10 seconds, a time that our users don't want to wait.
• even worse, when they click the "Mark as Junk" button it is not always for a real spam message. For example, think about the regular newsletters that they no longer want to read and that they decide to conveniently label as spamming instead of unsubscribe them in the proper way.

Therefore it is better to run these two tasks inside a cronjob every night (and this is going to solve the first issue), processing the messages stored in a folder where the users had copied only real spam or ham messages (then fixing the second as well).

Installing Dovecot and sieve on a vpopmail + qmail server

June 20, 2021 Roberto Puzzanghera 69 comments

• Overview
• Installing
• Configuring
• Download my dovecot's config files
• dovecot.conf
• 10-auth.conf
• auth-sql.conf.ext
• 10-director.conf
• 10-logging.conf
• Logrotate
• 10-mail.conf
• 10-master.conf
• 10-ssl.conf
• 15-lda.conf
• 15-mailboxes.conf
• 20-imap.conf
• 20-lmtp.conf
• 20-pop3.conf
• 90-plugin.conf
• 90-quota.conf

Changelog

• June 20, 2021
  -15-mailboxes.conf: added Junk.TeachSpam and Junk.TeachNotSpam mailboxes to store messages for the learning and reporting systems (more info here)
• March 17, 2021
  - 90-quota.conf: quota definition adjusted in this way quota = maildir:: as the GETQUOTA command was not returning the quota (thanks a.key)
• March 2, 2021
  - "one table per domain" support added (--disable-many-domains)
  - added domains limits support to password_query (you have to compile vpopmail --enable-mysql-limits)
• January 29, 2021
  - auth-sql.conf.ext now uses the userdb's prefetch driver in order to perform one single query when doing the auth
  - dovecot-sql.conf.ext has been modified to allow authentication both with real and alias domains, provided that you patched vpopmail accordingly. More info in this page.
  - the iterate_query in the sql driver now extracts the "user" field (was "username") as required by the docs.

Overview

• Info: http://www.dovecot.org/
• Documentation: http://wiki2.dovecot.org
• Mail Server overview: http://wiki2.dovecot.org/MailServerOverview
• Download: http://www.dovecot.org/releases/2.3/
• Version: dovecot-2.3.15

Dovecot is an open source IMAP and POP3 email server for Linux/UNIX-like systems, written with security primarily in mind. Dovecot is an excellent choice for both small and large installations. It's fast, simple to set up, requires no special administration and it uses very little memory.

Patching qmail

June 19, 2021 Roberto Puzzanghera 303 comments

• Latest stable combined patch for netqmail-1.06 v. 2021.06.19 (MD5)

Changelog

The complete changelog and patch info are inside the README.PATCH file.

• 2021.06.19
  -chkuser: defined extra allowed characters in sender/rcpt addresses and added the slash to the list (tx Thomas). diff here
• 2021.06.12
  -RSA key and DH parameters are created 4096 bit long also in Makefile-cert. qmail-smtpd.c and qmail-remote.c updated accordingly (tx Eric Broch).
  -Makefile-cert: the certs will be owned by vpopmail:vchkpw
• 2021.03.21
  -update_tmprsadh.sh: RSA key and DH parameters increased to 4096 bits
• 2020.12.04
  received.c: some adjustments to compile with gcc-10 (diff here)
• 2020.07.29
  -dk-filter: corrected a bug where dk-filter was using DKIMDOMAIN unconditionally. Now it uses DKIMDOMAIN only if _SENDER is null (tx Manvendra Bhangui).
• 2020.07.27
  -added a fix for CVE-2005-2513 (tx C)
• 2020.04.25
  -qmail-smtpd.c: added rcptcount = 0; in smtp_rset function to prevent the maxrcpto error if control/maxrcpt limit has been exceeded in multiple messages sent sequentially rather than in a single mail (tx Alexandre Fonceca)
• 2020.04.16
  -qmail-remote-logging patch added (more info here)
• 2020.04.10
  -DKIM patch updated to v. 1.28
  * outgoing messages from null sender ("<>") will be signed as well with the domain in env variable DKIMDOMAIN
  * declaring NODK env variable disables old domainkeys signature, while defining NODKIM disables DKIM.

Roundcube plugins

April 21, 2021 Roberto Puzzanghera 24 comments

UPDATE as of April 21, 2021: the markasjunk's multi_driver, when enabled, seems to prevent the display of the attached images. Why this driver is related to this problem is a mistery. Any comment on this will be welcome.

---

• Official repository: http://plugins.roundcube.net/

My enabled plugins are (at the moment):

• password, which is already included in the plugins folder
• managesieve, which writes sieve scripts to filter the incoming mails (reject, move to specific folders etc.). Note that in order to use it you must have Dovecot managesieve enabled.
• SpamAssassin-User-Prefs-SQL, which writes the spamassassin user preferences in the DB. The user will be allowed to create a black/white list, to adjust the required_score and so on.
• markasjunk. You can add the sender's email address to the blacklist, or run a command such as sa_learn. Requires sauprefs.
• rcguard. This plugin logs failed login attempts and requires users to go through a reCAPTCHA verification process when the number of failed attempts go too high.
• Context Menu. Adds context menus to the message list, folder list and address book. Menu includes the abilities mark messages as read/unread, delete, reply and forward.
• autologon. Autologin from external Site e.g. (CMS, Portal ...)
• logout_redirect. Modified version to only redirect to the homepage (depending on the domain part of the default identity)
• newmail_notifier. can notify new mail focusing browser window and changing favicon, playing a sound and  displaying desktop notification (using webkitNotifications feature).
• carddav. CardDav client. You can sync your addressbook against a CardDav server like nextcloud or SoGO.
• enigma adds support for viewing and sending of signed and encrypted messages in PGP (RFC 2440) and PGP/MIME (RFC 3156) format

Installing and configuring vpopmail

April 18, 2021 Roberto Puzzanghera 70 comments

• vpopmail version: 5.4.33
• Combined patch v. 2021.07.08
• Changelog
• More info here
• README.vdelivermail

Vpopmail provides an easy way to manage virtual email domains and non /etc/passwd email accounts on your mail servers.

The purpose of this note is to show how to use Mysql as the authentication system. Having a users database also offers the advantage of communicating with the database via PHP, and creating web-based user interfaces to manage accounts.

Patch details

The patch we'll apply puts together the following patches:

• sql-aliasdomains patch, which makes vpopmail save the aliasdomains to MySQL. This makes the dovecot sql auth driver aware of the aliasdomains, provided that you modify the sql query as well (see the dovecot page for more info).
• defaultdelivery patch, which makes vpopmail to copy your favourite delivery agent (stored in QMAILDIR/control/defauldelivery) into the .qmail-default file of any newly created domain, overriding the default vpopmail's behaviour, where vpopmail copies its delivery agent vdelivermail. You have to configure with --enable-defaultdelivery to enable this.
  If the functionality is disabled (--disable-defaultdelivery, which is the default option) vdelivermail is installed with the "delete" option instead of "bounce-no-mailbox", which is not reasonable anymore.
• dovecot-pwd_query patch
  If you want to use the dovecot's sql auth driver with one table for each domain (--disable-many-domains) you have to heavily customize your password query. With this patch vpopmail installs the sql procedure and functions in the database when you create a new domain. The procedure can be called by dovecot to perform the auth.
  The sql stuff supports aliasdomains and mysql limits and will be loaded from ~/vpopmail/etc/pwd-query_disable-many-domains.sql. You can customize the sql procedure editing this file.
  You have to configure with --enable-mysql-bin=PATH as we have to install the procedure calling the mysql bin as a shell command (no way to load an sql query from a file in C language, comments welcome).
• gcc-10-compat patch, which gets vpopmail to compile with gcc-10

Dovecot vpopmail-auth driver removal. Migrating to the SQL driver

March 9, 2021 Roberto Puzzanghera 36 comments

Those who are still using the Dovecot's vpopmail auth driver should consider a migration to another backend, as on January 4, 2021 dovecot-2.3.13 was released and the vpopmail auth driver removed (more info here).

I'll show below how to support domain aliases with the sql driver both with all domains in the same vpopmail table and with one table for each domain (--disable-many-domains). You can find how to setup the driver in this page. A short reference to vpopmail's vconvert program is presented toward the bottom of this page, in case one is planning to switch to sql.

If you browse the comments below you'll find some other nice solutions to replace the vpopmail driver:

• Tyler Simkin posted his auth.lua file (enhanced by Rick Richards to work with encrypted passwords)
• Laurent Bercot posted a solution based on passwd-file driver
• Pablo Murillo improved the sql password_query to work with one table for each domain
• erdgeist showed how to convert cdb accounts to postgres

Saving vpopmail's aliasdomains to MySQL

As some commentators have pointed out, switching to the dovecot's sql auth driver can be painful if one has domain aliases. I will show below how to make dovecot aware of the vpopmail's aliasdomains, so that a user who tries to login with a domain alias can pass the authentication.

The idea is to save the pairs alias/domain in a new "aliasdomains" MySQL table, for example:

MariaDB [vpopmail]> SELECT * FROM aliasdomains; 
+----------------------+----------------------+ 
| alias                | domain               | 
+----------------------+----------------------+ 
| alias.net            | realdomain.net       | 
+----------------------+----------------------+

...and then modify the dovecot's sql query in order to select the user's domain from this table in case the domain is an alias or from the vpopmail table otherwise.

I patched vpopmail so that it  will transparently do the sql stuff when creating/deleting the alias in the usual way by means of the vaddaliasdomain/vdeldomain vpopmail's programs.

Adding clamav-unofficial-sigs

October 30, 2020 Roberto Puzzanghera 0 comments

• More info here

The clamav-unofficial-sigs script provides a simple way to download, test, and update third-party signature databases provided by Sanesecurity, FOXHOLE, OITC, Scamnailer, BOFHLAND, CRDF, Porcupine, Securiteinfo, MalwarePatrol, Yara-Rules Project, urlhaus, etc. The script will also generate and install cron, logrotate, and man files.

qmailadmin

September 1, 2020 Roberto Puzzanghera 64 comments

• Author:  Inter7
• Version: 1.2.16
• Download the sources from  http://sourceforge.net/projects/qmailadmin/files/
• Combined patch v. 20210312

qmailAdmin is a free software package that provides a web interface for managing a qmail system with virtual domains. It provides admin for adding/deleting users, Aliases, Forwards, Mailing lists and Autoresponders.

Combined patch details

• qmailadmin-skin, a patch that I created during covid-19 spare time, provides a new responsive skin to the control panel. It modifies everything under the html dir and many .c files in order to adjust the html embedded into the source files. Added a stylesheet style.css in the images folder and a couple of png files for the qmail logo. It should be much easier to modify the qmailadmin's skin from now on.
• patch to call cracklib in order to check for the password strenght. This should avoid unsafe accounts created by domain administrators such as "test 123456".
• A patch (thanks to Tony, original author unknown) which gets qmailadmin to have authentication failures logged. This makes possible to ban malicious IPs via fail2ban. It is required to create the log file /var/log/qma-auth.log initially and assign write priviledges to apache.
• ezmlm-idx 7 compatibility patch (author unknown), which restores the compatibility with ezmlm-idx-7 (thanks to J.D. Trolinger for the advice).
• a fix to the catchall account (thanks to Luca Franceschini).
• another fix to autorespond.c to correct the way .qmail files are modified

Changelog

• 2021.03.12
  -patch cleanup
• 2020.09.02
  -mod_user.html: cleaned the html as it was printing unneeded strings
• 2020.08.10
  - mod_user.html: added the "value" attribute to the name/gecos input tag
  - Makefile.in: added a line to install the css, as already done for Makefile.am
  (tx Pablo Murillo)
• 2020.05.22
  - mod_user.html: removed the "required" attribute on password field, to allow modifications in case of no password change

 

• 1
• 2
• Next