Setting up a script for the Spamassassin's learning and reporting systems

June 20, 2021 Roberto Puzzanghera 0 comments

• Download the shell script

Now that we have the spam filters in place we have to train our bayesian system and report our spam to Razor, Pyzor and Spamcop.

The obvious thing that come in mind at this point could be to call sa_learn and spamassassin --report in cascade when clicking in the Roundcube webmail's "Mark as Junk" button (look at markasjunk plugin cmd_learn and multi_driver drivers), but this option has a couple of downsides:

• the learning process, the resulting journal syncing and the connection to several filtering networks takes up to 10 seconds, a time that our users don't want to wait.
• even worse, when they click the "Mark as Junk" button it is not always for a real spam message. For example, think about the regular newsletters that they no longer want to read and that they decide to conveniently label as spamming instead of unsubscribe them in the proper way.

Therefore it is better to run these two tasks inside a cronjob every night (and this is going to solve the first issue), processing the messages stored in a folder where the users had copied only real spam or ham messages (then fixing the second as well).

Installing Dovecot and sieve on a vpopmail + qmail server

June 20, 2021 Roberto Puzzanghera 67 comments

• Overview
• Installing
• Configuring
• Download my dovecot's config files
• dovecot.conf
• 10-auth.conf
• auth-sql.conf.ext
• 10-director.conf
• 10-logging.conf
• Logrotate
• 10-mail.conf
• 10-master.conf
• 10-ssl.conf
• 15-lda.conf
• 15-mailboxes.conf
• 20-imap.conf
• 20-lmtp.conf
• 20-pop3.conf
• 90-plugin.conf
• 90-quota.conf

Changelog

• June 20, 2021
  -15-mailboxes.conf: added Junk.TeachSpam and Junk.TeachNotSpam mailboxes to store messages for the learning and reporting systems (more info here)
• March 17, 2021
  - 90-quota.conf: quota definition adjusted in this way quota = maildir:: as the GETQUOTA command was not returning the quota (thanks a.key)
• March 2, 2021
  - "one table per domain" support added (--disable-many-domains)
  - added domains limits support to password_query (you have to compile vpopmail --enable-mysql-limits)
• January 29, 2021
  - auth-sql.conf.ext now uses the userdb's prefetch driver in order to perform one single query when doing the auth
  - dovecot-sql.conf.ext has been modified to allow authentication both with real and alias domains, provided that you patched vpopmail accordingly. More info in this page.
  - the iterate_query in the sql driver now extracts the "user" field (was "username") as required by the docs.

Overview

• Info: http://www.dovecot.org/
• Documentation: http://wiki2.dovecot.org
• Mail Server overview: http://wiki2.dovecot.org/MailServerOverview
• Download: http://www.dovecot.org/releases/2.3/
• Version: dovecot-2.3.14

Dovecot is an open source IMAP and POP3 email server for Linux/UNIX-like systems, written with security primarily in mind. Dovecot is an excellent choice for both small and large installations. It's fast, simple to set up, requires no special administration and it uses very little memory.

Patching qmail

June 19, 2021 Roberto Puzzanghera 298 comments

• Latest stable combined patch for netqmail-1.06 v. 2021.12.19 (MD5)

Changelog

The complete changelog and patch info are inside the README.PATCH file.

• 2021.06.19
  -chkuser: defined extra allowed characters in sender/rcpt addresses and added the slash to the list (tx Thomas). diff here
• 2021.06.12
  -RSA key and DH parameters are created 4096 bit long also in Makefile-cert. qmail-smtpd.c and qmail-remote.c updated accordingly (tx Eric Broch).
  -Makefile-cert: the certs will be owned by vpopmail:vchkpw
• 2021.03.21
  -update_tmprsadh.sh: RSA key and DH parameters increased to 4096 bits
• 2020.12.04
  received.c: some adjustments to compile with gcc-10 (diff here)
• 2020.07.29
  -dk-filter: corrected a bug where dk-filter was using DKIMDOMAIN unconditionally. Now it uses DKIMDOMAIN only if _SENDER is null (tx Manvendra Bhangui).
• 2020.07.27
  -added a fix for CVE-2005-2513 (tx C)
• 2020.04.25
  -qmail-smtpd.c: added rcptcount = 0; in smtp_rset function to prevent the maxrcpto error if control/maxrcpt limit has been exceeded in multiple messages sent sequentially rather than in a single mail (tx Alexandre Fonceca)
• 2020.04.16
  -qmail-remote-logging patch added (more info here)
• 2020.04.10
  -DKIM patch updated to v. 1.28
  * outgoing messages from null sender ("<>") will be signed as well with the domain in env variable DKIMDOMAIN
  * declaring NODK env variable disables old domainkeys signature, while defining NODKIM disables DKIM.

Roundcube plugins

April 21, 2021 Roberto Puzzanghera 24 comments

UPDATE as of April 21, 2021: the markasjunk's multi_driver, when enabled, seems to prevent the display of the attached images. Why this driver is related to this problem is a mistery. Any comment on this will be welcome.

---

• Official repository: http://plugins.roundcube.net/

My enabled plugins are (at the moment):

• password, which is already included in the plugins folder
• managesieve, which writes sieve scripts to filter the incoming mails (reject, move to specific folders etc.). Note that in order to use it you must have Dovecot managesieve enabled.
• SpamAssassin-User-Prefs-SQL, which writes the spamassassin user preferences in the DB. The user will be allowed to create a black/white list, to adjust the required_score and so on.
• markasjunk. You can add the sender's email address to the blacklist, or run a command such as sa_learn. Requires sauprefs.
• rcguard. This plugin logs failed login attempts and requires users to go through a reCAPTCHA verification process when the number of failed attempts go too high.
• Context Menu. Adds context menus to the message list, folder list and address book. Menu includes the abilities mark messages as read/unread, delete, reply and forward.
• autologon. Autologin from external Site e.g. (CMS, Portal ...)
• logout_redirect. Modified version to only redirect to the homepage (depending on the domain part of the default identity)
• newmail_notifier. can notify new mail focusing browser window and changing favicon, playing a sound and  displaying desktop notification (using webkitNotifications feature).
• carddav. CardDav client. You can sync your addressbook against a CardDav server like nextcloud or SoGO.
• enigma adds support for viewing and sending of signed and encrypted messages in PGP (RFC 2440) and PGP/MIME (RFC 3156) format

Installing and configuring vpopmail

April 18, 2021 Roberto Puzzanghera 57 comments

• vpopmail version: 5.4.33
• Combined patch v. 2021.04.18 (details below)
• More info here
• README.vdelivermail

Vpopmail provides an easy way to manage virtual email domains and non /etc/passwd email accounts on your mail servers.

The purpose of this note is to show how to use Mysql as the authentication system. Having a users database also offers the advantage of communicating with the database via PHP, and creating web-based user interfaces to manage accounts.

Changelog

• April 18, 2021
  - code cleanup
• March 27, 2021
  - bug fixes in the defaultdelivery patch: increased the buffer for the .qmail-default file path, as in particular cases of long path/domain names it will result truncated. Fixed another bug where the .qmail.default file where opened twice.
  - now if vdelivermail is installed the "delete" option will be used instead of "bounce-no-mailbox", which is not reasonable anymore
• March 9, 2021
  - the vpopmail patch now installs the sql code needed for "one table per domain" (--disable-many-domains) in ~/vpopmail/etc/pwd-query_disable-many-domains.sql and creates the sql procedure if needed. Of course this add-on to vpopmail will be completely transparent when you compile with the default option --enable-many-domains
• March 2, 2021
  - added mysql-limits support. See changes in dovecot-sql password_query as well.
• February 26, 2021
  - added a "defaultdelivery patch" to the package of patches. configure --enable-defaultdelivery to enable it.
• February 15, 2021
  - fix in the configure file. An autoreconf is needed as I modified the configure.in and Makefile.am files
• February 10, 2021
  - a C program vsavealiasdomains can now save all the existing domain aliases to MySQL. It can be useful in case of migrations to the dovecot's sql auth driver.
• February 5, 2021
  - sql-aliasdomains patch added and combined patch released
• December 12, 2020
  - patch to get vpopmail compatible with gcc-10

Patch details

The patch we'll apply puts together the following patches:

• sql-aliasdomains patch, which makes vpopmail save the aliasdomains to MySQL. This makes the dovecot sql auth driver aware of the aliasdomains, provided that you modify the sql query as well (see the dovecot page for more info).
• defaultdelivery patch, which makes vpopmail to copy your favourite delivery agent (stored in QMAILDIR/control/defauldelivery) into the .qmail-default file of any newly created domain, overriding the default vpopmail's behaviour, where vpopmail copies its delivery agent vdelivermail. You have to configure with --enable-defaultdelivery to enable this.
  If the functionality is disabled (--disable-defaultdelivery, which is the default option) vdelivermail is installed with the "delete" option instead of "bounce-no-mailbox", which is not reasonable anymore.
• dovecot-pwd_query patch
  If you want to use the dovecot's sql auth driver with one table for each domain (--disable-many-domains) you have to heavily customize your password query. With this patch vpopmail installs the sql procedure and functions in the database when you create a new domain. The procedure can be called by dovecot to perform the auth.
  The sql stuff supports aliasdomains and mysql limits and will be loaded from ~/vpopmail/etc/pwd-query_disable-many-domains.sql. You can customize the sql procedure editing this file.
  You have to configure with --enable-mysql-bin=PATH as we have to install the procedure calling the mysql bin as a shell command (no way to load an sql query from a file in C language, comments welcome).
• gcc-10-compat patch, which gets vpopmail to compile with gcc-10

Dovecot vpopmail-auth driver removal. Migrating to the SQL driver

March 9, 2021 Roberto Puzzanghera 36 comments

Those who are still using the Dovecot's vpopmail auth driver should consider a migration to another backend, as on January 4, 2021 dovecot-2.3.13 was released and the vpopmail auth driver removed (more info here).

I'll show below how to support domain aliases with the sql driver both with all domains in the same vpopmail table and with one table for each domain (--disable-many-domains). You can find how to setup the driver in this page. A short reference to vpopmail's vconvert program is presented toward the bottom of this page, in case one is planning to switch to sql.

If you browse the comments below you'll find some other nice solutions to replace the vpopmail driver:

• Tyler Simkin posted his auth.lua file (enhanced by Rick Richards to work with encrypted passwords)
• Laurent Bercot posted a solution based on passwd-file driver
• Pablo Murillo improved the sql password_query to work with one table for each domain
• erdgeist showed how to convert cdb accounts to postgres

Saving vpopmail's aliasdomains to MySQL

As some commentators have pointed out, switching to the dovecot's sql auth driver can be painful if one has domain aliases. I will show below how to make dovecot aware of the vpopmail's aliasdomains, so that a user who tries to login with a domain alias can pass the authentication.

The idea is to save the pairs alias/domain in a new "aliasdomains" MySQL table, for example:

MariaDB [vpopmail]> SELECT * FROM aliasdomains; 
+----------------------+----------------------+ 
| alias                | domain               | 
+----------------------+----------------------+ 
| alias.net            | realdomain.net       | 
+----------------------+----------------------+

...and then modify the dovecot's sql query in order to select the user's domain from this table in case the domain is an alias or from the vpopmail table otherwise.

I patched vpopmail so that it  will transparently do the sql stuff when creating/deleting the alias in the usual way by means of the vaddaliasdomain/vdeldomain vpopmail's programs.

Changelog

• March 9, 2021
  - the vpopmail patch now installs the sql code needed for "one table per domain" (--disable-many-domains) in ~/vpopmail/etc/pwd-query_disable-many-domains.sql and creates the sql procedure if needed. Of course this add-on to vpopmail will be completely transparent when you compile with the default --enable-many-domains option.
  I think that with this one we are arrived. Please test and report bugs, as this is fresh code.
• March 2, 2021
  - "one table per domain" support added (--disable-many-domains)
  - mysql-limits support added
• February 15, 2021
  - fix in the configure file. An autoreconf is needed as I modified the configure.in and Makefile.am files
• February 10, 2021
  - a C program vsavealiasdomains can now save all the existing domain aliases to MySQL. It can be useful in case of migrations to the dovecot's sql auth driver.
• Feb 5, 2021
  - The patch has been improved: the sql-aliasdomains stuff is now done by means of the vpopmail's C programs and functions.
• Feb 3, 2021
  - new patch and script released. Just configure --enable-sql-aliasdomains (default) and forget. The dbtable will be created the first time you will create an aliasdomain.
• Jan 18, 2021
  - now everything is inside a vpopmail patch. The aliasdomain sql records will be created/deleted transparently when using vaddaliasdomain/vdeldomain in the usual way, provided that you have created the aliasdomains dbtable
• Jan 17, 2021
  - I modified the dovecot's sql query so that a pair real_domain/real_domain is not needed anymore in the dbtable
• Jan 13, 2021
  - added support for sql aliasdomains

Adding clamav-unofficial-sigs

October 30, 2020 Roberto Puzzanghera 0 comments

• More info here

The clamav-unofficial-sigs script provides a simple way to download, test, and update third-party signature databases provided by Sanesecurity, FOXHOLE, OITC, Scamnailer, BOFHLAND, CRDF, Porcupine, Securiteinfo, MalwarePatrol, Yara-Rules Project, urlhaus, etc. The script will also generate and install cron, logrotate, and man files.

qmailadmin

September 1, 2020 Roberto Puzzanghera 64 comments

• Author:  Inter7
• Version: 1.2.16
• Download the sources from  http://sourceforge.net/projects/qmailadmin/files/
• Combined patch v. 20210312

qmailAdmin is a free software package that provides a web interface for managing a qmail system with virtual domains. It provides admin for adding/deleting users, Aliases, Forwards, Mailing lists and Autoresponders.

Combined patch details

• qmailadmin-skin, a patch that I created during covid-19 spare time, provides a new responsive skin to the control panel. It modifies everything under the html dir and many .c files in order to adjust the html embedded into the source files. Added a stylesheet style.css in the images folder and a couple of png files for the qmail logo. It should be much easier to modify the qmailadmin's skin from now on.
• patch to call cracklib in order to check for the password strenght. This should avoid unsafe accounts created by domain administrators such as "test 123456".
• A patch (thanks to Tony, original author unknown) which gets qmailadmin to have authentication failures logged. This makes possible to ban malicious IPs via fail2ban. It is required to create the log file /var/log/qma-auth.log initially and assign write priviledges to apache.
• ezmlm-idx 7 compatibility patch (author unknown), which restores the compatibility with ezmlm-idx-7 (thanks to J.D. Trolinger for the advice).
• a fix to the catchall account (thanks to Luca Franceschini).
• another fix to autorespond.c to correct the way .qmail files are modified

Changelog

• 2021.03.12
  -patch cleanup
• 2020.09.02
  -mod_user.html: cleaned the html as it was printing unneeded strings
• 2020.08.10
  - mod_user.html: added the "value" attribute to the name/gecos input tag
  - Makefile.in: added a line to install the css, as already done for Makefile.am
  (tx Pablo Murillo)
• 2020.05.22
  - mod_user.html: removed the "required" attribute on password field, to allow modifications in case of no password change

 

simscan

August 4, 2020 Roberto Puzzanghera 50 comments

• Info: https://github.com/qmail/simscan/releases
• John Simpson's simscan page (patch and a lot of info)
• README
• Download (local copy)
• Combined patch used
• Version: simscan-1.4.1
• Old 1.4.0 repo: http://sourceforge.net/projects/simscan/files/

Simscan is a simple program that enables the qmail smtpd service to reject viruses, spam, and block attachments during the SMTP conversation so the processing load on the email system is kept to a minimum.

Combined patch details

Version 1.4.1 is a fork of the original simscan by Inter7. The sources have been polished and modernized a bit and contain a number of bug fixes and patches, including almost all the patches by jms (the only missing one is the "debug" patch which we will apply below) and the bug fix by Gustavo Castro that I had in my previous bundle of patches. Therefore the new patch simply adds the following:

• the jms "debug" patch, to improve the debugging of simscan on qmail-smtpd log;
• a bug fix by Bob Greco where a received message with multiple 'local' recipients executes spamc as null user and not as the user extracted from the first local recipient.

Running OpenBoard in a window

May 10, 2021 Roberto Puzzanghera 6 comments

• OpenBoard home page
• Download my patch
• Github discussion
• Slackbuild (Slackware users)

These days I'm forced again to do lessons from remote. My school asked me to refer to Google Meet for the videoconferences and one thing I disliked was the Jam interactive whiteboard, which is completely inadequate for scientific subjects. On the other hand OpenBoard, my favourite board tool that I successfully use with Zoom, seemed not to be recognized as an application to be shared, because it runs fullscreen.

After some googleing I found a patch from this guy (I big thank for his work!) which forces OpenBoard to run in a window, but at the cost of passing a variable at compilation time. I modified the logic of that patch so that a user can set how OpenBoard will run just modifying an option in the config file. The "run windowed" feature is disabled by default, so it will not bother those teachers who are already familiar with the interface, but it can be easily switched on by advanced users.

• 1
• 2
• Next