SQL valias with sieve solution for qmail. New patches and Roundcube plugin

November 5, 2023 by Roberto Puzzanghera 6 comments

Changelog

• Nov 5, 2023
  -bug fix: vpopmail defaultdelivery patch: it won't create the .qmail file in case control/defaultdelivery already has vdelivermail, in order to prevent a vpopmail loop
  -qmailforward RC plugin: it won't create the copy record if $config['qmailforward_defaultdelivery'] contains 'vdelivermail'

---

Among the various problems that I set out to fix sooner or later, I think I've finally solved one: the impossibility of having the vpopmail aliases saved in the database (--enable-valias) and the sieve rules at the same time.

But before presenting the solutions, let's clarify the problem to be solved, which has also been discussed in several threads of this blog. It is no coincidence that the solution I identified was partly inspired by the (re)reading of some  of the visitors posts.

The problem

Usually vpopmail copies its delivery agent (vdelivermail) into the .qmail-default file of newly created domains. This makes it impossible to use Dovecot filters based on sieve rules, as this would require Dovecot-LDA.

On the other hand, if we set the Dovecot delivery agent to .qmail-default in order to have the sieve rules, when the user uses the Dovecot filters to generate a forward (s)he will unintentionally cause the violation of the SPF, because qmail is out of the question and srsfilter cannot be launched to rewrite the sender's address.

The best thing would be for the user to manage the aliases via dot-qmail, but this can only be done by the domain administrator, while the aliases stored on SQL database are not an option, as Dovecot has nothing to do with them.

These are problems that we know well due to the protest emails of users who see their forwarding messages bounced back because of the SPF/DKIM/DMARC check failure. But now we're going to fix it once and for all :-)

Patching qmail

November 20, 2023 by Roberto Puzzanghera 479 comments

• Latest stable combined patch for netqmail-1.06 v. 2023.11.20 (MD5)
• Changelog
• Readme

Changelog

• Nov 20, 2023
  -dkim:
  * The patch now by default excludes X-Arc-Authentication-Results
  * dkim can additionally use the environment variable EXCLUDE_DKIMSIGN to include colon separated list of headers to be excluded from signing (just like qmail-dkim). If -X option is used with dk-filter, it overrides the value of EXCLUDE_DKIMSIGN.
• Sep 26, 2023
  -surblfilter logs the rejected URL in the qmail-smtpd log. It can now inspect both http and https URLs.
  -Improvements in man dkim.9, qmail-dkim.9 and surblfilter.9
• Sep 5, 2023
  -DKIM patch upgraded to v. 1.42
  *dk-filter.sh: "source $envfn" has been replaced with ". $envfn" in oder to work for pure bourne shells
  *minor corrections to the man pages
• Aug 20, 2023 (diff)
  -install a sample control/smtpplugins file in case it does not exist yet, to avoid "unable to read control" crash.
• Jul 5, 2023 (diff)
  -vpopmail-dir.sh: now uses getent to gain compatibility with alpine/docker (tx BenV)
• Jun 30, 2023
  -DKIM patch upgraded to v. 1.41
  *dknewkey will allow domains in control/domainkey
  *Made a few adjustments to the man pages and dkimsign.cpp for DKIMDOMAIN to work with qmail-smtpd (in case some configures qmail-smtpd to sign instead of the usual dk-filter/qmail-remote)
  -The broken link based on pobox.com in the default SPF error explanation was changed to https://mxtoolbox.com/SuperTool.aspx?action=spf
• Jun 18, 2023 (diff)
  -vpopmail uid and gid are determined dinamically instead of assigning 89:89 ids by default
  -vpopmail install directory determined dinamically (was /home/vpopmail). Now the variable in the conf-cc file is determined as well.
  Feel free to post any issue in the comments as I'm not sure that /bin/sh will work in all Linux.
• Feb 19, 2023
  - dkim patch upgraded to v. 1.37
  * ed25519 support​ (RFC 8463)
  * dropped old yahoo's domainkeys stuff (no longer need the libdomainkeys.a library)

Installing and configuring vpopmail

November 11, 2023 by Roberto Puzzanghera 110 comments

• Inter7's original page
• vpopmail version: 5.4.33
• Combined patch v. 2023.11.11
• Changelog
• More info here
• README.vdelivermail

Vpopmail provides an easy way to manage virtual email domains and non /etc/passwd email accounts on your mail servers.

The purpose of this note is to show how to use Mysql as the authentication system. Having a users database also offers the advantage of communicating with the database via PHP, and creating web-based user interfaces to manage accounts.

Changelog

• Nov 11, 2023
  - defauldelivery patch: vmakedotqmail won't create users' .qmail if control/defaultdelivery already has vdelivermail.
• Oct 23, 2023
  - defaultdelivery patch: .qmail file won't be created if control/defaultdelivery already has vdelivermail.
• Oct 13, 2023
  - added "s/qmail cdb" patch, which gets vpopmail to locate correctly the qmail assign.cdb for s/qmail users. s/qmail users should configure vpopmail with --enable-sqmail-cdb
• Sep 5, 2023
  - changed configuration option --enable-logging=e (was p). Now failed attempts will be logged with no password shown.
• Aug 27, 2023
  - new combined patch. More info here
  * The logic of the defaultdelivery patch/feature has been revised. If configured with --enable-defauldelivery vpopmail will save control/defauldelivery in the user's .qmail and vdelivermail LDA in the domain's .qmail-default file. This will achieve multiple benefits: you have qmail forwards and sieve together and valias available. The valias table schema was changed as well.

read more

daemontools

September 3, 2023 by Roberto Puzzanghera 18 comments

• Info here: http://cr.yp.to/daemontools.html
• Download: http://cr.yp.to/daemontools/daemontools-0.76.tar.gz

daemontools is a collection of tools for managing UNIX services. It monitors qmail services and saves error messages to one or more logs.

Changelog

• Sep 3, 2023
  -Buffer Overflow fixed in timestamp.c (patch multilog-readable_datetime, Ubuntu 22.04). It was causing empty log files. (thanks Bai Borko and KPC)
• Jun 30, 2023
  -added my multilog_readable-datetime patch which replaces the timestamp in the log lines with a human readable datetime:
  
  

  2023-06-28 16:17:26.501272173 tcpserver: status: 0/200/0

  Do not install it if you prefer to stick with the timestamp.
  -if you install this patch you have to download again the convert-multilog program. In case you decide to stick with the original timestamp, then use the original one. (diff)

read more

Denying bad DNS HELO/EHLOs

August 16, 2023 by Roberto Puzzanghera 2 comments

Changelog

• Aug 17, 2023
  - C++ version (testing)
• Aug 13, 2023
  - v. 8.2.0 bug fix: segfault in case of no result in DNS record
  - default action changed to GNLR
• Jul 27, 2023
  - P will now pass through making all filters ignored. You are invited to remove this option if already using this program.
  - added G filter, i.e. HELO/EHLO with an invalid syntax are denied (random strings but also typing errors like sagredo..eu).
  - compile with -lpcre
  - code revision. Please report any issue.

The RFC-821 Section 3.5 states that 

The sender-SMTP MUST ensure that the <domain> parameter in a HELO command is a valid principal host domain name for the client host. As a result, the receiver-SMTP will not have to perform MX resolution on this name in order to validate the HELO parameter.

The HELO receiver MAY verify that the HELO parameter really corresponds to the IP address of the sender. However, the receiver MUST NOT refuse to accept a message, even if the sender's HELO command fails verification.

Not denying clients with a bad HELO/EHLO DNS can be considered a wise thing, just to avoid to update too frequently our welcomelist for those clients who didn't set up their DNS properly.

On the other hand, it is a matter of fact that most spammers use fake domains -sometimes our own domains-, or even random strings or not solving domains, as their HELO/EHLOs.

For example, consider the following log lines (I have plenty of them in my logs):

2022-02-01 10:19:53.142643500 helo-dns-check: HELO [yq3H9cDKgS] from [109.237.103.41] doesn't solve
2022-02-01 09:53:05.772497500 helo-dns-check: HELO [sagredo.eu] is a local domain but IP [183.240.55.119] is not a RELAYCLIENT

I think that at least such kind of failures should be blocked.

I'll explain below how to set up a filter which denies clients with one of these particular DNS failures:

1. HELO/EHLOs domains with an invalid syntax. Random strings but also typing errors like sagredo..eu will be banned.
2. fake HELO/EHLOs containing one of our domains, when RELAYCLIENT is NOT set and the HELO/EHLO matches one of our IPs. You can safely turn on this one.
3. not solving HELO/EHLOs domains (no A record). You'll get some false positive if you turn this on, as clients whose administrator forgot to add the A record will be banned.
4. clients whose remote IP doesn't match the A record. This is completely against RFC-821, so my configuration will not refuse these connections, just log them.

Installing a Let's Encrypt certificate for your qmail, dovecot and apache servers

August 6, 2023 by Roberto Puzzanghera 24 comments

Changelog

• Aug 6, 2023
  The certificates installation is now based on dehydrated. The previous documentation based on certbot will be left as is at the bottom of this page, but it won't be updated anymore.
• May 18, 2023
  added the option --key-type rsa to the certbot command, to avoid that certbot will silently default to ECDSA the private key format, which results not understandable by my openssl-1.1. In this way the format of the private key will be RSA. More info here.

 

• Let's Encrypt site

To enable HTTPS on your website, you need to get a certificate (a type of file) from a Certificate Authority (CA). Let’s Encrypt is a CA. In order to get a certificate for your website’s domain from Let’s Encrypt, you have to demonstrate control over the domain. With Let’s Encrypt, you do this using software that uses the ACME protocol which typically runs on your web host.

Upgrading notes

In case you have to replace a working installation of certificates based on certbot, you don't need to take any precautions as they won't be overwritten by the new ones generated by dehydrated. Therefore it is possible to run our tests on the production server itself. Of course it will be good to use a test domain while tinkering with Apache.

dehydrated installation

• github dehydrated page
• AlienBob article  (for Slackware users, but the informations are generic and suitable for all Linux distributions) 

Dehydrated is a client for signing certificates with an ACME-server (e.g. Let's Encrypt) implemented as a relatively simple (zsh-compatible) bash-script. This client supports both ACME v1 and the new ACME v2 including support for wildcard certificates!

dehydrated is a simple shell program that requires no dependencies, unlike the official certbot program, which needs a very long set of python libraries. Maintaining all this volume of programs proved to be quite time expensive, which is why I decided to switch to dehydrated.

Configuring the Sender Rewriting Scheme (SRS) on qmail

July 11, 2023 by Roberto Puzzanghera 2 comments

• open-spf.org
• libsrs2 site
• Marcelo Coelho's patch for qmail

SPF "breaks" email forwarding. SRS is a way to fix it. SRS is a simple way for forwarding MTAs to rewrite the sender address. The original concept was published in draft-mengwong-sender-rewrite and further expanded on in a paper by Shevek.

Configuration

Configure srsfilter, so that it will be called when an email for the srs user is received:

echo "| /var/qmail/bin/srsfilter" > /var/qmail/alias/.qmail-srs-default

Then create and configure a virtual domain to be used exclusively for SRS purposes. Be aware that this virtual domain should not be created by the usual vadddomain program, as it exists just to run srsfilter via the alias/.qmail-srs-default account that we created before and its definition is different from the vpopmail's virtual domains.

echo srs.mydomain.tld:srs >> /var/qmail/control/virtualdomains

Refer to the Life With Qmail bible to understand the logic behind, expecially for what virtual domains, aliases, .qmail and extensions addresses  are concerned. An explanation is also provided below in the testing section.

Add srs.mydomain.tld to rcpthosts so that qmail-smtpd will know that it has to deliver locally all messages for that domain. Do not add it to control/locals otherwise the virtualdomains file will be ignored and srsfilter will not be run.

echo srs.mydomain.tld >> /var/qmail/control/rcpthosts

Add srs.mydomain.tld in the srs_domain control file, so that srsfilter will use it in the rewritten address for all virtual hosts. Let's also create the srs_secret file, as well. It is a random string to generate and check SRS addresses.

echo srs.mydomain.tld > /var/qmail/control/srs_domain
echo "xxxxxxxxxxxxxxxxxxxxxx" > /var/qmail/control/srs_secrets

These are the only mandatory settings; look at the links above to have informations about all the other configuration parameters.

Of course we have to provide an MX record and also an SPF record like this to the newly created srs_domain in our DNS:

srs.mydomain.tld. IN TXT "v=spf1 a mx -all"

We should have already created an SPF record for the control/me domain as well. If not, let's do it now.

We can now restart qmail and test our SRS system.

Installing and configuring Spamassassin

June 25, 2023 by Roberto Puzzanghera 36 comments

• Info: http://spamassassin.apache.org/
• Docs: http://spamassassin.apache.org/full/4.0.x/doc/
• Latest version: 4.0.0
• Download: http://spamassassin.apache.org/downloads.cgi

SpamAssassin is a mature, widely-deployed open source project that serves as a mail filter to identify Spam. SpamAssassin uses a variety of mechanisms including header and text analysis, Bayesian filtering, DNS blocklists, and collaborative filtering databases. SpamAssassin runs on a server, and filters spam before it reaches your mailbox.

Changelog

• Jun 25, 2023
  - The ExtractText notes have been revised and corrected by Gabriel Torres
• Dec 27, 2022
  - SA upgraded to v. 4.0.0
• Jul 14, 2021
  - added DCC setup (next page)
  - moved the configuration of Razor, Pyzor and Spamcop to a separate page