Merry Xmas and happy new... patch!

Massive Christmas present by my italian friend Luca Franceschini of digitalmind. He merged his combo with my combined patch (2016.12.02 version) adding several (heavily customized) patches and functionalities. Luca is a C programmer and an expert system administrator who manages big servers.

The most powerful addition is his "qlog patch", which records detailed logs of qmail-smtpd. Once you will apply the new patch the qmail-smtpd log will be shown as follows:

@400000005855db3020335304 CHKUSER accepted sender: from <sender@senderdomain.com|remoteinfo/auth:|chkuser-identify:> remote <helo:smtp.senderdomain.com|remotehostname:unknown|remotehostip:1.2.3.4> rcpt <> : sender accepted
@400000005855db302064cefc CHKUSER accepted rcpt: from <sender@senderdomain.com|remoteinfo/auth:|chkuser-identify:> remote <helo:smtp.senderdomain.com|remotehostname:unknown|remotehostip:1.2.3.4> rcpt <user@rcptdomain.com> : found existing recipient
@400000005855db3021a8d434 qmail-smtpd[11928]: rbl: ip=1.2.3.4 query=4.3.2.1.zen.spamhaus.org result=ignore message=''
@400000005855db3028802424 qmail-smtpd[11928]: rbl: ip=1.2.3.4 query=4.3.2.1.b.barracudacentral.org result=ignore message=''
@400000005855db3028806a74 policy_check: remote sender@senderdomain.com -> local user@rcptdomain.com (UNAUTHENTICATED SENDER)
@400000005855db302880ef44 policy_check: policy allows transmission
@400000005855db3028811e24 qlogenvelope: result=accepted code=250 reason=rcptto detail=chkuser helo=smtp.senderdomain.com mailfrom=sender@senderdomain.com rcptto=user@rcptdomain.com relay=no rcpthosts=yes size= authuser= authtype= encrypted= sslverified=no localip=10.0.0.4 localport=25 remoteip=1.2.3.4 remoteport=43618 remotehost= qp= pid=11928
@400000005855db3226010eac simscan:[11928]:CLEAN (1.30/9.50):1.8847s:####### Subject ############## 1.2.3.4:sender@DEM.senderdomain.com:user@rcptdomain.com
@400000005855db322a890f9c mail recv: pid 11928 from <sender@DEM.senderdomain.com> qp 11934
@400000005855db322a891b54 qmail-smtpd: message accepted: sender@DEM.senderdomain.com from 1.2.3.4 to user@rcptdomain.com helo smtp.senderdomain.com
@400000005855db322a892324 qlogreceived: result=accepted code=250 reason=queueaccept detail= helo=smtp.senderdomain.com mailfrom=sender@senderdomain.com rcptto=user@rcptdomain.com relay=no rcpthosts= size=2689 authuser= authtype= encrypted= sslverified=no localip=10.0.0.4 localport=25 remoteip=1.2.3.4 remoteport=43618 remotehost= qp=11934 pid=11928

You can store the "qlogenvelope" lines in separate files organized by date for backup purposes, if you like, by means of a scriptlet (look at the new qmail-smtpd/log/run file in the Configuring page).

You should have noticed that the "qmail-smtpd:" log line (due to the logging Kyle Wheeler's patch) is still present despite some redundancy, for the fact that it is more explicit than the new "qlog" in some cases.

The qmail-dnsbl logic from the Fabio Busatto patch was slightly changed (see below), and moved after chkuser in order to decrease the number of calls to spamhaus and the other block lists.

Another big addition is the qregex patch, that I've never added before because of collisions with the dkim patch in qmail-smtpd.c. Many of the functions of the qregex were improved by Manvendra Banghui, while Luca Franceschini added the new control file 'badhelonorelay' and renamed "badmailto" to "badrcptto". The original logic of the qregex patch remains intact.

For those who have my previous patch installed there are some minor adjustments to consider:

• the libdomainkey.a library is now linked from /usr/lib, so you have to copy that library there
• the old qmail-tap was replaced with an improved version by Mihai Secasiu. Look below to understand how to modify your control/taps file

Have fun!

Changelog

This is the detailed changelog with respect to the patch I published on december 2 2016:

• qregex (by  Andrew St. Jean http://www.arda.homeunix.net/downloads-qmail/, contributors: Jeremy Kitchen, Alex Pleiner, Thanos Massias. Original patch by Evan Borgstrom)
  adds the ability to match address evelopes via Regular Expressions (REs) in the qmail-smtpd process.
  Added new control file 'badhelonorelay', control/badmailto renamed control/badrcptto (Tx Luca Franceschini).
• brtlimit
  added control/brtlimit and BRTLIMIT variable to limit max invalid recipient errors before closing the connection (man qmail-control, patch derived from http://netdevice.com/qmail/patch/goodrcptto-12.patch)
• validrcptto (code grabbed from several patches with additional features: http://qmail.jms1.net/patches/validrcptto.cdb.shtml, http://netdevice.com/qmail/patch/goodrcptto-ms-12.patch, http://patch.be/qmail/badrcptto.html)
  It works in conjunction with chkuser with both cdb and mysql accounts. Look here for details
• rbl (code and logic from rblsmtpd and qmail-dnsbl patch http://qmail-dnsbl.sourceforge.net/)
  added support for whitelists, TXT and A queries, configurable return codes 451 or 553 with custom messages
  More info here
• reject-relay-test (by Russell Nelson http://qmail.org/qmail-smtpd-relay-reject)
  It gets qmail to reject relay probes generated by so-called anti-spammers. These relay probes have '!', '%' and '@' in the local (username) part of the address.
• added DISABLETLS environment variable, useful if you want to disable TLS on a desired port
• added FORCEAUTHMAILFROM environment variable to REQUIRE that authenticated user and 'mail from' are identical
• fixed little bug in 'mail from' address handling (see the patch by Andre Opperman at http://qmail.cr.yp.narkive.com/kBry6GJl/bug-in-qmail-smtpd-c-addrparse-function)
• added SMTPAUTHMETHOD, SMTPAUTHUSER and SMTP_AUTH_USER env variables for external plugins (see http://qmail-spp.sourceforge.net/doc/)
• qlog patch (Luca Franceschini): smtpd logging with fixed format (note: 'size' field is evaluated only when control/databytes or DATABYTES are set)

an entry 'qlogenvelope' is generated after accepting or rejecting every recipients in the envelope phase, example:

qlogenvelope: result=rejected code=553 reason=rblreject detail=b.barracudacentral.org helo=test.machine.it mailfrom=test@domain.com rcptto=test@pippo.com relay=no rcpthosts=yes size= authuser= authtype= encrypted= sslverified=no localip=15.15.15.15 localport=25 remoteip=14.143.30.83 remoteport=57502 remotehost= qp= id=39156
qlogenvelope: result=accepted code=250 reason=rcptto detail=chkuser helo=test mailfrom=test@test.com rcptto=test@pippo.com relay=no rcpthosts=yes size= authuser= authtype= encrypted= sslverified=no localip=15.15.15.15 localport=25 remoteip=12.181.218.154 remoteport=57742 remotehost= qp= pid=37357

an entry 'qlogreceived' is generated after DATA (message accepted o rejected by qmail-queue)

qlogreceived: result=accepted code=250 reason=queueaccept detail= helo=test.machine.it mailfrom=test@domain.com rcptto=test@gmail.com relay=yes rcpthosts= size= authuser=pippo@pippo.com,pluto@pippo.com authtype=login encrypted=tls sslverified=no localip=192.168.200.162 localport=25 remoteip=192.168.200.162 remoteport=52602 remotehost= qp=30982 pid=30980

• reject null senders patch
  useful in special cases if you temporarily need to reject the null sender (although it breaks RFC compatibility). You just need to put 1 (actually any number different from 0) in your control/rejectnullsenders to reject the null sender with a 421 error message.
• qmail-taps-extended (http://patchlog.com/patches/qmail-taps-extended/ by Michai Secasiu, derived from Inter7's qmail-tap patch http://notes.sagredo.eu/files/qmail/patches/qmail-tap.diff)
  Provides the ability to archive each email that flows through the system. Now you can decide to archive only messages from or to certain email addresses. More info here.

unsigned int rejnsmf = 0;

@400000005890b1cd22b50c44 starting delivery 23226: msg 137479 to local domain.xxx-mailinglist@domain.xxx
@400000005890b1cd22b5102c status: local 1/10 remote 0/204
@400000005890b1cd2329944c delivery 23226: failure: ezmlm-reject:_fatal:_I_do_not_reply_to_bounce_messages_(#5.7.2)/