Setting up your firewall with fail2ban

July 16, 2023 by Roberto Puzzanghera 21 comments

Fail2ban scans log files (e.g. /var/log/apache/error_log) and bans IPs that show the malicious signs -- too many password failures, seeking for exploits, etc. Generally Fail2Ban is then used to update firewall rules to reject the IP addresses for a specified amount of time, although any arbitrary other action (e.g. sending an email) could also be configured. Out of the box Fail2Ban comes with filters for various services (apache, courier, ssh, etc).

I will show shortly how to install and configure fail2ban to ban malicious IPs, especially those related to the qmail-dnsrbl patch. This will avoid to be banned ourselves by spamhaus, which is free up to 100.000 queries per day.

fail2ban requires that you have a firewall as nftables or iptables active.

Changelog

  • Jul 15, 2023
    - the installation and the configuration has been revised in order to work on Debian, where python2 is missing (tx Gabriel Torres)
  • Nov 20, 2022
    - switched all actions to nftables, as it has now replaced iptables and fail2ban has support for it. Just replace "iptables" with "nftables" in your jails.
  • Nov 18, 2022
    - fail2ban upgraded to v. 1.0.2
    - jails now have a different action's declaration (iptables[type=multiport] instead of iptables-multiport[])
    - added a short note on how to configure the server with a network bridge

Downloading and installing

On Debian you may want to install these packages before installing fail2ban:

apt-get install python3-pyinotify python3-systemd 2to3

Now proceed to the installation:

cd /usr/local/src
wget --no-check-certificate https://github.com/fail2ban/fail2ban/archive/1.0.2.tar.gz --output-document=fail2ban-1.0.2.tar.gz
tar xzf fail2ban-1.0.2.tar.gz
cd fail2ban-1.0.2
chown -R root:root .

The setup script is based on python2. If your system is fine with python2 then go on with it:

./setup.py install

If your system only has python3 (like Debian) you have to run fail2ban-2to3 before the setup:

./fail2ban-2to3
python3 setup.py install

Running script

In order to start the server you can use an init script that you can copy from the files/ folder of the source dir, where you can probably find one suitable for your distribution. I use a very simple one (download here); I don't remember where I found it.

Download it in /usr/local/bin or wherever you want, run it and remember to launch it at boot time as well:

cd /usr/local/bin
wget https://notes.sagredo.eu/files/qmail/fail2banctl
chmod +x fail2banctl
fail2banctl start

logrotate script

You can copy a logrotate script from the install dir as follows:

cp files/fail2ban-logrotate /etc/logrotate.d/fail2ban

Fail2Ban main configuration file fail2ban.local

It is a good practice not to modify the /etc/fail2ban/*.conf files, but edit a customized file with a .local extension, which will be read by the server after each .conf to eventually overwrite the lines that you modified. Furthermore, the .local files should contain only the modified lines, to avoid that, after an upgrade, obsolete or incompatible settings are kept and continue to overwrite the corresponding .conf. In what follows I'll assume that you have created a .local file to overwrite the .conf file that we are talking about.

To understand the terminology and how fail2ban works you are invited to read the manual (quite concise and easy to read).

This is the only option I touched with respect to the original .conf file:

[Definition]
allowipv6 = auto

jails configuration

Enable the jails according to your needs. This is what I have in my jail.local, concerning the qmail/dovecot part:

[qmail-smtp]
enabled  = true
filter   = qmail-smtp
action   = nftables[type=multiport, name=SMTP, port="25,465,587"]
           sendmail-whois-lines[name=SMTP, logpath="%(logpath)s"]
logpath  = /var/log/qmail/smtpd/current
maxretry = 5
bantime  = 1d
findtime = 1h

[qmail-submission]
enabled  = true
filter   = qmail-smtp
action   = nftables[type=multiport, name=SUBMISSION, port="25,465,587"]
           sendmail-whois-lines[name=SUBMISSION, logpath="%(logpath)s"]
logpath  = /var/log/qmail/submission/current
maxretry = 5
bantime  = 1d
findtime = 1h

[vpopmail]
enabled  = true
filter   = vpopmail
action   = nftables[type=multiport, name=VPOPMAIL, port="25,465,587"]
           sendmail-whois-lines[name=VPOPMAIL, logpath="%(logpath)s"]
# check your syslog mail related log (mail.log in some systems)
logpath  = /var/log/maillog
maxretry = 5
bantime  = 86400
findtime = 3600

[qmailadmin]
enabled  = true
filter   = qmailadmin
action   = nftables[type=multiport, name=QMA, port="80,443"]
           sendmail-whois-lines[name=QMA, logpath="%(logpath)s"]
logpath  = /var/log/qma-auth.log
maxretry = 4
bantime  = 1d
findtime = 1h

[roundcube-auth]
enabled  = true
filter   = roundcube-auth
action   = nftables[type=multiport, name=RC, port="80,443"]
           sendmail-whois-lines[name=RC, logpath="%(logpath)s"]
logpath  = /var/www/roundcube/logs/userlogins.log
maxretry = 4
bantime  = 1d
findtime = 1h

[dovecot-pop3]
enabled  = true
filter   = dovecot
action   = nftables[type=multiport, name=POP3, port="993,995"]
           sendmail-whois-lines[name=POP3, logpath="%(logpath)s"] 
logpath  = /var/log/dovecot/dovecot.log
maxretry = 6
bantime  = 1h
findtime = 1h

[dovecot-imap]
enabled  = true
filter   = dovecot
action   = nftables[type=multiport, name=IMAP, port="993,995"]
           sendmail-whois-lines[name=IMAP, logpath="%(logpath)s"]
logpath  = /var/log/dovecot/dovecot.log
maxretry = 6
bantime  = 1h
findtime = 1h

As you can see, we have three jails, so fail2ban will look for the files qmail-smtp.conf, vpopmail.conf and dovecot.conf under the filter.d directory. I'll show the content of these files below.

The qmail-smtp jail is related to a filter named "qmail-smtp", which matches lines in the qmail-smtpd log mainly related to the qlogenvelope line, which records almost all kind of rejects. The filter has to be declared in the filter.d/qmail-smtp.conf file.

Similarly, the vpopmail jail will try to ban clients trying to guess the users' password in the submission port, while the dovecot jail will do the same as far as imap/pop3 is concerned.

Remember to add your server's IP and any other trusted IP in the DEFAULT section, just to avoid to ban yourself expecially in case someone is imitating your own IP (spoofing):

[DEFAULT]

#
# MISCELLANEOUS OPTIONS
#

# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
# ban a host which matches an address in this list. Several addresses can be
# defined using space separator.
ignoreip = 127.0.0.1/8 10.0.0.1/8 <my-external-ip>

Here is the content of the filter files:

qmail-smtp.conf

# Fail2Ban filters for qmail-smtp patched for qmail-dnsbl (http://qmail-dnsbl.sourceforge.net), chkuser (http://opensource.interazioni.it/qmail/chkuser.html) and greetdelay
#
# Here is an example of log lines that this filter is going to hit:
# 
# @40000000545076ad1de678ec GREETDELAY from 77.65.15.93: client sent data before greeting
# @4000000055154dc40e884894 qmail-smtpd: timeout: (null) from 95.141.38.94 to (null) helo host220-227-149-62.serverdedicati.aruba.it
#
# All lines concerning chkuser, qmail-dnsbl and others like these are now catched by the qlogenvelope line:
#
# @4000000059f5194706e649ec CHKUSER accepted sender: from <sender@remotedomain.xy|remoteinfo/auth:|chkuser-identify:> remote <helo:free-112-191.mediaworksit.net|remotehostname:unknown|remotehostip:95.140.112.191> rcpt <> : sender accepted
# @4000000059f519470be7b0fc CHKUSER accepted rcpt: from <sender@remotedomain.xy|remoteinfo/auth:|chkuser-identify:> remote <helo:free-112-191.mediaworksit.net|remotehostname:unknown|remotehostip:95.140.112.191> rcpt <localuser@localdomain.xy> : found existing recipient
# @4000000059f519470be860c4 qmail-smtpd[20003]: rcptcheck: checking <localuser@localdomain.xy> at 95.140.112.191
# @4000000059f519470c084ca4 qmail-smtpd[20003]: rcptcheck: ignore address <localuser@localdomain.xy> at 95.140.112.191
# @4000000059f5195c1f6d7e7c qmail-smtpd[20003]: rbl: ip=95.140.112.191 query=191.112.140.95.zen.spamhaus.org result=ignore message=''
# @4000000059f5195c211f1294 qmail-smtpd[20003]: rbl: ip=95.140.112.191 query=191.112.140.95.b.barracudacentral.org result=reject message='Client host blocked using Barracuda Reputation, see http://www.barracudanetworks.com/reputation/?r=1&ip=95.140.112.191'
# @4000000059f5195c211f2234 qlogenvelope: result=rejected code=553 reason=rblreject detail=b.barracudacentral.org helo=free-112-191.mediaworksit.net mailfrom=sender@remotedomain.xy rcptto=localuser@localdomain.xy relay=no rcpthosts=yes size= authuser= authtype= encrypted= sslverified=no localip=10.0.0.4 localport=25 remoteip=95.140.112.191 remoteport=15630 remotehost= qp= pid=20003
# 2022-02-18 16:23:03.719762500 helo-dns-check: blocked with: HELO doesn't match IP [91.121.144.116]
#
# Be aware that the following regex match only my patched chkuser at https://notes.sagredo.eu/en/qmail-notes-185/patching-qmail-82.html
# If you are using a standard version of chkuser you can refer to this page for the correct filter: http://wiki.qmailtoaster.com/index.php/Fail2Ban

[Definition]

failregex = qlogenvelope: result=rejected .* remoteip=<HOST>
 helo-dns-check: blocked with: .* \[<HOST>\]
 GREETDELAY from <HOST>: client sent data before greeting
 qmail-smtpd: reject \(auth not available\): \(null\) from <HOST>

ignoreregex =

# DEV Notes:
#
# Author: Roberto Puzzanghera

vpopmail.conf

[Definition]
# Jul 10 12:05:53 qmail vpopmail[3076]: vchkpw-submission: vpopmail user not found helpdesk@yourdomain.xy:191.233.70.140
# Jul 22 17:31:46 qmail vpopmail[6383]: vchkpw-submission: password fail (pass: 'dasdas') postmaster@yourdomain.xy:1.2.3.4

failregex = vchkpw-submission: .* user not found .*:<HOST>$
 vchkpw-submission: password fail .*:<HOST>$

# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
ignoreregex =

# DEV Notes:
#
# Author: Roberto Puzzanghera

qmailadmin.conf

# bans qmailadmin login attempts looking for lines like this
# 2015/05/27 15:45:58 user:postmaster@domain.xy ip:1.2.3.4 auth:failed [@domain.xy]
# qmailadmin must be patched with http://notes.sagredo.eu/sites/notes.sagredo.eu/files/qmail/patches/qmailadmin/qmailadmin-1.2.16-log.patch
# (thanks to Tony)

[INCLUDES]
before = common.conf

[Definition]
failregex = ip:<HOST> auth:failed

ignoreregex =

roundcube-auth.local

The filter roundcube-auth.conf already exists, so we'll overwrite it.

# Fail2Ban configuration file for roundcube webmail
#
# Author: Roberto Puzzanghera
# 15/07/2022
#
# Log line to match (the 1st one in case of rc behind a firewall) 
# [12-Jul-2022 08:56:39 +0200]: <3lq5onb8> Failed login for postmaster from 10.0.0.2 (X-Forwarded-For: 2.42.23.100) in session 3lq5onb87b7oqnc7 (error: 1) 
# [01-Sep-2014 00:07:11 +0200]: IMAP Error: Login failed for sisgri@iol.it from 151.55.133.38. AUTHENTICATE PLAIN: Authentication failed. in /usr/local/www/htdocs/roundcubemail-1.0.2/progr>

[INCLUDES]
before = common.conf

[Definition]
failregex = Failed login for  from .* \(X-Forwarded-For: <HOST>\)
            IMAP Error: (FAILED login|Login failed) for .*? from <HOST>\.    

ignoreregex =

dovecot.local

The filter dovecot.conf already exists, so we'll overwrite it.

# Fail2Ban filter Dovecot authentication and pop3/imap server
#
# Jul 22 23:33:29 auth-worker(27283): Info: sql(user@yourdomain.xy:1.2.3.4): Password mismatch
# Jul 22 23:33:31 imap-login: Info: Disconnected (auth failed, 1 attempts in 2 secs): user=<user@yourdomain.xy>, method=PLAIN, rip=1.2.3.4, lip=5.6.7.8, session=<k2t5+c7+5AAKAAAC>
# Jul 22 23:34:04 auth-worker(27283): Info: sql(adminww@yourdomain.xy:1.2.3.4): unknown user
# Jul 22 23:34:06 imap-login: Info: Disconnected (auth failed, 1 attempts in 2 secs): user=<adminww@yourdomain.xy>, method=PLAIN, rip=1.2.3.4, lip=5.6.7.8, session=<ONqY+87+7gAKAAAC>

[Definition]
failregex = \(\S*,<HOST>(?:,\S*)?\): (?:unknown user|invalid credentials|Password mismatch)

ignoreregex =

# DEV Notes:
#
# Author: Roberto Puzzanghera

actions configuration

Locate to the action.d directory and modify the action you are using to your needs, always copying the original .conf file in a .local

nftables-common.local

Create a action.d/nftables-common.local file that we'll use to overwrite nftables.conf.

I did the following modification to ban all protocols as default, instead of banning the only tcp

protocol = tcp,udp,udplite,sctp

sendmail-common.local

Finally you may want to overwrite the file /etc/fail2ban/action.d/sendmail-common.conf just to set the recipient email address where to send the alerts

# Fail2Ban configuration file
#
# Common settings for sendmail actions

[Init]

# Recipient mail address
#
dest = postmaster@mydomain.tld

# Sender mail address
#
sender = fail2ban@mydomain.tld

usage

When you modify something, you have to reload the jails in this way:

fail2banctl reload

or simply

fail2ban-client reload

Look at the jail list

# fail2ban-client status
Status: fail2ban
Status
|- Number of jail:      3
`- Jail list:           vpopmail, qmail-smtp, dovecot

Before you turn on a jail it's always a good practice to test your newly created filter against a log file as follows:

# fail2ban-regex /var/log/qmail/smtpd/current /etc/fail2ban/filter.d/qmail-smtp.conf

Running tests

=============

Use   failregex file : /etc/fail2ban/filter.d/qmail-smtp.conf
Use         log file : /var/log/qmail/smtpd/@40000000532f677b088a7854.s

Results
=======

Failregex: 65 total
|-  #) [# of hits] regular expression
|   1) [58] qmail-smtpd: message rejected \(qmail-dnsbl\) .* from 
|   2) [3] CHKUSER rejected rcpt: from <.*> remote <.*remotehostip:> .* : not existing recipient$
|   3) [4] CHKUSER rejected relaying: from <.*> remote <.*remotehostip:> .* : client not allowed to relay$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1596] TAI64N
`-

Lines: 1596 lines, 0 ignored, 65 matched, 1531 missed
Missed line(s): too many to print.  Use --print-all-missed to print all 1531 lines

fail2ban with a network bridge

In case your services are in a localnet and the packets to be filtered go through a network bridge, then your nftables rules must be written into the FORWARD chain instead of the INPUT one, like fail2ban normally does.

Modify your nftables-common.local action as follows

# Option:  chain_hook 
# Notes.:  refers to the kind of chain to be created 
# Values:  [ prerouting | input | forward | output | postrouting ]  Default: input 
# 
chain_hook = forward 
# was input

Comments

A few pointers for the fail2ban tutorial

Hi Roberto!

1. If your Linux distro is using Python 3, the installation won't work, and you will get an error when trying to start fail2ban. The manual is very confusing in explaining this.

Here I have Debian installed, to I had to install the following packages:

apt-get install python3-pyinotify python3-systemd 2to3

Instead of the ./setup.py install command, you should run instead:

./fail2ban-2to3
python3 setup.py install

2. The link for downloading the start/stop script is broken (http://notes.sagredo.eu/files/qmail/rc.fail2ban)

The correct link is https://notes.sagredo.eu/files/qmail/fail2banctl

Note: here with me, Fail2ban was installed at /usr/local/bin, so I needed to adjust the script

3. At the following line in your tutorial:

Finally you may want to overwrite the file /etc/fail2ban/action.d/sendmail-common.conf just to set the recipient email address where to send the alerts

AFAIK the code presented should be actually saved as sendmail-common.local, not sendmail-common.conf

Actually, this part should be moved to the "sendmail-common.local" section, under "actions configuration"

4. nftables.local

Using the instructions provided, fail2ban cannot start:

[6364]: ERROR Failed during configuration: Error in action definition 'nftables[type=multiport, name=RC, port="80,443"]': File contains no section headers.
file: '/etc/fail2ban/action.d/nftables.local', line: 1
'protocol = tcp,udp,udplite,sctp\n'

I need to add an [Init] section:

[Init]
protocol = tcp,udp,udplite,sctp

5. Got this warning:

[7603]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'

Fixed it by creating /etc/fail2ban/fail2ban.local with:

[Definition]
allowipv6 = auto

Enjoy!

Gabriel.

Reply |

A few pointers for the fail2ban tutorial

Thank you, Gabriel.

I added your instructions in this page. It's better to define nftables-common.local, which is included by default, instead of nftables.local

Reply |

Fail2ban alternative

Fail2ban requires obsolete setuptools and feature. It seems it is not maintained, so I published my simple IP blocker to github.

https://github.com/yohgaki/ssblocker

Reply |

Fail2ban alternative

Thanks. Fail2ban is alive as a new major version was released yesterday

Reply |

qlogenvelope: reason=authfailed

Hi Roberto,

Having a lot of

qlogenvelope: result=rejected code=535 reason=authfailed detail= helo=[177.103.237.193] mailfrom= rcptto= relay=no rcpthosts= size= athuser=thiru@domain.com authtype= encrypted= sslverified=no

Fail2ban did not pick them up. About 20 of them within 1 min

Please help.

Thanks
nic

Reply |

qlogenvelope: reason=authfailed

What do you have in your fail2ban rule? Can you post the entire log line?

In my example in the fail2ban page I have

failregex = qlogenvelope: result=rejected .* remoteip=<HOST>

but this cannot match the line you posted, as it misses the remoteip string

Reply |

qlogenvelope: reason=authfailed

I have the same as you. Duplicated from the site

failregex = qlogenvelope: result=rejected .* remoteip=<HOST>
GREETDELAY from <HOST>: client sent data before greeting
qmail-smtpd: read failed: \(null\) from <HOST>

ignoreregex =

Reply |

qlogenvelope: reason=authfailed

Can you check that the filter was actually loaded:

# fail2banctl status
fail2ban Status
Status
|- Number of jail: 18
`- Jail list: qmail-smtp, qmail-submission [........]

If yes check that the filter is matching lines against your log files

# fail2ban-regex /var/log/qmail/smtpd/current /etc/fail2ban/filter.d/qmail-smtp.conf

Running tests
=============

Use failregex filter file : qmail-smtp, basedir: /etc/fail2ban
Use log file : /var/log/qmail/smtpd/current
Use encoding : ISO-8859-1


Results
=======

Failregex: 88 total
|- #) [# of hits] regular expression
| 1) [24] qlogenvelope: result=rejected .* remoteip=<HOST>
| 3) [64] qmail-smtpd: read failed .* from <HOST>

Reply |

qlogenvelope: reason=authfailed

My results is very different from yours.

Results
=======

Failregex: 3 total
|- #) [# of hits] regular expression
| 3) [3] qmail-smtpd: read failed .* from
`-

Ignoreregex: 0 total

Reply |

qlogenvelope: reason=authfailed

Are sure that  you have those qlogenvelope lines in the log files?

In that case try to write from scratch the filter, because it's not working

Reply |

Updates

Hi,

Fail2ban is now on github. So the tutorial should be updated as:

cd /usr/local/src
git clone https://github.com/fail2ban/fail2ban.git
cd fail2ban
chown -R root:root .
./setup.py install

Also, the manual is now at https://github.com/fail2ban/fail2ban/wiki so the below paragraph should be updated:

To understand the terminology and how fail2ban works you are invited to read the manual (quite concise and easy to read).

Reply |

Updates

Thanks for pointing out that. I suggest to download the sources in this way because this page concerns the 0.10 version that I have already tested with my configutation, otherwise the 0.11-dev version will be downloaded

Reply |

Updates

Cool, no worries.

Also, I made a mistake, I recommended the wiki, but the manual available for version 0.80 and linked in the tutorial is more complete.

Reply |

qmail-smtpd fail2ban

Hi, fail2ban is not blocking

qmail-smtpd: read failed: (null) from 58.221.58.238 to (null) helo ylmf-pc

Any idea?

Thanks
nic

Reply |

Re: qmail-smtpd fail2ban

I can confirm that it is hitting those events here... double check everything

Reply |

ylmf-pc - fail2ban

Hi Roberto,

qmail-smtp fail2ban is not catching ylmf-pc

@400000005682c0591bee0064 qmail-smtpd: read failed: (null) from 159.122.81.186 to (null) helo ylmf-pc

What can i do to get it working?

Thanks

nic

Reply |

Hi Nic,

Hi Nic,

according to my examples, fail2ban is tuned with a "qmail-smtpd: read failed:" string, not with a particular helo string. In my example there must be two such events within an hour (findtime parameter).

If you want to match other string you have to modify your qmail filter accordingly

Reply |

Wrong log path for vpopmail / qmail-submission

Hi,

In /etc/fail2ban/jail.local, in the [vpopmail] section, you have:

logpath = /var/log/maillog

But in the scripts setting up qmail-submission (which is the service listening on port 587) the log file path is set to:

#!/bin/sh exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t /var/log/qmail/submission

(See http://notes.sagredo.eu/node/83)

Regards

/ Otto

Reply |

That's correct

That's correct, Otto. The failed login attempts are logged exactly there.

Reply |

Sorry, I was expecting the

Sorry, I was expecting the submission service to log to that fiile, but I suppose the login comes earlier in the chain and those errors are routed to a custom location?

However, after searching through  my logs, the failed login attempts from the submission service are logged in:

/var/log/mail.log

(Please notice the dot).

Not sure wher this is configured?

Regards

/ Otto

Reply |

vpopmail and syslog

You are right. The log are directed to syslog (look around line no. 679 of vchkpw.c). My slackware put all mail.* logs in /var/log/maillog, so it's worth that I clarify this point as soon as possible.

But you can manage what syslog is going to write at configure time:

--enable-logging=OPT       Log to syslog: n=nothing, e=errors only (default), y=all attempts, p=errors with passwords, v=verbose (all attempts, with passwords).
--enable-log-name=TEXT     Set syslog name vpopmail.

The second one is not very clear to me, but maybe can adjust the log file name.

As an alternative you may want to record your logs on mysql

--enable-sql-logging       Enable authentication logging to MySQL/Postgres.

but this is not convenient if you are going to use fail2ban

Reply |