Installing and configuring Spamassassin

• Info: http://spamassassin.apache.org/
• Docs: http://spamassassin.apache.org/full/3.4.x/doc/
• Latest version: 3.4.4
• Download: http://spamassassin.apache.org/downloads.cgi

SpamAssassin is a mature, widely-deployed open source project that serves as a mail filter to identify Spam. SpamAssassin uses a variety of mechanisms including header and text analysis, Bayesian filtering, DNS blocklists, and collaborative filtering databases. SpamAssassin runs on a server, and filters spam before it reaches your mailbox.

Changelog

• Nov 18 2020
  - solved some priviledge problems with the reports of the RC's markasjunk plugin, which is going to write inside the log dir and read the razor's identity file.
  - moved all log files into /var/log/spamassassin (apache group now has +w priv). spamdctl and logrotate scripts modified accordingly
• Jul 15 2020
  - added Razor2, Pyzor and Spamcop configuration (thanks to Gabriel Torres for the hint).

Upgrading spamassassin to version 3.4.3 and 3.4.4

Here is how to update quickly:

qmailctl stop
spamdctl stop
perl -MCPAN -e shell
cpan> o conf prerequisites_policy ask
cpan> force notest install  Mail::SpamAssassin Mail::SpamAssassin::Plugin::Razor2
cpan> force install BSD::Resource
cpan> quit
sa-update

I enabled all new plugins from /etc/mail/spamassassin/v343.pre.

Finally you have to apply this modification to the txrep table, as the column count was renamed (tx Tony Fung):

ALTER TABLE `txrep` CHANGE `count` `msgcount` INT(11) NOT NULL DEFAULT '0';

Then restart spamd and qmail

spamdctl start
qmailctl start

Install

Create the spamd user and group, prepare config and log dirs:

mkdir -p /etc/mail/spamassassin /home/spamd /var/log/spamassassin

groupadd spamd
useradd -g spamd -d /home/spamd spamd
chown -R spamd:spamd /home/spamd

chgrp apache /var/log/spamassassin
chmod g+w /var/log/spamassassin

You have to assign +w priviledges to apache in the log dir because if you decide to store there your spam reports created with the Roundcube's markasjunk plugin, which is runned by apache.

Razor2

• Homepage
• Features
• Download
• Docs

Vipul's Razor is a distributed, collaborative, spam detection and filtering network. Through user contribution, Razor establishes a distributed and constantly updating catalogue of spam in propagation that is consulted by email clients to filter out known spam. Detection is done with statistical and randomized signatures that efficiently spot mutating spam content. User input is validated through reputation assignments based on consensus on report and revoke assertions which in turn is used for computing confidence values associated with individual signatures.

Download and install razor-agent-sdk and razor-agent:

cd /usr/local/src
wget https://downloads.sourceforge.net/project/razor/razor-agents-sdk/2.07/razor-agents-sdk-2.07.tar.bz2 
wget https://downloads.sourceforge.net/project/razor/razor-agents/2.85/razor-agents-2.85.tar.bz2

tar xjf razor-agents-sdk-2.07.tar.bz2
cd razor-agents-sdk-2.07
chown -R root:root .
perl Makefile.PL
make
make install

cd ..
tar xjf razor-agents-2.85.tar.bz2
cd razor-agents-2.85
chown -R root:root .
perl Makefile.PL
make
make install

Razor2 requires reporters to be registered. This lets reporters build a reputation over time, so their reports and revocations are weighed according to their reputation value.

Register yourself and create the config directory:

mkdir -p /etc/mail/spamassassin/.razor
razor-admin -home=/etc/mail/spamassassin/.razor -register
razor-admin -home=/etc/mail/spamassassin/.razor -create
razor-admin -home=/etc/mail/spamassassin/.razor -discover

Set up permissions

chgrp apache /etc/mail/spamassassin/.razor/identity-*
chmod 640 /etc/mail/spamassassin/.razor/identity-*
chmod 644 /etc/mail/spamassassin/.razor/razor-agent.log

Again, you have to assign +r priviledges to apache to let it call Razor reports via RC's markasjunk plugin.

Tell Razor2 where it lives adding this line to /etc/mail/spamassassin/.razor/razor-agent.conf and setup the log dir

razorhome = /etc/mail/spamassassin/.razor/ logfile = /var/log/spamassassin/razor-agent.log

We will enable Razor2 once spamassassin has been installed.

Pyzor

• Homepage
• Download

Pyzor is a collaborative, networked system to detect and block spam using digests of messages. Using Pyzor client a short digest is generated that is likely to uniquely identify the email message.

You can install Pyzor via pip or pip3

pip3 install pyzor

or, as an alternative, you can download the source and install in the usual way

cd /usr/local/src
wget https://files.pythonhosted.org/packages/75/9d/e38a18d8c932f397537cda0d03a606314611fe1ebd8b24ed8fdd4df23191/pyzor-1.0.0.tar.gz
tar xzf pyzor-1.0.0.tar.gz
cd pyzor-1.0.0
chown -R root:root .
python setup.py build 
python setup.py install

Create the pyzor directory:

mkdir -p /etc/mail/spamassassin/.pyzor
chown spamd:spamd /etc/mail/spamassassin/.pyzor

We will enable pyzor later.

Spamcop

• Homepage
• How reporting service works

Spamcop is a known spam blocking list which works also as a reporting system sending warning information to the internet service provider responsible for hosting the services used by the spammer (web sites and email sending sites). SpamCop also uses the information to generate SpamCop's free blocking list.

Register an account here. Note: the captcha filter is not working here on my chromium browser; it works with firefox. At the end of the procedure you will get a unique e-mail address (something like submit.xxxxxxxxxxxxxxxxxxx@spam.spamcop.net) that you have to pass to spamassassin in order to send the reports to spamcop. We will see below how to set up SA accordingly.

Spamassassin

Finally install spamassassin via cpan

perl -MCPAN -e shell
o conf commit prerequisites_policy ask
install Mail::SpamAssassin
quit

Installation notes for Slackware users

• REQUIRED module missing: HTML::Parser
• REQUIRED module missing: Net::DNS
• REQUIRED module missing: NetAddr::IP
• REQUIRED module missing: BSD::Resource
• optional module missing: Digest::SHA1
• optional module missing: Mail::SPF
• optional module missing: Razor2
• optional module missing: IO::Socket::INET6
• optional module missing: IO::Socket::SSL
• optional module missing: Mail::DKIM
• optional module missing: LWP::UserAgent
• optional module missing: HTTP::Date
• optional module missing: Encode::Detect
• optional module missing: Geo::IP
• optional module missing: IO::Socket::IP
• optional module missing: Net::Patricia

These modules are missing and must be installed from CPAN. Some of them have dependencies as well...

At the end this is how I have installed everything. Reply yes if dependencies are found, install in this order and force install when needed.

perl -MCPAN -e shell
o conf prerequisites_policy ask

force notest install Socket6 IO::Socket IO::Socket::INET6 LWP MD5 CPAN::DistnameInfo Mail::DKIM

Installed prerequisites of Net::DNS:

force notest install Test::More MIME::Base64 Digest::MD5 Digest::HMAC_MD5 Net::IP

Continue installing (always from CPAN):

force notest install Net::Ping Net::DNS Time::HiRes Digest::SHA1 Getopt::Long Digest::Nilsimsa URI::Escape HTML::Parser HTTP::Date IO::Zlib Archive::Tar  Mail::SPF
force notest install Mail::SPF::Query Net::Ident IO::Socket::SSL Mail::DomainKeys Mail::DKIM LWP::UserAgent HTTP::Date Encode::Detect BSD::Resource

Install these modules

force notest install Storable DB_File Net::SMTP BerkeleyDB
force notest install Geo::IP IO::Socket::IP Net::Patricia

Finally, if everything is ok install spamassassin and Razor  via CPAN

force notest install  Mail::SpamAssassin Mail::SpamAssassin::Plugin::Razor2

I had to skip the tests because of many errors... anyway it works.

Configuring

You can find the config files into /etc/mail/spamassassin

> cd /etc/mail/spamassassin
> ls
init.pre  local.cf  v310.pre  v312.pre  v320.pre  v330.pre

local.cf

# Add *****SPAM***** to the Subject header of spam e-mails
# rewrite_header Subject *****SPAM*****
# put here your subnet
trusted_networks 10.0.0.
# Set the threshold at which a message is considered spam (default: 5.0)
required_score 5.0
use_bayes 1
bayes_auto_learn 1
use_txrep 1
txrep_factory Mail::SpamAssassin::SQLBasedAddrList

Tell spamassassin where Razor2 and Pyzor live

razor_config /etc/mail/spamassassin/.razor/razor-agent.conf 
pyzor_options --homedir /etc/mail/spamassassin/.pyzor 
pyzor_timeout 20

Finally configure Spamcop. spamcop_to_address is the address where to submit your reports, while spamcop_from_address is an email where you want to receive a feedback from the reporting system each time a spam message has been reported.

spamcop_from_address postmaster@yourdomain.tld 
spamcop_to_address submit.xxxxxxxxxxxxxxx@spam.spamcop.net

init.pre

# RelayCountry - add metadata for Bayes learning, marking the countries
# a message was relayed through
#
# Note: This requires the IP::Country::Fast Perl module
#
loadplugin Mail::SpamAssassin::Plugin::RelayCountry

# URIDNSBL - look up URLs found in the message against several DNS
# blocklists.
#
loadplugin Mail::SpamAssassin::Plugin::URIDNSBL

# Hashcash - perform hashcash verification.
#
loadplugin Mail::SpamAssassin::Plugin::Hashcash

# SPF - perform SPF verification.
#
loadplugin Mail::SpamAssassin::Plugin::SPF

v310.pre

Enable Razor2, Pyzor and Spamcop

# Pyzor - perform Pyzor message checks. 
# 
loadplugin Mail::SpamAssassin::Plugin::Pyzor 

# Razor2 - perform Razor2 message checks. 
# 
loadplugin Mail::SpamAssassin::Plugin::Razor2 

# SpamCop - perform SpamCop message reporting 
# 
loadplugin Mail::SpamAssassin::Plugin::SpamCop

Testing

Run this debug command. If you get no error you are ready to run the daemon.

sudo -u spamd -H spamassassin -D --lint

Check if the headers are inserted:

echo -e "From: myself@mymailserver.net\nTo:myfriend@domain.net\nSubject: test\n\n" | spamc

Received: from localhost by qmail.mymailserver.net
 with SpamAssassin (version 3.3.1);
 Tue, 30 Nov 2010 23:18:37 +0100
From: myself@mymailserver.net
To: myfriend@domain.net
Subject: test
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-04-18) on qmail.mymailserver.net
X-Spam-Flag: YES
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.4 required=5.0 tests=BAYES_99,FREEMAIL_FROM,
 MISSING_DATE,MISSING_MID,NO_RECEIVED,NO_RELAYS,TVD_SPACE_RATIO,
 T_TO_NO_BRKTS_FREEMAIL autolearn=no version=3.3.1

In order to test the learning system save a raw spam message into spam.txt and run sa-learn in this way (supposing that postmaster@yourdomain.tld is the email recipient)

sa-learn --debug --spam -upostmaster@yourdomain.tld spam.txt

This is how to test that the message is reported to Razor, Pyzor and Spamcop you have to pass spamassassin the --report option:

spamassassin --debug --report --nocreate-prefs < spam.txt

sa-update

sa-update updates the rules (it requires gpg 1.4). Before running spamassassin for the first time download the rules:

sa-update

Add to your crontab this line to update the rules once a day

# spamassassin update
30 3 * * * /usr/bin/sa-update --nogpg -v &

The -v option will produce an email notification to postmaster.

Running spamassassin

Download the startup script from here. You have to replace the IP of your firewall and place it in /usr/local/bin/spamdctl or /etc/rc.d/rc.spamd and make it executable. Check that the path where you daemon has been installed (/usr/local/bin/spamd or /usr/local/bin/spamd) matches the one in the run script.

NB: in what follows <external-IP/firewall-IP> is the IP address of your mail server, as seen from the internet.

#!/bin/sh

# Spamd init script for Slackware
# August, 2th 2003
# Martin Ostlund, nomicon
# Modified slightly by Troy Belding for Qmailrocks - February 23, 2004
#
# Modified by Roberto Puzzanghera - September 02, 2014
# November 17, 2020: moved log file to /var/log/spamassassin/spamd.log

DAEMON=/usr/local/bin/spamd
NAME=spamd
SNAME=spamdctl
DESC="SpamAssassin Mail Filter Daemon"
LOGFILE=/var/log/spamassassin/spamd.log
PIDFILE="/var/run/$NAME.pid"
PNAME="spamd"

DOPTIONS="-x -u spamd -A 127.0.0.1,<external-IP/firewall-IP> -s $LOGFILE -H /home/spamd -d --pidfile=$PIDFILE"

KILL="/bin/kill"
KILLALL="/bin/killall"
# Defaults - don''t touch, edit /etc/mail/spamassassin/local.cf
ENABLED=0
OPTIONS=""

set -e

case "$1" in
start)
echo -n "Starting $DESC: "
$DAEMON $OPTIONS $DOPTIONS

echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
$KILL -9 `cat $PIDFILE`
/bin/rm $PIDFILE
echo "$NAME."
;;
restart|force-reload)
echo -n "Restarting $DESC: "
$0 stop
$0 start

echo "$NAME."
;;
*)
ME=/usr/local/bin/$SNAME
echo "Usage: $ME {start|stop|restart|force-reload}" >&2
exit 1
;;
esac

exit 0

Now check that spamd is running:

> spamdctl start
> ps axfu
root      1859  0.1  3.4 139360 61044 ?        Ss   19:00   0:01 /usr/bin/spamd -x -u spamd -A 127.0.0.1,<external-IP> -H /home/spamd -d --pidfile=/var/run/spamd.pid
spamd     1860  0.0  3.2 139360 58984 ?        S    19:00   0:00  \_ spamd child
spamd     1861  0.0  3.2 139360 58984 ?        S    19:00   0:00  \_ spamd child

Type spamd -c to learn how to use spamd. See also http://spamassassin.apache.org/full/3.4.x/doc/spamd.html

Starting spamassassin at boot time

To start spamassassin at boot time put your startup script in your rc.local:

/usr/local/bin/spamdctl start &

logrotate

Create a file /etc/logrotate.d/spamd like this (slackware) to rotate daily your spamd logs:

cat > /etc/logrotate.d/spamd << __EOF__
/var/log/spamassassin/spamd.log /var/log/spamassassin/razor-agent.log {
rotate 5
daily
missingok
notifempty
delaycompress
postrotate
   [ -f '/var/run/spamd.pid' ] && (kill -HUP `cat /var/run/spamd.pid`) || exit 0
endscript
}
__EOF__

--enable-spam-auth-user=y 

Util.c: loadable library and perl binaries are mismatched (got handshake key 0xdb00080, needed 0xde00080),

/usr/local/bin/tcpserver -v -R -l mail.watchmusic.com -x /etc/tcp.smtp.cdb -c 50 -u 1008 -g 1003 0 465 /usr/local/bin/stunnel /var/qmail/control/stunnel_smtpd.conf

foreground = yes
cert = /var/qmail/control/servercert.pem
exec = /var/qmail/bin/qmail-smtpd
execargs = /var/www/vpopmail/bin/vchkpw /bin/true

sslserver -e -vR -l myserverfqdn -c 30 -u 508 -g 503 -x /etc/tcp.ssmtp.cdb 0.0.0.0 465 qmail-smtpd myserverfqdn /var/www/vpopmail/bin/vchkpw /bin/true

error: Can't locate loadable object for module Geo::IP in @INC (@INC contains: /usr/local/share/perl5 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at /usr/local/share/perl5/Geo/IP.pm line 42.

cd /usr/local/src
wget https://github.com/maxmind/geoip-api-perl/archive/v1.45.tar.gz
tar zxvf https://github.com/maxmind/geoip-api-perl/archive/v1.45.tar.gz
cd geo-api-perl-1.45
perl Makefile.PL
make
make install
ldconfig

$ /var/qmail/bin/simscanmk