Testing simscan

January 12, 2018 by Roberto Puzzanghera 12 comments

Send a message mailtest.txt to a user and check the simscan's response:

> echo "This is a test message" > mailtest.txt
> QMAILQUEUE=/var/qmail/bin/simscan SIMSCAN_DEBUG=4 /var/qmail/bin/qmail-inject user@yourdomain.tld < mailtest.txt 

simscan: cdb looking up 
simscan: cdb for found clam=yes,spam=yes,spam_hits=9.5,attach=.vbs:.lnk:.scr:.wsh:.hta:.pif
simscan: pelookup clam = yes
simscan: pelookup spam = yes
simscan: pelookup spam_hits = 9.5
simscan: Per Domain Hits set to : 9.500000
simscan: pelookup attach = .vbs:.lnk:.scr:.wsh:.hta:.pif
simscan: attachment flag attach = .vbs:.lnk:.scr:.wsh:.hta:.pif
simscan: add_attach called with .vbs:.lnk:.scr:.wsh:.hta:.pif
simscan: .vbs is attachment number 0
simscan: .lnk is attachment number 1
simscan: .scr is attachment number 2
simscan: .wsh is attachment number 3
simscan: .hta is attachment number 4
simscan: .pif is attachment number 5
simscan: starting: work dir: /var/qmail/simscan/1515774500.399139.5534
simscan: F envelope is root@mail.yourdomain.tld
simscan: T1 envelope is user@yourdomain.tld
simscan: pelookup: called with root@mail.yourdomain.tld
simscan: pelookup: domain is qmailtest.sagredo.eu
simscan: cdb looking up qmailtest.sagredo.eu
simscan: pelookup: local part is root
simscan: lpart: local part is **
simscan: cdb looking up root@mail.yourdomain.tld
simscan: pelookup: called with user@yourdomain.tld
simscan: pelookup: domain is qtest.sagredo.eu
simscan: cdb looking up qtest.sagredo.eu
simscan: pelookup: local part is test
simscan: lpart: local part is **
simscan: cdb looking up user@yourdomain.tld
simscan: checking attachment textfile0 against .vbs
simscan: checking attachment textfile0 against .lnk
simscan: checking attachment textfile0 against .scr
simscan: checking attachment textfile0 against .wsh
simscan: checking attachment textfile0 against .hta
simscan: checking attachment textfile0 against .pif
simscan: checking attachment addr.1515774500.399139.5534 against .vbs
simscan: checking attachment addr.1515774500.399139.5534 against .lnk
simscan: checking attachment addr.1515774500.399139.5534 against .scr
simscan: checking attachment addr.1515774500.399139.5534 against .wsh
simscan: checking attachment addr.1515774500.399139.5534 against .hta
simscan: checking attachment addr.1515774500.399139.5534 against .pif
simscan: checking attachment msg.1515774500.399139.5534 against .vbs
simscan: checking attachment msg.1515774500.399139.5534 against .lnk
simscan: checking attachment msg.1515774500.399139.5534 against .scr
simscan: checking attachment msg.1515774500.399139.5534 against .wsh
simscan: checking attachment msg.1515774500.399139.5534 against .hta
simscan: checking attachment msg.1515774500.399139.5534 against .pif
simscan: cdb looking up version attach
simscan: runned_scanners is attach: 1.4.0
simscan: found 1.4.0
simscan: calling clamdscan
simscan: clamdscan: /usr/local/qmail/simscan/1515774500.399139.5534: OK
simscan: clamdscan: 
simscan: clamdscan: ----------- SCAN SUMMARY -----------
simscan: clamdscan: Infected files: 0
simscan: clamdscan: Time: 0.004 sec (0 m 0 s)
simscan: cdb looking up version clamav
simscan: runned_scanners is attach: 1.4.0 clamav: 0.99.2/m:58/d:24215
simscan: found 0.99.2/m:58/d:24215
simscan: normal clamdscan return code: 0
simscan: calling spamc
simscan: calling /usr/local/bin/spamc spamc -u user@yourdomain.tld
simscan: cdb looking up version spam
simscan: runned_scanners is attach: 1.4.0 clamav: 0.99.2/m:58/d:24215 spam: 3.4.1
simscan: found 3.4.1
simscan:[5533]:CLEAN (3.80/9.50):0.3657s::(null):root@mail.yourdomain.tld:user@yourdomain.tld
simscan: done, execing qmail-queue
simscan: qmail-queue exited 0

simscan and chkuser in action in the smtpd log

This is what you are going to see in the smtpd log with the following settings inside simcontrol:

spam=yes,spam_passthru=yes

Simscan will pass through the spam regardless of its score. Hopefully the spam will be stored in the Junk folder by a sieve rule.

@TIMESTAMP CHKUSER accepted sender: from <spammer@somewhere.net::> remote  rcpt <> : sender accepted
@TIMESTAMP CHKUSER accepted rcpt: from <spammer@somewhere.net::> remote  rcpt <user@yourdomain.net> : found existing recipient
@TIMESTAMP simscan:[26411]:PASSTHRU (16.60/5.00):4.3012s:Body text.:[remoteIP]:spammer@somewhere.net:user@yourdomain.net

spam=yes,spam_passthru=no,spam_hits=9.5

spam will be rejected in case its score is above 9.5, never passed to the user:

@TIMESTAMP CHKUSER accepted sender: from <spammer@somewhere.net::> remote  rcpt <> : sender accepted
@TIMESTAMP CHKUSER accepted rcpt: from <spammer@somewhere.net::> remote  rcpt <user@yourdomain.net> : found existing recipient
@TIMESTAMP simscan:[26411]:SPAM REJECT (16.60/5.00):4.3012s:Body text.:[remoteIP]:spammer@somewhere.net:user@yourdomain.net

spam=yes,spam_hits=9.5

I strongly suggest this configuration.

In the following case if the score is between 5.0 and 9.5 the message will be passed through by simscan even though spamassassin has labeled it as spam:

@TIMESTAMP CHKUSER accepted sender: from <spammer@somewhere.net::> remote  rcpt <> : sender accepted
@TIMESTAMP CHKUSER accepted rcpt: from <spammer@somewhere.net::> remote  rcpt <user@yourdomain.net> : found existing recipient
@TIMESTAMP simscan:[26411]:SPAM CLEAN (6.40/9.50):4.3012s:Body text.:[remoteIP]:spammer@somewhere.net:user@yourdomain.net

And this should be the header of such a message:

X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-01-18) on mail.myserver.net
X-Spam-Flag: YES X-Spam-Level: ****** X-Spam-Status: Yes, score=6.4 required=5.0 tests=BAYES_99,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CHECK,RDNS_NONE autolearn=no version=3.3.1

Note that the message's status IS spam, but the score is below 9.5 and it was not rejected by simscan.

Troubleshooting

@400000004dd263d6378e6b74 qmail-smtpd: message delayed (mail server temporarily rejected message (#4.3.0)): xxxx@xxxxx.xx from y.y.y.y to zzzzzz@zzzz.zz helo mail.xxxxx.xx

I saw an error like this in a server where both spamd and clamd were stopped. The problem was solved simply running the servers again.

Comments

spamc simscam

I am having a strange issue, if I test simscam using mailtest.txt procedure, it calls spamc normally and everything just happen, but when simscam is called by qmailqueue running on the whole process, only spamc is not called, I have added debug=3 5 and I can't get the reason why its not been called. I thought about softlimit, but I am not sure. Any hints ?

Reply |

spamc simscam

simscan doesn't pass messages over 250k to spamassassin. Could it be the cause?

Reply |

spamc simscam

first of all, thanks for replying it in the first place. Nope, that's not the cause, cos I am using plain text emails to test it, they are really small. spamc wont be called when running from qmail-queue in regular procedure, but if I test it using your echo whatever > mailtest.txt and set env debug and use qmail-inject it will be called by simscan. I have just installed regex support, now it runs attach scan, regex scan and clamdscan perfectly, but no errors no mentions to spamc being called.

the only mentions of spamc are:

@400000005f32e41230469ea4 simscan:[5451]: cdb for found clam=yes,spam=yes,spam_hits=9.5,attach=.vbs:.lnk:.scr:.wsh:.hta:.pif
@400000005f32e4123047fe34 simscan:[5451]: pelookup clam = yes
@400000005f32e4123049a02c simscan:[5451]: pelookup spam = yes
@400000005f32e412304b8c5c simscan:[5451]: pelookup spam_hits = 9.5
@400000005f32e412304d6504 simscan:[5451]: Per Domain Hits set to : 9.50000

but the last msgs I get:

@400000005f32e4130bea8f5c simscan:[5451]: runned_scanners is regex: 1.4.1 attach: 1.4.1 clamav: 0.102.4/m:59/d:25900

I have tried running spamd as clamav user, I have tried passing socks file argument to both spamd and spamc, using localhost, whatever I do, it works on qmail-inject but not on qmail-smtpd

weird isnt it ?

Reply |

spamc simscam

I suppose that you don't have anything in your spamd log. I would try to inspect the intire qmail-smtpd process with strace in this way

strace -Ff -o /tmp/strace.log -p <pid>

where pid is the qmail-smtpd/tcpserver process id

EDIT: Try also to increase your softlimit, even if in case of softlimit errors you should have and error in your qmail-smtpd log

Reply |

spamc simscam

Not a single call to spamc, the impression I got is that the the enable-spam is somehow not defined when running on qmail-smtpd, how's that even possible. the strace shows no call to spamc or any of its debug error messages, like the code skipped the spam part of it.

Reply |

Bug in clamav 0.99.2

Dear Roberto

As far as the "mail server temporarily rejected message" message is concerned the real cause is not so innocent as it seems...

It is an old bug in clamav 0.99.2 that has been triggered by a recent signature update in clamav db.

So one has to upgrade clamav imediately. 

It is related to the fact that deleted tmp files used by clamav scan do not close their file descriptors and progressivelly the system limit is reached causing a DOS in mail servers with heavy load...

Check it out with: pidof clamd | xargs -I{} ls -l /proc/{}/fd

See:

[clamav-users] deleted files eating up file descriptors
http://lists.clamav.net/pipermail/clamav-users/2018-January/005742.html

Temp work around:

--------------------------------

[clamav-users] deleted files eating up file descriptors
http://lists.clamav.net/pipermail/clamav-users/2018-January/005744.html

Reply |

Bug in clamav 0.99.2

Thanks Bob..

Reply |

can we check for sender only

I have this problem, I tried to block email containing .html extension from senders. It works.

But then, I wanted to allow .html extension from certain domains. It never got through because simscan checks the same rule for both senders and recipients.

For instance, I allow senders from gmail to send .html extension to local domain exampledotcom like this rule:

gmaildotcom:clam=yes,spam=yes,spam_hits=12
:clam=yes,spam=yes,spam_hits=12,attach=.html

It will never get through because while gmaildotcom match the first line and is allowed, the recipients match the second line and is not allowed.

Is there a solution to this so that simscan will only check for sender?

Reply |

can we check for sender only

I think that the logic is correct and it should work as you expect, but (sorry if I ask) do you actually have something like gmail.co and not gmaildotcom, right?

Reply |

can we check for sender only

yeah, gmaildotcom is just an example.

So, there is no way to check for sender only..

Thanks Roberto

Best Regards

Reply |

simscan issue

Czy chodzi?o Ci o: mam taki problem z sim scan
I have a problem with simscan

simscan: check_spam detected spam refuse message
@400000004f6d08c5185a65dc simscan: exit error code: 82
@400000004f6d08c5185b8ad4 qmail-smtpd: message rejected (Your email is considered spam (1001.30 spam-hits)):

Reply |

can you please provide

Mihoo, can you please provide the smapd.log details just to see why you got such an high spamassassin score? Do you have userprefs settings?

Reply |