ezmlm-idx

May 17, 2014 by Roberto Puzzanghera 18 comments

ezmlm is an easy-to-use, high-speed mailing list manager for qmail. ezmlm-idx is an extension of the original ezmlm software of D.J. Bernstein.

Since version 7.2.0 ezml-idx is able to alter the From: address in messages from domains with strict SPF policies as announced here http://lists.untroubled.org/?list=ezmlm&cmd=showmsg&msgnum=14238. This will solve DMARC reject problems with users having yahoo or AOL addresses. Some changes were made in 7.2.1 and 7.2.2 to improve the process (read here http://lists.untroubled.org/?list=ezmlm&cmd=showmsg&msgnum=14256).

Compile:

wget http://ezmlm.untroubled.org/archive/7.2.2/ezmlm-idx-7.2.2.tar.gz
tar xzf ezmlm-idx-7.2.2.tar.gz
cd ezmlm-idx-7.2.2
chown -R root:root .
cd lang
ln -s en_US default
cd ..
make
make man

Test:

# ./ezmlm-test

getconfopt library:   OK
ezmlm-make:           OK
Using subdb plugin:   std
ezmlm-reject:         OK
ezmlm-[un|is]sub[n]:  OK
ezmlm-checksub:       OK
ezmlm-send:           OK
ezmlm-tstdig:         OK
ezmlm-weed:           OK
ezmlmrc contents:     OK
ezmlm-clean:          OK
ezmlm-store:          OK
ezmlm-return:         OK
ezmlm-warn (1/2):     OK
ezmlm-manage (1/2):   OK
ezmlm-request:        OK
ezmlm-split:          OK
ezmlm-gate:           OK
ezmlm-idx:            OK
ezmlm-get (index):    OK
ezmlm-get (get):      OK
ezmlm-get (thread):   OK
ezmlm-get (digest):   OK
ezmlm-manage (2/2):   OK
ezmlm-moderate:       OK
ezmlm-warn (2/2):     OK
ezmlm-archive:        OK
ezmlm-dispatch:       OK                                                                                                                    
dispatch editor:      OK                                                                                                                    
decode sender:        OK                                                                                                                    
Verifying message header and body contents...                                                                                               
flags:                OK                                                                                                                    
substitutions:        OK                                                                                                                    
messages:             OK                                                                                                                    
subscribe nop:        OK                                                                                                                    
unsubscribe nop:      OK                                                                                                                    
subscribe probe:      OK                                                                                                                    
unsubscribe probe:    OK
subscribe:            OK
unsubscribe:          OK
moderated subscribe:  OK
moderated unsub:      OK
subscribe by mod:     OK
unsubscribe by mod:   OK
copylines:            OK
ezmlm-send:           OK
ezmlm-send trailer:   OK
omitbottom (-manage): OK
omitbottom (-get):    OK
Cleaning up...

Install:

make install

Install mysql support

NB: I'm not using the sql support anymore, so the following section is not updated. I leave here my old notes on how to setup mysql for ezmlm-idx for future reference.

The main advantages are that you are using an address database system that can easily be accessed from any number of other programs via ODBC, perl, java, PHP, ... You can easily hook up ezmlm with your customer database, etc. ezmlm programs compiled with SQL support (and when available also those compiled with support for other SQL servers) are entirely backwards compatible. You can mix SQL dbs with normal ezmlm dbs, and convert lists between them.

Unfortunately the mysql support is not working here. When I create a list with mysql support via qmailadmin the database is not populated and I get an error like this in the qmail-send log:

@400000005364eb332209c884 delivery 2761: deferral: ezmlm-checksub:_fatal:_Could_not_load_plugin_/usr/local/lib/ezmlm/sub-10.0.0.3.so:_/usr/local/lib/ezmlm/sub-10.0.0.3.so:_cannot_open_shared_object_file:_No_such_file_or_directory/

10.0.0.3 is the IP of mysql host. The program is looking for sub-10.0.0.3.so instead of sub-mysql.so.

Create the database

# mysql -u root -p

CREATE USER 'ezmlm'@'[mailserver-IP]' IDENTIFIED BY '[ezmlm-password]';

GRANT USAGE ON * . * TO 'ezmlm'@'[mailserver-IP]' IDENTIFIED BY '[ezmlm-password]' WITH MAX_QUERIES_PER_HOUR 0
MAX_CONNECTIONS_PER_HOUR 0
MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;

CREATE DATABASE IF NOT EXISTS `ezmlm` ;
GRANT ALL PRIVILEGES ON `ezmlm` . * TO 'ezmlm'@'[mailserver-IP]';

If you have mysql installed in a non standard path (default prefix is /usr), you may have to adjust some conf files, and also to patch the source. I have mysql under /usr/local/mysql and here is what I have modified:

# nano conf-cc
gcc -O -g -I/usr/local/mysql/include

# nano conf-ld
cc -g -rdynamic -B /usr/local/mysql/lib

At this point I faced some compilation breaks, maybe because I have mysql in /usr/local/mysql. I solved applying the following patch:

diff -ruN ezmlm-idx-7.2.0/sub-mysql.c ezmlm-idx-7.2.0-patched/sub-mysql.c
--- ezmlm-idx-7.2.0/sub-mysql.c 2014-05-03 00:07:29.000000000 +0200
+++ ezmlm-idx-7.2.0-patched/sub-mysql.c 2014-05-03 14:53:43.000000000 +0200
@@ -6,7 +6,7 @@
 #include "subdb.h"
 #include <sys/types.h>
 #include <mysql.h>
-#include <mysql/errmsg.h>
+#include <errmsg.h>
 #include <mysqld_error.h>
 
 static stralloc line = {0};

Compile:

make mysql

Test:

# ./ezmlm-test -s mysql -u ezmlm -p [ezmlm-password] -h [mailserver-IP]

getconfopt library:   OK
ezmlm-make:           OK
Using subdb plugin:   mysql
ezmlm-reject:         OK
ezmlm-[un|is]sub[n]:  OK
ezmlm-checksub:       OK
ezmlm non-SQL:        OK
ezmlm SQL:            OK
ezmlm-send:           OK
ezmlm-tstdig:         OK
ezmlm-weed:           OK
ezmlmrc contents:     OK
ezmlm-clean:          OK
ezmlm-store:          OK
ezmlm-return:         OK
ezmlm-warn (1/2):     OK
ezmlm-manage (1/2):   OK
ezmlm-request:        OK
ezmlm-split:          OK
ezmlm-gate:           OK
ezmlm-idx:            OK
ezmlm-get (index):    OK
ezmlm-get (get):      OK
ezmlm-get (thread):   OK
ezmlm-get (digest):   OK
ezmlm-manage (2/2):   OK
ezmlm-moderate:       OK
ezmlm-warn (2/2):     OK
ezmlm-archive:        OK
ezmlm-dispatch:       OK
dispatch editor:      OK
decode sender:        OK
Verifying message header and body contents...
flags:                OK
substitutions:        OK
subscribe nop:        OK
unsubscribe nop:      OK
subscribe probe:      OK
unsubscribe probe:    OK
subscribe:            OK
unsubscribe:          OK
moderated subscribe:  OK
moderated unsub:      OK
subscribe by mod:     OK
unsubscribe by mod:   OK
copylines:            OK
ezmlm-send:           OK
ezmlm-send trailer:   OK
omitbottom (-manage): OK
omitbottom (-get):    OK
Cleaning up...

Now you are ready to install:

make install

Comments

download link error

Hi,

The link for download http://ezmlm.untroubled.org/archive/7.2.2/ezmlm-idx-7.2.2.tar.gz is broken. Do you have the source, please?

Thank you

Joao

Reply |

download link error

Hi Roberto, thank you for replying!

Do you know about this error?

# ./ezmlm-test
getconfopt library: OK
ezmlm-make: OK
Using subdb plugin: std
ezmlm-reject: OK
ezmlm-[un|is]sub[n]: OK
ezmlm-checksub: OK
ezmlm-send: OK
ezmlm-tstdig: OK
ezmlm-weed: OK
ezmlmrc contents: OK
ezmlm-clean: OK
ezmlm-store: OK
ezmlm-return: OK
ezmlm-warn (1/2): OK
ezmlm-manage (1/2): OK
ezmlm-request: OK
ezmlm-split: OK
ezmlm-gate: OK
ezmlm-idx: OK
ezmlm-get (index): OK
ezmlm-get (get): OK
ezmlm-get (thread): OK
ezmlm-get (digest): OK
ezmlm-manage (2/2): OK
ezmlm-moderate: OK
ezmlm-warn (2/2): OK
ezmlm-archive: OK
ezmlm-dispatch: OK
dispatch editor: Error delivering confirmed post

Reply |

download link error

I've never seen it. Tomorrow I'll try to find information in the code

Reply |

download link error

I had a look at the code, but i'ts not so easy to understand what's going on with that error... I'm sorry

Reply |

download link error

Hi Roberto, thank you for replying.

I'm trying to compile in rocky linux 9.
I'll try in rocky linux 8.

Thanks
joao

Reply |

How do I add List-Unsubscribe header to email?

Hi,

Recently I come across the email reputation "Your message does not contain a List-Unsubscribe header".

How do I add the "List-Unsubscribe" into the email's header?

If found from https://mailtrap.io/blog/list-unsubscribe-header/

There is not qmail implementation method.

Thx.

Reply |

How do I add List-Unsubscribe header to email?

Hi, which mailing list program are you using? ezmlm-idx already has this feature...

Reply |

I get stuck on ezml-test

Hi

After compiling everything I get stuck in ezml-test

Testing with and without sql I get the same result after a while

Coud be some problem with the domain used for test (example.com), because I have a lot of filter on who can sent mail from the server where I'm doing the test ?

ezmlm-archive: Segmentation fault (core dumped)
failed after "num" was restored

I created the DB and I see that there is a lot of tables on it now

list
list_allow
list_allow_slog
list_cookie
list_deny
list_deny_slog
list_digest
list_digest_cookie
list_digest_mlog
list_digest_slog
list_mlog
list_mod
list_mod_slog
list_slog

Do I have to create a file with the connection parameters some where ?

Is very confusing this point

I think if I don't put anywhere user/pass/db there is no way that the program can conect to the db :D

One more thing I don't understand very well is if all the domains use the same DB to store the lists info, or I can give a different DB to every domain ?

I'm very LOST !!!!

Reply |

I get stuck on ezml-test

I haven't used it in a long time and I honestly don't remember much anymore. To troubleshoot your breaks I would look at strace in the following way

strace -o test.log ./ezmlm-test

Reply |

I get stuck on ezml-test

Finally I do a trace ( truss ) on FreeBSD, but I don't know what is happening

I don't have time tu debug, so, I will use it without mysqlcry

stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/num",0x7fffffffe6c8) ERR#2 'No such file or directory'
stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/archive/authors",0x7fffffffe6c8) ERR#2 'No such file or directory'
stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/archive/lock",0x7fffffffe6a8) ERR#2 'No such file or directory'
stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/archive/subjects",0x7fffffffe688) ERR#2 'No such file or directory'
stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/archive/threads",0x7fffffffe668) ERR#2 'No such file or directory'
stat("/var/ports/usr/ports/mail/ezmlm-idx/work/ezmlm-idx-7.2.2/__TSTDIR/archnum",0x7fffffffe648) ERR#2 'No such file or directory'
vfork() = 88390 (0x15946)
wait4(-1,{ EXITED,val=0 },0x0,0x0) = 88390 (0x15946)
fork() = 88391 (0x15947)
wait4(-1,{ SIGNALED,sig=SIGSEGV,cored },0x0,0x0) = 88391 (0x15947)
stat("/usr/share/nls/C/libc.cat",0x7fffffffe1c8) ERR#2 'No such file or directory'
stat("/usr/share/nls/libc/C",0x7fffffffe1c8) ERR#2 'No such file or directory'
stat("/usr/local/share/nls/C/libc.cat",0x7fffffffe1c8) ERR#2 'No such file or directory'
stat("/usr/local/share/nls/libc/C",0x7fffffffe1c8) ERR#2 'No such file or directory'
write(2,"Segmentation fault (core dumped)"...,33) = 33 (0x21)
fcntl(1,F_DUPFD_CLOEXEC,0xa) = 11 (0xb)
dup2(2,1) = 1 (0x1)
write(1,"failed after "num" was restored"...,32) = 32 (0x20)
dup2(11,1) = 1 (0x1)
close(11) = 0 (0x0)
exit(0x64)
process exit, rval = 100

Reply |

compiling ezmlm-idx

also after make

must be added:

make mysql

it do not makes it by default

Reply |

compiling ezmlm-idx

yes, as you can see above this is explained later. The mysql installation is described separately

Reply |

compile ezmlm-idx

just before make u must create a file describing your mysql installation path

>conf-mysql

In my case it contains:

/usr

else plugin-mysql will not be compiled

Reply |

compile ezmlm-idx

as described above, the mysql installation path is declared inside conf-cc and conf-ld

Reply |

FreeBSD 10.3 Install fail -help?

./ezmlm-test -s mysql -u ezmlm -p [ezmlm-password] -h [mailserver-IP]

getconfopt library:   OK
ezmlm-make:           OK
Using subdb plugin:   mysql
ezmlm-make: fatal: Could not load plugin /package/ezmlm-idx-7.2.2/sub-mysql.so: Cannot open "/package/ezmlm-idx-7.2.2/sub-mysql.so"
ezmlm-make failed to add subdb config info

Reply |

Hopefully someone else can be

Hopefully someone else can be of any help, as I can't do tests on freeBSD

Reply |

ezmlm strange error

Hi guyst,

new day, new strange error :-)

from /var/log/qmail/qmail/current:

delivery 152: deferral: ezmlm-send:_fatal:_error_fetching_DNS_record_for_mxtoolbox.com:_temporary_failure/

I have been receiving mail from mxtoolbox without problems for years, but they might have changed something; anyway, the recipient is a mailing list (all subscribers are local users), the mail is stuck in the queue and it is not delivered; I googled for this ezmlm-idx error with no luck, and I really do not understand why ezmlm-send should be doing a DNS lookup on the sender address...

thank you !

Reply |