Before continuing, some details about this:
- Yes, I know that exists others solutions.
Yes, I know with Linux can obtain the same results.
Yes, I know that it seems a crazy solution.
Yes, I know (in the practice) that this solutions generate a moderate cpu usage. I have a Pentium IV with 75% of cpu usage (this can change with new features from MT... see scripts explanation) and we are a little ISP.
Yes, yes, yes...
but
I use MT from six years ago (when John Tully & Arnis Riekstins answered the company mails themselves) and always I try to resolve any networking necessity whit MT.
I had the trust that can resolve this whit MT.
and after thinking about this, I can't never sleep all night from many days, so, for my health and wife I made it !!!
I use 3 scripts:
itwbspam (run every second)
spamtocheck (run every 10 seconds)
tempclear (run every 30 minutes)
Script itwbspam (remove the # lines to work, I comment the most important parts):
Code: Select all
:set SCRIPT itwbspam;
# Always I think that MT permit one script running at the same time, for my surprise is not, so I
# use this to run at the same time one script (simulating a threads).
# You can reduce this, if your cpu usage is high or your mail traffic is reduced.
# In my case, less than 10 permit detect if the incoming SMTP connection is an spam or not
# quickly, and the cpu usage is reasonable.
# If you need run only one script at the same time, set at < 1 because the script it's count the
# same one when running.
:if ([:len [/system script job find script=$SCRIPT]] < 10) do={ \
# I use a firewall rule to add ips to this list (see below in firewall rules).
:set ALIST checkspam;
:foreach IPT in [/ip firewall address-list find list=$ALIST] do={ \
:set COMM [/ip firewall address-list get $IPT comment];
# For not process the same ip in more than one script, I put a comment to prevent this.
# Maybe in a future we can put a comment in a DYNAMIC list (I don't know why we can do that)
# so that can reduce the cpu usage.
:if ($COMM = "") do={ \
/ip firewall address-list set $IPT comment=checking;
# Now you need to revert the ip for querying a dnsbl server and know if the server that try to
# connect to your SMTP server is a spam or not.
:set IP [/ip firewall address-list get $IPT address]; :set IPL $IP;
:set l [:len $IP]; :set n [:find $IP .];
:set A [:pick $IP 0 $n];
:set IP [:pick $IP ($n + 1) 15]; :set n [:find $IP .];
:set B [:pick $IP 0 $n];
:set l [:len $IP]; :set IP [:pick $IP ($n + 1) 15]; :set n [:find $IP .];
:set C [:pick $IP 0 $n];
:set l [:len $IP]; :set IP [:pick $IP ($n + 1) 15];
:set D $IP;
:set COUNT 0;
# Now ask to black lists.
:set DSBL .dnsbl.sorbs.net;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP != "0.0.0.0") do={ :set COUNT ($COUNT + 1); }
:set DSBL .blackholes.five-ten-sg.com;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP != "0.0.0.0") do={ :set COUNT ($COUNT + 1); }
:set DSBL .dnsbl.tqmcube.com;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP != "0.0.0.0") do={ :set COUNT ($COUNT + 1); }
:set DSBL .zen.spamhaus.org; :set SPAM 127.0.0.4;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP = $SPAM) do={ :set COUNT ($COUNT + 1); }
:set DSBL .bl.spamcop.net; :set SPAM 127.0.0.2;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP = $SPAM) do={ :set COUNT ($COUNT + 1); }
# You can add so many black lists as you want, only you need set the DSBL and SPAM variable.
# If the ip returned from the dnsbl server coincides whit SPAM, the COUNT variable add
# by one.
# From servers like sorbs.net or five-ten-sg that return more than one value, I don't use the SPAM
# variable and check only the value if different to 0.0.0.0 and I assuming is spam when is
# different.
:if ($COUNT < 3) do= { \
# If the ip server COUNT lees than 3, I assuming maybe need more checks, so you can add
# others black lists to be safe that this ip it's not a spam server !.
:set DSBL .list.dsbl.org; :set SPAM 127.0.0.2;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP = $SPAM) do={ :set COUNT ($COUNT + 1); }
:set DSBL .dnsbl-1.uceprotect.net; :set SPAM 127.0.0.2;
:set IP [:resolve ($D . "." . $C . "." . $B . "." . $A . $DSBL)];
:if ($IP = $SPAM) do={ :set COUNT ($COUNT + 1); }
}
# Now according to the value in COUNT you can decide which is the address list that it
# corresponds to each case. In my case this is the best solution:
# if the server COUNT 1, is can or not a spam server, so it's fine to me and pass,
# if have 2, I think they are amateurs and they need to sample with other ISPs,
# if have 3 or more... hey, leave me alone !!!
:if ($COUNT > 1) do= { \
:if ($COUNT = 2) do={ /ip firewall address-list add list=tempspamlight address=$IPL; }
:if ($COUNT = 3) do={ /ip firewall address-list add list=tempspammedium address=$IPL; }
:if ($COUNT > 3) do={ /ip firewall address-list add list=tempspamhard address=$IPL; }
# Yes, you need to use STATIC list because we can't add a DYNAMIC list from script or
# timeout in a STATIC list (I don't know why, if we can in the future, this reduce considerably
# the cpu usage) and maybe, this server it's hacking so you can't assume that this server
# is a spam server only for a while, so you need to add a timeout value (see below in
# firewall rules).
} else= { \
/ip firewall address-list add list=tempsmtp address=$IPL;
}
# Once the server is cataloged you need delete the ip from the temp list
/ip firewall address-list remove $IPT;
}
}
}
Script spamtocheck: this script purpose it's only transform a DYNAMIC list to STATIC list because in DYNAMIC list you can't put a comment (why !!!???) or counter, or anything that can I use to prevent the script itwbspam take an ip that is checking in another script itwbspam at the same time.
Code: Select all
:set ALIST itwbspam; :set BLIST checkspam;
:local IPLIST [/ip firewall address-list find dynamic=yes list=$ALIST];
:foreach IP in $IPLIST do={ \
:set IPL [/ip firewall address-list get $IP address];
/ip firewall address-list add list=$BLIST address=$IPL;
/ip firewall address-list remove $IP;
}
Code: Select all
:set ALIST tempsmtp;
:for each IP in [/ip firewall address-list find list=$ALIST] do={ /ip firewall address-list remove $IP; }
:set ALIST tempspamlight;
:for each IP in [/ip firewall address-list find list=$ALIST] do={ /ip firewall address-list remove $IP; }
chain=smtp protocol=tcp src-port=25 src-address-list=staticwhitelist action=accept
:set ALIST tempspammedium;
:for each IP in [/ip firewall address-list find list=$ALIST] do={ /ip firewall address-list remove $IP; }
:set ALIST tempspamhard;
:for each IP in [/ip firewall address-list find list=$ALIST] do={ /ip firewall address-list remove $IP; }
Firewall rules:
I have an separate server before my SMTP server, so I put this rules in filter rules/XXX chain, maybe you need change this according to your configuration. Obviously I have more than this rules, I mention here only it related whit this scripts.
First you need to be safe that your customers can arrive to the server. In my case I have a relay server configured in different ip for the SMTP in the same server, but if you use the same ip server to receive and send emails you need this rule.
IMPORTANT, if you have customers out side your network whit this scripts they can't send mails from the SMTP server any more unless if your SMTP server are connecting to internet whit the same box where you run this scripts. In this case you need to add a few mangle rules to know which connection it's established for your SMTP server and mark the packet to pass over the spam verification.
Code: Select all
;;; Accept network customers !!!
chain=smtp protocol=tcp dst-port=25 src-address-list=customers action=accept
On the other hand, I have a static white list address from ISPs or Companies that I know don't make spam or are big ISP and they SMTP servers appears in more than one black list, so I need this servers pass over the spam verification (staticwhitelist). For example, here in Argentina, one of the biggest ISP is Speedy and their SMTP servers appears in more than one black list.
Code: Select all
;;; Accept white list !!!
chain=smtp protocol=tcp dst-port=25 src-address-list=whitelist action=accept
chain=smtp protocol=tcp dst-port=25 src-address-list=staticwhitelist action=accept
chain=smtp protocol=tcp src-port=25 src-address-list=whitelist action=accept
chain=smtp protocol=tcp src-port=25 src-address-list=staticwhitelist action=accept
Code: Select all
;;; Pass the temp to smtpok
chain=smtp src-address-list=tempsmtp action=add-src-to-address-list address-list=smtpok address-list-timeout=12h
Code: Select all
;;; Accept tempsmtp !!!
chain=smtp protocol=tcp dst-port=25 src-address-list=smtpok action=accept
chain=smtp protocol=tcp src-port=25 src-address-list=smtpok action=accept
;;; NOT accept static spam !!
chain=smtp protocol=tcp dst-port=25 src-address-list=staticspam action=reject reject-with=icmp-network-unreachable
Code: Select all
;;; Pass the temp to spam
chain=smtp src-address-list=tempspamlight action=add-src-to-address-list address-list=spamlight address-list-timeout=2h
chain=smtp src-address-list=tempspammedium action=add-src-to-address-list address-list=spammedium address-list-timeout=6h
chain=smtp src-address-list=tempspamhard action=add-src-to-address-list address-list=spamhard address-list-timeout=2d
Code: Select all
;;; Pass the temp to spam
chain=smtp src-address-list=tempspamlight action=add-src-to-address-list address-list=spamlight address-list-timeout=2h
chain=smtp src-address-list=tempspammedium action=add-src-to-address-list address-list=spammedium address-list-timeout=6h
chain=smtp src-address-list=tempspamhard action=add-src-to-address-list address-list=spamhard address-list-timeout=2d
;;; Not accept temps
chain=smtp protocol=tcp dst-port=25 src-address-list=itwbspam action=reject reject-with=icmp-host-unreachable
chain=smtp protocol=tcp dst-port=25 src-address-list=checkspam action=reject reject-with=icmp-host-unreachable
chain=smtp protocol=tcp dst-port=25 src-address-list=tempsmtp action=reject reject-with=icmp-host-unreachable
;;; NOT accept spamhard !!
chain=smtp protocol=tcp dst-port=25 src-address-list=spamhard action=reject reject-with=icmp-network-unreachable
;;; NOT accept spammedium !!
chain=smtp protocol=tcp dst-port=25 src-address-list=spammedium action=reject reject-with=icmp-network-unreachable
;;; NOT accept spamlight !!
chain=smtp protocol=tcp dst-port=25 src-address-list=spamlight action=reject reject-with=icmp-network-unreachable
Code: Select all
;;; Add to check
chain=smtp protocol=tcp dst-port=25 tcp-flags=syn connection-state=new src-address-list=!checkspam action=add-src-to-address-list address-list=itwbspam address-list-timeout=20m
chain=smtp protocol=tcp dst-port=25 tcp-flags=syn connection-state=new src-address-list=!checkspam action=reject reject-with=icmp-host-unreachable
chain=smtp protocol=tcp src-port=25 connection-state=established action=log log-prefix=""
Basically the process can resume like this:
- One server try to connect to our SMTP server
If the ip is in the white list address the packet is delivery to our server
If the ip server is in the address list smtpok the packet is delivery to our server
If the packet arrives to the rule that check is a new connection, the ip is added to the address list itwbspam
The script spamtocheck run every 10 seconds and move the ip address in the DYNAMIC list itwbspam to a STATIC list checkspam because we can't add a comment in DYNAMIC lists !!
The script itwbspam run every 1 second and check every ip address in the STATIC list checkspam to determine this ip is a spam server or not, if is spam then go to a tempspam list categorie, is not go to tempsmtp
Code: Select all
#!/bin/bash
#
today=`date "+%m%d%y"`;
workdir="/home/mt";
targets=(you ip or ips server);
cd $workdir
echo > wlists.rsc
# whitelist
echo :foreach subnet in [/ip firewall address-list find list=whitelist] do=\{ /ip firewall address-list remove \$subnet \} >> wlists.rsc
wget -t 2 -O dnswl.txt http://www.dnswl.org/data/generic-dnswl
egrep "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" dnswl.txt | awk '{print $1}' | awk -F";" '{print $1}' | grep -v ";" | sort +1 | awk '{print "/ip firewall address-list add l
ist=whitelist address=" $1}' >> wlists.rsc
wget -t 2 -O rediris.txt http://www.rediris.es/abuses/eswl/data/mtawl.postfix
egrep "[0-9]{1,3}" rediris.txt | awk '$1 > 0 " " $1 {print "/ip firewall address-list add list=whitelist address="$1"/32"}' >> wlists.rsc
echo >> wlists.rsc
# Remove duplicate lines
sort wlists.rsc | uniq > wlistsu.rsc
# Reduce to small files to be more manageable for MT
split wlistsu.rsc -a 5 -d -l 500 spwlists
# foreach target($targets)
for X in ${targets[@]}; do
echo $X;
for I in `find -name "spwlists?????"`; do
mv $I "$I.rsc";
echo "put $I.rsc" | ftp $X 21
done
for I in `find /home/mt/ -name "spwlists?????.rsc" -printf "%f\n" | sort`; do
echo "procesando $I";
ssh -l black -i /home/mt/black.dsa $X "/import $I" >> /home/mt/wlists.output
done
for I in `find /home/mt/ -name "spwlists?????.rsc" -printf "%f\n" | sort`; do
echo "delete $I" | ftp $X 21
rm -f $I;
done
done
- When we have more attributes to manipulate from script a STATIC address list it will be reduce the cpu usage a lot.
If MT adds a counter to the address list, then we can controlling how many times the ip in the smtpok address list is hitting, so, we can assuming that if there are more than 100.000 hits in a particular ip then this server is not spammer and we can move to a white list whit a timeout of 1 month.
When the dnsbl server returns an ip like 10.0.0.7 or 10.0.0.10 is definitively a BIG SPAMMER. To know this it need to fix the itwbspam script to accept more than one ip from a dnsbl server.
P.S.: if you add more black lists, please share it whit us !!!
P.S.: if you add more white lists, please share it whit us !!!
Enjoy,
Bernardo Pita
Argentina