Page 1 of 1

script to add ip to list based on log - help needed

Posted: Sun Mar 06, 2022 4:57 pm
by kinglestat
Hi folks,

I am trying to make a scrict which adds valid login IPs to a list, and bad requests to another list, but frankly I can't figure out the Mikrotik language

this is some test code I copied from the forum and adapted
:local tmp [:toarray [/log print  where time>([/system clock get time] - 1h)]];
#for some reason this line echoes the log to the console and nothing is put in array

:foreach i in=$tmp do={
:local logLine  $i ;
#:local put [:find $logLine "dns" -1];
#:put $value
:local put $i;
}
if I change /log print to /log print as-value it is put in array, but I can't use the where clause
I would apreciate some help

Re: script to add ip to list based on log - help needed

Posted: Sun Mar 06, 2022 8:50 pm
by Jotne
Here is what I do use to get last 5m log of massages that do contains: negotiation failed
You should be able to modify it to get the data you like
:local loglist [:toarray [/log find  time>([/system clock get time] - 5m) message~"negotiation failed"]]

Re: script to add ip to list based on log - help needed

Posted: Mon Mar 07, 2022 2:16 pm
by rextended
if the time is 00:00 the "[/system clock get time] - 5m" is negative time value -00:05:00 and do not find the previous day log from 23:55
also, on previous day the date is indicated differently...
Remember? :-P

Re: script to add ip to list based on log - help needed

Posted: Mon Mar 07, 2022 3:26 pm
by rextended
On step 7 is present the final usable script.

step 1: use my RegEx to identify all log lines that containing one IP:
((25[0-5]|(2[0-4]|[01]?[0-9]?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]?[0-9]?)[0-9])

on script test on terminal, \ must be added before ? on 6.x version
/log print where message~"((25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])"

step 2: RegEx can not be used for pick or find...
must be noticed all interested logs like:
SYSTEM: user admin logged in from 100.64.0.33 via winbox
login failure for user admin from 100.64.0.33 via winbox

step 3: ignore the PREFIX and find a pattern for the rules like:
when the string start with "user admin logged in from " and end with " via winbox" put the IP between on ok list
when the string start with "login failure for user admin from " and end with " via winbox" put the IP between on fail list

step 4: how obtain all logs than containing inside a valid IP with a cycle:
/log
:foreach rlog in=[find where message~"((25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])"] do={
    :local rmess [get $rlog message]
    :put $rmess
}

step 5: distinguish between actions (igoring the use of complex scripting with array, etc....:
THE SPACE MUST BE CONSIDERED
/log
:global okmsg "user admin logged in from "
:global failmsg "login failure for user admin from "
:global endmsg " via winbox"
:foreach rlog in=[find where message~"((25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])"] do={
    :local rmess [get $rlog message]
    :if (($rmess~$okmsg) and ($rmess~$endmsg)) do={
         :put "OK: $rmess"
    }
    :if (($rmess~$failmsg) and ($rmess~$endmsg)) do={
         :put "FAIL: $rmess"
    }
}

step 6: with previous founded pattern, extract the IPs
/log
:global okmsg "user admin logged in from "
:global failmsg "login failure for user admin from "
:global endmsg " via winbox"
:foreach rlog in=[find where message~"((25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])"] do={
    :local rmess [get $rlog message]
    :if (($rmess~$okmsg) and ($rmess~$endmsg)) do={
         :put "OK: $rmess"
         :local ipinside [:pick $rmess ([:find $rmess $okmsg -1] + [:len $okmsg]) [:find $rmess $endmsg -1]]
         :put "IP inside is $ipinside"
    }
    :if (($rmess~$failmsg) and ($rmess~$endmsg)) do={
         :put "FAIL: $rmess"
         :local ipinside [:pick $rmess ([:find $rmess $failmsg -1] + [:len $failmsg]) [:find $rmess $endmsg -1]]
         :put "IP inside is $ipinside"
    }
}

step 7: log the operations, remove the useless "put" for terminal, define the lists and add the IPs, if not already added, for prevent errors...
/log
:global okmsg "user admin logged in from "
:global failmsg "login failure for user admin from "
:global endmsg " via winbox"
:global listok "list_success_winbox_attempt"
:global listfail "list_failed_winbox_attempt"
:foreach rlog in=[find where message~"((25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])\\.){3}(25[0-5]|(2[0-4]|[01]\?[0-9]\?)[0-9])"] do={
    :local rmess [get $rlog message]
    :if (($rmess~$okmsg) and ($rmess~$endmsg)) do={
         :local ipinside [:pick $rmess ([:find $rmess $okmsg -1] + [:len $okmsg]) [:find $rmess $endmsg -1]]
         /ip firewall address-list
         :if ([:len [find where list=$listok and address=$ipinside]] = 0) do={
             add list=$listok address=$ipinside
             :log warning "IP $ipinside added to $listok"
         }
    }
    :if (($rmess~$failmsg) and ($rmess~$endmsg)) do={
         :local ipinside [:pick $rmess ([:find $rmess $failmsg -1] + [:len $failmsg]) [:find $rmess $endmsg -1]]
         /ip firewall address-list
         :if ([:len [find where list=$listfail and address=$ipinside]] = 0) do={
             add list=$listfail address=$ipinside
             :log error "IP $ipinside added to $listfail"
         }
    }
}
REMEMBER TO REMOVE THE \ BEFORE ? IF USED INSIDE A SCRIPT OR SCHEDULER


Final notices:
The script do not prevent if one IP on whitelist go also on blacklist if sometime fail login.
for do that must be replaced
:if ([:len [find where list=$listfail and address=$ipinside]] = 0) do={
with
:if ([:len [find where (list=$listfail or list=$listok) and address=$ipinside]] = 0) do={
but if first found the failed login, the successful login do not remove IP from blacklist,
for do that must be added a new line
remove [find where list=$listfail and address=$ipinside]
just before ":log warning"

More complex scripting can be doed if the start / end patterns are putted on one bidimensional array,
with a cycle "foreach" pair of elements on the array for find IPs for more success/fail events at same time.

Another approach is possible, like first finding the start string, than later check if inside is present a valid IP,
But I prefer to find at start only the lines with at least one valid IP inside

Re: script to add ip to list based on log - help needed

Posted: Mon Mar 07, 2022 5:28 pm
by Jotne
if the time is 00:00 the "[/system clock get time] - 5m" is negative time value -00:05:00 and do not find the previous day log from 23:55
for me that is an bug. If you work with time, you do work with time and 00:00 -5m should be 23:55.

Re: script to add ip to list based on log - help needed

Posted: Thu Mar 10, 2022 8:41 am
by terenceagius
Thanks guys. I will test and let you know how it goes. And splunk, thanks, I wrote my own; uses accounting. It has some bugs but its useful to me.

Re: script to add ip to list based on log - help needed

Posted: Sat Mar 12, 2022 12:32 pm
by kinglestat71
@rextended
Your script gives an error on line 6; that line works by itself but not in the script

Re: script to add ip to list based on log - help needed

Posted: Sun Mar 13, 2022 1:33 am
by rextended
line 6 is
:global listfail "list_failed_winbox_attempt"
is impossible that give an error, on any 6 / 7 routeros version...

If the error is "expected end of command (line 6 column 38)" probably you do not copy and paste also the first line of the script: "/log"

Re: script to add ip to list based on log - help needed

Posted: Fri Jul 14, 2023 4:54 pm
by tasyaraiva
@rextended
I have tested your script and work

About this line with condition user is admin
:global failmsg "login failure for user admin from "
and try login from winbox
:global endmsg " via winbox"

How if dynamic user and service (winbox, ssh, ftp) like brute force attack?

Re: script to add ip to list based on log - help needed

Posted: Fri Jul 14, 2023 6:18 pm
by rextended
replace bot occurencies of admin and winbox with the character asterisk *

Re: script to add ip to list based on log - help needed

Posted: Sat Jul 15, 2023 4:37 am
by tasyaraiva
Like this ?
:global failmsg "login failure for user * from "
:global endmsg " via *"

I have tested but not work without error (RoS v 7.10)