Community discussions

MikroTik App
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Need Help In Layer7 || isolation between Browsing , download

Wed May 26, 2010 11:40 pm

i used layer 7 for identifying the context of web pages

and i want to do something that's important

and want to make Mikrotik sense and make difference between download traffic and browsing traffic

To make it isolate between Browsing and download

and better thing i thought about was

extension of files

like
exe , rar , zip , iso , daa , avi and etc.....

and iam doing this by putting for example exe word as Regular Expression in Regexp Textbox in layer 7 filter

and make rule in Firewall Mangle to mark packet that contain layer 7 condition
as download packet

and in the Queue what ever simple Queue or Queue Tree i shape the traffic with the nice speed i want to.

the big problem i just found

that the layer 7 search about the word exe in the entire code of the HTML page

Head and Body

not only the <head> Head Code </head>

and any one know that we want layer 7 search just in the Head code "Search Between just Head Tags" because The URL in contained in between Head Tags

and if it searched about the word "flv" for example in the body code of the web page there is flashes and many web pages contain small flv files in the body code and by this layer 7 when find flv work in the body code of the page filter it as download

and take for another example if some one explain in a tutorial or topic the extensions or just mentioned any of the extensions i shape in the body of the page

the page then is shaped

so any one know the Regular Expression code that make layer 7 search just in only the Head of the HTML Code

and make it not to search in the Body "<body> Body Code </body> "of the page please tell us
Last edited by hazemamer7 on Fri May 28, 2010 10:49 am, edited 1 time in total.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Need Help In Layer7 || isolation between Browsing , down

Thu May 27, 2010 7:47 am

you have to realize that downloading = browsing only thing that is different is size of objects transferred. Wile you can find some patterns using layer-7, use of it for purpose you want to would not be good solution due to resource use of layer-7 and quite substantial chance of false positives.

Focus on connection bytes, as objects and pictures of web page will be considerably smaller than say.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7186
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Thu May 27, 2010 8:38 am

add <head> tags to your regexp, but as janisk said searching those patterns will take a lot of rotuer's resources.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Thu May 27, 2010 12:48 pm

add <head> tags to your regexp, but as janisk said searching those patterns will take a lot of rotuer's resources.
ok not matter the resources that it consume if it works

Specifications of my server " Quad Core Processor and 4 GB of RAM "

So not matter

but how i add <head> to the word exe for example in regexp

loke this "<head>exe"
or what

i want to make layer7 just earch exclusively between

<html>
<head>............href="www.example.com/file.exe"......... </head>

<body>...........This page is to learn extensions and how it work for example exe is the extension of the executable files ...... bla bla bla </body>

</html>

if we notice here the exe word just came twice , one time between head tags and the other time between body tags

and any one know that body is the context of the entire web page
and head tags just contain the title and URL and some fewer tags
so we
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7186
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Thu May 27, 2010 2:26 pm

At first you have to learn how to write regular expressions.
Here is good starting point: http://www.regular-expressions.info/

Then write regular expression that matches ".exe" inside <head> ... </head> tags.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Thu May 27, 2010 5:31 pm

At first you have to learn how to write regular expressions.
Here is good starting point: http://www.regular-expressions.info/

Then write regular expression that matches ".exe" inside <head> ... </head> tags.

ok i read this topic many days ago very carefully , but i can not till now write the right code

to make the layer7 search just in between Head Tags

<head>Head Code</head>

any one know the code just make it easier to me and write the code here

and take exe for example.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Fri May 28, 2010 9:30 am

ok making it much easier to any one want to help

this code of Regular Expressions take any extension , not specific extension

\.[^.\\/:*?"<>|\r\n]+$


and this code take the entire HTML web Page file

<html>.*?<head>.*?<title>.*?</title>.*?</head>.*?<body[^>]*>.*?</body>.*?</html>

and this one can capture exe extension if it came between two tags

(<head[^>]*>)(exe)(</head>)

it will take exe if the page code was like this

<head>exe</head>

but will not take affect if the code of the page was like this

<head>any other words here besides exe</head>


with these codes any one can help us ??

these are last thing i could reach

and i think with more customize to one of the last codes we can get the code
of search for specific extension just between <head> Head Code </head> Tags
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Fri May 28, 2010 2:17 pm

(<head[^>]*>)(.*exe)(</head>)
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Fri May 28, 2010 4:29 pm

(<head[^>]*>)(.*exe)(</head>)
First of all thanks for paying attention to my question

This worked true

but for just if the example applied on this code


<head>any words here before exe</head>


and no space between exe</head> or even any other character

if it just take spaces of characters no code will be applied

and you can see yourself in layer7 as a test or using RegexBuddy Programs

i added .* before exe and after it and it works

(<head[^>]*>)(.*exe.*)(</head>)


and i have tested it , it worked successfully on like this HTML code

<head><title>Download Page</title><a href="http://download.microsoft.com/download/ ... </a></head>

but still one problem

if the HTML code was like previous one but the </head> tags close came in another line not in the same line RE code never take effect on HTML code

so need a simple add-in to the last RE code

(<head[^>]*>)(.*exe.*)(</head>)

to make it what ever (</head>) came in the same line or in another line will take effect also.

Thanks in advanced to anyone pay attention.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 12:18 am

No one can give us any hope ???
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 1:16 am

what is RE code?.. :?

maybe try <head[^>]*>.*exe.*(\r\n)*.*</head> or something like that...
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 4:15 am

Ok fine

thanks for paying attention to me

RE i mean with it Regular Expressions

the last code worked for me fine

but i noticed that if we have a line such this

http://download.microsoft.com/download/ ... 86-ENU.exe

and it not included in the head tag the code will not work

so we need another simple code that say that if http word & exe word came in the same line catch it

so we can catch any http traffic contain the executable files "exe files"

thanks in advance.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 9:28 am

L7 inspects only first 2KB of data... your detecting may fail because of that...

p.s. if the page is linked to some .exe file, it doesn't mean the file will be downloaded...
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 9:43 am

p.s. if the page is linked to some .exe file, it doesn't mean the file will be downloaded...
that is why use connection-bytes in the first place, so if connection takes longer than say, 2MB (for flash page to load) the throttle it down a notch or two.

Only difference between downloading and browsing is that downloading transfers more bytes per connection.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 01, 2010 10:43 am

p.s. if the page is linked to some .exe file, it doesn't mean the file will be downloaded...
that is why use connection-bytes in the first place, so if connection takes longer than say, 2MB (for flash page to load) the throttle it down a notch or two.

Only difference between downloading and browsing is that downloading transfers more bytes per connection.
ok fine i know that very well

but i want just to add something

if i just added the word exe in the layer7 regexp
and if the link of the file contain the word exe

for example the extension was .exe

the rule of mangle applied very well and the shaping of Queue work very well
the only wrong thing happening if the HTML page contain in it's body code the word exe
for example someone mentioned the word executable and the exe part of it and etc...

so the only problem here
we want to add condition to the word exe in regexp textbox so we can say with this codition

if the word exe came in the same line with the word http

capture it ok i think you got it now
for example

http://anydomain.com/anyurl.exe

here http and exe just come in the same line if this line if download link ok

i want a code for this

to say if the http & exe came in the same line capture it

thanks for who will help in advanced.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Jun 02, 2010 2:45 am

https?://.*\.exe
?..
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Sat Jun 05, 2010 1:50 am

https?://.*\.exe
?..
when i used this in RegexBuddy Software it worked correctly
but when using it in layer7 it did not work

and i will give you the code and you can try it yourself


/ip firewall layer7-protocol
add comment="" name=exe regexp="https\?://.*\\.exe"

/ip firewall mangle
add action=mark-packet chain=forward comment="" disabled=no dst-port=80 layer7-protocol=exe new-packet-mark=\
"Exe Download" passthrough=yes protocol=tcp src-address=192.168.0.0/24


the same code when i replace "https\?://.*\\.exe" with just exe

the mangle capture the traffic but with the code you gave me it did not work

and of course we can not use just exe word due to the reason i mentioned before
if the word exe came in the body code of the HTML Page it will be captured

so we want to make our code search Exclusively in between URL's "http://......words....exe"

and you can take a download link for example contains exe file to try on it


http://download.microsoft.com/download/ ... 86-ENU.exe

and finally i want to add that you are the most one who was active with me
and most helpfull one for me so if this problem has been solved or not
thanks at all.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Sun Jun 06, 2010 1:05 pm

I see the main reason:
/ip firewall mangle
add action=mark-packet chain=forward comment="" disabled=no dst-port=80 layer7-protocol=exe new-packet-mark=\
"Exe Download" passthrough=yes protocol=tcp src-address=192.168.0.0/24
you check only upload traffic, not traffic from the website. in upload (requests), it looks like
GET /path/to.exe HTTP/1.1
Host: http://www.example.com
<bla-bla-bla>
there's no "http://some/thing.exe" string in those packets
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Sun Jun 06, 2010 8:42 pm

I see the main reason:
/ip firewall mangle
add action=mark-packet chain=forward comment="" disabled=no dst-port=80 layer7-protocol=exe new-packet-mark=\
"Exe Download" passthrough=yes protocol=tcp src-address=192.168.0.0/24
you check only upload traffic, not traffic from the website. in upload (requests), it looks like
GET /path/to.exe HTTP/1.1
Host: http://www.example.com
<bla-bla-bla>
there's no "http://some/thing.exe" string in those packets

At all as i said to you before thank you very much

I have reached the method which enables me to shape download traffic Via extensions of files
and i will give you the Entire code right here soon to you Allah willing.
It worked for me 100 %

if i download such a exe , flv, rar , zip..........bla..bla..bla files
the traffic is shaped if any else browsing traffic
traffic is not shaped

and thanks very much for you effort with me.
Last edited by hazemamer7 on Tue Nov 05, 2013 7:11 am, edited 1 time in total.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 08, 2010 2:48 am

As i promised that i will give you the code
here is it

/ip firewall layer7-protocol
add comment="" name="Extension \" .exe \"" regexp="\\.(exe)"
add comment="" name="Extension \" .rar \"" regexp="\\.(rar)"
add comment="" name="Extension \" .zip \"" regexp="\\.(zip)"
add comment="" name="Extension \" .7z \"" regexp="\\.(7z)"
add comment="" name="Extension \" .cab \"" regexp="\\.(cab)"
add comment="" name="Extension \" .asf \"" regexp="\\.(asf)"
add comment="" name="Extension \" .mov \"" regexp="\\.(mov)"
add comment="" name="Extension \" .wmv \"" regexp="\\.(wmv)"
add comment="" name="Extension \" .mpg \"" regexp="\\.(mpg)"
add comment="" name="Extension \" .mpeg \"" regexp="\\.(mpeg)"
add comment="" name="Extension \" .mkv \"" regexp="\\.(mkv)"
add comment="" name="Extension \" .avi \"" regexp="\\.(avi)"
add comment="" name="Extension \" .flv \"" regexp="\\.(flv)"
add comment="" name="Extension \" .pdf \"" regexp="\\.(pdf)"
add comment="" name="Extension \" .wav \"" regexp="\\.(wav)"
add comment="" name="Extension \" .rm \"" regexp="\\.(rm)"
add comment="" name="Extension \" .mp3 \"" regexp="\\.(mp3)"
add comment="" name="Extension \" .mp4 \"" regexp="\\.(mp4)"
add comment="" name="Extension \" .ram \"" regexp="\\.(ram)"
add comment="" name="Extension \" .rmvb \"" regexp="\\.(rmvb)"
add comment="" name="Extension \" .dat \"" regexp="\\.(dat)"
add comment="" name="Extension \" .daa \"" regexp="\\.(daa)"
add comment="" name="Extension \" .iso \"" regexp="\\.(iso)"
add comment="" name="Extension \" .nrg \"" regexp="\\.(nrg)"
add comment="" name="Extension \" .bin \"" regexp="\\.(bin)"
add comment="" name="Extension \" .vcd \"" regexp="\\.(vcd)"

/ip firewall mangle
add action=mark-connection chain=prerouting comment="7z DOWNS" disabled=no layer7-protocol="Extension \" .7z \"" new-connection-mark="7z DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="7z DOWNS" disabled=no new-packet-mark=7z passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="asf DOWNS" disabled=no layer7-protocol="Extension \" .asf \"" new-connection-mark="asf DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="asf DOWNS" disabled=no new-packet-mark=asf passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="avi DOWNS" disabled=no layer7-protocol="Extension \" .avi \"" new-connection-mark="avi DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="avi DOWNS" disabled=no new-packet-mark=avi passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="bin DOWNS" disabled=no layer7-protocol="Extension \" .bin \"" new-connection-mark="bin DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="bin DOWNS" disabled=no new-packet-mark=bin passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="flv DOWNS" disabled=no layer7-protocol="Extension \" .flv \"" new-connection-mark="flv DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="flv DOWNS" disabled=no new-packet-mark=flv passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="iso DOWNS" disabled=no layer7-protocol="Extension \" .iso \"" new-connection-mark="iso DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark= "iso DOWNS" disabled=no new-packet-mark=iso passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mkv DOWNS" disabled=no layer7-protocol="Extension \" .mkv \"" new-connection-mark="mkv DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mkv DOWNS" disabled=no new-packet-mark=mkv passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="exe DOWNS" disabled=no layer7-protocol="Extension \" .exe \"" new-connection-mark="exe DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="exe DOWNS" disabled=no new-packet-mark=exe passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mov DOWNS" disabled=no layer7-protocol="Extension \" .mov \"" new-connection-mark="mov DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mov DOWNS" disabled=no new-packet-mark=mov passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mp3 DOWNS" disabled=no layer7-protocol="Extension \" .mp3 \"" new-connection-mark="mp3 DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mp3 DOWNS" disabled=no new-packet-mark=mp3 passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mp4 DOWNS" disabled=no layer7-protocol="Extension \" .mp4 \"" new-connection-mark="mp4 DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mp4 DOWNS" disabled=no new-packet-mark=mp4 passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mpeg DOWNS" disabled=no layer7-protocol="Extension \" .mpeg \"" new-connection-mark="mpeg DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mpeg DOWNS" disabled=no new-packet-mark=mpeg passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="mpg DOWNS" disabled=no layer7-protocol="Extension \" .mpg \"" new-connection-mark="mpg DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="mpg DOWNS" disabled=no new-packet-mark=mpg passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="nrg DOWNS" disabled=no layer7-protocol="Extension \" .nrg \"" new-connection-mark="nrg DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="nrg DOWNS" disabled=no new-packet-mark=nrg passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="pdf DOWNS" disabled=no layer7-protocol="Extension \" .pdf \"" new-connection-mark="pdf DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="pdf DOWNS" disabled=no new-packet-mark=pdf passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="ram DOWNS" disabled=no layer7-protocol="Extension \" .ram \"" new-connection-mark="ram DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="ram DOWNS" disabled=no new-packet-mark=ram passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="rar DOWNS" disabled=no layer7-protocol="Extension \" .rar \"" new-connection-mark="rar DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="rar DOWNS" disabled=no new-packet-mark=rar passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="rm DOWNS" disabled=no layer7-protocol="Extension \" .rm \"" new-connection-mark="rm DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="rm DOWNS" disabled=no new-packet-mark=rm passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="rmvb DOWNS" disabled=no layer7-protocol="Extension \" .rmvb \"" new-connection-mark="rmvb DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="rmvb DOWNS" disabled=no new-packet-mark=rmvb passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="wav DOWNS" disabled=no layer7-protocol="Extension \" .wav \"" new-connection-mark="wav DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="wav DOWNS" disabled=no new-packet-mark=wav passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="wma DOWNS" disabled=no layer7-protocol="Extension \" .wma \"" new-connection-mark="wma DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="wma DOWNS" disabled=no new-packet-mark=wma passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="wmv DOWNS" disabled=no layer7-protocol="Extension \" .wmv \"" new-connection-mark="wmv DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="wmv DOWNS" disabled=no new-packet-mark=wmv passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="zip DOWNS" disabled=no layer7-protocol="Extension \" .zip \"" new-connection-mark="zip DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="zip DOWNS" disabled=no new-packet-mark=zip passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="youtube DOWNS" disabled=no layer7-protocol="YouTube " new-connection-mark="youtube DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="youtube DOWNS" disabled=no new-packet-mark=youtube passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="daa DOWNS" disabled=no layer7-protocol="Extension \" .daa \"" new-connection-mark="daa DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="daa DOWNS" disabled=no new-packet-mark=daa passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="dat DOWNS" disabled=no layer7-protocol="Extension \" .dat \"" new-connection-mark="dat DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="dat DOWNS" disabled=no new-packet-mark=dat passthrough=no protocol=tcp


add action=mark-connection chain=prerouting comment="vcd DOWNS" disabled=no layer7-protocol="Extension \" .vcd \"" new-connection-mark="vcd DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="vcd DOWNS" disabled=no new-packet-mark=vcd passthrough=no protocol=tcp

add action=mark-connection chain=prerouting comment="cab DOWNS" disabled=no layer7-protocol="Extension \" .cab \"" new-connection-mark="cab DOWNS" passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting comment="" connection-mark="cab DOWNS" disabled=no new-packet-mark=cab passthrough=no protocol=tcp






This shape download + upload with only 1 Mb/sec to the whole Network



if you want just to shape download you can use in layer7 regexp this rule for example

/ip firewall layer7-protocol
add comment="" name="Extension \" .exe \"" regexp="^.*get.+\\.exe.*\$"


and apply it to the rest of the extensions


if you want to shape upload only use this code in layer7


/ip firewall layer7-protocol
add comment="" name="Extension \" .exe \"" regexp="^.*set.+\\.exe.*\$"


********************************************************************

and you can shape Download Traffic by 64 kb on one IP like 192.168.0.2 using simple Queue instead of Queue Tree like this


/queue simple
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="" direction=both disabled=no dst-address=0.0.0.0/0 interface=all limit-at=1M/1M max-limit=1M/1M name="download max" packet-marks=exe,flv,rar,zip,7z,bin,mp3,mp4,wav,pdf,asf,avi,mov,wmv,mpg,mpeg,rm,iso,nrg,daa,vcd,cab,3gp,ram,rmvb,mkv,wma,youtube parent=none priority=8 queue=default-small/default-small total-queue=default-small

and then make 64 Kb a speed for the Specified user in this parent group and refer to it's parent the first rule like this

add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="" direction=both disabled=no dst-address=0.0.0.0/0 interface=all limit-at=64k/64k max-limit=64k/64k name="User \" 192.168.0.2 \"" parent="download max" priority=8 queue=default-small/default-small target-addresses=192.168.0.2/32 total-queue=default-small


using this we can Shape IP 192.168.0.2 Download Traffic By 64 Kb
or what ever the speed we want to do it for the client.



the only problem this code is not working with loadbalancing ADSL Lines
is there any way to make it work ???
any one know how to do both Load Balancing + Shape download Traffic please tell us .


Notice : All code above applied on 3.3 Mikrotik OS Version
Last edited by hazemamer7 on Wed Jul 21, 2010 5:03 am, edited 3 times in total.
 
ted
just joined
Posts: 20
Joined: Sat Feb 13, 2010 9:52 am
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jun 15, 2010 2:05 pm

wonderful post, rules working 100%

thank you
 
rodolfo
Long time Member
Long time Member
Posts: 553
Joined: Sat Jul 05, 2008 11:50 am

Re: Need Help In Layer7 || isolation between Browsing , down

Sat Jun 26, 2010 11:12 pm

a question: why you cannot mangle conn and packet in forward queue ?

thanks
 
fewi
Forum Guru
Forum Guru
Posts: 7717
Joined: Tue Aug 11, 2009 3:19 am

Re: Need Help In Layer7 || isolation between Browsing , down

Sat Jun 26, 2010 11:26 pm

http://wiki.mikrotik.com/wiki/Manual:Packet_Flow

Global-in happens at the end of pre routing, so you have to mark your packets before that. Global-out you could mark in forward if you wanted to.
 
rodolfo
Long time Member
Long time Member
Posts: 553
Joined: Sat Jul 05, 2008 11:50 am

Re: Need Help In Layer7 || isolation between Browsing , down

Sun Jun 27, 2010 11:25 am

ah, ok, I can mangle forward if I use interfaces, not global-in.

thanks fewi
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Jul 21, 2010 4:10 am

Correction has been made to the final code above so any one tried it before try it just now it works very well and sorry for latency
 
heleopless
Frequent Visitor
Frequent Visitor
Posts: 78
Joined: Mon Jan 03, 2011 3:03 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Jan 05, 2011 3:11 am

very very very well
thanks my master
 
khonyot
just joined
Posts: 1
Joined: Tue Aug 24, 2010 10:31 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Jan 25, 2011 9:20 pm

TQ TQ TQ..... good ping time right now, while others loading youtube, poker, facebook and on-line games.
 
rano
just joined
Posts: 1
Joined: Tue Jul 26, 2011 10:44 am

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Aug 09, 2011 8:09 am

thank you. it works perfectly :D
 
ThunderBRZ
just joined
Posts: 13
Joined: Sun Feb 17, 2008 6:35 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Thu Sep 01, 2011 11:27 am

i need it to mark a new route for download flv videos...

works???

how to???
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Thu Sep 01, 2011 11:57 am

you should mark routing from the very first packet of the connection. at that time, you don't know, what will be inside the connection - will it be some .flv download, or Skype session. so - you cannot
 
dcrystalj
just joined
Posts: 16
Joined: Thu Mar 10, 2011 12:19 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Oct 11, 2011 4:44 pm

/ip firewall layer7-protocol

here you actually didn't mark a youtube and than in mangle you did so how it can work??

I have already qos ok working except youtube HD don't even try to take connection. youtube is loading to slow to watch simultaniously and connection is not even 10% used.
I don't use L7 protocol yet

Please help
 
cylent
Member
Member
Posts: 383
Joined: Sun May 28, 2006 10:30 am

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Apr 04, 2012 7:15 am

very nice.

now i get to catch all those .cab windows updates machines!
 
sleimanr
Member Candidate
Member Candidate
Posts: 176
Joined: Mon Oct 18, 2010 9:31 am

Re: Need Help In Layer7 || isolation between Browsing , down

Sat Aug 10, 2013 4:16 pm

hi,
i have 2 interface i want to route exe,zip etc.. to second interface?


thxx
 
sleimanr
Member Candidate
Member Candidate
Posts: 176
Joined: Mon Oct 18, 2010 9:31 am

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Aug 13, 2013 1:48 am

pleasee
 
User avatar
mahnet
Long time Member
Long time Member
Posts: 654
Joined: Tue Jul 07, 2009 9:11 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Aug 13, 2013 4:54 am

very nice.

now i get to catch all those .cab windows updates machines!
how do u catch up the machines with above codes.
We can just see the packets used in mangles, not individual machines.

ok got it. it appears in the Firewall>connections
Last edited by mahnet on Thu Aug 15, 2013 4:21 am, edited 2 times in total.
 
sleimanr
Member Candidate
Member Candidate
Posts: 176
Joined: Mon Oct 18, 2010 9:31 am

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Aug 14, 2013 3:04 am

i want to catch and send to second interfce
how send to send dsl lines?
 
sleimanr
Member Candidate
Member Candidate
Posts: 176
Joined: Mon Oct 18, 2010 9:31 am

Re: Need Help In Layer7 || isolation between Browsing , down

Fri Sep 13, 2013 1:06 am

????
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Need Help In Layer7 || isolation between Browsing , down

Wed Sep 18, 2013 1:14 am

you cannot. L7 detects already established connections. if connection is established, you cannot 'redirect' it to another line. seems like you need Squid for such things
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Tue Nov 05, 2013 7:09 am

Dear Mr Chupaka :
this line can help in detect files extensions in L7 and it needs only 1 mangle rule ,

/ip firewall layer7-protocol
add name=test regexp="^(.*)(.exe|.rar|.zip|.7z|.iso|.nrg|.daa|.dat|.vcd|.flv|.mp4|.mov|.avi|.asf|.mpg|.mpeg|.wmv|. ra|.rar|.mp3|.wav|.ram|.rmvb|.mkv|.cab|.chm|.ctf|. flg)(.*)\\\$|videoplayback"


after i tested my first code or this code some few links get bypassed from speed shaping for example :


https://dl.google.com/chrome/win/BFA757 ... taller.exe

https://ia601004.us.archive.org/16/item ... raffic.avi

can work with me for get one code that never bypass any download traffic without shaping ?
 
quake1000
just joined
Posts: 9
Joined: Sun Dec 30, 2012 8:04 pm

Re: Need Help In Layer7 || isolation between Browsing , down

Sat Dec 28, 2013 10:34 am

Hi

I need Layer 7 Regexp for Transfer Units

for example .. when end user downloading any file the Regexp change the unit from KB/s to kb/s

Thanks
You do not have the required permissions to view the files attached to this post.
 
hazemamer7
newbie
Topic Author
Posts: 28
Joined: Mon May 03, 2010 5:58 am
Location: Egypt

Re: Need Help In Layer7 || isolation between Browsing , down

Sun Jan 05, 2014 6:15 pm

Hi

I need Layer 7 Regexp for Transfer Units

for example .. when end user downloading any file the Regexp change the unit from KB/s to kb/s

Thanks
you mean that when you use my code, L7 transfer KB to kb ? or another code?
by the way take a look in this topic.
http://forum.mikrotik.com/viewtopic.php?f=9&t=42441
 
TorrentFreak
just joined
Posts: 6
Joined: Fri Feb 14, 2014 10:50 am

Re: Need Help In Layer7 || isolation between Browsing , down

Thu Mar 13, 2014 9:38 pm

Nice Work , I've tried the script and It's very helpful , but now what about the torrent leeching !!
It gets the full speed and I can't manage any p2p connection :(
Any suggestions ?!

Who is online

Users browsing this forum: No registered users and 6 guests