L7 RegExp for .mp4 and .flv file types

1littlewisp · Tue Nov 01, 2011 8:36 pm

Looking to identify video content coming from youtube. I've already got the content filter sending all traffic containing "youtube.com" to the Youtube chain. Within the Youtube chain I want to pick out only .flv and .mp4 content. To the best of my understanding, the right way to do this is to use L7 filters. I do not know regular expressions. Does anyone have regexp strings that I could use to achieve my goal?

sadeghrafie · Tue Nov 01, 2011 9:02 pm

I checked http://l7-filter.sourceforge.net/protocols
but it doesn't have any regexp for mp4 and flv. But it have some useful regexp and you may need them in future.

1littlewisp · Tue Nov 01, 2011 9:33 pm

I checked http://l7-filter.sourceforge.net/protocols
but it doesn't have any regexp for mp4 and flv. But it have some useful regexp and you may need them in future.

Yeah, I've already searched the forum and found that most L7 threads inevitably link to that site. Looked there first.

Muqatil · Wed Nov 02, 2011 12:54 am

/ip firewall layer7-protocol
add name="Extension \" .mp4 \"" regexp="\\.(mp4)"
add name="Extension \" .flv \"" regexp="\\.(flv)"

sadeghrafie · Wed Nov 02, 2011 8:46 am

Are these really work?
I should go and try

1littlewisp · Wed Nov 02, 2011 3:50 pm

/ip firewall layer7-protocol
add name="Extension \" .mp4 \"" regexp="\\.(mp4)"
add name="Extension \" .flv \"" regexp="\\.(flv)"

Thanks, Muqatil. I'll give those a try and post the results. It looks like this format would work for pretty much any sort of file extension, right?

Muqatil · Thu Nov 03, 2011 12:16 am

yes it does. And it works for me

1littlewisp · Fri Dec 02, 2011 11:04 pm

Okay, so I think I've got the L7 filters put together right but the counter for the "packet-mark" rules is not getting ticked when I hit youtube and start a video. Here's how I'm identifying the traffic:

/ip firewall mangle
add action=jump chain=prerouting content=youtube.com disabled=no jump-target=\
    youtube
add action=add-dst-to-address-list address-list=Youtube address-list-timeout=\
    5m chain=youtube comment=\
    "All packets processed in this chain should be added to Youtube list." \
    disabled=no dst-port=80 protocol=tcp
add action=mark-connection chain=youtube disabled=no dst-address-list=Youtube \
    new-connection-mark=youtube_conn passthrough=yes
add action=mark-packet chain=output connection-mark=youtube_conn disabled=no \
    layer7-protocol="Extension \".mp4 \"" new-packet-mark=youtube_mp4 \
    passthrough=yes
add action=mark-packet chain=output connection-mark=youtube_conn disabled=no \
    layer7-protocol="Extension \".mp4 \"" new-packet-mark=youtube_flv \
    passthrough=yes

And here are the filters:

/ip firewall layer7-protocol
add name="Extension \".mp4 \"" regexp="\\.(mp4)"
add name="Extension \".flv \"" regexp="\\.(flv)"

Did I typo or am I just conceptualizing this wrong?

Muqatil · Sat Dec 03, 2011 2:26 am

Why did you set chain=output on your packet marker? That chain checks only the packets generated by the router itself. if i did not misunderstood your conf, change it to forward chain

1littlewisp · Sat Dec 03, 2011 6:04 pm

/ip firewall mangle
add action=jump chain=prerouting content=youtube.com disabled=no jump-target=\
    youtube
add action=add-dst-to-address-list address-list=Youtube address-list-timeout=\
    5m chain=youtube comment=\
    "All packets processed in this chain should be added to Youtube list." \
    disabled=no dst-port=80 protocol=tcp
add action=mark-connection chain=youtube disabled=no dst-address-list=Youtube \
    new-connection-mark=youtube_conn passthrough=yes
add action=mark-packet chain=forward connection-mark=youtube_conn disabled=no \
    layer7-protocol="Extension \".mp4 \"" new-packet-mark=youtube_mp4 \
    passthrough=yes src-address-list=Youtube
add action=mark-packet chain=forward connection-mark=youtube_conn disabled=no \
    layer7-protocol="Extension \".mp4 \"" new-packet-mark=youtube_flv \
    passthrough=yes src-address-list=Youtube

Okay, I corrected a couple of things here. First of all, I wasn't specifying the address list I wanted it to pull from. Second, I did have it in the output chain. I'm running a proxy on this device. My logic was that since it was a proxy redirect, it would be seen as coming from the router, but you're right. It should be in the forward chain.

The idea here is to have packets containing the "youtube.com" string get kicked to the "Youtube" chain. There, the IPs will be added to an address list. IPs on the address list get marked with the "youtube_conn" mark and sent through the L7 filter. The "packet-mark" rules still aren't getting hit.

1littlewisp · Sat Dec 03, 2011 6:23 pm

BTW, if you know of a more effective way to accomplish what I'm attempting here, I would not be opposed to a complete redesign. All I want is to slow *just* video traffic from youtube.

Muqatil · Sun Dec 04, 2011 12:15 am

Okay, I corrected a couple of things here. First of all, I wasn't specifying the address list I wanted it to pull from. Second, I did have it in the output chain. I'm running a proxy on this device. My logic was that since it was a proxy redirect, it would be seen as coming from the router, but you're right. It should be in the forward chain.

The idea here is to have packets containing the "youtube.com" string get kicked to the "Youtube" chain. There, the IPs will be added to an address list. IPs on the address list get marked with the "youtube_conn" mark and sent through the L7 filter. The "packet-mark" rules still aren't getting hit.

Actually i didn't know about the proxy service. Forward Chain would not be used by your traffic. Your logic was correct

Can you use prerouting chain or is it used for other purposes?

1littlewisp · Sun Dec 04, 2011 7:40 pm

Okay, I corrected a couple of things here. First of all, I wasn't specifying the address list I wanted it to pull from. Second, I did have it in the output chain. I'm running a proxy on this device. My logic was that since it was a proxy redirect, it would be seen as coming from the router, but you're right. It should be in the forward chain.

The idea here is to have packets containing the "youtube.com" string get kicked to the "Youtube" chain. There, the IPs will be added to an address list. IPs on the address list get marked with the "youtube_conn" mark and sent through the L7 filter. The "packet-mark" rules still aren't getting hit.
Actually i didn't know about the proxy service. Forward Chain would not be used by your traffic. Your logic was correct

Can you use prerouting chain or is it used for other purposes?

What did you have in mind for the prerouting chain? Can you give me an example?

L7 RegExp for .mp4 and .flv file types

L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types

Re: L7 RegExp for .mp4 and .flv file types