Content filtering with web proxy - adult sites
Posted: Fri Sep 24, 2010 5:43 pm
Steps to setup Mikrotik Router for content filtering. I should mention this is on a quad core pc with 4GB of Ram and 500GB of hard drive space running routeros v4.11 x86 Level 6 license as a central router.
I did many searches on this topic and found no clear example of how to do this except people suggesting using squid proxy or other hardware. This still a work in progess, and I welcome thoughts and opinions on how to improve it.
Problem : Have client that want access to porn and some that want it filtered.
Step 1.) Setup router to let everyone on the internet, including all QoS, firewall filtering, and Userman Test as in other tutorials.
Step 2.) In Userman Test, under profiles, limitations, under constraints set address list to "kid_friendly" on the profile used by client wanting content filtered. You may have to setup two different profiles and limitations. One profile and limitation with rate limits and address list set to something like "normal" and another profile and limitation with the address llist set to "kid_friendly". Userman Test is very picky and all profiles might have to be removed, reboot router, then added back for the list to populate correctly. Address list associations are somehow cached and do not take affect until router is rebooted.
Step 3.) Add these rules to the firewall filter:
;;; block adult www
chain=forward action=add-dst-to-address-list dst-address=!10.0.0.0/8 address-list=adult_block
address-list-timeout=0s layer7-protocol=kid_friendly-www
;;; block adult www
chain=forward action=drop dst-address=!10.0.0.0/8 src-address-list=kid_friendly layer7-protocol=kid_friendly-www
Step 4.)Now add this rule to the firewall NAT:
;;; Kid Friendly Web Proxy - BAD Sites
chain=dstnat action=redirect to-ports=8080 protocol=tcp src-address-list=kid_friendly dst-address-list=adult_block
dst-port=80
Step 5.) Now add words to the Layer 7 - Protocol to catch and filter:
4 kid_friendly - WWW ^.*(get|GET).+(choice|words|to|filter).*$
Step 6.) Enable and configure web proxy as suggested in other tutorials. Then add under access add these two rules.
;;; Not for kids URL (Enter word with vertical bar)
Dst. Host : (choice|words|to|filter) deny
;;; Block All
Dst. Host : deny
This may be the hard way of doing it but so far it is working. There are some false positives, in which the dst host address can manually be added and placed before the web proxy rules denying access. In this setup, only clients wanting to be filtered will be filtered, and then only hit the web proxy if a webpage is requested from the adult_blocked list. It also allows other clients that does not wish to be filtered to help populate the block list. I hope some of the experts will chime in with some advice. Hope this helps someone else.
I did many searches on this topic and found no clear example of how to do this except people suggesting using squid proxy or other hardware. This still a work in progess, and I welcome thoughts and opinions on how to improve it.
Problem : Have client that want access to porn and some that want it filtered.
Step 1.) Setup router to let everyone on the internet, including all QoS, firewall filtering, and Userman Test as in other tutorials.
Step 2.) In Userman Test, under profiles, limitations, under constraints set address list to "kid_friendly" on the profile used by client wanting content filtered. You may have to setup two different profiles and limitations. One profile and limitation with rate limits and address list set to something like "normal" and another profile and limitation with the address llist set to "kid_friendly". Userman Test is very picky and all profiles might have to be removed, reboot router, then added back for the list to populate correctly. Address list associations are somehow cached and do not take affect until router is rebooted.
Step 3.) Add these rules to the firewall filter:
;;; block adult www
chain=forward action=add-dst-to-address-list dst-address=!10.0.0.0/8 address-list=adult_block
address-list-timeout=0s layer7-protocol=kid_friendly-www
;;; block adult www
chain=forward action=drop dst-address=!10.0.0.0/8 src-address-list=kid_friendly layer7-protocol=kid_friendly-www
Step 4.)Now add this rule to the firewall NAT:
;;; Kid Friendly Web Proxy - BAD Sites
chain=dstnat action=redirect to-ports=8080 protocol=tcp src-address-list=kid_friendly dst-address-list=adult_block
dst-port=80
Step 5.) Now add words to the Layer 7 - Protocol to catch and filter:
4 kid_friendly - WWW ^.*(get|GET).+(choice|words|to|filter).*$
Step 6.) Enable and configure web proxy as suggested in other tutorials. Then add under access add these two rules.
;;; Not for kids URL (Enter word with vertical bar)
Dst. Host : (choice|words|to|filter) deny
;;; Block All
Dst. Host : deny
This may be the hard way of doing it but so far it is working. There are some false positives, in which the dst host address can manually be added and placed before the web proxy rules denying access. In this setup, only clients wanting to be filtered will be filtered, and then only hit the web proxy if a webpage is requested from the adult_blocked list. It also allows other clients that does not wish to be filtered to help populate the block list. I hope some of the experts will chime in with some advice. Hope this helps someone else.