Hi everyone,
i have a small network- at peak with 25 clients and i want to use web proxy but it seems from the tutorial that I've come acrossthe authors were working on a routerOs installed on a PC and not a routerboard.

If that is the case, how do i connect an external PC that will be caching content to a RouterBoard since an RB has small drive space and besides because of constant read/write need of Cache doesn’t make an RB a good candidate. my RB is 951-2n

Please I wouldn’t like to use squid though just the built-in web proxy from mikrotik.

thanks

http://forum.mikrotik.com/viewtopic.php?f=2&t=70400


*** just try to follow the rules & don't copy-paste.


then give me feedback what do you want actually & send me your configuration



best regards

thanks for you input but am sorry to say that as a newbie to mikrotik, I couldn't make much sense out of all those rules.

besides I couldn't figure out at what point the thread showed that
- an external drive was been connected and
- cache been enabled and working

i was only able to read the "web Proxy' section of the rule.

So if you could come down to my basic level and hold my hand I will really appreciate it.

thanks

ok.

I think your mikrotik product RB951-2n is not support any Memory card or Usb stick.
So you have to need an external PC that will be caching content. Otherwise you can't do it.
Now you will make a Squid server or External Mikrotik Web proxy server for giving your client this type of service. I will suggest you how to built a web proxy server. just tell me what do you want ?



Best regards

I would rather like to go your suggestion and build an external mikrotik web server for caching instead of squid cache.


thanks and I will be counting on your guidance for that.

regards, max

if you want to build an external Mikrotik web server for caching,
so you have to need a pc which have P3 or P4 Processor, 2GB Ram, 2 hard disk
(one is for operating system & 2nd is for data caching), 2 Lan Card & Thermal casing.
then you will make a good web-proxy server.


best regards

Same HW config would result in a more capable squid-proxy-cache. As there is the opportunity to cache videos, for example. Or to improove disk-IO using aufs.

i have Mikrotik system in pc when i try web proxy it's working fast but after some day's come slow i disable web proxy internet back speed

i need to know web proxy in mikrotik bad or i have this problem only or need to make web proxy cache in another pc external and what configuration
i think the master pc have (2 LAN - 1WAN )
the second pc have 2 LAN

i'm right or wrong ??

@ ba7abak
Actually Mikrotik Web-proxy is a Basic proxy-server. Not for customization proxy service.
So Master pc have (1 LAN - 2 WAN )
the second pc have 2 LAN (if it's Mikrotik)
Otherwise second pc have to need (1 Lan - 1 WAN) [for Squid server]

@ reinerotto
New user can not properly setup & maintaince squid-proxy-cache. But you can easy install Mikrotik Web-proxy in a single pc with load balancing & failover. Actually here is topic web proxy for small network & minimum user.

best regards

@dotnet:
>New user can not properly setup & maintaince squid-proxy-cache.<
Users, new to Mikrotik, also can not properly set up a cache on MT

But, in principle you are correct, that setting up mikrotik proxy is simpler compared to squid.
So for very small networks, MT proxy/cache can be good enough. In case of increasing no of users, with higher throughput demands, squid is definitely the better choice. Or, in case you need to save as much traffic as possible, when having an expensive or low-bandwidth WAN (satellite, mobile, ISDN etc.). Because then also you can add further services very easily, like content filtering or ad removal. Or caching dynamic content, like youtube. Impossible to be done with MTs cache.

@ reinerotto

Thanks. i know what is Squid/LUSCA (Nginx,Ruby,storeurl.pl)
if anyone want to advise from me, i will try to give him suggestion.


best regards

Same HW config would result in a more capable squid-proxy-cache. As there is the opportunity to cache videos, for example. Or to improove disk-IO using aufs.

I have put countless number of hours in just getting ip-tables configuration to stay permanent after reboot of ubuntu yet all the online guide i've followed seems not to get it work. that's why i don' want to go through the squid 'hussle'

but that was my first thought though

thanks.

I have put countless number of hours in just getting ip-tables configuration to stay permanent after reboot of ubuntu

I am wondering, why you had to fiddle around with iptables etc. I connected a MT-box to a squid-PC simply by defining the squid-PC to be an upstream (parent) proxy to the MT-proxy. This will force all http-traffic thru squid. squid-PC needs to be a router, too, of course.
It will be a hirarchical config then, MT to be the router (default gateway) for the LAN, not directly connected to WAN.
Then squid.conf can be more or less "default", besides size of cache_dir.
Only usage of iptables I can image is for the protection of the ports of the squid-PC, to inhibit inbound connections from the WAN, besides port 22, basically.
So

LAN-with-clients-------MT-------------------squid-pc--------------------WAN
proxy (no cache) parent-proxy (cache)

Regarding special squid.conf: The forum to meet the "squid specialists" (like me on
http://squid-web-proxy-cache.1019090.n4.nabble.com/
There you also get info from the developers, if necessary.

The path to follow:
Set up MT with http-proxy (without cache) for your clients first.
Setup squid-PC (without active squid) as a router.
Disconnect MT from WAN, connect MT to squid-PC, connect squid-PC to WAN
Then configure squid, incl. caching, modify MT-proxy to use squid-PC as upstream.

regarding iptables on ubuntu: As a "brutal hack" you can edit something like this into crontab:
@reboot /etc/iptables/firewall_up.sh

/etc/iptables/firewall_up.sh:
#!/bin/bash
#started after boot, so we have to wait a bit for the systems upstart to be completed
sleep 180
iptables .....

Coming from Open-SuSE originally, I also used this, until I figured out the "elegant" method. I know your pain

if you want to build an external Mikrotik web server for caching,
so you have to need a pc which have P3 or P4 Processor, 2GB Ram, 2 hard disk
(one is for operating system & 2nd is for data caching), 2 Lan Card & Thermal casing.
then you will make a good web-proxy server.


best regards

Please i have the hardware ready now. can you guide me to what to do next?

thanks,
max

@max

please install the Mikrotik Software in your pc. then you will active Web-proxy & activate your secondary hard disk for data caching.

Then what's your method for your client pc's ? I will solve your problem with your method or policy.
please show me your previous Router configuration.



best regards

/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=192.168.2.1/24 network=192.168.2.0 broadcast=192.168.2.255 interface=Wan (Connected with Mikrotik & Proxy-server by Cross cable which is your Wan)

/ip route
add dst-address=0.0.0.0/0 gateway=192.168.2.1 distance=1 check-gateway=ping

/ip firewall nat
add action=redirect chain=dstnat disabled=no dst-port=80 in-interface=Local protocol=tcp to-ports=3128
add action=masquerade chain=srcnat disabled=no out-interface=Wan

/ip dns
set allow-remote-requests=yes cache-max-ttl=1w cache-size=25000KiB max-udp-packet-size=512 servers=8.8.8.8, 8.8.4.4

/ip proxy
enabled: yes
src-address: 0.0.0.0
port: 3128
parent-proxy: 0.0.0.0
parent-proxy-port: 0
cache-administrator: Dotnet
max-cache-size: unlimited
cache-on-disk: yes
max-client-connections: 5000
max-server-connections: 5000
max-fresh-time: 3d
serialize-connections: no
always-from-cache: no
cache-hit-dscp: 4
cache-drive: primary-slave


best regards

please tell me the effect of "always from cache"

@ karina

Don't use "always from cache" for Mikrotik Web-proxy.
This is one kind of force command.
So it will be missing sometimes & user will be get an error page (By default).
Bcoz Mikrotik Web-proxy is a Basic Web-proxy.


Best regards

Thanks Dotnet, karma awarded for being helpfull

@ max

what about your Mikrotik-Proxy ?

@ max

what about your Mikrotik-Proxy ?

Hello pal,

i've set it up and it seems to be working. but over all, I seem to have gained jx a marginal difference.

But am still monitoring and researching on some petty issues but if I get stuck, I will ask for your help.

But one more thing, how do i prevent a specific site from being cached.

I want to prevent facebook.com from been cached and don't know the rule to apply.
Your help will be appreciated.

thanks

@ max

what about your Mikrotik-Proxy ?

hello Boss,

what i've noticed is that caching only works when am caching in RAM but fails to work when i select the "Cache on disk" option.

What could be my problem?
I had to drives both SATA and under System-->Stores --> Disks. they both show as SATA1 and SATA2 with their respective sizes intact yet whether i choose to cache on SATA1 or SATA2, it won't work unless I deselect Cache on disk then it start s caching on RAM.

help needed here please,
max

@max

You can block file download by Mikrotik Proxy. Example
/ip proxy access add path=*.avi action=deny
/ip proxy access add path=*.flv action=deny
/ip proxy access add path=*.mkv action=deny

You can block any web site via domain name. Example:
/ip proxy access add action=deny disabled=no dst-host=facebook.com
/ip proxy access add action=deny disabled=no dst-host=www.facebook.com

you may stop using facebook. but don't stop caching any web site. bcoz it's a basic web-proxy.
not for more customization.


You must 1st select your storage drive properly or activate your storage drive then select "cache on disk"

These images below show Web Proxy Enabled, Web Proxy copied to and activated on SATA2 yet the status show that nothing is being cached.

those few activities on Hits and "Hits sent to client" was those related to deselecting "Cache on disk"


looking forward to your assistance

thanks

Have you got your Dst Nat rule set correctly for http traffic?

Have you got your Dst Nat rule set correctly for http traffic?

thanks for your input Karina,

I've taken screen shot of my dst Nat rule for you to review if I'm missing something.

Every help will be appreciated. I'm kinda getting desperate now since intense online search seems to yield no result.

thanks again,

@max

what's your method for internet user ?
pppoe or ip base ???
have you any load balancing ?
pls export your configuration ...

best regards

Only thing i would add is the src address range of the subnet you want to proxy, I dont think this would cause your issue though

@max

what's your method for internet user ?
pppoe or ip base ???
have you any load balancing ?
pls export your configuration ...

best regards

hotspot and firewall configuration below

/ip hotspot export

/ip firewall export
thanks,
max

@max

i don't know what's your actual hotspot configuration. Here is a Example:

/ip address
add address=192.168.0.1/24 comment=Out to users disabled=no interface=LAN network=192.168.0.0
add address=192.168.1.6/24 comment=INTERNET disabled=no interface=WAN network=192.168.1.0

/ip pool
add name=hs-pool-1 ranges=192.168.0.10-192.168.0.255

/ip dns
set allow-remote-requests=yes cache-max-ttl=1w cache-size=10000KiB max-udp-packet-size=512 servers=8.8.8.8,8.8.4.4

/ip dhcp-server
add address-pool=hs-pool-1 authoritative=after-2sec-delay bootp-support=static disabled=no interface=LAN lease-time=1h name=dhcp1

/ip dhcp-server config set store-leases-disk=5m

/ip dhcp-server network add address=192.168.0.0/24 comment="hotspot network" gateway=192.168.0.1

/ip hotspot profile
set default dns-name="" hotspot-address=0.0.0.0 html-directory=hotspot http-cookie-lifetime=3d http-proxy=0.0.0.0:0 login-by=http-chap name=default rate-limit="" smtp-server=0.0.0.0 split-user-domain=no use-radius=no

add dns-name=login.dotnet.com hotspot-address=192.168.0.1 html-directory=hotspot http-cookie-lifetime=1d http-proxy=0.0.0.0:0 login-by=http-chap name=hsprof1 rate-limit="" smtp-server=0.0.0.0 split-user-domain=no use-radius=no

/ip hotspot
add address-pool=hs-pool-1 addresses-per-mac=1 disabled=no idle-timeout=15m interface=LAN keepalive-timeout=none name=hotspot1 profile=hsprof1

/ip hotspot user profile
set default idle-timeout=15m keepalive-timeout=2m name=default shared-users=1 status-autorefresh=1m transparent-proxy=no

add address-pool=hs-pool-1 advertise=no idle-timeout=none keepalive-timeout=2m name="512k Limit" open-status-page=always rate-limit=512k/512k shared-users=1 status-autorefresh=1m transparent-proxy=yes

add address-pool=hs-pool-1 advertise=no idle-timeout=none keepalive-timeout=2m name="256k Limit" open-status-page=always rate-limit=256k/256k shared-users=1 status-autorefresh=1m transparent-proxy=yes

/ip hotspot service-port set ftp disabled=yes ports=21

/ip hotspot walled-garden ip add action=accept disabled=no dst-address=192.168.0.1

/ip hotspot set numbers=hotspot1 address-pool=none

/ip firewall nat add action=masquerade chain=srcnat disabled=no

/ip hotspot user
add disabled=no name=admin password=123 profile=default
add disabled=no name=dotnet password=1234 profile="512k Limit" server=hotspot1
add disabled=no name=dotnet-256k password=1234 profile="256k Limit" server=hotspot1

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 scope=30 target-scope=10


***The Hotspot itself is already a proxy, so you're looping to yourself. To bypass the automatic proxy for authenticated users, insert the following NAT rule:
Code:
/ip nat firewall
add chain=pre-hotspot dst-address=!local hotspot=auth action=accept

That has other side effects. If it does not work well for you you can try rewriting your proxy rules to work in the 'output' rather than the 'forward' chain, but that will also have side effects.

Overall the cleanest solution would be to use a third party proxy.

Yes you can, but you need to force guests to use the proxy after they sign in. This can be done with a simple NAT rule or you can check to enable "use transparent proxy" in the user profile. The transparent proxy only works for HTTP, not HTTPS.
http://wiki.mikrotik.com/wiki/Manual:IP/Proxy

With the NAT rule it looks something like this and needs to come before the hotspot rules in the firewall, you can also put it on the pre-hotspot chain:
Code:
/ip firewall nat
add chain=dst-nat action=redirect to-port=8080 dst-port=80 protocol=tcp hotspot=auth src-address=192.168.1.0/24

If you want to do this for only certain profiles then you need to use it at the profile level, or use a dynamic address list that a guest is going to be added to upon signing in, another option in the user profiles, or done with a Radius attribute.

*** Acutally Hotspot with web-proxy is a complicated, i think it's need to better separate box from Hotspot. Then Mikrotik web proxy will give you good result. You must be delete your old web-proxy setting for storage, and make a new web-proxy setting.


best regards

all this while i had sent a message to mikrotik support now they say i should upgrade to V6 but when i go to System --> Packages and check for updates it doesn't update it tells me my system is up to date.

I've downloaded the v6 both the update package aldd "All package" can you please tell me how i can update manually via winbox to v6 please? i don't want to do anything that willl ruin my license

thanks

How did you try to upgrade? I was running 5.25, I went to system - packages, hit the "check for updates" button, hit the upgrade button and within a min or 2 I was back up with V6, everything working fine except the web proxy had disabled itself. probebly one of the smoothest upgrades I have ever done

This is probably a really stupid question but does your router you are trying to upgrade have access to the internet? I am thinking there is something very basic wrong here, hence the reason its been overlooked as is so often the case

@max

open your winbox then drag & drop all package of 6.0 in you winbox file menu.
then go to system >> upgrade >> upgrade package source >> click + sign >>
router ip address >> admin >> password (router password) >> Apply.
then you will reboot the system. i hope your system will upgrade automatically.


best regards

@max

open your winbox then drag & drop all package of 6.0 in you winbox file menu.
then go to system >> upgrade >> upgrade package source >> click + sign >>
router ip address >> admin >> password (router password) >> Apply.
then you will reboot the system. i hope your system will upgrade automatically.


best regards

unluckily for me, this didn't work

@max

it's a basic upgrade or upload system for Mikrotik Router.
you may follow the link for reference:
http://wiki.mikrotik.com/wiki/Manual:Upgrading_RouterOS


best regards

@max

open your winbox then drag & drop all package of 6.0 in you winbox file menu.
then go to system >> upgrade >> upgrade package source >> click + sign >>
router ip address >> admin >> password (router password) >> Apply.
then you will reboot the system. i hope your system will upgrade automatically.


best regards

unluckily for me, this didn't work

The best way to upgrade manually is just to add all the .npk packages you downloaded from all packages, click on file, then drag them to that place of thru webfig, click on file, then browse to add or upload all the packages, then login thru winbox, and ru this command, /system reboot, then y for yes, the stuff must upgrade. NB: make sure u are upgrading with the correct device or architecture (pc is different from rb-7xx, 9xx etc).

@max

i don't know what's your actual hotspot configuration. Here is a Example:


***The Hotspot itself is already a proxy, so you're looping to yourself. To bypass the automatic proxy for authenticated users, insert the following NAT rule:
Code:
/ip nat firewall
add chain=pre-hotspot dst-address=!local hotspot=auth action=accept

That has other side effects. If it does not work well for you you can try rewriting your proxy rules to work in the 'output' rather than the 'forward' chain, but that will also have side effects.
Can you explain or write the script or code for this cus i want to try and implement web proxy and hotspot on the same system but with external usb drive of about 2gb storage space.
Overall the cleanest solution would be to use a third party proxy.

Yes you can, but you need to force guests to use the proxy after they sign in. This can be done with a simple NAT rule or you can check to enable "use transparent proxy" in the user profile. The transparent proxy only works for HTTP, not HTTPS.
http://wiki.mikrotik.com/wiki/Manual:IP/Proxy

With the NAT rule it looks something like this and needs to come before the hotspot rules in the firewall, you can also put it on the pre-hotspot chain:
Code:
/ip firewall nat
add chain=dst-nat action=redirect to-port=8080 dst-port=80 protocol=tcp hotspot=auth src-address=192.168.1.0/24

Plz elaborate on the above let me understand it very well.

If you want to do this for only certain profiles then you need to use it at the profile level, or use a dynamic address list that a guest is going to be added to upon signing in, another option in the user profiles, or done with a Radius attribute.

*** Acutally Hotspot with web-proxy is a complicated, i think it's need to better separate box from Hotspot. Then Mikrotik web proxy will give you good result. You must be delete your old web-proxy setting for storage, and make a new web-proxy setting.


best regards

I will like to know the best way to achieve this setup of hotspot and web proxy on the same system or just using rb951