Page 1 of 1

How to use Transparent Web Proxy with 2 Internet Connections

Posted: Fri May 27, 2005 7:11 pm
by steve@thebiginter.net
This is the method I use to get the most out of 2 X 2mbps ADSL Connections. I'm using the built in Squid Proxy, and all proxy traffic uses both ADSL lines. All non http traffic just uses one ADSL line.
Let me tell you this is very fast Internet. I use a 1Gb RAM / Intel 3GHz CPU / 120Gb HDD just to have a big fast cache. I find the cache efficiency is 30 - 40%, so it's well worth having. I only use Winbox; here is what I did:

General Info
Router PC has 3 NIC's
1 for Local LAN IP 192.168.50.111 private address, 1 for each Internet Gateway with routable IP on each NIC.
Local LAN is NAT'd on the Gateway
Proxy running on port 8080

After adding the necessary IP Addresses to the NIC's, just add a route to one of the Gateways into the route list. This will be the catchall route.

Enable the Proxy as 'Transparent', and set the source IP to be 192.168.50.111 (the same as your LAN NIC)

In Firewall - NAT
Add a dst-nat rule to divert http requests to the Proxy:
Dst Address NOT 192.168.50.111
Protocol 6
Dst Port 80
In Interface LAN (your LAN NIC)
Action Redirect to Port 8080

There are another 2 rules needed in Firewall - NAT. These just NAT both Internet NICs onto your private LAN. They go like this:

Chain - srcnat
Out Interface - Gateway1
Action src-nat
To Addresses <IP of Gateway1 NIC>
To Ports 0-65535

Chain - srcnat
Out Interface - Gateway2
Action src-nat
To Addresses <IP of Gateway2 NIC>
To Ports 0-65535

Next, we move on to the Mangle, where we must mark traffic that has come from the Web Proxy and is destined for a Website. This mark will be called 'Proxy Traffic':

Chain - output
Src. Address - 192.168.50.111
Protocol - 6
Dst. Port - 80
Action - Mark Routing
New Routing Mark - Proxy Traffic

Now, we can move back to the Route List, and add an ECMP (dual ADSL lines in this case) Route. We can then define the traffic that will use this route as 'Proxy Traffic'.

Destination - 0.0.0.0
Gateway - <ADSL Gateway1; ADSL Gateway2>
Check Gateway - arp
Mark - Proxy Traffic

---------------------------------------------------------------------------

For interesting stats about the RouterOS Proxy, I use the new Kraken Reports. This now works with v2.9 logfiles, and tells you good stuff like
Every site a user has been on, and what time
How much www traffic each user has downloaded
Cache hit rate per user and in total.

Kraken is currently donerware, and I'm nothing to do with them btw.

hope that makes sense,

Steve Head
TBI

Posted: Tue May 31, 2005 9:15 pm
by tbutcher
When you say "very fast" I assume downloads are still limited to the max of one adsl line i.e. 2mbps. I am also interested in what amout of your traffic uses the web proxy, on our network only a small part of the traffic is HTTP.

Thanks.

Tim

Posted: Wed Jun 01, 2005 2:39 am
by steve@thebiginter.net
When you say "very fast" I assume downloads are still limited to the max of one adsl line i.e. 2mbps. I am also interested in what amout of your traffic uses the web proxy, on our network only a small part of the traffic is HTTP.

Thanks.

Tim
Yeah I mean perceptably fast web browsing; Getright will use both lines (sometimes) for downloads.
This setup is designed for schools / businesses where the most important use is Web Access. I try to divide up the remaining services with route mark to balance things up a bit.
Even with an underlying hum of p2p, the proxy helps a lot. I can't give a figure as to what amount of traffic uses the Web, as I have quite a few Mikrotiks out there, all in different scenarios.

Steve Head
TBI

Posted: Fri Jun 03, 2005 9:12 pm
by adminis
I wave the problem to setup my routher proxy to use both gateways :?:

When I put the firewall rules by your setup my proxy don't work at all :cry:
and if start working (afther changing some rules) it use only one gateway and then stop resonding afther while.

Can you help me to find out why :?:

Posted: Fri Jun 03, 2005 9:51 pm
by tbutcher
I have only been able to make it work using chain type of input not output.

Tim

Posted: Sat Jun 04, 2005 10:31 pm
by steve@thebiginter.net
Summary
Internet Gateways are at 81.5.187.49 and 82.152.139.121
Routers Public IP's are 81.5.187.53/29 and 82.152.139.122/29
Routers Private IP is 192.168.50.254/24
To complete setup, use Winbox to:
IP - Routes - On the 'ECMP Route for Web Proxy Users'
Check Gateway 'arp'
Mark: 'Proxy Traffic'





[admin@TBI] ip address> print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 ;;; TBI LAN IP Address
192.168.50.254/24 192.168.50.0 192.168.50.255 LAN
1 ;;; IP on Old Eclipse Connection (Fax Line)
81.5.187.53/29 81.5.187.48 81.5.187.55 Old_Eclipse
2 ;;; IP on New Eclipse Connection (Phone Line)
82.152.139.122/29 82.152.139.120 82.152.139.127 New_Eclipse
[admin@TBI] ip address>



[admin@TBI] ip> web-proxy print
enabled: yes
src-address: 192.168.50.254
port: 8080
hostname: "proxy"
transparent-proxy: yes
parent-proxy: 0.0.0.0:0
cache-administrator: "steve@thebiginter.net"
max-object-size: 4096KiB
cache-drive: system
max-cache-size: unlimited
max-ram-cache-size: unlimited
status: running
reserved-for-cache: 8392704KiB
reserved-for-ram-cache: 2048KiB
[admin@TBI] ip>




[admin@TBI] ip firewall> mangle print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; Mark Traffic coming from the Proxy with a Routing Mark
chain=output src-address=192.168.50.254 protocol=tcp dst-port=80
action=mark-routing new-routing-mark=Proxy Traffic
[admin@TBI] ip firewall>




[admin@TBI] ip firewall> nat print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; NAT for TBI Network to 1st Gateway (Old_Eclipse)
chain=srcnat out-interface=Old_Eclipse action=src-nat
to-addresses=81.5.187.53 to-ports=0-65535

1 ;;; NAT for TBI Network to 2nd Gateway (New_Eclipse)
chain=srcnat out-interface=New_Eclipse action=src-nat
to-addresses=82.152.139.122 to-ports=0-65535

2 ;;; Redirects all http requests to the Proxy on Port 8080
chain=dstnat in-interface=LAN dst-address=!192.168.50.254 protocol=tcp
dst-port=80 action=redirect to-ports=8080





[admin@TBI] ip> route print
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf
# DST-ADDRESS G GATEWAY DISTANCE INTERFACE
0 ADC 81.5.187.48/29 Old_Eclipse
1 ADC 82.152.139.120/29 New_Eclipse
2 ADC 192.168.50.0/24 LAN
3 A S ;;; Catchall Route to Old_Eclipse
0.0.0.0/0 r 81.5.187.49 Old_Eclipse
4 A S ;;; ECMP Route for Web Proxy Users
0.0.0.0/0 r 81.5.187.49 Old_Eclipse
r 82.152.139.121 New_Eclipse
[admin@TBI] ip>

Posted: Tue Jun 28, 2005 10:28 pm
by tbutcher
What version of Mikrotik are you using? As soon as I add a source address my proxy stops working.

Tim

Posted: Mon Aug 01, 2005 9:37 pm
by eugenevdm
I have tried this set up and I have to say it's the fastest Internet I have ever seen. It works really well. I have some questions:

1. In layman's terms, why does one have to use proxy-arp with this setup?

2. It looks like downloads break. Sometimes when there is not so much traffic on the lines it's fine, but during peak hours downloads break. I haven't been able to isolate or reproduce the problem, all I know is quite a few of my customers (myself including) are having this problem. I suppose it's because a connection gets broken in between proxy requests. Does anyone have other ideas?

3. Does anyone know the approximate way ECMP chooses it's routes?

Posted: Sun Aug 28, 2005 6:23 pm
by kjagus
2. It looks like downloads break. Sometimes when there is not so much traffic on the lines it's fine, but during peak hours downloads break. I haven't been able to isolate or reproduce the problem, all I know is quite a few of my customers (myself including) are having this problem. I suppose it's because a connection gets broken in between proxy requests. Does anyone have other ideas?
ECMP is not connection-orientated - so it is useless in Mikrotik. Using ECMP will always give you broken downloads or disconnected chats.

Posted: Sun Aug 28, 2005 6:45 pm
by changeip
3. Does anyone know the approximate way ECMP chooses it's routes?
Yes, equally. (E)qual (C)ost ... If you want anything else you need to use policy routing.

Sam