Dual gateways, wrong NAT, and impossible routes?

changeip · Mon May 31, 2004 8:23 pm

Hello,

We've had some problems at our data center in the past with dual gateways and Mikrotik - so we went ahead and setup a test environment to figure this out.

We've setup two cable modems. One is using a static IP, the other is a dynamic IP. I can setup both as Static for the time being, but same result. In Mikrotik we have specified both IP addresses on their respective networks. We have also setup a masq rule for each network to tell the outgoing packets to use the interfaces IP it goes out on.

The problem is the routing - if we try to add routes to both gateways using 0.0.0.0/0 as the destination one route always ends up getting flagged as INVALID. If we add both gateways to the default route then we end up with NAT not working - it sends packets out both cable modems with the source IP as only the first one.

I think this is a bug in mikrotik ... does anyone else have dual gateways working with each being on their own interface?

Any help would be appreciated... if I don't get this working I might have to setup 2 mikrotik routers, one for each gateway, which I dont want to do.

Here is a screenshot of the config showing that one gateway goes invalid:

andrewluck · Mon May 31, 2004 11:07 pm

As far as I can see, your problems are being caused by trying to have 2 default routes. Only one can be active at any one time. Consider what happens mid tcp connection; a packet hits the router, but the router has no way of knowing which route is being used for this connection. If it chooses the wrong one then the connection will break.

Policy based routing can be used to fix this. It chooses a route based upon the type of traffix e.g. all your SMTP traffic can use one route whilst HTTP uses the other.

Regards

Andrew

changeip · Mon May 31, 2004 11:30 pm

Mikrotik supports ECMP, which means it will load-balance the gateways. I should be able to use 2 or more gateways and have it work ... it just seems the NAT gets screwed up because it doesn't take the preferred source correctly - since you can only specify a single preferred source for multiple listings.

From the manual on:
http://www.mikrotik.com/docs2.8/ip/route.main

More than one gateway for one destination network may be used. This approach is called 'Equal-Cost Multi-Path Routing' and is used for load balancing (Note that this does not provide failover). With ECMP, a router potentially has several available next hops towards any given destination. A new gateway is chosen for each new source/destination IP pair. This means that, for example, one FTP connection will use only one link, but new connection to a different server will use other link. This also means that routes to often-used sites will always be over the same provider. But on big backbones this should distribute traffic fine. Also this has another good feature - single connection packets do not get reordered and therefore do not kill TCP performance.

Equal cost multipath routes can be created by routing protocols (RIP or OSPF), or adding a static route with multiple gateways (in the form gateway=x.x.x.x,y.y.y.y) The routing protocols may create routes with equal cost automatically, if the cost of the interfaces is adjusted properly. For more information on using the routing protocols, please read the corresponding section of the Manual.

Note! In routing process, the router decides which route it will use to send out the packet. Afterwards, when the packet is masqueraded, its source address is taken from the preferred-source field.

This should work - even if I list 2 default gateways with different costs, or a single gateway entry with 2 entries ... Anyone else got this working right while using masquerading?

I don't want to use policy routing - I want to use the bandwidth from each pipe for all traffic... which the manual states is possible. Each IP source/dest pair gets the alternate link, so no tcp reordering problems happen.

rz8168 · Tue Jun 01, 2004 2:28 pm

I just started a thread questioning how to setup load balancing in mktk. But as far as I know, gateway should be in x.x.x.x,y.y.y.y form but that doesn't seem to work as I tried. I do get traffic balanced out in 1 out of 500 balancing tests. Most the time, traffic go through one interface instead of both. I wonder how to set it up correctly.

andrewluck · Tue Jun 01, 2004 9:59 pm

I've only looked at equal cost routing with RIP & OSPF, but, as you point out, it should work with static routes on these routers.

The following procedure will setup a route with 2 gateways:

To set the 192.168.0.0/16 network is reachable via both 10.10.10.2 and 10.10.10.254 gateways:
[admin@MikroTik] ip route> set 0 gateway=10.10.10.2,10.10.10.254
[admin@MikroTik] ip route> print
Flags: X - disabled, I - invalid, D - dynamic, J - rejected,
C - connect, S - static, r - rip, o - ospf, b - bgp
# DST-ADDRESS G GATEWAY DISTANCE INTERFACE
0 S 192.168.0.0/16 r 10.10.10.2 1 Local
r 10.10.10.254 Local
1 S 0.0.0.0/0 r 10.10.10.1 1 Public
2 DC 10.10.10.0/24 r 0.0.0.0 0 Public
[admin@MikroTik] ip route>

Unfortunately, I can't test this on my current network setup.

Regards

Andrew[/i]

changeip · Tue Jun 01, 2004 10:06 pm

Yes, this looks right... problem is that I cannot set the preferred source for each route, only the entire entry. I believe that masquerading using the preferred source field, and if blank, is supposed to choose the right IP from the outgoing interface, but I think its not ...

Any comment from Mikrotik?

Sam

rz8168 · Wed Jun 02, 2004 12:52 pm

shouldn't anyone from mikrotik give us a hand on this issue?

Wed Jun 02, 2004 1:06 pm

Mikrotik supports ECMP, which means it will load-balance the gateways. I should be able to use 2 or more gateways and have it work ...

Equal cost multipath routes can be created by routing protocols (RIP or OSPF), or adding a static route with multiple gateways (in the form gateway=x.x.x.x,y.y.y.y) The routing protocols may create routes with equal cost automatically, if the cost of the interfaces is adjusted properly. For more information on using the routing protocols, please read the corresponding section of the Manual.

what you need to do is to remove both your static routes and add one like this:

ip route> add gateway=68.7.20.1,68.15.30.129

if you like one of them to have priority over the other, repeat that address twice in the above line.

if you want to have a specific source address to go through a specific gateway, use policy routing. there you will create a rule that says if the source address is this, use this routing table etc.