Hi,

I have read lots of posts without really getting the full picture re. bonding of links between two locations.

I have a datacenter in Europe with a high speed Internet connection coming in to a Mikrotik box.
A branch office in Asia can not get a high speed connection, but is instead using two ADSL lines (2/1 Mbps) and load balance over these when accessing Internet. They are not using a Mikrotik today.
This has worked fine until now for normal traffic, but recently they started to send large files between the offices, and since we are load balancing, the file transfer will select one of the ADSL links and use only this during the transfers.

My question is if we replace the router in the branch office with a Mikrotik, would we then be able to use bonding to allow the file transfer to make use of both ADSL lines in parallell and thus speed up the file transfers?

I understand that there are issues with latency, packet loss, and packets arriving in the wrong order, but this would be worth trying even if we only get some minor improvement. I would even be able to install a third ADSL in the branch office to further increase the transfer speed if this is working..!

If the answer is yes - please could someone post some sample configuration. Or would it be like http://wiki.mikrotik.com/wiki/Bonding_Examples but with only one link in location #2 ..? If so - can we have just one slave on the bonding master in one end and two slaves in the other?

Sorry for asking.. But I can't test myself since I don't have the equipment needed (yet..).


Thanks!

yes, you simply establish some tunnels between those routers (eoip or pptp or something) and add them to bonding

Hi

I tried this now. It works technically, but unfortunately the throughput is miserable..

The strange thing is that the MT Bandwidth Test shows very decent numbers (1.5 Mbps + 1.5 Mbps = 3.0 Mbps total), but when I route my laptop through these routers to the Internet, I only get around 200 kbps throughput to the Internet. Needless to say, the throughput to Internet locally on the backbone network is very much better..

Is there a known explanation for this..?

My solution is to use one RB750 in each end, both running ROS4.11. On the client side there is a local LAN connected to ether1 and two ADSL 2/1 Mb modems connected to ether2&ether3. On the backbone side there is a LAN with fast Internet access connected to ether1.
Since the modems get dynamic IP from the client side provider, I configure the MTs to use PPTP to establish the connection and get known static IP-addresses, then EoIP on top of that to get interfaces that can be bonded, and then I create a bonding-interface to join the two eoip-interfaces. An IP-address is then assigned to the bonding-interface on the backbone-side, and this is used as the default gateway from the client side.

These are the main important configs;

Backbone side:

/ppp secret
add caller-id="" comment="" disabled=no limit-bytes-in=0 limit-bytes-out=0 local-address=10.3.1.1 name=tunnel1 password=password profile=default remote-address=\
10.3.1.11 routes="" service=pptp
add caller-id="" comment="" disabled=no limit-bytes-in=0 limit-bytes-out=0 local-address=10.3.1.2 name=tunnel2 password=password profile=default remote-address=\
10.3.1.12 routes="" service=pptp

/interface eoip
add arp=enabled comment="" disabled=no l2mtu=65535 mac-address=02:7E:02:59:90:1C mtu=1500 name=eoip-tunnel1 remote-address=10.3.1.11 tunnel-id=1
add arp=enabled comment="" disabled=no l2mtu=65535 mac-address=02:BA:17:DC:E7:65 mtu=1500 name=eoip-tunnel2 remote-address=10.3.1.12 tunnel-id=2

/interface bonding
add arp=enabled arp-interval=100ms comment="" disabled=no down-delay=0s lacp-rate=30secs link-monitoring=none mii-interval=100ms mode=balance-rr mtu=1500 name=\
bonding1 primary=none slaves=eoip-tunnel1,eoip-tunnel2 transmit-hash-policy=layer-2 up-delay=0s

/ip address
add address=10.4.1.1/24 broadcast=10.4.1.255 comment="" disabled=no interface=bonding1 network=10.4.1.0

/ip route
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=[Internet gateway] scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=[Client side LAN] gateway=10.4.1.11 scope=30 target-scope=10




Client side:

/interface pptp-client
add add-default-route=no allow=mschap2 comment="" connect-to=a.b.c.d dial-on-demand=no disabled=no max-mru=1458 max-mtu=1458 mrru=disabled name=pptp-out1 \
password=password profile=default user=tunnel1
add add-default-route=no allow=mschap2 comment="" connect-to=a.b.c.d+1 dial-on-demand=no disabled=no max-mru=1458 max-mtu=1458 mrru=disabled name=pptp-out2 \
password=password profile=default user=tunnel2

add arp=enabled comment="" disabled=no l2mtu=65535 mac-address=02:5A:89:47:36:57 mtu=1458 name=eoip-tunnel1 remote-address=10.3.1.1 tunnel-id=1
add arp=enabled comment="" disabled=no l2mtu=65535 mac-address=02:67:B8:46:8C:0E mtu=1458 name=eoip-tunnel2 remote-address=10.3.1.2 tunnel-id=2

add arp=enabled arp-interval=100ms arp-ip-targets=10.4.1.1 comment="" disabled=no down-delay=0s lacp-rate=30secs link-monitoring=none mii-interval=100ms mode=\
balance-rr mtu=1458 name=bonding1 primary=none slaves=eoip-tunnel1,eoip-tunnel2 transmit-hash-policy=layer-2 up-delay=0s

/ip address
add address=10.2.7.2/24 broadcast=10.2.7.255 comment="" disabled=no interface=ether2 network=10.2.7.0
add address=10.2.9.2/24 broadcast=10.2.9.255 comment="" disabled=no interface=ether3 network=10.2.9.0
add address=10.4.1.11/24 broadcast=10.4.1.255 comment="" disabled=no interface=bonding1 network=10.4.1.0

/ip route
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.4.1.1 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=a.b.c.d/32 gateway=10.2.7.1 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=a.b.c.d+1/32 gateway=10.2.9.1 scope=30 target-scope=10




I would be very grateful for feedback if someone has tried anything similar, or even has a theoretical idea for an explanation..?

Thanks

I tried a setup like this many times
It just didn't work correctly over ADSL lines during my tests and also showed rather erratic behaviour.

I figured that this problem results from slightly deviating round-trip latency times through the two ADSL lines,
which makes packets arrive out of order. Unfortunately, there is no way to control this in RouterOS.

UDP Tests should show the correct amount of bandwith, as ordering of packets is not important there.

Thanks hedele

The part that makes me most curious here, is why the MT builtin Bandwidth testserver provides an almost perfect result between the two routers.
Could it be related to what you say, that maybe the MT bandwidth tester is using UDP packets?

I have made a few more tests, and seen that result varies if I fool around with the MTU settings on pptp, eoip and bonding interfaces.
I was wondering if the smaller MTU on PPTP could confuse the bonding interface some way.. But I'm not really sure what I'm looking for..

Well i think by default the bandwidth test is using UDP, but you can make it use TCP as well.
You will see rather bad results using TCP bandwidth test

MLPPP does have mechanisms to reorder packets, so by using multilink PPPoE across the two EoIP interfaces, you
actually can get almost double the throughput. Only catch with that: Mikrotik does support ML-PPPoE as a client, but
not as server So you would need at least a Cisco 26xx or something to work as PPPoE server.

Maybe it also works if you set MRRU of at least 1524 bytes for a single-link PPPoE client (over the bonding interface), as that should also trigger usage of ML-PPPoE

what bonding mode do you use? bonding-rr is FAIL

Balance-rr is a FAIL? That's the only one that stripes packets accross multiple interfaces and gives the full aggregate bandwidth with one download, which was the question in the first place. And now you say it's a fail?

ADSL lines add random jitter to packets latency, so TCP speed is being decreased dramatically with bonding-rr

EoIP over PPTP makes things even worse

I'm running bonded adsl lines just fine, though I'm using VLAN's, not eoip's. Maybe my adsl lines are not that rubbish. Latency does not jitter significantly if using the same equipment on all lines. Usually the latency jitter stays well within 1ms when measured with smokeping. TCP throughput is just about exactly the combined speed of all lines with what I've tested.

It is a different story of course, if you cannot control your line (CPE and CO) or are bonding something long distances.