Community discussions

MikroTik App
 
cmassey
just joined
Topic Author
Posts: 20
Joined: Fri Nov 08, 2019 8:06 am

Load balancing with MPLS (breaks LACP)

Tue Sep 14, 2021 9:29 pm

I currently have a network that is routed using OSPF and intend on enabling MPLS and using BGP to send public traffic over MPLS-TE's. I have many sites where I have 2 radio links as backhaul to aggregate bandwidth. Currently, I have LACP enabled to balance traffic based on layer 3+4. When I enable MPLS on the network, LACP will no longer be able to balance the traffic. In the future there will be VPLS tunnels to each site as well.

Has anyone found a good way to balance traffic across 2 links with MPLS enabled? I don't believe there's a way to create 2 VPLS tunnels to the same location so I couldn't just separate the the links into 2 OSPF paths to split the traffic that way.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2182
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Load balancing with MPLS (breaks LACP)

Wed Sep 15, 2021 2:19 am

MPLS ECMP is coming in RouterOS v7
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: Load balancing with MPLS (breaks LACP)

Wed Sep 15, 2021 4:08 am

MPLS traffic can be load balanced across the members of a bonding interface if you use balance-rr instead of 802.3ad.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2182
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Load balancing with MPLS (breaks LACP)

Wed Sep 15, 2021 9:04 am

MPLS traffic can be load balanced across the members of a bonding interface if you use balance-rr instead of 802.3ad.
It sure can, but please remember this will only work reliably when it is Mikrotik <---> Mikrotik or Mikrotik <---> Linux with both ends configured for balance-rr
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: Load balancing with MPLS (breaks LACP)

Wed Sep 15, 2021 9:22 am

Yes - I was going to mention that both sides would have to support balance-rr, which is mostly just MikroTik and Linux. I have heard that HPE supports balance-rr, but I've never tried it on HPE devices.
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: Load balancing with MPLS (breaks LACP)

Wed Sep 15, 2021 10:56 pm

I don't believe there's a way to create 2 VPLS tunnels to the same location so I couldn't just separate the the links into 2 OSPF paths to split the traffic that way.
You can have as many tunnels between the two points as you want. Just use a unique vpls-id.

Since you are using mpls-te you could add a second loopback ip for the tunnel endpoints and set its primary path to use the interface you want.
 
rpra
newbie
Posts: 45
Joined: Tue Jan 24, 2012 8:43 pm

Re: Load balancing with MPLS (breaks LACP)

Thu Sep 23, 2021 12:33 pm

This is a good question.
balance-rr is not a good solution and should be avoided.

To make flow-aware load-balancing router needs to use some more information to calculate hash value.
With "pure" IP traffic it uses 5-tuple, but with MPLS it's more challenging.
Some ways to do that:
1. Flow-label in tunnel. Additional label that have to be calculated at the endpoints and added to label stack. Must be supported by both sides, and transit router should use is for hash calculation.
2. Inspect VPLS/PWs packets deeper to the IP headers and take src/dst values. Works only if there is no additional encapsulation in the tunnel (ex. pppoe).

I think MT should add some of this mechanisms to ROS7, especially if they want to enable MPLS ECMP.

Load-balancing by flow-label or IP header in MPLS on the switches LAGs must be supported by switch-chip. I don't know if Mikrotik switches can support this.
 
cmassey
just joined
Topic Author
Posts: 20
Joined: Fri Nov 08, 2019 8:06 am

Re: Load balancing with MPLS (breaks LACP)

Fri Sep 24, 2021 5:53 am

It looks like the MPLS ECMP coming in v7 is really what I'm looking for. I just don't know how long it will be until Mikrotik adds it. I'm assuming there isn't any information about that? I don't believe there's a roadmap.

I have about 1/3 Mikrotik switches and the rest vendors which don't support balance-rr. I suppose I could change out the other switches but I was a little worried about packets in the same flows coming in out of order.
 
User avatar
ScottReed
Member Candidate
Member Candidate
Posts: 115
Joined: Thu Sep 24, 2009 9:47 pm
Location: Montana / Western Massachusetts

Re: Load balancing with MPLS (breaks LACP)

Thu Nov 07, 2024 7:47 pm

Sorry to resurrect an old post, but I am facing the same issue. I use a CCR1036 on 6.47.10 as a traffic aggregation point at our core for our WISP network. Said router is uplinked to our core switch stack via 2x 10Gbps interfaces in a 802.3ad bond using l3+l4 hashing. This router establishes two VPLS tunnels per site router to carry VLAN tagged public IP and VLAN tagged private IP (NAT) traffic. Absolutely none of this traffic is load balanced and one of the two sfp+ interfaces in the bond takes the brunt of the work. The issue is becoming more prevalent as we grow with this one interface coming within 90% of saturation during peak while the other interface is at about only 20% capacity. balance-rr is not an option here.

Further in the network at the sites, I see the same behavior where we use 802.3ad bonded interfaces between 2116's and 317's. Only one of the two member interfaces will take the traffic load. What's interesting is that these sites exhibiting this behavior are all on new releases of RouterOS 7. I might try the balance-rr method where I have Mikrotik <--> Mikrotik cases.

As stated earlier in this thread by another user, the Mikrotik is probably not inspecting the MPLS/PW headers and therefore unable to calculate a hash value.

I opened SUP-170589 yesterday about this behavior. At this point I am trying to figure out if this is a limitation of RouterOS, a misconfiguration on my part, or time to just throw in a 2216 in place of the 1036 and let the problem migrate over to a device with higher capacity interfaces!

Who is online

Users browsing this forum: No registered users and 8 guests