Page 1 of 1

BGP ECMP (multipathing)

Posted: Thu Jan 30, 2014 9:14 pm
by Crami
Hi There

I have setup where the Customer router (Router Boards) is connected to a MPLS ELAN (VPLS) cloud provided by a third party. We, the ISP are connected redundantly to that cloud with two Cisco Routers. I now set up two BGP
sessions from the Customer Routers to the ISP Routers for redundancy and announce the Prefix the Customer gets assigned and send a default route to them from the Cisco routers.

I now like to do load balancing over these two default gateways. I know that MikroTik has the standpoint that BGP can not do ECMP by design, but all other vendors I know can do it (Cisco, Juniper, Brocade, Fortinet etc.)

As the two BGP sessions do not end on the same router I can not use the loopback method mentioned in the BGP ECMP manual here: http://wiki.mikrotik.com/wiki/Manual:BG ... interfaces
But they reference a Cisco page to explain that other vendors do it the same way and that this is only the right way... fortunately on this same page Cisco explains how BGP multipath works! http://www.cisco.com/en/US/tech/tk365/t ... 45bf.shtml (Load Sharing When Dual-Homed to One Internet Service Provider (ISP) Through a Single Local Router)

Will MikroTik reconsider their standpoint about multipath BGP or do I have to switch back to Cisco for the customer routers?

Regards

Matthias

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 11:35 am
by mrz
It is a hack and we also can do it.
http://wiki.mikrotik.com/wiki/Manual:BG ... interfaces

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 1:05 pm
by Crami
Please explain how this is different to what I already wrote. Loopbacks are not possible. There are two (2) BGP sessions to two (2) different routers. So the destination can not be the same.

Or am I wrong ?

Regards

Matthias

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 1:35 pm
by mrz
Read that cisco article and mikrotik article, they both uses the same principle and loopback address.

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:05 pm
by Crami
Have you red the section: Load Sharing When Dual-Homed to One Internet Service Provider (ISP) Through a Single Local Router


This is what I like to accomplish. The other examples are clear and is common practice, I agree...

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:12 pm
by mrz
This is exactly the same setup as Load Sharing with the Loopback Address as a BGP Neighbor except that remote end has two routers.
Setup on mikrotik router would be the same, both remote peers establishes BGP session to loopback address.

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:18 pm
by Crami
Remote peers are not the problem. I want two default routes on the Mikrotik to the remote peers. That does not change anything if I make a session to remote peer loopbacks.

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:25 pm
by mrz
So to make it clear you have two BGP R1 and R2 running iBGP between then. They bothhave peering to ISPs router (R3).
And you want to have ECMP routes on R1 and R2?

shell code

R1------------\
 |             \
ibgp           ISP R3
 |             /
R2------------/

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:34 pm
by Crami
No, the other way around:
    +-------------- ISP_R1 -+
    |   EBGP                |
 R1-+                       | IBGP
    |   EBGP                |
    +-------------- ISP_R2 -+
R1 is Mikrotik
ISP_R1 and R2 are Cisco

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:46 pm
by mrz
Then the same setup as described in the manual
R1 has loopback address and both ISP routers should establish connection to loopback. Exact same config is also in cisco article.

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 2:59 pm
by Crami
Yes, exactly ... But with the little command "maximum-paths 2" in the bgp config!

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 3:26 pm
by mrz
It just limits ecmp gateways to two. Mikrotik does not limit amount of gateways.

Re: BGP ECMP (multipathing)

Posted: Fri Jan 31, 2014 3:45 pm
by Crami
So then the missing A on route 1 does not mean anything ?
> /ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  0.0.0.0/0                          212.25.9.1               20
 1  Db  0.0.0.0/0                          212.25.9.2               20
If I make traceroutes from a public ip behind the mikrotik (no nat) to several locations, it always takes the path over route 0.

Re: BGP ECMP (multipathing)

Posted: Fri Feb 14, 2014 10:35 am
by Crami
And now there is silence ...
Are you now convinced that there should be a "maximum-path" feature like in cisco ios to accomplish this? Or am I just to boring to be eligible for a rely.

Re: BGP ECMP (multipathing)

Posted: Mon Mar 03, 2014 11:42 pm
by Chupaka
maybe, post at least your BGP config first?..

Re: BGP ECMP (multipathing)

Posted: Tue Mar 04, 2014 10:28 am
by Crami
Shure, here ist the relevan config of the mikrotik router.
# feb/02/1970 15:21:57 by RouterOS 6.7
# software id = XHEM-GX3Y
#
/interface vlan
add comment="CES" interface=sfp1 l2mtu=4070 mtu=4000 name=sfp1-ces vlan-id=3214
/ip neighbor discovery
set sfp1-ces comment="CES"
/interface wireless security-profiles
/routing bgp instance
set default as=64512 router-id=z.y.x.w
add address=z.y.x.w/26 interface=sfp1-ces network=z.y.x.0
/routing bgp network
add network=a.b.c.d/27 synchronize=no
/routing bgp peer
add name=peerA remote-address=z.y.x.1 remote-as=8758 ttl=default update-source=sfp1-ces
add name=peerB remote-address=z.y.x.2 remote-as=8758 ttl=default update-source=sfp1-ces
peerA and peerB are Cisco Routers, which only announce a default route.
I'm happy for any inputs

Re: BGP ECMP (multipathing)

Posted: Tue Mar 04, 2014 1:55 pm
by mrz
Ok, sorry for that. Yes you are right your intended setup is currently not possible on MikroTik routers. BGP will install only one gateway.

Re: BGP ECMP (multipathing)

Posted: Wed Mar 05, 2014 10:10 am
by Crami
Thanks for your confirmation. At least I know now that I'm not doing something wrong. Are there plans to support this? Would be a very nice and appreciated feature.

Re: BGP ECMP (multipathing)

Posted: Wed Oct 22, 2014 7:04 am
by ttanner
I am really disappointed to see that this is not supported. We have 4 CCR1036-8G-2S+EMs in production that I need to do this to coming up.

What can we do to get this feature implemented? We have been migrating away from Cisco to Mikrotik, but missing features like this are really hurting us.

Re: BGP ECMP (multipathing)

Posted: Sat Nov 01, 2014 12:03 pm
by ada
Please add suport for many (2-3) ways for gateway for BGP

Re: BGP ECMP (multipathing)

Posted: Sat Nov 08, 2014 12:53 am
by engineertote
very interested

MT guys ,, can you please update the post if you are going to add this feature or not ?

Re: BGP ECMP (multipathing)

Posted: Thu Nov 26, 2015 1:44 pm
by omago
Any updates on this topic ?
Is BGP ECMP routing on Mikrotik's roadmap ?

Regards,

Jon

Re: BGP ECMP (multipathing)

Posted: Mon Nov 30, 2015 5:41 pm
by StubArea51
Would like to know about this as well - Maybe this is a v7 feature?

Re: BGP ECMP (multipathing)

Posted: Tue Dec 01, 2015 8:42 pm
by Bergante
It would be really useful.

BGP multipathing can be used, for example, to turn a router into a load balancer for a cluster of servers (web, DNS, etc) and it's a technique used in many large installations.

This article describes it very well.
http://bits.shutterstock.com/2014/05/22 ... -software/

This depends on a multipath BGP implementation that can do a flow or hash based load balancing. In a configuration like this several servers would announce the same IP address via BGP and the routers would distribute connections among them. Flow based balancing (or some kind of IP address hashing) is needed so that all the packets belonging to a TCP connection are sent to the same servers.

This technique is very cheap to implement (unless you have to use expensive routers for it) and in my opinion it's much more reliable and scalable than the typical load balancer "magic boxes".

So, do it, please!

Re: BGP ECMP (multipathing)

Posted: Tue Oct 03, 2017 1:31 am
by C3H5N3O9
Any updates on this topic ?
Is BGP ECMP routing on Mikrotik's roadmap ?

Regards,

Re: BGP ECMP (multipathing)

Posted: Thu Feb 01, 2018 6:41 pm
by chubbs596
Any updates on this topic ?
Is BGP ECMP routing on Mikrotik's roadmap ?

Re: BGP ECMP (multipathing)

Posted: Thu Feb 01, 2018 6:47 pm
by pe1chl
I would guess that it is all in the RouterOS v7 super duper routing code!

Re: BGP ECMP (multipathing)

Posted: Wed Sep 05, 2018 7:35 pm
by Crami
More than four (4) years since my initial post, and still there is no proper ECMP support ...

Re: BGP ECMP (multipathing)

Posted: Thu Sep 06, 2018 3:15 am
by schadom
More than four (4) years since my initial post, and still there is no proper ECMP support ...

Welcome to Mikrotik... I guess they should remove the "Router" from "RouterOS", as it seems there had been no progress on the routing engine for years now. Instead hundreds of fixes for the wifi and access point stuff are appearing in every new ROS release. Mikrotik nowadays is a great access point and WISP supplier, but not a company I'd buy another edge/core router or switch from. Seems their focus and market orientation has changed after releasing the CCR-series. Sad story, as there would have been potential.

Re: BGP ECMP (multipathing)

Posted: Sat Sep 08, 2018 9:49 pm
by StubArea51
I've been wanting to see this as well, but i'd rather have recursive routing in IPv6 for BGP fixed first. :-)

Re: BGP ECMP (multipathing)

Posted: Tue Nov 13, 2018 11:16 am
by artferrr
I've been wanting to see this as well, but i'd rather have recursive routing in IPv6 for BGP fixed first. :-)

maybe in ROS7 years after 10 I’m very impressed with the support of mikrotik for 2019, it is planned to fully transfer the kernel to devices from other manufacturers.

Re: BGP ECMP (multipathing)

Posted: Thu May 02, 2019 2:04 pm
by heribertos
I solved this for me with Routing Filters under Set Next-Hop-in with multiple Gateway-Addresses. This works for me even for BGP.

Please observe, that in CISCO the max-path n does 2 things: (1) Installation of best path and (n-1) looser pathes (2) Forming an ECMP Route from the multiple routes to the same destination prefix. The first part is not needed in MikroTik, because MikroTik installs always all BGP Routes in the Routing Table, of couse not neccessarily activ. Only the second part is required and could be done manually, by routing filters or by scripts.

Second thing to observe is, you will get never ever on any router 2 routes active pointing to the same destination prefix.

Re: BGP ECMP (multipathing)

Posted: Thu May 02, 2019 4:25 pm
by pe1chl
Second thing to observe is, you will get never ever on any router 2 routes active pointing to the same destination prefix.
But that is the idea of ECMP.... have 2 or more routes for the same prefix pointing to a different path, and the router has some mechanism to use them alternately for outgoing traffic.

Re: BGP ECMP (multipathing)

Posted: Thu May 02, 2019 5:17 pm
by heribertos
Sorry, for my confusing wording. I think we mean the same.

ECMP Route is for me 1 route with multiple gateways, it appears as 1 destination prefix in the table. Of course, that are 2 or more routes , but considered as 1 with respect to administrative distance.

With 2 active routes I ment 2 seperate entries in the table, and each one is marked as active, as the OP in his last post was asking, why 1 default route route is active and the second is not..
My comment is, that the second will never get active.

No issue. We mean the same.

Regards

Re: BGP ECMP (multipathing)

Posted: Tue Jul 02, 2019 11:18 pm
by RcRaCk2k
Came to that point today and i am disappointed that mikrotik developers are not able to give us some information if that feature is on a todo list or not.

Re: BGP ECMP (multipathing)

Posted: Wed Jul 03, 2019 4:00 am
by troybowman
Please add my +1 vote for ECMP, too.

Re: BGP ECMP (multipathing)

Posted: Fri Aug 16, 2019 10:01 pm
by Railander
as a workaround, ECMP works perfectly with OSPF instead of BGP, assuming your setup allows for it.
alternatively, the tip provided by heribertos using routing filters forcing the "Next-Hop-in" to your desired multipath gateways might work, though you'll lose the fully dynamic aspect of BGP by essentially enforcing static ECMP routes.

with that said, i'd really like to have ECMP supported in MPLS LDP. it's really weird that it works fine with OSPF but when you run LDP on top of it it just picks whatever gateway is listed first on the route, this is terrible for load-balancing VPLS.

Re: BGP ECMP (multipathing)

Posted: Mon Jun 22, 2020 9:36 pm
by ploquets
I solved this for me with Routing Filters under Set Next-Hop-in with multiple Gateway-Addresses. This works for me even for BGP.


Thanks for this, I could workaround this issue and now working!

Re: BGP ECMP (multipathing)

Posted: Thu Jul 02, 2020 9:50 pm
by RcRaCk2k
How have i to setup "Set In Nexthop" on Actions-Tab as @heribertos suggested?
What value have to be in there?

Re: BGP ECMP (multipathing)

Posted: Fri Sep 04, 2020 8:42 am
by peterh
with that said, i'd really like to have ECMP supported in MPLS LDP. it's really weird that it works fine with OSPF but when you run LDP on top of it it just picks whatever gateway is listed first on the route, this is terrible for load-balancing VPLS.
It is even worse. RouterOS "BGP" (I am using quotes because it is not RFC 4271 compliant) picks not just the first gateway for equal cost routes, but even for UNEQUAL cost routes, since it does not care whatsoever for IGP costs to reach the BGP next hop gateway. This not only violates RFC 4271 but essentially also means that RouterOS routes packets in an BGP or MPLS cloud essentially in a random fashion. Yes, packets reach the target, but to go from New York to Boston, RouterOS first sends you first to Los Angeles, then Seattle, detour through Chicago, and then to Boston, since its "BGP" has no understanding of the IGP topology in an AS.

With any other BGP implementation on the market (even free Open Source solutions), you get true recursive lookups of the BGP next hops within the IGP routing table including their costs, as it is described in RFC 4271. And yes, with most you even get ECMP.

Re: BGP ECMP (multipathing)

Posted: Fri Sep 04, 2020 9:00 am
by Crami
You are absolutely right. That's why I do not use mikrotik for larger scale BGP deployments. They should use something prooven as routing suite, like frr or bird underneath.

Re: BGP ECMP (multipathing)

Posted: Fri Sep 04, 2020 4:40 pm
by pe1chl
It is even worse. RouterOS "BGP" (I am using quotes because it is not RFC 4271 compliant) picks not just the first gateway for equal cost routes, but even for UNEQUAL cost routes, since it does not care whatsoever for IGP costs to reach the BGP next hop gateway. This not only violates RFC 4271 but essentially also means that RouterOS routes packets in an BGP or MPLS cloud essentially in a random fashion. Yes, packets reach the target, but to go from New York to Boston, RouterOS first sends you first to Los Angeles, then Seattle, detour through Chicago, and then to Boston, since its "BGP" has no understanding of the IGP topology in an AS.
That became clear to me pretty quickly when I started using RouterOS BGP, and I quickly decided to use a different AS number for every site.
(of course such decisions are only possible in a closed network not BGP-connected to internet)

Re: BGP ECMP (multipathing)

Posted: Fri Sep 04, 2020 4:44 pm
by peterh
The sad thing about this is that since the last 10 years, we see a creeping trend for "BGP everywhere". It is no longer just a WAN protocol for nation-wide ISPs and occasional Internet Exchange peerings, it first creeped into LANs and campus networks through MPLS and now we see whole data centre networks being constructed purely on BGP (i.e. RFC7938). That way you can scrap the underlying OSPF/ISIS completely and run on one protocol - BGP everywhere - from your spine to your leaf switches to your Internet peerings. MikroTik has catapulted itself out of that race with its current implementation. BGP ECMP is an absolute must in the year 2020 (actually it already was in 2010).

Re: BGP ECMP (multipathing)

Posted: Fri Sep 04, 2020 5:38 pm
by pe1chl
Similar to what that RFC proposes, I quickly switched to EBGP-only to avoid a lot of strange routing issues that I encountered at first when I had used the same AS for a couple of sites in the same region so there was a mix of IBGP/EBGP.
I now use IBGP only between routers at the same site that are directly linked.
Of course still no ECMP but at least no unpredictable routing.

Re: BGP ECMP (multipathing)

Posted: Tue Jul 13, 2021 4:16 pm
by jr0dd
@normis Is there any hope on ever getting ECMP implemented for BGP?! Considering it's the standard it should already be in ros.

ugh i guess there is no way to tag people on this forum

Re: BGP ECMP (multipathing)

Posted: Tue Jul 13, 2021 4:36 pm
by StubArea51
It's on the roadmap for protocol support in the v7 status page

https://help.mikrotik.com/docs/display/ ... col+Status

Image

Re: BGP ECMP (multipathing)

Posted: Tue Jul 13, 2021 6:15 pm
by jr0dd
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(

Re: BGP ECMP (multipathing)

Posted: Tue Jul 13, 2021 6:26 pm
by Cablenut9
It's on the roadmap for protocol support in the v7 status page
https://help.mikrotik.com/docs/display/ ... col+Status
I just need OSPF route conversion from v6 and then I'm golden.

Re: BGP ECMP (multipathing)

Posted: Wed Jul 14, 2021 3:36 am
by nz_monkey
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(
Mikrotik spent years putting the groundwork in place, building the framework for the new routing engine to ensure it would scale and be easy to maintain. They also hired a bunch more developers.

You might be surprised at how quickly features are added from this point forward...

Re: BGP ECMP (multipathing)

Posted: Wed Jul 14, 2021 4:09 am
by joegoldman
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(
Mikrotik spent years putting the groundwork in place, building the framework for the new routing engine to ensure it would scale and be easy to maintain. They also hired a bunch more developers.

You might be surprised at how quickly features are added from this point forward...
Its a big software dev problem I come across it all the time - 80% of the work is background "do-nothing" code (to a customer perspective) so the 20% of the frontend is super quick, zippy, easy to add and remove things, make it modular etc. Once you get over that hump it becomes so much quicker.

Re: BGP ECMP (multipathing)

Posted: Wed Jul 14, 2021 7:13 am
by nichky
@mrz

coming back to your first post

has anything changed since then?
https://wiki.mikrotik.com/wiki/Manual:B ... interfaces

I'm playing with multihop=yes and the target-scope that im receiving is 30 by default without any filter rule.

Re: BGP ECMP (multipathing)

Posted: Thu Jul 15, 2021 1:21 am
by jr0dd
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(
Mikrotik spent years putting the groundwork in place, building the framework for the new routing engine to ensure it would scale and be easy to maintain. They also hired a bunch more developers.

You might be surprised at how quickly features are added from this point forward...
Well that would be real nice if this could move to a stable release real soon. ROS has just felt stagnant for the last couple years. Hardware moving forward while the os at stand still.

Re: BGP ECMP (multipathing)

Posted: Fri Jul 16, 2021 10:46 am
by nz_monkey
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(
Mikrotik spent years putting the groundwork in place, building the framework for the new routing engine to ensure it would scale and be easy to maintain. They also hired a bunch more developers.

You might be surprised at how quickly features are added from this point forward...
Well that would be real nice if this could move to a stable release real soon. ROS has just felt stagnant for the last couple years. Hardware moving forward while the os at stand still.
You must not have been paying much attention!

RouterOS v6 has seen an immense number of new features and bugfixes in the last 2 years. And on top of that RouterOS v7 has entered beta with an entirely new "ground up" routing stack, Wireguard support, Multi-Chassis LAG and other long-requested features.

Re: BGP ECMP (multipathing)

Posted: Sun Jul 18, 2021 2:24 am
by tstivers
Thanks. Not very hopeful on it being any time soon since v7 has been in the works for years now :'(
Mikrotik spent years putting the groundwork in place, building the framework for the new routing engine to ensure it would scale and be easy to maintain. They also hired a bunch more developers.

You might be surprised at how quickly features are added from this point forward...
Well that would be real nice if this could move to a stable release real soon. ROS has just felt stagnant for the last couple years. Hardware moving forward while the os at stand still.
In fairness, their routers are dirt cheap even by consumer router standards and offer massive value *even without* BGP ECMP. I'd like to see it, too. But their devices aren't exactly priced to support a rapid pace of development while remaining stable, either.

Re: BGP ECMP (multipathing)

Posted: Mon Aug 16, 2021 9:25 pm
by eguzman
More than four (4) years since my initial post, and still there is no proper ECMP support ...


Well, is August 2021, still not. I found your post trying to solve the exactly same issue :(

Re: BGP ECMP (multipathing)

Posted: Sat Oct 16, 2021 10:09 am
by tuxtof
same for me please

Re: BGP ECMP (multipathing)

Posted: Thu Feb 10, 2022 7:38 pm
by RcRaCk2k
7.2rc3 is out now - and no ECMP implementation so far.

Re: BGP ECMP (multipathing)

Posted: Fri Sep 22, 2023 1:50 am
by eguzman
7.11.2 out, september 2023, still no ecmp support :(

Re: BGP ECMP (multipathing)

Posted: Mon Dec 18, 2023 9:04 pm
by RcRaCk2k
7.13 Stable out now, NO ECMP Support

Re: BGP ECMP (multipathing)

Posted: Sun Mar 31, 2024 12:33 am
by ivanflash84
7.14.2 stable and still no bgp ECMP.

Re: BGP ECMP (multipathing)

Posted: Sun Mar 31, 2024 3:06 pm
by tikmakro
This is such a huge gap in Mikrotik's feature set. Ubiquity have had BGP multipathing in EdgeOS for a long time now. To not even have a timeline or response for years on this thread suggests it is not a priority despite this topic having among the most views on this forum. This omission will be a deal-breaker for our continued use of RouterOS.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 9:34 am
by DarkNate
Don't forget BGP UCMP, i.e. weighted ECMP with link bandwidth awareness.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 1:01 pm
by markonen
I've been wondering if there's some sort of confusion at MikroTik about what exactly we're asking for.

Maybe someone at MT thinks they need to implement BGP ADD_PATH before working on this? Some of the comments I've seen over the years from them seem to point that way.

I bet most of us would be very happy with a single ECMP route per BGP peer, with no ADD_PATH required.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 1:43 pm
by mrz
There is no confusion, for OP setup ADD PATH is necessary.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 1:54 pm
by markonen
I'm sorry but I still don't understand why. My read was that the OP had two BGP sessions with a default coming from each, and wanted the ECMP route over those. So no BGP session is carrying more than one route per prefix. What did I miss, where's the need for ADD PATH?

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 2:17 pm
by mrz
but you can already have ECMP in any other case, each instance session can install its own best route and form ecmp.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 6:22 pm
by markonen
Okay, confusion confirmed, although I can't rule out the confusion being in my head :D I don't understand what this has to do with ADD PATH.

The OP (like me) wants to have the defaults received from two BGP peers automatically result in ECMP routing. This is what's happening in all the other vendor gear I use (Cisco, Juniper, Bird on Linux, Dell OS10, etc), without configuring ADD PATH.

I think you're saying that there's a workaround: just have dedicated BGP instances for each peer, each of the defaults will look like the best path within that instance and get installed. Great. But that's not ECMP. There's nothing looking at the route costs to determine that they're equal. Every route from every peer is deemed equal, so as soon as you're talking about anything other than defaults, it falls down.

The way I'd expect ECMP to work without ADD PATH is that if you get far enough in the BGP route selection algorithm (basically at the point where you're comparing router IDs) both/all of the routes get inserted in the FIB (the main routing table in the non-ASIC case). That's what everyone else is doing. It doesn't have to change any BGP messages, at all, so I don't understand this functionality being gated behind ADD PATH. A single best path is still selected and advertised to peers. It's just that if there are routes with the same cost as the best path, they get inserted as well. That's it!

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 7:03 pm
by pe1chl
That only works (reliably) when you are facing the simple situation of two BGP peers that send you a locally originated route (e.g. a default route) over 1 hop.
When the network is a little more complicated (several routers with several links between them, i.e. a partial mesh) such simple hacks may develop into a routing loop.

Re: BGP ECMP (multipathing)

Posted: Wed Apr 03, 2024 9:07 pm
by markonen
Can you help me figure out a napkin scenario where it actually does cause a loop? I tried to think up one but couldn’t, at least not with EBGP and sane tie-breaking rules (like those I described).

The way you enable this on Junos (just add "multipath;" on the bgp level) and IOS (set "maximum-paths" to more than 1) really seems like standard operating procedure rather than "a hack" and is documented without much of a warning sign. So it's weird to me that this is ok for them, and others, and unacceptable here. They do support add path, though, so there's that.

I'm far from an expert at this so I hope I'll be educated. I'm getting somewhat worried about what I'm overlooking since I'm current running my Junos default free zone routers with multipath enabled, without add path, on a complex (well, not totally trivial) network, pretty much oblivious to any problems.

Re: BGP ECMP (multipathing)

Posted: Thu Apr 04, 2024 12:36 pm
by mrz
I agree that for BGP to install ECMP routes, add path is not necessary, but since both ADD PATH and ECMP would need changes in best path selection code, those features should be linked and implemented at the same time.
But that's not ECMP. There's nothing looking at the route costs to determine that they're equal.
In contrary it actually is ECMP. If you try you will see that v7 adds multiple routes with + flag to the same destination and all of those routes are active, meaning that these routes are ECMP routes. And if you want to exclude for example one peer from adding ECMP route change the distance with routing filters.

Re: BGP ECMP (multipathing)

Posted: Thu Apr 04, 2024 3:20 pm
by pe1chl
Can you help me figure out a napkin scenario where it actually does cause a loop? I tried to think up one but couldn’t, at least not with EBGP and sane tie-breaking rules (like those I described).
The issue is that in a more complicated network, a route that you get from a peer may ultimately be coming from/via yourself.
Normally BGP would avoid that but when you install routes beyond what BGP considers the best path, such paths may form.

Re: BGP ECMP (multipathing)

Posted: Sat Apr 06, 2024 11:12 am
by DarkNate
but you can already have ECMP in any other case, each instance session can install its own best route and form ecmp.
I'm not sure what this means. Have employees at MikroTik never deployed BGP multipathing on Cisco, Juniper or even straight FRR on Ubuntu? There is no add-path involved. That is misinformation.
https://docs.frrouting.org/en/latest/bg ... -bandwidth

What does "each instance session" mean?

If I have two peers:
peer1: 1.1.1.1
peer2: 2.2.2.2

Am I supposed to have two BGP sessions with each or what?

Re: BGP ECMP (multipathing)

Posted: Sun Apr 07, 2024 1:43 am
by mrz
To my knowledge none of the MT employees have said that add path is required to install ECMP routes.
And if you are so familiar with Cisco, Juniper and FRR and have at least once deployed BGP on the RouterOS you should know what BGP instance is.
There is a lot of info all over the internet on what BGP instance is, hint, try typing "bgp multi instance" in the google search box.

Re: BGP ECMP (multipathing)

Posted: Sun Apr 07, 2024 4:50 am
by DarkNate
To my knowledge none of the MT employees have said that add path is required to install ECMP routes.
And if you are so familiar with Cisco, Juniper and FRR and have at least once deployed BGP on the RouterOS you should know what BGP instance is.
There is a lot of info all over the internet on what BGP instance is, hint, try typing "bgp multi instance" in the google search box.
You mrz are the one who said we need add path based on your public messages on this thread, stop making shit up.

In Cisco BGP multi-instance doesn't mean much, at best this is what it means:
https://community.cisco.com/t5/routing/ ... 05#M342994

On Juniper BGP multi-instance doesn't mean much beyond this:
https://www.reddit.com/r/Juniper/commen ... nos_mx204/

In both vendors, an instance can refer to an AFI (same ASN on both sides, same IP, but different AFI) or simply local-as spoofing/breaking the contiguous AS of a router.

Stop making shit up, this has absolutely NOTHING to do with BGP multipathing, and I've deployed BGP multipathing on Cisco, Juniper and Huawei.

Here's Cisco BGP multipathing, NOTHING to do with multi-instance:
https://vxplanet.com/2019/01/03/bgp-loa ... balancing/

Stop propagating fake information.

Re: BGP ECMP (multipathing)

Posted: Sun Apr 07, 2024 5:32 am
by Amm0
I take @mrz's comment to be that are NOT related other than if their changing BGP code, they want to do both at same time. e.g.
I agree that for BGP to install ECMP routes, add path is not necessary,
but since both ADD PATH and ECMP would need changes in best path selection code, those features should be linked and implemented at the same time.

Re: BGP ECMP (multipathing)

Posted: Mon May 20, 2024 10:01 pm
by RcRaCk2k
I fully agree with @markonen and @DarkNate

One BGP-Instance, 2 BGP Peers, if both peers send 0.0.0.0/0 or any other prefix with 2-Hop or 1-Hop reachable so the costs are exactly the same, it has to be imported as ECMP-Route.

Having two BGP-Instances is not the same. Because if 0.0.0.0/0 is announced once via 2-Hop and the another with 4-Hops, both routes get installed as ECMP-Route because both instances have seen there routes only for there instance as best route. This will result in an unwanted routing.

BGP Additional Pathes is not nessecary for ECMP Routing, because with ADD-PATH only informations will be sent to the neighbor that normally filtered out by best path selection.

Also if a router have 2 Pathes and forms an ECMP, a neighbour have not to know wich two pathes are selected. Only send the first path on an ECMP-Route as long ADD-PATH configuration is not supported or not configured. ADD-PATH have to be configured for RX/TX path but is not neccessary.

Re: BGP ECMP (multipathing)

Posted: Sat Dec 07, 2024 5:49 pm
by RcRaCk2k
Is ECMP for BGP on a roadmap?

Re: BGP ECMP (multipathing)

Posted: Sun Dec 08, 2024 11:24 am
by pe1chl
It seems like BGP in general is a bit off the current attention hotspot... in 7.17rc there are no announced BGP changes at all, even though there are several new known bugs (introduced over the 7.15 .. 7.16 span) and of course some unimplemented features too.