Community discussions

MikroTik App
 
mankomal
Member Candidate
Member Candidate
Topic Author
Posts: 106
Joined: Fri Nov 24, 2006 8:56 am

BGP Multihoming

Thu Jul 14, 2016 1:45 pm

Hello 

Trying to do BGP multihoming facing a few challenges, 
We have 2 ISPs we have total of 3x /22 pools , we want to broadcast 2x /22 pools out of ISP1 and 1x/22 out of ISP2 
In simple scenario i.e. we broadcast only the pools that need to broadcast everything works good 
but when we try to cross broadcast i.e. failover broadcast our ISP2 network practically shuts down 
We tried prepending advertisement from ISP1, we have tried to put MED of 100 on advertisement thru ISP1 but nothing works what is more strange that we ca see that prepend now works but still preferred path is the prepended path
Image
What could be going wrong can any one help me 
thanks 
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Multihoming

Thu Jul 14, 2016 5:24 pm

In your posted output from the looking glass  - the second path has local_pref of 90.
Local pref is more important than as-path length, so the first path having local_pref 100 will win every single time.

You need to ask the ISP what communities you may send to them to modify local_pref on their side. (this is a very common feature that ISPs provide to their customers, but each one has their own set of communities) Instead of prepending your ASN to the AS-PATH, you need to set a community which will tell that ISP to put the local_pref lower than 90 so that they will not use the direct connection to you unless the other path goes away.

What's currently happening is that the ISP itself is going to prefer to go directly to you for that prefix, regardless of the AS-prepends... but those AS-prepends will still appear to anyone beyond the ISP, so basically they're less likely to choose that ISP to reach you, but the ISP itself is guaranteed to prefer to go straight to you. Apparently this ISP has a significant amount of traffic for you, and you do not want even that traffic on this link.

If this ISP doesn't offer you the ability to send communities that modify local_pref, then you can also announce two /23 sub-prefixes of the /22 into the other ISP. The more specific route will just win out over the /22 prefix regardless of any BGP metrics, so unless your other ISP (or any entity out there) aggregates them, they'll rule the roost for your inbound traffic.

The reason MED didn't fix the problem:

MED (multi-exit discriminator) only works between two ASNs which have multiple links between them (not necessarily between the same pair of routers).
The goal of MED is to show the remote peer some information about how far your own network must carry traffic to reach a destination.
The remote peer can evaluate two competing paths to reach a prefix in your network, and see that path 1 must go further inside your network than path 2, so the remote peer can choose path 2 and forward the packet towards whatever gateway leads to path 2.

Therefore, MEDs are meaningless between two different ISPs because ISP1 only has a single exit to reach your network.

Even if they were meaningful, MED is a much lower priority in the decision process - MED is considered on step 6, where LocalPref is step 2.
 
mankomal
Member Candidate
Member Candidate
Topic Author
Posts: 106
Joined: Fri Nov 24, 2006 8:56 am

Re: BGP Multihoming

Thu Jul 14, 2016 5:28 pm

In your posted output from the looking glass  - the second path has local_pref of 90.
Local pref is more important than as-path length, so the first path having local_pref 100 will win every single time.

You need to ask the ISP what communities you may send to them to modify local_pref on their side. (this is a very common feature that ISPs provide to their customers, but each one has their own set of communities) Instead of prepending your ASN to the AS-PATH, you need to set a community which will tell that ISP to put the local_pref lower than 90 so that they will not use the direct connection to you unless the other path goes away.

What's currently happening is that the ISP itself is going to prefer to go directly to you for that prefix, regardless of the AS-prepends... but those AS-prepends will still appear to anyone beyond the ISP, so basically they're less likely to choose that ISP to reach you, but the ISP itself is guaranteed to prefer to go straight to you. Apparently this ISP has a significant amount of traffic for you, and you do not want even that traffic on this link.

If this ISP doesn't offer you the ability to send communities that modify local_pref, then you can also announce two /23 sub-prefixes of the /22 into the other ISP. The more specific route will just win out over the /22 prefix regardless of any BGP metrics, so unless your other ISP (or any entity out there) aggregates them, they'll rule the roost for your inbound traffic.

The reason MED didn't fix the problem:

MED (multi-exit discriminator) only works between two ASNs which have multiple links between them (not necessarily between the same pair of routers).
The goal of MED is to show the remote peer some information about how far your own network must carry traffic to reach a destination.
The remote peer can evaluate two competing paths to reach a prefix in your network, and see that path 1 must go further inside your network than path 2, so the remote peer can choose path 2 and forward the packet towards whatever gateway leads to path 2.

Therefore, MEDs are meaningless between two different ISPs because ISP1 only has a single exit to reach your network.

Even if they were meaningful, MED is a much lower priority in the decision process - MED is considered on step 6, where LocalPref is step 2.
Hello Zerobyte thanks for the update I will try to get communities from my upstream ISPs and try with that method and post a result at, a later date 
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Multihoming

Thu Jul 14, 2016 5:32 pm

Alright - one thing I noticed upon re-reading my post - the looking glass you're showing is obviously not in your ISPs network, so take my actual local_pref recommended values as only examples - not sure what Vodafone is using, but certainly they can give you their published BGP policy.
 
mankomal
Member Candidate
Member Candidate
Topic Author
Posts: 106
Joined: Fri Nov 24, 2006 8:56 am

Re: BGP Multihoming

Thu Jul 14, 2016 5:44 pm

Alright - one thing I noticed upon re-reading my post - the looking glass you're showing is obviously not in your ISPs network, so take my actual local_pref recommended values as only examples - not sure what Vodafone is using, but certainly they can give you their published BGP policy.
Zerobyte 
The looking glass is of SPRINT i see both ISPs using different sets of communities to publish to Sprint , waiting for a reply from my ISPs to see what communities they will share will take a call on this then 
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Multihoming

Thu Jul 14, 2016 5:51 pm

You should also find out whether your ISPs have their own looking glasses as well.
(or public route servers you can access)

Obviously this is the best way to see what your advertisements look like to your ISPs themselves, because looking at LGs further away from you is useful too, but many networks' policies have been applied by that point, so keep that in mind.
 
mankomal
Member Candidate
Member Candidate
Topic Author
Posts: 106
Joined: Fri Nov 24, 2006 8:56 am

Re: BGP Multihoming

Mon Jul 18, 2016 10:09 am

You should also find out whether your ISPs have their own looking glasses as well.
(or public route servers you can access)

Obviously this is the best way to see what your advertisements look like to your ISPs themselves, because looking at LGs further away from you is useful too, but many networks' policies have been applied by that point, so keep that in mind.
Having little to no luck on BGP communities from Upstream so I started with Prepending the lesser preferred path now what happens is that this path still receives inbound traffic to my network for specified pools but no traffic of the pools is going thru it though. Its downloading around 100M and upload is 0M not even bits . 
Does Prepending path takes time to propagate?  I can see changes in LG almost immediately.
If I use AS-PATH preference or next-hop preference what will happen if this PATH or hop goes down anyway to automate disabling the same or change on the go by using internal BGP communities, 
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Multihoming

Mon Jul 18, 2016 4:44 pm

Having little to no luck on BGP communities from Upstream so I started with Prepending the lesser preferred path now what happens is that this path still receives inbound traffic to my network for specified pools but no traffic of the pools is going thru it though. Its downloading around 100M and upload is 0M not even bits . 
Does Prepending path takes time to propagate?  I can see changes in LG almost immediately.
If I use AS-PATH preference or next-hop preference what will happen if this PATH or hop goes down anyway to automate disabling the same or change on the go by using internal BGP communities, 
These are settings that affect your OUTBOUND preference, not inbound. The only things you can do to affect inbound preference:
(these are arranged in what I consider to be most desirable to least desirable)
1) Send communities to ISP which modify the ISPs routing policy. This is the most common and most effective method, and if you send communities which instruct the ISP to lower the local_pref of your routes, this can have a completely different (and more powerful) effect on the behavior of traffic inbound towards your network.
2) set MEDs for multiple connections to same ISP (not applicable to you, I think)
3) AS-PREPEND - may not be effective to traffic originating from the ISP's own network because they may (and very often this is the case) have a local_pref value which sets customer routes as higher-priority over routes learned from their other peers. You could prepend it 50 times and it wouldn't matter because AS-PATH length is a less important metric to BGP than local_pref. This prepended route WILL affect which of your ISPs other networks may choose to reach you, but it very likely has zero effect on the ISP's own routing decisions inside its own network.
4) If all else fails, send more specific routes to another carrier. This is guaranteed to work because the first rule of routing is to choose the most specific route. A /23 route matching the destination will ALWAYS be chosen before a /22 route matching the same prefix. If you're announcing a /22 to Vodafone, you can announce the two /23 sub-prefixes of this /22 block into your other carrier's BGP table, and that will win out completely, regardless of Vodafone's  policy-based rules.

Communities are the most common, but also the hardest to easily discuss here because communities can do lots of things depending on what actions the ISP offers you. Some ISPs only have a very short list: don't advertise to other carriers/customers, lower local_pref, prepend once/twice/thrice to other carriers. Some have more choices than Nieman Marcus, and let you do things like "prepend once to this specific peer" or "do not advertise to European peers." Something like "set my local pref to very low" can have a big effect - causing the ISP to ignore your route as long as it is available anywhere else, or it can cause it to be "less attractive" to certain peers, or to everyone outside their network, etc.
 
mankomal
Member Candidate
Member Candidate
Topic Author
Posts: 106
Joined: Fri Nov 24, 2006 8:56 am

Re: BGP Multihoming

Tue Oct 04, 2016 4:53 pm

Having little to no luck on BGP communities from Upstream so I started with Prepending the lesser preferred path now what happens is that this path still receives inbound traffic to my network for specified pools but no traffic of the pools is going thru it though. Its downloading around 100M and upload is 0M not even bits . 
Does Prepending path takes time to propagate?  I can see changes in LG almost immediately.
If I use AS-PATH preference or next-hop preference what will happen if this PATH or hop goes down anyway to automate disabling the same or change on the go by using internal BGP communities, 
These are settings that affect your OUTBOUND preference, not inbound. The only things you can do to affect inbound preference:
(these are arranged in what I consider to be most desirable to least desirable)
1) Send communities to ISP which modify the ISPs routing policy. This is the most common and most effective method, and if you send communities which instruct the ISP to lower the local_pref of your routes, this can have a completely different (and more powerful) effect on the behavior of traffic inbound towards your network.
2) set MEDs for multiple connections to same ISP (not applicable to you, I think)
3) AS-PREPEND - may not be effective to traffic originating from the ISP's own network because they may (and very often this is the case) have a local_pref value which sets customer routes as higher-priority over routes learned from their other peers. You could prepend it 50 times and it wouldn't matter because AS-PATH length is a less important metric to BGP than local_pref. This prepended route WILL affect which of your ISPs other networks may choose to reach you, but it very likely has zero effect on the ISP's own routing decisions inside its own network.
4) If all else fails, send more specific routes to another carrier. This is guaranteed to work because the first rule of routing is to choose the most specific route. A /23 route matching the destination will ALWAYS be chosen before a /22 route matching the same prefix. If you're announcing a /22 to Vodafone, you can announce the two /23 sub-prefixes of this /22 block into your other carrier's BGP table, and that will win out completely, regardless of Vodafone's  policy-based rules.

Communities are the most common, but also the hardest to easily discuss here because communities can do lots of things depending on what actions the ISP offers you. Some ISPs only have a very short list: don't advertise to other carriers/customers, lower local_pref, prepend once/twice/thrice to other carriers. Some have more choices than Nieman Marcus, and let you do things like "prepend once to this specific peer" or "do not advertise to European peers." Something like "set my local pref to very low" can have a big effect - causing the ISP to ignore your route as long as it is available anywhere else, or it can cause it to be "less attractive" to certain peers, or to everyone outside their network, etc.

Zerobyte,

Thanks for your help finally got it working by using unsummarized routes in one ISP BGP and summarized routes in other ISP

Cheers
 
1obro
just joined
Posts: 4
Joined: Fri Oct 10, 2014 7:00 pm

Re: BGP Multihoming

Sun Oct 09, 2016 3:10 pm

If I understood correctly.
You want to use your secondary link only, and only, as the primary goes down.
You also could ask your secondary upstream provider (vodafone india) if they support lowering the local preference of your announcement.

For example;
Cogent supports this.
When you are tagging your announcement with the community 174:70 (communities cogent: https://onestep.net/communities/as174/ )
This will lower the local preference of your announcement, thus cogent will prefer the route via their peer (in this case it would be via NTT, as NTT is the TIER1 transit of your primary)
this means that your announcement via your secondary wont be seen unless your primary goes down.

Who is online

Users browsing this forum: No registered users and 3 guests