Hi folks,
When I have redistribute connected enabled for a BGP instance, all my connected routes are being advertised to our external peer, private etc, even though I have route filters set.
How can I redistribute connected and still have the sub-nets I want to advertised filtered?

Thanks,
Q.

You likely have made a mistake somewhere.
The route filters will filter the routes that result from "redistribute connected".
So your route filters are probably wrong...

Spot on i needed to specify protocol connect in my deny filter:
add action=discard chain=AllowToGTT protocol=connect,bgp target-scope=0-255

Thanks for pointing me in the right direction

Is this for MPLS/VPNv4 or for public Internet routing?
If the latter, then you really shouldn't redistribute connected (or static or any other protocol)

Just add networks for the prefixes you wish to announce.

Hi,
I'm using eBGP.
Q.

Hi,
I'm using eBGP.
Q.

Okay, so basically you should add networks to your BGP instance so that they originate properly. Redistributing routes into a protocol is necessary in some cases, but eBGP is probably the one that needs it the least. Redistributed routes are considered inferior by routing protocols, and while they have their place, it's generally a bad idea to just redistribute everything into a routing protocol. BGP is a protocol that already behaves in a very similar fashion to having "redistribute static" enabled...

Add your prefix(es) like this: Whenever an exactly-matching route is active in the routing table, BGP will start announcing this prefix to its peers - and any peer whose out-filter allows the prefix will receive the announcement.

One more piece of BGP theory: It's generally better to have a default-deny policy as an out-filter than it is to have a default-accept with certain prefixes blocked - especially to an upstream ISP. If your network gets a reputation for leaking bad routes into the routing table, then your AS might get blocked by some carriers and exchanges. In general, you want to make sure that your out-filters will not allow ANYTHING out to the public Internet EXCEPT for your specific prefixes. Of course there are exceptions to this - you may be operating a private peering with an organization with which you've agreed to share information that will not be leaked to the global routing table - e.g. prefixes longer than /24 and/or private RFC1918 address ranges, etc. That's completely fine as long as you trust the peer not to export such things into the global table.

Hi Zero,
If lets say I add network 8.8.8.0/22 but my filter will only allow 8.8.8.0/24 to peer A. Will it work OK?
So from what you say if I have an interface on my router 8.8.8.1/24 I should announce it by adding 8.8.8.1/24 to Networks, and this is best practice?

Thanks for your tips.
Q.

Hi Zero,
If lets say I add network 8.8.8.0/22 but my filter will only allow 8.8.8.0/24 to peer A. Will it work OK?
So from what you say if I have an interface on my router 8.8.8.1/24 I should announce it by adding 8.8.8.1/24 to Networks, and this is best practice?

Thanks for your tips.
Q.

No problem.

To answer your question, no - 8.8.8.0/22 is a different prefix than 8.8.8.0/24 - if you want your router to announce 8.8.8.0/24, then you need to explicitly add that prefix to your networks list. Your router can announce 8.8.8.0/22 and 8.8.8.0/24 at the same time - the Internet considers these to be completely independent announcements. Of course the /24 prefix will always win over the /22 prefix for other routers deciding how to reach, say, 8.8.8.97 because the /24 is more specific.

Think of your routing table like a pool with fish swimming around in it - and each fish represents a prefix, such as 8.8.8.0/24
Think of the BGP process like a spear-fisherman who will spear a certain kind of fish, and the networks list is the list of fish he will catch.
If your routing table has 8.8.8.0/24 as an active destination, then BGP will spear that fish and put it in the basket of prefixes being announced. If there are no 8.8.8.0/24 swimming in the waters of the IP routing table, then there will not be any 8.8.8.0/24 fish in the basket. If the route 8.8.8.0/24 is active in the routing table, then the BGP process will announce it. If it's not active in the routing table, then BGP will not announce it.

This match must be exact.

8.8.8.128/25 is a subnet of 8.8.8.0/24, but it's not an exact match - it's only "half a match" which is not a match. Since it doesn't contain 100% of the range specified, BGP will not advertise 8.8.8.0/24 based on 8.8.8.128/25
8.8.8.0/23 is a supernet of 8.8.8.0/24, but it's not an exact match - it CONTAINS a match, but it is not an actual match. Since it's not specific enough, BGP will not announce 8.8.8.0/24 based on a route to 8.8.8.0/23

If you want to originate routes for both your main /22 prefix AND for your /24 subnet of that prefix, then add both prefixes to the networks list.

If one of your router's interfaces is configured as 8.8.8.1/24, then this will cause the route 8.8.8.0/24 to be active in the routing table as a connected route - this is a match, and thus BGP will originate the the /24 prefix. But this won't satisfy the /22 supernet. Usually, the master prefix is "nailed down" by adding a static black hole route to the master prefix.
/ip route add dst=8.8.8.0/22 type=blackhole
This is a good catch-all which discards traffic to addresses that you've not yet assigned anywhere, and it will satisfy the criteria for BGP to originate the /22 prefix into the global routing table.

The difference between ORIGINATING a route, and ANNOUNCING a route:

So using the fisherman analogy again - the networks list tells the fisherman which fish to catch, but the filter rule chains are what determines which fish the fisherman will sell to a particular customer. The fisherman may have many more fish in his basket but he will only sell the ones that match the filter rules. Stretching the analogy further - realize that when the fisherman goes to market, he may have lots of fish that he didn't catch himself. Suppose the fisherman has also bought lots of other fish from other fishermen.

These "purchased fish" are analogous to the routes your router has learned from other ISPs. You definitely do NOT want to allow routes from ISP1 to be sent to ISP2 - this will cause ISP2 to use YOU to reach the Internet for any destination that looks more appealing via your network than any other path available to ISP2. This is the most important reason you need to be very conservative with BGP filters - allow EXACTLY what you intend to advertise, and discard everything that has not matched your list.

How filters work:

Let's say you've added both the /22 and /24 prefixes to your networks list, and added the blackhole /22 route. Now your BGP process has both 8.8.8.0/24 and 8.8.8.0/22 in its collection of prefixes. When speaking to a BGP peer, Your router will use the out-filter to limit what information BGP may send to that peer.

Example 1: This will allow both prefixes 8.8.8.0/22 and 8.8.8.0/24 to be advertised - because the prefix 8.8.8.0/24 matches (is completely contained within) the 8.8.8.0/22 prefix, AND the rule allows matching prefixes whose length is 22-24 bits.


Example 2: This chain would ONLY allow the /22 prefix, and would drop the /24 prefix:
That's because when you don't specify a prefix length, the route must exactly match the prefix length specified.


Example 3:
Another fancy thing you could do is allow any /24 prefix which is a subset of your master /22, but not to allow the /22 prefix itself: In this example, 8.8.12.0/24 would not pass this filter because while it IS a /24 prefix, it is NOT contained in 8.8.8.0/22

I hope this information is helpful for you in understanding how BGP behaves - obviously there is a LOT more to BGP theory, behavior, and network design, but it really helps to have a firm foundational understanding of what's going on under the hood so that you can establish good best practice habits as soon as possible, which will help keep you from painting yourself into a corner that's hard to undo later on down the road.

Is this for MPLS/VPNv4 or for public Internet routing?
If the latter, then you really shouldn't redistribute connected (or static or any other protocol)

Just add networks for the prefixes you wish to announce.

I agree that redistribute static is probably not a good idea, but why not redistribute connected?
It appears that setting networks for BGP is just a complicated way of achieving manually what redistribute connected does automatically...
On our hamnet I just enable redistribute connected and have no BGP network set, I have routing filters to limit the BGP routing to our /8
network.   What is wrong with this?  It appears to work very well, every address added to a router interface is automatically routed everywhere.

This is going to be a bit long and rambly, because at the end of the day, it's not quite as tangible a difference between redistributed BGP routes and properly-originated routes like in other protocols like OSPF and EIGRP for instance.....

But here's the exact difference:
BGP considers equal-prefix-length route candidates with the following decisions:
1) highest-value Weight metric
2) highest-value Local_Preference metric
3) Locally Originated > Aggregated > Redistributed
4) AS-Path length
(many more)

1 is actually not even in the BGP standard - it was introduced by Cisco as a proprietary extension, and has been implemented in other platforms, like Mikrotik. It's just a way to make a route preferable to only the local router, but not affect any other routers in the AS. The route messages don't have a field for weight.

2 is the most important metric in standard BGP - and it's global within a single AS - meaning that if you stamp a prefix with pref=200, then that's going to take priority throughout your entire BGP cloud. This is an administratively-set metric used to dictate routing policy. If you don't set this, then all routes default to pref=100, and get settled by other means, the FIRST of which is:

3 - was the route originated locally, or bundled up in some other group of networks, or was it redistributed.

Thus, a redistributed route is already going to be inferior to any naturally-originated route regardless of any other natural criteria. Basically, when you redistribute a route into a protocol (not just BGP, but ANY protocol) - what you're doing is like hear/say - "I heard that you can get to such-and-such if you come here first" - Locally-originated routes are considered to be authoritative.

I know this seems all hypothetical, pedantic, and "meh, it works, so who cares?" but it's fundamental in understanding how to work with routing protocols in concert with how they're designed to work. If a network is based on some protocol, such as OSPF for instance, then it makes sense to architect the network so that it reflects the behavior of OSPF, instead of trying to shoe-horn some other behavior that doesn't actually reflect the behavior of OSPF. If you know how OSPF is going to handle things, and you're using OSPF, then you can design your network according to the behavior and not get any ugly surprises when something bizarre happens.

So - redistributing connected routes into BGP vs. specifying desired networks to be originated....
It's a little bit academic, I suppose, but basically, BGP is intended to be a high-level protocol that maps the "big picture" of the entire Internet, not an IGP which makes a very detailed map of a regional network and figures out the best way to get from anywhere to anywhere on that map. You can USE it like an IGP in the case of Hamnet because Hamnet is a confederation of islands of control and responsibility. BGP is well-suited for complex policy-based routing between organizations, which is why it fits for HamNet, but it's not well-suited as an IGP - it's slow to converge and it requires lots of explicit configuration. This last point is precisely because BGP is designed to scale to the entire globe. If some guy could just connect into the global BGP table and dump anything into the global table, there would be chaos. It's important that the protocol require you to explicitly tell it WHAT you want to inject into its table, and to whom you share this table, and precisely which components of this table you will share, and how you present those shared components. (i.e. share this route, but make it look really really bad - I don't want anyone to actually USE it unless there's just no other way possible)

Then there's the even more esoteric benefit of establishing good habits. When you do things according to the way they're designed to work, you'll find that you naturally tend to deploy it in ways that work well for it, and that you will avoid bad configurations because they don't work well with the protocol. Following best practice makes you understand the protocol more completely and more able to quickly spot bad design, more quickly troubleshoot things, and avoid painting yourself into a corner or setting yourself up for "whoops" moments later.

If you just tell it to redistribute connected routes, then as you stated, any time you put an IP address on your router - it's going into BGP. If you forget to properly filter your routes, then you could leak wrong information into the table, which can take a bit of time to propagate back out of the table (BGP is relatively slow to converge). It's just easier to not have things floating around in your BGP table that you would never want to advertise anyway. Not only that, but BGP is designed to be a global-scale map of the Internet, so naturally a router in Hong Kong doesn't want to receive an update because an interface in Toronto went down, then up, then down, then up again.... 

Ok I guess the case of hamnet is a bit different from what a internet BGP table would require.
In hamnet we have an "isolated" /8 network in which we use BGP internally.  It is divided mostly in /16
networks for countries, and we further divide that into /22 networks for regions, which have one or
more locations on which one or more smaller subnets may be connected.   The routers on each
location have an AS number, so iBGP within the locations and eBGP between them.

I have experimented with different settings of network in BGP and it has never become clear to me
what is exactly happening and how to avoid some unwanted effects (like advertisement of an unreachable
network).  With redistribute connected it all seems to work OK.

I have not yet noticed any slow convergence, but we have only 115 routes in our country's table at the
moment.   This will be very different from an internet table.

Probably I will study the matter of manual network configuration again, to see if there is really enough of
an advantage to change over to them (and have more work when subnets are added or deleted, which
happens all the time now the network is being expanded more and more)

Of course I agree with you that it is not a good idea to re-distribute all kinds of random routes, only I don't
think a connected subnet is a random route...

Of course I agree with you that it is not a good idea to re-distribute all kinds of random routes, only I don't
think a connected subnet is a random route...

As long as you don't have any interface on the router with an IP address that's out of your range (e.g. a private 192.168.x.x interface?)
This falls into the same realm as the discussion about why it's bad to use your left foot on the brake pedal in a car. All reasons for it just seem academic and disconnected from any reality for teenagers learning to drive, because "it works fine for me" - most parents end up saying "just don't do it 'cause it's wrong, okay?"  ;)
I was in a long thread with a Hamnet user here on the forums (it may have even been you) who was using BGP for some type of topology favoring with local_pref vs. as-prepend, and the details escape me, but I seem to recall that after hitting my head on the wall with my duplication of the person's topology, I just scrapped it and built it from scratch as I would have done on my onw, and things seemed to work better, and it was hard to nail down just exactly why... there was just some rough difference between what I know of best-practice-wise and what the other guy was doing, and I was able to get pretty consistent and comparable behavior in my lab with my own configs. That's why I'm such a "best practice" advocate. Once it becomes ingrained in your brain how to do certain steps, then it's easy to rattle off a topology in a lab or real environment that "just works well" and you encounter less cases of things just acting different than they seem they should.

Of course I agree with you that it is not a good idea to re-distribute all kinds of random routes, only I don't
think a connected subnet is a random route...

As long as you don't have any interface on the router with an IP address that's out of your range (e.g. a private 192.168.x.x interface?)
This falls into the same realm as the discussion about why it's bad to use your left foot on the brake pedal in a car. All reasons for it just seem academic and disconnected from any reality for teenagers learning to drive, because "it works fine for me" - most parents end up saying "just don't do it 'cause it's wrong, okay?"  ;)
I was in a long thread with a Hamnet user here on the forums (it may have even been you) who was using BGP for some type of topology favoring with local_pref vs. as-prepend, and the details escape me, but I seem to recall that after hitting my head on the wall with my duplication of the person's topology, I just scrapped it and built it from scratch as I would have done on my onw, and things seemed to work better, and it was hard to nail down just exactly why... there was just some rough difference between what I know of best-practice-wise and what the other guy was doing, and I was able to get pretty consistent and comparable behavior in my lab with my own configs. That's why I'm such a "best practice" advocate. Once it becomes ingrained in your brain how to do certain steps, then it's easy to rattle off a topology in a lab or real environment that "just works well" and you encounter less cases of things just acting different than they seem they should.

The local networks are not propagated in our BGP routing because of the filters.  I understand that this is an aspect of hamnet that cannot be applied as easily in general.
We just filter everything outside 44.0.0.0/8 except 0.0.0.0/0 and it works OK automatically.   This may not be as easy when you want to route the entire internet or when your network is a mix of RFC1918 subnets and those addresses are also used on networks you do not want to propagate.
Indeed I was the same person of that discussion.   The change to just use "redistribute connected" was made in that same timeframe, when I was trying to get everything working exactly as desired and I had to combat unwanted effects.   That is why I mentioned that I may test it again, because in such situations you sometimes end up making changes that were not required after all.

Right now we are running a very simple config, example from a typical router:
/routing filter
add bgp-communities=44137:10050 chain=hamnet-in set-bgp-local-pref=50
add action=accept chain=hamnet-in prefix=44.0.0.0/8 prefix-length=8-32
add action=accept chain=hamnet-in prefix=0.0.0.0/0
add action=discard chain=hamnet-in
add action=accept chain=hamnet-out prefix=44.0.0.0/8 prefix-length=8-32
add action=accept chain=hamnet-out prefix=0.0.0.0/0
add action=discard chain=hamnet-out
/routing bgp instance
set default as=4220403700 redistribute-connected=yes router-id=44.137.37.1
/routing bgp peer
add default-originate=if-installed hold-time=15s in-filter=hamnet-in name=\
    hilversum out-filter=hamnet-out remote-address=44.137.62.25 remote-as=\
    4220403600 ttl=1
add default-originate=if-installed in-filter=hamnet-in name=gw-44-137 \
    out-filter=hamnet-out remote-address=44.137.61.25 remote-as=4220406100 \
    ttl=1

With this config in all the radio-connected routers, and with the central VPN gateway (gw-44-137) adding community 44137:10050 to the routes it advertises via the VPN tunnels, it all works very nicely.
During my debugging the setting of default-originate was sometimes not correct for all the peer definitions and I was pulling my hair to find why it was asymmetric.
I remember that when using explicit network ranges in BGP I had some problems with routes that were taking precedence over others in unexpected ways, so I have to research that again to determine if the root cause of that problem has been already solved.
I guess I need to have the same network ranges as added to interfaces, and with synchronize=yes, then remove the redistribute-connected, and it should work.
It still adds double maintenance.  It would be convenient when the "IP->addresses" setting page had a reference to the "BGP networks" so that one could simply tick a checkmark when defining an interface address to add the same range as a BGP advertised network too.

I suspect that a naturally-originated prefix may have been winning out over a redistributed prefix because redistributed prefixes lose to aggregate routes and originated routes on the 3rd decision point. Normally, adding and removing prefixes isn't this big of a deal because it's not changing as much. Your organization is using it as an IGP, but given the fact that it's a confederation of individuals, that makes sense.

I think the compromise between redistributing and originating every single prefix (no matter how small) would be to originate an aggregate prefix or to just announce the main prefix at your own borders, and use some other IGP inside your own sphere.

i.e. if Hamnet gives you 44.137.37.0/24 to play with, just announce that prefix at your borders and then route the individual pieces internally w/o announcing every detail of your internal topology to the rest of Hamnet. This is actually the intended regime of BGP because it's an EGP - because if I'm going to drive from Dallas to Houston, I don't need to know that the south bypass highway in Los Angeles just got closed due to an overturned truck.

Again - if the goal is in fact to use BGP as an IGP on Hamnet, then that's how it is, and giving full topology info is going to be required. Also, I'd like to reiterate that I don't see this discussion as telling you what to do, I'm just sharing my understanding about best practices - obviously you're free to do what you want to / need to in your unique circumstances....

I suspect that a naturally-originated prefix may have been winning out over a redistributed prefix because redistributed prefixes lose to aggregate routes and originated routes on the 3rd decision point. Normally, adding and removing prefixes isn't this big of a deal because it's not changing as much. Your organization is using it as an IGP, but given the fact that it's a confederation of individuals, that makes sense.

I think the compromise between redistributing and originating every single prefix (no matter how small) would be to originate an aggregate prefix or to just announce the main prefix at your own borders, and use some other IGP inside your own sphere.

i.e. if Hamnet gives you 44.137.37.0/24 to play with, just announce that prefix at your borders and then route the individual pieces internally w/o announcing every detail of your internal topology to the rest of Hamnet. This is actually the intended regime of BGP because it's an EGP - because if I'm going to drive from Dallas to Houston, I don't need to know that the south bypass highway in Los Angeles just got closed due to an overturned truck.

Again - if the goal is in fact to use BGP as an IGP on Hamnet, then that's how it is, and giving full topology info is going to be required. Also, I'd like to reiterate that I don't see this discussion as telling you what to do, I'm just sharing my understanding about best practices - obviously you're free to do what you want to / need to in your unique circumstances....

I appreciate your comments because they offer another viewpoint and you have experience with BGP in other contexts!
I have started with adding BGP networks instead of using redistribute connected.  I made a small script that outputs the bgp network commands reading the ip address info from a config file (to avoid nasty typo errors), and re-configured 8 routers to do some testing.
It appears to work OK, the only change I see in the tables is the origin of the routes changes from incomplete to igp.   That is a good thing.
The network quickly absorbs the changes, I have not noticed any outages.  Probably my earlier problems had a different cause.

I have experimented with aggregation, both in BGP itself and in route filters (just filtering routes with a long prefix) and as you already suspect, it is not easy because we use BGP as an IGP for our partly meshed network.  On a "leaf node" it is easy to advertise only a larger subnet even when smaller subnets are handed out to users, but as soon as there is other connectivity from a single router or between users, it causes problems that are difficult to predict.  A factor in that is that we, besides radio links, have a central gateway system (to internet) where a subset of the routers are connected via VPN links as a backup.  This makes the network topology a bit weird, a mix between routers connected to their geographical neighbors and a star to a central point.  Difficult to know what prefixes you need and what you don't need to distribute!

We also advertise the /29 networks used to form links between routers, so it is possible to connect to the remote access point when a link is down, assuming an alternative route exists.  This of course leads to "a lot" of routes in the table, but it is handy for debugging and error recovery.  And we have only some 115 routes now, so it should not be a problem.   Maybe later

So for now I announce everything inside the country, and only plan to aggregate at links that cross the border.  Then we see the entire country as a single AS and only send and receive prefixes of 8-16 bits.

I would like to add one correction to my earlier explanations about the BGP path selection process.
I was incorrect about my explanation of Step 3.
Step 3 means "prefer routes that the router has originated itself, over routes that it has learned from eBGP/iBGP" - and it will prefer networks/redistributed routes above aggregates.
In short: trust yourself before trusting some other bgp speaker.

Step 5 is what I was explaining step 3 to be:
origin=IGP > origin=EGP > origin=incomplete

I think of this as "stratum of information source" - so IGP is a stratum 1 - it's the very source of authority for that route. EGP is a form of hear-say, but based on another exterior gateway protocol. (EGP is obsolete and I've never even heard of it being implemented anywhere anymore, but that doesn't mean it doesn't exist somewhere), and incomplete origin means that someone somewhere said that it's reachable, and the BGP originator is merely the last BGP speaker you will encounter before following whatever other protocols might lead you to the destination. In a routing protocol perspective. Following an incomplete route is like buying a plane ticket to reach a destination where there isn't actually any airport. Yeah, you can get to backwater-Maryland via a plane ticket to LAX... but you'll have LOTS of driving to do after that, but the airlines' ticketing system doesn't account for that - it just shows you what the various flights cost, and when they go, etc.

So if I were to get a ham license and some radios and such and hop onto hamnet, would my network be assigned its own ASN (obviously a private ASN) and speak eBGP with my neighboring nodes, or does hamnet use iBGP?

So if I were to get a ham license and some radios and such and hop onto hamnet, would my network be assigned its own ASN (obviously a private ASN) and speak eBGP with my neighboring nodes, or does hamnet use iBGP?

It depends on how it is arranged locally.  There are different routing islands that use local conventions and talk to eachother only via internet tunnels.
Some use BGP over those tunnels, but most tunnels are configured "statically" by using a table of subnets manually maintained at a registration website.
The German/Austrian/Swiss hamnet uses a mix of iBGP/eBGP and I think even OSPF.  AS contain a number of sites in a region.
Here in the Netherlands we use iBGP only within a site (some sites have more than one router), and always eBGP between sites.  So a region has only one site.
An end-user does not need to run BGP if they are a leaf node (connect to only a single site).  So most users just get a directly connected subnet.
However, for users wanting more than one link (radio and/or VPN) it is possible to run BGP and they get their own AS number.   We use 32-bit private AS numbers.
The convention is to use AS numbers formatted as 42xxxyyyyy where xxx is a Mobile Country Code (e.g. 204 for the Netherlands) and yyyyy is a number managed by coordination within the countries only.   Larger countries have several MCCs so enough AS numbers are available.