BGP Failover issues

victorsoares · Fri Aug 10, 2018 5:44 pm

Hello everyone.

Let me try to explain my scenario. I have two BGP sessions from two different AS coming to my CCR1072-1G-8S+. Both of them are active and working. They both have the same speeds (3Gbps full).
I want to make BGP 1 (superimagem) primary and BGP 2 (nip) secondary. I have set BGP weight, local preference and MED on filters for both iBGP's. I'm going to post my filters and my BGP configurations below. What am I doing wrong?

Filters:

# aug/10/2018 11:27:40 by RouterOS 6.42.6
# software id = SLL9-BT75
#
# model = CCR1072-1G-8S+
# serial number = --------------
/routing filter
add action=accept chain=ebgp_superimagem_v4_in prefix=0.0.0.0/0 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=discard chain=ebgp_superimagem_v4_in set-bgp-local-pref=200 \
    set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept bgp-communities="" chain=ebgp_superimagem_v4_out prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=200 set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=accept chain=ebgp_superimagem_v4_out disabled=yes prefix=\
    XXX.XXX.XXX.X/24 set-bgp-local-pref=100 set-bgp-weight=300
add action=reject chain=ebgp_superimagem_v4_out prefix=XXX.XXX.XXX.X/30
add action=discard chain=ebgp_superimagem_v4_out set-bgp-local-pref=200 \
    set-bgp-med=10 set-bgp-weight=300
add action=accept chain=ebgp_nip_v4_in prefix=0.0.0.0/0 set-bgp-local-pref=0 \
    set-bgp-med=50 set-bgp-weight=100
add action=discard chain=ebgp_nip_v4_in set-bgp-local-pref=0 set-bgp-med=50 \
    set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=accept chain=ebgp_nip_v4_out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=0 set-bgp-med=50 set-bgp-weight=100
add action=discard chain=ebgp_nip_v4_out set-bgp-local-pref=0 set-bgp-med=50 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=0.0.0.0/0 set-bgp-local-pref=0 \
    set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/23 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/23 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/23 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-in prefix=XXX.XXX.XXX.X/24 set-bgp-local-pref=\
    0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=discard chain=ibgp-mpv-in set-bgp-local-pref=0 set-bgp-prepend=3 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/23 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/23 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out prefix=XXX.XXX.XXX.X/24 \
    set-bgp-local-pref=0 set-bgp-med=20 set-bgp-prepend=3 set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/23 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=XXX.XXX.XXX.X/24 \
    set-bgp-weight=100
add action=accept chain=ibgp-mpv-out disabled=yes prefix=0.0.0.0/0 \
    set-bgp-weight=100
add action=discard chain=ibgp-mpv-out set-bgp-weight=100

BGP:

# aug/10/2018 11:28:00 by RouterOS 6.42.6
# software id = SLL9-BT75
#
# model = CCR1072-1G-8S+
# serial number = --------------
/routing bgp instance
set default as=XXXXX client-to-client-reflection=no router-id=X.X.X.X
add as=XXXXX ignore-as-path-len=yes name=superimagem router-id=\
    X.X.X.X
add as=1 client-to-client-reflection=no name=interno redistribute-connected=\
    yes router-id=X.X.X.X
add as=XXXXX ignore-as-path-len=yes name=nip router-id=X.X.X.X
/routing bgp network
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/23 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/23 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/23 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
add network=XXX.X.XXX.0/24 synchronize=no
/routing bgp peer
add in-filter=ebgp_nip_v4_in instance=nip multihop=yes name=BGP-NIP \
    nexthop-choice=force-self out-filter=ebgp_nip_v4_out remote-address=\
    XXX.XXX.XXX.X remote-as=XXXXX ttl=default update-source=vlan_nip_ubatuba
add in-filter=ebgp_superimagem_v4_in instance=superimagem multihop=yes name=\
    BGP-SUPERIMAGEM-V4 out-filter=ebgp_superimagem_v4_out remote-address=\
    XXX.XXX.XXX.X remote-as=XXXXX ttl=default update-source=\
    vlan_super_ubatuba
add in-filter=ibgp-mpv-in instance=interno multihop=yes name=BGP-INTERNO \
    nexthop-choice=force-self out-filter=ibgp-mpv-out remote-address=\
    XXX.XXX.XXX.X remote-as=1 ttl=default update-source=vlan_caragua
add disabled=yes in-filter=ibgp-mpv-in name=BGP-CARAGUA out-filter=\
    ibgp-mpv-out remote-address=191.7.144.1 remote-as=263320 ttl=default

paulct · Fri Aug 10, 2018 6:00 pm

What is the issue?

victorsoares · Fri Aug 10, 2018 6:05 pm

I'm sorry, completely forgot to talk about the issue itself. The thing is that when both sessions are up, BGP 2 (NIP) gets more traffic that BGP 1, and it should be the opposite.

StubArea51 · Fri Aug 10, 2018 8:43 pm

So we need to understand whether you're trying to influence traffic coming in from your upstream (normally a "download" for a user) or traffic from your network going outbound (normally an "upload" for a user)

There are different ways to influence BGP depending on what you're trying to do.

victorsoares · Sun Aug 12, 2018 12:34 am

Traffic coming upstream, downstream is pretty much sorted at this point. I just want that RX on my links goes to BGP 1 (superimagem)

TFyre · Sun Aug 12, 2018 12:06 pm

What you are trying to achieve is close to impossible (50/50 equal balance).

Your inbound traffic is not as much affected by yourself as by your upstream peers & their peers.

Referring to: https://wiki.mikrotik.com/wiki/Manual:B ... _Algorithm

Generally trying to force traffic flow in a certain path is not advised since you can sometimes introduce latency or other adverse effects.

Possible solutions (to affect how the rest of the world sees your network):

BGP Prepends to either upstream peer. This will let your network LOOK like its X hops further, so the another path might be selected as the best path
Route de-aggregation. You can advertise more specific prefixes to either of the 2 upstreams

Route de-aggregation:
Lets assume you have 155.155.0.0/22 as your assigned prefix
You advertise 155.155.0.0/22 to superimagem & nip
You advertise 155.155.0.0/23 to superimagem
You advertise 155.155.2.0/23 to nip

Depending on what sits behind either of the 2x /23's traffic would be somewhat more balanced. You also get the full fail over in case either upstream goes down since you advertise your full prefix to both

victorsoares · Mon Aug 13, 2018 3:47 pm

Thank you for your help. I already tried prepending, and it got me to a certain point, but I think that the de-aggregation might be the best way. I'll try this and see how it works. Thanks again for the help!

BGP Failover issues

BGP Failover issues

Re: BGP Failover issues

Re: BGP Failover issues

Re: BGP Failover issues

Re: BGP Failover issues

Re: BGP Failover issues

Re: BGP Failover issues

Who is online