MikroTik

Community discussions
https://forum.mikrotik.com/

BFD, ipv6 & bgp multihop problem

https://forum.mikrotik.com/viewtopic.php?t=210206

Page 1 of 1

BFD, ipv6 & bgp multihop problem

Posted: Fri Aug 16, 2024 2:57 pm
by joj89
Hi,

I have a setup with OSPFv3 with BFD for distributing loobacks and link-networks, on top of that i am trying to run BGP (multi-hop Loopback to Loopback) with BFD.
All this for IPV6 and a separate instances with ospfv2 + bfd and BGP (multihop Loopback to Loopback) + BFD for IPv4.

BFD works fine for IPv4 and the OSPFv3 on all interfaces.
On BGP ipv6 however it works sometimes... after newly being configured and after reboot it never works.
Both routers on either end of the session sends BFD packets according to the counter.

it dose not matter if i run the loopback on bridge interface or the default lo interface.
if i shut of L3 HW offload or if i am running the link vlan interface directly on hw interface or bridge interface, no difference.

have any one else had this problem or have i done anything just wrong?

my test setup looks something like this. (only labb no prod :P)

Core router ccr2216 (border router something similar but ccr2116+cake config on the interfaces)

export
Code: Select all
/interface bridge
add frame-types=admit-only-vlan-tagged mtu=9000 name=bridge1 protocol-mode=none vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp28-1 ] l2mtu=9216 mtu=9000
set [ find default-name=sfp28-2 ] l2mtu=9216 mtu=9000
set [ find default-name=sfp28-9 ] fec-mode=fec91 l2mtu=9216 mtu=9000
set [ find default-name=sfp28-10 ] fec-mode=fec91 l2mtu=9216 mtu=9000
set [ find default-name=sfp28-11 ] l2mtu=9216 mtu=9000
set [ find default-name=sfp28-12 ] l2mtu=9216 mtu=9000
/interface vlan
add interface=bridge1 mtu=9000 name=vlan3000 vlan-id=3000
add interface=bridge1 mtu=9000 name=vlan3002 vlan-id=3002
add interface=bridge1 mtu=9000 name=vlan3004 vlan-id=3004
add interface=bridge1 mtu=9000 name=vlan3100 vlan-id=3100
add interface=bridge1 mtu=9000 name=vlan3102 vlan-id=3102
/interface bonding
add mode=802.3ad mtu=9000 name=bond1 slaves=sfp28-11,sfp28-12
add mode=802.3ad mtu=9000 name=bond2 slaves=sfp28-9,sfp28-10
/port
set 0 name=serial0
/routing bgp template
set default as=xxxyy disabled=no hold-time=1m input.affinity=alone keepalive-time=20s nexthop-choice=default output.affinity=alone router-id=xxx.xxx.128.229 routing-table=main vrf=main
/routing id
add disabled=no id=xxx.xxx.128.229 name=Loopback select-dynamic-id="" select-from-vrf=main
/routing ospf instance
add disabled=no name=ASxxxyy-IPv4 router-id=Loopback routing-table=main
add disabled=no name=ASxxxyy-IPv6 router-id=Loopback routing-table=main version=3
/routing ospf area
add disabled=no instance=ASxxxyy-IPv4 name=ospf-area-0-IPv4
add disabled=no instance=ASxxxyy-IPv6 name=ospf-area-0-IPv6
/interface bridge port
add bridge=bridge1 frame-types=admit-only-vlan-tagged interface=bond1
add bridge=bridge1 frame-types=admit-only-vlan-tagged interface=sfp28-1
add bridge=bridge1 frame-types=admit-only-vlan-tagged interface=sfp28-2
add bridge=bridge1 frame-types=admit-only-vlan-tagged interface=bond2
/ip firewall connection tracking
set udp-timeout=10s
/interface bridge vlan
add bridge=bridge1 tagged=bond1,bridge1 vlan-ids=3000
add bridge=bridge1 tagged=sfp28-1,bridge1 vlan-ids=3002
add bridge=bridge1 tagged=sfp28-2,bridge1 vlan-ids=3004
add bridge=bridge1 tagged=bond2,bridge1 vlan-ids=3100
add bridge=bridge1 tagged=bond2,bridge1 vlan-ids=3102
/interface ethernet switch
set 0 l3-hw-offloading=yes
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether1 network=192.168.88.0
add address=xxx.xxx.128.229 interface=bridge1 network=xxx.xxx.128.229
add address=xxx.xxx.128.2/30 interface=vlan3000 network=xxx.xxx.128.0
add address=xxx.xxx.128.9/30 interface=vlan3002 network=xxx.xxx.128.8
add address=xxx.xxx.128.17/30 interface=vlan3004 network=xxx.xxx.128.16
add address=xxx.xxx.128.66/28 interface=vlan3100 network=xxx.xxx.128.64
add address=xxx.xxx.128.82/28 interface=vlan3102 network=xxx.xxx.128.80
/ip firewall address-list
add address=xxx.xxx.128.0/26 list=bgp_allow_bfd_ipv4
add address=xxx.xxx.128.64/26 list=bgp_allow_bfd_ipv4
add address=xxx.xxx.128.192/26 list=bgp_allow_bfd_ipv4
/ip route
add blackhole comment="Blackhole, Default Route" disabled=no distance=1 dst-address=0.0.0.0/0 gateway="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=0.0.0.0/8
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=172.16.0.0/12
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=192.168.0.0/16
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=10.0.0.0/8
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=169.254.0.0/16
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=127.0.0.0/8
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=224.0.0.0/4
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=198.18.0.0/15
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=192.0.0.0/24
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=192.0.2.0/24
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=198.51.100.0/24
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=203.0.113.0/24
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=100.64.0.0/10
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=240.0.0.0/4
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=192.88.99.0/24
add blackhole comment="Blackhole route for RFC6890 (limited broadcast)" disabled=no dst-address=255.255.255.255/32
/ipv6 route
add blackhole comment="Blackhole route for RFC6890" disabled=no dst-address=::1/128
add blackhole comment="Blackhole route for RFC6890" disabled=no dst-address=::/128
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=64:ff9b::/96
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=::ffff:0.0.0.0/96
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=100::/64
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2001::/23
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2001::/32
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2001:2::/48
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2001:db8::/32
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2001:10::/28
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=2002::/16
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no distance=1 dst-address=fc00::/7 gateway="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add blackhole comment="Blackhole route for RFC6890 (aggregated)" disabled=no dst-address=fe80::/10
/ipv6 address
add address=2xxx:xxxx:0:1::2455/128 advertise=no interface=bridge1
add address=2xxx:xxxx:0:c:0:2454:2455:2/126 advertise=no interface=vlan3000
add address=2xxx:xxxx:0:e:0:2452:2455:1/126 advertise=no interface=vlan3002
add address=2xxx:xxxx:0:10:0:2453:2455:1/126 advertise=no interface=vlan3004
add address=2xxx:xxxx:0:11:0:2455:3100:1 advertise=no interface=vlan3100
add address=2xxx:xxxx:0:12:0:2455:3102:1 advertise=no interface=vlan3102
/ipv6 firewall address-list
add address=2xxx:xxxx::/56 list=allow_bfd_ipv6
add address=fe80::/10 list=allow_bfd_ipv6
/routing bfd configuration
add address-list=bgp_allow_bfd_ipv4 disabled=no
add address-list=allow_bfd_ipv6 disabled=no
add disabled=no
/routing bgp connection
add address-families=ip as=xxxyy cluster-id=1.1.1.1 connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp name=\
    BBR2455-to-BBR2454-IPv4 nexthop-choice=default output.affinity=alone remote.address=xxx.xxx.128.228/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-AGR2461 \
    nexthop-choice=default output.affinity=alone .default-originate=always remote.address=xxx.xxx.128.233/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-AGR2460 \
    nexthop-choice=default output.affinity=alone .default-originate=always remote.address=xxx.xxx.128.232/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-AGR2459 \
    nexthop-choice=default output.affinity=alone .default-originate=always remote.address=xxx.xxx.128.231/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-AGR2458 \
    nexthop-choice=default output.affinity=alone .default-originate=always remote.address=xxx.xxx.128.230/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-BDR2453-IPv4 \
    nexthop-choice=default output.affinity=alone remote.address=xxx.xxx.128.227/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ip as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=xxx.xxx.128.229 .role=ibgp-rr name=BBR2455-to-BDR2452-IPv4 \
    nexthop-choice=default output.affinity=alone remote.address=xxx.xxx.128.226/32 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=main
add address-families=ipv6 as=xxxyy cluster-id=1.1.1.1 connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=2xxx:xxxx:0:1::2455 .role=ibgp multihop=yes \
    name=BBR2455-to-BBR2454-IPv6 nexthop-choice=default output.affinity=alone remote.address=2xxx:xxxx:0:1::2454/128 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes \
    vrf=main
add address-families=ipv6 as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=2xxx:xxxx:0:1::2455 .role=ibgp-rr multihop=yes name=\
    BBR2455-to-BDR2452-IPv6 nexthop-choice=default output.affinity=alone remote.address=2xxx:xxxx:0:1::2452/128 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=\
    main
add address-families=ipv6 as=xxxyy connect=yes disabled=no hold-time=1m input.affinity=alone keepalive-time=20s listen=yes local.address=2xxx:xxxx:0:1::2455 .role=ibgp-rr multihop=yes name=\
    BBR2455-to-BDR2453-IPv6 nexthop-choice=default output.affinity=alone remote.address=2xxx:xxxx:0:1::2453/128 .as=xxxyy router-id=xxx.xxx.128.229 routing-table=main templates=default use-bfd=yes vrf=\
    main
/routing ospf interface-template
add area=ospf-area-0-IPv4 disabled=no networks=xxx.xxx.128.229/32 passive
add area=ospf-area-0-IPv6 disabled=no networks=2xxx:xxxx:0:1::2455/128 passive
add area=ospf-area-0-IPv4 cost=20 disabled=no interfaces=vlan3000 networks=xxx.xxx.128.0/30 type=ptp use-bfd=yes
add area=ospf-area-0-IPv6 cost=20 disabled=no interfaces=vlan3000 networks=2xxx:xxxx:0:c:0:2454:2455:0/126 type=ptp use-bfd=yes
add area=ospf-area-0-IPv4 cost=100 disabled=no interfaces=vlan3002 networks=xxx.xxx.128.8/30 type=ptp use-bfd=yes
add area=ospf-area-0-IPv6 cost=100 disabled=no interfaces=vlan3002 networks=2xxx:xxxx:0:e:0:2452:2455:0/126 type=ptp use-bfd=yes
add area=ospf-area-0-IPv4 cost=100 disabled=no interfaces=vlan3004 networks=xxx.xxx.128.16/30 type=ptp use-bfd=yes
add area=ospf-area-0-IPv6 cost=100 disabled=no interfaces=vlan3004 networks=2xxx:xxxx:0:10:0:2453:2455:0/126 type=ptp use-bfd=yes
add area=ospf-area-0-IPv4 cost=50 disabled=no interfaces=vlan3100 networks=xxx.xxx.128.64/28 priority=200 use-bfd=yes
add area=ospf-area-0-IPv6 cost=50 disabled=no interfaces=vlan3100 networks=2xxx:xxxx:0:11::/64 priority=200 use-bfd=yes
add area=ospf-area-0-IPv4 cost=51 disabled=no interfaces=vlan3102 networks=xxx.xxx.128.80/28 priority=200 use-bfd=yes
add area=ospf-area-0-IPv6 cost=51 disabled=no interfaces=vlan3102 networks=2xxx:xxxx:0:12::/64 priority=200 use-bfd=yes
routing/bfd/session/ print
Code: Select all

20   multihop=yes vrf=main remote-address=2xxx:xxxx:0:1::2452 local-address=2xxx:xxxx:0:1::2455 state=down state-changes=0 desired-tx-interval=200ms actual-tx-interval=1s required-min-rx=200ms remote-min-rx=1us multiplier=5 packets-rx=0 
     packets-tx=5

Re: BFD, ipv6 & bgp multihop problem

Posted: Wed Oct 23, 2024 4:59 pm
by joj89
Anyone?

Re: BFD, ipv6 & bgp multihop problem

Posted: Wed Oct 23, 2024 6:39 pm
by Larsa
I’d say it’s way too hard to figure out the network topology without a clear network diagram.

Re: BFD, ipv6 & bgp multihop problem

Posted: Tue Nov 26, 2024 3:23 am
by PhilB
I'm seeing similar.

I think the problem is that MT BFD is sending packets from a physical interface IPv6 address and not the configured local IPv6 address for the BGP peer, even though BFD figures this out just fine for IPv4 multi-hop peers.

Try doing a packet sniff with IPv6 Address set to one of your loopback peer addresses, proto 17 (UDP), and dst port 4784 and see if you see the same as me.

When I do this, I can see inbound BFD from a JunOS neighbour with the expected source and destination addresses, and the mikrotik sending from one of the physical interface IPv6 addresses in the OSPF area instead of the BGP local-address :?

If I add an IPv6 NAT rule like this (where xxxx:xx::x is the local loopback IP of the MT):
Code: Select all
/ipv6 firewall nat add action=src-nat chain=srcnat dst-address-list=IPv6Loopbacks dst-port=4784 protocol=udp src-address=!xxxx:xx::x/128 src-port=49152-65535 to-address=xxxx:xx::x/128
to force the mikrotik to rewrite the packets on their way out, BFD comes right up. Obviously you now have the expense of rewriting those packets. :roll:

I guess I'll have another look at my config to make sure it's really not something stupid I have done, and then I'll try reporting a bug.
All times are UTC+02:00
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/