Multi-hop BGP not forwarding (for MetalLB load balancing)

pubudeux · Sun Oct 25, 2020 8:11 pm

Hey all - I am trying to configure BGP with MetalLB to achieve load-balancing in my Kubernetes cluster.

So far, I have set up the following:

My RouterBoard has a BGP instance 65530 on the router

In between the Kubernetes cluster and the router is a managed switch with L3 routing between VLANs, so the BGP peers are set up as multi-hop
Here are the routes on the L3 switch: (the cluster nodes that I'm trying to load balance are in the VLAN20 with subnet 10.20.0.0/16)

Protocol	Destination Network	Next Hop	Distance	Metric	Interface Name
Static 0.0.0.0/0 10.0.0.1 1 0 VLAN1
Connected 10.0.0.0/24 10.0.0.2 0 1 VLAN1 
Connected 10.20.0.0/16 10.20.0.2 0 1 VLAN20

Here is the BGP peer setup:

/routing bgp peer
add multihop=yes name=peer1 remote-address=10.20.10.1 remote-as=65529 ttl=default
add multihop=yes name=peer2 remote-address=10.20.10.2 remote-as=65529 ttl=default
add multihop=yes name=peer4 remote-address=10.20.10.4 remote-as=65529 ttl=default

There is a connection established between all the peers and they successfully advertise routes:

 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                          x.x.x.x                1
 1 ADC  10.0.0.0/16        10.0.0.1        bridge                    0
 2 A S  10.10.0.0/16                       10.0.0.2                  1
 3 A S  10.20.0.0/16                       10.0.0.2                  1
 4 A S  10.30.0.0/16                       10.0.0.2                  1
 5 ADC  x.x.x.x/x      x.x.x.x     ether1                    0
 6 ADC  172.16.0.0/16      172.16.0.0      bridge                    0
 7 ADb  172.16.100.1/32                    10.20.10.1               20
 8  Db  172.16.100.1/32                    10.20.10.2               20
 9  Db  172.16.100.1/32                    10.20.10.4               20

Note the 172.16.100.1 route dynamically added by the BGP peers. When I ping this address it hits the router, but immediately drops the packet. I am trying to get this IP to route to the peer IP in order to load balance connections.

Currently this 172 IP is not working, i have also tried a cidr in the VLAN 20 subnet, which didnt work, as well as a cidr in the same overall subnet. Not sure what I'm missing.

Multi-hop BGP not forwarding (for MetalLB load balancing)

Multi-hop BGP not forwarding (for MetalLB load balancing)

Who is online