ROSv7 BGP-VPLS Setup Help

JdccDevel · Wed Mar 09, 2022 9:58 pm

Has anyone been able to get BGP-VPLS working on ROSv7 ? If so, could you help and/or share your config?

I'm trying to get a BGP-VPLS setup working on some CCR2004 Routers. As of right now, they're running 7.1.3
Unfortunately, they came with ROSv7 from the factory, so as far as I can tell, there's no way to downgrade them to ROSv6.
I've been using this wiki article for reference, and following as best I can while modifying the BGP config for the changes in ROSv7

https://wiki.mikrotik.com/wiki/Manual:BGP_based_VPLS

Here's where I'm at right now:

I've got OSPF and MPLS working, and BGP Sessions established. (I'm only working with 3 routers so far, but there will be a lot more later.)

As far as I can tell, one of the BGP Instances is setup as a Route Reflector, (With the disturbing lack of feedback / status / debugging info from the BGP setup, it's really hard to know for sure.)
I've setup my BGP-VPLS instance, and .... nothing. (As far as I can see, considering I can't see what's going on with BGP at all.)

Here's the sections of my config that appear to be relevant for R1 (172.28.0.1)

/interface bridge
add comment="VPLS link" name="Multisite Core VPLS"
/routing bgp template
set default address-families=ip cluster-id=172.28.0.1 disabled=no router-id=172.28.0.1 routing-table=main
add address-families=ip,l2vpn as=65100 cluster-id=172.28.0.1 disabled=no name=iBGP-Template router-id=172.28.0.1 routing-table=main
/interface bridge port
add bridge="Multisite Core VPLS" interface=ether11
/interface vpls bgp-vpls
add bridge="Multisite Core VPLS" bridge-horizon=1 comment="Multisite Core VLAN" disabled=no export-route-targets=65100:1 import-route-targets=65100:1 name="Multisite Core" pw-control-word=default pw-l2mtu=1500 pw-type=raw-ethernet rd=65100:1 site-id=1
/routing bgp connection
add address-families=ip,l2vpn as=65100 cluster-id=172.28.0.1 disabled=no listen=no local.address=172.28.0.1 .role=ibgp-rr name=Peer remote.address=172.28.1.1 router-id=172.28.0.1 routing-table=main templates=iBGP-Template
add address-families=ip,l2vpn as=65100 cluster-id=172.28.0.1 disabled=no listen=yes local.address=172.28.0.1 .role=ibgp-rr name=CLIENT-CPE remote.address=172.28.0.0/24 router-id=172.28.0.1 routing-table=main templates=iBGP-Template

The other router (172.28.1.1) is identical for these options, except replace 172.28.0.x with 172.28.1.x

Any help and/or ideas you might have on what I can do to get this working would be appreciated.

TFyre · Sun Mar 13, 2022 2:39 pm

Looks like the BGP-VPLS session with (ibgp-rr-client) does not send its local bgp-vpls configuration to the RR

Network5 · Wed Mar 23, 2022 1:38 pm

Hi,
I'm trying to set up a BGP-VPLS tunnel to our backbone from a ROSv7 router and ROSv6 backbone. (We have some backbone nodes running on CCR2004s and they are restarting randomly so, the idea is to upgrade to v7, where they get rid of that problem.)

Now:
OSPF is working fine,
MPLS/LDP it seems is working fine,
BGP is established.

Non-BGP VPLS is running but doesn't work, (actually I can restart the router v7 with a ping) but there is no way to set up BGP signalled VPLS.

In the v7 BPG peer is missing the update-source parameter. In v6 is used to point to loopback bridge.

clambert · Wed Mar 23, 2022 2:29 pm

I think that everything related to MP-BGP is not ready yet, but there is no clear information from Mikrotik.
For a while it was possible to follow the state of development through the following link:
https://help.mikrotik.com/docs/display/ ... col+Status
But it has not received updates with the release of the latest versions of RouterOS v7.

StubArea51 · Wed Mar 23, 2022 3:53 pm

In the v7 BPG peer is missing the update-source parameter. In v6 is used to point to loopback bridge.

update-source is now local.address.

here is an example:

name="IPv4-rtr-edge-02.jan1.us.ipa.net" 
     remote.address=100.127.32.2/32 .as=65012 
     local.address=100.127.32.3 .role=ibgp-rr 
     connect=yes listen=yes routing-table=main router-id=100.127.32.3 templates=default as=65012 address-families=ip cisco-vpls-nlri-len-fmt=auto-bits 
     output.redistribute=connected .filter-chain=ipv4-bgp-rr-deny-default-match-communities .network=bgp-networks 
     input.filter=ipv4-bgp-rr-permit-default-match-communities

CyB0rg · Sun May 15, 2022 8:35 pm

Hi,
I'm trying to set up a BGP-VPLS tunnel to our backbone from a ROSv7 router and ROSv6 backbone. (We have some backbone nodes running on CCR2004s and they are restarting randomly so, the idea is to upgrade to v7, where they get rid of that problem.)

Now:
OSPF is working fine,
MPLS/LDP it seems is working fine,
BGP is established.

Non-BGP VPLS is running but doesn't work, (actually I can restart the router v7 with a ping) but there is no way to set up BGP signalled VPLS.

In the v7 BPG peer is missing the update-source parameter. In v6 is used to point to loopback bridge.

L2VPN distribution in BGP was broken in 7.1 and 7.2.
It kind of works in the latest testing 7.3beta40.

I have BGP signalled VPLS up and running between 2 peers, however the route reflection does not work as expected therefore full mesh VPLS between all the participating peers is not possible

Really sad that they have released v7 as "stable" without all routing protocols working.

C.

clambert · Mon May 16, 2022 5:35 am

In the post viewtopic.php?t=185574#p930840, @mrz reported that version ROS v7.3beta40 would have fixed BGP signaled VPLS. However, the first tests I have done have not been successful. I see the connections being established and disconnected continuously.

mjperry82 · Wed May 25, 2022 1:53 am

I do think 7.3beta40 solved the issue with BGP signaling of VPLS connections. But I think there are still some cross platform issues. We are trying to run VPLS between Mikrotik and Juniper. The Lab device running 7.3beta40 will now connect to other Mikrotik PE's, but not to a Juniper PE. All Mikrotik PE's get full mesh VPLS interfaces, and the Juniper PE shows a connection to all of the Mikrotik PE's, but the ROSv7 PE doesn't show a connection to the Juniper PE. The ROSv7 PE also doesn't mark the vpls interfaces as BGP. Perhaps 7.3beta40 is sending BGP signaling, but not correctly making use of BGP signaling? All of the PEs connect through Juniper P1 which is also the route reflector.

Juniper P1
vMX 18.2R1.9

set version 18.2R1.9
set system host-name P1
set system syslog user * any emergency
set system syslog file messages any notice
set system syslog file messages authorization info
set system syslog file interactive-commands interactive-commands any
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/30
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 10.0.0.5/30
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces ge-0/0/2 unit 0 family inet address 10.0.0.9/30
set interfaces ge-0/0/2 unit 0 family mpls
set interfaces ge-0/0/3 unit 0 family inet address 10.0.0.13/30
set interfaces ge-0/0/3 unit 0 family mpls
set interfaces lo0 unit 0 family inet address 10.1.1.5/32
set routing-options router-id 10.1.1.5
set protocols rsvp interface ge-0/0/0.0 
set protocols rsvp interface ge-0/0/1.0
set protocols rsvp interface ge-0/0/2.0
set protocols rsvp interface ge-0/0/3.0
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols mpls interface ge-0/0/2.0
set protocols mpls interface ge-0/0/3.0
set protocols bgp traceoptions file bgp-debug
set protocols bgp traceoptions flag open
set protocols bgp local-address 10.1.1.5
set protocols bgp local-as 65500
set protocols bgp group ibgp-route-reflect type internal
set protocols bgp group ibgp-route-reflect family inet unicast
set protocols bgp group ibgp-route-reflect family l2vpn signaling
set protocols bgp group ibgp-route-reflect cluster 1.1.1.1
set protocols bgp group ibgp-route-reflect neighbor 10.1.1.1 peer-as 65500
set protocols bgp group ibgp-route-reflect neighbor 10.1.1.2 peer-as 65500
set protocols bgp group ibgp-route-reflect neighbor 10.1.1.3 peer-as 65500
set protocols bgp group ibgp-route-reflect neighbor 10.1.1.4 peer-as 65500
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0
set protocols ospf area 0.0.0.0 interface ge-0/0/3.0
set protocols ospf area 0.0.0.0 interface lo0.0 passive
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
set protocols ldp interface ge-0/0/2.0
set protocols ldp interface ge-0/0/3.0
set protocols ldp interface lo0.0

Juniper PE1
vMX 18.2R1.9

set version 18.2R1.9
set system host-name vMX-PE1
set system syslog user * any emergency
set system syslog file messages any notice
set system syslog file messages authorization info
set system syslog file interactive-commands interactive-commands any
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.2/30
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/9 encapsulation ethernet-vpls
set interfaces ge-0/0/9 unit 0 family vpls
set interfaces lo0 unit 0 family inet address 10.1.1.1/32
set routing-options router-id 10.1.1.1
set protocols rsvp interface ge-0/0/0.0
set protocols mpls label-switched-path pe1-to-pe2 from 10.1.1.1
set protocols mpls label-switched-path pe1-to-pe2 to 10.1.1.2
set protocols mpls label-switched-path pe1-to-pe3 from 10.1.1.1
set protocols mpls label-switched-path pe1-to-pe3 to 10.1.1.3
set protocols mpls label-switched-path pe1-to-pe4 from 10.1.1.1
set protocols mpls label-switched-path pe1-to-pe4 to 10.1.1.4
set protocols mpls interface ge-0/0/0.0
set protocols bgp local-address 10.1.1.1
set protocols bgp local-as 65500
set protocols bgp group ibgp-group type internal
set protocols bgp group ibgp-group family inet unicast
set protocols bgp group ibgp-group family l2vpn signaling
set protocols bgp group ibgp-group neighbor 10.1.1.5 peer-as 65500
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface lo0.0
set routing-instances vpls-1 instance-type vpls
set routing-instances vpls-1 interface ge-0/0/9.0
set routing-instances vpls-1 route-distinguisher 10.1.1.1:1
set routing-instances vpls-1 vrf-target target:1.1.1.1:1
set routing-instances vpls-1 protocols vpls traceoptions file vpls-debug
set routing-instances vpls-1 protocols vpls traceoptions flag all
set routing-instances vpls-1 protocols vpls control-word
set routing-instances vpls-1 protocols vpls site-range 30
set routing-instances vpls-1 protocols vpls no-tunnel-services
set routing-instances vpls-1 protocols vpls site vpls-1-pe1 site-identifier 17
set routing-instances vpls-1 protocols vpls site vpls-1-pe1 interface ge-0/0/9.0

Mikrotik CHR
ROSv6 PE2

# may/24/2022 21:15:29 by RouterOS 6.48.5
# software id = 
#
#
#
/interface bridge
add name=VPLS-BR-1
add name=lo0
/mpls traffic-eng tunnel-path
add name=dynamic
/interface traffic-eng
add disabled=no from-address=10.1.1.2 name=pe2-to-pe1 primary-path=dynamic record-route=yes to-address=10.1.1.1
add disabled=no from-address=10.1.1.2 name=pe2-to-pe3 primary-path=dynamic record-route=yes to-address=10.1.1.3
add disabled=no from-address=10.1.1.2 name=pe2-to-pe4 primary-path=dynamic record-route=yes to-address=10.1.1.4
/routing bgp instance
set default as=65500 router-id=10.1.1.2
/routing ospf instance
set [ find default=yes ] mpls-te-area=backbone mpls-te-router-id=lo0 router-id=10.1.1.2
/interface bridge port
add bridge=VPLS-BR-1 interface=ether4
/interface vpls bgp-vpls
add bridge=VPLS-BR-1 export-route-targets=1.1.1.1:1 import-route-targets=1.1.1.1:1 name=vpls-1 route-distinguisher=10.1.1.2:1 site-id=2
/ip address
add address=10.0.0.6/30 interface=ether1 network=10.0.0.4
add address=10.1.1.2 interface=lo0 network=10.1.1.2
/mpls ldp
set enabled=yes lsr-id=10.1.1.2 transport-address=10.1.1.2
/mpls ldp interface
add interface=lo0
add interface=ether1
/mpls traffic-eng interface
add bandwidth=1Gbps interface=ether1
/routing bgp peer
add address-families=ip,l2vpn name=RR-10.1.1.5 remote-address=10.1.1.5 remote-as=65500 update-source=lo0
/routing ospf interface
add interface=lo0 passive=yes
add interface=ether1
/routing ospf network
add area=backbone network=10.1.1.2/32
add area=backbone network=10.0.0.4/30
/system identity
set name=ROSv6-PE2

Mikrotik CHR
ROSv7-PE4 (Interfaces re-ordered to get ROS and Eve-NG to agree on names)

# may/24/2022 21:50:33 by RouterOS 7.3beta40
# software id = 
#
/interface bridge
add name=VPLS-BR-1
add name=lo0
/interface ethernet
set [ find default-name=ether3 ] name=ether1
set [ find default-name=ether1 ] name=ether2
set [ find default-name=ether2 ] name=ether3
/mpls traffic-eng path
add name=dynamic
/port
set 0 name=serial0
/routing ospf instance
add disabled=no mpls-te-address=10.1.1.4 mpls-te-area=0.0.0.0 name=ospf2 router-id=10.1.1.4 routing-table=main
/routing ospf area
add disabled=no instance=ospf2 name=backbone
/interface bridge port
add bridge=VPLS-BR-1 interface=ether4
/interface vpls bgp-vpls
add bridge=VPLS-BR-1 export-route-targets=1.1.1.1:1 import-route-targets=1.1.1.1:1 name=vpls-1 pw-control-word=enabled pw-type=vpls rd=10.1.1.4:1 site-id=20
/ip address
add address=10.1.1.4 interface=lo0 network=10.1.1.4
add address=10.0.0.14/30 interface=ether1 network=10.0.0.12
/mpls ldp
add afi=ip lsr-id=10.1.1.4 transport-addresses=10.1.1.4
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ip interface=ether1 transport-addresses=10.1.1.4
/mpls traffic-eng interface
add bandwidth=1Gbps interface=ether1
/mpls traffic-eng tunnel
add from-address=10.1.1.4 name=pe4-to-pe1 primary-path=dynamic to-address=10.1.1.1
add from-address=10.1.1.4 name=pe4-to-pe2 primary-path=dynamic to-address=10.1.1.2
add from-address=10.1.1.4 name=pe4-to-pe3 primary-path=dynamic to-address=10.1.1.3
/routing bgp connection
add address-families=ip,l2vpn as=65500 local.address=10.1.1.4 .role=ibgp name=RR-10.1.1.5 remote.address=10.1.1.5 .as=65500 router-id=10.1.1.4
/routing ospf interface-template
add area=backbone disabled=no interfaces=lo0 passive
add area=backbone disabled=no interfaces=ether1
/system identity
set name=ROSv7-PE4

Juniper PE1
VPLS connections:

root@vMX-PE1> show vpls connections 
Layer-2 VPN connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present 
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                      
LD -- local site signaled down   CF -- call admission control failure      
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not available
BK -- Backup connection          ST -- Standby connection
PF -- Profile parse failure      PB -- Profile busy
RS -- remote site standby        SN -- Static Neighbor
LB -- Local site not best-site   RB -- Remote site not best-site
VM -- VLAN ID mismatch           HS -- Hot-standby Connection

Legend for interface status 
Up -- operational                       
Dn -- down

Instance: vpls-1
Edge protection: Not-Primary
  Local site: vpls-1-pe1 (17)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     May 24 19:52:52 2022           1
      Remote PE: 10.1.1.2, Negotiated control-word: Yes (Null)
      Incoming label: 262170, Outgoing label: 39
      Local interface: lsi.1048580, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls vpls-1 local site 17 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No
    3                         rmt   Up     May 24 20:38:20 2022           1
      Remote PE: 10.1.1.3, Negotiated control-word: Yes (Null)
      Incoming label: 262171, Outgoing label: 40
      Local interface: lsi.1048581, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls vpls-1 local site 17 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No
    20                        rmt   Up     May 24 20:43:15 2022           1
      Remote PE: 10.1.1.4, Negotiated control-word: Yes (Null)
      Incoming label: 262156, Outgoing label: 17
      Local interface: lsi.1048583, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls vpls-1 local site 17 remote site 20
      Flow Label Transmit: No, Flow Label Receive: No

ROSv6 PE2
VPLS interfaces

[admin@ROSv6-PE2] > interface vpls print detail 
Flags: X - disabled, R - running, D - dynamic, B - bgp-signaled, C - cisco-bgp-signaled 
 0 RDB name="vpls2" mtu=1500 l2mtu=1500 mac-address=02:B7:52:32:BB:71 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=10.1.1.1 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=yes vpls=vpls-1 

 1 RDB name="vpls3" mtu=1500 l2mtu=1500 mac-address=02:42:34:91:CB:AB arp=enabled arp-timeout=auto disable-running-check=no remote-peer=10.1.1.3 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=yes vpls=vpls-1 

 2 RDB name="vpls5" mtu=1500 l2mtu=1500 mac-address=02:1D:1F:FC:E2:11 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=10.1.1.4 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=yes vpls=vpls-1

ROSv7 PE4
VPLS interfaces

[admin@ROSv7-PE4] > interface/vpls/print detail 
Flags: X - disabled, R - running; D - dynamic; B - bgp-signaled, C - cisco-bgp-signaled 
 0 RD  name="vpls1" mtu=1500 mac-address=02:A4:52:68:94:48 arp-timeout=auto peer=10.1.1.2 pw-type=vpls pw-l2mtu=1500 pw-control-word=enabled 
       bridge=VPLS-BR-1 bgp-vpls=vpls-1 bgp-vpls-prfx="veId=2,veBlockOffset=16&10.1.1.2:1" 

 1 RD  name="vpls2" mtu=1500 mac-address=02:82:6E:FA:CA:57 arp-timeout=auto peer=10.1.1.3 pw-type=vpls pw-l2mtu=1500 pw-control-word=enabled 
       bridge=VPLS-BR-1 bgp-vpls=vpls-1 bgp-vpls-prfx="veId=3,veBlockOffset=16&10.1.1.3:1"

mrz · Wed May 25, 2022 11:08 am

Perhaps you need to set the same pw-type also in ROS v7.
At least on older junipers you had to match pw-type in order for tunnel to establish.

What you have now is on ROS v6 pw-type is set to "raw-ethernet", on ROS v7 you have it set to "vpls". If you want to operate with mismatched pw-types, if I remember correctly juniper had option "ignore-encapsulation-mismatch ".

mjperry82 · Wed May 25, 2022 3:27 pm

What you have now is on ROS v6 pw-type is set to "raw-ethernet", on ROS v7 you have it set to "vpls".

That is only on the dynamic /interface vpls, not the explicitly created /interface vpls bgp-vpls. I see now that the ROSv6 export doesn't show it, but on both ROS versions the /interface vpls bgp-vpls is set to pw-type "vpls. Here is a print from the ROSv6:

[admin@ROSv6-PE2] > interface vpls bgp-vpls print detail 
Flags: X - disabled, I - inactive 
 0   name="vpls-1" route-distinguisher=10.1.1.2:1 import-route-targets=1.1.1.1:1 export-route-targets=1.1.1.1:1 site-id=2 bridge=VPLS-BR-1 bridge-cost=50 
     bridge-horizon=none use-control-word=yes pw-mtu=1500 pw-type=vpls

I did add this option to the Juniper configuration, it did not change anything.

If you want to operate with mismatched pw-types, if I remember correctly juniper had option "ignore-encapsulation-mismatch ".

set routing-instances vpls-1 instance-type vpls
set routing-instances vpls-1 interface ge-0/0/9.0
set routing-instances vpls-1 route-distinguisher 10.1.1.1:1
set routing-instances vpls-1 vrf-target target:1.1.1.1:1
set routing-instances vpls-1 protocols vpls traceoptions file vpls-debug
set routing-instances vpls-1 protocols vpls traceoptions flag all
set routing-instances vpls-1 protocols vpls control-word
set routing-instances vpls-1 protocols vpls site-range 30
set routing-instances vpls-1 protocols vpls no-tunnel-services
set routing-instances vpls-1 protocols vpls site vpls-1-pe1 site-identifier 17
set routing-instances vpls-1 protocols vpls site vpls-1-pe1 interface ge-0/0/9.0
set routing-instances vpls-1 protocols vpls ignore-encapsulation-mismatch

jackrabbit · Mon Jun 06, 2022 1:26 am

BLG signaled VPLS on v7.3rc2 seems to be working in my lab. Compatible with v6 BGP VPLS peers. Thanks @mrz and MikroTik team. Hopeful that inter-compatibility with other vendors can be resolved.

clambert · Tue Jun 07, 2022 2:57 pm

In the post viewtopic.php?t=185574#p930840, @mrz reported that version ROS v7.3beta40 would have fixed BGP signaled VPLS. However, the first tests I have done have not been successful. I see the connections being established and disconnected continuously.

With the help of Support, I was able to successfully set up Cisco VPLS BGP-based auto-discovery in ROS 7.3rc1.

elelec · Wed Sep 28, 2022 9:01 am

moderator note: do not quote preceding post, use "Post Reply"

Post your config please

clambert · Sat Oct 01, 2022 5:19 am

This is the setup I managed to get working:

/routing bgp vpls add bridge=bridge-cisco-vpls bridge-cost=1 bridge-horizon=1 cisco-id=192.168.100.10&65000:100 export-route-targets=65000:100 import-route-targets=65000:100 name=cisco-vpls pw-control-word=default pw-l2mtu=1500 pw-type=raw-ethernet rd=65000:100

However, it is important to note that I have not been able to get Cisco VPLS working using a Cisco IOS 12 route reflector. The route reflector appears and disappears as a VPLS interface and the CPU consumption goes up to 100%. I have reported this situation [SUP-83173], still without a solution.

JdccDevel · Tue Oct 04, 2022 1:19 am

My testing is showing that, as of ROS 7.5, The "ibgp rr" local role is not working properly for l2vpn.

For l2vpn announcements, it's behaving like normal ibgp, i.e. Not Reflecting. (During my testing, IPv4 routes were reflected properly on the same connection, but not l2vpn.) As such, I was unable to get BGP-VPLS working in an environment where all routers were running ROS version 7.5

I WAS able to get BGP-VPLS working, but I had to use a ROSv6 device as a route reflector. Everything else is running 7.5, and so far it appears to be working in my testing.

I've seen other posts describing issues with the "ibgp rr" role (flapping, etc), but I figured I'd post my findings, to describe specifically what I found.

Hopefully there's a fix soon. So far, as far as I can tell, there's no release target for a fix.

nichky · Tue Oct 04, 2022 6:43 am

@mrz

is there any chance that u can fix the name of BGP-VPLS, if i specificater the name like vpls-x, i want to stay as it is, i dont want when bgp drops for some reason to get vplsx

Network5 · Thu Nov 24, 2022 12:45 am

Hi,
Today I was able to setup a BGP signalled VPLS configuration. It's running natively on LDPv6, no IPv4 for transport.
The main issues were setting up BGP, so in dual stack environment just set the L2VPN and L4VPN only on IPv4 or IPv6 bgp session. If both, none will work. Remains the RR issue: unless a bgp mash is done between routers only adjacent routers will signal the VPLS tunnel to each other.

I'm reporting the configuration of one router. Basically on the other routers the loopback addresses are different. 10.255.255.0/24 and 2001

ffff::/64 are loopback addresses.

LDP configuration:

/mpls interface
add disabled=no input=yes interface=sfp-sfpplus11 mpls-mtu=1650
add disabled=no input=yes interface=sfp-sfpplus12 mpls-mtu=1650
/mpls ldp
add afi=ipv6 disabled=no loop-detect=yes lsr-id=10.255.255.201 transport-addresses=2001:0db8:ffff::201 vrf=main
/mpls ldp accept-filter
add accept=yes disabled=no prefix=2001:0db8:ffff::/64 vrf=main
add accept=no disabled=no prefix=::/0 vrf=main
add accept=no disabled=no prefix=0.0.0.0/0 vrf=main
/mpls ldp advertise-filter
add advertise=yes disabled=no prefix=2001:0db8:ffff::/64 vrf=main
add advertise=no disabled=no prefix=::/0 vrf=main
add advertise=no disabled=no prefix=0.0.0.0/0 vrf=main
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=sfp-sfpplus11 transport-addresses=2001:0db8:ffff::201
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=sfp-sfpplus12 transport-addresses=2001:0db8:ffff::201

BGP VPLS configuration:

/routing bgp vpls
add bridge=test bridge-cost=1 bridge-horizon=1 disabled=no export-route-targets=65000:999 import-route-targets=65000:999 name=test1 \
    pw-control-word=default pw-l2mtu=1600 pw-type=vpls rd=65000:999 site-id=201 vrf=main

BGP connection:

add address-families=ip as=65000 cisco-vpls-nlri-len-fmt=auto-bits connect=yes disabled=no input.affinity=alone .filter=ibgp-in-4 listen=yes \
    local.address=10.255.255.201 .role=ibgp multihop=yes name=BGPv4-R1 nexthop-choice=force-self output.affinity=alone .default-originate=always \
    .keep-sent-attributes=yes .redistribute=connected remote.address=10.255.255.211/32 .as=65000 router-id=10.255.255.201 routing-table=main \
    templates=iBGP-AS56000 vrf=main
add address-families=ipv6,l2vpn,l2vpn-cisco,vpnv4 as=65000 cisco-vpls-nlri-len-fmt=auto-bits connect=yes disabled=no input.affinity=alone .filter=\
    ibgp-in-6 listen=yes local.address=2001:0db8:ffff::201 .role=ibgp multihop=yes name=BGPv6-R1 nexthop-choice=force-self output.affinity=alone \
    .default-originate=always .keep-sent-attributes=yes .no-client-to-client-reflection=no .redistribute=connected remote.address=\
    2001:0db8:ffff::211/128 .as=65000 router-id=10.255.255.201 routing-table=main templates=iBGP-AS65000 vrf=main

If someone has got the RR working I'll be glad to know.

xmasin · Thu Nov 24, 2022 6:35 pm

I have same problem with VPNv4 and RR setup. I have found, that it could be caused by LDP. Can you, please, post LDP status between routers? If is more than one hop between end router and route reflector on intermediate router I see end router in LDP in passive state at it seems, that it cause, that BGP session between end router and RR is not established. When I disable LDP BGP is established immediately.

mpls/ldp/neighbor/print 
Flags: D, I - INACTIVE; O, T - THROTTLED; p - PASSIVE
Columns: TRANSPORT, LOCAL-TRANSPORT, PEER, ADDRESSES
#     TRANSPORT  LOCAL-TRANSPORT  PEER        ADDRESSES    
0 DO  10.0.0.2   10.0.0.3         10.0.0.2:0  10.0.0.2     
                                              10.0.1.2     
                                              10.0.1.5     
                                              172.29.119.11
                                              192.168.21.1 
                                              192.168.22.1 
1 DOp 10.0.0.4   10.0.0.3         10.0.0.4:0  10.0.0.4     
                                              10.0.1.10    
                                              172.29.119.13

Network5 · Thu Nov 24, 2022 11:40 pm

Below you can find the LDP status. R1 and R3 are edge routers, R2 is the middle one. (There is also a BGP session between R1 and R3 to keep the VPLS working from R1 to R3.)
(Link-local addresses has been removed and the IPv6 block replaced with the demo one.) The topology is going further from R1, hence c41. As you sad, the passive is present in the middle router LDP status.

R1

Flags: D, I - INACTIVE; O, T - THROTTLED
Columns: TRANSPORT, LOCAL-TRANSPORT, PEER, ADDRESSES
#    TRANSPORT            LOCAL-TRANSPORT       PEER              ADDRESSES                
0 DO 2001:0db8:ffff::c41  2001:0db8:ffff::a101  10.255.255.41:0   2001:0db8:101e:2::a2     
                                                                  2001:0db8:1301::1        
                                                                  2001:0db8:1301:f000::a1  
                                                                  2001:0db8:1301:f001::a1  
                                                                 2001:0db8:ffff::c41      
                                                                  fe80::
1 DO 2001:0db8:ffff::201  2001:0db8:ffff::a101  10.255.255.201:0  2001:0db8:2:2a::ae:2      
                                                                  2001:0db8:0:420::2       
                                                                  2001:0db8:101f:1::1      
                                                                  2001:0db8:101f:101::1    
                                                                  2001:0db8:ffff::201      
                                                                  fe80::

R2

Flags: D, I - INACTIVE; O, T - THROTTLED; p - PASSIVE
Columns: TRANSPORT, LOCAL-TRANSPORT, PEER, ADDRESSES
#     TRANSPORT             LOCAL-TRANSPORT      PEER              ADDRESSES                
0 DOp 2001:0db8:ffff::a101  2001:0db8:ffff::201  10.255.255.31:0   2001:0db8:101e::a1       
                                                                   2001:0db8:101e:1::a1     
                                                                   2001:0db8:101e:2::a1     
                                                                   2001:0db8:101f:1::2      
                                                                   2001:0db8:ffff::a101     
                                                                   fe80::             
                                                                   
1 DOp 2001:0db8:ffff::211   2001:0db8:ffff::201  10.255.255.211:0  2001:0db8:101f:101::2    
                                                                   2001:0db8:ffff::211      
                                                                   fe80::

R3


Flags: D, I - INACTIVE; O, T - THROTTLED
Columns: TRANSPORT, LOCAL-TRANSPORT, PEER, ADDRESSES
#    TRANSPORT            LOCAL-TRANSPORT      PEER              ADDRESSES                
0 DO 2001:0db8:ffff::201  2001:0db8:ffff::211  10.255.255.201:0  2001:0db8:2:2a::ae:2      
                                                                 2001:0db8:0:420::2       
                                                                 2001:0db8:101f:1::1      
                                                                 2001:0db8:101f:101::1    
                                                                 2001:0db8:ffff::201      
                                                                 fe80::

Hope it helped.

vipismez · Fri Nov 03, 2023 6:04 am

20231103112555.jpg

I found that the real problem on ROS7 is that BGP-RR, after receiving NLRI from other RR clients, changes next-hop to itself when reflecting to other clients

Network5 · Fri Nov 03, 2023 9:52 am

Could be the reason why BGP-VPLS is not working in between peers that have not direct BGP session. I have to test another strange behaviour, but have to do further testing: if the setup uses two (redundant) RR, the tunnels will be flapping undefinetly.

What ROS7 version did you used?
Please open a support ticket!

ROSv7 BGP-VPLS Setup Help

ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Re: ROSv7 BGP-VPLS Setup Help

Who is online