Problems with MPLS IPv4 VPN

Crami · Tue Jun 18, 2013 4:32 pm

Hi

I have a new CCR1016-12G which I like to use as part of my MPLS network. I attached it to a Brocade XMR via VLAN 2006. On this Interface I have enabled OSPF and LDP. I made 2 BGP sessions to my VPNv4 Route Reflectors and configured a VRF on the CCR and on the two Cisco Routers.
On the CCR I have a ipip tunnel in this VRF and on the Cisco I have a loopback interface in the VRF.

I can sometimes ping from a RouterBoard behind the CCR to the loopback on the Cisco. It usualy works for 10-20s. After that it stops working. Sometimes it spontaniously starts working again for 10-20s. I can force it to work, when I change anything in the routing table on the CCR, eg. adding/removing or disabling/enabling a static route, or disabling or enabling one of the BGP sessions.

Does anyone have a similar setup or similar troubles? Any hint's are highly welcome.

Tue Jun 18, 2013 11:06 pm

Hi Crami

I have a similar setup.

Please email support@mikrotik.com with your problem and a supout.rif they are actively working on problems with L3VPN at the moment.

bearmeister · Wed Jun 19, 2013 2:11 am

Hi All,

I was about to make a post on this very issue.

I am running 6.0rc14 on RB1200 and RB750 platforms. ( For some reason 6.0 candidate release crashes regularly, 6.0rc14 less so )

Having come from Cisco(15yrs+) and Juniper (2yrs+) to Mikrotik (6months+) I would have thought that establishing a basic back to back MPLS VPN between two routers using Loopbacks with a couple of "LAN" computers to simulate a station at each site would have been a no brainer, but after some time I have consider this feature of RouterOS faulty !

I've run my config by a couple of MT engineers who seem to think this should be pretty straight forward also, but their experience with these features is primarily in 5.x. I would be interested to hear your thoughts.

I have
P2P GRE tunnel between two Mikrotiks, lets call then PE1 and PE2
Each PE has it's own Loopback0 IP (PE1 10.240.2.1 PE2 10.240.2.2)
OSPF (P2P) is running on the GRE with Loopback0 as passive
LDP is running on the GRE with Loopback0 as the source
VPNv4 BGP is running between Loopback of PE1 and PE2 for VRF prefix distribution
Each PE has a single LAN interface in VRF2256
PE1's in-VRF IP is 192.168.26.254/24 and PE2 is 192.168.127.254/24

When I apply the config to a new router straight from copy & paste, everything comes up as expected. OSPF exchanges Loopbacks, BGP comes up, label distribution is good, MPLS forwarding table is good. BGP Prefix to Label association is correct and the paths are working. The device on LAN1 IP 192.168.26.150 can ping device on LAN2 IP 192.168.127.150 and the world is a happy place. A no brainer !

However, after an arbitrary period of time the pings between the two computers will stop. The tables on the Mikrotik are unchanged with respect to routes and labels. Debug log shows nothing untoward at the moment the ping stops dead in the water. The most time I have got this working is 965 pings, but almost always this dies after 5-20 successful pings.

Like you, changing anything to do with the route table causes the MPLS VPN to start working again, even though the change has no relationship to the traffic that is flowing through the router. For example, I add a Loopback2256 into VRF2256 on a PE. Adding a loopback in the VRF on the PE has no bearing on the existing labels and prefixes relating to the LAN-LAN PC's pinging each other, there is no visible change to any tables, yet MPLS will start working again, briefly. When it stops, you can do a similar thing, anything that pokes the routing table appears to cause it to start working again momentarily. It's like the MPLS table is saying it is doing what it should, but it isn't.

See screenshot attached

bearmeister · Wed Jun 19, 2013 2:14 am

config for PE2

/routing bgp instance
set default as=65501 router-id=10.240.2.2
/routing ospf instance
set [ find default=yes ] mpls-te-area=backbone mpls-te-router-id=Loopback0 router-id=10.240.2.2
/ip address
add address=192.168.127.254/24 interface=ether1 network=192.168.127.0
add address=10.240.2.2/32 interface=Loopback0 network=10.240.2.2
add address=192.168.5.81/24 interface=ether5 network=192.168.5.0
add address=10.200.2.6/30 interface=AGGR1-GRE network=10.200.2.4
/ip dhcp-server network
add address=192.168.127.0/24 dns-server=8.8.8.8 gateway=192.168.127.254 netmask=24 ntp-server=0.0.0.0 wins-server=0.0.0.0
/ip route vrf
add export-route-targets=65501:2256 import-route-targets=65501:2256 interfaces=ether1 route-distinguisher=65501:2256 routing-mark=C2256
/mpls
set propagate-ttl=no
/mpls ldp
set enabled=yes lsr-id=10.240.2.2 transport-address=10.240.2.2 use-explicit-null=yes
/mpls ldp interface
add transport-address=10.240.2.2
/routing bgp instance vrf
add redistribute-connected=yes routing-mark=C2256
/routing bgp peer
add address-families=vpnv4 name=PE2 remote-address=10.240.2.1 remote-as=65501 route-reflect=no ttl=default update-source=Loopback0
/routing ospf interface
add interface=Loopback0 network-type=broadcast passive=yes
/routing ospf network
add area=backbone network=10.200.0.0/16
add area=backbone network=10.240.0.0/16

Wed Jun 19, 2013 2:58 am

Hi Guys,

L3VPN is currently unusable on RouterOS, and has been since at least 5.0rc days.

There are multiple issues with L3VPN on RouterOS:

1. BGP route withdraw are received but not actioned on L3VPN endpoints (results in stale routes)
2. Loopback interfaces within a VRF are reachable from within other VRF's on the same router.
3. Routing engine crashes on L3VPN endpoints
4. Redistribution of "Other BGP" in a VRF does not work

Issue 1 and 2 have been there since 5.0rc days, possibly since mpls-test days... 2 and 3 appear to be more recent regressions.

Please email support@mikrotik.com with Attn:Maris in the subject, include detailed descriptions of your problem and supout.rif from all routers involved.

Mikrotik are actively working on these issues at the moment, and the more information they get the quicker it will be fixed.

bearmeister · Wed Jun 19, 2013 3:25 am

That is very disappointing !

I used the MUM slideshows and Mikrotik Manual as my guide for doing this on RouterOS, aside from the fact the most recent version I can see is 3.17 or thereabouts, it all seems to work and in fact cites "no failures here" with pings going back and forward. There is no caveats/gotchas or any suggestions this is broken which it obviously is

I then read Greg Sowell's blog entry "Why is Mikrotik pushing MPLS" and figured well, if it worked in 3.17 surely it would work now. Bit the dust on that one didn't I hahah !!!!

Without sounding too stupid, does Mikrotik publish a list of known caveats like Cisco/Juniper for each release so at least when one sees a knob that *should* work, if its fundamentally broken, this is identified during the release?

I read the changelogs and that is great for showing what is resolved, but it doesn't show what remains broken (despite the fact the knobs are there giving it the appearance that it should work for more than 30 seconds.) I like to read over the known caveats list before I go jumping in.

Spent a fair bit of time bashing this out before I reached the conclusion "this is just outright broken!" so to hear that it is a known issue and has been for a long time to a new comer who has endured a baptism of fire is a bit painful. Suggestions on how to avoid a similar experience in the future? Is there a definitive reference I should be looking at when considering what features work and what don't on a given release?

This appears to be ongoing for some time, what kind of timeline does Mikrotik have on a working MPLS VPN given they are pushing "Cloud Core Routers" ? I'm used to Cisco who would consider this a serious issue and would not push a new release out (citing a bunch of fixes) whilst such a fundamental aspect of the code remains accessible to the user, yet non functional due to immaturity?

I'm not expecting Mikrotik to be Cisco, I'm just trying to reset my expectations based on where Mikrotik is at so I don't end up bashing my head on something which I discover was always broken

Wed Jun 19, 2013 4:22 am

This appears to be ongoing for some time, what kind of timeline does Mikrotik have on a working MPLS VPN given they are pushing "Cloud Core Routers" ? I'm used to Cisco who would consider this a serious issue and would not push a new release out (citing a bunch of fixes) whilst such a fundamental aspect of the code remains accessible to the user, yet non functional due to immaturity?

Completely agree. We are wanting to deploy our first 100% Mikrotik L3VPN network using the current CCR as well as the new -2S+ model but until these issues are resolved we cannot.

I have raised the issues I have discovered multiple times with support over the past few years and have previously been told "All problems will be fixed in the new routing" but never got a timeline on when this would make it in to RouterOS

while we have been able to successfully use L2VPN and static VPLS, L3VPN has been completely unusable due to the stale routes issue.

The ticket I logged last week is the first time they have actively engaged with me to work through to a resolution. So we are now heading towards the possibility that this bug will finally be fixed

Please email them, as the only way we will get the problems resolved is to work with Mikrotik support through their official support mechanisms, which is currently email.

Crami · Wed Jun 19, 2013 8:00 pm

I sent a mail to support@ today in the morning, but have not heard anything until now. Lets see what they come up with.

norpan · Wed Jun 26, 2013 10:16 am

I have sent a mail to support last night about this too.
With v6.1 L3VPN stops working after a very short time, LDP is running, MPLS forwarding table is ok, BGP routes are still there. It seems, at least in my tests, that the vrf is "leaking".

My setup:
ether1 - MPLS interface
ether2 - VRF 2:20, IP 10.12.0.1/24

My test:
I had a PC connected to ether2 with IP 10.12.0.10 and ran a ping against 8.8.8.8, and when the issue occured I sniffed traffic on ether1 (ie: tool sniffer quick interface=ether1 ip-address=10.12.0.0/24) which should not return anything since i'm running L3VPN, but it did.

My observation:
Basically the packets are IP routed, instead of getting a label pushed and forwarded as MPLS-packet.
A interesting thing is that I didn't have a route that matched 8.8.8.8 in the main routing-table, so it cant be using that.

Can anyone verify this and see if v6 is forwarding L3VPN-packets as IP instead of MPLS?

Wed Jun 26, 2013 12:24 pm

Hi Norpan,

Your issue is different from the issue that has been discussed here, but being a problem related to MPLS is still quite interesting.

Have you looked at the packets "outside" the Mikrotik, e.g. captured them with wireshark on a mirror port? That would be really interesting if it is indeed forwarding packets unlabeled.

norpan · Wed Jun 26, 2013 2:02 pm

Hi All,
When I apply the config to a new router straight from copy & paste, everything comes up as expected. OSPF exchanges Loopbacks, BGP comes up, label distribution is good, MPLS forwarding table is good. BGP Prefix to Label association is correct and the paths are working. The device on LAN1 IP 192.168.26.150 can ping device on LAN2 IP 192.168.127.150 and the world is a happy place. A no brainer !

However, after an arbitrary period of time the pings between the two computers will stop. The tables on the Mikrotik are unchanged with respect to routes and labels. Debug log shows nothing untoward at the moment the ping stops dead in the water. The most time I have got this working is 965 pings, but almost always this dies after 5-20 successful pings.

Like you, changing anything to do with the route table causes the MPLS VPN to start working again, even though the change has no relationship to the traffic that is flowing through the router. For example, I add a Loopback2256 into VRF2256 on a PE. Adding a loopback in the VRF on the PE has no bearing on the existing labels and prefixes relating to the LAN-LAN PC's pinging each other, there is no visible change to any tables, yet MPLS will start working again, briefly. When it stops, you can do a similar thing, anything that pokes the routing table appears to cause it to start working again momentarily. It's like the MPLS table is saying it is doing what it should, but it isn't.

Well, it may or may not be the same issue, but bearmeister's description of his problem fits exactly with what I have discovered. A 'poke' in the routing table makes things work again, but only for a short while.
I have just now setup a lab with two routers in virtualbox, and basic l3vpn where the problem shows up almost instantly.

* R2: ping routing-table=vrf1_1 src-address=192.168.2.1 192.168.1.1
* R1: tool sniff quick interface=ether1 ip-address=192.168.2.0/24

And when the ping dies with timeout, I get hits on the capture.

R1

/interface bridge
add name=lo0
/ip address
add address=1.1.1.1/32 interface=lo0 network=1.1.1.1
add address=10.1.1.1/24 interface=ether1 network=10.1.1.0
add address=192.168.1.1/24 interface=ether2 network=192.168.1.0
/ip route vrf
add export-route-targets=1:1 import-route-targets=1:1 interfaces=ether2 \
    route-distinguisher=1:1 routing-mark=vrf1_1
/mpls interface
set [ find default=yes ] mpls-mtu=1500
/mpls ldp
set enabled=yes lsr-id=1.1.1.1 transport-address=1.1.1.1
/mpls ldp interface
add interface=ether1
/routing bgp instance vrf
add redistribute-connected=yes routing-mark=vrf1_1
/routing bgp peer
add address-families=vpnv4 name=R2 remote-address=2.2.2.2 remote-as=65530 \
    update-source=1.1.1.1
/routing ospf interface
add interface=lo0 passive=yes
/routing ospf network
add area=backbone network=10.1.1.0/24
add area=backbone network=1.1.1.1/32
/system identity
set name=R1

R2

/interface bridge
add name=lo0
/ip address
add address=2.2.2.2/32 interface=lo0 network=2.2.2.2
add address=10.1.1.2/24 interface=ether1 network=10.1.1.0
add address=192.168.2.1/24 interface=ether2 network=192.168.2.0
/ip route vrf
add export-route-targets=1:1 import-route-targets=1:1 interfaces=ether2 \
    route-distinguisher=1:1 routing-mark=vrf1_1
/mpls interface
set [ find default=yes ] mpls-mtu=1500
/mpls ldp
set enabled=yes lsr-id=2.2.2.2 transport-address=2.2.2.2
/mpls ldp interface
add interface=ether1
/routing bgp instance vrf
add redistribute-connected=yes routing-mark=vrf1_1
/routing bgp peer
add address-families=vpnv4 name=R1 remote-address=1.1.1.1 remote-as=65530 \
    update-source=2.2.2.2
/routing ospf interface
add interface=lo0 passive=yes
/routing ospf network
add area=backbone network=10.1.1.0/24
add area=backbone network=2.2.2.2/32
/system identity
set name=R2

Thu Jun 27, 2013 3:13 am

Hi Norpan,

Sorry I did not read your post completely. It does sound related.

Hopefully Mikrotik can fix these issues quickly

norpan · Thu Jun 27, 2013 11:33 am

@nz_monkey: That's ok, I was not sure at first either but the more I look at it, and troubleshoot, the clearer it get's. At least that's what I'm telling myself.

I just hope that MT support can acknowledge the problem and figure it out, and get it patched really soon...

I did another setup last night with two RB750 running with the config I posted earlier, with the same result. Almost instanty the ping over the vrf fails.
With some poking around with routing and/or ldp I can get it running again, sometimes for a few seconds and at a few times it felt stable. But after a reboot it comes back.
So now I have the same issue on tile, x86 and mipsbe, which tells me that it's not architecture related. I have not tried any rc or beta releases if it gets introduced in a specific version, but 6.0 and 6.1 behaves the same.

From my point of view MPLS L3VPN (at PE) is broken in v6, which means that CCR's can't be used at all.

Or is there anyone running v6 on a PE-router and it's stable? If so, which version are you running?

norpan · Thu Jun 27, 2013 11:56 am

Testing with 6.0beta2 now, definitely more stable than 6.1.

norpan · Thu Jun 27, 2013 2:39 pm

v6.0rc2 worked for over an hour, that have never happened for me with 6.0/6.1.
Unfortunately I have managed to kill one of the routers during my upgrade/downgrade so my testing is done for now.

norpan · Thu Jun 27, 2013 10:05 pm

Summary of todays testing, versions up to 6.0rc13 it is stable so far.
From v6.0rc14 and up the issue occurs and MT support has been able to reproduce it in their test.

Hoping for a patch to come out soon.

Crami · Thu Jun 27, 2013 10:24 pm

Is it normal that MT does not reply to support mails ? Was my first request, just wondering.

norpan · Thu Jun 27, 2013 11:10 pm

What I can recall they have at least answered within 1 working day, haven't you got a response at all?

If you still have the issues in your first post, can you try with 6.0rc13 to see if it works better?

If you don't have it, here is a link to the torrent for all architectures:
http://www.mikrotik.com/download/router ... 13.torrent

edit: it doesn't have to be rc13, anything pre-6.0rc14 should work better.

Fri Jun 28, 2013 12:53 am

Norpan I think your issue is different from my primary issue. The issue I have is stale routes within a VRF, e.g. a withdraw is received by the PE router but the route is never actually withdrawn from the FIB. This bug has been in RouterOS since at least 5.0rc's.

In my lab I have 6.1 as a route originator, and 5.12 as well as 6.0 and 6.1 as PE devices and the issue occurs on all 3 versions.

Fri Jun 28, 2013 1:05 am

Is it normal that MT does not reply to support mails ? Was my first request, just wondering.

I think it depends on:

- How busy they are
- The subject of the issue
- How major the problem is
- If it is related to the "new & cool" feature

For some RouterOS components e.g. MPLS it seems there are less people in the Mikrotik support team focusing on it, I have only ever received responses from Maris for anything to do with MPLS. Where as for most other components I have had interactions with a bunch of different Mikrotik staff.

I have found generally if I log a ticket about a "new & cool" feature I will get a response quickly and any bugs get fixed quickly. If I log a ticket about something that is neglected and/or complex e.g. MPLS or HWMP the response is generally slower.

norpan · Fri Jun 28, 2013 1:30 am

Norpan I think your issue is different from my primary issue. The issue I have is stale routes within a VRF, e.g. a withdraw is received by the PE router but the route is never actually withdrawn from the FIB. This bug has been in RouterOS since at least 5.0rc's.

In my lab I have 6.1 as a route originator, and 5.12 as well as 6.0 and 6.1 as PE devices and the issue occurs on all 3 versions.

I agree that we speak of two different issues, I have discovered those stale routes too. I have also opened a case about that, I get those when I have two routers actively redistributing the same prefix.
If this is the same as what you are experiencing.

My last mail to support:

Well, i still don't think that is the case, if you look at this, both is from
the
same router (2.2.2.2):

## BGP ###

VPNv4 ROUTES
Flags: L - label-present
0 L route-distinguisher=1:1 dst-address=172.16.1.0/24 interface=ether2
in-label=18 bgp-ext-communities="RT:1:1"

Here you only have the local route in BGP table, ok?

## IP route ###

ROUTE
5
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
0 ADC dst-address=172.16.1.0/24 pref-src=172.16.1.2 gateway=ether2
gateway-status=ether2 reachable distance=0 scope=10 routing-mark=vrf

1 Db dst-address=172.16.1.0/24 gateway=1.1.1.1
gateway-status=1.1.1.1 recursive via 10.1.1.1 ether1 distance=200
scope=40 target-scope=30 routing-mark=vrf bgp-local-pref=100
bgp-origin=incomplete bgp-ext-communities="RT:1:1"

2 ADo dst-address=1.1.1.1/32 gateway=10.1.1.1
gateway-status=10.1.1.1 reachable via ether1 distance=110 scope=20
target-scope=10 ospf-metric=20 ospf-type=intra-area

3 ADC dst-address=2.2.2.2/32 pref-src=2.2.2.2 gateway=lo0
gateway-status=lo0 reachable distance=0 scope=10

4 ADC dst-address=10.1.1.0/24 pref-src=10.1.1.2 gateway=ether1
gateway-status=ether1 reachable distance=0 scope=10

But here you have a BGP route where 1.1.1.1 is gateway, and 1.1.1.1 has stopped
redistributing any routes.
It's not active, but still left over in IP-tables.

My last post was regarding Crami's issues, which could be related to what I have been troubleshooting the last couple of days.

With some reservation for misunderstanding of our issues.

Fri Jun 28, 2013 2:28 pm

Currently we can repeat two problems related to VRFs.
Please wait until we fix them, then we can make further tests and see if here mentioned problems are related.

Fri Jul 19, 2013 7:52 am

Has anyone else had any response from Mikrotik Support on these issues ?

They seem to have gone silent on my ticket.

samsung172 · Mon Jul 22, 2013 6:01 am

this has been known for a while. I first posted about this I 2012 :

http://forum.mikrotik.com/viewtopic.php?f=19&t=68374

also later http://forum.mikrotik.com/viewtopic.php ... rf#p367265 and http://forum.mikrotik.com/viewtopic.php ... rf#p369918

There is also a support ticket : Ticket#2013061066000546

The Vrf, has been unstable in all releases of 6.x

Mon Jul 22, 2013 6:53 am

this has been known for a while. I first posted about this I 2012 :

The Vrf, has been unstable in all releases of 6.x

My experience is that L3VPN on RouterOS has been unusable for much longer. I tried it in 5.0rc, 5.12, 5.16 and then gave up since Mikrotik said it would be fixed in the "new routing". I recently needed L3VPN again and started testing on 6.0 then 6.1 and now 6.2 and have encountered the same issue with stale routes on all of those releases. Maybe that is different from the problem you have, as I cannot see if you are just using VRF, or if you are using L3VPN as well.

Unfortunately I have received no response from Mikrotik on my ticket since the 4th of July.

samsung172 have you logged a ticket on this recently ?

samsung172 · Mon Jul 22, 2013 11:16 am

the lastest mail, with supout from the 2 devices connected was sendt 03.07.2013. I have not recived anny answer from this. post was like this:

Finaly i was able to make a supout after the router was upgraded to 6.x. Today I was testing 6.1 to se if it differ any. I does not.

Simple setup:

R1-R2-R3
172.31.0.4-172.31.0.25-172.31.0.41 (at loopback)
Fiber-R1-licenced 400mb/s-R2-Rb800 with rocket –R3.

Pic1: From the router with problem,
Pic2 – Log to se how ofen problem occur (r3 up down up down)
Pic3 - Log from R1. To se what “happening here”. (ospf down)
Pic4 – after router (r2) have reebooted by itself.

Suppout files:
Suppout.rif=R2 (with problem)
Suppout2.rif ) R1 (connected to the router with problem)

-----Opprinnelig melding-----
Fra: MikroTik support [Maris] [mailto:support@mikrotik.com]
Sendt: 10. juni 2013 13:41
Til: Thomas Andreassen
Emne: Re: [Ticket#2013061066000546] VRF

Hello,

Try to disable vrfs and then generate supout files. Maybe there will be some useful info.

Regards,
Maris

06/10/2013 14:35 - Thomas Andreassen wrote:

> I have tested at a lot of routers, and the same bahavior to all. Only
> solution
is
> 6.rc12.
>
>
>
> I have tried, but even serial cable, don’t make the suppout.rif (it
> start, but just hang)
>
>
>
>
>
>
>
>
>
>
>
> At <http://forum.mikrotik.com/viewtopic.php ... hilit=+vrf>
> http://forum.mikrotik.com/viewtopic.php ... hilit=+vrf i was
> describing the behavior.
>
>
>
> All is ok, until I put the vrf into /ip route. Its no
>
>
>
> I have a backup of config. Works in 6.rc12 but not at 6.0.
>
>
>
> I see that the route distinguisher is "unknown". This behavior exist
> in all
6.xxx
> versions. Not in 5.xxx
>
>
>
> /ip route vrf
>
> add export-route-targets=0.0.0.0:0 import-route-targets=0.0.0.0:0 \
>
> route-distinguisher="(unknown)" routing-mark=vrf.internet
>
>
>
>
>
>
>
> Thomas
>
>
>
>
>
> -----Opprinnelig melding-----
> Fra: MikroTik support [Maris] [mailto:support@mikrotik.com]
> Sendt: 10. juni 2013 12:18
> Til: Thomas Andreassen
> Emne: Re: [Ticket#2013061066000546] VRF
>
>
>
> Hello,
>
>
>
> There is no difference in code from rc12 to 6.0 in VRFs.
>
> Probably in rc12 you did not trigger the same behavior.
>
>
>
> To get supout file you can connect serial cable and generate one via
> serial terminal.
>
>
>
> Regards,
>
> Maris
>
>
>
> 06/10/2013 13:06 - Thomas Andreassen wrote:
>
>
>
> > Hello. Earlier in 6.xx there was an vrf issue, making cpu going 100%.
>
> > In
>
> > 6.rc14 it was ok, and the router did not go to 100% CPU. In 6.0 it
>
> > seems like the issue is back, and I cannot have 6.0 to any router
>
> > having MPLS,bgp,ospf and vrf.
>
> >
>
> >
>
> >
>
> > Is this bug registrated and will be fixed? Earlier you asked me for
> > a
>
> > supout to this device, but its impossible since cpu is 100%, and
>
> > router just "hanging". Its impossible to even connect to the router.
> > I
>
> > have to switch off ospf, bgp and pmls interface in other side, and
>
> > connect trough mac-telnet after a while, to give the router a
> > default
>
> > gateway. After this its possible to downgrade again.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > Thomas A
>
> >
>
> >
>
> >
>
> >
>
>

Mon Jul 22, 2013 11:56 am

samsung172 your issue sounds very similar to mine.

I am using OSPF, BGP (L3VPN signalling), VRF and also route leaking.

I notice that the route leaking partially works on 6.2, e.g. Static/Connected routes appear to be leaked, but if you redistribute "Other BGP" it does not work.

So the issues I have encountered so far:

- Route leaking only partially works
- Routes are not always withdrawn (BGP withdraw is received, but routing process crashes, cpu reads 25% and router becomes unresponsive)
- BGP Cluster ID is not used correctly, e.g. if you have two route reflectors, set the cluster ID on both to 10.1.1.0 and then look at the packets you will see the cluster ID hash is different from each route reflector....

Hopefully Mikrotik can find the time to update my ticket soon, even if they are still researching the problem it would be nice to be kept informed.

samsung172 · Tue Jul 23, 2013 10:55 am

I have not seen the problems with route leaking (I don't leak routes), but the biggest issue, is that once I put the vrf config to /ip route, the CPU goes to 100%. Then it not even possible to make a supout. If you se my old post's the routers goes crasy.

It's really a pain in the ass, since its impossible to use CCR's in places I want a vrf. (typical at CPE). I can use in core net, since the MPLS/VPLS, BGP and OSPF work, as long as I don't want to have a IP from a vrf to a interface. It still forward the routingtable by the ibgp

It seems like rc6.12 is working. (but then with its other problem)

Thu Jul 25, 2013 12:32 am

Some bad news.

I received an update from Mikrotik support to let me know that these problems are not a priority and that they are busy working on other problems.

This means that IPv4 / L3VPN as well as clustered BGP remain unusable in production on Mikrotik RouterOS.

samsung172 · Thu Jul 25, 2013 10:57 pm

Really bad info. Since we cannot start the rollout of CCR's before the vrf issue is solved in 6.x. (works like a charm in all 5.x).

Thu Jul 25, 2013 11:54 pm

Really bad info. Since we cannot start the rollout of CCR's before the vrf issue is solved in 6.x. (works like a charm in all 5.x).

The situation is the same for us. Unfortunately L3VPN does not work on v5 either or we could run it on x86.

These features have been in RouterOS since late 3.x releases, and have never worked correctly. As mentioned earlier I have previously reported the issues and been fobbed off by support with the standard "this is fixed in new routing".

Saying I am frustrated is an understatement. The product does not do what it says in the data sheets.

norpan · Wed Aug 07, 2013 11:13 am

I have not seen the problems with route leaking (I don't leak routes), but the biggest issue, is that once I put the vrf config to /ip route, the CPU goes to 100%. Then it not even possible to make a supout. If you se my old post's the routers goes crasy.

It's really a pain in the ass, since its impossible to use CCR's in places I want a vrf. (typical at CPE). I can use in core net, since the MPLS/VPLS, BGP and OSPF work, as long as I don't want to have a IP from a vrf to a interface. It still forward the routingtable by the ibgp

It seems like rc6.12 is working. (but then with its other problem)

For me the last working version is rc13, with everything after that the vrf stops within minutes at most, tried latest build of v6.2 yesterday without success.

Crami · Mon Aug 19, 2013 6:46 pm

I have tried it with the new 6.2 release now, same result... As expected because the releas notes do not state anything in this direction.
Downgraded now to 6.0rc13 and it worked for s short time, but now I can't get it to work at all...

Also noticed that VRF's are not isolated from each other. I can ping IP's in one VRF configured on the CCR from a laptop connected to another VRF...

This is anoying ...

When can we expect a fix ?

Regards

Matthias

murgatroid · Wed Aug 28, 2013 5:25 am

I'm currently experiencing the same behaviour with MPLS/IPv4 VPN's as well.

5.25 - Mostly works. Occasionally hit 100% cpu on routing process. Frequency increases when wireless links (running this over long distance) suffer from flaps
6.0rc14 - Up to here, behaviour is consistent.
6.1 - VRF is available for 1 or 2 seconds after coming up. Then nothing.
6.2 - as 6.1

What changed?

Downgraded back to 5.25 to get life back into the network.

It's actually quite disappointing that this "feature" is available, but is buggy and has been for a long time, and is not even considered important enough to address. It will be fixed in the "new routing"? I'm not holding my breath. It hasn't been fixed for how many years now?

On the positive side, mpls/bgp/l2vpn works just fine. Doesn't fix this issue though.

Crami · Wed Aug 28, 2013 10:10 am

Pity is also that I don not have the luxury of going back to 5.x because on the CCR there is only newer releases ...

Wed Aug 28, 2013 12:16 pm

It's actually quite disappointing that this "feature" is available, but is buggy and has been for a long time, and is not even considered important enough to address. It will be fixed in the "new routing"? I'm not holding my breath. It hasn't been fixed for how many years now?

On the positive side, mpls/bgp/l2vpn works just fine. Doesn't fix this issue though.

I agree 100% on all points raised.

We have numerous L2VPN implementations with Mikrotik and they work flawlessly, the problems are all related to L3 functionality e.g. VRF's, RIB/FIB mismatches, OSPF instances, route leaking, route filters, bfd.

My feeling is that there are just too many fundamental things wrong with routing in 5.x/6.x to put a band-aid on that will get this functionality to a usable point. I have logged numerous tickets that seemed to be making progress, only to have them end in "will be fixed in new routing" or in silence.

MIkrotik have a lot on their plate at the moment with the new architectures, and I am sure we will see the "new routing" as soon as it is ready (my pick is it will arrive in v7 betas) so all we can do is sit back and wait, or move to a more resilient MPLS platform.

samsung172 · Fri Sep 27, 2013 8:30 am

vrf is now OK in my setup. Its fixed in 6.5rc1

norpan · Fri Sep 27, 2013 9:26 am

I also have good results so far with 6.5rc1, but I'll give it some more time before I go and say that it's fixed.

samsung172 · Sat Sep 28, 2013 12:35 am

I also have good results so far with 6.5rc1, but I'll give it some more time before I go and say that it's fixed.

At least its not 100% in CPU, and the router's have now about 2 days uptime. In pre 6.5 (exept 6.rc13) there was about 2-3 min uptime before cpu goes 100%.

murgatroid · Mon Oct 07, 2013 7:14 am

Is it long enough to be considered 'more time' now? How's it running?

Fri Oct 11, 2013 3:03 am

Is it long enough to be considered 'more time' now? How's it running?

It is stable on our network. Thanks Mikrotik, this should tide us through until v7

samsung172 · Fri Oct 11, 2013 11:53 pm

Rock solid and stable here. All my routers is now upgraded to 6.5.

We have just ordered 16 new CCR's, and will start the rollout of a lot of new equipment now (18Ghz 400 MB/s fiber radioes)

If MT removes the stupid 1 box per channel in scanlist to 6.x units now. I will also make a upgrade to all unlicensed wireless units.

crtee · Sun Feb 09, 2014 2:54 am

Hi there,

sorry to bring this up again, haven't been in touch with this recently, but are L3VPNs really safe so far? Are there still any known stability problems or did they finally manage to fix it?

Sun Feb 09, 2014 3:34 am

Hi there,

sorry to bring this up again, haven't been in touch with this recently, but are L3VPNs really safe so far? Are there still any known stability problems or did they finally manage to fix it?

Hi crtee,

I can say they are stable on 6.5 as that is what we have running in production. Any newer version YMMV.

We have had one issue doing CE originated routes to our PE router where the initial send of the routes appeared in the VRF but were not reachable, dropping the BGP session and re-establishing fixed it. So I suspect there may still be a few little issues, but nothing that stops us from being able to use it in production.

Given the recent dramas with RouterOS stability I am scared to even attempt an upgrade past 6.5 for fear that it will break something.

crtee · Sun Feb 09, 2014 5:36 pm

Hi crtee,

I can say they are stable on 6.5 as that is what we have running in production. Any newer version YMMV.
[...]
Given the recent dramas with RouterOS stability I am scared to even attempt an upgrade past 6.5 for fear that it will break something.

Thanks for your reply. I'm already at 6.9, however I'll give it a try and post my results here.

crtee · Mon Feb 17, 2014 11:54 pm

Thanks for your reply. I'm already at 6.9, however I'll give it a try and post my results here.

Okay, my conclusion so far: don't try to run L3VPN and a full IPv4 BGP table on the same box. The routing process is unable to keep up with everything, maybe unless you take some top-end Xeon box, put it in liquid nitrogen and crank the clock beyond 5 Ghz.

However, VPLS works great. Waiting for 7.0 and the "new routing engine"

Tue Feb 18, 2014 12:41 am

Waiting for 7.0 and the "new routing engine"

Mikrotik have not confirmed when "new routing" will arrive. I have been hearing about it from them for about 3 years now, but still have not seen it.

Hopefully it does come in v7 and we see it soon!

There are a number of issues/missing features on v4/5/6 that are affecting us

- No RIPv2 from VRF's so cannot use it for PE-CE
- Cannot view BGP advertisements sent to/received from a peer when running PE-CE from a VRF
- Cannot view L2VPN information sent/received from a BGP peer
- Cannot view L3VPN information sent/received from a BGP peer
- BGP is not multi-core optimized making complex filters, full tables and large updates very very slow
- A lot of configuration is still CLI only, e.g. BGP VRF out-filters
- Config database sync issues, often the running config is different from what is shown in Winbox/CLI.
- VPLS tunnel state changes are not logged
- Can not specify which VRF router management services are available from, e.g. WinBox, SSH, Webfig
- Cannot specify which VRF PPP connections terminate to
- OSPF interface to instance mapping seems inconsistent with the rest of RouterOS (uses instance ID's rather than names)

samsung172 · Tue Feb 18, 2014 1:33 am

Waiting for 7.0 and the "new routing engine"
Mikrotik have not confirmed when "new routing" will arrive. I have been hearing about it from them for about 3 years now, but still have not seen it.

Hopefully it does come in v7 and we see it soon!

There are a number of issues on v4/5/6 that are affecting us

- No RIPv2 from VRF's so cannot use it for PE-CE
- Cannot view BGP advertisements sent to/received from a peer when running PE-CE from a VRF
- Cannot view L2VPN information sent/received from a BGP peer
- Cannot view L3VPN information sent/received from a BGP peer
- BGP is not multi-core optimized making complex filters, full tables and large updates very very slow
- A lot of configuration is still CLI only, e.g. BGP VRF out-filters
- Config database sync issues, often the running config is different from what is shown in Winbox/CLI.
- VPLS tunnel state changes are not logged
- Can not specify which VRF router management services are available from, e.g. WinBox, SSH, Webfig
- Cannot specify which VRF PPP connections terminate to

I run witout any troble. Still - annoing to not be able to set sevice/vrf. Solution is to run more boxes, and put services utside the vrf's.

I dont run full bgp to CE, just instide core/MPLS cloud, using "ebgp" just annoncing IP's inside the actual vrf to CE ( outside the vrf itself).
l2vpn Distributed by ospf (MPLS)

Tue Feb 18, 2014 2:23 am

We are running reliably now on 6.5, we just find compared to other platforms we run (Juniper, Cisco and Extreme) that RouterOS is lacking a lot of basic "service provider" routing features.

We are currently doing BGP for PE-CE, which works fine but is a lot more configuration than using RIPv2. Ideally we would like to use BGP for PE-CE at large customer sites, and RIPv2 for smaller branch sites. This is a pretty standard model for service providers, as it means you can use engineers with less experience to configure the CE routers for the bulk of the sites.

Not being able to view advertisements on PE-CE connections is a major though, generally we do not have access to the CE routers as they are managed by a 3rd party (as is normal for service providers) and we have no idea what prefixes are being sent to/received from CE routers and if route filters are working correctly. Quite scary.

samsung172 · Wed Feb 19, 2014 1:28 am

I still cannot se exactly why CE Devices want to know about all vrf's and routing's in MPLS cloud. It should know about it own routes, and Gateways. If running a bgp outside the MPLS, but inside the vrf, CE should still have the full Routing table that is usful to Customer, and would update the vrf With internal "own" subnets/routes. I just let CE know about the default (Routing table) via ospf (if CE should have l2VPN) and just running a bgp, without internal confederation and l3VPN at last point (PE-CE) (without default Routing table). This also make the problem about what vrf to run services to to dissappere. Its a bit annoing to not have managment to Devices, but its possible to have a link net to just have mgmt, and not run full ospf/bgp/MPLS to just do managment. Usualy my vrf's is Public IP's from different ISP's and my default via ospf, is rfc1918 for ospf/MPLS/loopback. Bgp via vpn4, l2vpn-cisco and confederation inside different AS set. All MPLS routers would have its own internal AS in the confederation and produce the non agregated table for AS set that peer With global transits.

I have setup MPLS/BGP/OSPF to Cisco, juniper, and ZTE. (some test to huawei) I have seen no problems at all to Routing. (but i use as less parameters as possible). I run into more problems using rip and ospf, than bgp, connecting to other vendors equipment. And there is other more annoing problems, like 100mb/s to Cisco, that make the CCR's freeze etc. (We had 1500 CPE Down today, couse of this problem a technican put wrong GE on a cisco to 100mb/s) (No its not fixed in 6.10, like it supposed to)

Using this, i will se all annonced at CE Devices, and have prefix Count "counting" at PE. CE (and PE) would show both prefix Count and advertisements to the bgp. Usually its not that interessed, and in most cases, it could be aggregated into "pipes" since CE dont have to manny PE Devices as Gateways. The PE always shows routes annyway. Both in advertisements and VPN4 Routes.

Wed Feb 19, 2014 4:00 am

I still cannot se exactly why CE Devices want to know about all vrf's and routing's in MPLS cloud. It should know about it own routes, and Gateways. If running a bgp outside the MPLS, but inside the vrf, CE should still have the full Routing table that is usful to Customer, and would update the vrf With internal "own" subnets/routes. I just let CE know about the default (Routing table) via ospf (if CE should have l2VPN) and just running a bgp, without internal confederation and l3VPN at last point (PE-CE) (without default Routing table). This also make the problem about what vrf to run services to to dissappere. Its a bit annoing to not have managment to Devices, but its possible to have a link net to just have mgmt, and not run full ospf/bgp/MPLS to just do managment. Usualy my vrf's is Public IP's from different ISP's and my default via ospf, is rfc1918 for ospf/MPLS/loopback. Bgp via vpn4, l2vpn-cisco and confederation inside different AS set. All MPLS routers would have its own internal AS in the confederation and produce the non agregated table for AS set that peer With global transits.

The issue we have is that there is no working equivalent of the IOS/JunOS commands:

show ip bgp vpnv4 vrf vrf-nzr-wan neighbors 172.16.95.1 advertised-routes
show ip bgp vpnv4 vrf vrf-nzr-wan neighbors 172.16.95.1 received-routes
show ip bgp vpnv4 vrf vrf-nzr-wan neighbors 172.16.95.1 routes

in RouterOS. This prevents us from seeing what prefixes we are advertising/receiving/using to/from CE routers with BGP. Most of the time we are just advertising everything in the customers VRF, but occasionally we want to block leaked management ranges from being advertised to the customers CE device, at the moment we cannot easily confirm that our filters are working correctly, or what ranges we are receiving from a peer inside a routing instance.

And there is other more annoing problems, like 100mb/s to Cisco, that make the CCR's freeze etc. (We had 1500 CPE Down today, couse of this problem a technican put wrong GE on a cisco to 100mb/s) (No its not fixed in 6.10, like it supposed to)

Yes this problem is extremely annoying, we too are still experiencing this issue. Connect a CCR to a Cisco device and set the port at 100mbit FDX, after a random period of time (from within hours, to 6 weeks) connectivity will drop completely, you check the port config on the CCR and notice it somehow now has a speed of 1gbps listed, the only fix at the moment is to reboot the CCR

We had to dig a bunch of HP1800's out of retirement to sit between CCR's and Cisco devices and convert from 1gbps/AUTO to 100/FDX just so our client networks stopped dropping out.

richinuk · Sat Apr 25, 2015 1:00 am

Hey guys.

It's been a while since this was last discussed - what's the current feel on stability of the layer 3 VPN's, particularly with BGP as PE-CE protocol? Good for production?

Rich

Sat Apr 25, 2015 1:42 am

Hey guys.

It's been a while since this was last discussed - what's the current feel on stability of the layer 3 VPN's, particularly with BGP as PE-CE protocol? Good for production?
Rich

Hi Rich,

Yes it works well. We run 6.5 and 6.19 iin production with L3VPN and use BGP for PE-CE routing.

If you are filtering routes just be careful sometimes(a lot of the time) route filters won't work at all and you will need to delete them and re-re-add them. This has been an ongoing problem with RouterOS. It will likely be fixed in v7

Andrew

richinuk · Sat Apr 25, 2015 2:30 am

Sweet. Thanks for that.

Rich

StubArea51 · Tue Apr 28, 2015 8:07 pm

We have also seen a lot of stability improvements running L3VPN in RouterOS since it was fixed a few versions ago. We are also using BGP for the PE / CE handoff for most deployments.

Now we just need fast reroute

Wed Apr 29, 2015 12:27 am

Now we just need fast reroute

Also need LSP ping and TE Auto-tunnel !

Did anyone else notice mplsguy from Mikrotik has disappeared ?
He was brilliant at giving answers on the more complex aspects of RouterOS.

StubArea51 · Fri May 01, 2015 12:38 am

Interesting...I wonder who deals with MPLS more often at MikroTik? We are always working on complex multi-vendor MPLS networks that involve MikroTik and it's nice to have a product expert to bounce the high level development stuff off of.

Fri May 01, 2015 5:21 am

I have dealt with two people at Mikrotik in regards to MPLS over the past 5 years. In the last 2 years though I have only dealt with one.

Mikrotik support seem to allocate certain engineers to the more specialist areas e.g. wireless and MPLS.
For other areas I have had responses from many different people.

StubArea51 · Wed Jun 03, 2015 8:36 pm

We are working on an MPLS network that includes x86 routers and routerboards which has been running well on MPLS with VPLS on 6.24, but in converting to L3VPN recently, we experienced some issues with LDP neighbors flapping. We haven't completely ruled out MTU issues in the transport, but did see a few bugs as we experimented with different versions.

Just curious to see of those of you running MPLS networks what your experience has been using MPLS/BGP/OSPF in Versions 6.20 through 6.29

digicomtech · Wed May 11, 2016 6:04 pm

Hi,
would like to know if some have experiment issue with bug fix version 6.30 and more, using BGP/MPLS/VPNv4 ?

Don't see any other post on these since june 2015... I guess all is fine ?!

Regards,
Michael Plourde

StubArea51 · Wed May 11, 2016 6:55 pm

Many of our customers are service providers and run MPLS/BGP/OSPF on MikroTik hardware as well as integrating MikroTik with vendors like Cisco, Juniper, Brocade, etc. We have been recommending and using the bugfix version since it came out and it has been extremely successful and stable.

The latest bugfix right now is 6.32.4 and it is very stable on MPLS/BGP and OSPF networks. Be sure and migrate all of your routers to the same bugfix version as soon as it is practical to get the the best stability.

k44 · Fri Aug 19, 2016 11:33 am

Anyone here seen high CPU usage spikes in 'unclassified' due to flapping bgp vpnv4 route ?
I have few vpnv4 prefixes flapping and it terribly affects performance of my CHR (6.37rc16) installation.

buset1974 · Sat Mar 02, 2019 10:34 pm

I have dealt with two people at Mikrotik in regards to MPLS over the past 5 years. In the last 2 years though I have only dealt with one.

Mikrotik support seem to allocate certain engineers to the more specialist areas e.g. wireless and MPLS.
For other areas I have had responses from many different people.

Hi,
i have the same experience with you til today 6.44, regarding bgp withdraw issue on mpls.
once they promise it will fix on v7, but v7 never show up.
we have mikrotik mpls on our production, but to solved this issue i have to use some cisco PE instead.
problem happen if you use multihoming BGP between CE to 2 CE.

thx

Mon Mar 04, 2019 9:26 am

I have dealt with two people at Mikrotik in regards to MPLS over the past 5 years. In the last 2 years though I have only dealt with one.

Mikrotik support seem to allocate certain engineers to the more specialist areas e.g. wireless and MPLS.
For other areas I have had responses from many different people.
Hi,
i have the same experience with you til today 6.44, regarding bgp withdraw issue on mpls.
once they promise it will fix on v7, but v7 never show up.
we have mikrotik mpls on our production, but to solved this issue i have to use some cisco PE instead.
problem happen if you use multihoming BGP between CE to 2 CE.

thx

I am all too familiar with this issue.. it is a VPNV4 NLRI update problem.

Mikrotik support confirmed it back in 2014 but there is still no fix.

buset1974 · Wed Mar 06, 2019 6:15 am

I have dealt with two people at Mikrotik in regards to MPLS over the past 5 years. In the last 2 years though I have only dealt with one.

Mikrotik support seem to allocate certain engineers to the more specialist areas e.g. wireless and MPLS.
For other areas I have had responses from many different people.
Hi,
i have the same experience with you til today 6.44, regarding bgp withdraw issue on mpls.
once they promise it will fix on v7, but v7 never show up.
we have mikrotik mpls on our production, but to solved this issue i have to use some cisco PE instead.
problem happen if you use multihoming BGP between CE to 2 CE.

thx
I am all too familiar with this issue.. it is a VPNV4 NLRI update problem.

Mikrotik support confirmed it back in 2014 but there is still no fix.

Yes, in Multi homing CE, bgp attribute were ignored when best path withdraw.
prefix with longer path were chosen and last prefix advertised seem were ignored.

Can all be rewrite and fix in v6, so we don't have to wait v7 years.

thx

richinuk · Wed Apr 15, 2020 1:40 am

Hey all,

Can I confirm with you guys that this is the same issue(s) I've been trying to debug recently with duplicate prefix adverts into a VPN.

Scenario (lab) - simple 3 PE node MPLS network, and for the most part, working as expected. 2 CE's, each connected to one of the PE's. Both CE's injecting a default route 0.0.0.0/0.

Drawing2.jpg

Issue 1: The VRF has the same RD on all PE's. After PE1 and PE2 receive the default from their respective CE's, there is a back-and-forth Add/Withdraw of this route between PE1 and PE2. This cycle repeats for anything from a few seconds to hours. In fact, this happens for any prefix that is advertised into the VPN more than once.

Issue 2: I can fix issue 1 by using unique RD's per VRF on each PE, and PE3 will receive both defaults. However PE3 doesn't choose the most preferred route despite manipulation of attributes (localpref). The only thing that seems to make a difference is manipulation of the distance.

If this is a known issue, I can stop wasting time trying to debug

Cheers,
Rich

Wed Apr 15, 2020 8:03 am

Hi Rich,

Yes that sounds like a related issue.

Email support@mikrotik.com and let them know. They cant do anything to fix it in RouterOS v6, but hopefully it will put a little pressure on them to get v7 routing out the door.

richinuk · Wed Apr 15, 2020 12:31 pm

Sweet, thank-you. I can stop banging my head against the wall!

Cheers,
Rich

Wed Apr 15, 2020 1:12 pm

Regarding Issue1:
RDs should be unique, so this is not a RouterOS bug, but misconfiguration.

Regarding Issue2:
Yes we are aware of route selection problems in VRFs, unfortunately you will have to wait for ROS v7 updates.

richinuk · Thu Apr 16, 2020 3:54 am

Hi Mrz,

Certainly excellent news. I look forward to seeing the improvements.

Regarding issue 1, a single common RD is certainly a valid config and the overwhelming number of examples on the internet (including Mikrotik's documentation) use the same RD across all PE's for the same customer/VPN. Many authors recommend using a single RD in order to preserve memory on the router (although this may be less of an issue these days). In fact, it took me quite some time to find references from Cisco, et al that describes the behaviour with multihomed sites, or multiple sites advertising a default-route into the VPN, in conjunction with a single RD whereby all PE's that have multiple copies of RD/prefix in their VPNV4 table will withdraw their version of the route in favour of a single route being advertised from the preferred CE. This results in a stable convergence, but only 1 copy of the route in the VPNv4 table. The use of unique RD's per PE within the same customer VPN allows multiple copies of that route to co-exist and therefore each PE can make their own decision based on the BGP path selection algorithm and allow for faster failover. This is also the case when using a route-reflector that doesn't support multipath.

The problem I see with RouterOS (and this is the bug) is that a PE receiving a conflicting route (one from it's CE, one from the VPNv4 mesh) will withdraw its copy from the VPNv4 adverts, and 1 second later, re-advertise its copy, and 1 second later withdraw, etc. This loop can go on for minutes or hours until some race-condition seems to break the cycle. Based on what you've said about issues with VPN route selection, I suspect issue 1 is a result of the PE incorrectly selecting the preferred route and getting into a cycle of adds/withdraws.

Rich

Thu Apr 16, 2020 7:23 am

Yes we are aware of route selection problems in VRFs, unfortunately you will have to wait for ROS v7 updates.

hknet · Mon Mar 22, 2021 3:41 am

this keeps on giving it seems.

we recently tested MT's ROS6.48.1 on RB3011 as PE talking to its CE via ospf.

only when we introduced redundancy this got weird:
it seems only one PE will report its CE-routes to the route-reflectors.
as soon as we filter some CE route from being announced to the RRs, the other MT will then announce this route.
it looks like as long as a vpnv4 route is known for some subnet this box will not announce the same vpnv4 route to RRs.

also if changes are done to the MT boxes sometimes they simply loose mpls connectivity,
we even reached a moment when ldp was not working anymore and only a reboot could fix this.

so the final decision seems to be: single PE device for relatively simple MPLS - works.

any more sophisticated (sic!) setup (like redundancy) seems to break things hard.

we asked MT support for help, maybe we can get help - if it says ROS7 I guess we will have to install cisco boxes again...

hknet · Fri Mar 26, 2021 1:51 pm

we asked MT support for help, maybe we can get help - if it says ROS7 I guess we will have to install cisco boxes again...

well got this exact answer: will not be fixed in V6, will have to hope for ROSv7 to get mpls and bgp vpnv4 working in a multipath-setup.

Who is online