VRF routing issue on 7.14

IOException · Fri Mar 01, 2024 12:30 am

Hello. I have RB4011 and just upgraded from 7.13.5 to 7.14 and got very confusing issue.
I have a Wireguard interface which belongs to a VRF, and I have a route to 0.0.0.0/0 in vrf_table through that wg_tunnel. I tried to repeat the issue in a more clear environment with hap ac^2 and issue confirmed. That's my config (everything other is just default):

/interface wireguard
add listen-port=13231 mtu=1420 name=wg_interface
/interface vlan
add interface=bridge name=vlan10 vlan-id=10
/interface bridge vlan
add bridge=bridge tagged=bridge,ether5 vlan-ids=10
/ip vrf
add interfaces=wg_interface,vlan10 name=vrf_table
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
wg_interface@vrf_table pref-src=0.0.0.0 routing-table=vrf_table scope=30 \
suppress-hw-offload=no target-scope=10 vrf-interface=wg_interface
/ip dhcp-client
add comment=defconf interface=ether1

And everything works as intended: wg_interface connects to a peer via ISP and provides access to /0 for vlan10, and other clients use dynamic route to 0.0.0.0/0 which is created from DHCP client (use ISP directly).

After updating to 7.14 I see that in the routing table:

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
wg_interface@vrf_table pref-src=0.0.0.0 routing-table=vrf_table scope=30 \
suppress-hw-offload=no target-scope=10 vrf-interface=wg_interface

When I export config, BUT in winbox this route is shown as belonged to the main table and also works that way! I have no internet access anymore fot anyone, router can't check updates, wg_interface doesnt connect until I disable this rule. After reverting back to 7.13.5 everything goes back to normal.

Is that my config broken (this routing rule) or something wrong with the 7.14 update?
Probably, I shouldn't specify vrf-interface option, so what is that option for and why everything is fine with 7.13.5 and not with 7.14?
Thanks.

accarda · Fri Mar 01, 2024 7:42 am

Actually I think v7.14 creates several issues with interfaces and VRF.
I have issues in my config with IPIP interfaces not working properly after v7.14.
Basically the IPIP interface does not get installed in the VRF table, but it stays in the main routing table.
Also at startup I get an error that using VRF while setting the IPIP interface (in the remote address side) to check whether VRF is enabled, which it is.
And finally also IP service being set on VRF does not respond to requests anymore.
So I switched back to 7.13.5 for now and everything returned working fine as before.
I have reported this both to support and to 7.14 forum post (viewtopic.php?t=205097&sid=60a2235908db ... e#p1059546); I can see others have the same issues with VRF and some type of interface.
In my case VLAN and L2TP worked fine with VRF and 7.14, but not IPIP tunnel.

nichky · Fri Mar 01, 2024 12:26 pm

clear we can see those two interfaces are in to the main table as they have +.
So your VRF is not working that for sure.

accarda · Fri Mar 01, 2024 1:46 pm

I have re-installed again v7.14 so that now I was able to generate a supout to provide Mikrotik.
In my case once the router restarts I get these type of error messages related to the IPIP interfaces:
failed to set vrf for interface ipip-if, please check if selected vrf instance is enabled

Hopefully Mikrotik will be able to reproduce it to find a fix for it.

jordanp123 · Fri Mar 01, 2024 2:35 pm

I'm with you 100% experiencing the exact issue. I've had to downgrade all of my gear that I had initially upgraded until something is figured out. In my case the IP address route that is connected to the WG interface is dynamically assigned to the main routing table even when that interface belongs to my separate VRF and not to the main.

mrz · Fri Mar 01, 2024 2:54 pm

Thanks, confirmed.

IOException · Fri Mar 01, 2024 6:48 pm

Also seems like when I remove vtf-interface parameter, everything gets fine on 7.14.
What is that option for, where can I find explaination? Didn't see those in help.mikrotik.com, what changes when I leave that parameter empty? Considering that I specify gateway with vrf_table.

mrz · Fri Mar 01, 2024 6:56 pm

vrf-interface parameter is just another way to say in which vrf route will be added. VRF is then chosen based on which VRF interface belongs, but since after reboot IPIP tunnel is assigned to "main", route is also moved to main.

aburnedinpixel · Fri Mar 01, 2024 10:07 pm

I reported this for 7.14beta9 (SUP-142491), not happy to still get this on stable. At least now it's probably getting fixed soon.

nichky · Sat Mar 02, 2024 4:43 am

i'm confused with that feature. e.g. once u establish pppoe- client automatically is ading in to vrf interface even though it is in to the main table.

Didnt check with dhcp-client , but im predy sure that is behavior the same

Jamesits · Sat Mar 02, 2024 4:28 pm

7.14 completely breaks my VRF setup which has been working fine since 7.x. Wireguard connected route now goes to main instead of its associated VRF. And (maybe due to the first issue) one of my VRF default routes is automatically moved to main during the upgrade, breaking my Internet access.

Railander · Wed Mar 06, 2024 7:09 pm

had same issue, here's the workaround i found.

after updating to 7.14, noticed that the addesses in EoIP interfaces members of the VRF were falling on main VRF instead of created VRF, thus causing the issue, as reported by others.
upgrading to 7.15 beta did not solve it.

the solution was deleting EoIP and creating again and putting it back in VRF.

accarda · Fri Mar 08, 2024 11:37 am

I received a notification from Mikrotik about the case I submitted that they fixed it and will be released soon.

MightySlaytanic · Wed Jul 24, 2024 11:39 pm

I've just updated from 7.12 to 7.15.2 and it broke something on my VRF setup.
I have a lan_bridge in vlan 254 along with eth4 and I've created an interface vlan lan_bridge.254 with an IP address that usually was placed in a specific vrf along with a pope interface. The hosts on eth4 vlan 254 can not ping anymore if they lan_bridge.254 interface is in the VRF, while if I move the interface outside the VRF it can be pinged. Did you find any solution to the VRF problems related to 7.14 and 7.15?

/ip vrf
add interfaces=pppoe-out-rasp,lan_bridge.254 name=VRF-RASP

/interface vlan
add comment="254 Vlan" interface=lan_bridge name=lan_bridge.254 vlan-id=254

/interface bridge vlan
add bridge=lan_bridge tagged=lan_bridge,eth4 vlan-ids=254

/ip address
add address=192.168.254.1/24 comment="254 Network" interface=lan_bridge.254 network=192.168.254.0

If I disable VRF-RASP then hosts on eth4 can ping 192.168.254.1

MightySlaytanic · Mon Jul 29, 2024 6:52 pm

My issue could be related to the following behavior change starting from 7.14:

Started from version 7.14 when interfaces are added in VRF - virtual VRF interface is created automatically. If it is needed to match traffic which belongs to VRF interface, VRF virtual interface should be used in firewall filters, for example:

/ip vrf add interfaces=ether5 name=vrf5
/ip firewall filter add chain=input in-interface=vrf5 action=accept

If there are several interfaces in one VRF but it is needed to match only one of these interfaces - marks should be used. For example:

/ip vrf add interface=ether15,ether16 vrf=vrf1516
/ip firewall mangle
add action=mark-connection chain=prerouting connection-state=new in-interface=ether15 new-connection-mark=input_allow passthrough=yes
/ip firewall filter
add action=accept chain=input connection-mark=input_allow

This is totally nonsense IMHO, we should be able to match specific in/out interfaces even if they’re in VRF as always, without messing with mangle table

MightySlaytanic · Tue Jul 30, 2024 7:47 pm

I can confirm that my vrf setup was broken due to the change in firewall filter table where you can no more match specific input/output interfaces if those interfaces are put within a VRF.

For more info see the note about 7.14 and VRF here: https://help.mikrotik.com/docs/pages/vi ... eId=328206

clambert · Tue Aug 06, 2024 4:43 pm

Does anyone have any updates regarding this issue? We are still experiencing issues with VRF on 7.15.3.

nichky · Wed Aug 07, 2024 6:19 am

what exactly ?

clambert · Wed Aug 07, 2024 2:12 pm

We have issues with certain interfaces working and others not within the same VRF. We have created a ticket [SUP-160981], but we have not yet received a response from Mikrotik.

nichky · Wed Aug 07, 2024 2:15 pm

curious to hear more about that.
Can we get more details of the issues that you are experiencing

clambert · Wed Aug 07, 2024 7:07 pm

We have been observing that in some occasions, certain interfaces work and others do not within the same VRF. We have seen this behavior both on devices that were upgraded from v6 to v7, and on devices configured from scratch on v7.
Below is a basic configuration that does not work. It is worth noting that if we remove the VRF interface, the ping works correctly:

/ip vrf
add interfaces=IL_VRF_TEST name=VRF_TEST
...
/interface list
add name=IL_VRF_TEST
/interface list member
add interface=sfp.111 list=IL_VRF_TEST
...
/ip address
add address=10.1.1.9/30 interface=sfp.111 network=10.1.1.8

Below are the connectivity tests:

> ping 10.1.1.10 src-address=10.1.1.9 vrf=VRF_TEST count=1 
Columns: SEQ, HOST, STATUS
SEQ  HOST           STATUS 
  0  10.1.1.10  timeout

If we remove the interface from the VRF (removing it from the interface list), the connectivity is correct:

> ping 10.1.1.10 src-address=10.1.1.9  count=1            
Columns: SEQ, HOST, SIZE, TTL, TIME
SEQ  HOST           SIZE  TTL  TIME 
  0  10.1.1.10    56   64  414us

Apachez · Wed Aug 07, 2024 8:28 pm

Do you get the same if you dont use an interface list but like manually add each and every interface to the VRF (lets start with just 2 of them or so)?

As a bonus question on similar topic - is it possible to (similar to how you can on Arista and VyOS) get to the bash mode when logged in to a Mikrotik?

Would be handy to see what happens in the "backend" of the Linux kernel and its settings when these errors occurs (I have for example one that the DNS service doesnt work with VRF's which is also confirmed by others but still zero response from Mikrotik on that case).

clambert · Wed Aug 07, 2024 9:30 pm

The behavior is the same both when using interface lists and when explicitly declaring interfaces in the VRF configuration section.
As far as I know, it is not possible to access the underlying linux console in RouterOS (it would be great!).

nichky · Thu Aug 08, 2024 2:55 am

@clambert

im doing this on 7.15.3

/interface list
add name=IL_VRF_TEST

/ip vrf
add interfaces=IL_VRF_TEST name=VRF_TEST

/interface list member
add interface=ether1 list=IL_VRF_TEST

/ip address
add address=10.10.10.1/30 interface=ether1 network=10.10.10.0

> ping 10.10.10.2 vrf=VRF_TEST   src-address=10.10.10.1 count=1
Columns: SEQ, HOST, SIZE, TTL, TIME
SEQ  HOST        SIZE  TTL  TIME    
  0  10.10.10.2    56   64  5ms506us

clambert · Thu Aug 08, 2024 2:45 pm

@Nichky, I'm also running 7.15.3. I removed the configuration and reconfigured it in the order you show, but it still doesn't work.

nichky · Thu Aug 08, 2024 2:50 pm

can we get export file of both routers?

clambert · Thu Aug 08, 2024 5:33 pm

Exploring the configuration I found that by disabling the following queue the ping began to respond:

/queue simple
add max-limit=1500M/1500M name=QUEUE-TEST target=sfp.111

Below are the connectivity tests:

> ping 10.1.1.2 vrf=VRF_TEST src-address=10.1.1.1 count=1
Columns: SEQ, HOST, SIZE, TTL, TIME
SEQ  HOST           SIZE  TTL  TIME 
  0  10.1.1.2   56   64  232us

I cannot understand the relationship that may exist.

jaclaz · Thu Aug 08, 2024 8:25 pm

But it should be an "old" (evidently not resolved) issue:
viewtopic.php?t=201990

clambert · Fri Aug 09, 2024 2:17 pm

I hadn't found that post. Obviously it's the same problem.

nichky · Fri Aug 09, 2024 2:31 pm

@clambert

i think that totally makes sense, since that interface is in the VRF.
At the QoS have you tried to use Targed=VRF_TEST instead?

clambert · Fri Aug 09, 2024 3:04 pm

The problem I find in using the VRF_TEST VRF interface as the target is that within the VRF there are several interfaces, which have different bandwidth control policies.

jaclaz · Fri Aug 09, 2024 3:21 pm

target=sfp.111@VRF_TEST ?

clambert · Fri Aug 09, 2024 3:39 pm

@jaclaz

> queue/simple/set 0 target=sfp.111@VRF_TEST
invalid value for argument target:
    input does not match any value of target-interface
    invalid value for argument target-address:
        value of ip-address must contain '/'
        value of ipv6-address expects IPv6 prefix

The documentation talks about using mangle rules to mark traffic, but for it to work in simple queues, they can only be used in the prerouting chain. This means that I cannot apply marking to traffic going out the sfp.111 interface.

jaclaz · Fri Aug 09, 2024 3:51 pm

Well, at least we tried.
It would have been too d@mn simple to have the interface notation that is just fine in route/gateway also work in queues.

clambert · Fri Aug 09, 2024 6:22 pm

I was able to find a workaround to control outgoing traffic through the interface:

/ip firewall filter
add action=drop chain=output comment=QUEUE-TEST limit=!1500M,1500M:bit out-interface=sfp.111

VRF routing issue on 7.14 [SOLVED]

VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14 [SOLVED]

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Re: VRF routing issue on 7.14

Who is online