VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

alexioma · Wed Apr 03, 2024 4:18 pm

Dear members,

I need help with a particular setup where one MT router CCR2004 needs to be connected with multiple switches where a local lan is present with a fix subnet. In specific there are many identical subnet (each in a dedicate VLAN) with same overlapping IP subnet 192.168.1.0/24 - let's call them the "devices (or dev) vlan".

There are also several device in each of the identical VLAN with same IP address that cannot be modified. On top there is a service vlan (say 10.2.4.0/24) where my PC will be localted that needs to access each individual VLAN with a distinct IP.

My understanding is that, to achieve a working setup, VRF must be used in order to isolate routing of each individual VLAN with overlapping IP. Then mangle should be used to allow traffic from service vlan to and back from each dev vlan. NAT should then do the trick of converting a public IP reachable from main table (where service vlan is located) to the dev vlan in each VRF.

So far I was able to emulate this scenario in a simple EVE-NG lab as follows:

Screenshot 2024-04-03 at 2.59.04 PM.png

In this example:
- Router is a CHR with ROS 7.14.2
- Main is the serivce pc IP 10.2.4.10
- PC30 is one of the device PC with IP 192.168.1.10 (VRF30)
- PC40 is one of the device PC with IP 192.168.1.10 (VRF40)

This is the config of the CHR:

# 2024-04-03 13:15:19 by RouterOS 7.14.2
# software id = 
#
/ip vrf
add interfaces=ether2 name=vrf30
add interfaces=ether3 name=vrf40

/ip address
add address=10.2.4.1/24 interface=ether1 network=10.2.4.0
add address=10.2.30.1/24 interface=ether2 network=10.2.30.0
add address=10.2.40.1/24 interface=ether3 network=10.2.40.0
add address=192.168.1.1/24 interface=ether2 network=192.168.1.0

/ip firewall mangle
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.30.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf30
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf30 \
    in-interface=ether2 new-routing-mark=main passthrough=no
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.40.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf40
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf40 \
    in-interface=vrf40 new-routing-mark=main passthrough=no
    
/ip firewall nat ## possibly wrong
add action=netmap chain=dstnat dst-address=10.2.30.0/24 to-addresses=\
    192.168.1.0/24
add action=masquerade chain=srcnat out-interface=ether2

/ip route
add dst-address=10.2.30.0/24 gateway=vrf30@vrf30
add dst-address=10.2.40.0/24 gateway=vrf40@vrf40

I can get ping working from main PC to PC30 and PC40 if they are set on their "public IP" like 10.2.30.10 but cannot get it to translate if they are set to 192.168.1.10.

Any help would be of great assistance.

Thank you in advance.

A

jaclaz · Wed Apr 03, 2024 5:15 pm

Besides and before the netmap line, I don't understand what are these settings:

/ip route
add dst-address=10.2.30.0/24 gateway=vrf30@vrf30
add dst-address=10.2.40.0/24 gateway=vrf40@vrf40

With these commands:

/ip vrf
add interfaces=ether2 name=vrf30
add interfaces=ether3 name=vrf40

you are essentially creating two new routing tables called vrf30 and vrf40, to which - respectively - ether2 and ether3 belong to.

So interfaces to set as gateway should be either:
ether2@vrf30 and ether3@vrf40
or
10.2.30.1@vrf30 and 10.2.40.1@vrf40

Or did you attribute (outside the posted configuration) the vrf30 and vrf40 names to the interfaces? If you have done that, maybe you could change the naming as to avoid possible confusion.

alexioma · Thu Apr 04, 2024 1:07 am

Thanks for the reply.

vrf30 and vrf40 put in the ip/route are indeed the vrf created in ip/vrf/add

Adding vrfXX instead of the interface like ether2 or 3 is a practice mentioned in Mikrotik manual for "[url https://help.mikrotik.com/docs/pages/vi ... eId=328206]Static VRF-Lite Connected route leaking[/url]". I could easily change the configuration but:
- If I specify the interface like gateway=ether2@vrf30, it behaves in the exact same way and can ping the un-NATed IP like 10.2.40.10 (if these IP is set on the VirtPC)
- If I specify the gateway address I get an inactive route on main losing the intermediate gateway

Here's an example:

[admin@MikroTik] > ip route/pr detail                    
Flags: D - dynamic; X - disabled, I - inactive, A - active; 
c - connect, s - static, r - rip, b - bgp, o - ospf, i - is-is, d - dhcp, v - >
H - hw-offloaded; + - ecmp 
   DAc   dst-address=10.2.4.0/24 routing-table=main gateway=ether1 
         immediate-gw=ether1 distance=0 scope=10 suppress-hw-offload=no 
         local-address=10.2.4.1%ether1 

 0  IsH  dst-address=10.2.30.0/24 routing-table=main pref-src="" 
         gateway=10.2.30.1@vrf30 immediate-gw="" distance=1 scope=30 
         target-scope=10 suppress-hw-offload=no 

 1  As   dst-address=10.2.40.0/24 routing-table=main pref-src="" 
         gateway=ether3@vrf40 immediate-gw=ether3 distance=1 scope=30 
         target-scope=10 suppress-hw-offload=no 

   DAc   dst-address=10.2.40.0/24 routing-table=vrf40 gateway=ether3@vrf40 
         immediate-gw=ether3 distance=0 scope=10 suppress-hw-offload=no 
         local-address=10.2.40.1%ether3@vrf40 

   DAc   dst-address=10.2.30.0/24 routing-table=vrf30 gateway=ether2@vrf30 
         immediate-gw=ether2 distance=0 scope=10 suppress-hw-offload=no 
         local-address=10.2.30.1%ether2@vrf30

In no way I can reach the VirtPC on their native IP (i.e. 192.168.1.10) of course, after setting the virtPC on such IP.

I've tried applying dstnat and srcnat based on connection marking but this still doesn't work.

Here's the latest setup which, as before, works only on "non overlapping" and does not work with address translation to the local native IP of the device:

/ip vrf
add interfaces=ether2 name=vrf30
add interfaces=ether3 name=vrf40


/ip address
add address=10.2.4.1/24 interface=ether1 network=10.2.4.0
add address=10.2.30.1/24 interface=ether2 network=10.2.30.0
add address=10.2.40.1/24 interface=ether3 network=10.2.40.0
add address=192.168.1.1/24 interface=ether2 network=192.168.1.0

/ip firewall mangle
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.30.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf30
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf30 \
    in-interface=vrf30 new-routing-mark=main passthrough=no
add action=mark-connection chain=prerouting in-interface=vrf30 \
    new-connection-mark=from-vfr30 passthrough=no src-address=192.168.1.0/24
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.40.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf40
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf40 \
    in-interface=vrf40 new-routing-mark=main passthrough=no
add action=mark-connection chain=prerouting in-interface=vrf40 \
    new-connection-mark=from-vfr40 passthrough=no src-address=192.168.1.0/24

/ip firewall nat
add action=netmap chain=dstnat connection-mark=from-main-to-vrf30 \
    dst-address=10.2.30.10 to-addresses=192.168.1.10
add action=netmap chain=srcnat connection-mark=from-vfr30 src-address=\
    192.168.1.10 to-addresses=10.2.30.10

/ip route
add dst-address=10.2.30.0/24 gateway=ether2@vrf30
add dst-address=10.2.40.0/24 gateway=ether3@vrf40

jaclaz · Thu Apr 04, 2024 2:01 am

I see.
I wasn't sure which one of the two syntaxes (or both) were the right one, so it is:
ether2@vrf30
i.e.
interface_name@vrf_name

This way it is easier to distinguish the names.

In mangle you have
in-interface=vrf30
but do you really have an interface called vrf30?
Or you have just the ether2 one?

alexioma · Thu Apr 04, 2024 11:59 am

Thanks for the reply, but I think I found the solution.

This is my working configuration:

/ip vrf
add interfaces=ether4 name=vrf50
add interfaces=ether2 name=vrf30
add interfaces=ether3 name=vrf40

/ip address
add address=10.2.4.1/24 interface=ether1 network=10.2.4.0
add address=192.168.1.1/24 interface=ether2 network=192.168.1.0
add address=192.168.1.1/24 interface=ether4 network=192.168.1.0
add address=192.168.1.1/24 interface=ether3 network=192.168.1.0

/ip firewall mangle
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.30.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf30
add action=mark-routing chain=prerouting dst-address=10.2.30.0/24 \
    new-routing-mark=vrf30
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf30 \
    in-interface=vrf30 new-routing-mark=main passthrough=no
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.40.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf40
add action=mark-routing chain=prerouting dst-address=10.2.40.0/24 \
    new-routing-mark=vrf40
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf40 \
    in-interface=vrf40 new-routing-mark=main passthrough=no
add action=mark-connection chain=prerouting connection-state=new dst-address=\
    10.2.50.0/24 in-interface=ether1 new-connection-mark=from-main-to-vrf50
add action=mark-routing chain=prerouting dst-address=10.2.50.0/24 \
    new-routing-mark=vrf50
add action=mark-routing chain=prerouting connection-mark=from-main-to-vrf50 \
    in-interface=vrf50 new-routing-mark=main passthrough=no
/ip firewall nat
add action=netmap chain=dstnat dst-address=10.2.30.0/24 to-addresses=\
    192.168.1.0/24
add action=netmap chain=dstnat dst-address=10.2.40.0/24 to-addresses=\
    192.168.1.0/24
add action=netmap chain=dstnat dst-address=10.2.50.0/24 to-addresses=\
    192.168.1.0/24

/ip route
add disabled=no dst-address=192.168.1.0/24 gateway=ether2@vrf30
add dst-address=192.168.1.0/24 gateway=ether3@vrf40 routing-table=vrf40
add dst-address=192.168.1.0/24 gateway=ether4@vrf50 routing-table=vrf50

I was able to get this working setup thanks to this post.

Sharing here as a reference for other users.

Any suggestion on how to improve the config is welcome.

jaclaz · Thu Apr 04, 2024 12:14 pm

Happy you got it.

IMHO Sob's posts should be printed, framed and hanged on the wall (or more simply collected in an index of some sort easily findable/browsable) as they more often than not contain exceptionally useful info, very well explained.

As a general note/comment, from the very little experience with Mikrotik/RouterOS, almost anything can be done in more than one way, so - as you stated - it would be very useful to get other members opinions on the methods used and possibly suggested alternatives, but for now, if it works, it works.

You still have in mangle lines with:
in-interface=vrf30
in-interface=vrf40
in-interface=vrf50

I still believe that in your configuration an interface called vrf30, vrf40 or vrf50 does not exist, so that mangle rule may be not applied.

but it is entirely possible that there is a sort of (implicit) "aliasing" going on in the RouterOS internals.

alexioma · Wed Apr 10, 2024 6:52 pm

I concur about Sob's posts.

As for the vrf* interfaces: when you program it via terminal the vrf* interface gets autocompleted with "tab" when as an in-interface option. Thus ROS knows it - not sure why - and should consider it. With that rule, it wasn't working.

Best.

VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Re: VRF-Lite with Mangle and dstNAT to reach multiple VLANs with overlapping IPs

Who is online