Wireguard - specific setup

pawlisko · Sun Mar 27, 2022 12:43 am

Hi all,

So I am in the process of changing my setup. I've already changed RB1100AHx4Dude for RB5009, I am working on creating a proper guest network (I have Cisco Wireless - WLC + WAPs) with proper VLANs, etc. But now I want to move out from IPSec to Wireguard.
Presently I have 4 always on tunnels - 3 to different countries (Poland, Germany, UK) and one dedicated for one of my VMs at home). I am controlling country sites with IP lists. See my config (with some redactions

)

BackupWireGuard.rsc

Question: how I can do the same with WG instead of IPSec. I have some knowledge but if you could create it "for dummies" I would appreciate

Thanks,

Sob · Sun Mar 27, 2022 1:10 am

It shouldn't be difficult. Add new WG interfaces (e.g. wgPL, wgDE, wgUK), configure them according to provider's instructions, and then you should need something like:

/routing table
add fib name=vpnPL
add fib name=vpnDE
add fib name=vpnUK
/ip route
add dst-address=0.0.0.0/0 gateway=wgPL routing-table=vpnPL
add dst-address=0.0.0.0/0 gateway=wgDE routing-table=vpnDE
add dst-address=0.0.0.0/0 gateway=wgUK routing-table=vpnUK
/ip firewall mangle
add chain=prerouting in-interface=LAN connection-mark=VPN-IP-PL action=mark-routing new-routing-mark=vpnPL
add chain=prerouting in-interface=LAN connection-mark=VPN-IP-DE action=mark-routing new-routing-mark=vpnDE
add chain=prerouting in-interface=LAN connection-mark=VPN-IP-UK action=mark-routing new-routing-mark=vpnUK
/ip firewall nat
add chain=srcnat out-interface=wgPL action=masquerade 
add chain=srcnat out-interface=wgDE action=masquerade 
add chain=srcnat out-interface=wgUK action=masquerade

Mangle rules should be after your connection marking rules, and also add connection-mark=no-mark to those.

pawlisko · Sun Mar 27, 2022 2:27 am

It shouldn't be difficult. Add new WG interfaces (e.g. wgPL, wgDE, wgUK), configure them according to provider's instructions, and then you should need something like:

Awesome what about code for dedicated connection? So lets say WG interface wg-ded-US (another from my provider). This will send entire traffic for IP: 10.20.30.250 to that tunnel without mangle

Sob · Sun Mar 27, 2022 4:33 am

Same config as for others (routing table, route, masquerade), only routing rule instead of mangle rule:

/routing rule
add src-address=10.20.30.250/32 action=lookup table=vpnUS

anav · Sun Mar 27, 2022 5:00 am

Imagine if MT had firewall address lists useable in routing rules how easy this would be, no mangles..........

pawlisko · Sun Mar 27, 2022 5:11 am

Same config as for others (routing table, route, masquerade), only routing rule instead of mangle rule:

is it a way to do it via lists?

OK, let me ask a big picture.

What I have now are IPSec tunnels 3 mangeled to countries, 1 dedicated for an one IP
Sometimes I have to send entire traffic of another device (which usually is a part mangled) to that country which already has tunel, and I am doing this by creating dedicated tunnel for that list

/ip ipsec mode-config add connection-mark=VPN-IP-PL name="VPN - Poland" responder=no use-responder-dns=yes
/ip ipsec mode-config add connection-mark=VPN-IP-DE name="VPN - Germany" responder=no use-responder-dns=yes
/ip ipsec mode-config add connection-mark=VPN-IP-UK name="VPN - UK" responder=no use-responder-dns=yes
/ip ipsec mode-config add connection-mark=no-mark name="VPN - US" responder=no use-responder-dns=no
/ip ipsec mode-config add connection-mark=no-mark name="VPN - US - Dedicated" responder=no src-address-list=local-us
/ip ipsec mode-config add connection-mark=no-mark name="VPN - UK - Dedicated" responder=no src-address-list=local-uk
/ip ipsec mode-config add connection-mark=no-mark name="VPN - Poland - Dedicated" responder=no src-address-list=local-pl
/ip firewall address-list add address=10.20.30.56 list=local-us
/ip firewall address-list add address=10.20.30.56 list=Own-VPN
/ip firewall address-list add address=10.20.30.16 disabled=yes list=Own-VPN
/ip firewall address-list add address=10.20.30.16 list=local-uk
/ip firewall address-list add address=10.20.30.17 list=local-uk
/ip firewall address-list add address=10.20.30.17 disabled=yes list=Own-VPN
/ip firewall mangle add action=mark-connection chain=prerouting dst-address-list=VPN-IP-Poland new-connection-mark=VPN-IP-PL passthrough=yes src-address-list=!Own-VPN
/ip firewall mangle add action=mark-connection chain=prerouting dst-address-list=VPN-IP-Germany new-connection-mark=VPN-IP-DE passthrough=yes src-address-list=!Own-VPN
/ip firewall mangle add action=mark-connection chain=prerouting dst-address-list=VPN-IP-UK new-connection-mark=VPN-IP-UK passthrough=yes src-address-list=!Own-VPN
/system script add dont-require-permissions=yes name=IP16-17-VPN-UK-ON owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="ip ipsec peer enable \"KeepSolidVPN - UK - Dedicated\"\  ## new tunnel
    \nip firewall address-list enable 26694" ## enable Own-VPN list
/system script add dont-require-permissions=yes name=AppleTV-VPN-UK-OFF owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="ip ipsec peer disable \"KeepSolidVPN - UK - Dedicated\"\ ## delete tunnel
    \nip firewall address-list disable 26694" ## disable Own-VPN list

So normally IP 16 or 17 are going through mangle, but when I enable them at Own-VPN list and get them new tunnel they are going exclusively over there. Probably there is a smarter way to do it without the need for new tunnel.

Is there a similar way of doing this via wg through lists? Or should I use routing rules?

Sob · Sun Mar 27, 2022 5:41 am

Just use right mangle rules in right order, e.g.:

/ip firewall mangle
add chain=prerouting connection-mark=no-mark src-address-list=local-pl action=mark-connection new-connection-mark=VPN-IP-PL passthrough=yes
add chain=prerouting connection-mark=no-mark src-address-list=local-de action=mark-connection new-connection-mark=VPN-IP-DE passthrough=yes
add chain=prerouting connection-mark=no-mark src-address-list=local-uk action=mark-connection new-connection-mark=VPN-IP-UK passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-Poland action=mark-connection new-connection-mark=VPN-IP-PL passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-Germany action=mark-connection new-connection-mark=VPN-IP-DE passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-UK action=mark-connection new-connection-mark=VPN-IP-UK passthrough=yes

This way you can do anything. Priority depends on the order of rules. If you need other exceptions, just add them before these. Don't forget connection-mark=no-mark, to avoid re-marking already marked connection.

@anav: AFAIK address lists and routes live in completely different part of Linux kernel, so it may not be easy to link them together.

pawlisko · Sun Mar 27, 2022 7:16 am

This way you can do anything. Priority depends on the order of rules. If you need other exceptions, just add them before these. Don't forget connection-mark=no-mark, to avoid re-marking already marked connection.

Sorry for stupid question but what do you mean by "Don't forget connection-mark=no-mark, to avoid re-marking already marked connection"

Thanks,

Sob · Sun Mar 27, 2022 5:49 pm

It's another condition. When you have rules with passthrough=yes (and you need that), it means that processing won't stop there, but will continue with following rules. Let's say that with the six rules in my last post the first one matches (because source is in local-pl list), but the last one matches too (because destination is in VPN-IP-UK list). So first the connection gets VPN-IP-PL mark, but if there wasn't connection-mark=no-mark, then a moment later it would get VPN-IP-UK mark.

msatter · Sun Mar 27, 2022 6:15 pm

On that I have the question if there is in all the routing options in v7 something like /ip/route/rules that routes based on the connection-mark depending up or downstream?

Now in Mangle, you have put x-number of connections markers lines and the the same number of mark markers lines to be able to select which route has to be used. I mostly put those blocks in an own Chain and a Jump that filters first if traffic should be handled by that dedicated Chain.

Sob · Sun Mar 27, 2022 6:29 pm

I'm not sure what you mean, try to describe it in more detail. But generally I wouldn't say that router sees upstream and downstream as anything special, it just has some routes.

pawlisko · Sun Mar 27, 2022 10:03 pm

It's another condition. When you have rules with passthrough=yes (and you need that), it means that processing won't stop there, but will continue with following rules. Let's say that with the six rules in my last post the first one matches (because source is in local-pl list), but the last one matches too (because destination is in VPN-IP-UK list). So first the connection gets VPN-IP-PL mark, but if there wasn't connection-mark=no-mark, then a moment later it would get VPN-IP-UK mark.

/ip firewall mangle
add chain=prerouting connection-mark=no-mark src-address-list=local-pl action=mark-connection new-connection-mark=VPN-IP-PL passthrough=yes
add chain=prerouting connection-mark=no-mark src-address-list=local-de action=mark-connection new-connection-mark=VPN-IP-DE passthrough=yes
add chain=prerouting connection-mark=no-mark src-address-list=local-uk action=mark-connection new-connection-mark=VPN-IP-UK passthrough=yes
add chain=prerouting connection-mark=no-mark src-address-list=local-us action=mark-connection new-connection-mark=VPN-IP-UK passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-Poland action=mark-connection new-connection-mark=VPN-IP-PL passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-Germany action=mark-connection new-connection-mark=VPN-IP-DE passthrough=yes
add chain=prerouting connection-mark=no-mark dst-address-list=VPN-IP-UK action=mark-connection new-connection-mark=VPN-IP-UK passthrough=yes
add chain=prerouting connection-mark=VPN-IP-PL action=mark-routing new-routing-mark=vpnPL
add chain=prerouting connection-mark=VPN-IP-DE action=mark-routing new-routing-mark=vpnDE
add chain=prerouting connection-mark=VPN-IP-UK action=mark-routing new-routing-mark=vpnUK
add chain=prerouting connection-mark=VPN-IP-US action=mark-routing new-routing-mark=vpnUS

Lets assume that these are my mangle rules.
IP 200 (in local-pl list) - based on this mangle entire traffic is sent through vpnPL (regardless if destination IP suggest another tunnel)
IP 201 (in local-uk list) - based on this mangle entire traffic is sent through vpnUK (regardless if destination IP suggest another tunnel)
IP 202 (in local-us list) - based on this mangle entire traffic is sent through vpnUS (regardless if destination IP suggest another tunnel)
IP 203 (in local-de list) - based on this mangle entire traffic is sent through vpnDE (regardless if destination IP suggest another tunnel)
IP 204 (in no local-* lists) - based on this mangle traffic goes to vpnPL when requested IP is in VPN-IP-Poland, to vpnUK when requested IP is in VPN-IP-UK, to WAN when IP is not in VPN-IP-* lists
IP 205 (in no local-* lists) - based on this mangle traffic goes to WAN (destination IP not in any list VPN-IP-* lists - i.e. 1.1.1.1)
IP 206 (in local-us list) - based on this mangle traffic goes to vpnUS (same destination address as IP205 - i.e 1.1.1.1)

Am I right?

msatter · Sun Mar 27, 2022 10:59 pm

I'm not sure what you mean, try to describe it in more detail. But generally I wouldn't say that router sees upstream and downstream as anything special, it just has some routes.

I mention upstream because it makes a difference in which direction you route. Connection marking lives in connection tracking and looks solely at the connection...in any direction.

Searching docs, I did not find connection/routing mark being mentioned, other then in Mangle.

Sob · Mon Mar 28, 2022 5:07 am

@pawlisko: If 206 is in local-us, then it will be the same as 202, i.e. third rule will match and it will get connection mark from it. It's simple whatever matches first will be used.

@msatter: Unless I'm misundertanding you, router doesn't care about direction, it has routes and uses them. Yes, default one covers more addresses than others, but I wouldn't say that router is moved by that very much.

pawlisko · Mon Mar 28, 2022 6:07 am

@pawlisko: If 206 is in local-us, then it will be the same as 202, i.e. third rule will match and it will get connection mark from it. It's simple whatever matches first will be used.

/interface wireguard add listen-port=51821 mtu=1420 name=KeepSolidVPN-Germany
/interface wireguard add listen-port=51822 mtu=1420 name=KeepSolidVPN-P2P
/interface wireguard add listen-port=51823 mtu=1420 name=KeepSolidVPN-Poland
/interface wireguard add listen-port=51824 mtu=1420 name=KeepSolidVPN-UK
/routing table add comment="Table for WireGuard - Poland" disabled=no fib name=wg-pl
/routing table add comment="Table for WireGuard - Germany" disabled=no fib name=wg-de
/routing table add comment="Table for WireGuard - UK" disabled=no fib name=wg-uk
/routing table add comment="Table for WireGuard - P2P" disabled=no fib name=wg-p2p
/interface wireguard peers add allowed-address=0.0.0.0/0 endpoint-address=*CUT-Private* endpoint-port=51820 interface=KeepSolidVPN-Poland persistent-keepalive=25s
/interface wireguard peers add allowed-address=0.0.0.0/0 endpoint-address=*CUT-Private* endpoint-port=51820 interface=KeepSolidVPN-UK persistent-keepalive=25s
/interface wireguard peers add allowed-address=0.0.0.0/0 endpoint-address=*CUT-Private* endpoint-port=51820 interface=KeepSolidVPN-Germany persistent-keepalive=25s
/interface wireguard peers add allowed-address=0.0.0.0/0 endpoint-address=*CUT-Private* endpoint-port=51820 interface=KeepSolidVPN-P2P persistent-keepalive=25s
/ip address add address=*CUT-Private* interface=KeepSolidVPN-Poland network=*CUT-Private*
/ip address add address=*CUT-Private* interface=KeepSolidVPN-UK network=*CUT-Private*
/ip address add address=*CUT-Private* interface=KeepSolidVPN-Germany network=*CUT-Private*
/ip address add address=*CUT-Private* interface=KeepSolidVPN-P2P network=*CUT-Private*
/ip firewall address-list add address=10.20.30.16 list=local-uk
/ip firewall address-list add address=10.20.30.90 list=local-pl
/ip firewall address-list add address=10.20.30.56 list=local-p2p
*CUT-Obviously more IPs from local network*
/ip firewall address-list add address=64.18.128.0/20 comment=POLAND list=IP-Poland
/ip firewall address-list add address=217.197.80.0/20 comment=GERMANY list=IP-Germany
/ip firewall address-list add address=103.69.204.0/22 comment="UNITED KINGDOM" list=IP-UK
*CUT-22k addresses, changed list name - removed VPN- as it was confusing with connection marks*
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=VPN-IP-PL passthrough=yes src-address-list=local-pl
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=VPN-IP-UK passthrough=yes src-address-list=local-uk
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=VPN-IP-P2P passthrough=yes src-address-list=local-p2p
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=IP-Poland new-connection-mark=VPN-IP-PL passthrough=yes
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=IP-Germany new-connection-mark=VPN-IP-DE passthrough=yes
/ip firewall mangle add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-list=IP-UK new-connection-mark=VPN-IP-UK passthrough=yes
/ip firewall mangle add action=mark-routing chain=prerouting connection-mark=VPN-IP-PL new-routing-mark=wg-pl passthrough=yes
/ip firewall mangle add action=mark-routing chain=prerouting connection-mark=VPN-IP-DE new-routing-mark=wg-de passthrough=yes
/ip firewall mangle add action=mark-routing chain=prerouting connection-mark=VPN-IP-UK new-routing-mark=wg-uk passthrough=yes
/ip firewall mangle add action=mark-routing chain=prerouting connection-mark=VPN-IP-P2P new-routing-mark=wg-p2p passthrough=yes
/ip firewall nat add action=masquerade chain=srcnat comment="Default Masqarade Rule" out-interface=WAN
/ip firewall nat add action=masquerade chain=srcnat out-interface=KeepSolidVPN-Poland
/ip firewall nat add action=masquerade chain=srcnat out-interface=KeepSolidVPN-Germany
/ip firewall nat add action=masquerade chain=srcnat out-interface=KeepSolidVPN-UK
/ip firewall nat add action=masquerade chain=srcnat out-interface=KeepSolidVPN-P2P
/ip firewall nat add action=masquerade chain=srcnat comment="Hairpin NAT Rule" dst-address=!10.20.30.1 src-address=10.20.30.0/23
/ip firewall nat add action=masquerade chain=srcnat comment="Hairpin NAT Rule - WG Home" dst-address=!10.20.50.1 src-address=10.20.50.0/24
/ip firewall nat add action=dst-nat chain=dstnat comment="WireGuard on MikroTik - Home Main Network" dst-port=52850 protocol=udp to-addresses=10.20.50.1 to-ports=52850
/ip firewall nat add action=dst-nat chain=dstnat comment="WireGuard on MikroTik - Home Guest Network" dst-port=52860 protocol=udp to-addresses=10.20.60.1 to-ports=52860
/ip route add disabled=no dst-address=10.20.50.0/24 gateway=10.20.30.1 routing-table=main suppress-hw-offload=no
/ip route add disabled=no dst-address=10.20.60.0/24 gateway=10.20.30.1 routing-table=main suppress-hw-offload=no
/ip route add dst-address=0.0.0.0/0 gateway=KeepSolidVPN-UK routing-table=wg-uk
/ip route add dst-address=0.0.0.0/0 gateway=KeepSolidVPN-P2P routing-table=wg-p2p
/ip route add dst-address=0.0.0.0/0 gateway=KeepSolidVPN-Germany routing-table=wg-de
/ip route add dst-address=0.0.0.0/0 gateway=KeepSolidVPN-Poland routing-table=wg-pl
/routing rule add action=lookup disabled=no src-address=10.20.30.56/32 table=wg-uk

So this is my config and it does not work.

I know that all wireguard connections are set up properly, but there is some sort of error in mangles and basically, connections are timing out. If I disable mangles then I can do dedicated from 10.20.30.56 (see /routing rule) and it works like charm (confirmed with different IP checkers. I checked on other computers and it is only affecting IP-Country lists. Anything else works perfectly with WAN connection.

Where did I made an error?

Sob · Tue Mar 29, 2022 4:51 am

It's those rules with action=mark-routing, they currently work for both directions, but you want them only for outgoing traffic from LAN, so add in-interface=LAN to them. And they can also have passthrough=no (which will speed up processing by 0.000something%, so nothing to really care about, but passthrough=yes is useless for them; but only for these four).

pawlisko · Wed Mar 30, 2022 4:49 am

It's those rules with action=mark-routing, they currently work for both directions, but you want them only for outgoing traffic from LAN, so add in-interface=LAN to them. And they can also have passthrough=no (which will speed up processing by 0.000something%, so nothing to really care about, but passthrough=yes is useless for them; but only for these four).

@Sob

First of all - huge thank you. It is working now as designed.
Second - I didn't do any speed comparison testing with IPSec but will report back whenever I will do it. My IPSec tunnels (AES-256-GCM) @600Mbps were drawing about 45% CPU usage.

Last but not least - is there a way to create IP lists made of other lists?
I need this to create one firewall filter instead of a few.
Example: I have lists: local-pl, local-uk, local-p2p - is there a way to create a single list i.e. "locals" that consist of all those lists so I can put it in a single firewall filter instead of creating multiple lists?

Sob · Wed Mar 30, 2022 5:05 am

Unfortunately, there's no list of lists so far. The sad part is that Linux (which RouterOS is based on) supports it, together with other useful list types, but they are not exposed in RouterOS. Hopefully one day...

pawlisko · Thu Mar 31, 2022 1:13 am

Unfortunately, there's no list of lists so far. The sad part is that Linux (which RouterOS is based on) supports it, together with other useful list types, but they are not exposed in RouterOS. Hopefully one day...

So let me ask you even crazier question - do you know how to create list with MAC addresses?

I need to do list with IPv6 and I have stateless address allocation. So the only way to limit connectivity is through a MAC address. Any idea how to do it there?

Sob · Thu Mar 31, 2022 4:52 am

AFAIK, there's no such thing in RouterOS.

pawlisko · Mon Apr 04, 2022 3:13 pm

AFAIK, there's no such thing in RouterOS.

Based on this thread I created Reddit post: https://www.reddit.com/r/mikrotik/comme ... s_with_vpn

Hopefully, it will be a good guide for people like me looking for how to create and use wg VPN service using MT.

@Sob - if you are on reddit and want to be tagged there please DM with details.

Wireguard - specific setup

Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup

Re: Wireguard - specific setup