Wireguard - access from VRF

osc86 · Sun Aug 18, 2024 4:59 pm

I've never thought, I have to create a thread about a simple thing like routing, but I've come to a point where I don't know how to proceed.
The setup is rather simple, I have a CHR instance running in a remote dc, and I need to get wireguard working between the CHR and a remote peer.
There's a vrf called dc that has access to the public internet. It's the only uplink, no internet connectivity in the main routing table.
I've tried adding routing/rules aswell as using mangle rules to steer the traffic without success. I see packets getting routed to the internet correctly, but the return traffic never makes it back to the wireguard interface, and I've no idea why.
I assume, that the wireguard data plane, which handels the underlay network connection including handshakes always use the main routing table. Unfortunately, this isn't documented in any way.
my config:

[sysop@CHR] /ip/vrf> /ip/route pr where routing-table=dc
Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, s - STATIC
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#     DST-ADDRESS        GATEWAY            DISTANCE
0  As 0.0.0.0/0          195.XXX.XXX.193@dc         1
  DAc 195.XXX.XXX.192/26  MGMT@dc                   0

[sysop@CHR] /ip/vrf> /routing/rule/pr where table=dc
Flags: X - disabled, I - inactive 
 4   dst-address=87.XXX.XXX.191/32 action=lookup-only-in-table table=dc

[sysop@CHR] /ip/vrf> /tool sniffer quick interface=MGMT port=51820
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE  TIME   NUM  DIR  SRC-MAC            DST-MAC            SRC-ADDRESS           DST-ADDRESS           PROTOCOL  SIZE  CPU
MGMT       3.385    1  ->   00:50:56:00:91:5E  84:C1:C1:78:98:CE  195.XXX.XXX.206:51800  87.XXX.XXX.191:51820  ip:udp     190    0
MGMT       3.402    2  <-   84:C1:C1:78:98:CE  00:50:56:00:91:5E  87.XXX.XXX.191:51820  195.XXX.XXX.206:51800  ip:udp     134    0

[sysop@CHR] /interface/wireguard/peers> pr proplist=name,last-handshake,rx,tx
Columns: NAME, RX, TX   
4 peer6   0  231.7KiB

anav · Sun Aug 18, 2024 5:26 pm

Post the CHR config and the main router config
/export file=anynameyouwish ( minus router serial #, any public WANIP information, keys etc. )

osc86 · Sun Aug 18, 2024 6:41 pm

As the config contains too much sensitive info, I've removed most parts of it. That's the config currently running on the CHR.
The remote wireguard peer is a UDM, nothing special.
As expected, it's still not working.

/interface ethernet
set [ find default-name=ether2 ] disable-running-check=no name=MGMT
set [ find default-name=ether1 ] disable-running-check=no name=PVE
/interface wireguard
add listen-port=51800 mtu=1420 name=WG.UDM
/ip smb users
set [ find default=yes ] disabled=yes
/ip vrf
add interfaces=MGMT name=dc
/snmp community
set [ find default=yes ] addresses=0.0.0.0/0 disabled=yes
add addresses=::/0 authentication-protocol=SHA1 encryption-protocol=AES name=wkkadmin security=private
/system logging action
add name=netwatch target=memory
add name=ospf target=memory
add name=firewall target=memory
add name=wireguard target=memory
/certificate settings
set crl-download=yes crl-use=yes
/ip neighbor discovery-settings
set discover-interface-list=all protocol=lldp,mndp
/ip settings
set max-neighbor-entries=8192 rp-filter=loose tcp-syncookies=yes
/interface wireguard peers
add allowed-address=0.0.0.0/0,::/0 endpoint-address=87.XXX.XXX.191 endpoint-port=51820 interface=WG.UDM name=peer6 persistent-keepalive=25s preshared-key="XXX" public-key=\
    "XXX"
/ip address
add address=195.XXX.XXX.206/26 interface=MGMT network=195.XXX.XXX.192
add address=192.168.11.7/24 interface=WG.UDM network=192.168.11.0
/ip cloud
set ddns-enabled=yes ddns-update-interval=1m update-time=yes
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip firewall address-list
add address=87.XXX.XXX.191 list=ADMIN
add address=88.XXX.XXX.54 list=ADMIN
/ip firewall filter
add action=accept chain=input comment="ESTABLISHED, RELATED, UNTRACKED" connection-state=established,related,untracked tcp-flags=""
add action=drop chain=input comment=INVALID connection-state=invalid
add action=accept chain=input comment=ADMIN src-address-list=ADMIN
add action=accept chain=input comment=ICMP protocol=icmp
add action=drop chain=input comment="INPUT DEFAULT DROP" log=yes log-prefix=INP_BLOCK
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add dst-address=0.0.0.0/0 gateway=195.XXX.XXX.193@dc routing-table=dc
/ip service
set telnet disabled=yes vrf=*1
set ftp disabled=yes
set www disabled=yes vrf=*1
set ssh vrf=dc
set www-ssl vrf=*1
set api disabled=yes vrf=*1
set winbox vrf=dc
set api-ssl disabled=yes vrf=*1
/ip smb shares
set [ find default=yes ] directory=/pub
/ip ssh
set always-allow-password-login=yes host-key-size=4096 host-key-type=ed25519 strong-crypto=yes
/ipv6 nd
set [ find default=yes ] advertise-dns=no ra-delay=0s ra-lifetime=10m
/ipv6 nd prefix default
set preferred-lifetime=10m10s valid-lifetime=25w5d
/routing rule
add action=lookup-only-in-table disabled=no dst-address=87.XXX.XXX.191/32 table=dc
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name=CHR
/system logging
add action=disk topics=critical
add action=netwatch topics=netwatch,!debug
add action=ospf topics=ospf
add action=firewall topics=firewall
add action=wireguard topics=wireguard
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system ntp server
set enabled=yes multicast=yes
/system ntp client servers
add address=17.253.34.253
add address=17.253.52.125

anav · Sun Aug 18, 2024 7:41 pm

Do you understand wireguard?
If you do then you would realize this is nonsensical for the Server for handshake
/interface wireguard peers
add allowed-address=0.0.0.0/0,::/0 endpoint-address=87.XXX.XXX.191 endpoint-port=51820 interface=WG.UDM name=peer6 persistent-keepalive=25s preshared-key="XXX" public-key=\
"XXX"

At the CHR it should be
/interface wireguard peers
add allowed-address=192.168.11.X interface=WG.UDM name=peer6 preshared-key="XXX" public-key=\
"XXX"

and where is the allowed IP settings for the client router???
/interface wireguard peers
add allowed-address=192.168.11.Y interface=WG.UDM name=peerRouter preshared-key="YYY" public-key=\
"YYY"

and where is the input chain wireguard rule to allow incoming handshakes to the CHR........
/ip firewall filter
add action=accept chain=input comment="ESTABLISHED, RELATED, UNTRACKED" connection-state=established,related,untracked tcp-flags=""
add action=drop chain=input comment=INVALID connection-state=invalid
add action=accept chain=input comment=ADMIN src-address-list=ADMIN
add action=accept chain=input comment=ICMP protocol=icmp
add action=drop chain=input comment="INPUT DEFAULT DROP" log=yes log-prefix=INP_BLOCK

Since I think this is now a prank post, I am outta here........

osc86 · Sun Aug 18, 2024 8:17 pm

yes, I do understand the concept of wireguard very well. The allowed-address parameter simply limits the ip communication INSIDE the tunnel and has nothing to do with the handshake or the underlay network communication. The CHR is initiating the connection, so I have to specify the remote peer and port.
0.0.0.0/0;::/0 allows incoming connections from any IP/IPv6 SOURCE address (again, inside the tunnel). It is not relevant for the described problem.
Furthermore, the remote peer ip address (87.XXX.XXX.191) is part of the ADMIN address list, permitted to connect.

anav · Sun Aug 18, 2024 9:10 pm

Then you have never stated clearly there is another router acting as the Wireguard server...... that is the router config I need to see.

osc86 · Mon Aug 19, 2024 3:01 pm

solved it by using a separate routing table instead of a vrf. Not an ideal solution, but at least it works.
My guess is, that this kind of setup currently just doesn't work with wireguard. Needs proper vrf support, like the other core services such as ssh, winbox, www..

divB · Tue Nov 26, 2024 2:39 am

Same issue, very badly waiting for WireGuard vrf support

nichky · Tue Nov 26, 2024 6:57 am

there is no feature for that, i get that.
But why is that hard to add it manually?

spippan · Fri Jan 03, 2025 3:46 pm

Same issue, very badly waiting for WireGuard vrf support

same.
VRF support for Wireguard Interfaces! => viewtopic.php?p=1117383#p1117383

anav · Fri Jan 03, 2025 6:13 pm

I agree nichky, seems like people just dont know how to use wireguard properly

Truth be told I havent used VRF but I think thats a BGP issue. Attempting to use BGP and wireguard VPN .........

As to my first statement, dont use overlapping subnets ;-PPP

spippan · Mon Jan 06, 2025 11:57 am

how does asking for VRF support for wireguard and allegedly "not knowing how to properly use wireguard" relate to each other??

anav · Mon Jan 06, 2025 4:28 pm

Was just poking you in the eye LOL.

spippan · Tue Jan 07, 2025 12:00 am

Wireguard - access from VRF [SOLVED]

Wireguard - access from VRF

Re: Wireguard

Re: Wireguard

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF [SOLVED]

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF

Re: Wireguard - access from VRF