Wireguard - branches to main site - one works, other not

BrandonSk · Sat Feb 10, 2024 11:38 pm

Hello.
I am switching from ipsec to wireguard. But for some reason one branch can communicate with the main location while the other branch not. Configs seem to me identical, but maybe I am missing something.

Sites overview
Site 1 - main site:
Public IP: 1.1.1.1
Local networks: 10.201.22.0/24, 10.201.25.0/24, ... (+ some other, but not relevant for this topic)
Wireguard interface (wgEDI) addresses:
172.16.0.1/30
172.16.1.1/30
172.16.3.1/30
Port: 12321 (udp allowed on firewall input)
Static routes defined for branches' LANs with gateway being the wireguard interface wgEDI

Site 2 - working branch:
Public IP: 2.2.2.2
Local networks: 10.201.1.0/24
Wireguard interface (wgSKL) addresses:
172.16.1.2/30
Port: 12321 (udp allowed on firewall input)
Static routes to 10.201.22.0/24 and to 10.201.25.0/24 defined with gateway wgSKL

Site 3 - not working branch:
Public IP: NO (behind NAT)
Local networks: 10.201.3.0/24
Wireguard interface (wgHOD) addresses:
172.16.3.2/30
Port: 12321 (udp allowed on firewall input)
Static routes to 10.201.22.0/24 and to 10.201.25.0/24 defined with gateway wgHOD
On ether1 ("WAN") I have address 192.168.100.2 and gateway 192.168.100.1. This is from ISP's router. I have full access to that router, but I cannot replace it with Mikrotik. So I must be behind it's NAT. I have disabled firewall on that router just to make sure the issue is not there.
The actual WAN address for that ISP router is dynamic one.
[ ISP ] <---> [ Zyxel ISP router ] (192.168.100.1) <---> (192.168.100.2) [ Mikrotik ] (10.201.3.1) <---> [ LAN 10.201.3.0/24 ]

CONFIGs
Site1

# 2024-02-10 20:37:22 by RouterOS 7.13.2
# software id = GEFA-6CF8
#
# model = RB4011iGS+
# serial number = D4440D1022D0
#NOTE: Do not mind the bridge and vlan setups = work in progress...
/interface bridge
add admin-mac=08:55:31:12:92:3B auto-mac=no comment=defconf name=bridge.local \
    priority=0x9000
add name=bridge.servers
/interface ethernet
set [ find default-name=ether1 ] name=e1.WAN
set [ find default-name=ether2 ] comment="LAN - trunk to crs109 - 22, 1978" \
    name=e2.t.crs109
set [ find default-name=ether3 ] comment="LAN - tbd - free" name=e3
set [ find default-name=ether4 ] name=e4.srvs.private.vlan
set [ find default-name=ether5 ] name=e5.free
/interface wireguard
add listen-port=13231 mtu=1420 name=wgEDI
/interface vlan
add comment="LAN + WiFi" interface=bridge.local name=vlan22 vlan-id=22
add interface=bridge.local name=vlan25 vlan-id=25
add comment="Servers private vlan" interface=e4.srvs.private.vlan name=\
    vlan1112 vlan-id=1112
add comment=Management interface=bridge.local name=vlan9999 vlan-id=1978
/interface list
add comment=defconf name=WAN
add comment="Local + VPN clients" include=dynamic name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec profile
add dh-group=modp1024 enc-algorithm=aes-128 name=prof-Sushinet-S2S
/ip ipsec peer
add address=2.2.2.2/32 disabled=yes exchange-mode=ike2 name=SKL \
    profile=prof-Sushinet-S2S
add address=3.3.3.3/19 exchange-mode=ike2 name=HOD passive=yes \
    profile=prof-Sushinet-S2S send-initial-contact=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256,sha1 enc-algorithms=\
    aes-256-cbc,aes-128-cbc
add auth-algorithms="" enc-algorithms=aes-128-gcm lifetime=4h name=\
    prop-Sushinet-S2S pfs-group=none
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp_servers ranges=10.201.25.100-10.201.25.254
add name=dhcp_emerg.mgmt ranges=192.168.98.2-192.168.98.254
add name=dhcp_lan.and.wifi ranges=10.201.22.100-10.201.22.254
add name=dhcp_management ranges=10.99.99.100-10.99.99.254
add name=pool_l2tp ranges=10.201.22.70-10.201.22.79
add name=dhcp_servers.private ranges=10.11.12.200-10.11.12.220
/ip dhcp-server
add address-pool=default-dhcp disabled=yes interface=bridge.local lease-time=\
    10m name=defconf
add address-pool=dhcp_servers interface=bridge.servers name=dhcp.servers
add address-pool=dhcp_emerg.mgmt interface=sfp-sfpplus1 name=\
    dhcp.emer.management
add address-pool=dhcp_lan.and.wifi interface=vlan22 name=dhcp.vlan22
add address-pool=dhcp_management interface=vlan9999 name=dhcp.management
add address-pool=dhcp_servers.private interface=vlan1112 name=\
    dhcp.private.servers
/ppp profile
set *FFFFFFFE bridge=bridge.local dns-server=10.201.22.1,10.201.25.5 \
    local-address=10.201.22.1 remote-address=pool_l2tp
/interface bridge port
add bridge=bridge.local comment=defconf interface=e2.t.crs109
add bridge=bridge.local comment=defconf interface=e3
add bridge=bridge.local comment=defconf interface=e5.free
add bridge=bridge.servers comment=defconf interface=ether6
add bridge=bridge.servers comment=defconf interface=ether7
add bridge=bridge.servers comment=defconf interface=ether8
add bridge=bridge.servers comment=defconf interface=ether9
add bridge=bridge.servers comment=defconf interface=ether10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge.local comment="LAN & WiFi" tagged=e2.t.crs109,bridge.local \
    vlan-ids=22
add bridge=bridge.local comment=Management tagged=e2.t.crs109,bridge.local \
    vlan-ids=1978
add bridge=bridge.local comment="Servers private" tagged=e4.srvs.private.vlan \
    vlan-ids=1112
add bridge=bridge.local comment=Servers tagged=e2.t.crs109,bridge.local \
    vlan-ids=25
/interface l2tp-server server
set enabled=yes use-ipsec=yes
/interface list member
add comment=defconf interface=bridge.local list=LAN
add comment=defconf interface=e1.WAN list=WAN
add interface=bridge.servers list=LAN
add interface=vlan22 list=LAN
add interface=vlan9999 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
/interface wireguard peers
add allowed-address=172.16.0.2/32,10.201.19.0/24 comment="PST mAPlite" \
    interface=wgEDI public-key=\
    "9999..."
add allowed-address=172.16.1.2/32,10.201.1.0/24 comment=SKL \
    endpoint-address=2.2.2.2 endpoint-port=13231 interface=wgEDI \
    public-key="2222..."
add allowed-address=172.16.3.2/32,10.201.3.0/24,192.168.100.0/24 comment=\
    HOD interface=wgEDI public-key=\
    "3333..."
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=\
    bridge.local network=192.168.88.0
add address=1.1.1.1 comment="ISP public IP" interface=e1.WAN \
    network=1.1.1.1
add address=10.201.25.1/24 comment="LAN - servers" interface=bridge.servers \
    network=10.201.25.0
add address=192.168.98.1/24 comment="emerg. management" interface=\
    sfp-sfpplus1 network=192.168.98.0
add address=10.201.22.1/24 comment="VLAN + wifi" interface=vlan22 network=\
    10.201.22.0
add address=10.99.99.1/24 comment="VLAN Management" interface=vlan9999 \
    network=10.99.99.0
add address=10.11.12.1/24 comment="Servers private vlan" interface=vlan1112 \
    network=10.11.12.0
add address=172.16.0.1/30 comment="Wireguard Local" interface=wgEDI network=172.16.0.0
add address=172.16.3.1/30 interface=wgEDI network=172.16.3.0
add address=172.16.1.1/30 interface=wgEDI network=172.16.1.0
/ip dhcp-client
add comment=defconf interface=e1.WAN
add interface=e4.srvs.private.vlan
/ip dhcp-server network
add address=10.11.12.0/24 comment="Servers private" dns-server=10.11.12.1 \
    gateway=10.11.12.1
add address=10.99.99.0/24 gateway=10.99.99.1
add address=10.201.22.0/24 dns-server=10.201.25.5,88.212.8.8,88.212.8.88 \
    gateway=10.201.22.1
add address=10.201.25.0/24 dns-server=10.201.25.5,88.212.8.8,88.212.8.88 \
    gateway=10.201.25.1
add address=192.168.88.0/24 comment=defconf dns-server=\
    10.201.25.5,192.168.88.1 gateway=192.168.88.1
add address=192.168.98.0/24 gateway=192.168.98.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
add forward-to=10.201.25.5 regexp=".*\\.mydomain\\.com\$" type=FWD
/ip firewall address-list
add address=10.201.19.0/24 list="PST"
add address=10.201.22.90-10.201.22.99 list=Management
add address=10.201.22.129 comment=Test list=Management
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=accept chain=input dst-port=8291 in-interface-list=LAN \
    protocol=tcp
add action=accept chain=input comment="Wireguard - EDI" dst-port=13231 \
    in-interface-list=WAN protocol=udp
add action=accept chain=input comment="L2TP VPN" dst-port=500,1701,4500 \
    in-interface=e1.WAN protocol=udp
add action=accept chain=input comment="L2TP VPN" in-interface=e1.WAN \
    protocol=ipsec-esp
add action=accept chain=input comment="L2TP VPN" in-interface=e1.WAN \
    protocol=ipencap
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=tcp
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=udp
add action=add-src-to-address-list address-list=HODO_new_IP \
    ...
add action=add-src-to-address-list address-list=ping-knock2 \
    ...
add action=add-src-to-address-list address-list=ping-knock1 \
    ...
add action=add-src-to-address-list address-list=ping-knock1 \
    ...
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward dst-address=10.99.99.0/24 src-address-list=\
    !Management
add action=accept chain=forward dst-address=10.201.25.0/24 in-interface=wgEDI
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add disabled=yes peer=SKL
add peer=HOD
/ip ipsec policy
add disabled=yes dst-address=\
    10.201.1.0/24 peer=SKL proposal=prop-Sushinet-S2S src-address=\
    10.201.16.0/20 tunnel=yes
add comment="HOD-LAN" dst-address=10.201.3.0/24 level=unique peer=\
    HOD proposal=prop-Sushinet-S2S src-address=10.201.16.0/20 tunnel=\
    yes
add comment="HOD-to-intermediateNAT" dst-address=\
    192.168.100.0/24 level=unique peer=HOD proposal=prop-Sushinet-S2S \
    src-address=10.201.16.0/20 tunnel=yes
/ip route
add disabled=no dst-address=10.201.1.0/24 gateway=wgEDI routing-table=\
    main suppress-hw-offload=no
add disabled=no distance=1 dst-address=10.201.3.0/24 gateway=wgEDI \
    pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=10
add disabled=no dst-address=192.168.100.0/24 gateway=wgEDI \
    routing-table=main suppress-hw-offload=no
/ppp secret
add name=xxx profile=default-encryption service=l2tp
/tool netwatch
add disabled=no down-script="/system script run newHODOip" host=10.201.3.1 \
    interval=1m timeout=1s type=simple

Site2

# 2024-02-10 20:42:35 by RouterOS 7.10.2
# software id = U7D4-T1XK
#
# model = RB750Gr3
# serial number = CC210E3A152D
/interface bridge
add admin-mac=2C:C8:1B:9F:3E:B8 auto-mac=no comment=defconf name=bridge-local
/interface l2tp-server
add name=l2tp-in1-xxx user=xxx
/interface wireguard
add listen-port=13231 mtu=1420 name=wgSKL
/caps-man datapath
add bridge=bridge-local client-to-client-forwarding=yes local-forwarding=yes \
    name=dtpSushinet
/caps-man security
add authentication-types=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm \
    name=secLAN
/caps-man configuration
add datapath=dtpSushinet mode=ap name=sushinet security=\
    secLAN ssid=Sushinet
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-192-cbc
add enc-algorithms=3des name=l2tp_vpn
/ip pool
add name=default-dhcp ranges=10.201.1.150-10.201.1.220
/ip dhcp-server
add address-pool=default-dhcp authoritative=after-2sec-delay interface=\
    bridge-local lease-time=2h name=default
/port
set 0 name=serial0
/ppp profile
add dns-server=10.201.1.1 local-address=10.201.1.1 name=sstp remote-address=\
    default-dhcp
add bridge=bridge-local local-address=10.201.1.1 name=l2tp remote-address=\
    default-dhcp
/routing bgp template
set default disabled=no output.network=bgp-networks
/routing ospf instance
add disabled=no name=default-v2
/routing ospf area
add disabled=yes instance=default-v2 name=backbone-v2
/caps-man access-list
add action=accept disabled=no interface=any signal-range=-100..120 \
    ssid-regexp=""
add action=reject disabled=no interface=any signal-range=-120..-101 \
    ssid-regexp=""
/caps-man manager
set enabled=yes
/caps-man provisioning
add action=create-dynamic-enabled master-configuration=sushinet name-format=\
    prefix-identity name-prefix=cap-SKL
/interface bridge port
add bridge=bridge-local comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge-local comment=defconf ingress-filtering=no interface=ether3
add bridge=bridge-local comment=defconf ingress-filtering=no interface=ether4
add bridge=bridge-local comment=defconf ingress-filtering=no interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface l2tp-server server
set allow-fast-path=yes default-profile=l2tp use-ipsec=required
/interface list member
add comment=defconf interface=bridge-local list=LAN
add comment=defconf interface=ether1 list=WAN
/interface ovpn-server server
set auth=sha1,md5
/interface wireguard peers
add allowed-address=172.16.1.1/32,10.201.25.0/24,10.201.22.0/24 \
    endpoint-address=1.1.1.1 endpoint-port=13231 interface=wgSKL \
    public-key="1111..."
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=\
    bridge-local network=192.168.88.0
add address=10.201.1.1/24 comment="default configuration" interface=\
    bridge-local network=10.201.1.0
add address=2.2.2.2 comment="WAN public address" interface=ether1 \
    network=2.2.2.2
add address=172.16.1.2/30 interface=wgSKL network=172.16.1.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=10.201.1.0/24 comment="default configuration" dns-server=\
    10.201.1.1 gateway=10.201.1.1
/ip dns
set allow-remote-requests=yes cache-max-ttl=2d
/ip dns static
add forward-to=10.201.25.5 regexp=".*\\.mydomain\\.com\$" type=FWD
add address=10.201.1.1 name=router
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall address-list
add address=1.1.1.1 comment=Edisonova list="Trusted IPs"
add address=1.1.1.1 comment=Edisonova list=Sushinet_Networks
add address=10.201.3.0/24 list=Sushinet_Networks
add address=10.201.1.0/24 list=Sushinet_Networks
add address=10.201.22.0/24 list=Sushinet_Networks
add address=10.201.25.0/24 list=Sushinet_Networks
add address=10.19.78.0/24 list=Sushinet_Networks
add address=192.168.88.0/24 list=Sushinet_Networks
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=drop chain=input comment="intruders DROP" src-address-list=\
    Intruders
add action=accept chain=input comment="DNS only internal requests" dst-port=\
    53 in-interface-list=!WAN protocol=tcp
add action=accept chain=input comment="DNS only internal requests" dst-port=\
    53 in-interface-list=!WAN protocol=udp
add action=accept chain=input dst-port=13231 in-interface-list=WAN protocol=\
    udp
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input dst-port=22 protocol=tcp src-address-list=\
    Sushinet_Networks
add action=accept chain=input comment="mikrotik WinBox" dst-port=8291 \
    protocol=tcp src-address-list=Sushinet_Networks
add action=accept chain=input comment="L2TP VPN" in-interface-list=WAN \
    protocol=ipsec-esp
add action=jump chain=input comment="Brute-force UDP CHECK" connection-state=\
    new dst-port=500,1701,4500 in-interface-list=WAN jump-target=ipsec_chain \
    protocol=udp
add action=accept chain=input comment=ipsec dst-port=500,1701,4500 \
    in-interface=ether1 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="SECURE DROP ALL - forward" disabled=\
    yes
add action=drop chain=input comment="SECURE DROP ALL - input" disabled=yes
add action=add-src-to-address-list address-list=Intruders \
    ...
add action=add-src-to-address-list address-list=ipsec_stage2 \
    ...
add action=add-src-to-address-list address-list=ipsec_stage1 \
    ...
add action=return chain=ipsec_chain
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip firewall raw
add action=drop chain=prerouting in-interface-list=WAN src-address-list=\
    Intruders
/ip route
add disabled=no dst-address=10.201.25.0/24 gateway=wgSKL routing-table=\
    main suppress-hw-offload=no
add disabled=no dst-address=10.201.22.0/24 gateway=wgSKL routing-table=\
    main suppress-hw-offload=no
/ppp secret
add name=xxx profile=l2tp service=l2tp
/routing bfd configuration
add disabled=no
/system identity
set name=SKL-router-hEX

Site3

# 2024-02-10 20:41:19 by RouterOS 7.13.2
# software id = KDL8-VF33
#
# model = RBD53iG-5HacD2HnD
# serial number = E7290E690E26
/interface bridge
add admin-mac=2C:C8:1B:C5:FE:22 auto-mac=no comment=defconf name=bridge \
    port-cost-mode=short
/interface wireguard
add listen-port=13231 mtu=1420 name=wgHOD
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN

/ip ipsec profile
add dh-group=modp1024 enc-algorithm=aes-128 name=prof-Sushinet-S2S
/ip ipsec peer
add address=88.212.60.238/32 exchange-mode=ike2 name=EDI profile=\
    prof-Sushinet-S2S
/ip ipsec proposal
set [ find default=yes ] disabled=yes enc-algorithms=aes-256-cbc,aes-192-cbc
add auth-algorithms="" enc-algorithms=aes-128-gcm lifetime=4h name=\
    prop-Sushinet-S2S pfs-group=none
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp ranges=10.201.3.100-10.201.3.254
/ip dhcp-server
add address-pool=dhcp interface=bridge lease-time=10m name=defconf
/routing bgp template
set default disabled=no output.network=bgp-networks
/routing ospf instance
add disabled=no name=default-v2
/routing ospf area
add disabled=yes instance=default-v2 name=backbone-v2
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=wlan1 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=wlan2 \
    internal-path-cost=10 path-cost=10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface detect-internet
set lan-interface-list=LAN wan-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=wgHOD list=LAN
/interface ovpn-server server
set auth=sha1,md5
/interface wireguard peers
add allowed-address=172.16.3.1/32,10.201.25.0/24,10.201.22.0/24 endpoint-address=\
	1.1.1.1 endpoint-port=13231 interface=\
    wgHOD public-key="1111..."
/ip address
add address=10.201.3.1/24 interface=bridge network=10.201.3.0
add address=172.16.3.2/30 interface=wgHOD network=172.16.3.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server lease
add address=10.201.3.10 client-id=1:0:1d:ec:17:3c:f8 mac-address=\
    00:1D:EC:17:3C:F8 server=defconf
add address=10.201.3.11 client-id=1:7c:dd:90:d6:dc:9f mac-address=\
    7C:DD:90:D6:DC:9F server=defconf
/ip dhcp-server network
add address=10.201.3.0/24 comment=defconf dns-server=10.201.3.1 gateway=\
    10.201.3.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip dns static
add address=192.168.88.1 comment=defconf disabled=yes name=router.lan
add forward-to=10.201.25.5 regexp=".*\\.mydomain\\.com\$" type=FWD
add address=10.201.3.1 name=router.lan
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=accept chain=input comment="Wireguard EDI" dst-port=13231 \
    in-interface-list=WAN protocol=udp
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input dst-port=500,4500 protocol=udp
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=accept chain=forward disabled=yes dst-address=10.201.16.0/20 \
    src-address=10.201.3.0/24
add action=accept chain=forward disabled=yes dst-address=10.201.3.0/24 \
    src-address=10.201.16.0/20
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add peer=EDI
/ip ipsec policy
set 0 disabled=yes
add dst-address=10.201.16.0/20 level=unique peer=\
    EDI proposal=prop-Sushinet-S2S src-address=10.201.3.0/24 tunnel=yes
add dst-address=10.201.16.0/20 peer=EDI proposal=prop-Sushinet-S2S src-address=\
    192.168.100.0/24 tunnel=yes
/ip route
add comment="Wireguard - Enable" disabled=no distance=1 dst-address=\
    10.201.22.0/24 gateway=wgHOD pref-src="" routing-table=main scope=\
    30 suppress-hw-offload=no target-scope=10
add comment="Wireguard - Enable" disabled=no distance=1 dst-address=\
    10.201.25.0/24 gateway=wgHOD pref-src="" routing-table=main scope=\
    30 suppress-hw-offload=no target-scope=10
/routing bfd configuration
add disabled=no
/system identity
set name=HOD-hAPac3

Connection between Site1 and Site2 works perfectly. I have removed ipsec and everything works fine.
Connection between Site1 and Site3 does not work. Seems wg connection is established - I can ping wg interfaces (from Site 1 I can ping Site's 3 wg interfaces address 172.16.3.2 and vice versa, from Site 3 I can ping 172.16.3.1). Please note, that ipsec tunnel is up, but the pings work with ipsec enabled or disabled, thus I believe the wireguard connection is up and running.
I cannot reach LANs however. E.g. when I ping from Site 3 a server address 10.201.25.9, it works if ipsec tunnel is up, but as soon as I shut it down, the ping times-out.

I suspected a routing issue. But since the routes are set-up the same way as for Site 2, I really do not know where the problem could be.
Any advice is appreciated!

Thank you!

Mesquite · Sun Feb 11, 2024 3:03 am

No problems......

(1) One does not need input chain rule on the routers that are Client for Handshake. Only the Server for handshake requires input chain rule.
No issues with the single wireguard interface having three addresses either.

(2) One question did you want the two other routers to access each others shares?

(3) Did you want to add a remote client ( the admin for example PC or smartphone or both ) to be able to reach all three routers via wireguard for configuration access??

(4) Your whole bridge and vlan structure is hosed........... Suggest you only use one bridge
all subnets should be vlans, the single bridge should not do any DHCP.
check out --> viewtopic.php?t=143620 for some ideas...

(5) You failed to mention another device a maplite?? It has a local LAN subnet of 10.201.19.0/24 which you identified in the first set of Wireguard Peer settings.

(6) The second wireguard peer is setup incorrectly as though the MAIN router was a client not a server for handshake.
From:
add allowed-address=172.16.1.2/32,10.201.1.0/24 comment=SKL \
endpoint-address=2.2.2.2 endpoint-port=13231 interface=wgEDI \
public-key="2222..."

TO:add allowed-address=172.16.1.2/32,10.201.1.0/24 comment=SKL \
interface=wgEDI public-key="2222..."

(7) The third device you have included the WANIP subnet of the third router ( the LANIP subnet of the ISP router ) as an allowed IP. Why??

(8) Why do you have a forward chain rule for to allow WG to 10.201.25.0/24 and NOT 10.201/22.0/24???

(9) Your firewall rules also need much work overall.

(10) Okay so you want to be able to wireguard and reach an ISP router LAN subnet Ip address on the third network device.
If thats the case then you want to be sure on the third device to sourcnat the wg traffic out the WAN, as the ISP router knows nothing about any of the remote subnets.

(11) You are missing the IP route for the maplite 10.201.19.0/24
+++++++++++++++++++++++++++++++++++++++++++++++++++++

Second device:

(12) Peer Settings
From:
/interface wireguard peers
add allowed-address=172.16.1.1/32,10.201.25.0/24,10.201.22.0/24 \
endpoint-address=1.1.1.1 endpoint-port=13231 interface=wgSKL \
public-key="1111..."

TO:
/interface wireguard peers
add allowed-address=172.16.1.0/24,10.201.25.0/24,10.201.22.0/24 \
endpoint-address=1.1.1.1 endpoint-port=13231 interface=wgSKL \
public-key="1111..." persistent-keep-alive=35s

(13) No need for input chain rule for wireguard handshake..

Third device:

(14) Peer Settings: same issue with format on wg adddress and missing keep alive, plus WRONG endpoint address!!
from:
/interface wireguard peers
add allowed-address=172.16.3.1/32,10.201.25.0/24,10.201.22.0/24 endpoint-address=\
24.124.23.66 endpoint-port=13231 interface=\
wgHOD public-key="1111..."

TO:
/interface wireguard peers
add allowed-address=172.16.3.0/24,10.201.25.0/24,10.201.22.0/24 endpoint-address=\
1.1.1.1 endpoint-port=13231 interface=\
wgHOD public-key="1111..." persistent-keep-alive=30s

(15) Okay good you have WG as part of the LAN interface. That should suffice to ensure the traffic exiting the tunnel for a 192.168.100.x address goes out
your WANIP of 192.168.100.2. Thus the ISP router will know where to send the response and the mikrotik will unsourcenat it back to the proper wireguard address for destination handling.......

(16) you can remove the input chain rule for wireguard handshake, not done on this router.

-

BrandonSk · Sun Feb 11, 2024 7:50 am

Hello.

Thanks for looking at my setup.
Instead of quoting, let me just comment on your points:
(1) Fixed - removed the input rules at Site 2 and Site 3.

(2) Please clarify what you mean by "shares" (devices?).
I do want devices from LANs to access devices at different Site. Particularly, I want devices at Site 2 subnet (10.201.1.0/24) and Site 3 subnet (10.201.3.0/24) to be able to access devices at Site 1 subnet (mainly the 10.201.25.0/24, other subnets I can configure later on).
To reach devices from Site 1 subnet (10.201.22.0/24) on either Site 2 or Site 3 is not that important, but I would not mind if it works.

(3)
No. But now that you mentioned it...

But first things first. Let's get these site-to-site tunnels working first (especially the site 3, which is still not working).
After that I would be happy if you point me in the right direction to achieve what you just suggested (if I understand correctly, I would VPN into the router at site 1 [for which I use l2tp vpn at the moment] and through that connection I would be able to access routers at site 2 and site3 as well... I would certainly not mind that.).

(4) Agree and I am aware of that. As the note at the beginning of the config for site 1 says, please disregard these configs. It is work in progress, where I am also in the process of changing some other parts of infrastructure (and moving from CRS1xxx to CRS/CSS3xx) at Site 1 and this will get fixed once everything is in place. But thanks for pointing that up.

(5) That's yet another location. Not important though for the issue at hand though. This location is down at the moment. Maybe I will get it up and running today just to see if it is really just the Site 3 that has an issue. This location used to work before and at that site the Mikrotik also sits behind another router and is NATed.

(6) Fixed...

(7) To be able to access the Zyxel router if necessary and configure it. This is not a hard requirement, but nice to have. It works via existing ipsec tunnel, so I thought it could work via WG too.

(8) As mentioned in (2), the 10.201.25.0/24 (servers subnet) is key at this moment. If that works, adding the 22 subnet should be a matter of including it in the rule.
(Note: overall, I will play with the access rules much more, but for now I would like to keep it as open as possible in order to minimize unintented firewall blocks.)

(9) Yes... relates to (4) and (8). There are some historical rules that need to be eventually removed and new ones that will come in place.

(10) I guess this is solved by what you wrote in (15)? But as said in (7), this is not the main goal. Main goal is to have devices from sites 2 and 3 be able to reach subnet(s) at site 1.

(11) Will fix that once that location gets connected (see (5))

(12) Fixed.
(although I would like to understand, why is it important to have the .0/24 at those configs... At site 1 the ip addresses assigned to wg interface for each location are configured as x.y.z.1/30, thus having only .1 and .2 as usable range... for Site 2 the communication worked fine with the allowed address being .1/32 ... So I am just curious, why would I need to expand the allowed addresses to .0/24 range?)

(13) Fixed. Rule removed.

(14) Fixed the peer config.
As for end-point address - it was correct, it just slipped through when I redacted configs, as it was my actual public ip. I changed it in the original post to 1.1.1.1 and I would appreciate if you could also edit your response and remove it from there. Thanks.

(15) This relates to (10)?

(16) Fixed. Rule removed.

Updated parts of WG configs:
Site 1

/interface wireguard
add listen-port=13231 mtu=1420 name=wgEDI
/interface wireguard peers
add allowed-address=172.16.0.2/32,10.201.19.0/24 comment="PST mAPlite" interface=wgEDI public-key="9999..."
add allowed-address=172.16.1.2/32,10.201.1.0/24 comment=SKL interface=wgEDI public-key="2222..."
add allowed-address=172.16.3.2/32,10.201.3.0/24,192.168.100.0/24 comment=HOD interface=wgEDI public-key="3333..."

Site 2

/interface wireguard
add listen-port=13231 mtu=1420 name=wgSKL
/interface wireguard peers
add allowed-address=172.16.1.0/24,10.201.25.0/24,10.201.22.0/24 endpoint-address=1.1.1.1 endpoint-port=13231 interface=wgSKL persistent-keepalive=35s public-key="1111..."

Site 3

/interface wireguard
add listen-port=13231 mtu=1420 name=wgHOD
/interface wireguard peers
add allowed-address=172.16.3.0/24,10.201.25.0/24,10.201.22.0/24 comment=EDI endpoint-address=1.1.1.1 endpoint-port=13231 interface=wgHOD public-key="1111..."

Changes to firewalls - firewalls remain as they were, except I only removed the input rules for port 13231 at site 2 and site 3 as suggested.
Changes to static routes - None.

Unfortunately, the situation remains as before. Site 3 still cannot reach the 10.201.25.0/24 subnet at site 1.

Cheers,
B.

UPDATE:
I tested the location with mAP Lite (10.201.19.0/24 subnet) and after I added the proper route at Site 1, then it started working immediately.
So if configs look ok, there must be something special going on at Site 3 with that Zyxel router in-between. Strange remains, that 172.16.3.1 and .2 can ping each other even if ipsec tunnel is shutdown (indicating the wg tunnel is up), but subnets won't communicate.

Mesquite · Sun Feb 11, 2024 3:04 pm

(12) Allowed IPs is the key to success. Its how the WG algorithm/routing works.
Allowed ips serve two purposes, matching destination address outbound and filtering and on the inbound just filtering.
It always considers the other end the remote end.
So for outbound, is the destination address on a peer list, if so, select that peer and let the traffic enter the tunnel.
For inbound, is the source address of the incoming remote traffic on the list, if yes, allow it to enter.

It should be clear why we try (are exceptions) to not set 0.0.0.0/0 for peer address on the server device........
All outbound traffic will match that particular peer and no other peers will get traffic. (order can be used to try to reduce this error).

Thus the MAIN router is server for handshake, should to identify each peer precisely in all of its peer client settings.
The clients do not, and in fact its better to allow them the full range of wg addresses. Often if they need internet access their settings are wide open 0.0.0.0/0
In the case where clients do not need internet access they put the IP range of the wireguard subnet and any remote subnets (coming in, or local folks need access to).
(Note: for any remote subnets, one needs IP routes for them). If anything, its to avoid writing down each wireguard IP, so maybe its a lazy approach LOL.

What does this approach accomplish ( wireguard subnet in allowed IPs at client devices)
Think of the outgoing........
In this regard, if you are the admin sitting behind router2 or router3 you can reach router3 or router2 because you have a wide variance in your allowed IPs
otherwise you would have to write every wireguard IP in the allowed IPs
Think of the incoming.....
Any road warrior, can reach any router, otherwise you would have to write every wireguard IP in the allowed IPs.

Mesquite · Sun Feb 11, 2024 3:10 pm

If they can ping each other, it would seem as if the wireguard tunnel is established but something else is going on.
By the way I dont see the persistent-keep alive set on Site 3 ????

Lets see the full config on MAIN and full config on SITE 3 again please.

The upstream zyxel router should not interfere but perhaps something on the SITE3 config needs work.

BrandonSk · Sun Feb 11, 2024 4:41 pm

Hello,

below are the "full" configs. I redatcted actual public IPs.
I also removed parts: /system/scheduler /system/script and /tool/email
(all information in those section relates to the auto-update and backup script you can find here)

Site 1:

# 2024-02-11 14:34:51 by RouterOS 7.13.2
# software id = GEFA-6CF8
#
# model = RB4011iGS+
# serial number = D4440D1022D0
/interface bridge
add admin-mac=08:55:31:12:92:3B auto-mac=no comment=defconf name=bridge.local \
    priority=0x9000
add name=bridge.servers
/interface ethernet
set [ find default-name=ether1 ] name=e1.WAN
set [ find default-name=ether2 ] comment="LAN - trunk to crs109 - 22, 9999" \
    name=e2.t.crs109
set [ find default-name=ether3 ] comment="LAN - tbd - free" name=e3
set [ find default-name=ether4 ] name=e4.srvs.private.vlan
set [ find default-name=ether5 ] name=e5.free
/interface wireguard
add listen-port=13231 mtu=1420 name=wgEDI
/interface vlan
add comment="LAN + WiFi" interface=bridge.local name=vlan22 vlan-id=22
add interface=bridge.local name=vlan25 vlan-id=25
add comment="Servers private vlan" interface=e4.srvs.private.vlan name=\
    vlan1112 vlan-id=1112
add comment=Management interface=bridge.local name=vlan9999 vlan-id=9999
/interface list
add comment=defconf name=WAN
add comment="Local + VPN clients" include=dynamic name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec profile
add dh-group=modp1024 enc-algorithm=aes-128 name=prof-Sushinet-S2S
/ip ipsec peer
add address=2.2.2.2/32 disabled=yes exchange-mode=ike2 name=SKL \
    profile=prof-Sushinet-S2S
add address=3.3.3.3/19 exchange-mode=ike2 name=HOD passive=yes \
    profile=prof-Sushinet-S2S send-initial-contact=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256,sha1 enc-algorithms=\
    aes-256-cbc,aes-128-cbc
add auth-algorithms="" enc-algorithms=aes-128-gcm lifetime=4h name=\
    prop-Sushinet-S2S pfs-group=none
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp_servers ranges=10.201.25.100-10.201.25.254
add name=dhcp_emerg.mgmt ranges=192.168.98.2-192.168.98.254
add name=dhcp_lan.and.wifi ranges=10.201.22.100-10.201.22.254
add name=dhcp_management ranges=10.99.99.100-10.99.99.254
add name=pool_l2tp ranges=10.201.22.70-10.201.22.79
add name=dhcp_servers.private ranges=10.11.12.200-10.11.12.220
/ip dhcp-server
add address-pool=default-dhcp disabled=yes interface=bridge.local lease-time=\
    10m name=defconf
add address-pool=dhcp_servers interface=bridge.servers name=dhcp.servers
add address-pool=dhcp_emerg.mgmt interface=sfp-sfpplus1 name=\
    dhcp.emer.management
add address-pool=dhcp_lan.and.wifi interface=vlan22 name=dhcp.vlan22
add address-pool=dhcp_management interface=vlan9999 name=dhcp.management
add address-pool=dhcp_servers.private interface=vlan1112 name=\
    dhcp.private.servers
/port
set 0 name=serial0
set 1 name=serial1
/ppp profile
set *FFFFFFFE bridge=bridge.local dns-server=10.201.22.1,10.201.25.5 \
    local-address=10.201.22.1 remote-address=pool_l2tp
/user group
add name=cert_upload policy="ssh,ftp,write,!local,!telnet,!reboot,!read,!polic\
    y,!test,!winbox,!password,!web,!sniff,!sensitive,!api,!romon,!rest-api"
/interface bridge port
add bridge=bridge.local comment=defconf interface=e2.t.crs109
add bridge=bridge.local comment=defconf interface=e3
add bridge=bridge.local comment=defconf interface=e5.free
add bridge=bridge.servers comment=defconf interface=ether6
add bridge=bridge.servers comment=defconf interface=ether7
add bridge=bridge.servers comment=defconf interface=ether8
add bridge=bridge.servers comment=defconf interface=ether9
add bridge=bridge.servers comment=defconf interface=ether10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge.local comment="LAN & WiFi" tagged=e2.t.crs109,bridge.local \
    vlan-ids=22
add bridge=bridge.local comment=Management tagged=e2.t.crs109,bridge.local \
    vlan-ids=9999
add bridge=bridge.local comment="Servers private" tagged=e4.srvs.private.vlan \
    vlan-ids=1112
add bridge=bridge.local comment=Servers tagged=e2.t.crs109,bridge.local \
    vlan-ids=25
/interface l2tp-server server
set enabled=yes use-ipsec=yes
/interface list member
add comment=defconf interface=bridge.local list=LAN
add comment=defconf interface=e1.WAN list=WAN
add interface=bridge.servers list=LAN
add interface=vlan22 list=LAN
add interface=vlan9999 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
/interface wireguard peers
add allowed-address=172.16.0.2/32,10.201.19.0/24 comment="PST mAPlite" \
    interface=wgEDI public-key=\
    "9999..."
add allowed-address=172.16.1.2/32,10.201.1.0/24 comment=SKL interface=\
    wgEDI public-key="2222..."
add allowed-address=172.16.3.2/32,10.201.3.0/24,192.168.100.0/24 comment=\
    HOD interface=wgEDI public-key=\
    "3333..."
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=\
    bridge.local network=192.168.88.0
add address=1.1.1.1 comment="public IP" interface=e1.WAN \
    network=1.1.1.1
add address=10.201.25.1/24 comment="LAN - servers" interface=bridge.servers \
    network=10.201.25.0
add address=192.168.98.1/24 comment="emerg. management" interface=\
    sfp-sfpplus1 network=192.168.98.0
add address=10.201.22.1/24 comment="VLAN + wifi" interface=vlan22 network=\
    10.201.22.0
add address=10.99.99.1/24 comment="VLAN Management" interface=vlan9999 \
    network=10.99.99.0
add address=172.16.0.1/30 comment="wgEDI to PST" interface=wgEDI \
    network=172.16.0.0
add address=10.11.12.1/24 comment="Servers private vlan" interface=vlan1112 \
    network=10.11.12.0
add address=172.16.3.1/30 comment="wgEDI to HOD" interface=wgEDI network=172.16.3.0
add address=172.16.1.1/30 comment="wgEDI to SKL" interface=wgEDI network=172.16.1.0
/ip dhcp-client
add comment=defconf interface=e1.WAN
add interface=e4.srvs.private.vlan
/ip dhcp-server lease
add address=10.201.25.5 client-id=1:0:11:32:17:40:c1 comment=\
    "NAS" mac-address=00:11:32:17:40:C1 \
    server=dhcp.servers
add address=10.201.25.8 client-id=1:a:d9:92:e1:3e:13 comment=\
    "Radius" mac-address=0A:D9:92:E1:3E:13 server=\
    dhcp.servers
add address=10.201.25.25 client-id=1:7a:bc:43:82:8b:96 comment=backups \
    mac-address=7A:BC:43:82:8B:96 server=dhcp.servers
add address=10.201.25.3 comment=XCP-NG mac-address=10:60:4B:92:66:68 server=\
    dhcp.servers
add address=10.201.25.50 client-id=1:0:1d:ec:b:8a:27 mac-address=\
    00:1D:EC:0B:8A:27 server=dhcp.servers
add address=10.201.25.9 client-id=1:b8:27:eb:72:5a:fd comment="rPI" \
    mac-address=B8:27:EB:72:5A:FD server=dhcp.servers
add address=10.201.25.13 client-id=\
    ff:eb:a6:69:86:0:1:0:1:28:b1:b:47:2e:fc:cc:be:9c:be comment=\
    "XOA VA" mac-address=0A:6A:EB:A6:69:86 server=dhcp.servers
add address=10.201.25.10 client-id=\
    ff:8d:a9:cb:4d:0:1:0:1:2c:89:c1:2c:8e:5f:8d:a9:cb:4d comment=\
    "main host" mac-address=8E:5F:8D:A9:CB:4D server=dhcp.servers
add address=10.201.22.51 client-id=1:0:1d:ec:b:8a:27 mac-address=\
    00:1D:EC:0B:8A:27 server=dhcp.vlan22
add address=10.201.22.20 client-id=1:80:af:ca:22:81:f8 mac-address=\
    80:AF:CA:22:81:F8 server=dhcp.vlan22
add address=10.201.22.99 client-id=1:fc:e2:6c:2a:2a:e0 mac-address=\
    FC:E2:6C:2A:2A:E0 server=dhcp.vlan22
add address=10.201.22.21 mac-address=80:AF:CA:22:81:FA server=dhcp.vlan22
add address=10.201.22.22 mac-address=80:AF:CA:22:82:0A server=dhcp.vlan22
add address=10.11.12.5 client-id=1:0:11:32:17:40:c2 comment=\
    "SERVERS_PRIVATE - NAS" mac-address=00:11:32:17:40:C2 server=\
    dhcp.private.servers
add address=10.11.12.10 comment="SERVERS_PRIVATE - xcp" mac-address=\
    10:60:4B:92:66:69 server=dhcp.private.servers
add address=10.11.12.12 client-id=1:8e:c1:46:e9:43:e1 comment=\
    SERVERS_PRIVATE-backups mac-address=8E:C1:46:E9:43:E1 server=\
    dhcp.private.servers
/ip dhcp-server network
add address=10.11.12.0/24 comment="Servers private" dns-server=10.11.12.1 \
    gateway=10.11.12.1
add address=10.99.99.0/24 gateway=10.99.99.1
add address=10.201.22.0/24 dns-server=10.201.25.5,88.212.8.8,88.212.8.88 \
    gateway=10.201.22.1
add address=10.201.25.0/24 dns-server=10.201.25.5,88.212.8.8,88.212.8.88 \
    gateway=10.201.25.1
add address=192.168.88.0/24 comment=defconf dns-server=\
    10.201.25.5,192.168.88.1 gateway=192.168.88.1
add address=192.168.98.0/24 gateway=192.168.98.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
add forward-to=10.201.25.5 regexp=".*\\.mydomain\\.com\$" type=FWD
/ip firewall address-list
add address=10.201.19.0/24 list="PST"
add address=10.201.22.90-10.201.22.99 list=Management
add address=10.201.22.129 list=Management
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment=\
    "Disable access to router from Pista's network" disabled=yes protocol=tcp \
    src-address-list="PST"
add action=accept chain=input comment="Allow winbox access from LAN (further r\
    estrictions may be in IP->Services)" dst-port=8291 in-interface-list=LAN \
    protocol=tcp
add action=accept chain=input comment="Wireguard - EDI" dst-port=13231 \
    in-interface-list=WAN protocol=udp
add action=accept chain=input comment="L2TP VPN" dst-port=500,1701,4500 \
    in-interface=e1.WAN protocol=udp
add action=accept chain=input comment="L2TP VPN" in-interface=e1.WAN \
    protocol=ipsec-esp
add action=accept chain=input comment="L2TP VPN" in-interface=e1.WAN \
    protocol=ipencap
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=tcp
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=udp
add action=add-src-to-address-list address-list=HODO_new_IP \
    address-list-timeout=2m30s chain=input comment="ICMP knock - step3" \
    in-interface=e1.WAN log=yes packet-size=xxx protocol=icmp \
    src-address-list=ping-knock2
add action=add-src-to-address-list address-list=ping-knock2 \
    address-list-timeout=20s chain=input comment="ICMP knock - step2" \
    in-interface=e1.WAN log=yes packet-size=yyy protocol=icmp \
    src-address-list=ping-knock1
add action=add-src-to-address-list address-list=ping-knock1 \
    address-list-timeout=20s chain=input comment="ICMP knock - step 1" \
    in-interface=e1.WAN packet-size=zzz protocol=icmp
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward dst-address=10.99.99.0/24 src-address-list=\
    !Management
add action=accept chain=forward comment="wgEDI to servers" dst-address=10.201.25.0/24 in-interface=wgEDI
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=accept chain=forward comment="tmp PST restriction - DISABLED" disabled=yes \
    dst-address=10.201.25.9 src-address-list="PST"
add action=drop chain=forward comment="tmp PST restriction - DISABLED" \
    disabled=yes src-address-list="PST"
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat comment=\
    "NAS" dst-port=\
    XXX protocol=tcp to-addresses=10.201.25.5 to-ports=YYY
/ip ipsec identity
add comment="<medium>" disabled=yes peer=SKL
add peer=HOD
/ip ipsec policy
add comment="DISABLED - Wireguard works for SKL" disabled=yes dst-address=\
    10.201.1.0/24 peer=SKL proposal=prop-Sushinet-S2S src-address=\
    10.201.16.0/20 tunnel=yes
add comment="HOD LAN" dst-address=10.201.3.0/24 level=unique peer=\
    HOD proposal=prop-Sushinet-S2S src-address=10.201.16.0/20 tunnel=\
    yes
add comment="HOD - LAN between Orange and Mikrotik" dst-address=\
    192.168.100.0/24 level=unique peer=HOD proposal=prop-Sushinet-S2S \
    src-address=10.201.16.0/20 tunnel=yes
/ip route
add disabled=no dst-address=10.201.1.0/24 gateway=wgEDI routing-table=\
    main suppress-hw-offload=no
add disabled=no distance=1 dst-address=10.201.3.0/24 gateway=wgEDI \
    pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=10
add disabled=no distance=1 dst-address=192.168.100.0/24 gateway=wgEDI \
    pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=10
add disabled=no dst-address=10.201.19.0/24 gateway=wgEDI routing-table=\
    main suppress-hw-offload=no
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www address=10.201.22.0/24 disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
/ppp secret
add name=xxx profile=default-encryption service=l2tp
/system clock
set time-zone-name=xxx
/system logging
set 0 topics=info,!dhcp
add disabled=yes topics=ipsec,!packet
add topics=info,script
/system note
set show-at-login=no
/tool mac-server
set allowed-interface-list=static
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool netwatch
add disabled=no down-script="/system script run newHODOip" host=10.201.3.1 \
    interval=1m timeout=1s type=simple

Site 3:

# 2024-02-11 14:34:00 by RouterOS 7.13.2
# software id = KDL8-VF33
#
# model = RBD53iG-5HacD2HnD
# serial number = E7290E690E26
/interface bridge
add admin-mac=2C:C8:1B:C5:FE:22 auto-mac=no comment=defconf name=bridge \
    port-cost-mode=short
/interface wireguard
add listen-port=13231 mtu=1420 name=wgHOD
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
add authentication-types=wpa2-psk eap-methods="" group-ciphers=tkip,aes-ccm \
    mode=dynamic-keys name=Sushinet supplicant-identity="" unicast-ciphers=\
    tkip,aes-ccm
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX \
    disabled=no distance=indoors frequency=auto \
    installation=indoor mode=ap-bridge security-profile=Sushinet ssid=\
    Sushinet wireless-protocol=802.11
set [ find default-name=wlan2 ] band=5ghz-a/n/ac channel-width=\
    20/40/80mhz-XXXX disabled=no distance=indoors frequency=\
    auto installation=indoor mode=ap-bridge security-profile=Sushinet ssid=\
    Sushinet wireless-protocol=802.11
/ip ipsec profile
add dh-group=modp1024 enc-algorithm=aes-128 name=prof-Sushinet-S2S
/ip ipsec peer
add address=1.1.1.1/32 exchange-mode=ike2 name=EDI profile=\
    prof-Sushinet-S2S
/ip ipsec proposal
set [ find default=yes ] disabled=yes enc-algorithms=aes-256-cbc,aes-192-cbc
add auth-algorithms="" enc-algorithms=aes-128-gcm lifetime=4h name=\
    prop-Sushinet-S2S pfs-group=none
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp ranges=10.201.3.100-10.201.3.254
/ip dhcp-server
add address-pool=dhcp interface=bridge lease-time=10m name=defconf
/routing bgp template
set default disabled=no output.network=bgp-networks
/routing ospf instance
add disabled=no name=default-v2
/routing ospf area
add disabled=yes instance=default-v2 name=backbone-v2
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=wlan1 \
    internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=wlan2 \
    internal-path-cost=10 path-cost=10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface detect-internet
set lan-interface-list=LAN wan-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=wgHOD list=LAN
/interface ovpn-server server
set auth=sha1,md5
/interface wireguard peers
add allowed-address=172.16.3.0/24,10.201.25.0/24,10.201.22.0/24 comment=\
    EDI endpoint-address=1.1.1.1 endpoint-port=13231 interface=\
    wgHOD persistent-keepalive=35s public-key=\
    "3333..."
/ip address
add address=10.201.3.1/24 interface=bridge network=10.201.3.0
add address=172.16.3.2/30 interface=wgHOD network=172.16.3.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server lease
add address=10.201.3.10 client-id=1:0:1d:ec:17:3c:f8 mac-address=\
    00:1D:EC:17:3C:F8 server=defconf
add address=10.201.3.11 client-id=1:7c:dd:90:d6:dc:9f mac-address=\
    7C:DD:90:D6:DC:9F server=defconf
/ip dhcp-server network
add address=10.201.3.0/24 comment=defconf dns-server=10.201.3.1 gateway=\
    10.201.3.1
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip dns static
add address=192.168.88.1 comment=defconf disabled=yes name=router.lan
add forward-to=10.201.25.5 regexp=".*\\.mydomain\\.com\$" type=FWD
add address=10.201.3.1 name=router.lan
/ip firewall address-list
add address=1.1.1.1 comment=EDI list="Trusted IPs"
add address=1.1.1.1 comment=EDI list=Sushinet_Networks
add address=10.201.3.0/24 list=Sushinet_Networks
add address=10.201.1.0/24 list=Sushinet_Networks
add address=10.201.22.0/24 list=Sushinet_Networks
add address=10.201.25.0/24 list=Sushinet_Networks
add address=10.19.78.0/24 list=Sushinet_Networks
add address=192.168.88.0/24 list=Sushinet_Networks
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input dst-port=8291 protocol=tcp src-address=\
    10.201.0.0/16
add action=accept chain=input dst-port=22 protocol=tcp src-address=\
    10.201.0.0/16
add action=accept chain=input dst-port=500,4500 protocol=udp
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add peer=EDI
/ip ipsec policy
set 0 disabled=yes
add comment="HOD LAN" dst-address=10.201.16.0/20 level=unique peer=\
    EDI proposal=prop-Sushinet-S2S src-address=10.201.3.0/24 tunnel=yes
add comment="HOD - LAN between Orange and Mikrotik" dst-address=\
    10.201.16.0/20 peer=EDI proposal=prop-Sushinet-S2S src-address=\
    192.168.100.0/24 tunnel=yes
/ip route
add comment="Wireguard - Enable" disabled=no distance=1 dst-address=\
    10.201.22.0/24 gateway=wgHOD pref-src="" routing-table=main scope=\
    30 suppress-hw-offload=no target-scope=10
add comment="Wireguard - Enable" disabled=no distance=1 dst-address=\
    10.201.25.0/24 gateway=wgHOD pref-src="" routing-table=main scope=\
    30 suppress-hw-offload=no target-scope=10
/ip service
set telnet disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ip ssh
set always-allow-password-login=yes strong-crypto=yes
/routing bfd configuration
add disabled=no
/system clock
set time-zone-name=Europe/Bratislava
/system identity
set name=HOD-hAPac3
/system leds
set 0 interface=wlan1 leds=led1,led2,led3,led4,led5 type=\
    wireless-signal-strength
set 1 leds=poe-led type=poe-out
/system logging
set 0 topics=info,!wireless,!dhcp
add disabled=yes topics=ipsec,!packet
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system ntp client servers
add address=90.176.21.0
add address=194.239.123.230
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

I spotted one thing just now.
On Site 1, the route definitions - for Site 3 both routes (10.201.3.0/24 adn 192.168.100.0/24) contain definition of additional parameters, in comparison to route definitions for (working) Site 2 and recently added Site 4 (PST).
These parameters are pref-src="", scope=30, target-scope=10.
Even when I try to remove these parameters in Winbox (pushing "up arrow" to greyout the input), they get added back in once I click Apply/OK.
I have no idea what is causing that and whether this could have an impact. Just an observation.

Cheers,
B.

BrandonSk · Sun Feb 11, 2024 5:32 pm

SOLVED (almost)

Hi,
well, the thing about the routes I mentioned in previous post got me thinking... why there is pref-src=""? (in my translation Mikrotik is saying I don't know which source to use... so there must be more than one...)

Well, it turns out that turning off the ipsec at Site 3 side was not enough. I still had active IPSEC policies at Site 1, and despite the ipsec tunnel being down, the router saw 2 possible options for reaching 10.201.3.0/24 subnet and obviously was choosing the ipsec one to use...

Once I disabled ipsec at both sides, I get the connection from Site 3 to Site 1.
What does not work is from Site 1 to Site 3 clients, but (a) it is not a priority; (b) I will play with configs later as I have to go now. If I don't succeed, I will be back here

Big thanks for your help nevertheless!

Cheers,
B.

BrandonSk · Sun Feb 11, 2024 10:30 pm

Follow-up for whoever might find this useful.
Disabling ipsec at Site 1 got it working in the direction Site 3 to Site 1.
But for some reason I still was getting the pref-src="" for the route 192.168.100.0/24 and I could not get rid of it.

So instead I decided to specify the preferred source for both subnets (10.201.3.0/24 and 192.168.100.0/24) as hard-coded 172.16.3.1.
After doing so, now also devices at Site 1 can reach devices at Site 3.
Final note: a mentioned before, Site 1 is getting reworked - firewall, vlans, etc. So it may happen that once I clean it up, the above mystery will be gone. But for now I am happy with the solution.

The /ip/route/export at Site 1 is now like this:

add disabled=no dst-address=10.201.1.0/24 gateway=wgEDI routing-table=main suppress-hw-offload=no
add disabled=no distance=1 dst-address=10.201.3.0/24 gateway=wgEDI pref-src=172.16.3.1 routing-table=main scope=30 \
    suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=192.168.100.0/24 gateway=wgEDI pref-src=172.16.3.1 routing-table=main scope=30 \
    suppress-hw-offload=no target-scope=10
add disabled=no dst-address=10.201.19.0/24 gateway=wgEDI routing-table=main suppress-hw-offload=no

Now, @ Mesquite - I would be interested how to make your original point #3 to work

EDIT:

Now, @ Mesquite - I would be interested how to make your original point #3 to work

Nevermind. I got it already working from another post about wireguard (it actually might have been also yours).
I added 2 firewall rules to forward table at Site 1:
1) accept traffic coming in from bridge-local and destined to wgEDI
2) vice versa, accept traffic from wgEDI to bridge.local

Now when I VPN in through my l2tp vpn, I can reach routers at all 3 sites! Awesome. Thanks.

Mesquite · Mon Feb 12, 2024 6:21 am

Yup, its a matter of firewall rules input chain to configure router and forward chain to reach subnets.
If you want the two client routers to reach other then on the RELAY (main) router make firewall rule as such
add chain=forward action=accept in-interface=wireguardname out-interface=wireguardname comment="relay rule"

Basically router 2 has to connect peer to peer with router 1, the traffic exits the tunnel and has to be allowed to reenter the tunnel for the peer to peer connection with router 3.

Wireguard - branches to main site - one works, other not [SOLVED]

Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not

Re: Wireguard - branches to main site - one works, other not [SOLVED]

Re: Wireguard - branches to main site - one works, other not