Wireguard peer interface irregularly stop working

DL7JP · Fri Apr 01, 2022 8:19 pm

I am running a WG server since 7.1beta6, now with 7.1.5. Through all versions I experience the same problem: Some WG peers irregularly become disconnected and cannot reconnect, the tunnel just stops working and no traffic is going through it any more. Reconnecting from the client fails, the peer interface shows neither incoming nor outgoing traffic; on the client I see a couple of kb outgoing traffic, but only few byte incoming every 20 seconds or so (accummulated values are usually 92, 124, 156, 188,...).

The problems shows up irregularly, sometimes after a few hours, sometimes after a few days, but not on all clients. It is not client-specific, it happens with iOS, Android and also Windows clients. I experienced this with several configurations of WG servers, so I am quite sure this is a WG-internal problem not related to firewall settings, routing tables or the like.

Disabling and enabling the peer on the server instantly solves the problem and the client can reconnect. It seems like the public key of the client gets corrupted in the memory of the router and has to be read in again. Does anyone experience this as well? Any ideas for a solution? I am thinking about writing a script that regularly loops over all peers and disables and enables them.

Fri Apr 01, 2022 8:43 pm

First things first ...
Have you enabled keep-alive messages on the client side ?

Please post config of your Mikrotik.
/export file=<anynameyouwish> (default ROS7 = hide-sensitive)
Clean the export for any left-overs of private info and then post between Code quotes.

mkx · Fri Apr 01, 2022 8:46 pm

Have you enabled keep-alive messages on the client side ?

Oh my ... now we can just wait for @mozerd to drop by and explain that wireguard is a peer to peer protocol, so no clients around.

On the serious note: by all means try enabling keepalives (even on both ends) and see if this helps.

Fri Apr 01, 2022 8:49 pm

Slip of the tongue...

DL7JP · Fri Apr 01, 2022 10:36 pm

I tried with and without keepalives, makes no difference.

Fri Apr 01, 2022 10:48 pm

In that case. Please provide config.

DL7JP · Sat Apr 02, 2022 1:14 am

Here's the config, I refrained from stripping anything since there might be side effects, so it's a bit complex, I hope I cut out all sensitive information. The config (sort of) grew historically, so all comments on the config are appreciated, also beyond WG.

The router has two public IPs and provides remote access to a small (< 20) group of people in the context of amateur radio; the working principle is quite simple: It accepts incoming WG connections from DFN and routes all traffic via bridge-LST. SSTP and L2TP dialin is configured but only used as a fallback if WG does not work. Port-knocking is only needed for SSTP and L2TP. I realised that a few WG peers are in /24 rather than /32, but I guess this is not of significance.

# apr/01/2022 20:57:57 by RouterOS 7.1.5
# software id = MZRN-97EY
#
# model = RB450Gx4
/interface bridge
add name=bridge-LST
/interface ethernet
set [ find default-name=ether1 ] name=DFN
/interface wireguard
add listen-port=XXXX mtu=1420 name=wireguard1
/interface list
add name=LAN
add name=WAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add comment="PPP Dialin Local" name=Dialin-IPs-Local ranges=\
    10.AAA.BBB.1-10.AAA.BBB.150
add comment="PPP Dialin remote" name=DialinIPs-Remote ranges=\
    10.AAA.BBB.151-10.AAA.BBB.250
/port
set 0 name=serial0
/ppp profile
add change-tcp-mss=yes local-address=Dialin-IPs-Local name=Dialin-ITSEC \
    on-down=Telegram-notify-hangup on-up=Telegram-notify-dialin \
    remote-address=DialinIPs-Remote use-encryption=required use-mpls=yes
set *FFFFFFFE use-encryption=required
/routing table
add fib name=viaDFN
/system logging action
add name=TelegramNotify target=memory
/interface bridge port
add bridge=bridge-LST ingress-filtering=no interface=ether2
add bridge=bridge-LST ingress-filtering=no interface=ether3
add bridge=bridge-LST ingress-filtering=no interface=ether4
add bridge=bridge-LST ingress-filtering=no interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=!WAN
/ipv6 settings
set disable-ipv6=yes
/interface l2tp-server server
set authentication=mschap2 default-profile=Dialin-ITSEC enabled=yes \
    use-ipsec=required
/interface list member
add interface=bridge-LST list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=wireguard1 list=LAN
add interface=DFN list=WAN
/interface sstp-server server
set authentication=mschap2 enabled=yes tls-version=only-1.2
/interface wireguard peers
add allowed-address=10.100.100.2/32 interface=\
    wireguard1 public-key=" ...uVqYmPyVR9eboiMBiU="
add allowed-address=10.100.100.3/32 interface=wireguard1 \
    public-key="...pWg24k+tTtCFypq1ryVnw="
add allowed-address=10.100.100.4/32 interface=wireguard1 \
    persistent-keepalive=1m public-key=\
    "...Rab5Q3Qh4DgSDBg0="
add allowed-address=10.100.100.5/32 interface=\
    wireguard1 public-key="...JfLZyWEj9boA9Yz36Az4="

(peers 6-28 deleted)

add allowed-address=10.100.100.29/24 interface=wireguard1 \
    persistent-keepalive=1m public-key=\
    "...bKeowbNGQsuPvK5s8kpGqlE="
/ip address
add address=DDD.EEE.FFF.114/27 interface=DFN network=DDD.EEE.FFF.96
add address=10.100.100.1/24 interface=wireguard1 network=10.100.100.0
/ip dhcp-client
add interface=bridge-LST
/ip dns
set allow-remote-requests=yes
/ip firewall address-list
add address=0.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=172.16.0.0/12 comment=RFC6890 list=not_in_internet
add address=192.168.0.0/16 comment=RFC6890 list=not_in_internet
add address=10.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=169.254.0.0/16 comment=RFC6890 list=not_in_internet
add address=127.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=224.0.0.0/4 comment=Multicast list=not_in_internet
add address=198.18.0.0/15 comment=RFC6890 list=not_in_internet
add address=192.0.0.0/24 comment=RFC6890 list=not_in_internet
add address=192.0.2.0/24 comment=RFC6890 list=not_in_internet
add address=198.51.100.0/24 comment=RFC6890 list=not_in_internet
add address=203.0.113.0/24 comment=RFC6890 list=not_in_internet
add address=100.64.0.0/10 comment=RFC6890 list=not_in_internet
add address=240.0.0.0/4 comment=RFC6890 list=not_in_internet
add address=192.88.99.0/24 comment="6to4 relay Anycast [RFC 3068]" list=\
    not_in_internet
add address=AAA.BBB.11.0/24 comment="ITSEC  subnet" list=allowed_to_router
add address=10.AAA.232.0/24 comment=wireguard disabled=yes list=\
    allowed_to_router
add address=10.AAA.BBB.0/24 comment="PPP (L2TP and SSTP) addresses" list=\
    allowed_to_router
add address=10.100.100.0/24 comment=wireguard list=allowed_to_router
add address=10.100.101.0/24 comment=wireguard list=allowed_to_router
/ip firewall filter
add action=fasttrack-connection chain=input comment=\
    "fasttrack establised and related" connection-state=established,related \
    hw-offload=yes
add action=accept chain=input comment="Accept establised and related" \
    connection-state=established,related
add action=accept chain=forward comment=TESTESTEST connection-state=related \
    disabled=yes
add action=add-src-to-address-list address-list=Port-Knock1 \
    address-list-timeout=1s chain=input comment=\
    "********** Port Knocking - Knock1" dst-port=12345 protocol=tcp
add action=add-src-to-address-list address-list=Port-Knock2 \
    address-list-timeout=1s chain=input comment=Knock2 dst-port=12346 \
    protocol=tcp src-address-list=Port-Knock1
add action=add-src-to-address-list address-list=Port-Knock-Safe \
    address-list-timeout=1h23m chain=input comment="Add to Safe" dst-port=\
    12347 log-prefix="Port-Knock IP white listed: " protocol=tcp \
    src-address-list=Port-Knock2
add action=drop chain=input comment="drop if stored" dst-port=12347 \
    log-prefix="LOGALERT " protocol=tcp src-address-list=\
    Port-Knock-Safe_permanent
add action=add-src-to-address-list address-list=Port-Knock-Safe_permanent \
    address-list-timeout=none-static chain=input comment="store  permanently" \
    dst-port=12347 log=yes log-prefix="LOGALERT " protocol=tcp \
    src-address-list=Port-Knock2
add action=add-src-to-address-list address-list=Port-Knock1 \
    address-list-timeout=8s chain=input comment=\
    "********** Port Knocking - Knock (long clearance)" dst-port=12348 \
    protocol=tcp
add action=add-src-to-address-list address-list=Port-Knock2 \
    address-list-timeout=8s chain=input comment=Knock2 dst-port=12349 \
    protocol=tcp src-address-list=Port-Knock1
add action=add-src-to-address-list address-list=Port-Knock3 \
    address-list-timeout=8s chain=input comment=Knock3 dst-port=12350 \
    protocol=tcp src-address-list=Port-Knock2
add action=add-src-to-address-list address-list=Port-Knock-Safe \
    address-list-timeout=8h chain=input comment="Add to Safe" dst-port=12351 \
    log-prefix="Port-Knock IP white listed: " protocol=tcp src-address-list=\
    Port-Knock3
add action=drop chain=input comment="drop if already stored" dst-port=12351 \
    log-prefix="LOGALERT " protocol=tcp src-address-list=\
    Port-Knock-Safe_permanent
add action=add-src-to-address-list address-list=Port-Knock-Safe_permanent \
    address-list-timeout=none-static chain=input comment=\
    "Add to permanent storage" dst-port=12351 log=yes log-prefix="LOGALERT " \
    protocol=tcp src-address-list=Port-Knock3
add action=accept chain=input comment=\
    "PING Accept icmp from DFN from Port Knock Safe addresses" \
    connection-state="" disabled=yes in-interface=DFN log=yes log-prefix=\
    "DFN icmp:" protocol=icmp src-address-list=Port-Knock-Safe
add action=accept chain=input comment=\
    "DFN: Accept SSTP  from port-knock Safe addresses" dst-port=443 \
    in-interface=DFN protocol=tcp src-address-list=Port-Knock-Safe
add action=accept chain=input comment=\
    "DFN: Accept L2TP from port-knock Safe addresses" dst-port=1701,500,4500 \
    in-interface=DFN protocol=udp src-address-list=Port-Knock-Safe src-port=\
    ""
add action=accept chain=input in-interface=DFN protocol=ipsec-ah \
    src-address-list=Port-Knock-Safe
add action=accept chain=input comment="DFN: accept Wireguard incoming" \
    dst-port=XXXX in-interface=DFN log=yes log-prefix="WG1  in" protocol=udp
add action=accept chain=input comment="DFN: accept icmp" in-interface=DFN \
    protocol=icmp
add action=drop chain=input comment="DFN: drop all else" in-interface=DFN
add action=accept chain=input dst-port=8291,80,22,23,443 in-interface=\
    bridge-LST protocol=tcp
add action=accept chain=input comment=\
    "Accept intput from known IPs (Lst and VPN IPs) " log-prefix=XXX \
    src-address-list=allowed_to_router
add action=drop chain=input comment=CATCHALL log-prefix="C i:"
add action=fasttrack-connection chain=forward comment="***********************\
    ****************************** FORWARD: FastTrack" connection-state=\
    established,related hw-offload=yes
add action=accept chain=forward comment="Established, Related" \
    connection-state=established,related
add action=drop chain=forward comment="Drop invalid" connection-state=invalid \
    disabled=yes log=yes log-prefix=invalid
add action=drop chain=forward comment=\
    "Drop arriving packets that are not dst-NAT`ted" connection-nat-state=\
    !dstnat connection-state=new in-interface=DFN log=yes log-prefix=!NAT
add action=jump chain=forward comment="jump to ICMP filters" jump-target=icmp \
    protocol=icmp
add action=accept chain=forward comment=\
    "Accept allowed addresses (inclding VPN IPs)" src-address-list=\
    allowed_to_router
add action=drop chain=forward comment=CATCHALL log=yes log-prefix="C fw: "
add action=accept chain=icmp comment=\
    "********************************************* ICMP chain: echo reply" \
    icmp-options=0:0 protocol=icmp
add action=accept chain=icmp comment="net unreachable" icmp-options=3:0 \
    protocol=icmp
add action=accept chain=icmp comment="host unreachable" icmp-options=3:1 \
    protocol=icmp
add action=accept chain=icmp comment=\
    "host unreachable fragmentation required" icmp-options=3:4 protocol=icmp
add action=accept chain=icmp comment="allow echo request" icmp-options=8:0 \
    protocol=icmp
add action=accept chain=icmp comment="allow time exceed" icmp-options=11:0 \
    protocol=icmp
add action=accept chain=icmp comment="allow parameter bad" icmp-options=12:0 \
    protocol=icmp
add action=drop chain=icmp comment="deny all other types"
/ip firewall nat
add action=masquerade chain=srcnat disabled=yes log=yes log-prefix="mDFN: " \
    out-interface=DFN
add action=masquerade chain=srcnat log-prefix="masq WG: " out-interface=\
    bridge-LST
/ip route
add dst-address=0.0.0.0/0 gateway=DDD.EEE.FFF.118 routing-table=viaDFN
/ipv6 firewall address-list
add address=fe80::/16 list=allowed
/ppp secret
add name=jp profile=Dialin-ITSEC
add name=hAP-ITSEC profile=Dialin-ITSEC
/routing rule
add action=lookup-only-in-table disabled=no src-address=DDD.EEE.FFF.114/32 \
    table=viaDFN
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name=ITSEC-Wireguard
/system logging
add action=TelegramNotify topics=error
add action=TelegramNotify topics=critical
add action=TelegramNotify topics=account
/system ntp client
set enabled=yes
/system ntp client servers
add address=pool.ntp.org
/system package update
set channel=development
/system routerboard settings
set cpu-frequency=auto
/system scheduler
add name=Telegram on-event=Report-Boot-To-Telegram policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-time=startup
add interval=2m name=ReportLogsToTelegram on-event=TelegramLogParser policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-time=startup
add comment="jun/05/2021 18:00:01" interval=5m name=ScanLogEntriesAndNotify \
    on-event="/system script run ScanLogEntriesAndNotify" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-date=may/29/2020 start-time=00:00:00
/system script
add dont-require-permissions=no name=Report-Boot-To-Telegram owner=admin \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source=":delay 61s\r\
    \n\r\
    \n/tool fetch url=\"https://api.telegram.org/botX:Y/sendMessage\\\?chat_id=Z&text=BOOTED \$[/\
    system identity get name] \$[/system clock get time] \$[/system clock get \
    date] UPTIME \$[/system resource get uptime] \" keep-result=no"
add dont-require-permissions=no name=ScanLogEntriesAndNotify owner=admin \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source="#:log info (\"Test: Starting logmon\");\r\
    \n# BEGIN SETUP\r\
    \n:local scheduleName \"ScanLogEntriesAndNotify\"\r\
    \n:local emailAddress \"email@example.com\"\r\
    \n:local startBuf [:toarray [/log find message~\"LOGALERT\"]]\r\
    \n#:local removeThese {\"telnet\";\"whatever string you want\"}\r\
    \n:local removeThese {\"ipfw.drop.WAN\"}\r\
    \n# END SETUP\r\
    \n\r\
    \n# warn if schedule does not exist\r\
    \n:if ([:len [/system scheduler find name=\"\$scheduleName\"]] = 0) do={\r\
    \n  /log warning \"[LOGMON] ERROR: Schedule does not exist. Create schedul\
    e and edit script to match name\"\r\
    \n}\r\
    \n\r\
    \n# get last time\r\
    \n:local lastTime [/system scheduler get [find name=\"\$scheduleName\"] co\
    mment]\r\
    \n# for checking time of each log entry\r\
    \n:local currentTime\r\
    \n# log message\r\
    \n:local message\r\
    \n \r\
    \n# final output\r\
    \n:local output\r\
    \n\r\
    \n:local keepOutput false\r\
    \n# if lastTime is empty, set keepOutput to true\r\
    \n:if ([:len \$lastTime] = 0) do={\r\
    \n  :set keepOutput true\r\
    \n}\r\
    \n\r\
    \n\r\
    \n:local counter 0\r\
    \n# loop through all log entries that have been found\r\
    \n:foreach i in=\$startBuf do={\r\
    \n \r\
    \n# loop through all removeThese array items\r\
    \n  :local keepLog true\r\
    \n  :foreach j in=\$removeThese do={\r\
    \n#   if this log entry contains any of them, it will be ignored\r\
    \n    :if ([/log get \$i message] ~ \"\$j\") do={\r\
    \n      :set keepLog false\r\
    \n    }\r\
    \n  }\r\
    \n  :if (\$keepLog = true) do={\r\
    \n   \r\
    \n   :set message [/log get \$i message]\r\
    \n\r\
    \n#   LOG DATE\r\
    \n#   depending on log date/time, the format may be different. 3 known for\
    mats\r\
    \n#   format of jan/01/2002 00:00:00 which shows up at unknown date/time. \
    Using as default\r\
    \n    :set currentTime [ /log get \$i time ]\r\
    \n#   format of 00:00:00 which shows up on current day's logs\r\
    \n   :if ([:len \$currentTime] = 8 ) do={\r\
    \n     :set currentTime ([:pick [/system clock get date] 0 11].\" \".\$cur\
    rentTime)\r\
    \n    } else={\r\
    \n#     format of jan/01 00:00:00 which shows up on previous day's logs\r\
    \n     :if ([:len \$currentTime] = 15 ) do={\r\
    \n        :set currentTime ([:pick \$currentTime 0 6].\"/\".[:pick [/syste\
    m clock get date] 7 11].\" \".[:pick \$currentTime 7 15])\r\
    \n      }\r\
    \n   }\r\
    \n    \r\
    \n#   if keepOutput is true, add this log entry to output\r\
    \n   :if (\$keepOutput = true) do={\r\
    \n     :set output (\$output.\$currentTime.\" \".\$message.\"\\r\\n\")\r\
    \n   }\r\
    \n#   if currentTime = lastTime, set keepOutput so any further logs found \
    will be added to output\r\
    \n#   reset output in the case we have multiple identical date/time entrie\
    s in a row as the last matching logs\r\
    \n#   otherwise, it would stop at the first found matching log, thus all f\
    ollowing logs would be output\r\
    \n    :if (\$currentTime = \$lastTime) do={\r\
    \n     :set keepOutput true\r\
    \n     :set output \"\"\r\
    \n   }\r\
    \n  }\r\
    \n\r\
    \n#   if this is last log entry\r\
    \n  :if (\$counter = ([:len \$startBuf]-1)) do={\r\
    \n#   If keepOutput is still false after loop, this means lastTime has a v\
    alue, but a matching currentTime was never found.\r\
    \n#   This can happen if 1) The router was rebooted and matching logs stor\
    ed in memory were wiped, or 2) An item is added\r\
    \n#   to the removeThese array that then ignores the last log that determi\
    ned the lastTime variable.\r\
    \n#   This resets the comment to nothing. The next run will be like the fi\
    rst time, and you will get all matching logs\r\
    \n   :if (\$keepOutput = false) do={\r\
    \n#     if previous log was found, this will be our new lastTime entry    \
    \_ \r\
    \n     :if ([:len \$message] > 0) do={\r\
    \n        :set output (\$output.\$currentTime.\" \".\$message.\"\\r\\n\")\
    \r\
    \n      }\r\
    \n    }\r\
    \n  }\r\
    \n  :set counter (\$counter + 1)\r\
    \n}\r\
    \n\r\
    \n# If we have output, save new date/time, and send email\r\
    \nif ([:len \$output] > 0) do={\r\
    \n  /system scheduler set [find name=\"\$scheduleName\"] comment=\$current\
    Time\r\
    \n#  /tool e-mail send to=\"\$emailAddress\" subject=\"MikroTik alert \$cu\
    rrentTime\" body=\"\$output\" \r\
    \n  /tool fetch \"https://api.telegram.org/botX:Y/sendMessage\?chat_id=Z&text=\$[/system iden\
    tity get name] \$output\" \r\
    \n  #/log info \"[LOGMON] New login entry found in logs, send email\"\r\
    \n}\r\
    \n"
add dont-require-permissions=no name=Telegram-notify-dialin \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source=":local callerId \$\"caller-id\"\r\
    \n\r\
    \n/log warning message=\"VPN Dialin at \$[/system clock get date] \$[/syst\
    em clock get time] caller: \$callerId \"\r\
    \n/tool fetch url=\"https://api.telegram.org/Y/sendMessage\\\?chat_id=Z&text=ITSEC Wireg\
    uard: VPN Dialin at \$[/system clock get date] \$[/system clock get time] \
    caller: \$callerId \" keep-result=no "
add dont-require-permissions=no name=Telegram-notify-hangup \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source=":local callerId \$\"caller-id\"\r\
    \n/log warning message=\"VPN hangup at \$[/system clock get date] \$[/syst\
    em clock get time] peer: \$callerId \"\r\
    \n/tool fetch url=\"https://api.telegram.org/Y/sendMessage\\\?chat_id=Z&text=ITSEC Wireg\
    uard: VPN hangup at \$[/system clock get date] \$[/system clock get time] \
    peer: \$callerId \" keep-result=no "
add dont-require-permissions=no name=TelegramLogParser owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=":\
    global currentBuf [ :toarray [ /log find buffer=TelegramNotify  ] ] ;\r\
    \n:global currentLineCount [ :len \$currentBuf ] ;\r\
    \n:global lastLineCount ;\r\
    \n\r\
    \n:global message \"\";\r\
    \n\r\
    \n:if ( \$lastLineCount <  \$currentLineCount ) do={ \r\
    \n\t:set lastLineCount \$currentLineCount ; \r\
    \n\t:set message [/log get [ :pick \$currentBuf (\$currentLineCount-1) ] m\
    essage];\r\
    \n                /tool fetch url=\"https://api.telegram.org/X:Y/sendMessage\\\?chat_id=-Z&\
    text=\$[/system identity get name] \$[/system clock get date] \$[/system c\
    lock get time]  \$message \" keep-result=no \r\
    \n                 } \r\
    \n"
/tool bandwidth-server
set enabled=no
/tool graphing interface
add
/tool graphing resource
add
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool romon
set enabled=yes
/tool romon port
add disabled=no forbid=yes interface=DFN

anav · Sat Apr 02, 2022 4:27 am

Why is this peer /24 vice 32? (allowed IPs)

add allowed-address=10.100.100.29/24 interface=wireguard1 \
persistent-keepalive=1m public-key=\
"...bKeowbNGQsuPvK5s8kpGqlE="

What is the purpose of this routing rule???
/routing rule
add action=lookup-only-in-table disabled=no src-address=DDD.EEE.FFF.114/32 \
table=viaDFN

and this route
/ip route
add dst-address=0.0.0.0/0 gateway=DDD.EEE.FFF.118 routing-table=viaDFN

DL7JP · Sat Apr 02, 2022 10:38 am

> Why is this peer /24 vice 32? (allowed IPs)
Actually it is a typo, but it does not seem to matter.

WG peers connect to DDD.EEE.FFF.114, and the routing rule causes WG-server responses to use the same interface. The traffic from inside the WG-Tunnels is routed via the default route (bridge-LST).

Sat Apr 02, 2022 10:46 am

Typo or not, it does matter.
If that peer is a spoke in your hub/ spoke setup, it should be /32.

DL7JP · Sat Apr 02, 2022 11:06 am

> If that peer is a spoke in your hub/ spoke setup, it should be /32.

What is the effect of /24 vs. /32? The peer with /24 can connect and traffic goes through the tunnel.

Sat Apr 02, 2022 11:08 am

Yes but ALL traffic from that subnet will go through that peer.
Also traffic not intended to go there.
Wireguard will not know what needs to go where.

You should not have any overlap in the allowed addresses on your device.
And /24 is a big overlap...

DL7JP · Sat Apr 02, 2022 11:18 am

> Yes but ALL traffic from that subnet will go through that peer.
Hm ... but the configuration works. I would think the WG server sends traffic using the most specific subnet.

There are a few other peers in /24, it might be that exactly these stop working after a couple of hours. I will check this.

mkx · Sat Apr 02, 2022 11:29 am

> Yes but ALL traffic from that subnet will go through that peer.
Hm ... but the configuration works. I would think the WG server sends traffic using the most specific subnet.

Perhaps it does ... when all WG tunnels work as desired. But what if one of peers with /32 isn't? Then the /24 might take over and the /32 would never re-establish ... so why trying your luck if your "working" setup is marginal at least ... and you came here because it's not really working.

We've seen weirder problems and fixing seemingly pretty unrelated mis-configurations made a change. One of facts with ROS is that only a small number of mis-configurations are flagged as error and that's partially because ROS is so flexible that finding invalid configuration automatically is a daunting task.

fragtion · Sat Apr 02, 2022 11:56 am

I've been having this problem too. I think I've managed to narrow it down to the UDP Stream Timeout setting. Especially if one of the routers is behind (double-?)NAT or DMZ router with DUAL WAN failover setup. In my case the router running ROSv7 has a router running ROSv6 as its gateway with DST-NAT for the wg ports.

If the gateway's WAN IP address changes, the WG sessions fail to reconnect automatically after the Endpoint IP is changed, unless you change the ports and disable/re-enable both the parent and peer interface

If I disconnect the wg peer for over 3 minutes then it can reconnect successfully again. So yeah it has something to do with the UDP Stream Timeout.

There are some threads on the MikroTik forum describing this issue, and it seems to affect other UDP services too like SIP phones and UDP-based OpenVPN tunnels, but no elegant solutions besides for disabling the interface for a duration greater than the UDP Stream Timeout which is 3 minutes by default on RouterOS. Hopefully more users of you can help me test this theory if this is the only reason for WG interfaces failing to automatically re-establish connections where it should

See these threads:
viewtopic.php?t=129048
viewtopic.php?t=35196
That's exactly what seems to be happening, except now for WG instead of SIP connections

DL7JP · Sat Apr 02, 2022 12:33 pm

So firstly thanks to all here, very competent forum!

I changed all peers to /32 In the past all worked fine for 1-2 days then tunnels got stuck. So, I will now wait for a few days, see if the problem comes up again and report back.

UDP timing might also be a reason, but in the setup here there is no NAT and no IP change.

mkx · Sat Apr 02, 2022 2:05 pm

UDP timing might also be a reason, but in the setup here there is no NAT and no IP change.

Unless you control all network devices on all the route (which gives problems) you can never be entirely sure that there's no device breaking long-lasting UDP connections with sporadic traffic. It doesn't have to be NAT.

Znevna · Sat Apr 02, 2022 2:11 pm

I don't see how UDP timing could be the reason why a peer couldn't reach the peer with a nice and shiny open port to it.

mkx · Sat Apr 02, 2022 2:17 pm

I don't see how UDP timing could be the reason why a peer couldn't reach the peer with a nice and shiny open port to it.

The only case when you can be sure of "a nice and shiny open port to it" is if peers are directly connected by a passive connection (UTP cable, fibre with plain SFP modules) where you can actually see any link outs.

anav · Sat Apr 02, 2022 3:24 pm

WG peers connect to DDD.EEE.FFF.114, and the routing rule causes WG-server responses to use the same interface. The traffic from inside the WG-Tunnels is routed via the default route (bridge-LST).

Yeah but there is something weird about your config that I cannot put my finger on....

Perhaps its because you have a bridge BUT NO IP ADDRESS FOR THE BRIDGE ???
/ip address
add address=DDD.EEE.FFF.114/27 interface=DFN network=DDD.EEE.FFF.96
add address=10.100.100.1/24 interface=wireguard1 network=10.100.100.0

and yet you have bridge ports ??
/interface bridge port
add bridge=bridge-LST ingress-filtering=no interface=ether2
add bridge=bridge-LST ingress-filtering=no interface=ether3
add bridge=bridge-LST ingress-filtering=no interface=ether4
add bridge=bridge-LST ingress-filtering=no interface=ether5

But then I see you have an IP DHCP Client........ which is another pile of confusing.
/ip dhcp-client
add interface=bridge-LST

Clearly the WAN is ether1
/interface ethernet
set [ find default-name=ether1 ] name=DFN

But the bridge is associated with the LAN.
/interface list member
add interface=bridge-LST list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=wireguard1 list=LAN
add interface=DFN list=WAN

My conclusion is that either
a. the config works and I dont have a clue how OR
b. the config is fundamentally flawed and needs to be fixed.

DL7JP · Sat Apr 02, 2022 3:46 pm

Well, LAN is indeed be not be a good choice of a name for bridge-LST: It is connected to a (firewalled) subnet with public IP addresses, a seperate gateway to the Internet, and a dhcp server. So the router can reach the Internet via "WAN" and "LAN". WG peers are connecting from "outside" to the WAN-interface, and the traffic coming through these tunnels goes via LAN towards the Internet.

anav · Sat Apr 02, 2022 4:33 pm

Doesnt compute for me. BUT since no one else has piped in, it must be a special config that works.

Sat Apr 02, 2022 4:36 pm

A picture would make things a lot more clear. Don't you think so ?

DL7JP · Sat Apr 02, 2022 5:17 pm

A picture would make things a lot more clear. Don't you think so ?

Hm, not sure how to add a picture here...
https://www.dropbox.com/s/dk5khwruya9qz9x/WG.png

Sat Apr 02, 2022 6:05 pm

Now I'm even more confused ...

I assume eth1-5 are all the same device ?
Why represent them in 2 separate blocks then ?
Where is your ISP connection which should be connected to eth1 ?

Having a look at the detailed config:
What is providing DHCP on your network (since nothing is visible in your config) ? Your ISP modem ? Then it should be on the picture.

Why's that scr-nat between bridge-LST and WG-server there and going out over bridge-LST ? I don't understand it's use.

Might also help to clarify what you are trying to accomplish with your setup.

anav · Sat Apr 02, 2022 7:19 pm

...... Done for the op.

Capture1.JPG

fragtion · Sat Apr 02, 2022 7:48 pm

For now I'm using scripting to try get around these issues. The following script should completely reset a wg tunnel if the IP address associated with the endpoint's hostname changes, and it will also block traffic out of the tunnel for 5 minutes (to work around the UDP Stream Timeout issue) if no handshake was received during the past 3 minutes prior to the script being run. This will allow any remote upstream routers to drop the broken connection (assuming their timeout value is below the 5 minute interval of the scheduled script)

The downside to running this script is that, if the host is actually down or unreachable, the logs will be spammed & a firewall rule changed every 5 minutes

Assume remote host peer port is 12345 (yours may obviously differ) - this port must be set accordingly in the firewall rule below, to match the endpoint port of the peer in question

#1. Prepare for scripts

/system/scheduler add interval=5m name=check-wg-all on-event="/system/script/run check-wg-all" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=apr/02/2022 start-time=12:00:00

/system script add dont-require-permissions=no name=check-wg-all owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=\
    ":foreach item in=[/system/script/find where name~\"check-wg-\" and name!=\"check-wg-all\"] do={ /system/script/run \$item }"

/ip firewall filter add action=drop chain=output comment=out-wg-remotehost disabled=yes dst-port=12345 protocol=udp

#2. Add script for each wg tunnel with name "check-wg-<scriptname>". eg check-wg-remotehost:

:local resolvedIP [:resolve "some.remote-host.com"];
:local interface "wg-remotehost";
:local peeraddr "daDASJDOJdoijasiDJBNDBAIAaodjasidjasiojd=";

:local peer [/interface/wireguard/peers/find where public-key=$peeraddr];
:local currentIP [/interface/wireguard/peers/get $peer endpoint-address];

:if ([:find $currentIP $resolvedIP] < 0) do={
    /interface/wireguard/peers set $peer endpoint-address=$resolvedIP;
    /interface/wireguard/peers disable $peer
    /interface/wireguard disable $interface
    /interface/wireguard enable $interface
    /interface/wireguard/peers enable $peer
    /log info "Wireguard Peer $peeraddr (on $interface) endpoint-address changed from $currentIP to $resolvedIP";
}
:delay 5
if (([/interface/wireguard/peers/get $peer last-handshake] > 180) || ([:len [/interface/wireguard/peers/get $peer last-handshake]] = 0)) do={
  if ([/ip/firewall/filter/get [find where comment="out-"."$interface"] disabled] = true) do={
    /log info "Wireguard Peer $peeraddr (under $interface) unresponsive - Temporarily disabling...";
    /ip/firewall/filter/set [find where comment="out-"."$interface"] disabled=no
  } else={
    /log info "Wireguard Peer $peeraddr (under $interface) unresponsive - Trying to restore...";
    /ip/firewall/filter/set [find where comment="out-"."$interface"] disabled=yes
  }
}

Improvements/optimizations & further code suggestions welcome

DL7JP · Sat Apr 02, 2022 7:52 pm

Thanks for embedding the picture!

> I assume eth1-5 are all the same device ?
Yes, they are all part of the router working as a WG-Server

> Why represent them in 2 separate blocks then ?
They are in different networks.

> Where is your ISP connection which should be connected to eth1 ?
The router has to seperate Internet connections: eth1 has a static public IP, and bridge-LST (eth1-4) gets a dynamically assigned public IP (via dhcp) providing the default route.

> Why's that scr-nat between bridge-LST and WG-server there and going out over bridge-LST ? I don't understand it's use.
It is NAT-ing the WG-peers (in 10.100.100.0/24) towards bridge-LST.

> Might also help to clarify what you are trying to accomplish with your setup.
It works like a standard VPN server providing Internet via a WG-tunnel, with the exception that clients connect to a dedicated IP (at eth1 / DFN). Internet connections are then routed via the bridge-LST (the default route).

DL7JP · Sat Apr 02, 2022 7:57 pm

For now I'm using scripting to try get around these issues. The following script should completely reset a wg tunnel

Thanks, I will have a look at it!

anav · Sat Apr 02, 2022 8:03 pm

Already discussed in other threads, all relevant Script discussed here, go to para 6.
(6) MYNETNAME - SPECIAL CONSIDERATION FOR ENDPOINT VIA ANY DYNDNS URL (reresolve DNS)

viewtopic.php?t=182340

anav · Sat Apr 02, 2022 8:13 pm

[quote=DL7JP post_id=923404 time=1648918329 user_id=65078]
Thanks for embedding the picture!

> I assume eth1-5 are all the same device ?
Yes, they are all part of the router working as a WG-Server

> Why represent them in 2 separate blocks then ?
They are in different networks.

> Where is your ISP connection which should be connected to eth1 ?
The router has to seperate Internet connections: eth1 has a static public IP, and bridge-LST (eth1-4) gets a dynamically assigned public IP (via dhcp) providing the default route.

> Why's that scr-nat between bridge-LST and WG-server there and going out over bridge-LST ? I don't understand it's use.
It is NAT-ing the WG-peers (in 10.100.100.0/24) towards bridge-LST.

> Might also help to clarify what you are trying to accomplish with your setup.
It works like a standard VPN server providing Internet via a WG-tunnel, with the exception that clients connect to a dedicated IP (at eth1 / DFN). Internet connections are then routed via the bridge-LST (the default route).
[/quote][/i]

Almost there!

1. The ISP is providing multiple Public IPs.
2. One Public IP is assigned to Ether1 but SERVES NO PURPOSE ???? WAIT< its strictly for the purpose of creating the WG tunnel as a server need public reachable IP!!!
I dont see any traffic from the LAN going out this wanip ????
3. ALL clients existing behind the router coming in on ports that belong to the bridge will go out the second public IP associated to the bridge.
4. All external wireguard clients coming into the router will go out the internet via the bridge and the second public IP
5. The admin wants all the source address from wireguard incoming clients to be changed to that of the second public IP.
I dont know why but thats the admins call........suggest wants to hide source IP from internet.

?????????? but how can your ISP provide both a static fixed IP and dynamic secondary public IPs, ????

DL7JP · Sat Apr 02, 2022 8:26 pm

1. The ISP is providing multiple Public IPs.
2. One Public IP is assigned to Ether1 but SERVES NO PURPOSE ???? WAIT< its strictly for the purpose of creating the WG tunnel as a server need public reachable IP!!!
I dont see any traffic from the LAN going out this wanip ????
3. ALL clients existing behind the router coming in on ports that belong to the bridge will go out the second public IP associated to the bridge.
4. All external wireguard clients coming into the router will go out the internet via the bridge and the second public IP
5. The admin wants all the source address from wireguard incoming clients to be changed to that of the second public IP.
I dont know why but thats the admins call........suggest wants to hide source IP from internet.

Congrats, good thinking! All points are correct - just I don't understand what you mean by 3. The router does nothing but VPN service.

> ?????????? but how can your ISP provide both a static fixed IP and dynamic secondary public IPs, ????
No ISP in the classical sense is involved here. I simply have access to two completely independent subnets with public IPs and Internet connectivity.

anav · Sat Apr 02, 2022 8:34 pm

Ahh so there are not clients behind this router, it is strictly a vpn conduit for external clients connected via wireguard??

I am still stuck on ether1 carrying multiple WANIPs ??
If you assign the bridge as an IP DHCP client............... it must get a gateway and WANIP from somewhere.
Is there another port the router uses for ISP???
Something I am not grasping.

....

DL7JP · Sat Apr 02, 2022 8:45 pm

I am still stuck on ether1 carrying multiple WANIPs ??
If you assign the bridge as an IP DHCP client............... it must get a gateway and WANIP from somewhere.
Is there another port the router uses for ISP???
Something I am not grasping. ....

ether1 (renamed to DFN) is not part opf the bridge and has these settings:

/ip address add address=DDD.EEE.FFF.114/27 interface=DFN network=DDD.EEE.FFF.96 
/ip route add dst-address=0.0.0.0/0 gateway=DDD.EEE.FFF.118 routing-table=viaDFN
/routing rule add action=lookup-only-in-table disabled=no src-address=DDD.EEE.FFF.114/32 table=viaDFN

The bridge (ether2-5) gets the network configuration via dhcp. Both networks are different.

anav · Sat Apr 02, 2022 10:34 pm

I am still stuck on ether1 carrying multiple WANIPs ??
If you assign the bridge as an IP DHCP client............... it must get a gateway and WANIP from somewhere.
Is there another port the router uses for ISP???
Something I am not grasping. ....
ether1 (renamed to DFN) is not part opf the bridge and has these settings:
Code: Select all
/ip address add address=DDD.EEE.FFF.114/27 interface=DFN network=DDD.EEE.FFF.96 
/ip route add dst-address=0.0.0.0/0 gateway=DDD.EEE.FFF.118 routing-table=viaDFN
/routing rule add action=lookup-only-in-table disabled=no src-address=DDD.EEE.FFF.114/32 table=viaDFN
The bridge (ether2-5) gets the network configuration via dhcp. Both networks are different.

I understand they are different but then please clarify.
a. ONLY One cable is connected to ether1 from a source ( what is the source it is not clear)
b. No other physical connections exist.
c. It appears one gets a valid IP address on ether1 which you use as a WAN address where the INPUT CHAIN rules apply.
d. You have a bridge that you assign as an IP DHCP Client TO WHAT ??
e. How does this omnipotent magic source supply two different types of IP addresses to your router. (one for ether1 and one for the bridge), especially if one is static fixed and the other is dynamic which rules out use of vlans or blocks of IPS. me thinks your hallucinating

I know they are two different networks but the lan bridge how is it reaching/connected to the internet ...........

Sat Apr 02, 2022 10:44 pm

I'm glad I'm not the only one being confused...

DL7JP · Sat Apr 02, 2022 11:35 pm

> a. ONLY One cable is connected to ether1 from a source ( what is the source it is not clear)
either1 (DFN, WAN) is assigned DDD.EEE.FFF.114/27, a cable goes into a port on a switch that is in this network. Another cable from one of the bridge ports (LAN) goes into another port on a switch that is in another network ; "/ip dhcp-client add interface=bridge-LST" gets an IP address for the bridge..

> b. No other physical connections exist.
As explained above, both the bridge as well as either1 are connected to seperate switch ports

> c. It appears one gets a valid IP address on ether1 which you use as a WAN address where the INPUT CHAIN rules apply.
yes

> d. You have a bridge that you assign as an IP DHCP Client TO WHAT ??
there is a dhcp server in the network the bridge is connected to (see above).

> e. How does this omnipotent magic source supply two different types of IP addresses to your router. (one for ether1 and one for the bridge), especially if one is static
> fixed and the other is dynamic which rules out use of vlans or blocks of IPS. me thinks your hallucinating

no magic involed: The two networks are simply delivered to different ports on switches I connect the router to. It's like having two ISPs, one providing a static IP, the other assigning the IP via dhcp.

anav · Sat Apr 02, 2022 11:51 pm

Consider the mystery solved, Hoelvo, you are sane!! The world has not gone bananas, just the standard case of poor OP communications LOL

Now the truth comes out, like pulling teeth.
One would think that for such a non standard setup the explanation would have been clearer.

Your diagram is not helpful and I would say adds to the confusion, as it show no etherport being connected external to the router.

This would have been appropriate, and a comment that the bridge is connected to an external switch etc for WAN2 etc.....
...

Capture5.jpg

DL7JP · Sat Apr 02, 2022 11:58 pm

Your diagram is not helpful and I would say adds to the confusion, as it show no etherport being connected external to the router.

You're right, the picture is now clearer. If you are in your own setup, you don't see the problems others might have to understand it.

anav · Sun Apr 03, 2022 1:21 am

Your diagram is not helpful and I would say adds to the confusion, as it shows no etherport being connected external to the router.
You're right, the picture is now clearer. If you are in your own setup, you don't see the problems others might have to understand it.

Well for morons like me that have very little exposure to anything other than a home network, we need eXtra help!

DL7JP · Sun Apr 03, 2022 2:05 am

[/quote]
Well for morons like me that have very little exposure to anything other than a home network, we need eXtra help!
[/quote]

To confuse you even more

, WAN2 in your picture is actually https://hamnetdb.net/map.cgi - a (sort of) parallel version of the public internet.

anav · Sun Apr 03, 2022 3:18 am

I see the Czechias are smart and avoid that disasterly confusing internet LOL.

MrBonding · Wed Oct 05, 2022 1:09 pm

I'm facing a similar issue with WG, I have a Mikrotik device working as a Wireguad client (LTE connection) since it's behind a CGNAT I'm connecting to my Wireguard server that is not behind a CGNAT.
The problem is that when I reboot/start the client device, the Wireguard connection does not establish itself, I have to disable and enable the peer to connect to it. I have set the persistent keepalive to 25 on both, client and server peer. On the other hand, I have an SSTP tunnel for backup that works nicely.

Here are the configs:
Client:

# oct/01/2022 15:41:15 by RouterOS 7.5
# software id = QCU2-3MJT
#
# model = RBD53GR-5HacD2HnD
# serial number = XXX
/interface bridge add name=bridge1
/interface bridge add name=bridge_mngmt
/interface ethernet set [ find default-name=ether1 ] comment=Management
/interface ethernet set [ find default-name=ether2 ] 
/interface ethernet set [ find default-name=ether3 ] 
/interface ethernet set [ find default-name=ether4 ] 
/interface ethernet set [ find default-name=ether5 ] 
/interface wireguard add listen-port=13831 mtu=1420 name="Wireguard" private-key="XX"
/interface lte apn add apn=XXX
/interface lte set [ find default-name=lte1 ] allow-roaming=yes apn-profiles=XXX band="" 
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/interface wireless security-profiles add authentication-types=wpa2-psk mode=dynamic-keys name=XXXXX supplicant-identity="" wpa2-pre-shared-key=XXXXXX
/interface wireless set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX country=francedisabled=no frequency=auto mode=ap-bridge security-profile=XX ssid=XX
/interface wireless set [ find default-name=wlan2 ] country=france disabled=no frequency=auto security-profile=XXXX ssid=XXXX
/ip pool add name=dhcp_pool0 ranges=192.168.2.2-192.168.2.254
/ip dhcp-server add address-pool=dhcp_pool0 interface=bridge_mngmt name=dhcp1
/interface sstp-client add connect-to=XXXX disabled=no name=SSTP password=XXX profile=default-encryption user=XXXX
/interface bridge port add bridge=bridge1 interface=ether2
/interface bridge port add bridge=bridge1 interface=ether3
/interface bridge port add bridge=bridge1 interface=ether4
/interface bridge port add bridge=bridge1 interface=ether5
/interface bridge port add bridge=bridge_mngmt interface=ether1
/interface bridge port add bridge=bridge_mngmt interface=wlan1
/interface bridge port add bridge=bridge_mngmt interface=wlan2
/ip neighbor discovery-settings set discover-interface-list=all
/interface wireguard peers add allowed-address=172.62.0.1/32,192.168.0.0/16,10.100.0.0/24 endpoint-address="IP ADDRESS, not a DNS" endpoint-port=XXX interface=Wireguard persistent-keepalive=25s public-key=X
/ip address add address=172.62.0.2/24 interface=Wireguard=172.62.0.0
/ip address add address=10.100.0.1/24 interface=bridge1 network=10.100.0.0
/ip address add address=192.168.2.1/24 interface=bridge_mngmt network=192.168.2.0
/ip dhcp-server network add address=192.168.2.0/24 gateway=192.168.2.1
/ip dns set allow-remote-requests=yes
/ip firewall filter add action=accept chain=forward
/ip firewall nat add action=masquerade chain=srcnat out-interface=lte1
/ip route add comment="Route of Office PC" disabled=no dst-address=192.168.101.0/24 gateway=Wireguard routing-table=main suppress-hw-offload=no
/ip service set telnet disabled=yes
/ip service set ftp disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/ip ssh set always-allow-password-login=yes
/system identity set name=XXXXX
/tool romon set enabled=yes

Server config:

# oct/05/2022 11:55:23 by RouterOS 7.5
# software id = 9Q0W-FI01
#
# model = RB4011iGS+
# serial number = 
/interface bridge add name=bridge
/interface ethernet set [ find default-name=ether1 ] comment="WAN"
/interface ethernet set [ find default-name=ether2 ] comment="Link 192.168.255.0/30"
/interface ethernet set [ find default-name=ether3 ] 
/interface wireguard add listen-port=13231 mtu=1420 name=Wireguard_X
/interface wireguard add listen-port=14213 mtu=1420 name=Wireguard_XX
/interface wireguard add listen-port=13831 mtu=1420 name=Wireguard_XXX
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/ip pool add name=VPN-SSTP ranges=172.63.0.0/24
/port set 0 name=serial0
/port set 1 name=serial1
/ppp profile add local-address=172.63.0.1 name=SSTP remote-address=VPN-SSTP
/dude set enabled=yes
/interface bridge port add bridge=bridge interface=ether3
/interface l2tp-server server set enabled=yes
/interface sstp-server server set enabled=yes
/interface wireguard peers add allowed-address=10.100.10.2/32 comment=Test interface=Wireguard_X public-key=
/interface wireguard peers add allowed-address=10.100.10.6/32 comment=Roadwarrior_02 interface=Wireguard_X public-key=
/interface wireguard peers add allowed-address=172.61.0.3/32,10.50.2.0/24 interface=Wireguard_X persistent-keepalive=25s public-key=
/interface wireguard peers add allowed-address=172.61.0.2/32,10.50.2.0/24 interface=Wireguard_X persistent-keepalive=25s public-key=
/interface wireguard peers add allowed-address=172.61.0.10/32,10.50.2.0/24 interface=Wireguard_X persistent-keepalive=25s public-key=
/interface wireguard peers add allowed-address=10.10.10.2/32 comment="Management" interface=Wireguard_XX public-key=
/interface wireguard peers add allowed-address=172.61.0.254/24,10.50.2.0/24 comment=Management interface=Wireguard_X persistent-keepalive=25s public-key=
/interface wireguard peers add allowed-address=172.61.0.11/32,10.50.2.0/24 comment=Test interface=Wireguard_X persistent-keepalive=25s public-key=
/interface wireguard peers add allowed-address=172.62.0.2/32,10.100.0.0/24,192.168.0.0/16 comment=Problematic_Wireguard interface=Wireguard_XXX persistent-keepalive=25s public-key=
/ip address add address=192.168.255.2/30 interface=ether2 network=192.168.255.0
/ip address add address=192.168.90.1/24 interface=bridge_marratxi network=192.168.90.0
/ip address add address=172.61.0.1/24 interface=Wireguard_X network=172.61.0.0
/ip address add address=10.10.10.1/24 interface=Wireguard_XX network=10.10.10.0
/ip address add address=172.62.0.1/24 interface=Wireguard_XXX network=172.62.0.0
/ip cloud set ddns-enabled=yes ddns-update-interval=10m
/ip dhcp-client add interface=ether1
/ip dhcp-server network add address=192.168.55.0/24 gateway=192.168.55.1
/ip dhcp-server network add address=192.168.90.0/24 gateway=192.168.90.1
/ip firewall nat add action=masquerade chain=srcnat out-interface=ether1
/ip firewall nat add action=src-nat chain=srcnat disabled=yes dst-address-list=172.61.0.0/24 to-addresses=172.61.0.1
/ip route add comment="Net 192.168.101.0/24 router" disabled=no distance=1 dst-address=192.168.101.0/24 gateway=192.168.255.1 pref-src=0.0.0.0 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ip route add disabled=no dst-address=10.1.200.0/24 gateway=Wireguard_XX routing-table=main suppress-hw-offload=no
/ip route add disabled=no distance=2 dst-address=10.50.2.0/24 gateway=Wireguard_X pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ip route add disabled=no distance=1 dst-address=10.100.0.0/24 gateway=172.62.0.2 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ip service set telnet disabled=yes
/ip service set ftp disabled=yes
/ip service set www disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/snmp set enabled=yes
/system clock set time-zone-name=Europe
/system identity set name=XXX
/tool graphing interface add
/tool graphing resource add
/tool romon set enabled=yes

On the server I have multiple Wireguard servers, the one that's giving me problems is the Wireguard_XXX, the others are not fully tested, so I will not say for sure that those are working 100% correctly, which means that they could be having the same issue.

In summary, does someone have a suggestion why this is happening? I belive having to have a script to solve that issue is not really an elegant solution...

Many thanks for you help!

Znevna · Wed Oct 05, 2022 1:48 pm

It's nice that you hijacked someone else's topic.
Also nice that you edited the private info from the configs.
Except the serial number from the client.
And you did so much anonymizing that we can't tell what you're using as endpoint, a DNS or IP?
Because if it's a DNS there are posts around the forum explaining why it doesn't work after a reboot, and what can you do about it. Even mentioned in this topic, here: viewtopic.php?p=960461#p923407
Cheers.

MrBonding · Wed Oct 05, 2022 2:11 pm

It's nice that you hijacked someone else's topic.
Also nice that you edited the private info from the configs.
Except the serial number from the client.
And you did so much anonymizing that we can't tell what you're using as endpoint, a DNS or IP?
Because if it's a DNS there are posts around the forum explaining why it doesn't work after a reboot, and what can you do about it. Even mentioned in this topic, here: viewtopic.php?p=960461#p923407
Cheers.

I thought it may be better to continue on that thread since it's almost about the same, and someone else in the future might be facing the same problem... Don't know if I did it right.
Thanks for pointing out the serial number issue

You are completely right about DNS and IP, I knew about that issue so I'm using an IP, not a DNS, the problem still presists.

Thanks!

Wed Oct 05, 2022 2:16 pm

Does your "client" device have a valid public IP (CGNAT or not doesn't matter) BEFORE wireguard service is started on that device ?
Check the log files if that info is there.

If not, we're talking basically about the same problem.

WG protocol starts, does not know where to go to and shuts down. No retry.
Toggling the peer status retriggers the whole process. If that happens AFTER the device has a valid IP, handshake will take place and packets start to flow.

What might help in that case, provide a startup script with a delay of 30s or so which does nothing but toggling that peer.

Still, the scripting part where "the other side" is checked and if needed peer status is toggled, is still the best workaround also for other causes.
Until they include something like that in ROS (which should be done, in my view).

ivicask · Wed Oct 05, 2022 2:21 pm

Yeah its a problem that mikrotik needs to fix, simplest WG setup and peer doesnt work on like 50% reboots(STATIC IP NO DNS), i have to add netwatch script in order to retoggle it and fix it after boot..
And on some locations peers just die permanently and i need to create new WG interface(client side) with fresh key in order to work again, its like keys gets corrupted or something..

MrBonding · Wed Oct 05, 2022 5:05 pm

Does your "client" device have a valid public IP (CGNAT or not doesn't matter) BEFORE wireguard service is started on that device ?
Check the log files if that info is there.

If not, we're talking basically about the same problem.

WG protocol starts, does not know where to go to and shuts down. No retry.
Toggling the peer status retriggers the whole process. If that happens AFTER the device has a valid IP, handshake will take place and packets start to flow.

What might help in that case, provide a startup script with a delay of 30s or so which does nothing but toggling that peer.

Still, the scripting part where "the other side" is checked and if needed peer status is toggled, is still the best workaround also for other causes.
Until they include something like that in ROS (which should be done, in my view).

Hi!.
It's an LTE connection I believe there is not a valid public IP before it starts since it takes a while.
I'm trying to understand the problem, but there is a thing that does not make sense to me. The peer I'm disabling/enabling is on the server side with no endpoint (since the endpoint it's on the client), so when I'm rebooting the peer (disable/enable) that means that somehow that client is trying to make a connection to the server. In your scenario that does not make sense since there's no retry (and anything that happens on the server is seen on the client).

Any explanation?

Thanks!

anav · Wed Oct 05, 2022 5:33 pm

Too bad you didnt start a fresh thread, too confusing to address your issues with half answers, half explanations, no diagrams etc.......................

MrBonding · Wed Oct 05, 2022 5:40 pm

Too bad you didnt start a fresh thread, too confusing to address your issues with half answers, half explanations, no diagrams etc.......................

I thought it may be better to continue on that thread since it's almost about the same issue, and someone else in the future might be facing the same problem... Don't know if I did it right.

It's in my best interest to provide full answers and explanations for you so we can understand and solve the problem, I don't really know what you mean by half answers/explanations, anyhow I did not post any network diagram as I think it is not necessary for that issue, a simple Wireguard Client/Server. If you need a diagram to better solve the issue I can happily draw it for you.

Many thanks for your help.

anav · Wed Oct 05, 2022 7:45 pm

Please do then I will have a look at the configs

Wed Oct 05, 2022 9:14 pm

@MrBonding
No direct possible idea why it fails in your case and even less why it starts working.
For that we should have more details on the internal processes inside ROS ( which we have not).

anav · Wed Oct 05, 2022 9:35 pm

Before I even attempt to figure out what is going on in the config..........
You need to clearly state
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
a. If any wireguard clients users are going directly to the MT CLIENT ( I think the answer is no, my assumption is all go to server first)
b. If any wireguard clients are going to the MT server and then going to the MT CLIENT if so which ones?
c. If any local MT Client users are going to the MT SERVER and if so which subnets and to where:
i. for lan device access?
ii for internet access?
d. For admin on MT client to config MT server?
++++++++++++++++++++++++++++++++++++++++++++

e. For local users on MT server are they going to MT Client device and if so which subnets and to where
i. lan devices on MT client
ii. for internet access
f. For admin on MT server to config MT client?

+++++++++++++++++++++++++++++++++++

It will make much more sense to me for your settings if I know whos who in the zoo and going where..............

Wed Oct 05, 2022 9:48 pm

Point a
(since I also have such a setup with SXT LTE)
An LTE device behind CGNAT is never directly accessible, to my knowledge.
It always has to pass a device with fixed or dynamic IP publicly accessible.
Then a tunnel can be made and only then it becomes reachable from elsewhere.

anav · Wed Oct 05, 2022 9:50 pm

Kewl, thanks........... sounds as bad as proprietary Starlink NAT.........

retom · Mon Oct 17, 2022 6:10 am

I am running a WG server since 7.1beta6, now with 7.1.5. Through all versions I experience the same problem: Some WG peers irregularly become disconnected and cannot reconnect, the tunnel just stops working and no traffic is going through it any more. Reconnecting from the client fails, the peer interface shows neither incoming nor outgoing traffic; on the client I see a couple of kb outgoing traffic, but only few byte incoming every 20 seconds or so (accummulated values are usually 92, 124, 156, 188,...).

I have a similar problem, with exactly the same symptoms
WG Server with static ip, client android phone from 3G and 4G.

Only disable/enable interface can help me. Rebooting device not solve problem.
It seems to me that this happens with an unstable cellular connection

Mon Oct 17, 2022 3:26 pm

Makes sense, no ?
If the connection is not stable, how do you expect the link to be made ?

As for 3G/4G connections ... I already used Wireguard connection to home when driving in France on the highway (I was on the passenger seat).
Perfect connection to home (with most likely several antenna hops in between) until we came into 'no mans land' (everyone knows them, those areas with fields with vegetation as far as you can see and nothing else).
No connection = no link to home. Logical.
Once I had cellular connection again, the link automagically came up again. Nothing extra to be done.

raistlin · Fri Oct 21, 2022 5:51 am

I am running a WG server since 7.1beta6, now with 7.1.5. Through all versions I experience the same problem: Some WG peers irregularly become disconnected and cannot reconnect, the tunnel just stops working and no traffic is going through it any more. Reconnecting from the client fails, the peer interface shows neither incoming nor outgoing traffic; on the client I see a couple of kb outgoing traffic, but only few byte incoming every 20 seconds or so (accummulated values are usually 92, 124, 156, 188,...).

I have a similar problem, with exactly the same symptoms
WG Server with static ip, client android phone from 3G and 4G.

Only disable/enable interface can help me. Rebooting device not solve problem.
It seems to me that this happens with an unstable cellular connection

I have the same problem on 7.4.1.
I found that it work again after disable and enable the peer.

retom · Mon Oct 24, 2022 6:42 am

No connection = no link to home. Logical.
Once I had cellular connection again, the link automagically came up again. Nothing extra to be done.

Everything is exactly like that, but sometimes after 1-2 days of normal operation, the peer stops working. And this is observed mainly 3g / 4g.

retom · Sun Nov 13, 2022 4:30 pm

I have the same problem on 7.4.1.
I found that it work again after disable and enable the peer.

I also noticed that:
If disable/enable peers from winbox or from webfig, wireguar with this peers start working.

If i disable/enable with script, peers not working


:log warning "WireGuard disable peers"
 /interface/wireguard/peers disable [find comment~"*restart*"]
:delay 15000ms
:log warning "WireGuard enable peers"
 /interface/wireguard/peers enable [find comment~"*restart*"]

Montecri · Tue Mar 21, 2023 7:20 pm

Hi! RouterOS 7.8 here and the exact same behavior. Smartphone on 4G connected through Wireguard to RB4011, can't even reach the router itself. If I disable and then enable the peer it will work instantly.

Did you find a fix?

Thanks.

- Cristiano

anav · Tue Mar 21, 2023 7:31 pm

@retom
@Montecri

If you two [*****.***] actually read the thread..........
viewtopic.php?p=991310#p923407

This one is recommended:
**** FOR ADVANCED USERS ------- Courtesy of Sob/Dave ( called elegant by Chupaka even )

MrBonding · Tue Oct 10, 2023 10:51 pm

I believe this was solved on 7.10 with:
wireguard - retry "endpoint-address" DNS query on failed resolve;

llamajaja · Wed Oct 11, 2023 1:42 am

Yes I think so!

starlingus · Tue Jan 16, 2024 6:35 pm

RB5009 with ROS 7.12.1, encoutering the same issue with Wiregauard.

caionew · Mon Jan 29, 2024 4:09 pm

RB5009UG+S+
with version 7.10.2, upgrading now to latest stable. 7.12.1

same issue... wireguard start to work after disabling and enabling the peer...
Connection is stable... just cant reach anything on the network.

Alpha7456321 · Thu Mar 21, 2024 10:12 am

CCR2004-1G-12S-2XS
OS router 7.13.5

Hi,

We are on March 21, 2024, the problem is still present.
Peer connection in 5G (Android, iOS, macOS, Windows), fixed ISP IP, etc.
Peers connect, then can no longer until I deactivate them and activate them again.
I cannot use Wireguard in these conditions even if the configuration is simpler.
Unless someone finally has a stable solution or if an update corrects this problem.