v7.14.3 [stable] is released!

pedkoschi · Sun Mar 10, 2024 12:11 pm

Good day,

IKEv2/IPSec PSK Android native VPN connection : ipsec identity not fund error

Android 14 (maybe 13 too) initialliy sends IPsec-ID as configured in the phone connection (In ROS: ID_I) but an empty remote id (ID_R).
Later it requests ID_R to check if configured vpn server matches ROS id. It closes the connection if it doesnt match.

Problem:

a) if i configure the identity in ROS to have both ID's (my id type:fqdn, my id: my.public.dns and remote id type: fqdn, id: ipsec.vpn for example)
ROS will not find that identity because it seems that it checks both id's (but ID_R from android is emtpy)

b) if i configure the identity in ROS to have empty value for my id, ROS will pick up that identity but report that empty value later when requseted
Finally android will send a DELETE to abort the connection.

i dont know if android behaves correctly, however it is possible to bypass checking "my id" value when selecting identity in case of empty value?

pedkoschi · Sun Mar 10, 2024 1:08 pm

Good day,

feature request: ipsecprofiles / ipsec peers
ipsec/profile (phase1) multiple values for hash and prf algorithms
--and / or--
mutilple identical ipsec peers (with the exception of profile selection)

feature request:

ipsec/profile (phase1) multiple values for hash and prf algorithms
--and / or--
mutilple identical ipsec peers (with the exception of profile selection)

Windows demands the same algo for both hash and prf. for example i can set up SHA256/SHA256 or SHA1/SHA1,
but e.g android 11 accepts only hash:SH256 and prf:SHA1 for IKEv2/PSK VPN.

It is impossible to handle both types of clients here.
Auto setting for prf seems to use SHA1.

It is possible to create multiple profiles with different settings,
but only one of them can be selected in the corresponding peer definition.

Maybe it will be possible in the future to extend profile config to accept more than one value for hash/prf
or/and
to handle all peers which only differ in the profile definition without the error: this entry is unreachable?

many thanks

Network5 · Mon Mar 11, 2024 2:02 am

@pedkoschi - Use PS to update your Windows IKEv2 configuration to the encryption level that suits for both clients:

PS C:\Users\user> Set-VpnConnectionIPsecConfiguration -ConnectionName "xxxx" -AuthenticationTransformConstants SHA256128 -CipherTransformConstants AES256 -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -PfsGroup PFS2048 -DHGroup Group14 -PassThru -Force

AuthenticationTransformConstants : SHA256128
CipherTransformConstants : AES256
DHGroup : Group14
IntegrityCheckMethod : SHA256
PfsGroup : PFS2048
EncryptionMethod : AES256

This is also possible for Mac clients with .mobileconfig profiles using Apple Configurator.

Mon Mar 11, 2024 10:46 am

What's new in 7.14.1 (2024-Mar-08 14:50):

*) bgp-vpn - use VRF interface as gateway for leaked connected routes;
*) chr - fixed Xen and Vultr missing ethernet (introduced in v7.14);
*) chr - fixed bogus messages printed out while booting up the system (introduced in v7.14);
*) console - fixed do/while implementation not working with variables (introduced in v7.14);
*) ethernet - fixed default names for CRS310-8G+2S+ device (introduced in v7.14);
*) lte - fixed R11e-LTE-US modem dial-up;
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wireguard - do not attempt to connect to peer without specified endpoint-address;

dadaniel · Mon Mar 11, 2024 11:33 am

Bug in 7.14.1 on CRS328-24P-4S+: Blink button and blink cli command does not work

blink.png

fragtion · Mon Mar 11, 2024 11:39 am

Nice update. Good to see the improvement for wireguard peers which will fix a lot of the log spam, and of course the Chr ethernet fix which was quite serious/urgent 👍viva la MikroTik.. ROS v7 ftw

Now we just need smaller packages for 15/16 MB devices please 😉 or at least some function or way to reclaim bogus used space without needing to netinstall it?

infabo · Mon Mar 11, 2024 11:55 am

or at least some function or way to reclaim bogus used space without needing to netinstall it?

I wish for this ever since. Completely non-understandable why there no such command exists. ROS hides the "real" filesystem behind a facade and gives us no option to get rid of crappy tempfiles.

infabo · Mon Mar 11, 2024 11:56 am

Bug in 7.14.1 on CRS328-24P-4S+: Blink button and blink cli command does not work

It worked in 7.14?

shyrwall · Mon Mar 11, 2024 12:08 pm

Didn't say Cisco/Juniper don't have tons of bugs but I've never come across a "recommended release" that does not allow the product to boot after upgrade. Especially now in recent years when there's an impact/compability check etc before upgrading.
Ok but that is where you are going wrong. The MikroTik release tag "stable" does not make it a "recommended release".
Even "long-term" doesn't do that. You need extra research to know if a certain release is recommended for installation.

Bricked in 7.14.1 also even with this,
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

I guess I'm waiting for the "*) We actually mean stable this time. And it really works for CR2004-1G-2XS-PCIe" according to your logic.

dadaniel · Mon Mar 11, 2024 12:14 pm

It worked in 7.14?

don't know, just wanted to let know it does not work in the current version

infabo · Mon Mar 11, 2024 12:47 pm

What is the latest version you know it still worked?

infabo · Mon Mar 11, 2024 12:50 pm

Bricked in 7.14.1 also even with this,
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

I guess I'm waiting for the "*) We actually mean stable this time. And it really works for CR2004-1G-2XS-PCIe" according to your logic.

You need to watch out for changelog entry like:

* sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14.1)

jordanp123 · Mon Mar 11, 2024 12:54 pm

What's new in 7.14.1 (2024-Mar-08 14:50):

*) bgp-vpn - use VRF interface as gateway for leaked connected routes;
*) chr - fixed Xen and Vultr missing ethernet (introduced in v7.14);
*) chr - fixed bogus messages printed out while booting up the system (introduced in v7.14);
*) console - fixed do/while implementation not working with variables (introduced in v7.14);
*) ethernet - fixed default names for CRS310-8G+2S+ device (introduced in v7.14);
*) lte - fixed R11e-LTE-US modem dial-up;
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wireguard - do not attempt to connect to peer without specified endpoint-address;

I'm still seeing failures with the VRF's being assigned to main. In the new Testing release it appears to be fixed but in this 14.1 release its still broken for me anyway. I rolled back my RB5009 after updating this mornign.

diamuxin · Mon Mar 11, 2024 1:14 pm

Hi,

Access to CHANGELOG information via CLI does not work:

[admin@MikroTik] > :put ([/tool fetch "https://upgrade.mikrotik.com/7.14.1/CHANGELOG" output=user as-value] -> "data")
failure: Fetch failed with status 404

Thx.

erlinden · Mon Mar 11, 2024 1:18 pm

Probably because there is no changelog on the provided URL, @diamuxin. Also from the browser it results in a 404.

Mon Mar 11, 2024 1:22 pm

Missing "routeros" directory:

https://upgrade.mikrotik.com/routeros/7.14.1/CHANGELOG

Hi,

Access to CHANGELOG information via CLI does not work:

[admin@MikroTik] > :put ([/tool fetch "https://upgrade.mikrotik.com/7.14.1/CHANGELOG" output=user as-value] -> "data")
failure: Fetch failed with status 404

Thx.

diamuxin · Mon Mar 11, 2024 1:28 pm

Missing "routeros" directory:

https://upgrade.mikrotik.com/routeros/7.14.1/CHANGELOG

Hi,

Access to CHANGELOG information via CLI does not work:

[admin@MikroTik] > :put ([/tool fetch "https://upgrade.mikrotik.com/7.14.1/CHANGELOG" output=user as-value] -> "data")
failure: Fetch failed with status 404

Thx.

Right, sorry!

BR.

hova888 · Mon Mar 11, 2024 1:30 pm

hap ac3 7.14.1 I have the same problem as on 7.14. Container pihole does not work. Сomplete reinstallation of the container does not help. Downgrade 7.13.5 helps get the container working again

infabo · Mon Mar 11, 2024 1:31 pm

What's new in 7.14.1 (2024-Mar-08 14:50):

*) bgp-vpn - use VRF interface as gateway for leaked connected routes;
*) chr - fixed Xen and Vultr missing ethernet (introduced in v7.14);
*) chr - fixed bogus messages printed out while booting up the system (introduced in v7.14);
*) console - fixed do/while implementation not working with variables (introduced in v7.14);
*) ethernet - fixed default names for CRS310-8G+2S+ device (introduced in v7.14);
*) lte - fixed R11e-LTE-US modem dial-up;
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wireguard - do not attempt to connect to peer without specified endpoint-address;

It is really stunning how many lines say "introduced in v7.14". This was already the case in 7.13. Please don't push "major" releases too early. Give your developers time to fix crucial bugs. Improve your internal QA process. So for example now we all know that VRF broke in 7.14. Add this knowledge to your testing process/setup and we will never ever run again in these same issues. Of course this needs automation and not just some additional manpower and 2 more entries in a hand-written checklist.

Regarding the urge to release:

viewtopic.php?p=1059444#p1058995

We have reproduced multiple issues regarding VLAN MTU not applying correctly or resetting to default after reboot. Unfortunately, it is too late to incorporate the fixes into 7.14, so those will be available in the upcoming 7.15beta

It is never too late to incorporate fixes. I don't know who "pushed" or "urged" you to release 7.14. Wasn't the best idea it seems to me.

Kanzler · Mon Mar 11, 2024 1:38 pm

hap ac3 7.14.1 I have the same problem as on 7.14. Container pihole does not work. Сomplete reinstallation of the container does not help. Downgrade 7.13.5 helps get the container working again

Very strange, my pihole works on hap ac3 with 7.14.1. Can you show your config?

Chupaka · Mon Mar 11, 2024 1:48 pm

What's new in 7.14.1 (2024-Mar-08 14:50):
*) wireguard - do not attempt to connect to peer without specified endpoint-address;

Needs more work. When a peer without endpoint-address connects and then disconnects - RouterOS still floods the log with useless messages (probably because now the peer does have remote address under the hood - the address of the last message from peer)

hova888 · Mon Mar 11, 2024 1:52 pm

hap ac3 7.14.1 I have the same problem as on 7.14. Container pihole does not work. Сomplete reinstallation of the container does not help. Downgrade 7.13.5 helps get the container working again
Very strange, my pihole works on hap ac3 with 7.14.1. Can you show your config?

now 7.13.5 pihole work not problem ( instruction install https://blog.unixhost.pro/ru/2022/10/us ... -routeros/)

Kanzler · Mon Mar 11, 2024 2:05 pm

@hova888,
In any case, we need to check the log.

hova888 · Mon Mar 11, 2024 2:07 pm

@hova888,
In any case, we need to check the log.

How to do it?

Kanzler · Mon Mar 11, 2024 2:15 pm

You need to enable logging in the container settings and then check the log messages to understand what the error is.

hova888 · Mon Mar 11, 2024 2:29 pm

You need to enable logging in the container settings and then check the log messages to understand what the error is.

this is log 7.13.5 not problems

Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service s6rc-oneshot-runner: starting
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service s6rc-oneshot-runner successfully started
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service fix-attrs: starting
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service fix-attrs successfully started
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service legacy-cont-init: starting
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service legacy-cont-init successfully started
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service cron: starting
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service cron successfully started
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service _uid-gid-changer: starting
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service _uid-gid-changer successfully started
Mar/11/2024 14:20:21 container,info,debug s6-rc: info: service _startup: starting
Mar/11/2024 14:20:21 container,info,debug Starting docker specific checks & setup for docker pihole/pihole
Mar/11/2024 14:20:21 container,info,debug Setting capabilities on pihole-FTL where possible
Mar/11/2024 14:20:21 container,info,debug Applying the following caps to pihole-FTL:
Mar/11/2024 14:20:21 container,info,debug * CAP_CHOWN
Mar/11/2024 14:20:21 container,info,debug * CAP_NET_BIND_SERVICE
Mar/11/2024 14:20:21 container,info,debug * CAP_NET_RAW
Mar/11/2024 14:20:21 container,info,debug * CAP_NET_ADMIN
Mar/11/2024 14:20:21 container,info,debug * CAP_SYS_NICE
Mar/11/2024 14:20:21 container,info,debug Ensuring basic configuration by re-running select functions from basic-install.sh
Mar/11/2024 14:20:21 container,info,debug
Mar/11/2024 14:20:21 container,info,debug Installing configs from /etc/.pihole...
Mar/11/2024 14:20:21 container,info,debug Existing dnsmasq.conf found... it is not a Pi-hole file, leaving alone!
Mar/11/2024 14:20:21 container,info,debug Installing /etc/dnsmasq.d/01-pihole.conf...
[K [✓] Installed /etc/dnsmasq.d/01-pihole.conf
Mar/11/2024 14:20:25 container,info,debug Installing /etc/.pihole/advanced/06-rfc6761.conf...
[K [✓] Installed /etc/dnsmasq.d/06-rfc6761.conf
Mar/11/2024 14:20:27 container,info,debug
Mar/11/2024 14:20:27 container,info,debug Installing latest logrotate script...
Mar/11/2024 14:20:27 container,info,debug Existing logrotate file found. No changes made.
Mar/11/2024 14:20:28 container,info,debug [i] Assigning password defined by Environment Variable
Mar/11/2024 14:20:28 container,info,debug [✓] New password set
Mar/11/2024 14:20:29 container,info,debug [i] Added ENV to php:
Mar/11/2024 14:20:29 container,info,debug "TZ" => "Europe/Kyiv",
Mar/11/2024 14:20:29 container,info,debug "PIHOLE_DOCKER_TAG" => "",
Mar/11/2024 14:20:29 container,info,debug "PHP_ERROR_LOG" => "/var/log/lighttpd/error-pihole.log",
Mar/11/2024 14:20:29 container,info,debug "CORS_HOSTS" => "",
Mar/11/2024 14:20:29 container,info,debug "VIRTUAL_HOST" => "MikroTik",
Mar/11/2024 14:20:29 container,info,debug [i] Using IPv4 and IPv6
Mar/11/2024 14:20:29 container,info,debug
Mar/11/2024 14:20:30 container,info,debug [i] Installing latest Cron script...
[K [✓] Installing latest Cron script
Mar/11/2024 14:20:30 container,info,debug [i] Preexisting ad list /etc/pihole/adlists.list detected (exiting setup_blocklists early)
Mar/11/2024 14:20:30 container,info,debug [i] Existing DNS servers detected in setupVars.conf. Leaving them alone
Mar/11/2024 14:20:30 container,info,debug [i] Applying pihole-FTL.conf setting LOCAL_IPV4=0.0.0.0
Mar/11/2024 14:20:30 container,info,debug [i] FTL binding to default interface: eth0
Mar/11/2024 14:20:33 container,info,debug [i] Enabling Query Logging
Mar/11/2024 14:20:35 container,info,debug [i] Testing lighttpd config: Syntax OK
Mar/11/2024 14:20:35 container,info,debug [i] All config checks passed, cleared for startup ...
Mar/11/2024 14:20:35 container,info,debug [i] Docker start setup complete
Mar/11/2024 14:20:35 container,info,debug
Mar/11/2024 14:20:35 container,info,debug [i] pihole-FTL (no-daemon) will be started as root
Mar/11/2024 14:20:35 container,info,debug
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service _startup successfully started
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service pihole-FTL: starting
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service pihole-FTL successfully started
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service lighttpd: starting
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service lighttpd successfully started
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service _postFTL: starting
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service _postFTL successfully started
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service legacy-services: starting
Mar/11/2024 14:20:35 container,info,debug Checking if custom gravity.db is set in /etc/pihole/pihole-FTL.conf
Mar/11/2024 14:20:35 container,info,debug s6-rc: info: service legacy-services successfully started
Mar/11/2024 14:20:38 container,info,debug [i] Neutrino emissions detected...
Mar/11/2024 14:20:39 container,info,debug
[K [✓] Pulling blocklist source list into range
Mar/11/2024 14:20:39 container,info,debug
Mar/11/2024 14:20:45 container,info,debug [i] Preparing new gravity database...
[K [✓] Preparing new gravity database
Mar/11/2024 14:20:58 container,info,debug [i] Creating new gravity databases...
[K [✓] Creating new gravity databases
Mar/11/2024 14:20:59 container,info,debug [i] Using libz compression
Mar/11/2024 14:20:59 container,info,debug
Mar/11/2024 14:20:59 container,info,debug [i] Target: https://raw.githubusercontent.com/Steve ... ster/hosts
Mar/11/2024 14:21:03 container,info,debug [i] Status: Pending...
[K [✓] Status: Retrieval successful
Mar/11/2024 14:21:25 container,info,debug
[✓] Parsed 173658 exact domains and 0 ABP-style domains (ignored 1 non-domain entries)
Mar/11/2024 14:21:25 container,info,debug Sample of non-domain entries:
Mar/11/2024 14:21:25 container,info,debug - "0.0.0.0"
Mar/11/2024 14:21:25 container,info,debug
Mar/11/2024 14:21:25 container,info,debug [i] List stayed unchanged
Mar/11/2024 14:21:25 container,info,debug
Mar/11/2024 14:21:25 container,info,debug [i] Target: https://adguardteam.github.io/AdGuardSD ... filter.txt
Mar/11/2024 14:21:27 container,info,debug [i] Status: Pending...
[K [✓] Status: Retrieval successful
Mar/11/2024 14:21:36 container,info,debug
[✓] Parsed 2 exact domains and 62518 ABP-style domains (ignored 589 non-domain entries)
Mar/11/2024 14:21:36 container,info,debug Sample of non-domain entries:
Mar/11/2024 14:21:36 container,info,debug - "||ads.livetv*.me^"
Mar/11/2024 14:21:36 container,info,debug - "||counter*.stat.ovh^"
Mar/11/2024 14:21:36 container,info,debug - "||fxhpaoxqyajvmdg"
Mar/11/2024 14:21:36 container,info,debug - "||rgfftupf"
Mar/11/2024 14:21:36 container,info,debug - "||clk.rtpdn*.com^"
Mar/11/2024 14:21:36 container,info,debug
Mar/11/2024 14:21:37 container,info,debug [i] List has been updated
Mar/11/2024 14:21:37 container,info,debug
Mar/11/2024 14:21:43 container,info,debug [i] Building tree...
[K [✓] Building tree
Mar/11/2024 14:21:43 container,info,debug [i] Swapping databases...
[K [✓] Swapping databases
Mar/11/2024 14:21:43 container,info,debug [✓] The old database remains available
Mar/11/2024 14:21:46 container,info,debug [i] Number of gravity domains: 236178 (236178 unique domains)
Mar/11/2024 14:21:47 container,info,debug [i] Number of exact blacklisted domains: 0
Mar/11/2024 14:21:47 container,info,debug [i] Number of regex blacklist filters: 0
Mar/11/2024 14:21:47 container,info,debug [i] Number of exact whitelisted domains: 4
Mar/11/2024 14:21:47 container,info,debug [i] Number of regex whitelist filters: 0
Mar/11/2024 14:21:47 container,info,debug [i] Cleaning up stray matter...
[K [✓] Cleaning up stray matter
Mar/11/2024 14:21:47 container,info,debug
Mar/11/2024 14:21:47 container,info,debug [✓] FTL is listening on port 53
Mar/11/2024 14:21:47 container,info,debug [✓] UDP (IPv4)
Mar/11/2024 14:21:47 container,info,debug [✓] TCP (IPv4)
Mar/11/2024 14:21:47 container,info,debug [✓] UDP (IPv6)
Mar/11/2024 14:21:47 container,info,debug [✓] TCP (IPv6)
Mar/11/2024 14:21:47 container,info,debug
Mar/11/2024 14:21:47 container,info,debug [✓] Pi-hole blocking is enabled
Mar/11/2024 14:21:47 container,info,debug
Mar/11/2024 14:21:54 container,info,debug Pi-hole version is v5.17.3 (Latest: v5.17.3)
Mar/11/2024 14:21:54 container,info,debug web version is v5.21 (Latest: v5.21)
Mar/11/2024 14:21:54 container,info,debug FTL version is v5.25.1 (Latest: v5.25.1)
Mar/11/2024 14:21:54 container,info,debug Container tag is: 2024.02.2
Mar/11/2024 14:21:54 container,info,debug

Paternot · Mon Mar 11, 2024 2:38 pm

Bricked in 7.14.1 also even with this,
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

The upgrade is done by the installed version, not the new on.

If you upgrade from (say) 7.13 to 7.14, then the upgrade package is downloaded by the running 7.13, tested, extracted and installed by the running 7.13
Only after the boot the new 7.14 will be running.

So, when they say "improved system stability" you will only see the benefits after the first boot, when the new version starts running. In some cases maybe not even then, since they would need for the firmware to be upgraded too.

hova888 · Mon Mar 11, 2024 2:42 pm

You need to enable logging in the container settings and then check the log messages to understand what the error is.

log container 7.14.1

hova888 · Mon Mar 11, 2024 2:49 pm

You need to enable logging in the container settings and then check the log messages to understand what the error is.

What versions of the solution will you have for my problem pihole 7.14.1 log above in the screenshot?

shyrwall · Mon Mar 11, 2024 3:02 pm

Bricked in 7.14.1 also even with this,
*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

I guess I'm waiting for the "*) We actually mean stable this time. And it really works for CR2004-1G-2XS-PCIe" according to your logic.
You need to watch out for changelog entry like:

* sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14.1)

I soldered a push button switch to my lab card now so atleast I'm a bit more ready.

DEHAAS · Mon Mar 11, 2024 3:28 pm

Tunnel interfaces are still going to the main VRF instead of the configured VRF in 7.14.1.

osc86 · Mon Mar 11, 2024 7:45 pm

*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);

#notfixed - it's getting ridiculous..

diamuxin · Mon Mar 11, 2024 8:07 pm

Mr. Support, with the latest update 7.14.1 the logging problems with the handshake have NOT been solved, both in the wireguard road-warrior links and with BTH.

1. I have connected with the RW, I navigate a little bit and when I disconnect, the messages in the log appear again, it does not stop after 20 attempts.
2. I have activated the BTH with the app and the same thing, the messages start coming out as if there was no tomorrow. The funny thing is that they come out whether you are connected or not.

Any idea?

Etz · Mon Mar 11, 2024 8:16 pm

Must be something config specific, I do use wireguard for road-warrior setup and I have 0 such logs.
I don't use or have BTH though.

noah · Mon Mar 11, 2024 8:17 pm

Tunnel interfaces are still going to the main VRF instead of the configured VRF in 7.14.1.

Yes, i have the same issue, that my Wireguard Interface for my Road Warrior Clients go to the main VRF, instead of the dedicated VRF.

diamuxin · Mon Mar 11, 2024 8:23 pm

Must be something config specific, I do use wireguard for road-warrior setup and I have 0 such logs.
I don't use or have BTH though.

Try to connect with road-warrior and when you disconnect you will see that the handshak messages start coming out, this started since the first versions of 7.13.

Etz · Mon Mar 11, 2024 9:02 pm

Nope, I don't...
Just tested, deliberately.
That is why I am actually wondering, maybe some specific config setting is involved.

zajadacz · Mon Mar 11, 2024 9:14 pm

Can someone confirm issues with new SMB service? I tried to use hAP ac2 as SMB server, but it crashes when I'm transferring larger file (~8GB) to SSD
drive connected to USB port. I have submitted ticket (SUP-146120). New version (7.14.1) is released but there is no information in changelog about fixes in new SMB service. On 7.13.5 everything works fine. Tested with the same configuration and drive.

Grant · Mon Mar 11, 2024 9:46 pm

7.14.1
this wireguard bug has not been fixed

Handshake for peer did not complete after 5 seconds, retrying (try 2)
Handshake for peer did not complete after 20 attempts, giving up

diamuxin · Mon Mar 11, 2024 10:33 pm

Nope, I don't...
Just tested, deliberately.
That is why I am actually wondering, maybe some specific config setting is involved.

Maybe, let's wait and see if someone from the staff can enlighten us.
Thx.

sajt · Mon Mar 11, 2024 11:36 pm

7.14.1 ccr1009 directly connected routes still not added to vrf after update. downgrade to 7.13.5 fixes the issue.

Kentzo · Tue Mar 12, 2024 1:33 am

After the update to 7.14.1 (and possibly 7.14 as well) my hAP ac lite (RB952Ui-5ac2nD) cannot maintain wireless clients anymore

Appears to be an unrelated error in configuration that manifested only after a reboot.

ksteink · Tue Mar 12, 2024 4:43 am

3 CRS326-24G tested:
- 1 Upgraded without problems
- 1 Failed to upgrade (and storage reports 0% free space but I can dump a copy of the configuration and a backup without a problem)
- 1 Bricked (I need to do a Netinstall)

I am not thrilled with the outcome of these upgrades and since 7.13 these new versions are too buggy for production.

krafg · Tue Mar 12, 2024 4:57 am

What's new in 7.14.1 (2024-Mar-08 14:50):
*) lte - fixed R11e-LTE-US modem dial-up;

Yes, it fixed, now the question is why on all my devices I have a loopback interface?

Thanks and regards.

Reinis · Tue Mar 12, 2024 6:44 am

Yes, it fixed, now the question is why on all my devices I have a loopback interface?

It was always there but hidden, now is exposed. Use cases for it are many and discussed plenty in previous page and in other topics

Etz · Tue Mar 12, 2024 8:47 am

Nope, I don't...
Just tested, deliberately.
That is why I am actually wondering, maybe some specific config setting is involved.
Maybe, let's wait and see if someone from the staff can enlighten us.
Thx.

We can compare the configs, just in case:

/interface wireguard
add comment="vpn:wireguard1" listen-port=443 mtu=1420 name=wireguard1

/interface wireguard peers
add allowed-address=<redacted>.10/32,<redacted>:13ff::10/128 client-address=<redacted>.10/32,<redacted>:13ff::10/128 client-dns=\
    <redacted>.1,<redacted>:1300::1 client-endpoint=<redacted> comment=vpn:peer1 interface=wireguard1 preshared-key=\

Znevna · Tue Mar 12, 2024 9:42 am

What's new in 7.14.1 (2024-Mar-08 14:50):
*) wireguard - do not attempt to connect to peer without specified endpoint-address;
Needs more work. When a peer without endpoint-address connects and then disconnects - RouterOS still floods the log with useless messages (probably because now the peer does have remote address under the hood - the address of the last message from peer)

If you enable wireguard logging on other platforms it doesn't behave the same? Because I think it does.

zervan · Tue Mar 12, 2024 9:47 am

I was updating CRS328-4C-20S-4S+ from 7.13.3 and CRS328-24P-4S+ from 7.13.5 - both have failed:

setting up elf image... not an elf header, kernel loading failed

I had to netinstall both of them. Is this normal?
I am not sure, but it is possible that before update I have marked wireless package to uninstall.

Znevna · Tue Mar 12, 2024 9:51 am

Boy, you had some expensive bricks there for a while.

diamuxin · Tue Mar 12, 2024 9:53 am

We can compare the configs, just in case:

Thanks for sharing:

My roadwarrior configuration:

/interface wireguard
add comment="vpn: roadwarrior" listen-port=54321 mtu=1420 name=wg-rw

/interface wireguard peers
add allowed-address=<redacted>.30.2/32 comment="My Mobile" interface=wg-rw public-key="<redacted>XSbE/y0w2x0kVFYCsILdOBkIsk+w0="

How do you see it? It has always worked fine with previous versions.

Guscht · Tue Mar 12, 2024 10:10 am

Any good reasons to increase from 10 to 30 seconds:

*) firewall - increased default "udp-timeout" value from 10s to 30s;

All deployed devices - v6 and v7 - are configured for 10 seconds.
But I'd like to stay consistent with new deployments. What are the thoughts/background about this step? There must be s story or at least a good reason to do so.

UpRunTech · Tue Mar 12, 2024 10:26 am

I suspect there is an issue with 7.14 and CRS354. After upgrading CAPs would log brief disconnects to CAPSMAN and internet performance was poor. Downgrading to 7.13.5 restored performance.

cowgirl · Tue Mar 12, 2024 10:44 am

*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

Apparently the back port doesn't work in 7.14.1 - still crashing.....

Time 2024-03-11 15:50:52
Buffer memory
Topics
system
error
critical
Message kernel failure in previous boot

Etz · Tue Mar 12, 2024 10:46 am

We can compare the configs, just in case:
Thanks for sharing:

My roadwarrior configuration:
Code: Select all
/interface wireguard
add comment="vpn: roadwarrior" listen-port=54321 mtu=1420 name=wg-rw

/interface wireguard peers
add allowed-address=<redacted>.30.2/32 comment="My Mobile" interface=wg-rw public-key="<redacted>XSbE/y0w2x0kVFYCsILdOBkIsk+w0="
How do you see it? It has always worked fine with previous versions.

No idea, to be honest, but I have not seen any of those log messages.
I was kind of expecting to see, some keepalive or something in your config.

Tue Mar 12, 2024 11:19 am

WireGuard fix solves an issue where RouterOS was trying to connect to peers which have not been used yet and there is no known endpoint-address. WireGuard is a bidirectional tunnel. Once you connect peer, it knows "last remote endpoint" information and tries to connect to it. Fix solved the issue just for unused peers. Used peers do behave as expected - bidirectional tunnels trying to reach the remote endpoint - either configured or last used.

m4rk3J · Tue Mar 12, 2024 11:36 am

Any good reasons to increase from 10 to 30 seconds:
*) firewall - increased default "udp-timeout" value from 10s to 30s;

All deployed devices - v6 and v7 - are configured for 10 seconds.
But I'd like to stay consistent with new deployments. What are the thoughts/background about this step? There must be s story or at least a good reason to do so.

I had issues with MS Windows RDP and after changing default 10s to 30s.. problem dissapeared.
But I don't have any idea, why the same value on ROS v6 works well.
I had to get it up and running quickly, so I didn't bother with it anymore.

erlinden · Tue Mar 12, 2024 11:53 am

WireGuard fix solves an issue where RouterOS was trying to connect to peers which have not been used yet and there is no known endpoint-address.

I'm running into the situation that after enabling and disabling RoadWarrior on a client device (where endpoint is not set on the router), the router is still trying to connect until it reaches 20 times and then it logs:

WG-Interface: [whatever]: Handshake for peer did not complete after 20 attempts, giving up

But...this line is repeated 5 times (with 2 minutes in between).
Is this expected behaviour?

Code:

/interface wireguard
add listen-port=13231 mtu=1420 name=WG-Interface
/interface wireguard peers
add allowed-address=10.0.0.51/32 comment="Mobiel erlinden" interface=WG-Interface public-key="[whatever]"

diamuxin · Tue Mar 12, 2024 12:16 pm

WireGuard fix solves an issue where RouterOS was trying to connect to peers which have not been used yet and there is no known endpoint-address. WireGuard is a bidirectional tunnel. Once you connect peer, it knows "last remote endpoint" information and tries to connect to it. Fix solved the issue just for unused peers. Used peers do behave as expected - bidirectional tunnels trying to reach the remote endpoint - either configured or last used.

@strods, thanks for responding.

I understand what you have explained. But there is a problem because those results invade the log continuously with those messages and to avoid it I have created a logging filter (!wireguard), so I stop seeing them but, what happens if there are other important notices of the wireguard topic? they would not be seen.

Is there any way to create filtering rules? e.g. I want to avoid displaying messages containing the word "handshake for peer" and the rest of the wireguard messages are displayed..

EDIT: Couldn't MikroTik modify those messages to be in the debug topic instead of info? that would solve it.

Thx.

osc86 · Tue Mar 12, 2024 12:21 pm

Is this expected behaviour?

of course this is expected behavior. people finally need to stop thinking of wireguard being just another server / client vpn software, which gives up after x unsuccessful connection attempts; it is not! once a peer has learned a peer's remote address, it will try to establish a connection indefinitely, so log messages regarding connection retries / handshake messages will appear indefinitely. In case the remote peer is not reachable for whatever reason (ip address has changed but didn't initialize a new connection, client went offline, blocked by a filter, ...), handshake timeout messages are being generated. If you're all so annoyed by this, just set up your logging correctly.

diamuxin · Tue Mar 12, 2024 12:42 pm

of course this is expected behavior. people finally need to stop thinking of wireguard being just another server / client vpn software, which gives up after x unsuccessful connection attempts; it is not! once a peer has learned a peer's remote address, it will try to establish a connection indefinitely, so log messages regarding connection retries / handshake messages will appear indefinitely. In case the remote peer is not reachable for whatever reason (ip address has changed but didn't initialize a new connection, client went offline, blocked by a filter, ...), handshake timeout messages are being generated. If you're all so annoyed by this, just set up your logging correctly.

Of course, you are right, wireguard does not understand if the connection is fixed (site to site) or road warrior (site to mobile) and checks indefinitely. For this they would have to create a "feature" that allows to select a maximum number of checks (rw) or unlimited (in case of sts).

Thank you for clarifying this.

erlinden · Tue Mar 12, 2024 12:43 pm

If you're all so annoyed by this, just set up your logging correctly.

Perhaps my question wasn't clear: what is Wireguard "giving up" after 20 attempts?
Besides...my complete wireguard logging:

 10:29:18 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:29:36 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:30:13 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:30:45 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:31:01 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:31:17 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:31:51 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:32:42 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:32:57 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:33:03 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:33:08 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:34:43 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:35:55 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:36:00 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:36:06 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:36:42 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:38:24 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 20 attempts, giving up
 10:38:45 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:40:31 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 20 attempts, giving up
 10:40:46 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:42:31 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 20 attempts, giving up
 10:42:47 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:44:30 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 20 attempts, giving up
 10:44:48 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 5 seconds, retrying (try 2)
 10:46:35 wireguard,info WG-Interface: [public key]: Handshake for peer did not complete after 20 attempts, giving up

After the last giving up it actually gave up.

mkx · Tue Mar 12, 2024 12:55 pm

*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

You should read the line for what it is: "SFP - improved stability" (on some certain device). You simply should not read it like "improved stability of CCR2004-1G-XS-PCIe" because it's not about it.

Chupaka · Tue Mar 12, 2024 2:06 pm

/interface/wireguard/peers
:foreach i in=[find where disabled=no endpoint-address="" current-endpoint-address!=""] do={
  :local LastHandshake [get $i last-handshake]
  :if ($LastHandshake > [:totime "3m"]) do={
    enable $i
  }
}

This effectively unsets current-endpoint-address at missing peers after 3 minutes

eworm · Tue Mar 12, 2024 2:07 pm

This may cause a short interruption in packet flow... Perhaps add something like "last-handshake>2m"?

Chupaka · Tue Mar 12, 2024 2:08 pm

"($LastHandshake > [:totime "3m"])"?..

eworm · Tue Mar 12, 2024 2:17 pm

A decimal with suffix "m" is handled as time automatically, as long as you skip the quotes. And two minutes should be fine, that is the interval last-handshake is updated.

Yes, your code works as expected, I did not answer precisely. What I wanted to say is that you do not even need to enter the loop for most peers. That is what I would use:

:foreach Peer in=[ /interface/wireguard/peers/find where disabled=no endpoint-address="" current-endpoint-address!="" last-handshake>2m] do={
  /interface/wireguard/peers/enable $Peer;
}

cowgirl · Tue Mar 12, 2024 3:15 pm

*) sfp - improved system stability for CR2004-1G-2XS-PCIe (introduced in v7.14);

You should read the line for what it is: "SFP - improved stability" (on some certain device). You simply should not read it like "improved stability of CCR2004-1G-XS-PCIe" because it's not about it.

its exactly about it, at least for me, it fixed the issue with beta6, but the back port doesn´t

Tue Mar 12, 2024 3:34 pm

Version 7.15 will include WireGuard peer functionality "responder". Then you will be able to mark your peer as "responder". Then the peer will not spam your log, if you enable this option manually.

diamuxin · Tue Mar 12, 2024 4:30 pm

Version 7.15 will include WireGuard peer functionality "responder". Then you will be able to mark your peer as "responder". Then the peer will not spam your log, if you enable this option manually.

Great news!
Thank you very very much.

pedkoschi · Tue Mar 12, 2024 5:37 pm

@pedkoschi - Use PS to update your Windows IKEv2 configuration to the encryption level that suits for both clients:

PS C:\Users\user> Set-VpnConnectionIPsecConfiguration -ConnectionName "xxxx" -AuthenticationTransformConstants SHA256128 -CipherTransformConstants AES256 -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -PfsGroup PFS2048 -DHGroup Group14 -PassThru -Force

This is also possible for Mac clients with .mobileconfig profiles using Apple Configurator.

Thanks for the reply. Of course, I did something similar to avoid using weak encryption.
Windows wants to use auth: sha256 and prf hmac-sha256 here, the problem arises when trying to serve both windows and android 11 , which wants auth: sha256 and prf hmac-sha1

pe1chl · Tue Mar 12, 2024 6:03 pm

Yeah, that is always a problem with IPsec. Some OS requiring settings that other OSes refuse.

krafg · Tue Mar 12, 2024 6:21 pm

Yes, it fixed, now the question is why on all my devices I have a loopback interface?
It was always there but hidden, now is exposed. Use cases for it are many and discussed plenty in previous page and in other topics

Thanks for response.

Regards.

jsadler · Tue Mar 12, 2024 7:00 pm

Issue with EOIP Tunnel Source Address...

This was working at 7.13.5 but now seems broken at 7.14.1 - Previously, when you set a "Local Address" for an EOIP tunnel then this will be the local address used to source traffic. The remote end will see traffic originating from that address.

Unfortunately, it now appears like the 'Local Address' is not honoured and the router may reply from a different interface..

This is quite frustrating.

Cheers,
Jono.

osc86 · Tue Mar 12, 2024 8:58 pm

@jsadler I'm using eoip + local address without issues in 7.14.1.
When you say that the local address is not being honored, the first thing you should check is the connection table (IP->Firewall->Connections).
Set up a destination address filter that matches the value of dst-addr of the eoip interface. Enable the additional column "Reply Src. Address" and verify, that your connection is not accidentally being snat'ed.

pedkoschi · Tue Mar 12, 2024 8:59 pm

Yeah, that is always a problem with IPsec. Some OS requiring settings that other OSes refuse.

Hi, please don't give up easily.

it is not an exception but rather common practice to handle multiple phase1 selectors per peer.

Currently we have this rule:.
"
If the remote peer's address matches this prefix, then the peer configuration is used in authentication and establishment of Phase 1. If several peer's addresses match several configuration entries, the most specific one (i.e. the one with the largest netmask) will be used.
"

What if there are still two or more matches?

Just select the one with the "best" matching profile :)

petardo · Tue Mar 12, 2024 9:00 pm

Hi,
I had a strange wg issue (actually with 7.12.1, but the behavior didn't change after upgrading to 7.14.1 either):

Configuration:
Peer A initiates a connection to peer B on peer B's given public address : port.

Issue:
There was a network reconfiguration by the service provider on site A.
After this, peer A didn't receive peer B's reply anymore, no handshake occurred.

Resolution:
After clearing the random wg port on interface A, peer A created a new random port and the connection has been restored.
Probably the previous port didn't work anymore.

Question:
Is this behavior normal?
Wouldn't it be possible that a initiator peer changes it's random source port if no handshake occurs within a given period of time?

jsadler · Tue Mar 12, 2024 9:13 pm

@jsadler I'm using eoip + local address without issues in 7.14.1.
When you say that the local address is not being honored, the first thing you should check is the connection table (IP->Firewall->Connections).
Set up a destination address filter that matches the value of dst-addr of the eoip interface. Enable the additional column "Reply Src. Address" and verify, that your connection is not accidentally being snat'ed.

@osc86 interesting that you're not seeing issues, I've rolled back to 7.13.5 and issue has gone away. I was short on time so didn't get to look deep into this, I'll have to lab it up later, but what I did see on the broken scenario before I rolled back was the firewall connection table showing the correct 'reply src address' however when I packet capture further north in our network I see the router replying from its linknet address instead of the local address that it should have used per the EOIP tunnel. The linknet isn't routable elsewhere in our network so the traffic is dropped before it hits the destination. Note that the linknet IP that is the incorrect source and the EOIP local address both sit on the same vlan interface which faces elsewhere in the network.

Cheers,
Jono.

pe1chl · Tue Mar 12, 2024 9:55 pm

Yeah, that is always a problem with IPsec. Some OS requiring settings that other OSes refuse.
Hi, please don't give up easily.

it is not an exception but rather common practice to handle multiple phase1 selectors per peer.

Currently we have this rule:.
"
If the remote peer's address matches this prefix, then the peer configuration is used in authentication and establishment of Phase 1. If several peer's addresses match several configuration entries, the most specific one (i.e. the one with the largest netmask) will be used.
"

That doesn't help when your goal is to have "road warrior" connectivity (i.e. remote prefix is 0.0.0.0/0)...
Yes indeed it can be used to have very strong encryption to one specific peer (e.g. for a tunnel) and default settings for the remainder.

What if there are still two or more matches?

Just select the one with the "best" matching profile :)

I don't think that can be done in IPsec. It is the protocol that limits that, not the implementation.

osc86 · Tue Mar 12, 2024 10:24 pm

@jsadler sounds like your setup is a bit more complicated than mine. I've assigned dedicated ip addresses (/30) to loopback interfaces which are explictely used as src and dst addresses for eoip interfaces on both sides. (still using bridges instead of lo). Addresses are distributed using ospf, connections between them are secured by wireguard / ipsec and connection tracking is disabled for loopback networks.

pedkoschi · Wed Mar 13, 2024 12:03 am

That doesn't help when your goal is to have "road warrior" connectivity (i.e. remote prefix is 0.0.0.0/0)...
Yes indeed it can be used to have very strong encryption to one specific peer (e.g. for a tunnel) and default settings for the remainder.

Hi, why?

Suppose I have
peer1 for src 0.0.0.0/0
and
peer2 for src 0.0.0.0/0

peer1 is linked to profile1 wich accepts a set of parameters for phase1 (=set1)
peer2 is linked to profile2 wich accepts another set of parameters for phase1 (=set2)

Now
first client offers a proposal matching set1 > accept and take peer1
second client offers a proposal matching set2 > accept and take peer2
third client offers a proposal not matching any of the defined profiles > abort phase1

I don't think that can be done in IPsec. It is the protocol that limits that, not the implementation.

the protocol just accepts one of client's phase1 proposal(s) or not.
at least forti and watchguard can handle any combination of hash/prf/enc in phase1 for any source ip or network.

pe1chl · Wed Mar 13, 2024 12:24 am

You can only indentify different peers before starting parameter negotiation when using exchange mode "agressive" instead of the default "main". That will again be a problem when you want to abide by the rules of several different client OSes.

Kentzo · Wed Mar 13, 2024 12:52 am

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

What is "dark-mode", is it the "all-leds-off" LEDs setting?

infabo · Wed Mar 13, 2024 9:08 am

fragtion · Wed Mar 13, 2024 3:06 pm

Hi,
I had a strange wg issue (actually with 7.12.1, but the behavior didn't change after upgrading to 7.14.1 either):

Configuration:
Peer A initiates a connection to peer B on peer B's given public address : port.

Issue:
There was a network reconfiguration by the service provider on site A.
After this, peer A didn't receive peer B's reply anymore, no handshake occurred.

Resolution:
After clearing the random wg port on interface A, peer A created a new random port and the connection has been restored.
Probably the previous port didn't work anymore.

Question:
Is this behavior normal?
Wouldn't it be possible that a initiator peer changes it's random source port if no handshake occurs within a given period of time?

I've experienced this too. In fact it has been the leading cause of wireguard sessions failing in my experience. Good job finding the culprit and solution: resetting the configured port of the wireguard interface (not peer) - I arrived at the same solution.

I think what's happening is that your wg "client" side router is behind another router (double NAT / CG-NAT) and the upstream router has a cached UDP connection (UDP timeout has not expired) which fails when its public IP changes.
So yes, changing the source port for that connection (via the interface port) seems to be the only realistic solution. Unless MikroTik can implement a native inbuilt solution for this (maybe randomize source port for each connection attempt if the interface port isn't explicitly set, as this would be an ideal solution for such wg endpoints behind double nat), then your best bet is to script a checking mechanism on scheduler, to randomize the port based on last handshake or such

Perhaps a higher keepalive timer could also help for tunnels that have *very* low/intermittent traffic, as that could allow the upstream udp connection to time-out while no traffic is taking place between the peers (which wouldn't be possible if the keepalive keeps pinging via the broken upstream connection, effectively [and quite ironically] keeping the link in a dead state). Not the most reliable workaround but could help for some fringe setups

iustin · Wed Mar 13, 2024 6:45 pm

- changed default "fan-min-speed-percent" from 0% to 12%;
- improved fan control on CRS3xx and CCR1016-12S-1S+r2;
I have no fan control at all after this 7.14 update. The fan of my CRS310-1G-5S-4S+IN runs with annoying maximum speed and reports ~+-13000rpm regardless which settings i put in the health settings menu. CPU temp was 39°C and Board around 33°C with default settings for the temperature control.

Only a full rollback to 7.13.5 and loading the backup solved it immediately. Please have a look into this.

Same here, 7.14.1, the CRS310-5S-4S+ is running full speed. No tweaking of the values seem to fix this, it's 13K RPM continuously. Mikrotik, please fix this!

sinisa · Wed Mar 13, 2024 7:14 pm

Just installed wifi-qcom-ac on a pair of p-t-p RBSXTsqG-5acD (first on local station-(pseudo)bridge), then on remote AP (with some scripting to run at boot and configure wifi1 interface and assign it IP address), prepared to "travel" to other side of town to netinstall if something should go wrong.
Everything went just fine, both devices are up and running, brigde seems to be stable and all that.

Now, previously I was monitoring the signal levels, SNR and throughput via SNMP. I already knew in advance that there is no SNMP with new Wifi, but I am using this opportunity (besides for some bragging) to nicely ask (again):
Mikrotik, pretty please, with cherry on top, stop adding new features, useless on small/older devices like hAP ac2, cAP ac, SXTsq (you know what I mean, 16MB flash), and restore functionality that we used to have before (like SNMP for Wifi interfaces).

Thank you!

Btw. SXTsq has level 3 license, I could not run AP on it before, but now my "master" device that used to be "bridge" is called "ap". Does it mean that I could have more than one client now if I wanted, not that I need that at this moment but just being curious? Or there is some limit of 1 client in driver? Have to ask, it is too far away to test right now...

patrickmkt · Wed Mar 13, 2024 7:36 pm

I am still having trouble with the /tool/SMS/Allowed-Number that disappears every time you do a /tool/sms/set receive-enable=yes

stmx38 · Wed Mar 13, 2024 11:07 pm

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;
What is "dark-mode", is it the "all-leds-off" LEDs setting?

LED Settings
The listed devices support turning off their LEDs (LED dark mode), however, some LEDs still cannot be turned off due to the device design factors.

/system leds setting all-leds-off

DarkNate · Thu Mar 14, 2024 12:35 am

Has the arm64 VPLS asymmetric bug, been fixed in 7.14.1? Can I now push at least 10Gbps VPLS using a CCR2004?

AndiiiHD · Thu Mar 14, 2024 8:50 am

Same here, 7.14.1, the CRS310-5S-4S+ is running full speed. No tweaking of the values seem to fix this, it's 13K RPM continuously. Mikrotik, please fix this!

i found the answer somewhere (on reddit i think, but can't find the link) the CRS310-1G-5S-4S+IN fan is only able to handle two different fan states. These are OFF and ON (with max speed). Its not controllable in any other way. So you should set the "fan min speed percent" value to 0 again i.e. set the value back to 0% from 12%. This fixed it for me now. Fan only spins on startup then gets off. Sadly SwitchOS does not have this feature so its max noise there and i am forced to use RouterOS.

clte19ax · Thu Mar 14, 2024 2:05 pm

Help, this update broke my containers. I just get "execve: No such file or directory" in the log on both of my containers when I try to start them.

Here are the relevant settings:

The containers are on a USB drive that seems to be working fine. What's the error, how can I get them working again?

I have the same issue. After some tinkering pihole started to work again, but one other container had to be reinstalled, something got corrupted. Be careful with this update if you run containers.

UPDATE: There is something wrong with USB mounts. This needs to be investigated!

optio · Thu Mar 14, 2024 9:43 pm

@clte19ax Maybe there is a issue that containers are not stopped gracefully before system shutdown (or there is short timeout for waiting). On mine device Pi-hole container is in stopping state about 2-3s before goes to stopped when manually stopped, some others a bit shorter, but overall, device shutdowns quite fast even when many containers are running so I'm guessing they are not stopped gracefully. Just because of that I'm stopping containers manually before shutdown/reboot to be safe.
If system is shuts down while some process is writing on disk that can lead to corruptions.

robmaltsystems · Thu Mar 14, 2024 10:16 pm

I am still having trouble with the /tool/SMS/Allowed-Number that disappears every time you do a /tool/sms/set receive-enable=yes

Related problem on 7.14.1 on (underpowered) hAP lite. /tool/sms/export always throws an error. I've never used sms... I've reported the fault to Mikrotik. Worked okay on 7.12.1 and on hAP ax2.

dzievamarcos · Thu Mar 14, 2024 10:45 pm

Help, this update broke my containers. I just get "execve: No such file or directory" in the log on both of my containers when I try to start them.

Here are the relevant settings:

The containers are on a USB drive that seems to be working fine. What's the error, how can I get them working again?
I have the same issue. After some tinkering pihole started to work again, but one other container had to be reinstalled, something got corrupted. Be careful with this update if you run containers.

UPDATE: There is something wrong with USB mounts. This needs to be investigated!

Thanks for letting me know.

iustin · Thu Mar 14, 2024 11:22 pm

Same here, 7.14.1, the CRS310-5S-4S+ is running full speed. No tweaking of the values seem to fix this, it's 13K RPM continuously. Mikrotik, please fix this!
i found the answer somewhere (on reddit i think, but can't find the link) the CRS310-1G-5S-4S+IN fan is only able to handle two different fan states. These are OFF and ON (with max speed). Its not controllable in any other way. So you should set the "fan min speed percent" value to 0 again i.e. set the value back to 0% from 12%. This fixed it for me now. Fan only spins on startup then gets off. Sadly SwitchOS does not have this feature so its max noise there and i am forced to use RouterOS.

Ooh, awesome, thank you! I thought 12% is the target when it is running, not in both cases. Well, lesson learned, thanks a lot!

clte19ax · Fri Mar 15, 2024 9:18 am

@clte19ax Maybe there is a issue that containers are not stopped gracefully before system shutdown (or there is short timeout for waiting). On mine device Pi-hole container is in stopping state about 2-3s before goes to stopped when manually stopped, some others a bit shorter, but overall, device shutdowns quite fast even when many containers are running so I'm guessing they are not stopped gracefully. Just because of that I'm stopping containers manually before shutdown/reboot to be safe.
If system is shuts down while some process is writing on disk that can lead to corruptions.

Yes, I agree that it could be the case, but the cause is not a reboot, but after this update my USB Flash keeps randomly disappearing. If I unplug/plug it in again then it starts to work again.

BWC · Fri Mar 15, 2024 11:13 am

AWS sent me a notification that there is a new ROS Image available, does this mean updates will work now without the Instance type t3 -> t2 -> t3 limbo when freshly deployed?

https://aws.amazon.com/marketplace/pp/p ... gn6js6av54

--Michael
yes

I created a fresh t3 instance with the 7.14 image, assigned my EIP and tried to update to 7.14.1, it's broken now.

So, Updates still not working, am I right?

--Michael

BWC · Sat Mar 16, 2024 1:11 pm

It was not mentioned in the changelog, but I wasn't able to install my own CA and Server certificate in 7.14, it just did not do anything, no log, no error message no nothing.

In 7.14.1 I was able to import the certs as usual, so there might be some changes.

--Michael

infabo · Sat Mar 16, 2024 1:14 pm

Maybe your certificate file you want to import contains spaces or special chars 🤔

BWC · Sat Mar 16, 2024 2:34 pm

Maybe your certificate file you want to import contains spaces or special chars 🤔

I changed the filename to ca.crt to avoid any trouble, but that didn't helped, until 7.14.1.

The CN of the CA contain spaces, but the CN of the Server did not.

7.14.x is a mixed bag for me, CHR updates on AWS still crash the VM, Certificate import seems to fixed but otherwise it works as usual.

--Michael

petardo · Sat Mar 16, 2024 4:37 pm

fragtion, Thanks for reply.

Unless Mikrotik can implement a native inbuilt solution for this (maybe randomize source port for each connection attempt if the interface port isn't explicitly set, as this would be an ideal solution for such wg endpoints behind double nat)

Someone from Mikrotik, are you going to enable a feature like this in a future release? Or do you have any other idea to solve the problem?

kinjakinja · Sun Mar 17, 2024 2:18 pm

haplite RB941-2nD with 7.14.1 (wifi and OS via netinstall, fresh input not restore cfg)

1. logout failure via winbox(3.40-x86) following with 100% usage for few minutes
2. kernel crash and self reboot
3. intermittent wan traffic halted

rollback to 7.14 problem 2 & 3 solved.
problem1 exists.

Sun Mar 17, 2024 3:29 pm

You may want to stay on 7.12 or even ros6 for that device.

robmaltsystems · Sun Mar 17, 2024 3:55 pm

I'm tempted to concur. The hap lite (smips) really does struggle with ROS 7. I suspect it's a little too underpowered and certainly doesn't have enough storage.

kinjakinja · Sun Mar 17, 2024 4:46 pm

i had a hap-ac2 ran 6.49.8 , A audience ran 6.48.10 as a AP and a spare hap-lite(current 7.14)
i tried the haplite on 7.2, 7.7, 7.13.5(horrible) 7.14 & 7.14.1

surprisingly the hap lite with 7.14 is extremely responsively on the wan connection even faster than my hap-ac2(6.49.8)
*except the admin login winbox failure, everhting else seems fine(inc. wifi)

i'll keep the haplite with 7.14 and ran for a week,
if it is good i may upgrade the main router ac2 to 7.14 and keep the haplite as a spare

MrRobotdev · Sun Mar 17, 2024 5:01 pm

Hi everyone,

I have several different MT devices and all now are at 7.14.1 without any visible problems (God only knows if everything is working as expected).

Among others i have my first MT - a hAP lite TC that cannot be upgraded due to hard disk limitations of 16 MB.

I have overcome this problem WITHOUT NETINSTALL easily following this procedure and i hope it will help others also:
1. Uninstall the wireless package.
2. Reboot.
3. Update the core package left behind without any problems.
4. Reboot.
5. Routerboard upgrade.
6. Reboot.
7. Download the appropriate (for example now the 7.14.1) wireless package for my device.
8. Copy the package to Mikrotik (drag and drop) through Winbox.
9. Reboot.
10 The MT has all the packages updated and the most important has kept all my configuration for the wifi (that has been uninstalled and installed again).

I dont know if this the correct way to do it, but it is working - at least for me.

Cheers

resetsa · Sun Mar 17, 2024 5:35 pm

Hi all! please help with route leak from VRF.
I will do as in example

/ip vrf
add interfaces=lte1 name=VRF_LTE
add interfaces=wlan2,wlan5 name=VRF_WIFI
/ip address
add address=192.168.103.33/27 interface=wlan2 network=192.168.103.32
add address=192.168.103.65/27 interface=wlan5 network=192.168.103.64
/routing bgp vpn
add export.redistribute=connected .route-targets=63000:200 import.route-targets=63000:100 label-allocation-policy=per-vrf name=bgp-mpls-vpn-1 route-distinguisher=63000:200 vrf=VRF_WIFI
add export.redistribute=modem .route-targets=63000:100 import.route-targets=63000:200 label-allocation-policy=per-vrf name=bgp-mpls-vpn-2 route-distinguisher=63000:100 vrf=VRF_LTE

and result

route print detail afi=ip4
...
Ay   afi=ip4 contribution=active dst-address=192.168.103.32/27 routing-table=VRF_LTE label=4294967295 gateway=VRF_WIFI@VRF_WIFI immediate-gw=VRF_WIFI distance=200 scope=40 target-scope=10 belongs-to="bgp-mpls-vpn-2-bgp-mpls-vpn-1-connected-export-import" bgp.ext-communities=rt:63000:200 .atomic-aggregate=no .origin=incomplete 
...
Ay   afi=ip4 contribution=active dst-address=0.0.0.0/0 routing-table=VRF_WIFI label=4294967295 gateway=VRF_LTE@VRF_LTE immediate-gw=VRF_LTE distance=200 scope=40 target-scope=10 belongs-to="bgp-mpls-vpn-1-bgp-mpls-vpn-2-modem-export-import" bgp.ext-communities=rt:63000:100 .atomic-aggregate=no .origin=incomplete debug.fwp-ptr=0x20242480

but not work, maybe reason for this immediate-gw equal VRF? maybe immediate-gw must point at interface in VRF?

robmaltsystems · Sun Mar 17, 2024 6:51 pm

I dont know if this the correct way to do it, but it is working - at least for me.

I had to do something similar. Would netinstall also be a way to update? Not as easy but I assume it doesn't have to have in effect two copies of the OS on there for a time.

connectlife · Mon Mar 18, 2024 12:41 pm

Hi everyone, we have a CCR2216-1G-12XS-2XQ updated with ROS 7.14.1 which randomly restarts every 2/3 days. After the restart we notice the KERNEL FAILURE error in the LOGs. Does this happen to anyone else?

SUP-147284

Znevna · Mon Mar 18, 2024 1:02 pm

Anything fancy in the config?
I have a 2116 that's still on 7.14beta something.

connectlife · Mon Mar 18, 2024 1:06 pm

Anything fancy in the config?
I have a 2116 that's still on 7.14beta something.

Nothing special.. BGP/OSPF/BFD. With v7.14 the problem never occurred..

Hominidae · Mon Mar 18, 2024 11:03 pm

...updated two hap ac^3 from 7.13.1 to 7.14.1....both devices are running wifi-qcom-ac, vlans for ssids in bridge filtering mode and booting in CAP mode
The bridge/vlan config got messed up completely during upgrade

although all of the wifi interfaces got provisioned by the capsman host in the correct order (wifi1, wifi2, wif3, ..., wifi7) not all wifi interfaces were still present in the bridge ports (but there were no unknown ports listed)
the PVIDs allocated to each wifi interface got messed up, hence vlan filtering got messed up, too...resulting in wifi clients being allocated / associated with wrong vlans and dhcp allocated IPs.
the lo/loopback interface was added to the bridge ports config

...had to manually add all wifi interfaces to the bridge ports again, as well as allocating their correct PVIDs in the VLAN config.
I also deleted the loopback interface from the bridge ports config (shouldn't I do so?... or rather if it were to stay, which PVID should I assign to it??)

mabels · Mon Mar 18, 2024 11:11 pm

Hi,

i upgrade from v7.13.1 to v7.14 on a CRS326-24G-2S+RM a week ago and after this upgrade the devices was not working anymore.
I had been not away from the device so there was no analysis possible. Now I'm back at the device an the only visible reaction after a power cycle is the blue power led.

What can i do to get the device working again?

thx in advance

meno

Hominidae · Mon Mar 18, 2024 11:40 pm

What can i do to get the device working again?

...netinstall, as always.

infabo · Tue Mar 19, 2024 1:59 am

a netinstall in need is a friend indeed.

sirca · Tue Mar 19, 2024 4:03 am

Hi,
I upgraded, packages and firmware, from 7.13.5 to 7.14.1
2x CRS326-24G
CRS305
CRS317
CHR
All upgraded without issues but for CRS317 the "idle" fan1-speed and fan2-speed went from 0 to 3780RPM as reported by winbox and cli:

[admin@MikroTik_CRS317] /system/health> print
Columns: NAME, VALUE, TYPE
#  NAME             VALUE  TYPE
0  cpu-temperature  35     C   
1  fan1-speed       3765   RPM 
2  fan2-speed       3720   RPM 
3  psu1-state       ok         
4  psu2-state       ok         
[admin@MikroTik_CRS317] /system/health>

Best Regards :)

sirca · Tue Mar 19, 2024 4:26 am

Hi again,
related to my previous post, I found following statement in documentation:

fan-min-speed-percent:
[...]
*NOTE: the default value may vary based on FAN controller chip and/or specific model requirements. From RouterOS verson 7.14 default value is set to 12, all previous versions have 0.
[...]

health.png

konstantinas · Tue Mar 19, 2024 8:44 am

*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
#notfixed - it's getting ridiculous..

Confirmed. I've spent a few hours to overhaul firewall configuration for the new VRF concept in 7.14.1 where you cannot filter separate interfaces in a VRF and then noticed that at least EoIP tunnel interface routes are not getting into an assigned VRF after reboot. You can recreate the interface and it will work until you reboot again.

If you have firewall rules on INPUT or FORWARD for any interface that are assigned to some non-main VRF you might consider to redesign it. Also if you have any EoIP interfaces in VRF just don't upgrade yet because it is almost certainly going to break.

Archous · Tue Mar 19, 2024 10:15 am

#notfixed - it's getting ridiculous..

Confirmed. I've spent a few hours to overhaul firewall configuration for the new VRF concept in 7.14.1 where you cannot filter separate interfaces in a VRF and then noticed that at least EoIP tunnel interface routes are not getting into an assigned VRF after reboot. You can recreate the interface and it will work until you reboot again.

If you have firewall rules on INPUT or FORWARD for any interface that are assigned to some non-main VRF you might consider to redesign it. Also if you have any EoIP interfaces in VRF just don't upgrade yet because it is almost certainly going to break.

We still are having this same issue as well.

ips · Tue Mar 19, 2024 1:05 pm

Hi again,
related to my previous post, I found following statement in documentation:

fan-min-speed-percent:
[...]
*NOTE: the default value may vary based on FAN controller chip and/or specific model requirements. From RouterOS verson 7.14 default value is set to 12, all previous versions have 0.
[...]

health.png

If I remember correctly, there are some messages in this discussion about fan speed. Something like "in some models, fans can only be switched off or going at max speed, so the default value set to 12 means that fans are always at max speed. Setting the value to 0 restores the previous situation.". But please check.

AndiiiHD · Tue Mar 19, 2024 2:03 pm

i wrote this here

sirca · Wed Mar 20, 2024 2:36 am

I think in CRS317 there are several fan "levels" other than 0 and 100, because at startup, during boot or reboot, both fans spin more over the level they seat after boot is complete (12%->3750RPM).

3dfx · Wed Mar 20, 2024 9:55 am

After upgrading to v7.14 I lost connectivity between OpenVPN server and clients in Ethernet mode. The connection is established normally and automatic routes are created, but still peers are inaccessible.
Anyone experiencing similar behaviour?

Edit: It's the same with v7.15beta4. Reverting back to v.7.13.5 solves the issue for me.

Still the same with v7.14.1...

Edit:
OK, switching the STP of the bridge from RSTP to none makes the peers reachable...

bratislav · Wed Mar 20, 2024 12:48 pm

Hi again,
related to my previous post, I found following statement in documentation:

fan-min-speed-percent:
[...]
*NOTE: the default value may vary based on FAN controller chip and/or specific model requirements. From RouterOS verson 7.14 default value is set to 12, all previous versions have 0.
[...]

health.png

Depending on the fan with PWM controller (as used in CRS317) it may have different minimum speed and anything bellow some percentage (for example 30 percent) would run at same RPM unless percentage is zero (no PWM signal) in which case it will stop. By the way PWM signal percentage is more of a suggestion and it is not guaranteed that actual speed would exactly correspond to a percentage of a max fan RPM...

Railander · Wed Mar 20, 2024 9:04 pm

with the linux loopback interface being now exposed, the loopback address ::1 shows up in /ipv6/address and /ipv6/route
this creates unexpected behavior of OSPF's "redistribute connected" advertising ::1
IPv4 does not have this issue, likely as it does not show up in /ip/address or /ip/route

> ipv6/route/print where dst-address in ::/16
Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, o - OSPF; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, DISTANCE
     DST-ADDRESS  GATEWAY                                    DISTANCE
D oH ::1/128      fe80::de2c:6eff:fe38:c7e6%vlan13-sfpplus3       110
DAc  ::1/128      lo

> ip/route/print where dst-address in 127.0.0.0/8

pedkoschi · Wed Mar 20, 2024 9:46 pm

You can only indentify different peers before starting parameter negotiation when using exchange mode "agressive" instead of the default "main". That will again be a problem when you want to abide by the rules of several different client OSes.

hi pe1chl,
In each mode, the first thing you get is SA and VID.
At this point, you know the IP and initiator's proposals as well as everything about NAT-T, fragmentation, etc.
Theoretically, you can re-use any information available. It is a nonsensical example
but you could require Aes256 encryption if the initiator doesn't support fragmentation.

RouterOS decides which peer configuration to use based on the initiator IP (and its own interface IP) from top to bottom.
The associated P1 configuration(profile) has to match the initiator's proposal.
In this scenario, the P1 configuration is the result of the selection process and not an input parameter.

In contrast, e.g. FortiOS has a list of parameters (P1 selectors), which takes all criteria into account in the selection, including initiator's IP.

Example
selector1 = (Encryption = AES256 AND Hash = SHA256 AND SourceIP = any)
selector2 = (Encryption = AES256 AND Hash = SHA1 AND SourceIP = any)

In this example, if an initiator from the network 0.0.0.0/0 offers Encryption = AES256 / Hash = SHA1, selector2 is used.
For another initiator from 0.0.0.0/0 that offers Encryption = AES256 / Hash = SHA256 selector1 is used.

I can even merge both into one:
selector3 = ((Encryption = AES256 AND Hash = SHA256) OR (Encryption = AES256 AND Hash = SHA1)) AND SourceIP = any

In this scenario, the P1 configuration is an input parameter for the selection process to decide whether the negotiation
will be continued or canceled.

In "ROS logic" something similar can be done by
a) processing all matching peer configurations (and associated profile configrations), not only the top peer config in the list, until both parties agree

or / and

b) extending the profile configuration which allows multiple combinations of SHA/PRF like already possible for encryption algorithm and dh-group.

And.. i dont want to complain, but rather wish an improvement for everyone involved..
cheers!

pedkoschi · Wed Mar 20, 2024 9:53 pm

I'm trying to get Windows 10 native IKEv2 VPN working with RouterOS 14.1.

but it doesn't work. I have tried many times but unfortunately I'm always getting this 'invalid payload' and the log looks like this:

22:00:03 ipsec -> ike2 request, exchange: SA_INIT:0 92.218.169.238[500] 5bf276184133cc0c:0000000000000000
22:00:03 ipsec ike2 respond
22:00:03 ipsec payload seen: SA (48 bytes)
22:00:03 ipsec payload seen: KE (136 bytes)
22:00:03 ipsec payload seen: NONCE (52 bytes)
22:00:03 ipsec payload seen: NOTIFY (8 bytes)
22:00:03 ipsec payload seen: NOTIFY (28 bytes)
22:00:03 ipsec payload seen: NOTIFY (28 bytes)
22:00:03 ipsec payload seen: VID (24 bytes)
22:00:03 ipsec,debug 1e2b516905991c7d7c96fcbfb587e46100000009
22:00:03 ipsec payload seen: VID (20 bytes)
22:00:03 ipsec,debug fb1de3cdf341b7ea16b7e5be0855f120
22:00:03 ipsec payload seen: VID (20 bytes)
22:00:03 ipsec,debug 26244d38eddb61b3172a36e3d0cfb819
22:00:03 ipsec payload seen: VID (24 bytes)
22:00:03 ipsec,debug 01528bbbc00696121849ab9a1c5b2a5100000002
22:00:03 ipsec processing payload: SA
22:00:03 ipsec IKE Protocol: IKE
22:00:03 ipsec  proposal #1
22:00:03 ipsec   enc: aes256-cbc
22:00:03 ipsec   prf: hmac-sha256
22:00:03 ipsec   auth: sha256
22:00:03 ipsec   dh: modp1024
22:00:03 ipsec matched proposal:
22:00:03 ipsec  proposal #1
22:00:03 ipsec   enc: aes256-cbc
22:00:03 ipsec   prf: hmac-sha256
22:00:03 ipsec   auth: sha256
22:00:03 ipsec   dh: modp1024
22:00:03 ipsec processing payload: KE
22:00:03 ipsec,debug => shared secret (size 0x80)
22:00:03 ipsec,debug ed4db930 da48c382 16c8b6d0 0e7e3c96 3c0ed077 375c1ad9 c442a294 ae125bd6
.... 
22:00:03 ipsec ike2 respond finish: request, exchange: SA_INIT:0 92.218.169.238[500] 5bf276184133cc0c:0000000000000000
22:00:03 ipsec processing payload: NONCE
22:00:03 ipsec adding payload: SA
22:00:03 ipsec,debug => (size 0x30)
22:00:03 ipsec,debug 00000030 0000002c 01010004 0300000c 0100000c 800e0100 03000008 02000005
22:00:03 ipsec,debug 03000008 0300000c 00000008 04000002
22:00:03 ipsec adding payload: KE
22:00:03 ipsec,debug => (size 0x88)
22:00:03 ipsec,debug 00000088 00020000 e871eec4 262ed58b d0daead4 fecf2e3b 631792de a3dea688
....
22:00:03 ipsec,debug 578bfb75 2be8d777
22:00:03 ipsec adding payload: NONCE
22:00:03 ipsec,debug => (size 0x1c)
22:00:03 ipsec,debug 0000001c d43c330a de7696ba e63586d7 c7cb3fe8 c06e8cf5 c6897518
22:00:03 ipsec adding notify: NAT_DETECTION_SOURCE_IP
22:00:03 ipsec,debug => (size 0x1c)
22:00:03 ipsec,debug 0000001c 00004004 f26bd54c 523a1686 e779aa1d 81dec242 86a73555
22:00:03 ipsec adding notify: NAT_DETECTION_DESTINATION_IP
22:00:03 ipsec,debug => (size 0x1c)
22:00:03 ipsec,debug 0000001c 00004005 44c34d90 2b8ce597 d5941902 a7fe024c d6d44692
22:00:03 ipsec adding notify: IKEV2_FRAGMENTATION_SUPPORTED
22:00:03 ipsec,debug => (size 0x8)
22:00:03 ipsec,debug 00000008 0000402e
22:00:03 ipsec adding payload: CERTREQ
22:00:03 ipsec,debug => (size 0x5)
22:00:03 ipsec,debug 00000005 04
22:00:03 ipsec <- ike2 reply, exchange: SA_INIT:0 92.218.169.238[500] 5bf276184133cc0c:748693d32f3f7b78
22:00:03 ipsec,debug ===== sending 309 bytes from 192.168.178.2[500] to 92.218.169.238[500]
22:00:03 ipsec,debug 1 times of 309 bytes message will be sent to 92.218.169.238[500]
22:00:03 ipsec,debug => skeyseed (size 0x20)
22:00:03 ipsec,debug cc750a6b bb60a237 7639d7f0 17f9d42f 572c3fd9 c8140c79 16ae7350 0b6e4b86
22:00:03 ipsec,debug => keymat (size 0x20)
22:00:03 ipsec,debug 1d5bb68f e3a796ba 3a66245a 346f9ba8 9529ddda 103acf02 73489b6f e38ba767
22:00:03 ipsec,debug => SK_ai (size 0x20)
22:00:03 ipsec,debug ee762705 107e5f72 0acee098 05124f20 50949121 dbafd085 923f57eb 4de1e677
22:00:03 ipsec,debug => SK_ar (size 0x20)
22:00:03 ipsec,debug 90389239 c6ff6fab d763950b 1bacbe5d afbad831 06c71e9f f364ae21 4c2c1477
22:00:03 ipsec,debug => SK_ei (size 0x20)
22:00:03 ipsec,debug 53a081d4 236c84b6 745bfca7 1d240545 30da4001 5d8a5cc1 7d1c0518 8567806d
22:00:03 ipsec,debug => SK_er (size 0x20)
22:00:03 ipsec,debug 496a502f 3980d2c3 e3844135 80c14ebf 0ef42c0f e2d67894 ed2e4d2e 9002f8a8
22:00:03 ipsec,debug => SK_pi (size 0x20)
22:00:03 ipsec,debug f8a8a7a0 bd019204 8f50773b 39b4d2e5 ac6efe9a d31cb592 72740a88 82977221
22:00:03 ipsec,debug => SK_pr (size 0x20)
22:00:03 ipsec,debug b8e6de6d 63087a3d 850c06c3 d1e5ea8f 21f3440c 7ffe0a30 1dfb55de 940804d7
22:00:03 ipsec,info new ike2 SA (R): peer-ikev2 192.168.178.2[500]-92.218.169.238[500] spi:748693d32f3f7b78:5bf276184133cc0c
22:00:03 ipsec processing payloads: VID
22:00:03 ipsec peer is MS Windows (ISAKMPOAKLEY 9)
22:00:03 ipsec processing payloads: NOTIFY
22:00:03 ipsec   notify: IKEV2_FRAGMENTATION_SUPPORTED
22:00:03 ipsec   notify: NAT_DETECTION_SOURCE_IP
22:00:03 ipsec   notify: NAT_DETECTION_DESTINATION_IP
22:00:03 ipsec (NAT-T) REMOTE LOCAL
22:00:03 ipsec KA list add: 192.168.178.2[4500]->92.218.169.238[4500]
22:00:03 ipsec fragmentation negotiated
22:00:03 ipsec,debug ===== received 580 bytes from 92.218.169.238[4500] to 192.168.178.2[4500]
22:00:03 ipsec -> ike2 request, exchange: AUTH:1 92.218.169.238[4500] 5bf276184133cc0c:748693d32f3f7b78
22:00:03 ipsec payload seen: SKF (552 bytes)
22:00:03 ipsec processing payload: ENC (not found)
22:00:03 ipsec processing payload: SKF
22:00:03 ipsec => invalid payload (first 0x100 of 0x228)
22:00:03 ipsec 23000228 00010015 6c54892a 2f32e614 c86353b2 116e9e12 93c7b5c8 1fb2f423
....
....
22:00:03 ipsec reply notify: INVALID_SYNTAX
22:00:03 ipsec adding notify: INVALID_SYNTAX

I have tried an older client (Windows 7) (IKEV2_FRAGMENTATION_SUPPORTED not supported)
In this case it works and the log looks like this:

22:04:44 ipsec -> ike2 request, exchange: SA_INIT:0 92.218.169.238[500] 30662d9e3dd4f417:0000000000000000
22:04:44 ipsec ike2 respond
22:04:44 ipsec payload seen: SA (256 bytes)
22:04:44 ipsec payload seen: KE (136 bytes)
22:04:44 ipsec payload seen: NONCE (52 bytes)
22:04:44 ipsec payload seen: NOTIFY (28 bytes)
22:04:44 ipsec payload seen: NOTIFY (28 bytes)
22:04:44 ipsec processing payload: SA
22:04:44 ipsec IKE Protocol: IKE
22:04:44 ipsec  proposal #1
22:04:44 ipsec   enc: 3des-cbc
22:04:44 ipsec   prf: hmac-sha1
22:04:44 ipsec   auth: sha1
22:04:44 ipsec   dh: modp1024
....
22:04:44 ipsec matched proposal:
22:04:44 ipsec  proposal #4
22:04:44 ipsec   enc: aes256-cbc
22:04:44 ipsec   prf: hmac-sha256
22:04:44 ipsec   auth: sha256
22:04:44 ipsec   dh: modp1024
22:04:44 ipsec processing payload: KE
22:04:44 ipsec,debug => shared secret (size 0x80)
22:04:44 ipsec,debug cacd86b3 2ac9f519 30f24759 918faec8 69eaf627 4e4f72ed 5b017fcf c93445ce
....
22:04:44 ipsec ike2 respond finish: request, exchange: SA_INIT:0 92.218.169.238[500] 30662d9e3dd4f417:0000000000000000
22:04:44 ipsec processing payload: NONCE
22:04:44 ipsec adding payload: SA
22:04:44 ipsec,debug => (size 0x30)
22:04:44 ipsec,debug 00000030 0000002c 04010004 0300000c 0100000c 800e0100 03000008 02000005
22:04:44 ipsec,debug 03000008 0300000c 00000008 04000002
22:04:44 ipsec adding payload: KE
22:04:44 ipsec,debug => (size 0x88)
22:04:44 ipsec,debug 00000088 00020000 4a43fee7 2414d79d bd3871af 89962461 59c21fa2 b1cf8f3c
....
22:04:44 ipsec adding payload: NONCE
22:04:44 ipsec,debug => (size 0x1c)
22:04:44 ipsec,debug 0000001c 6ab6c036 5d00e7cf 8625e9b1 377ab58a 9ab47d9d 681c0320
22:04:44 ipsec adding notify: NAT_DETECTION_SOURCE_IP
22:04:44 ipsec,debug => (size 0x1c)
22:04:44 ipsec,debug 0000001c 00004004 0f3ee9df 28abb5ca b3583693 fcdf2cd9 e19b1d26
22:04:44 ipsec adding notify: NAT_DETECTION_DESTINATION_IP
22:04:44 ipsec,debug => (size 0x1c)
22:04:44 ipsec,debug 0000001c 00004005 2e4de6b6 7ceca631 f9790f64 5eded1c3 2e17f433
22:04:44 ipsec adding notify: IKEV2_FRAGMENTATION_SUPPORTED
22:04:44 ipsec,debug => (size 0x8)
22:04:44 ipsec,debug 00000008 0000402e
22:04:44 ipsec adding payload: CERTREQ
22:04:44 ipsec,debug => (size 0x5)
22:04:44 ipsec,debug 00000005 04
22:04:44 ipsec <- ike2 reply, exchange: SA_INIT:0 92.218.169.238[500] 30662d9e3dd4f417:c54b477ba08eff43
22:04:44 ipsec,debug ===== sending 309 bytes from 192.168.178.2[500] to 92.218.169.238[500]
22:04:44 ipsec,debug 1 times of 309 bytes message will be sent to 92.218.169.238[500]
22:04:44 ipsec,debug => skeyseed (size 0x20)
22:04:44 ipsec,debug fdab80a4 4c283ba7 65b2ef51 8261a6ae 64e87139 fe416bb8 9bddecc2 4050601c
22:04:44 ipsec,debug => keymat (size 0x20)
22:04:44 ipsec,debug eeb82b4b c0fee115 2944b6df 2fec55cb 0f772b7d 378117d5 d70c7d56 ae6fd1d0
22:04:44 ipsec,debug => SK_ai (size 0x20)
22:04:44 ipsec,debug 02fca05c 95929c53 b80ddec7 42f14400 04889fff 43ef1b3a aac69eb0 8b89b8dd
22:04:44 ipsec,debug => SK_ar (size 0x20)
22:04:44 ipsec,debug c2f84726 ff8781a2 74bd8b45 6bf1dede 8f4ddeb4 ca1c58b6 b15c8c0f e33f21cf
22:04:44 ipsec,debug => SK_ei (size 0x20)
22:04:44 ipsec,debug 86a362eb 14bbbda7 cc6fe62b 26b3b1ef 80f1a019 0dacbb4e 8ec49d9c 23830b66
22:04:44 ipsec,debug => SK_er (size 0x20)
22:04:44 ipsec,debug 53e8890d 757e3f55 ea3e839d e1471a70 fc80758e fc567724 d1604a67 4943008d
22:04:44 ipsec,debug => SK_pi (size 0x20)
22:04:44 ipsec,debug 4e3039db 552a5d7c ece012c9 9d002bcf e8fa76d2 c77514af 80a29583 95fb78ed
22:04:44 ipsec,debug => SK_pr (size 0x20)
22:04:44 ipsec,debug c24a9936 582ade02 361b6684 71f84189 73de9413 a6c5265e c230570c 6212fa01
22:04:44 ipsec,info new ike2 SA (R): peer-ikev2 192.168.178.2[500]-92.218.169.238[500] spi:c54b477ba08eff43:30662d9e3dd4f417
22:04:44 ipsec processing payloads: VID (none found)
22:04:44 ipsec processing payloads: NOTIFY
22:04:44 ipsec   notify: NAT_DETECTION_SOURCE_IP
22:04:44 ipsec   notify: NAT_DETECTION_DESTINATION_IP
22:04:44 ipsec (NAT-T) REMOTE LOCAL
22:04:44 ipsec KA list add: 192.168.178.2[4500]->92.218.169.238[4500]
22:04:44 ipsec,debug ===== received 13488 bytes from 92.218.169.238[4500] to 192.168.178.2[4500]
22:04:44 ipsec -> ike2 request, exchange: AUTH:1 92.218.169.238[4500] 30662d9e3dd4f417:c54b477ba08eff43
22:04:44 ipsec payload seen: ENC (13460 bytes)
22:04:44 ipsec processing payload: ENC
22:04:44 ipsec,debug => iv (size 0x10)
22:04:44 ipsec,debug 745a3ad3 ddf4c60f 93d6930c ed188636
22:04:44 ipsec,debug decrypted packet
22:04:44 ipsec payload seen: ID_I (12 bytes)
22:04:44 ipsec payload seen: CERTREQ (13245 bytes)
22:04:44 ipsec payload seen: NOTIFY (8 bytes)
22:04:44 ipsec payload seen: CONFIG (24 bytes)
22:04:44 ipsec payload seen: SA (80 bytes)
22:04:44 ipsec payload seen: TS_I (24 bytes)
22:04:44 ipsec payload seen: TS_R (24 bytes)
22:04:44 ipsec processing payloads: NOTIFY
22:04:44 ipsec   notify: MOBIKE_SUPPORTED
22:04:44 ipsec ike auth: respond
22:04:44 ipsec processing payload: ID_I
22:04:44 ipsec ID_I (ADDR4): 192.168.2.142
22:04:44 ipsec processing payload: ID_R (not found)
22:04:44 ipsec processing payload: AUTH (not found)
22:04:44 ipsec processing payloads: NOTIFY
22:04:44 ipsec   notify: MOBIKE_SUPPORTED
22:04:44 ipsec ID_R (DER DN): C=DE, S=NRW, O=Home, CN=MT-CA
22:04:44 ipsec,debug => auth nonce (size 0x30)
22:04:44 ipsec,debug 9667e8ce ad0ff9db ef4db673 50fb39cd 8d4bf6d0 74195765 64c63c34 e84b6341
22:04:44 ipsec,debug 77dd0d44 2c1eaa82 a3eb2176 438c2f41
22:04:44 ipsec,debug => SK_p (size 0x20)
22:04:44 ipsec,debug c24a9936 582ade02 361b6684 71f84189 73de9413 a6c5265e c230570c 6212fa01
22:04:44 ipsec,debug => idhash (size 0x20)
22:04:44 ipsec,debug e91e774a 46d3d002 0f5e3773 67b7dc15 2713334f 3c7d6c7f 86d738c2 a708c460
22:04:44 ipsec,debug => my auth (size 0x100)
22:04:44 ipsec,debug 28affb55 ca8d5b92 e75cafd9 ec796184 ce48c793 8e12b4fa 704abe06 e326609d
...
22:04:44 ipsec adding payload: ID_R
22:04:44 ipsec,debug => (size 0x44)
22:04:44 ipsec,debug 00000044 09000000 303a310b 30090603 55040613 02444531 0c300a06 03550408
22:04:44 ipsec,debug 0c034e52 57310d30 0b060355 040a0c04 486f6d65 310e300c 06035504 030c054d
22:04:44 ipsec,debug 542d4341
22:04:44 ipsec adding payload: AUTH
22:04:44 ipsec,debug => (first 0x100 of 0x108)
22:04:44 ipsec,debug 00000108 01000000 28affb55 ca8d5b92 e75cafd9 ec796184 ce48c793 8e12b4fa
....
22:04:44 ipsec Certificate:
22:04:44 ipsec   serialNr:  ....
22:04:44 ipsec   issuer:    ....
22:04:44 ipsec   subject:   ...
22:04:44 ipsec   notBefore: Mon Mar 18 16:19:20 2024
22:04:44 ipsec   notAfter:  Thu Mar 18 16:29:20 2049
22:04:44 ipsec   selfSigned:1
22:04:44 ipsec   extensions:
22:04:44 ipsec     key usage: key-cert-sign, crl-sign
22:04:44 ipsec     basic constraints: isCa: TRUE
22:04:44 ipsec     subject key id:  ....
22:04:44 ipsec   signed with: SHA256+RSA
22:04:44 ipsec [RSA-PUBLIC]
22:04:44 ipsec modulus: ....
22:04:44 ipsec publicExponent: 10001
22:04:44 ipsec adding payload: CERT
22:04:44 ipsec,debug => (first 0x100 of 0x349)
22:04:44 ipsec,debug 00000349 04308203 40308202 28a00302 01020210 6c4a2ae3 844cc08f 4c95037b
....
22:04:44 ipsec Certificate:
22:04:44 ipsec   serialNr:  ....
22:04:44 ipsec   issuer:    ....
22:04:44 ipsec   subject:   ....
22:04:44 ipsec   notBefore: Mon Mar 18 10:18:50 2024
22:04:44 ipsec   notAfter:  Thu Mar 18 10:18:50 2049
22:04:44 ipsec   selfSigned:0
22:04:44 ipsec   extensions:
22:04:44 ipsec     key usage: digital-signature, key-encipherment, data-encipherment, key-agreement, key-cert-sign
22:04:44 ipsec     extended key usage: tls-server, tls-client
22:04:44 ipsec     subject key id:  ....
22:04:44 ipsec     authority key id:....
22:04:44 ipsec     subject alternative name: 
22:04:44 ipsec       DNS: ....
22:04:44 ipsec   signed with: SHA256+RSA
22:04:44 ipsec [RSA-PUBLIC]
22:04:44 ipsec modulus: ....
22:04:44 ipsec adding payload: CERT
22:04:44 ipsec,debug => (first 0x100 of 0x3ac)
22:04:44 ipsec,debug 000003ac 04308203 a3308202 8ba00302 01020210 7a76d202 e82291a9 4d989fb2
....
22:04:44 ipsec adding payload: EAP
22:04:44 ipsec,debug => (size 0x9)
22:04:44 ipsec,debug 00000009 01000005 01
22:04:44 ipsec <- ike2 reply, exchange: AUTH:1 92.218.169.238[4500] 30662d9e3dd4f417:c54b477ba08eff43
22:04:44 ipsec,debug ===== sending 2304 bytes from 192.168.178.2[4500] to 92.218.169.238[4500]
22:04:44 ipsec,debug 1 times of 2308 bytes message will be sent to 92.218.169.238[4500]
22:05:03 ipsec,debug KA: 192.168.178.2[4500]->92.218.169.238[4500]
22:05:03 ipsec,debug 1 times of 1 bytes message will be sent to 92.218.169.238[4500]
....

What about RouterOS 6.47.8? (does not support IKEV2_FRAGMENTATION_SUPPORTED)
It plays nicely with Windows 10:

15:43:36 ipsec,debug ===== received 416 bytes from 89.1.175.13[61155] to 192.168.178.3[500] 
15:43:36 ipsec -> ike2 request, exchange: SA_INIT:0 89.1.175.13[61155] 83537d7b19796f4b:0000000000000000 
15:43:36 ipsec ike2 respond 
15:43:36 ipsec payload seen: SA (48 bytes) 
15:43:36 ipsec payload seen: KE (136 bytes) 
15:43:36 ipsec payload seen: NONCE (52 bytes) 
15:43:36 ipsec payload seen: NOTIFY (8 bytes) 
15:43:36 ipsec payload seen: NOTIFY (28 bytes) 
15:43:36 ipsec payload seen: NOTIFY (28 bytes) 
15:43:36 ipsec payload seen: VID (24 bytes) 
15:43:36 ipsec,debug 1e2b516905991c7d7c96fcbfb587e46100000009 
15:43:36 ipsec payload seen: VID (20 bytes) 
15:43:36 ipsec,debug fb1de3cdf341b7ea16b7e5be0855f120 
15:43:36 ipsec payload seen: VID (20 bytes) 
15:43:36 ipsec,debug 26244d38eddb61b3172a36e3d0cfb819 
15:43:36 ipsec payload seen: VID (24 bytes) 
15:43:36 ipsec,debug 01528bbbc00696121849ab9a1c5b2a5100000002 
15:43:36 ipsec processing payload: NONCE 
15:43:36 ipsec processing payload: SA 
15:43:36 ipsec IKE Protocol: IKE 
15:43:36 ipsec  proposal #1 
15:43:36 ipsec   enc: aes256-cbc 
15:43:36 ipsec   prf: hmac-sha256 
15:43:36 ipsec   auth: sha256 
15:43:36 ipsec   dh: modp1024 
15:43:36 ipsec matched proposal: 
15:43:36 ipsec  proposal #1 
15:43:36 ipsec   enc: aes256-cbc 
15:43:36 ipsec   prf: hmac-sha256 
15:43:36 ipsec   auth: sha256 
15:43:36 ipsec   dh: modp1024 
15:43:36 ipsec processing payload: KE 
15:43:36 ipsec,debug => shared secret (size 0x80) 
15:43:36 ipsec,debug fccf5856 459b96fb 4797ef3e 8bf19677 01adb24a e561073a 6d7ab22a af09df6b 
....
15:43:36 ipsec adding payload: SA 
15:43:36 ipsec,debug => (size 0x30) 
15:43:36 ipsec,debug 00000030 0000002c 01010004 0300000c 0100000c 800e0100 03000008 02000005 
15:43:36 ipsec,debug 03000008 0300000c 00000008 04000002 
15:43:36 ipsec adding payload: KE 
15:43:36 ipsec,debug => (size 0x88) 
15:43:36 ipsec,debug 00000088 00020000 31549f63 d9dd7561 71fbe128 79163a4b 0c1b50ce 7abea09a 
....
15:43:36 ipsec,debug 84467320 5ceae650 
15:43:36 ipsec adding payload: NONCE 
15:43:36 ipsec,debug => (size 0x1c) 
15:43:36 ipsec,debug 0000001c a6f931ed db6c9502 aa239977 ca4c202d 3a3b852e b8fb1252 
15:43:36 ipsec adding notify: NAT_DETECTION_SOURCE_IP 
15:43:36 ipsec,debug => (size 0x1c) 
15:43:36 ipsec,debug 0000001c 00004004 eaac1121 3800363d 26dccd35 486f3f34 01599949 
15:43:36 ipsec adding notify: NAT_DETECTION_DESTINATION_IP 
15:43:36 ipsec,debug => (size 0x1c) 
15:43:36 ipsec,debug 0000001c 00004005 7b215569 b5fc2583 33839f2a bbca7514 e5d613eb 
15:43:36 ipsec adding payload: CERTREQ 
15:43:36 ipsec,debug => (size 0x5) 
15:43:36 ipsec,debug 00000005 04 
15:43:36 ipsec <- ike2 reply, exchange: SA_INIT:0 89.1.175.13[61155] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 301 bytes from 192.168.178.3[500] to 89.1.175.13[61155] 
15:43:36 ipsec,debug 1 times of 301 bytes message will be sent to 89.1.175.13[61155] 
15:43:36 ipsec,debug => skeyseed (size 0x20) 
15:43:36 ipsec,debug d22e3f74 ccfe9600 badc7cec 4bf8e56d a656561b 2a1b4c89 8aed5aff 76508741 
15:43:36 ipsec,debug => keymat (size 0x20) 
15:43:36 ipsec,debug 2750a0f1 22c4b3b0 5e2e735f b7fd8695 8f04e9f2 c44c56fd 031b167a 4937450d 
15:43:36 ipsec,debug => SK_ai (size 0x20) 
15:43:36 ipsec,debug c750b975 65b188f2 31f277ac b732c672 473c9d26 0877eac5 3becfd9b a733d96f 
15:43:36 ipsec,debug => SK_ar (size 0x20) 
15:43:36 ipsec,debug eb152a64 ae927d46 d4ce54e7 5ac38d20 32521893 7eb70591 a5b06118 1aa942d8 
15:43:36 ipsec,debug => SK_ei (size 0x20) 
15:43:36 ipsec,debug 039dd29f 370a4ec5 4c20695f 4c0b573b cd3a121d 4a24c696 82a3101b cf041a19 
15:43:36 ipsec,debug => SK_er (size 0x20) 
15:43:36 ipsec,debug afe9bed8 defb1d4d f3072d05 6af3fd02 ac6d1b09 192fde62 e266bd17 6021e5a4 
15:43:36 ipsec,debug => SK_pi (size 0x20) 
15:43:36 ipsec,debug 1d5d5da3 7626d4df a53d6db2 2b4d9c0c b3ea61fa 71ca4a8d 36b89879 031a7809 
15:43:36 ipsec,debug => SK_pr (size 0x20) 
15:43:36 ipsec,debug 24fb3220 68c34ca9 75edf830 da97f3bd f2d5adb0 19d61e4c 72c3ba96 39c6faf4 
15:43:36 ipsec,info new ike2 SA (R): 192.168.178.3[500]-89.1.175.13[61155] spi:d2793fca2b643b58:83537d7b19796f4b 
15:43:36 ipsec processing payloads: VID 
15:43:36 ipsec peer is MS Windows (ISAKMPOAKLEY 9) 
15:43:36 ipsec processing payloads: NOTIFY 
15:43:36 ipsec   notify: IKEV2_FRAGMENTATION_SUPPORTED 
15:43:36 ipsec   notify: NAT_DETECTION_SOURCE_IP 
15:43:36 ipsec   notify: NAT_DETECTION_DESTINATION_IP 
15:43:36 ipsec (NAT-T) REMOTE LOCAL 
15:43:36 ipsec KA list add: 192.168.178.3[4500]->89.1.175.13[61155] 
15:43:36 ipsec,debug ===== received 10208 bytes from 89.1.175.13[61156] to 192.168.178.3[4500] 
15:43:36 ipsec -> ike2 request, exchange: AUTH:1 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec peer ports changed: 61155 -> 61156 
15:43:36 ipsec KA remove: 192.168.178.3[4500]->89.1.175.13[61155] 
15:43:36 ipsec,debug KA tree dump: 192.168.178.3[4500]->89.1.175.13[61155] (in_use=1) 
15:43:36 ipsec,debug KA removing this one... 
15:43:36 ipsec KA list add: 192.168.178.3[4500]->89.1.175.13[61156] 
15:43:36 ipsec payload seen: ENC (10180 bytes) 
15:43:36 ipsec processing payload: ENC 
15:43:36 ipsec,debug => iv (size 0x10) 
15:43:36 ipsec,debug 28dd69e2 50c75f22 c46c67fd 1f4312c1 
15:43:36 ipsec,debug => plain payload (trimmed) (first 0x100 of 0x279d) 
15:43:36 ipsec,debug 2600000c 01000000 c0a8b2c5 290026c5 04121a24 b7518bf7 9ce2597b 396d2143 
....
15:43:36 ipsec,debug decrypted 
15:43:36 ipsec payload seen: ID_I (12 bytes) 
15:43:36 ipsec payload seen: CERTREQ (9925 bytes) 
15:43:36 ipsec payload seen: NOTIFY (8 bytes) 
15:43:36 ipsec payload seen: CONFIG (24 bytes) 
15:43:36 ipsec payload seen: SA (44 bytes) 
15:43:36 ipsec payload seen: TS_I (64 bytes) 
15:43:36 ipsec payload seen: TS_R (64 bytes) 
15:43:36 ipsec processing payloads: NOTIFY 
15:43:36 ipsec   notify: MOBIKE_SUPPORTED 
15:43:36 ipsec ike auth: respond 
15:43:36 ipsec processing payload: ID_I 
15:43:36 ipsec ID_I (ADDR4): 192.168.178.197 
15:43:36 ipsec processing payload: ID_R (not found) 
15:43:36 ipsec processing payload: AUTH (not found) 
15:43:36 ipsec processing payloads: NOTIFY 
15:43:36 ipsec   notify: MOBIKE_SUPPORTED 
15:43:36 ipsec ID_R (FQDN): xxxxxxx.xxxxx.xx
15:43:36 ipsec adding payload: ID_R 
15:43:36 ipsec,debug => (size 0x18) 
15:43:36 ipsec,debug 00000018 02000000 70656472 6f66622e 7370646e 732e6575 
15:43:36 ipsec cert: CN=xxxxxxx.xxxxx.xx.eu,C=DE,ST=NRW,L=,O=Home,OU=,SN= 
15:43:36 ipsec adding payload: CERT 
15:43:36 ipsec,debug => (first 0x100 of 0x3ac) 
15:43:36 ipsec,debug 000003ac 04308203 a3308202 8ba00302 01020210 7a76d202 e82291a9 4d989fb2 
....
15:43:36 ipsec processing payload: NONCE 
15:43:36 ipsec,debug => auth nonce (size 0x30) 
15:43:36 ipsec,debug ab05427f 0c8fbeb2 decd96d6 3da2b3ec f6b9841d c847cddc 3bf9f056 c6847a6b 
15:43:36 ipsec,debug 5570c140 c5fb833d fe6bf318 a019bfd4 
15:43:36 ipsec,debug => SK_p (size 0x20) 
15:43:36 ipsec,debug 24fb3220 68c34ca9 75edf830 da97f3bd f2d5adb0 19d61e4c 72c3ba96 39c6faf4 
15:43:36 ipsec,debug => idhash (size 0x20) 
15:43:36 ipsec,debug 79788e54 7ba9e0ca 99e3ddf7 504c9d47 ea36e95c 37e23a97 8ca5a5ff 19114741 
15:43:36 ipsec,debug => my auth (size 0x100) 
15:43:36 ipsec,debug 942d7114 54a2c241 bc5c4423 5f11553a a0f24776 892ccba5 997aad16 fc5a0afd 
.... 
15:43:36 ipsec adding payload: AUTH 
15:43:36 ipsec,debug => (first 0x100 of 0x108) 
15:43:36 ipsec,debug 00000108 01000000 942d7114 54a2c241 bc5c4423 5f11553a a0f24776 892ccba5 
.... 
15:43:36 ipsec adding payload: EAP 
15:43:36 ipsec,debug => (size 0x9) 
15:43:36 ipsec,debug 00000009 01000005 01 
15:43:36 ipsec <- ike2 reply, exchange: AUTH:1 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 1520 bytes from 192.168.178.3[4500] to 89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 1524 bytes message will be sent to 89.1.175.13[61156] 
15:43:36 ipsec,debug ===== received 80 bytes from 89.1.175.13[61156] to 192.168.178.3[4500] 
15:43:36 ipsec -> ike2 request, exchange: AUTH:2 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec payload seen: ENC (52 bytes) 
15:43:36 ipsec processing payload: ENC 
15:43:36 ipsec,debug => iv (size 0x10) 
15:43:36 ipsec,debug 0c84d24c 8dc179f8 00253e3a 078ffb7b 
15:43:36 ipsec,debug => plain payload (trimmed) (size 0xe) 
15:43:36 ipsec,debug 0000000e 0200000a 0161646d 696e 
15:43:36 ipsec,debug decrypted 
15:43:36 ipsec payload seen: EAP (14 bytes) 
15:43:36 ipsec processing payloads: NOTIFY (none found) 
15:43:36 ipsec processing payload: EAP 
15:43:36 ipsec update peer's identity: 192.168.178.197 -> admin 
15:43:36 radius,debug new request 55:41 code=Access-Request service=ipsec called-id=192.168.178.3 
15:43:36 radius,debug sending 55:41 to 192.168.178.2:1812 
15:43:36 radius,debug,packet sending Access-Request with id 27 to 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0xdf0716b589f1a4f880c3267b04a44bb4 
15:43:36 radius,debug,packet     User-Name = "admin" 
15:43:36 radius,debug,packet     Called-Station-Id = "192.168.178.3" 
15:43:36 radius,debug,packet     Calling-Station-Id = "89.1.175.13" 
15:43:36 radius,debug,packet     NAS-Port-Id = 0x0000000a 
15:43:36 radius,debug,packet     NAS-Port-Type = 5 
15:43:36 radius,debug,packet     Service-Type = 2 
15:43:36 radius,debug,packet     Event-Timestamp = 1710945816 
15:43:36 radius,debug,packet     Framed-MTU = 1400 
15:43:36 radius,debug,packet     EAP-Message = 0x0200000a0161646d696e 
15:43:36 radius,debug,packet     Message-Authenticator = 0x99a41153a662788660d61d83f6f38759 
15:43:36 radius,debug,packet     NAS-Identifier = "MikroTik" 
15:43:36 radius,debug,packet     NAS-IP-Address = 192.168.178.3 
15:43:36 radius,debug,packet received Access-Challenge with id 27 from 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0xdd590399c4590ce9a53e035d997fe99c 
15:43:36 radius,debug,packet     EAP-Message = 0x0101001b1a01010016103cfe103150bf 
15:43:36 radius,debug,packet       d2e32cdd36442552b79a20 
15:43:36 radius,debug,packet     State = 0xed3b5c31ccdc32d6b2e5c7801b908a30 
15:43:36 radius,debug,packet     Message-Authenticator = 0x8e0f9d8c8cd864233f38c6cfa95fbd1e 
15:43:36 radius,debug received reply for 55:41 
15:43:36 ipsec adding payload: EAP 
15:43:36 ipsec,debug => (size 0x1f) 
15:43:36 ipsec,debug 0000001f 0101001b 1a010100 16103cfe 103150bf d2e32cdd 36442552 b79a20 
15:43:36 ipsec <- ike2 reply, exchange: AUTH:2 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 128 bytes from 192.168.178.3[4500] to 89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 132 bytes message will be sent to 89.1.175.13[61156] 
15:43:36 ipsec,debug ===== received 144 bytes from 89.1.175.13[61156] to 192.168.178.3[4500] 
15:43:36 ipsec -> ike2 request, exchange: AUTH:3 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec payload seen: ENC (116 bytes) 
15:43:36 ipsec processing payload: ENC 
15:43:36 ipsec,debug => iv (size 0x10) 
15:43:36 ipsec,debug 368dae72 34a1f57e ca0c5ee7 682b8bd2 
15:43:36 ipsec,debug => plain payload (trimmed) (size 0x44) 
15:43:36 ipsec,debug 00000044 02010040 1a020100 3b31e910 4d4030fc 62262036 ca54051b 532b0000 
....
15:43:36 ipsec,debug 646d696e 
15:43:36 ipsec,debug decrypted 
15:43:36 ipsec payload seen: EAP (68 bytes) 
15:43:36 ipsec processing payloads: NOTIFY (none found) 
15:43:36 ipsec processing payload: EAP 
15:43:36 radius,debug new request 55:42 code=Access-Request service=ipsec called-id=192.168.178.3 
15:43:36 radius,debug sending 55:42 to 192.168.178.2:1812 
15:43:36 radius,debug,packet sending Access-Request with id 28 to 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0x7290ad9698232cb00a29392a912e9452 
15:43:36 radius,debug,packet     User-Name = "admin" 
15:43:36 radius,debug,packet     Called-Station-Id = "192.168.178.3" 
15:43:36 radius,debug,packet     Calling-Station-Id = "89.1.175.13" 
15:43:36 radius,debug,packet     NAS-Port-Id = 0x0000000a 
15:43:36 radius,debug,packet     NAS-Port-Type = 5 
15:43:36 radius,debug,packet     Service-Type = 2 
15:43:36 radius,debug,packet     Event-Timestamp = 1710945816 
15:43:36 radius,debug,packet     Framed-MTU = 1400 
15:43:36 radius,debug,packet     State = 0xed3b5c31ccdc32d6b2e5c7801b908a30 
15:43:36 radius,debug,packet     EAP-Message = 0x020100401a0201003b31e9104d4030fc 
15:43:36 radius,debug,packet       62262036ca54051b532b000000000000 
15:43:36 radius,debug,packet       0000a4af1d3d69c31195ff47e7b8d48c 
15:43:36 radius,debug,packet       2dcd9d6a6e86f7b4b6870061646d696e 
15:43:36 radius,debug,packet     Message-Authenticator = 0x328dc4060f481438a2c8810fb308b613 
15:43:36 radius,debug,packet     NAS-Identifier = "MikroTik" 
15:43:36 radius,debug,packet     NAS-IP-Address = 192.168.178.3 
15:43:36 radius,debug,packet received Access-Challenge with id 28 from 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0xdf1a57e9789b1af0a0522703a69e89c8 
15:43:36 radius,debug,packet     EAP-Message = 0x010200331a0301002e533d3138463334 
15:43:36 radius,debug,packet       34344537304635384439383737323245 
15:43:36 radius,debug,packet       46324131333945324134443139463936 
15:43:36 radius,debug,packet       303741 
15:43:36 radius,debug,packet     State = 0xed3b5c31ccdc32d6b2e5c7801b908a30 
15:43:36 radius,debug,packet     Message-Authenticator = 0xc93b0dfa560754af46e195207cfc0096 
15:43:36 radius,debug received reply for 55:42 
15:43:36 ipsec adding payload: EAP 
15:43:36 ipsec,debug => (size 0x37) 
15:43:36 ipsec,debug 00000037 01020033 1a030100 2e533d31 38463334 34344537 30463538 44393837 
15:43:36 ipsec,debug 37323245 46324131 33394532 41344431 39463936 303741 
15:43:36 ipsec <- ike2 reply, exchange: AUTH:3 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 256 bytes from 192.168.178.3[4500] to 89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 260 bytes message will be sent to 89.1.175.13[61156] 
15:43:36 ipsec,debug ===== received 80 bytes from 89.1.175.13[61156] to 192.168.178.3[4500] 
15:43:36 ipsec -> ike2 request, exchange: AUTH:4 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec payload seen: ENC (52 bytes) 
15:43:36 ipsec processing payload: ENC 
15:43:36 ipsec,debug => iv (size 0x10) 
15:43:36 ipsec,debug 088d4831 fe9107a9 6a09a6ca bcfb73dc 
15:43:36 ipsec,debug => plain payload (trimmed) (size 0xa) 
15:43:36 ipsec,debug 0000000a 02020006 1a03 
15:43:36 ipsec,debug decrypted 
15:43:36 ipsec payload seen: EAP (10 bytes) 
15:43:36 ipsec processing payloads: NOTIFY (none found) 
15:43:36 ipsec processing payload: EAP 
15:43:36 radius,debug new request 55:43 code=Access-Request service=ipsec called-id=192.168.178.3 
15:43:36 radius,debug sending 55:43 to 192.168.178.2:1812 
15:43:36 radius,debug,packet sending Access-Request with id 29 to 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0x3bbedf88ed2148156ef154b11bba926e 
15:43:36 radius,debug,packet     User-Name = "admin" 
15:43:36 radius,debug,packet     Called-Station-Id = "192.168.178.3" 
15:43:36 radius,debug,packet     Calling-Station-Id = "89.1.175.13" 
15:43:36 radius,debug,packet     NAS-Port-Id = 0x0000000a 
15:43:36 radius,debug,packet     NAS-Port-Type = 5 
15:43:36 radius,debug,packet     Service-Type = 2 
15:43:36 radius,debug,packet     Event-Timestamp = 1710945816 
15:43:36 radius,debug,packet     Framed-MTU = 1400 
15:43:36 radius,debug,packet     State = 0xed3b5c31ccdc32d6b2e5c7801b908a30 
15:43:36 radius,debug,packet     EAP-Message = 0x020200061a03 
15:43:36 radius,debug,packet     Message-Authenticator = 0x3ca52bf120318c6643a0e45cbb9b04b9 
15:43:36 radius,debug,packet     NAS-Identifier = "MikroTik" 
15:43:36 radius,debug,packet     NAS-IP-Address = 192.168.178.3 
15:43:36 radius,debug,packet received Access-Accept with id 29 from 192.168.178.2:1812 
15:43:36 radius,debug,packet     Signature = 0x14560b54e470efa876b03c29d7b41f03 
15:43:36 radius,debug,packet     EAP-Message = 0x03020004 
15:43:36 radius,debug,packet     MS-MPPE-Recv-Key = 0xa0e08a7a898fa0cbc4a689a757c00067 
15:43:36 radius,debug,packet       db5c87c71f86d6d8132d16abe4ac82c7 
15:43:36 radius,debug,packet       4aeb 
15:43:36 radius,debug,packet     MS-MPPE-Send-Key = 0xc02bc16a210705d53b209143f0f45c4f 
15:43:36 radius,debug,packet       e32c727fbe30dd280b6e3a7809876d4c 
15:43:36 radius,debug,packet       c147 
15:43:36 radius,debug,packet     Class = 0xc782769ce84e504f 
15:43:36 radius,debug,packet     Message-Authenticator = 0x7d66467dc3273b4d2759f63a179bdfc0 
15:43:36 radius,debug received reply for 55:43 
15:43:36 ipsec,debug => EAP MSK (size 0x20) 
15:43:36 ipsec,debug cfcbde44 44148e41 4d45abef b02b5f7b 8ce9bb75 202411be fd32a395 60101fde 
15:43:36 ipsec adding payload: EAP 
15:43:36 ipsec,debug => (size 0x8) 
15:43:36 ipsec,debug 00000008 03020004 
15:43:36 ipsec <- ike2 reply, exchange: AUTH:4 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 272 bytes from 192.168.178.3[4500] to 89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 276 bytes message will be sent to 89.1.175.13[61156] 
15:43:36 ipsec,debug ===== received 112 bytes from 89.1.175.13[61156] to 192.168.178.3[4500] 
15:43:36 ipsec -> ike2 request, exchange: AUTH:5 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec payload seen: ENC (84 bytes) 
15:43:36 ipsec processing payload: ENC 
15:43:36 ipsec,debug => iv (size 0x10) 
15:43:36 ipsec,debug 6682a308 a77dbe4f 4f1f16cf 3872d434 
15:43:36 ipsec,debug => plain payload (trimmed) (size 0x28) 
15:43:36 ipsec,debug 00000028 02000000 5fbce3ed 3fa0d8c6 b9c0ed59 cecb2c37 085669c1 6b57200a 
15:43:36 ipsec,debug 18b6f38a 2251fd82 
15:43:36 ipsec,debug decrypted 
15:43:36 ipsec payload seen: AUTH (40 bytes) 
15:43:36 ipsec processing payloads: NOTIFY (none found) 
15:43:36 ipsec processing payload: AUTH 
15:43:36 ipsec requested auth method: SKEY 
15:43:36 ipsec,debug => peer's auth (size 0x20) 
15:43:36 ipsec,debug 5fbce3ed 3fa0d8c6 b9c0ed59 cecb2c37 085669c1 6b57200a 18b6f38a 2251fd82 
15:43:36 ipsec,debug => auth nonce (size 0x18) 
15:43:36 ipsec,debug a6f931ed db6c9502 aa239977 ca4c202d 3a3b852e b8fb1252 
15:43:36 ipsec,debug => SK_p (size 0x20) 
15:43:36 ipsec,debug 1d5d5da3 7626d4df a53d6db2 2b4d9c0c b3ea61fa 71ca4a8d 36b89879 031a7809 
15:43:36 ipsec,debug => idhash (size 0x20) 
15:43:36 ipsec,debug 56563c47 8425dee8 4cc5b536 e89e3928 48e096ee e30eaba3 6987887b 458188df 
15:43:36 ipsec,debug => calculated peer's AUTH (size 0x20) 
15:43:36 ipsec,debug 5fbce3ed 3fa0d8c6 b9c0ed59 cecb2c37 085669c1 6b57200a 18b6f38a 2251fd82 
15:43:36 ipsec,info,account peer authorized: 192.168.178.3[4500]-89.1.175.13[61156] spi:d2793fca2b643b58:83537d7b19796f4b 
15:43:36 ipsec processing payloads: NOTIFY 
15:43:36 ipsec   notify: MOBIKE_SUPPORTED 
15:43:36 ipsec peer wants tunnel mode 
15:43:36 ipsec processing payload: CONFIG 
15:43:36 ipsec   attribute: internal IPv4 address 
15:43:36 ipsec   attribute: internal IPv4 DNS 
15:43:36 ipsec   attribute: internal IPv4 NBNS 
15:43:36 ipsec   attribute: MS internal IPv4 server 
15:43:36 ipsec,info acquired 192.168.9.254 address for 89.1.175.13, admin 
15:43:36 ipsec processing payload: TS_I 
15:43:36 ipsec 0.0.0.0/0 
15:43:36 ipsec [::/0] 
15:43:36 ipsec processing payload: TS_R 
15:43:36 ipsec 0.0.0.0/0 
15:43:36 ipsec [::/0] 
15:43:36 ipsec TSi in tunnel mode replaced with config address: 192.168.9.254 
15:43:36 ipsec canditate selectors: 0.0.0.0/0 <=> 192.168.9.254 
15:43:36 ipsec canditate selectors: [::/0] <=> [::/0] 
15:43:36 ipsec processing payload: SA 
15:43:36 ipsec IKE Protocol: ESP 
15:43:36 ipsec  proposal #1 
15:43:36 ipsec   enc: aes256-cbc 
15:43:36 ipsec   auth: sha256 
15:43:36 ipsec searching for policy for selector: 0.0.0.0/0 <=> 192.168.9.254 
15:43:36 ipsec generating policy 
15:43:36 ipsec matched proposal: 
15:43:36 ipsec  proposal #1 
15:43:36 ipsec   enc: aes256-cbc 
15:43:36 ipsec   auth: sha256 
15:43:36 ipsec ike auth: finish 
15:43:36 ipsec ID_R (FQDN): xxxxxxx.xxxxx.xx
15:43:36 ipsec processing payload: NONCE 
15:43:36 ipsec,debug => auth nonce (size 0x30) 
15:43:36 ipsec,debug ab05427f 0c8fbeb2 decd96d6 3da2b3ec f6b9841d c847cddc 3bf9f056 c6847a6b 
15:43:36 ipsec,debug 5570c140 c5fb833d fe6bf318 a019bfd4 
15:43:36 ipsec,debug => SK_p (size 0x20) 
15:43:36 ipsec,debug 24fb3220 68c34ca9 75edf830 da97f3bd f2d5adb0 19d61e4c 72c3ba96 39c6faf4 
15:43:36 ipsec,debug => idhash (size 0x20) 
15:43:36 ipsec,debug 79788e54 7ba9e0ca 99e3ddf7 504c9d47 ea36e95c 37e23a97 8ca5a5ff 19114741 
15:43:36 ipsec,debug => my auth (size 0x20) 
15:43:36 ipsec,debug 0d5903a7 455633b1 6368e02a 447fab19 e2ce3dd7 03f4097f 666b893a d07b51ca 
15:43:36 ipsec cert: CN=xxxxxxx.xxxxx.xx,C=DE,ST=NRW,L=,O=Home,OU=,SN= 
15:43:36 ipsec adding payload: CERT 
15:43:36 ipsec,debug => (first 0x100 of 0x3ac) 
15:43:36 ipsec,debug 000003ac 04308203 a3308202 8ba00302 01020210 7a76d202 e82291a9 4d989fb2 
....
15:43:36 ipsec adding payload: ID_R 
15:43:36 ipsec,debug => (size 0x18) 
15:43:36 ipsec,debug 00000018 02000000 70656472 6f66622e 7370646e 732e6575 
15:43:36 ipsec adding payload: AUTH 
15:43:36 ipsec,debug => (size 0x28) 
15:43:36 ipsec,debug 00000028 02000000 0d5903a7 455633b1 6368e02a 447fab19 e2ce3dd7 03f4097f 
15:43:36 ipsec,debug 666b893a d07b51ca 
15:43:36 ipsec adding notify: INITIAL_CONTACT 
15:43:36 ipsec,debug => (size 0x8) 
15:43:36 ipsec,debug 00000008 00004000 
15:43:36 ipsec preparing internal IPv4 address 
15:43:36 ipsec preparing internal IPv4 netmask 
15:43:36 ipsec preparing internal IPv6 subnet 
15:43:36 ipsec preparing internal IPv4 DNS 
15:43:36 ipsec adding payload: CONFIG 
15:43:36 ipsec,debug => (size 0x2c) 
15:43:36 ipsec,debug 0000002c 02000000 00010004 c0a809fe 00020004 ffffffff 000d0008 00000000 
15:43:36 ipsec,debug 00000000 00030004 c0a80901 
15:43:36 ipsec initiator selector: 192.168.9.254 
15:43:36 ipsec adding payload: TS_I 
15:43:36 ipsec,debug => (size 0x18) 
15:43:36 ipsec,debug 00000018 01000000 07000010 0000ffff c0a809fe c0a809fe 
15:43:36 ipsec responder selector: 0.0.0.0/0 
15:43:36 ipsec adding payload: TS_R 
15:43:36 ipsec,debug => (size 0x18) 
15:43:36 ipsec,debug 00000018 01000000 07000010 0000ffff 00000000 ffffffff 
15:43:36 ipsec adding payload: SA 
15:43:36 ipsec,debug => (size 0x2c) 
15:43:36 ipsec,debug 0000002c 00000028 01030403 079f8ec2 0300000c 0100000c 800e0100 03000008 
15:43:36 ipsec,debug 0300000c 00000008 05000000 
15:43:36 ipsec <- ike2 reply, exchange: AUTH:5 89.1.175.13[61156] 83537d7b19796f4b:d2793fca2b643b58 
15:43:36 ipsec,debug ===== sending 1360 bytes from 192.168.178.3[4500] to 89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 1364 bytes message will be sent to 89.1.175.13[61156] 
15:43:36 ipsec,debug => child keymat (size 0x80) 
15:43:36 ipsec,debug bd44af42 53740c4b 0615b4f1 82a1516b 19db63ee a8f18c68 8a9c45be 3a3c4e56 
.... 
15:43:36 ipsec IPsec-SA established: 89.1.175.13[61156]->192.168.178.3[4500] spi=0x79f8ec2 
15:43:36 ipsec IPsec-SA established: 192.168.178.3[4500]->89.1.175.13[61156] spi=0xe631bbce 
15:43:36 ipsec,debug recv DHCP inform from 192.168.9.254 
15:43:36 ipsec,debug sending DHCP reply 
15:43:36 ipsec,debug 1 times of 300 bytes message will be sent to 192.168.9.254[68] 
15:43:36 ipsec,debug KA: 192.168.178.3[4500]->89.1.175.13[61156] 
15:43:36 ipsec,debug 1 times of 1 bytes message will be sent to 89.1.175.13[61156] 
....

Next, i updated RouterOS to version 6.49.13 (IKEV2_FRAGMENTATION_SUPPORTED supported).
It doesn't work, exactly the same behavior as actual ROS. Here the log:

16:30:59 ipsec -> ike2 request, exchange: SA_INIT:0 89.1.175.13[61157] e832f37d5a1a84a9:0000000000000000 
16:30:59 ipsec ike2 respond 
16:30:59 ipsec payload seen: SA (48 bytes) 
16:30:59 ipsec payload seen: KE (136 bytes) 
16:30:59 ipsec payload seen: NONCE (52 bytes) 
16:30:59 ipsec payload seen: NOTIFY (8 bytes) 
16:30:59 ipsec payload seen: NOTIFY (28 bytes) 
16:30:59 ipsec payload seen: NOTIFY (28 bytes) 
16:30:59 ipsec payload seen: VID (24 bytes) 
16:30:59 ipsec,debug 1e2b516905991c7d7c96fcbfb587e46100000009 
16:30:59 ipsec payload seen: VID (20 bytes) 
16:30:59 ipsec,debug fb1de3cdf341b7ea16b7e5be0855f120 
16:30:59 ipsec payload seen: VID (20 bytes) 
16:30:59 ipsec,debug 26244d38eddb61b3172a36e3d0cfb819 
16:30:59 ipsec payload seen: VID (24 bytes) 
16:30:59 ipsec,debug 01528bbbc00696121849ab9a1c5b2a5100000002 
16:30:59 ipsec processing payload: NONCE 
16:30:59 ipsec processing payload: SA 
16:30:59 ipsec IKE Protocol: IKE 
16:30:59 ipsec  proposal #1 
16:30:59 ipsec   enc: aes256-cbc 
16:30:59 ipsec   prf: hmac-sha256 
16:30:59 ipsec   auth: sha256 
16:30:59 ipsec   dh: modp1024 
16:30:59 ipsec matched proposal: 
16:30:59 ipsec  proposal #1 
16:30:59 ipsec   enc: aes256-cbc 
16:30:59 ipsec   prf: hmac-sha256 
16:30:59 ipsec   auth: sha256 
16:30:59 ipsec   dh: modp1024 
16:30:59 ipsec processing payload: KE 
16:30:59 ipsec,debug => shared secret (size 0x80) 
16:30:59 ipsec,debug 620660f4 d5b45fcf e620ce7d acdec84b 226e7127 e31385fa 0bff5a6b b499cab7 
....
16:30:59 ipsec adding payload: SA 
16:30:59 ipsec,debug => (size 0x30) 
16:30:59 ipsec,debug 00000030 0000002c 01010004 0300000c 0100000c 800e0100 03000008 02000005 
16:30:59 ipsec,debug 03000008 0300000c 00000008 04000002 
16:30:59 ipsec adding payload: KE 
16:30:59 ipsec,debug => (size 0x88) 
16:30:59 ipsec,debug 00000088 00020000 b2cb043c 78648ad8 cb3b5408 13992ea0 a4303b53 e6b3090a 
....
16:30:59 ipsec,debug 4d9d4b70 a6f8f7dc 
16:30:59 ipsec adding payload: NONCE 
16:30:59 ipsec,debug => (size 0x1c) 
16:30:59 ipsec,debug 0000001c fe55002c 02c86d21 49b3d595 0e8e0c9c aef9f35e 79808ca3 
16:30:59 ipsec adding notify: NAT_DETECTION_SOURCE_IP 
16:30:59 ipsec,debug => (size 0x1c) 
16:30:59 ipsec,debug 0000001c 00004004 7baeb0ca 3e017b0e 1a076a25 52f96443 8189c9e8 
16:30:59 ipsec adding notify: NAT_DETECTION_DESTINATION_IP 
16:30:59 ipsec,debug => (size 0x1c) 
16:30:59 ipsec,debug 0000001c 00004005 7639864c 3fc2990f d6450142 62b314d8 bac1aaa4 
16:30:59 ipsec adding notify: IKEV2_FRAGMENTATION_SUPPORTED 
16:30:59 ipsec,debug => (size 0x8) 
16:30:59 ipsec,debug 00000008 0000402e 
16:30:59 ipsec adding payload: CERTREQ 
16:30:59 ipsec,debug => (size 0x5) 
16:30:59 ipsec,debug 00000005 04 
16:30:59 ipsec <- ike2 reply, exchange: SA_INIT:0 89.1.175.13[61157] e832f37d5a1a84a9:a8d753933f764b81 
16:30:59 ipsec,debug ===== sending 309 bytes from 192.168.178.3[500] to 89.1.175.13[61157] 
16:30:59 ipsec,debug 1 times of 309 bytes message will be sent to 89.1.175.13[61157] 
16:30:59 ipsec,debug => skeyseed (size 0x20) 
16:30:59 ipsec,debug 77829c02 abc9f543 7d5f6aab 9883de37 c3e1d14b fa487175 e6e8235c bd6bee92 
16:30:59 ipsec,debug => keymat (size 0x20) 
16:30:59 ipsec,debug 9d7d8d28 70f1e50b 4289e28c 9aebf425 2f1a2619 892ac0d9 93990cd9 429230d1 
16:30:59 ipsec,debug => SK_ai (size 0x20) 
16:30:59 ipsec,debug cee1ca6d cffa6e61 5bb682dd 6c51ba2a c571ba28 0a289619 a224e847 57dac787 
16:30:59 ipsec,debug => SK_ar (size 0x20) 
16:30:59 ipsec,debug 818001bb 402fb660 811d04f5 cc7fcf09 4afc6483 e56b8d6d 5f9bd748 fadeb21c 
16:30:59 ipsec,debug => SK_ei (size 0x20) 
16:30:59 ipsec,debug d186c701 74f05689 56ad7798 fac87f4b 1ce7f1ce d54fed03 8e1c98db 5ecb3d7e 
16:30:59 ipsec,debug => SK_er (size 0x20) 
16:30:59 ipsec,debug d0b8fa6c 6fc67c05 6b5b527c b2b771db ce5070bd f4c9cfda c335eccd 949e8fd3 
16:30:59 ipsec,debug => SK_pi (size 0x20) 
16:30:59 ipsec,debug 1e7e50e3 4d3a9b61 3b59b309 8d7ef298 8ef77013 f602438f 2f1a90f8 2c43a546 
16:30:59 ipsec,debug => SK_pr (size 0x20) 
16:30:59 ipsec,debug da50cd40 90c98787 c3aee0f4 7b615b90 091a3f8c b7b81ba3 ae4de9bc 7c89d0c5 
16:30:59 ipsec,info new ike2 SA (R): peer-ikev2 192.168.178.3[500]-89.1.175.13[61157] spi:a8d753933f764b81:e832f37d5a1a84a9 
16:30:59 ipsec processing payloads: VID 
16:30:59 ipsec peer is MS Windows (ISAKMPOAKLEY 9) 
16:30:59 ipsec processing payloads: NOTIFY 
16:30:59 ipsec   notify: IKEV2_FRAGMENTATION_SUPPORTED 
16:30:59 ipsec   notify: NAT_DETECTION_SOURCE_IP 
16:30:59 ipsec   notify: NAT_DETECTION_DESTINATION_IP 
16:30:59 ipsec (NAT-T) REMOTE LOCAL 
16:30:59 ipsec KA list add: 192.168.178.3[4500]->89.1.175.13[61157] 
16:30:59 ipsec fragmentation negotiated 
16:30:59 ipsec,debug ===== received 580 bytes from 89.1.175.13[61158] to 192.168.178.3[4500] 
16:30:59 ipsec -> ike2 request, exchange: AUTH:1 89.1.175.13[61158] e832f37d5a1a84a9:a8d753933f764b81 
16:30:59 ipsec peer ports changed: 61157 -> 61158 
16:30:59 ipsec KA remove: 192.168.178.3[4500]->89.1.175.13[61157] 
16:30:59 ipsec,debug KA tree dump: 192.168.178.3[4500]->89.1.175.13[61157] (in_use=1) 
16:30:59 ipsec,debug KA removing this one... 
16:30:59 ipsec KA list add: 192.168.178.3[4500]->89.1.175.13[61158] 
16:30:59 ipsec payload seen: SKF (552 bytes) 
16:30:59 ipsec processing payload: ENC (not found) 
16:30:59 ipsec processing payload: SKF 
16:30:59 ipsec => invalid payload (first 0x100 of 0x228) 
16:30:59 ipsec 23000228 00010015 97fbc6d8 10f936cb 6ec07f5e 9dbd06ce 45294b31 5c92c706 
.... 
16:30:59 ipsec reply notify: INVALID_SYNTAX 
16:30:59 ipsec adding notify: INVALID_SYNTAX 
....

I suspect that the problem starts after version 6.47.8 when IKEV2_FRAGMENTATION_SUPPORTED was implemented,
especially since the Windows 10 native client works as axpected with other vendors
and fragmentation enabled on both endpoints.

TosLin · Fri Mar 22, 2024 9:41 am

My R11e-LTE-US keeps on boot loop. I Can't upgrade to this version.

This version creates a weird loopback interface that keeps running but LTE2 never boots up.

Downgrading to 7.13.5 problem solved.

LTE firmware is up to date on both LTE interfaces and RouterBOARD firmware keeps on 7.13.5 on all devices.

Tested on LtAP. I also have an RBM33G to do the tests.

Regards.

I had the same problem, after netinstall I could run 7.14

pe1chl · Fri Mar 22, 2024 11:05 am

You can only indentify different peers before starting parameter negotiation when using exchange mode "agressive" instead of the default "main". That will again be a problem when you want to abide by the rules of several different client OSes.
RouterOS decides which peer configuration to use based on the initiator IP (and its own interface IP) from top to bottom.
The associated P1 configuration(profile) has to match the initiator's proposal.
In this scenario, the P1 configuration is the result of the selection process and not an input parameter.

What I mean is: identify different peers that are all on dynamic addresses (so remote IP has to be 0.0.0.0/0).

Of course when peers are static you can make peer-specific configuration, but when you have road warriors of different OS types, you will have a problem because all of them have to work with the same profile, which may be unacceptable for one of them.
(because OS maintainers arbitrarily "deprecated" certain modes, while others my not support the required modes).

You are correct that local interface IP is another factor in deciding which profile to use, so when you have multiple IP addresses you can often work around this issue.

Having multiple options in the same profile is not always a solution, because e.g. PFS group can have only one value, and also because some OSes simply fail when there are modes in the profile that the do not know about (some bug in the selection mechanism).

All of this is not a RouterOS issue, it is more of an IPsec issue combined with overzealous "security improvements" by OS maintainers.

pedkoschi · Sat Mar 23, 2024 12:36 pm

What I mean is: identify different peers that are all on dynamic addresses (so remote IP has to be 0.0.0.0/0).

Me too

...have to work with the same profile, which may be unacceptable for one of them.

That's the point.
You can define many identical peers, but only the first one will win.

Just allow for multiple identical peer configs and it will give you multiple different profile config candidates.
Then take the "best matching" profile.
Best matching could be the one with the fewest allowed Dh-groups .. or whatever developers prefer.

...and also because some OSes simply fail when there are modes in the profile that the do not know about

Agreed, but having more options and in (infrequent) case of trouble omitting them its better than not having them.
However, taking multiple profiles into account is enough, extending profile config is nice to have if the first would be implemented.
But of course it is possible:

...because e.g. PFS group can have only one value

I'm in doubt (however PFS is phase 2.. but we are talking about phase 1):

Based on my experience ROS's implementation cannot keep up with others in this aspect.

Have a nice weekend!

doneware · Mon Mar 25, 2024 10:49 am

user-manager experienced some strange errors with 7.14.1 after upgrade on a hEX.
VPN users complained that they cannot log in, and i checked the logs:

 09:31:14 ipsec,error radius timeout

then i checked user-manager, and saw this:

[user@router] /user-manager> export 
# 2024-03-25 09:33:23 by RouterOS 7.14.1
# software id = FFFF-EEEE
#
# model = RB750Gr3
# serial number = UWUWUWWE
#error exporting "/user-manager/attribute"
#error exporting "/user-manager/limitation"
#error exporting "/user-manager/profile"
#error exporting "/user-manager/user"
#error exporting "/user-manager/user/group"
#error exporting "/user-manager"
#error exporting "/user-manager/advanced"
#error exporting "/user-manager/profile-limitation"
#error exporting "/user-manager/router"
#error exporting "/user-manager/user-profile"
[user@router] /user-manager> /sys package/print 
Columns: NAME, VERSION, BUILD-TIME, SIZE
# NAME          VERSION  BUILD-TIME           SIZE     
0 routeros      7.14.1   2024-03-08 12:50:23  9.3MiB   
1 user-manager  7.14.1   2024-03-08 12:50:23  372.1KiB 
2 wireless      7.14.1   2024-03-08 12:50:23  1572.1KiB

did not have much time to troubleshoot, i had to reboot it on the spot. the database is on an usb thumb drive with fat32 FS, so it is very likely the issue was related to media access errors. the system recovered and it's now operational. if it occurs again, will investigate deeper.

[user@router] /user-manager>  /disk/print 
Flags: B - BLOCK-DEVICE; M - MOUNTED; p - PARTITION
Columns: SLOT, MODEL, SERIAL, INTERFACE, SIZE, FREE, FS
#     SLOT        MODEL                      SERIAL                    INTERFACE                  SIZE           FREE  FS   
0 B   usb1        Kingston DataTraveler 2.0  00241D8CE51B1F90993F3CEC  USB 2.00 480Mbps  7 757 398 016                      
1 BMp usb1-part1                             @512-7'757'398'016                          7 757 397 504  7 635 136 512  fat32

(not the best FS choice i admit)

regardless of that, there was no clear indication in the logs that the problem is related to the filesystem. it would be nice if user-manager would spit out messages with error severity if it is not able to read its files.

chiem · Tue Mar 26, 2024 3:27 am

DNS doesn't seem stable in this release:

[admin@ccr2116] /ip/dns> export
# 2024-03-25 17:50:06 by RouterOS 7.14.1
# software id = 7WRZ-DFWZ
#
# model = CCR2116-12G-4S+
# serial number = XXXXXXXXXXX
#error exporting "/ip/dns" (timeout)
#error exporting "/ip/dns/static" (timeout)

Wed Mar 27, 2024 3:01 pm

What's new in 7.14.2 (2024-Mar-27 09:48):

*) defconf - do not override default DHCP server lease time;
*) defconf - fixed 5ghz-ax channel width for L11, L22 devices;
*) ethernet - fixed interface disable for CRS326-4C+20G+2Q;
*) ethernet - improved port speed downshift functionality for CRS326-4C+20G+2Q;
*) leds - fixed LEDs for L22 device;
*) lte - fixed firmware upgrade not found issue for Chateau LTE12 (introduced in v7.14.1);
*) ssh - require "policy" user policy when adding public key;
*) timezone - updated timezone information from "tzdata2024a" release;
*) traffic-flow - improved system stability;
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wifi-qcom - added configuration.distance setting to enable operation over multi-kilometer distances (CLI only);

jordanp123 · Wed Mar 27, 2024 3:36 pm

Looks like the VRF issue with main is fixed but now I'm noticing that none of my firewall rules for interfaces in the VRF (not main) are being hit, on 7.14.2, rolling back to 7.13.5 and it works like normal.

infabo · Wed Mar 27, 2024 4:20 pm

jordanp123, thanks for testing the stable release! 👍

DarkNate · Wed Mar 27, 2024 4:37 pm

What's new in 7.14.2 (2024-Mar-27 09:48):

*) wifi-qcom - added configuration.distance setting to enable operation over multi-kilometer distances (CLI only);

I can't seem to find this on the documentation. How does it work? How do we configure it? How are the values mapped or parsed internally?

erlinden · Wed Mar 27, 2024 4:59 pm

I can't seem to find this on the documentation. How does it work? How do we configure it? How are the values mapped or parsed internally?

Your wish...

Maximum link distance in kilometers, needs to be set for long-range outdoor links. The value should reflect the distance to the AP or station that is furthest from the device. Unconfigured value allows usage of 3KM links.

https://help.mikrotik.com/docs/display/ ... properties

DarkNate · Wed Mar 27, 2024 5:18 pm

Your wish...

Maximum link distance in kilometers, needs to be set for long-range outdoor links. The value should reflect the distance to the AP or station that is furthest from the device. Unconfigured value allows usage of 3KM links.
https://help.mikrotik.com/docs/display/ ... properties

So if I set "10" value, does this mean 10KM?

Wed Mar 27, 2024 5:22 pm

Yes, "10" should be set for 10KM link.

anav · Wed Mar 27, 2024 7:13 pm

Thats 6.2 miles for you DN, and for me..... 5.4 nm. :-)

pe1chl · Wed Mar 27, 2024 7:18 pm

They should have used lightseconds instead :-)
Then we can all agree and no internal conversion is required...

mkx · Wed Mar 27, 2024 7:38 pm

But would then it be speed of light in vacuum or in some thick air with large refractive index?

infabo · Wed Mar 27, 2024 7:57 pm

We already agreed: it is called "International System of Units". And metre is one of the base units. So end of discussion. 🤣

Amm0 · Wed Mar 27, 2024 8:11 pm

But would then it be speed of light in vacuum or in some thick air with large refractive index?

Well, in the case of distance= it already a proxy for time. 10km equates to some timing intervals at the end of the day.

Now the constant "indoor" means I don't trust Mikrotik math, or perhaps chaos theory.

mkx · Wed Mar 27, 2024 8:33 pm

@Amm0: exactly, proper setting would be something like propagation-delay-max with integer setting (>=1) and unit of microseconds (and 10km would roughly translate into 33 microseconds). But imagine chaos this would cause among most AP admins.

Constant indoor would translate into 1 microsecond or around 1.5 furlong ;-)

t0mm13b · Wed Mar 27, 2024 8:38 pm

What's new in 7.14.2 (2024-Mar-27 09:48):

*) lte - fixed firmware upgrade not found issue for Chateau LTE12 (introduced in v7.14.1);

As a Chateau LTE 12 owner, I cannot even get past v7.14, let alone, v7.14.1 or further, as no storage, so how is that going to work.

pe1chl · Wed Mar 27, 2024 9:13 pm

As a Chateau LTE 12 owner, I cannot even get past v7.14, let alone, v7.14.1 or further, as no storage, so how is that going to work.

That is a different problem... it has been mentioned often enough already.
For now, what you could try is to export and backup the configuration and then netinstall it.
But only do that when you have some experience with this matter, or when you don't mind losing the configuration.
(e.g. because it is close to default)

infabo · Wed Mar 27, 2024 9:39 pm

You can netinstall 7.14.2 your device with "keep configuration". Should give you plenty of 200kb free space again. 😂😂😂😂

Do yourself a favor and downgrade to 7.13.5 when already netinstalling. Last stable version with reasonable free space (around 600kb) for a Chateau LTE12.

t0mm13b · Thu Mar 28, 2024 12:26 am

You can netinstall 7.14.2 your device with "keep configuration". Should give you plenty of 200kb free space again. 😂😂😂😂

Do yourself a favor and downgrade to 7.13.5 when already netinstalling. Last stable version with reasonable free space (around 600kb) for a Chateau LTE12.

I shall do a happy dance when doing a downgrade to reclaim 200Kb free space! 😂

Archous · Thu Mar 28, 2024 7:54 am

What's new in 7.14.2 (2024-Mar-27 09:48):

*) defconf - do not override default DHCP server lease time;
*) defconf - fixed 5ghz-ax channel width for L11, L22 devices;
*) ethernet - fixed interface disable for CRS326-4C+20G+2Q;
*) ethernet - improved port speed downshift functionality for CRS326-4C+20G+2Q;
*) leds - fixed LEDs for L22 device;
*) lte - fixed firmware upgrade not found issue for Chateau LTE12 (introduced in v7.14.1);
*) ssh - require "policy" user policy when adding public key;
*) timezone - updated timezone information from "tzdata2024a" release;
*) traffic-flow - improved system stability;
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wifi-qcom - added configuration.distance setting to enable operation over multi-kilometer distances (CLI only);

Third times the charm? Or is it? :|
Already seeing VRF devices come back that have had our management VRF down since 7.14.

konradnh · Thu Mar 28, 2024 10:33 am

After upgrading to v7.14 I lost connectivity between OpenVPN server and clients in Ethernet mode. The connection is established normally and automatic routes are created, but still peers are inaccessible.
Anyone experiencing similar behaviour?

Edit: It's the same with v7.15beta4. Reverting back to v.7.13.5 solves the issue for me.
Still the same with v7.14.1...

Edit:
OK, switching the STP of the bridge from RSTP to none makes the peers reachable...

+1

Two RB3011 same issue on 7.14.2

Thu Mar 28, 2024 11:01 am

Hi, konradnh and 3dfx.

Regarding the issues that's related to bridge R/M/STP. Could you please create a supout.rif file and send it to our support team? Additionally, provide some extra information such as which ports are experiencing the issue, what devices are connected to these ports (e.g. hosts, other bridges, switches).

Check the status of the bridge ports and the ports of neighboring bridges/switches to ensure they are in the "forwarding" state:

/interface/bridge/port monitor [find]

Remember, simply monitoring bridge ports locally might not reveal the issue, as traffic could be blocked by neighboring switches. If a bridge port is connected to a network beyond your control (e.g. a port is part of a WAN), consider configuring "edge=yes" for those ports.

Let us know the SUP number when you send these details.

RouterOS version 7.14 included some updates related to R/M/STP, aimed at making the protocol more compliant with the standard. The problem you're encountering now might indicate that previous versions didn't function as intended.

Thank you.

Smoerrebroed · Thu Mar 28, 2024 12:58 pm

Installation went well without issues, but an issue that seems related to DFS probing on AX devices which has been present since 7.14 still persists. So I keep 5 GHz WiFi turned off on those devices that keep locking up. :-/

Ullinator · Thu Mar 28, 2024 1:03 pm

Installation went well without issues, but an issue that seems related to DFS probing on AX devices which has been present since 7.14 still persists. So I keep 5 GHz WiFi turned off on those devices that keep locking up. :-/

What kind of issue?
I personally run 6 CAP AX with 7.14.2 without any problems.

S8T8 · Thu Mar 28, 2024 3:03 pm

Maximum link distance in kilometers, needs to be set for long-range outdoor links. The value should reflect the distance to the AP or station that is furthest from the device. Unconfigured value allows usage of 3KM links.

setting configuration.distance=1 will improve anything for indoor devices?

Thu Mar 28, 2024 3:19 pm

No, there is no reason to set the distance for indoor devices. Distance value should only be adjusted for devices running the wifi-qcom package, not the wifi-qcom-ac, and it should only be set for outdoor links that are above 3KM in length.

Archous · Thu Mar 28, 2024 4:59 pm

What's new in 7.14.2 (2024-Mar-27 09:48):

*) defconf - do not override default DHCP server lease time;
*) defconf - fixed 5ghz-ax channel width for L11, L22 devices;
*) ethernet - fixed interface disable for CRS326-4C+20G+2Q;
*) ethernet - improved port speed downshift functionality for CRS326-4C+20G+2Q;
*) leds - fixed LEDs for L22 device;
*) lte - fixed firmware upgrade not found issue for Chateau LTE12 (introduced in v7.14.1);
*) ssh - require "policy" user policy when adding public key;
*) timezone - updated timezone information from "tzdata2024a" release;
*) traffic-flow - improved system stability;
*) vrf - fixed VRF interfaces being moved to main table after reboot (introduced in v7.14);
*) wifi-qcom - added configuration.distance setting to enable operation over multi-kilometer distances (CLI only);
Third times the charm? Or is it? :|
Already seeing VRF devices come back that have had our management VRF down since 7.14.

Nope. Still broken VRFs for IP services. @Edpa did you test this release with the previously mentioned issues on IP services not working in VRF anymore?

See below:

[jmoore@arch-homerville-pop-487-holly-oob-r1] > /tool/sniffer/quick interface=ether15 port=22
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE  TIME   NUM  DIR  SRC-MAC            DST-MAC            SRC-ADDRESS          DST-ADDRESS             PROTOCOL  SIZE  CPU
ether15    3.8      1  <-   00:0C:29:29:D7:66  DC:2C:6E:8A:96:E5  172.16.30.254:35375  172.16.30.199:22 (ssh)  ip:tcp      66    1
ether15    4.791    2  <-   00:0C:29:29:D7:66  DC:2C:6E:8A:96:E5  172.16.30.254:35375  172.16.30.199:22 (ssh)  ip:tcp      66    1
ether15    6.799    3  <-   00:0C:29:29:D7:66  DC:2C:6E:8A:96:E5  172.16.30.254:35375  172.16.30.199:22 (ssh)  ip:tcp      66    1

[jmoore@arch-homerville-pop-487-holly-oob-r1] > /ip/service/print
Flags: X - DISABLED, I - INVALID
Columns: NAME, PORT, CERTIFICATE, VRF
#   NAME     PORT  CERTIFICATE  VRF
0 X telnet     23               management
1 X ftp        21
2   www        80               management
3   ssh        22               management
4 X www-ssl   443  none         main
5 X api      8728               main
6 X winbox   8291               main
7 X api-ssl  8729  none         main
[jmoore@arch-homerville-pop-487-holly-oob-r1] > /ip/vrf/print
Flags: X - disabled; * - builtin
 0    name="management" interfaces=ether15,wireguard1

 1  * name="main" interfaces=all
[jmoore@arch-homerville-pop-487-holly-oob-r1] > /ip/firewall/filter/print
Flags: X - disabled, I - invalid; D - dynamic
 0    ;;; Allow VPN
      chain=input action=accept in-interface=wireguard1 log=no log-prefix=""

 1    ;;; Allow VPN
      chain=forward action=accept in-interface=wireguard1 log=no log-prefix=""

 2    ;;; Allow BGP
      chain=input action=accept protocol=tcp dst-port=179 log=no log-prefix=""

 3    ;;; Allow Established/Related
      chain=input action=accept connection-state=established,related,untracked log=no log-prefix=""

 4    ;;; Allow Wireguard
      chain=input action=accept protocol=udp in-interface-list=WAN dst-port=13231

 5    ;;; Allow ICMP to WANs
      chain=input action=accept protocol=icmp log=no log-prefix=""

 6    ;;; Allow to management
      chain=input action=accept in-interface=ether15 log=no log-prefix=""

 7    chain=input action=drop
[jmoore@arch-homerville-pop-487-holly-oob-r1] > /system/routerboard/print
       routerboard: yes
             model: CCR2004-16G-2S+
     serial-number: HBJ07H8VK5V
     firmware-type: al64
  factory-firmware: 7.0.4
  current-firmware: 7.14.2
  upgrade-firmware: 7.14.2
[jmoore@arch-homerville-pop-487-holly-oob-r1] >

Archous · Thu Mar 28, 2024 5:06 pm

UGH. Just to follow up from my previous post. It seems the firewall filtering logic with VRFs has also changed. I can now get IP services to respond on my broken router if I add:

/ip/firewall/filter/add in-interface=management action=accept place-before=1

Where "management" is the automatically generated VRF interface for my VRF named "management". This worked before with the following rule:

/ip/firewall/filter/add action=accept chain=input comment="Allow to management" in-interface=ether15

Where "ether15" is a layer 3 interface added to the VRF "management". This needs to be fixed ASAP as backward compatibility with firewall rules for VRF interfaces is now broken.

Archous · Thu Mar 28, 2024 5:08 pm

#notfixed - it's getting ridiculous..

Confirmed. I've spent a few hours to overhaul firewall configuration for the new VRF concept in 7.14.1 where you cannot filter separate interfaces in a VRF and then noticed that at least EoIP tunnel interface routes are not getting into an assigned VRF after reboot. You can recreate the interface and it will work until you reboot again.

If you have firewall rules on INPUT or FORWARD for any interface that are assigned to some non-main VRF you might consider to redesign it. Also if you have any EoIP interfaces in VRF just don't upgrade yet because it is almost certainly going to break.

This confirms the same behavior I am seeing in my previous posts.

jordanp123 · Thu Mar 28, 2024 5:52 pm

UGH. Just to follow up from my previous post. It seems the firewall filtering logic with VRFs has also changed. I can now get IP services to respond on my broken router if I add:
Code: Select all
/ip/firewall/filter/add in-interface=management action=accept place-before=1
Where "management" is the automatically generated VRF interface for my VRF named "management". This worked before with the following rule:
Code: Select all
/ip/firewall/filter/add action=accept chain=input comment="Allow to management" in-interface=ether15
Where "ether15" is a layer 3 interface added to the VRF "management". This needs to be fixed ASAP as backward compatibility with firewall rules for VRF interfaces is now broken.

After reviewing your post and double checked my issues with the VRF firewall rules and I'm seeing exactly the same thing. All of the existing configs/VRF's I have are broken because of this change in the firewall rule config.

sawa · Thu Mar 28, 2024 8:47 pm

The firmware update on the latest stable v7.14.2 behaves interestingly.
If it is already installed and run
/system package update
Then the firmware will be "updated" from v7.14.2 to v7.14.1

No caching proxies (on my network).
This behavior is only in Kazakhstan.

 > sys package update check-for-updates 
            channel: stable
  installed-version: 7.14.2
     latest-version: 7.14.1
             status: New version is available

Milecus · Fri Mar 29, 2024 7:34 am

Most likely related to timezone. UTC+6 changed to UTC+5. But this behavior is still strange.

pe1chl · Fri Mar 29, 2024 11:03 am

I don't think that is related to timezone, the availability of new versions is not determined by time but by fetching a specific URL that returns the "latest version", which is then compared to the installed version. As you see, it may even be lower.
What this indicates is that the server that is checked for updates has not (yet) been updated to the most recent version.
There may be a server in the pool that somehow did not catch the update, or it may be that the provider or some other agency has redirected upgrade.mikrotik.com to their own server and has not upgraded it.

Check what "upgrade.mikrotik.com" resolves to. E.g. here:

host upgrade.mikrotik.com
upgrade.mikrotik.com is an alias for global-balancer-e.mikrotik.com.
global-balancer-e.mikrotik.com has address 159.148.147.251
global-balancer-e.mikrotik.com has IPv6 address 2a02:610:7501:3000::251

Then check if the returned address belongs to MikroTik:

whois 159.148.147.251
inetnum:        159.148.147.226 - 159.148.147.255
netname:        MIKROTIKLSSIA
abuse-c:        AR23365-RIPE
descr:          MIKROTIKLS SIA
country:        LV

(you can also do the same check for "update.mikrotik.com" which is also involved in the process)
When you get a different result, it may be wise to configure different DNS servers than those offered by the ISP, and even to use DoH.

rb9999 · Fri Mar 29, 2024 1:22 pm

bridge vlans in 7.14.1 and 7.14.2 seem a bit broken. i had to revert config on rb951g as devices on particular vlan (one bridge, untagged, pvid set on one port, tagged on uplink) didn't obtain an ip. had to downgrade to 7.13.5 and now it works yet again.

sirbryan · Fri Mar 29, 2024 5:06 pm

7.14.x reverts/restores BGP as-path filter behavior on egress to previous behavior. Why isn't this mentioned in the release notes?

Before and up to 7.7, any outgoing as-path would be filtered based on an incoming AS's number (their ASN was first in the match list). Sometime after 7.7 (not sure which release), your own ASN was added to that list, requiring all as-path filters to be adjusted accordingly.

Now in 7.14, that has been reverted without explanation. All my outgoing filters were broken. The fix was to remove my own ASN from the filter in order for downstream announcements to be advertised to the upstream providers.

sawa · Fri Mar 29, 2024 9:17 pm

Check what "upgrade.mikrotik.com" resolves to. E.g. here:

The problem was in DNS: upgrade.mikrotik.com resolved to 159.148.147.204.
The DNS server change helped.

NetTecture · Sat Mar 30, 2024 12:07 am

I upgraded from 7.14.0 to 7.14.2 and since then I hava a serious problem with the outgoing NAT'd connections. I get plenty of errors that result in long loading times.

My firewall has a drop invalid connection packets as second to last entry and after the upgrade - it was working fine before - i get plenty of
forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,RST), 192.168.88.19:52394->52.210.81.44:443, len 40
or
forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,FIN), 192.168.88.19:52383->52.210.81.44:443, len 40

I tried in single computer use and... did anything change? any timeout that is worth changing back? I have no idea whether this behavior was there first, but - since the update, browsing and other operations are seriously tricky and often websites time out first, then load good on second try.

The router in question is a 5009.In and out are on 1g ports.

thogrin · Sat Mar 30, 2024 9:13 am

sms auro-erase is missing

mkx · Sat Mar 30, 2024 11:29 am

forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,RST), 192.168.88.19:52394->52.210.81.44:443, len 40
or
forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,FIN), 192.168.88.19:52383->52.210.81.44:443, len 40

These two should be benign ... both indicate dropping of packets, belonging to connections which were in dying stage anyway (RST flag means that sending party is breaking the connection, FIN flag means that sending party is acknowledging that the connection is finishing).

So there must be something else going on ...

pe1chl · Sat Mar 30, 2024 11:49 am

I had one hAP ac2 brick itself after the upgrade and resetting it didn't bring it back to life. Fortunately I had a hAP ax3 on-hand spare to swap it with. The other hAP ac2 and lower models I'm managing I've had to pause auto-updates on (and just in time.) Bottom-line in my opinion, the ac2 and lower models will never be upgraded again and I'm considering them EOL after 7.13.x.

This has been discussed many times on the forum already. I advise to go back to 7.12.1 (or 7.12.2) using netinstall and remain on that version until MikroTik have sorted out this mess.

t0mm13b · Sat Mar 30, 2024 12:36 pm

You can netinstall 7.14.2 your device with "keep configuration". Should give you plenty of 200kb free space again. 😂😂😂😂

Do yourself a favor and downgrade to 7.13.5 when already netinstalling. Last stable version with reasonable free space (around 600kb) for a Chateau LTE12.
I shall do a happy dance when doing a downgrade to reclaim 200Kb free space! 😂

Hello netinstall my old friend,
v7.13.5 installed with wifi-qcom-ac annnd 528 KiB free

/happy dance! 😂

Maggiore81 · Sat Mar 30, 2024 2:57 pm

I also think that the 7.13.5 (till now) it is the more stable for us.
I woule love that MT gets this branch for the longterm... maybe we can see 7.13.6 with more bugfixes.

Smoerrebroed · Sat Mar 30, 2024 5:43 pm

What kind of issue?
I personally run 6 CAP AX with 7.14.2 without any problems.

I run five cAP ax, two hAP ax3, and one wAP ac, and one of the cAP ax always locks up after some time (a few hours, usually), making a power cycle necessary. This also happens after netinstall with a fresh config, which is the same across all devices. I even swapped two cAP ax, and it seems that the lock-up is somehow related to where the device is located, probably seeing DFS activitiy or something.

This is worse than with 7.13.x where only the 5 GHz network would stop working and be in somewhat of a nirvana state (until a reboot). Now the complete AP locks up and needs a reboot.

NetTecture · Sat Mar 30, 2024 5:49 pm

forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,RST), 192.168.88.19:52394->52.210.81.44:443, len 40
or
forward: in:bridge out:up-etisalat, connection-state:invalid src-mac 30:9c:23:28:5e:0d, proto TCP (ACK,FIN), 192.168.88.19:52383->52.210.81.44:443, len 40

These two should be benign ... both indicate dropping of packets, belonging to connections which were in dying stage anyway (RST flag means that sending party is breaking the connection, FIN flag means that sending party is acknowledging that the connection is finishing).

So there must be something else going on ...

Well, among other things I just found and fixed the UDP timeout (which is amazing, Mikrotik changing it for new setups but not changing it for existing installations where the user has not changed the default value - talk about breaking systems) which fixed SOME of the issues (RDP it seems).

Now I just read about some MTU trickery broken that "we are not going to fix because we plan to fix it in 7.15" which may well be it.

Given those I am really out of ideas. This is a setup that has been working flawlessly the last years, now I get significant issues down to PING not working over this one router to all machines (which change randomly after hours).

Jotne · Sat Mar 30, 2024 10:25 pm

There seems to be a bug in history of commands that do break script and MT are not able to reproduce.
So can all test the command:

/system/history/print

and see if it gives error. If so maybe send a support file to MT.
More info here:
viewtopic.php?p=1066861#p1066861

woland · Sun Mar 31, 2024 2:18 pm

Hi! Using a single CAP AX here, no CAPSMAN. 2 SSIDs on both radios into different VLANs. Upgraded from 7.12.2 directly to 7.14.2. Which went surprisingly well (previous attempt to 7.14.0 resulted in netinstall to 7.12.2).
Performance is great, now even the farthest corner of my flat is perfectly covered.
A strange, mainly cosmetic issue appeared, band/channel marked as unknown or greyed out (WinBox 3.40):

Screenshot_20240331_130410.png

Also in th CLI>

[admin@kk-ap1] /interface/wifi> print detail 
Flags: M - master; D - dynamic; B - bound; X - disabled, I - inactive, R - running 
 0 M B  default-name="wifi2" name="ap1-kk2g_kk" l2mtu=1560 mac-address=48:A9:8A:BA:1F:8A arp-timeout=auto radio-mac=48:A9:8A:BA:1F:9D configuration=kk-2g 
        configuration.mode=ap 
        security=kk datapath=dp-vl3 channel=2gax steering=kk-steer 

 1   BR name="ap1-kk2g_kk-guest" l2mtu=1560 mac-address=4A:A9:8A:BA:1F:8B arp-timeout=auto master-interface=ap1-kk2g_kk configuration=kk-guest-2g 
        configuration.mode=ap 
        security=kk-guest datapath=dp-vl5 steering=kk-steer 

 2 M B  default-name="wifi1" name="ap1-kk5g_kk" l2mtu=1560 mac-address=48:A9:8A:BA:1F:8C arp-timeout=auto radio-mac=48:A9:8A:BA:1F:9C configuration=kk-5g 
        configuration.mode=ap .tx-power=30 
        security=kk datapath=dp-vl3 channel=5gax steering=kk-steer 

 3   BR name="ap1-kk5g_kk-guest" l2mtu=1560 mac-address=48:A9:8A:BA:1F:8D arp-timeout=auto master-interface=ap1-kk5g_kk configuration=kk-guest-5g 
        configuration.mode=ap 
        security=kk-guest datapath=dp-vl5 channel=5gax 
        channel.band=(unknown) .width=(unknown) 
        steering=kk-steer

Also in Wifi/Radios if I click on the "Provision" button, it then just deletes the matching radio config and disables it.
Yeah, somehow logical, as I did not enter any provisioning rules for my single CAP, but not very user friendly behavior...

Here is my full WIFI config (I did NOT enter "add channel=5gax channel.band="(unknown)" .width="(unknown)", that just magically appeared, BUG?):

/interface wifi channel
add band=5ghz-ax disabled=no name=5gax width=20/40/80mhz
add band=2ghz-ax disabled=no name=2gax width=20mhz
/interface wifi security
add authentication-types=wpa2-psk,wpa3-psk disabled=no encryption=ccmp,gcmp,ccmp-256,gcmp-256 ft=yes name=kk-guest wps=disable
add authentication-types=wpa2-psk,wpa3-psk disabled=no encryption=ccmp,gcmp,ccmp-256,gcmp-256 ft=yes name=kk wps=disable
/interface wifi steering
add disabled=no name=kk-steer neighbor-group=dynamic-kk-6a8706d1 rrm=yes wnm=yes
add disabled=no name=kk-guest-steer neighbor-group=dynamic-kk-guest-f69bd7d7 rrm=yes wnm=yes
/interface wifi
set [ find default-name=wifi2 ] channel=2gax configuration=kk-2g configuration.mode=ap datapath=dp-vl3 disabled=no mac-address=48:A9:8A:BA:1F:8A name=ap1-kk2g_kk security=kk \
    steering=kk-steer
add configuration=kk-guest-2g configuration.mode=ap datapath=dp-vl5 disabled=no mac-address=4A:A9:8A:BA:1F:8B master-interface=ap1-kk2g_kk name=ap1-kk2g_kk-guest security=kk-guest \
    steering=kk-steer
set [ find default-name=wifi1 ] channel=5gax configuration=kk-5g configuration.mode=ap datapath=dp-vl3 disabled=no mac-address=48:A9:8A:BA:1F:8C name=ap1-kk5g_kk security=kk \
    steering=kk-steer
add channel=5gax channel.band="(unknown)" .width="(unknown)" configuration=kk-guest-5g configuration.mode=ap datapath=dp-vl5 disabled=no mac-address=48:A9:8A:BA:1F:8D \
    master-interface=ap1-kk5g_kk name=ap1-kk5g_kk-guest security=kk-guest steering=kk-steer
/interface wifi configuration
add channel=5gax country=Hungary datapath=dp-vl3 disabled=no mode=ap multicast-enhance=enabled name=kk-5g security=kk ssid=kk tx-power=30
add channel=2gax country=Hungary datapath=dp-vl3 disabled=no mode=ap multicast-enhance=enabled name=kk-2g security=kk ssid=kk tx-power=20
add channel=2gax country=Hungary datapath=dp-vl5 disabled=no mode=ap multicast-enhance=enabled name=kk-guest-2g security=kk-guest ssid=kk-guest steering=kk-guest-steer tx-power=20
add channel=5gax country=Hungary datapath=dp-vl5 disabled=no mode=ap multicast-enhance=enabled name=kk-guest-5g security=kk-guest ssid=kk-guest steering=kk-guest-steer tx-power=30
/interface wifi datapath
add bridge=br0 disabled=no name=dp-vl3 vlan-id=3
add bridge=br0 disabled=no name=dp-vl6 vlan-id=6
add bridge=br0 disabled=no name=dp-vl5 vlan-id=5

Sit75 · Mon Apr 01, 2024 4:12 pm

I see 2 wrong things in RouterOS 7.14.2 (and probably in complete 7.x trail): 1) Memory leakage the most probably related to Queue Tree implementation (fq-codel) and 2) Missing packets properly marked in Queue Tree hierarchical QoS. See picture. Registered under SUP-147911.

biomesh · Mon Apr 01, 2024 5:20 pm

Could the "memory leak" be due to 0 disk space available?

mkx · Mon Apr 01, 2024 5:29 pm

Could the "memory leak" be due to 0 disk space available?

It might ... because ROS might be caching writes to flash. AFAIK that's not what linux kernel usually does though.

pe1chl · Mon Apr 01, 2024 5:44 pm

Anyway, when you have 0KB HDD space available your device is a time bomb. It will not survive a reboot.
So now you need to netinstall it, and after that you can try again.

sirbryan · Mon Apr 01, 2024 8:16 pm

Well, among other things I just found and fixed the UDP timeout (which is amazing, Mikrotik changing it for new setups but not changing it for existing installations where the user has not changed the default value - talk about breaking systems) which fixed SOME of the issues (RDP it seems).

Now I just read about some MTU trickery broken that "we are not going to fix because we plan to fix it in 7.15" which may well be it.

Given those I am really out of ideas. This is a setup that has been working flawlessly the last years, now I get significant issues down to PING not working over this one router to all machines (which change randomly after hours).

Mikrotik has "general rule" about not touching existing configs, except during major upgrades where a config update is necessary. Usually, changing connection tracking settings falls under that "not a major upgrade" category.

The default NAT UDP timeout has been 10s for a very long time--years, if not decades. That's for "unacknowledged" connections (i.e. client doesn't respond using the same IP:PORT combination; UDP streams have a 3 minute timeout). Bumping it to 30s keeps the NAT entry in the router a little longer, most likely to accommodate high-latency, low-bandwidth/congested situations.

The MTU situation isn't new to RouterOS 7...it's been there as long as I can remember. Specifically it's a problem where VLAN MTU's that differ from the VLAN's parent bridge MTU are reset to either the bridge MTU or 1500 just after boot-up. (In my case it's always 1500, down from 2024 or 4000 or 9000, whatever my radio links are capable of.). More often than not, I see it on my switch-chip devices (CRS300's, CCR2116); I don't recall it as much on my RB4011's and RB5009's as much.

I have an RB5009 where I've started noticing it randomly stop talking to devices on one of the ports. It takes a reboot to fix it. No amount of port-bouncing or bridge tinkering works. I suspect it's seeing occasional route loops or some other packet it doesn't like and it silently shuts down the port. It usually happens if I've been messing physically with a device directly connected or downstream of the failing port. Noticed it on 7.11.2, still happens (very rarely) on 7.13.5.

Archous · Tue Apr 02, 2024 12:26 pm

Well, among other things I just found and fixed the UDP timeout (which is amazing, Mikrotik changing it for new setups but not changing it for existing installations where the user has not changed the default value - talk about breaking systems) which fixed SOME of the issues (RDP it seems).

Now I just read about some MTU trickery broken that "we are not going to fix because we plan to fix it in 7.15" which may well be it.

Given those I am really out of ideas. This is a setup that has been working flawlessly the last years, now I get significant issues down to PING not working over this one router to all machines (which change randomly after hours).
Mikrotik has "general rule" about not touching existing configs, except during major upgrades where a config update is necessary. Usually, changing connection tracking settings falls under that "not a major upgrade" category.

The default NAT UDP timeout has been 10s for a very long time--years, if not decades. That's for "unacknowledged" connections (i.e. client doesn't respond using the same IP:PORT combination; UDP streams have a 3 minute timeout). Bumping it to 30s keeps the NAT entry in the router a little longer, most likely to accommodate high-latency, low-bandwidth/congested situations.

The MTU situation isn't new to RouterOS 7...it's been there as long as I can remember. Specifically it's a problem where VLAN MTU's that differ from the VLAN's parent bridge MTU are reset to either the bridge MTU or 1500 just after boot-up. (In my case it's always 1500, down from 2024 or 4000 or 9000, whatever my radio links are capable of.). More often than not, I see it on my switch-chip devices (CRS300's, CCR2116); I don't recall it as much on my RB4011's and RB5009's as much.

I have an RB5009 where I've started noticing it randomly stop talking to devices on one of the ports. It takes a reboot to fix it. No amount of port-bouncing or bridge tinkering works. I suspect it's seeing occasional route loops or some other packet it doesn't like and it silently shuts down the port. It usually happens if I've been messing physically with a device directly connected or downstream of the failing port. Noticed it on 7.11.2, still happens (very rarely) on 7.13.5.

Except they did just this with VRFs and firewall rules. Existing firewall configs are now broken that reference L3 input interfaces attached to a VRF.

nmt1900 · Tue Apr 02, 2024 12:52 pm

I have an RB5009 where I've started noticing it randomly stop talking to devices on one of the ports. It takes a reboot to fix it. No amount of port-bouncing or bridge tinkering works. I suspect it's seeing occasional route loops or some other packet it doesn't like and it silently shuts down the port. It usually happens if I've been messing physically with a device directly connected or downstream of the failing port. Noticed it on 7.11.2, still happens (very rarely) on 7.13.5.

Are you sure your (R/M)STP works properly? Maybe you should check bridge port status for this port and check which bridge is indicated as root bridge. If root bridge is not detected properly there might be a need to tune STP priority for bridges...

sirbryan · Tue Apr 02, 2024 2:37 pm

Mikrotik has "general rule" about not touching existing configs, except during major upgrades where a config update is necessary. Usually, changing connection tracking settings falls under that "not a major upgrade" category.

Except they did just this with VRFs and firewall rules. Existing firewall configs are now broken that reference L3 input interfaces attached to a VRF.

Note the quotes around "general rule." Does the upgrade process actually change the configuration, or just modify previous behavior and require the user to adjust the configuration accordingly? The latter happens much more frequently, unfortunately, without an automatic fix.

sirbryan · Tue Apr 02, 2024 2:42 pm

I have an RB5009 where I've started noticing it randomly stop talking to devices on one of the ports. It takes a reboot to fix it. No amount of port-bouncing or bridge tinkering works. I suspect it's seeing occasional route loops or some other packet it doesn't like and it silently shuts down the port. It usually happens if I've been messing physically with a device directly connected or downstream of the failing port. Noticed it on 7.11.2, still happens (very rarely) on 7.13.5.
Are you sure your (R/M)STP works properly? Maybe you should check bridge port status for this port and check which bridge is indicated as root bridge. If root bridge is not detected properly there might be a need to tune STP priority for bridges...

I don't doubt there's something along those lines, but this configuration has worked for four years on an RB4011 and now the 5009. It's on customer-facing ports with VLAN's that don't leave the router. There is undoubtedly some kind of "perfect storm" scenario that causes it to happen, and I wouldn't be surprised if it was STP-based, except it doesn't show up in the router as STP blocking the port when it does happen.

pe1chl · Tue Apr 02, 2024 5:42 pm

I am using a 5009 with several ports with VLANs (all on a common VLAN-aware bridge) and I have not yet observed such a problem...
I have set the STP mode to "none", as I always do in places where there is no need for STP.

LightnetBarry · Tue Apr 02, 2024 6:18 pm

Since updating one of our CCR1072 routers from v7.7 to v7.11.2 and now v7.14 we are seeing constantly incrementing CPU load resulting in packet loss until a reboot.

1072v7.png

Router is terminating circa 1000 PPPoE sessions, routing up to 6Gbps IPv4 & IPv6
Routing is primarily OSPF with circa 8000 LSA and iBGP with 3500 prefixes.
A significant portion of routed traffic is via ECMP

A similar CCR1072 with circa 400 PPPoE sessions and 5Gbps via ECMP does not show this behaviour on v7.11.2

PPPoE sessions renew on a 24 hour basis

loloski · Tue Apr 02, 2024 6:27 pm

Please create a different thread so that others might be able to help you and by the looks of it is this a one big box doing everything how about NAT? if yes you might rethink your strategy

LightnetBarry · Tue Apr 02, 2024 6:35 pm

We don't do NAT. Everything is routed, there's one forward chain FW rule to deal with private addresses. Sure there are ~1000 queues, but it is, as you say, a big box! (for our heavier traffic we've moved up to CCR2216s)
I've posted it here as it looks like a bug introduced between v7.7 and v7.11.2 but not yet fixed as of v7.14.2
I don't see how anyone but Mikrotik devs can really help, it's not a configuration issue...

sirbryan · Tue Apr 02, 2024 6:54 pm

I am using a 5009 with several ports with VLANs (all on a common VLAN-aware bridge) and I have not yet observed such a problem...
I have set the STP mode to "none", as I always do in places where there is no need for STP.

Unfortunately, STP is needed on this router due to some uplink redundancy that risks being dumped into the same Layer 2 segment during physical network changes.

That said, one of the affected ports was configured as Edge, the others were not. I've moved all customer-facing ports to Edge to see if that has any effect. It hasn't happened for a couple weeks now, so I suspect the actual cause might be lower on the stack (Ethernet hardware oddities).

sirbryan · Tue Apr 02, 2024 7:05 pm

I don't see how anyone but Mikrotik devs can really help, it's not a configuration issue...

Sounds like a good reason to open a ticket, then, if you haven't already.

The forum is for users to share what works and what doesn't with each other, with only the slight possibility that a dev might see it.

loloski · Tue Apr 02, 2024 7:07 pm

We don't do NAT. Everything is routed, there's one forward chain FW rule to deal with private addresses. Sure there are ~1000 queues, but it is, as you say, a big box! (for our heavier traffic we've moved

In our experience 1072 is more suitable as edge router doing BGP and OSPF only and disable connection tracking for maximum performance and we move PPPoE + Queue on a separate 1036/2116 dedicated box, I know this is not the right answer you are looking for I'm just sharing with you the experience we've learned on that 1072 box it's not suitable for PPPoE termination it has 72 weak cores and Queue is heavy hitter in terms of CPU it needs higher core clock. We have seen this behavior even on 1036 for 800 customer it's working fine as soon as you hit 1K mark then CPU goes haywire and packet loss can now be observed

LightnetBarry · Tue Apr 02, 2024 7:26 pm

Thanks,

I'll open a ticket also. Somehow I expected the devs to monitor the release threads !

We've not seen any issues with PPPoE on the 1072 (this exact router among others) until we went past v7.7
We have <300 sessions on a 2004 and it's dying (about on a par with a 1009, not even matching a 1016) so I don't know about clock speed...

We've seen the 1072 struggle significantly with BGP and multiple full tables though whereas the 2216 just flies along.

loloski · Tue Apr 02, 2024 7:34 pm

Yes 2216 and 2116 is a different beast :) I hope MT support would be able to help you out along the way

jordanp123 · Wed Apr 03, 2024 6:53 pm

Ended up adding a rule in the VRF to get it to work around whatever is going on with the firewall rules and VRF's. Before I had a rule that went from Guest (VLAN) out to the Wireguard tunnel for a VPN. So it was firewall rule (forward) allow from guest to WG Tunnel. With the VRF changes that doesn't work anymore. I had to add a rule that goes from the VRF (The now exposed interface) to the WG Tunnel and it works. Which is very confusing to me. I left the previous firewall rules in place just disabled but even before disabling no packets would hit them.

kinjakinja · Wed Apr 03, 2024 8:48 pm

i had a hap-ac2 ran 6.49.8 , A audience ran 6.48.10 as a AP and a spare hap-lite(current 7.14)
i tried the haplite on 7.2, 7.7, 7.13.5(horrible) 7.14 & 7.14.1

surprisingly the hap lite with 7.14 is extremely responsively on the wan connection even faster than my hap-ac2(6.49.8)
*except the admin login winbox failure, everhting else seems fine(inc. wifi)

i'll keep the haplite with 7.14 and ran for a week,
if it is good i may upgrade the main router ac2 to 7.14 and keep the haplite as a spare

reporting: had discover why the old haplite running 7.14.1 has problem with winbox
if i choose to save session with winbox connection, after a few logins, it will start causing 100% cpu usage on it.
symptoms of problems after a few login in by winbox:
- wan start slowing down.
- multiple kick out from winbox
finally the haplite will freeze to slow to death all you can do it unplug the miniusb power.

to solving the problem, delete all the files in
drive://Users/username/AppData/Roaming/Mikrotik
(i'm running win10)
run winbox without saving password & sessions.
by deleting all the files in above directory... i had no login problems or slow wan anymore. smooth for days and weeks.
on my case, seems the problem was caused by the winbox saved session problem.

Had upgraded the hap-ac2 to 7.14.2 via netinstall so far is good and fast
where everyone are concerned the freeHDD space, mine is 924Kb
run following:
multi wan+failover, ECMP, PCC, Wifi , NTP & graphing

whatever · Thu Apr 04, 2024 12:33 am

where everyone are concerned the freeHDD space, mine is 924Kb

I guess you are not using the wifi-qcom-ac driver package.

Network5 · Thu Apr 04, 2024 8:34 am

I have a tricky problem that came out during MPLS debugging:

Supposed to have three routers, Router A - CCR2004, Router B - CCR2004, and Router C - CCR2216.
Routers A and B are linked with AirFiber 24 and the same is between B and C. AirFiber supports jumbo frame.
If pinging from A to B with MTU 9000 and DF it works, perfectly. The same from B to C or viceversa.
But if pinging from A to C a "packet too large and cannot be fragmented" message is shown. :-)

The router has a fresh update to 7.14.2 and the same behavior as with 7.12.1.
Does anyone have similar experiences?

kinjakinja · Thu Apr 04, 2024 12:37 pm

where everyone are concerned the freeHDD space, mine is 924Kb
I guess you are not using the wifi-qcom-ac driver package.

i'm just using the wireless package, i had no idea, will the qcom-ac preform better?
i tried once but fail to configue the 5G channel.

dwnldr · Thu Apr 04, 2024 11:50 pm

Hi Guys !

I encountered such bug on some previous version of v7 RouterOS, where the WiFi interface IGNORES the security profile, and the result is unsecured network. Tested on wap AP, cap AC XL, mAntbox and recently today on hapAx3, with 7.14.2. For instance 2.4ghz&5ghz interface is configured with the SAME security profile. Result is secured 2.4ghz and unsecured 5ghz network. When i fill manually the columns under "security" tab, they are working, neither with profile. hapAc3 worked OK. No idea why is this happening...