MikroTik

Posted: **Sun Jan 21, 2024 7:07 pm**

Indeed, they are in denial. We can only hope that 7.12.1 will be labeled long-term and maintained with security fixes...

Posted: **Sun Jan 21, 2024 7:11 pm**

Less significant, means it doesnt fit into the business planning ( aka profit models and future product planning ). Any change requires resources and those are tightly controlled.
@normis I agree with Pe1chl, 7.12.2? whatever was the last one, may be an excellent candidate for long term stable.

Posted: **Sun Jan 21, 2024 7:30 pm**

Indeed, they are in denial. We can only hope that 7.12.1 will be labeled long-term and maintained with security fixes...

+1

Posted: **Mon Jan 22, 2024 5:46 am**

Having a long term branch only with fixes, one can dream..

Posted: **Mon Jan 22, 2024 10:00 am**

MT is not willing to maintain a long term branch right now. The ROS 7 train is going without long-term stops ahead. I assume "forking" now a long term branch would increase their development effort. not immediately, but within months the branches diverge and and will be hard to apply only patches while the stable branch keep going on.

Posted: **Mon Jan 22, 2024 10:26 am**

Well they have to do something if 7.14 is getting unusable on 16MB devices.
The split I've mentioned above could be done easier than getting back to split packages, I think. They'd just have to implement a way for devices to get the proper firmware for them (since there will be more firmwares for the same architecture...).

Posted: **Mon Jan 22, 2024 11:15 am**

I assume "forking" now a long term branch would increase their development effort. not immediately, but within months the branches diverge and and will be hard to apply only patches while the stable branch keep going on.

It should not be that hard. Of course there will be no development in that branch, only backporting of critical fixes.
Just as it has always been in the v6 long-term branch.
I (and several others) just want to make sure we can remain on 7.12.x until the whole wifi/flashspace hoopla has been neatly solved.

Posted: **Mon Jan 22, 2024 11:45 am**

v6 long term wasn't like that, it had huge version bumps with a ton of new features and obviously new bugs, it had like what, 2-3 bugfix releases per major version?

Posted: **Mon Jan 22, 2024 11:50 am**

Yeah, sometimes they backported fixes that did not even work in the release they came from. I guess developers are human.
But it would be nice if we could get commitment to maintain 7.12.x with critical fixes until the showstopping bugs in 7.13+ have been ironed out a bit. Maybe when 7.16 is released or so.

Posted: **Mon Jan 22, 2024 1:10 pm**

Indeed, they are in denial. We can only hope that 7.12.1 will be labeled long-term and maintained with security fixes...

Or one could go back to RouterOS v6 long-term, it is still supported and will be for some time as far as I can tell...

Posted: **Mon Jan 22, 2024 1:13 pm**

Or one could go back to RouterOS v6 long-term, it is still supported and will be for some time as far as I can tell...

Nope. Mikrotik still sells devices with 16MB SPI flash onboard, and many (all?) of those are shipped with v7 from the factory, which makes downgrading them to v6 impossible.

Posted: **Mon Jan 22, 2024 3:21 pm**

Bug with Push Routes feature for OpenVPN.

We push a lot of routes (about 100) to OpenVPN client, using OpenVPN Access Server.

When we try to push these using ROS 7.14.B7, it loses the plot.

Bug is volume related. We have tested pushing up to 11 routes and that is OK. Try for 12 and it does not work (in our case).

Every time it goofs up, the OpenVPN server is left unstable, requiring router restart.

We've tried removing bound IP Address, bound PPP interface, plus disable OVPN server in the PPP dialogue then re enabling. Nothing will recover the OVPN stack to working condition.

Can we please log this as a bug, and have dev team do some stress testing for high volume routes? (like 100 or 200 or 300?).

____________
Some more info. Running on CHR instance so x86 code base.

We intend to use the CHR instance to give us a static IP that we can VPN to then route through to access remote sites which are firewalled to one source IP. Likely we will end up with 100's++ of pushed routes; current count is 137.

When we got to 12 routes pushed. we gets lots of 'interfaces' created, which go red in Winbox. At that stage, we can reduce the route push count to 11 and make a new connection. However, when we stick in like 20 or 30, we get the instability requiring full reboot. Not feasible to test further to see the level that happens at, as already broken at 12!
_____________
Same behaviour 7.14.B8

Posted: **Mon Jan 22, 2024 3:31 pm**

But with proper ICMP troubleshooting is so much nicer and it is so much friendlier to the software as it allows immediate and accurate error reporting to the user.

Unfortunately, a widely used desktop operating system decides to hide the useful error information from the user...
(and also it does not take NO for an answer... it just retries as usual)

Still, that blackhole for IPv6 addresses is a good thing as it prevents traffic bouncing back- and forward over the subscriber line when probes are done for subnets not configured on the router. Typically users use only a very small number of their allocated subnets, so most of their space is unroutable and the default route on their CPE would send it back to the provider.

Posted: **Mon Jan 22, 2024 3:57 pm**

And also helps potential attackers to scan IPv6 address space much more effectively.

And why do you consider SOHO differently than DCs and other corporate installations?

Only trusted side of the LAN gets the privilege of proper response.

I treat it differently with respect to “wasted cpu cycles” comment. SOHO router will likely sit way below 100% utilization and such will likely have resources to echo with appropriate ICMP errors. My hAP ac lite could handle it.

Still, that blackhole for IPv6 addresses is a good thing as it prevents traffic bouncing back- and forward over the subscriber line when probes are done for subnets not configured on the router. Typically users use only a very small number of their allocated subnets, so most of their space is unroutable and the default route on their CPE would send it back to the provider.

I don’t disagree with that. It is good that this issue finally gets some recourse and I do warmly welcome it. Old behavior of default routing back was shameful.

But I also want to see a builtin solution for proper errors that is as easy to set up as blackhole routes.

At the very least I don’t want this release to break my configuration that routes unallocated PD to the “trap” interface which sends errors via the firewall.

Posted: **Mon Jan 22, 2024 4:03 pm**

And also helps potential attackers to scan IPv6 address space much more effectively. I don't know if potentual benefits actually outweigh drawbacks.

And why do you consider SOHO differently than DCs and other corporate installations?

Obscurity is not security. I don't care if they can ping my hosts, my hosts have stateful firewalls enabled to drop unsolicited inbound other than ICMPv6 itself.

The problem with ICMPv4/v6 replies on a router instead of blackhole is, it's easy to DoS/DDoS the CPU of the router with these packets. If it's blackhole, you cannot DoS/DDoS the CPU.

Posted: **Mon Jan 22, 2024 4:27 pm**

Less significant, means it doesnt fit into the business planning ( aka profit models and future product planning ). Any change requires resources and those are tightly controlled.
@normis I agree with Pe1chl, 7.12.2? whatever was the last one, may be an excellent candidate for long term stable.

7.12.x is definitely “most stable”, I'm running it in both SP, DC and home networking. +1 to make 7.12.x as long-term.

BGP works, OSPF, MPLS etc, there are bugs, but not crashing type of bugs, good enough for long-term channel of ROSv7

Posted: **Mon Jan 22, 2024 5:29 pm**

there are bugs, but not crashing type of bugs, good enough for long-term channel of ROSv7

LOL, agree. And running out of disk space is a quick way to have the "crashing type of bugs" IMO.

Only issue is scripting in the "7.12.x as long-term" suggestion – there is JSON support and other very nice changes in v7.14 that be lost.

Posted: **Mon Jan 22, 2024 6:43 pm**

There will always be enhancements that will be missed when staying at an older version. Not relevant.

Posted: **Tue Jan 23, 2024 10:27 am**

It seems Let's Encrypt doesn't work with 7.14beta7. It returns an error: "progress: [error] http client error, please make sure device is connected to internet and letsencrypt servers are reachable". It worked with 7.13.1 before upgrading to 7.14beta7 and it also works after downgrading to 7.13.2 using the identical RouterOS configuration. Can anyone replicate?

Posted: **Tue Jan 23, 2024 3:44 pm**

What's new in 7.14beta8 (2024-Jan-22 21:07):

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);
*) bgp - allow to leak routes between local VRFs;
*) bth - added simple "Back To Home Users" manager under IP/Cloud menu;
*) console - added "show-at-cli-login" option to display a note before telnet login (CLI only);
*) defconf - fixed Audience scanning-for-wps-ap timeout;
*) dns - do not add new entries to cache if "cache-size" is reached;
*) dns - fixed DNS service crash when DoH used (introduced in v7.14beta4);
*) fetch - added "head" option for "http-method";
*) fetch - allow specifying link-local address in FTP mode;
*) fetch - fixed fetch when using "src-path" with SFTP mode (introduced in v7.13);
*) firewall - fixed underlying CAPsMAN tunnel reusing packet marks of encapsulated packets;
*) firewall - fixed underlying VXLAN/EoIP tunnel reusing packet marks of encapsulated packets;
*) health - show voltage when powering KNOT R through Micro-USB;
*) iot - added bluetooth whitelist wildcard asterisk support;
*) iot - improvements to GPIO behavior on boot;
*) iot - improvements to LoRa CUPS;
*) iot - removed bluetooth whitelist maximum entry limit of 8;
*) ipv6 - made "valid" and "lifetime" parameters dynamic for SLAAC IPv6 addresses;
*) leds - fixed modem signal strength for RBSXTR&R11e-LTE (introduced in v7.14beta6);
*) lte - added "at-chat" support for Sierra Wireless EM9293 5G modem;
*) lte - fixed Simcom modem support in 0x9001 USB composition;
*) modem - improved stability when performing modem FOTA upgrade;
*) netinstall-cli - check package and device architecture before formatting;
*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;
*) package - reduced package size for SMIPS;
*) poe-out - fixed "power-cycle" for CRS354-48P-4S+2Q+ device (introduced in v7.13);
*) sms - fixed SMS inbox for FG621-EA modem (introduced in v7.13);
*) sms - fixed SMS sending from WinBox and WebFig (introduced in v7.13);
*) sms - improved system stability when working with SMS;
*) snmp - updated timeout log;
*) switch - fixed "cpu-flow-control" for RB3011 (introduced in v7.14beta3);
*) switch - fixed Atheros switch port configuration export (introduced in v7.14beta3);
*) switch - fixed Ethernet disable/enable for CRS310-8G+2S+ devices;
*) system - properly close HTTP/S connections initiated by the router;
*) system - properly start HTTP/S connections initiated by the router if non-default port is used (introduced in v7.14beta3);
*) traffic-flow - use 64bit counters for v9 and IPFIX flows;
*) traffic-generator - improved system stability when receiving bogus traffic;
*) webfig - fixed setting the user's password;
*) wifi - improved handling of CAP connections in dual CAPsMAN scenario;
*) wifi-qcom - improved system stability when using FastPath (introduced in v7.13);

Posted: **Tue Jan 23, 2024 4:08 pm**

Let's Encrypt problem persists with 7.14beta8.

Posted: **Tue Jan 23, 2024 5:36 pm**

*) ipv6 - made "valid" and "lifetime" parameters dynamic for SLAAC IPv6 addresses;

What does this actually mean?

Posted: **Tue Jan 23, 2024 5:58 pm**

What's new in 7.14beta8 (2024-Jan-22 21:07):

*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;

Did anyone already check by how much?
(i.e. how much more free space do you have after upgrade to this version on a device with wireless package)

Posted: **Tue Jan 23, 2024 6:00 pm**

Just compare .npk...
2.686.976 - 2.244.753 byte = 442.223 byte...

Removed the file
wil6436-cube-sa-tg.brd 13.472 byte
wil6436-cube-tg.brd 13.472 byte
wil6436-tg.fw 986.764 byte

So, uncompressed on NAND/Flash are 1.013.663 byte (~1MiB) less
This compensate for some device this:

!) rose-storage - moved SMB service in the RouterOS bundle;

Posted: **Tue Jan 23, 2024 7:11 pm**

What's new in 7.14beta8 (2024-Jan-22 21:07):

*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;
Did anyone already check by how much?
(i.e. how much more free space do you have after upgrade to this version on a device with wireless package)

7.13.2 -> 14.4 MiB of 15.3 MiB used : 5% free
7.14beta8 -> 14.2 MiB of 15.3 MiB used : 7% free

On my hAP ac²

Posted: **Tue Jan 23, 2024 7:12 pm**

Some time ago I filed a bug report about RA’s advertised prefix “valid” and “lifetime” not respecting corresponding values of DHCPv6 Client PD (it used values from the `default` submenu instead).

Perhaps that’s what they addressed here?

Posted: **Tue Jan 23, 2024 7:29 pm**

So they advertise the lifetime of the received prefix now? how sad.

Posted: **Tue Jan 23, 2024 7:35 pm**

(accidentally edited instead of new post)

Posted: **Tue Jan 23, 2024 7:50 pm**

What's new in 7.14beta8 (2024-Jan-22 21:07):

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);

I still can't get it to work... I would like to understand if I'm wrong or is there a new way with CLI

Posted: **Tue Jan 23, 2024 7:52 pm**

If the idea is more unification (ROSE SMB in routeros package)...

IMO it should be the the otger way around ... throw SMB out if core ROS. For those who want to use their ROS device as NAS, they can install (full) ROSE. If they want to run NAS services, they better use device with decent storage size.

Posted: **Tue Jan 23, 2024 8:09 pm**

If the idea is more unification (ROSE SMB in routeros package)...
IMO it should be the the otger way around ... throw SMB out if core ROS.

I obviously agree. Now MT's counterpoint has been there is overhead in packaging (and, likely, add complexity in upgrade/migration).

So, in practical terms, MQTT is "broken" from 7.12.1 to 7.13+ for 16MB flash devices.

Posted: **Tue Jan 23, 2024 8:14 pm**

IMO it should be the the otger way around ... throw SMB out if core ROS.
I obviously agree. Now MT's counterpoint has been there is overhead in packaging ...

If understood MT's reasoning correct, this was argument for inclusion of routing protocols (BGP, OSPPF, whatnot) into base package and the reason was that ROS doesn't run separate routing daemons.

But this clearly isn't the case with SMB since MT was able to replace one SMB with another.

There are only two legitimate reasons for not moving certain functionality to optional package:

package dependencies
if too many packages (not necessarily core package) depend on some functionaliry, then its indeed easier to package that functionality into base package
if functiobality is indeed part of core linux kernel or a common service (think busybox) and only parts of UI could be split into separate package

All the rest boils down to amount of work for package managers and IMO MT should work hard make new ROS releases compatible with not-so-ancient 16MB-only-flash devices. If not other to make up for blatantly ignoring voices of wisdom which raised concerns about introducing tiny-flash devices only a few years ago.

Posted: **Tue Jan 23, 2024 8:24 pm**

So they advertise the lifetime of the received prefix now? how sad.

Do you prefer stale prefixes to linger for 30 days instead?

RFC requires valid and lifetime values in advertisements of prefixes derived from PD to not exceed the parent PD. Note that if PD renews to the same value, no renumbering will actually take place, assuming your SLAAC hosts are not buggy.

Posted: **Tue Jan 23, 2024 8:52 pm**

Also minor logging bug when you do run out of disk space when attempting an upgrade. After reboot, the logs show:

The (null) looks wrong, either package name or nothing be better.

Posted: **Tue Jan 23, 2024 8:56 pm**

*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;

Doubt this change does much if people want to use wifi-qcom-ac because that did stay the same, and possibly grew a little. I think people expected to reduce the size of the main RouterOS package or wifi-qcom-ac.

Posted: **Tue Jan 23, 2024 9:23 pm**

It seems that

/tool fetch

does no longer include a "total" value - only "downloaded".
Is this expeded/hidden feature or a bug?

Posted: **Tue Jan 23, 2024 10:32 pm**

So they advertise the lifetime of the received prefix now? how sad.
Do you prefer stale prefixes to linger for 30 days instead?

RFC requires valid and lifetime values in advertisements of prefixes derived from PD to not exceed the parent PD. Note that if PD renews to the same value, no renumbering will actually take place, assuming your SLAAC hosts are not buggy.

No, I'd like the ability to set my own lifetime (lower) and not use the lifetime offered by the ISP, and I don't know if that's possible anymore? Since the changelog is rather vague.

Posted: **Tue Jan 23, 2024 10:40 pm**

No, I'd like the ability to set my own lifetime (lower) and not use the lifetime offered by the ISP, and I don't know if that's possible anymore? Since the changelog is rather vague.

Agree, there must remain a possibility to change these values administratively. Changelog quality is piss-poor.

Posted: **Tue Jan 23, 2024 11:28 pm**

Agree, there must remain a possibility to change these values administratively. Changelog quality is piss-poor.

+10000

Posted: **Tue Jan 23, 2024 11:32 pm**

MT should put out a hotfix for let's Encrypt

Why?
This is a beta and should not be used in production. Just wait until it are fixed.

Posted: **Tue Jan 23, 2024 11:32 pm**

This is a beta and should not be used in production.

I quote

Posted: **Tue Jan 23, 2024 11:37 pm**

wifi-qcom-ac contains the firmware for QCA9984 and IPQ4018/4019. Most devices just have one of these chipsets. Audience has both - but 128MB storage. But most devices could gain at least 700kb of free space - just by not having useless firmware installed.

MT, it is noble to reduce wireless package size. But splitting wifi-qcom-ac by chipset would have a huge impact and would be - IMHO - easy to achieve. Low hanging fruits.

Posted: **Wed Jan 24, 2024 12:45 am**

*) traffic-flow - use 64bit counters for v9 and IPFIX flows;

Wow! Finally!!! It only took 8+ years!!!
Better late than never I guess :)

Posted: **Wed Jan 24, 2024 12:50 am**

MikroTik development never stops! also years have a different meaning in Latvia, for them it's been like 8 months since it was reported.

Posted: **Wed Jan 24, 2024 4:35 am**

the changelog is rather vague.

Changelog quality is piss-poor.

I think an easy way to improve this would be to link to the relevant bug tracker case (MikroTik has one of these, right?) which would hopefully have more tech stuff for people to read over if they want.

Posted: **Wed Jan 24, 2024 4:46 am**

I think an easy way to improve this would be to link to the relevant bug tracker case (MikroTik has one of these, right?) which would hopefully have more tech stuff for people to read over if they want.

There is a per-user support tool, but it's not for tracking: they close tickets once they are acknowledged and you don't get updates.

Posted: **Wed Jan 24, 2024 9:22 am**

MT, it is noble to reduce wireless package size. But splitting wifi-qcom-ac by chipset would have a huge impact and would be - IMHO - easy to achieve. Low hanging fruits.

Perhaps there's another way instead of further package splitting: include a pre-install script into package ... that script detects actual hardware on device, where npk is being installed and only install .ko necessary to drive actually present hardware.

Rationale: the devices in question all use RAM disks as storage root, which is used as storage when .npk files are downloaded (both when using ROS package upgrade function and if manually pushed to device) and RAM size is normally not a problem. Which means that a bit larger (a few MB) npk files are not actual problem. Problem is space used after installation and including only necessary files onto installed ROS would help a lot.

I guess this would require another required ROS stop version because the behaviour explained above depends on behaviour of already running ROS version (if current ROS version updaters don't include ability to execute pre/post-install scripts). And this possibly requires change in netinstall as well: either include ability to execute pre/post-install scripts from .npk being installed or (less favourable) handle the same functionality "natively" (but this would require frequent changes of netinstall binary to have it in line with npk files).
This kind of solving the problem might actually allow for merger packages wifi-qcom and wifi-qcom-ac into one again ... this depends on other differences in packages (apart from which .ko modules are present in which npk).

Posted: **Wed Jan 24, 2024 11:07 am**

And this possibly requires change in netinstall as well

As well as the way CAPs are upgraded from CAPsMAN (both versions), I guess. And yet we might not see the whole picture...

Posted: **Wed Jan 24, 2024 11:11 am**

I still think that different RouterOS bundles for different devices with the same architecture would be the easier way to do. Just a way to prevent users from installing the wrong bundle for their device is needed.

Posted: **Wed Jan 24, 2024 12:05 pm**

the changelog is rather vague.

Changelog quality is piss-poor.
I think an easy way to improve this would be to link to the relevant bug tracker case (MikroTik has one of these, right?) which would hopefully have more tech stuff for people to read over if they want.

I have proposed several times that they make a database of change log lines where each entry has the short description as now seen in the release topic but there are also other elements like a more detailed explanation, a pointer to the relevant manual page where more info may be shown, a version where the reason for the change was introduced, etc.
Then a script that extracts this info can be used to automatically generate the changelog for the version, but a webpage can be provided to browse the recent changes and see more info e.g. by clicking on links or by hovering over the line.

And, it could have a search field where two version numbers can be selected and it will show all changes between those two versions, which you can use to get all relevant changes when you want to upgrade from an earlier release.
E.g. you want to upgrade from 7.12.1 to 7.14beta7 then you select those versions and you see all changes between those versions.
That would be all release notes from versions in between MINUS the notes that fix problems "introduced in >7.12.1".
The forum topics for new version release would include a link to that webpage with the search fields set correctly for that version.

That same query would be used to display the changes when you try to do an upgrade in the router's packages menu.
(now it shows only the notes for the version you want to install and you may miss some critcial notes about changes in versions between the version you are running and the version you want to install, in this example the wifi packages changes)

I would think it also is less work for MikroTik that way, because they can automatically put the relevant changes into that database from their version control system, rather than having to manually generate the changes lines.

Posted: **Wed Jan 24, 2024 12:21 pm**

MT, it is noble to reduce wireless package size. But splitting wifi-qcom-ac by chipset would have a huge impact and would be - IMHO - easy to achieve. Low hanging fruits.

Perhaps there's another way instead of further package splitting: include a pre-install script into package ... that script detects actual hardware on device, where npk is being installed and only install .ko necessary to drive actually present hardware.

Absolutely! But if they consider this approach they can revert the package split (wifi-qcom and wifi-qcom-ac) they did with 7.13. Then just a single "wifi-qcom" package is all we need and the package manager installs only the hardware-relevant kernel modules and chipset firmware. At least I think this should be possible, as NPK upgrade packages you upload manually are also larger than free disk space and are possibly save in some RAM tmpfs or alike.

Posted: **Wed Jan 24, 2024 12:28 pm**

I have proposed several times that they make a database of change log lines where each entry has the short description as now seen in the release topic but there are also other elements like a more detailed explanation, a pointer to the relevant manual page where more info may be shown, a version where the reason for the change was introduced, etc.

I absolutely hope and pray that the changelog is not a programmatically auto-created changelog from e.g. their VCS or alike.

*) wifi-qcom - improved system stability when using FastPath (introduced in v7.13);

No serious software developer would write such commit or changelog messages.

So somewhere the "real" detailed changelog may exist. The changelog we see publicly is maybe curated manually by someone from testing-department.

Posted: **Wed Jan 24, 2024 12:32 pm**

And this possibly requires change in netinstall as well
As well as the way CAPs are upgraded from CAPsMAN (both versions), I guess.

Maybe not ... I guess that CAPsMAN initiated upgrades of CAPs are actually more or less handled by CAPs the same way as "manually initiated upgrades from within ROS" are ... only the kick to do it comes from CAPsMAN (instead of a GUI button click) and npks are downloaded from different source (CAPsMAN instead of mikrotik's download server).

But you're right, only MT devs can tell if my idea is feasible or not.

Posted: **Wed Jan 24, 2024 12:46 pm**

MikroTik_RouterOSv7.14beta8_BGP_RouteLeak_VRF.png

Hurrah!

E interact with MikroTik devices since 2007-2008...
And Just now... But it is a beginning.

Now we need to check if the behavior is correct with L3VPN / IPv6...

Posted: **Wed Jan 24, 2024 2:14 pm**

Code: Select all
*) wifi-qcom - improved system stability when using FastPath (introduced in v7.13);
No serious software developer would write such commit or changelog messages.

There are some standard phrases used in those changelog messages. It can be good because it reduces the line length.
"improved system stability when..." = the router crashed when you did this, we have solved that.
"improved stability when receiving bogus traffic" often refers to a fix for a CVE.
etc.

It is fine with me for a brief changelog, but for every line there really should be a record with more detail.
The lack of detail also leads to noise on the announce topic, that could be avoided by presenting the info that people will ask for anyways.

Posted: **Wed Jan 24, 2024 2:24 pm**

*) vlan - fixed non-running VLAN interface after failed MTU change;

I don't know if related, but since 7.13, and also in 7.14beta8 I have to disable/enable the VLAN interface to make PPPoE on that interface work.

Posted: **Wed Jan 24, 2024 2:35 pm**

*) dns - fixed DNS service crash when DoH used (introduced in v7.1beta4);

@EdPa
Is the "introduced in version v7.1beta4" information correct? Or should it be v7.14beta4?

Posted: **Wed Jan 24, 2024 3:11 pm**

@DanMos79, thanks. It should be v7.14beta4.

Posted: **Wed Jan 24, 2024 4:11 pm**

Maybe not ... I guess that CAPsMAN initiated upgrades of CAPs are actually more or less handled by CAPs the same way as "manually initiated upgrades from within ROS" are ... only the kick to do it comes from CAPsMAN (instead of a GUI button click) and npks are downloaded from different source (CAPsMAN instead of mikrotik's download server).

I mean, by default, CAPsMAN serves locally installed packages to remote CAPs if architecture matches. Now, if local packages are only partially installed, that will have to be taken into account.

Also, I suspect the npks are not really "packages" like rpm, deb, etc. They seem to be SquashFS images, likely with some proprietary extensions to allow for consistency checks / tampering prevention. So they are likely just copied to the device during upgrade / netinstall, and then mounted on boot, so the whole arrangement may need to be redesigned to make partial package installation possible.

Posted: **Wed Jan 24, 2024 4:24 pm**

I mean, by default, CAPsMAN serves locally installed packages to remote CAPs if architecture matches. Now, if local packages are only partially installed, that will have to be taken into account.

Correct and it's all or nothing (learned it from observing an upgrade not going like I wanted until I noticed that next to wifi-qcom.npk I also needed to included base ROS package on the folder used for this purpose or it wouldn't work).
All packages need to be available on local controller or all need to be provided via storage.
Capsman upgrade does not mix both sources.

Posted: **Wed Jan 24, 2024 6:06 pm**

Maybe not ... I guess that CAPsMAN initiated upgrades of CAPs are actually more or less handled by CAPs the same way as "manually initiated upgrades from within ROS" are ... only the kick to do it comes from CAPsMAN (instead of a GUI button click) and npks are downloaded from different source (CAPsMAN instead of mikrotik's download server).
I mean, by default, CAPsMAN serves locally installed packages to remote CAPs if architecture matches. Now, if local packages are only partially installed, that will have to be taken into account.

I don't know about new CAPsMAN, the old simply served packages from local storage. And it had to have complete npk files, which have nothing in common with installed files. Also it was possible to upload npks for different platforms and CAPs then selected the files they required.

Also, I suspect the npks are not really "packages" like rpm, deb, etc. They seem to be SquashFS images, ...

If one opens npk file with 7z (either GUI version in Windows or CLI version in Linux) it's possible to extract individual files. This doesn't proove anything. But I seriously doubt that npks are simple SquashFS files which are dumped on flash by installer because this way it wouldn't be possible to install any of optional packages. I'm sure that there is copying (or un-archiving) of files involved, but quite probably it's "dumb" copying of whole archives.
Alas, some intelligence is possible, "intelligent" upgrade from 7.12.1 with wifiwave2 to 7.13 with correctly chosen wifi-qcom variety does proove it (but it's likely that this intelligence is built into ROS upgrade command).

Posted: **Wed Jan 24, 2024 6:12 pm**

Alas, some intelligence is possible, "intelligent" upgrade from 7.12.1 with wifiwave2 to 7.13 with correctly chosen wifi-qcom variety does proove it (but it's likely that this intelligence is built into ROS upgrade command).

It is present in 7.12 since that's the last version you had to upgrade to BEFORE you could move to 7.13 (in a normal way, not referring to manual upload or netinstall).
So this move was already being prepared before we even knew 7.13 was coming.

Posted: **Wed Jan 24, 2024 6:27 pm**

My point is that it's possible to add intelligence into installer ... if it can select different packages, then it could also selectively copy contents of npks. If MT devs made necesary changes that is.

Posted: **Wed Jan 24, 2024 7:11 pm**

I don't know about new CAPsMAN, the old simply served packages from local storage.

Not necessarily, see https://help.mikrotik.com/docs/display/ROS/CAPsMAN

package-path (string |; Default: ) Folder location for the RouterOS packages. For example, use "/upgrade" to specify the upgrade folder from the files section. If empty string is set, CAPsMAN can use built-in RouterOS packages, note that in this case only CAPs with the same architecture as CAPsMAN will be upgraded.

If you leave package-path empty it will use packages that are actually installed on the manager, not separate npk files. It's the same in new capsman.

Posted: **Wed Jan 24, 2024 7:29 pm**

Regarding the following item:

dns - do not add new entries to cache if "cache-size" is reached;

I don't know about you guys (and this is not a written rule), but my standard expectation when talking about a DNS cache is some sort of Most Recently Used (MRU) cache or similar.

I would expect that some cached entries that are older / less used / perhaps a combination of both would be removed to give space for the new records in need of being cached. Not to simply stop saving to the cache when there is not enough room.

Posted: **Wed Jan 24, 2024 7:46 pm**

I had the same thought on the cache... Older items should be discarded, to have newer ones in cache.

Posted: **Wed Jan 24, 2024 7:52 pm**

Discarding a record early is not a problem. Just keeping it longer than TTL is.

Posted: **Wed Jan 24, 2024 9:11 pm**

You can already set a "Cache Max TTL" when you have a lot of DNS cache and are unable to increase the cache size due to memory limitations of the router. E.G. set it to 02:00:00.
On typical router models used in heavy-use situations you can easily set the cache size 10x higher than default (i.e. add a zero after the size).

Posted: **Wed Jan 24, 2024 10:18 pm**

Sure when there is no more cache space it would be best when the DNS resolver drops old records first.
However, by doing the above tuning you can likely avoid that situation occurring altogether.

Posted: **Wed Jan 24, 2024 10:31 pm**

Also, I suspect the npks are not really "packages" like rpm, deb, etc. They seem to be SquashFS images, likely with some proprietary extensions to allow for consistency checks / tampering prevention. So they are likely just copied to the device during upgrade / netinstall, and then mounted on boot, so the whole arrangement may need to be redesigned to make partial package installation possible.

I think the better/easiest way would be to use one package to each close related groups of devices. I will not say "one package for each device", but something close to it. This way there is no need to manage dependencies, no thinking about whatsoever. Just one package to (say) hAp AX2, another for hAp AX3.

There would be no need to upgrade or modify CAPSMAN. As far as it would be concerned, there would be now 20 devices types, instead of just 3.

Posted: **Wed Jan 24, 2024 10:35 pm**

You can't expect to cache the whole Internet. So some cache management is required. Common approaches are LRU and displacement of records with shortest remaining TTL. Not caching new records when cache is full is a questionable implementation, something an SDE intern may do. I could understand such implementation when trying to avoid churn for media with limited write cycles like flash, but when we are talking DRAM this makes no sense at all.

Posted: **Wed Jan 24, 2024 10:38 pm**

Discarding a record early is not a problem. Just keeping it longer than TTL is.
I run a DNS server and don't recall that. Now I have to do some reading on that.

The TTL just says when the DNS server should fetch the record again. It's perfectly acceptable to ditch it earlier, if we have memory constraints. What shouldn't be done is to keep it after the TTL was expired.

Think about it: the DNS resolver caches the answers, up to the TTL expire time. All well and good. But what if the server runs out of memory? The we discard the least used, even if it didn't get up to the TTL. Sure, more memory (more cache) would be better - but we live in an imperfect world.

Posted: **Wed Jan 24, 2024 11:01 pm**

You can't expect to cache the whole Internet. So some cache management is required. Common approaches are LRU and displacement of records with shortest remaining TTL.

The latter would probably not be a good approach today.
Currently, TTL from the upstream DNS servers is usually either about one hour, or less than a minute.
The latter being used for loadsharing/failover solutions.
Dropping the shortest remaining TTL first will likely drop recently requested (and probably re-requested) records.

Posted: **Wed Jan 24, 2024 11:36 pm**

Short TTL in upstream DNS servers might be there for a reason: to not cache those entries for too long..
Evicting "soon" or already expired entries is a "good thing"

Posted: **Wed Jan 24, 2024 11:42 pm**

Short TTLs ~0-5s are used for load balancing, but the web is full of really long TTLs. NS TTLs are usually hours, if not days like TLD delegations. SOA TTLs may also be really long, but I don't know if ROS resolver is doing negative caching - hopefully not, since it's easy do do wrong. CNAMEs usually have quite long TTLs and there is no harm in evicting them from cache early. What really makes no sense is consistent cache misses once the cache is at capacity.

Posted: **Thu Jan 25, 2024 12:02 am**

I just noticed that the syntax gateway=example-interface@example-vrf in /ip(v6)/route/add dst-address=0.0.0.0/0 gateway=example-interface@example-vrf routing-table=example-vrf (together with /ip/vrf/add interfaces=example-interface name=example-vrf) is not working properly since v7.14beta6:

- /ip(v6)/route/print detail shows flag I - inactive for that static route; and
- ping 8.8.8.8 vrf=example-vrf shows no route to host.

This same configuration has worked just fine for the last couple of years.

Am I missing something? Is it a bug or did something change now that VRFs are exposed as interfaces?

Posted: **Thu Jan 25, 2024 2:24 am**

can we get more info about :

*) bgp - allow to leak routes between local VRFs;

how that works?

Posted: **Thu Jan 25, 2024 11:51 am**

Short TTLs ~0-5s are used for load balancing, but the web is full of really long TTLs. NS TTLs are usually hours, if not days like TLD delegations. SOA TTLs may also be really long, but I don't know if ROS resolver is doing negative caching - hopefully not, since it's easy do do wrong. CNAMEs usually have quite long TTLs and there is no harm in evicting them from cache early. What really makes no sense is consistent cache misses once the cache is at capacity.

The main problem is that MikroTik are developing their own DNS resolver, instead of leaving that to the experts.
There are a couple of open-source resolvers and they usually have open bugtrackers, and browsing these you can easily see how tricky it all is.
Still, they naively start coding and make all the same mistakes again.
E.g. what you mention with CNAME records and the name they are pointing to having a different TTL. What should the resolver do when the client requests an A record, the record in the cache is a CNAME and it points to a record that already has expired.
Answer is obvious, yet this has gone wrong multiple times already. Probably when another programmer takes over the code and thinks "wow this is a mess, let's clean that up a bit because I think it is much too complicated!".
And then we (the RouterOS users) again get a nonfunctional resolver which a couple of versions later is fixed with a "introduced in 7.xx" remark in the changelog.

When will that finally end???

Posted: **Thu Jan 25, 2024 11:55 am**

CNAME records and the name they are pointing to having a different TTL.
What should the resolver do when the client requests an A record, the record in the cache is a CNAME and it points to a record that already has expired.

The Playstation problem plague...

Posted: **Thu Jan 25, 2024 12:31 pm**

Seeing on the changelog of both 7.13.3 and this 7.14beta how many problems are being fixed that were introduced in 7.13 or later, I still think the 7.12.1 release should be promoted to long-term.
There apparently are a lot of changes being made, and a lot of associated mistakes (for whatever reason).
7.12.1 seems to be a stable fallback for many, and it should be supported like that.

Posted: **Thu Jan 25, 2024 12:46 pm**

Seeing on the changelog of both 7.13.3 and this 7.14beta how many problems are being fixed that were introduced in 7.13 or later, I still think the 7.12.1 release should be promoted to long-term

That's why I stick with version 7.12, as it's much more stable. I also believe that the 7.12 should become to long-term.

Posted: **Thu Jan 25, 2024 1:24 pm**

That's why I stick with version 7.12, as it's much more stable.

I have no problem with 7.13.1 on several routers with some complex configuration

I also believe that the 7.12 should become to long-term.

Believe is something you do in a church :)

Posted: **Thu Jan 25, 2024 1:35 pm**

I have no problem with 7.13.1 on several routers with some complex configuration

I have various issues with 7.13.x that are not present in 7.12. Specifically, there are WiFi signal deteriorations on hap ac3(wifi-qcom-ac). As far as I know, similar issues are reported by some other users. Tickets have been opened.

Believe is something you do in a church :)

Since I don't attend church, I've decided to believe in version 7.12 as the long-term here :)))))))))))

Posted: **Thu Jan 25, 2024 6:38 pm**

ROS 7.14 very nice. Now firewall for VPRN traffic to VRF on CPE mikrotik routers just completely and silently ignored . DNAT to VRF also don`t work.
Well done.

Posted: **Thu Jan 25, 2024 7:10 pm**

ROS 7.14 very nice. Now firewall for VPRN traffic to VRF on CPE mikrotik routers just completely and silently ignored . DNAT to VRF also don`t work.
Well done.

1- v7.14 is still in beta
2- I'm having a similar problem with static routes in VRFs: viewtopic.php?t=202612&start=300#p1051523

Posted: **Thu Jan 25, 2024 7:22 pm**

1- v7.14 is still in beta
2- I'm having a similar problem with static routes in VRFs: viewtopic.php?t=202612&start=300#p1051523

try to use ipaddress%inface@table

Posted: **Thu Jan 25, 2024 7:27 pm**

[...] What really makes no sense is consistent cache misses once the cache is at capacity.
The main problem is that MikroTik are developing their own DNS resolver
[...]
When will that finally end???

IMO it starts by documenting what Mikrotik's expected DNS caching behavior is WRT TTL:
https://help.mikrotik.com/docs/display/ ... S-DNSCache

Doc is totally vague as what Mikrotik thinks should happen — it just generically mentions the cache exists & the cache params descriptions don't elucidate much either. There are a few RFC involved & open to interpretation, so what's a bug isn't so clear cut. If the help described the caching rules, that go a long way since there be some baseline as to what's the expected behavior.

Posted: **Thu Jan 25, 2024 7:37 pm**

Well of course the expected operation is "when A record is requested an CNAME record exists in the cache but destination of CNAME record does not, re-request the appropriate pointed record and wait for the response, and return the result to the client only after that response has been received (and put in cache)".
However, more than once that was not the actual behavior, and debugging the resulting problems is difficult.
(most client devices do not issue error messages when receiving bodged DNS replies and just fail to work correctly)
I read about playstations failing to work, but I encountered problems with video streaming services (Viaplay) as well.
As a user you get the task to test and debug the thing... apparently it is unreasonable to expect it to just work.

Posted: **Thu Jan 25, 2024 7:41 pm**

Can developers from Microtik split base package into multiple packages by functionality? Not everyone needs everything. And devices like hap ac2 are only 5 years on market. It is really not old device. People usually buy routers for many more years. But now on 7.13.3 it has 500kB free space only with base package and wifi installed. It makes bad impresion on microtik devices. It don’t think people buy mikrotik just to use it as basic wifi router. But with such small hdd it is not possible now to use it for anything else than basic wifi router. And sad is that in time when i bought hap ac2 there was not much to choose from.

Posted: **Thu Jan 25, 2024 9:12 pm**

Perhaps there's another way instead of further package splitting: include a pre-install script into package ... that script detects actual hardware on device, where npk is being installed and only install .ko necessary to drive actually present hardware.

Is there any way how to clean it manually after package was installed? I would like to add zerotier package but i need at least 300kB of space. I would be hapy to remove this .ko files manualy as i need only firmware for hap ac2 hardware but i don’t know how.

Posted: **Thu Jan 25, 2024 9:12 pm**

But now on 7.13.3 it has 500kB free space only with base package and wifi installed.

This Hungarian Master Distributor just made a video about this topic, and on their hAP ac2 after installing wifi-qcom-ac running RouterOS 7.13.2 the free space left is 696 KiB, So with 7.13.3 there will be less than that. Thankfully there's subtitles on the video, so it can be understood by everyone. Resources window is shown at 10:17 -> https://www.youtube.com/watch?v=Vek2thSptn8

Posted: **Thu Jan 25, 2024 9:37 pm**

Yes with 7.13.3 i have exactly 500.0KiB. I am thankfull for allowing us to use WPA 3 with hap ac2 and to have wifi-qcom-ac package. It is really great. But unfortunately it is at costs of ability to install other packages. Free space was problem even before but now it is really not enough. Any size optimalization would be really welcomed. I really don’t want to buy new router only few years after i bought this one. Personally i don’t like creating unnecesary e-waste.

Posted: **Thu Jan 25, 2024 9:41 pm**

The main problem is that MikroTik are developing their own DNS resolver, instead of leaving that to the experts.

Agreed. Even major DNS resolvers and large cloud DNS providers have bugs - think recent CloudFlare DNS Root zone DNSSEC issue. Expecting a functional bug free resolver from developers who are not DNS experts is a stretch.
Luckily with container support it is now possible to avoid using Mikrotik DNS altogether and spin up a container with Unbound, Bind, Knot or dnsmasq.
The only difficulty is learning curve for configuration format. It would have been much nicer if Mikrotik was just translating ROS config to native open source resolver config, instead of trying to reinvent the wheel.

Posted: **Thu Jan 25, 2024 10:08 pm**

That is what I mean! E.g. "unbound" provides everything the RouterOS resolver can do now, plus additional things that are still on the list of things-to-do, like DNSSEC support. I'm sure once work is done on that, all the bugs come back to haunt them.

Posted: **Fri Jan 26, 2024 2:32 pm**

That is what I mean! E.g. "unbound" provides everything the RouterOS resolver can do now, plus additional things that are still on the list of things-to-do, like DNSSEC support. I'm sure once work is done on that, all the bugs come back to haunt them.

There are many good dns resolver options to choose from.
Unbound is great and supports everything but may be overkill for a small router...
There is also knot-resolver which seems to be a bit lighter.
Dnsmasq also comes to mind that supports DNSSEC and also provides very useful DHCP and TFTP servers.
And most Linux distributions today have systemd-resolved by default.
So yes there are options but the main issue IMHO is that it has to be small enough to fit even on smallest MikroTik routers which already seems to be struggling with space...

Posted: **Fri Jan 26, 2024 3:03 pm**

The current "resolver" already includes many things that are not part of a standard resolver, like static entries, forwards for a domain, DoH, etc.
But unbound can do them all, plus more. I don't know which of the others would have all those capabilities, but maybe there are indeed more.
Space indeed is always an issue (unfortunately, but that has already been discussed above... all my early MikroTik routers have 128MB flash)

Posted: **Sat Jan 27, 2024 4:32 pm**

That is what I mean! E.g. "unbound" provides everything the RouterOS resolver can do now, plus additional things that are still on the list of things-to-do, like DNSSEC support. I'm sure once work is done on that, all the bugs come back to haunt them.
There are many good dns resolver options to choose from.
Unbound is great and supports everything but may be overkill for a small router...
There is also knot-resolver which seems to be a bit lighter.
Dnsmasq also comes to mind that supports DNSSEC and also provides very useful DHCP and TFTP servers.
And most Linux distributions today have systemd-resolved by default.
So yes there are options but the main issue IMHO is that it has to be small enough to fit even on smallest MikroTik routers which already seems to be struggling with space...

I manage my own resolver with Unbound/DNSSEC to root on an RPi4 which is well fast enough for my needs. Can't see me changing anytime soon.

Posted: **Sat Jan 27, 2024 4:37 pm**

Log spamming connect/disconnect SA timouts etc seems to have stopped now I'm using 7.14.beta8 which is good news. I see Mikrotik has new WiFi6 devices out. I'm sure it wouldn't be anything to do with that know would it.....

Oh and also I see the UK Power levels still ain't been fixed either.....
interface/wifi/radio/reg-info country="United Kingdom" 0

interface/wifi/radio/reg-info country="United Kingdom" 0
  ranges: 2402-2482/20
          5170-5250/23/indoor
          5250-5330/23/indoor/dfs
          5490-5730/30/dfs
          5735-5875/14

Posted: **Sat Jan 27, 2024 5:57 pm**

The main problem is that MikroTik are developing their own DNS resolver, instead of leaving that to the experts.

I have always seen the Mikrotik DNS "server" , as a DNS forwarder, with just limited functions as DNS resolver.
There is much more in a DNS server, than forwarding: like secondary DNS server (zone copy), SOA for a domain, etc etc

https://serverfault.com/questions/10605 ... s-resolver

Posted: **Sat Jan 27, 2024 6:21 pm**

I have always seen the Mikrotik DNS "server" , as a DNS forwarder, with just limited functions as DNS resolver.

Agreed.

Given the numerous 16MB devices & space is "tight", the 7.14beta8 change makes perfect sense IMO... e.g. prevent a disk full condition from DNS cache overflowing:

*) dns - do not add new entries to cache if "cache-size" is reached;

But is anything actually "broken" with DNS in latest build?

Posted: **Sat Jan 27, 2024 6:29 pm**

The typical categorization for DNS servers are Authoritative servers (like PowerDNS or NSD for example) that only serve domains that are authoritative for and Recursive servers (like Unbound) that are your typical DNS resolver. I'd say forwarders fall into the same category.
Of course many solutions provide both functionalities (like BIND).
These are core design principles when writing the software.

I'd say MikroTik's DNS is somewhere in the middle by being a forwarder first and foremost with a little authoritative functionality thrown in.
In general, I avoid using MikroTik DNS where I can and I opt in for BIND or some other proven solution.

Posted: **Sat Jan 27, 2024 8:15 pm**

That is why I wrote "resolver" instead of just resolver on Fri Jan 26, 2024 2:03 pm above.
It isn't only a resolver. But, some of the other mentioned products (at least "unbound") have similar functionality.
In many routers, the capability to serve a .local zone and to forward queries for some domain to another internal server (usually a Windows DC) is included in the DNS resolver just as it is in RouterOS.
And usually it is better integrated as well. E.g. the host names received by the DHCP server are auto-added to the .local zone by the DNS resolver. That is covered in another topic in the forum that recently woke up again.

Posted: **Sun Jan 28, 2024 2:55 am**

with BGP-VPLS can you stop dynamic names on that?

Posted: **Sun Jan 28, 2024 8:36 pm**

with BGP-VPLS can you stop dynamic names on that?

See here, dynamic name isn't only issue:
viewtopic.php?t=201638#p1052392

Posted: **Mon Jan 29, 2024 4:22 am**

*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;

While this is true:

Starting from netinstall'ed 7.12.1, it was able to do a /system/package upgrade to 7.14beta8 with zerotier, gps, routeros, and wireless (automatically since started at 7.12.1).

That was going from 7.12.1 directly to 7.14beta8. HOWEVER... I just noticed that if you upgraded from 7.12.1 via stable 7.13.3, and then try a 7.14beta8 install from 7.13.3, it fails and does NOT even respond to MAC-based winbox (and even if power cycled and default config reset using button).

I'd imagine the reduced wireless.npk size does not help if you already had it from 7.13... but does seem 7.13 -> 7.14 upgrade path may be broken with zerotier & "classic" wireless on 16MB flash devices.

Posted: **Mon Jan 29, 2024 5:11 am**

X86 won't boot up with onboard RTL;8125B nci. It report kernel panic due to new RTL8169 driver introduced in ROS 7.12.

bad-state(-2049, 3fffffff)
[    0.000000] Linux version 5.6.3-64 (agent@cicd-a06.mt.lv) (gcc version 11.1.0 (GCC)) #1 SMP Mon Jan 22 19:16:19 UTC 2024
[    0.000000] Command line: 
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: xstate_offset[3]:  576, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  640, xstate_sizes[4]:   64
[    0.000000] x86/fpu: Enabled xstate features 0x1b, context size is 704 bytes, using 'compacted' format.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000003dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000003e000-0x000000000003ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000040000-0x000000000009efff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009f000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000000fffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000010000000-0x0000000012150fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000012151000-0x00000000758f1fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000758f2000-0x0000000079145fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000079146000-0x000000007929dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007929e000-0x000000007960afff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007960b000-0x0000000079a1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000079a20000-0x000000007a775fff] usable
[    0.000000] BIOS-e820: [mem 0x000000007a776000-0x000000007a821fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007a822000-0x000000007abfffff] usable
[    0.000000] BIOS-e820: [mem 0x000000007ac00000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000d0000000-0x00000000d0ffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000d3709000-0x00000000d3709fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fe042000-0x00000000fe044fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fe900000-0x00000000fe902fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed01000-0x00000000fed01fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff800000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000047fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: update [mem 0x6b727018-0x6b737057] usable ==> usable
[    0.000000] e820: update [mem 0x6b727018-0x6b737057] usable ==> usable
[    0.000000] e820: update [mem 0x6b6fd018-0x6b726457] usable ==> usable
[    0.000000] e820: update [mem 0x6b6fd018-0x6b726457] usable ==> usable
[    0.000000] e820: update [mem 0x6b6d3018-0x6b6fc457] usable ==> usable
[    0.000000] e820: update [mem 0x6b6d3018-0x6b6fc457] usable ==> usable
[    0.000000] extended physical RAM map:
[    0.000000] reserve setup_data: [mem 0x0000000000000000-0x000000000003dfff] usable
[    0.000000] reserve setup_data: [mem 0x000000000003e000-0x000000000003ffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000040000-0x000000000009efff] usable
[    0.000000] reserve setup_data: [mem 0x000000000009f000-0x00000000000fffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000100000-0x000000000fffffff] usable
[    0.000000] reserve setup_data: [mem 0x0000000010000000-0x0000000012150fff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000012151000-0x000000006b6d3017] usable
[    0.000000] reserve setup_data: [mem 0x000000006b6d3018-0x000000006b6fc457] usable
[    0.000000] reserve setup_data: [mem 0x000000006b6fc458-0x000000006b6fd017] usable
[    0.000000] reserve setup_data: [mem 0x000000006b6fd018-0x000000006b726457] usable
[    0.000000] reserve setup_data: [mem 0x000000006b726458-0x000000006b727017] usable
[    0.000000] reserve setup_data: [mem 0x000000006b727018-0x000000006b737057] usable
[    0.000000] reserve setup_data: [mem 0x000000006b737058-0x00000000758f1fff] usable
[    0.000000] reserve setup_data: [mem 0x00000000758f2000-0x0000000079145fff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000079146000-0x000000007929dfff] usable
[    0.000000] reserve setup_data: [mem 0x000000007929e000-0x000000007960afff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x000000007960b000-0x0000000079a1ffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000079a20000-0x000000007a775fff] usable
[    0.000000] reserve setup_data: [mem 0x000000007a776000-0x000000007a821fff] reserved
[    0.000000] reserve setup_data: [mem 0x000000007a822000-0x000000007abfffff] usable
[    0.000000] reserve setup_data: [mem 0x000000007ac00000-0x000000007fffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000d0000000-0x00000000d0ffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000d3709000-0x00000000d3709fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fe042000-0x00000000fe044fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fe900000-0x00000000fe902fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fed01000-0x00000000fed01fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000ff800000-0x00000000ffffffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000100000000-0x000000047fffffff] usable
[    0.000000] efi: EFI v2.70 by American Megatrends
[    0.000000] efi:  ACPI 2.0=0x795a5000  ACPI=0x795a5000  SMBIOS=0x79836000  SMBIOS 3.0=0x79835000  MEMATTR=0x72808018  MPS=0xfcb90  ESRT=0x749cbc98  RNG=0x79862818 
[    0.000000] efi: seeding entropy pool
[    0.000000] SMBIOS 3.2.0 present.
[    0.000000] DMI: HARDKERNEL ODROID-H2/ODROID-H2, BIOS 5.13 07/27/2020
[    0.000000] tsc: Detected 1996.800 MHz processor
[    0.000014] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000019] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000033] last_pfn = 0x480000 max_arch_pfn = 0x400000000
[    0.000035] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC  
[    0.000037] last_pfn = 0x7ac00 max_arch_pfn = 0x400000000
[    0.004644] found SMP MP-table at [mem 0x000fce00-0x000fce0f]
[    0.004659] esrt: Reserving ESRT space from 0x00000000749cbc98 to 0x00000000749cbcd0.
[    0.004666] e820: update [mem 0x749cb000-0x749cbfff] usable ==> reserved
[    0.004695] kexec: Reserving the low 1M of memory for crashkernel
[    0.004698] Using GB pages for direct mapping
[    0.004702] BRK [0x02401000, 0x02401fff] PGTABLE
[    0.004705] BRK [0x02402000, 0x02402fff] PGTABLE
[    0.004707] BRK [0x02403000, 0x02403fff] PGTABLE
[    0.005379] BRK [0x02404000, 0x02404fff] PGTABLE
[    0.005527] BRK [0x02405000, 0x02405fff] PGTABLE
[    0.005684] BRK [0x02406000, 0x02406fff] PGTABLE
[    0.006954] Secure boot disabled
[    0.006962] ACPI: Early table checksum verification disabled
[    0.006967] ACPI: RSDP 0x00000000795A5000 000024 (v02 ALASKA)
[    0.006972] ACPI: XSDT 0x00000000795A50B0 0000D4 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.006981] ACPI: FACP 0x00000000795AD9B0 000114 (v06 ALASKA A M I    01072009 AMI  00010013)
[    0.006989] ACPI: DSDT 0x00000000795A5230 008776 (v02 ALASKA A M I    01072009 INTL 20160930)
[    0.006995] ACPI: FACS 0x000000007960A080 000040
[    0.006999] ACPI: FPDT 0x00000000795ADAD0 000044 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.007004] ACPI: FIDT 0x00000000795ADB20 00009C (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.007009] ACPI: MCFG 0x00000000795ADBC0 00003C (v01 ALASKA A M I    01072009 MSFT 00000097)
[    0.007013] ACPI: DBG2 0x00000000795ADC00 000072 (v00 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007018] ACPI: DBGP 0x00000000795ADC80 000034 (v01 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007023] ACPI: HPET 0x00000000795ADCC0 000038 (v01 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007027] ACPI: LPIT 0x00000000795ADD00 00005C (v01 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007032] ACPI: APIC 0x00000000795ADD60 000084 (v04 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007036] ACPI: NPKT 0x00000000795ADDF0 000065 (v01 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007041] ACPI: SSDT 0x00000000795ADE60 0010B3 (v02 INTEL  UsbCTabl 00000003 BRXT 0100000D)
[    0.007046] ACPI: SSDT 0x00000000795AEF20 00014E (v01 Intel_ Platform 00001000 INTL 20160930)
[    0.007051] ACPI: SSDT 0x00000000795AF070 0000B1 (v01 Intel_ ADebTabl 00001000 INTL 20160930)
[    0.007056] ACPI: SSDT 0x00000000795AF130 0005BC (v02 PmRef  Cpu0Ist  00003000 INTL 20160930)
[    0.007061] ACPI: SSDT 0x00000000795AF6F0 000775 (v02 CpuRef CpuSsdt  00003000 INTL 20160930)
[    0.007066] ACPI: SSDT 0x00000000795AFE70 00035F (v02 PmRef  Cpu0Tst  00003000 INTL 20160930)
[    0.007071] ACPI: SSDT 0x00000000795B01D0 0001E6 (v02 PmRef  ApTst    00003000 INTL 20160930)
[    0.007076] ACPI: SSDT 0x00000000795B03C0 00286F (v02 SaSsdt SaSsdt   00003000 INTL 20160930)
[    0.007081] ACPI: BGRT 0x00000000795B2C30 000038 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.007085] ACPI: DMAR 0x00000000795B2C70 0000A8 (v01 INTEL  GLK-SOC  00000003 BRXT 0100000D)
[    0.007090] ACPI: WDAT 0x00000000795B2D20 000104 (v01                 00000000      00000000)
[    0.007095] ACPI: WSMT 0x00000000795B2E30 000028 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.007105] ACPI: Local APIC address 0xfee00000
[    0.007115] Reserving 48MB of memory at 64MB for crashkernel (System RAM: 16201MB)
[    0.007148] Zone ranges:
[    0.007150]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.007152]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.007155]   Normal   [mem 0x0000000100000000-0x000000047fffffff]
[    0.007157] Movable zone start for each node
[    0.007158] Early memory node ranges
[    0.007160]   node   0: [mem 0x0000000000001000-0x000000000003dfff]
[    0.007162]   node   0: [mem 0x0000000000040000-0x000000000009efff]
[    0.007163]   node   0: [mem 0x0000000000100000-0x000000000fffffff]
[    0.007165]   node   0: [mem 0x0000000012151000-0x00000000758f1fff]
[    0.007166]   node   0: [mem 0x0000000079146000-0x000000007929dfff]
[    0.007168]   node   0: [mem 0x0000000079a20000-0x000000007a775fff]
[    0.007169]   node   0: [mem 0x000000007a822000-0x000000007abfffff]
[    0.007171]   node   0: [mem 0x0000000100000000-0x000000047fffffff]
[    0.007926] Zeroed struct page in unavailable ranges: 46647 pages
[    0.007929] Initmem setup node 0 [mem 0x0000000000001000-0x000000047fffffff]
[    0.007931] On node 0 totalpages: 4147657
[    0.007933]   DMA zone: 64 pages used for memmap
[    0.007934]   DMA zone: 156 pages reserved
[    0.007937]   DMA zone: 3996 pages, LIFO batch:0
[    0.008020]   DMA32 zone: 7401 pages used for memmap
[    0.008022]   DMA32 zone: 473645 pages, LIFO batch:63
[    0.018586]   Normal zone: 57344 pages used for memmap
[    0.018588]   Normal zone: 3670016 pages, LIFO batch:63
[    0.093187] Reserving Intel graphics memory at [mem 0x7c000000-0x7fffffff]
[    0.093490] ACPI: PM-Timer IO Port: 0x408
[    0.093493] ACPI: Local APIC address 0xfee00000
[    0.093501] ACPI: LAPIC_NMI (acpi_id[0x01] high level lint[0x1])
[    0.093503] ACPI: LAPIC_NMI (acpi_id[0x02] high level lint[0x1])
[    0.093504] ACPI: LAPIC_NMI (acpi_id[0x03] high level lint[0x1])
[    0.093506] ACPI: LAPIC_NMI (acpi_id[0x04] high level lint[0x1])
[    0.093540] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-119
[    0.093543] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.093546] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[    0.093548] ACPI: IRQ0 used by override.
[    0.093550] ACPI: IRQ9 used by override.
[    0.093554] Using ACPI (MADT) for SMP configuration information
[    0.093556] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.093562] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.093601] [mem 0x80000000-0xcfffffff] available for PCI devices
[    0.093604] Booting paravirtualized kernel on bare hardware
[    0.093608] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.093616] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:4 nr_node_ids:1
[    0.094006] percpu: Embedded 53 pages/cpu s153240 r32768 d31080 u524288
[    0.094017] pcpu-alloc: s153240 r32768 d31080 u524288 alloc=1*2097152
[    0.094019] pcpu-alloc: [0] 0 1 2 3 
[    0.094054] Built 1 zonelists, mobility grouping on.  Total pages: 4082692
[    0.094057] Kernel command line: crashkernel=48M@64M gpt 
[    0.096570] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.097815] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.097865] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.189982] Memory: 16136352K/16590628K available (8194K kernel code, 775K rwdata, 1460K rodata, 5804K init, 1556K bss, 454276K reserved, 0K cma-reserved)
[    0.190052] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.190119] rcu: Hierarchical RCU implementation.
[    0.190121] rcu: 	RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=4.
[    0.190124] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.190125] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.190151] NR_IRQS: 33024, nr_irqs: 1024, preallocated irqs: 16
[    0.190940] Console: colour dummy device 80x25
[    0.190974] printk: console [tty0] enabled
[    0.190983] ACPI: Core revision 20200110
[    0.191237] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 99544814920 ns
[    0.191340] APIC: Switch to symmetric I/O mode setup
[    0.196235] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.241303] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x3990bec8342, max_idle_ns: 881590769617 ns
[    0.241308] Calibrating delay loop (skipped), value calculated using timer frequency.. 3993.60 BogoMIPS (lpj=19968000)
[    0.241312] pid_max: default: 32768 minimum: 301
[    0.244079] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.244121] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.244388] x86/cpu: User Mode Instruction Prevention (UMIP) activated
[    0.244406] process: using mwait in idle threads
[    0.244410] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.244411] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.244414] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.244416] Spectre V2 : Mitigation: Enhanced IBRS
[    0.244418] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.244420] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.244422] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.244547] Freeing SMP alternatives memory: 12K
[    0.244645] TSC deadline timer enabled
[    0.244654] smpboot: CPU0: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz (family: 0x6, model: 0x7a, stepping: 0x8)
[    0.244779] Performance Events: PEBS fmt3+, Goldmont plus events, 32-deep LBR, full-width counters, Intel PMU driver.
[    0.244791] ... version:                4
[    0.244792] ... bit width:              48
[    0.244793] ... generic registers:      4
[    0.244795] ... value mask:             0000ffffffffffff
[    0.244796] ... max period:             00007fffffffffff
[    0.244797] ... fixed-purpose events:   3
[    0.244798] ... event mask:             000000070000000f
[    0.244843] rcu: Hierarchical SRCU implementation.
[    0.245146] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[    0.245214] smp: Bringing up secondary CPUs ...
[    0.245313] x86: Booting SMP configuration:
[    0.245316] .... node  #0, CPUs:      #1 #2 #3
[    0.248374] smp: Brought up 1 node, 4 CPUs
[    0.248377] smpboot: Max logical packages: 1
[    0.248380] smpboot: Total of 4 processors activated (15974.40 BogoMIPS)
[    0.249376] devtmpfs: initialized
[    0.249639] random: get_random_bytes called from _stext+0x493077/0x569958 with crng_init=0
[    0.249731] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.249737] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.249814] thermal_sys: Registered thermal governor 'step_wise'
[    0.249991] NET: Registered protocol family 16
[    0.250175] cpuidle: using governor ladder
[    0.250250] ACPI: bus type PCI registered
[    0.250252] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.250311] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.250315] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
[    0.250324] PCI: Using configuration type 1 for base access
[    0.251307] cryptd: max_cpu_qlen set to 1000
[    0.251307] ACPI: Added _OSI(Module Device)
[    0.251307] ACPI: Added _OSI(Processor Device)
[    0.251307] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.251307] ACPI: Added _OSI(Processor Aggregator Device)
[    0.251307] ACPI: Added _OSI(Linux-Dell-Video)
[    0.251307] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.251307] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.261435] ACPI: 9 ACPI AML tables successfully acquired and loaded
[    0.322209] ACPI: Dynamic OEM Table Load:
[    0.322220] ACPI: SSDT 0xFFFF88846D8A8400 000190 (v02 PmRef  Cpu0Cst  00003001 INTL 20160930)
[    0.323213] ACPI: Dynamic OEM Table Load:
[    0.323221] ACPI: SSDT 0xFFFF88846D8A8600 0001E6 (v02 PmRef  ApIst    00003000 INTL 20160930)
[    0.323996] ACPI: Dynamic OEM Table Load:
[    0.324003] ACPI: SSDT 0xFFFF88846E2FBC00 0000C9 (v02 PmRef  ApCst    00003000 INTL 20160930)
[    0.325810] ACPI: Interpreter enabled
[    0.325822] ACPI: (supports S0 S5)
[    0.325823] ACPI: Using IOAPIC for interrupt routing
[    0.325869] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.326233] ACPI: Enabled 12 GPEs in block 00 to 7F
[    0.331587] ACPI: Power Resource [WRST] (on)
[    0.335521] ACPI: Power Resource [FN00] (on)
[    0.336382] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.336390] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI HPX-Type3]
[    0.336484] acpi PNP0A08:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    0.337159] PCI host bridge to bus 0000:00
[    0.337163] pci_bus 0000:00: root bus resource [io  0x0070-0x0077]
[    0.337166] pci_bus 0000:00: root bus resource [io  0x0000-0x006f window]
[    0.337168] pci_bus 0000:00: root bus resource [io  0x0078-0x0cf7 window]
[    0.337171] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.337174] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.337176] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000dffff window]
[    0.337178] pci_bus 0000:00: root bus resource [mem 0x000e0000-0x000fffff window]
[    0.337181] pci_bus 0000:00: root bus resource [mem 0x7c000001-0x7fffffff window]
[    0.337183] pci_bus 0000:00: root bus resource [mem 0x80000000-0xcfffffff window]
[    0.337185] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xefffffff window]
[    0.337187] pci_bus 0000:00: root bus resource [mem 0xfea00000-0xfeafffff window]
[    0.337190] pci_bus 0000:00: root bus resource [mem 0xfed00000-0xfed003ff window]
[    0.337192] pci_bus 0000:00: root bus resource [mem 0xfed01000-0xfed01fff window]
[    0.337194] pci_bus 0000:00: root bus resource [mem 0xfed03000-0xfed03fff window]
[    0.337197] pci_bus 0000:00: root bus resource [mem 0xfed06000-0xfed06fff window]
[    0.337199] pci_bus 0000:00: root bus resource [mem 0xfed08000-0xfed09fff window]
[    0.337201] pci_bus 0000:00: root bus resource [mem 0xfed80000-0xfedbffff window]
[    0.337203] pci_bus 0000:00: root bus resource [mem 0xfed1c000-0xfed1cfff window]
[    0.337206] pci_bus 0000:00: root bus resource [mem 0xfee00000-0xfeefffff window]
[    0.337209] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.337226] pci 0000:00:00.0: [8086:31f0] type 00 class 0x060000
[    0.337607] pci 0000:00:02.0: [8086:3185] type 00 class 0x030000
[    0.337622] pci 0000:00:02.0: reg 0x10: [mem 0xa0000000-0xa0ffffff 64bit]
[    0.337629] pci 0000:00:02.0: reg 0x18: [mem 0x90000000-0x9fffffff 64bit pref]
[    0.337635] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
[    0.337652] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.338042] pci 0000:00:0e.0: [8086:3198] type 00 class 0x040300
[    0.338069] pci 0000:00:0e.0: reg 0x10: [mem 0xa1710000-0xa1713fff 64bit]
[    0.338097] pci 0000:00:0e.0: reg 0x20: [mem 0xa1000000-0xa10fffff 64bit]
[    0.338158] pci 0000:00:0e.0: PME# supported from D0 D3hot D3cold
[    0.338831] pci 0000:00:0f.0: [8086:319a] type 00 class 0x078000
[    0.338864] pci 0000:00:0f.0: reg 0x10: [mem 0xa1723000-0xa1723fff 64bit]
[    0.338987] pci 0000:00:0f.0: PME# supported from D3hot
[    0.339337] pci 0000:00:12.0: [8086:31e3] type 00 class 0x010601
[    0.339357] pci 0000:00:12.0: reg 0x10: [mem 0xa1714000-0xa1715fff]
[    0.339366] pci 0000:00:12.0: reg 0x14: [mem 0xa1722000-0xa17220ff]
[    0.339374] pci 0000:00:12.0: reg 0x18: [io  0xf090-0xf097]
[    0.339383] pci 0000:00:12.0: reg 0x1c: [io  0xf080-0xf083]
[    0.339391] pci 0000:00:12.0: reg 0x20: [io  0xf060-0xf07f]
[    0.339400] pci 0000:00:12.0: reg 0x24: [mem 0xa1721000-0xa17217ff]
[    0.339459] pci 0000:00:12.0: PME# supported from D3hot
[    0.339772] pci 0000:00:13.0: [8086:31d8] type 01 class 0x060400
[    0.339846] pci 0000:00:13.0: PME# supported from D0 D3hot D3cold
[    0.340199] pci 0000:00:13.2: [8086:31da] type 01 class 0x060400
[    0.340293] pci 0000:00:13.2: PME# supported from D0 D3hot D3cold
[    0.340628] pci 0000:00:13.3: [8086:31db] type 01 class 0x060400
[    0.340703] pci 0000:00:13.3: PME# supported from D0 D3hot D3cold
[    0.341055] pci 0000:00:14.0: [8086:31d6] type 01 class 0x060400
[    0.341129] pci 0000:00:14.0: PME# supported from D0 D3hot D3cold
[    0.341468] pci 0000:00:14.1: [8086:31d7] type 01 class 0x060400
[    0.341545] pci 0000:00:14.1: PME# supported from D0 D3hot D3cold
[    0.341927] pci 0000:00:15.0: [8086:31a8] type 00 class 0x0c0330
[    0.341952] pci 0000:00:15.0: reg 0x10: [mem 0xa1700000-0xa170ffff 64bit]
[    0.342024] pci 0000:00:15.0: PME# supported from D3hot D3cold
[    0.342367] pci 0000:00:17.0: [8086:31b4] type 00 class 0x118000
[    0.342393] pci 0000:00:17.0: reg 0x10: [mem 0xa1720000-0xa1720fff 64bit]
[    0.342425] pci 0000:00:17.0: reg 0x18: [mem 0xa171f000-0xa171ffff 64bit]
[    0.342782] pci 0000:00:17.1: [8086:31b6] type 00 class 0x118000
[    0.342808] pci 0000:00:17.1: reg 0x10: [mem 0xa171e000-0xa171efff 64bit]
[    0.342822] pci 0000:00:17.1: reg 0x18: [mem 0xa171d000-0xa171dfff 64bit]
[    0.343189] pci 0000:00:17.2: [8086:31b8] type 00 class 0x118000
[    0.343214] pci 0000:00:17.2: reg 0x10: [mem 0xa171c000-0xa171cfff 64bit]
[    0.343228] pci 0000:00:17.2: reg 0x18: [mem 0xa171b000-0xa171bfff 64bit]
[    0.343586] pci 0000:00:17.3: [8086:31ba] type 00 class 0x118000
[    0.343611] pci 0000:00:17.3: reg 0x10: [mem 0xa171a000-0xa171afff 64bit]
[    0.343625] pci 0000:00:17.3: reg 0x18: [mem 0xa1719000-0xa1719fff 64bit]
[    0.344016] pci 0000:00:1c.0: [8086:31cc] type 00 class 0x080501
[    0.344045] pci 0000:00:1c.0: reg 0x10: [mem 0xa1718000-0xa1718fff 64bit]
[    0.344061] pci 0000:00:1c.0: reg 0x18: [mem 0xa1717000-0xa1717fff 64bit]
[    0.344461] pci 0000:00:1f.0: [8086:31e8] type 00 class 0x060100
[    0.344917] pci 0000:00:1f.1: [8086:31d4] type 00 class 0x0c0500
[    0.345004] pci 0000:00:1f.1: reg 0x10: [mem 0xa1716000-0xa17160ff 64bit]
[    0.345085] pci 0000:00:1f.1: reg 0x20: [io  0xf040-0xf05f]
[    0.345575] pci 0000:01:00.0: [15b3:1015] type 00 class 0x020000
[    0.345784] pci 0000:01:00.0: reg 0x10: [mem 0x84000000-0x85ffffff 64bit pref]
[    0.346085] pci 0000:01:00.0: reg 0x30: [mem 0xa1200000-0xa12fffff pref]
[    0.346696] pci 0000:01:00.0: PME# supported from D3cold
[    0.347070] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x2 link at 0000:00:13.0 (capable of 63.008 Gb/s with 8 GT/s x8 link)
[    0.347205] pci 0000:01:00.1: [15b3:1015] type 00 class 0x020000
[    0.347403] pci 0000:01:00.1: reg 0x10: [mem 0x82000000-0x83ffffff 64bit pref]
[    0.347706] pci 0000:01:00.1: reg 0x30: [mem 0xa1100000-0xa11fffff pref]
[    0.348275] pci 0000:01:00.1: PME# supported from D3cold
[    0.348624] pci 0000:00:13.0: PCI bridge to [bus 01]
[    0.348630] pci 0000:00:13.0:   bridge window [mem 0xa1100000-0xa12fffff]
[    0.348635] pci 0000:00:13.0:   bridge window [mem 0x82000000-0x85ffffff 64bit pref]
[    0.348694] pci 0000:02:00.0: [10ec:8125] type 00 class 0x020000
[    0.348731] pci 0000:02:00.0: reg 0x10: [io  0xe000-0xe0ff]
[    0.348758] pci 0000:02:00.0: reg 0x18: [mem 0xa1600000-0xa160ffff 64bit]
[    0.348776] pci 0000:02:00.0: reg 0x20: [mem 0xa1610000-0xa1613fff 64bit]
[    0.348893] pci 0000:02:00.0: supports D1 D2
[    0.348895] pci 0000:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.349007] pci 0000:00:13.2: PCI bridge to [bus 02]
[    0.349011] pci 0000:00:13.2:   bridge window [io  0xe000-0xefff]
[    0.349015] pci 0000:00:13.2:   bridge window [mem 0xa1600000-0xa16fffff]
[    0.349076] pci 0000:03:00.0: [10ec:8125] type 00 class 0x020000
[    0.349113] pci 0000:03:00.0: reg 0x10: [io  0xd000-0xd0ff]
[    0.349139] pci 0000:03:00.0: reg 0x18: [mem 0xa1500000-0xa150ffff 64bit]
[    0.349157] pci 0000:03:00.0: reg 0x20: [mem 0xa1510000-0xa1513fff 64bit]
[    0.349265] pci 0000:03:00.0: supports D1 D2
[    0.349267] pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.349382] pci 0000:00:13.3: PCI bridge to [bus 03]
[    0.349387] pci 0000:00:13.3:   bridge window [io  0xd000-0xdfff]
[    0.349391] pci 0000:00:13.3:   bridge window [mem 0xa1500000-0xa15fffff]
[    0.349457] pci 0000:04:00.0: [10ec:8125] type 00 class 0x020000
[    0.349514] pci 0000:04:00.0: reg 0x10: [io  0xc000-0xc0ff]
[    0.349541] pci 0000:04:00.0: reg 0x18: [mem 0xa1400000-0xa140ffff 64bit]
[    0.349559] pci 0000:04:00.0: reg 0x20: [mem 0xa1410000-0xa1413fff 64bit]
[    0.349662] pci 0000:04:00.0: supports D1 D2
[    0.349664] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.349781] pci 0000:00:14.0: PCI bridge to [bus 04]
[    0.349785] pci 0000:00:14.0:   bridge window [io  0xc000-0xcfff]
[    0.349789] pci 0000:00:14.0:   bridge window [mem 0xa1400000-0xa14fffff]
[    0.349855] pci 0000:05:00.0: [10ec:8125] type 00 class 0x020000
[    0.349891] pci 0000:05:00.0: reg 0x10: [io  0xb000-0xb0ff]
[    0.349921] pci 0000:05:00.0: reg 0x18: [mem 0xa1300000-0xa130ffff 64bit]
[    0.349939] pci 0000:05:00.0: reg 0x20: [mem 0xa1310000-0xa1313fff 64bit]
[    0.350061] pci 0000:05:00.0: supports D1 D2
[    0.350063] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.350182] pci 0000:00:14.1: PCI bridge to [bus 05]
[    0.350186] pci 0000:00:14.1:   bridge window [io  0xb000-0xbfff]
[    0.350190] pci 0000:00:14.1:   bridge window [mem 0xa1300000-0xa13fffff]
[    0.350216] pci_bus 0000:00: on NUMA node 0
[    0.350583] ACPI: PCI Interrupt Link [LNKA] (IRQs *3 4 5 6 10 11 12 14 15)
[    0.350682] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 *4 5 6 10 11 12 14 15)
[    0.350777] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)
[    0.350872] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)
[    0.350966] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 10 11 12 14 15) *7
[    0.351061] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 11 12 14 15) *9
[    0.351155] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 *10 11 12 14 15)
[    0.351249] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 10 *11 12 14 15)
[    0.351942] iommu: Default domain type: Translated 
[    0.351996] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.352000] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.352005] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.352007] vgaarb: loaded
[    0.352079] SCSI subsystem initialized
[    0.352136] libata version 3.00 loaded.
[    0.352138] ACPI: bus type USB registered
[    0.352159] usbcore: registered new interface driver usbfs
[    0.352170] usbcore: registered new interface driver hub
[    0.352190] usbcore: registered new device driver usb
[    0.352211] EDAC MC: Ver: 3.0.0
[    0.352390] Registered efivars operations
[    0.352454] PCI: Using ACPI for IRQ routing
[    0.381548] PCI: pci_cache_line_size set to 64 bytes
[    0.381697] e820: reserve RAM buffer [mem 0x0003e000-0x0003ffff]
[    0.381699] e820: reserve RAM buffer [mem 0x0009f000-0x0009ffff]
[    0.381701] e820: reserve RAM buffer [mem 0x6b6d3018-0x6bffffff]
[    0.381703] e820: reserve RAM buffer [mem 0x6b6fd018-0x6bffffff]
[    0.381704] e820: reserve RAM buffer [mem 0x6b727018-0x6bffffff]
[    0.381706] e820: reserve RAM buffer [mem 0x749cb000-0x77ffffff]
[    0.381708] e820: reserve RAM buffer [mem 0x758f2000-0x77ffffff]
[    0.381709] e820: reserve RAM buffer [mem 0x7929e000-0x7bffffff]
[    0.381711] e820: reserve RAM buffer [mem 0x7a776000-0x7bffffff]
[    0.381713] e820: reserve RAM buffer [mem 0x7ac00000-0x7bffffff]
[    0.381880] clocksource: Switched to clocksource tsc-early
[    0.381956] pnp: PnP ACPI init
[    0.382045] system 00:00: [io  0x0680-0x069f] has been reserved
[    0.382048] system 00:00: [io  0x0400-0x047f] could not be reserved
[    0.382051] system 00:00: [io  0x0500-0x05fe] has been reserved
[    0.382054] system 00:00: [io  0x0600-0x061f] has been reserved
[    0.382056] system 00:00: [io  0x164e-0x164f] has been reserved
[    0.382064] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.382321] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    0.382324] system 00:01: [mem 0xfea00000-0xfeafffff] has been reserved
[    0.382327] system 00:01: [mem 0xfed01000-0xfed01fff] has been reserved
[    0.382329] system 00:01: [mem 0xfed03000-0xfed03fff] has been reserved
[    0.382333] system 00:01: [mem 0xfed06000-0xfed06fff] has been reserved
[    0.382336] system 00:01: [mem 0xfed08000-0xfed09fff] has been reserved
[    0.382338] system 00:01: [mem 0xfed80000-0xfedbffff] has been reserved
[    0.382341] system 00:01: [mem 0xfed1c000-0xfed1cfff] has been reserved
[    0.382344] system 00:01: [mem 0xfee00000-0xfeefffff] could not be reserved
[    0.382350] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.382561] pnp 00:02: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.382740] pnp: PnP ACPI: found 3 devices
[    0.389245] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.389259] pci 0000:00:13.0: PCI bridge to [bus 01]
[    0.389265] pci 0000:00:13.0:   bridge window [mem 0xa1100000-0xa12fffff]
[    0.389269] pci 0000:00:13.0:   bridge window [mem 0x82000000-0x85ffffff 64bit pref]
[    0.389274] pci 0000:00:13.2: PCI bridge to [bus 02]
[    0.389278] pci 0000:00:13.2:   bridge window [io  0xe000-0xefff]
[    0.389283] pci 0000:00:13.2:   bridge window [mem 0xa1600000-0xa16fffff]
[    0.389289] pci 0000:00:13.3: PCI bridge to [bus 03]
[    0.389292] pci 0000:00:13.3:   bridge window [io  0xd000-0xdfff]
[    0.389297] pci 0000:00:13.3:   bridge window [mem 0xa1500000-0xa15fffff]
[    0.389303] pci 0000:00:14.0: PCI bridge to [bus 04]
[    0.389307] pci 0000:00:14.0:   bridge window [io  0xc000-0xcfff]
[    0.389311] pci 0000:00:14.0:   bridge window [mem 0xa1400000-0xa14fffff]
[    0.389317] pci 0000:00:14.1: PCI bridge to [bus 05]
[    0.389321] pci 0000:00:14.1:   bridge window [io  0xb000-0xbfff]
[    0.389325] pci 0000:00:14.1:   bridge window [mem 0xa1300000-0xa13fffff]
[    0.389333] pci_bus 0000:00: resource 4 [io  0x0070-0x0077]
[    0.389335] pci_bus 0000:00: resource 5 [io  0x0000-0x006f window]
[    0.389338] pci_bus 0000:00: resource 6 [io  0x0078-0x0cf7 window]
[    0.389340] pci_bus 0000:00: resource 7 [io  0x0d00-0xffff window]
[    0.389342] pci_bus 0000:00: resource 8 [mem 0x000a0000-0x000bffff window]
[    0.389345] pci_bus 0000:00: resource 9 [mem 0x000c0000-0x000dffff window]
[    0.389347] pci_bus 0000:00: resource 10 [mem 0x000e0000-0x000fffff window]
[    0.389349] pci_bus 0000:00: resource 11 [mem 0x7c000001-0x7fffffff window]
[    0.389352] pci_bus 0000:00: resource 12 [mem 0x80000000-0xcfffffff window]
[    0.389354] pci_bus 0000:00: resource 13 [mem 0xe0000000-0xefffffff window]
[    0.389356] pci_bus 0000:00: resource 14 [mem 0xfea00000-0xfeafffff window]
[    0.389358] pci_bus 0000:00: resource 15 [mem 0xfed00000-0xfed003ff window]
[    0.389361] pci_bus 0000:00: resource 16 [mem 0xfed01000-0xfed01fff window]
[    0.389363] pci_bus 0000:00: resource 17 [mem 0xfed03000-0xfed03fff window]
[    0.389365] pci_bus 0000:00: resource 18 [mem 0xfed06000-0xfed06fff window]
[    0.389368] pci_bus 0000:00: resource 19 [mem 0xfed08000-0xfed09fff window]
[    0.389370] pci_bus 0000:00: resource 20 [mem 0xfed80000-0xfedbffff window]
[    0.389372] pci_bus 0000:00: resource 21 [mem 0xfed1c000-0xfed1cfff window]
[    0.389374] pci_bus 0000:00: resource 22 [mem 0xfee00000-0xfeefffff window]
[    0.389377] pci_bus 0000:01: resource 1 [mem 0xa1100000-0xa12fffff]
[    0.389379] pci_bus 0000:01: resource 2 [mem 0x82000000-0x85ffffff 64bit pref]
[    0.389381] pci_bus 0000:02: resource 0 [io  0xe000-0xefff]
[    0.389384] pci_bus 0000:02: resource 1 [mem 0xa1600000-0xa16fffff]
[    0.389386] pci_bus 0000:03: resource 0 [io  0xd000-0xdfff]
[    0.389388] pci_bus 0000:03: resource 1 [mem 0xa1500000-0xa15fffff]
[    0.389390] pci_bus 0000:04: resource 0 [io  0xc000-0xcfff]
[    0.389392] pci_bus 0000:04: resource 1 [mem 0xa1400000-0xa14fffff]
[    0.389395] pci_bus 0000:05: resource 0 [io  0xb000-0xbfff]
[    0.389397] pci_bus 0000:05: resource 1 [mem 0xa1300000-0xa13fffff]
[    0.389514] NET: Registered protocol family 2
[    0.389675] tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    0.389707] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.390021] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.390213] TCP: Hash tables configured (established 131072 bind 65536)
[    0.390257] UDP hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.390317] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.390443] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.391074] PCI: CLS 64 bytes, default 64
[    0.417263] DMAR: Host address width 39
[    0.417267] DMAR: DRHD base: 0x000000fed64000 flags: 0x0
[    0.417290] DMAR: dmar0: reg_base_addr fed64000 ver 1:0 cap 1c0000c40660462 ecap 9e2ff0505e
[    0.417292] DMAR: DRHD base: 0x000000fed65000 flags: 0x1
[    0.417301] DMAR: dmar1: reg_base_addr fed65000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.417304] DMAR: RMRR base: 0x000000790e4000 end: 0x00000079103fff
[    0.417306] DMAR: RMRR base: 0x0000007b800000 end: 0x0000007fffffff
[    0.417325] DMAR: No ATSR found
[    0.417383] DMAR: dmar0: Using Queued invalidation
[    0.417393] DMAR: dmar1: Using Queued invalidation
[    0.417797] pci 0000:00:00.0: Adding to iommu group 0
[    0.428692] pci 0000:00:02.0: Adding to iommu group 1
[    0.428761] pci 0000:00:0e.0: Adding to iommu group 2
[    0.428836] pci 0000:00:0f.0: Adding to iommu group 3
[    0.428909] pci 0000:00:12.0: Adding to iommu group 4
[    0.428982] pci 0000:00:13.0: Adding to iommu group 5
[    0.429053] pci 0000:00:13.2: Adding to iommu group 6
[    0.429130] pci 0000:00:13.3: Adding to iommu group 7
[    0.429200] pci 0000:00:14.0: Adding to iommu group 8
[    0.429272] pci 0000:00:14.1: Adding to iommu group 9
[    0.429376] pci 0000:00:15.0: Adding to iommu group 10
[    0.429465] pci 0000:00:17.0: Adding to iommu group 11
[    0.429482] pci 0000:00:17.1: Adding to iommu group 11
[    0.429498] pci 0000:00:17.2: Adding to iommu group 11
[    0.429514] pci 0000:00:17.3: Adding to iommu group 11
[    0.429582] pci 0000:00:1c.0: Adding to iommu group 12
[    0.432062] pci 0000:00:1f.0: Adding to iommu group 13
[    0.432081] pci 0000:00:1f.1: Adding to iommu group 13
[    0.432225] pci 0000:01:00.0: Adding to iommu group 14
[    0.432350] pci 0000:01:00.1: Adding to iommu group 15
[    0.432425] pci 0000:02:00.0: Adding to iommu group 16
[    0.432498] pci 0000:03:00.0: Adding to iommu group 17
[    0.432570] pci 0000:04:00.0: Adding to iommu group 18
[    0.432645] pci 0000:05:00.0: Adding to iommu group 19
[    0.432713] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    0.434776] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x3990bec8342, max_idle_ns: 881590769617 ns
[    0.434805] clocksource: Switched to clocksource tsc
[    0.435522] workingset: timestamp_bits=46 max_order=22 bucket_order=0
[    0.437462] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.437554] ntfs3: Max link count 4000
[    0.437576] 9p: Installing v9fs 9p2000 file system support
[    0.441867] NET: Registered protocol family 38
[    0.441882] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.441884] io scheduler mq-deadline registered
[    0.441886] io scheduler kyber registered
[    0.442923] efifb: probing for efifb
[    0.442942] efifb: framebuffer at 0x90000000, using 8100k, total 8100k
[    0.442944] efifb: mode is 1920x1080x32, linelength=7680, pages=1
[    0.442945] efifb: scrolling: redraw
[    0.442948] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    0.443075] Console: switching to colour frame buffer device 240x67
[    0.451293] fb0: EFI VGA frame buffer device
[    0.451399] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    0.451428] ACPI: Power Button [PWRB]
[    0.451522] Monitor-Mwait will be used to enter C-1 state
[    0.451536] Monitor-Mwait will be used to enter C-2 state
[    0.451549] Monitor-Mwait will be used to enter C-3 state
[    0.451558] ACPI: \_SB_.CPU0: Found 3 idle states
[    0.452631] ACPI: \_SB_.CPU1: Found 3 idle states
[    0.453013] ACPI: \_SB_.CPU2: Found 3 idle states
[    0.453353] ACPI: \_SB_.CPU3: Found 3 idle states
[    0.454066] thermal LNXTHERM:00: registered as thermal_zone0
[    0.454069] ACPI: Thermal Zone [TZ01] (38 C)
[    0.454159] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.454669] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    0.454671] AMD-Vi: AMD IOMMUv2 functionality not available on this system
[    0.455248] loop: module loaded
[    0.455313] isci: Intel(R) C600 SAS Controller Driver - version 1.2.0
[    0.455366] megasas: 07.713.01.00-rc1
[    0.455391] mpt3sas version 33.100.00.00 loaded
[    0.455562] 3ware Storage Controller device driver for Linux v1.26.02.003.
[    0.455576] 3ware 9000 Storage Controller device driver for Linux v2.26.02.014.
[    0.455606] VMware PVSCSI driver - version 1.0.7.0-k
[    0.455620] hv_vmbus: registering driver hv_storvsc
[    0.456013] ahci 0000:00:12.0: version 3.0
[    0.466333] ahci 0000:00:12.0: AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    0.466338] ahci 0000:00:12.0: flags: 64bit ncq sntf pm clo only pmp pio slum part deso sadm sds apst 
[    0.466787] scsi host0: ahci
[    0.467023] scsi host1: ahci
[    0.467082] ata1: SATA max UDMA/133 abar m2048@0xa1721000 port 0xa1721100 irq 127
[    0.467086] ata2: SATA max UDMA/133 abar m2048@0xa1721000 port 0xa1721180 irq 127
[    0.468083] scsi host2: pata_legacy
[    0.468133] ata3: PATA max PIO4 cmd 0x1f0 ctl 0x3f6 irq 14
[    0.792960] ata1: SATA link down (SStatus 4 SControl 300)
[    0.961698] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    0.963200] ata2.00: ATA-8: SSD32G, P201903, max UDMA/133
[    0.963208] ata2.00: 61865984 sectors, multi 0: LBA48 
[    0.964305] ata2.00: configured for UDMA/133
[    0.964505] scsi 1:0:0:0: Direct-Access     ATA      SSD32G           903  PQ: 0 ANSI: 5
[    0.964934] sd 1:0:0:0: [sda] 61865984 512-byte logical blocks: (31.7 GB/29.5 GiB)
[    0.964949] sd 1:0:0:0: [sda] Write Protect is off
[    0.964952] sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    0.965016] sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.968462]  sda: sda1 sda2
[    0.968969] sd 1:0:0:0: [sda] Attached SCSI disk
[    1.021902] scsi host2: pata_legacy
[    1.021951] ata4: PATA max PIO4 cmd 0x170 ctl 0x376 irq 15
[    1.231453] Fusion MPT base driver 3.04.20
[    1.231456] Copyright (c) 1999-2008 LSI Corporation
[    1.231462] Fusion MPT SPI Host driver 3.04.20
[    1.231500] Fusion MPT SAS Host driver 3.04.20
[    1.231527] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.231528] ehci-pci: EHCI PCI platform driver
[    1.231542] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.231544] ohci-pci: OHCI PCI platform driver
[    1.231557] uhci_hcd: USB Universal Host Controller Interface driver
[    1.231824] xhci_hcd 0000:00:15.0: xHCI Host Controller
[    1.231832] xhci_hcd 0000:00:15.0: new USB bus registered, assigned bus number 1
[    1.232960] xhci_hcd 0000:00:15.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810
[    1.232966] xhci_hcd 0000:00:15.0: cache line size of 64 is not supported
[    1.233237] hub 1-0:1.0: USB hub found
[    1.233255] hub 1-0:1.0: 9 ports detected
[    1.234734] xhci_hcd 0000:00:15.0: xHCI Host Controller
[    1.234739] xhci_hcd 0000:00:15.0: new USB bus registered, assigned bus number 2
[    1.234743] xhci_hcd 0000:00:15.0: Host supports USB 3.0 SuperSpeed
[    1.234923] hub 2-0:1.0: USB hub found
[    1.234944] hub 2-0:1.0: 7 ports detected
[    1.236106] usbcore: registered new interface driver usb-storage
[    1.236131] usbcore: registered new interface driver usbserial_generic
[    1.236136] usbserial: USB Serial support registered for generic
[    1.236145] usbcore: registered new interface driver ftdi_sio
[    1.236150] usbserial: USB Serial support registered for FTDI USB Serial Device
[    1.236197] i8042: PNP: No PS/2 controller found.
[    1.236263] mousedev: PS/2 mouse device common for all mice
[    1.236383] intel_pstate: Intel P-state driver initializing
[    1.236799] usbcore: registered new interface driver usbhid
[    1.236800] usbhid: USB HID core driver
[    1.236806] hv_utils: Registering HyperV Utility Driver
[    1.236808] hv_vmbus: registering driver hv_utils
[    1.236809] hv_vmbus: registering driver hv_balloon
[    1.236953] NET: Registered protocol family 17
[    1.237015] 9pnet: Installing 9P2000 support
[    1.237023] mpls_gso: MPLS GSO support
[    1.237050] IPI shorthand broadcast: enabled
[    1.237052] SSE version of gcm_enc/dec engaged.
[    1.238405] sched_clock: Marking stable (1247840294, -9443492)->(1242387190, -3990388)
[    1.238450] registered taskstats version 1
[    1.240223] Freeing unused kernel image (initmem) memory: 5804K
[    1.281388] Write protecting the kernel read-only data: 12288k
[    1.282206] Freeing unused kernel image (text/rodata gap) memory: 2044K
[    1.282399] Freeing unused kernel image (rodata/data gap) memory: 588K
[    1.282401] Run /init as init process
[    1.282402]   with arguments:
[    1.282403]     /init
[    1.282404]   with environment:
[    1.282404]     HOME=/
[    1.282405]     TERM=linux
[    1.282406]     crashkernel=48M@64M
[    1.282479] process '/init' started with executable stack
[    1.283879] EXT4-fs (sda2): mounting ext3 file system using the ext4 subsystem
[    1.288781] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
[    1.338636] random: fast init done
[    1.601390] usb 1-3: new high-speed USB device number 2 using xhci_hcd
[    1.632480] hub 1-3:1.0: USB hub found
[    1.632512] hub 1-3:1.0: 4 ports detected
[    2.051419] usb 1-3.3: new full-speed USB device number 3 using xhci_hcd
[    2.291147] input: RAPOO RAPOO 5G Wireless Device as /devices/pci0000:00/0000:00:15.0/usb1/1-3/1-3.3/1-3.3:1.0/0003:24AE:2003.0001/input/input1
[    2.351802] hid-generic 0003:24AE:2003.0001: input: USB HID v1.10 Keyboard [RAPOO RAPOO 5G Wireless Device] on usb-0000:00:15.0-3.3/input0
[    2.355740] input: RAPOO RAPOO 5G Wireless Device Mouse as /devices/pci0000:00/0000:00:15.0/usb1/1-3/1-3.3/1-3.3:1.1/0003:24AE:2003.0002/input/input2
[    2.355843] input: RAPOO RAPOO 5G Wireless Device System Control as /devices/pci0000:00/0000:00:15.0/usb1/1-3/1-3.3/1-3.3:1.1/0003:24AE:2003.0002/input/input3
[    2.421478] input: RAPOO RAPOO 5G Wireless Device Consumer Control as /devices/pci0000:00/0000:00:15.0/usb1/1-3/1-3.3/1-3.3:1.1/0003:24AE:2003.0002/input/input4
[    2.421546] hid-generic 0003:24AE:2003.0002: input,hiddev96: USB HID v1.10 Mouse [RAPOO RAPOO 5G Wireless Device] on usb-0000:00:15.0-3.3/input1
[    2.458242] EXT4-fs (sda2): re-mounted. Opts: (null)
[    2.458247] ext3 filesystem being remounted at /system/flash supports timestamps until 2038 (0x7fffffff)
[    2.461360] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    2.535214] NET: Registered protocol family 1
[    2.556274] logring: default buf size:8192, console_loglevel:1
[    2.563837] printk: console [tty0] disabled
[    2.564410] printk: console [logring0] enabled
[    2.601233] NET: Registered protocol family 10
[    2.601505] Segment Routing with IPv6
[    2.865217] softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
[    2.875528] nf_conntrack: table size 1048576 (4 * 262144)
[    2.883152] skb_bin_update_max_len: 0->1664
[    2.883207] gre: register ipv4
[    2.883210] gre: register ipv6
[    2.889114] sizeof(struct phy):760
[    2.897758] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    3.389958] lm87: unknown parameter 'force_lm87' ignored
[    3.543219] libphy: r8169: probed
[    3.543228] r8169 0000:02:00.0: no dedicated PHY driver found for PHY ID 0x00000000, maybe realtek.ko needs to be added to initramfs?
[    3.543244] ------------[ cut here ]------------
[    3.543251] Kernel BUG at mdiobus_free+0x15/0x32 [libphy@0xffffffffa0141000] [verbose debug info unavailable]
[    3.543257] invalid opcode: 0000 [#1] SMP
[    3.543261] CPU: 0 PID: 160 Comm: modprobed Kdump: loaded Not tainted 5.6.3-64 #1
[    3.543263] Hardware name: HARDKERNEL ODROID-H2/ODROID-H2, BIOS 5.13 07/27/2020
[    3.543267] RIP: 0010:mdiobus_free+0x15/0x32 [libphy@0xffffffffa0141000]
[    3.543271] Code: 10 48 89 ef 89 44 24 04 e8 7d 04 00 00 8b 44 24 04 5a 5b 5d c3 8b 87 98 04 00 00 83 f8 01 75 05 e9 c4 fc ff e0 83 f8 03 74 02 <0f> 0b c7 87 98 04 00 00 04 00 00 00 48 81 c7 a0 04 00 00 e9 ac 4e
[    3.543273] RSP: 0018:ffffc900001e3cb0 EFLAGS: 00010297
[    3.543276] RAX: 0000000000000002 RBX: ffffc900001e3cb8 RCX: 0000000000000000
[    3.543278] RDX: ffff88846d9b1348 RSI: ffff88846991b2f8 RDI: ffff88846bfa6000
[    3.543280] RBP: ffff88846991b2e0 R08: 0000000000000286 R09: ffff88846989aac0
[    3.543282] R10: 0000000000000001 R11: 0000000000000001 R12: ffff88846d9b10a8
[    3.543284] R13: 0000000000000005 R14: ffff88846989aac0 R15: 0000000000000000
[    3.543287] FS:  0000000000000000(0000) GS:ffff88846fc00000(0063) knlGS:00000000f7ffc7a4
[    3.543289] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
[    3.543291] CR2: 0000000030046020 CR3: 000000046abf5000 CR4: 0000000000140eb0
[    3.543292] Call Trace:
[    3.543296]  ? _stext+0x331da0/0x569958
[    3.543299]  ? _stext+0x32eb1d/0x569958
[    3.543302]  ? _stext+0x32ed80/0x569958
[    3.543304]  ? _stext+0x32eee9/0x569958
[    3.543306]  ? _stext+0x32ef91/0x569958
[    3.543308]  ? _stext+0x32ef02/0x569958
[    3.543310]  ? _stext+0x32d253/0x569958
[    3.543312]  ? _stext+0xc49c5/0x569958
[    3.543314]  ? _stext+0x32e174/0x569958
[    3.543316]  ? _stext+0x32f5d9/0x569958
[    3.543318]  ? 0xffffffffa0170000
[    3.543320]  ? _stext+0x2160/0x569958
[    3.543322]  ? _stext+0x141ba5/0x569958
[    3.543324]  ? _stext+0x14443b/0x569958
[    3.543326]  ? _stext+0xed218/0x569958
[    3.543329]  ? _stext+0xef4b3/0x569958
[    3.543332]  ? _stext+0x2ffd/0x569958
[    3.543334]  ? __schedule+0x978a0/0x297195[/quote]

Posted: **Mon Jan 29, 2024 7:31 pm**

MikroTik_RouterOSv7.14beta8_BGP_RouteLeak_VRF.png

Hurrah!

E interact with MikroTik devices since 2007-2008...
And Just now... But it is a beginning.

Now we need to check if the behavior is correct with L3VPN / IPv6...

viewtopic.php?t=181846#p1051300

Posted: **Mon Jan 29, 2024 7:33 pm**

can we get more info about :

*) bgp - allow to leak routes between local VRFs;

how that works?

there is a topic on that. also i spotted the current behaviour in v7.4 but it disappeared and now is back again.
leak from VRF to VRF somehow starts to exchange routing information but forwarding still is in the works. got an open SUP on that (SUP-138970)

viewtopic.php?t=181846#p1051300

Posted: **Tue Jan 30, 2024 1:54 am**

Hello, Mikrotik team!

I would like to say THANK YOU VERY MUCH for this improvement in 7.14 beta8:

*) traffic-flow - use 64bit counters for v9 and IPFIX flows

Waiting for release 😎

I hope we will mark this https://github.com/pavel-odintsov/fastn ... ssues/1000 as resolved very soon.

Posted: **Tue Jan 30, 2024 11:39 am**

I would like to say THANK YOU VERY MUCH for this improvement in 7.14 beta8:
Code: Select all
*) traffic-flow - use 64bit counters for v9 and IPFIX flows

Yes, that is great! I have been asking for that for many years...

Posted: **Tue Jan 30, 2024 12:10 pm**

sooo looong beta :) very good Mikrotik :) so RC should be very soon I gues :)

Posted: **Wed Jan 31, 2024 5:27 pm**

When mixing MTU sizes on VLAN interfaces on the bridge, anything that's not 1500 starts up as 1500, breaking OSPF adjacencies and causing PMTU issues. And now, even if I run my script at startup to "fix" the VLAN interface's MTU (by disabling and enabling all of them >1500), something post-startup changes them back.

I've also seen them revert to 1500 after weeks of uptime due to an unknown event.

(Filed this as SUP-142470, a follow-up to SUP-98026, which was marked closed in July. This is still an issue in 7.13.2 and 7.14b8.)

Posted: **Fri Feb 02, 2024 1:08 pm**

What's new in 7.14beta9 (2024-Feb-01 11:09):

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);
*) arp - added ARP status;
*) bridge - avoid per-VLAN host flushing on HW offloaded bridge;
*) bridge - fixed host flush on BPDU-guard port disable (introduced in v7.14beta3);
*) bridge - improved protocol-mode MSTP functionality;
*) bridge - removed "mst-config-digest" from MSTI menu;
*) bridge - removed MVRP support, the development will continue in v7.15 "beta";
*) certificate - improved certificate validation performance;
*) console - added "show-at-cli-login" option to display a note before telnet login;
*) console - fixed delayed output from ":grep" command in certain cases;
*) console - fixed incorrect behavior of ":onerror" command in certain cases;
*) defconf - added log about configuration reset due to pressed reset button;
*) defconf - fixed Audience scanning-for-wps-ap timeout;
*) defconf - increased LTE interface wait time;
*) defconf - updated health settings on configuration revert;
*) disk - added exFAT and NTFS mount/read/write support;
*) fetch - allow to use certificate and check-certificate parameters only in HTTPS mode;
*) fetch - fixed incorrect "src-path" error message when "upload=yes";
*) fetch - print all "Set-Cookies" headers in response;
*) health - added limited manual control over fans for CCR1016r2, CCR1036r2 devices;
*) health - changed default "fan-min-speed-percent" from 0% to 12%;
*) health - updated health properties for CCR1016r2, CCR1036r2 devices;
*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;
*) leds - fixed modem LED indication for SXT LTE 3-7;
*) lte - fixed APN authentication for FG621-EA modem;
*) lte - fixed Simcom modem support in 0x9000; 0x9002, 0x9002; 0x901a and 0x901b USB compositions;
*) lte - optimized "at-chat" response reading;
*) ovpn - improved system stability when using HW encryption on ARM64 devices (introduced in v7.13);
*) package - added "size" property;
*) package - reduced "wireless" package size for ARM, ARM64, MIPSBE, MMIPS devices;
*) poe-out - driver optimization for AF/AT controlled boards;
*) ptp - fixed "default" and "g8275.1" profiles go into "slave" instead of "uncalibrated" state;
*) ptp - fixed default values for "802.1as" profile;
*) ptp - fixed flags in Announce message;
*) ptp - fixed potential error in packet exchange;
*) ptp - make clock go into grandmaster state if slave port goes down;
*) sfp - fixed corrupted Tx traffic at 10Gbps rate on RB4011 in rare cases;
*) snmp - added "bgpLocalAs" and "bgpIdentifier" OID reporting;
*) snmp - fixed "bgpPeerFsmEstablishedTime" OID reporting;
*) sstp - added support for "aes256-gcm-sha384" encryption;
*) switch - fixed Atheros-8327 switch rules (introduced in v7.14beta3);
*) switch - fixed reserved multicast receive on Atheros-8327, QCA8337 switches for R/STP bridge;
*) system - correctly handle HTTP 1xx and 204 response status codes (introduced in v7.14beta6);
*) system - fixed "cpu-frequency" for CRS3xx ARM devices;
*) system - properly assign destination port for HTTP/S connections initiated by the router (introduced in v7.13);
*) webfig - fixed routing table filter under "IP/Routes" menu;
*) webfig - improved stability when adding new entries under "IP/Routes" menu;
*) wifi - added "station-pseudobridge" interface mode;
*) wifi-qcom - enable display of regulatory information on L11,L22 devices;
*) wifi-qcom - improve system stability for L11, L22 devices;
*) winbox - fixed status under "Bridge/Ports" menu (introduced in v7.14beta3);
*) winbox - improved status values under "System/PTP" menu;

Posted: **Fri Feb 02, 2024 1:22 pm**

Another bundle size increase for exfat and ntfs support?

Posted: **Fri Feb 02, 2024 1:29 pm**

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);

These are mention on various version. Are there any change between beta8 or beta9?
Or this this is a list of change since 7.13???

Posted: **Fri Feb 02, 2024 1:33 pm**

Still no progress on IS-IS ?

Posted: **Fri Feb 02, 2024 1:35 pm**

The ovpn fix-up will be pushed also on the stable tree ?

Posted: **Fri Feb 02, 2024 1:42 pm**

SMB sharing that works with 7.13.3 no longer works with 7.14beta9...

smb.png

Does the drive need to be reformatted?
It is recognized but cannot be accessed in any way.

Posted: **Fri Feb 02, 2024 1:44 pm**

Another bundle size increase for exfat and ntfs support?

No the size is the same as beta8

Posted: **Fri Feb 02, 2024 1:50 pm**

Awesome update. Although since upgrading to the 7.14betaX the wireguard/peers page seems very wonky on webfig it doesn't display properly. Is anyone else seeing this (sorry if it's already been reported)?

Posted: **Fri Feb 02, 2024 2:55 pm**

*) snmp - added "bgpLocalAs" and "bgpIdentifier" OID reporting;
*) snmp - fixed "bgpPeerFsmEstablishedTime" OID reporting;

Finally /me parties

I would love to see IPv6 BGP peers as well...

Posted: **Fri Feb 02, 2024 3:43 pm**

Hi,
after upgrading my CRS326-24S+2Q+ from 7.14Beta8 to Beta9 the memory consumption is very high. Free RAM is between 9-10MB, befor it was ~16-20MB free.
The upgrafe itself didn´t went smooth, the first update failed, and after the second successful try the switch rebootet directly with an “kernel error”.
After the third reboot the switch is now up and running, but with very low free RAM, so I expect in the near future other reboots triggert by Watchdog because of low RAM.

@MT: The overall development in 7.14 is consuming more and more RAM, please keep an eye on this.

Creatring a first supout.rif fails after several minutes without any logs or errors….
The second try was successfull……
Overall for this model no good beta update so far..
Ticket opened! (SUP-142702)

Posted: **Fri Feb 02, 2024 3:51 pm**

Sorry guys, it become less and less useful due to size of basic packages. I have made pure netinstall with netinstall 7.14beta9 with saving configuration and after that only 140 kB was left !!! See pictures.

Posted: **Fri Feb 02, 2024 5:41 pm**

SMB sharing that works with 7.13.3 no longer works with 7.14beta9...

I had/have ROSE installed. So in 7.14beta9, I see "D" dynamic entries for share+user from previous ROSE smb-export... configuration. That works (although does not support reconnection, same as before). But when I add new "static" shares and/or users using winbox /ip/smb — they don't work unless allowed users is EMPTY. Is this expected if using a ROSE raid volume from same device?

Also, the doc re SMB and ROSE are still rather incomplete.

Posted: **Fri Feb 02, 2024 5:42 pm**

7.13.2 --> Free Space = 848 KiB
7.13.3 --> Free Space = 864 KiB
7.14beta8 --> Free Space = 1096 KiB
7.14beta9 --> Free Space = 988 KiB

hAP ac² with the wireless package

Posted: **Fri Feb 02, 2024 6:24 pm**

m1.png

f1.png

Posted: **Fri Feb 02, 2024 6:25 pm**

Just did a quick upgrade to an hAP ac lite with a wireless interface working as station-pseudobridge and the other wireless interface managed by a remote CAPSMAN, from ROS7.13.3 to 7.14beta9.

Lost layer3 connectivity (ping, ssh, telnet,...) mac-telnet and mac-ping working.

Connecting to an ethernet port I could'nt ping to the other network nodes on the other side of the wireless bridge (no route to host)

CAPSMAN was still working (connection is via layer2).

Reverting to v7.13.3 restored layer 3 connectivity...

Posted: **Fri Feb 02, 2024 6:37 pm**

Last beta broke my RB4011 wireless. Now is not working anymore.
Very good job!!

Posted: **Fri Feb 02, 2024 6:40 pm**

Last beta broke my RB4011 wireless. Now is not working anymore.
Very good job!!

Maybe you installed the qcom-ac package without careful reading? It will break the 2GHz WiFi. To have 2GHz WiFi on the 4011 you need to keep the "wireless" package, and you cannot have both "wireless" and "qcom-ac".

Posted: **Fri Feb 02, 2024 8:34 pm**

Beta and production do not go hand in hand. Do not complain!

Posted: **Fri Feb 02, 2024 8:55 pm**

Please split router OS to multiple packages. Make SMB separated package, VPN supports as separated packages etc. There is still less and less free space on hap ac2 and similar 16MB hdd devices. It bought hap ac2 only few years ago. And router is not kind of device people are buying every year or two. Please make it futureproof for people with 16MB storage.

Posted: **Fri Feb 02, 2024 9:12 pm**

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);

While it may be good to retire legacy SMB service from ROS (so I welcome the second bullet) I think that moving SMB service from ROSE to main bundle is a capital failure. With 7.13 MT stirred lots of confusion about wireless packages. So it wouldn't hurt much more to keep SMB in optional package. I'm willing to bet that this change would affect way less users than the change about wireless. And would help way more people in maintaining their 15.3MB devices running trouble-free.

Posted: **Fri Feb 02, 2024 9:28 pm**

Maybe legacy SMB needed more space than the new one. Could that be?

Posted: **Fri Feb 02, 2024 9:34 pm**

DoH is starting too early...

Posted: **Fri Feb 02, 2024 9:36 pm**

It means your router does a DNS query before everything is up and running. Maybe you configured NTP with a DNS name, or some other service like that.
Just ignore it, it will fix itself later.

Posted: **Fri Feb 02, 2024 9:43 pm**

Maybe legacy SMB needed more space than the new one. Could that be?

I think they didn't want two different SMB implementations, one if you installed ROSE and another from the "classic" samba-like one. That part makes sense – although imagine new ROSE-based SMB might break a few folks who used "classic SMB" – since I tried add user & didn't work (see above).

Hopefully they got the message on the "package problem"... Not sure listing feature to break out helps much here – think everyone has an opinion on what they remove. ;). No one here knows there build process/dependancies. The bigger problem is we're at step one with Mikrotik: recognizing the problem.

Posted: **Fri Feb 02, 2024 9:46 pm**

Maybe legacy SMB needed more space than the new one. Could that be?

It doesn't matter. What matters is tgat there's now rose-storage optiobal package which neatly packs various network file sharing protocols (SMB, NFS, iSCSI, etc.) and it's a great opportunity to declare that if somebody wants to use RB as NAS server (including SMB), then this is entirely optional feature. They might even rename rose-storage to "nas" to make clear to everybody what it was (WTH does rose stand for?). Yeah, it'll break plenty of configs, many users will blindly upgrade without reading changelogs. But it would be a relatively clean cut. And a few (tens? hundreds?) of kB more free space for all those who never used SMB would be very welcome. It might be enough to allow to install e.g. zerotier package?

Posted: **Fri Feb 02, 2024 10:15 pm**

It's a great idea to rename to "nas"! I love roses, like well Guns'n'Roses, rose flowers, rose perfume and so on… but how it is related to SMB I really don't understand!

Posted: **Fri Feb 02, 2024 10:38 pm**

ROS
Enterprise Storage

Posted: **Fri Feb 02, 2024 10:46 pm**

Rename it to "storage". Keep it simple, stupid.

Posted: **Fri Feb 02, 2024 10:46 pm**

I am able to achieve a maximum of 464K free HDD space with a clean netinstall and no configuration except for the default scripts. When I stripped the configuration (advanced QoS removal), I'm able to live with 350-400K free HDD. But anything close to 1MB is absolutely unattainable. It's a pity that when reaching 0 KB of free space on the HDD, the router becomes unbootable and a netinstall is required.

Posted: **Sat Feb 03, 2024 12:35 am**

It's a great idea to rename to "nas"! I love roses, like well Guns'n'Roses, rose flowers, rose perfume and so on… but how it is related to SMB I really don't understand!

the rose-storage package does not only offer the "nas" function, but also the "client" function. In fact that is the only part
of it that I am using. With rose-storage you can access a NAS share from within the router, i.e. you can mount a folder within
the router Files section onto a share on your NAS. (that "NAS" of course could be another MikroTik router)

It does not only work with SMB, but also with NFS, iSCSI, and NVMe/TCP.

Posted: **Sat Feb 03, 2024 12:41 am**

It's a great idea to rename to "nas"! I love roses, like well Guns'n'Roses, rose flowers, rose perfume and so on… but how it is related to SMB I really don't understand!
the rose-storage package does not only offer the "nas" function, but also the "client" function. In fact that is the only part
of it that I am using. With rose-storage you can access a NAS share from within the router, i.e. you can mount a folder within
the router Files section onto a share on your NAS. (that "NAS" of course could be another MikroTik router)

It does not only work with SMB, but also with NFS, iSCSI, and NVMe/TCP.

NAS (like QNAP, Synology) can mount another NASes, also it can run Docker, can run Photo Gallery, Video Surveillance, remote access, of course NFS, iSCSI and so on. Anyway, "rose" is counter-intuitive naming. All in all, it's not that important, just "rose" here, "rose" there, and finally we'll stuck with something what nobody will understand :-)

Posted: **Sat Feb 03, 2024 1:17 am**

But why not just: ROSS. ROS Storage.

Posted: **Sat Feb 03, 2024 2:48 am**

It means your router does a DNS query before everything is up and running. Maybe you configured NTP with a DNS name, or some other service like that.
Just ignore it, it will fix itself later.

Yes, I ignore it, the intention here was just to show how something as simple as starting a service at the right time would avoid an error appearing at the beginning of startup.

Regards

Posted: **Sat Feb 03, 2024 5:47 am**

i've been advised that the LTE issues will be fixed in the 7.14 stable version .

Let see how it goes

Posted: **Sat Feb 03, 2024 10:48 am**

Another bundle size increase for exfat and ntfs support?
No the size is the same as beta8

I'm sorry, you're right.

11,619,964 routeros-7.14beta8-arm.npk
11,742,852 routeros-7.14beta9-arm.npk

It's exactly the same size even, i don't know how they managed that.

Posted: **Sat Feb 03, 2024 11:23 am**

Because it was already part of beta8. https://mikrotik.com/download/changelog ... 7c85965dfe

*) disk - added exFAT and NTFS mount/read/write support (CLI only);

Posted: **Sat Feb 03, 2024 11:27 am**

Ah, MikroTik changelogs.
Anyway, the size is obviously not the same, probably another reason for those 120KB.
I've checked the changelog posted above for beta8 and there's no mention of that. viewtopic.php?p=1051007#p1051007
Not that I'm surprised by the missing information from changelogs.

Posted: **Sat Feb 03, 2024 4:57 pm**

But when I add new "static" shares and/or users using winbox /ip/smb — they don't work unless allowed users is EMPTY.

Same also for me, I’m not using ROSE but the build-in SMB and after the update no longer works.
Unfortunately I can't use ROSE due to lack of space.

Posted: **Sat Feb 03, 2024 5:30 pm**

It means your router does a DNS query before everything is up and running. Maybe you configured NTP with a DNS name, or some other service like that.
Just ignore it, it will fix itself later.
Yes, I ignore it, the intention here was just to show how something as simple as starting a service at the right time would avoid an error appearing at the beginning of startup.

It is not as simple as you think. Some of those services may be circularly dependent. E.g. a VPN may require DNS to determine the endpoint, while the user may want to run DNS over his VPN.
I simply do not use DoH, it serves no useful purpose for me.

Posted: **Sat Feb 03, 2024 5:45 pm**

Unfortunately I can't use ROSE due to lack of space.

And that's the brutal irony: You might want to use client parts of ROSE/NAS on a 16MB device to mount an external disk. While now you're getting SMB server parts to potentially share the 16MB flash? The "full" ROSE is only applicable on a few devices, but parts of ROSE/NAS are useful on more but it does not fit.

And a lot of users just like remove features they don't need, yet you cannot. And good reasons to want to: less attack surface, increase available storage, less stuff in the UI, or fit "extra" packages that aren't extra in your use cases.

Posted: **Sun Feb 04, 2024 7:41 pm**

Regarding this

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

How do I enjoy this for hAP-ax3? I tried with ip leds settings but there is not a "dark mode" anywhere (set all leds off option was already there before this release, but it only shutdown front leds).

Thanks!

Posted: **Mon Feb 05, 2024 1:01 pm**

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);

Great move, now remove pptp and l2tp and create a legacy vpn package for it

Posted: **Mon Feb 05, 2024 2:15 pm**

Regarding this

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

How do I enjoy this for hAP-ax3? I tried with ip leds settings but there is not a "dark mode" anywhere (set all leds off option was already there before this release, but the only shutdown front leds).

Thanks!

My God i wonder the same, im so irritated by LEDS going nuts during night, why isnt it possible to make ALL leds(front and ones on LAN ports) go complete off?

Posted: **Mon Feb 05, 2024 2:35 pm**

The easiest solution for irritating LEDs: apply black tape. I have done that "forever" on MikroTik equipment, e.g. because of the blue powerled (torch).

Posted: **Mon Feb 05, 2024 8:53 pm**

Installed beta9 on my CCR2004-1G-2XS-PCIe this morning without success. After powering the system on the two red leds turn off and the Mgmt Ethernet Ports comes up but the Router is not reachable and under my Linux host the interface isn't able to obtain an IP address anymore. I tried to use the reset button but it doesn't do anything. Netinstall failes. 192.168.88.1 isn´t reachable. Tried to pull up the link using an old Hub and a newer Gigabit Switch, as well as connecting it directly to my Laptop without success.
Any advice?

Solved: Solution -> Mac-telnet -> set boot to Ethernet -> netinstall -> back to the latest stable!

Posted: **Tue Feb 06, 2024 3:35 pm**

The ovpn fix-up will be pushed also on the stable tree ?

I think that means.. no :(

Posted: **Tue Feb 06, 2024 5:08 pm**

Is there anything known about a "wifi-mediatek" package?

Posted: **Tue Feb 06, 2024 6:34 pm**

I don't think MikroTik uses any MediaTek wifi chips?

Posted: **Tue Feb 06, 2024 7:05 pm**

Just found this string inside the "wifi-qcom" npk. We'll see whats coming. ;)

Posted: **Tue Feb 06, 2024 7:23 pm**

I found a setting on AX Lite LTE related to esim.

Posted: **Tue Feb 06, 2024 8:02 pm**

I found a setting on AX Lite LTE related to esim.

I think that's been there (at least on other models)... But it's not usable AFAIK & nothing about it in docs.

It maybe related to DW5821e-eSIM modems, but IDK.

Posted: **Tue Feb 06, 2024 11:36 pm**

I don't think MikroTik uses any MediaTek wifi chips?

maybe an upcoming wifi7 device.

Posted: **Wed Feb 07, 2024 12:44 pm**

What's new in 7.14beta10 (2024-Feb-06 15:47):

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);
*) arp - added ARP status;
*) console - improved stability when using autocomplete with "export";
*) defconf - fixed configuration script on KNOT devices if "ppp-out" interface is removed;
*) defconf - fixed firewall rule for IPv6 UDP traceroute;
*) dhcpv6-client - updated error logging when multiple prefixes received on renew;
*) disk - added global disk "settings" menu (CLI only);
*) l3hw - fixed IPv6 host offloading in certain cases;
*) package - added "size" property;
*) poe-out - improved 802.3at classification and measurement accuracy;
*) poe-out - improved cable test for hAP ac3 and hAP ax3 devices;
*) ptp - added "aes67" and "smpte" profiles;
*) ptp - added configurable "domain" and "priority2" parameters;
*) ptp - added support for Management message forwarding in BC;
*) route - fixed gateways of locally imported vpnv4 routes;
*) route-filter - fixed AS path matchers when input and output chains are used;
*) sfp - fixed corrupted Tx traffic at 10Gbps rate on CCR2004-16G-2S+ in rare cases;
*) smb - added option to specify SMB service mode as "auto";
*) supout - added PTP section;
*) system - expose "lo" and "vrf" interfaces;

Posted: **Wed Feb 07, 2024 1:04 pm**

Beta 10? What is happening? :) This will be the long therm ?

Posted: **Wed Feb 07, 2024 1:06 pm**

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);

Build-in SMB still doesn't work with macOS and Windows clients
Maybe I don't understand how it should be used... can you explain it to us?
Thanks

Posted: **Wed Feb 07, 2024 1:12 pm**

Correction, old SMB did not work with macOS.
New v7.14 SMB works with all Operating Systems now. And is fast too.
There is a different config now, so you may need to adjust first.

Posted: **Wed Feb 07, 2024 1:44 pm**

Correction, old SMB did not work with macOS.

You are right (Thanks for listening to us 😊)

There is a different config now, so you may need to adjust first.

So after the update from 7.13.3 is better to delete old SMB share and make a new one?

Posted: **Wed Feb 07, 2024 1:47 pm**

When 7.14 is not in beta, config should be migrated normally. During beta all kind of bugs can be possible, so be careful.

Posted: **Wed Feb 07, 2024 3:39 pm**

*) sfp - fixed corrupted Tx traffic at 10Gbps rate on CCR2004-16G-2S+ in rare cases;

Could we have more details on this issue?

Posted: **Wed Feb 07, 2024 3:55 pm**

Is there anything known about a "wifi-mediatek" package?

That will be for Mikrotik's coming WiFi 7 devices

or potentially for WiFi 6 AP's. The MediaTek chips are more energy efficient and output less heat so as long as they can match the Qualcomm driver stability and feature support would be a great option.

Posted: **Wed Feb 07, 2024 7:18 pm**

Just informing incase it isn't known. Adding wireguard peers is broken on webfig in the 7.14betaX series. The wireguard/peer page is messed up and the add new button doesn't work.

Tested on CCR2216.

Posted: **Wed Feb 07, 2024 7:24 pm**

Correction, old SMB did not work with macOS.
New v7.14 SMB works with all Operating Systems now. And is fast too.

I tested this again. And does seem to work, from RB1100AHx4 with previously existing ROSE+RAID1 to a macOS Sonoma(Intel). ⌘K in Finder, smb://<routeros_ip> then it prompts for share. Both the "dynamic" user from smb-user= and new ones added & new shares. MacOS cache credentials, somewhere/sometime... so maybe that's why I thought it didn't work before. Over 1G ethernet direct to router, 53MB/sec or 420Mb/sec if my math is right (e.g. 1G file transfers in 19 seconds:

time cp /tmp/1gsample.txt /Volumes/raid1-part1
cp /tmp/1gsample.txt /Volumes/raid1-part1  0.00s user 0.86s system 4% cpu 19.227 total

While useful to have users with ROSE shares on RB1100... my poor little 16MB LTE devices didn't need SMB server.

Now if there were only DNS-SD (e.g. mDNS) support, in an extra-package... SMB server/shares WOULD just "magically" appear in "Network" section, and not require a less friendly ⌘K with URL ;)

Posted: **Wed Feb 07, 2024 8:30 pm**

RB1100AHx4 with previously existing ROSE+RAID1

Hi Amm0, does it also work without the ROSE package and without RAID?

Posted: **Wed Feb 07, 2024 9:24 pm**

RB1100AHx4 with previously existing ROSE+RAID1
Hi Amm0, does it also work without the ROSE package and without RAID?

That's a fair question: do the SMB server parts of ROSE bloating the routeros package, actually work without ROSE being installed... Geez, I'd like to think so... But dunno.

Posted: **Wed Feb 07, 2024 10:20 pm**

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

this is joke?

Screenshot_1.png

Screenshot_2.png

photo_2024-02-07_23-18-48.jpg

Posted: **Wed Feb 07, 2024 10:46 pm**

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

this is joke?

Strange.
My AX3 really shuts down ALL leds when I apply that setting, including the big blue one (finally !!).
But I'm still not sure what the blue led is for

EDIT: have to try 7.14b10. I see that AX3 is still on 7.14b8.
EDIT2: pfieuw, same result. All off.

Posted: **Thu Feb 08, 2024 12:57 am**

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

this is joke?

Screenshot_1.png

Screenshot_2.png

photo_2024-02-07_23-18-48.jpg

Aren't all those fields supposed to be enabled before you shut them off?

Posted: **Thu Feb 08, 2024 1:11 am**

What's new in 7.14beta10 (2024-Feb-06 15:47):

!) rose-storage - moved SMB service in the RouterOS bundle;
!) smb - removed legacy SMB service (replaced with newer and faster ROSE SMB service);
*) arp - added ARP status;
*) console - improved stability when using autocomplete with "export";
*) defconf - fixed configuration script on KNOT devices if "ppp-out" interface is removed;
*) defconf - fixed firewall rule for IPv6 UDP traceroute;
*) dhcpv6-client - updated error logging when multiple prefixes received on renew;
*) disk - added global disk "settings" menu (CLI only);
*) l3hw - fixed IPv6 host offloading in certain cases;
*) package - added "size" property;
*) poe-out - improved 802.3at classification and measurement accuracy;
*) poe-out - improved cable test for hAP ac3 and hAP ax3 devices;
*) ptp - added "aes67" and "smpte" profiles;
*) ptp - added configurable "domain" and "priority2" parameters;
*) ptp - added support for Management message forwarding in BC;
*) route - fixed gateways of locally imported vpnv4 routes;
*) route-filter - fixed AS path matchers when input and output chains are used;
*) sfp - fixed corrupted Tx traffic at 10Gbps rate on CCR2004-16G-2S+ in rare cases;
*) smb - added option to specify SMB service mode as "auto";
*) supout - added PTP section;
*) system - expose "lo" and "vrf" interfaces;

unfortunately no BGP/VRF local route leaking updates...

Posted: **Thu Feb 08, 2024 1:21 am**

On beta 8
*) bgp - allow to leak routes between local VRFs;

If you are looking for proper implementation through RD i don't think it will happen today :(

Posted: **Thu Feb 08, 2024 1:27 am**

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;

this is joke?

Screenshot_1.png

Screenshot_2.png

photo_2024-02-07_23-18-48.jpg
Aren't all those fields supposed to be enabled before you shut them off?

I am pretty sure that's the right answer. No joke.

Posted: **Thu Feb 08, 2024 8:47 am**

unfortunately no BGP/VRF local route leaking updates...

*) route - fixed gateways of locally imported vpnv4 routes;

Posted: **Thu Feb 08, 2024 4:51 pm**

Any chance of a Link for 7.14beta8 arm64 plus wifi-qcom I gotta roll back!
Edit: I got it....

Posted: **Thu Feb 08, 2024 6:50 pm**

Aren't all those fields supposed to be enabled before you shut them off?
I am pretty sure that's the right answer. No joke.

its not working

Posted: **Thu Feb 08, 2024 8:07 pm**

It is working for me.

Create supout and contact support.
They should be able to shed some light on it ( oh no, wait, you don't want light...sorry, couldnt resist)

Posted: **Thu Feb 08, 2024 8:12 pm**

They should be able to shed some light on it ( oh no, wait, you don't want light...sorry, couldnt resist)

You ALMOST got it. I think he wanted to shed some light FROM it...

Posted: **Thu Feb 08, 2024 9:25 pm**

Please bring IS-IS ipv6 :(

Posted: **Fri Feb 09, 2024 1:04 am**

It is working for me.

Create supout and contact support.
They should be able to shed some light on it ( oh no, wait, you don't want light...sorry, couldnt resist)

can you make photo in the dark with ax, please?

Posted: **Fri Feb 09, 2024 8:14 am**

Before and after.

The dim blue light shows on the photo (slow shutter because of light conditions) but is not visible at all with naked eye.
Ports on the back are completely off.

Again, if it doesn't work for you, contact support.

Posted: **Fri Feb 09, 2024 9:18 am**

Are the ETH ports still flashing?
I’m trying to understand how this differs from simply scheduling:

/system leds settings set all-leds-off=immediate

Posted: **Fri Feb 09, 2024 9:42 am**

Please bring IS-IS ipv6

Have you even tried?

Posted: **Fri Feb 09, 2024 1:17 pm**

*) leds - added "dark-mode" functionality for hAP ax3 and Chateau ax series devices;
Will this be addd for Hap ax2 as well ?
All my front leds stay on with this... /system leds settings set all-leds-off=immediate

Posted: **Fri Feb 09, 2024 3:49 pm**

I'd like to see this implemented for hap ax2 as well. Could get rid of that black tape...

Posted: **Sat Feb 10, 2024 11:34 pm**

Please bring IS-IS ipv6 :(
Have you even tried?

Ofcourse not.
There is no changelog for IS-IS.
And still flaged as red here,
https://help.mikrotik.com/docs/display/ ... l+Overview

If its ready, at least tell us in changelog.

Posted: **Sun Feb 11, 2024 11:56 am**

Any chance of a Link for 7.14beta8 arm64 plus wifi-qcom I gotta roll back!
Edit: I got it....

As I thought, things on my Hap ax2 are great and wifi is very stable on 7.14beta8 as seen below. But 7.14beta9/10 not great massive SA timeout spam.

Posted: **Sun Feb 11, 2024 6:56 pm**

If its ready, at least tell us in changelog.

Couldn’t agree more

To MikroTik team :

Please allow redistribution of IS-IS routes from other routing protocol

And after IS-IS is complete, hopefully there will be EVPN and SR-MPLS with TI-LFA

Posted: **Sun Feb 11, 2024 7:27 pm**

Couldn’t agree more

To MikroTik team :

Please allow redistribution of IS-IS routes from other routing protocol

And after IS-IS is complete, hopefully there will be EVPN and SR-MPLS with TI-LFA

And hopefully will be SRv6 also..

Posted: **Mon Feb 12, 2024 9:53 am**

Version 7.14rc1 has been released.
viewtopic.php?t=204440

MikroTik

v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!

Re: v7.14beta [testing] is released!