Thanks to a good friend at work we now have an ASUS wifi router with WiFi6 on it to join the network. This is mainly because people were complaining about the wifi stability issues regarding Apple devices and Mikrotik (seems to be a bad combo), and according to this friend they never have issues with this asus router, so there we are.
Either way, at this time we run a separate DNS server for the network (one with piHole and unbound, which we are very happy with),
DHCP is done by the RB4011iGS+5HacQ2HnD, which I would probably also like to keep that way. This RB4011 also is connected to the WAN (an internet modem) right now, but I could change that to be done by the ASUS router (it does have a much faster CPU, and probably higher throughput than the RB4011, but I haven't tested that).
This ASUS RT-AX89X has a WAN port, but since it's mostly to do with its wifi abilities, I'm not sure how to set up the network now.
All ports, including the SFP+ port, of the RB4011 are in use, and 4 of its 1G LAN ports are set up as bound ports (2Gb) to two local servers.

Knowing all this, how would you people implement this ASUS into this network? I could use it as a mere switch and AP, but I think I'll also prefer to have it do NAT and so use ASUS' WAN port, take over that role from the RB4011.
Would then having the RB4011 do DHCP for the entire network make it slow down anything?

And how would I best connect the two routers? I could create a bonding link between the two (2Gb), so that at least it's faster than just one LAN to LAN port. The ASUS offers 802.3ad on its first two LAN-ports. Just found out I can also use the 10G SFP+ port on the ASUS as a LAN port, so that way I can use a DAC SFP+ cable between the RB4011 and the RX89X, sweet!

Either way, I will switch off the WiFi on the RB4011, and use the WiFi6 ability on the AX89X, as a minimum.

Just curious on how any of you would set this up, and why.
(Firewalling to/from the interwebs is not much of an issue, since the internet modem WAN side already does CGNAT, so I'm basically two NATs away from the internet, at all times. I can only run public servers/sevices using ssh port-links through an external server anyway. So security is not much of a worry here, as long as we have no idiots on the network clicking links and/or running stupid IoT devices as botnets etc. but as this is an IT-company, they've never had such issues.)

Since the ASUS RT-model, is not capable of reading vlans, concur that it should be the router and consider getting a managed switch for the rest and giveing the RB4011 to a family member/friend.
THe RB4011 is more than adequate for your routing needs and has far better functional granularity including the use of VLANs, but in this case, not a good fit for the ASUS.
A better focussed purchase (replace WIFI) would have been a business class AP.

A better focussed purchase (replace WIFI) would have been a business class AP.

Agreed. I am running a RB4011 with four Meraki Enterprise access points at home. My kids will tell you that I have the best WiFi around...

You should place the Asus in AP mode only … this turns of the NAT and router functions of the Asus. Then you can exploit the Ethernet ports on the Asus … the WiFi of the Asus will become available to everyone on your network … in AP mode you can exploit the Asus WAN port … it just becomes another Ethernet port … connect it to one of your 4 ports used by the servers … then plug the removed server port into the Asus free port etc. the Asus requires that you understand how to exploit it etc. your Tik will be the router, firewall, dhcp, dns etc.

The Asus is an excellent [outstanding] WiFi device but the person that configure it to work with your Tik must be capable and understand how to exploite … it’s not trivial

A better focussed purchase (replace WIFI) would have been a business class AP.

Agreed. I am running a RB4011 with four Meraki Enterprise access points at home. My kids will tell you that I have the best WiFi around...

hahahaha, how big is your house? Two TPLINKS could do the same job ;-PP

hahahaha, how big is your house? Two TPLINKS could do the same job ;-PP

1665 sq ft with a detached garage. Most of the house is covered from the AP here in the family room (very back wall of the house). There is an outdoor AP on the front wall of the house under the eve. It's there primarily for the cameras in the front yard, and the cameras and GPS in the truck. The third one is in the detached garage. There is a bunch of stuff in the garage. The last one is inside a steel storage shed behind the garage. My older son hangs out in the shed and has dismal coverage without an AP inside the steel. It is set for about as low power as I can make it since for the most part the signal is only needed inside the shed - total distance less than 25 feet. I could likely do the house with a single AP, but from a networking standpoint, the locations are convenient (very close to network switches).

hahahaha, how big is your house? Two TPLINKS could do the same job ;-PP

1665 sq ft with a detached garage. Most of the house is covered from the AP here in the family room (very back wall of the house). There is an outdoor AP on the front wall of the house under the eve. It's there primarily for the cameras in the front yard, and the cameras and GPS in the truck. The third one is in the detached garage. There is a bunch of stuff in the garage. The last one is inside a steel storage shed behind the garage. My older son hangs out in the shed and has dismal coverage without an AP inside the steel. It is set for about as low power as I can make it since for the most part the signal is only needed inside the shed - total distance less than 25 feet.

Very nicely done --- TP-Link like the EAP640HD would have been the same number of AP's but less costly ..... The key however is that your family are very satisfied far more important than the cost equation.

You should place the Asus in AP mode only … this turns of the NAT and router functions of the Asus. Then you can exploit the Ethernet ports on the Asus … the WiFi of the Asus will become available to everyone on your network … in AP mode you can exploit the Asus WAN port … it just becomes another Ethernet port … connect it to one of your 4 ports used by the servers … then plug the removed server port into the Asus free port etc. the Asus requires that you understand how to exploit it etc. your Tik will be the router, firewall, dhcp, dns etc.

The Asus is an excellent [outstanding] WiFi device but the person that configure it to work with your Tik must be capable and understand how to exploite … it’s not trivial

Guys I'd want to thank you for the tips and input. I first had the asus running as the WAN-LAN device, but it turns out being quite unstable. As with most of these closed source home-routers by asus, they are stacked with 'fancy' gamer-functionalities we don't really need, which make it rather unstable. It has for example a syslog, but it keeps booting with the factory time in the logs, which is really annoying, so until it sets its time using ntp, it logs everything as if it happened years ago. Typical ASUS hackjob. There's nothing in the logs that explains why it loses connectivity at random times.
So, what I've just decided to do is what mozerd proposed;
I'll use the ASUS in AP mode only, put the 100Mb links I have in the ethernet ports of the ASUS. Use one 10Gbe port between the RB4011 and the asus, which will suffice.
So glad I saved a RB4011 config from before I put the asus in, just a matter of returning the mikrotik to that, and then change the ASUS into a switch/AP only. Allows me to run the noisy fan of the RT-AX89X at its lowest too, because it doesn't have to do any firewalling, NAT, DNS or DHCP and barely should warm up.
Compared to the Mikrotik ROS this is quite a silly messy WebUI that ASUS created, lacking options and vaguely trying to look geeky. Can't even put any scripts in, unless it would be supported by Merlin fw, which it isn't..

Just an FYI. Not selling the RB4011, it is truly way more stable, if you disregard the wireless of it. And to be fair, the signal strength of the wireless chips/antennas of the RB4011 way outperform that of the ASUS. Sadly, it lacks the WiFi6, and doesn't work well with Apple/IOS devices, which is the main reason we put it here.

Understood Jult, until MT comes out with WIFI6, the experience you have matches most of the advice given here by people that speak truth.
As for mozerds advice, he forgot one important bit of evidence and that is that the ASUS does not do vlans, so if you were expecting to have multiple
subnets running on the ASUS, you are mistaken, I think it will handle one main network and then a separate (within the asus only) guest network.
Eventually they tie into the same lan subnet though. It may be possible not sure of asus software to ensure that not only are the guest blocked from the other WLANs, but they are
also blocked from the main LAN the home users are on ( put in other words, access to the internet only).
Do not expect to be able to have multiple home, guest, vidcam, multimedia, etc. subnets on wifi.