I am planing to buy this switch, to use it to connect 3x ESX hosts and a NetApp Filer via 10Gbit. Is this switch potent enough to operate all of its 24x 10Gbit ports "non-blocking" at full speed? I won't need all if its ports, but just to make sure.
Also I plan to use RouterOS instead of SwitchOS - will there be a performance hit by doing so?

The actual setup is, I want to use two switches - for redundancy purposes. They are both connected to each other with 1x 10Gbit. Each ESX host will use 4x 10Gbit "lanes/cables/ports" - two will go to the first switch, and two to the second switch. The ESX host itself has two different network cards with 2x 10Gbit ports each. So, should any piece of hardware fail (the network card in the ESX host, or a switch), network connectivity should remain.

Is this setup feasible?

It tells you in the brochure. What else do you need to know? RouterOS is fine as long as you don't try to use the CRS as a router. It is setup for switching. To understand your needs, you need to note the ports in use, direction of traffic, and the speed.

Two 40 Gbps QSFP+ ports and twenty four 10 Gbps SFP+ ports, which provides total non-blocking throughput of 320 Gbps and switching capacity of 640 Gbps with forwarding rate of 252 Mpps with most common packets

The actual setup is, I want to use two switches - for redundancy purposes. They are both connected to each other with 1x 10Gbit. Each ESX host will use 4x 10Gbit "lanes/cables/ports" - two will go to the first switch, and two to the second switch. The ESX host itself has two different network cards with 2x 10Gbit ports each. So, should any piece of hardware fail (the network card in the ESX host, or a switch), network connectivity should remain.

Is this setup feasible?

When planning your network please keep in mind that according to the current documentation Multi-chassis Link Aggregation Group (MLAG) "is not compatible with L3 hardware offloading. When using MLAG, the L3 hardware offloading must be disabled."

Thank you for pointing out the MLAG situation! Two questions regarding MLAG:

1) What exactly does using MLAG mean, respectively what's the downside of not using MLAG, and treating the 2 switches as independent individuals? If I understand it correctly, MLAG means that a connected client sees the switches as one physical unit. What's the advantage of that? In case one switch fails there is a seamless continuation of network connectivity without any downtime/delay what so ever? Ease of management, since only "one" switch needs to be configured/monitored/maintained? Is there any performance hit when NOT using MLAG?

2) Although we intend to run RouterOS on it, we do not intend to use any routing functionality on these two switches, but use it as pure-hardware switches only. Using RouterOS is only out of convenience, since we are familiar with RouterOS and know how to maintain/monitor it remotely. AFAIK there is no downside in using RouterOS, right? With that in mind, I assume it is fine to use the two switches in our setup with RouterOS running in MLAG mode?

Yes, MLAG means that LACP connections from one ESXi host split across switches will appear (to ESXi) to be going back to a single switch and will load-balance across both switches.

There is no performance decrease using Router OS if you're just switching. My CRS300 (305,309,310,317,326,328) switches all hum along on RouterOS 7 with hardly any CPU load on them.

Dont buy it if you intend to use MLAG. MLAG is severely broken in Mikrotik and they have demonstrated zero interest in fixing it.. Lots of talk.. no action.. Ive been sitting on a pair of CRS317-1G-16S+ for over a year now and MLAG is still as broken (more so) than it has ever been.

Thanks @roadracer96
Is it true that MLAG is unusable in Mikrotik Switches in it's current state? Can other users confirm that?
So are you running the switches without MLAG instead?
I wonder if @sirbryan is using MLAG, because I have the impression (from his post) that he is happy with the switches?
What's other users experience with MLAG on CRS switches?

We only intend to use these 2 switches to hook up 3x ESXi's with a NetApp filer. What is important to us is reliable connectivity (especially in case one device should fail), full speed of the 10Gbit ports. If on top of that we take advantage of LACP with "increased liability" and load balancing, that's a plus of course.

Easy of management (stacking, "manage as one device") is NOT important to us, since there's only these 2 switches to manage.

Mlag doesn't fail over transparently. If the primary device restarts or fails. The lacp system id and STP bridge ID of the secondary reverts causing complete outage for several seconds.

It works. Unless you want it to work for faiover. Not even close. It's half assed.

Spend the money and get a good switch If you need this functionality.

Hi @OriiOn

While MLAG/VCP is obviously better, people have used the redundant switches without MLAG support for long time.
If you use iSCSI for connectivity to NetApp, there should be no issue at all, you create multi-path configuration on two different subnets. For NFS and VM traffic you would use port groups with multiple uplinks connected to the different switches. NetApp can do similar configuration using the broadcast domains/failover groups.

Hi @OriiOn

While MLAG/VCP is obviously better, people have used the redundant switches without MLAG support for long time.
If you use iSCSI for connectivity to NetApp, there should be no issue at all, you create multi-path configuration on two different subnets. For NFS and VM traffic you would use port groups with multiple uplinks connected to the different switches. NetApp can do similar configuration using the broadcast domains/failover groups.

Keep in mind. Using port groups in vmware breaks some networking. It's the worst idea anyone ever came up with and for some reason people still cling to using it. 2 words. Multicast and promiscuous mode. Mlag/lacp fixes this.

Thanks for your replies!

@roadracer96: so which MLAG/LACP compliant switch would you recommend? The ones I am seeing from Cisco or HP cost €5000 and up - per piece. Compared to what we spent on the 3 ESXi's that seems unproportional. Ok, the NetApp filer did cost more (€20.000). But it seems unreasonable to spend €10K for two switches just to connect three machines? Or am I unrealistic? In other words, are there MLAG/LACP switches at around €2K per piece that suits our needs?

And I agree, I would like to avoid having to do complicated setups on the ESXi/VM's and/or NetApp filer, if there IS a technology (MLAG/LACP) that takes care of reliability and load-balancing nicely.