Hello,

I am upgrading my existing home network/lab, a CRS109-8G-1S-2HnD. I wanted to set up 10G between my Router, Switch, NAS(s), Proxmox Servers and workstation. My physical connection needs don't require a router or switch with a lot of ports. So I just got a CCR2004-16G-2S+ and a CRS305-1G-4S+. I've installed 10G NICs in my servers (Broadcom) and workstation (Intel). The servers are connected with 1.5m DAC cables, my workstation with a 9m DAC and a 1m DAC to crossconnect between the CCR2004's sfp1 port and the CRS305's sfp1 port. All interfaces show 10G link w/ no errors or dropped links. I left the CRS305 running the RouterOS vs switching to the SwitchOS mode, mostly because I'm very familiar with RouterOS, not so much with SwitchOS. I know there are several ways w/ Mikrotik to set up vlans and trunking. How I've done it over the years is I set up a vlan_interface_bridge with the vlans I use on my network on that bridge. Then I set up the Bridge VLANs with the ID's and the interfaces as Tagged or Untagged. Generally I find I have to set up the Bridge Port PVID with the vlan ID I want when setting up a port as not tagged. I've done this on both devices and and effectively have the interfaces in trunking mode. I've set up a /30 between the two devices on one of the vlans. When I run a bandwidth test from the switch, TCP, Random Data, in receive mode, I'm only seeing as high as 401.5Mbps but avgs around 380Mbps. UDP avgs around 1160Mbps. For TCP, that's no better than my 1G network prior my upgrade. What have I missed or completely misunderstand? Have I made a mistake in getting the CRS305? CPU peaks to about 27% on the CCR but 100% on the CRS. So that tells me something. I DO NOT have "Use IP Firewall" enabled on the bridge and left Allow Fast Path enabled. I've also set Spanning Tree Protocol mode to "none" and left "Ingress Filtering" disabled. There's no Queues defined. I haven't upgraded the CRS to v7, didn't think at this point it would have any bearing. Would really appreciate your experiences. If I haven't provided enough info, I'm happy to provide any additional.

Thanks

The CPU on the CRS isn't capable of handling 10Gb, use suitably powerful external devices for bandwidth testing.

Thanks @tdw A few questions if I may. First question, I've seen some youtube vids ( @TheNetworkTrip & @Inquirinity ) showing how to set up Vlans using the Switch instead of setting them up using the Bridge->VLANs which uses the cpu. This passes the traffic directly using the switch chip vs going through the cpu. You have to set up a Bridge w/o VLAN Filtering and add all the relative ports under the Bridge->Ports tab ensuring that each has "Hardware Offload" enabled. I've done that. But the Switch config box doesn't show a VLANs tab like I see in the vids. In both videos the switches were using v6.47.1, v6.48.2 respectively. BTW, I upgraded the CRS305 to v7.7. I have also referenced (https://wiki.mikrotik.com/wiki/Manual:I ... Offloading). Second question, should I switch to SwitchOS? And third question, if the cpu isn't capable, then why is it sold as a 10G switch? Am I to understand that the CRS305 will "PASS" 10G, but that you can't "TEST" it using the CRS305?

Thanks again

bridge can offload certain functions to underlying hardware switch. And CRS3xx supported offloading VLAN filtering to underlying hardware right from beginning. So here's the reason not to follow random youtube videos without any considerations.
Many devices received L2 HW offload since, your CCR included. Your CRS1xx did not, so you have to know your hardware when considering options of configuring certain things, such as VLANs.

Answer to second question is subjective. SwOS has easier interface but lacks lots of advanced functionality.

Switch is fine for passing traffic between its ports (i.e. switching). Bandwidth test (function of RouterOS), however, uses CPU to generate (and consume) traffic and built-in test is not exactly performance optimized.

Thanks @mkx You are absolutely correct in your comment regarding "random" youtube videos. You do have to know your hardware and the context of both your hardware and what you are trying to accomplish. But, we all have benefited by research and using youtube as a resource so I don't want to poo poo it as an effective resource to use and to avoid. And I don't think that's what you are saying, but just be careful and have a clear understanding not taking what you see verbatim. In may case I understood from the feature matrix that the CRS305 did support hardware offload, etc. I consider myself quite familiar with RouterOS; I've configured and managed 100's of mtik devices used in a number of WISPs our company has acquired over the years. In every case, I was using RouterOS and so only had brief experience with SwOS. And me personally, found SwOS a bit clunky to use. But that's only my opinion. So if I understand you, and now thinking about it it makes sense, that the Bandwidth Test tool uses the CPU. So even if I continue to use RouterOS to configure my vlans, I should see the increased bandwidth capability when simply passing traffic through the switch across the ports/vlans. I admit, I had not yet set up an iperf or perfsonar node/vm and tested through the switch. I will do that to confirm. Super appreciate the feedback.

Thanks