As I understand it, under "Test Results" the whole capacity of the device gets tested, ie. using all ports w/o the uplink port(s).
This is good, but there seems to be a significant difference in the used test methods between switches and routers:

For example for the switch CRS326-24G-2S+RM this result for 1518 byte packets is given:
Switching Non blocking Layer 2 throughput 3,576.1 kpps = 43,427.8 Mpbs
Switching Non blocking Layer 2 capacity 3,576.1 kpps = 86,855.7 Mbps

Whereas for the router RB4011iGS+RM for 1518 byte packets:
Bridging none (fast path) 806,4 kpps = 9,792.9 Mbps
Routing none (fast path) 806,4 kpss = 9,792.9 Mbps

Now, if we do the math for full-duplex (ie. simultanous bi-directional traffic ) for the first test record of each device:
3576.1 * 1000 * 1518 * 8 / (1000 * 1000) / 24 = 1809.5 Mbps / port / full duplex (ie. avg of the total 24 ports w/o the uplink ports)
and for the router:
806.4 * 1000 * 1518 * 8 / (1000 * 1000) / 10 = 979.3 Mbps / port / full duplex (ie. avg of the total 10 ports w/o the uplink port)

So, either the router was not tested bi-directional, unlike the switches, or the router achieves only half of the Gigabit throughput it should have
(ie. the router should have twice of the above throughput rate if it were tested with bidirectional traffic, like in the case of the switch).
I think the test method for the router did not cover the bidirectionality, right?

Or, is maybe my assumption wrong?

Thx

Links to the Test Result pages:
https://mikrotik.com/product/CRS326-24G ... estresults
https://mikrotik.com/product/rb4011igs_rm#testresults

If you connect two devices to same switch with Gbps ports and run test between those two devices through switch. Then configure both devices to transmit data to the other device with max possible rate of 1Gbps.
Which makes switch to handle 2Gbps in total ... while two ports on the switch are used. So average throughput per used port is 1Gbps.

The example above shows that simple calculations without considering the nature of tests may be misleading.

As to the CRS326 test results: I'd be much surprised if the test didn't use all ports, including the two SFP+ ports. And those add 10Gbps each to throughput. If you substract 20Gbps from test result (43.4Gbps), you get 23.4Gbps and divided that to 24 ports, the result is roughly 1Gbps.

As to the RB4011 test results: the bottleneck there are interconnects between CPU and both switch chips which are 2.5Gbps each. When you consider a few test scenarios you'll probably always end up with 10Gbps max throughput and it'll be a mixture of traffic between pairs if Gbps ports and traffic between a few Gbps ports and that SFP+ port.

If you connect two devices to same switch with Gbps ports and run test between those two devices through switch. Then configure both devices to transmit data to the other device with max possible rate of 1Gbps.
Which makes switch to handle 2Gbps in total ... while two ports on the switch are used. So average throughput per used port is 1Gbps.

The example above shows that simple calculations without considering the nature of tests may be misleading.

As to the CRS326 test results: I'd be much surprised if the test didn't use all ports, including the two SFP+ ports. And those add 10Gbps each to throughput. If you substract 20Gbps from test result (43.4Gbps), you get 23.4Gbps and divided that to 24 ports, the result is roughly 1Gbps.

As to the RB4011 test results: the bottleneck there are interconnects between CPU and both switch chips which are 2.5Gbps each. When you consider a few test scenarios you'll probably always end up with 10Gbps max throughput and it'll be a mixture of traffic between pairs if Gbps ports and traffic between a few Gbps ports and that SFP+ port.

Hi, thanks, what do you think about the max practical throughput of these devices if ten Gigabit clients at the same time are getting data
from a high-speed server attached to the device via the SFP+ 10G port? Will each client get near Gigabit throughput or will it be much lower?

And maybe a related question:
in the following spec (of a foreign device), what does "Backplane 128Gb/s" mean?
Speeds: 24x RJ-45 (10/​100/​1000Base-T), 4x SFP+ (1G/​10Gb/​s)
Backplane: 128Gb/​s, 95.2Mp/​s, 9kB Jumboframes, 1.5MB Packetbuffer, 16k MAC addresses

Thx

I would expect CRS326 to be able to support the scenario (1x 10Gbps server serving 10x 1Gbps clients) while RB4011 could not. The reason being the mentioned interconnects (2.5Gbps per 5-port group) limiting achievable throughput in same scenario to 5Gbps total throughput (average 500Mbps per client).

It's hard to tell what exactly "backplane throughput" means without knowing block diagram and other device details. I suspect it's a marketing BS to emphasize there's no hidden limitation - 128Gbps is sum of full duplex speeds of all ports (2x 24x1Gbps + 2x 4x10Gbps). A marketing person could not present similar figure for RB4011 - sum of full duplex port speeds is 40Gbps (2x 10x1Gbps + 2x 10Gbps), while backplane capacity is only 30Gbps (2x 2x2.5Gbps + 2x10Gbps). The specs you posted show another neat number: 95.2 Mp/s (million packets per second). One can calculate smallest average packet size needed to allow switch to reach max throughput: 128Gbps/95.2Mpps/8bits = 168 bytes ... which is not something excessive large even in typical office environemnt. Meaning that max rate of packet forwarding likely won't limit the throughput, even more so if one skips the full-duplex BS (a packet ingressing through one port eventually egresses through another port, using up capacity of two ports hence summing up full-duplex capacity of all ports is BS) which then means that full switch throughput of 64Gbps can be reached by using tiny 84 byte packets.

I want to provide as many as possible full-duplex Gigabit connections with Gigabit throughput
to a local data server. Let's say for 10+ clients.

To make this real I was thinking I would need 10G equipment (a 10G switch,
a 10G NIC and the 10G cabling with the 10G SFP+ GBICs at each side).
Currently everything is just Gigabit (RJ-45).

But then I had this IMHO cool idea:
Add just as many quad-port Gigabit NICs to the server and attach each
port to the switch. There are 3 PCIe slots avail on the mainboard,
So I can put 3 quad-port GbE NICs to it, meaning 3x4 GbE ports + the onboard GbE port
makes in total 13 GbE ports. This means with a 24 port unmanaged/dumb GbE switch
n = 24 - 1 - 13 = 10 clients can get full Gigabit speed to & from this server (full-duplex)
at the same time. One port of the switch is used for uplink/gateway.

I'll try this out next week initially with just one quad-port NIC,
for this I just already have ordered some cheap used hardware at ebay...

Although the server is old, but I think it should still be powerful enough
to master the many NICs as it works mostly from RAM (DB caching etc.).

What do you think of this idea?

I guess your idea is plausible.

even more so if one skips the full-duplex BS (a packet ingressing through one port eventually egresses through another port, using up capacity of two ports hence summing up full-duplex capacity of all ports is BS)

I think the stated full-duplex speed is measured different. If the full-duplex speed is 2x the half-duplex one, and if the half-duplex one is equal to wirespeed, THEN we can say it is a non blocking switch.

I don't remember seeing one, but we could argue that someone could sell a switch with enough processing to do 150% wirespeed of a half duplex link, instead of the 200% we usually see. This hypothetical switch would not be non blocking - since we couldn't get it to switch 100% of wirespeed up AND down on all ports, at the same time.

True, it is hard to think of a situation where it would happen (all ports using close enough of 100% wirespeed up and down, at the same time) - but it´s the definition that matters.

@paternot: I'm not arguing usability of full-duplex port, I'm just wondering about definition of throughput in tests. I admit I've never read any test protocol specification, but from test results it seems to me that the published results are something like as follows:

Take two devices and connect them to a switch. Then start iperf in udp mode on both devices, one as server (accepting connections) and one as client (initiating connections). Then start a single direction UDP test at wire speed (e.g. 1Gbps). So 1Gbps traffic flows from client device to server device and none in reverse direction (it's UDP). For switch that's 1Gbps ingress on one port and 1Gbps egress on another port.

For me, that's 1Gbps throughput. For marketing people (and probably testers) that's 2Gbps throughput. AFAIK that's 1Gbps flowing through switch "backplane" and I guess switch will handle each packet in one single stretch ...

The only case when the full-duplex numbers would actually make sense would be if the switch/router would be source/sink of data streams itself ... meaning that data stream flowing through one port is completely unrelated to any data flowing through any other port ...

@paternot: I'm not arguing usability of full-duplex port, I'm just wondering about definition of throughput in tests. I admit I've never read any test protocol specification, but from test results it seems to me that the published results are something like as follows:

Take two devices and connect them to a switch. Then start iperf in udp mode on both devices, one as server (accepting connections) and one as client (initiating connections). Then start a single direction UDP test at wire speed (e.g. 1Gbps). So 1Gbps traffic flows from client device to server device and none in reverse direction (it's UDP). For switch that's 1Gbps ingress on one port and 1Gbps egress on another port.

For me, that's 1Gbps throughput. For marketing people (and probably testers) that's 2Gbps throughput. AFAIK that's 1Gbps flowing through switch "backplane" and I guess switch will handle each packet in one single stretch ...

The only case when the full-duplex numbers would actually make sense would be if the switch/router would be source/sink of data streams itself ... meaning that data stream flowing through one port is completely unrelated to any data flowing through any other port ...

No, that's not what they use as full duplex.

Case 1:
Take two computers, as You said. On server and one client. Make the client send 1 Gbps to the server. One switch port sees 1 Gbps incoming, and another port sees 1 Gbps outgoing. This is what they call "1 Gbps throughput."

Case 2:
Take the same two computers. But now both are server and client, at the same time. Each one will send 1Gbps AND receive 1Gbps. This is what they call "2Gbps full duplex". And it is true, since each port will see 2Gbps of traffic: one Gbps up and another Gbps down.

It is the same thing with network cards. A Gb network card will say that it is an "one gigabit network card", and in the specs You will see "max traffic (full duplex) is two Gbps". Go on, take a look. I've seen it since fast ethernet cards for sure - and I believe the ethernet (10 Mbps) ones would be the same.

I want to provide as many as possible full-duplex Gigabit connections with Gigabit throughput
to a local data server. ...
Add just as many quad-port Gigabit NICs to the server and attach each
port to the switch. ... I'll try this out next week initially with just one quad-port NIC,

I just tried this out: works well. I had to specify different metric 1 to 5 to the now 5 GbE interfaces of the server.
I'm getting around 950 Mbps on all 5 interfaces of the server at the same time, going thru an unmanaged/dumb 24-port switch.
But the drawback of this solution is of course the many additional cables needed, and the additional switch ports for them, of course.
Nevertheless, it's indeed a good budget, ie. "poor man's" alternative to offer Gigabit throughput to/from a server for more than one client at the same time.
But I think I still will buy also the CRS326-24G-2S+RM as I need to gain some experience also with such 10G hardware.

I test CCR routers very simple with simple static routing. And SFP+ use VLAN's.

CCR's with full 100% CPU utilization seem not really usable to have this in production under these kinds of load...nice for a lab-stress-test to max it out but...
Do you have stats on effective packet drops under these loads ?
Hard to believe it is "0" with all cores at 100% etc.

Interesting is that, when you turn on the Firewall filter drop "Invalid" rule (all MT CCR an hEX) -MT (hEX S) integrated BandWidth test produces a lot of invalid packets and traffic drop by 25%
But if this test is performed using Virtual Linux Servers (KVM VM) and the bridge interface is on SFP + so many Invalid packages VM linux server cannot be produced using SSH or NFS or Windows client connect to Linux SAMBA VM.