hw offloading, uplink 10Gbit. Other 1Gbit. Speed through 10Gbit to 1Gbit is under 1Gbit... about 600-700Mbit. But, 1Gbit to 1Gbit port is speed full 1Gbit. Flow control on/off... no matter.
This same problem was on CRS326. Do you test traffic at all different port speeds?
I need a solution. Immediately.
Again disappointing Mikrotik.
CRS328-24P-4S+RM speed problem
Last edited by hapi on Fri May 25, 2018 4:04 pm, edited 1 time in total.
Re: CRS328-24P-4S+RM speed problem
I have noticed some of the same problems on my CRS328-24P-4S+RM device, it does look to be related in some way to latency or packet buffers inside the switch chip.
Looking at some other posts on the forum, it appears this problem has/is with most of the CRS models. Would be nice if mikrotik could respond to this.
An iperf3 on low latency local network from 10G port to 1G port does around 930Mbit both ways, while if you put something on the 1G port that has higher latency like internet connection it can drop down to almost nothing (50-130Mbit) in the 10G to 1G direction, while the 1G to 10G direction is 930Mbit+
I think it is some kind of buffer problem on the hw level, as the problem can be solved if you can limit (queue) the 10G traffic going onto the 1G port to below 1G, then the speed goes up to full. So its likely some kind of burst problem.
Looking at some other posts on the forum, it appears this problem has/is with most of the CRS models. Would be nice if mikrotik could respond to this.
An iperf3 on low latency local network from 10G port to 1G port does around 930Mbit both ways, while if you put something on the 1G port that has higher latency like internet connection it can drop down to almost nothing (50-130Mbit) in the 10G to 1G direction, while the 1G to 10G direction is 930Mbit+
I think it is some kind of buffer problem on the hw level, as the problem can be solved if you can limit (queue) the 10G traffic going onto the 1G port to below 1G, then the speed goes up to full. So its likely some kind of burst problem.
Re: CRS328-24P-4S+RM speed problem
I have noticed some of the same problems with my CRS328-24P-4S+RM device, it looks to be related to latency, and maybe packet buffers in the switch chip.
On the forum there are other posts of people having similar issues with other CRS models. It would be nice if mikrotik could respond to this issue.
An iperf3 on a local low latency network from 10G port to 1G port gives around 930Mbit both ways, but if something with higher latency is used like an internet connection it can drop down to almost nothing (30-130Mbit) in the 10G to 1G direction, while 1G to 10G is full speed.
It appears to be some kind of burst problem on the hw level, as the problem can be solved by limiting (queue) the traffic coming into the 10G port, that is destined to the 1G port to below 1G speeds.
First reply post didnt get though for some reason, so I try again, sorry if 2 replies show up.
On the forum there are other posts of people having similar issues with other CRS models. It would be nice if mikrotik could respond to this issue.
An iperf3 on a local low latency network from 10G port to 1G port gives around 930Mbit both ways, but if something with higher latency is used like an internet connection it can drop down to almost nothing (30-130Mbit) in the 10G to 1G direction, while 1G to 10G is full speed.
It appears to be some kind of burst problem on the hw level, as the problem can be solved by limiting (queue) the traffic coming into the 10G port, that is destined to the 1G port to below 1G speeds.
First reply post didnt get though for some reason, so I try again, sorry if 2 replies show up.
Re: CRS328-24P-4S+RM speed problem
i doubt flow control will fix that.
this issue is known as interface oversubscription or speed conversion drop. you fill the outgoing interface’s buffers with bursty traffic. happens if you have 10g ingress and 1g egress connection, or if multiple GE ports receive traffic that must leave on a single GE interface. the solution is usually having bigger buffers on the egress ports which can hold more packets but this causes increased latency.
this issue is known as interface oversubscription or speed conversion drop. you fill the outgoing interface’s buffers with bursty traffic. happens if you have 10g ingress and 1g egress connection, or if multiple GE ports receive traffic that must leave on a single GE interface. the solution is usually having bigger buffers on the egress ports which can hold more packets but this causes increased latency.
Re: CRS328-24P-4S+RM speed problem
Nonsense. I'll wait for someone who knows what he's talking about. There is a problem with the transition from 1G to 100M to the last CRS. The same goes for 10G to 1G.
Just have a 10Gbit uplink and only one 1Gbit device and the traffic fluctuates and 1Gbit can not be reached. At nomikrotik devices it works normally.
Just have a 10Gbit uplink and only one 1Gbit device and the traffic fluctuates and 1Gbit can not be reached. At nomikrotik devices it works normally.
Re: CRS328-24P-4S+RM speed problem
be my guestAt nomikrotik devices it works normally.
https://www.cisco.com/c/en/us/support/d ... ut-dr.html
Re: CRS328-24P-4S+RM speed problem
nice, but the problem arises even when the 800Mbit queue is used on the uplink router to crs where the buffers in the CRS should work at rest so it does not work anyway. It still can not reach even 800Mbit. Similarly, it can be converted to 1G to 100M ports. This normally handles other devices. CRS no.
Re: CRS328-24P-4S+RM speed problem
Is there any new news ? with 6.43.8
Re: CRS328-24P-4S+RM speed problem
I think I'm in the same case on my CRS328-24P-4S+
a bandwidth-test beetween my CRS328 and a RB5009 on a dedicated 10G line give only 369.4Mbps in one way and 737.4Mbps in the other.
I found this since I use the RB5009 as my main internet router connection.
If I plug a computer directly on the RB5009 on a 1G port I have with 680Mb upload and around 930Mb down with speedtest.net. This is good.
Then if I plug a 10G SFP beetween RB5009 and CRS328 and I connect a PC on CRS328 the same test speed on speedtest.net give 642Mb upload but 349Mb download.
If I run my CRS328 with SwitchOS there is no trouble.
I use RouterOs for historical raison and ssh access to power on and off Poe port remotely. I dont find a way to do this with switch os.
Firmware on RB5009 : 7.15.3
Firmware on CRS328 : 7.15.3
a bandwidth-test beetween my CRS328 and a RB5009 on a dedicated 10G line give only 369.4Mbps in one way and 737.4Mbps in the other.
I found this since I use the RB5009 as my main internet router connection.
If I plug a computer directly on the RB5009 on a 1G port I have with 680Mb upload and around 930Mb down with speedtest.net. This is good.
Then if I plug a 10G SFP beetween RB5009 and CRS328 and I connect a PC on CRS328 the same test speed on speedtest.net give 642Mb upload but 349Mb download.
If I run my CRS328 with SwitchOS there is no trouble.
I use RouterOs for historical raison and ssh access to power on and off Poe port remotely. I dont find a way to do this with switch os.
Firmware on RB5009 : 7.15.3
Firmware on CRS328 : 7.15.3
Re: CRS328-24P-4S+RM speed problem
good info, thank you for sharing, can be interesting to apply this info on new qos-hw-offloading feature of CRS 3xx/5xx switchesbe my guestAt nomikrotik devices it works normally.
https://www.cisco.com/c/en/us/support/d ... ut-dr.html
qos-hw-offloading
https://help.mikrotik.com/docs/pages/vi ... =189497483
Now, referring to the topic in question about CRS switch speed problem:
First thing to do is to verify if CRS switch is properly configured as a switch using a single bridge with vlan-filtering to allow hardware acceleration to operate
Re: CRS328-24P-4S+RM speed problem
Bandwidth-test (as ROS feature) is extremely CPU-bound, so in most cases it's measuring CPU performance rather than link performance. Specially so for devices with fast links and relatively weak CPU (CRS328 in your case).a bandwidth-test beetween my CRS328 and a RB5009
So: just don't use bandwidth-test on both devices, rather test through them by using two proper PCs, one connected to each end, and run iperf test between those two PCs.
Re: CRS328-24P-4S+RM speed problem
So I think I simplified at maximum my config.
Has I said in previous post, you can see that my external speedtest done with speedtest.net are not good when my PC is connected to CRS328 in router mode.
I'd like to stay in RouterMode to be able to on off poe port remotely with an ssh connection. If there is a way to do this with SwitchOS I will change mode because same test with SwitchOS are just very good. I can manage to learn how to use API if I can do i this way.
As same test are ok with SwitchOS I think I should be able to have a really simple RouterOS config which could do the same a SwitchOS !
Here Is my CRS328 config:
Has I said in previous post, you can see that my external speedtest done with speedtest.net are not good when my PC is connected to CRS328 in router mode.
I'd like to stay in RouterMode to be able to on off poe port remotely with an ssh connection. If there is a way to do this with SwitchOS I will change mode because same test with SwitchOS are just very good. I can manage to learn how to use API if I can do i this way.
As same test are ok with SwitchOS I think I should be able to have a really simple RouterOS config which could do the same a SwitchOS !
Here Is my CRS328 config:
Code: Select all
/interface bridge add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no comment=defconf name=bridge port-cost-mode=short
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether1 internal-path-cost=10 path-cost=10
[...] # all port same config
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4 internal-path-cost=10 path-cost=10
/interface bridge settings set allow-fast-path=no
/interface ethernet switch set 0 l3-hw-offloading=yes
/interface list
add name=WAN
add name=LAN
add name=INTERNET
add name=VLAN
/interface list member add interface=bridge list=LAN
/routing bgp template set default disabled=no output.network=bgp-networks
/routing ospf instance add disabled=no name=default-v2
/routing ospf area add disabled=yes instance=default-v2 name=backbone-v2
/routing bfd configuration add disabled=no
/ipv6 settings set disable-ipv6=yes
/ip firewall connection tracking set udp-timeout=10s
/ip neighbor discovery-settings set discover-interface-list=all
/ip settings set max-neighbor-entries=8192
/ip address add address=192.168.0.253/24 interface=bridge network=192.168.0.0
/ip dns set allow-remote-requests=yes servers=192.168.0.53
/system routerboard settings set boot-os=router-os
/system swos
set address-acquisition-mode=static allow-from-ports="p1,p2,p3,p4,p5,p6,p7,p8,\
p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27\
,p28,p29,p30,p31,p32" identity=switch static-ip-address=192.168.0.253
/tool bandwidth-server set authenticate=no
/tool mac-server set allowed-interface-list=LAN
Re: CRS328-24P-4S+RM speed problem
I had the same issue you had when we got our first switch.
I understand you are using routeros for management, but you could use untagged vlan with a ip address /30
At the SwitchOS you configure one of the ip address and in the mikrotik the other one.
I manage plenty of switches with this config, I can reprovision Vlans and poe. Everything using the web interface. The obly downside is romon.
If you are interested I could figure out a mini tutorial showing what I mean.
I understand you are using routeros for management, but you could use untagged vlan with a ip address /30
At the SwitchOS you configure one of the ip address and in the mikrotik the other one.
I manage plenty of switches with this config, I can reprovision Vlans and poe. Everything using the web interface. The obly downside is romon.
If you are interested I could figure out a mini tutorial showing what I mean.
Re: CRS328-24P-4S+RM speed problem
Hi, someone aske me on Reddit to post the full conf. It's here, the above was not completly full.
Code: Select all
# 2024-09-23 10:32:46 by RouterOS 7.15.3
# software id = RJBF-0VLJ
#
# model = CRS328-24P-4S+
# serial number = myserial
/interface bridge
add admin-mac=xx:xx:xx:xx:xx:xx auto-mac=no comment=defconf name=bridge \
port-cost-mode=short
/interface ethernet
set [ find default-name=ether1 ] comment="" poe-out=off
set [ find default-name=ether2 ] comment="" poe-out=off
set [ find default-name=ether4 ] comment=""
set [ find default-name=ether14 ] comment=""
set [ find default-name=ether18 ] comment=""
set [ find default-name=ether22 ] comment=""
set [ find default-name=ether24 ] comment=""
set [ find default-name=sfp-sfpplus1 ] comment=""
set [ find default-name=sfp-sfpplus3 ] comment=""
set [ find default-name=sfp-sfpplus4 ] comment=""
/interface list
add name=WAN
add name=LAN
add name=INTERNET
add name=VLAN
/interface lte apn set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/ip smb users set [ find default=yes ] disabled=yes
/port set 0 name=serial0
/routing bgp template set default disabled=no output.network=bgp-networks
/routing ospf instance add disabled=no name=default-v2
/routing ospf area add disabled=yes instance=default-v2 name=backbone-v2
/user group
add name=user1 policy="read,winbox,api,!local,!telnet,!ssh,!ftp,!reboot,!\
write,!policy,!test,!password,!web,!sniff,!sensitive,!romon,!rest-api"
add name=user2 policy="ssh,read,winbox,api,!local,!telnet,!ftp,!reboot\
,!write,!policy,!test,!password,!web,!sniff,!sensitive,!romon,!rest-api"
/interface bridge nat
add action=accept chain=srcnat disabled=yes
add action=accept chain=dstnat disabled=yes
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether1 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether9 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether10 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether11 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether12 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether13 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether14 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether15 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether16 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether17 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether18 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether19 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether20 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether21 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether22 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether23 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether24 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4 internal-path-cost=10 path-cost=10
add bridge=bridge interface=sfp-sfpplus2 internal-path-cost=10 path-cost=10
/interface bridge settings set allow-fast-path=no
/ip firewall connection tracking set udp-timeout=10s
/ip neighbor discovery-settings set discover-interface-list=all
/ip settings set max-neighbor-entries=8192
/ipv6 settings set disable-ipv6=yes
/interface ethernet switch set 0 l3-hw-offloading=yes
/interface list member add interface=bridge list=LAN
/interface ovpn-server server set auth=sha1,md5
/ip address add address=192.168.0.253/24 interface=bridge network=192.168.0.0
/ip dns set allow-remote-requests=yes servers=192.168.0.53
/ip service set telnet disabled=yes set ftp disabled=yes set www-ssl disabled=no
/ip smb shares set [ find default=yes ] directory=/flash/pub
/routing bfd configuration add disabled=no
/system clock set time-zone-name=Europe/Paris
/system identity set name=Switch
/system note set show-at-login=no
/system ntp client set enabled=yes
/system routerboard settings set boot-os=router-os
/system swos
set address-acquisition-mode=static allow-from-ports="p1,p2,p3,p4,p5,p6,p7,p8,\
p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27\
,p28,p29,p30,p31,p32" identity=switch static-ip-address=192.168.0.253
/tool bandwidth-server set authenticate=no
/tool mac-server set allowed-interface-list=LAN
/user aaa set default-group=groupe1
Re: CRS328-24P-4S+RM speed problem
Pretty sure that is your problem.Code: Select all.... TRIM .... /interface bridge settings set allow-fast-path=no .... TRIM ....