RB2011 - Expected performance?

robbz · Fri Mar 11, 2016 2:27 am

Hi!

I've been a Routerboard used since 2008. I've been super happy with my routerboard (3 different ones). They've been running stable with few issues.

Recently I bought a RB2011 (RB2011UiAS-2HnD-IN) to replace my RB433UAH that i've been using for my home network. The reason being that I wanted to utilize the full WAN speed that I have. RB433 only has 10/100 interfaces while RB2011 has gigabit interfaces. I had some issues configuring the new routerboard running a newer routerOS but managed to get things working in the end. I am using the same amount of filters as on my RB433. At 100mbit throughput (in multiple streams) on the RB433, I was utilizing about 50% cpu and less when I was doing one stream, but I always was getting about 95-100mbit.

Now, using speedtest - I can get about 110mbit on the RB2011 (physical port) which is better than the RB433, however I do a daily tar and backup of a remote server - and since it's located in Europe - I use fdt (http://monalisa.cern.ch/FDT/). However - I never get more than 60-70mbit and about 90-95% CPU usage on the RB2011 when using 16 streams. This was maxing line speed of the RB433 and not using more than 50% CPU.

I've changed these two guys just to make sure it's nothing else - and I can replicate this behavior every single time I do the switch. I've never gotten more than 70mbit thorughput on the RB2011 using fdt while I was always maxing the RB433 line speed - at less CPU usage.

Now - I doubt the RB433UAH (680mhz) cpu is faster (bought 2010) than the RB2011 (bought 2016) - but I might be wrong here. Anyone has any insight on what kind of performance I should expect from the RB2011? With very few rules and all - is this all I should get? Is the RB433UAH a better performer than the RB2011UiAS-2HnD-IN?

# mar/10/2016 16:18:54 by RouterOS 6.30.2
# software id = FAKE-YKEM
#
/interface bridge
add admin-mac=E4:8D:8C:36:3C:2A auto-mac=no name=bridge-local
/interface ethernet
set [ find default-name=ether1 ] mac-address=<redacted> name=ether1-gateway
set [ find default-name=ether2 ] name=ether2-master-local
set [ find default-name=ether3 ] master-port=ether2-master-local name=ether3-slave-local
set [ find default-name=ether4 ] master-port=ether2-master-local name=ether4-slave-local
set [ find default-name=ether5 ] master-port=ether2-master-local name=ether5-slave-local
set [ find default-name=ether6 ] name=ether6-master-local
set [ find default-name=ether7 ] master-port=ether6-master-local name=ether7-slave-local
set [ find default-name=ether8 ] master-port=ether6-master-local name=ether8-slave-local
set [ find default-name=ether9 ] master-port=ether6-master-local name=ether9-slave-local
set [ find default-name=ether10 ] master-port=ether6-master-local name=ether10-slave-local
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-Ce country="united states" disabled=no distance=indoors frequency=auto l2mtu=1600 mode=ap-bridge wireless-protocol=802.11
add disabled=no l2mtu=1600 mac-address=<redacted> master-interface=wlan1 name=wlan3 ssid=MikroTikN wds-default-bridge=bridge-local
/ip neighbor discovery
set ether1-gateway discover=no
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk mode=dynamic-keys wpa-pre-shared-key=bananaboat wpa2-pre-shared-key=<redacted>
add authentication-types=wpa2-psk management-protection=allowed mode=dynamic-keys name=guest_wifi wpa2-pre-shared-key=<redacted>
/interface wireless
add mac-address=<redacted> master-interface=wlan1 name=wlan2 security-profile=guest_wifi ssid=MikroTik-Guest wds-default-bridge=bridge-local
/ip pool
add name=pool1 ranges=192.168.77.101-192.168.77.250
/ip dhcp-server
add address-pool=pool1 disabled=no interface=bridge-local name=dhcp1
/interface bridge port
add bridge=bridge-local interface=ether2-master-local
add bridge=bridge-local interface=ether6-master-local
add bridge=bridge-local interface=sfp1
add bridge=bridge-local interface=wlan1
/ip accounting
set enabled=yes
/ip accounting web-access
set accessible-via-web=yes address=192.168.77.0/24
/ip address
add address=192.168.88.1/24 comment="default configuration" disabled=yes interface=bridge-local network=192.168.88.0
add address=192.168.77.1/24 interface=bridge-local network=192.168.77.0
/ip dhcp-client
add comment="default configuration" dhcp-options=hostname,clientid disabled=no interface=ether1-gateway
/ip dhcp-server lease
add address=192.168.77.10 mac-address=<redacted> server=dhcp1
add address=192.168.77.11 mac-address=<redacted> server=dhcp1
add address=192.168.77.12 mac-address=<redacted> server=dhcp1
/ip dhcp-server network
add address=192.168.77.0/24 dns-server=192.168.77.1 gateway=192.168.77.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 disabled=yes name=router
add address=192.168.77.1 disabled=yes name=router
/ip firewall filter
add chain=input connection-state=established,related
add chain=input src-address=192.168.77.0/24
add chain=forward connection-state=established,related
add chain=forward src-address=192.168.77.0/24
add chain=input protocol=udp
add action=drop chain=input protocol=icmp
add action=drop chain=input dst-port=22 protocol=tcp src-address-list=ssh_blacklist
add action=add-src-to-address-list address-list=ssh_blacklist address-list-timeout=4w2d chain=input connection-state=new dst-port=22 protocol=tcp src-address-list=ssh_stage3
add action=add-src-to-address-list address-list=ssh_stage3 address-list-timeout=1m chain=input connection-state=new dst-port=22 protocol=tcp src-address-list=ssh_stage2
add action=add-src-to-address-list address-list=ssh_stage2 address-list-timeout=1m chain=input connection-state=new dst-port=22 protocol=tcp src-address-list=ssh_stage1
add action=add-src-to-address-list address-list=ssh_stage1 address-list-timeout=1m chain=input connection-state=new dst-port=22 protocol=tcp
add action=drop chain=input dst-port=21 protocol=tcp src-address-list=ftp_blacklist
add chain=output content="530 Login incorrect" dst-limit=1/1m,4,dst-address/1m protocol=tcp
add action=add-dst-to-address-list address-list=ftp_blacklist address-list-timeout=3h chain=output content="530 Login incorrect" protocol=tcp
add action=log chain=input log-prefix="DROP INPUT"
add chain=input port=22 protocol=tcp
add chain=input disabled=yes port=8291 protocol=tcp
add action=drop chain=input connection-state=invalid
add action=drop chain=forward connection-state=invalid
add chain=forward
add chain=input disabled=yes dst-port=80 protocol=tcp src-address=<redacted>
add chain=input dst-port=80 protocol=tcp src-address=<redacted>
add action=drop chain=input
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" out-interface=ether1-gateway
add action=dst-nat chain=dstnat dst-port=222 in-interface=ether1-gateway protocol=tcp to-addresses=<redacted> to-ports=22
/ip proxy
set cache-path=web-proxy1 max-cache-size=none parent-proxy=0.0.0.0
/ip traffic-flow target
add address=192.168.77.12:9995 version=5
/lcd
set time-interval=weekly
/lcd interface pages
set 0 interfaces=sfp1,ether1-gateway,ether2-master-local,ether3-slave-local,ether4-slave-local,ether5-slave-local,ether6-master-local,ether7-slave-local,ether8-slave-local,ether9-slave-local,ether10-slave-local
/system clock
set time-zone-autodetect=no time-zone-name=America/Los_Angeles
/system ntp client
set enabled=yes primary-ntp=213.239.154.12 secondary-ntp=85.12.35.12
/system routerboard settings
set protected-routerboot=disabled
/tool graphing interface
add
/tool graphing queue
add
/tool graphing resource
add
/tool mac-server
set [ find default=yes ] disabled=yes
add interface=ether2-master-local
add interface=ether3-slave-local
add interface=ether4-slave-local
add interface=ether5-slave-local
add interface=ether6-master-local
add interface=ether7-slave-local
add interface=ether8-slave-local
add interface=ether9-slave-local
add interface=ether10-slave-local
add interface=sfp1
add interface=wlan1
add interface=bridge-local
/tool mac-server mac-winbox
set [ find default=yes ] disabled=yes
add interface=ether2-master-local
add interface=ether3-slave-local
add interface=ether4-slave-local
add interface=ether5-slave-local
add interface=ether6-master-local
add interface=ether7-slave-local
add interface=ether8-slave-local
add interface=ether9-slave-local
add interface=ether10-slave-local
add interface=sfp1
add interface=wlan1
add interface=bridge-local
/tool romon port
add

Fri Mar 11, 2016 2:53 am

Seems you can try fasttrack. It should help dramatically.

robbz · Tue Mar 15, 2016 7:19 pm

Jarda, thanks for your reply. I got fasttrack in as described in http://wiki.mikrotik.com/wiki/Manual:Wiki/Fasttrack - and although it helped the throughput only increased by 20% or so. It's nowhere near 100mbit tho. Avaraging at about 70mbit now. with peaks at 75.

I've noticed, since I upgraded to the latest ROS on my 433, it's now using 100% CPU when I am maxing the interface. I get about 98-99mbit with the RB433.

I will try to reset to factory defaults and see what kind of througput I will get through the 2011. If it's not improved I'll send it back.

robbz · Tue Mar 15, 2016 9:51 pm

Another discovery:

Since I upgraded the RB433 to 6.34.2, the CPU usage was way higher when doing transfers or whatever I was doing. I was unaware of fasttrack - so thanks for pointing that out. It helped some on the RB2011 but I didn't get anywhere near the throughput of the RB433 (same NAT, same Firewall filters).

I then enabled fasttrack on the RB433 to see if it helped with CPU usage - and BOOM, I'm down to the same CPU usage as before the 6.34.2 update.

So, did the 6.34.2 update just make things slower unless you had fasttrack enabled? LOL, it sure seems like it. Now I am doing 98-99mbit of throughput with about 40-50% CPU usage.

The RB2011 seems like a lost cause. The default configuration did improve throughput some - but it's still nowhere near what I get with the RB433.

I'm ordering a CCR1009 and some wireless AP's instead. The RB2011 is a great idea. Would love to have everything integrated in that little neat box - but it performs like shit for me.

vortex · Tue Mar 15, 2016 9:58 pm

I download at 500Mbps with NAT and fasttrack using the RB2011 at 65% CPU.

But I don't use PPPoE.

Tue Mar 15, 2016 11:06 pm

Looks like robbz does not use pppoe. Anyway, there should be enabled fasttrack for pppoe in last rc version too.
The playing with interface queues could help too. Try the only hardware queues for all ethernets or ethernet default.
The firewall looks also strange, for example accepting all udp in input chain, what sense it makes?

robbz · Wed Mar 16, 2016 2:54 am

I'm not sure what that udp rule is all about as I probably added it at some point in the past 5-6 years.

Might have something to do with tftp and so on... not sure..

Now, I need to add something that might have gotten missed in my initial post. If I do a speedtest - I can get more than 100mbit on the RB2011. What the RB2011 has issues with is concurrent hight throughput - meaning, if I have 16-20 downloads of 5+ mbit each - it will max out the CPU and will not perform. I am sure what differs from your setup vortex, and I would expect to get the same speed from the RB2011 as from my RB433 since the CPU in the RB2011 should be about 10-15% faster according to online resources.

I even went as far as disabling all other ports than eth0 and eth1 so there was nothing weird with the switch chip and so on, but didn't change.

I'll play some more with it tonight but don't feel like wasting too much time on it. I have a CCR1009 on order already.. Was thinking to try a 850Gx2 too since I'm separating WLAN and Routing to two different devices anyway with the CCR

Wed Mar 16, 2016 9:47 am

there will be surely some misconfiguration problem. But if you have ccr1009 on the way to you, it will be working great even with badly optimized firewall.
It could be good to see the profile what consumes the cpu when you load it by the described traffic. Do you use any special queues?
You can also try to update to latest bugfix 6.32.4 and update the firmware subsequently.

robbz · Mon Mar 21, 2016 9:28 pm

Sorry for this late reply. I'm just busy juggling life and work

I am not using any queues and the firewall has worked fine for years. I have not done any changes in the past year - and looking at the graph - CPU usage on the 433 went up from an average of 10% to and average of 60-70% by upgrading from 6.0 to 6.34.2.

Just need to add that with the default config I get the same issues on the RB2011 (which was sent back last week).

I have not yet installed the CCR1009 - but I also got a 850Gx2 just for fun. I'll play around with these when I have time. For now - the 433 is giving me what I need - so I'm not sweating it.

So, How do I file a bug - since this surely has to be a bug.

vortex · Mon Mar 21, 2016 9:34 pm

I think the bridge might be killing your performance.

I had to get rid of it to reach 500Mbps on the RB2011.

Wed Mar 23, 2016 1:02 am

Playing with interface queues could also help. Try to set ethernet default where only hardware queue or vice versa.

RB2011 - Expected performance?

RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?

Re: RB2011 - Expected performance?