50% bandwidth loss RB2011UiAS

servaris · Sun Mar 26, 2017 5:24 am

Hi,
Getting loss of more than 50% DL speed when behind the RB2011UiAS. There is an issue with upload speed ISP said will be fixed.

Tests below were performed from wired Desktop

Behind RB2011UiAS

bandwidth-test-rb2011.png

Directly connected to Cable Modem

bandwidth-test-cablemodem.png

Running the Bandwidth test from Winbox

bandwidth-test-tcp.png

Have simple firewall and very few nat rules
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" log=yes log-prefix=masquerage \
out-interface=ether1 to-addresses=0.0.0.0
add action=dst-nat chain=dstnat comment="Desktop SSH" dst-port=10069 in-interface=ether1 log=yes \
protocol=tcp to-addresses=192.168.25.15 to-ports=22
add action=dst-nat chain=dstnat comment="Other Box" dst-port=10050 in-interface=ether1 log=yes protocol=\
tcp to-addresses=192.168.25.252 to-ports=10050

/ip firewall filter
add action=drop chain=input dst-port=80 in-interface=ether1 protocol=tcp
add action=drop chain=input comment="drop ssh 22 brute forcers" dst-port=22 protocol=tcp \
src-address-list=ssh_blacklist
add action=add-src-to-address-list address-list=ssh_blacklist address-list-timeout=1w3d chain=input \
connection-state=new dst-port=22 protocol=tcp src-address-list=ssh_stage3
add action=add-src-to-address-list address-list=ssh_stage2 address-list-timeout=20s chain=input \
connection-state=new dst-port=22 protocol=tcp src-address-list=ssh_stage1
add action=add-src-to-address-list address-list=ssh_stage1 address-list-timeout=1m chain=input \
connection-state=new dst-port=22 protocol=tcp
add action=drop chain=input comment="drop pptp brute forcers" dst-port=1723 protocol=tcp \
src-address-list=pptp_blacklist
add action=add-src-to-address-list address-list=pptp_blacklist address-list-timeout=1w3d chain=input \
connection-state=new dst-port=1723 protocol=tcp src-address-list=pptp_stage3
add action=add-src-to-address-list address-list=pptp_stage3 address-list-timeout=1m chain=input \
connection-state=new dst-port=1723 protocol=tcp src-address-list=pptp_stage2
add action=add-src-to-address-list address-list=pptp_stage2 address-list-timeout=1m chain=input \
connection-state=new dst-port=1723 protocol=tcp src-address-list=pptp_stage1
add action=add-src-to-address-list address-list=pptp_stage1 address-list-timeout=1m chain=input \
connection-state=new dst-port=1723 protocol=tcp
add action=drop chain=input connection-state=invalid,new dst-port=53 in-interface=ether1 protocol=udp

LCD is disabled
eth1 = WAN
Ether3 is a slave of ether2
1 Bridge with ports wlan1, eth2 and eth3 (Dynamic) all other eth ports are not used.
1 DHCP server for Bridge

So why the loss of bandwidth? What might I do to fix it.
Thank you for your time and help.

mistry7 · Sun Mar 26, 2017 10:22 am

Fastrack and Fastpath
Search for this in wiki

pukkita · Sun Mar 26, 2017 11:44 am

Can you post Interfaces > ether1 Overall, Rx, Tx and Status after 10 minutes of bw test? What's System > Routerboard Current/Upgrade Firmware?

servaris · Mon Mar 27, 2017 3:54 pm

Hi Pukkita,
Bandwidth test TCP > 10 minutes

bandwith-tcp-both-10min.png

Eth1 Overall stats > 10 minutes

ether1-stats.png

Received email from support suggesting to run bandwidth test using UDP, Bandwidth test UDP > 10 minutes. UDP bandwidth looks great but aren't most things TCP?

bandwith-udp-both-10min.png

Thanks to Mistry7 for suggesting fasttrack

servaris · Mon Mar 27, 2017 3:58 pm

Fastrack and Fastpath
Search for this in wiki

Thanks for the suggestion Mistry7

New firewall rules added for Fasttrack

firewall.png

pukkita · Mon Mar 27, 2017 4:05 pm

Yes, most protocols use TCP.

What your results with UDP suggest, is something is hosing TCP traffic.

UDP doesn't acknowledge packet delivery, whereas TCP does, hence the difference.

Can you please post Tx stats, Rx stats, and Status tabs?

Do you have another routerboard? Is this an old one with graphing enabled?

servaris · Mon Mar 27, 2017 4:34 pm

Hi Pukkita,

Below are images af all

status.png

rx-stats.png

overall-stats.png

servaris · Mon Mar 27, 2017 4:39 pm

This site only allows 3 images!

ethernet.png

loop-protect.png

general.png

servaris · Mon Mar 27, 2017 4:41 pm

tx-stats.png

traffic.png

rx-stats.png

dgnevans · Tue Mar 28, 2017 1:00 pm

Are you running any queues? have you tried to remove the port you connect to from the bridge and run test outside the bridge?

philamonster · Mon Apr 03, 2017 11:09 pm

RB2011UiAS-RM 6.38.5
ether1 = WAN
ether2 = MASTER (Cisco SG300-10), 6 virtual ints as gateway for corresponding DHCP scopes
ether3 = SLAVE2 (HP ProCurve 1810g)
ether4 = SLAVE2 (260GSP)

I am seeing this as well since at least 6.37.4 bugfix. I also moved over to current 6.38.5 to see if there was any difference and am still seeing the same thing. Gigabit fiber that pulls ~945mbit directly plugged into ONT on both macOS and Linux laptop, averaging around 500-650mbit with RB2011. Upload is unaffected, ~96mbit w/100mbit provisioned.

RB2011

rb2011_level3.PNG

macOS connected to ONT

ONT.PNG

ip filters

 0  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 
 1    ;;; blacklist
      chain=input action=drop src-address-list=blacklist in-interface=ether1 log=yes log-prefix="DROP_BLACKLIST   " 
 2    ;;; Drop DNS UDP in new
      chain=input action=drop connection-state=new protocol=udp in-interface=ether1 dst-port=53 log=yes 
      log-prefix="DROP_DNS_UDP" 
 3    ;;; drop DNS TCP in new
      chain=input action=drop connection-state=new protocol=tcp in-interface=ether1 dst-port=53 log=yes 
      log-prefix="DROP_DNS_TCP" 
 4    ;;; accept guest to lan
      chain=input action=accept protocol=udp src-address=172.16.33.0/24 dst-address=172.16.33.1 dst-port=53 log=no 
      log-prefix="" 
 5    chain=forward action=accept connection-state=established,related,new protocol=tcp src-address=172.16.33.0/24 
      dst-address=10.200.32.24 dst-port=443,36443,25565 log=no log-prefix="GUEST_ACCEPT" 
 6    chain=forward action=accept protocol=tcp src-address=172.16.33.0/24 dst-address=10.200.32.20 dst-port=31443 log=no 
      log-prefix="" 
 7    ;;; drop all guest to lan
      chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.100.25.0/24 log=no log-prefix="" 
 8    chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.100.27.0/24 log=no log-prefix="" 
 9    chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.200.16.0/23 log=no log-prefix="" 
10    chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.200.32.0/24 log=no log-prefix="GUEST_232" 
11    chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.200.48.0/23 log=no log-prefix="" 
12    chain=forward action=drop src-address=172.16.33.0/24 dst-address=10.200.64.0/24 log=no log-prefix="" 
13    ;;; drop guest to gateways
      chain=input action=drop src-address=172.16.33.0/24 dst-address=10.100.25.1 log=yes log-prefix="GUEST_100" 
14    chain=input action=drop src-address=172.16.33.0/24 dst-address=172.16.33.1 log=yes log-prefix="GUEST_1633" 
15    chain=input action=drop src-address=172.16.33.0/24 dst-address=10.100.27.1 log=yes log-prefix="GUEST_127" 
16    ;;; drop wifi to admin
      chain=input action=drop src-address=10.200.48.0/23 dst-address=10.100.25.1 log=yes log-prefix="DROP_WIFI_2_ADMIN" 
17    ;;; drop Time Machine from VPN
      chain=forward action=drop src-address=10.200.16.0/24 dst-address=10.200.64.239 log=no log-prefix="" 
18    ;;; L2TP/IPSEC VPN
      chain=input action=accept connection-state=new protocol=udp in-interface=ether1 dst-port=500,4500 log=yes 
      log-prefix="L2TP_IPSEC" 
19    chain=input action=accept connection-state=new protocol=udp in-interface=ether1 dst-port=1701 log=yes 
      log-prefix="L2TP_IPSEC_pol" ipsec-policy=in,ipsec 
20 XI  chain=input action=accept connection-state=new connection-nat-state="" protocol=ipsec-esp in-interface=ether1 log=yes 
      log-prefix="L2TP_IPSEC50" 
21 XI  chain=input action=accept connection-state=new protocol=ipsec-ah in-interface=ether1 log=yes log-prefix="L2TP_IPSEC51" 
22    ;;; reject icmp, net prohibited
      chain=input action=reject reject-with=icmp-net-prohibited connection-state=established,related 
      connection-nat-state=dstnat protocol=icmp in-interface=ether1 log=yes log-prefix="REJECT_ICMP" 
23    ;;; public btest - mikrotik
      chain=input action=accept protocol=udp src-address=207.32.195.2 log=no log-prefix="" 
24 XI  chain=input action=accept protocol=udp src-address=50.235.23.218 log=no log-prefix="" 
25    ;;; default configuration - accept
      chain=input action=accept connection-state=established,related log=no log-prefix="" 
26    ;;; default configuration - drop unsolicited
      chain=input action=drop in-interface=ether1 log=yes log-prefix="DROP      -  " 
27    ;;; default configuration
      chain=forward action=fasttrack-connection connection-state=established,related log=no log-prefix="" 
	  28    ;;; default configuration
      chain=forward action=accept connection-state=established,related log=no log-prefix="" 
29    ;;; default configuration
      chain=forward action=drop connection-state=invalid log=no log-prefix=""

NAT

 0    ;;; default config
      chain=srcnat action=masquerade out-interface=ether1 log=no log-prefix="" 
 1    ;;; subsonic
      chain=dstnat action=dst-nat to-addresses=10.200.32.24 to-ports=36443 protocol=tcp in-interface=ether1 dst-port=36443 log=no log-prefix="" 
 2    ;;; mozsync
      chain=dstnat action=dst-nat to-addresses=10.200.32.20 to-ports=31443 protocol=tcp in-interface=ether1 dst-port=31443 log=no log-prefix="" 
 3    ;;; ownCloud
      chain=dstnat action=dst-nat to-addresses=10.200.32.24 to-ports=443 protocol=tcp in-interface=ether1 dst-port=443 log=no log-prefix=""

btest_udp_rb2011.PNG

I'm also seeing less than expected results using btest locally across vlans that traverse the RB2011. iperf and SMB to/from Windows and Linux hosts are unaffected and pull 105+MB/sec with CPU not touching 100%. In the screen above using btest CPU is always maxed out. With this same config prior to 6.37.4 & 6.38.5 the RB2011 was pulling between 840-880mbit/sec solid, never wavered over the last 12 months.

pukkita · Wed Apr 05, 2017 9:23 am

In the screen above using btest CPU is always maxed out. With this same config prior to 6.37.4 & 6.38.5 the RB2011 was pulling between 840-880mbit/sec solid, never wavered over the last 12 months.

Do you mean on a btest done on the 2011 itself?

toxicfusion · Wed Apr 05, 2017 6:00 pm

Do you have a drop all input rule?? Please create (Security reasons). Create your required DST-NAT rules beforehand.

Furthermore, be sure have filter rule: (This to allow local LAN traffic)
chain=forward action=accept src-address=172.16.33.0/24

I have numerous RB2011 out in production in the wild, no said issues with WAN performances. Especially when using port1 (Gigabit port). Please try speed test again after above rules..

philamonster · Wed Apr 05, 2017 7:23 pm

Thank you for replies.

@pukkita - Yes, the btest on rb2011 itself. I presume this is to be expected? The browser-based speed test never pushes CPU past 75% at current speed results.

@toxicfusion
I do have rule to drop incoming connections:

...
26    ;;; default configuration - drop unsolicited
      chain=input action=drop in-interface=ether1 log=yes log-prefix="DROP      -  "
...

And 172.16.33.0/24 is guest network that I don't allow to touch non-guest networks with the exception of hosted services on vlan232. At least that is my plan. I'll have to look at rules again specifically as these were configured some time ago and functioned as intended.

I've downgraded to 6.37.4 bugfix and further back to current 6.37.1 and still see this issue. Now on 6.37.5 bugfix. I've spoken with my ISP and given that when I connect a device directly to ONT and get full speed I am provisioned for they claim the issue most likely resides in the rb2011. They moved me from one provider to another about a month ago but again, no issues when connected to ONT on macOS or linux box though I immediately noticed after the switch when going through rb2011. There was also some damage to my ISP's network around the same time due to high winds but claims are now made that everything has been restored. The quality of my connection does not seem to be affected regardless.

I really have no idea at this point what to even try short of restoring the rb2011 to factory defaults and starting over.

pukkita · Thu Apr 06, 2017 11:47 am

Thank you for replies.

@pukkita - Yes, the btest on rb2011 itself. I presume this is to be expected? The browser-based speed test never pushes CPU past 75% at current speed results.

Yes, because you're taxing the 2011 CPU with btest, whereas passing traffic through the 2011 don't need the additional btest process.

Did you check System > Routerboard Current Firmware?

I really have no idea at this point what to even try short of restoring the rb2011 to factory defaults and starting over.

Better reset it to no defaults.

toxicfusion · Thu Apr 06, 2017 5:50 pm

for your "chain=input action=drop in-interface=ether1 log=yes log-prefix="DROP"

do not log this. eats up flash cycles / memory. Just my personal preference.

No need to run btest. try running www.speedtest.net

philamonster · Thu Apr 13, 2017 3:57 am

So everything seems to have taken care of itself. I defaulted the router out and still had issues. Loaded previous config and re-opened a ticket with my ISP but didn't bother to check speeds for a couple days. Consistently now getting ~850-880mbit/sec without much deviation after last two days across various test sites. When I originally purchased the router there were similar issues though much more severe. My ISP has to have made adjustments both times for these issues to have cleared up. Monitoring and waiting to hear back....

kevintitus81 · Thu Apr 13, 2017 5:02 pm

What is your wan link negotiating at? I have seen some issues in the past where the ISP side (link partner) was advertising half duplex, and so my WAN link was linking at half capacity.

I would check that out, make sure the link partner is advertising and linking to the proper speed/duplex. Once the ISP forced their CPE to gigabit link I saw improved download speeds.

philamonster · Fri Apr 14, 2017 3:20 am

Auto-negotiation set to enabled. Everything seems to be normal now but I will keep an eye on it.

advertising: 10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
link-partner-advertising: 10M-half,10M-full,100M-half,100M-full,1000M-full

50% bandwidth loss RB2011UiAS

50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Re: 50% bandwidth loss RB2011UiAS

Who is online