Routerboard/RouterOS losing serious connection

gmac63 · Sat Jan 27, 2007 5:51 am

All,

I am not sure of the root cause of this but I need some input on why my Routerboard 532A, RouterOS 2.9.39 is seriously dropping connection when I connect as a bridge. Here is my configuration:

                  [Routerboard 532A/RouterOS]
<-->WAN (MetroE)<------>bridge(eth2/eth3)<------>LAN (HP Procurve switch)<-->
                           MGMT(eth1)<----------------^

When connected in this manner, I lose 45-50% of the packets, as I see from pinging from a WAN side device to a LAN side device. When I remove the Routerboard, the connection is 100%, no packet loss.

This is very frustrating as I want to use this as a traffic/bandwidth control device. Currently I cannot even get a consistent ping.

I can send my exported config if that would help.

All help is greatly appreciated.

-Wes Yates

gmac63 · Sat Jan 27, 2007 5:07 pm

To clarify, I have eth1 as a Management port to do config and traffic graphing with winbox. The other two ports on the 532 are bridged. Even with no queueing (other than the natural default) and no packet marking, there should be nothing to cause the bridge to lose packets/connectivity.

The MetroE is a BellSouth Fiber WAN, 10 Mbs, full duplex. The LAN is of course 100Mbs full duplex.

The LAN consists of about 100 workstations connecting to a main site. There are 7 other remote sites that connect to the main site for a total of 9 sites. Traffic out of the one site in question is about 4-5 Mbps

This issue does not seem to be present on my home network between my cable Internet adapter (modem) and my LAN's gateway/router (Linux running SmoothWall).

-Wes

bushy · Sat Jan 27, 2007 8:32 pm

Try turning off "Auto negotiation " on the ethernet ports , i think it can disagree with some gear on the far end.

gmac63 · Sun Jan 28, 2007 7:52 pm

Try turning off "Auto negotiation " on the ethernet ports , i think it can disagree with some gear on the far end.

Oh, Yes I agree. I indeed did and failed to mention that Auto negotiation was off at the time. I had to force 10/full on both ends so I turned Auto negotiation off.

Any other possibilities?

Thank you for responding.

-Wes

andrewluck · Sun Jan 28, 2007 7:59 pm

Bad cable?

Regards

Andrew

gmac63 · Sun Jan 28, 2007 11:01 pm

Bad cable?

Regards

Andrew

The Tech at the site thought of that and changed them. No luck. Good thought tho! Thanks.

-Wes

andrewluck · Sun Jan 28, 2007 11:23 pm

Post your bridge config here.

Regards

Andrew

gmac63 · Mon Jan 29, 2007 5:27 am

Ok, its rather long, but...

# jan/10/2007 09:33:20 by RouterOS 2.9.35
# software id = JZ94-3TT
#
/ interface ethernet 
set MGMT name="MGMT" mtu=1500 mac-address=00:0C:42:0F:02:C1 arp=enabled \
    disable-running-check=yes auto-negotiation=yes full-duplex=yes \
    cable-settings=default mdix-enable=yes speed=100Mbps comment="" disabled=no 
set WAN name="WAN" mtu=1500 mac-address=00:0C:42:0F:02:C2 arp=enabled \
    disable-running-check=yes auto-negotiation=no full-duplex=yes \
    cable-settings=default speed=10Mbps comment="" disabled=no 
set LAN name="LAN" mtu=1500 mac-address=00:0C:42:0F:02:C3 arp=enabled \
    disable-running-check=yes auto-negotiation=no full-duplex=yes \
    cable-settings=default speed=10Mbps comment="" disabled=no 
/ interface wireless security-profiles 
set default name="default" mode=none authentication-types="" unicast-ciphers="" \
    group-ciphers="" wpa-pre-shared-key="" wpa2-pre-shared-key="" \
    eap-methods=passthrough tls-mode=no-certificates tls-certificate=none \
    static-algo-0=none static-key-0="" static-algo-1=none static-key-1="" \
    static-algo-2=none static-key-2="" static-algo-3=none static-key-3="" \
    static-transmit-key=key-0 static-sta-private-algo=none static-sta-private-key="" \
    radius-mac-authentication=no group-key-update=5m 
/ interface wireless align 
set frame-size=300 active-mode=yes receive-all=no audio-monitor=00:00:00:00:00:00 \
    filter-mac=00:00:00:00:00:00 ssid-all=no frames-per-second=25 audio-min=-100 \
    audio-max=-20 
/ interface wireless snooper 
set multiple-channels=yes channel-time=200ms receive-errors=no 
/ interface wireless sniffer 
set multiple-channels=no channel-time=200ms only-headers=no receive-errors=no \
    memory-limit=10 file-name="" file-limit=10 streaming-enabled=no \
    streaming-server=0.0.0.0 streaming-max-rate=0 
/ interface bridge 
add name="L56Bridge" mtu=1500 arp=enabled stp=yes priority=32768 ageing-time=5m \
    forward-delay=15s garbage-collection-interval=5s hello-time=2s max-message-age=20s \
    comment="" disabled=no 
/ interface bridge port 
add interface=WAN bridge=L56Bridge priority=128 path-cost=10 comment="" disabled=no 
add interface=LAN bridge=L56Bridge priority=128 path-cost=10 comment="" disabled=no 
/ interface l2tp-server server 
set enabled=no max-mtu=1460 max-mru=1460 authentication=pap,chap,mschap1,mschap2 \
    default-profile=default-encryption 
/ interface pptp-server server 
set enabled=no max-mtu=1460 max-mru=1460 authentication=mschap1,mschap2 \
    keepalive-timeout=30 default-profile=default-encryption 
/ ip ipsec proposal 
add name="default" auth-algorithms=sha1 enc-algorithms=3des lifetime=30m lifebytes=0 \
    pfs-group=modp1024 disabled=no 
/ ip accounting 
set enabled=no account-local-traffic=no threshold=256 
/ ip accounting web-access 
set accessible-via-web=no address=0.0.0.0/0 
/ ip service 
set telnet port=23 address=0.0.0.0/0 disabled=yes 
set ftp port=21 address=0.0.0.0/0 disabled=yes 
set www port=80 address=0.0.0.0/0 disabled=no 
set ssh port=22 address=0.0.0.0/0 disabled=no 
set www-ssl port=443 address=0.0.0.0/0 certificate=none disabled=yes 
/ ip upnp 
set enabled=no allow-disable-external-interface=yes show-dummy-rule=yes 
/ ip arp 
/ ip socks 
set enabled=no port=1080 connection-idle-timeout=2m max-connections=200 
/ ip dns 
set primary-dns=0.0.0.0 secondary-dns=0.0.0.0 allow-remote-requests=no \
    cache-size=2048KiB cache-max-ttl=1w 
/ ip dns static 
add name="L56DNS" address=10.1.4.7 ttl=1d 
/ ip traffic-flow 
set enabled=no interfaces=all cache-entries=4k active-flow-timeout=30m \
    inactive-flow-timeout=15s 
/ ip address 
add address=10.2.56.50/16 network=10.2.0.0 broadcast=10.2.255.255 interface=MGMT \
    comment="added by setup" disabled=no 
/ ip proxy 
set enabled=no src-address=0.0.0.0 port=8080 parent-proxy=0.0.0.0:0 \
    cache-administrator="webmaster" max-disk-cache-size=none \
    max-ram-cache-size=unlimited cache-only-on-disk=no maximal-client-connections=1000 \
    maximal-server-connections=1000 max-object-size=4096KiB max-fresh-time=3d 
/ ip neighbor discovery 
set MGMT discover=yes 
set WAN discover=yes 
set LAN discover=yes 
set L56Bridge discover=yes 
/ ip route 
add dst-address=0.0.0.0/0 gateway=10.2.2.40 scope=255 target-scope=10 comment="added by \
    setup" disabled=no 
/ ip firewall mangle 
add chain=prerouting protocol=tcp src-port=20000-50000 dst-port=20-21 \
    action=mark-packet new-packet-mark=src_FTPpassive passthrough=no comment="" \
    disabled=no 
add chain=prerouting protocol=tcp src-port=20-21 dst-port=20000-50000 \
    connection-mark=dst_FTP_passive_conn action=mark-packet \
    new-packet-mark=dst_FTPpassive passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=20-21 action=mark-packet \
    new-packet-mark=src_FTPactive passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=20-21 action=mark-packet \
    new-packet-mark=dst_FTPactive passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=445 action=mark-packet \
    new-packet-mark=src_microsoft-ds passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=445 action=mark-packet \
    new-packet-mark=dst_microsoft-ds passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=80 action=mark-packet new-packet-mark=dst_80 \
    passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=80 action=mark-packet new-packet-mark=src_80 \
    passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=443 action=mark-packet \
    new-packet-mark=dst_443 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=443 action=mark-packet \
    new-packet-mark=src_443 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=137-139 action=mark-packet \
    new-packet-mark=src_NetBIOS passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=137-139 action=mark-packet \
    new-packet-mark=dst_NetBIOS passthrough=yes comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=524 action=mark-packet \
    new-packet-mark=src_524 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=524 action=mark-packet \
    new-packet-mark=dst_524 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=873 action=mark-packet \
    new-packet-mark=src_873 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=873 action=mark-packet \
    new-packet-mark=dst_873 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp src-port=8080 action=mark-packet \
    new-packet-mark=src_8080 passthrough=no comment="" disabled=no 
add chain=prerouting protocol=tcp dst-port=8080 action=mark-packet \
    new-packet-mark=dst_8080 passthrough=no comment="" disabled=no 
/ ip firewall connection tracking 
set enabled=yes tcp-syn-sent-timeout=5s tcp-syn-received-timeout=5s \
    tcp-established-timeout=1d tcp-fin-wait-timeout=10s tcp-close-wait-timeout=10s \
    tcp-last-ack-timeout=10s tcp-time-wait-timeout=10s tcp-close-timeout=10s \
    udp-timeout=10s udp-stream-timeout=3m icmp-timeout=10s generic-timeout=10m \
    tcp-syncookie=no 
/ ip firewall service-port 
set ftp ports=21 disabled=no 
set tftp ports=69 disabled=no 
set irc ports=6667 disabled=no 
set h323 disabled=no 
set quake3 disabled=no 
set gre disabled=no 
set pptp disabled=no 
/ ip hotspot service-port 
set ftp ports=21 disabled=no 
/ ip hotspot profile 
set default name="default" hotspot-address=0.0.0.0 dns-name="" html-directory=hotspot \
    rate-limit="" http-proxy=0.0.0.0:0 smtp-server=0.0.0.0 login-by=cookie,http-chap \
    http-cookie-lifetime=3d split-user-domain=no use-radius=no 
/ ip hotspot user profile 
set default name="default" idle-timeout=none keepalive-timeout=2m status-autorefresh=1m \
    shared-users=1 transparent-proxy=yes open-status-page=always advertise=no 
/ ip dhcp-server config 
set store-leases-disk=5m 
/ system logging 
add topics=info prefix="" action=memory disabled=no 
add topics=error prefix="" action=memory disabled=no 
add topics=warning prefix="" action=memory disabled=no 
add topics=critical prefix="" action=echo disabled=no 
/ system logging action 
set memory name="memory" target=memory memory-lines=100 memory-stop-on-full=no 
set disk name="disk" target=disk disk-lines=100 disk-stop-on-full=no 
set echo name="echo" target=echo remember=yes 
set remote name="remote" target=remote remote=0.0.0.0:514 
/ system upgrade mirror 
set enabled=no primary-server=0.0.0.0 secondary-server=0.0.0.0 check-interval=1d \
    user="" 
/ system clock manual 
set time-zone=-05:00 dst-delta=+00:00 dst-start="jan/01/1970 00:00:00" \
    dst-end="jan/01/1970 00:00:00" 
/ system watchdog 
set reboot-on-failure=yes watch-address=none watchdog-timer=yes no-ping-delay=5m \
    automatic-supout=yes auto-send-supout=no 
/ system console 
add port=serial0 term="" disabled=no 
/ system identity 
set name="L56TC_MT" 
/ system note 
set show-at-login=yes note="  Welcome to the L56 Traffic Control System\n  Press <?> \
    for command menu.\n\n" 
/ system routerboard settings 
set baud-rate=9600 boot-delay=5s boot-device=nand-if-fail-then-ethernet \
    enter-setup-on=any-key cpu-mode=power-save memory-test=no cpu-frequency=264MHz \
    boot-protocol=bootp enable-jumper-reset=yes 
/ system ntp server 
set enabled=no broadcast=no multicast=no manycast=yes 
/ system ntp client 
set enabled=yes mode=unicast primary-ntp=66.187.233.4 secondary-ntp=66.187.224.4 
/ port 
set serial0 name="serial0" baud-rate=auto data-bits=8 parity=none stop-bits=1 \
    flow-control=hardware 
/ ppp profile 
set default name="default" use-compression=default use-vj-compression=default \
    use-encryption=default only-one=default change-tcp-mss=yes comment="" 
set default-encryption name="default-encryption" use-compression=default \
    use-vj-compression=default use-encryption=yes only-one=default change-tcp-mss=yes \
    comment="" 
/ ppp aaa 
set use-radius=no accounting=yes interim-update=0s 
/ queue type 
set default name="default" kind=pfifo pfifo-limit=50 
set ethernet-default name="ethernet-default" kind=pfifo pfifo-limit=50 
set wireless-default name="wireless-default" kind=sfq sfq-perturb=5 sfq-allot=1514 
set synchronous-default name="synchronous-default" kind=red red-limit=60 \
    red-min-threshold=10 red-max-threshold=50 red-burst=20 red-avg-packet=1000 
set hotspot-default name="hotspot-default" kind=sfq sfq-perturb=5 sfq-allot=1514 
add name="L56-DEFAULT" kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier="" \
    pcq-total-limit=2000 
add name="default-small" kind=pfifo pfifo-limit=10 
/ queue simple 
add name="baseBandwidthUp" dst-address=0.0.0.0/0 interface=WAN parent=none \
    direction=download priority=1 queue=L56-DEFAULT/L56-DEFAULT \
    limit-at=10000000/10000000 max-limit=10000000/10000000 total-queue=default-small \
    disabled=no 
add name="baseBandwidthDown" dst-address=0.0.0.0/0 interface=LAN parent=none \
    direction=download priority=1 queue=L56-DEFAULT/L56-DEFAULT \
    limit-at=10000000/10000000 max-limit=10000000/10000000 total-queue=default-small \
    disabled=no 
add name="8080Up" dst-address=0.0.0.0/0 interface=all parent=baseBandwidthUp \
    packet-marks=dst_8080 direction=both priority=2 queue=L56-DEFAULT/L56-DEFAULT \
    limit-at=0/0 max-limit=5000000/5000000 burst-limit=7000000/7000000 \
    burst-threshold=1544000/1544000 burst-time=5s/5s total-queue=default-small \
    disabled=yes 
add name="windowsCopyDown" dst-address=0.0.0.0/0 interface=all parent=baseBandwidthDown \
    packet-marks=src_microsoft-ds direction=both priority=2 \
    queue=L56-DEFAULT/L56-DEFAULT limit-at=0/0 max-limit=1000000/1000000 \
    burst-limit=2000000/2000000 burst-threshold=1544000/1544000 burst-time=5s/5s \
    total-queue=default-small disabled=yes 
add name="8080Down" dst-address=0.0.0.0/0 interface=all parent=baseBandwidthDown \
    packet-marks=src_8080 direction=both priority=2 queue=L56-DEFAULT/L56-DEFAULT \
    limit-at=0/0 max-limit=5000000/5000000 burst-limit=7000000/7000000 \
    burst-threshold=1544000/1544000 burst-time=5s/5s total-queue=default-small \
    disabled=yes 
add name="windowsCopyUp" dst-address=0.0.0.0/0 interface=all parent=baseBandwidthUp \
    packet-marks=dst_microsoft-ds direction=both priority=2 \
    queue=L56-DEFAULT/L56-DEFAULT limit-at=0/0 max-limit=1000000/1000000 \
    burst-limit=2000000/2000000 burst-threshold=1544000/1544000 burst-time=5s/5s \
    total-queue=default-small disabled=yes 
/ queue tree 
add name="public-out" parent=LAN packet-mark="" limit-at=0 queue=default priority=8 \
    max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 
add name="private-in" parent=WAN packet-mark="" limit-at=0 queue=default priority=8 \
    max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no 
/ user 
add name="admin" group=full address=0.0.0.0/0 comment="system default user" disabled=no 
/ user group 
add name="read" policy=local,telnet,ssh,reboot,read,test,winbox,password,web,!ftp,!write\
    ,!policy 
add name="write" policy=local,telnet,ssh,reboot,read,write,test,winbox,password,web,!ftp\
    ,!policy 
add name="full" policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,passwor\
    d,web 
/ user aaa 
set use-radius=no accounting=yes interim-update=0s default-group=read 
/ radius incoming 
set accept=no port=1700 
/ snmp 
set enabled=no contact="" location="" 
/ snmp community 
set public name="public" address=0.0.0.0/0 read-access=yes 
/ tool bandwidth-server 
set enabled=yes authenticate=yes allocate-udp-ports-from=2000 max-sessions=10 
/ tool mac-server ping 
set enabled=yes 
/ tool e-mail 
set server=0.0.0.0 from="<>" 
/ tool sniffer 
set interface=LAN only-headers=no memory-limit=10 file-name="" file-limit=10 \
    streaming-enabled=no streaming-server=0.0.0.0 filter-stream=yes \
    filter-protocol=ip-only filter-address1=0.0.0.0/0:0-65535 \
    filter-address2=0.0.0.0/0:0-65535 
/ tool graphing 
set store-every=5min 
/ tool graphing interface 
add interface=LAN allow-address=0.0.0.0/0 store-on-disk=yes disabled=no 
/ routing ospf 
set router-id=0.0.0.0 distribute-default=never redistribute-connected=no \
    redistribute-static=no redistribute-rip=no redistribute-bgp=no metric-default=1 \
    metric-connected=20 metric-static=20 metric-rip=20 metric-bgp=20 
/ routing ospf area 
set backbone area-id=0.0.0.0 type=default translator-role=translate-candidate \
    authentication=none prefix-list-import="" prefix-list-export="" disabled=no 
/ routing bgp 
set enabled=no as=1 router-id=0.0.0.0 redistribute-static=no redistribute-connected=no \
    redistribute-rip=no redistribute-ospf=no 
/ routing rip 
set redistribute-static=no redistribute-connected=no redistribute-ospf=no \
    redistribute-bgp=no metric-static=1 metric-connected=1 metric-ospf=1 metric-bgp=1 \
    update-timer=30s timeout-timer=3m garbage-timer=2m

MGMT = ether1
WAN = ether2
LAN = ether3

andrewluck · Mon Jan 29, 2007 11:09 pm

Just the interface and bridge config would have been sufficient for now :shock:

Two of your links are fixed at 10MBit full-duplex. This is unusual as 10Mbit is usually associated with hubs which are half-duplex.

Regards

Andrew

gmac63 · Tue Jan 30, 2007 5:44 am

Just the interface and bridge config would have been sufficient for now

Two of your links are fixed at 10MBit full-duplex. This is unusual as 10Mbit is usually associated with hubs which are half-duplex.

Regards

Andrew

Andrew,
Right, so I didn't know how much of the config might be in error or the root cause if any, so I sent it all.

Yes, MetroE in the US has a network adapter that is 10Mbit full duplex and setting any device other causes even more issues. This is really the setting for this.

So i the case of the Routerboard/RouterOS, setting an interface at full duplex in bridge mode is not advised?

Regards

-Wes

andrewluck · Tue Jan 30, 2007 6:35 pm

No problem with full-duplex that I'm aware of. I was just puzzled why your LAN connection is 10Mbit full-duplex. If your LAN is a switch then that should probably be set to 100Mbit full. If it's a hub, then 10Mbit half.

Regards

Andrew

gmac63 · Tue Jan 30, 2007 7:35 pm

No problem with full-duplex that I'm aware of. I was just puzzled why your LAN connection is 10Mbit full-duplex. If your LAN is a switch then that should probably be set to 100Mbit full. If it's a hub, then 10Mbit half.

Regards

Andrew

Yes, and rightly so. The LAN side of the Routerboard is an HP Procurve Switch set to 10Mbit/full as is required by the MetroE. Therefor I have set the bridge to reflect that as well.

Questions stands thought, any ideas why the Routerboard/RouterOS/my config is such an issue??? I can't install anymore until I know what might be a cause.

Currently, I have the Routerboard inline with my network here at home:

                        [Routerboard 532A/RouterOS]
<-->"WAN" (Cable modem)<--->bridge(eth2/eth3)<--->"LAN" (Linux based gateway)<-->Switch<--> rest of my LAN
                                 MGMT(eth1)<---------------------------------------^

Currently, there are no connectivity issues.

Regards

-Wes

andrewluck · Tue Jan 30, 2007 10:46 pm

Does the Procurve show any errors on the 532's port?

Only other thing I can suggest is to reset the config on the 532 and setup the bare minimum to bridge the ports. See if you still have the problem.

Regards

Andrew

gmac63 · Wed Jan 31, 2007 3:52 am

Does the Procurve show any errors on the 532's port?

Only other thing I can suggest is to reset the config on the 532 and setup the bare minimum to bridge the ports. See if you still have the problem.

Regards

Andrew

Good call. Initially the Procurve did have CRC redundancy errors on a previous run when I did not have 10Mbit/full/Auto Neg off, but the most recent try (still giving dropped connections), the Procurve showed no errors.

I have since reset the 532 and am using it "bare bones" here at my local home network to see if I get this issue.

Thank you again for helping and if anything in the config stands out as the culprit, let me know.

Regards
-Wes

gmac63 · Fri Mar 09, 2007 3:58 pm

The original and continuing issue seems to originate under the following conditions:

- The routerboard 532A is onsite at a location that has a fiber WAN adapter at the DEMARC.
- The adapters are older
- Any connecting device (switch,router) has to be set to 10Mb/Full Duplex (NO autoneg)
- The 532A does not seem to have this issue when connected to a newer filber WAN adapter set to 100Mb/Full Duplex (can be autoneg enabled)
- The 532A does introduce about 1ms more latency which is expected but does still cause brief loss of connections.

Oddly enough at one site, there naturally exists traffic "spikes" every 35 seconds lasting 6 seconds each. During this time, the 532A loses its connectivity for about 3-4 of those seconds. Traffic at that time has been measured to 2.5-3Mbit.

The issue also does not seem to exist when the device is installed until about 4-6 hours later. I spent an hour on the local LAN monitoring connectivity and found little to no loss of packets ( method: ping ).

Any more suggestions/information/advice is greatly welcome. I'm running out of options.

-Wes

Routerboard/RouterOS losing serious connection

Routerboard/RouterOS losing serious connection

More configuration infomation

UPDATE for anyone who can help...

Who is online