Master SSID dropping to kbs speeds on all cAP ac units

Kaldek · Mon Oct 10, 2022 2:49 pm

UPDATE: This issue appears resolved. Please scroll down to the posts from late December 2022

Hi folks, I have four cAP ac units all managed via CAPsMAN with two SSIDs, one for the regular network and one for IoT devices. It has been this way for a couple of years with no issues whatsoever. However, it seems somewhere around ROS 7.3.1 or so, my master SSID on all APs regularly drops to between 50kbs and 1Mbs maximum throughput. This is happening what feels like every 30 minutes or so, and lasts a couple of minutes each time. My slave SSID is not affected, ever. If you change SSIDs, the speeds go back to normal.
All devices are affected. PCs, Android phones, iPads, you name it. The issue does not occur on all APs at the same time; if you roam to a different AP, it is likely the issue is not happening on that AP at that time.

Ethernet is unaffected. It is not a router issue or an issue with the VLAN.

I have tried a whole bunch of things, none of which have made a difference:
- Messing with channels
- Messing with channel widths
- Messing with rates (NOTE! "Messing with rates" did not include going back to default rates)
- Using an RB4011 (WiFi model) to see if it is related to the IPQ-4019 in the cAP ac
- Local Forwarding vs. CAPsMAN forwarding
- Connecting the APs directly to the router (I have an RB5009 PoE model, as well as a CRS328).
- Downgrading to earlier ROS versions (I did not go lower than 7.2.3)
- Upgrading to beta versions (currently on 7.6beta10)

Here is the CAPsMAN config. Feel free to pick it apart, but note that it worked for a long time with no evident issues.

/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=XX frequency=5180 name=5G-CH36-Bedroom tx-power=30
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2412 name=2G-CH1-Bedroom tx-power=15
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2437 name=2G-CH6-Garage tx-power=13
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2462 name=2G-CH11-Living tx-power=15
/caps-man channel add band=5ghz-onlyac control-channel-width=20mhz extension-channel=XXXX frequency=5260 name=5G-CH52-Hallway tx-power=40
/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=Ce frequency=5500 name=5G-CH100-Garage tx-power=20
/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=Ce frequency=5745 name=5G-CH149-Living tx-power=40
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2412 name=2G-CH1-Hallway tx-power=12
/caps-man rates add basic=12Mbps comment=12Mbs+ name=2Ghz-rate supported=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps
/caps-man rates add basic=18Mbps,24Mbps,36Mbps,48Mbps,54Mbps comment=18Mbs+ name=5ghz-rate supported=24Mbps,36Mbps,48Mbps,54Mbps
/caps-man security add authentication-types=wpa2-psk disable-pmkid=yes encryption=aes-ccm group-encryption=aes-ccm group-key-update=1h name=wpa2psk
/caps-man access-list add action=accept allow-signal-out-of-range=10s disabled=no interface=all signal-range=-85..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=reject allow-signal-out-of-range=10s comment="Kick host off AP if signal is bad" disabled=no interface=all signal-range=-120..-86 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man configuration add datapath.bridge=bridge .client-to-client-forwarding=yes .local-forwarding=no .vlan-id=10 .vlan-mode=use-tag name=config_IOT security=wpa2psk ssid=melonhead-iot
/caps-man configuration add datapath.bridge=bridge .client-to-client-forwarding=yes .local-forwarding=no .vlan-id=1 .vlan-mode=no-tag name=config_LAN security=wpa2psk ssid=melonhead
/caps-man interface add channel=2G-CH6-Garage configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:E2 master-interface=none name=Garage-2g radio-mac=08:55:31:02:66:E2 radio-name=Garage-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:E2 master-interface=Garage-2g name=Garage-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Garage-2g
/caps-man interface add channel=5G-CH100-Garage configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:E3 master-interface=none name=Garage-5g radio-mac=08:55:31:02:66:E3 radio-name=Garage-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:E3 master-interface=Garage-5g name=Garage-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Garage-5g
/caps-man interface add channel=2G-CH1-Hallway configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:AE master-interface=none name=Hallway-2g radio-mac=08:55:31:02:66:AE radio-name=Hallway-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:AE master-interface=Hallway-2g name=Hallway-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Hallway-2g
/caps-man interface add channel=5G-CH52-Hallway configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:AF master-interface=none name=Hallway-5g radio-mac=08:55:31:02:66:AF radio-name=Hallway-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:AF master-interface=Hallway-5g name=Hallway-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Hallway-5g
/caps-man interface add channel=2G-CH11-Living configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:94 master-interface=none name=Living-2g radio-mac=08:55:31:02:66:94 radio-name=Living-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:94 master-interface=Living-2g name=Living-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Living-2g
/caps-man interface add channel=5G-CH149-Living configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:95 master-interface=none name=Living-5g radio-mac=08:55:31:02:66:95 radio-name=Living-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:95 master-interface=Living-5g name=Living-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Living-5g
/caps-man interface add configuration=config_LAN disabled=no l2mtu=1600 mac-address=48:8F:5A:3B:C6:42 master-interface=none name=ap-bedroom-2g radio-mac=48:8F:5A:3B:C6:42 radio-name=488F5A3BC642 rates=2Ghz-rate
/caps-man interface add channel=2G-CH1-Bedroom configuration=config_IOT disabled=no l2mtu=1600 mac-address=4A:8F:5A:3B:C6:42 master-interface=ap-bedroom-2g name=ap-bedroom-2g-iot radio-mac=00:00:00:00:00:00 radio-name=4A8F5A3BC642
/caps-man interface add channel=5G-CH36-Bedroom configuration=config_LAN disabled=no l2mtu=1600 mac-address=48:8F:5A:3B:C6:43 master-interface=none name=ap-bedroom-5g radio-mac=48:8F:5A:3B:C6:43 radio-name=bedroom-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=4A:8F:5A:3B:C6:43 master-interface=ap-bedroom-5g name=ap-bedroom-5g-iot radio-mac=00:00:00:00:00:00 radio-name=4A8F5A3BC643
/caps-man manager set enabled=yes
/caps-man manager interface set [ find default=yes ] forbid=yes
/caps-man manager interface add disabled=no interface=bridge
/caps-man provisioning add action=create-enabled master-configuration=config_LAN name-format=identity slave-configurations=config_IOT

Note that I have not tried using a non-CAPsMAN configuration on any devices yet.

Marino · Mon Oct 10, 2022 3:11 pm

There might be something wrong with your rates:
/caps-man rates add basic=18Mbps,24Mbps,36Mbps,48Mbps,54Mbps comment=18Mbs+ name=5ghz-rate supported=24Mbps,36Mbps,48Mbps,54Mbps

You miss 18Mbps in your supported rate. And check your basic rate settings. Might be the best to switch them off see how it goes.

Kaldek · Tue Oct 11, 2022 1:09 am

There might be something wrong with your rates:
/caps-man rates add basic=18Mbps,24Mbps,36Mbps,48Mbps,54Mbps comment=18Mbs+ name=5ghz-rate supported=24Mbps,36Mbps,48Mbps,54Mbps

You miss 18Mbps in your supported rate. And check your basic rate settings. Might be the best to switch them off see how it goes.

Those are intentional as per guidance from Ron Touw's videos on WiFi Layer 1 tuning for Mikrotik. And that config has been in place for over 18 months without issue.
Also, as stated the secondary SSID is unaffected and it is subject to the same radio settings as it is just as Virtual AP.

Marino · Tue Oct 11, 2022 3:28 pm

This is the correct setting from Ron Touw guidance:
/caps-man rates add basic=12Mbps comment=12Mbs+ name=2Ghz-rate supported=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps

Check the setting for 5Ghz rate or try to disable them and use the standard rates.

Note for BartozP => please don't edit my posts. Thanks

Kaldek · Fri Oct 14, 2022 2:23 am

This is the correct setting from Ron Touw guidance:
/caps-man rates add basic=12Mbps comment=12Mbs+ name=2Ghz-rate supported=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps

Check the setting for 5Ghz rate or try to disable them and use the standard rates.

I had a good 24 hours where the issue didn't occur before it triggered again, but I've now put these settings in place to see if it improves anything.
It is still very odd that this didn't happen for a long time and the only thing that has changed is ROS versions. Sure, I could have added additional WiFi devices into the house that I've forgotten about but it wouldn't be much.

Kaldek · Sun Oct 23, 2022 5:24 am

All APs and my router are now running ROS 7.6 stable for 4 days. So far, I have not seen this issue return. I don't know if it was 7.6 that fixed the issue or the rate settings.

robmaltsystems · Sun Oct 23, 2022 3:04 pm

Out of interest, are you running v7.x on the cAP acs? I've got a site that's proving quite challenging - Wi-Fi fine most of the time but occasionally it goes horrible. The router is underpowered so we're considering replacing it with a more powerful unit - and going to v7.x at the same time.

Kaldek · Mon Oct 24, 2022 4:28 am

Out of interest, are you running v7.x on the cAP acs?

Yeah, my cAP ac units are nearly always ahead of my router but everything has been running 7.X. All devices are on 7.6 (stable) now.

Kaldek · Mon Nov 28, 2022 1:57 am

Well folks this issue came back, and hard. No changes were made aside from my RB5009 losing power recently and all the network gear rebooting. All of the cAP ac units are powered by the RB5009 (it's the PoE model).
I have disabled CAPsMAN and gone back to individual AP configs to see if this resolves the problem. All devices are running ROS 7.6.

There is other weirdness too, and I would like to hear from the community:

Now that I'm not using CAPsMAN, my WiFi speeds have picked up by about 50Mb/s and I wasn't even using CAPsMAN forwarding; all the APs were local forwarding.
My secondary SSID is on VLAN 10. CAPsMAN puts the virtual WiFi interfaces (e.g. wifi3/wifi4) into the bridge as tagged interfaces. However if I do this using individual AP configs, no traffic from the VLAN can flow; the wlan3/wlan4 interfaces need to be PVID 10 and not tagged

I really am at a loss as to what the occasional speed drops to the kbs range are caused by when using CAPsMAN or even direct configs. CAPsMAN vs direct config is posted here for comparison:

/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=XX frequency=5180 name=5G-CH36-Bedroom tx-power=30
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2412 name=2G-CH1-Bedroom tx-power=15
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2437 name=2G-CH6-Garage tx-power=15
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2462 name=2G-CH11-Living tx-power=15
/caps-man channel add band=5ghz-onlyac control-channel-width=20mhz extension-channel=XX frequency=5260 name=5G-CH52-Hallway tx-power=40
/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=Ce frequency=5500 name=5G-CH100-Garage tx-power=20
/caps-man channel add band=5ghz-n/ac control-channel-width=20mhz extension-channel=Ce frequency=5745 name=5G-CH149-Living tx-power=40
/caps-man channel add band=2ghz-g/n control-channel-width=20mhz extension-channel=disabled frequency=2412 name=2G-CH1-Hallway tx-power=12
/caps-man rates add basic=12Mbps comment=12Mbs+ name=2Ghz-rate supported=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps
/caps-man rates add basic=12Mbps comment=12Mbs+ name=5ghz-rate supported=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps
/caps-man security add authentication-types=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm name=wpa2psk
/caps-man access-list add action=accept allow-signal-out-of-range=10s comment="Accep Nest Doorbell regardless of signal" disabled=no mac-address=D8:EB:46:CB:FB:79 signal-range=-120..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=accept allow-signal-out-of-range=10s comment="Accept Tesla Powerwall regardless of signal" disabled=no mac-address=CC:88:26:18:76:B2 signal-range=-120..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=accept allow-signal-out-of-range=10s comment="Allow home theatre fan regardless of signal" disabled=no mac-address=50:8A:06:82:EF:DA signal-range=-120..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=accept allow-signal-out-of-range=10s comment="Accept Airtouch Control Panel" disabled=no mac-address=E4:83:2A:C5:02:7E signal-range=-90..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=accept allow-signal-out-of-range=10s disabled=no interface=all signal-range=-83..120 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man access-list add action=reject allow-signal-out-of-range=10s comment="Kick host off AP if signal is bad" disabled=no interface=all signal-range=-120..-84 ssid-regexp="" time=0s-1d,sun,mon,tue,wed,thu,fri,sat
/caps-man configuration add datapath.bridge=bridge .client-to-client-forwarding=yes .local-forwarding=yes .vlan-id=10 .vlan-mode=use-tag name=config_IOT security=wpa2psk ssid=melonhead-iot
/caps-man configuration add datapath.bridge=bridge .client-to-client-forwarding=yes .local-forwarding=yes name=config_LAN security=wpa2psk ssid=melonhead
/caps-man interface add channel=2G-CH6-Garage configuration=config_LAN disabled=no mac-address=08:55:31:02:66:E2 master-interface=none name=Garage-2g radio-mac=08:55:31:02:66:E2 radio-name=Garage-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:E2 master-interface=Garage-2g name=Garage-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Garage-2g
/caps-man interface add channel=5G-CH100-Garage configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:E3 master-interface=none name=Garage-5g radio-mac=08:55:31:02:66:E3 radio-name=Garage-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:E3 master-interface=Garage-5g name=Garage-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Garage-5g
/caps-man interface add channel=2G-CH1-Hallway configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:AE master-interface=none name=Hallway-2g radio-mac=08:55:31:02:66:AE radio-name=Hallway-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:AE master-interface=Hallway-2g name=Hallway-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Hallway-2g
/caps-man interface add channel=5G-CH52-Hallway configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:AF master-interface=none mtu=0 name=Hallway-5g radio-mac=08:55:31:02:66:AF radio-name=Hallway-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:AF master-interface=Hallway-5g name=Hallway-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Hallway-5g
/caps-man interface add channel=2G-CH11-Living configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:94 master-interface=none name=Living-2g radio-mac=08:55:31:02:66:94 radio-name=Living-2g rates=2Ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:94 master-interface=Living-2g name=Living-2g-iot radio-mac=00:00:00:00:00:00 radio-name=Living-2g
/caps-man interface add channel=5G-CH149-Living configuration=config_LAN disabled=no l2mtu=1600 mac-address=08:55:31:02:66:95 master-interface=none mtu=0 name=Living-5g radio-mac=08:55:31:02:66:95 radio-name=Living-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=0A:55:31:02:66:95 master-interface=Living-5g name=Living-5g-iot radio-mac=00:00:00:00:00:00 radio-name=Living-5g
/caps-man interface add configuration=config_LAN disabled=no l2mtu=1600 mac-address=48:8F:5A:3B:C6:42 master-interface=none name=ap-bedroom-2g radio-mac=48:8F:5A:3B:C6:42 radio-name=488F5A3BC642 rates=2Ghz-rate
/caps-man interface add channel=2G-CH1-Bedroom configuration=config_IOT disabled=no l2mtu=1600 mac-address=4A:8F:5A:3B:C6:42 master-interface=ap-bedroom-2g name=ap-bedroom-2g-iot radio-mac=00:00:00:00:00:00 radio-name=4A8F5A3BC642
/caps-man interface add channel=5G-CH36-Bedroom configuration=config_LAN disabled=no l2mtu=1600 mac-address=48:8F:5A:3B:C6:43 master-interface=none mtu=0 name=ap-bedroom-5g radio-mac=48:8F:5A:3B:C6:43 radio-name=bedroom-5g rates=5ghz-rate
/caps-man interface add configuration=config_IOT disabled=no l2mtu=1600 mac-address=4A:8F:5A:3B:C6:43 master-interface=ap-bedroom-5g name=ap-bedroom-5g-iot radio-mac=00:00:00:00:00:00 radio-name=4A8F5A3BC643
/caps-man manager set ca-certificate=auto certificate=auto enabled=yes package-path=/ upgrade-policy=suggest-same-version
/caps-man manager interface set [ find default=yes ] forbid=yes
/caps-man manager interface add disabled=no interface=bridge
/caps-man provisioning add action=create-enabled master-configuration=config_LAN name-format=identity slave-configurations=config_IOT

And here is the direct (not CAPsMAN) config for one of the APs. All APs are the same except for frequencies and that I don't do any of the basic rate edits.

/interface bridge add admin-mac=48:8F:5A:3B:C6:41 auto-mac=no comment=defconf ingress-filtering=no name=bridge vlan-filtering=yes
/interface bridge port add bridge=bridge ingress-filtering=no interface=ether2
/interface bridge port add bridge=bridge ingress-filtering=no interface=wlan1
/interface bridge port add bridge=bridge ingress-filtering=no interface=wlan2
/interface bridge port add bridge=bridge ingress-filtering=no interface=ether1
/interface bridge port add bridge=bridge interface=wlan3 pvid=10
/interface bridge port add bridge=bridge interface=wlan4 pvid=10
/interface bridge vlan add bridge=bridge tagged=bridge,ether1 vlan-ids=10

interface wireless channels add band=2ghz-g/n frequency=2412 list=2G-List name=2G-CH1 width=20
/interface wireless channels add band=5ghz-n/ac extension-channel=Ce frequency=5180 list=5G-List name=5G-CH36 width=20

/interface wireless security-profiles add authentication-types=wpa2-psk mode=dynamic-keys name=LAN 
/interface wireless security-profiles add authentication-types=wpa2-psk mode=dynamic-keys name=IOT 
/interface wireless set [ find default-name=wlan1 ] band=2ghz-g/n disabled=no frequency=2G-CH1 installation=indoor mode=ap-bridge security-profile=LAN ssid=melonhead wps-mode=disabled
/interface wireless set [ find default-name=wlan2 ] band=5ghz-n/ac channel-width=20/40mhz-Ce disabled=no distance=indoors frequency=5G-CH36 installation=indoor mode=ap-bridge security-profile=LAN ssid=melonhead wireless-protocol=802.11 wps-mode=disabled
/interface wireless add disabled=no mac-address=4A:8F:5A:3B:C6:42 master-interface=wlan1 name=wlan3 security-profile=IOT ssid=melonhead-iot vlan-id=10 wds-default-bridge=bridge wps-mode=disabled
/interface wireless add disabled=no mac-address=4A:8F:5A:3B:C6:43 master-interface=wlan2 name=wlan4 security-profile=IOT ssid=melonhead-iot vlan-id=10 wds-default-bridge=bridge wps-mode=disabled

robmaltsystems · Mon Nov 28, 2022 10:18 am

The site I mentioned is still using CAPsMAN on v6. I was about to switch the four cAP ac access points to direct configuration, but the client has reported that it's been okay since we implemented a nightly reboot of the entire system (not ideal!).

I had switched one access point over to direct with what sounds like a similar configuration except not using VLAN on the main network, just for the guest network. I'm rushing out now so will get the detailed configuration later. Pretty sure it's using tagging.

Kaldek · Mon Nov 28, 2022 12:59 pm

Well, scratch my good news. I had yet another dropout, even on the direct config.

Once again, the same behaviour:

Only happens on the Master SSID, not the Slave SSID (Virtual interface)
Lasts for a minute or two and them comes back

I've done so many different things which haven't worked. So frustrating.

anav · Mon Nov 28, 2022 8:01 pm

I never used capsman and never had issues.

erlinden · Mon Nov 28, 2022 8:51 pm

Is it prohibited to the 5GHz radio, or does it also happen on the 2.4GHz radio?
Do all accesspoints have the same behaviour?
Does the logging give you any clue?

Sure the VLAN section is correct? I'm missing the frame-types=admit-only-untagged-and-priority-tagged on the /interface brdige port.
But my knowledge on VLAN is very limited (though I know I prefer using VLAN's for all networks).

Perhaps (also) have a look at this great topic:
viewtopic.php?t=143620

Ca6ko · Mon Nov 28, 2022 9:15 pm

You spend so much time dealing with the problem of low data rate through the radio channel and never once!!! scanned the airwaves.
The search for problems with data transmission over the air has to start with the radio!
The beacons of the main access point and the virtual one are shifted in time. If there is an interfering station on the same channel, the beacons periodically coincide.

Kaldek · Mon Nov 28, 2022 11:09 pm

Hold on chum, just because I haven't specifically stated that, doesn't mean I didn't do it. This radio environment has channels carefully selected so that there are either no other radios on the same channel, or those radios have a seen signal of lower than -92dB.

Kaldek · Mon Nov 28, 2022 11:10 pm

Righto, I might be onto something here. Yesterday before the issue occurred again I had re-entered the cusom rates on all APs.
I have now set all APs to the default rates, so let's see how stable this is.

The one remaining additional edit is access lists which boot hosts off an AP if the signal is -83dB or worse.

toxicfusion · Tue Nov 29, 2022 2:23 am

Are all your AP's the cAP AC units? I had similar random 5Ghz drop-outs using very similar CAPsMan deployment for a restaurant. I believe the cAP AC was faulty / flaky -- 5Ghz was not stable. I ended up replacing the cAP AC unit with another brand [tp-link]. as the MikroTik cAP XL AC [larger AC, higher gain] was back-ordered.

I've had better performance and feel the XL AC units are more reliable.

I recall seeing a few RouterOS notes about "fix 5Ghz stability on AC". I'd have to find the releases where MikroTik noted it.

robmaltsystems · Tue Nov 29, 2022 10:36 am

The venue I'm struggling with is a busy bar and ""fix 5Ghz stability on AC" resonates with me. What appears to happen is that 5GHz stops working (took me a while to twig) and devices are falling back to the very heavily congested 2.4GHz band. So, speed plumets from a respectable 200-300Mbps to around 20Mbps. We're now rebooting the access points each night which appears to help.

Kaldek · Tue Nov 29, 2022 11:24 am

The venue I'm struggling with is a busy bar and ""fix 5Ghz stability on AC" resonates with me. What appears to happen is that 5GHz stops working (took me a while to twig) and devices are falling back to the very heavily congested 2.4GHz band.

In my case the clients don't leave the 5Ghz band. And, it's only on the master SSID on that band - the secondary SSID is still fast. Speeds are anywhere from 15Kb/s to 250Kb/s but no more.

Kaldek · Tue Nov 29, 2022 11:25 am

Are all your AP's the cAP AC units? I had similar random 5Ghz drop-outs using very similar CAPsMan deployment for a restaurant. I believe the cAP AC was faulty / flaky -- 5Ghz was not stable.

They are, but I also replaced one with an RB4011 (in CAPsMAN, so its radio config was identical) and it still happened.

Anyway I'm waiting to see if the issue is resolved now that I've removed the alterations to the supported Basic Rates.

Tue Nov 29, 2022 11:29 am

I've had better performance and feel the XL AC units are more reliable.

How would that be explained ?
Apart from antenna they are HW-wise identical. Same CPU, same wifi chip, same memory, same flash, ... everything the same.

robmaltsystems · Tue Nov 29, 2022 11:31 am

Also, I read a review of them that wasn't exactly glowing. Not sure where it was but I think they were disappointed with the radios.

bpwl · Tue Nov 29, 2022 11:14 pm

Ron Touw is an expert on wifi tuning. However if you apply one of his tuning tips to the wrong environment you make things worse.

Why would one rise the basic rate above 6 Mbps?
1. Because there are a lot of SSID/AP points, broadcasting at 6 Mbps and consuming 0.5% of the airtime in the channel for every SSID+AP instance. (If you have 3 SSID on 2 AP, you consume 3% of all airtime in that channel, with 18Mbps as lowest basic rate only 1%)
2. Because there is a lot of multicast data to be transferred, and you did not set the Multicast Helper to Full. Lots of Multicast data, sent as multicast in wifi, will almost always fully destroy a good wifi environment.

Why NOT rise the basic rate above 6 Mbps?
1. Because the environment is not fully clear field without disturbances and short range connections. Not being able to correctly receive the beacon (at basic rate) will give connection drops. With multicast and beacon transmitted there is no CRC check, no ACK, and no retransmit, as is with unicast. (Unicast is retried nx7 times before the transmission fails on disconnect)
2. Because you prefer unnoticable retries above full disconnects. Setting high minimum basic rates gives self-inflicted disconnects. *
3. Because you want to use the full physical range of the AP-client connections

Be carefull with tuning, and know why you are doing some setting. Just copying tips, even from the best guru, can be very contra-productive.
Just one remark: extension channel XX or XXXX is giving you what channel? It is undefined with XX and XXXX. So it can be different with every interface restart, or every CAPsMAN channel reselection (which I hope is set to OFF)

* : the same is true for high minimum supported rates.:You are not letting the wifi driver lower the inerface rate, and will get disconnected instead.
I'm pleased that disabling lower supported rates and lower basic rates is not possible anymore in 802.11ac (VHT MCS rates are 0-9 supported, and 0-7 (0 till 7) basic rates). No way to remove the lower rates!

bpwl · Tue Nov 29, 2022 11:36 pm

action=reject allow-signal-out-of-range=10s comment="Kick host off AP if signal is bad" disabled=no interface=all signal-range=-120..-86

Even with a good signal (>-86dBm) for 9.99 sec the host is still rejected !?

Kaldek · Wed Nov 30, 2022 1:56 am

You're misreading it. It's "-86 or worse for more than ten seconds" (-120 to -86) not "-86 or better".

Kaldek · Wed Nov 30, 2022 2:04 am

Just one remark: extension channel XX or XXXX is giving you what channel? It is undefined with XX and XXXX. So it can be different with every interface restart, or every CAPsMAN channel reselection (which I hope is set to OFF)

This is a bit of a throwback to when I didn't know "what was best" for which channel is the control (management) channel and which is the extension channel. It's Ce/eC with 40Mhz channel width and (I probably don't recall this correctly) Ceee/eCee/eeCe/eeeC with 80Mhz channels. My understanding of this is that if there is cross-channel interference from another AP, your management channel should be the same as their management channel. There's some really good data on this from the Wireless LAN Professionals Youtube channel on how wireless performance can be crushed if there are two APs on the same "channel" but their control channels don't overlap, compared to if you have the same management channel as the interfering AP.

According to documentation and what I've witnessed, the XX or XXXX setting doesn't affect channel selection (e.g, it's still "channel 36") but it just affects which of the channels are the control channel. Not relevant for 20Mhz channel widths of course.

Kaldek · Wed Nov 30, 2022 2:12 am

1. Because the environment is not fully clear field without disturbances and short range connections. Not being able to correctly receive the beacon (at basic rate) will give connection drops. With multicast and beacon transmitted there is no CRC check, no ACK, and no retransmit, as is with unicast. (Unicast is retried nx7 times before the transmission fails on disconnect)
2. Because you prefer unnoticable retries above full disconnects. Setting high minimum basic rates gives self-inflicted disconnects. *
3. Because you want to use the full physical range of the AP-client connections

I'm deciphering that as meaning that for an already connected client even with an RSSI of -41dB that the client will miss the beacon, disconnect itself, reconnect itself, miss the beacon again, etc etc.

I turned on debug logging for each AP and did not see disconnect/reconnect entries though.

Kaldek · Wed Nov 30, 2022 7:32 am

It's been just over a day and I've not had any performance issues.

I'm going to let this bed in for a few days to a week, to see if it was indeed an alteration of the basic rates settings.

mkx · Wed Nov 30, 2022 9:18 am

According to documentation and what I've witnessed, the XX or XXXX setting doesn't affect channel selection (e.g, it's still "channel 36") but it just affects which of the channels are the control channel. Not relevant for 20Mhz channel widths of course.

Channel layout is pretty much standardized, you can see the table here. Many devices tolerate "non standard" channel layouts, but some are picky and won't use e.g. 80MHz channel width if channel layout is not correct, they will revert to 40MHz or even 20MHz.
As far as I observed, XX or XXXX setting selects correct (standard) channel layout. If one manually sets frequency (which is center frequency of control channel), then this mostly indirectly defines also channel layout used. Example: if one uses XX (40MHz) and sets frequency to 5240, then actual channel layout will be eC ... if frequency is set to 5220, then actual channel layout will be Ce ... and in both cases it'll be 40MHz channel numbered 46.

The question of control channel co-existence with adjacent APs still remains though. It would be nice if somebody with more experience with equipment of different wireless vendors would write how other vendors tend to select channel layouts when left to automatic settings.

bpwl · Wed Nov 30, 2022 10:21 am

Easy to test that XX and XXXX setting, and see what you get in Status. It worked well for me at the edge (5180) of the possible range to be in line with the standard list, but it went wrong higher up in the frequencies. viewtopic.php?t=183744#p918305

Have not found yet another brand that has the XX or XXXX option, to check if those are "smarter".

bpwl · Wed Nov 30, 2022 10:26 am

I'm deciphering that as meaning that for an already connected client even with an RSSI of -41dB that the client will miss the beacon, disconnect itself, reconnect itself, miss the beacon again, etc etc.

I turned on debug logging for each AP and did not see disconnect/reconnect entries though.

Is was meant for the other posts where the CAP ac is called flacky or faulty.
One missed beacon does probably not drop the connection yet, but multicast (basic rate) packet data will be lost, what does not happen with unicast.

bpwl · Wed Nov 30, 2022 10:41 am

Even with a good signal (>-86dBm) for 9.99 sec the host is still rejected !?
You're misreading it. It's "-86 or worse for more than ten seconds" (-120 to -86) not "-86 or better".

I did not express the case well. If you get out of range (signal drops < -86dBm, that is in the -120..-86 range) for a bigger time than the "signal-out-of-range" in the allow rule, you get kicked out, for at least the duration of the signal-out-of-range time in the deny rule. Here 10 sec.

Now consider a home (or bar) with 2 AP, where people are busy and travel around getting in and out of range of either of the AP at random pace. They need to be kicked out to avoid the "sticky client" and now connect to the nearby AP. But when they get back in range of the first AP, they will not be accepted for 10 sec. It's easy to get in the case where MT roaming is said not to work, because the client is denied access when been out of range. (Happens often with indoor AP's in separate rooms, or with indoor/outdoor AP, or when people walk in/out of range )

Kaldek · Wed Nov 30, 2022 11:30 pm

Well, bad news. The issue occurred again. I do have some more interesting facts however:

During the incident, any devices connected to the secondary/slave SSID had no performance issues (200+Mbs)
During the incident, the affected device stayed connected and reported 400/400 Tx/Rx speeds with an RSSI of -42dB
The speed issue would happen in cycles. A few minutes of OK, followed by a few minutes of terrible speeds

At this point I'm out of ideas. I guess I'll have to put myself through the drama of sending supout.rif files and "please do a netinstall" from support.

bpwl · Thu Dec 01, 2022 1:05 am

This looks like not a wifi (RF signal) problem as such.
400/400Mbps interface rate is excellent for 40MHz/2S/SGI connection. No connection drop-out or unexpected roaming.
The virtual WLAN (2nd SSID) keeps performing well on the same channel, you say. So airtime and co-channel interference is no problem, no channel overload. No adjacent channel interference as this would drop the interface rate after a while. (Because of Faulty transmissions : TX HW Frames >> TX Frames)

There are many other problems possible, linked to the first SSID data traffic path.

Kaldek · Thu Dec 01, 2022 4:54 am

I've just noticed that even with the basic rates set back to "default" it's kept the old config with the specified rates.
Very weird. I'm now resetting the WLAN configuration for the interfaces and creating them again.

There are many other problems possible, linked to the first SSID data traffic path.

The next time this happens I'm going to run a bandwidth test from the RB5009 to the AP where it's happening and see if there are any L2 Ethernet issues on the native VLAN.
I do seem to recall that evening pinging the AP's own IP address on the bridge (from an affected wireless client) was slow though, so the ethernet based test may not be useful.

Kaldek · Thu Dec 01, 2022 5:21 am

If you get out of range (signal drops < -86dBm, that is in the -120..-86 range) for a bigger time than the "signal-out-of-range" in the allow rule, you get kicked out, for at least the duration of the signal-out-of-range time in the deny rule. Here 10 sec.

Interesting. I did not know the client was placed in a banned list for a period of time. I didn't see this in the documentation anywhere!

Thu Dec 01, 2022 8:58 am

It is there but with a mental twist...

There are the following parameters for access list rules:

client matching parameters:
address - MAC address of the client
interface - optional interface to compare with the interface to which client actually connects to
time - time of day and days when rule matches
signal-range - range in which client signal must fit for the rule to match
allow-signal-out-of-range - option which permits client's signal to be out of the range always or for some time interval

https://help.mikrotik.com/docs/display/ ... AccessList

Kaldek · Thu Dec 01, 2022 11:58 am

Yeah, I read that one. It doesn't say that the client is added to a ban list though, and regardless of RSSI even if it moves back into range.

bpwl · Thu Dec 01, 2022 6:09 pm

There is an important question mark in viewtopic.php?p=969849#p969648
As happens more with Mikrotik, the documentation is not very specific.

All depends on how it is implemented (programmed).
If "allow-signal-out-of-range" means that the rule will remain active even if the signal is not within the specified range anymore for the set time.
Everyone understands the "authentication" case. You are still allowed to authorize, even is your signal is not in the specified range anymore. The "authentication" rules matches and is granted for the specified time.

If that logic is applied for the rule with "no-authentication", then that rule would still apply even if the signal strength is not in that deny-range anymore for the set 'allow-out-of-range' time.

But then there is "Only the first matching rule is applied." What happens if the 'other' range rule comes before the rule with "allow-out-of-range". Which one is applied then?
There should be no difference in handling for the "authentication" and "no-authentication" rules, unless the programmer decided to make it different.

I think, some reverse engineering is required, by doing some tests with long "allow-out-of-range" settings.

I had it wrong. Both rules seem to be implemented differently. The client is not locked out. Not even when the no-authentication comes before the authentication in the list.
Experiment is quite confusing, with the "banned/last failure", but the lockout is not as I expected.

Klembord-2.jpg

LOG start when out of range for authorize, after being in range long ago.
Reconnected to WLAN2 to check the LOG. Hope this had no influence. But if ROS maintains a MAC address based table it could have been the case.
Should have used 2 clients for the experiment.

Klembord-3.jpg

Done the test with a second client device. The WLAN2 login was not the cause to switch from denied to accepted, before the allow timer expired.

So now it seems that the "allow signal out of range" timer only applies to an established connection. When the association drops, the rule timer drops.
This answers the question how that timer would have been used with other properties (VLAN mode, VLAN id, private password) combined with signal range.
Avoiding a flapping connection for some time, with the "no authentication" rule is not possible this way. You cannot deny a client access for a defined time because it recently failed the signal range required.
A disconnect will invalidate/terminate any "allow out of range" timer. So if you lose connection while you work in the "allow signal out of range" time , and try to reconnect, this will fail because you lost the right to authenticate with signal out of range.
You have to be again in the required signal range to reconnect.

There is clearly such a thing as "banned/last failure" but that seems to be only for some non adjustable seconds, not related to the "allow signal out of range" timer.

Sorry for posting my initial confusion.

Kaldek · Mon Dec 05, 2022 3:01 am

Small update - the issue is still occurring but at the moment it is less often, and only seems to last for a few seconds. So few in fact that if I reach for my laptop which is wired via Ethernet I don't even have time to run some ethernet-level tests before the issue goes away again.

Kaldek · Sat Dec 10, 2022 6:05 am

The latest update is that this issue appears to have become rare. It's happened a couple of times but only for brief periods.

I can only surmise at this point that there is something which is affected by the rates settings but not tied to it?

I can only leave it as-is and keep updating this thread if the behaviour changes.

Kaldek · Sun Dec 18, 2022 7:28 am

Another update, 8 days since the last post.

No detected issues during this time.

Kaldek · Tue Dec 20, 2022 7:31 am

I have now moved all APs back to being under CAPsMAN management. No issues detected in the following 24 hours.

I'm going to call this one soon with a "solved" status about this being related to the custom rates.

Kaldek · Thu Dec 22, 2022 12:57 am

Folks I'm going to call this one resolved. The solution was to not alter the default rates, either in CAPsMAN or if configuring the APs directly.

Who is online