Hello everyone,
I inherited a geographic backbone made up of about 70 wireless devices (all Mikrotik, various models), where I'm having big stability problems.

The problem is that communications often suffer interruptions, even of about ten seconds and as the diameter of the network increases, ie the number of wireless devices, the problem gets worse. Sometimes WinBox disconnects after only a few minutes of activity and immediately afterwards it is already possible to log in. During the night, typically when there is less traffic, the problem is much less evident.

The backbone is divided into 6 segments, each physically connected to the wired network through a Mikrotik RB260GS switch. With an Access port, each switch encapsulates the traffic of its segment in a VLAN which is then conveyed over the wired network (Trunk). The wireless devices are therefore not aware of the existence of any VLAN.

The backbone is fully bridged: most of the wireless links are configured as ap-bridge <-> station-bridge (ROS L4), the rest as bridge <-> station-bridge (ROS L3). There are both P-t-P and P-t-MP links and they are all quite stable (almost all with link down = 0). Each wireless device only ever uses two interfaces, usually wlan1 and eth1, both bridged.

Except for manual changes, the devices that make up the backbone are always the same and RSTP is active in all of them with the default settings. (!) ROS 6.47.9 Long Term is installed in all wireless devices, SwOS 2.13 in the switches.

I'm monitoring the network with The Dude and, barring the above outages, all devices are reachable. However, I noticed that during the day the root-bridge changes frequently and I thought it might depend on the stability of some wireless link. Is it plausible?

Probably the best solution is to switch to a routed network but now I have to find a quick solution.

What could be the cause?
Does the election of a different root-bridge involve the loss of communication?
Since there are currently no redundant links and they are not provided, could disabling RSTP improve the situation?
If, on the other hand, there is actually a loop, how can I identify it without having to disconnect pieces of network in turn?

Please give me some advice.
Thank you!

On PtP with one ether and one wlan, how you can made a loop?
The problem of loop can happen only on a router/switch also set the admin mac on bridge interface, everytime choice the "WAN" interface as mac address

On PtP with one ether and one wlan, how you can made a loop?

Yes, a loop on this type of device is quite unlikely ...

However, since the wireless backbone originates from 6 different points in the wired network and 90% of the links are PtMP, a loop is still possible. Unfortunately, between a Station device and the AP that relays the wreless signal there is not always a switch or when there is sometimes it is not managed...

also set the admin mac on bridge interface, everytime choice the "WAN" interface as mac address

To prevent the devices from obtaining an IP address other than those reserved in the DHCP server, all bridges have the Mac Admin set but at the moment with the MAC of eth1. Would changing it with that of the wlan in use improve anything?

If I set the right RSTP priority in the various bridges, can I improve / fix the problem?
What about my the other questions?

Thanks for now.

1) However, since the wireless backbone originates from 6 different points in the wired network and 90% of the links are PtMP, a loop is still possible. Unfortunately, between a Station device and the AP that relays the wreless signal there is not always a switch or when there is sometimes it is not managed...

2) ...all bridges have the Mac Admin set...

3) If I set the right RSTP priority in the various bridges, can I improve / fix the problem?

4) What about my the other questions?

1) the only place where loop can happen are on switch / router, each client of the PTMP already have client isolation, true?

2) just for set the admin MAC, if you already have do it everywhere, do not matter.

3) I do not know the config of the other devices, on the forum is hard to reply without know all the variables on your network

4) If I not have reply to some question, probably is like poit 3) or I not have the time or I do not know the topic

I am managing a similar wifi-linked network and it is quite usual that brief interruptions occur, especially in summertime when there often are tropospheric effects that can influence both your links and the sensing of RADAR (DFS) that causes frequency changes and interruptions.
Indeed it is quite irritating that winbox disconnects after brief interruptions. I have requested that this is made configurable so you can select a longer timeout period, and also to be able to configure the "polling interval" so there is not so much traffic when you have opened windows like the firewall filters or connections. However, no reaction from MikroTik.

I would recommend to make the network routed instead of bridged, that usually solves lots of issues. But it may be a big undertaking to change it.
We use a (nearly) fully routed network with separate /29 subnets for links, and use BGP and BFD to autoroute it.

If the devices are all L2 and the problem increases as the network gets bigger, one of the problems you could be facing is spanning tree diameter with 70 devices out there. Generally, once you exceed 7 switch hops, the STP/RSTP timers need to be modified or the send/receive of BPDUs to calculate a loop-free topology will never completely finish and areas of the network will be susceptible to loops.

This is a good explanation (even though it's a Cisco doc, this problem affects all vendor implementations of STP and RSTP)

https://www.cisco.com/c/en/us/support/d ... #calculate

There are a few ways to mitigate this

1) Move to MSTP - it calculates diameter very differently and is better for large L2 topologies - up to 255 "hops"
2) Begin a migration to a routed network which can easily scale to thousands of devices.

Here is an article I wrote specifically for WISPs on migrating from bridged to routed.
https://stubarea51.net/2019/09/15/wisp- ... to-routed/

Ultimately, the problem you have likely isn't any one configuration setting, but rather the overall design - large L2 topologies in wireless networks are generally not a good idea.

Check the size (diameter) of your network. Old STP is 7 hops, we all know. RSTP can go up to 40 hops, but the default set is 20 (max-message-age) ! As is the case for MSTP (max-hops)

viewtopic.php?t=169122#p829283

Thanks for the tips guys, very helpful.
I come back to the topic because I had to change the network topology for other reasons but the problem remained. At the moment the network is conceptually similar to the following:


The configuration of the wireless devices has not changed: they always use only two interfaces (ether1 and wlan1), both bridged and RSTP is enabled with the default parameters.

For diagnostic purposes and to simulate the reduction of the diameter of the network (reduce the "active" bridges) I have temporarily disabled RSTP in the few RB260GS switches between AP and ST.
In these conditions, counting only the bridges in the AP and ST, the diameter of the real network (eg from SW0.n to SW1.n) is equal to 15, the wireless links crossed are a total of 7 and the network stability improves considerably.

If I re-enable RSTP in these switches I exceed the default max hops value currently in use and go back to the beginning.
So, I ask for scruple, if between each AP and ST I put (if there is not) a managed switch (eg RB260GS) with RSTP enabled and the right priority, I can disable RSTP on AP and ST without the risk of tripping over some loop?

I noticed that by disabling RSTP in the RB260GS the devices connected downstream of each port (eg AP1.2 and ST1.n) have the same root bridge but it is different from that of the devices connected to other ports (eg AP1.1). It appears that under these conditions each port blocks BPDU traffic. Possible?

Finally, for troubleshooting purposes, within The Dude I would like to view the current root bridge of the devices in the related labels. I saw that displays it but how do I extract only the portion of text following “root-bridge-id”?

Thank you.

You should rather put a ROUTER between the links, not a SWITCH.
Do not invest in RB260GS but in routers like RB750Gr3 (when you are on a tight budget) or RB3011/RB4011.
This allows you to break up the large broadcast domain.
The first place to do that is obviously where you now have SW0.0 but when there are other places in the network where 3 links come together, do it there too.

Furthermore, MikroTik switches are known to be not compliant with the standard that BPDU frames should never be forwarded. When that is your reason for inserting a switch, I would recommend to select another brand.

Yes, switching to a routed network is certainly in the plans but I have to get there step by step.

I am already using the RB750Gr3s almost everywhere to derive routed subnets from this backbone and create VPNs: small but powerful!
I'm using the RB260GS because with the 4 PoE ports I easily feed / control the antennas. I had read that they were bored but I did not remember what they were about. In the changelog I read some improvements related to RSTP from the 2.7 release with which they come out of the factory: did the incompatibilities remain even in SwOS 2.13? If they worked well with 5/6 units I would have filled the gaps between AP and ST and, waiting to switch to a routed network, I might have finished / solved.

Since the backbone is built exclusively with Mikrotik products and has no "intentional" redundant links, I wanted confirmation on the possibility of disabling RSTP on wireless devices and enabling it only on RB260GS.
Again, thanks.

On PtP with one ether and one wlan, how you can made a loop?
The problem of loop can happen only on a router/switch

Code: Select all

/int bri
set [find] protocol=none

also set the admin mac on bridge interface, everytime choice the "WAN" interface as mac address

Sorry rextended, you've probably already answered my question but just to avoid misunderstanding, given my network can I actually disable RSTP on wireless devices and enable it only on RB260GS?
Thank you.

the right is: (i have correct the error on my previous post about thaht)


Yes: it's the simple answer,
but it means:
do not use any form of STP (protocol-mode=none) when a peripheral has only one input (logical) and one output (logical), it makes no sense.
for example: all ptp and all ap with only one ethernet and only one wlan, all SLAVE PTMP, all AP with one ether and more than one wifi, but with different band (2/5)

Instead activate at least the RSTP in the peripherals that might have a loop (all devices with more than one (logical) in or more than one (logical) out interface)
for example: all switch, all MASTER PTMP, all AP with one (or more) ethernet but with more than one wifi with same band (both 2 or both 5)

Since

all ap with only one ethernet and only one wlan

so when you say

all MASTER PTMP

you mean all MASTERS PTMP with more than two interfaces?

In my case all the wireless devices configured as AP / MASTER PTMP have only two interfaces and therefore I can disable RSTP.
I learned?

i do not specify numer of interface on "all MASTER PTMP" because you have more than one wireless device linked,
and loop can happen only if two remote client are bridge together with a switch, for example load balancing or failover,
and if something go wrong, cause a loop

You can turn off RSTP on slave/client wireless device of PTMP, but on master PTMP, for the above reason, is better to turn it on

Great, all clear.
Grazie, ciao.