MikroTik

Hi everyone,

First off. I am no stranger to VLAN's and networking. I started playing around with this stuff on my old ProCurve switches in the late 2000's and since have successfully configured on ProCurve, HP, Aruba, and Mikrotik switches, as well as Unifi AP's and controllers, as well as managing those VLAN's in both ESXi and on Linux systems with KVM and LXC.

To be fair, while I have 5 Mikrotik switches in the house (CRS326-24S+2Q+, CRS317-1G-16S+, two CSS326-24G-2S+ and a little CRS305-1G-4S+) but I always opted to just use these in SwOS mode, as all I was doing was some layer 2 switching with VLAN's, no need for routing, so in retrospect maybe I should not have taken on my first RouterOS project and my first Mikrotik Wifi experience at the same time, but it is what it is. Trial by fire I guess.


Background (optional reading):

I have used Unifi wireless Access points since ~2010. In that time Ubiquiti Networks has morphed from a cool startup with really neat implementations that were upsetting the industry and were great for "pro-sumers" like myself, to being just another evil Enterprise vendor trying to twist users arms into doing things their way only, pushing more and more cloud nonsense, etc. I started getting annoyed when they decided to not just be an access point company, but also push their routers and switches, and integrate everything into the same controller, and show big red failure notifications when it couldn't find the Unifi routing and switching hardware (because I was using my own router and switches). The last straw came a few years ago when at the same time as they were having a highly publicized attack on their cloud services, they decided to discontinue the local server version of their Unifi Video product and force users onto their cloud.

I decided then and there I was going to get off of Unifi products as soon as I can, before they decided to force my local Unifi controller onto the cloud as well. I just didn't get around to it until now, because quite frankly, the 802.11ac WAP's I was using continued to work fine, and I really didn't have the need for more wireless bandwidth than that, as I am really more of a wired guy, and prefer all things wired.

But I finally got around to the "getting rid of Unifi AP's" project, in part because the Linux Unifi controller install I had broke and went into MongoDB dependency hell preventing me from updating it without breaking the controller, when the recent xz utils vulnerability came out, forcing me to either patch and break the controller spending hours trying to fix it again, or just make now the time I finally move away from Unifi. I chose the latter.

Since I already had (and liked) a bunch of Mikrotik switches, I decided to give Mikrotik AP's a try this time around and ordered two CAP ax units.


The Project:

In the current configuration I had three Unifi UAP-AC-LR WAP's. I didn't need that many (used to live in a larger house where I needed them for coverage) so I decided to get two Mikrotik CAP ax WAP's and just put one on each side of the house, as drop in replacements for two of the Unifi AP's (the third is unnecessary)

the VLAN trunks were already set up and working on the Unifi WAP's, so I figured it would just be a drop in replacement and off to the races.

I did decide that I kind of liked the "manage all of my Wifi from a central location" experience, and since I didn't want asymmetrical configuration with CAPSMAN running on one of the CAP's but not on the other, I just decided to install CHR in a VM on my KVM box, replacing the Unifi controller, and serving as a simple central controller for the two AP's.

When I went to set these up, I found out the hard way that by default these access points are set up for routing (which I find really weird, as I've never once wanted to have a WIFI access point do any routing, but whatever)

I already have a custom built 2U Xeon server running OPNSense for 10gig routing and firewall services which I am able to push gigabit speeds over WireGuard with, so I kind of wanted to keep it as the main router, and not worry about anything else.

Just like the existing Unifi setup the Wireless network needs to have three SSID's:

1.) Main LAN (VLAN 1)
2.) Untrusted IoT (VLAN 22)
3.) Guest (VLAN 5)


Initial Setup:

For this step, I relied on the Mikrotik Youtube video for CAPSMAN here:

https://www.youtube.com/watch?v=37aff6d14Xk

Things that are known to work are the VLAN's on the switches and OPNSense router, as they worked just fine in the previous configuration with all the VLAN's already configured.

Here is a high level diagram (with non-relevant servers and clients omitted)
Before plugging in the CAP ax units for the first time, I had already configured my CHR VM install to work in bridge mode, assigned it a static IP and enabled CAPSMAN in it.

I initially struggled with getting the CHR install CAPSMAN to find and be able to control the two CAP ax units because I wasn't expecting an AP to be set up as a Router, with Firewall and NAT enabled.

Once I got past that hurdle though, I was able to log into the devices, and on each of them:
1.) Update to latest RouterOS (7.14.2)
2.) Disable NAT and all services (DHCP, etc.)
3.) Expand the default bridge1 (encompassing the two wireless devices, wifi1 and wifi2 and ether2) to include ether1 as well, as this is the port I need to use without NAT as it is the port that supports incoming PoE.
4.) Change IP to static and set it to an IP matching my Main LAN range (10.0.1.0/24)
5.) Enable CAP, and point it towards the IP for my CHR Capsman
6.) configure each of the wireless devices to be managed by CAPSMAN, and set the datapath to use bridge1

Success. The four wireless devices now showed up in CAPSMAN. Time to start configuring SSID's.

There are a lot more configuration options than I am used to for wifi. I decided to start with a rather basic configuration to get things up and running quickly, figuring I can always fine tune things later. No one likes downtime. (especially my untrusted wifi thermostats while we are still in heating season)

First created a security profile named "Main". It is configured with WPA2 PSK, and WPA3 PSKm with CCMP, GCMP, CCMP 256 and GCMP 256 enabled. Additionally a passphrase has been assigned.

Secondly I created a configuration profile for the main LAN, also named "Main", and use my previously configured security provfile. Set it to "ap" mode, assigned an SSID, and did not assign a VLAN. I also assigned the country to the United States. (Strangely enough, the configuration keeps forgetting my country setting. Not sure why, or if it is important)

I then set all four of the wifi devices to use the "main" configuration profile.

And it worked. The SSID appeared immediately. Well.. Sortof. My Android phone is able to see and log into the SSDI's. As is my Linux laptop. The same laptop when dual booted to Windows 10, does not see the Mikrotik SSID's at all for some reason. At first I thought maybe it was because the aging Intel Wireless-AC 7260 Mini-PCie WLAN card couldn't handle WPA3, but then why did it work in Linux? And I also have WPA2 enabled. It also shouldn't be a cipher problem, as I have everything except TKIP enabled (which I will never enable) so it should be working. Either way, I digress. Might be an outdated driver issue or something. It's not a big deal, I can troubleshoot that later.


The Problem

This is where I was starting to feel pretty confident. Things were working! (And damn are these Mikrotik CAP ax units fast compared to the old Unifi AC LR units I used to use.)

Part of the issue I think I am having is that most guides/posts I find when googling are for the old CAPSMAN used with 802.11ac WAPs, not the new CAPSMAN for ax devices.

Anyway, I started clicking around looking for a way to add my second and third SSID's (to be configured with VLANs 22 and 5) and I got stuck.

Some googling later, and it turns out that in order to have multiple SSID's on the same radio, I have to create slave devices subordinate to the main wireless devices. So in the CHR management interface, I created a slave for each of the four wifi devices.

Then I created a second security profile (same authentication/cipher options as previously known to work in the "main" profile, and a second configuration profile using that security profile, a different SSID and also adding VLAN 22 to datapath settings, and assigned it to the slaves.

I got it partially right, because almost immediately I saw my pesky untrusted thermostats show up under registration. This is were things broke down though. The DHCP server on my OPNSense router (set to VLAN 22 and previously working with Unifi on this VLAN) never sees the two thermostats. I also cannot log in to the SSID for VLAN22 from my laptop in Linux. While the main LAN works, it just tries and tries and tries and then fails.

Things I have tried:

1.) Enabling VLAN filtering on the bridges set up on the CAP's to make sure they can transfer the VLAN data from the wifi devices out to the switches. I configured them as follows: EtherType 0x8100, PVID 1 (matching main LAN). Frame Types admit all, Ingress Filtering, Yes. This appears to ahve had no effect. It doesn't help VLAN22 get through to the upstream CSS326 switch, but it also doesn't appear to break main lan traffic, despite it not being tagged with VLAN 1.

2.) I tried changing the configuration profile on the main LAN to tag it with VLAN 1, and that just resulted in the main LAN wifi becoming unusable. So VLAN traffic is just not getting through to the upstream CSS326 switches.

3.) I tried creating VLAN22 devices on the CAPs linked to the bridge, but this doesn't seem to have done anyhting (and I am not even sure it is supposed to be necessary, unless I want to assign the CAPs an IP on the VLAN for DHCP or something like that, which I don't.)

4.) While it shouldn't be necessary (as no wifi traffic hits the CHR, as it only communicates configuration data to the CAPS over the main LAN) I also did #2 and #3 above to the CHR install as well, with no effect.

5.) When setting up the second SSID's I was a little bit during the configuration as notably in the configuration in CAPSMAN, in the dropdowns under datapath, the only bridge I see is the bridge configured on the CHR VM, not the bridges on the CAP devices which the wifi devices actually need to use to reach the CSS326 switch. This seems like a potential issue. To test tthis I ran some high bandwidth activity over wifi, and looked for traffic activity and found that while I observed high traffic on the wifi devices during my test, I saw no traffic at all on the configured bridge, which is very strange, as the traffic would have HAD TO cross the bridge to reach the switch. I have no idea what is going on there, but suspect this may be a contributing cause for my issues.

6.) I did some more googling, and reading of the Mikrotik Wiki. I came across the "Layer2 Misconfiguration" page, and notably the subsection for "VLAN Interface on a Slave Device"


Essentially what this article says is that a slave device cannot carry VLAN, as that needs to be on the master.

I can't for the life of me figure out how to fix it though.

As mentioned above, I tried setting VLAN1 on the master Wifi device, and that just resulted in not being able to communicate with the upstream CSS326 switch.

It would seem (to me, unless I got something wrong) that the only way to configure additional SSID's is by using slave devices, but if slave devices can't assign VLAN's, then how is this supposed to work?

The most likely reason you'd want additional SSID's is if you are going to use VLAN's on them, and that would necessitate slave devices.

I am thus thoroughly confused about the RouterOS way of doing things, and would greatly appreciate anyone who could give me a few suggestions.

I appreciate anyone who would take the time to read through everything I have written. You are the true community forum heroes.

Thanks again, and appreciate any input at all!

--Matt

This part of the documentation should be sufficient:

https://help.mikrotik.com/docs/display/ ... Fi-CAPsMAN:

When interested, only read the part of the wifi-qcom driver beneath.

Also as a reformed UNIF--KER...

I was bit hard by the shortcoming of Mikrotik's AC radios.

I did a few hundred caps-man system, but configured 1 cap system with the WAVE2 drivers, and that was the end of that.

Caps-man is awesome in concept and gives you instant info about roams and stuff... but Mikrotik JUST GOT 1 OUTDOOR RADIO out there for AX and its 24 Volts. And from the ANT series were I used to really laugh and all the problems other Mikrotik radio users would bitch about.

I was forced back to Ruckus long ago and have since added Cambium. Both work on STANDARDS for VLANs that make them easy to deal with across multiple manufactures. But while their interfaces may not give me the instant access to data like caps-man did when I wasted thousands of hours on it and lost maybe 50,000 replacing radios... Ruckus and Cambium work better as RADIOS SERVING CLIENTS.

This part of the documentation should be sufficient:

https://help.mikrotik.com/docs/display/ ... Fi-CAPsMAN:

When interested, only read the part of the wifi-qcom driver beneath.

I will take a look, thank you.

Also as a reformed UNIF--KER...

I was bit hard by the shortcoming of Mikrotik's AC radios.

I did a few hundred caps-man system, but configured 1 cap system with the WAVE2 drivers, and that was the end of that.

Caps-man is awesome in concept and gives you instant info about roams and stuff... but Mikrotik JUST GOT 1 OUTDOOR RADIO out there for AX and its 24 Volts. And from the ANT series were I used to really laugh and all the problems other Mikrotik radio users would bitch about.

I was forced back to Ruckus long ago and have since added Cambium. Both work on STANDARDS for VLANs that make them easy to deal with across multiple manufactures. But while their interfaces may not give me the instant access to data like caps-man did when I wasted thousands of hours on it and lost maybe 50,000 replacing radios... Ruckus and Cambium work better as RADIOS SERVING CLIENTS.

I have to admit, I share some of your frustrations.

I'm no spring chicken when it comes to setting up networking, but Mikrotik's RouterOS is just beyond convoluted for absolutely no good reason.

Mikrotik has some great hardware, and they sell it at surprisingly decent pricing. As noted above I have 5 mikrotik switches of differing performance levels in active use (and one on a box I'm not using right now) and they are all excellent products performing very well, and doing their job.

I think part of the problem with Mikrotik is this "everything is a router" mentality.

Like, with the CAP ax. There is absolutely no reason why - by default - a wireless access point should set itself up as a router with one port NAT:ed and firewalled. It makes zero sense. That has to be 0.001% of the use cases for a product like this. The overwhelming majority of users are likely going to set up an access point on an internal preconfigured network with NAT/firewall and DHCP already set up, and no NAT/Firewall/DHCP needed needed on the device.

Now to add insult to injury, Ether1, the port you need for PoE in is set up as a WAN port by default, frustrating initial setup. On the flipside - however - if you were to plug Ether2 into your switch you might tank your entire network as you wind up with dueling DHCP servers.

But the problem isn't just that "everything is a router". It is also that RouterOS is seemingly in a state of eternal Beta with a known errata list the size of a J.R.R. Tolkien novel. What they call Betas are closer to Alphas and what they call stable releases are closer to Beta. I keep running into functionality that does not work as intended, and even when it does work as intended it requires users to get way too much into the weeds of the details of master and slave interfaces and how this impacts VLAN gheaders, etc. etc. Something any product like this should handle on its own and never bother a user with.

if I were them, I'd rewrite RouterOS from scratch (or get rid of it completely) to make it more usable and intuitive as a router product. (Maybe look to pfSense or OPNSense for inspiration), and limit it to being used on a handful of dedicated router products, and transition everything else to something like SwOS-only.

There is no reason every goddamn switch and wireless access point should by default be setup like a router. It seems so divorced from the products real world uses to be mind boggling to understand what they are thinking. Most of the products (while excellent in Layer2 applications) don't even have the CPU power to support anything but very basic routing, so why even bother offering RouterOS on them?

Anyway, now I am just frustrated and ranting.

So, as recommended in this other thread with a very similar topic I went back and reset the CAP ax units to CAPSMAN mode.

I then re-paired them with CAPSMAN on the CAPSMAN device.

I applied my main SSID config and security profiles. So far so good. Main SSID works.

In CAPSMAN I added slave interfaces for all four WIFI devices. Since someone suggested NOT using configuration profiles, I set up the SSID and VLAN ports (22 in my case) and applied the security profile directly on the interfaces.

For a brief second it looked like it was working. Devices seem to have connected and grabbed IP's from the DHCP server assigned to VLAN 22, but it was not to be. The devices were quickly non-responsive.

Then I logged in on each of the the CAPs themselves and clicked on the auto-generated bridge (bridgeLocal) and added vlan22 as follows:

Tagged: bridge and Ether1
Untagged: The two slave Wifi interfaces
Also, as mentioned in that thread, I DID NOT enable VLAN filtering in the bridge, as I was told that breaks things.

Now I'm able to get my phone to connect to the SSDI on VLAN22 and it actually pulls an ip from the VLAN22 DHCP, but nothing else can seem to connect to it. Not my Laptop (which connects just fine to the main LAN wifi, with everything the same except VLAN) nor the thermostats this VLAN is for.

Edit:

The phone connecting to the VLAN 22 SSID seems highly intermittent. It works sometimes, but not other times, and the pings on th elocal networkj to the phone when it is connected are all over the place from single digit ms all the way up to 200ms.

Anyone have any suggestions or pointers where I went wrong?

I'm getting pretty close to just returning these access points and getting something else. (Ruckus maybe?) Amazon says I have until Tuesday to do so.

Mikrotik really needs to fix their wireless products.

Mattlach,

You can tweak and mess with it for as long as you like. A Mikrotik radio is NOT GOING TO COME CLOSE TO THE CLIENT SERVING ABILITIES OF A RUCKUS RADIO.

Believe me... I wish it wasn't the case. And since Mikrotik has finally adopted manufacture drivers... The performance on their radios improved considerably. But there is really no comparison.

Since you are buying from Amazon... Get the Ruckus now. Put them side by side and test them. First off the VLAN settings are clear as day. But the beamflex+ on current radios will just demolish Mikrotik in a noisy environment.

I have HUNDREDS of installs out there going back over a decade, with Mikrotik routing and Ruckus Radios. They work great together. In that... They actually work.

And some old devices don't even see wifi 6 radios. There is a simple disable wifi 6 on RUCKUS under SSIDs. Great for dealing with older IOT or cheap radios.

Besides your opinion (that I share) about the good Mikrotik guys having their own (wrong) definitiions of Beta and Stable and the abysmal lack of valid documentation about nearly everything the RouterOS can do, it seems to me like you *somehow* chose a "wrong" (in the sense of steeper) path.

I would understand if you had many access points, but for only two of them, are you sure that the (IMHO) added complication of the CHR instance and of Capsman is worth it?

See:
viewtopic.php?t=204733&hilit=capsman#p1057385

the prevalent opinion seems that capsman - while not being (for those expert with it ) particularly more complex - is anyway advised as a refinement of an already working "independent" setup.

If I were you I would go by steps, try one of the two setting it as a "normal" AP (without capsman), and if you find difficulties post the whole configuration as per:
viewtopic.php?t=203686#p1051720
so that hopefully more expert members can spot the cause of the problem(s) then replicate the configuration on the other AP, and only when everything works as expected, attempt to "convert" the configuration to capsman managed.

Well,

I wanted to make this work, but I got to the last day of my return window, and I had not yet been able to resolve my issue, so I decided to send them back.

I have popped my old Unifi 802.11ac units back in for the time being (and all the VLAN's work great again).

Going to have to think about what I do next. Maybe a set of Ruckus R650's. They cost more than 3x more, but if I can get them to work...

I'm still happy with my switches (using SwOS) but I don't think I'll be touching RouterOS again any time soon.

For me the key setting was to go on the CAP-AXs edit bridge to vlan-filtering=enabled, disable ingressfiltering and then assign vlan id to the uplink interface as tagged. As then the whole thing started working... unfortunately the documentation misses this part
https://help.mikrotik.com/docs/display/ ... Fi-CAPsMAN under CAPsMAN - CAP VLAN configuration example