Basic Understanding Bridge, VLAN, Switch, ...

plani · Sun Jan 03, 2021 5:03 pm

Hi all together and a happy new year.

As I'm levelling up my network, I reached the point everybody here sooner or later digs into: VLAN configuration. After reading the last two weeks here and there, believing to understand the not always so clear documentation and following the tutorial from pcunite here: viewtopic.php?t=143620, I finally spend 2 hours last night to restore the basic / default configuration. As I like to understand what I do, before I do it, I thought I am going to summarise my basic understanding with the goal that:

* Somebody can confirm my understanding or redirect me where I turned wrong
* Others find this useful when stumbling over this.

In case of details: I'm running three CRS326-24G-2S+ on RouterOS 6.48, but that shouldn't make a difference except with regard to Hardware-offloading. A router RB3011UiAS and two APs (one cAP and one wAP).

So let's start with the switch. The main task of the switch is connecting devices, attached through the ports, with each other. This is called a bridge. An unmanaged Switch is basically set up as one bridge with all ports connected.

What I could do to separate networks would be to create for example 3 bridges and adding the ports to the different switches. This could look like the following:

switch1_bridgedp.png

Now all devices on the green ports are hooked to the bridge named bridge1, all devices connected to the blue ports are hooked to the bridge named bridge2 and so on. All green devices can see each other on Layer2 as this would be a separate broadcast domain. As would be the blue and red ports. One could not see ports of another colour.

1.1 Question: Is my understanding correct that this would be a simple and basic approach to separate networks on Layer2?

As simple confirmation or correction would be appreciated while I prepare the second post.

plani · Sun Jan 03, 2021 5:59 pm

So, now let us extend this concept to three switches:
But first lets give those coloured networks a name/meaning:

GREEN is the usernetwork, or just LAN

BLUE is the guest network

RED is for now just another separated Network

We already know switch1 from the fist post. Switch2 will only have users and guests connected. So we again create two bridges and assign the ports to the bridges.
Switch3 mainly has ports used by users of the LAN (GREEN) and 2 ports for the "other separated LAN"(RED).
In order for devices on one switch to reach devices on the other switch, I need to connect every bridge on one switch with the bridge on the other switch. That way I have 3 separated Layer2 networks across 3 switches, which looks like this:

switch123_bridged.png

The downside is, that I would need per switch and bridge a dedicated cable, so 4 cables in this example. This leads me to VLANs shortly, but first my questions:
2.1 Is the approach described above correct, at least in theory as nobody would do this practically. I'm just laying this out to show the logic behind it, so the question actually is, is the logic correct?
2.2 Configuring names to a bridge are a) locally and b) can be chosen free, right? So, I could use the name "volvo" for a bridge instead of "bridge1" or "br1" and further, I can use "bridge1" on switch1 and "bridge1" on another switch and there would be no conflicts or anything as one doesn't know from the other.

plani · Sun Jan 03, 2021 6:23 pm

So, as mentioned previously, this approach of configuring multiple bridges on a switch and connecting each bridge with the bridge on another switch is not elegant, to say the least. This is where VLANs come in to play.
The logic behind VLANs is simply said:

Asociate a specific Port to a virtual LAN (VLAN) as we did with the bridge earlier (coloured Networks)

Tag the data with a Label to which VLAN it belongs (replacing colour by Number)

Make the virtual LANs known on the participating switches

Transfer the data of all VLANs through one physical connection (Cable) from one Switch to another

Ensure that data only reaches those ports that belong to the same VLAN

And this is where I get stuck. I thought of a VLAN as a configuration-item similar to the bridge. So basically I would create a "VLAN" as an Interface and associate the ports, just as I did previously creating a bridge as an interface and adding the ports. But according to pcunite's tutorial we define a bridge and tag the ports added to the bridge.
Question 3.1 Am I totaly wrong or is the a configurable device "VLAN" similar to "BRIDGE"?

So let's say we configure the switch1 again using VLANs:
Ports 1-8 (green) would be tagged VLAN-ID 100
Ports 9-16 (blue) would be tagged VLAN-ID 200
Ports 17-24 (red) would be tagged VLAN-ID 99
Question 3.2 The SFP+-Ports 1 and to 2 are not tagged. Would they then default to VLAN-ID 1? Saying it different: Do ports that have not been assigned a VLAN-ID default to any default-ID?

mkx · Sun Jan 03, 2021 11:19 pm

I guess you missed most of stuff written by @pcunite in the article you linked in the first post. When you properly configure VLANs, they can partially overlap and ports where they overlap are trunk ports. Due to this reason you don't need multiple physical connections between two switches, single connection can carry all VLANs.
Your setup is still done according to multiple bridges concept, but VLANs are not like multiple bridges. I suggest you to forget about bridges separating LANs on L2 inside single switch. Just read the tutorial again.

anav · Mon Jan 04, 2021 3:49 am

In short, on each device, (be it router, switch, access point), you only need one bridge per device and one defines the vlans they need to run through that device.
DHCP for the vlans is only done normally on one device. The tutorial has all the answers.

What you initially described was simply PORT based vlans, which is not uncommon and has its uses.
Tagged VLANs using vlanIDs, has more flexibility.

On other thought, you can use SFP ports as the main connector between devices (if they both have an SFP port etc) as one only needs one trunk port on each side of a device.

Read the tutorial as Yoda stated. ;-P

plani · Mon Jan 04, 2021 9:02 am

Thanks for the responses, and yes, I went over the tutorial again. Let me ask my question differently:

a) Instead of creating a new Bridge, I could use the Bridge from the default config of the CRS326 where all Ports are configured with hw-offloading and pvid=1. I would then need to add VLAN-ID 1 to the trunk port. By doing this, I would simply transition the current flat network in a VLAN-based flat network, right?

b) Trunkports should be configured with all known / planned VLANs, even if the Switch might not have a port associated with one of the VLANs, right?

c) Following scenario: My PC is connected to switch3. What happens if I configure the VLANs on switch3 and activate VLAN-filtering and the trunk port, which is connected to switch1, is connected to switch1 where no VLANs are yet configured and vlan-filtering is set to no? Either the VLANs get joined again on switch3, which means I still can access devices on switch3 from my PC or the packets get lost, as switch3 doesn't know what to do with tagged packets and discards them.

d) Out of curiosity: According to the tutorial and other posts, it is recommended to configure VLANs on a bridge, so basicaly using /interfaces/bridge/vlan. Why not, or for what is then /interface/vlan and where is the difference/advantage/disadvantage?

Thanks for your help.

tdw · Mon Jan 04, 2021 3:45 pm

a) Instead of creating a new Bridge, I could use the Bridge from the default config of the CRS326 where all Ports are configured with hw-offloading and pvid=1. I would then need to add VLAN-ID 1 to the trunk port. By doing this, I would simply transition the current flat network in a VLAN-based flat network, right?

If you change the existing bridge to have vlan-filtering=yes nothing should break. The default PVID values will dynamically add the bridge ports and bridge interface as untagged members of VLAN 1. It is easy to cut yourself off making bridge and/or VLAN changes, using safe mode in Winbox is suggested.

As with any other ports the bridge interface which connects packets passing through the bridge with services on the Mikrotik itself can be configured to be either hybrid or all-tagged - there isn't a "correct" way, it is down to choice.

b) Trunkports should be configured with all known / planned VLANs, even if the Switch might not have a port associated with one of the VLANs, right?

It depends. If you are never going use a particular VLAN there isn't much point making it available as it cuts down on unnecessary broadcast/multicast traffic being sent over the trunk, if you are potentially going to use a VLAN then it would be sensible to add them when setting the device up.

c) Following scenario: My PC is connected to switch3. What happens if I configure the VLANs on switch3 and activate VLAN-filtering and the trunk port, which is connected to switch1, is connected to switch1 where no VLANs are yet configured and vlan-filtering is set to no? Either the VLANs get joined again on switch3, which means I still can access devices on switch3 from my PC or the packets get lost, as switch3 doesn't know what to do with tagged packets and discards them.

If the trunk is hybrid the untagged traffic will flow as before. Any tagged VLANs will pass through non-VLAN-aware bridges as they act in the same way as inexpensive unmanaged switches. In your example as switch1 knows nothing about VLANs any tagged packets from switch3 would be sent to the other bridge ports but there would be no IP services (routing, DHCP, etc.)

d) Out of curiosity: According to the tutorial and other posts, it is recommended to configure VLANs on a bridge, so basicaly using /interfaces/bridge/vlan. Why not, or for what is then /interface/vlan and where is the difference/advantage/disadvantage?

Originally linux (and thus Mikrotik) bridges knew nothing of VLANs, it is just another ethertype in the packet, so you would create VLAN interfaces on physical ports and add the interfaces for a particular VLAN ID onto their own bridge. You can still do this, but there are a number of issues as described https://wiki.mikrotik.com/wiki/Manual:L ... figuration and it is slightly less efficient than using a single VLAN-aware bridge now that functionality is available.

plani · Mon Jan 04, 2021 8:53 pm

Thank you for the response, let me try to put it in my words, as I'm not sure if I really got the point:

a) Instead of creating a new Bridge, I could use the Bridge from the default config of the CRS326 where all Ports are configured with hw-offloading and pvid=1. I would then need to add VLAN-ID 1 to the trunk port. By doing this, I would simply transition the current flat network in a VLAN-based flat network, right?
If you change the existing bridge to have vlan-filtering=yes nothing should break. The default PVID values will dynamically add the bridge ports and bridge interface as untagged members of VLAN 1. It is easy to cut yourself off making bridge and/or VLAN changes, using safe mode in Winbox is suggested.

As with any other ports the bridge interface which connects packets passing through the bridge with services on the Mikrotik itself can be configured to be either hybrid or all-tagged - there isn't a "correct" way, it is down to choice.

a) 1) As I read, VLAN 1 is a special VLAN. If I understand it correct, VLAN1 is so to say the default VLAN where all interfaces fall in to, when not specified/tagged otherwise. So I could basically say VLAN1 is a "catch-all" VLAN? What are then the "default PVID values" as I understand there is only one default PVID Value which is "1"? Or is the "s" in "values" mistaken?
a) 2) By "add the bridge ports and bridge interface as untagged members of VLAN 1" you mean the port is used like an 'Access Port' according to the wording used by pcunite in his article. The incoming traffic from that port (Packages/Frames) are tagged. Did I get that correct?

I also don't understand the term hybrid in this context, but I#ll come to that later. Let me first set a scenario. I'll be using the CRS-Switch again. The default bridge named "bridge" is in place and all Ports are added to the bridge, using

add bridge=bridge interface=ether3 hw=yes pvid=1

.
except the ports 20 to 24 which have pvid=100:

/interface bridge port
add bridge=bridge interface=ether20 hw=yes pvid=100
add bridge=bridge interface=ether21 hw=yes pvid=100
add bridge=bridge interface=ether22 hw=yes pvid=100
add bridge=bridge interface=ether23 hw=yes pvid=100
add bridge=bridge interface=ether24 hw=yes pvid=100

d) If I now do

/interface bridge set bridge vlan-filtering=yes

I would assume to have 2 VLANs created. Devices on Ports 1-20 can see each other, not knowing from devices on Ports 21-24 and vice versa. Is that right?
e) Further I believe that traffic leaving the ports would be tagged (and as such not usable for standard devices like a printer) but incoming traffic is assumed untagged and will be tagged with the VLAN-id on entering the bridge as stated in the "pvid=xyz" part? Is that right, or did I take a wrong turn? Will only untagged traffic be tagged or all incoming traffic?

f) Assuming I'm right so far, I'd use

/interface bridge vlan 
add bridge=bridge untagged=ether21,ether22,ether23,ether24 vlan-ids=100

in order that the traffic leaving ether21-ether24 gets the tag removed, or more specific, gets the tag labelled "VLAN 100" removed. Right?

Sorry, I hope I don't sound too dumb, but each time I think I got the logic and go over it again, it just falls to pieces again. But this step by step approach helps me alot.

mkx · Mon Jan 04, 2021 9:59 pm

a) VLAN 1 is nothing special per-se. Wgat makes it special is that it's used by default configuration. For this very reason it's best to avoid using VLAN ID 1 in explicit configurations

b) yes. Port with pvid set tags untagged ingress frames and untags frames belonging to that vid on egress. Frames are tagged inside bridge.
There are 3 types of ports:
- access where outside of switch (on the wire) only untagged frames exist
- trunk where outside of switch only tagged frames exist. Can be single or multiple different VLAN IDs
- hybrid where outside of switch there are both untagged and tagged frames. Port is access port for a select VLAN

d) yes

e) no, frames belonging to VLAN ID same as pvid setting will get untagged on egress. Setting pvid on port automatically makes it untagged member (bullet f) unless you explicitly set it tagged member (but then you'll have discrepancy). On ingress only untagged frames will get tagged, already tagged will either pass unaltered or get dropped (depending on other port settings)

f) yes

Just keep in mind: VLAN-related settings in /interface bridge port are for ingress and /interface bridge vlan settings are for egress.

And the answer to the question in one of previous posts: bridge has two personalities: 1) something like a switch and 2) interface for CPU (gets created implicitly). Settings in /interface bridge are affecting behaviour of bridge "as a switch" while /interface vlan are about bridge interface (these vlan interfaces allow ROS to communicate with particular VLANs via bridge, e.g. IP routing, DHCP service, etc.)

plani · Tue Jan 05, 2021 9:49 am

Great, I'm getting there. Let me summarise:

Incoming traffic, called ingress, is configured in "/interface bridge port" and by default behaves as follows:
- Incoming traffic that is untagged gets tagged with a VLAN-Id according to the set pvid for that port
- Incoming traffic that is already tagged gets passed on unaltered
Further I can control this behaviour to only accept tagged or untagged frames or both by using something like:

/interface bridge port set bridge=bridge ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged etherX

a) Is that correct?

Outgoing traffic, referred to egress, is configured in "/interface bridge vlan". If nothing is configured, traffic tagged with a VLAN-id same as the port-specific pvid gets untagged. With "/interface brdige vlan" I can control
[*]which VLAN is broadcasted to which interface and
[*]if it is tagged or untagged leaving that port.
Lets look again at the CRS-Switch configured with all ports to VLAN1 except Ports 21-24 which are assigned to VLAN100 the follwing setting:

/interface bridge vlan add bridge=bridge  tagged=ether1,ether24 untagged=ether21,ether22, vlan-ids=100

Traffic leaving:
- ether 1 will be tagged with VLAN-ID 100
- ether21 will be untagged according to the above setting
- ether22 will be untagged according to the above setting
- ether23 will be untagged as the pvid is set to 100
- ether24 will be tagged with VLAN-ID 100
b) So the "untagged=ether21,ether22" is basically redundant right?
c) I must not think of "one" traffic on a port, but in multiple traffics based on different VLANs. So ether1 will receive packets from VLAN 1 and VLAN 100. Traffic from VLAN 1 will get untagged due to the pvid setting of the port, where as packets from the VLAN 100 will egress tagged based on the setting in "/interface bridge vlan", right?
d) Could I say that the "/interface bridge vlan" settings mean "Share traffic from VLAN XYZ with the ports a,b,c,d but remove the tag when leaving a,d and keep the tag when leaving through port b and c"? I mean, it not only explicitly states which VLAN-id is tagged or untaged on a specific port but implicitly states which ports (tagged and untagged) shall have knowledge of a certain VLAN, right?
e) How shall I picture this vlan-table in "/interface bridge vlan"? Basically like a spreadsheet with all colums named to the ports and all rows to the VLAN-id and each field then states either "assigned and tagged", "assigned and untagged" or "unassigned"?

Your answers a very appreciated. If I got this right so far, there are only two more topics until I believe I know enough to start transforming my network.

mkx · Wed Jan 06, 2021 12:17 am

Yes, yes but helps when reading config, yes, yes, something like that (I prefer to think of it as "VLAN port membership").

plani · Wed Jan 06, 2021 12:26 pm

Ok, now starting to play around.

1.) First question: I add "/interface bridge vlan add bridge=bridge tagged=ether1,ether10 untagged=ether21,ether22 vlan-ids=100". Then I decide to remove ether10 from that VLAN. So I issue "/interface bridge vlan add bridge=bridge tagged=ether1 untagged=ether21,ether22 vlan-ids=100" which does not work because the entry for VLAN 100 already exists. So I use "/interface bridge vlan set bridge=bridge tagged=ether1 untagged=ether21,ether22 vlan-ids=100" which requires me to enter the number of the 'data-set' referring to the VLAN 100. Instead of entering the number of the table row I can replace "vlan-ids=100" by "[find vlan-ids=100]". Am I right assuming that there always is only one entry per VLAN-ID in the /interface/bridge/vlan table?

2.) If I look at the ethernet interfaces using "/interface ethernet print" or "/interface ethernet print detail" I won't see the pvid and don't see the association with the bridge named bridge. Where as when I "/interface bridge port print" I see the ethernet interfaces associated with the bridge and their pvid. To my question: I can not investigate the state/configuration of a physical ethernet port solely by looking at "/interface ethernet", I would always have to check /interface bridges and maybe other places, or am I missing something?

Taking a closer look at the bridge. The bridge itself, as @mkx stated earlier has two personalities.
3.) Looking at the bridge as "the virtual device that does the switching between the added ports" it does not have any VLAN-related configuration by itself, besides setting vlan-filtering=yes/no and setting the stp to rstp, mstp or the like. But it does not have a VLAN-ID or pvid per se, right? At least it doesn't show using "/interface bridge print" and it doesn't show up using "/interface bridge port print". Looking at the bridge as a 'switching device' there also wouldn't be the need for. Do I understand that right?

4.a) Looking now at the bridge as the link to the underlying RouterOS. In general one would want to "get on the switch/router" in order to do the configuration etc. I can take an ether-port, configure it with a proper IP-Setting and voila, I can access the switch/router on the OS-Level. At least that's what I've done on my RB3011. If now all ports on the CRS-Switch are added to a bridge, I can to the IP-configuration for that bridge and again, voila, I can access the OS-level. Now activating vlan-filtering: Traffic including access to the RouterOS comes from an access-port, get's tagged, passes back/forth/through the bridge, but never to the bridge itself, as it would need to be a member of the same VLAN as from where the connection request originated. Is that logic right?
4.b) Looking at pcunites scripts, I can add the bridge to a vlan by tagging it: "add bridge=BR1 tagged=BR1,sfp1,sfp2 vlan-ids=99". Does the bridge not have a pvid-setting? Or is it an "invisible default pvid=1"?
4.c) Would it work if I configure IP on the bridge as stated earlier and then add the bridge to the vlan untagged, for example by: "add bridge=BR1 tagged=sfp1,sfp2 untagged=BR1 vlan-ids=99" and if not, why not?

Sorry for the long post, but the last part will probably derive from question #4 so let me just as well put it in here and save another, last post:
After understanding all previous logic, I now can create VLANs on a switch using access ports, span the VLANs over switches using a trunk port. Connected devices can now talk to each other in the same VLAN. But now I want access from my PC in VLAN100 to a server/router/... on another VLAN. A blunt simple approach would be to configure some ports on my RB3011, for example by giving ether5,6,7,8,9,10 different IP-configurations and adding DHCP-Server and then plug them into the CRS. on the CRS I would assign the incoming ports originating from the RB3011 to different VLANs and voila, I have IP-Service in each VLAN including DHCP and could route traffic from one VLAN to another over the Router.
5.a) Technically, logically this would work, right?
The downside is, that this would limit the number of VLANs to the number of available Ports on the RB3011 and I assume it is not the "usual" way you would do this. So, according to what I have read and what is used in pcunite's script, I create a "virtual device/interface" using: "/interface vlan add interface=BR1 name=BASE_VLAN vlan-id=99". This creates an Interface named BASE_VLAN which I can use the same way as e.g ether8 previously for configuring IP.
5.a) Is there a difference between using the VLAN-Interface or the "ether-to-access-port" approach in terms of performance, limitations?
5.b) The RB3011 has two switch chips, connecting port 1-5 and 6-10. Does it make a difference if I add the VLAN-Interface on the first 5 ports together with the WAN-Port or should I rather move it as planned to the second because it then might utilise the hardwareoffloading better on the vlans or something?

Thanks for bearing with me so long, I already started to put the configuration together as I believe I nearly have everything together.

mkx · Wed Jan 06, 2021 1:36 pm

1) yes ... one vlan entry per bridge. So ideally you would actually include [ find bridge=bridge vlan-ids=100 ] in the set command.

2) /interface ethernet has no correlation with settings in /interface bridge. OTOH VLANs are not physical properties of ethernet ports (and are not properties of only ethernet ports), they are kind of logical properties of bridge ports. In short: you always check status in the configuration tree where you also configure the same properties.

3) bridge the switch-like property doesn't have pvid settings. But bridge the interface does, you can set it with /interface bridge set [ find bridge=bridge] pvid=100. Actually default setting is pvid=1 and like other default settings it's not shown in usual configuration export. If you set pvid on bridge interface, then whatever you do L3-wise directly on bridge interface (just like you would on all-untagged setup) will actually be done for VLAN ID 100 ... and bridge interface becomes untagged member of VLAN ID 100. In this case you should not use /interface vlan to create vlan interface for vlan-id=100 ...
Yeah, gets messy, that's why I strongly suggest to always use tagged stuff inside ROS device (and inside LAN infrastructure perimeter which means using solely tagged-only trunks between routers/switches).
You can see actual VID state by running /interface bridge vlan print (all actual VLAN settings) and /interface bridge port print (the PVID column ... which sometimes is not relevant if port has frame-types=admit-only-vlan-tagged set). Mind that the implicit bridge interface is not shown in the later printout.

4a) yes

4b) it has "invisible" pvid=1 set, see the answer above.

4c) see the answer above. In addition to setting bridge interface as untagged member of said VLAN, you'd have to set pvid=99 on /interface bridge (see the answer above).

5a) yes.
The correct approach would be to use single connection between switch and router (either single ethernet cable or multiple configured in a bond) in a VLAN trunk configuration (tagged member of multiple VLANs). Do it the same on router side. This way only single physical connection consumes port space both on switch and router. Downside is potential throughput bottleneck, but in case of most mikrotik routers the bottleneck will be routing performance of the router.
Then construct multiple /interface vlan interfaces, one per each of VLANs which should be routed, and configure L3 (IP) stuff on them. Then router will automatically try to route between VLANs unless particular configuration (most frequently that's firewall setup) blocks such connections.

5b) tagging/untagging frame has some performance overhead. Bridge has some overhead as well. In case of CRS3xx both are offloaded to hardware and things are equal performance wise.
That's true only for switching functionality. When it comes to L3 (IP) performance, most likely it will be the general purpose CPU of device which will be the bottleneck in any case. But in any case, the bridge/vlan-interface approach is clean switch-like config and I recommend it every time.

5c) Have a look at block diagram, available from product page. As you can see, each of switch chips have kind of bottleneck with interconnection to CPU. If traffic is destined to CPU in any case (e.g. WAN traffic should always hit firewall), then it doesn't matter which switch chip it passes. However it is beneficial to separate it from LAN traffic if only to give LAN a bit more bandwidth on the switch chip - CPU interconnect. Add the fact that HW-offloaded switching is only available between ports handled by same switch chip (so you can have 5 LAN ports if using eth6-eth10 group). OTOH, RB3011 doesn't offload VLAN config if done per guide by @pcunite. OTOH2, this doesn't matter at all if only single link is used between LAN (switches, ...) and router. HW offload only comes into play for intra-VLAN traffic, passing between bridge ethernet ports. HW offload doesn't do anything for traffic passing any other bridge ports (including bridge interface).
Beware of the oddity with SFP cage connection ... when it's used, switch chip for ports eth6-eth10 looses one of interconnects.

plani · Wed Jan 06, 2021 7:28 pm

Thanks for the details @mkx
Just zu clarify regarding the bridge as interface: Both ways can be used:

Define L3 on the bridge, add pvid for desired VLAN on the bridge and add the bridge as untagged member of designated VLAN in /interface bridge vlan
Create VLAN-Device by "/interface vlan add interface=BRIGDGENAME name=VLAN_interface_name vlan-id=SOMEVLAN", add the bridge as tagged member to "/interface bridge vlan" while the L3 configuration then is done in the VLAN_interface_name interface.

a) Both ways serve the same purpose and the difference is basically the consistency in the way one does the configuration. Yes, this gets messy, but did I get it right? Actually, assuming there is no real difference, I would prefer the first approach, as this would be, at least in my mind, be analogue to an ether-Interface that is included in the bridge-vlan. Ether has a wire a to a device and a logical link to the bridge, the bridge is "virtually wired" to the RouteOS and also linked logically to the vlan-bridge....messy....forget the last sentence.

Now my final questions on how I concrete plan to migrate the network to VLANs:
I got three CRS326 spread in the house and one RB3011. Switch 1 is the core-switch to whom switch 2 and 3 connect. I am connected with my PC to switch3. Currently no VLANs, so on the CRS all (physical) ports are member of a bridge named bridge with pvid=1, just as the default-config has set it up. The RB3011 has no bridge, port 1 and 2 connect to the wan-router and port 10 to switch1. The following picture shows it,leaving out the WAN-Part.

Basic Structure.png

Instead of a big bang and locking myself out again as I tried with pcunites scripts, I want to migrate step by step. Basically I want to configure a VLAN separated Network with all VLANs configured, trunks established but in a first step having all Devices (=Accessports) on VLAN 1 and the ROS-Devices on VLAN 99. When that worked, meaning everything functions as today and is reachable, I will start to transfer (reconfigure) Accessports to other VLANs.

1. Step: On the Router, Create a bridge and add ether5 as a trunk port. Create the 8 VLAN-Interfaces for all 8 VLANs to create the L3/IP configuration points (IP Addressing, DHCP) for all VLANs. Run vlan-filtering=yes on the bridge. Also connect ether10 with switch1 to keep connectivity for the time being, as Ether5 (Trunkport) will not work. (ether10 is currently configured as gateway for the LAN, so nothing changes and thinking of it, I should not IP-config the VLAN-Interface for VLAN1 on ether5 just yet)

2. Step: Switch2 (furthest away from my PC) gets port 1 configured as trunk with all 8 VLANs, including VLAN 1 tagged, all other ports get defined as untagged VLAN 1 ports (Accessports). Additionaly create a VLAN-device for the bridge and add it to VLAN=99. When activating "/interface bridge set bridge vlan-filtering=yes" I'm sure my SSH-Connection gets terminated and all devices attached to the CRS will be not reachable for the time being.

3. Step: Same as the first step, but on the coreswitch (Switch1) except that I configure not 1 but 3 Trunk Ports, one to each Switch and one to the Router (ether5 on the router). After activating vlan-filtering, I shouldn't have access any more to switch1, switch2 and the router.

4. Step: Repeat Step2 on switch3 with one trunk port that connect to switch1. After activating vlan-filtering the ssh-connection might break or not. But latest after reconnecting my PC should be a device within VLAN1 from the vlan-seperated network. Within VLAN1 I can connect to the other devices (Server, Printer) as all should be on VLAN1. I should also be able to connect to the Switches and the router as the traffic from vlan1 to vlan99 should be routed by default on the router (currently no firewall settings) based on the IP-Adressing.

My final questions:
a) Does that way of migration make sense or did I overlook something or miss any caveats?
b) I wouldn't set "ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged" for the time being, as I'm not sure yet what the filtering does exactly. But when it is not activated, could it be, that even after setting vlan-filtering=yes on switch2, that the trunk ports accepts traffic from switch1 and tags it (ingress) as VLAN1 which makes it a hybrid port? On the way back, the traffic leaving the switch2 towards switch1 again, gets or stays tagged as vlan1 (egress), whereas switch1, still having vlan-filtering=no doesn't care about the tag and is happy to share the packets with all ports as usual? That would mean, devices on switch2 would still be reachable throughout the migration.
c) After I transfered all devices from VLAN 1 to their respective target VLAN by changing the configuration of the accessports, I would keep all other (currently unused) ports in VLAN 1 as a catch-all VLAN and set the firewall on the router to deny any traffic (src=IP-Range of VLAN1). This way, anybody pluging a device into a network socket in the house, will get an IP but nothing else and complain ;-) and based on the IP I can conclude that he/she landed in VLAN1 and then can transfer his/her accessport to the right VLAN. Any reason speaking against this approach?
d) After migration I would like to install the two Mikrotik APs and try Capsman. One gets connected to switch1 and the other to switch3. The APs shall radio 2-3 WLANs which in turn are associated with 2-3 different VLANs. I would then need to transform the two ports where the APs are connected into trunk-ports, right?

Looking forward to your response. I'm anxious to start and already run through the house, taking notes which device is plugged into which port....

mkx · Thu Jan 07, 2021 9:04 pm

a) ceveat is: try to keep away from using VLAN ID 1. It's used as default PVID in ROS (and some other vendors) and if you're not carefull, it can cause confusion. Specially so as this setting is not shown in configuration exports (it only shows non-default configuration)

b) these two settings are more or less security settings. E.g. if port should be access port, then tagged frames should not pass it on ingress. Without these settings they will pass and is thus possible to inject some frames to "alien" VLAN. It should work fine without these settings and you can set them after you verify your setup works correctly

c) you'll have general problem with mixing non-vlan and vlan traffic belonging the same IP subnet. I'd simply create a new IP subnet and would then move devices from current to new subnet. Meanwhile router would allow communication between both subnets with some limitations (most of discovery protocols work only within single IP subnet without special helper services).

d) yes

plani · Thu Jan 07, 2021 9:27 pm

thanks @mkx, I just don't understand Answer c) Maybe there is a misunderstanding. Every VLAN will get it's own /24 IP-Network, So an device plugging in to a "catch-all-VLAN1" will get address from e.g. 192.168.199.0/24, where the actual LAN for example resides in VLAN100 using 192.168.100.0/24. I would not use 2 IP-Ranges on one VLAN or 2 VLANs on one IP-Range.

mkx · Fri Jan 08, 2021 7:22 pm

Well, I guess you thought out everything. So just go ahead and implement the LAN upgrade.

Jotne · Fri Jan 08, 2021 7:52 pm

Have a look at my thread here:
viewtopic.php?t=138232
I did learn how the new one bridge for all vlan did work, step by step.

plani · Sun Jan 10, 2021 8:42 pm

Hi everyone, Frustration is at a maximum, hope someone can give me a lead where to Look for the following issue:

I set up the vlans on all three Switches, as discussed in this thread and pcunites HowTo. Same on the router, setting up a bridge, adding ether6 and the VLAN-Devices for Layer3. After some frustrated chaos phase, finding typos and copy/paste errors in my runbook I have a partially working network with VLAN separation, meaning that everything connected to the router and switch1 and switch2 works:
* Layer3 connectivity
* Layer3 routing from ohne vlan network to another
* Layer2 access from RouterOS/Smartphone-App to RouterOS

Now my problem is Switch3 as in previous post pictured:
* Access from PC to the Network doesn't work, only to devices attached to the same vlan on switch3, like printer and accesspoint
* Access to the switch itself (RouterOS) on L3 works from PC when plugged into an accessport for vlan99=MGMT-LAN
...and thats it. Not even a ping to router or mgmt-interface on switch1 even wenn attached to mgmgt-vlan.

The curious part is, wenn using the Mikrotik-App on my smartphone, connected via wifi to the Accesspoint, attached to switch3 I see the router and switch 1 and 2. I also can mac-telnet from the app to them, but not to switch3 allthough it lays directly between the smartphone/accesspoint and switch1. Other than that, Layer3 Access via the Accesspoint is again, only to the devices in vlan10, plugged into switch3(printer,pc).

Anybody an idea? I didn't user vlan1. And I didn't set VLAN security, meaning ingress-filtering or so.

Compared all three switch configurations now for 2 days....can't find anything. Anybody an idea?

tdw · Sun Jan 10, 2021 8:54 pm

Post the actual configs from /export hide-sensitive

plani · Sun Jan 10, 2021 9:06 pm

Here it is the config from switch3:

# jan/02/1970 03:25:51 by RouterOS 6.48
# 
# model = CRS326-24G-2S+
/interface bridge
add admin-mac=48:8F:5A:B6:2D:98 auto-mac=no comment=defconf name=bridge \
    protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=bridge name=MGMT_IFACE vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=defconf interface=ether1
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3 pvid=70
add bridge=bridge comment=defconf interface=ether4 pvid=70
add bridge=bridge comment=defconf interface=ether5 pvid=10
add bridge=bridge comment=defconf interface=ether6 pvid=10
add bridge=bridge comment=defconf interface=ether7 pvid=10
add bridge=bridge comment=defconf interface=ether8 pvid=10
add bridge=bridge comment=defconf interface=ether9 pvid=10
add bridge=bridge comment=defconf interface=ether10 pvid=10
add bridge=bridge comment=defconf interface=ether11 pvid=10
add bridge=bridge comment=defconf interface=ether12 pvid=10
add bridge=bridge comment=defconf interface=ether13 pvid=10
add bridge=bridge comment=defconf interface=ether14 pvid=10
add bridge=bridge comment=defconf interface=ether15 pvid=10
add bridge=bridge comment=defconf interface=ether16 pvid=10
add bridge=bridge comment=defconf interface=ether17 pvid=10
add bridge=bridge comment=defconf interface=ether18 pvid=10
add bridge=bridge comment=defconf interface=ether19 pvid=10
add bridge=bridge comment=defconf interface=ether20 pvid=10
add bridge=bridge comment=defconf interface=ether21 pvid=10
add bridge=bridge comment=defconf interface=ether22 pvid=10
add bridge=bridge comment=defconf interface=ether23 pvid=99
add bridge=bridge comment=defconf interface=ether24 pvid=99
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge comment=defconf interface=sfp-sfpplus2
/interface bridge vlan
add bridge=bridge tagged=ether1,ether2 untagged="ether5,ether6,ether7,ether8,e\
    ther9,ether10,ether11,ether12,ether13,ether14,ether15,ether16,ether17,ethe\
    r18,ether19,ether20,ether21,ether22" vlan-ids=10
add bridge=bridge tagged=ether1,ether2 untagged=ether3,ether4 vlan-ids=70
add bridge=bridge tagged=bridge,ether1,ether2 untagged=ether23,ether24 \
    vlan-ids=99
add bridge=bridge tagged=ether1,ether2 vlan-ids=50
add bridge=bridge tagged=ether1,ether2 vlan-ids=80
add bridge=bridge tagged=ether1,ether2 vlan-ids=90
add bridge=bridge tagged=ether1,ether2 vlan-ids=100
add bridge=bridge tagged=ether1,ether2 vlan-ids=200
add bridge=bridge tagged=ether1,ether2 vlan-ids=300
/ip address
add address=172.28.99.243/24 interface=MGMT_IFACE network=172.28.99.0
/ip cloud
set update-time=no
/ip dns
set servers=192.168.2.220
/ip route
add distance=1 gateway=172.28.99.254
/ip ssh
set strong-crypto=yes
/system identity
set name=switch3-office
/system ntp client
set enabled=yes primary-ntp=172.28.254.1
/system routerboard settings
set boot-os=router-os

plani · Sun Jan 10, 2021 9:25 pm

Here the export from switch1, where everything works:

# jan/10/2021 20:18:59 by RouterOS 6.48
# 
# model = CRS326-24G-2S+
/interface bridge
add admin-mac=C4:AD:34:E5:2F:D2 auto-mac=no comment=defconf name=bridge \
    protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=bridge name=MGMT_IFACE vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/system logging action
set 3 remote=192.168.2.22
/user group
add name=prometheus policy="read,api,!local,!telnet,!ssh,!ftp,!reboot,!write,!\
    policy,!test,!winbox,!password,!web,!sniff,!sensitive,!romon,!dude,!tikapp\
    "
/interface bridge port
add bridge=bridge comment=defconf interface=ether1
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5 pvid=10
add bridge=bridge comment=defconf interface=ether6 pvid=10
add bridge=bridge comment=defconf interface=ether7 pvid=10
add bridge=bridge comment=defconf interface=ether8 pvid=10
add bridge=bridge comment=defconf interface=ether9 pvid=10
add bridge=bridge comment=defconf interface=ether10 pvid=10
add bridge=bridge comment=defconf interface=ether11 pvid=10
add bridge=bridge comment=defconf interface=ether12 pvid=10
add bridge=bridge comment=defconf interface=ether13 pvid=10
add bridge=bridge comment=defconf interface=ether14 pvid=10
add bridge=bridge comment=defconf interface=ether15 pvid=10
add bridge=bridge comment=defconf interface=ether16 pvid=10
add bridge=bridge comment=defconf interface=ether17 pvid=10
add bridge=bridge comment=defconf interface=ether18 pvid=10
add bridge=bridge comment=defconf interface=ether19 pvid=10
add bridge=bridge comment=defconf interface=ether20 pvid=10
add bridge=bridge comment=defconf interface=ether21 pvid=10
add bridge=bridge comment=defconf interface=ether22 pvid=10
add bridge=bridge comment=defconf interface=ether23 pvid=10
add bridge=bridge comment=defconf interface=ether24 pvid=10
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge comment=defconf interface=sfp-sfpplus2
/interface bridge vlan
add bridge=bridge tagged=ether1,ether2,ether3,ether4 untagged="ether5,ether6,e\
    ther7,ether8,ether9,ether10,ether11,ether12,ether13,ether14,ether15,ether1\
    6,ether17,ether18,ether19,ether20,ether21,ether22,ether23,ether24" \
    vlan-ids=10
add bridge=bridge tagged=ether1 vlan-ids=50
add bridge=bridge tagged=ether1 vlan-ids=70
add bridge=bridge tagged=ether1 vlan-ids=80
add bridge=bridge tagged=ether1 vlan-ids=90
add bridge=bridge tagged=ether1 vlan-ids=100
add bridge=bridge tagged=ether1 vlan-ids=200
add bridge=bridge tagged=ether1 vlan-ids=300
add bridge=bridge tagged=ether1,ether2,ether3,ether4,bridge vlan-ids=99
/ip address
add address=172.28.99.241/24 interface=MGMT_IFACE network=172.28.99.0
/ip dns
set servers=192.168.2.220
/ip route
add distance=1 gateway=172.28.99.254
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name=switch1-core
/system logging
add action=remote topics=critical
add action=remote topics=error
add action=remote topics=interface
/system ntp client
set enabled=yes primary-ntp=172.28.254.1
/system routerboard settings
set boot-os=router-os

Not sure if it means anything, but when doing "/interface bridge vlan print" I see another VLAN with vlanID=1, marked as dynamic and having no tagged and no untagged ports set, but has currently untagged ports, which are the trunk ports. But that is on all three Switches the case.

plani · Mon Jan 11, 2021 9:12 am

It works, a big thank you to @mkx and all others who helped me understand VLANs on RouterOS and get this up and running and to @pcunites HowTo on VLANs for the thorough examples.

And for all who stumble over this thread at a later time and wonder what the solution was to my last issue: It's a bit embarrassing to say, but it basically was the wrong cable. The trunk-port from Switch1 was actually connected to an access-port on Switch3.....

tdw · Mon Jan 11, 2021 1:48 pm

It had to be something simple like that, nothing obviously incorrect in the configurations. I did notice that VLAN IDs 10 & 99 are tagged on ether1-4 on switch1, however VLAN IDs 50, 70, 80, 90, 100, 200 & 300 are only tagged on ether1 so will not propagate on the trunks connected to ether2-4.

Regarding VLAN 1 appearing in /interface bridge vlan print, the pvid= settings for the ports under /interface bridge (for the implicit bridge-to-CPU port) and /interface bridge port dynamically add untagged membership of the ports to /interface bridge vlan.

Due to this you can leave the untagged= settings out under /interface bridge vlan, then changing the untagged membership of a port only requires the PVID changing, otherwise you have to remember to change both the PVID and untagged membership.

All ports are initially access (untagged only) ports, adding tagged VLANs makes them hybrid (untagged and tagged VLANs) ports. If you wish to removed the untagged membership to become a trunk (tagged VLANs only) port you should add ingress-filtering=yes frame-types=admit-only-vlan-tagged to the ports.

plani · Mon Jan 11, 2021 3:49 pm

Thanks tdw,

I left out the ingress-filtering for the time being as I wanted to do one step after the other. Regarding the trunkports on switch1 only having tagged 10 and 99 is right too. As the other VLANs (50,70,...) are not yet populated, I just left them out and add them now, as the basics are working. Thanks for taking a look. Now going to dive into capsman.

Anybody an idea how I can mark this thread as solved? I tried to change the title, but I doesn't show up.

Basic Understanding Bridge, VLAN, Switch, ... [SOLVED]

Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ... [SOLVED]

Re: Basic Understanding Bridge, VLAN, Switch, ...

Re: Basic Understanding Bridge, VLAN, Switch, ... [SOLVED]