Sharing one physical trunk port with two bridges

mr2web · Fri Jan 19, 2024 7:37 pm

Hello,
I new to mikrotik and have spent a long time researching how to set up my MikroTik CCR2116-12G-4S+ to do what I need it to do. Up to this point I really can't understand what I'm doing wrong, but clearly I am doing something wrong. I really hope that someone can point me in the direction to get this sorted.

I have read many articles, seen loads of YT tutorials and read many forum posts here without being able to understand what I'm doing wrong.

Here is a simple schematic over my environment:

SimplefiedInfrastructure_2.png

Here is my usecase with references to the simple schematics attached to this post:

I like the following machines to be able to communicate:
- A and E, B and F, C and G, D and H on as close to wire speed as possible. These pairs should be separated from each other (pair to pair).
- J to be able to talk to E and K to G without any demand on wire speed. No other traffic should be routed to J and K than responses from E and G. Naturally J and G should not be able to talk to each other nor should E not be able to reach K.

A, B, C and D are virtual machines running on a hypervisor terminating them two trunk connections ("sfp-sfpplus3 – TRUNK1" and "sfp-sfpplus2 – TRUNK2"). Non of these VMs should be able to talk to each other.

E, F, G, H are physical machines with Gigabit Ethernet ports.

J and K are virtual machines running on another hypervisor. J and K are to share a physical trunk connection ("ether2 – TRUNK3") to the router.

This environment is not connected to internet nor have the need for any DHCP server functionality. All machines will have their IPs set staticky

Here is my export result:

[@MikroTik] > export
# RouterOS 7.13

/interface bridge
add name=BR0 vlan-filtering=yes
add name=BR1 vlan-filtering=yes
add name=BR2 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether2 ] name="ether2 - TRUNK3"
set [ find default-name=sfp-sfpplus2 ] name="sfp-sfpplus2 - TRUNK1"
set [ find default-name=sfp-sfpplus3 ] name="sfp-sfpplus3 - TRUNK2"
/interface vlan
add interface="sfp-sfpplus2 - TRUNK1" name=VLAN10 vlan-id=10
add interface="sfp-sfpplus2 - TRUNK1" name=VLAN11 vlan-id=11
add interface=BR2 name=VLAN19 vlan-id=19
add interface="sfp-sfpplus3 - TRUNK2" name=VLAN20 vlan-id=20
add interface="sfp-sfpplus3 - TRUNK2" name=VLAN21 vlan-id=21
add interface=BR2 name=VLAN29 vlan-id=29
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=BR0 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether4 pvid=10
add bridge=BR0 frame-types=admit-only-vlan-tagged interface=\
    "sfp-sfpplus2 - TRUNK1"
add bridge=BR1 frame-types=admit-only-vlan-tagged interface=\
    "sfp-sfpplus3 - TRUNK2"
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether7 pvid=20
add bridge=BR0 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether5 pvid=11
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether8 pvid=21
add bridge=BR0 frame-types=admit-only-vlan-tagged interface=VLAN19
add bridge=BR1 frame-types=admit-only-vlan-tagged interface=VLAN29
add bridge=BR2 frame-types=admit-only-vlan-tagged interface="ether2 - TRUNK3"
/ip firewall connection tracking
set enabled=yes
/interface bridge vlan
add bridge=BR0 tagged="sfp-sfpplus2 - TRUNK1" vlan-ids=10
add bridge=BR0 tagged="sfp-sfpplus2 - TRUNK1" vlan-ids=11
add bridge=BR1 tagged="sfp-sfpplus3 - TRUNK2" vlan-ids=20
add bridge=BR1 tagged="sfp-sfpplus3 - TRUNK2" vlan-ids=21
add bridge=BR0 tagged=VLAN19 vlan-ids=19
add bridge=BR1 tagged=VLAN29 vlan-ids=29
/ip address
add address=192.168.125.11/24 interface=VLAN10 network=192.168.125.0
add address=10.0.11.11/24 interface=VLAN11 network=10.0.11.0
add address=192.168.125.11/24 interface=VLAN20 network=192.168.125.0
add address=10.0.21.11/24 interface=VLAN21 network=10.0.21.0
add address=192.168.125.11/24 interface=VLAN19 network=192.168.125.0
add address=192.168.125.11/24 interface=VLAN29 network=192.168.125.0
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key

What I think I'm stuck on is sharing the "ether2 – TRUNK3" connection with BR0 and BR1 having VLAN19 to go to BR0 and VLAN29 to go to BR1.

Any suggestions would be much appreciated. And I know my usecase might seem a bit odd but its for a test environment where I'm going to run tests towards the machines E, F, G and H. There of no access to outside this environment.

If i have missed providing any information please do not hesitate to make me aware of it.

I have tried to understand the viewtopic.php?t=143620 thread but clearly not been able to sort my issues described above.
I like to understand where I have gone wrong.

With hopeful regards
Toby

bpwl · Fri Jan 19, 2024 9:47 pm

Reading someones VLAN design, is always hard for me. Even with a very nice drawing , it helps, but still I didn't grasp it. It is probably me.

I give what I understand of VLAN and bridges, but in this matter, making a mistake is easy done.

Can you explain the purpose of using 3 bridges, and not just one?
Splitting VLAN in one bridge works fine.
The only double I see in multiple versus one, is the untagged traffic, that was separated per bridge, now would be common in the one bridge.
That looks easy to resolve, give it a separate VLAN number in the one bridge, and untag it where needed.

Ethernet interfaces normally are ports on the bridge, the VLANs used there are set in the bridge VLAN definitions, as tagged or untagged to some ports (ethernet interfaces)

Then no VLANs should be defined on the ethernet interfaces itself, only on the bridge.

It does exist ... VLAN on an ethernet interface, but then that VLAN is a different VLAN than the VLAN on the bridge or other interface even with the same number.

Interface VLAN 10,11,20,21 should be picked up at the bridge as interface, not at the SFP-TRUNK as interfaces, if these SFP-TRUNK interfaces are ports on the bridge.
Just like VLAN 19 and 29 are VLAN of the bridge.
But the VLAN interfaces themselves should not be added again to the bridge as ports, if the supporting interface already is a port.
The VLAN filtering of the bridge should give the access to those VLAN.

SFP Being ports of a bridge makes them "slave" interfaces, and the VLAN and DHCP and other IP functions of slave interfaces are handled only by the bridge, not by the interface.
If VLAN 19 and 29 are defined on a non-port-connected ethernet interface, then AFAIK the VLAN interfaces themselves can be added to a bridge on their own, typical as already untagged ports.

Confusing all this. I know, and I try to be carefull in what I say here.

anav · Fri Jan 19, 2024 11:43 pm

Really a straightforward setup for the most part, the question I have is why do you have TWO ethernet ports going to the hypervisor on the left??
You only need one port going to a smart switch for example if it was there instead of the hyper visor.

Thus I would need to understand what the hypervisor is doing>>> I would be afraid of some connection between those hypervisor ports and looping etc..............

Concur, with bpwl AND THE LINK!!! One bridge, all vlans with interface bridge, and bridge does no DHCP.

The one that struck me is there should be a trusted vlan to all smart devices ( and all smart devices should get an IP on that vlan).
I dont see that as the hypervisor left side and right side dont have a common vlan. Vlans are cheap LOL.

Once you have answered the questions in both posts, I will have a bit better understanding to tackle the config.

+++++++++++++++++++++++++++

By the way to comment coherently on your config, the whole export minus router serial number, public WANIP info, keys etc would be needed.
For example I dont see any FW rules, interface lists etc...... A config is the sum of many integrated parts!!!

aoakeley · Sat Jan 20, 2024 4:08 pm

Which specific bit are you having an issue with?
You have said what you want to do, and that you need help, but what bit is not working?

I would start by simplifying it and first make sure everything can talk on the correct ports with tagged and untagged packets.
Then optimize performance and split into separate bridges if you want
Then apply firewall rules, and other settings like frame types etc.

Start with this where it is all in a single bridge and go from there.

/interface bridge
add name=BR vlan-filtering=yes

/interface vlan
add interface=BR name=vlan10 vlan-id=10
add interface=BR name=vlan11 vlan-id=11
add interface=BR name=vlan19 vlan-id=19
add interface=BR name=vlan20 vlan-id=20
add interface=BR name=vlan21 vlan-id=21
add interface=BR name=vlan29 vlan-id=29

/interface bridge port
add bridge=BR interface=ether7-access-vlan20 pvid=20
add bridge=BR interface=ether5--access-vlan11 pvid=11
add bridge=BR interface="ether2 - TRUNK3"
add bridge=BR interface=ether4-access-vlan10 pvid=10
add bridge=BR interface=ether8-access-vlan21 pvid=21
add bridge=BR interface="sfp-sfpplus2 - TRUNK1"
add bridge=BR interface="sfp-sfpplus3 - TRUNK2"

/interface bridge vlan
add bridge=BR tagged="sfp-sfpplus2 - TRUNK1,BR" untagged=ether7-access-vlan20 vlan-ids=20
add bridge=BR tagged="sfp-sfpplus2 - TRUNK1,BR" untagged=ether8-access-vlan21 vlan-ids=21
add bridge=BR tagged="sfp-sfpplus3 - TRUNK2,BR" untagged=ether4-access-vlan10 vlan-ids=10
add bridge=BR tagged="sfp-sfpplus3 - TRUNK2,BR" untagged=ether5--access-vlan11 vlan-ids=11
add bridge=BR tagged="ether2 - TRUNK3,BR" vlan-ids=19
add bridge=BR tagged="ether2 - TRUNK3,BR" vlan-ids=29

#### Sometimes adding a DHCP Server is useful for testing.
/ip address
add address=192.168.10.1/24 interface=vlan10 network=192.168.10.0
add address=10.0.11.1/24 interface=vlan11 network=10.0.11.0
add address=192.168.125.1/24 interface=vlan20 network=192.168.125.0
add address=10.0.21.1/24 interface=vlan21 network=10.0.21.0
add address=192.168.126.1/24 interface=vlan19 network=192.168.126.0
add address=192.168.129.1/24 interface=vlan29 network=192.168.129.0

/ip pool
add name=dhcp_pool0 ranges=192.168.10.2-192.168.10.254
add name=dhcp_pool1 ranges=10.0.11.2-10.0.11.254
add name=dhcp_pool2 ranges=192.168.126.2-192.168.126.254
add name=dhcp_pool3 ranges=192.168.125.2-192.168.125.254
add name=dhcp_pool4 ranges=10.0.21.2-10.0.21.254
add name=dhcp_pool5 ranges=192.168.129.2-192.168.129.254

/ip dhcp-server
add address-pool=dhcp_pool0 interface=vlan10 name=dhcp1
add address-pool=dhcp_pool1 interface=vlan11 name=dhcp2
add address-pool=dhcp_pool2 interface=vlan19 name=dhcp3
add address-pool=dhcp_pool3 interface=vlan20 name=dhcp4
add address-pool=dhcp_pool4 interface=vlan21 name=dhcp5
add address-pool=dhcp_pool5 interface=vlan29 name=dhcp6

/ip dhcp-server network
add address=10.0.11.0/24 dns-server=1.1.1.1 domain=vlan11.lan gateway=10.0.11.1
add address=10.0.21.0/24 dns-server=1.1.1.1 domain=vlan21.lan gateway=10.0.21.1
add address=192.168.10.0/24 dns-server=1.1.1.1 domain=vlan10.lan gateway=192.168.10.1
add address=192.168.125.0/24 dns-server=1.1.1.1 domain=vlan20.lan gateway=192.168.125.1
add address=192.168.126.0/24 dns-server=1.1.1.1 domain=vlan19.lan gateway=192.168.126.1
add address=192.168.129.0/24 dns-server=1.1.1.1 domain=vlan29.lan gateway=192.168.129.1

mr2web · Mon Jan 22, 2024 4:09 pm

Reading someones VLAN design, is always hard for me. Even with a very nice drawing , it helps, but still I didn't grasp it. It is probably me.

I give what I understand of VLAN and bridges, but in this matter, making a mistake is easy done.

Can you explain the purpose of using 3 bridges, and not just one?
Splitting VLAN in one bridge works fine.
The only double I see in multiple versus one, is the untagged traffic, that was separated per bridge, now would be common in the one bridge.
That looks easy to resolve, give it a separate VLAN number in the one bridge, and untag it where needed.

Ethernet interfaces normally are ports on the bridge, the VLANs used there are set in the bridge VLAN definitions, as tagged or untagged to some ports (ethernet interfaces)

Then no VLANs should be defined on the ethernet interfaces itself, only on the bridge.

It does exist ... VLAN on an ethernet interface, but then that VLAN is a different VLAN than the VLAN on the bridge or other interface even with the same number.

Interface VLAN 10,11,20,21 should be picked up at the bridge as interface, not at the SFP-TRUNK as interfaces, if these SFP-TRUNK interfaces are ports on the bridge.
Just like VLAN 19 and 29 are VLAN of the bridge.
But the VLAN interfaces themselves should not be added again to the bridge as ports, if the supporting interface already is a port.
The VLAN filtering of the bridge should give the access to those VLAN.

SFP Being ports of a bridge makes them "slave" interfaces, and the VLAN and DHCP and other IP functions of slave interfaces are handled only by the bridge, not by the interface.
If VLAN 19 and 29 are defined on a non-port-connected ethernet interface, then AFAIK the VLAN interfaces themselves can be added to a bridge on their own, typical as already untagged ports.

Confusing all this. I know, and I try to be carefull in what I say here.

Thanks for your time and effort answering and trying to understand my perhaps a bit odd usecase and setup.
Sorry for late reply. I have been away for most of the weekend.

Can you explain the purpose of using 3 bridges, and not just one?

In my usecase and the schematics above I have tried to describe a concept as simple as possible yet meeting my future goals. This concept is to be scaled up a little when its fully operational, yet for testing and learning purposes.
My REAL current setup is exactly what I shown above, there is nothing else going on at the moment. But there is a goal beyond this concept that will be just a bit larger in scale. As I intend to run multiple flooding tests in parallel in this environment I need to have the throughput to be able to saturate the physical machines multiple gigabit ports (in the concept they only have one each, but in the future they will have multiple ports per machine). In total spread over a few physical machines I plan to run tests against up to 12 Ethernet ports, if I find it doable that is. I will illiterately scale the solution to find where its maximum lies.
So to make sure them test sets not to interfere with each other I put them in different bridges (BR0 and BR1) as well over separate physical SFP+ ports.
BR2 is my attempt to keep them VM's, connected to VLAN19 and 29, to share a physical trunk port, yet being members of separate bridges (BR0 and BR1) but later on with firewall rules preventing packages from the flooding tests to leak from VLAN10 and VLAN20 to them VM's. This as them VM's will be also connected to another network that I do not like risking my flooding tests to leak into.

It kind of makes sense to me to keep them SFP+-trunk ports separate as they carry logical bound traffic. In my end-goal each physical SFP+ port cares for a set of physical machines logically belonging together. Trying to keep things separated is for me to keep this, to me, complex concept in order. But perhaps I dig my own grave that way?

So I have tried to keep the concept as simple as possible, but still as I have it IRL, for me to be able to learn and understand better as well for any of You kind people trying to push me int the right direction to grasp what I'm trying to achieve and expose more easily where I'm going wrong.
My goal is to keep this concept for my tests. Though I stand humble in-front of the fact I'm new to MikroTik and am not a very experienced networking guy, but I do believe I know the basics to some grade so I probably have done some design mistakes for sure. In short, what I like to achieve is to run these flooding tests in parallel without affecting each other. This is why I got this slightly more powerful router in the first place.

Splitting VLAN in one bridge works fine.
The only double I see in multiple versus one, is the untagged traffic, that was separated per bridge, now would be common in the one bridge.
That looks easy to resolve, give it a separate VLAN number in the one bridge, and untag it where needed.

I have tried to design for "no" untagged traffic except beyond the access ports that is. But I understand that good practises is to care for "all" traffic.

I have tried to stay clear of hybrid ports as well as they might complicate things even further for me?

Ethernet interfaces normally are ports on the bridge, the VLANs used there are set in the bridge VLAN definitions, as tagged or untagged to some ports (ethernet interfaces)

Then no VLANs should be defined on the ethernet interfaces itself, only on the bridge.

It does exist ... VLAN on an ethernet interface, but then that VLAN is a different VLAN than the VLAN on the bridge or other interface even with the same number.

Interface VLAN 10,11,20,21 should be picked up at the bridge as interface, not at the SFP-TRUNK as interfaces, if these SFP-TRUNK interfaces are ports on the bridge.
Just like VLAN 19 and 29 are VLAN of the bridge.
But the VLAN interfaces themselves should not be added again to the bridge as ports, if the supporting interface already is a port.
The VLAN filtering of the bridge should give the access to those VLAN.

Ok, I have moved the VLAN10, 11 to belong to BR0 and VLAN20, 21 to belong to BR1.
Still have a working connection in between A and E as when them VLANs was on the SFP+-interfaces.
I have removed VLAN19, 29 as bridge ports and added them as bridge vlans (VLAN19 to BR0 and VLAN29 to BR1)
BUT still have no connection in between E and J.

One thing I have a hard time getting my head around is the "ether2 - TRUNK3" port is not part of BR0 nor BR1 as a physical Ethernet port only can be member of one bridge, right?
This even if the BR0 and BR1 have a vlan with tag 19 to BR0 and 29 to BR1 wouldn't BR0 and BR1 need to have knowledge of "ether2 - TRUNK3" as bridge port? But as a physical Ethernet port only can be part of one bridge that will not work. I feel stuck in my head...

What would u suggest me to do to achieve having "ether2 - TRUNK3" as a shared trunk port for J only be able to communicate over VLAN10 over port 443 and 8080 (TCP) and K to be able to communicate over VLAN29 on port 443 and 8080 (TCP)? this as this is what I'm trying to achieve but are probably on the wrong track here.

I did adjust the addresses as well for VLAN19, 20 and 29. dono if that affects anything really.

Here are my current export:

@MikroTik] > export
# RouterOS 7.13
# model = CCR2116-12G-4S+
/interface bridge
add name=BR0 vlan-filtering=yes
add name=BR1 vlan-filtering=yes
add name=BR2 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether2 ] name="ether2 - TRUNK3"
set [ find default-name=sfp-sfpplus2 ] name="sfp-sfpplus2 - TRUNK1"
set [ find default-name=sfp-sfpplus3 ] name="sfp-sfpplus3 - TRUNK2"
/interface vlan
add interface=BR0 name=VLAN10 vlan-id=10
add interface=BR0 name=VLAN11 vlan-id=11
add interface=BR2 name=VLAN19 vlan-id=19
add interface=BR1 name=VLAN20 vlan-id=20
add interface=BR1 name=VLAN21 vlan-id=21
add interface=BR2 name=VLAN29 vlan-id=29
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=BR0 frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=10
add bridge=BR0 frame-types=admit-only-vlan-tagged interface="sfp-sfpplus2 - TRUNK1"
add bridge=BR1 frame-types=admit-only-vlan-tagged interface="sfp-sfpplus3 - TRUNK2"
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=20
add bridge=BR0 frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=11
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=21
add bridge=BR2 frame-types=admit-only-vlan-tagged interface="ether2 - TRUNK3"
/ip firewall connection tracking
set enabled=yes
/interface bridge vlan
add bridge=BR0 tagged="sfp-sfpplus2 - TRUNK1" vlan-ids=10,11
add bridge=BR1 tagged="sfp-sfpplus3 - TRUNK2" vlan-ids=20,21
add bridge=BR0 tagged="ether2 - TRUNK3" vlan-ids=19
add bridge=BR1 tagged="ether2 - TRUNK3" vlan-ids=29
/ip address
add address=192.168.125.12/24 interface=VLAN10 network=192.168.125.0
add address=10.0.11.12/24 interface=VLAN11 network=10.0.11.0
add address=192.168.125.14/24 interface=VLAN20 network=192.168.125.0
add address=10.0.21.12/24 interface=VLAN21 network=10.0.21.0
add address=192.168.125.13/24 interface=VLAN19 network=192.168.125.0
add address=192.168.125.15/24 interface=VLAN29 network=192.168.125.0
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key
[@MikroTik] >

Thank you ever so much again for taking the time, much appreciated!

anav · Mon Jan 22, 2024 5:09 pm

Sure I can look, and will respect your wishes to have separate bridges etc.............

1. I'm kinda org freak so moved rules around per bridge basis for easy understanding.

2. The biggest error I see is not tagging the bridge..... as per --> viewtopic.php?t=143620

3. The second error stems from WHY I INSIST ON MANUALLY UNTAGGING ON /interface bridge vlans. So that one can cross-check with /bridge ports.
Yes, it is not mandatory, but only for those that know what they are doing.............. and even then its not something I do. The key to keep in mind is that each /interface bridge vlan LINE in the configuration is to express the tagging and untagging of a single VLAN. You can combine multiple vlan-ids, ONLY IF, the tagging and untagging for each vlan is identical.
Easy to think it is if you dont put in the untagging.................... On the other hand, you can combine 19,29 !!

/interface bridge vlan
add bridge=BR0 tagged="sfp-sfpplus2 - TRUNK1" vlan-ids=10,11 <------
add bridge=BR1 tagged="sfp-sfpplus3 - TRUNK2" vlan-ids=20,21 <-------
add bridge=BR0 tagged="ether2 - TRUNK3" vlan-ids=19
add bridge=BR1 tagged="ether2 - TRUNK3" vlan-ids=29

4. Not unexpected to get mixed up trying different approaches an example is this line in the /interface bridge vlan settings.....
add bridge=BR0 tagged="ether2 - TRUNK3" vlan-ids=19[/b] Clearly vlan19 has nothing to do with BR0!!
add bridge=BR1 tagged="ether2 - TRUNK3" vlan-ids=29 Ditto, 29 nothing to do with BR1!!

# RouterOS 7.13
# model = CCR2116-12G-4S+
/interface bridge
add name=BR0 vlan-filtering=yes
add name=BR1 vlan-filtering=yes
add name=BR2 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether2 ] name="ether2 - TRUNK3"
set [ find default-name=sfp-sfpplus2 ] name="sfp-sfpplus2 - TRUNK1"
set [ find default-name=sfp-sfpplus3 ] name="sfp-sfpplus3 - TRUNK2"
/interface vlan
add interface=BR0 name=VLAN10 vlan-id=10
add interface=BR0 name=VLAN11 vlan-id=11
add interface=BR1 name=VLAN20 vlan-id=20
add interface=BR1 name=VLAN21 vlan-id=21
add interface=BR2 name=VLAN19 vlan-id=19
add interface=BR2 name=VLAN29 vlan-id=29
/interface bridge port
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=10
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=11
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="sfp-sfpplus2 - TRUNK1"
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="sfp-sfpplus3 - TRUNK2"
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=20
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=21
add bridge=BR2 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="ether2 - TRUNK3"
/ip firewall connection tracking
set enabled=yes
/interface bridge vlan
add bridge=BR0 tagged=BR0,"sfp-sfpplus2 - TRUNK1" untagged=ether4 vlan-ids=10
add bridge=BR0 tagged=BR0,"sfp-sfpplus2 - TRUNK1" untagged=ether5 vlan-ids=11
add bridge=BR1 tagged=BR1,"sfp-sfpplus3 - TRUNK2" untagged=ether7 vlan-ids=20
add bridge=BR1 tagged=BR1,"sfp-sfpplus3 - TRUNK2" untagged=ether8 vlan-ids=21
add bridge=BR2 tagged=BR2,"ether2 - TRUNK3" vlan-ids=19,29
/ip address
add address=192.168.125.12/24 interface=VLAN10 network=192.168.125.0
add address=10.0.11.12/24 interface=VLAN11 network=10.0.11.0
add address=192.168.125.14/24 interface=VLAN20 network=192.168.125.0
add address=10.0.21.12/24 interface=VLAN21 network=10.0.21.0
add address=192.168.125.13/24 interface=VLAN19 network=192.168.125.0
add address=192.168.125.15/24 interface=VLAN29 network=192.168.125.0
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key
[@MikroTik] >

mr2web · Mon Jan 22, 2024 6:13 pm

Really a straightforward setup for the most part, the question I have is why do you have TWO ethernet ports going to the hypervisor on the left??
You only need one port going to a smart switch for example if it was there instead of the hyper visor.

Thus I would need to understand what the hypervisor is doing>>> I would be afraid of some connection between those hypervisor ports and looping etc..............

Concur, with bpwl AND THE LINK!!! One bridge, all vlans with interface bridge, and bridge does no DHCP.

The one that struck me is there should be a trusted vlan to all smart devices ( and all smart devices should get an IP on that vlan).
I dont see that as the hypervisor left side and right side dont have a common vlan. Vlans are cheap LOL.

Once you have answered the questions in both posts, I will have a bit better understanding to tackle the config.

+++++++++++++++++++++++++++

By the way to comment coherently on your config, the whole export minus router serial number, public WANIP info, keys etc would be needed.
For example I dont see any FW rules, interface lists etc...... A config is the sum of many integrated parts!!!

Thanks for your time and effort answering and trying to understand my perhaps a bit odd usecase and setup. I have submitted an answer to bpwl, please have a look if possible. I hope I have answered most of your questions in that reply. I will add some answers here as well just so I feel I have done my best to provide the required information.
Sorry for late reply. I have been away for most of the weekend.

Thus I would need to understand what the hypervisor is doing>>> I would be afraid of some connection between those hypervisor ports and looping etc..............

The hypervisors are terminating the trunk connections routing the vlans to the intended VM's. Each device on the each vlan on the right hand side hypervisor has no other connections. I far as I can see there do not seen very likely for any risk of loops. The hypervisor on the left holds two VM's that should have access to VLAN10 and VLAN20 only over port 8080 and 443 (TCP). Else these VM's have also access to another network not part of this concept and schematics, but will be needed in the future. There of why I like to limit their access to these vlans. The intention is for these VM's to set up the physical test machines, trigger tests and gather test results. Why they need to share a trunk port is due to the lack of physical port on the router. BUT I really like to understand how this can be achieved as I see the possibility to share physical trunk ports among multiple bridges quite useful in the future.
I have considered buying a separate vlan aware switch to get around this need of sharing a trunk port.

The one that struck me is there should be a trusted vlan to all smart devices ( and all smart devices should get an IP on that vlan).

I have seen this option for marking a bridge port as "trusted" but I have yet not looked into it. Will do that right after finishing these replies.

I will have to look up the possibility to make a vlan also trusted when I find out what "trusted" actual means.

I dont see that as the hypervisor left side and right side dont have a common vlan. Vlans are cheap LOL.

Don't know what you are thinking of here, but it got me thinking of how to actual trigger these tests on the VM's on machines marked with A, B, C,and D from the VM's on the left hypervisor marked with J and K. Thanks for pushing me there.

Probably need a separate vlan for trigger the tests. As stated vlans are cheep!

Do u feel like I have answered all questions of have I missed out on any? most answers are probably in the answer to pbwl. Perhaps more questions have popped up?
I really do appreciate the effort you put into my learning.

Thanks!

anav · Mon Jan 22, 2024 6:34 pm

No worries, --> no not a selection for trusted, trusted meaning in concept, different thing!

As far as trusted, subnet or vlan yes, and NOT a trusted port (that gets into physical security which is a whole other entity).

A trusted subnet (home) or management subnet (business), is the subnet where all smart devices get their IP address from.
No non-admins (or untrusted users should be on this network).

Typically trusted subnet is made part of a management interface list. This is done in MT because that interface list can be used in neighours discovery, so that winbox can find all the mikrotiks ( routers, APs, switches) and display them for easy access. The management subnet ( for devices ) is typically combined with other subnets or interfaces where the ADMIN may typically want to access the router from for config purposes. Lets say, wireguard subnet, management (smart devices subnet) and an isolated work subnet and they form the complete management interface list.
Now we can use that interface list in the too mac-server mac-winbox interface setting _ which is one of the security parameters allowing/limiting winbox access.
++++++++++++

mr2web · Mon Jan 22, 2024 7:44 pm

Sure I can look, and will respect your wishes to have separate bridges etc.............

1. I'm kinda org freak so moved rules around per bridge basis for easy understanding.

2. The biggest error I see is not tagging the bridge..... as per --> viewtopic.php?t=143620

3. The second error stems from WHY I INSIST ON MANUALLY UNTAGGING ON /interface bridge vlans. So that one can cross-check with /bridge ports.
Yes, it is not mandatory, but only for those that know what they are doing.............. and even then its not something I do. The key to keep in mind is that each /interface bridge vlan LINE in the configuration is to express the tagging and untagging of a single VLAN. You can combine multiple vlan-ids, ONLY IF, the tagging and untagging for each vlan is identical.
Easy to think it is if you dont put in the untagging.................... On the other hand, you can combine 19,29 !!

/interface bridge vlan
add bridge=BR0 tagged="sfp-sfpplus2 - TRUNK1" vlan-ids=10,11 <------
add bridge=BR1 tagged="sfp-sfpplus3 - TRUNK2" vlan-ids=20,21 <-------
add bridge=BR0 tagged="ether2 - TRUNK3" vlan-ids=19
add bridge=BR1 tagged="ether2 - TRUNK3" vlan-ids=29

4. Not unexpected to get mixed up trying different approaches an example is this line in the /interface bridge vlan settings.....
add bridge=BR0 tagged="ether2 - TRUNK3" vlan-ids=19[/b] Clearly vlan19 has nothing to do with BR0!!
add bridge=BR1 tagged="ether2 - TRUNK3" vlan-ids=29 Ditto, 29 nothing to do with BR1!!

# RouterOS 7.13
# model = CCR2116-12G-4S+
/interface bridge
add name=BR0 vlan-filtering=yes
add name=BR1 vlan-filtering=yes
add name=BR2 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether2 ] name="ether2 - TRUNK3"
set [ find default-name=sfp-sfpplus2 ] name="sfp-sfpplus2 - TRUNK1"
set [ find default-name=sfp-sfpplus3 ] name="sfp-sfpplus3 - TRUNK2"
/interface vlan
add interface=BR0 name=VLAN10 vlan-id=10
add interface=BR0 name=VLAN11 vlan-id=11
add interface=BR1 name=VLAN20 vlan-id=20
add interface=BR1 name=VLAN21 vlan-id=21
add interface=BR2 name=VLAN19 vlan-id=19
add interface=BR2 name=VLAN29 vlan-id=29
/interface bridge port
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=10
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=11
add bridge=BR0 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="sfp-sfpplus2 - TRUNK1"
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="sfp-sfpplus3 - TRUNK2"
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=20
add bridge=BR1 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=21
add bridge=BR2 ingress-filtering=yes frame-types=admit-only-vlan-tagged interface="ether2 - TRUNK3"
/ip firewall connection tracking
set enabled=yes
/interface bridge vlan
add bridge=BR0 tagged=BR0,"sfp-sfpplus2 - TRUNK1" untagged=ether4 vlan-ids=10
add bridge=BR0 tagged=BR0,"sfp-sfpplus2 - TRUNK1" untagged=ether5 vlan-ids=11
add bridge=BR1 tagged=BR1,"sfp-sfpplus3 - TRUNK2" untagged=ether7 vlan-ids=20
add bridge=BR1 tagged=BR1,"sfp-sfpplus3 - TRUNK2" untagged=ether8 vlan-ids=21
add bridge=BR2 tagged=BR2,"ether2 - TRUNK3" vlan-ids=19,29
/ip address
add address=192.168.125.12/24 interface=VLAN10 network=192.168.125.0
add address=10.0.11.12/24 interface=VLAN11 network=10.0.11.0
add address=192.168.125.14/24 interface=VLAN20 network=192.168.125.0
add address=10.0.21.12/24 interface=VLAN21 network=10.0.21.0
add address=192.168.125.13/24 interface=VLAN19 network=192.168.125.0
add address=192.168.125.15/24 interface=VLAN29 network=192.168.125.0
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key
[@MikroTik] >

Thanks for your quick response.

I have read your response a few times and am still processing it. Most of it make so much sense. But I have a few gaps that I'm processing and I am doing yet another run on the "article" everyone is pointing to. It is good so I'm happy to read it another time, hopefully with fresh eyes and new angles to take the information to me.

I will get back to you when I'm ready as I have not been able to get connections working in between J and E but the connection in between A and E works as in the past and I'll need to use the knowledge gained from your reply as well as the other replies and other sources.

As I have written quite extensive replies to the other two reposes I felt like giving you a heads up on that I'm still processing your reply and that I'm very glad you took the effort to support me in my learning.

anav · Mon Jan 22, 2024 7:48 pm

No worries, just send $$$ kidding!!

mr2web · Mon Jan 22, 2024 8:00 pm

Which specific bit are you having an issue with?
You have said what you want to do, and that you need help, but what bit is not working?

I would start by simplifying it and first make sure everything can talk on the correct ports with tagged and untagged packets.
Then optimize performance and split into separate bridges if you want
Then apply firewall rules, and other settings like frame types etc.

Start with this where it is all in a single bridge and go from there.

/interface bridge
add name=BR vlan-filtering=yes

/interface vlan
add interface=BR name=vlan10 vlan-id=10
add interface=BR name=vlan11 vlan-id=11
add interface=BR name=vlan19 vlan-id=19
add interface=BR name=vlan20 vlan-id=20
add interface=BR name=vlan21 vlan-id=21
add interface=BR name=vlan29 vlan-id=29

/interface bridge port
add bridge=BR interface=ether7-access-vlan20 pvid=20
add bridge=BR interface=ether5--access-vlan11 pvid=11
add bridge=BR interface="ether2 - TRUNK3"
add bridge=BR interface=ether4-access-vlan10 pvid=10
add bridge=BR interface=ether8-access-vlan21 pvid=21
add bridge=BR interface="sfp-sfpplus2 - TRUNK1"
add bridge=BR interface="sfp-sfpplus3 - TRUNK2"

/interface bridge vlan
add bridge=BR tagged="sfp-sfpplus2 - TRUNK1,BR" untagged=ether7-access-vlan20 vlan-ids=20
add bridge=BR tagged="sfp-sfpplus2 - TRUNK1,BR" untagged=ether8-access-vlan21 vlan-ids=21
add bridge=BR tagged="sfp-sfpplus3 - TRUNK2,BR" untagged=ether4-access-vlan10 vlan-ids=10
add bridge=BR tagged="sfp-sfpplus3 - TRUNK2,BR" untagged=ether5--access-vlan11 vlan-ids=11
add bridge=BR tagged="ether2 - TRUNK3,BR" vlan-ids=19
add bridge=BR tagged="ether2 - TRUNK3,BR" vlan-ids=29

#### Sometimes adding a DHCP Server is useful for testing.
/ip address
add address=192.168.10.1/24 interface=vlan10 network=192.168.10.0
add address=10.0.11.1/24 interface=vlan11 network=10.0.11.0
add address=192.168.125.1/24 interface=vlan20 network=192.168.125.0
add address=10.0.21.1/24 interface=vlan21 network=10.0.21.0
add address=192.168.126.1/24 interface=vlan19 network=192.168.126.0
add address=192.168.129.1/24 interface=vlan29 network=192.168.129.0

/ip pool
add name=dhcp_pool0 ranges=192.168.10.2-192.168.10.254
add name=dhcp_pool1 ranges=10.0.11.2-10.0.11.254
add name=dhcp_pool2 ranges=192.168.126.2-192.168.126.254
add name=dhcp_pool3 ranges=192.168.125.2-192.168.125.254
add name=dhcp_pool4 ranges=10.0.21.2-10.0.21.254
add name=dhcp_pool5 ranges=192.168.129.2-192.168.129.254

/ip dhcp-server
add address-pool=dhcp_pool0 interface=vlan10 name=dhcp1
add address-pool=dhcp_pool1 interface=vlan11 name=dhcp2
add address-pool=dhcp_pool2 interface=vlan19 name=dhcp3
add address-pool=dhcp_pool3 interface=vlan20 name=dhcp4
add address-pool=dhcp_pool4 interface=vlan21 name=dhcp5
add address-pool=dhcp_pool5 interface=vlan29 name=dhcp6

/ip dhcp-server network
add address=10.0.11.0/24 dns-server=1.1.1.1 domain=vlan11.lan gateway=10.0.11.1
add address=10.0.21.0/24 dns-server=1.1.1.1 domain=vlan21.lan gateway=10.0.21.1
add address=192.168.10.0/24 dns-server=1.1.1.1 domain=vlan10.lan gateway=192.168.10.1
add address=192.168.125.0/24 dns-server=1.1.1.1 domain=vlan20.lan gateway=192.168.125.1
add address=192.168.126.0/24 dns-server=1.1.1.1 domain=vlan19.lan gateway=192.168.126.1
add address=192.168.129.0/24 dns-server=1.1.1.1 domain=vlan29.lan gateway=192.168.129.1

Thanks for your time and effort answering and trying to understand my perhaps a bit odd usecase and setup. I have submitted an answer to bpwl and anav, please have a look if possible if you like a slightly deeper insight to what I'm trying to achieve and learn.
Sorry for late reply. I have been away for most of the weekend.

I'm still processing all the answers and I just like to give you heads up on that have read your response and I'm having another go at the "article" all is referring to. I'm trying to put all good advice and knowledge together to understand how to get the router to do what I need it to do.

I like your suggested approach, I have actually been down that road kind of, perhaps not exactly your suggested road, but trying to get all to be able to communicate and I have gotten all to work. but not with a shared trunk port over two bridges. I kind off need this due to lack of physical ports on the router. but naturally I can get a vlan aware switch to get around that. But I like to understand if it is possible. I feel like it should but I might be vrong.

Well, its getting late and I have been at this quite a lot today. will get back on it tomorrow again.

Thanks for your reply!!! I'm still processing it so I'll most likely get back to u at a later point.

Cheers

Sharing one physical trunk port with two bridges

Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Re: Sharing one physical trunk port with two bridges

Who is online