Community discussions

MikroTik App
 
pxcorp
just joined
Topic Author
Posts: 11
Joined: Thu Jun 15, 2017 7:48 pm

RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 15, 2017 8:30 pm

Hello i have problems with packet tagging on RB3011. I configured VLANs on both switch chips a set switch1-cpu and switch2-cpu port to secure mode and header add if missing.
Then I instantiate VLAN 200 interfaces on both master ports and bridget it together. In this configuration I observe significant packet loose between some clients in VLAN200 if traffic goes thru bridge.
After little peaking with torch tool I found that sometimes switch chip omit add vlan tag on cpu port if packet comes from access port (untaged) of switch. From trunk port (already tagged) it probably allways works (cpu gets packet with vlan). Also when device is on access and trunk port on same switch it works - so problem is only with tagging on cpu port.
I expected that when I set "add if missing" on cpu port then CPU sould not get untagged packet, but according to attached picture sometimes VLAN 200 is added and sometimes not.
Legend:
10.100.0.100 computer attached to ether5 (header always strip,mode secure,vlan 200) - used to send ping requests
10.100.0.16 device sttached to ether9 (header add if missing,mode secure)
10.100.0.1 RB3011 itself

Same behavior found with ROS 6.40rc19, 6.40rc21, 6.39.2
When I looked in diagram of RB3011 I suspect that only one link between each switch and CPU is controlled by header mode settings, but second is left in leave as is mode. And this break tagging from access ports if packet is send to cpu via second link. Also behavior is very similar to case when bonded interfaces have broken one link between them.

configuration of switch in test setup:
/interface ethernet
set [ find default-name=ether4 ] name=eth4
set [ find default-name=ether5 ] comment="Dohledovy port" master-port=eth4 name=eth5
set [ find default-name=ether6 ] comment="maste LAN port (VLANy)" name=eth6
set [ find default-name=ether7 ] master-port=eth6 name=eth7
set [ find default-name=ether8 ] master-port=eth6 name=eth8
set [ find default-name=ether9 ] master-port=eth6 name=eth9
set [ find default-name=ether10 ] master-port=eth6 name=eth10
set [ find default-name=ether1 ] master-port=eth4 name=eth1
set [ find default-name=ether2 ] master-port=eth4 name=eth2
set [ find default-name=ether3 ] master-port=eth4 name=eth3
/interface ethernet switch port
set 0 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 1 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 2 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 3 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 4 default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
set 5 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 6 default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
set 7 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 8 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 9 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 10 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
set 11 default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure
/interface ethernet switch vlan
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=101
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=102
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=103
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=104
add independent-learning=no ports=eth6,eth7,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=200
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=201
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=202
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=switch2 vlan-id=1
add independent-learning=no ports=eth1,eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=10
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=51
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=101
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=102
add independent-learning=no ports=eth2,eth3,eth4,eth5,switch1-cpu switch=switch1 vlan-id=200
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=202
add independent-learning=no ports=eth1,eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=1
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=50
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 vlan-id=11
You do not have the required permissions to view the files attached to this post.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10544
Joined: Mon Jun 08, 2015 12:09 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 15, 2017 10:33 pm

The correct setting for an untagged port is default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
I would not use the default-vlan-id=1 vlan-header=add-if-missing vlan-mode=secure setting.
Make another VLAN (100 for example) and use that for the "other VLAN", vlan 1 remains unused.
 
pxcorp
just joined
Topic Author
Posts: 11
Joined: Thu Jun 15, 2017 7:48 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 15, 2017 10:58 pm

Access port (ether 5) have such setting:
set 4 default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
If I understand principe of settings for tagging inside switch it is necessary to explicitly add tags to packet which came from access port to trunk port
So for trunk and CPU port I need to set add VLAN tag to packet which come from access port (set vlan-header=add-if-missing vlan-mode=secure)
Otherwise packet from access ports (originally untagged) go to trunk port without tag, which is wrong. Same apply for CPUport if I wan to use single interface for VLAN 200 in trunk and access ports.
Workaround can be to set cpu port to leave header as is and bridge eth5 to VLAN200 from both switches, but this way communication from eth5 allways go thru CPU (ether in case where source and target is on same switch)
 
pe1chl
Forum Guru
Forum Guru
Posts: 10544
Joined: Mon Jun 08, 2015 12:09 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 15, 2017 11:38 pm

I explain you how is the correct setting to make it work.
When you want to devise your own way, that's fine!
However, I think you started the topic because you have a problem, so maybe it is not the correct way.
Good luck!
 
pxcorp
just joined
Topic Author
Posts: 11
Joined: Thu Jun 15, 2017 7:48 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Fri Jun 16, 2017 9:34 am

Hello,
it is not my intent to do in my own strage way, but understand what I am doing, so I expected no only "do it this way" but also why it should be done this way/what happend inside switch chipset.
I think that networking isn't "black magic", or at least shouldn't be - network administrator should know what happends with packet inside network, not only set configuration to something that he somehow find to working.
Later I also found notice in Wiki that for switch used in RB3011 vlan-header mode doesn't apply if vlan-mode is set to secure - adding/stripping tag is controlled by default vlan ID of port.

Maybe I should better explain what I am trying to configure:
Ether2,3,4,6,8,9,10 is VLAN trunk, in all this trunks exist VLAN200 (which is device management network)
Ether5,Ether7 is access port for such network (VLAN200)
RB3011 has also own address in this network
My intent was to progrm all vlan stuff to switch chips in RB3011 and then add vlan interface (VLAN200) to master ports for each group, bridge this two vlan interfaces together and add routers IP to this bridge.
But this configuration seems not working when packet come from access ports and have to go thru bridge.

I also give a try supposed changes in configuration (removed VLAN 1 from switch, set header-mode to leave-as-is), but problem with communication become even worse - now I have problem not only to access devices on trunks but also router itself (connection randomly drops) from access port. But this may be just result of changed configuration, previously i found that packet loose is dependent on MACs, IPs, etc.. of communicatin devices.

New config (I also add to listing items related to bridging, to explain what is intended by configuration):
Post edited: added /interface ethernet switch vlan (forget to copy from cfg when posting)

/interface ethernet
set [ find default-name=ether4 ] name=eth4
set [ find default-name=ether5 ] comment="Dohledovy port" master-port=eth4 \
name=eth5
set [ find default-name=ether6 ] comment="maste LAN port (VLANy)" name=eth6
set [ find default-name=ether7 ] master-port=eth6 name=eth7
set [ find default-name=ether8 ] master-port=eth6 name=eth8
set [ find default-name=ether9 ] master-port=eth6 name=eth9
set [ find default-name=ether10 ] master-port=eth6 name=eth10
set [ find default-name=ether1 ] master-port=eth4 name=eth1
set [ find default-name=ether2 ] master-port=eth4 name=eth2
set [ find default-name=ether3 ] master-port=eth4 name=eth3

/interface ethernet switch port
set 0 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 1 default-vlan-id=1000 vlan-mode=secure
set 2 default-vlan-id=1000 vlan-mode=secure
set 3 default-vlan-id=1000 vlan-mode=secure
set 4 default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
set 5 default-vlan-id=1000 vlan-mode=secure
set 6 default-vlan-id=200 vlan-header=always-strip vlan-mode=secure
set 7 default-vlan-id=1000 vlan-mode=secure
set 8 default-vlan-id=1000 vlan-mode=secure
set 9 default-vlan-id=1000 vlan-mode=secure
set 10 default-vlan-id=1000 vlan-mode=secure
set 11 default-vlan-id=1000 vlan-mode=secure

/interface ethernet switch vlan
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=101
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=102
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=103
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=104
add independent-learning=no ports=eth6,eth7,eth8,eth9,eth10,switch2-cpu \
switch=switch2 vlan-id=200
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=201
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=202
add independent-learning=no ports=eth1,eth2,eth3,eth4,switch1-cpu switch=\
switch1 vlan-id=10
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=51
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=101
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=102
add independent-learning=no ports=eth2,eth3,eth4,eth5,switch1-cpu switch=\
switch1 vlan-id=200
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=202
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=50
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=11
add independent-learning=no ports=eth2,eth3,eth4,switch1-cpu switch=switch1 \
vlan-id=1000
add independent-learning=no ports=eth6,eth8,eth9,eth10,switch2-cpu switch=\
switch2 vlan-id=1000


/interface vlan
add comment="dohled APcek" interface=eth6 name=vlan200a vlan-id=200
add comment="dohled APcek" interface=eth4 name=vlan200b vlan-id=200
/interface bridge
add comment="bridge pro dohled" name=brg_mgmt protocol-mode=none
/interface bridge port
add bridge=brg_mgmt interface=vlan200a
add bridge=brg_mgmt interface=vlan200b
add address=10.100.0.1/24 comment="dohledova IP" interface=brg_mgmt network=10.100.0.0
 
pe1chl
Forum Guru
Forum Guru
Posts: 10544
Joined: Mon Jun 08, 2015 12:09 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Fri Jun 16, 2017 10:54 am

It could be caused by STP. When you have STP switches in your network or other MikroTik devices running
older firmware there are STP incompatabilities. Turn off STP on the bridge (protocol none) and see if that helps.
Also, set a MAC address on the bridge so that it is not determined automatically (you can copy the automatically
determined MAC with the 02 bit of the upperbyte set).

In principle your switch config is correct now. The explanation of the switch features on the wiki is a bit
unclear and mostly be-example, which is bad when you want to know the exact details. I struggled with it
as well before I understood how to make the "some ports trunk, some ports access for a specific VLAN" config
that you want and I know that add-if-missing is not what you want to do. always-strip with default-vlan set
also implies always-add in the other direction, so no need to fiddle with that further in the switch or router.

In my configs I don't use the untagged VLAN at the router (cpu port) side. I think you have that as well now. I use only
tagged VLANs there.
 
pxcorp
just joined
Topic Author
Posts: 11
Joined: Thu Jun 15, 2017 7:48 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Fri Jun 16, 2017 11:25 am

Hello,
in test setup all devices other than computer was mikrotiks (HAPac lite) with 6.40rc19 or rc21, spanning tree is disabled everywhere on bridges (but I check it again for sure). Changing brg_mngmt MAC to generated one was first thing that I tried - but without success (it only changed which device was visible and which not).
Currently config don't use untagged traffic to CPU, althought VLAN1000 is defined on switch, nothing is connected to it in interfaces settings (nothing directly on master ports and no VLAN1000 interface).

This morning I get another external switch do some aditional testing to closer specify conditions when it works and when not :
trunk port - trunk port communication - this allways work either when port is on different switch chips (traffic go thru bridge)
access port - trunk port on same switch chip - this allways work
router (cpu port) - trunk port - this allways works
router (cpu port) - access port - not sure if it works correctly, randomly connection get dropped
access port - trunk port on oposite switch chip - this exhibit problems in communication with some clients:
I found also that packet lost depends on IP and/or MAC of client - I have 3 clients connected on eth9 via external switch and can communicate only with one from eth5 access port, but router itself see all clients. So definitely it is problem with traffic from access port that go thru CPU (bridge).
Also behavior changed slightly when SFP is inserted - but because it cannot be done when powered i don't know if it is result of rebooting or change in internal topology due to SFP.

Update:
STP/RSTP is disabled on all bridges in test setup (I checked all connected devices)
router (cpu port) - access port communication not working correctly, exhibit same problems as access port to trunk port via bridge (probably I've luck in choosing IP/MAC for previous tests)
when SFP is inserted (just power off,insert,power on - no configuration change) then switch 2 communication looks like working correctly. Switch 1 still have some problems but in much less cases.
 
plum
just joined
Posts: 13
Joined: Mon May 22, 2017 3:40 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Sat Jun 17, 2017 11:48 am

eth6 and eth4 are trunks too, right? so why are they on "always strip"? doesn't that make them access ports?


In my configs
It would really be awesome to see an export of a working VLAN example in this forum!
 
pe1chl
Forum Guru
Forum Guru
Posts: 10544
Joined: Mon Jun 08, 2015 12:09 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Sat Jun 17, 2017 1:00 pm

Remember that the numbering of the ports in the switch is not the same as the naming of the ports in the interface section.
The switch starts numbering ports at zero. In interfaces you can use any name but they normally start at ether1.

There is little point in posting "a working config" because it depends on what you want to do. There are examples in the wiki.
The "default-vlan-id=xxxx vlan-header=always-strip vlan-mode=secure" is a working config for an untagged port on vlan xxxx.
(contrary to what you may think, "always-strip" implies "always-add" in the other direction)
 
mickey
just joined
Posts: 6
Joined: Sun Mar 19, 2017 3:29 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 29, 2017 9:04 pm

I have observed that too actually , this was my post with helpful picture viewtopic.php?f=13&t=120661&p=593266#p593266
On this router shared Vlan between two switches is kind of magic. Sometimes reboot helps, sometimes assign static ip, then changing to new and reconnect device helping. It's totally weird because RB2011 have two switches and no problems with shared vlans.
 
pxcorp
just joined
Topic Author
Posts: 11
Joined: Thu Jun 15, 2017 7:48 pm

Re: RB3011 UiAS switch1/2-cpu vlan-header mode problem

Thu Jun 29, 2017 10:30 pm

Mickey:
Maybe this is because RB3011 and RB2011 have different internal connection between CPU and switches (RB3011 have two links to each switch and RB2011 only one to each).
I don't find any configuration of switch that works, so I have to bridge access ports to VLANs in CPU instead of switch (to get this working you have to set IVL on default CPU VLAN and bridged VLANs otherwise switch gets totally confused).
I suspect that problem is in internal bonding between two links to CPU - also symptoms look like if you have bonded interfaces and one link have significant error rate (this can also explain why behavior changes with IP addresses etc... - bonding engine choose path for packet by hashing packets header). Also my least experiments with SFP (inserting SPF disconnect one link from one switch) talks for this explanation of problem.