CRS VLAN mystery

r2504 · Sat Sep 27, 2014 12:57 am

Since I'm new to the CRS series, I'm trying to get VLAN's working but I experience some very strange issues.

First of all, this is my test setup;

/interface ethernet
set [ find default-name=sfp1 ] name=sfp1 master-port=none
set [ find default-name=ether2 ] name=ether2 master-port=sfp1

# Interfaces VLAN
/interface vlan
add name=VLAN50 vlan-id=50 interface=sfp1

# Switch VLAN
/interface ethernet switch vlan
add ports=sfp1,switch1-cpu,ether2 vlan-id 50 learn=yes

# Switch VLAN tagging
/interface ethernet switch egress-vlan-tag
add vlan-id 50 tagged-ports=switch1-cpu

# Switch In VLAN
/interface ethernet switch ingress-vlan-translation
add customer-vlan-format=any customer-vid=0 new-customer-vid=50 ports=ether2 sa-learning=yes

/ip dhcp-client
add interface=VLAN50

/ip pool
add name=VLAN50-pool ranges=192.168.50.1-192.168.50.100

/ip dhcp-server network
add address=192.168.50.0/24 gateway=192.168.50.254 dns-server=8.8.8.8

/ip dhcp-server
add name=VLAN50-DHCP interface=VLAN50 lease-time=01:00:00 address-pool=VLAN50-pool

The issue is now that the DHCP client doesn't get an address... but now it becomes even more strange. Port 2 is in fact connected to my core switch but on an access port (which internally is linked to VLAN 255 - but this is offcourse stripped when leaving the access port- which is my provider). If I change the VLAN id in the configuration of the CRS above to also use 255... the DHCP client DOES get an address ???

How can this be... the CRS shouldn't know at all about the VLAN on my core switch ! I used wireshark to see if somehow VLAN tagged traffic is coming out if it but it doesn't ???

Due to all this I played a bit more and replaced the DHCP client by an IP-address linked to VLAN50. The strange thing is that when I now connect a laptop (which has a static IP in the same subnet) to port 2, I can ping it fine... however if I connect another laptop to port 2 the same ping doesn't work anymore (while on my core switch both laptops work fine) ???

Am I missing something... or is this some magic going on ? Any additional information about what each Switch VLAN menu does would be a great help.

r2504 · Sun Sep 28, 2014 3:01 pm

Part of the mystery has been solved... or maybe just not.

The laptop that had issues with the ping request is a Windows 8 with Hyper-V enabled, hence it has a virtual switch. Once I deleted this configuration, the ping worked fine. I don't know however if this is a good thing as the laptop did work fine on my own network which also has access ports (and VLAN's)... so what makes it special that this CRS configuration doesn't work with my laptop ?

And besides that... the CRS still doesn't get an IP address from on its DHCP client ???

r2504 · Mon Sep 29, 2014 12:07 am

Tired of this mystery and the possible influence of my core switch (and my ISP running in VLAN id 255 over my home swicthes) I unplugged my home network from the ISP's cable modem and directly connected the CRS to it with the above configuration. Now I didn't get an IP-address, no matter which VLAN id I selected.

This makes me believe something is wrong with VLAN's in combination with DHCP clients on 6.19... unless I'm really missing something and still don't understand the whole VLAN stuff on a CRS yet

At this moment I'm really disappointed/frustrated on this CRS range and Mikrotik giving hardly any decent information on VLAN's

Mon Sep 29, 2014 9:55 am

This seems to be some configuration issue, because after importing your posted configuration on my CRS125 and enabling DHCP client (which was disabled by default) I got an IP address on VLAN50 from untagged DHCP server connected to ether2.
Could you post a full configuration export from your CRS?

r2504 · Mon Sep 29, 2014 9:20 pm

Please find attached the export. It is the result of a reset configuation and running script above (till the DHCP client).

The DHCP client remains in "searching"... until I change all VLAN id references to either 100 (my internal LAN - running a Microsoft Server 2K8R2 DHCP server) or 255 (my ISP - running his own DHCP server).

I also included two Wiresharks (taken via a hub in between port 2 and my core switch).

DHCPDiscoverLAN50.pcap is the one using VLAN id 50 which doesn't get an address.
DHCPDiscoverLAN100.pcap is the one using VLAN id 100 which does get an address.

The only important difference I see so far is the "Bootp flags" which is 0x8000 (Broadcast) when it doesn't work, and 0x000 (Unicast) when it does work ???

Why do I see other bootp flags when the DHCP client is on either VLAN id 50 or 100 ???

r2504 · Mon Sep 29, 2014 10:46 pm

Digging a bit deeper on this issue, I found the thread http://forum.mikrotik.com/viewtopic.php?f=2&t=49224 which states that the Mikrotik DHCP client starts with 0x8000 and switches after 15 seconds to 0x0000 which seems to be correct based on a new Wireshark I just ran on the CRS.

Question remains... what is different when the CRS VLAN is/is not equal to the one behind the access port of my core switch ???

r2504 · Wed Oct 01, 2014 12:28 am

After some tests and running Wiresharks, I found out that my core switch is "eating up" the DHCP request from the CRS !

This is however only happening if the Original DHCP request was tagged with a VLAN id different from the one internally on my core switch.

CRS (VLAN id 50) - no switch but direct connection to server ... DHCP server sees packet.
CRS (VLAN id 50) - Core Switch (VLAN id 100) ... on access port to DHCP server is nothing to see !!!!
CRS (VLAN id 100) - Core Switch (VLAN id 100) ... on access port to DHCP server I can see the packet.
PC (untagged) - Core Switch (VLAN id 100)... on access port to DHCP server I can see the packet.

Note: Packets on the wire never have VLAN tagging, all ports used are access ports.

So what is causing this since the packets entering my core switch (which is a TP-Link 3216) are untagged and hence have no trace at all of the Original VLAN id ???

Looking further at the wireshark in more detail, the only difference I see is the UDP checksum which is different. Could it be that since the core switch is re-adding a VLAN id internally, that the checksum only becomes valid if the VLAN id matches the original VLAN id in the CRS before it was stripped when leaving it ? Would it be so that my core switch is "eating up" packets which don't have a valid UDP checksum ?

If that's correct... who's fault is it ? Is it the CRS calculating a bad UDP checksum when the packet leaves port 2... or is it my core switch doing bad UDP checksum validations when the packet is handled by it ???

r2504 · Wed Oct 01, 2014 12:41 am

Theory (partially) busted... I enabled UDP checksum validation in wireshark and packets leaving the CRS are always valid.

This brings me again to my initial question... how does my core switch know which VLAN id the untagged packet originally had ?

And why is it dropping those where it doesn't match his internal VLAN id ?

r2504 · Thu Oct 02, 2014 8:17 pm

After having reported this issue to TP-Link (my core swicth) and doing some investigation it finally turned out that my laptop wasn't capturing VLAN information.

Once I edited the registry to make this information visible to Wireshark... I found out that the CRS is NOT untagging my packets ???

This means that the configuration at the start of this topic isn't doing what I expected...

# Switch VLAN tagging
/interface ethernet switch egress-vlan-tag
add vlan-id 50 tagged-ports=switch1-cpu

Shouldn't this part of the ROS cause packets leaving ether2 to become untagged ???

Could someone please explain the relation between the switch menu's VLAN, VLAN Tagging and In. VLAN Tran. ?

Fri Oct 03, 2014 9:44 am

Code: Select all
# Switch VLAN tagging
/interface ethernet switch egress-vlan-tag
add vlan-id 50 tagged-ports=switch1-cpu
Shouldn't this part of the ROS cause packets leaving ether2 to become untagged ???

"egress-vlan-tag" entry alone does not do anything. It works together with Port, Protocol or MAC based VLAN assingments. In Port based VLAN setup it is with Ingress VLAN translation rules. So to make packets leaving ether2 to become untagged you also need:

/interface ethernet switch ingress-vlan-translation
add customer-vid=0 new-customer-vid=50 ports=ether2 sa-learning=yes

which is already in your configuration...

Could someone please explain the relation between the switch menu's VLAN, VLAN Tagging and In. VLAN Tran. ?

VLAN menu defines allowed VLANs on ports, everything not defined is filtered when "unknown-forwarding" is disabled. It does not do anything with tagging/untagging.
Egress VLAN Tag menu sets VLAN trunk ports.
Ingress VLAN Translation menu modifies VLAN tag of packets, but the actual changes of the VLAN tags occur in egress, so that is why both "Egress VLAN Tag" and "Ingress VLAN Translation" are required to make connection.

I think I have found the cause of this problem in "test1.rsc" configuration. You have set two "master-ports", but it restricts VLAN functionality on CRS.
From wiki:
Note: Multiple master-port configuration is designed as fast and simple port isolation solution, but it limits part of VLAN functionality supported by CRS switch-chip. For advanced configurations use one master-port within CRS switch chip for all ports, configure VLANs and isolate port groups with port isolation profile configuration.

In your case, to simplify overall configuration, I suggest you to change ether2 port to have ether1 as master-port and set VLAN50 interface on ether1. If you want to use sfp1 port too, add to swtiched port group where ether1 is a master-port. Basically, it all will work with one master-port.

r2504 · Sat Oct 04, 2014 1:11 am

The configuration I used to test was indeed limited to a single port to make testing easier, however I didn't realise the consequences of it. I started however initially with all ports having sfp1 as master port so I tried that configuration again... but it doesn't make any difference

Given the attached configuration... the DHCP client keeps sending TAGGED packets out of ether2 !

While I guess I understand how VLAN's are working on a CRS... none of the examples found so far have either an IP-address linked to it, nor a DHCP client or server (actually they don't do more than what you would do with a normal managed switch). In that context it would be fine to explain a bit more on when it is needed to specify the master and/or switch1-cpu port to the VLAN's.

But first of all... it would be nice if packets would leave ether2 UNTAGGED

nstorm · Tue Oct 07, 2014 3:44 pm

I think I have found the cause of this problem in "test1.rsc" configuration. You have set two "master-ports", but it restricts VLAN functionality on CRS.
From wiki:
Note: Multiple master-port configuration is designed as fast and simple port isolation solution, but it limits part of VLAN functionality supported by CRS switch-chip. For advanced configurations use one master-port within CRS switch chip for all ports, configure VLANs and isolate port groups with port isolation profile configuration.

In your case, to simplify overall configuration, I suggest you to change ether2 port to have ether1 as master-port and set VLAN50 interface on ether1. If you want to use sfp1 port too, add to swtiched port group where ether1 is a master-port. Basically, it all will work with one master-port.

Does that means that if you have more than 2 master ports you can't use switch VLAN function, even if you use them within the same port group?
For example I have this config and I doesn't works as intended. I would like to have ether1-2 as a switch trunk ports (bonded) and also trunk ports (passing tagged VLANs), with ether9-16 on VLAN 91 PVID and ether17-24 VLAN 92 PVID. While I have ether3-4 added as another port group with ether3 master, I simply cannot ping vlan91 interface IP while connected to port 9.
Besides, can I have master port (ether1 in this example) as a member of switch trunk (bonding) group? Or is better to set master to a "stand-alone" port?

nstorm · Wed Oct 08, 2014 3:58 pm

Ok, I've tried setting every port to a ether3 as a master, and temporary removing trunk from switch configuration. It doesn't works right for me, right now I can ping IP on VLAN91 interface from any port, cept for those I've specially allocated as vlan91 access ports ;P
But I guess that is because I've created a vlan interface (/interface vlan) and attached it to the same bridge. The whole thing with p different switch chip and RB configuration is just very confusing. Some manual should be added on how they should interoperate with each other.
Basically can someone help me with sample config, because I'm already having a headache from trying to understand the logic behind all this.
I got CRS125 device, and I need ports 1-2 set up as a hardware static bond (trunk in terms of /interface ethernet switch) and a trunk port with tagged VLANs 91 and 92, and half of the remaining ports set as untagged access ports for these VLANs respectively. And I need CRS125 to have IPs on both VLANs for configuration access, no need for a routing or similar, just the ssh/web access from both VLANs.

r2504 · Wed Oct 08, 2014 4:07 pm

I guess I found the issue... using ether1 as mast port for all other ports seems to have solved the issue (but I still need some extensive testing). When I use sfp1 as master port for all other ports VLAN's were behaving strange (access port still contained VLAN tags).

nstorm · Thu Oct 09, 2014 12:48 pm

For example I have this config and I doesn't works as intended. I would like to have ether1-2 as a switch trunk ports (bonded) and also trunk ports (passing tagged VLANs), with ether9-16 on VLAN 91 PVID and ether17-24 VLAN 92 PVID. While I have ether3-4 added as another port group with ether3 master, I simply cannot ping vlan91 interface IP while connected to port 9.
Besides, can I have master port (ether1 in this example) as a member of switch trunk (bonding) group? Or is better to set master to a "stand-alone" port?

Ok, I've managed to get that working myself. I've just set a master port to ether3 (outside of trunk group), not sure if this is required though. Every port is a slave of ether3. I've set up VLANs via CRS switch ASIC (confusing part that we have 2 means of managing VLANs via switch chip and RB in software) and added a vlan interfaces on RB, bound to ether3 port. Then created an IP on vlan interface.
Most of the issues I was facing in previous config was from having an RB bridge with both switch port and vlan interface. It wasn't obvious to me that the RB bridge will transparently tag/untag and bridge VLANs together even if I had them setup in a different way via switch ASIC.
I've disabled the bridge completely and that helped. I needed bridge only for (R)STP feature... seems like I can do that if I add the bridge with only ether3 port attached it won't leak VLANs. But it will be a single instance of STP for both VLANs unfortunately. I know work for SPB are in progress...

CRS VLAN mystery

CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery

Re: CRS VLAN mystery