VLANs on switch chip

mkx · Tue Apr 28, 2020 10:20 pm

On a RB951G I have VLANs configured "the old school" way - on the switch chip. In ROS v6 this setup works great and is fully HW-offloaded.

I can't get the same/similar setup work on v7 at all. Initially I tried to upgrade from 6.45.7, but after reboot the device did not come back to the network. Mind that it was not in reboot loop, it emitted the normal sequence of beeps and then it sit still.

A note: the firewall part of config is flawed right now because I choked on the problem explained below. But IMHO it shouldn't affect behaviour in the way unit misbehaves right now.

Then I netinstalled the device and started to set things up from default config. Here's the complete export of config in a state where it still works:

# apr/28/2020 19:10:24 by RouterOS 7.0beta5
# software id = Y9I0-I2S0
#
# model = 951G-2HnD
# serial number = 5D610532E411
/interface bridge
add admin-mac=E6:8D:8C:C6:11:C4 auto-mac=no name=bridge protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] name=ether1-lan
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX disabled=no distance=indoors frequency=auto installation=indoor mode=ap-bridge ssid=MikroTik-C611C9 wireless-protocol=802.11
/interface vlan
add interface=bridge name=vlan-2 vlan-id=2
add interface=bridge name=vlan-40 vlan-id=40
add interface=bridge name=vlan-41 vlan-id=41
add interface=bridge name=vlan-42 vlan-id=42
/interface ethernet switch port
set 0 vlan-mode=fallback
set 1 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set 3 default-vlan-id=41 vlan-header=always-strip vlan-mode=secure
set 4 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set 5 vlan-header=add-if-missing vlan-mode=fallback
/interface list
add comment="home network" name=LAN
add comment="guests (over WiFi)" name=guest
add comment="IPTV set top box internet" name=IPTV
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge name=defconf
/interface bridge port
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether4
add bridge=bridge interface=ether5
add bridge=bridge interface=wlan1
add bridge=bridge interface=ether1-lan
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface ethernet switch vlan
add independent-learning=yes ports=switch1-cpu,ether1-lan,ether2,ether5 switch=switch1 vlan-id=42
add independent-learning=yes ports=switch1-cpu,ether4,ether1-lan switch=switch1 vlan-id=41
/interface list member
add comment=defconf interface=bridge list=LAN
add interface=vlan-42 list=LAN
add interface=vlan-41 list=LAN
add interface=ether1-lan list=LAN
add interface=ether5 list=LAN
add interface=ether4 list=LAN
/ip address
add address=192.168.42.6/23 interface=vlan-42 network=192.168.42.0
add address=192.168.41.2/24 interface=vlan-41 network=192.168.41.0
add address=192.168.40.2/24 interface=vlan-40 network=192.168.40.0
/ip cloud
set update-time=no
/ip dhcp-client
add comment=defconf interface=ether1-lan
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns
set servers=192.168.42.11
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new \
    in-interface-list=*2000010
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=*2000010
/ip route
add dst-address=0.0.0.0/0 gateway=192.168.42.1
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
add address=::224.0.0.0/100 comment="defconf: other" list=bad_ipv6
add address=::127.0.0.0/104 comment="defconf: other" list=bad_ipv6
add address=::/104 comment="defconf: other" list=bad_ipv6
add address=::255.0.0.0/104 comment="defconf: other" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
#error exporting /ipv6/route/rule
#error exporting /routing/policy/selection
/system clock
set time-zone-autodetect=no time-zone-name=UTC
/system ntp client
set enabled=yes
/system ntp client servers
add address=192.168.42.10
add address=192.168.42.11
/system package update
set channel=development
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

I've included full config (mostly not to hide something). However, what bothers me is the following part:

If it's configured like this, I can access management interface via ether5:

/interface ethernet switch port
set 0 vlan-mode=fallback
set 1 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set 3 default-vlan-id=41 vlan-header=always-strip vlan-mode=secure
set 4 default-vlan-id=42 vlan-header=always-strip vlan-mode=secure
set 5 vlan-header=add-if-missing vlan-mode=fallback
/interface ethernet switch vlan
add independent-learning=yes ports=switch1-cpu,ether1-lan,ether2,ether5 switch=switch1 vlan-id=42
add independent-learning=yes ports=switch1-cpu,ether4,ether1-lan switch=switch1 vlan-id=41

If I change setting of switch1-cpu interface in /interface ethernet switch port to

/interface ethernet switch port
set 5 vlan-header=add-if-missing vlan-mode=check # or secure

then I loose access to management interface (sitting on vlan-42 interface). And I loose both IP and MAC access, so the problem is clearly in L2 setup.

I guess related problem is that if I connect ether1-lan (which should be trunk port with all frames tagged) to a trunk port on a switch (same port works just fine if same RB951 is running ROS v6), I also don't have any connectivity to this device regardless the setting for switch1-cpu interface and I wonder if ROS v7 configures AR8327 correctly?

What am I doing wrong?

huntah · Wed Apr 29, 2020 8:49 pm

Code:
/interface ethernet switch port
set 5 vlan-header=add-if-missing vlan-mode=check # or secure

This is switch1-cpu .. I always set this like this:

/interface ethernet switch port
set 5 vlan-header=leave-as-is vlan-mode=secure

Try it.. if it works like this..
Or dou you think this not correct for switch-cpu "port"..

mkx · Wed Apr 29, 2020 10:59 pm

@huntah, thanks for reply.

Previous to posting my OP I also tried with vlan-header=leave-as-is ... worked just the same as with my posted setting. However if I tried to set vlan-mode=secure or vlan-mode=check, I lost access to device's CPU.

From official manual:

fallback - checks tagged traffic against the VLAN Table for ingress traffic, forwards all untagged traffic. If ingress traffic is tagged and egress port is not found in the VLAN table for the appropriate VLAN ID, then traffic is dropped. If a VLAN ID is not found in the VLAN Table, then traffic is forwarded. Used to allow known VLANs only in specific ports.
check - checks tagged traffic against the VLAN Table for ingress traffic, drops all untagged traffic. If ingress traffic is tagged and egress port is not found in the VLAN table for the appropriate VLAN ID, then traffic is dropped.
secure - checks tagged traffic against the VLAN Table for ingress traffic, drops all untagged traffic. Both ingress and egress port must be found in the VLAN Table for the appropriate VLAN ID, otherwise traffic is dropped.

I marked the differences between fallback and check as well as between check and secure. Below I'm concertrating on difference between fallback and check because I can't see if there's difference between check and secure.

Using torch on bridge (I guess that's the closest I could get at interface switch1-cpu) it seems that switch is doing its job tagging ingress packets on access port (ether5 in particular). So it seems it's not the first difference that is dropping the traffic. In theory torch shows traffic over interface in both directions, however on my device this is only true for software interfaces (bridge, vlan interfaces) but not for hardware interfaces (ether), for those it only shows Rx traffic.

So this leaves the second difference, where in fallback mode switch forwards packet "if a VLAN ID is not found in VLAN table" ... As if my configuration is missing something that "stores VLAN ID in VLAN table". As both in check and secure modes behaviour seems to be the same for egress packets, I assume it's the egress path which somehow is blocked. And for interface switch1-cpu egress probably means direction from switch chip towards bridge/CPU.

VLANs on switch chip

VLANs on switch chip

Re: VLANs on switch chip

Re: VLANs on switch chip