CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

joshuapl · Fri Mar 22, 2024 3:26 pm

Hi,

I'm fighting with configuration of my CRS354. I have a bridge set over all ports, some of them are set as access ports (defined both by PVID in /interface/bridge/port and by being set as "untagged" in /interface/bridge/vlan), some are set as trunk ports.

Bridge setup is simple:

0 R ;;; defconf
     name="bridge" mtu=auto actual-mtu=1500 l2mtu=1584 arp=enabled arp-timeout=auto mac-address=78:9A:18:5A:89:32 protocol-mode=rstp fast-forward=yes igmp-snooping=no auto-mac=yes ageing-time=5m
     priority=0x8000 max-message-age=20s forward-delay=15s transmit-hold-count=6 vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all ingress-filtering=yes dhcp-snooping=no

The vlan I'm having problem with has ID 111, it is set on ether7 and ether48 as untagged, and spf-spfplus1 as tagged:

/interface bridge port add bridge=bridge comment="WAN 2" frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=111
/interface bridge port add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether48 pvid=111
/interface bridge port add bridge=bridge comment=defconf interface=sfp-sfpplus1
/interface bridge vlan add bridge=bridge comment="WAN 2" tagged=sfp-sfpplus1 untagged=ether7,ether48 vlan-ids=111

These ports are meant to do the following:
- secondary WAN input from the Internet Provider on ether7
- WAN goes to the first router via ether48
- WAN goes to another switch (CRS326) via sfp-sfpplus1 where it is being used by another router

The problem is... it does not work.
And when I try sniffing on this port by /tool/sniffer/quick interface=ether7, it shows only 802.2 protocol frames.
However, once I disable that ether7 from bridge - it immediately shows real traffic there. Once enabled in port - returns to show only 802.2 frames.

It looks like this:

[admin@Router354-1-PPD3-S3-1] /interface/bridge> /tool/sniffer/quick interface=ether7
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, PROTOCOL, SIZE, CPU
INTERFACE  TIME    NUM  DIR  SRC-MAC            DST-MAC            PROTOCOL  SIZE  CPU

ether7     14.215    8  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00  802.2       64    0
ether7     16.215    9  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00  802.2       64    0
ether7     18.215   10  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00  802.2       64    0

(here I'm disabling port from bridge, by /interface/bridge/port disable numbers=6)

ether7     29.973   20  <-   AC:7A:56:89:CF:C0  FF:FF:FF:FF:FF:FF  xxx.yyy.183.153: who has xxx.yyy.183.156?               arp            60    0
ether7     30.215   21  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00                                                          802.2          64    0
ether7     30.783   22  <-   AC:7A:56:89:CF:C0  FF:FF:FF:FF:FF:FF  xxx.yyy.183.153: who has xxx.yyy.183.155?               arp            60    0
ether7     30.914   23  ->   78:9A:18:5A:89:44  33:33:00:00:00:16  fe80::7a9a:18ff:fe5a:8944                  ff02::16     ipv6:icmpv6   130    0
ether7     30.944   24  ->   78:9A:18:5A:89:44  33:33:00:00:00:16  fe80::7a9a:18ff:fe5a:8944                  ff02::16     ipv6:icmpv6    90    0
ether7     31.074   25  ->   78:9A:18:5A:89:44  33:33:00:00:00:16  fe80::7a9a:18ff:fe5a:8944                  ff02::16     ipv6:icmpv6    90    0
ether7     31.415   26  <-   98:3F:60:AF:29:70  01:00:0C:CC:CC:CD                                                          802.2          64    0
ether7     31.634   27  ->   78:9A:18:5A:89:44  33:33:00:00:00:16  fe80::7a9a:18ff:fe5a:8944                  ff02::16     ipv6:icmpv6   130    0
ether7     32.215   28  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00                                                          802.2          64    0
ether7     33.416   29  <-   98:3F:60:AF:29:70  01:00:0C:CC:CC:CD                                                          802.2          64    0
ether7     33.802   30  <-   AC:7A:56:89:CF:C0  FF:FF:FF:FF:FF:FF  xxx.yyy.183.153: who has xxx.yyy.183.155?               arp            60    0
ether7     34.215   31  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00                                                          802.2          64    0
ether7     35.425   32  <-   98:3F:60:AF:29:70  01:00:0C:CC:CC:CD                                                          802.2          64    0
ether7     35.516   33  <-   AC:7A:56:89:CF:C0  FF:FF:FF:FF:FF:FF  xxx.yyy.183.153: who has xxx.yyy.183.157?               arp            60    0

(and here I'm enabling port in bridge again)

ether7     36.215   34  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00                                                          802.2          64    0
ether7     36.89    35  <-   38:94:ED:CB:03:26  01:00:0C:CC:CC:CC                                                          802.2          93    0
ether7     37.517   36  <-   98:3F:60:AF:29:70  01:00:0C:CC:CC:CD                                                          802.2          64    0
ether7     37.522   37  ->   78:9A:18:5A:89:44  01:80:C2:00:00:00                                                          802.2          53    0
ether7     38.215   38  <-   CC:3E:5F:DF:87:80  01:80:C2:00:00:00                                                          802.2          64    0

Could anyone explain me what is going on? Why does bridge block all incoming traffic?
What am I doing wrong?

And it is not the problem with built-in sniffer, I have another Mikrotik device on port ether48 sniffing and there are just 802.2 frames. Also, I have checked that the network on port ether48 does not work - I have connected xxx.yyy.183.157 to this port and it is unreachable, but if I connect this device directly to the cable that goes into ether7 - instantly works.

Any ideas?

loloski · Fri Mar 22, 2024 3:45 pm

try this and adapt to your situation you missed where the bridge should be tagged as well

/interface/bridge/add pvid=4094 frame-types=admit-only-vlan-tagged name=bridge # Best practice don't set pvid=1
/interface/bridge/port add interface=ether7 frame-types=admit-only-untagged-and-priority-tagged pvid=111 bridge=bridge
/interface/bridge/port add interface=ether48 frame-types=admit-only-untagged-and-priority-tagged pvid=111 bridge=bridge
/interface/bridge/vlan/add vlan-ids=111 tagged=bridge,sfp-sfpplus1 untagged=ether7,ether48 bridge=bridge
/interface/bridge/set vlan-filtering=yes numbers=0

joshuapl · Fri Mar 22, 2024 4:00 pm

Thanks @loloski

I have changed PVID to 4094, but it didn't change anything. Still exactly the same behaviour.
But when I execute /interface/bridge/vlan/print, I get:

7 D bridge         1                  qsfpplus1-1
                                      sfp-sfpplus3
                                      sfp-sfpplus1
                                      sfp-sfpplus4

I don't know why it is still on vlan-id 1.

loloski · Fri Mar 22, 2024 4:03 pm

if qsfpplus1-1 is your trunk port

/interface/bridge/vlan/add vlan-ids=111 tagged=bridge,qsfpplus1-1 untagged=ether7,ether48 bridge=bridge

mkx · Fri Mar 22, 2024 4:08 pm

This setting

/interface/bridge/add pvid=4094 frame-types=admit-only-vlan-tagged name=bridge # Best practice don't set pvid=1

doesn't change a thing ... PVID setting is irrelevant when frame-types property is set to admit-only-vlan-tagged. In addition, it only applies to bridge CPU-facing port, not to any other bridge port.

My guess: for some weird reason, the config doesn't get properly applied to switch chip (we've seen that before). Did you properly cold boot the switch after you changed the settings? Sometimes that helps (making switch chip forget old config).

joshuapl · Fri Mar 22, 2024 4:14 pm

Yes, qsfpplus1-1 is a trunk port - it is connected to qsfpplus1-1 on another CRS354 and sends all vlans there for further connectivity.

I did not have a chance to coldboot this switch. I've asked someone to do this, but if this is required, it would be a really bad thing - putting in doubt my whole idea of going into mikrotiks

joshuapl · Fri Mar 22, 2024 4:28 pm

Just to be clear: coldboot did not help,either

anav · Fri Mar 22, 2024 4:54 pm

Detailed network diagram and current config, and i will have it fixed in a jiffy pop.
What always puts the icing on the cake, if you understand your own planning is also.
a. identifying all the users/devices
b. identifying all the traffic they need to accomplish.

joshuapl · Fri Mar 22, 2024 5:01 pm

My config is quite simple - 7 input networks, including 3 WANs, spread around this CRS354 and 4 other CRS354/326:

This is the config, just without recent changes (VLAN 4094 instead of 1 set on bridge)

# jan/26/1970 00:14:52 by RouterOS 7.8
# software id = DU6V-AYZV
#
# model = CRS354-48G-4S+2Q+
# serial number = HF209CSJ9S2
/interface bridge
add comment=defconf name=bridge vlan-filtering=yes
/interface vlan
add interface=bridge name=management vlan-id=251
add interface=bridge name=park vlan-id=102
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment="PARK in" interface=ether1 pvid=102
add bridge=bridge comment="ZSZ in" interface=ether2 pvid=109
add bridge=bridge comment="ZSZ-VIP in" interface=ether3 pvid=108
add bridge=bridge comment="Kosiba in" interface=ether4 pvid=106
add bridge=bridge comment="Bistro in" interface=ether5 pvid=119
add bridge=bridge comment="External in" interface=ether6 pvid=118
add bridge=bridge comment=defconf interface=ether7 pvid=111
add bridge=bridge comment=defconf interface=ether8 pvid=109
add bridge=bridge comment=defconf interface=ether9 pvid=109
add bridge=bridge comment=defconf interface=ether10 pvid=109
add bridge=bridge comment=defconf interface=ether11 pvid=109
add bridge=bridge comment=defconf interface=ether12 pvid=109
add bridge=bridge comment=defconf interface=ether13 pvid=109
add bridge=bridge comment=defconf interface=ether14 pvid=109
add bridge=bridge comment=defconf interface=ether15 pvid=109
add bridge=bridge comment=defconf interface=ether16 pvid=109
add bridge=bridge comment=defconf interface=ether17 pvid=109
add bridge=bridge comment=defconf interface=ether18 pvid=109
add bridge=bridge comment=defconf interface=ether19 pvid=109
add bridge=bridge comment=defconf interface=ether20 pvid=109
add bridge=bridge comment=defconf interface=ether21 pvid=109
add bridge=bridge comment=defconf interface=ether22 pvid=109
add bridge=bridge comment=defconf interface=ether23 pvid=109
add bridge=bridge comment=defconf interface=ether24 pvid=109
add bridge=bridge comment=defconf interface=ether25 pvid=109
add bridge=bridge comment=defconf interface=ether26 pvid=109
add bridge=bridge comment=defconf interface=ether27 pvid=109
add bridge=bridge comment=defconf interface=ether28 pvid=109
add bridge=bridge comment=defconf interface=ether29 pvid=109
add bridge=bridge comment=defconf interface=ether30 pvid=109
add bridge=bridge comment=defconf interface=ether31 pvid=109
add bridge=bridge comment=defconf interface=ether32 pvid=109
add bridge=bridge comment=defconf interface=ether33 pvid=109
add bridge=bridge comment=defconf interface=ether34 pvid=109
add bridge=bridge comment=defconf interface=ether35 pvid=109
add bridge=bridge comment=defconf interface=ether36 pvid=109
add bridge=bridge comment=defconf interface=ether37 pvid=109
add bridge=bridge comment=defconf interface=ether38 pvid=109
add bridge=bridge comment=defconf interface=ether39 pvid=109
add bridge=bridge comment=defconf interface=ether40 pvid=109
add bridge=bridge comment=defconf interface=ether41 pvid=109
add bridge=bridge comment=defconf interface=ether42 pvid=109
add bridge=bridge comment=defconf interface=ether43 pvid=109
add bridge=bridge comment=defconf interface=ether44 pvid=109
add bridge=bridge comment=defconf interface=ether45 pvid=109
add bridge=bridge comment=defconf interface=ether46 pvid=118
add bridge=bridge comment=defconf interface=ether47 pvid=119
add bridge=bridge comment=defconf interface=ether48 pvid=111
add bridge=bridge comment=defconf interface=ether49 pvid=251
add bridge=bridge comment=defconf interface=qsfpplus1-1
add bridge=bridge comment=defconf interface=qsfpplus1-2
add bridge=bridge comment=defconf interface=qsfpplus1-3
add bridge=bridge comment=defconf interface=qsfpplus1-4
add bridge=bridge comment=defconf interface=qsfpplus2-1
add bridge=bridge comment=defconf interface=qsfpplus2-2
add bridge=bridge comment=defconf interface=qsfpplus2-3
add bridge=bridge comment=defconf interface=qsfpplus2-4
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge comment=defconf interface=sfp-sfpplus2
add bridge=bridge comment=defconf interface=sfp-sfpplus3
add bridge=bridge comment=defconf interface=sfp-sfpplus4
/interface bridge vlan
add bridge=bridge comment=Management tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4 untagged=ether49 vlan-ids=251
add bridge=bridge comment="ZSZ INF" tagged=qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4 untagged=ether2 vlan-ids=109
add bridge=bridge comment=Park tagged=qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4,bridge untagged=ether1 vlan-ids=102
add bridge=bridge comment="ZSZ VIP" tagged=qsfpplus1-1,sfp-sfpplus1 untagged=ether3 vlan-ids=108
add bridge=bridge comment=Kosiba tagged=qsfpplus1-1,sfp-sfpplus4,sfp-sfpplus3 untagged=ether4 vlan-ids=106
add bridge=bridge comment=Bistro tagged=sfp-sfpplus3 untagged=ether5 vlan-ids=119
add bridge=bridge comment=External tagged=qsfpplus1-1 vlan-ids=118
add bridge=bridge comment="WAN ZSZ" tagged=sfp-sfpplus1 vlan-ids=111
add bridge=bridge comment=ext tagged=sfp-sfpplus2 vlan-ids=18
/interface list member
add interface=ether49 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=ether25 list=LAN
add interface=ether26 list=LAN
add interface=ether27 list=LAN
add interface=ether28 list=LAN
add interface=ether29 list=LAN
add interface=ether30 list=LAN
add interface=ether31 list=LAN
add interface=ether32 list=LAN
add interface=ether33 list=LAN
add interface=ether34 list=LAN
add interface=ether35 list=LAN
add interface=ether36 list=LAN
add interface=ether37 list=LAN
add interface=ether38 list=LAN
add interface=ether39 list=LAN
add interface=ether40 list=LAN
add interface=ether41 list=LAN
add interface=ether42 list=LAN
add interface=ether43 list=LAN
add interface=ether44 list=LAN
add interface=ether45 list=LAN
add interface=ether46 list=LAN
add interface=ether47 list=LAN
add interface=ether48 list=LAN
add interface=qsfpplus1-1 list=LAN
add interface=qsfpplus1-2 list=LAN
add interface=qsfpplus1-3 list=LAN
add interface=qsfpplus1-4 list=LAN
add interface=qsfpplus2-1 list=LAN
add interface=qsfpplus2-2 list=LAN
add interface=qsfpplus2-3 list=LAN
add interface=qsfpplus2-4 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
/ip address
add address=192.168.251.1/24 comment=defconf interface=management network=192.168.251.0
add address=10.0.2.11/24 interface=park network=10.0.2.0
/ip firewall filter
add action=drop chain=input disabled=yes in-interface=*3F port=67 protocol=udp
/system identity
set name=Router354-1-PPD3-S3-1
/system routerboard settings
set boot-os=router-os enter-setup-on=delete-key

anav · Fri Mar 22, 2024 10:23 pm

None of your vlans are identified, so how can 354 be the router ??? etc.....
So my request for network diagram, but ignored, does not let me progress any further.
you have wan and lan list but dont see any IP DHCP client or route etc................

joshuapl · Sat Mar 23, 2024 1:35 am

None of your vlans are identified, so how can 354 be the router ??? etc.....
So my request for network diagram, but ignored, does not let me progress any further.
you have wan and lan list but dont see any IP DHCP client or route etc................

Sorry, maybe I was not precise enough.
I'm searching since two hours for anything that might be good to draw some reasonable diagram, once I get it I will prepare the network diagram.

Generally my 354 is going to be handling WAN and multiple LANs, but it is not a router by itself.

1. One WAN goes directly into local router (RB1100AHx4) and it handles DHCP and provides internet with SNAT to networks 10.0.2.0/24, 10.0.9.0/24, 10.0.8.0/24, 10.0.6.0/24, 10.0.18.0/24
2. Another WAN goes directly into local router and is provided to network 10.0.19.0/24
3. Third WAN goes into port ether7 (VLAN 111), then it is sent to local server with public address through port ether48 and also sent via tagged LAN 111 through SPF1 to remote switch (CRS326) where it is sent via access port to another few servers with public addressess.
4. Local router receives LANs from RB1100 into following ports:
- ether1 (VLAN 102, 10.0.2.0/24)
- ether2 (VLAN 109, 10.0.9.0/24)
- ether3 (VLAN 108, 10.0.8.0/24)
- ether4 (VLAN 106, 10.0.6.0/24)
- ether5 (VLAN 119, 10.0.19.0/24)
- ether6 (VLAN 118, 10.0.18.0/24)
5. Ports ether6 through ether45 are going into patchpanel and to end users in VLAN 109
6. Network 10.0.18.0/24 is available on Access port ether46 for monitoring reasons
7. Network 10.0.19.0/24 is available on Access port ether47 for monitoring reasons
8. Port 48 is connected directly to local server.

This is the core, other things are probably less important:
9. Some VLANs are sent via SFP1..4 to other CRS354/326 which propagate them to their ethernet access ports
10. All VLANs are provided via DAC by QSFP to another CRS354 in the same rack mount, so it can spread them through ehter1..48 to another patchpanels and via SFP1..4 to other Mikrotik switches, just like in (9)

I hope this is now clear enough.

The problem is, that on ports ether46, ether47, ether48 I'm not seeing any traffic that goes into corresponding ports ether6, ether5, ether7.

What is more confusing - connectivity between ether2 and ether8..ether45 works perfectly fine. Even when I change cables and connect 10.0.18.0/24 network to ether2, I instantly can access it through ether8..ether45. Same with 10.0.19.0/24, HOWEVER it does not help for WAN that originally goes into ehter7.

Any ideas what I have done wrong?

Just to be sure everything is up to date, here is current config:

/interface bridge
add comment=defconf frame-types=admit-only-vlan-tagged name=bridge pvid=4094 vlan-filtering=yes
/interface vlan
add interface=bridge name=management vlan-id=251
add interface=bridge name=park vlan-id=102
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment="PARK in" interface=ether1 pvid=102
add bridge=bridge comment="ZSZ in" frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=109
add bridge=bridge comment="ZSZ-VIP in" interface=ether3 pvid=108
add bridge=bridge comment="Kosiba in" interface=ether4 pvid=106
add bridge=bridge comment="Bistro in" interface=ether5 pvid=119
add bridge=bridge comment="External in" frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=118
add bridge=bridge comment="WAN 2" frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=111
add bridge=bridge comment=defconf interface=ether8 pvid=109
add bridge=bridge comment=defconf interface=ether9 pvid=109
add bridge=bridge comment=defconf interface=ether10 pvid=109
add bridge=bridge comment=defconf interface=ether11 pvid=109
add bridge=bridge comment=defconf interface=ether12 pvid=109
add bridge=bridge comment=defconf interface=ether13 pvid=109
add bridge=bridge comment=defconf interface=ether14 pvid=109
add bridge=bridge comment=defconf interface=ether15 pvid=109
add bridge=bridge comment=defconf interface=ether16 pvid=109
add bridge=bridge comment=defconf interface=ether17 pvid=109
add bridge=bridge comment=defconf interface=ether18 pvid=109
add bridge=bridge comment=defconf interface=ether19 pvid=109
add bridge=bridge comment=defconf interface=ether20 pvid=109
add bridge=bridge comment=defconf interface=ether21 pvid=109
add bridge=bridge comment=defconf interface=ether22 pvid=109
add bridge=bridge comment=defconf interface=ether23 pvid=109
add bridge=bridge comment=defconf interface=ether24 pvid=109
add bridge=bridge comment=defconf interface=ether25 pvid=109
add bridge=bridge comment=defconf interface=ether26 pvid=109
add bridge=bridge comment=defconf interface=ether27 pvid=109
add bridge=bridge comment=defconf interface=ether28 pvid=109
add bridge=bridge comment=defconf interface=ether29 pvid=109
add bridge=bridge comment=defconf interface=ether30 pvid=109
add bridge=bridge comment=defconf interface=ether31 pvid=109
add bridge=bridge comment=defconf interface=ether32 pvid=109
add bridge=bridge comment=defconf interface=ether33 pvid=109
add bridge=bridge comment=defconf interface=ether34 pvid=109
add bridge=bridge comment=defconf interface=ether35 pvid=109
add bridge=bridge comment=defconf interface=ether36 pvid=109
add bridge=bridge comment=defconf interface=ether37 pvid=109
add bridge=bridge comment=defconf interface=ether38 pvid=109
add bridge=bridge comment=defconf interface=ether39 pvid=109
add bridge=bridge comment=defconf interface=ether40 pvid=109
add bridge=bridge comment=defconf interface=ether41 pvid=109
add bridge=bridge comment=defconf interface=ether42 pvid=109
add bridge=bridge comment=defconf interface=ether43 pvid=109
add bridge=bridge comment=defconf interface=ether44 pvid=109
add bridge=bridge comment=defconf interface=ether45 pvid=109
add bridge=bridge comment=defconf interface=ether46 pvid=118
add bridge=bridge comment=defconf interface=ether47 pvid=119
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether48 pvid=111
add bridge=bridge comment=defconf interface=ether49 pvid=251
add bridge=bridge comment=defconf interface=qsfpplus1-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-4 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-4 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus2 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus3 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus4 pvid=4094
/interface bridge vlan
add bridge=bridge comment=Management tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4 untagged=ether49 vlan-ids=251
add bridge=bridge comment="ZSZ INF" tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged=ether2 vlan-ids=109
add bridge=bridge comment=Park tagged=bridge,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged=ether1 vlan-ids=102
add bridge=bridge comment="ZSZ VIP" tagged=qsfpplus1-1,sfp-sfpplus1 untagged=ether3 vlan-ids=108
add bridge=bridge comment=Kosiba tagged=qsfpplus1-1,sfp-sfpplus4,sfp-sfpplus3 untagged=ether4 vlan-ids=106
add bridge=bridge comment=Bistro tagged=sfp-sfpplus3 untagged=ether5 vlan-ids=119
add bridge=bridge comment=External tagged=sfp-sfpplus2 vlan-ids=118
add bridge=bridge comment="WAN ZSZ" tagged=sfp-sfpplus1 untagged=ether7,ether48 vlan-ids=111
/interface list member
add interface=ether49 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=ether25 list=LAN
add interface=ether26 list=LAN
add interface=ether27 list=LAN
add interface=ether28 list=LAN
add interface=ether29 list=LAN
add interface=ether30 list=LAN
add interface=ether31 list=LAN
add interface=ether32 list=LAN
add interface=ether33 list=LAN
add interface=ether34 list=LAN
add interface=ether35 list=LAN
add interface=ether36 list=LAN
add interface=ether37 list=LAN
add interface=ether38 list=LAN
add interface=ether39 list=LAN
add interface=ether40 list=LAN
add interface=ether41 list=LAN
add interface=ether42 list=LAN
add interface=ether43 list=LAN
add interface=ether44 list=LAN
add interface=ether45 list=LAN
add interface=ether46 list=LAN
add interface=ether47 list=LAN
add interface=ether48 list=LAN
add interface=qsfpplus1-1 list=LAN
add interface=qsfpplus1-2 list=LAN
add interface=qsfpplus1-3 list=LAN
add interface=qsfpplus1-4 list=LAN
add interface=qsfpplus2-1 list=LAN
add interface=qsfpplus2-2 list=LAN
add interface=qsfpplus2-3 list=LAN
add interface=qsfpplus2-4 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
/ip address
add address=192.168.251.1/24 comment=defconf interface=management network=192.168.251.0
add address=10.0.2.11/24 interface=park network=10.0.2.0
/ip firewall filter
add action=drop chain=input disabled=yes in-interface=*3F port=67 protocol=udp
/system identity
set name=Router354-1-PPD3-S3-1
/system routerboard settings
set boot-os=router-os enter-setup-on=delete-key

loloski · Sat Mar 23, 2024 3:18 am

-- The problem is, that on ports ether46, ether47, ether48 I'm not seeing any traffic that goes into corresponding ports ether6, ether5, ether7.

/interface/bridge/port
add bridge=bridge comment="Bistro in" frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=119
add bridge=bridge comment="External in" frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=118
add bridge=bridge comment="WAN 2" frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=111

add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether47 pvid=119
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether46 pvid=118
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether48 pvid=111

/interface/bridge/vlan
add bridge=bridge comment=Bistro tagged=bridge,sfp-sfpplus3 untagged=ether5,ether47 vlan-ids=119
add bridge=bridge comment=External tagged=bridge,sfp-sfpplus2 untagged=ether6,ether46 vlan-ids=118
add bridge=bridge comment="WAN ZSZ" tagged=bridge,sfp-sfpplus1 untagged=ether7,ether48 vlan-ids=111

joshuapl · Sat Mar 23, 2024 9:30 am

-- The problem is, that on ports ether46, ether47, ether48 I'm not seeing any traffic that goes into corresponding ports ether6, ether5, ether7.
Code: Select all
/interface/bridge/port
add bridge=bridge comment="Bistro in" frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=119

/interface/bridge/vlan
add bridge=bridge comment=Bistro tagged=bridge,sfp-sfpplus3 untagged=ether5,ether47 vlan-ids=119

Do you suggest adding bridge as untagged to selected VLANs and setting frame-types=admit-only-untagged-and-priority-tagged? I did it, unfortunately it does not help at all.
Still I don't understand why VLAN 109 on ether2 and ether8..ether45 works just fine - all traffic goes through between interfaces just like it should.

loloski · Sat Mar 23, 2024 12:37 pm

Draw a basic network diagram including vlan assignment so that we can easily help you, I just interpret what you said

anav · Sat Mar 23, 2024 2:14 pm

I would caveat that with enough detail that shows where all the WANs are comming from and which vlans are going to which device over which ports!! ( to which devices )

joshuapl · Sat Mar 23, 2024 2:58 pm

I'm on it, almost did the detailed network diagram

I'll post it with updated config

joshuapl · Sat Mar 23, 2024 4:59 pm

I would caveat that with enough detail that shows where all the WANs are comming from and which vlans are going to which device over which ports!! ( to which devices )

OK, so here is the network diagram, hope I didn't miss anything:

And here is the most recent config:

/interface bridge
add comment=defconf frame-types=admit-only-vlan-tagged name=bridge pvid=4094 vlan-filtering=yes
/interface vlan
add interface=bridge name=management vlan-id=251
add interface=bridge name=park vlan-id=102
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment="PARK in" frame-types=admit-only-untagged-and-priority-tagged interface=ether1 pvid=102
add bridge=bridge comment="ZSZ in" frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=109
add bridge=bridge comment="ZSZ-VIP in" frame-types=admit-only-untagged-and-priority-tagged interface=ether3 pvid=108
add bridge=bridge comment="Kosiba in" frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=106
add bridge=bridge comment="Bistro in" frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=119
add bridge=bridge comment="External in" frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=118
add bridge=bridge comment="WAN 2" frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=111
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether9 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether10 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether11 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether12 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether13 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether14 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether15 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether16 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether17 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether18 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether19 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether20 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether21 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether22 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether23 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether24 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether25 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether26 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether27 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether28 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether29 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether30 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether31 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether32 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether33 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether34 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether35 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether36 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether37 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether38 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether39 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether40 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether41 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether42 pvid=102
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether43 pvid=106
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether44 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether45 pvid=108
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether46 pvid=118
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether47 pvid=119
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether48 pvid=111
add bridge=bridge comment=defconf interface=ether49 pvid=251
add bridge=bridge comment=defconf interface=qsfpplus1-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-4 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-4 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus2 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus3 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus4 pvid=4094
/interface bridge vlan
add bridge=bridge comment=Management tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4 untagged=ether49 vlan-ids=251
add bridge=bridge comment="ZSZ INF" tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged="ether2,ether8,ether9,ether10,ether11,ether12,ether13,ether14,ether15,ether16,ether17,ether18,ethe\
    r19,ether20,ether21,ether22,ether23,ether24,ether25,ether26,ether27,ether28,ether29,ether30,ether31,ether32,ether33,ether34,ether35,ether36,ether37,ether38,ether39,ether40,ether41" vlan-ids=109
add bridge=bridge comment=Park tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged=ether1,ether42 vlan-ids=102
add bridge=bridge comment="ZSZ VIP" tagged=qsfpplus1-1,sfp-sfpplus1 untagged=ether3,ether45 vlan-ids=108
add bridge=bridge comment=Kosiba tagged=qsfpplus1-1,sfp-sfpplus4,sfp-sfpplus3 untagged=ether4,ether43 vlan-ids=106
add bridge=bridge comment=Bistro tagged=bridge,sfp-sfpplus3 untagged=ether5,ether47 vlan-ids=119
add bridge=bridge comment=External tagged=bridge,sfp-sfpplus2 untagged=ether6,ether38,ether46 vlan-ids=118
add bridge=bridge comment="WAN ZSZ" tagged=bridge,sfp-sfpplus1 untagged=ether7,ether48 vlan-ids=111
/interface list member
add interface=ether49 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=ether25 list=LAN
add interface=ether26 list=LAN
add interface=ether27 list=LAN
add interface=ether28 list=LAN
add interface=ether29 list=LAN
add interface=ether30 list=LAN
add interface=ether31 list=LAN
add interface=ether32 list=LAN
add interface=ether33 list=LAN
add interface=ether34 list=LAN
add interface=ether35 list=LAN
add interface=ether36 list=LAN
add interface=ether37 list=LAN
add interface=ether38 list=LAN
add interface=ether39 list=LAN
add interface=ether40 list=LAN
add interface=ether41 list=LAN
add interface=ether42 list=LAN
add interface=ether43 list=LAN
add interface=ether44 list=LAN
add interface=ether45 list=LAN
add interface=ether46 list=LAN
add interface=ether47 list=LAN
add interface=ether48 list=LAN
add interface=qsfpplus1-1 list=LAN
add interface=qsfpplus1-2 list=LAN
add interface=qsfpplus1-3 list=LAN
add interface=qsfpplus1-4 list=LAN
add interface=qsfpplus2-1 list=LAN
add interface=qsfpplus2-2 list=LAN
add interface=qsfpplus2-3 list=LAN
add interface=qsfpplus2-4 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
/ip address
add address=192.168.251.1/24 comment=defconf interface=management network=192.168.251.0
add address=10.0.2.11/24 interface=park network=10.0.2.0
/ip firewall filter
add action=drop chain=input disabled=yes in-interface=*3F port=67 protocol=udp
/system identity
set name=Router354-1-PPD3-S3-1
/system routerboard settings
set boot-os=router-os enter-setup-on=delete-key

and what can I say from perspective of monitoring server:
- traffic on 10.0.2.0/24 (VLAN 102) is visible
- traffic on 10.0.6.0/24 (VLAN 106) is visible
- traffic on 10.0.8.0/24 (VLAN 108) is visible
- traffic on 10.0.9.0/24 (VLAN 109) is visible
but:
- traffic 10.0.18.0/24 on port eth46 (VLAN 118) is not visible
- traffic 10.0.19.0/24 on port eth47 (VLAN 118) is not visible
- WAN traffic on port ether48 (VLAN 111) is not visible

Stormshield and RB3011 will be replaced by single RB1100AHx4 in next step.

Really counting on you guys, I can't find any reasonable explanation for what is happening.

anav · Sat Mar 23, 2024 5:51 pm

Okay to recap before I look at the monster and by the way, how one would expect to grasp that in bits and pieces of posts etc is amusing..........

You have three WANS and two ROUTERs in the mix.
- The Stormshield router gets WAN2 and provides DHCP for the following vlans: 102,106,108,109,111
- The RB3011 router gets WAN1 and provides DHCP for the following vlans 118,119
- WAN3 is hosted/terminated by WEBSERVER1 on vlan111 ( which came from ISP3 via a connection on Switch 354 )
- WAN4 is hosted/terminated by WEBSERVER2 on vlan111 ( which came from ISP3 via Switch 354 and then switch 326-1

Hence, Switch 354 is NOT a router.
WHERE IS THE MANAGEMENT VLAN................... at least it should come from RB3011

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Changes:
1. add comment=defconf name=bridge vlan-filtering=yes the rest was noise..............
2. Removed vlan102, not sure why you defined it all??
3. added ethernet interface for off bridge access and easy clean separate location to install and modify config .....!!
4. Your management vlan make NO sense to me at all, where is its SOURCE of origination, its not coming from the only two expected sources, the routers!
In fact it would appear you start at the switch but guess what, its a subnet and you dont have that on the switch, nor should you!!
So.............. I will assume 251 was a vlan subnet you made on the 3011 and passed to the Switch.
5. Brings up another WEIRDNESS, why do you have to you have TWO ports connecting the RB3011 to the SWITCH. Possible but not usual but NOT recommended.
6. Why do you have 5 ports from Stormshield to Switch ???? Possible but not usual. Since I dont know that device how do you separate subnets by port ???
7. It would appear that the vlans from Stormshield are NOT vlans and are subnets using port isolation ??? as the incoming ports 1-4 are untagged as well as 5,6 are untagged.
I can buy that from the stormshield but NO CANNOT do for RB3011 as its perfectly capable of using vlans and besides the management vlan has to come over as well.

In other words, I need to see and modify the RB3011 before proceeding.

joshuapl · Sat Mar 23, 2024 7:32 pm

Okay to recap before I look at the monster and by the way, how one would expect to grasp that in bits and pieces of posts etc is amusing..........

The whole network is quite complex, I agree

But generally the main thing is that some vlans on SWITCH354-1 behave differently than other and I'm pretty sure it's my fault, even though I don't see vast differences.

First thing: this is an inherited network built over 10 years ago and vastly modified since then, and it consists of proper fiber infrastructure connecting 5 buildings (SWITCH354-1 & 354-2 are in the main server room, connected by Direct Attach Cable). Sometimes a single ethernet cable goes into room with 10-20 computers on unmanaged switch. Current infrastructure is built on old Netgear switches, and there are completely no tagged vlans anywhere. I am introducing changes, but not all at once - that's why in the first stage I am leaving previous routers (Stormshield and RB3011) and I'm not changing anything in their configuration. And this is the reason for many links between routers and SWITCH354-1. I decided to replace old Netgears and TPLinks (6-7 various models) with Mikrotiks in order to have homogenic infrastructure and possibility to replace any switch with one CRS354 that is kept in case of emergency.

Once I replace all the core switches, I will go to the stage 2, where I will get rid of old routers - everything will be handled by RB1100AHx4, and RB3011 will be configured identically so in case of hardware failure it would replace RB1100AHx4. Of course, multiple links between router and SWITCH354-1 will be then replaced by fewer uplinks with VLANs.

You have three WANS and two ROUTERs in the mix.
- The Stormshield router gets WAN2 and provides DHCP for the following vlans: 102,106,108,109,111
- The RB3011 router gets WAN1 and provides DHCP for the following vlans 118,119
- WAN3 is hosted/terminated by WEBSERVER1 on vlan111 ( which came from ISP3 via a connection on Switch 354 )
- WAN4 is hosted/terminated by WEBSERVER2 on vlan111 ( which came from ISP3 via Switch 354 and then switch 326-1

That's correct. And of course both WEBSERVERs are not VLAN aware, they are connected to access ports.

Hence, Switch 354 is NOT a router.

Exactly

WHERE IS THE MANAGEMENT VLAN................... at least it should come from RB3011

Sorry, didn't mark it. It was set up initially on the switches in order to be sure, that I can access all the switches by connecting to ether49 management port on any of the CRS354s. I plan to have it available through VPN connection through separate LTE router - in case of all ISPs failing.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Changes:
1. add comment=defconf name=bridge vlan-filtering=yes the rest was noise..............

OK. I added this while trying to fix the problem.

2. Removed vlan102, not sure why you defined it all??

This is the main technical infrastructure. It covers trusted endpoints in all the buildings, LAN connectivity to some servers, CCTV network, terminals in sentry rooms, electronic locks, building automation, etc. It is also by default used for network administration. Yes, it is a possible security breach, but - as I'm saying - this is inherited infrastructure and changing it is on the TODO list, but not the highest priority.

3. added ethernet interface for off bridge access and easy clean separate location to install and modify config .....!!

I don't understand this one - should I remove ether49 with 192.168.251.0/24 network from the bridge? This would block be from using emergency access to any of the switches and it's not something I want to happen (this is a remote site, I'm not available there every day).

4. Your management vlan make NO sense to me at all, where is its SOURCE of origination, its not coming from the only two expected sources, the routers!
In fact it would appear you start at the switch but guess what, its a subnet and you dont have that on the switch, nor should you!!

So.............. I will assume 251 was a vlan subnet you made on the 3011 and passed to the Switch.

Now it is not connected anywhere, but it will be connected to dedicated LTE router accessible via VPN.

5. Brings up another WEIRDNESS, why do you have to you have TWO ports connecting the RB3011 to the SWITCH. Possible but not usual but NOT recommended.
6. Why do you have 5 ports from Stormshield to Switch ???? Possible but not usual. Since I dont know that device how do you separate subnets by port ???

Because it is first stage of migration to new infrastructure. Till now, there are no tagged VLANS, untagged LANs are going through weird connections like 10.0.9.0/24 from main server room goes via fiber to building 2 and then ethernet to building 3 - even though there is fiber between main server room and building 3... because there were not enough SFP ports on switches in building 3, etc. I'm introducing single fiber lines from main switch to other buildings, and I don't want to make instant replacement of the routers as well.

7. It would appear that the vlans from Stormshield are NOT vlans and are subnets using port isolation ??? as the incoming ports 1-4 are untagged as well as 5,6 are untagged.

Exactly. And I don't want to mess with introducing VLANs on Stormshield - for me, removing total mess from LANs is the priority.

I can buy that from the stormshield but NO CANNOT do for RB3011 as its perfectly capable of using vlans and besides the management vlan has to come over as well.

In other words, I need to see and modify the RB3011 before proceeding.

Currently, VLANs 118 and 119 are going to different switches that aren't connected in any way. It would be possible to modify it with introduction of Mikrotik core switches, but I didn't want to do this at once - reverting to original configuration would be much harder if anything goes wrong, especially taking into regard that with mikrotik you can't just do export and then import configuration.

And, besides VLANs 118/119, the main problem is the WAN2 that fails to go through SWITCH354-1. I can theoretically leave part of old equipment and send this signal to second building by recently discovered LAN cable that goes there, but I would prefer to have all switches replaced.

anav · Sat Mar 23, 2024 8:03 pm

Why the RB1100, its old news.
The RB5009 is cheaper and very capable.
However the real question is what are the Throughputs of your Three ISP connections?
ISP1 up/down ISP2 up/down ISP3 up/down NOW and planned.

If all three are 1gig, then even the 5009 is getting pushed.
If you plan on any 2.5 gig connections in the future, then the right purchase is the 2116.
Three sfp+ ports for WANS, and one SFP+ port to the the 354................

joshuapl · Sat Mar 23, 2024 8:36 pm

Why the RB1100, its old news.
The RB5009 is cheaper and very capable.
However the real question is what are the Throughputs of your Three ISP connections?
ISP1 up/down ISP2 up/down ISP3 up/down NOW and planned.

RB1011 was purchased before5009 was available. Unfortunately, we were held for a looong time due to CRS354 unaivailability

Simple failover for ISP1&2 might be introduced, but failures are extremely rare and this infrastructure is far from being critical - it's for several schools. Same for ISP3.

The overall throughut of all ISP's barely exceeds 1Gbps.

Nevertheless, replacing a router isn't a huge cost, so in the future we can do this

joshuapl · Tue Mar 26, 2024 12:36 am

7. It would appear that the vlans from Stormshield are NOT vlans and are subnets using port isolation ??? as the incoming ports 1-4 are untagged as well as 5,6 are untagged.
I can buy that from the stormshield but NO CANNOT do for RB3011 as its perfectly capable of using vlans and besides the management vlan has to come over as well.

In other words, I need to see and modify the RB3011 before proceeding.

OK, I have fixed VLANS 118 & 119 now. Both come through ether5 as tagged VLANS.
However, this didn't help at all.

While sniffing on port ether5 (/tool/sniffer/packet/print detail follow with set filter-interface=ether5), all I get is:

 1 time=1.83 num=2 direction=tx src-mac=78:9A:18:5A:89:42 dst-mac=01:00:0C:CC:CC:CC interface=ether5 protocol=802.2 size=125 cpu=0
 2 time=1.83 num=3 direction=tx src-mac=78:9A:18:5A:89:42 dst-mac=01:80:C2:00:00:0E interface=ether5 protocol=lldp size=145 cpu=0
 3 time=1.923 num=4 direction=tx src-mac=78:9A:18:5A:89:42 dst-mac=01:80:C2:00:00:00 interface=ether5 protocol=802.2 size=53 cpu=0
 4 time=3.926 num=5 direction=tx src-mac=78:9A:18:5A:89:42 dst-mac=01:80:C2:00:00:00 interface=ether5 protocol=802.2 size=53 cpu=0
 5 time=5.926 num=6 direction=tx src-mac=78:9A:18:5A:89:42 dst-mac=01:80:C2:00:00:00 interface=ether5 protocol=802.2 size=53 cpu=0

When I disable port ether5 on bridge by

/interface/bridge/port disable numbers=4

the proper traffic appears immediately:

54 time=56.115 num=55 direction=rx src-mac=B8:69:F4:87:FB:C9 dst-mac=FF:FF:FF:FF:FF:FF vlan=119 interface=ether5 protocol=arp size=64 cpu=0
55 time=57.114 num=56 direction=rx src-mac=B8:69:F4:87:FB:C9 dst-mac=FF:FF:FF:FF:FF:FF vlan=119 interface=ether5 protocol=arp size=64 cpu=0
56 time=57.831 num=57 direction=rx src-mac=B8:69:F4:87:FB:C8 dst-mac=FF:FF:FF:FF:FF:FF vlan=118 interface=ether5 protocol=arp size=64 cpu=0
57 time=57.978 num=58 direction=rx src-mac=44:94:FC:8D:41:E2 dst-mac=01:80:C2:00:00:00 interface=ether5 protocol=802.2 size=60 cpu=0
58 time=58.114 num=59 direction=rx src-mac=B8:69:F4:87:FB:C9 dst-mac=FF:FF:FF:FF:FF:FF vlan=119 interface=ether5 protocol=arp size=64 cpu=0
59 time=58.359 num=60 direction=rx src-mac=28:C6:8E:2C:B0:C8 dst-mac=FF:FF:FF:FF:FF:FF vlan=119 interface=ether5 protocol=arp size=64 cpu=0
60 time=58.824 num=61 direction=rx src-mac=B8:69:F4:87:FB:C8 dst-mac=FF:FF:FF:FF:FF:FF vlan=118 interface=ether5 protocol=arp size=64 cpu=0

The only change from previous config is that for /interface/bridge/vlan port ether5 is set as tagged for both vlans. Here is the result of "vlan print":

[admin@Router354-1-PPD3-S3-1] /interface/bridge> vlan/print
Flags: D - DYNAMIC
Columns: BRIDGE, VLAN-IDS, CURRENT-TAGGED, CURRENT-UNTAGGED
#   BRIDGE  VLAN-IDS  CURRENT-TAGGED  CURRENT-UNTAGGED
;;; Management
0   bridge       251  bridge
                      sfp-sfpplus1
                      sfp-sfpplus3
                      sfp-sfpplus4
                      qsfpplus1-1
;;; ZSZ INF
1   bridge       109  sfp-sfpplus1    ether2
                      sfp-sfpplus3
                      sfp-sfpplus4
;;; Park
2   bridge       102  bridge          ether1
                      qsfpplus1-1
                      sfp-sfpplus1
                      sfp-sfpplus3
                      sfp-sfpplus4
;;; ZSZ VIP
3   bridge       108  qsfpplus1-1     ether3
                      sfp-sfpplus1    ether45
;;; Kosiba
4   bridge       106  qsfpplus1-1     ether4
                      sfp-sfpplus3
                      sfp-sfpplus4
;;; Bistro
5   bridge       119  ether5          ether47
                      sfp-sfpplus3
;;; External
6   bridge       118  ether5          ether38
;;; WAN ZSZ
7   bridge       111  bridge          ether7
                      sfp-sfpplus1    ether48
8 D bridge      4094                  sfp-sfpplus1
                                      sfp-sfpplus3
                                      sfp-sfpplus4
                                      qsfpplus1-1
                                      ether5
                                      ether6

It seems like once port is enabled in the bridge, only 802.2 (what the hell is it?) are seen on the interface. Why?

tdw · Tue Mar 26, 2024 4:39 am

It seems like once port is enabled in the bridge, only 802.2 (what the hell is it?) are seen on the interface. Why?

Spanning tree, and the port will be ending up in the blocking state to prevent a network loop. STP & RSTP are not VLAN-aware, they allow or block all traffic be it untagged or tagged.

If you have multiple connections between VLAN-aware bridges / switches you should create a bond of all the links and apply the VLANs to the bond instead of trying to configure some VLANs on one link and different VLANs on other links.

joshuapl · Tue Mar 26, 2024 12:03 pm

If you have multiple connections between VLAN-aware bridges / switches you should create a bond of all the links and apply the VLANs to the bond instead of trying to configure some VLANs on one link and different VLANs on other links.

I don't have multiple bridges (since only first one is hardware-supported), and no multiple links between my switches - if any networks are going between switch A and B, then everything goes through single SFP link with all VLANs tagged.

Nevertheless, I'm still struggling with the main switch - even traffic on this one is not working as it should be, and I can't find what I did wrong.
No matter what I do, network from port ether5 (no matter if it is tagged or untagged) does not go to port 47 as it should.

tdw · Tue Mar 26, 2024 1:18 pm

That doesn't agree with your diagram, it shows ether5 and ether6 connected between the CRS and RB3011

joshuapl · Tue Mar 26, 2024 1:23 pm

That doesn't agree with your diagram, it shows ether5 and ether6 connected between the CRS and RB3011

First - it was irrelevant, since these are two separate networks.
And, also, as I have written, I also eliminated this. Now ether6 is not connected, and both VLAN 118 and 119 enter as tagged VLAN to ether5. You can see it on sniffer dump. And it didn't help at all, traffic from VLANs 118 and 119 on ether5 are not propagated to access ports ether47 and ether46

joshuapl · Tue Mar 26, 2024 6:16 pm

It seems like once port is enabled in the bridge, only 802.2 (what the hell is it?) are seen on the interface. Why?
Spanning tree, and the port will be ending up in the blocking state to prevent a network loop. STP & RSTP are not VLAN-aware, they allow or block all traffic be it untagged or tagged.

One more question: shouldn't I be using protocol-mode=mstp on the bridge, since everything is done via vlans?

And I have some updates... although I'm not very happy, since looks like some totally weird things are happening.
One thing: I changed VLAN 118 to be untagged on ports 6 and 46, VLAN 119 untagged on ports 5 and 47 - to have more flexibility with connecting cables to various port for testing.

1. The unfiltered WAN - where there are two access ports (ether7, ether48) and one trunk port (sfp-sfpplus1): there is no communication, unless... I disconnect ehternet cable from port 1 (VLAN 102, network 10.0.2.0/24). I can do it also by disabling port ether1 on the bridge. WHY? There is absolutely nothing special about ether1 if I compare port definitions.

2. The VLAN 118 (10.0.18.0/24) - this is even weirder: ports ether2 and ether8..ether41 are defined identically:

add bridge=bridge comment="ZSZ in" frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether9 pvid=109
.. etc.

When on RB3011 (10.0.18.1) I start pinging 10.0.18.252 (a small mikrotik router with this address set up) then I have following symptoms:
- RB3011 connected to ether6 (VLAN 118 untagged), target mikrotik on ether46 (VLAN 118 untagged): timeout, on target mikrotik sniffer shows only 802.2 packets
- RB3011 connected to ether2 (VLAN 109 untagged), target mikrotik on ether8 (VLAN 109 untagged): ping works, on target mikrotik sniffer shows proper traffic

BUT...!
- RB3011 connected to ether8 (VLAN 109 untagged), target mikrotik on ether2 (VLAN 109 untagged): timeout, on target mikrotik sniffer shows only 802.2 packets

That's insane! Traffic goes only in one direction?

Just to be fully transparent: here is current config:

/interface bridge
add comment=defconf frame-types=admit-only-vlan-tagged name=bridge protocol-mode=mstp pvid=4094 vlan-filtering=yes
/interface vlan
add interface=bridge name=management vlan-id=251
add interface=bridge name=park vlan-id=102
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment="PARK in" frame-types=admit-only-untagged-and-priority-tagged interface=ether1 pvid=102
add bridge=bridge comment="ZSZ in" frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=109
add bridge=bridge comment="ZSZ-VIP in" frame-types=admit-only-untagged-and-priority-tagged interface=ether3 pvid=108
add bridge=bridge comment="Kosiba in" frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=106
add bridge=bridge comment="Bistro in" interface=ether5 pvid=119
add bridge=bridge comment="External in" interface=ether6 pvid=118
add bridge=bridge comment="WAN 2" frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=111
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether8 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether9 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether10 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether11 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether12 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether13 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether14 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether15 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether16 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether17 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether18 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether19 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether20 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether21 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether22 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether23 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether24 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether25 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether26 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether27 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether28 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether29 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether30 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether31 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether32 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether33 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether34 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether35 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether36 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether37 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether38 pvid=118
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether39 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether40 pvid=109
add bridge=bridge comment="ZSZ out" frame-types=admit-only-untagged-and-priority-tagged interface=ether41 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether42 pvid=102
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether43 pvid=106
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether44 pvid=109
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether45 pvid=108
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether46 pvid=118
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether47 pvid=119
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged interface=ether48 pvid=111
add bridge=bridge comment=defconf interface=ether49 pvid=251
add bridge=bridge comment=defconf interface=qsfpplus1-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus1-4 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-1 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-2 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-3 pvid=4094
add bridge=bridge comment=defconf interface=qsfpplus2-4 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus2 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus3 pvid=4094
add bridge=bridge comment=defconf interface=sfp-sfpplus4 pvid=4094
/interface bridge vlan
add bridge=bridge comment=Management tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus3,sfp-sfpplus4 untagged=ether49 vlan-ids=251
add bridge=bridge comment="ZSZ INF" tagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged="ether2,ether8,ether9,ether10,ether11,ether12,ether13,ether14,ether15,ether16,ether17,ether18,ethe\
    r19,ether20,ether21,ether22,ether23,ether24,ether25,ether26,ether27,ether28,ether29,ether30,ether31,ether32,ether33,ether34,ether35,ether36,ether37,ether39,ether40,ether41" vlan-ids=109
add bridge=bridge comment=Park tagged=bridge,qsfpplus1-1,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 untagged=ether1,ether42 vlan-ids=102
add bridge=bridge comment="ZSZ VIP" tagged=qsfpplus1-1,sfp-sfpplus1 untagged=ether3,ether45 vlan-ids=108
add bridge=bridge comment=Kosiba tagged=qsfpplus1-1,sfp-sfpplus4,sfp-sfpplus3 untagged=ether4,ether43 vlan-ids=106
add bridge=bridge comment=Bistro tagged=sfp-sfpplus3 untagged=ether5,ether47 vlan-ids=119
add bridge=bridge comment=External tagged=sfp-sfpplus2 untagged=ether6,ether46 vlan-ids=118
add bridge=bridge comment="WAN ZSZ" tagged=sfp-sfpplus1 untagged=ether7,ether48 vlan-ids=111
/interface list member
add interface=ether49 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=ether25 list=LAN
add interface=ether26 list=LAN
add interface=ether27 list=LAN
add interface=ether28 list=LAN
add interface=ether29 list=LAN
add interface=ether30 list=LAN
add interface=ether31 list=LAN
add interface=ether32 list=LAN
add interface=ether33 list=LAN
add interface=ether34 list=LAN
add interface=ether35 list=LAN
add interface=ether36 list=LAN
add interface=ether37 list=LAN
add interface=ether38 list=LAN
add interface=ether39 list=LAN
add interface=ether40 list=LAN
add interface=ether41 list=LAN
add interface=ether42 list=LAN
add interface=ether43 list=LAN
add interface=ether44 list=LAN
add interface=ether45 list=LAN
add interface=ether46 list=LAN
add interface=ether47 list=LAN
add interface=ether48 list=LAN
add interface=qsfpplus1-1 list=LAN
add interface=qsfpplus1-2 list=LAN
add interface=qsfpplus1-3 list=LAN
add interface=qsfpplus1-4 list=LAN
add interface=qsfpplus2-1 list=LAN
add interface=qsfpplus2-2 list=LAN
add interface=qsfpplus2-3 list=LAN
add interface=qsfpplus2-4 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
/ip address
add address=192.168.251.1/24 comment=defconf interface=management network=192.168.251.0
add address=10.0.2.11/24 interface=park network=10.0.2.0
/ip firewall filter
add action=drop chain=input disabled=yes in-interface=*3F port=67 protocol=udp
/system identity
set name=Router354-1-PPD3-S3-1
/system routerboard settings
set boot-os=router-os enter-setup-on=delete-key
/tool sniffer
set filter-interface=ether48

anav · Tue Mar 26, 2024 9:01 pm

Problem is I stopped looking at this thread awhile ago doing to the moving datum.
Once you get all the final equipment in place, then will be able to devote time and energy to a static target.

joshuapl · Wed Mar 27, 2024 6:18 am

Problem is I stopped looking at this thread awhile ago doing to the moving datum.
Once you get all the final equipment in place, then will be able to devote time and energy to a static target.

Well, I will have to solve it by myself then

The problem is within single switch, other hardware has noting to do with it. and I still don't know what I did wrong. Either there is some mikrotik-specific issue where it does something in some weird way, or the hardware is somehow damaged.

I'll start from the scratch and check step by step when the connectivity fails, hope I'll find out.

mkx · Wed Mar 27, 2024 9:21 am

I'll start from the scratch and check step by step when the connectivity fails, hope I'll find out.

That's something I was about to suggest you. Start by netinstalling the switch and try to progress at desired setup without taking turns.

There were cases where visible configuration of device (the one shown by export or print) did not correspond with how device behaved. The suspicion is that ROS has some internal configuration table which gets out-of-sync with visible configuration. The only way out of such state is to reset configuration (or, even better, netinstall) ...

joshuapl · Wed Mar 27, 2024 12:34 pm

I'll start from the scratch and check step by step when the connectivity fails, hope I'll find out.

That's something I was about to suggest you. Start by netinstalling the switch and try to progress at desired setup without taking turns.

There were cases where visible configuration of device (the one shown by export or print) did not correspond with how device behaved. The suspicion is that ROS has some internal configuration table which gets out-of-sync with visible configuration. The only way out of such state is to reset configuration (or, even better, netinstall) ...

I thought I would get away with just resetting configuration - but forgot, that new CRS354 come now with preset passwords printed on stickers... So now I have to drive to the site, and probably will try netinstall.

BTW, when I tried updating software it said 7.12.1 is the highest version possible. However, when I want to download netinstall there is 7.14.1 Stable available as default... Should I go with that or rather use 7.12.1?

joshuapl · Wed Mar 27, 2024 4:24 pm

It seems like once port is enabled in the bridge, only 802.2 (what the hell is it?) are seen on the interface. Why?
Spanning tree, and the port will be ending up in the blocking state to prevent a network loop. STP & RSTP are not VLAN-aware, they allow or block all traffic be it untagged or tagged.

Hey All,

it seems that finally my configuration was OK.
What was actually causing the problems, was something... completely else.

ISP1 and ISP2 were connected directly to the main switch, but ISP3 was connected through a switch since it was spread between several local servers.
And it just occured, that there was a rogue ethernet coming from the other server room in second building, that was connected to this switch AND local network (VLAN 102).
And once I connected VLAN 102 and WAN from ISP3 to the CRS354-1, STP on that ISP's switch came into play, blocking WAN3.

Thanks for pointing out that STP is not VLAN-aware. I didn't know that.

mkx · Wed Mar 27, 2024 6:22 pm

First of all, I'm glad you found the problem.

BTW, when I tried updating software it said 7.12.1 is the highest version possible. However, when I want to download netinstall there is 7.14.1 Stable available as default... Should I go with that or rather use 7.12.1?

7.13 came with breaking change (wireless package was split from nain package) and ROS upgrader got aware of it only in 7.12.1 (or was it 7.12?) Since package download is done by upgrader in previous ROS version, direct upgrade from 7.11 or earlier to 7.13 or later would break many people's setup (missing wireless), the ROS built-in upgrades have to go via 7.12.1 ... after ROS 7.12 boots, it'll show later versions as upgrade candidates.
Netinstall allows to install any ROS version, selecting the right combination of optional packages is installer's responsibility.

CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge [SOLVED]

CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge [SOLVED]

Re: CRS3xx and vlans: access port doesn't see traffic unless it is removed from bridge

Who is online