Adding VETH to a bridge triggers a weird VLAN filtering behavior

stigger · Tue Feb 14, 2023 1:01 am

I have an RB5009 with the routeros 7.7 and the following bridge configuration:

/interface bridge
add admin-mac=DC:2C:6E:66:3A:E0 auto-mac=no frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether1
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether2
add bridge=bridge interface=ether3 pvid=132
add bridge=bridge interface=ether4 pvid=130
add bridge=bridge disabled=yes interface=veth1 pvid=132
/interface bridge vlan
add bridge=bridge tagged=bridge,ether1,ether2,ether3 vlan-ids=131
add bridge=bridge tagged=bridge,ether1 untagged=ether4 vlan-ids=130
add bridge=bridge tagged=bridge,ether1,ether2 untagged=ether3 vlan-ids=132

Everything worked perfectly until I tried to experiment with containers functionality. As soon as I enable the veth1 port, I start seeing a strange behavior on the untagged ether3 and ether4 ports: some packets are sent to the port untagged, and some are tagged with the corresponding VLAN. For example, here are two consequent TCP packets, one with the label, and one without:

This leads to a massive packet loss and degraded performance.

Is there anything that can be done to make it work?

Amm0 · Tue Feb 14, 2023 2:09 am

Not 100% sure. But perhaps don't assign an IP address to the VETH interface on the Mikrotik side (e.g. skip the /ip/address for VETH)? The /ip/addres creates a connected subnet in /ip/route – except it uses the bridge as the interface (since automatically connected subnet routes don't know about vlan-filtering). Unlike a VLAN, which listens on the bridge interface, VETH is a port, so little different behavior as a slave port to the bridge.

Since one end (e.g. the container) has an IP address, and it's bridged (so VLAN IP is RouterOS/CPU's L3 IP address" for the bridge, VETH shouldn't need an IP address. The container can use the VLAN's IP address if it needs on for the router.

stigger · Tue Feb 14, 2023 2:13 am

There is no address assigned to veth1 in `/ip/addresses`. There is a link-local ipv6 address which was created automatically, however removing it did not fix the problem.

Amm0 · Tue Feb 14, 2023 2:15 am

What does /ip/route show for the VETH?

stigger · Tue Feb 14, 2023 2:16 am

Nothing, it's not mentioned there.

Amm0 · Tue Feb 14, 2023 2:17 am

Also, you should use frame-type of "only untagged and priority tags" if the intend is for VETH to be an access port.

stigger · Tue Feb 14, 2023 2:20 am

Also, you should use frame-type of "only untagged and priority tags" if the intend is for VETH to be an access port.

But "admit all" should also work, shouldn't it? I mean, it's not necessary to restrict the accepted frame types, right?

Amm0 · Tue Feb 14, 2023 2:30 am

Well perhaps, but I believe ingress-filtering=yes is default & VETH isn't listed a tagged port. And your config gets a bit complex since bridge is VLAN tagged only. All may be fine, in fact likely a good idea...

But I'd just be explicit in the Bridge VLANs table on what interface should be tagged and untagged. Simpilar, If it's access port, declare it as such with "untagged only" frame-type (or if really "hybrid" then admit-all is totally right). But I don't see VETH as a tagged or untagged port in bridge's VLAN table. In theory, the PVID should add VETH it as an untagged port, but not if that's try if admit-all (thus the suggestions).

What's happening in the VLAN table with the VETH be the next place to look.

stigger · Tue Feb 14, 2023 2:37 am

The idea was to have it as an untagged port, and enabling it correctly adds it as an untagged port to /interface/bridge/vlan without explicit configuration. Other than that, there are no changes to the VLAN table caused by toggling the port.

Amm0 · Tue Feb 14, 2023 3:39 am

Well, the use of hybrid ports introduces a bunch of other variables and should be avoid IMO – truck or access port make troubleshoot easier. For example, it's ingress NOT egress filtering, so lots of ways tagged packets can get in. Could look the MACs in your packet tracker to know the devices involved, maybe that give another clue on path that's cause tagging/not tagging on ether3/4.

Also there is the subject RSTP, which is the default for a bridge, so something in spanning tree could be causing this too. So could try disabled RSTP to see if that fixes.

But I think I'm guessing now. You should post the full config, with some explanation of what's on the ports. I personally try to make everything explicit (and avoid hybrid ports) in bridge, except dynamic PVID for access ports.

stigger · Tue Feb 14, 2023 11:40 pm

I disabled RSTP and set "admit only untagged and priority tags" for ether4, making it an access-only port. The problem did not go away.

In any case, it looks like you agree that in general my configuration makes sense and this is more likely a bug, rather than a configuration error. Hopefully someone from RouterOS team will chime in (or someone else shows a configuration that's known to be working on RB5009).

Amm0 · Wed Feb 15, 2023 12:28 am

it looks like you agree that in general my configuration makes sense and this is more likely a bug, rather than a configuration error

I'm saying it's most likely a configuration error, NOT a bug. The packet flow is complex, but all this traceable through RouterOS:
https://help.mikrotik.com/docs/display/ ... n+RouterOS
So you kinda need to about to point to where it's jumping off the rails in those diagrams to say it's bug.

At this point, I'm just not sure what you're trying to do...so hard to know what's suppose be right. You might get more help if you followed viewtopic.php?t=182373 - mainly including the full configuration.

stigger · Wed Feb 15, 2023 12:50 am

I'm saying it's most likely a configuration error, NOT a bug. The packet flow is complex, but all this traceable through RouterOS:

Hm, so there might be an explanation for the fact that this behavior is triggered by adding a VETH to the bridge? Well, OK. I don't want to share my current configuration, but on the weekend I'll try to reproduce the problem on a fresh config, which I will be able to share.

At this point, I'm just not sure what you're trying to do...so hard to know what's suppose be right.

I'm not sure why you're not sure

I have several VLANs, the bridge has a couple of trunk ports, one access port, one hybrid port. The goal is to give access to a docker container from VLAN 132 by connecting veth1 to it. The "/interface bridge vlan" configuration sums everything up pretty well, there are no mistakes.

Amm0 · Wed Feb 15, 2023 12:56 am

Not knowing the source of a config error isn't same as a bug

I'm not sure why you're not sure I

I just meant with a partial configuration file, it's hard to know. And, those hybrid ports are tricky to get right, so if there was some way to avoid/cleanup the need, that's might avoid having to debug the specific config issue too

. RouterOS bridge is, well, confusing. e.g. just because some config logically looks right, there are a little subtle rules that come into play. Or if mistake elsewhere in config, too

stigger · Sun Feb 19, 2023 3:54 am

Amm0, you were right, the issue does not reproduce as easily as I initially thought.

Here is the full minimized config:

/interface bridge
add admin-mac=DC:2C:6E:66:3A:E0 auto-mac=no frame-types=\
    admit-only-vlan-tagged name=bridge protocol-mode=none vlan-filtering=yes
/interface veth
add address=192.168.129.2/24 gateway=192.168.129.1 name=veth1
/interface wireguard
add listen-port=13231 mtu=1392 name=vpn
/interface vlan
add interface=sfp-sfpplus1 name=isp-vlan vlan-id=40
add interface=bridge name=lan vlan-id=132
add interface=bridge name=via-vpn vlan-id=130
/interface pppoe-client
add add-default-route=yes disabled=no interface=isp-vlan keepalive-timeout=60 \
    name=isp-pppoe user=username
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether1
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged \
    interface=ether4 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged \
    interface=veth1 pvid=132
/interface bridge vlan
add bridge=bridge tagged=bridge,ether1 untagged=ether4 vlan-ids=130
add bridge=bridge tagged=bridge,ether1 vlan-ids=132
/interface wireguard peers
add allowed-address=0.0.0.0/0 endpoint-address=[...] endpoint-port=\
    [...] interface=vpn persistent-keepalive=15s public-key=\
    "[...]"
/ip address
add address=192.168.132.2/24 interface=lan network=192.168.132.0
add address=[...] interface=vpn network=[...]
add address=192.168.130.1/24 interface=via-vpn network=192.168.130.0
/ip firewall nat
add action=masquerade chain=srcnat
/ip route
add dst-address=88.198.248.254 gateway=vpn

From a client behind via-vpn/ether4 I'm doing

curl https://speed.hetzner.de/10GB.bin > /dev/null

(domain resolves to 88.198.248.254), and that reproduces the issue. Removing veth1 from the bridge fixes the problem. Removing the route to 88.198.248.254 fixes the problem.

thr · Sat Apr 15, 2023 8:39 pm

I'm seeing this same behaviour on 7.8 while trying to setup a mdns-repeating container for multiple bridge vlans, the moment veth is enabled tcp traffic originating from the router (including vpns) is crippled (speeds from 300+mbps down to 1mbps)

stigger · Sat Apr 15, 2023 10:33 pm

I'm seeing this same behaviour on 7.8 while trying to setup a mdns-repeating container for multiple bridge vlans, the moment veth is enabled tcp traffic originating from the router (including vpns) is crippled (speeds from 300+mbps down to 1mbps)

I've reported the issue to the support back in February. They reproduced it and confirmed that this is a bug, but could not provide an ETA for a fix.

Adding VETH to a bridge triggers a weird VLAN filtering behavior

Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior

Re: Adding VETH to a bridge triggers a weird VLAN filtering behavior