Community discussions

MikroTik App
  4. RouterOS
  5. Beginner Basics

Virtualized VLANs (for Proxmox)  [SOLVED]

Post Reply
 
3eff
just joined
Topic Author
Posts: 6
Joined: Tue Mar 26, 2024 6:52 pm

Virtualized VLANs (for Proxmox)

Thu Apr 04, 2024 9:31 pm

Hi everyone,
after resolving this issue viewtopic.php?t=206359 just wanted to know how to setup RouterOS 7 (without external switch) in order to have the following setup:
  • VLAN10, 192.168.10.0/24, for router and proxmox management
  • VLAN11, 192,168,11.0/24, for virtualized machines inside Proxmox nodes on VLAN10
I want to have a virtualized LAN in order to isolate "operative" infraestructure/LAN (VLAN11) inside the IT LAN (VLAN10).

Besides Proxmox and virtualized machines setup, what would be the correct configuration of tagget/untagged in the router. Now I have the following conf (only what I believe is important for VLAN conf) but only VLAN10 is working (internet, LAN connection), VLAN11 outside proxmox working ok, VLAN11 inside proxmox seems not to have network connectivity (inside and outside):
Code: Select all
/interface bridge
add name=aBridge port-cost-mode=short protocol-mode=none pvid=11 vlan-filtering=yes
/interface vlan
add interface=aBridge name=VLAN10 vlan-id=10
add interface=aBridge name=VLAN11 vlan-id=11
/interface list
add name=WAN
add name=LAN
[...]
/ip pool
add name=VLAN10_POOL ranges=192.168.10.100-192.168.10.200
add name=VLAN11_POOL ranges=192.168.11.100-192.168.11.200
/ip dhcp-server
add address-pool=VLAN10_POOL interface=VLAN10 lease-time=10m name=VLAN10_DHCP
add address-pool=VLAN11_POOL interface=VLAN11 lease-time=10m name=VLAN11_DHCP
[..]
/interface bridge port
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=ether2 internal-path-cost=10 path-cost=10 pvid=10
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=ether3 internal-path-cost=10 path-cost=10 pvid=10
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=wlan1 internal-path-cost=10 path-cost=10 pvid=10
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=wlan2 internal-path-cost=10 path-cost=10 pvid=11
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=ether4 internal-path-cost=10 path-cost=10 pvid=11
add bridge=aBridge frame-types=admit-only-untagged-and-priority-tagged interface=ether5 internal-path-cost=10 path-cost=10 pvid=11
[..]
/interface bridge vlan
add bridge=3TSBridge tagged=3TSBridge vlan-ids=101
add bridge=3TSBridge tagged=3TSBridge vlan-ids=100
/interface list member
add interface=ether1 list=WAN
add interface=VLAN10 list=LAN
add interface=VLAN11 list=LAN
/ip address
add address=192.168.10.1/24 interface=VLAN10 network=192.168.10.0
add address=192.168.11.1/24 interface=VLAN11 network=192.168.11.0
[..]
/ip dhcp-server network
add address=192.168.10.0/24 dns-server=192.168.10.1 gateway=192.168.10.1
add address=192.168.11.0/24 dns-server=192.168.11.1 gateway=192.168.11.1
/ip dns
set allow-remote-requests=yes
/ip firewall address-list
add address=192.168.10.0/24 list=LAN
add address=192.168.11.0/24 list=LAN
/ip firewall filter
add action=accept chain=input comment="Accept established/related/untracked" connection-state=established,related,untracked
add action=drop chain=input comment="Drop invalid" connection-state=invalid
add action=accept chain=input comment="Accept ICMP" protocol=icmp
add action=accept chain=input comment="Accept local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="Drop all not comming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="Accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="Accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="Fastrack for established/related" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="Accept forwardig of established/related/untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="Drop all WAN not NATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT out" ipsec-policy=out,none out-interface-list=WAN
[..]
Thanks in advance!
Juan Ignacio.
Top
 
User avatar
TheCat12
Member
Member
Posts: 429
Joined: Fri Dec 31, 2021 9:13 pm

Re: Virtualized VLANs (for Proxmox)

Thu Apr 04, 2024 10:07 pm

Could you post a diagram of some sort because your configuration is a bit confusing
Top
 
tdw
Forum Guru
Forum Guru
Posts: 2032
Joined: Sat May 05, 2018 11:55 am

Re: Virtualized VLANs (for Proxmox)

Fri Apr 05, 2024 12:22 am

The configuration doesn't make sense - you have name=aBridge in /interface bridge but references to bridge=3TSBridge in /interface bridge vlan.
Also, do not set the bridge-to-CPU PVID in /interface bridge to have the same ID as an /interface vlan attached to the bridge.
Top
 
3eff
just joined
Topic Author
Posts: 6
Joined: Tue Mar 26, 2024 6:52 pm

Re: Virtualized VLANs (for Proxmox)

Fri Apr 05, 2024 12:43 am

Sure @theCat12!
Hope this diagram helps...
Image

Right know, Proxmox (and its contained VMs) cannot access anything on 192.168.11.x/24 (just 192.168.11.2 that lives inside proxmox machine static IP).
@tdw, just a typo from playing with config: should be aBridge always, sorry. Sorry also for misunderstanding, which is the CPU PVID and where in the code I'm attaching it to de vlan attached to the bridge? (I believed it was CPU PVID was 0 for mikrotik)
Juan Ignacio,
Last edited by 3eff on Fri Apr 05, 2024 2:23 am, edited 1 time in total.
Top
 
User avatar
loloski
Member
Member
Posts: 420
Joined: Mon Mar 15, 2021 9:10 pm

Re: Virtualized VLANs (for Proxmox)  [SOLVED]

Fri Apr 05, 2024 1:09 am

You need a trunk port on both router and proxmox, this is how you are going to do it if your equipment is a CRS 3xx switch this is device specific , this is just a bare minimum and assume that your tagged port is ether1
Code: Select all
[admin@RT] > export
# 2024-04-04 22:01:36 by RouterOS 7.15beta9
# software id =
#
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface vlan
add interface=bridge name=VLAN10 vlan-id=10
add interface=bridge name=VLAN11 vlan-id=11
/ip pool
add name=dhcp_pool0 ranges=192.168.10.2-192.168.10.254
add name=dhcp_pool1 ranges=192.168.11.2-192.168.11.254
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether1 pvid=4094
/interface bridge vlan
add bridge=bridge tagged=bridge,ether1 vlan-ids=10
add bridge=bridge tagged=bridge,ether1 vlan-ids=11
/ip address
add address=192.168.10.1/24 interface=VLAN10 network=192.168.10.0
add address=192.168.11.1/24 interface=VLAN11 network=192.168.11.0

/ip dhcp-server
add address-pool=dhcp_pool0 interface=VLAN10 name=dhcp1
add address-pool=dhcp_pool1 interface=VLAN11 name=dhcp2
/ip dhcp-server network
add address=192.168.10.0/24 gateway=192.168.10.1
add address=192.168.11.0/24 gateway=192.168.11.1
/system identity
set name=RT
/system note
set show-at-login=no
/tool romon
set enabled=yes
Top
 
3eff
just joined
Topic Author
Posts: 6
Joined: Tue Mar 26, 2024 6:52 pm

Re: Virtualized VLANs (for Proxmox)

Fri Apr 05, 2024 1:40 am

Thanks @loloski,
I believed the solution was about trunk port, but did not know how to apply the the specific mikrotic implementation (tried randomly some changes related to tagged/untagged without success) I will try you solution in my scenario and then tell the about results!
Juan Ignacio.
You need a trunk port on both router and proxmox, this is how you are going to do it if your equipment is a CRS 3xx switch this is device specific , this is just a bare minimum and assume that your tagged port is ether1
Code: Select all
[admin@RT] > export
# 2024-04-04 22:01:36 by RouterOS 7.15beta9
# software id =
#
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface vlan
add interface=bridge name=VLAN10 vlan-id=10
add interface=bridge name=VLAN11 vlan-id=11
/ip pool
add name=dhcp_pool0 ranges=192.168.10.2-192.168.10.254
add name=dhcp_pool1 ranges=192.168.11.2-192.168.11.254
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether1 pvid=4094
/interface bridge vlan
add bridge=bridge tagged=bridge,ether1 vlan-ids=10
add bridge=bridge tagged=bridge,ether1 vlan-ids=11
/ip address
add address=192.168.10.1/24 interface=VLAN10 network=192.168.10.0
add address=192.168.11.1/24 interface=VLAN11 network=192.168.11.0

/ip dhcp-server
add address-pool=dhcp_pool0 interface=VLAN10 name=dhcp1
add address-pool=dhcp_pool1 interface=VLAN11 name=dhcp2
/ip dhcp-server network
add address=192.168.10.0/24 gateway=192.168.10.1
add address=192.168.11.0/24 gateway=192.168.11.1
/system identity
set name=RT
/system note
set show-at-login=no
/tool romon
set enabled=yes
Top
 
User avatar
loloski
Member
Member
Posts: 420
Joined: Mon Mar 15, 2021 9:10 pm

Re: Virtualized VLANs (for Proxmox)

Fri Apr 05, 2024 2:37 am

There are many ways how to make a VLAN in mikrotik, even if it works it doesn't mean it's correct because if you are doing that on non CRS 3xx series it will consume CPU, so be careful
Top
 
3eff
just joined
Topic Author
Posts: 6
Joined: Tue Mar 26, 2024 6:52 pm

Re: Virtualized VLANs (for Proxmox)

Thu Apr 11, 2024 6:05 am

Thank you @loloski,
just played a little more with
Code: Select all
admit-only-vlan-tagged
on aBridge and the specific port to use Proxmox and its virtual machines.
Have a nice week! Juan Ignacio.
Top
 
User avatar
Lokamaya
Member Candidate
Member Candidate
Posts: 101
Joined: Thu Nov 11, 2021 4:40 am
Location: Bandung

Re: Virtualized VLANs (for Proxmox)

Wed Oct 30, 2024 7:22 pm

You need a trunk port on both router and proxmox
Got it working with single NIC on my Proxmox server. This tutorial and "How To Create VLANs in Proxmox For a Single NIC" on Youtube make me understand why we need "admit-only-vlan-tagged" and add "pvid=4094" to the bridge port on Mikrotik.
Top
 
User avatar
Lokamaya
Member Candidate
Member Candidate
Posts: 101
Joined: Thu Nov 11, 2021 4:40 am
Location: Bandung

Re: Virtualized VLANs (for Proxmox)

Wed Oct 30, 2024 7:29 pm

#Bridge
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge1 vlan-filtering=yes
...

#Bridge Port: Facing to Proxmox
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether2 pvid=4094
Top
 
tdw
Forum Guru
Forum Guru
Posts: 2032
Joined: Sat May 05, 2018 11:55 am

Re: Virtualized VLANs (for Proxmox)

Sat Nov 02, 2024 1:58 am

Setting pvid= is irrelevant with frame-types=admit-only-vlan-tagged as untagged packets are discarded.
Top
 
User avatar
Lokamaya
Member Candidate
Member Candidate
Posts: 101
Joined: Thu Nov 11, 2021 4:40 am
Location: Bandung

Re: Virtualized VLANs (for Proxmox)

Sat Nov 02, 2024 11:46 am

Setting pvid= is irrelevant with frame-types=admit-only-vlan-tagged as untagged packets are discarded.
Yes, it's what is expected.

The default Proxmox network interface with bridge-vlan-aware look like this:
Code: Select all
auto lo
iface lo inet loopback

iface enp0s25 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.99.100/24
        gateway 192.168.99.1
        bridge-ports enp0s25
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

source /etc/network/interfaces.d/*
If we want to utilize all available vlans on single NIC, nothing works except with the configuration above, i.e. adding non-existent pvid. With this specific configuration, we can use any available vlans inside Proxmox or it's VMs.
Code: Select all
auto lo
iface lo inet loopback

auto enp0s25
iface enp0s25 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp0s25
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4000
#vlans: 99, 210, 220;

auto vmbr0.99
iface vmbr0.99 inet static
        address 192.168.99.100/24
        gateway 192.168.99.1
#vlan-mgmt

auto vmbr0.210
iface vmbr0.210 inet static
        address 192.168.210.100/24
#vlan-210

source /etc/network/interfaces.d/*
It should be noted that the value of iface's bridge-vids 2-4094 (default) must be changed into something like bridge-vids 2-4090, then we use the value outside that range for pvid in the Mikrotik bridge/port. I'm not a linux expert. So I don't know if this behavior applies to all linux bridges or only on Proxmox.

On VM inside Proxmox, simply tag the intended vlan:
Proxmox.JPG
You do not have the required permissions to view the files attached to this post.
Top
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 12979
Joined: Thu Mar 03, 2016 10:23 pm

Re: Virtualized VLANs (for Proxmox)

Sat Nov 02, 2024 1:50 pm

If we want to utilize all available vlans on single NIC, nothing works except with the configuration above, i.e. adding non-existent pvid.

Not entirely true. When setting VLAN-related thing on bridge and sub-items, things are pretty much divided:
  • items under bridge/port are about ingress behaviour
    setting bridge port with frame-types=admit-only-vlan-tagged makes pirt to reject any untagged frame on i gress. Also makes setting pvid completely irrellevant
  • items under bridge/vlan are about egress behaviour
    only frames, belonging to one of VLANs of which port is member, can egress that port. If port is set as untagged member of port, then VLAN header will be stripped upon leaving the port
  • items on bridge definition (under bridge) are either about bridge switch-like entity (very few of them) or about CPU-facing bridge port (most of items) ... frame-types is one of CPU-facing port properties

And then there are a few interactions between first two items above, some are automatic and some have to be enabled explicitly:
  • when port is set with PVID (and it allways is even if with default value of 1) and frame-types setting allows untagged on ingress, then port is automatically added as untagged member of corresponding VLAN
    This doesn't happen in your case as frame-types setting doesn't allow untagged frames on ingress
  • when ingress-filtering is enabled (by manual configuration) on bridge/port, then port VLAN membership (from bridge/vlan) is consulted when determinimg if a frame can be allowed to ingress or not.
    I.e. if port is member of VLANs with IDs 100 and 200 and there's frame with VID 300 "trying to enter", it'll be dropped with ingress-filtering ebabled ... but would be alowed to ingress with ingress-filtering disabled (which is default setting).

The proxmox config woukd be mirrored on proxmox-facing MT port lije this:
Code: Select all
/interface/bridge/port
add bridge=bridge interface=etherX ingress-filtering=yes frame-types=allow-only-vlan-tagged
/interface/bridge/vlan
add bridge=bridge vlan-ids=2-4094 tagged=etherX
The problem with config in the bridge/vlan section is that there can be only line targeting any given VLAN ID. So if there was a line like add bridge=bridge vlan-ids=300 tagged=bridge,etherX, configuration attempt would fail. It woukd have to be rewritten like this:
Code: Select all
/interface/bridge/vlan
add bridge=bridge vlan-ids=2-299,301-4094 tagged=etherX
add bridge=bridge vlan-ids=300 tagged=bridge,etherX
The config libe order doesn't matter, what matters is "puncturing holes" in VID range of the upper line (and I'm not entirely sure if the syntax shown for merging two intervals is actually valid). And this need for "puncturing tge interval" is the biggest PITA when it comes to the way VLANs are configured in ROS (but it's actually the same way as in linux bridge).
Top
Post Reply

Who is online

Users browsing this forum: faxxe and 14 guests
  4. RouterOS
  5. Beginner Basics