I'd like to set up my home lab with a few different VLANS like:

Trusted - Can go anywhere
Untrusted/Guest - Can only go out WAN
IoT - Can't go out WAN

I currently have a CRS309 to act as a core switch, and an RB5009 that I use as my router, along with an old Cisco managed switch, which I'm going to replace with an MT device, I also have 3 cAPs/wAPs.

I have a file server my daughter uses for video editing, and having 10Gbps from her PC to the file server really helps her.

Looking at the test results for the CRS309, it seems that it's nowhere near capable of supporting 10Gbps with firewall rules in effect.

What's a good hardware set that will enable 10Gbps+ with VLANs to segment traffic?

IoT - Can't go out WAN

They're called "Internet of Things" devices because they won't work without an Internet connection.

While there are things mis-called IoT that can work LAN-only, my point is that you might have cause to create a fourth VLAN.

the CRS309…[is] nowhere near capable of supporting 10Gbps with firewall rules in effect.

True, but you don't do this with firewall rules. You do it with bridge VLAN filtering. Firewalling you do on the router, which has the CPU grunt to do such things.

I'd like to set up my home lab with a few different VLANS like:

Trusted - Can go anywhere
Untrusted/Guest - Can only go out WAN
IoT - Can't go out WAN

I currently have a CRS309 to act as a core switch, and an RB5009 that I use as my router, along with an old Cisco managed switch, which I'm going to replace with an MT device, I also have 3 cAPs/wAPs.

I have a file server my daughter uses for video editing, and having 10Gbps from her PC to the file server really helps her.

Looking at the test results for the CRS309, it seems that it's nowhere near capable of supporting 10Gbps with firewall rules in effect.

What's a good hardware set that will enable 10Gbps+ with VLANs to segment traffic?

IMO - if you have a PC/work-station talking to a file-server - to get the fastest possible I/O speed/transfer-rate , consider doing the following:
- Put the PC/work-station and the file-server on the same network ( same IP range ).
- Layer-2 switch instead of Layer-3 routing. ( L-3 routing and L-3 firewall configs both use CPU resources --&-- L-2 hardware switching does not use the CPU , you get full port network port speed between the PC/work-station to/from your file-server.

Also , depending on your server/work-station and file-server , consider some research into the following:
- SMB vs NFS vs iSCSI transfer rates ( which is he fastest )
- sync vs async writes configured in your file-server system
- if you use a ZFZ file system on your file-server , add some additional RAM so that you end up with more L2-ARC cache


North Idaho Tom Jones

Yes connect the PC and the server to the same switch (even a cheap unmanaged switch is fine if all on same network).

@tangent, some folks like to send their camera recordings only to a local NAS or camera server and not have the camera etc talk to the cloud, or at least this is what I think they mean but totally acree its plain weird to cutoff iot from internet.

I wouldn’t call a traditional IPcam an “IoT” camera. It isn’t until you get into the Ring and Nest type stuff that I’d apply that label.

It isn’t pedantry because it affects which devices go on which VLAN and the design of those VLANs.

Looking at the test results for the CRS309, it seems that it's nowhere near capable of supporting 10Gbps with firewall rules in effect.

It is with L3 hardware offload. The CRS309 can do full L3 offload, including firewalling.

I'd like to set up my home lab with a few different VLANS like:

Trusted - Can go anywhere
Untrusted/Guest - Can only go out WAN
IoT - Can't go out WAN

I currently have a CRS309 to act as a core switch, and an RB5009 that I use as my router, along with an old Cisco managed switch, which I'm going to replace with an MT device, I also have 3 cAPs/wAPs.

I have a file server my daughter uses for video editing, and having 10Gbps from her PC to the file server really helps her.

Looking at the test results for the CRS309, it seems that it's nowhere near capable of supporting 10Gbps with firewall rules in effect.

What's a good hardware set that will enable 10Gbps+ with VLANs to segment traffic?

IMO - if you have a PC/work-station talking to a file-server - to get the fastest possible I/O speed/transfer-rate , consider doing the following:
- Put the PC/work-station and the file-server on the same network ( same IP range ).
- Layer-2 switch instead of Layer-3 routing. ( L-3 routing and L-3 firewall configs both use CPU resources --&-- L-2 hardware switching does not use the CPU , you get full port network port speed between the PC/work-station to/from your file-server.

Also , depending on your server/work-station and file-server , consider some research into the following:
- SMB vs NFS vs iSCSI transfer rates ( which is he fastest )
- sync vs async writes configured in your file-server system
- if you use a ZFZ file system on your file-server , add some additional RAM so that you end up with more L2-ARC cache


North Idaho Tom Jones

What if, for example, "Untrusted" VLAN is 10.1.1.0/24,"Semi-Trusted" VLAN is 10.1.2.0/24, "Fully-Trusted" VLAN is 10.1.3.0/24, and the file server is 10.1.4.1/24.

Untrusted and Semi-Trusted can access the File Server, but Untrusted, can't. How would I do that without needing firewall rules?

What brand ( operating system ) is the file server ?

One possible thing you might be able to do ( depending on your file-server ) ...
- WAN interface to your default gateway router to get out to the Internet
- Multiple inside LANs on your file-server ( for PCs inside one of your networks ). This allows any PC inside one of your networks to talk directly to one of the inside LAN IP addresses on your file-server without having to route ( aka go through a gateway to get to a different IP on your file-server ).

A PC workstation on the same IP address network as a file-server will always be faster than having to route through a router to get to a file-server.

...and with a fileserver you also need to look at aspects like NFS ACL's or SMB User-accounts etc.
Being able to "reach" your fileserver does not mean you can access it / use it.
Depending on the file-server model/OS , you can also apply a IP-ACL to exclude the "Untrusted" IP-range.
Sure the packets "will reach" the NIC of the file-server but who cares.

Most performant approach is have a NIC of the file-server directly into the "Trusted" or "Semi-Trusted" network.

All of this might help to obtain the goal of more then adequate secure access to files for certain devices on certain networks WITHOUT looking too much at the Mikrotik devices

I assume you are running a regular "home" network right ? No NSA or MIL-spec environment .....

Lil off topic - but still related to file-servers ...

Take a look at TrueNAS
I run a dozens of TrueNAS file servers. When configured correctly , they can be pretty fast.

For example , I have a TrueNAS file-server with 1-TB RAM and about 256-TB of solid-stade SSD drives with 100-GIg network interfaces and it can sustain 25 to 30-Gig transfer rates.

My slowest TrueNAS systems ( 128-Gig RAM with SATA 6-Gig disks ) , can burst up to 10-Gig then it will drop down to 6-GIg and sustain 6-Gig with no problems.

Lil off topic - but still related to file-servers ...

Take a look at TrueNAS
I run a dozens of TrueNAS file servers. When configured correctly , they can be pretty fast.

For example , I have a TrueNAS file-server with 1-TB RAM and about 256-TB of solid-stade SSD drives with 100-GIg network interfaces and it can sustain 25 to 30-Gig transfer rates.

My slowest TrueNAS systems ( 128-Gig RAM with SATA 6-Gig disks ) , can burst up to 10-Gig then it will drop down to 6-GIg and sustain 6-Gig with no problems.

Lil more offtopic ; are you using such machines to offer News/NNTP-services for your users ? Because you are runninng an ISP right ?
Can't see too much other use-cases requiring that insane amount of transfer-rates.
Or do you have a Netflix "Open Connect Appliance" of some deployed near your users.

We also run Netflix, Google & Akamai caches "closer" to our users but often these are "cots" hardware with classic 10G ports but a lot of machines (scale out)

Lil off topic - but still related to file-servers ...

Take a look at TrueNAS
I run a dozens of TrueNAS file servers. When configured correctly , they can be pretty fast.

For example , I have a TrueNAS file-server with 1-TB RAM and about 256-TB of solid-stade SSD drives with 100-GIg network interfaces and it can sustain 25 to 30-Gig transfer rates.

My slowest TrueNAS systems ( 128-Gig RAM with SATA 6-Gig disks ) , can burst up to 10-Gig then it will drop down to 6-GIg and sustain 6-Gig with no problems.

Lil more offtopic ; are you using such machines to offer News/NNTP-services for your users ? Because you are runninng an ISP right ?
Can't see too much other use-cases requiring that insane amount of transfer-rates.
Or do you have a Netflix "Open Connect Appliance" of some deployed near your users.

We also run Netflix, Google & Akamai caches "closer" to our users but often these are "cots" hardware with classic 10G ports but a lot of machines (scale out)

Re: ... are you using such machines to offer ...

We are an ISP doing fiber-to-the-home/business & wireless ( mikrotik's ) to the home. Also, we host/manage some large file-servers for a few of our 10-Gig connected business customers. All of our file-servers combined ( in-house and business customers ) perform daily full-backups - where we average transferring well over 100-Tib of file transfers to our backup servers. As you might guess , transferring 100+ TiB of data backups on a daily bases requires some very high bandwidth over/to/from some very fast NAS file-servers to/from some very fast network switches.

Sooo , with the original topic this forum thread being "Performance: 10Gbps - VLANs, and WiFi" , I was offering some related advice on how to make a network file-server run fast.

North Idaho Tom Jones

What if, for example, "Untrusted" VLAN is 10.1.1.0/24,"Semi-Trusted" VLAN is 10.1.2.0/24, "Fully-Trusted" VLAN is 10.1.3.0/24, and the file server is 10.1.4.1/24.

Untrusted and Semi-Trusted can access the File Server, but Untrusted, can't. How would I do that without needing firewall rules?

You could also add a routing-rule (aka "policy routing")

Routing > Rules

Sure this probably kills performance too, but its a valid approach.