Hello there,

I am using a CCR2004 router as my main internet router. Right behind it I am running two virtual OPNsense firewalls in HA mode which are also doing my inter VLAN routing.
All devices share the network informations via OSPF in a single area (0.0.0.0).

As the inter VLAN routing performance of OPNsense is not the best (only getting ~4.5 Gbit instead of ~10 Gbit) I would like to have kinda split routing:

WAN --> CCR2004 --> OPNsense (for general firewalling) --> CCR2004 (for VLAN routing).


What would be the best way to archieve this?

Anyone any idea?
I tried to play around with VRF but unfortunately when enabling any VRF I am no longer able to reach other devices (either devices via VPN or OPNsense itself)

If your Opensense devices are doing all the routing, what remains for CCR to do? Is it only switching?

CCR is responsible for my VPN tunnels, NAT/portforwarding, WAN traffic in general.

I am using it as my ISP only allows one MAC address on the interface. OPNsense in HA does not do that and I had troubles with that.

The only reason why I need to use OPNsense is IDS/IPS via Suricata and some Geoblocking to keep those Asian bots out of my network

So traffic from (V)LAN clients to internet will pass CCR twice (once from client towards Opensense and another time from opensense towards internet, similarly in the other direction). Which then requires marking of packets for routing according to ingress interface. Won't be easy on router's CPU either. And adding VRF into the mix doesn't make CCR's life any easier. And I'm not sure if CCR will be able to route at 10Gbps, official test results indicate real life routing capacity of around 4.5Gbps (and your setup will be pretty heavier than average, so I'd expect to see lower performance in your particular use case).

If you need simple inter-vlan routing on the LAN side of Opensense, then you may want to look into getting a decent L3 switch ... MT has some to offer, have a look at L3 hw offloading manual, it has some capability tables. Just beware: when looking at routing prefixes (or routes) numbers, directly connected networks count as large number of routes/prefixes, each host counts (i.e. /32 routes for IPv4 and /128 routes for IPv6). Meaning it's quite easy to exhaust the routing table and after that L3 switch will start routing using its (weak) CPU and performance will drop to the floor.

So best would be to either leave as it is or get another router for VLAN routing?

Yup. I've just edited my previous post with some idea and pointer.

Thanks for the info.
In general I am already using a CRS312-4C+8XG-RM as my core switch where my Proxmox servers are connected via a LACP bond.
So far they work pretty well but as I would need also some firewall rules like DMZ-VLAN is not allowed to access some hosts from other VLANs this switch would probably getting 100 % CPU very quickly.

My main issue why I started this thread is that I only get 5 Gigabit due to virtual OPNsense with only two physical interfaces.
If I run iperf3 from one VLAN to another VLAN the bandwith is divided by two as traffic needs to run through the cables twice.

Using CRS312 with some straight-forward firewall rules should be fine as it can offload fasttrack to HW. Which means that firewall uses CPU only to process packets which are starting new connections (and those are generally not so frequent ... unless some LAN device starts a DoS attack on L3 routing engine).