Page 1 of 1
Implementing address list-based routing with RouterOS v7
Posted: Thu Aug 03, 2023 7:18 pm
by medvm
Hello.
I'm trying to migrate my config of address list-based routing from RouterOS v6 to RouterOS v7.
Here's my RouterOS v6 config:
/ip/firewall/address-list> add list block_bypass
/ip/firewall/address-list> add address=example.org list=block_bypass
/ip/firewall/mangle> add action=mark-routing chain=prerouting dst-address-list=block_bypass new-routing-mark=to-vpn
/ip/route> add dst-address=0.0.0.0/0 gateway=vpn routing-mark=to-vpn
New RouterOS v7 config:
/ip/firewall/address-list> add list block_bypass
/ip/firewall/address-list> add address=example.org list=block_bypass
/routing/table/add name=to-vpn fib
ip/firewall/mangle/ add action=mark-routing chain=prerouting dst-address-list=block_bypass new-routing-mark=to-vpn
ip route/add dst-address=0.0.0.0/0 gateway=vpn routing-table=to-vpn
So the problem is that new config is working, but very badly. I see a lot of TCP Retransmissions, sites routed via vpn take several minutes to load, and do so not always on the first try.
Except for the new routing mark the configurations of the different RouterOS versions are identical.
It is not necessary to use routing mark, if someone can suggest a better address list-based routing option I would be very grateful.
Re: Implementing address list-based routing with RouterOS v7
Posted: Thu Aug 03, 2023 8:41 pm
by anav
Most of the extra firewall rules people put in are not required.
What are you experiencing that forces you to adopt 'extra' rules?
Do you have many servers open to the public??
Re: Implementing address list-based routing with RouterOS v7
Posted: Thu Aug 03, 2023 9:03 pm
by Amm0
I'm not sure it your firewall config. Although hard to know from just the snippet - it's does have the right pieces...
Perhaps not, but this sounds like an MTU problem... (e.g. working but slow and TCP retransmissions...)
- Is icmp allowed in the firewall?
- Is the MTU for the VPN interface "right"? e.g. each VPN has different overhead and your WAN might too.
Or maybe some other different defaults between V6 and V7 on VPN interface.
Re: Implementing address list-based routing with RouterOS v7
Posted: Thu Aug 03, 2023 9:06 pm
by msatter
When TCP connections are taking long or even not complete then think of MTU problems. For that I have the following rule in Mange:
add action=change-mss chain=forward comment="WireGuard & IKEv2 Sync" in-interface-list=PMTU-IN log-prefix=MSS new-mss=clamp-to-pmtu \
passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1232-65535
The PMTU-IN interface list contains the gateways that I want to be handled by this rule.
Re: Implementing address list-based routing with RouterOS v7
Posted: Thu Aug 03, 2023 10:47 pm
by medvm
When TCP connections are taking long or even not complete then think of MTU problems. For that I have the following rule in Mange:
add action=change-mss chain=forward comment="WireGuard & IKEv2 Sync" in-interface-list=PMTU-IN log-prefix=MSS new-mss=clamp-to-pmtu \
passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1232-65535
The PMTU-IN interface list contains the gateways that I want to be handled by this rule.
Thank you for the suggestion. I've tried to change the MSS even to 1000, but to no avail.
Funny thing: just routing from ip to ip works perfectly fine (ip routes), the problem seems to be only with mangle rule when dst-address list option is present. Quite strange.
Decided to recreate the address list completely, so now it works mostly fine. But straightforward routing still feels better when browsing.
Hate this kind of floating problems...
P.S. I am still open to suggestions how to solve this "address list-based routing" task more efficiently.
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 1:40 am
by Amm0
Keep in mind, changing MTU may not have an immediate effect... Connections are tracked and devices have caches of PMTUD. And also setting too low have introduce new problems.
But with @msatter this sounds like MTU issue, someplace...
Might want to post your full config. Still could be firewall, but if it was working before...mangle hasn't changed much since V6 so not my first guess here.
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 1:42 am
by Amm0
P.S. I am still open to suggestions how to solve this "address list-based routing" task more efficiently.
I'd say using /routing/rules instead of address-list. But if what you had was working in V6, be easier to figure out the likely one thing that didn't convert from V6 right, than redo the config...
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 2:40 am
by medvm
Keep in mind, changing MTU may not have an immediate effect... Connections are tracked and devices have caches of PMTUD. And also setting too low have introduce new problems.
But with @msatter this sounds like MTU issue, someplace...
Might want to post your full config. Still could be firewall, but if it was working before...mangle hasn't changed much since V6 so not my first guess here.
That's not an MTU issue, because direct routing like this works fine.
ip route/add dst-address=1.1.1.1/32 gateway=vpn
But routing through mangle (even with just dst-address option, without address lists, etc.) causing a lot of tcp retransmissions.
Seems like a bug to me, don't you think?
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 3:10 am
by Amm0
Hard to know, snippets really do NOT tell us much.
e.g.
- Is the VPN also in the main routing table as well, or is just in the new route-table?
- When you "tried changing the MTU", what exact did you try? Did you do ping test to verify the actual MTU? My concern is the default MTU may be different between V6 and V7.
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 12:25 pm
by msatter
You state re-transmissions then the question is your network sending out the request, or the other side because it did not got an acknowledgement from you?
Secondly did you see any traffic hitting the MTU rule I gave? With this one you don't need to state a wished MTU and it will adapt to the MTU size the destination is able to handle.
Thirdly, if the other side is answering on a related IP address you are then sending out through the main gateway. This is not very likely.
As last, I connection mark traffic that should go through the VPN so I can see in connection tracking the connection. You routing mark traffic based on an address-list so there it is not needed keep connection marking active after debugging.
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 3:42 pm
by medvm
You state re-transmissions then the question is your network sending out the request, or the other side because it did not got an acknowledgement from you?
Secondly did you see any traffic hitting the MTU rule I gave? With this one you don't need to state a wished MTU and it will adapt to the MTU size the destination is able to handle.
Thirdly, if the other side is answering on a related IP address you are then sending out through the main gateway. This is not very likely.
As last, I connection mark traffic that should go through the VPN so I can see in connection tracking the connection. You routing mark traffic based on an address-list so there it is not needed keep connection marking active after debugging.
I don't get it. Why you guys think about some MSS/MTU issues if routing the same addresses directly to the same gateway works flawlessly?
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 6:44 pm
by msatter
You missed the first question and the second one. I already stated why I think it's a MTU problem.
But you are free to request support from Mikrotik themselves by mailing them on
support@mikrotik.com
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 6:57 pm
by pe1chl
Another difference between v6 and v7 is that in v7 a routing mark (in IPv4) makes the router do only lookups in the specified table.
In v6 it was just a hint, and other routing rules and the main table were still in use. Not in v7.
So your alternative routing table must be complete. You may need to copy routes from the main table to the second routing table, or you may need to adapt your mangle rule to exclude certain cases form the routing mark (e.g. local addresses).
Also, make sure you don't have the "fasttrack" rule enabled, as this method of route mangle will NOT work with fasttrack enabled.
Disable or remove the fasttrack rule in the forward chain, and reboot the router.
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 7:11 pm
by medvm
Another difference between v6 and v7 is that in v7 a routing mark (in IPv4) makes the router do only lookups in the specified table.
In v6 it was just a hint, and other routing rules and the main table were still in use. Not in v7.
So your alternative routing table must be complete. You may need to copy routes from the main table to the second routing table, or you may need to adapt your mangle rule to exclude certain cases form the routing mark (e.g. local addresses).
Also, make sure you don't have the "fasttrack" rule enabled, as this method of route mangle will NOT work with fasttrack enabled.
Disable or remove the fasttrack rule in the forward chain, and reboot the router.
I was just about to write that I solved the problem by disabling fasttrack. Here's the solution
viewtopic.php?f=2&t=134048&p=659612#p659676
Re: Implementing address list-based routing with RouterOS v7
Posted: Fri Aug 04, 2023 7:19 pm
by anav
just add connection-mark=no-mark to fastrack rule............
Re: Implementing address list-based routing with RouterOS v7 [SOLVED]
Posted: Fri Aug 04, 2023 7:38 pm
by medvm
just add connection-mark=no-mark to fastrack rule............
I am using routing marking, so a correct way to implement the idea would be
/ip/firewall/filter> add action=fasttrack-connection routing-mark=!to-vpn in-interface=!vpn chain=forward
Works fine.
Re: Implementing address list-based routing with RouterOS v7
Posted: Sat Aug 05, 2023 1:47 am
by anav
Well done, adapting the idea to your needs!
Re: Implementing address list-based routing with RouterOS v7
Posted: Sat Aug 05, 2023 11:16 am
by msatter
You should give pe1chl the credits for giving you tips on the fasttrack rule.