Fasttrack and route marked packets

arnaldo · Wed Apr 18, 2018 3:24 am

I know that PBR does not play well with Fasttrack. So, I'm trying to exclude "route-marked" packets from being Fasttracked. To my understanding, the following filter rule should pick all packets that were not mangled with route-mark during prerouting:

/ip firewall chain=forward action=fasttrack-connection connection-state=established,related routing-mark=main

But, it's not working. As I understand, route-mark from prerouting should be reflected/available in the forwarding chain (otherwise why having the setting).

If I add a mangle rule to "connection-mark" based on the "route-mark" and use the connection-mark in the Fasttrack filter, it works. But one my setup that requires a few more rules (one for each route mark I use) and the route-mark rules (several) need to passthrough, to allow the connection-marking. Lot's of rules and not efficient.

Any ideas why checking for route-mark in the forwarding chain will not work?

Sob · Wed Apr 18, 2018 3:48 am

It works, but there seems to be something special about main routing mark.

If I do this, I see logged packets:

/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=test <some conditions>
/ip firewall filter
add action=log chain=forward log-prefix=test routing-mark=test

If I do this, I don't see anything:

/ip firewall filter
add action=log chain=forward log-prefix=main routing-mark=main

But if I do this, it works:

/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=main <some conditions>
/ip firewall filter
add action=log chain=forward log-prefix=main routing-mark=main

So it looks like packets don't have implicit routing marks. But option routing-table=main should do what you need.

sindy · Wed Apr 18, 2018 8:13 am

Regardless the above (i.e. whether

routing-mark=main

,

routing-mark=none

or

routing-table=main

matches or doesn't match packets without any routing mark explicitly assigned), there is another point: the

action=fasttrack-connection

rule matches also on received packets which are usually not routing-marked. For this and other reasons, it is much better to use connection marking on both incoming and outgoing connections and translate it to routing marking for outbound traffic. That way, the potentially complex conditions are evaluated only once per connection, when deciding whether to connection-mark its initial packet, and the routing-marking is then applied only on outbound packets and evaluates just a single simple condition

connection-mark=xxx

. The exception from the fasttrack rule is then also based on mere presence or a particular value of connection-mark.

arnaldo · Wed Apr 18, 2018 3:43 pm

Interesting, my understanding of "route-mark" with a value of "main" is that it means "no routing mark" (if that's true, why not use "no-mark"). This is very confusing.
Also, I've tried using routing-table, without success. Anyone knows when "routing-table" is set?

I will give the idea of conn-mark and then route-mark a try.

sindy · Wed Apr 18, 2018 4:16 pm

Interesting, my understanding of "route-mark" with a value of "main" is that it means "no routing mark" (if that's true, why not use "no-mark"). This is very confusing.

The confusion probably comes from the fact that it would be weird to call the default routing table "none". So an explicit name "main" has been assigned to the default routing table, but you can as well refer to it implicitly by not providing any table name when creating a route. On the other hand, not to assign any routing mark at all to a packet is quite normal, and it may also be useful to check for the mere absence or presence of a routing mark, regardless its actual value if present. Therefore, reserved identifiers "none" and "any" are used to match these two special cases in rules. Use of "any" to indicate the table to which a particular route belongs would be a clear nonsense, and use of "none" to indicate "default" would seem weird as said above, so for naming the default table, both nothing at all (empty name) and main can be used.

(and don't ask me what happens if you set

new-routing-mark=any

or

new-routing-mark=none

)

Also, I've tried using routing-table, without success. Anyone knows when "routing-table" is set?

EDIT: ignore the text in italics, see newer posts for explanation. I didn't want to argue with @Sob but I think it makes sense to use

routing-table

in rule chains evaluated after routing has been actually performed, i.e. when the routing table used is already known, like

output

and

postrouting

, so way too late for your purpose.

Sob · Wed Apr 18, 2018 5:24 pm

Please do argue with me, if you think that something I posted is not correct, could be improved, etc. I post either things I'm sure about, or I try to give them at least a quick test. This is the second case.

When I tried:

/ip firewall filter
add action=log chain=forward log-prefix="mark main" routing-mark=main
add action=log chain=forward log-prefix="table main" routing-table=main

without any marking, I had "table main" entries in log. And it makes sense to me that when packet is in forward chain and knows outgoing interface (as a result of routing decision), used routing table must be known already.

You're right about incoming connections, I didn't think about them.

sindy · Wed Apr 18, 2018 5:39 pm

Please do argue with me, if you think that something I posted is not correct, could be improved, etc. I post either things I'm sure about, or I try to give them at least a quick test. This is the second case.

Well, I should have written "interfere" rather than "argue", as you were dealing with one part of the issue so I wanted to concentrate only to the other one, i.e. the fasttracking rule "kicking in unexpected" due to matching packets in opposite direction, as one needs to understand how conntrack works to realize that that happens and why.

And you are right that once the packet is being handled by the forward (or input) chain, it must have been routed already, so by my own reasoning the match on

routing-table

in

chain=forward

should work. Weird.

CZFan · Wed Apr 18, 2018 6:33 pm

@Sob and @sindy, with all due respect, I love watching you guys argue / "interfere"

I learn so much from you guys, please continue

tippenring · Wed Apr 18, 2018 9:05 pm

@Sob and @sindy, with all due respect, I love watching you guys argue / "interfere" I learn so much from you guys, please continue

Fully agree with what @CZFan said.

Sob · Wed Apr 18, 2018 10:35 pm

What can I say, it's nice to know that what I post can be sometimes useful even for certified professionals.

arnaldo · Thu Apr 19, 2018 2:46 am

Thanks for all the insights. Using connection-marks was my original approach, but I went the other way around: route-mark -> connection-mark (not a bright idea)

I think the following code will work:

/ip firewall mangle 
add connection-state=new dst-address=10.10.0.0/16 chain=prerouting action=mark-connection new-connection-mark=vpn1 passthrough=yes
add chain=prerouting connection-mark=vpn1 in-interface-list=LAN action=mark-routing new-routing-mark=to_vpn1 passthrough=no
/ip firewall filter
add action=fasttrack-connection chain=forward connection-state=established,related connection-mark=no-mark

I run 4 always-on VPN and traffic is routed there based on a combination of source/destination addresses (there are more than 20 rules). I plan to group them by the target VPN circuit and place the corresponding route-mark just after each group. Maybe repeating all the "mark-routing" rules at the beginning can help, as packets other than the first will arrive "connection-marked" so they will skip the bulk of the rules in the chain. Something like:

/ip firewall mangle 
add chain=prerouting connection-mark=vpn1 in-interface-list=LAN action=mark-routing new-routing-mark=to_vpn1 passthrough=no
add connection-state=new dst-address=10.10.0.0/16 chain=prerouting action=mark-connection new-connection-mark=vpn1 passthrough=yes
add chain=prerouting connection-mark=vpn1 in-interface-list=LAN action=mark-routing new-routing-mark=to_vpn1 passthrough=no
/ip firewall filter
add action=fasttrack-connection chain=forward connection-state=established,related connection-mark=no-mark

There is a little overhead for the first packet in a connection, but the rest will be handled by the top rule(s). Traffic that is not touched by the mangle rules will be fasttracked.

Sounds reasonable?

sindy · Thu Apr 19, 2018 6:06 pm

There is a little overhead for the first packet in a connection, but the rest will be handled by the top rule(s). Traffic that is not touched by the mangle rules will be fasttracked.
Sounds reasonable?

As for me - yes, that's it. The less rules a packet of the most frequent kind has to pass the better.

arnaldo · Thu Apr 19, 2018 8:50 pm

Took some time to rewrite all the rules (changing route-marks to connection-marks) adjust passthrough and setting the new route-marks.
So far seems to be working fine.

The idea to add extra connection-mark->route-mark at the top saves most packages from having to traverse the whole chain. Combine this with fasttrack and accept(related,established) and packets will flow with minimal firewall processing.

Thanks everyone!

msatter · Thu Apr 19, 2018 10:42 pm

After connecting and routing marking, I look if a connecting is still NEW so I can fasttrack it. I think this is because of the no passthrough on marking the routing. I mark fasttrack in Mangle.

sindy · Fri Apr 20, 2018 11:44 am

After connecting and routing marking, I look if a connecting is still NEW so I can fasttrack it. I think this is because of the no passthrough on marking the routing. I mark fasttrack in Mangle.

I'm not sure I get you right.

Adding a connection mark in mangle (or anywhere else) does not change the state of a connection, it remains
Code: Select all
```
new
```
for that packet. Nor does setting
Code: Select all
```
passthrough=no
```
in the rule translating
Code: Select all
```
connection-mark
```
into
Code: Select all
```
new-routing-mark
```
.
I've never seen a fasttrack-connection rule to be applied on packets with
Code: Select all
```
connection-state=new
```
, so I've always assumed that part of conntrack processing has to be done on the first packet which is not
Code: Select all
```
new
```
before the connection can be fasttracked. So I accept packets with
Code: Select all
```
connection-mark=!none
```
before they reach the
Code: Select all
```
action=fasttrack-connection connection-state=established,related
```
rule, and that's it.

msatter · Fri Apr 20, 2018 1:32 pm

I am now more adapting rules to how RouterOS/traffic breaths and not that much according to the manual. It is big fun and you can reduce processing time that way.

So when a connection is routing marked and passthrough is no it will leave Mangle without passing through all the lines underneath. The next time the connection passes through mangle to be marked for routing is not new anymore but also on marking it leaves Mangle again and not even reach the fastracking line.

Normally I would also fasttrack on established, not a fan of related, but when I do it on new connections RouteOS only have to mark it once. Even when a connection is fast tracked it will have some slow packet for control and I hope Mikrotik handels the first packets of a fasttracked connection also as slow.

CZFan · Fri Apr 20, 2018 1:46 pm

...
[*]I've never seen a fasttrack-connection rule to be applied on packets with
Code: Select all
connection-state=new
, so I've always assumed that part of conntrack processing has to be done on the first packet which is not
Code: Select all
new
before the connection can be fasttracked. So I accept packets with
Code: Select all
connection-mark=!none
before they reach the
Code: Select all
action=fasttrack-connection connection-state=established,related
rule, and that's it.
[/list]

My view of why you don't want to fasttrack "new" connections is that you want the new connection to first go through your firewall to see if safe/trusted, once the new connection is seens as safe/trusted, the established/related part of that connection can be accepted/fasttracked

arnaldo · Sat Apr 21, 2018 4:13 am

My view of why you don't want to fasttrack "new" connections is that you want the new connection to first go through your firewall to see if safe/trusted, once the new connection is seens as safe/trusted, the established/related part of that connection can be accepted/fasttracked

That's how I understand it too. I view fasttracking as a "faster" way to forward related/established packets, so only the first packet needs to suffer through the firewall. With that in mind one does not need to worry to much in optimizing the rules, as most of the packets will skip the firewall chains, except for when using mangle for PBR.

As for being a fan of "related", my biggest example is SIP. Try it without using "related". If your SIP client/server is well behaved, you will have to open 10000 port for UDP (if your sw used the 10000-20000 range). WIth "related", you simply ignore SIP, if running SIP clients only, when the clients register with the external server (on port 5060), "related" will allow all RTP UDP sessions to flow without any need to open ports. And Mikrotik SIP ALG works perfectly, even if you use port forward to bypass ISP SIP filtering. They all require "related" to work.

Fasttrack and route marked packets

Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets

Re: Fasttrack and route marked packets