I am trying to understand fully packet, connection and route marking. Can anyone confirm any of the following?

Can a packet have a connection mark, a packet mark and a route mark at the same time?
Can a packet have multiple packet marks?
Is a connection mark associated with all packets related to the connection?

Some explanation of why I am looking at this may clarify things. I have routers (RB150 and Dell 1950 with RoS) located in a data centre that has multiple networks connected.

The routers I have used (Linux and Cisco) have traditionally ensured the reply packets leave on the interface they entered on if the packets target the router itself. Packets forwarding through the router follow traditional route tables.

The issue I have with RoS is that if I try to connect via ssh or winbox to any interface the replies try to leave by the default gateway and not the gateway associated with the entry interface. This then means the packets are incorrectly addressed for the upstream connection.

My plan to address this had been as follows:

1. Mark the connection to the router to identify the entry network e.g. wan2-in, wan2-in
2. Route mark packets if a connection mark exists e.g. if connection mark = wan1-in apply a route mark wan2-out
3. Setup policy route to route according to the route mark to the correct gateway

The effect I see at the moment is as follows:

1. If I apply a packet mark for packets from a selected network the packets are marked
2. If I apply a connection mark the connections get marked
3. If I apply a route mark selecting packets with connection mark they do not appear to mark.

To confirm mark I look at the count in the magle rule and also apply a filter rule in input or output to simply log packets that have the relevant mark in place

WIth this in place the policy route does not apply. I can ping the router gateway but when I try to connect via ssh or winbox the connection fails unless I connect to the interface that contains the default gateway. This suggests the policy route is not being applied and packets are leaving via the default gateway.

I have searched the forums and find this has come up quite a few times. It is clearly a FAQ and common config. If I can get some information and conclusively establish a procedure for this I think it would be excellent to document this and submit to the wiki. I am prepared to do this if it helps others.

If anyone can offer any help I would be very grateful. It is one small quirk in what is otherwise an excellent router product. Thanks in advance for your time and attention - all help much appreciated!

tucker -

Can a packet have a connection mark, a packet mark and a route mark at the same time?
Can a packet have multiple packet marks?
Is a connection mark associated with all packets related to the connection?

yes
no - not at the same time. You can however use a packet mark to filter a packet again to give it a different packet mark....
yes, as long as you connection tracking on and the 'helpers' for the particular protocol (if there is one) turned on.


First you mark the connection, then mark the packet, and finally add the route mark.
Where are you SSH'ing from? If from the router itself you'll need a special rule to deal with that - otherwise it should work as 'advertised'.

Be careful how you decide to mark connections / packets - MT will ONLY mark what you tell it...sometimes the obvious escapes 'us' on this - like selecting an interface and an IP and then wondering why the count is so low...

tucker -

Can a packet have a connection mark, a packet mark and a route mark at the same time?
Can a packet have multiple packet marks?
Is a connection mark associated with all packets related to the connection?

yes
no - not at the same time. You can however use a packet mark to filter a packet again to give it a different packet mark....
yes, as long as you connection tracking on and the 'helpers' for the particular protocol (if there is one) turned on.


First you mark the connection, then mark the packet, and finally add the route mark.

That is what I had tried and did not manage to get it to work. However I was applying the mangle in the prerouting chain and from what I can see at least some of the marking can take place only in input/output chains.

Where are you SSH'ing from? If from the router itself you'll need a special rule to deal with that - otherwise it should work as 'advertised'.

I had been ssh from external host and connecting to each of the public interfaces. The one with def g/w worked and the others did not. I have no need to connect from the local host and prefer not to use the routing routers for anything other than routers. Exception is I have one installed in the data centre to manage other routers e.g. mac telnet (if I do something silly!), upgrades, ntp server etc.

Be careful how you decide to mark connections / packets - MT will ONLY mark what you tell it...sometimes the obvious escapes 'us' on this - like selecting an interface and an IP and then wondering why the count is so low...

I think this is where things are going wrong. I had been attemting to mark traffic entering on a specific interface and not being selective on the IP address etc. I am wondering if I want to control traffic to the router should I mark packets on interface and to the assigned interface address of the router. My flow was:

Mark on prerouting for connection from interface
Mark on prerouting for packet that has connection mark
Mark on prerouting for route mark to ensure egress packets related to connection will policy route

I am quite certain I am not marking something I should or I am marking in the wrong place but I cannot see why or where. Any help you can offer would be great!

tucker -

Input and Output chains are only for packets destined / sourced at the router itself.

That being said - see below.

Quote:
Be careful how you decide to mark connections / packets - MT will ONLY mark what you tell it...sometimes the obvious escapes 'us' on this - like selecting an interface and an IP and then wondering why the count is so low...


I think this is where things are going wrong. I had been attemting to mark traffic entering on a specific interface and not being selective on the IP address etc. I am wondering if I want to control traffic to the router should I mark packets on interface and to the assigned interface address of the router. My flow was:

Mark on prerouting for connection from interface
Mark on prerouting for packet that has connection mark
Mark on prerouting for route mark to ensure egress packets related to connection will policy route

Ok - looking at the above quote - you have two things going on...one you're trying to mark packects/connections to/from users, the second is packets/connections to/from the router itself. Mangling is VERY specific...as I said earlier it ONLY does what you tell it.... :)

So it looks like your first order of business would be to mark connections, then packets and add routing marks for the router itself. Then you can use policy routing... A little trick ChangeIP uses is to add a rule in the firewall under dst-nat, use each interface IP of the router, chain - dstnat, dst port 1701, protocol udp, action re-direct to ports 0-65535. This essentially 'creates' a static entry in the routing table for that IP address. This seems to really make a difference when you have to route connections/packets to/from the router itself.

Well - this should get you pointed in the right direction. Keep the forum posted as to your progress...

Thanks again for the comments and feedback ... this has indeed pointed me in the direction.

I have run some trials and reduced the configurations. I now have something that seems simple and works. This is what I have done:

1. Mangle on input chain and mark connections selecting on in interface and destination address as the address assigned to the rb within that network subnet. I mark connection as wann-in n=1,2,3, ..
2. Mangle on output chain and route mark selecting on connection mark. I look for connection mark wann-in and route mark as wann-out
3. In policy routing I add a route that looks for route mark wann-out and routes to relevant gateway with preferred source as the IP of the router within the subnet associated with the relevant next hop

All of this works perfectly. I have tested with 5 WAN connections and can ssh/winbox to the router on all 5 addresses. I can even have 5 SSH connections open to all 5 addresses from same source address. All work fine and the connection tracker shows the correct connection marks.

I am looking at a couple of questions and would appreciate any comments:

1. I do not mark packets and do not see the need to. Your previous posts (and others I looked at on forum) suggested the need to mark packets. The only reason I can see for this would be for protocols that do not establish connections.
2. On connection mark I am not selecting connection state. Does it improve efficiency in terms of CPU usage or packet forward rate if I only mark connections for new connections?
3. If I mark on the input chain I do not think I need to select based on the destination IP address as it is implict by entering the input chain that the packet is destined for a local IP address of the router

As soon as I finalise this config I will document it in the hope that it may help others. I guess the Wiki is the best place for this.

Finally, I am working on a standard configuration addressing issues that seem to crop up time and time again as we (and others) use RoS. I have several other questions but rather than hijack this thread I have opened two additional threads. I would appreciate any comments you may have as I would like to include these in the documentation I am working on.

tucker -

There was a very good explanation of packet / connection marking in one of the recent posts - I'll try and find it later and post the link for you.

Basically if you mark the connection the related connections get marked too, so packet marking the related and the original packets requires a lot less processing because the router already 'knows' these packets. If you just mark packets the the router has to check each and every packet coming / going....

Going forward...

1. I do not mark packets and do not see the need to. Your previous posts (and others I looked at on forum) suggested the need to mark packets. The only reason I can see for this would be for protocols that do not establish connections.
2. On connection mark I am not selecting connection state. Does it improve efficiency in terms of CPU usage or packet forward rate if I only mark connections for new connections?
3. If I mark on the input chain I do not think I need to select based on the destination IP address as it is implict by entering the input chain that the packet is destined for a local IP address of the router

1) Not exactly true - queues work better this way (packet marks), you get some very granular control over data with packet marks, this can be very handy as you grow. I would reccommend that as soon as you have a solid working config that you start looking at marking packets for the different traffic that you want to allow to pass through your router. You'll see why as your understanding grows....

2) Only if you have a very, very slow cpu, small memory, otherwise, anything above 300Mhz, not handling a wireless card directly, can do ethernet speeds.

3) Not necessarily - it could actually be an IP in the same network that could enter this chain...say the IP of the router is 192.168.1.1/24, technically anything in the 192.168.1.0/24 could conceivably enter here... Better to make sure that ONLY the IP you are interested in makes it this far....

Haven't looked at your other threads yet - getting busy here right now...

Thom

Thanks for the comments.

I can see the benefit in marking packets and how it allows much better control on queue. As part of my plan is now to shape the lines I will review packet marking.

I can see how other traffic can enter the router in input and I only want to mark and route unicast traffic to the router address so best to mark with destination ip address as part of selection.

It seems to be working quite well now and I am trying a few configuration changes to see how they affect things. Your help has been great - much appreciated. I will get all of this documented and summarised as soon as I am happy with it.

tucker -
Here is that link to packet / connection marking I said I'd find....

http://forum.mikrotik.com/viewtopic.php?f=2&t=19770

Thom

Thanks for the link ... much appreciated!

The connection mark/track seems to work well. A bit more testing and tidying and will report back exact configuration then document it.

Thanks again for all the help!