Hi,

for a while now I've been trying to find a good setup in ROS to cover our needs (redundant VPN with policy routing/failover via Multi-WAN, details are not too important). While I have some generally working solution, I would prefer another one, but I'm stumbling about one certain problem again and again.

I will give an example for what I would like to do, and I do on purpose not just post a config, as I intend to show a more abstract picture, not just focus on 1 special config.

Connection from LAN should be routed through one of 2 GRE tunnels, and as different type of traffic should go through different tunnel, use Routing-Marks.

So I do the following: Packet "P" from PC to Router -> mangle Prerouting, mark connection "CON1"-> mangle prerouting, mark routing "RM1" if connection mark = "CON1"
Add route "R1" with routing-mark "RM1", ping check, distance 1 -> gateway1
Add route "R2" with routing-mark "RM1", ping check, distance 2 -> gateway2
Add route "R3" with no routing-mark, ping check, distance 3 -> gateway2
Add route "R4" with no routing-mark, ping check, distance 4 -> gateway1
(Dst-address is always same, gateway is same for R1/R4, and for R2/R3 ... so traffic type with RM1 would be preferably routed via gateway1, all other traffic would preferrably be routed via gateway2)


Now when I look at this:

http://wiki.mikrotik.com/wiki/File:IP_final.png

I would expect the after pre-routing, when the routing decision is done, it would match the route with best match, which should be R1 ... at it matches the dst-address AND the routing mark AND shortest distance.
However, when I then log the packet "P" in forward chain, it will show that the out-interface is gateway2. So it selected "R3". Even worse , if I disable R3 and R4, it will not work at all, which shows to me that the "Routing decision" after "PreRouting" does care about routing marks. And after that point i don't see any chance to force "P" into the correct tunnel.
Am I doing anything wrong, am I misunderstanding something, or this even a bug? Are there better ways to achieve this kind of policy routing?

Thank you in advance for your answers.

I may have to add something ...

after doing config reset on my test MetaRouters, I applied my test scenario fresh from start, and this time it seemed to work.

But then I applied IPSec tunnel policy, and here is where it starts to act up
(I swear i had the problem before even without IPSec, and not only in one configuration ... but maybe something got screwed up throughout my tests , who knows)

So what I see now is ... if IPSec policy is active, the log entry in the forward mangle chain will show the wrong out-interface, even though IPSec processing should follow only later in the packet flow. And I feel here is is where my trouble starts. Because then after IPSec is done, it won't change the out interface anymore , even though the routing mark seems to survive the IPSec encapsulation (log in mangle chain Output, filter routing mark, the IPSec packet is logged ... but the out interface does not get adjusted afterwards)

So maybe my question should be rephrased to: how can I do selective policy routing and combine that with IPSec?

My main idea for this looks like:

Traffic A GRE-TUNNEL1 -> ISP1
\ /
IPSec
/ \
Traffic B GRE-TUNNEL2 -> ISP2

(Failover must be possible for both A & B )

The GRE tunnels are because I have dynamic IPs in many locations, and dyndns is less compicated to do for tunnels then for IPSec.

Looking forward to any input you can give