Wed Jan 17, 2018 12:03 am
OK. What has made me ask again was that your routing marks etc. are in contrast with what I expect. So I'll give you my complete description of how I would do it.
There is the "tunnel payload" - the packets that need to be delivered to destination through the VPN tunnel. For them, you may use whatever routing strategy you consider best (I have no opinion here as I had to google what ECMP means), but this strategy has to work with only the two tunnels as gateways. The tunnels are represented by virtual interfaces of type l2tp-client, which get IP addresses from the VPN server(s).
These payload packets are enciphered and encapsulated into "transport packets" - the UDP packets carrying ESP carrying L2TP carrying the payload.
However, the routing rules for the transport packets have to be completely different from those used for payload packets. I assume that the two physical connections to the internet are connected to two distinct Ethernet ports of your Mikrotik, each with its own IP address and gateway. Possibly one of them is used as the primary one for all generic traffic, not fitting to any of the two groups discussed above.
So to make sure that each of your tunnels' transport packets will be sent using a different physical channel regardless what else may happen, you need to provide two /32 routes, one per each remote server IP, each using the "default" gateway of a different one of the two uplinks as that route's gateway. If the uplinks assign your IP address dynamically and the "default" gateway indicated may change with each assignment, you'll have to use scripting to keep track with the changes (i.e. to re-set the gateway of the individual route for a VPN server with each DHCP assignment received as a copy of the default gateway received).
By creating these two routes you also ensure that any dynamically obtained default route (as provided by the L2TP servers configured with add-default-route=yes) does not cause the second tunnel to get established through the first one. As the transport packets are locally originated, no src-nat is needed for them as they should leave with the IP of the interface where the uplink is terminated as source. Unless the uplinks are something exotic, this should be automatic.
The next thing to solve is that you have a bunch of "default" gateways assigned dynamically. You have addressed this by setting different default-route-distance values to the two l2tp-client interfaces, but it means that any routing strategy is effectively disabled because routing will choose the default route with lowest distance value unless it is unavailable. And if also your uplinks feed you with dynamic default routes, you need to configure the DHCP clients to assign even lower distances to them, so that the routes received from l2tp always win for the "tunnel payload".
Now unless you do something about it, the winning default route for generic traffic will be the same like for the "tunnel payload".
If you want different routes to be chosen for the tunnel payload and generic traffic, you'll need to use scripting to set routing marks to be checked to the various default routes, and you'll need to use firewall rules to assign routing marks to different kinds of traffic. Normally, I would give the uplink-provided default routes a high distance (low priority) and no routing mark, while the tunnel-provided default routes would have higher priority but would only be available to tunnel-payload packets which have been marked with a routing mark by firewall mangle rules based on destination address of the service due to which you undergo all this exercise.
However, as the dhcp client settings allow to attach a script to the address assignment event, l2tp client does not. So you have to invert the philosophy - you need to assign low distance (high priority) and routing mark to the uplink-provided default routes, and assign the same routing mark to the generic packets and no routing mark to the tunnel payload packets. This way, the uplink-provided routes become unavailable for the tunnel payload, so it will use those provided by l2tp.
And, finally, as you already do, you also have to provide the srcnat rules for the tunnel payload, as the l2tp VPN provider expects only packet with source address they have assigned to you.