MikroTik

I have a site to site tunnel between A and B, running over a PPTP tunnel. Assume A has 3 networks:

1.0.1.0/24
1.0.2.0/24
1.0.4.0/24

and B has 3 networks

1.0.4.0/24
1.0.5.0/24
1.0.3.0/24

I see routes on A and B sending traffic for the other over the 'AB Tunnel' gateway. On A the routes to B are flagged DAS, while on B the routes to A are flagged AS. I think it means A's routes to B are 'static' routes added when that PPP user profile connected (which added the routes 'dynamically'), while B's routes were added in IP ROUTES (since B is the initiator of the PPTP tunnel).

Next, I have a second tunnel up between A and B using IKEv2. I want to move the routes from the PPTP tunnel to the IKEv2 tunnel. I assume that I just need to add some routes, but there is no gateway offered (in WinBox) for my IKEv2 tunnel when I try to add a route! How do I route (eg: x.0.3.0/24) traffic over the IKE tunnel?

Based on some posts I tried marking traffic on A destined for B with "IKETUNNEL" (in IP FIREWALL MANGLE), then I thought I could add that mark on the Mode Config. But, on A (the responder) there is no connection mark option. I see an "ADDRESS" field available on the responder side, but no where in the documentation can I find out what that means. I on the initiator side I think I could do connection marking. But it feel weird to use different ways to route traffic based on initiator (marking) or responder (??).

Based on some other posts I thought that perhaps my IP Sec policy template for the IKE tunnel connection should determine subnets to match and send over the tunnel. However, I cannot create a single CIDR address that matches A or B's subnets. As well, create a second template policy seems to be ignored when the tunnel is brought up, and switching it to non-template causes saving the policy to fail with error. I have Generate Policy set to 'port strict' on A, for the B connection. In case that matters

BTW, I work at A and don't want to drive to B, so I'm hoping there is a way to do this incrementally (disabling the PPTP tunnel temporarily on A) so I can test before destroying the PPTP tunnel/routes

In bare IPsec, there are no routes, there are just the traffic selectors of policies. So if you want networks A1, A2, A3 to freely talk to networks B1, B2, B3, you need 9 policies to cover all possible src/dst pairs. Which quickly turns into a prescription to a headache. You can allow one side to create the policies dynamically, but the other side must nevertheless provide suggestions, still all 9 of them.

So if you are lucky enough to have a public IP at both ends, or at least the public side IP address of the NAT behond which one of the peers is connected to be static, set up an IPsec-encrypted IPIP (or GRE) tunnel between them, and use regular routing. The overhead of IPsec in tunnel mode is the same like the overhead of an IPIP tunnel secured by IPsec in transport mode.

If you are not so lucky, you have to use a tunnel mode of the IPsec, add auxiliary static IP address(es) to link using an IPsec policy, and establish the IPIP tunnel between those static addresses. This will cost you one more IP header per transport packet, though. So it may be better so use IPsec enrypted L2TP which can handle the NAT at the client side natively, and has the advantage of hiding any MTU issues by splitting and reassembling the payload packets at MLPPP level rather than fragmenting the transport packets on IP level - the latter is proven to cause issues on some networks that randomly drop fragments.

If you opt for the policies despite the warning, bear in mind that a statically configured policy intercepts the matching traffic even if the corresponding security association is not established, so if the traffic selector matches on the traffic you normally route via the PPTP, you'll have to drive. So it is better to let Site B generate policies dynamically according to the static ones at Site A - in that case, disabling the policy at site A destroys it at site B. Security-wise, to prevent traffic leakage via the default gateway if the tunnel goes down, it is better to have static policies at both sites, or to use a route to nowhere for the traffic (IPsec will "save" the traffic while up).

The mikrotiks A and B are both edge routers/firewalls, and both have public IP's which are dynamic.

I've been reading about the various options you suggest (balancing my need to minimize an already steep learning curve). I want to avoid learning a technology on the way out, and according to this document https://atlasvpn.com/blog/what-is-l2tp- ... col-to-use L2TP+IPsec is outdated and has been replaced with newer protocols like IKEv2. Using IPsec+GRE is interesting, and based on the howto guide (https://wiki.mikrotik.com/wiki/Manual:Interface/Gre ) is pretty simple to setup (and I assume since both ends have dynamic IP the tunnel can come up by specifying only the fqdn of the remote address).

Out of interest I still would like to learn how to do a pure IKEv2 tunnel. I would appreciate a bit more info on your suggestions:

1. Add auxiliary static IP addresses to A and B: Both routers already have 3 static internal/non-routable IP's each. Can I use those? If not, I know how the responder (A) can assign B a static IP, but do I need to create another interface on A and assign it another static IP? I haven't figure out why this is necessary, but I suspect it has to do with #2 below

2. Multiple policies for an identity: I'm having trouble with implementing this. I tried creating multiple policy templates on A in the group associated with my tunnel, but when B connects no policies are generated. I suspect because the dst address I'm setting (1.0.3.0/24) does not match/overlap the dynamic address given to B's end of the tunnel. But not sure.

If I try to create a (non-template) policy it always refuses to save.
Couldn't change IPsec policy <0.0.0.0/0;0->1.0.3.0/24:0> cannot convert template to policy (6)
Am I missing something obvious here?

I will keep GRE related questions out of this thread. I'm hoping this thread might help others with IKEv2 only related issues.

The mikrotiks A and B are both edge routers/firewalls, and both have public IP's which are dynamic.
...
I've been reading about the various options you suggest

GRE is even more outated than L2TP; IKE (v1) which is used as part of the standardized L2TP/IPsec combo is kind of outdated too but since both ends are under your administration, nothing prevents you from ignoring that standard and using IKEv2 to encrypt the L2TP (or GRE, or IPIP). The clear advantage of L2TP is the way it substitutes IP level fragmentation by a MLPPP-level one. Nothing else (bare IPsec, something-over-IPsec) has this "magic power" of not running into MTU/fragmentation issues.

And yes, if you have dynamic addresses but there is no NAT between the devices, you can use fqdn as the remote-address of IPIP or GRE tunnel destination, as well as as the address of an IPsec peer.

I just remember I had some surprising behaviour on reboot where an IPsec peer had a non-existent fqdn as address, as it has shadowed other peers for some reason - I don't remember the details but it was definitely wrong, normally the peers are automatically sorted by prefix length of the address, but this one ended up being the first one but catching all incoming requests nevertheless.

Out of interest I still would like to learn how to do a pure IKEv2 tunnel. I would appreciate a bit more info on your suggestions:

1. Add auxiliary static IP addresses to A and B: Both routers already have 3 static internal/non-routable IP's each. Can I use those? If not, I know how the responder (A) can assign B a static IP, but do I need to create another interface on A and assign it another static IP? I haven't figure out why this is necessary, but I suspect it has to do with #2 below

2. Multiple policies for an identity: I'm having trouble with implementing this. I tried creating multiple policy templates on A in the group associated with my tunnel, but when B connects no policies are generated. I suspect because the dst address I'm setting (1.0.3.0/24) does not match/overlap the dynamic address given to B's end of the tunnel. But not sure.

If I try to create a (non-template) policy it always refuses to save.
Couldn't change IPsec policy <0.0.0.0/0;0->1.0.3.0/24:0> cannot convert template to policy (6)
Am I missing something obvious here?

The auxiliary static addresses were necessary to deal with the dynamic IP of an external NAT, so that the IPIP or GRE tunnel could use a fixed remote-address; if you don't mind to rely on dynamic DNS updates (and you've got no other choice anyway) and there is no NAT, they are not needed.

With bare IPsec, you don't need them either - the policies may refer only to the subnets on both sites you actually want to make reachable to each other.

Policies always refer to a peer as a static configuration item, but their sa-dst-address is dynamically set to the remote-address taken from the active-peer data (it used to be different but it works this way for at least two years now). So if multiple initiators connect to the same local responder, policies for each of them can be dynamically created and get the correct sa-dst-address.

To let policies be dynamically created, you need to set generate-policy to something else than no on the identity row representing the remote peer; if you want to restrict the templates only to a few allowed ones and/or indicate a distinct proposal to be used rather than the default one, you have to add a policy template group, mark the newly added templates as belonging to that group, and set its name in the policy-template-group on the identity row.

Regarding the error message you get, I have no idea how you got there. If you choose to edit an existing template, it makes sense; if you choose to add a new policy from scratch, I cannot see why such a message should appear.

So Site A would look as follows:
/ip ipsec peer
add name=siteB address=siteb.ocg-ltd.ca exchange-mode=ike2 profile=...
add name=roadWarriors exchange-mode=ike2 passive=yes profile=...

/ip ipsec identity
add peer=siteB auth-mode=... generate-policy=no ...auth mode specific parameters...

/ip ipsec policy
add peer=siteB src-address=a1.sub.net/mask dst-addrtess=b1.sub.net/mask tunnel=yes proposal=...
...(7 more rows)
add peer=siteB src-address=a3.sub.net/mask dst-addrtess=b3.sub.net/mask tunnel=yes proposal=...

(bear in mind that if the road warriors connect to Site A but have to access subnets at Site B, you need even more policies)

During the debug phase, you want the policies at Site B to be generated dynamically, hence:
/ip ipsec peer
add name=siteA address=sitea.ocg-ltd.ca exchange-mode=ike2 profile=...

/ip ipsec policy group add name=forSiteA

/ip ipsec policy add group=forSiteA template=yes proposal=...

/ip ipsec identity
add peer=siteA auth-mode=... generate-policy=port-strict policy-template-group=forSiteA ...auth mode specific parameters...

Interesting, when I add a new policy via WinBox is creates a template by default that is switched to static policy and it refuses to apply. But using command line the policy was accepted. Maybe relates to the next sentence...

I already had a proposal and peer created, so I started by adding just a the new policy. But the policy immediately appears in red (invalid), with no explanation why. I printed all policies below. Can you explain why the new policy (9) is invalid?

And a related question, if I used 0.0.0.0/0 as my source, would that shortcut allow all my subnets in A to route to that subnet in B ? Or does that cause a selector issue, and I must use a specific subnet.

You're right that roadwarriors will want access to site B too (through A), so this may be getting out of hand unless I can use 0.0.0.0/0 as my source.

Can you explain why the new policy (9) is invalid?

On the command line, the policies are marked as I(nvalid) if the peer they are linked to is disabled. In Winbox, the I is not shown but instead they appear in red. Other than that, policy 9 is not shadowed by any of the previous ones (except the templates, which aren't relevant until some actual policies are eventually created from them). But RouterOS sometimes does behave weird when adding policies. So if the peer is enabled, try to move the policy around a bit or remove it and create it again.

And a related question, if I used 0.0.0.0/0 as my source, would that shortcut allow all my subnets in A to route to that subnet in B ? Or does that cause a selector issue, and I must use a specific subnet

The selectors are negotiated between the peers. So if you have src-address=0.0.0.0/0 and, say, dst-address=192.168.233.0/24 in a statically configured policy at site A, there must be a matching (mirror) policy at site B (a manually configured one or one created dynamically from a template). And if it is there, all the traffic of 192.168.233.0/24 at site B, including the one sent by the router itself to other devices in that subnet, is diverted to the tunnel to site A. The SA for the policy is not established if an exactly matching policy cannot be found or dynamically created at the remote peer. A template may be narrowed down to match the peer's offer but not extended; a statically configured policy cannot be adjusted to the peer's offer in any way.

Ok - yes I had disabled that peer because WinBox showed it as "This entry is unreachable". I suspect it is because peer #1 has address "::/0", the same as my Site B peer #2 below. Peer #1 is meant for road warriors connecting from anywhere. But since the address is the same as #2 I'm guessing no connection attempts will ever reach peer #2 ??

Does this mean you cannot have two IPsec peers with ::/0 addresses? Do I therefor have to use a single generic peer (::/0) and let the remote/my ID distinguish the remote peers later on? I tried that and associated my policy #9 with peer #1, but the policy still shows as invalid

I have a feeling I will end up creating an L2TP+IPsec tunnel in the end, but I don't wan't to give up on this until at least I have figured out a single subnet tunneled between sites.

Does this mean you cannot have two IPsec peers with ::/0 addresses?

Indeed, you cannot have multiple peers with identical (address, local-address, exchange-mode) tuple, as these three parameters are used to choose the best matching peer; if multiple peers with all these parameters identical exist, all incoming requests are handled by the first one.

Do I therefore have to use a single generic peer (::/0) and let the remote/my ID distinguish the remote peers later on?

This only works if the policies are created dynamically. You cannot link a static policy to a peer (as in configuration object) that represents multiple actual peers (remote initiators), i.e. to any peer with something else than a single IP address or fqdn as address.

So set address=masonville.ocg-ltd.ca passive=no at "Site B Tunnel Peer", hook the policy to it, and see what happens.

Ok, peer is now reachable, and new policy is associated with that peer is valid! I'm trying understand what I just did by disabling passive at the peer level. (Since I want site A to be responder, which I thought was like passive)

According to doc:

When passive mode is enabled will wait for remote peer to initiate IKE connection. Enabled passive mode also indicates that peer is xauth responder, and disabled passive mode - xauth initiator. When passive mode is disabled peer will try to establish not only phase1, but also phase2 automatically, if policies are configured or created during phase1.

I thought phase 2 always follows phase 1 automatically, so that part of the explanation doesn't mean much to me. But as s for initiating the IKE connection, I thought this is what the "responder" checkbox is for in the mode config? So peer is now passive, but Mode Config associate with the identity

If I just made site A the initiator, won't I encounter the same problem on site B (can't create a responder or will encounter the same issues as above)?

Is a solution to make the policies generate dynamically - assuming I can create 3 template policies for the seem peer...

The passive parameter should actually say passive-only - if it is set to no, the peer both attempts to actively initiate a connection towards the actual peer specified by address and responds to incoming connections from that peer (if any arrive). You cannot have an "initiator-only" peer.

And I was wrong when saying that you cannot link a statically configured policy to any peer with passive=yes - the statically configured policies are only marked as invalid if the address of their peer is not a single IP address or a hostname. However, only peers with passive=yes can have a subnet as address.

When you configure both the peer representing Site B at Site A (as you've just done) and the peer representing Site A at Site B, they will both initiate a connection, and they may even both succeed; one of the connections will eventually die off. An initiator without any statically configured policy will not initiate Phase 2 if I remember properly. So if you want Site B to create policies dynamically from templates, in order to be able to fall back to PPTP if something goes wrong, you cannot make Site A a responder-only one.

Now the identity for Site B (on site A) says Wrong mod-config. I tried editting the mode config but can't find what options need to be turned on/off to make the mode acceptable. What is it looking for?

A few things are still unclear...

1. What is the point of having a responder check box in the Mode Config, if the peer definition determines whether the peer initiates or responds? (Maybe I don't understand how Mode Config is used)

2. If I disable Responder flag in the mode config for my tunnel, I suddenly have access to 'connection mark'. Doesn't that eliminate the need for multiple policies since I can just mark packets with mangle, and tell my local site to forward only marked packets? (Going in a bit of a circle here)

3. So the steps upon incoming connection are:
a. match remote peer by address/local address/exchange mode tuple
b. verify peer by identity (certs/id)
c. send mode config to remote peer if local is a responder, otherwise setup connection mark/src address list
b. activate policies in group applied to the remote peer

(My head is spinning...this is proving way harder than I thought it would be.)

You only need to refer to a mode-config row on the identity row if you are an initiator asking for an IP address to use and list of remote subnets to access (split-include), or when you are a responder providing this information to the initiator. For a site2site VPN, none of these is necessary (although it may be useful to have all your remote routers manageable using addresses assigned by means of mode config and use statically configured policies only for the external traffic).

The responder item of the mode-config row determines whether that row will be used for the initiator role of for the responder one (remember you cannot configure an initiator-only peer by means of the passive parameter).

If used for an initiator peer, and if the values of connection-mark or src-address-list are specified, src-nat rules are dynamically created at the very top of the srcnat chain once the connection is established. The values of these fields are used as match conditions in these rules, causing any matching traffic to be src-nated to the address assigned by means of mode config, making it match the traffic selector of the dynamically created IPsec policy later on. The dst-address of these rules are taken from the received split-include list, so you end up with as many src-nat rules added as there are subnets on that list.

So the traffic selectors do not directly match on connection-mark; it is just this indirect relationship via source NAT which makes traffic marked with that connection-mark be diverted to the security association, the price being that the recipient doesn't know the actual source address. And if you eventually need to support incoming connections to devices in LAN subnets of such an initiator router to be set up via the tunnel, you have to use dst-nat (port forwarding) rules.

If used for a responder peer, the other items are relevant - address, address-pool, address-prefix-length, split-dns, split-include, static-dns, system-dns, use-responder-dns, all of those being reasonably self-explaining names, some of them even detailed in the manual. The only exception is the address-prefix-length for which I haven't noticed any effect and never got a response to my question from support.

3. So the steps upon incoming connection are:
a. match remote peer by address/local address/exchange mode tuple

correct

b. verify peer by identity (certs/id)

actually, not just verify - since the peer configuration item may represent multiple actual peers, you actually identify and authenticate the actual peer.

c. send mode config to remote peer if local is a responder, otherwise setup connection mark/src address list

correct, but only if configured that way. For site2site this step is rarely needed, as stated above.

d. activate policies in group applied to the remote peer

correct, also known as phase 2. But group is only related to dynamic creation of policies. Static policies are fully specified by the (peer, traffic selector, proposal) tuple; groups are a way to link templates to particular actual peers.

OK - I've pulled out a lot of hair on this IKEv2 tunnel experience. ENough to realize when to pull the plug. I'm going to switch to an L2TP tunnel at this point, and then see if I can add IPsec to it after it's working. (Which seems to be just a checkbox and a PSK).

I appreciate all of the help! Good learning experience....but I also wonder if the UI for IKEv2 needs to be rethought by Mikrotik to make it more accessible to less experience admins.

I want to avoid learning a technology on the way out, and according to this document https://atlasvpn.com/blog/what-is-l2tp- ... col-to-use L2TP+IPsec is outdated

You need to understand that the writer of that article is discussing a completely different scope and has a completely different objective than you have.
The name "VPN" today is often used for a service like that providerd by the company of the author, that provides a secure tunnel for a single user towards a service where all his traffic will be routed to/from the open internet.
What you are building now is the traditional usage of a VPN: to provide a secure tunnel between two closed networks over the public internet.
That has different requirements and different issues/problems, as you have noted by now.

It is like reading a paper written by a salesman from an air cargo company declaring that trains and ships are completely outdated and will all be replaced by planes. Which he happens to have.
But that is not reality, each transport has its advantages and disadvantages and there is no sign that ships are going away even when planes are faster and "more modern".

You're right - that doc was from a party with a particular interest in steering its users towards a particular solution.

After hours of more reading it seems like IPsec+L2TP (as suggested by Sindy) is both current and the most effective. I found little on pure IKEv2 site-to-site usage, but based on the policy headache I encountered I think this isn't suitable for sites with n subnets due to n^2 policies which need to be created (and other complexities). I'm still going to use pure IKEv2 for road warrior VPN, but switched to IPsec+L2TP for site to site.

I got the L2TP+IPsec up and running in hours, after spending days on pure IKEv2 for my site-to-site tunnel. I good warning for other intermediate network admins....

I think that is a good idea, yes. Any plain IPsec tunnel which transports between two locations with multiple networks, or multiple locations with a single network, will be a headache.
Such mechanisms are good for "single IP towards other network" usage like roadwarrior users and also for the type of VPN that communicates between a user and the open internet, but not for a complicated network.

When using L2TP/IPsec or other tunnel protcols (like IPIP/IPsec or GRE/IPsec) you only define the endpoints of the tunnels and not the type of traffic that can pass through it.
Thus you can separate those layers, as is an absolute must to keep clarity in what is really happening. Both in the routing and in the firewalling.

In a "star" network where the leafs have dynamic addresses, L2TP/IPsec is a good choice on MikroTik.
When your network is a little more complicated than what you sketched, consider using a routing protocol on top of it (e.g. BGP).
Easy to configure on RouterOS, especially in version 6, and it reliefs you of setting up all the static routes and changing them whenever the topology changes.
(e.g. when you add extra sites)