Setting up IKEv2 VPN Server behind NAT

danergo · Wed Sep 22, 2021 12:11 pm

Folks,

My Tik is behind NAT (due to ISP), and it gets an internal IP from them.
I want to setup an IKE2 VPN Server on this Tik to allow my Android phone to be part of the LAN.

Phone (192.168.0.10) ----> ISP1 (192.168.0.1, 4.3.2.1) -----> INTERNET -----> ISP2 (8.7.6.5, 192.168.1.1) -----> TIK (192.168.1.10) (192.168.2.1)

I have followed a tutorial from internet, with some adjustments. I'm currently on version 6.46.6.

These were my steps:

1.) Create the CA certificate which will validate all incoming connections

###Replace common name with Public DNS name of VPN appliance
/certificate add common-name="vpn.yourdomain.com Root CA" name=ca days-valid=3650
/certificate sign ca ca-crl-host=192.168.2.1

I'm unsure here about "ca-crl-host". Shall it be 192.168.2.1 or 192.168.1.10 (i.e. Tik's LAN or WAN IP)?

2.) Create and sign VPN Server's certificate

/certificate add common-name=vpn.yourdomain.com key-usage=tls-server name=vpnserver1 days-valid=3650
/certificate sign vpnserver1 ca=ca

3.) Create and sign a client certificate

/certificate add common-name=android key-usage=tls-client name=android days-valid=3650
/certificate sign android ca=ca

4.) Create a new bridge for VPN connections to make it easier to manage with custom rules

/interface bridge add name=vpn-bridge
/ip address add interface=vpn-bridge address=192.168.200.1/24 comment="VPN Bridge IP"

5.) Create a new pool from which VPN clients can get their IPs

/ip pool add name=vpn-pool ranges=192.168.200.100-192.168.200.200

6.) Create a new IPSec entity (ModeConfig, Proposal, Profile, Peer, Policy, Identity):

#Add mode config to use our pool with VPN Bridge's IP as DNS
/ip ipsec mode-config add 
address-pool=vpn-pool address-prefix-length=32 name=vpn-config system-dns=no static-dns=192.168.200.1

/ip ipsec proposal add 
name=vpn-clients auth-algorithms=sha1,sha256 enc-algorithms=aes-128-cbc,aes-256-cbc pfs-group=modp2048

/ip ipsec profile add 
dh-group=modp1024,modp2048 enc-algorithm=aes-256 hash-algorithm=sha256 name=vpn-profile

/ip  ipsec peer add 
name=vpn-peer address=0.0.0.0/0 exchange-mode=ike2 passive=yes profile=vpn-profile

/ip ipsec policy group add 
name=vpn

/ip ipsec policy add 
template=yes src-address=0.0.0.0/0 dst-address=192.168.200.0/24 proposal=vpn-clients group=vpn

/ip ipsec identity add 
peer=vpn-peer auth-method=digital-signature certificate=vpnserver1 generate-policy=port-strict mode-config=vpn-config policy-template-group=vpn

7.) Export certificates from Tik, and import into Android (successful), I can share these details, but no errors were happened, so let me know if needed.

8.) Try connecting: with vpn.yourdomain.com as address, and with CA and "android" client certificates. Android says "Unsuccessful" after 10-15seconds.

Extra info: I have forwarded UDP ports 500 and 4500 in ISP2 Router (to 192.168.1.10).

I have no clue on why it fails, can you please help me with either just looking into my config (I might misconfigured something), either with some tips on debug, or on what to check, what to see in Tik while android is trying to connect.

Thank you very much.

sindy · Wed Sep 22, 2021 7:37 pm

Everything you wrote seems fine to me. What you haven't written is how exactly did you export the client certificate for the Android and what are the properties of the Mikrotik's certificate.

If the client certificate is not created the proper way, where the client creates a certificate signing request and the CA signs that request, but it is instead completely created outside the client as you did, the export of the certificate must contain also the private key to it. And in order that it did, you must enter a passphrase when doing the export-certificate. Plus if you choose type=pem, the certificate itself and the private key are in separate files; if you choose type=pkcs12, a single file contains the certificate itself, the private key to it, and all the intermediate certificates up to the root CA's one included.

So the Android may find out it doesn't have a private key when it attempts to use the certificate. Or it may check the key-usage and not be happy with it (Mikrotik as an IPsec responder only looks for the tls-client bit but the Android may require something else). Or the Android may check Subject-Alt-Name rather than Common name in the responder's certificate (Strongswan does this). Or there may be some other issue in the communication between the two, which you might see in the log if you set /system logging add topics=ipsec,!packet, run /log print follow-only file=ipsec-startup where topics~"ipsec", then make one connection attempt, then break the /log print ..., download the file ipsec-startup.txt, and start reading it.

mbaute · Thu Sep 23, 2021 2:05 am

Hi danergo,

Try setting My ID type as address in your Identity config and put as value in My ID your public IP assigned to tik (i.e 8.7.6.5).

/ip ipsec identity add 
peer=vpn-peer auth-method=digital-signature certificate=vpnserver1 generate-policy=port-strict mode-config=vpn-config policy-template-group=vpn my-id=address:8.7.6.5

Regards,

danergo · Thu Sep 23, 2021 3:13 pm

mbaute:

"Public IP assigned to Tik": 8.7.6.5 is assigned to ISP's router, not to Tik.

Tik is behind this ISP's router, and it gots an internal IP assigned by ISP's router.

Is your suggestion considered this fact?

Thank you!

danergo · Thu Sep 23, 2021 5:26 pm

Sindy:

I have exported the certificates (both ca and both the client) from winbox, in PKCS format, and providing a passphrase.
Then I've sent over those two files onto my android phone, and then I have opened them, my phone asked me the passphrase, then certificate type ("VPN and app user certificate" or "Wifi certificate"), I have chosen the first one, then asked a name, entered "ca", and "client", then said import was successful.

After that, I have setup a new connection on Android, with these settings:

Then when tried connecting, android says "Unsuccessful".

I have done the log you suggested (thank you!):

I have replaced the actual values to the IP addresses I used here on top for demo purposes:

16:19:36 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[4500] 
16:19:36 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[4500] 
16:19:36 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[1025] 
16:19:36 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[1025] 
16:19:56 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[4500] 
16:19:56 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[4500] 
16:19:56 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[1025] 
16:19:56 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[1025] 
16:20:06 ipsec sending dpd packet 
16:20:06 ipsec <- ike2 request, exchange: INFORMATIONAL:574 4.3.2.1[4500] SOMEMAGICSTUFF 
16:20:06 ipsec,debug ===== sending 112 bytes from 192.168.1.10[4500] to 4.3.2.1[4500] 
16:20:06 ipsec,debug 1 times of 116 bytes message will be sent to 4.3.2.1[4500] 
16:20:06 ipsec,debug ===== received 96 bytes from 4.3.2.1[4500] to 192.168.1.10[4500] 
16:20:06 ipsec -> ike2 reply, exchange: INFORMATIONAL:574 4.3.2.1[4500] SOMEMAGICSTUFF 
16:20:06 ipsec payload seen: ENC (68 bytes) 
16:20:06 ipsec processing payload: ENC 
16:20:06 ipsec,debug => iv (size 0x10) 
16:20:06 ipsec,debug SOME OTHER MAGIC STUFF 
16:20:06 ipsec,debug => plain payload (trimmed) (size 0x0) 
16:20:06 ipsec,debug decrypted 
16:20:06 ipsec respond: info 
16:20:06 ipsec,debug reply ignored 
16:20:16 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[4500] 
16:20:16 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[4500] 
16:20:16 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[1025] 
16:20:16 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[1025] 
16:20:36 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[4500] 
16:20:36 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[4500] 
16:20:36 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[1025] 
16:20:36 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[1025]

sindy · Thu Sep 23, 2021 6:11 pm

I have exported the certificates (both ca and both the client) from winbox, in PKCS format, and providing a passphrase.

You should not have exported the CA certificate using the passphrase, as the private key of the CA should never leave the CA. But this is "only" a security issue (anyone having the CA certificate together with its private key can sign additional certificates that will be trusted by whoever trusts that CA), it does not explain why the connection attempt fails.

The log shows two ongoing IKE sessions, one from remote port 4500 and another one from remote port 1025. Do you have any other client than the Android successfully connected from behind ISP1?

In any case, the log is far too short and nothing resembling the initial establishment packet of the IKEv2 connection is visible there. So either nothing has arrived from the Android, or you haven't started the log sufficiently earlier than you've taken the connection attempt. As the log spans a minute, I rather suspect nothing has arrived from the Android.

So I'd say wait for 15 minutes since the last attempt taken from the Android, then try to log the startup again.

erkexzcx · Thu Sep 23, 2021 9:49 pm

viewtopic.php?f=23&t=175656

You're welcome

sindy · Thu Sep 23, 2021 10:12 pm

@erkexzcx, if I understand it correctly (no own experience so far), latest Androids support IKEv2 natively, i.e. you don't need to install Strongswan. The screenshots from the OP suggest that that's what he's dealing with here - the graphics does not resemble the one of the Strongswan app.

mbaute · Fri Sep 24, 2021 1:01 am

mbaute:

"Public IP assigned to Tik": 8.7.6.5 is assigned to ISP's router, not to Tik.

Tik is behind this ISP's router, and it gots an internal IP assigned by ISP's router.

Is your suggestion considered this fact?

Thank you!

Yes no problem with that, take it as a form of "extra identification" for IPsec, as -because you're natted-, SAs replying from tik are being sent with your private IP 192.168.1.10. With that ID you're telling: although my SA tells 192.168.1.10, reply to 8.7.6.5. At least this works on tiks in Google Cloud, where it is always a 1:1 NAT scenario.

Regards,

danergo · Fri Sep 24, 2021 8:51 am

You should not have exported the CA certificate using the passphrase, as the private key of the CA should never leave the CA. But this is "only" a security issue (anyone having the CA certificate together with its private key can sign additional certificates that will be trusted by whoever trusts that CA), it does not explain why the connection attempt fails.

I know, yes, I wanted to avoid any possible blocking to first "just" make it work, then finetune and make it secure (generating new certificates, and changing addresses, etc).

The log shows two ongoing IKE sessions, one from remote port 4500 and another one from remote port 1025. Do you have any other client than the Android successfully connected from behind ISP1?

Sindy, I also have a persistent session from another Tik, which we did together long time ago: viewtopic.php?t=160805.
This is surely working on UDP4500. I have no idea about UDP1025, this is not forwarded on ISP's router to Tik.
In my earlier log Android and the persistent session were behind the same ISP (which wouldn't make too much sense and completely out of my scope) - sorry for this.

I have recreated the logs when Android is using LTE, and now it's much larger, it does a lot of things. After ~160lines, it ends with this error:

07:36:53 ipsec payload seen: NOTIFY (8 bytes) 
07:36:53 ipsec create child: initiator finish 
07:36:53 ipsec processing payloads: NOTIFY 
07:36:53 ipsec   notify: TS_UNACCEPTABLE 
07:36:53 ipsec got error: TS_UNACCEPTABLE

So it seems Traffic Selector has some issues with me, however, it seems IKE2 request and reply is going back and forth between "4.3.2.1" and "192.168.1.10" (indicated with my original network demo IPs).

viewtopic.php?f=23&t=175656

You're welcome

Thanks. Android version 11 and above supports IKEv2 natively, no need to install Strongswan or any other 3rd party utility.

sindy · Fri Sep 24, 2021 9:44 am

we did together long time ago: viewtopic.php?t=160805.

The notification about your update to that thread from this July never made it to my mailbox. I assume you've resolved it?

I have no idea about UDP1025, this is not forwarded on ISP's router to Tik.

it's a source port from the perspective of the responder, so it need not be port-forwarded neither at the responder side nor at the initiator one. It's just that the src-nat in netfilter (used by RouterOS and many other Linux devices) prefers to use the original source port unless that would cause packets belonging to two distinct src-nated connections to be undistinguishable from each other at the WAN side because they'd use the very same remote address:remote port and local address:local port. So the first IPsec connection from 4.3.2.1 to 8.7.6.5:4500 is src-nated to 4.3.2.1:4500, whereas the second one to 4.3.2.1:1024, the third one to 4.3.2.1:1025 etc. The first IPsec connection to somewhere else than 8.7.6.5:4500 can be src-nated to 4.3.2.1:4500 again as the remote address:port is sufficient to distinguish it from the connection 4.3.2.1:4500 -> 8.7.6.5:4500.

So now as I've seen the actual reason why the Android connection fails to establish, I can reasonably assume that the connection from :1025 was the Android one. I'm not sure whether TS_UNACCEPTABLE breaks the whole session or only the Phase 2 whereas Phase 1 remains up. So it is well possible that the Mikrotik kept sending keepalives within the Phase 1 SA although Phase 2 SA failed to establish, whereas Android has reported an error due to the Phase 2 failure.

So it seems Traffic Selector has some issues with me, however, it seems IKE2 request and reply is going back and forth between "4.3.2.1" and "192.168.1.10" (indicated with my original network demo IPs).

Exactly, the authentification has succeeded, otherwise the process would not get as far as to Phase 2 initiation. However, I'd have to see the complete log to say whether the TS suggested by the responder was eventually wrong in some way (not that there was too much to go wrong in a TS itself).

danergo · Fri Sep 24, 2021 12:38 pm

The notification about your update to that thread from this July never made it to my mailbox. I assume you've resolved it?

No problem at all, it was some ISP error, it actually has resolved itself after some days (I was far away, couldn't reboot anything)

So it seems Traffic Selector has some issues with me, however, it seems IKE2 request and reply is going back and forth between "4.3.2.1" and "192.168.1.10" (indicated with my original network demo IPs).
Exactly, the authentification has succeeded, otherwise the process would not get as far as to Phase 2 initiation. However, I'd have to see the complete log to say whether the TS suggested by the responder was eventually wrong in some way (not that there was too much to go wrong in a TS itself).

Here is the complete log:

07:36:41 ipsec ph2 possible after ph1 creation 
07:36:41 ipsec init child for policy: 0.0.0.0/0 <=> 192.168.200.0/24 
07:36:41 ipsec init child continue 
07:36:41 ipsec offering proto: 3 
07:36:41 ipsec  proposal #1 
07:36:41 ipsec   enc: aes256-cbc 
07:36:41 ipsec   enc: aes128-cbc 
07:36:41 ipsec   auth: sha256 
07:36:41 ipsec   auth: sha1 
07:36:41 ipsec   dh: modp2048 
07:36:42 ipsec adding payload: NONCE 
07:36:42 ipsec,debug => (size 0x1c) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec adding payload: KE 
07:36:42 ipsec,debug => (first 0x100 of 0x108) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec adding payload: SA 
07:36:42 ipsec,debug => (size 0x48) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec initiator selector: 0.0.0.0/0 
07:36:42 ipsec adding payload: TS_I 
07:36:42 ipsec,debug => (size 0x18) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec responder selector: 192.168.200.0/24 
07:36:42 ipsec adding payload: TS_R 
07:36:42 ipsec,debug => (size 0x18) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec <- ike2 request, exchange: CREATE_CHILD_SA:4474 4.3.2.1[1025] SOMEMAGICS
07:36:42 ipsec,debug ===== sending 544 bytes from 192.168.1.10[4500] to 4.3.2.1[1025] 
07:36:42 ipsec,debug 1 times of 548 bytes message will be sent to 4.3.2.1[1025] 
07:36:42 ipsec,debug ===== received 240 bytes from 4.3.2.1[1025] to 192.168.1.10[4500] 
07:36:42 ipsec -> ike2 reply, exchange: CREATE_CHILD_SA:4474 4.3.2.1[1025] SOMEMAGICS
07:36:42 ipsec payload seen: ENC (212 bytes) 
07:36:42 ipsec processing payload: ENC 
07:36:42 ipsec,debug => iv (size 0x10) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug => plain payload (trimmed) (size 0x8) 
07:36:42 ipsec,debug SOMEMAGICS
07:36:42 ipsec,debug decrypted 
07:36:42 ipsec payload seen: NOTIFY (8 bytes) 
07:36:42 ipsec create child: initiator finish 
07:36:42 ipsec processing payloads: NOTIFY 
07:36:42 ipsec   notify: TS_UNACCEPTABLE 
07:36:42 ipsec got error: TS_UNACCEPTABLE 
07:36:47 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[4500] 
07:36:47 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[4500] 
07:36:47 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[1025] 
07:36:47 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[1025]

I couldn't figure out what goes really wrong, thank you so much for your help!

sindy · Fri Sep 24, 2021 1:03 pm

Here is the complete log:
...
I couldn't figure out what goes really wrong, thank you so much for your help!

What goes wrong is clear - this: responder selector: 192.168.200.0/24. The phone should get a single address from the pool via mode-config (or rather its IKEv2 equivalent to be precise).
Why it goes wrong is much harder to answer, as your mode-config clearly states address-prefix-length=32. If I remember properly, it should ask for the SA on its own, so possibly this is the difference to other devices/IPsec implementations, and there's a bug in Mikrotik that never surfaced because the initiator normally proposes an x.x.x.x/32 -> 0.0.0.0/0 traffic selector? If it was that, I wonder where RouterOS gets the /24 from.

So maybe try to set the address associated to the bridge from /24 to /32 and try again, but it's a blind shot and I can see no reasonable logic behind that.

danergo · Fri Sep 24, 2021 3:00 pm

@Sindy,

Maybe from this config:

/ip ipsec policy add 
template=yes src-address=0.0.0.0/0 dst-address=192.168.200.0/[b]24[/b] proposal=vpn-clients group=vpn

Anyway, I have recreated the Policy from scratch, and rebooted the router also.

Now, it's much worse than before: it seems no packet arrives from Android to Tik at all. However, after reboot, I've realised that my 2 active peers happened to be one initiator and one responder.

And somewhere else I remember you wrote something about this limitation, that in case of initiator I'll have some troubles.

So now to see the clear picture, I have two policies:

/ip ipsec policy
add comment=Tunnel group=default-Tunnel proposal=default-Tunnel template=yes
add comment=VPN dst-address=192.168.200.0/24 group=vpn proposal=vpn-clients src-address=0.0.0.0/0 template=yes

"Tunnel" policy accepted 2 dynamic policy, one of them happened to be initiator side, another one is responder side.

I have tried setting a /32 address to the VPN bridge, doesn't make any difference.

sindy · Fri Sep 24, 2021 3:30 pm

Well... the policy with a /24 at dst-address is a template, but I admit I never dug deep into what happens if the responder has to suggest a TS first because the initiator doesn't. So you may want to reduce the pool to a single address and set the same address as a /32 in the template, to a) verify the assumption and b) maybe solve the issue for the single phone.

One of the improvements of IKEv2 with regard to IKE(v1) is that it allows to reduce the proposed TS, so if this is the issue, it's the Android client that should respond with an update from the /24 to /32 rather than rejecting the TS completely. Does the log before Phase 2 show anything regarding the address assignment? Maybe this is what actually fails and the Android phone doesn't ask for an IP, or asks in a way that RouterOS doesn't understand? And as it gets no address, it doesn't ask for a SA, so RouterOS suggests the template, etc.

Of course, why you can see no incoming connection attempts after the reboot is a different question.

As for the initiator and responder - if both your static peers are set to passive=no, then indeed you have two IKE connections, at least for a while. But these should normally not use different ports as both share the same UDP stream.

danergo · Fri Sep 24, 2021 3:33 pm

I'm going into a much more easier direction now:

I've disabled the persistent tunnel, so now there are no active peers. This way I'll have the chance to test the VPN server from Windows, and this would be extremely helpful, as my Win is now behind another Tik, so logging will be perfect.

I'll get back to here once I have more details on this, and after my Win will work, I will move onto Android (because that is the original goal).

Thank you.

danergo · Fri Sep 24, 2021 4:24 pm

Guys, I've started from scratch and found one basic root-cause: UDP500 was not accepted on the input chain, and therefore it never reached the IKE2 routines.
After adding that, a much nicer and more verbose log was created which however needs some explanation for me, if you could do that:

15:00:23 ipsec,debug ===== received 658 bytes from 4.3.2.1[42618] to 192.168.1.10[500] 
15:00:23 ipsec -> ike2 request, exchange: SA_INIT:0 4.3.2.1[42618] MAGIC 
15:00:23 ipsec ike2 respond 
15:00:23 ipsec payload seen: SA (244 bytes) 
15:00:23 ipsec payload seen: KE (264 bytes) 
15:00:23 ipsec payload seen: NONCE (36 bytes) 
15:00:23 ipsec payload seen: NOTIFY (28 bytes) 
15:00:23 ipsec payload seen: NOTIFY (28 bytes) 
15:00:23 ipsec payload seen: NOTIFY (8 bytes) 
15:00:23 ipsec payload seen: NOTIFY (14 bytes) 
15:00:23 ipsec payload seen: NOTIFY (8 bytes) 
15:00:23 ipsec processing payload: NONCE 
15:00:23 ipsec processing payload: SA 
15:00:23 ipsec,debug unknown auth: #13 
15:00:23 ipsec,debug unknown prf: #6 
15:00:23 ipsec,debug unknown DH group: #24 
15:00:23 ipsec,debug unknown prf: #6 
15:00:23 ipsec,debug unknown DH group: #24 
15:00:23 ipsec IKE Protocol: IKE 
15:00:23 ipsec  proposal #1 
15:00:23 ipsec   enc: aes256-cbc 
15:00:23 ipsec   enc: aes128-cbc 
15:00:23 ipsec   prf: hmac-sha512 
15:00:23 ipsec   prf: unknown 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   prf: hmac-sha1 
15:00:23 ipsec   auth: sha512 
15:00:23 ipsec   auth: unknown 
15:00:23 ipsec   auth: sha256 
15:00:23 ipsec   auth: sha1 
15:00:23 ipsec   dh: unknown 
15:00:23 ipsec   dh: ecp384 
15:00:23 ipsec   dh: ecp256 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec   dh: modp1536 
15:00:23 ipsec  proposal #2 
15:00:23 ipsec   enc: aes256-gcm 
15:00:23 ipsec   enc: aes128-gcm 
15:00:23 ipsec   prf: hmac-sha512 
15:00:23 ipsec   prf: unknown 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   prf: hmac-sha1 
15:00:23 ipsec   dh: unknown 
15:00:23 ipsec   dh: ecp384 
15:00:23 ipsec   dh: ecp256 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec   dh: modp1536 
15:00:23 ipsec matched proposal: 
15:00:23 ipsec  proposal #1 
15:00:23 ipsec   enc: aes256-cbc 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   auth: sha256 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec processing payload: KE 
15:00:23 ipsec unknown DH group: #24 
[b]15:00:23 ipsec DH group number mismatch: 14 != 24[/b]
15:00:23 ipsec adding notify: INVALID_KE_PAYLOAD 
15:00:23 ipsec,debug => (size 0xa) 
15:00:23 ipsec,debug 0000000a 00000011 000e 
15:00:23 ipsec,debug ===== sending 38 bytes from 192.168.1.10[500] to 4.3.2.1[42618] 
15:00:23 ipsec,debug 1 times of 38 bytes message will be sent to 4.3.2.1[42618] 
15:00:23 ipsec,debug ===== received 658 bytes from 4.3.2.1[42618] to 192.168.1.10[500] 
15:00:23 ipsec -> ike2 request, exchange: SA_INIT:0 4.3.2.1[42618] MAGIC
15:00:23 ipsec ike2 respond 
15:00:23 ipsec payload seen: SA (244 bytes) 
15:00:23 ipsec payload seen: KE (264 bytes) 
15:00:23 ipsec payload seen: NONCE (36 bytes) 
15:00:23 ipsec payload seen: NOTIFY (28 bytes) 
15:00:23 ipsec payload seen: NOTIFY (28 bytes) 
15:00:23 ipsec payload seen: NOTIFY (8 bytes) 
15:00:23 ipsec payload seen: NOTIFY (14 bytes) 
15:00:23 ipsec payload seen: NOTIFY (8 bytes) 
15:00:23 ipsec processing payload: NONCE 
15:00:23 ipsec processing payload: SA 
15:00:23 ipsec,debug unknown auth: #13 
15:00:23 ipsec,debug unknown prf: #6 
15:00:23 ipsec,debug unknown DH group: #24 
15:00:23 ipsec,debug unknown prf: #6 
15:00:23 ipsec,debug unknown DH group: #24 
15:00:23 ipsec IKE Protocol: IKE 
15:00:23 ipsec  proposal #1 
15:00:23 ipsec   enc: aes256-cbc 
15:00:23 ipsec   enc: aes128-cbc 
15:00:23 ipsec   prf: hmac-sha512 
15:00:23 ipsec   prf: unknown 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   prf: hmac-sha1 
15:00:23 ipsec   auth: sha512 
15:00:23 ipsec   auth: unknown 
15:00:23 ipsec   auth: sha256 
15:00:23 ipsec   auth: sha1 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec   dh: unknown 
15:00:23 ipsec   dh: ecp384 
15:00:23 ipsec   dh: ecp256 
15:00:23 ipsec   dh: modp1536 
15:00:23 ipsec  proposal #2 
15:00:23 ipsec   enc: aes256-gcm 
15:00:23 ipsec   enc: aes128-gcm 
15:00:23 ipsec   prf: hmac-sha512 
15:00:23 ipsec   prf: unknown 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   prf: hmac-sha1 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec   dh: unknown 
15:00:23 ipsec   dh: ecp384 
15:00:23 ipsec   dh: ecp256 
15:00:23 ipsec   dh: modp1536 
15:00:23 ipsec matched proposal: 
15:00:23 ipsec  proposal #1 
15:00:23 ipsec   enc: aes256-cbc 
15:00:23 ipsec   prf: hmac-sha256 
15:00:23 ipsec   auth: sha256 
15:00:23 ipsec   dh: modp2048 
15:00:23 ipsec processing payload: KE 
15:00:24 ipsec,debug => shared secret (size 0x100) 
...
15:00:24 ipsec adding payload: SA 
15:00:24 ipsec,debug => (size 0x30) 
...
15:00:24 ipsec adding payload: KE 
15:00:24 ipsec,debug => (first 0x100 of 0x108) 
...
15:00:24 ipsec adding payload: NONCE 
15:00:24 ipsec,debug => (size 0x1c) 
...
15:00:24 ipsec adding notify: NAT_DETECTION_SOURCE_IP 
15:00:24 ipsec,debug => (size 0x1c) 
...
15:00:24 ipsec adding notify: NAT_DETECTION_DESTINATION_IP 
15:00:24 ipsec,debug => (size 0x1c) 
...
15:00:24 ipsec adding payload: CERTREQ 
15:00:24 ipsec,debug => (size 0x5) 
...
15:00:24 ipsec <- ike2 reply, exchange: SA_INIT:0 4.3.2.1[42618] MAGIC 
15:00:24 ipsec,debug ===== sending 429 bytes from 192.168.1.10[500] to 4.3.2.1[42618] 
15:00:24 ipsec,debug 1 times of 429 bytes message will be sent to 4.3.2.1[42618] 
15:00:24 ipsec,debug => skeyseed (size 0x20) 
...
15:00:24 ipsec,debug => keymat (size 0x20) 
...
15:00:24 ipsec,debug => SK_ai (size 0x20) 
...
15:00:24 ipsec,debug => SK_ar (size 0x20) 
...
15:00:24 ipsec,debug => SK_ei (size 0x20) 
...
15:00:24 ipsec,debug => SK_er (size 0x20) 
...
15:00:24 ipsec,debug => SK_pi (size 0x20) 
...
15:00:24 ipsec,debug => SK_pr (size 0x20) 
...
15:00:24 ipsec,info new ike2 SA (R): 192.168.1.10[500]-4.3.2.1[42618] spi:MAGIC
15:00:24 ipsec processing payloads: VID (none found) 
15:00:24 ipsec processing payloads: NOTIFY 
15:00:24 ipsec   notify: NAT_DETECTION_SOURCE_IP 
15:00:24 ipsec   notify: NAT_DETECTION_DESTINATION_IP 
15:00:24 ipsec   notify: IKEV2_FRAGMENTATION_SUPPORTED 
15:00:24 ipsec   notify: SIGNATURE_HASH_ALGORITHMS 
15:00:24 ipsec,debug 000200030004 
15:00:24 ipsec   notify: REDIRECT_SUPPORTED 
15:00:24 ipsec (NAT-T) REMOTE LOCAL 
15:00:24 ipsec KA list add: 192.168.1.10[4500]->4.3.2.1[42618] 
15:00:24 ipsec,debug ===== received 658 bytes from 4.3.2.1[42618] to 192.168.1.10[500] 
15:00:24 ipsec -> ike2 request, exchange: SA_INIT:0 4.3.2.1[42618] MAGIC
15:00:24 ipsec ike2 respond 
15:00:24 ipsec payload seen: SA (244 bytes) 
15:00:24 ipsec payload seen: KE (264 bytes) 
15:00:24 ipsec payload seen: NONCE (36 bytes) 
15:00:24 ipsec payload seen: NOTIFY (28 bytes) 
15:00:24 ipsec payload seen: NOTIFY (28 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec payload seen: NOTIFY (14 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec processing payload: NONCE 
15:00:24 ipsec sa init retransmitted  
15:00:24 ipsec,debug ===== sending 429 bytes from 192.168.1.10[500] to 4.3.2.1[42618] 
15:00:24 ipsec,debug 1 times of 429 bytes message will be sent to 4.3.2.1[42618] 
15:00:24 ipsec,debug ===== received 658 bytes from 4.3.2.1[42618] to 192.168.1.10[500] 
15:00:24 ipsec -> ike2 request, exchange: SA_INIT:0 4.3.2.1[42618] MAGIC
15:00:24 ipsec ike2 respond 
15:00:24 ipsec payload seen: SA (244 bytes) 
15:00:24 ipsec payload seen: KE (264 bytes) 
15:00:24 ipsec payload seen: NONCE (36 bytes) 
15:00:24 ipsec payload seen: NOTIFY (28 bytes) 
15:00:24 ipsec payload seen: NOTIFY (28 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec payload seen: NOTIFY (14 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec processing payload: NONCE 
15:00:24 ipsec sa init retransmitted  
15:00:24 ipsec,debug ===== sending 429 bytes from 192.168.1.10[500] to 4.3.2.1[42618] 
15:00:24 ipsec,debug 1 times of 429 bytes message will be sent to 4.3.2.1[42618] 
15:00:24 ipsec,debug KA: 192.168.1.10[4500]->4.3.2.1[42618] 
15:00:24 ipsec,debug 1 times of 1 bytes message will be sent to 4.3.2.1[42618] 
15:00:24 ipsec,debug ===== received 1472 bytes from 4.3.2.1[42278] to 192.168.1.10[4500] 
15:00:24 ipsec -> ike2 request, exchange: AUTH:1 4.3.2.1[42278] MAGIC 
15:00:24 ipsec peer ports changed: 42618 -> 42278 
15:00:24 ipsec KA remove: 192.168.1.10[4500]->4.3.2.1[42618] 
15:00:24 ipsec,debug KA tree dump: 192.168.1.10[4500]->4.3.2.1[42618] (in_use=1) 
15:00:24 ipsec,debug KA tree dump: 192.168.1.10[4500]->4.3.2.1[42618] (in_use=1) 
15:00:24 ipsec,debug KA tree dump: 192.168.1.10[4500]->4.3.2.1[42618] (in_use=1) 
15:00:24 ipsec,debug KA removing this one... 
15:00:24 ipsec KA list add: 192.168.1.10[4500]->4.3.2.1[42278] 
15:00:24 ipsec payload seen: ENC (1444 bytes) 
15:00:24 ipsec processing payload: ENC 
15:00:24 ipsec,debug => iv (size 0x10) 
15:00:24 ipsec,debug ff60e326 e1177c6f 31247509 d1d80824 
15:00:24 ipsec,debug => plain payload (trimmed) (first 0x100 of 0x57f) 
...
15:00:24 ipsec,debug decrypted 
15:00:24 ipsec payload seen: ID_I (25 bytes) 
15:00:24 ipsec payload seen: CERT (845 bytes) 
15:00:24 ipsec payload seen: CERTREQ (25 bytes) 
15:00:24 ipsec payload seen: AUTH (264 bytes) 
15:00:24 ipsec payload seen: CONFIG (16 bytes) 
15:00:24 ipsec payload seen: SA (124 bytes) 
15:00:24 ipsec payload seen: TS_I (24 bytes) 
15:00:24 ipsec payload seen: TS_R (24 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec payload seen: NOTIFY (12 bytes) 
15:00:24 ipsec payload seen: NOTIFY (24 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec payload seen: NOTIFY (8 bytes) 
15:00:24 ipsec processing payloads: NOTIFY 
15:00:24 ipsec   notify: MOBIKE_SUPPORTED 
15:00:24 ipsec   notify: ADDITIONAL_IP4_ADDRESS 
15:00:24 ipsec,debug 64438afe 
15:00:24 ipsec   notify: ADDITIONAL_IP6_ADDRESS 
15:00:24 ipsec,debug 200108f81d3e78ff6c446c1acf2028e5 
15:00:24 ipsec   notify: EAP_ONLY_AUTHENTICATION 
15:00:24 ipsec   notify: IKEV2_MESSAGE_ID_SYNC_SUPPORTED 
15:00:24 ipsec ike auth: respond 
15:00:24 ipsec processing payload: ID_I 
[b]15:00:24 ipsec ID_I (DER DN): myphone[/b]
15:00:24 ipsec processing payload: ID_R (not found) 
15:00:24 ipsec processing payload: AUTH 
15:00:24 ipsec processing payload: CERT 
[b]15:00:24 ipsec got CERT: myphone[/b] 
15:00:24 ipsec,debug => (size 0x348) 
...
...
...
[b]15:00:24 ipsec,error identity not found for peer: DER DN: myphone[/b] 
15:00:24 ipsec reply notify: AUTHENTICATION_FAILED 
15:00:24 ipsec adding notify: AUTHENTICATION_FAILED

First:

15:00:23 ipsec DH group number mismatch: 14 != 24

how can I make DH to not mismatch? (Or is this something I can leave alone?)

Second:

15:00:24 ipsec ID_I (DER DN): myphone
15:00:24 ipsec got CERT: myphone
15:00:24 ipsec,error identity not found for peer: DER DN: myphone

"myphone" is the name of the certificate on my Android. Android asked me a custom name during the import procedure and I gave this name. It sounds pretty obvious that Tik doesn't know this text, as I have never wrote this into Tik. What is the resolution?

Thank you

danergo · Fri Sep 24, 2021 4:50 pm

Try disabling and re-enabling the second identity (or both) and see whether it starts working then.

This indeed worked

Now Android is joined to this lovely Tik.

Other question with the DH mismatch would be still interesting though.

Thank you!

danergo · Fri Sep 24, 2021 8:52 pm

Folks,

I don't understand one another thing:

In active peers, I can see my VPN client perfectly. Its "Local Address" is "192.168.1.10", which is the IP of the Tik from outside (assigned by ISP). Why?

I mean where did we defined this, or how IPsec knows this?

Everything works perfectly, just I don't understand this, and I like learning new things.

sindy · Fri Sep 24, 2021 10:34 pm

Other question with the DH mismatch would be still interesting though.

According to the log, the Android proposes multiple DH groups in each Phase 1 proposal, one of which is not recognized by the RouterOS IPsec stack. So RouterOS accepts the proposal, but complains one more time about that. You can safely ignore this complaint.

In active peers, I can see my VPN client perfectly. Its "Local Address" is "192.168.1.10", which is the IP of the Tik from outside (assigned by ISP). Why?
I mean where did we defined this, or how IPsec knows this?

It's the local address of the Tik where you run the /ip ipsec active-peers command to/from which the connection with that peer has been established. There's no magic in it, this information is sourced locally, not received from the peer.

However, the peers do inform each other about the actual IP addresses and ports they use for the connection, in order to find out whether there is a NAT somewhere between them and adjust the transport accordingly. Each peer tells the other one what is its own address and port used for the connection, and from what address and port it has received the other peer's previous packet; this way, both peers have enough information to compare.

danergo · Sat Sep 25, 2021 9:20 am

Thank you, everything is clear/perfect now.

anav · Sun Sep 26, 2021 6:57 pm

viewtopic.php?f=23&t=175656

You're welcome

That link misses the most key points
where is the direction for input chain rules
If needed where is the direction for forward chain rules.
But what I find hardest is..........Where is the exit/entry point of the tunnel (behind the router) in relation to interfaces, or faux IP address or IP subnet etc.......

danergo · Mon Sep 27, 2021 9:30 pm

Folks,

I'm finetuning this solution, and something came across to me:

I have an active peer in IPsec/Active Peers tab, for my phone when it's connected onto VPN:

I have a webserver running in the LAN (192.168.2.100, port 80), and I added 2 logs into "Firewall/Filter rules" at the very top:

One with "RX" prefix: forward chain, TCP, dst port=80, action=log
One with "TX" prefix: forward chain, TCP, src port=80, action=log

(And also, I have a dstnat: TCP, dstport=80, from ether1_wan (with in interface list), action=dst-nat to 192.168.2.100)

However investigating the log, when my phone is on VPN, and accesses this webserver, it seems exactly the same as if the whole traffic is not going through the VPN but a normal port fw:

RX from WAN:
1.) in:ether1_wan out:bridge 4.3.2.1:10000->192.168.2.100:80, NAT 4.3.2.1:10000->(192.168.1.10:80->192.168.2.100:80)
TX from LAN:
2.) in:bridge out:ether1_wan 192.168.2.100:80->4.3.2.1:10000, NAT (192.168.2.100:80->192.168.1.10:80)->4.3.2.1:10000

Could you please validate this? I think it's going through the VPN, but these log lines makes some fear for me.

I don't understand why I don't see any mention for the "Dynamic address" of 192.168.200.200 anywhere in the chain.

Thank you!

sindy · Mon Sep 27, 2021 10:25 pm

There is a common problem - when the VPN tunnel becomes the default gateway for the VPN client, you have to make sure that it is not used for routing the transport packets, for obvious reasons. And whilst on Mikrotik, you must take care about this manually for all types of VPN except bare IPsec where RouterOS does some magic on its own (exepmting IPsec transport packets from matching IPsec traffic selectors), the Android implementation may have chosen to exempt the whole server address. If you have another public IP somewhere, on which you could sniff, you can verify this assumption.

danergo · Mon Sep 27, 2021 10:39 pm

Actually, if I check "myip" from Android during VPN, I can see TIK's public address. So I believe it all goes through VPN, just I had some fear because of the packet logs.

So you are saying, that this implementation I've done here in this thread can be considered as "bare IPsec", and RouterOS is just doing the magic for me?

danergo · Mon Sep 27, 2021 11:03 pm

Also, I've came across another thing, which I really need to solve.

It's kind of merging two threads, Sindy. We have created a nice EoIP tunnels with bonding over IPsec earlier.

I'm marking some packets to go into this tunnel instead getting out on public WAN.

This also works for this VPN clients, ie:

Phone~~~TIK0~~~TIK1~~~WebServer

My issue is that I can see the packets in TIK1 (coming through the tunnel between TIK0 and TIK1), but TIK1 can't send their replys back.

Why? Because:

Step1 (TIK0):

in:wan out:bonding 192.168.200.200:10000->9.10.11.12:80 (9.10.11.12 is TIK1)

Step2 (TIK1):

in:bonding out:wan 192.168.200.200:10000->9.10.11.12:80
in:wan out:bridge LASTIP_BEFORE_TIK1:10000->WEBSERVER_IN_TIK1_LAN:80, NAT LASTIP_BEFORE_TIK1:10000->(TIK1_WANIP:80->WEBSERVER_IN_TIK1_LAN:80)

Step3 (TIK1), reply:

in:bridge out:wan WEBSERVER_IN_TIK1_LAN:80->LASTIP_BEFORE_TIK1:10000, NAT (WEBSERVER_IN_TIK1_LAN:80->TIK1_WANIP:80)->LASTIP_BEFORE_TIK1:10000
!!!! in:wan out:wan 9.10.11.12:80->192.168.200.200:10000, NAT 9.10.11.12:80->(TIK1_WANIP:10000->192.168.100.200:10000)

This last line is the problem, as it puts the reply to WAN, instead of bonding. Now I could create a static route, to explicitly show TIK1, that it can reach 192.168.200.0/24 on the bonding, but it feels a hack (this IP Pool can be changed in TIK0, and it would break TIK1's config), thus I've tried, and it makes replys reaching back TIK0, but TIK0 can't forward those replys to Phone over IPsec VPN.

So I'm considering something more elegant. Like some masq, or srcnat in TIK0? Or anything? Do you maybe have some ideas on this? How is this to be solved elegant in MikroTik's beautiful world?

Thank you so much!

sindy · Mon Sep 27, 2021 11:32 pm

Actually, if I check "myip" from Android during VPN, I can see TIK's public address. So I believe it all goes through VPN, just I had some fear because of the packet logs.

Yes, all goes through VPN, except traffic to the public address of the responder (VPN server).

So you are saying, that this implementation I've done here in this thread can be considered as "bare IPsec", and RouterOS is just doing the magic for me?

RouterOS is doing the magic in terms that even if the src-address of a policy includes the WAN IP from which the IPsec transport packets are sent, and dst-address of the same policy is 0.0.0.0/0 (or at least includes the address to which the IPsec transport packets are sent), i.e. the transport packets would normally match that policy, the fact that they are IPsec transport packet prevents RouterOS from matching them to the policy, so they take the regular route.

Whereas the Android doesn't discriminate at such a deep level, and simply exempts all packets with dst-address of the responder, even other ones than IPsec transport ones, from the traffic selector matching.

sindy · Mon Sep 27, 2021 11:35 pm

So I'm considering something more elegant. Like some masq, or srcnat in TIK0? Or anything? Do you maybe have some ideas on this? How is this to be solved elegant in MikroTik's beautiful world?

The solution is included in this post. Start reading it from the last paragraph, which explains the relationship to your context.

danergo · Tue Sep 28, 2021 11:28 am

Sindy, thank you so much!

Yes, all goes through VPN, except traffic to the public address of the responder (VPN server).

At first, I didn't get this.

RouterOS is doing the magic in terms that even if the src-address of a policy includes the WAN IP from which the IPsec transport packets are sent, and dst-address of the same policy is 0.0.0.0/0 (or at least includes the address to which the IPsec transport packets are sent), i.e. the transport packets would normally match that policy, the fact that they are IPsec transport packet prevents RouterOS from matching them to the policy, so they take the regular route.

Yes, that's crystal clear I already knew this.

Whereas the Android doesn't discriminate at such a deep level, and simply exempts all packets with dst-address of the responder, even other ones than IPsec transport ones, from the traffic selector matching.

And wow, I can feel how my "language interpretation" evolves after drinking morning coffee: I can finally understand what's happening under the hood in this case. This was the reason why my WebServer in Tik's LAN (which is accessed via VPN) made those traffic logs, as they are indeed simple port forwards.

By any chance do you know how Windows is working in this term? Is it also not discriminating IPsec transport to dst-address of repsonder vs anything else to dst-address of responder?

Regarding this:

The solution is included in this post. Start reading it from the last paragraph, which explains the relationship to your context.

I got this, but I think I didn't describe my issue with enough details:

Please consider I've a working tunnel between 2 Tiks (TIK0 and TIK1). There is a service in TIK1's LAN, which I need to access from TIK0 (one lan client of it), and I want this traffic to go through the tunnel.
Therefore I have these mangle's in TIK0:

/ip firewall mangle
add action=accept chain=prerouting comment="est. rel. connections without conmark are accepted to prevent next rules" connection-mark=no-mark connection-state=established,related
add action=accept chain=prerouting comment="download packets must not be routing-marked" connection-state=established,related in-interface-list=wan|tunnel
add action=mark-routing chain=prerouting comment="connection marked packets marked for route also" connection-mark=into-tunnel new-routing-mark=into-tunnel passthrough=no
add action=mark-connection chain=prerouting comment="only initial packets of connections get here past the rules above" connection-state=new dst-address-list=to-tunnel-dsts new-connection-mark=into-tunnel passthrough=yes
add action=mark-routing chain=prerouting comment="initial packets of connections which evaded the rules above get here with no connection mark; we just repeat the mark-routing rules above" connection-mark=into-tunnel new-routing-mark=into-tunnel passthrough=no

Obviously,

to-tunnel-dsts

contains the address of TIK1, therefore traffic to TIK1 address will go through the tunnel. This works perfectly (I have static routes defined in both TIKs to the other TIK: reachable via tunnel). So, all traffic directed to

to-tunnel-dsts

go into the tunnel, and all the rest just simply go out on the default gateway.

However, the picture is now a bit more complex as I've introduced a remote VPN client to TIK0. This remote VPN client still need to use the tunnel (between TIKs) if it wants to communicate with the service in TIK1's LAN.

This doesn't mean a problem until a point:

in TIK0: above mangles are redirecting the traffic into the tunnel, even if it comes from the remote VPN client
in TIK1: I can see the traffic coming through the tunnel, and it also successfully reaches the service on TIK1's LAN
in TIK1: reply's route is not defined in ip/route, as the original packet contains the src address as the imaginary 192.168.200.200 (defined only in TIK0), and TIK1 has no idea on where shall it send this response.

I don't want to create a new static route in TIK1 for 192.168.200.0 via tunnel, as it might be changing if I decide to change that imaginary pool in TIK0. Plus, I might want to use the same imaginary pool in TIK1 for something else later.

Sorry for this, but I really don't see how mangles can help me more with this, as that imaginary IP seems making my situation hard.

danergo · Tue Sep 28, 2021 12:23 pm

So the problem is not in TIK1, I have many options to send back the reply:

Static route
Mangle
NAT

The problem is that after TIK0 receives the reply, it tries to send it to 192.168.200.200, which is the phone's Dynamic Address in IPsec, and this packet doesn't seem arriving into the phone.

So I might have to srcnat this IP pool into phone's Remote IP? Or IPsec shall simply take care of that?

sindy · Tue Sep 28, 2021 3:51 pm

The essence of the setup outlined in the post I've linked is "send the response packets of a connection via the same interface through which the initial request of that connection has arrived to you", and it doesn't matter much whether that interface is a real WAN or a tunnel. So at the router to which the server is connected, you don't need dedicated routes to particular destinations.

danergo · Tue Sep 28, 2021 4:53 pm

When Mangle marks the connection, does that mean for MikroTik, that it will have to use the "incoming interface of the packet" when sending back the response?

sindy · Tue Sep 28, 2021 5:08 pm

It's just one of the components of the complete setup. You also need another mangle rule, assigning a routing-mark (in fact, a routing table name) to packets sent from LAN side depending on the connection-mark value, and the routing table itself, typically consisting of just a single default route via the gateway reachable via that interface.

So you assign the connection-mark as the initial client->server packet arrives, making the connection context "remember" the interface to use, and you use the connection-mark when sending the server->client packets, translating it to a routing table name.

danergo · Tue Sep 28, 2021 11:12 pm

Ok, I got this, thank you.

Please consider this other scenario:

Phone ~~~ VPN ~~~ TIK0 ~~~ TUNNEL ~~~ TIK1 ~~~ WAN

Let's say phone first joins to TIK0 via VPN, and TIK0's mangle rules marks some connections and packets, to routing them (instead TIK0's default gateway) towards the TIK1 through the TUNNEL, letting my phone browsing the net if it was part of TIK1's LAN.

I can do mangle-rules in TIK1 for marking the connections (coming from TUNNEL), but as they have to go towards the default gateway (of TIK1), it feels unnecessary (or I don't really know how to do it properly).

Here (https://wiki.mikrotik.com/wiki/Manual:I ... all/Mangle) it's written:

The mangle marks exist only within the router, they are not transmitted across the network

And indeed, if the packet leaves TIK1, and then later a response for this packet arrives back, shall those reply packets contain my original marks? If so - I can do a route towards TUNNEL if has my connection-mark, and originating from WAN. (Actually, not exactly, as routes can't have source addresses or interfaces, so I'd need another mangle in TIK1: if originating from WAN, and has my earlier connection-mark, replace the connection-mark to something else, and for this something else connection-mark, I can add my static route towards TUNNEL)

But it seems I can't solve this.

sindy · Wed Sep 29, 2021 9:15 am

The statement regarding the mangle marks being virtual and not being added to the actual packet data is absolutely correct, but you miss some information.

The stateful firewall is built around a key component called connection tracker (conntrack module of Linux netfilter). This component maintains a database of "connections", which are TCP or SCTP sessions, UDP flows, ICMP echo/echo reply conversations, GRE or IPIP flows...

Let's take an example of UDP, as it requires a more complex handling than GRE or IPIP but a less complex one than TCP or SCTP. When a UDP packet arrives, the conntrack checks whether a UDP connection with the same source and destination address and port exists in the database. If it does, it determines the direction of the packet within that connection: if the source of the packet matches the one stored in the connection database, it is the "originating" direction, if the destination of the packet matches the source one stored in the connection database, it is the "reply" direction. In fact, it is a bit more complex as there is also src-nat and dst-nat, so four address:port tuples in total are stored per connection.

If no matching connnection is found, it gets created.

The conntrack module automatically attaches connection-state and connection-nat-state attributes to each packet, so that subsequent stages of firewall processing could use them to treat the packet specifically, depending on these attributes. Also the connection-mark attribute may get attached to the packet if the one being currently processed or some previous packet belonging to the same connection has hit an action=mark-connection rule, because the connection-mark value is stored as yet another attribute of the connection in the database.

So the fact that the response packet arriving from the server to TIK1 belongs to a particular tracked connection is determined from its source and destination address and port; unless you actively rewrite the connection-mark when processing that packet, the pre-existing value of the connection-mark attribute is retrieved from the database.

If you look at the rules in the linked post, you'll see that packets with connection-state value established, which means almost every mid-connection packet, are accepted in mangle/prerouting before they can reach any of the mark-connection rules. So as a result, the connection-mark is assigned only when processing the initial packet of the connection and remains unchanged until the connection ends.

danergo · Wed Sep 29, 2021 1:32 pm

Sindy, thank you so much for summarizing this for me, I can't be thankful enough.

Although, I have the intention that I won't need mangles: as for me it seems they are extremely helpful if (for example) you need to direct specific packets from the same subnet to not go towards default route(s).

However in my case, I have different subnets and different interfaces, so Tik anyway will be able find the reply-path, as if some packets arrived through the tunnel (on distinct interface and on distinct address), reply-packets will go back through that interface and address.

I've validated this, my packets are going back and forth, without adding any mangle-rules into TIK1.

I'm however still stuck at one point, I guess this will be the last, because everything else is working now:

In TIK0:
I'm logging "RX": packets received with dst-address of "whatismyip.com" (RX from client)
I'm logging "TX": packets received from src-address of "whatismyip.com" (TX to client)

In this scenario (Scenario1):

Phone ~~~ VPN ~~~ TIK0 ~~~ WAN

RX forward: in:wan out:wan src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN), PHONE_DYNAMIC_ADR:10000->WHATSMYIP:443, len 60
TX forward: in:wan out:wan src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN,ACK), WHATSMYIP:443->PHONE_DYNAMIC_ADR:10000, NAT WHATSMYIP:443->(WANIP_OF_TIK0:10000->PHONE_DYNAMIC_ADR:10000), len 52

This works as intended! Only little detail here is that I have no idea on why "RX" shows "wan" for "in" interface if that packet is clearly coming from the phone over IKE2 VPN.

However in this scenario (Scenario2):

Phone ~~~ VPN ~~~ TIK0 ~~~ TUNNEL ~~~ TIK1 ~~~ WAN

RX forward: in:wan out:tunnel, src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN), PHONE_DYNAMIC_ADR:10000->WHATSMYIP:443, len 60
TX forward: in:tunnel out:wan, src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN,ACK), WHATSMYIP:443->PHONE_DYNAMIC_ADR:10000, NAT WHATSMYIP:443->(TUNNEL_ADDRESS_ENDPOINT_IN_TIK0:10000->PHONE_DYNAMIC_ADR:10000), len 52

My phone is constantly retrying, it seems no TX packets are arriving back to it, which is strange as it seems the only difference is in NAT (WANIP_OF_TIK0 vs TUNNEL_ADDRESS_ENDPOINT_IN_TIK0).

Here are my NAT-rules:

add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=wan //by default for outgoing packets, this makes first scenario working
add action=src-nat chain=srcnat comment="conf: for tunneled connections" connection-mark=please-go-through-tunnel to-addresses=TUNNEL_ADDRESS_ENDPOINT_IN_TIK0 //I've created this for 2nd scenario working

Strange facts in order:

Scenario2 is not working
If I add "ipsec-policy=out,none" to my src-nat action, there are no "TX" logs printed from that point on.

Note: I have tried with masquerading instead of src-nat, but as that's just a special srcnat, not surprisingly, it didn't work.

Could you please help me with this NAT-thing, I think this is the final piece missing.

Thank you!

sindy · Wed Sep 29, 2021 6:35 pm

Although, I have the intention that I won't need mangles:
...
I've validated this, my packets are going back and forth, without adding any mangle-rules into TIK1.

Well - you've said before you wanted something "future-proof", i.e. something that would work even if you change the IP addresses used at the remote end of the tunnel. So the use of connection tracking to clip a connection to the in-interface of the initial packet addresses exactly this request. Of course, where plain routing based on destination addresses is enough, policy routing (mangle rules & multiple routinig tables) are clearly an overkill. So if you use src-nat on TIK0 also when sending packets to TIK1 via the tunnel, whatever actual address the phone has, its packets are seen as coming from TIK0's address inside the tunnel, so a route to TIK0's tunnel address at TIK1 is sufficient for everything to work.

Only little detail here is that I have no idea on why "RX" shows "wan" for "in" interface if that packet is clearly coming from the phone over IKE2 VPN.

Because the payload packet extracted from the IPsec transport one inherits the in-interface attribute from the transport one. In vanilla IPsec as implemented by Mikrotik, there is no Virtual Tunnel Interface for the IPsec payload. So to distinguish packets that came as payload ones of an IPsec SA from those that didn't, the ipsec-policy matcher is used - in,ipsec matches on packets that came within an SA, whereas in,none matches on packets that came outside an SA, like e.g. the IPsec transport packets.

However in this scenario (Scenario2):

The behavious is confusing, and so is the description. The second nat rule refers to some connection-mark but I don't know where that connection-mark is assigned. But the fact that adding ipsec-policy=out,none to the rule changes the behaviour indicates that the connection-mark is asigned indeed.

Since you can see the response packet to arrive from TIK1 via the tunnel, it means everything is fine all the way from TIK0 via TIK1 to whatsmyip and back to TIK0. In what you have posted, I can see no reason why the response packet should not make it back to the phone. But as the ipsec-policy=out,none in the srcnat rule has an impact, I assume that the request packet matches to some other IPsec policy. So if the srcnat rule changes its source address, this policy doesn't match it later on, the packet is really sent via the tunnel with the IP address of the local tunnel interface, and the response comes back via the tunnel. As the response gets un-src-nated, it starts inverse-matching that policy again, and because packets that should have come via an SA but didn't are silently dropped, it is not delivered to the phone. But that's a speculation, I have never tested this. When you exempt the packet from being srcnated by adding the ipsec-policy=out,none to the srcnat rule, it doesn't get srcnated, so although the filter rule log still says that the packet has been routed via the tunnel, in fact the policy matching supersedes the routing decision, so the packet is sent down the SA associated to that other policy, and gets lost somewhere on the remote peer, so no response comes back.

danergo · Wed Sep 29, 2021 9:06 pm

Well - you've said before you wanted something "future-proof", i.e. something that would work even if you change the IP addresses used at the remote end of the tunnel. So the use of connection tracking to clip a connection to the in-interface of the initial packet addresses exactly this request. Of course, where plain routing based on destination addresses is enough, policy routing (mangle rules & multiple routinig tables) are clearly an overkill. So if you use src-nat on TIK0 also when sending packets to TIK1 via the tunnel, whatever actual address the phone has, its packets are seen as coming from TIK0's address inside the tunnel, so a route to TIK0's tunnel address at TIK1 is sufficient for everything to work.

Sorry for being unclear about "future-proof" I meant for that only for the IKE2VPN in TIK0. What you wrote here is completely clear and true, thank you so much.

Because the payload packet extracted from the IPsec transport one inherits the in-interface attribute from the transport one. In vanilla IPsec as implemented by Mikrotik, there is no Virtual Tunnel Interface for the IPsec payload. So to distinguish packets that came as payload ones of an IPsec SA from those that didn't, the ipsec-policy matcher is used - in,ipsec matches on packets that came within an SA, whereas in,none matches on packets that came outside an SA, like e.g. the IPsec transport packets.

This was my idea too, thank you very much for confirming!

The behavious is confusing, and so is the description. The second nat rule refers to some connection-mark but I don't know where that connection-mark is assigned. But the fact that adding ipsec-policy=out,none to the rule changes the behavior indicates that the connection-mark is asigned indeed.

Sorry for made this confusing: both NAT rules shown here are in TIK0 (in both scenarios as-is, no changes in NAT rules, they are constant in both scenarios). Also, in both scenarios I'm showing the RX and TX in TIK0.

Actually there are only 2 differences between those 2 scenarios:

In Scenario1 Phone browses the internet as if it was part of TIK0's LAN, whereas in Scenario2 Phone browses the internet as if it was part of TIK1's LAN.
Scenario1 is working, whereas Scenario2 is not.

"please-go-through-tunnel" mangle is in TIK0, based on your earlier topic suggestion (here I felt those mangles are not needed to share in detail). It really just marks the connection when it needs to go towards TIK1, and also does the routing marks accordingly. I just tried using something as descriptive as possible, and this is what I found out: all packets belong to "please-go-through-tunnel" in TIK0 will use TIK1 as gateway instead of TIK0's default gateway.

Since you can see the response packet to arrive from TIK1 via the tunnel, it means everything is fine all the way from TIK0 via TIK1 to whatsmyip and back to TIK0. In what you have posted, I can see no reason why the response packet should not make it back to the phone.

I'm so happy for at least not being alone with this

But as the ipsec-policy=out,none in the srcnat rule has an impact, I assume that the request packet matches to some other IPsec policy. So if the srcnat rule changes its source address, this policy doesn't match it later on, the packet is really sent via the tunnel with the IP address of the local tunnel interface, and the response comes back via the tunnel. As the response gets un-src-nated, it starts inverse-matching that policy again, and because packets that should have come via an SA but didn't are silently dropped, it is not delivered to the phone. But that's a speculation, I have never tested this.

This can really be a possible case! This would be a triple20 in darts Sindy: as the current request packet is coming from IKE2 IPsec VPN from the phone, it is actually part of another IPsec policy. OH! And if this is true, then the "masquerade" NAT-rule is also not applied to the request coming from the phone in Scenario1! So in Scenario1, there is no NAT-rule based masquerading or src-natting at all. I shouldn't do that in Scenario2 either, but then I'll definitely have to use either Mangles or other basic routing in TIK1.

When you exempt the packet from being srcnated by adding the ipsec-policy=out,none to the srcnat rule, it doesn't get srcnated, so although the filter rule log still says that the packet has been routed via the tunnel, in fact the policy matching supersedes the routing decision, so the packet is sent down the SA associated to that other policy, and gets lost somewhere on the remote peer, so no response comes back.

Is srcnat-ing IPsec connections, even possible at all?

I've remove the "src-nat" from TIK0 (to not hurt any IPsec policies), and I added a new route "192.168.200.0/24" in TIK1 (to have the response packets chance to come back, i.e. 192.168.200.0/24 is the "Dst. Address" of the IPsec Policy that handles the Phone in TIK0):

Phone ---> IKE2_IPSEC ---> TIK0 ---> IKE2_IPSEC ---> TIK1 ---> MYIP
                                                                |
                                                                v
Phone ---- IKE2_IPSEC x--- TIK0 <--- IKE2_IPSEC <--- TIK1 <--- MYIP

Checking my earlier RX/TX log in TIK0 for this, I can see the packets are both receiving and transmitting:

RX forward: in:wan out:tunnel, src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN), PHONE_DYNAMIC_ADR:10000->WHATSMYIP:443, len 60
TX forward: in:tunnel out:wan, src-mac aa:bb:cc:dd:ee:ff, proto TCP (SYN,ACK), WHATSMYIP:443->PHONE_DYNAMIC_ADR:10000, len 52

I.e. here is no NAT involved in TIK0. Nothing touches the IPsec packets' addresses, however in TIK1, it obviously must do the masquerading when forwarding the original request to MYIP. From the view of TIK0, everything feels OK, but my phone is still not receive one single response over this complex multi-IPsec tunnel setup.

Do you maybe have any idea on what would worth checking, why this response packet is not able to fulfill IPsec's requirements?

Thank you so much!

sindy · Wed Sep 29, 2021 9:11 pm

Show me the complete anonymized configuration from TIK0. As adding the route to phone's address at TIK1 was enough to make the responses reach TIK0, the idea with another IPsec policy at TIK0 is clearly not the answer, so it must be some misconfiguration of the firewall at TIK0.

danergo · Wed Sep 29, 2021 9:29 pm

Okay, what infos do you need exactly? Firewall only? Or Firewall+IPsec? Or Firewall+Route+IPsec?

Thank you!

sindy · Wed Sep 29, 2021 9:38 pm

Just post the full output of /export hide-sensitive after substituting public IPs and usernames in /ppp secret section. You can also remove static dhcp leases. Certificates and user names are not part of export even without hide-sensitive.

The thing is that you never know where the issue is hidden. If you knew where to look for it, there would be no issue.

danergo · Wed Sep 29, 2021 10:33 pm

Thanks!

Can I share it with you somehow more privately, by any chance?

danergo · Thu Sep 30, 2021 4:26 pm

/interface bridge
add admin-mac=MAC_OF_BRIDGE auto-mac=no comment=defconf name=bridge
add name=site2site-tunnel-vbridge
add comment="Remote VPN Clients" name=vpn-clients-vbridge
/interface ethernet
set [ find default-name=ether1 ] name=wan
/interface eoip
add local-address=192.168.20.3 mac-address= mtu=1400 name=\
    site2site-eoip1 remote-address=192.168.25.2 tunnel-id=3
add local-address=192.168.20.4 mac-address= mtu=1400 name=\
    site2site-eoip2 remote-address=192.168.25.2 tunnel-id=4
/interface bonding
add comment="Site2Site bonding" name=site2site-eoips-bonding slaves=\
    site2site-eoip1,site2site-eoip2
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=incoming
/ip ipsec policy group
add name=default-Site2Site
add name=default-RemoteVPN
/ip ipsec profile
add enc-algorithm=aes-256,aes-128 hash-algorithm=sha256 name=default-Site2Site
add enc-algorithm=aes-256,aes-128 hash-algorithm=sha256 name=default-RemoteVPN
/ip ipsec peer
add address=peer1.TIK1 exchange-mode=ike2 name=\
    Site2SitePeer1 port=34501 profile=default-Site2Site send-initial-contact=no
add address=peer2.TIK1 exchange-mode=ike2 name=\
	Site2SitePeer1 port=34502 profile=default-Site2Site send-initial-contact=no
add exchange-mode=ike2 name=IKE2-Acceptor \
    passive=yes profile=default-RemoteVPN send-initial-contact=no
/ip ipsec proposal
add auth-algorithms=null enc-algorithms=null name=default-Site2Site pfs-group=none
add auth-algorithms=sha256,sha1 enc-algorithms=aes-256-cbc,aes-128-cbc name=\
    default-RemoteVPN pfs-group=modp2048
/ip pool
add name=lan ranges=192.168.2.100-192.168.2.200
add name=ike2-vpn ranges=192.168.200.100-192.168.200.200
/ip dhcp-server
add address-pool=lan disabled=no interface=bridge name=defconf
/ip ipsec mode-config
add address-pool=ike2-vpn address-prefix-length=32 name=vpnpool-systemdns
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas\
    sword,web,sniff,sensitive,api,romon,dude,tikapp"
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=wan list=WAN
add interface=wan list=incoming
add interface=site2site-eoips-bonding list=incoming
add interface=vpn-clients-vbridge list=LAN
/ip address
add address=192.168.2.1/24 comment=defconf interface=bridge network=\
    192.168.2.0
add address=192.168.20.3 interface=site2site-tunnel-vbridge network=192.168.20.3
add address=192.168.20.4 interface=site2site-tunnel-vbridge network=192.168.20.4
add address=192.168.30.2/30 interface=site2site-eoips-bonding network=192.168.30.0
add address=192.168.170.65/24 interface=wan network=192.168.170.0
add address=192.168.200.1/24 comment="VPN bridge" interface=\
    vpn-clients-vbridge network=192.168.200.0
/ip dns static
add address=a.b.c.d name=peer1.TIK1
add address=a.b.c.d name=peer2.TIK1
/ip firewall address-list
add address=EXTERNAL_DNSNAME list=ExternalIP
add address=myip.com list=ThroughSite2SiteTunnel
/ip firewall filter
add action=log chain=forward log=yes log-prefix=TX src-address-list=\
    ThroughSite2SiteTunnel
add action=log chain=forward dst-address-list=ThroughSite2SiteTunnel log=yes log-prefix=\
    RX
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=accept chain=input comment="conf: accept IPsec" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="conf: accept GRE (for IPsec tunnels)" \
    ipsec-policy=in,ipsec protocol=gre
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid log=yes
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="defconf: accept all traffic from site2site tunnel" \
    in-interface=site2site-eoips-bonding
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN log=yes
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-mark=no-mark connection-state=established,related
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=accept chain=prerouting comment="est. rel. connections without conm\
    ark are accepted to prevent next rules" connection-mark=no-mark \
    connection-state=established,related
add action=accept chain=prerouting comment=\
    "download packets must not be routing-marked" connection-state=\
    established,related in-interface-list=incoming
add action=mark-routing chain=prerouting comment=\
    "connection marked packets marked for route also" connection-mark=\
    ipsec-site2site-tunnel-route new-routing-mark=ipsec-site2site-tunnel-route passthrough=no
add action=mark-connection chain=prerouting comment=\
    "only initial packets of connections get here past the rules above" \
    connection-state=new dst-address-list=ThroughSite2SiteTunnel new-connection-mark=\
    ipsec-site2site-tunnel-route passthrough=yes
add action=mark-routing chain=prerouting comment="initial packets of connectio\
    ns which evaded the rules above get here with no routing mark; we just \
    repeat the mark-routing rules above" connection-mark=ipsec-site2site-tunnel-route \
    new-routing-mark=ipsec-site2site-tunnel-route passthrough=no
/ip firewall nat
add action=src-nat chain=srcnat comment="!!!! srcnat disabled !!!!" \
    connection-mark=ipsec-site2site-tunnel-route disabled=yes log=yes to-addresses=\
    192.168.30.2
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add comment="Site2Site tunnels IDs" my-id=fqdn:TIK0.1 peer=Site2SitePeer1 \
    policy-template-group=default-Site2Site
add my-id=fqdn:TIK0.2 peer=Site2SitePeer2 policy-template-group=default-Site2Site
add auth-method=digital-signature certificate=ike2-vpnserver comment=\
    "Remote tunnel(s)" generate-policy=port-strict mode-config=\
    vpnpool-systemdns peer=IKE2-Acceptor policy-template-group=default-RemoteVPN
/ip ipsec policy
add comment="Site2Site tunnel policies" dst-address=192.168.180.10/32 level=unique \
    peer=Site2SitePeer1 proposal=default-Site2Site sa-dst-address=w.x.y.z \
    sa-src-address=0.0.0.0 src-address=192.168.20.3/32 tunnel=yes
add dst-address=192.168.180.10/32 level=unique peer=Site2SitePeer2 proposal=\
    default-Site2Site sa-dst-address=w.x.y.z sa-src-address=0.0.0.0 \
    src-address=192.168.20.4/32 tunnel=yes
add comment=Remote dst-address=192.168.200.0/24 group=default-RemoteVPN \
    proposal=default-RemoteVPN src-address=0.0.0.0/0 template=yes
/ip route
add comment="Site2Site IPsec tunnel" distance=1 gateway=192.168.30.1 routing-mark=\
    ipsec-site2site-tunnel-route
add distance=2 gateway=192.168.170.254

Here we go, thank you!

sindy · Thu Sep 30, 2021 9:23 pm

From what I can see in your mangle rules:

0 action=accept chain=prerouting connection-mark=no-mark connection-state=established,related
1 action=accept chain=prerouting connection-state=established,related in-interface-list=incoming
2 action=mark-routing chain=prerouting connection-mark=ipsec-site2site-tunnel-route new-routing-mark=ipsec-site2site-tunnel-route passthrough=no
3 action=mark-connection chain=prerouting connection-state=new dst-address-list=ThroughSite2SiteTunnel new-connection-mark=ipsec-site2site-tunnel-route passthrough=yes
4 action=mark-routing chain=prerouting connection-mark=ipsec-site2site-tunnel-route new-routing-mark=ipsec-site2site-tunnel-route passthrough=no

the initial packet from the phone (SYN) to the destination has connection-state=new, so it evades rules 0 and 1 that match on connection-state=established,related, it evades rule 2 because it doesn't have connection-mark ipsec-site2site-tunnel-route, and gets connection-marked by rule 3 and then routing-marked by rule 4
the next packets from the phone, if they eventually come, skip rule 0 because they do have a connection-mark, and match rule 1 because their in-interface is wan, so they match on in-interface-list=incoming, hence they do not reach rule 2 to get routing-marked.

So what happens to Scenario 2 if you add ipsec-policy=in,none to rule 1?

The firewall log from Scenario 2 in one of our previous posts shows the SYN to be received on wan and aimed to the tunnel, and SYN+ACK to be received on tunnel and aimed to wan, but you haven't shown the following packet from the phone. If it is a SYN one again, it means that the SYN+ACK one didn't make it to the phone; if so, the idea above is still relevant but it is not the explanation why the TCP session doesn't establish.

danergo · Thu Sep 30, 2021 10:50 pm

So what happens to Scenario 2 if you add ipsec-policy=in,none to rule 1?

What happens? It solves all my issues

Thank you so much Sindy! This was a huge investigation, I'm very thankful!

danergo · Sat Nov 27, 2021 5:19 pm

Hello!

I've came across to a small finetune:

I have here a virtual bridge "vpn-clients-vbridge" with an assigned IP Address "192.168.200.1/24".

This virtual bridge is part of an interface list of "LAN" along with the physical bridge:

/interface list member
add comment=defconf interface=bridge list=LAN
add interface=vpn-clients-vbridge list=LAN

I also have a firewall filter rule:

add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN log=yes

And Tik is operating on this domain:

/ip address
add address=192.168.2.1/24 comment=defconf interface=bridge network=\
    192.168.2.0

When my phone connects to the Tik with VPN, it accesses all the local area servers, and printers (192.168.2.x).

However, my phone is not able to access the Tik itself (via the Mikrotik app), with IP 192.168.2.1.

I added this to the filter list (before the drop rule above):

add action=accept chain=input comment=\
    "defconf: accept all traffic from vpn-clients" src-address-list=\
    vpn-clients

(vpn-clients list only has one single item: 192.168.200.0/24)

After the addition of this, my phone is able to connect to Tik (192.168.2.1) via VPN.

My question: the "drop filter" above seems dropping my connection as the in_interface is not the virtual bridge, but the wan interface.

It seems that virtual bridge is not used anywhere, can I (should I) somehow make IPSec to use the virtual bridge as source of remote VPN clients?

Thank you!

danergo · Sat Nov 27, 2021 5:24 pm

One small addition:

If I modify the accept rule:

add action=accept chain=input comment=\
    "defconf: accept all traffic from vpn-clients" ipsec-policy=in,ipsec

This also works.

Which led me to another question: Which one is more preferred? (this, or the other one with the IP range?)

Accepting an in,ipsec policy on input isn't a security hole? Or this is only accepted after the ipsec is correctly negotiated and authenticated?

Thank you!

sindy · Sat Nov 27, 2021 8:11 pm

It seems that virtual bridge is not used anywhere, can I (should I) somehow make IPSec to use the virtual bridge as source of remote VPN clients?

You cannot make the bridge a source (or rather in-interface) of the IPsec payload. The in-interface of IPsec payload is always the in-interface of the IPsec transport packet that has brought that payload. The bridge is there just as a virtual IP interface, so that you could hook 192.168.200.1/24 to it; the question is whether it makes any sense to have this address at all.

Which led me to another question: Which one is more preferred? (this, or the other one with the IP range?)

Accepting an in,ipsec policy on input isn't a security hole? Or this is only accepted after the ipsec is correctly negotiated and authenticated?

There are two things to look at.

First, the ipsec-policy match condition is actually just a macro that checks the packet's respective headers against all existing policies either directly (ipsec-policy=out,...) or inversely (ipsec-policy=in,...). But the IPsec RFC mandates that a received packet inversely matching an existing IPsec policy was dropped if it did not come in via a security association linked to that policy. This is the case even if the policy is not currently active because the related SA is not established. So in this sense, that rule is not a security hole - the rule itself would indeed match the packet even if it came in the "wrong" way, but the IPsec stack (or, more likely, an invisible dynamic firewall filter rule) drops it before it can reven each the manually configured filter rule.

Second, in more complex networks, the mere fact that a packet came in via an established IPsec SA does not automatically mean that that packet is allowed to get wherever it wants to get - e.g. you may be using IPsec to connect to customers' networks and you only want a limited set of application connections to get through, so in such cases, you still have to combine the ipsec-policy match condition with additional ones, such as src-address(-list) or even in-interface(-list) ones (no one says all IPsec connections must be established via a single interface).

Regarding which one is better in your particular case - my feeling is that matching a single prefix (src-address) is less CPU-intensive than matching a (src-address,src-port,ip-protocol,dst-address,dst-port) quintuple to a list of traffic selectors. Security-wise, there is no difference in your particular case.

danergo · Sat Nov 27, 2021 8:20 pm

Thank you!

idanatzis · Sun Oct 27, 2024 4:40 am

Hello team,
I am writing under this post because I think my issue is simillar to the one here.
I have a mikrotik connected behind an isp so I have a local ip and the isp has the public one.
I am trying to make the mikrotik an ikev2 server But my laptop connection seems to fail constantly and the reason based on the windows message is the there is not connectivity because of nat etc.
I have the exact same configuration to a customer where the public ip is on the mikrotik and it works like a charm.
Does anybody has an idea to suggest?

sindy · Sun Oct 27, 2024 2:30 pm

Your description suggests that Windows have the same default behavior when acting as an IKEv2 initiator like when acting as an L2TP/IPsec client, i.e. to terminate the connection if the responder/server is behind a NAT. There are two ways to address this - you can use regedit to change this behavior and make the Windows accept a server behind a NAT, or you can add the same public IP that runs on your ISP router also to the Mikrotik and dst-nat the incoming traffic back to that public address, so that the NAT detection mechanisms of IPsec would not notice the NAT at responder side. The latter way is better for scenarios with multiple client devices but only works if there is NAT on the client side or if the ISP router can forward ESP (most cannot). And it may trigger issues with pushing the routing table to the client.

There are multiple topics dealing with this here but right now I've failed to google up any of them

So if this brief description is not sufficient, come back for more detailed instructions.

idanatzis · Sun Oct 27, 2024 3:07 pm

Hello,
Thank you very much for your answer.
I too was trying to find some threads for this issue But this was the only one that I found.

I tried and changed the regedit of the windows laptop with no success.

I saw the solution with the dst nat But it really got me confused, if somebody has an example of the config it would be great.
Also my isp router cannot forward esp protocol, But is that needed also on ikev2? I am forwarding 500 and 4500 ports. Also I though of forwarding udp port 50 which somebody in the internet said it worked But not for what I need to do.

sindy · Sun Oct 27, 2024 3:37 pm

UDP port 50 and IP protocol 50 are not the same thing, and too many people who don't have a clue post authoritative statements on the internet.
IP protocol 50 indeed is ESP, and ESP indeed gets encapsulated into UDP if there is NAT between the peers, but not using port 50.

ESP itself has no notion of ports, which is why NATs have a tough time handling it, so to reliably traverse NAT, it must be encapsulated into UDP. Which is one of the things the peers do if they detect the presence of NAT on the path between them. IKEv2 uses a different message format and procedures as compared to IKE (v1), but both create the same Phase 2 security associations that use the same ESP (or, rarely, AH).

The configuration for the "forth-and-back dstnat" is actually very easy, but before diving into it - does the CN and SAN of the certificate you use on the Mikrotik refer to your public IP, to the private IP of the Mikrotik, or to the FQDN?

idanatzis · Sun Oct 27, 2024 3:48 pm

It refers to the ddns fqdn.

sindy · Sun Oct 27, 2024 4:05 pm

OK, so the certificate contents is correct but the fact that the public IP is dynamic makes the configuration more complicated.

As for the registry change - as said I have never tried that with IKEv2, so I'm not sure whether the embedded Windows VPN client indeed has an issue with a NATed responder and if it does, whether the same bit in the registry that helps for L2TP/IPsec can be used to change the behavior. So maybe we have to choose another tree to bark at.

So just for a quick proof of concept - add a new bridge, name it br-lo and do not add any ports to it. Then attach the current public address as a /32 one to that bridge, and set up a dstnat rule that will dst-nat whatever comes in via WAN to UDP ports 500 and 4500 to this address. Assuming that you already have the port forwarding in place on the ISP router, that's all - now try connecting the client from the outside (e.g. via a mobile hotspot). At least 5 minutes must elapse since the last connection attempt from that client so that the non-dst-nated connection could time out from the connection tracking.

If that works, we'll have to find a way to track the current public address and keep the dst-nat rule and the internal copy of the address in accord. How often does the ISP change the public address?

idanatzis · Sun Oct 27, 2024 6:32 pm

hi,
I have already a script that trucks the ISP's public and the truth is that it does not change often.
Now that I made that change I get the message:
no policy found/generated

The config I am using is:
/interface bridge add name=bridge-loopback
/ip address add address=172.16.10.33 interface=bridge-loopback network=172.16.10.32/28
/ip pool add name="pool vpn.ike2.xyz" ranges=172.16.10.34-172.16.10.46
/certificate add name=CA.ike2.xyz country=MY state=Selangor locality=Cyberjaya organization=IKE2.xyz common-name=ca.ike2.xyz subject-alt-name=DNS:ca.ike2.xyz key-size=2048 days-valid=3650 trusted=yes key-usage=digital-signature,key-encipherment,data-encipherment,key-cert-sign,crl-sign
/certificate sign CA.ike2.xyz
/certificate add name=xxxxxxxxx.sn.mynetname.net country=MY state=Selangor locality=Cyberjaya organization=IKE2.xyz unit=VPN common-name=xxxxxxxxx.sn.mynetname.net subject-alt-name=DNS:xxxxxxxxx.sn.mynetname.net key-size=2048 days-valid=1095 trusted=yes key-usage=tls-server
/certificate sign xxxxxxxxx.sn.mynetname.net ca=CA.ike2.xyz
/certificate add name=ekl_arx_1@xxxxxxxxx.sn.mynetname.net country=MY state=Selangor locality=Cyberjaya organization=IKE2.xyz common-name=ekl_arx_1@xxxxxxxxx.sn.mynetname.net subject-alt-name=email:ekl_arx_1@xxxxxxxxx.sn.mynetname.net key-size=2048 days-valid=365 trusted=yes key-usage=tls-client
/certificate add copy-from=ekl_arx_1@xxxxxxxxx.sn.mynetname.net name=c1@xxxxxxxxx.sn.mynetname.net common-name=c1@xxxxxxxxx.sn.mynetname.net subject-alt-name=email:c1@xxxxxxxxx.sn.mynetname.net
/certificate sign c1@xxxxxxxxx.sn.mynetname.net ca=CA.ike2.xyz
/certificate export-certificate c1@xxxxxxxxx.sn.mynetname.net type=pkcs12 export-passphrase=xxxxxx
/certificate export-certificate CA.ike2.xyz
/ip ipsec mode-config add address-pool="pool vpn.ike2.xyz" address-prefix-length=32 name="modeconf vpn.ike2.xyz" split-include=0.0.0.0/0 static-dns=172.16.10.33 system-dns=no ;
/ip ipsec proposal add auth-algorithms=sha512,sha256,sha1 enc-algorithms=aes-256-cbc,aes-256-ctr,aes-256-gcm,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-gcm lifetime=8h name="proposal vpn.ike2.xyz" pfs-group=none
/ip ipsec profile add dh-group=modp2048,modp1536,modp1024 enc-algorithm=aes-256,aes-192,aes-128 hash-algorithm=sha256 name="profile vpn.ike2.xyz" nat-traversal=yes proposal-check=obey
/ip ipsec policy group add name="group vpn.ike2.xyz"
/ip ipsec policy add dst-address=172.16.21.0/30 group="group vpn.ike2.xyz" proposal="proposal vpn.ike2.xyz" src-address=0.0.0.0/0 template=yes sa-src-address=0.0.0.0 sa-dst-address=0.0.0.0 ipsec-protocols=esp level=require protocol=all action=encrypt
/ip ipsec peer add exchange-mode=ike2 address=0.0.0.0/0 local-address=xxxxxxxxx name="peer xxxxxxxxx.sn.mynetname.net" passive=yes send-initial-contact=yes profile="profile vpn.ike2.xyz"
/ip ipsec identity add auth-method=digital-signature certificate=xxxxxxxxx.sn.mynetname.net remote-certificate=c1@xxxxxxxxx.sn.mynetname.net generate-policy=port-strict match-by=certificate mode-config="modeconf vpn.ike2.xyz" peer="peer xxxxxxxxx.sn.mynetname.net" policy-template-group="group vpn.ike2.xyz" remote-id=user-fqdn:c1@xxxxxxxxx.sn.mynetname.net
/ip firewall mangle add action=change-mss chain=forward new-mss=1360 src-address=172.16.21.0/30 protocol=tcp tcp-flags=syn tcp-mss=!0-1360 ipsec-policy=in,ipsec passthrough=yes comment="IKE2: Clamp TCP MSS from 172.16.21.0/30 to ANY"
/ip firewall mangle add action=change-mss chain=forward new-mss=1360 dst-address=172.16.21.0/30 protocol=tcp tcp-flags=syn tcp-mss=!0-1360 ipsec-policy=out,ipsec passthrough=yes comment="IKE2: Clamp TCP MSS from ANY to 172.16.21.0/30"
/ip firewall nat add chain=srcnat src-address=172.16.21.0/30 out-interface=eth1 ipsec-policy=out,none action=masquerade comment="MSQRD IKE2:172.16.21.0/30

idanatzis · Sun Oct 27, 2024 6:49 pm

Hi there,
There was a configuration mistake.
I corrected it and everything is alright.
Thank you!

sindy · Sun Oct 27, 2024 6:59 pm

First, it makes the posts better readable if you post the configurations between [code] and [/code] tags (one way to get them is to press the </> button in the editing toolbar).

Next, what you posted is not a complete export of the actual configuration but rather a recording of the configuration steps you have executed, so some things may be missing there.

In any case, no policy found/generated indicates that the initial authentication has succeeded. So looking into it, there is a mismatch between the dst-address of the policy template and the pool you assign the address from - you give out addresses from 172.16.10.34-172.16.10.46 so the initiator suggests a policy with one of these addresses at its end but the policy template only allows 172.16.21.0/30 there, no wonder it fails.

Other than that - did you have to assign the public address to a bridge interface to make it work or it turned out not to be necessary after all? What you've posted does not answer this question.