Hello!

I'm from Croatia and I've setup myself a CHR router in Amsterdam. I've connected 3 locations to it with IKEv2 IPSEC, but I'm having a problem with a connection with one of them.
Two of the three locations works as expected, pinging to a public IP of a CHR and pinging to a private IP of a CHR through IPSEC is showing only marginal difference in latency. The remaining location is showing much higher latency when pinging through IPSEC instead of just pinging the Public IP. Are there any known common causes of such issues? The ISP at the location is not perfect but I suppose it shouldn't make such a drastic difference?
The location with difficulties in question runs on a WISP provider. The locations that work properly are on LTE and Fiber.

Can't deny that it might be caused by misconfiguration, it's gotten a bit messy and if nothing else works I will try and start everything from scratch once I'm back on the location with difficulties (which is 500km away from me)

Additional info: CHR is 6.49.1, the client on the questionable location is RB4011 with 6.49.2.

I've also now tried setting up IPSEC client on one of the APs (951G) at the location to the CHR, and it's the same issue.. Though the traffic is forwarded through the previously mentioned router.

Are there any known common causes of such issues? The ISP at the location is not perfect but I suppose it shouldn't make such a drastic difference?
The location with difficulties in question runs on a WISP provider. The locations that work properly are on LTE and Fiber.

Can't deny that it might be caused by misconfiguration, it's gotten a bit messy and if nothing else works I will try and start everything from scratch once I'm back on the location with difficulties (which is 500km away from me)

I have seen issues related to packet fragmentation, but there is no reason for packet fragmentation for 50-byte IMP packets.

Network-wise, the difference between pinging the public address of the CHR and pinging its internal one via the tunnel is that in the latter case, it's UDP traffic, so if your ISP for the problematic site applies some selective magic (e.g. prioritizes pings over other traffic to look better in benchmark tests), it could be the explanation.

Instead of travelling 500 km, I'd suggest to set up another type of VPN, e.g. an SSTP one, in parallel to the IPsec one. SSTP is slower than IPsec and it uses TCP as transport, which causes its own sort of problems if packet loss occurs (TCP meltdown), but it is equally secure as IPsec and it is sufficient to manage a remote router, so you can use it to change the IPsec configuration on it. But I don't think that what you see is caused by an IPsec misconfiguration - the packets are not lost, only delayed.

Are there any known common causes of such issues? The ISP at the location is not perfect but I suppose it shouldn't make such a drastic difference?
The location with difficulties in question runs on a WISP provider. The locations that work properly are on LTE and Fiber.

Can't deny that it might be caused by misconfiguration, it's gotten a bit messy and if nothing else works I will try and start everything from scratch once I'm back on the location with difficulties (which is 500km away from me)

I have seen issues related to packet fragmentation, but there is no reason for packet fragmentation for 50-byte IMP packets.

Network-wise, the difference between pinging the public address of the CHR and pinging its internal one via the tunnel is that in the latter case, it's UDP traffic, so if your ISP for the problematic site applies some selective magic (e.g. prioritizes pings over other traffic to look better in benchmark tests), it could be the explanation.

Instead of travelling 500 km, I'd suggest to set up another type of VPN, e.g. an SSTP one, in parallel to the IPsec one. SSTP is slower than IPsec and it uses TCP as transport, which causes its own sort of problems if packet loss occurs (TCP meltdown), but it is equally secure as IPsec and it is sufficient to manage a remote router, so you can use it to change the IPsec configuration on it. But I don't think that what you see is caused by an IPsec misconfiguration - the packets are not lost, only delayed.

Thanks for taking your time to help me!
My WISP is unreliable and unstable... using Ubiquity to connect people through trees and buildings. -70 signals are acceptable for them in not so clean spectrum.
But I suppose it should still work better than LTE in crowded place? And I don't think they would actually setup any kind of "selective magic" like that... though I wouldn't be surprised either.

I can access the router at the location through public IP as well, allowing Winbox until I finish with everything.. so it doesn't pose much of a problem to access while I'm reconfiguring IPSEC... it's just that I'm doubting I might have f'ed up that badly to have such a drastic difference latency-wise, and I don't want to go through the trouble of redoing everything over again. Though I might configure a currently unused router there with some basic configuration and tell them to replace the 4011 temporary so I can test things out.

But as I said, I tried 951 as well, it's just that it's still going through the 4011 so I suppose I can't say I tried everything out...

I can paste my configurations here but I'll need time to clean them up before posting them here.

Are there any other performance tests between routers that I can do beside the usual ICMP ping and bandwidth test?

I can paste my configurations here but I'll need time to clean them up before posting them here.

More eyes usually see more, hence worth it.

Are there any other performance tests between routers that I can do beside the usual ICMP ping and bandwidth test?

Just sniff at the WAN interface of the 4011 while pinging via IPsec. If you can see the jitter already in the HR->NL direction, the IPsec enryption is guilty; if you can only see it as late as in the NL->HR direction, it is the ISP.

/tool sniffer quick ip-address=ip.of.the.chr ip-protocol=ipsec-esp

(since you say you can access the 4011 remotely without a VPN, I assume it has got a public IP address directly on itself so bare ESP is used as the transport protocol).

I can paste my configurations here but I'll need time to clean them up before posting them here.

More eyes usually see more, hence worth it.

Are there any other performance tests between routers that I can do beside the usual ICMP ping and bandwidth test?

Just sniff at the WAN interface of the 4011 while pinging via IPsec. If you can see the jitter already in the HR->NL direction, the IPsec enryption is guilty; if you can only see it as late as in the NL->HR direction, it is the ISP.

/tool sniffer quick ip-address=ip.of.the.chr ip-protocol=ipsec-esp

(since you say you can access the 4011 remotely without a VPN, I assume it has got a public IP address directly on itself so bare ESP is used as the transport protocol).

Not sure if I can read it properly, but I would say the Jitter ain't caused by the 4011 here, right? EDIT:
When I sniff CHR and ping from 4011 When I sniff 4011 and ping from CHR I suppose this would also be some kind of proof that IPsec is working with barely any delay?
I never used sniff before, at least not like this. It's cool!

I suppose this would also be some kind of proof that IPsec is working with barely any delay?

Yup, IPsec encryption and decryption is not the reason of the jitter.

You may try a ping to the CHR's public address using slightly larger packets, matching the size of the ESP ones, to see whether it's the packet size or the IP protocol that makes the difference.

I suppose this would also be some kind of proof that IPsec is working with barely any delay?

Yup, IPsec encryption and decryption is not the reason of the jitter.

You may try a ping to the CHR's public address using slightly larger packets, matching the size of the ESP ones, to see whether it's the packet size or the IP protocol that makes the difference.

Ping is identical with size of 118 or 1400 bytes with Don't Fragment turned on, working just like ping with 50 bytes of data.