I have a router with 3 IPsec tunnels configured. Normally everything works as expected, but after few days all 3 tunnels stop at the same time. The only way I can correct the situation is to reboot the router. After the reboot all 3 tunnels are fully functional again.

I can't find any reason for this. My main questions is: how can I find what went wrong when it happens?

On the remote hosts there is nothing unusual in logs. Only entries about expired SAs and new SAs.

The endpoints can ping each other.

There are vaild phase1 (ISAKMP) associations and phase2 (SA) associations as well. They match on both sides. If I reset any of these on the router, new associations will be negotiated within seconds, but it does not help.

Remote osts are sending AH/ESP packets to the router, but no packets are coming from the router.

HW, SW is RB2011UiAS, 6.18. I have just upgraded to 6.19 and I wait for this error to occur again. This may take a week.

Check by netwatch if tunnel works. When it stops responding, remove SAs. It will link again.

Do both ends use the very same NTP server?
I had exactly this issue a year ago when my VPN RB was syncing with a different NTP server than the remote peer.
Bringing both on the same NTP server instantly solved the problem.
-Chris

Check by netwatch if tunnel works. When it stops responding, remove SAs. It will link again.

As I wrote, if I remove SAs, new SAs will be installed, but the connections remain unusable.

It happens to all connections in the same time. I guess there is nothing wrong with individual SAs, individual peers, and policies. Something seems to affect the whole IPsec function.

If it happens again, I will try to check if packets entering the router from LAN are leaving the router encapsulated on WAN. I have no ideas what else to check.

Do both ends use the very same NTP server?
I had exactly this issue a year ago when my VPN RB was syncing with a different NTP server than the remote peer.
Bringing both on the same NTP server instantly solved the problem.

Very strange. Is there an explanation for this? IMHO, IPsec should work even if they were not sychronized at all.

To answer your question: both ends use NTP, but not the same server. Remote hosts use a random server from a NTP pool for Europe. The router gets its time from the central server on the LAN which in turn is synchronized using the mentioned pool.

When removing sas it has to be done on both sides simultaneously. The synchronised time is really necessary.

When removing sas it has to be done on both sides simultaneously.

This is one of the things I have tried. It did not help.

OTOH, If I reboot only the router, the problem is gone. No action on the remote side necessary.

That's why I must disagree in this particular case.

The synchronised time is really necessary.

All hosts have exact NTP time from public stratum 2 or stratum 3 servers.

I just wrote what helped to me. Before I stopped to use ipsec I used such scripts that worked. Maybe your problem is different...

It took 8 days this time.

I tried it again and I have to repeat myself that none of these helps: Even if combined with a similar action on the remote site, it does not help. All that happens is that new phase1 and phase2 SAs appear quickly as they should.

I tried a ping from a remote site over the IPsec tunnel. The packet sniffer shows that:
- AH/ESP packets do arrive from WAN,
- decapsulated ping requests are sent to LAN,
- ping replies are coming back,
- but routerboard does not send any encapsulated AH/ESP packets.

I checked CPU usage and free memory. No problems there.

I have no idea what else to check or to reset. On a Linux host I would check the current state of SPD (Security Policy Database) in the kernel with setkey -DP.

After 45 minutes I rebooted the router to restore normal operation.

IPsec down again. I think I will write an automatic reboot script.

It looks to me like a RouterOS issue, but I got no "me too" replies. If there is anybody reading this and having multiple IPsec tunnels configured, I would appreciate a short note.

Hi vladop,
We are having an annoying issue like this too.
Slightly different to you, we have an IPsec between a pfsense(SiteA) and a Mikrotik(SiteB) and another IPsec between pfsense(SiteA) and pfSense(SiteC).
It works fine for hours, and then randomly stops. Usually when the Mikrotik(SiteB end aDSL drops for a minute or so, but not always.
When it happens both ends complain about time outs waiting for phase1.

If I reboot the Mikrotik(siteB) the IPsec comes back fine.
Nothing ever seems to go wrong with the SiteA <--> SiteC IPsec

The odd thing is that we replaced the SiteB pfsense with the Mikrotik (because we thought it a problem with SiteB) and it's not made much difference at all.

Reading below, I wonder if it is is some sort of timing problem......
The SiteA logs have(note I have edited them and changed the IPs and spi values etc):
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: [218.214.147.172] INFO: DPD: remote (ISAKMP-SA spi=0d35yyyyye0b9d5b0:1cb0yyyy46cb1) seems to be dead.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purging ISAKMP-SA spi=0d35a5byyyy5b0:1cb039yyyy46cb1.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purged IPsec-SA spi=510556195.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purged IPsec-SA spi=1307544530.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purged IPsec-SA spi=2888775497.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purged IPsec-SA spi=147348029.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: purged ISAKMP-SA spi=0d35a5byyyy5b0:1cb039yyyy46cb1.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: ISAKMP-SA deleted yy.yy.yy.yy[500]-xx.xx.xx.xx[500] spi:0d35a5b2yyyyy5b0:1cb0yyyyy346cb1
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: IPsec-SA request for xx.xx.xx.xx queued due to no phase1 found.
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: initiate new phase 1 negotiation: yy.yy.yy.yy[500]<=>xx.xx.xx.xx[500]
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: INFO: begin Identity Protection mode.

Any ideas?

Rgds Ben

The SiteA logs have(note I have edited them and changed the IPs and spi values etc):
2014.09.30-12:46:26 <yy.yy.yy.yy>: <30>Sep 30 12:46:26 racoon: [218.214.147.172] INFO: DPD: remote (ISAKMP-SA spi=0d35yyyyye0b9d5b0:1cb0yyyy46cb1) seems to be dead.
.....

Thank you for your note. I do not see anything unusual in the logs you have posted. I have disabled DPD, but our racoon logs look normal too.

Did you tried anything else except rebooting the mikrotik router?

We have two Internet connections. I have reconfigured one of the IPsec tunnels to other provider's link. I will wait what happens. If the problem is triggerred by a link outage, I will get 2 IPsec tunnels unusable instead of all 3.

For the record: during the last problem occurence, I had two simultaneous session between same hosts opened. One becomes dead and the other was fine for a while (less than a minute). Then it froze too.

I had similar problems but just with SIP (or certain UDP connections) over IPSEC. It was triggered by a port flap on an Internet modem or other Internet outage. The problem seems to have been addressed in 6.18/6.19.

However, before the firmware updates were released I noticed that fixing the router port speed of the modem dramatically reduced the problem (e.g. if the port shows the modem auto-negotiates at 100M-full, turn off auto-negotiation and fix the router port speed to 100M-full). I never turned the port speeds back to auto-negotiate after the firmware update, so it's possible the combination of firmware update to 6.18/6.19 and fixing the port speed were required to fix the problem.

We have two Internet connections. I have reconfigured one of the IPsec tunnels to other provider's link. I will wait what happens. If the problem is triggerred by a link outage, I will get 2 IPsec tunnels unusable instead of all 3.

It did break down again few days ago. All IPsec's were down no matter what interfaces were used for the communication.

I'm happy I don't have to explain it to a customer.

Small update: We did some upgrades. The router is running 6.23 and one of the ipsec endpoints is now using libreswan instead of racoon.

Unfortunately it did not help. I had to reboot this morning. The uptime was 2w4d but there was no significant traffic during Xmas and new year holliday time.

Hi. Have this error on 6.27. Anyone have solution?

I started this topic.

I hope I'm not wrong, but it seems that the problem is gone. I have 29 days uptime without ipsec problems.

I was upgrading often and some of the fixes in the RouterOS did it.

sorry to resurrect an old thread but I'm seeing what seems to be this issue on 6.35.2 on a CCR1016-12G.

i'm running l2tp with ipsec between the 2 local networks at each site. the l2tp tunnels stay up and the ipsec tunnel seems to establish with no errors but if you look at current bytes under installed SAs there is no traffic in one direction. as soon as i reboot the router traffic starts to flow in both directions.

Any help would be appreciated?

Thanks

I too have this problem, if the internet connection (PPPoE in my case) goes down and up quickly, it stalls on PH2 state "no phase 2", rebooting the router again, or removing the 'remote peers' re-establishes the links.

We are experiencing this in 5 different site to site connections.

All are RB3011 running either 6.38.1 and 6.38.5.

Had this too.
ipsec works, after x hours no traffice but tunnel is still up.

I found this:
https://mum.mikrotik.com/presentations/ ... 360368.pdf

I changed my nat rule.
Every example I found for an ipsec tunnel has a nat exclusion rule.
On page 53 of this pdf presentation I found a (new?) recommended way to do this.

/ip firewall raw
add action=notrack chain=prerouting srcaddress=10.1.101.0/24 dst-address=10.1.202.0/24
add action=notrack chain=prerouting srcaddress=10.1.202.0/24 dst-address=10.1.101.0/24

page 52:
Raw table
● Firewall RAW table allows to selectively bypass
or drop packets before connection tracking thus
significantly reducing the load on the CPU.
● If packet is marked to bypass connection
tracking:
– Packet de-fragmentation will not occur.
– NAT will be skipped.
– Options that depend on connection tracking will not
trigger (fasttrack-connection, mark-connection,
layer7 etc.)
– Will have connection-state=untracked.

Haven't tested for days yet, but today is the first day I have a tunnel up and running longer than 24h.
Maybe this did the trick and it might help someone who got here like me.

This is old, but it's the best hit I get on this issue. Ipsec was freezing every 2-3 days and requiring a reboot.

Based loosely on the docs + stuff written here I've added a couple of lines.

/ip firewall raw
add action=notrack chain=prerouting ipsec-policy=in,ipsec
add action=notrack chain=output ipsec-policy=out,ipsec

I've also removed all the conntrack stuff (bypass nat, fasttrack) I had before as it appears to be no longer needed. Hopefully this will resolve the issue as it did for the previous poster.