After Upgrade from 6.49.1 to 7.1 ipsec Site-Site not working

trolrolo · Tue Dec 07, 2021 2:55 pm

I have recently bought CCR2004 with 7.1 soft. I have Configured ipsec Site-Site tunnel according to instruction on many sites. Tunnel was established between CCR2004 soft 7.1 and RB3011 soft 6.49.1)
eg. https://www.informaticar.net/how-to-est ... k-routers/
Tunnel is established but there is not transfer between sites. Of course there is a rule in NAT (in the first place) that accepts packets form sites not to go through NAT.

I put the same configuration to router RB4011 with soft 6.49.1. Tunnel is established between RB4011 (6.49.1) and RB3011 (6.49.1). Everything is working OK communication between sites is working OK.

After upgrade RB4011 from 6.49.1 to 7.1 communication stopped working. No packed are transferred through IPSEC Tunnel. Tunnel itself is established, but no packet between sites are going....
Is there anything special that should I setup on 7.1 soft or this is just a bug of 7.1 soft (I have tried 7.1 rc7, but the problem persists). I don't have access to older soft from 7 version...

My configuration of IPSEC tunnel is simple:
IPSEC Configuration RB4011/CCR2004 v7.1 OS
LAN IP SRC RB4011/CCR2004 v7.1 OS: 192.168.10.1/24
LAN IP RB3011 (Poznan) v6.49.1 OS: 192.168.29.1/24

/ip ipsec profile add dh-group=modp1024 enc-algorithm=3des name=Phase2
/ip ipsec peer add address=111.111.111.22/32 name=Poznan profile=Phase2
/ip ipsec proposal add enc-algorithms=3des lifetime=1h name=Phase1
/ip ipsec identity add peer=Poznan remote-id=ignore secret=PoznanPassword
/ip ipsec add dst-address=192.168.29.0/24 peer=Poznan proposal=Phase1 src-address=192.168.10.0/24 tunnel=yes

IP NAT (in the beginning of roules)
/ip firewall nat add action=accept chain=srcnat dst-address=192.168.10.0/24 src-address=192.168.29.0/24
/ip firewall nat add action=accept chain=srcnat dst-address=192.168.29.0/24 src-address=192.168.10.0/24

Andreywys · Fri Dec 10, 2021 12:59 am

I am also confirming the problem.

theprojectgroup · Mon Dec 13, 2021 3:27 pm

Same here + L2TP IPSEC Clients can't connect.
CCR1016-12G

L2TP Clients connects and fails with error "server did not respond"

14:12:59 ipsec,info respond new phase 1 (Identity Protection): 212.114.xx.xx[500]<=>80.187.82.203[500]
14:12:59 ipsec received Vendor ID: RFC 3947
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-08
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-07
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-06
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-05
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-04
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-03
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
14:12:59 ipsec received Vendor ID: draft-ietf-ipsec-nat-t-ike-02\n
14:12:59 ipsec received Vendor ID: FRAGMENTATION
14:12:59 ipsec Fragmentation enabled
14:12:59 ipsec received Vendor ID: DPD
14:12:59 ipsec 80.187.xx.xxSelected NAT-T version: RFC 3947
14:12:59 ipsec sent phase1 packet 212.114.xx.xx[500]<=>80.187.82.203[500] 6808335ad5fa9786:355bce9f67760205
14:12:59 ipsec NAT detected: ME PEER
14:12:59 ipsec Adding remote and local NAT-D payloads.
14:12:59 ipsec sent phase1 packet 212.114.xx.xx[500]<=>80.187.82.203[500] 6808335ad5fa9786:355bce9f67760205
14:12:59 ipsec NAT-T: ports changed to: 80.187.82.203[14545]<=>212.114.xx.xx[4500]
14:12:59 ipsec KA list add: 212.114.xx.xx[4500]->80.187.82.203[14545]
14:12:59 ipsec 80.187.xx.xxignore INITIAL-CONTACT notification, because it is only accepted after phase1.
14:12:59 ipsec,info ISAKMP-SA established 212.114.xx.xx[4500]-80.187.82.203[14545] spi:6808335ad5fa9786:355bce9f67760205
14:13:00 ipsec respond new phase 2 negotiation: 212.114.xx.xx[4500]<=>80.187.82.203[14545]
14:13:00 ipsec searching for policy for selector: 212.114.xx.xx:1701 ip-proto:17 <=> 80.187.82.203:59792 ip-proto:17
14:13:00 ipsec generating policy
14:13:00 ipsec Adjusting my encmode UDP-Transport->Transport
14:13:00 ipsec Adjusting peer's encmode UDP-Transport(4)->Transport(2)
14:13:00 ipsec sent phase2 packet 212.114.xx.xx[4500]<=>80.187.82.203[14545] 6808335ad5fa9786:355bce9f67760205:00001611
14:13:00 ipsec IPsec-SA established: ESP/Transport 80.187.82.203[14545]->212.114.xx.xx[4500] spi=0xb40aa34
14:13:00 ipsec IPsec-SA established: ESP/Transport 212.114.xx.xx[4500]->80.187.82.203[14545] spi=0xbd73b8
14:13:00 ipsec -> ike2 request, exchange: INFORMATIONAL:634 13.95.9.128[4500] 6cbd8f62cb636534:95bd53d95feb7fe0
14:13:00 ipsec payload seen: ENC
14:13:00 ipsec processing payload: ENC
14:13:00 ipsec respond: info
14:13:00 ipsec <- ike2 reply, exchange: INFORMATIONAL:634 13.95.9.128[4500] 6cbd8f62cb636534:95bd53d95feb7fe0

theprojectgroup · Mon Dec 13, 2021 10:14 pm

Turns out it is working, same for l2tp ppp dial-in - but only right after a fresh boot.
After a few minutes all tunnel die.

trolrolo · Tue Dec 14, 2021 9:06 am

In my situation after reboot nothing changed tunnels where established but no transfer between them.
Maybe my test was on 28 tunnels...But with 6.49 everything was OK. Only soft upgrade to 7.1 and everything fall down.

I have wrote to support@mikrotik.com but no response....

Andreywys · Mon Dec 20, 2021 6:36 pm

Hey guys from developers, can you answer in this topic please.

mikruser · Mon Dec 20, 2021 9:45 pm

No. They already celebrate Christmas and annual bonuses.

atakacs · Tue Dec 21, 2021 8:09 am

Some "official" response would indeed be appreciated...

trolrolo · Tue Dec 21, 2021 8:50 am

support@mikrotik also have already Christmas.
Sorry but waiting 2 weeks for the answer that they products doesn't work (CCR with obligatory OS 7) is something that is not right.

bbartlomiej · Tue Dec 21, 2021 4:29 pm

For me the tunnel was working but I experience massive packet drops. WIreshark showed a lot of TCP DUPs and Retransmissions. After changing the underlay from IPSec to WireGuard it is smooth now. On top I have a GRE tunnel with OSPF. No DUPs and Retransmissions now.

mabooshi · Thu Dec 23, 2021 11:35 pm

I've tested 7.x version few days ago , there was a lot of problem with it, such as routing tunnels and so on.
Seriously I suggest all of you not to test 7.x version in production environments just use stable versions.

Andreywys · Sun Dec 26, 2021 9:41 pm

trolrolo,

Try to change 3des algorithm to aes-128 cbc

talavs · Mon Dec 27, 2021 7:43 pm

+1 Confirming this problem.
IPSEC tunnel and connections to remote computers via RDP works while on 6.49.1.
After upgrading to 7.1 IPSEC tunnel is established without errors, but I am unable to access remote resources. In IPSEC "Active peers" tab there are zero Rx Bytes/packets.

trolrolo · Tue Dec 28, 2021 10:46 am

Try to change 3des algorithm to aes-128 cbc

Tunnel works ok with 3des, Tunnel encryption should not have influence to routing. I have some old routers on the other side and I need to use 3des instead aes-128

usego · Wed Dec 29, 2021 12:01 pm

( I have to stop playing with new "stable" releases on holidays!

)

Same story there. Tunnels work for 5-10 seconds after 7.1.1 router reboots and stop then. I've "fixed" that by setting Check Gateway = none in routes

atakacs · Thu Jan 13, 2022 6:22 pm

Was there any resolution to that ? Still pretty much seeing the same problem :/

temnikiov · Fri Jan 21, 2022 4:24 pm

We had the same issue on RB1100AHx4 after upgrade from v6.49.2->v7.1.1.. IPSec VPN tunnel is estableshed fine but packets weren't routed to the tunnel. RDC to a client host stopped working after the upgrade.

Had to downgrade to v6. Found several bugs while the downgrade. All of the found bugs were related with IP\Firewall\Mangle fields mapping.

temnikiov · Fri Jan 21, 2022 4:55 pm

More details. We use 2 WANs and so have several IP\Firewall\mangle settings. While the downgrade I faced with field mapping bugs. [New] Connection mark and [New] Routing mark fields were filled incorrectly. I even lost connection to the office network and had to ask collegues to help. I'm not ready to check on production router but I propose there is the same bug with field mapping while upgrading V6->V7. And that may be the reason why the routing fails.

breal · Mon Jan 24, 2022 8:16 pm

Same problems here. http/https sessions are not working. icmp does go through the tunnel

I haven't been able to get my hex-s working again. Rollback to 6.49 didn't fix the issue, even after a clean wipe with netinstall.
It's driving me crazy

Andreywys · Thu Jan 27, 2022 4:01 pm

IPsec on RB2011 with fw 7.1 works fine, this problem i have on CCR.

breal · Fri Jan 28, 2022 10:09 am

The issues were related to MTU size.
I was able to solve this by lowering the interface MTU in combination with an MSS clamping rule.
IKEv2 now running fine on 7.1.1

Dude2048 · Fri Jan 28, 2022 11:47 am

What size did you use?

breal · Fri Jan 28, 2022 11:54 am

1422, but it depends on the authentication header size

Minddaugas · Thu Feb 10, 2022 11:55 am

Hi,

7.1.2 is out. Does anyone already tested if the problem persists in the latest RouterOS version?

icttech · Sat Feb 12, 2022 2:07 am

I'm having same issue with L2TP w/IPsec on static routes breaking after either side is reset or rebooted. v7.1.2 on ccr1009-7g-1c-1s+ . Not an issue before upgrading from 6.49.

mloaiza · Sat Feb 12, 2022 3:12 am

HI,

Having the same issue with l2tp/IPsec if I upgraded to 7.1.2.

"Client" side RB3011 is on 7.1.2
"Server" side CCR1036-12G-4S still 6.49.2

Buffer: Memory
Topics: ipsec
error

message : phase1 negotiation failed due to send error. 18X.17X.1XX.XXX[500]<=>20X.XX.2X.XXX[500] d13df58429b61179:0000000000000000

If I take the "client" back to 6.49.2, it works just fine.

ygckmzsp · Tue Feb 15, 2022 10:26 am

I have the same issue after upgrading from 6.48.1 to 6.49.2.
Also I've noticed that this issue related to connections with cert authorization in my case, my IPsec tunnel with PSK secret works.
PPTP connections stopped working too.

LetMeRepair · Thu Feb 17, 2022 1:23 pm

started various attempts with various CCR1009 with 7.x since official release. IPSec problems keep returning, after certain runtime (sometimes hours, sometimes days) CPU will raise towards 90-100%, IPSec tunnel shows established, but routes through the tunnel are failing because ping check fails. Disabling IPSec policy makes the associated tunnels run stable.

MTU is an interesting thing to check, will play a little once the problems return.

Cray · Fri Feb 25, 2022 3:17 pm

I'm having this issue with latest 7.1.3 release.

IPSec site-to-site tunnels seems to work between ROS 6.x <-> ROS 7.x, but two ROS 7.x routers are unable to keep traffic flowing after tunnel has been established.

IPSec tunnel negotiates phase 1 and phase 2 successfully (policy state: established) but traffic just stop flowing after 10-15 seconds.

All works perfectly with exactly same configuration if either (or both) end of the link is downgraded to ROS 6.49.3.

End result being that after several attempts of diagnosing the issue I'm unable to get reliable site-to-site tunneling work between ROS v7 devices.
Configuration used is in no way "exotic" for an IPSec tunneling.

Has anyone been able to find out what exactly is stopping the traffic flow in ROS7?

hkusulja · Fri Feb 25, 2022 7:20 pm

Hello,
After upgrade from 6.49.1 to 7.1 i did have some issues with ipsec and no traffic. And it was due to CPU 100% and no cpu resources for traffic.
Simple one time reboot fixed permanently issue.

mloaiza · Mon Feb 28, 2022 2:53 am

I still see same issue with l2tp/Ipsec (client/server) not able to connect.

Client side is on 7.1.3 and server side is on 6.49.3. I also try 7.1.3 on both sides with the same issues.

JanWerner · Thu Mar 24, 2022 12:53 am

Hi. RouterOS 7.1.5 the same problem, even if tunnel is up. Something goes wrong with L2TP after upgrade. After reboot tunnels with ipsec encryption won't up (I have two). There is problem with RDP connection to RDS from IKE2 road warrior client through site-to-site l2tp tunnel with ipsec encryption. Client can't establish connection at securing stage, or establish it but the connection is slow, the picture freezes. If I use MPPE128 stateless encryption for L2TP tunnels all is fine.

l2tp routeros 7.1.5.drawio (1).png

sidex84 · Wed Apr 06, 2022 2:42 pm

Hi. RouterOS 7.1.5 the same problem, even if tunnel is up. Something goes wrong with L2TP after upgrade. After reboot tunnels with ipsec encryption won't up (I have two). There is problem with RDP connection to RDS from IKE2 road warrior client through site-to-site l2tp tunnel with ipsec encryption. Client can't establish connection at securing stage, or establish it but the connection is slow, the picture freezes. If I use MPPE128 stateless encryption for L2TP tunnels all is fine.
l2tp routeros 7.1.5.drawio (1).png

A similar situation. I have many routers on the network. And this problem is observed only on two hEX S. Updated to 7.2. Did not help.

negge · Tue Apr 26, 2022 10:11 am

This is still an issue with 7.2.1 on my CCR1009-8G-1S-1S+. Sometimes, usually right after a reboot, no L2TP/IPsec tunnels can be established and the router CPU usage hovers around 80%. Rebooting is the only "solution" I've found.

Andreywys · Wed Apr 27, 2022 7:17 am

I have same situation.

evince · Wed Apr 27, 2022 10:27 am

Same problem for me,

L2TP clients are not able ton connect to my my hub vpn when ipsec is enabled.

lopar · Fri Apr 29, 2022 11:11 pm

Same problem.
Updated CCR1016-12G from 6.48.6 to 7.2.1 and lost all ipsec-based tunnels: site-to-site and l2tp\ipsec.
Everything came back to work after downgrade to 6.48.6.

Deantwo · Wed May 11, 2022 12:42 pm

IP NAT (in the beginning of roules)
/ip firewall nat add action=accept chain=srcnat dst-address=192.168.10.0/24 src-address=192.168.29.0/24
/ip firewall nat add action=accept chain=srcnat dst-address=192.168.29.0/24 src-address=192.168.10.0/24

Probably won't solve this main issue discussed here, but those two NAT rules are better replaced with a single rule like this:

/ip firewall nat
add action=accept chain=srcnat comment="IPsec no-NAT" ipsec-policy=out,ipsec

Set it as the first NAT rule in the srcnat chain and it will prevent all source-NAT of outgoing IPsec traffic. No need to create a separate rule per policy.

rManz · Thu Jun 16, 2022 10:05 am

I had the same problem, helped me add "ipsec-policy=out, none" option to masquerade rule on both sides and restarted devices.

/ip firewall nat
add action=masquerade chain=srcnat comment=masquerade ipsec-policy=out,none out-interface=wan to-addresses=0.0.0.0

Andreywys · Fri Jun 17, 2022 9:56 am

Support answer about this problem

A connection request (SA-INIT) is sent, but nothing is received back. Also there is "s" (source NAT) flag for the connection under IP->Firewall->Connections menu:
585 Cs udp xxx.xxx.xxx.xxx:4500 yyy.yyy.yyy.yyy:4500
Indicating the source address of the packets has changed. Please make sure your masquerade or any other source NAT rule is not wrongfully changing the packets.

-----------------

I have rule
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT WAN" ipsec-policy=out,none out-interface=ether1

------------------

That rules has no effect to the IPsec management traffic that I pointed out to you has the source NAT applied to it. You need to apply masquerade (source NAT) to the traffic coming from your LAN devices only. Do not NAT the traffic that originates on the router itself.

I don't really understand what I need to do. Can someone explain to me what rule I should add/change?

hotab · Sun Jul 03, 2022 6:33 pm

I managed to fix it for me.

Two very helpful threads:
viewtopic.php?t=143990
viewtopic.php?t=85703

The trick was to do MSS clamping, I used MSS of 1350, like this rule:

5 chain=forward action=change-mss new-mss=1350 tcp-flags=syn protocol=tcp
src-address=x.x.x.x/24 dst-address=y.y.y.y/24 tcp-mss=!0-1350 log=no
log-prefix=""

6 chain=forward action=change-mss new-mss=1350 tcp-flags=syn protocol=tcp
src-address=y.y.y.y/24 dst-address=x.x.x.x/24 tcp-mss=!0-1350 log=n

The trick was to have it on mikrotiks on _both_ ends of the tunnel. I am not sure if it is a must to have this on both ends, but in my case it worked like a charm.

One RouterOS is 7.3.1, the other is a slightly dated 6.43.10

bennhana · Tue Jul 19, 2022 3:42 pm

I have followed what others have suggested on this forum but nothing worked. I ended up downgrading to version 6.49

marcmerz · Tue Oct 25, 2022 1:06 am

I have followed what others have suggested on this forum but nothing worked. I ended up downgrading to version 6.49

I ended up downgrading from 7.6 to 6.49.7 for exact the same reason: I could not connect via L2TP/IPSEC to one of my servers after the upgrade. Why does Mikrotik screw something up which worked like a charm?

trolrolo · Tue Oct 25, 2022 9:17 am

They replied to my email with that problem to support@mikrotik after a half year !sic (quick response). That wrote that they don't provide support with configuration. They provide only support if there is a bug. They cannot see that is the problem of upgrade.

Mikrotik is going in bad direction...

marcmerz · Tue Oct 25, 2022 7:00 pm

nvm ...i changed my Windows Server from using RAS to use Wireguard and now i connect via Wireguard from my RB4011 to it.

Life is too short to waste your time with such bs

Andreywys · Tue Oct 25, 2022 8:38 pm

They replied to my email with that problem to support@mikrotik after a half year !sic (quick response). That wrote that they don't provide support with configuration. They provide only support if there is a bug. They cannot see that is the problem of upgrade.

Mikrotik is going in bad direction...

Same situation...

Who is online