L2TP over IPsec multiple connections problem
Posted: Sat Feb 02, 2019 7:04 pm
Hello.
I have very strange problem with l2tp over ipsec in 6.43.x versions.
I have devices CCR1036-12G-4S have 3 pppoe broadband ISP with 100mbut UL/DL as L2TP server and many RB1100 as remote clients, every device have 2 pppoe broadband links with 100mbit UL/DL. All devices have white static IP without any limitations of protocols. When I reboot all WAN is instantly up on CCR1036 without any problems but then in log file I have many "retransmissions" of l2tp clients which going from RB1100 any no one of them is not connecting. If I trying to UP all of L2TP over IPSec connections to different WAN's IP of CCR1036 at the same time: SA's is creating but with no encryption and then delete as "dead peers".
If I manual off second and third ISP - SA's is creating, and L2TP clients is connecting, when all of them is works I going to "up" ISP2 and ISP3. Then secondary L2TP over IPSec connections is working.
Scheme:
CCR1036 <--------------------------> RB1100
WAN1<----L2TP over IPSec 1--->WAN1
WAN2<----L2TP over IPSec 2--->WAN2
WAN3
LAN-----------------------------------------LAN
192.168.1.0/24----------------------192.168.2.0
Two PPP profiles for peers of each conection:
/ppp profile
set *0 use-compression=no
add change-tcp-mss=yes comment="Site-to-site VPN, \EE\F1\ED\EE\E2\ED\FB\E5 \EA\
\E0\ED\E0\EB\FB \EC\E0\E3\E0\E7\E8\ED<->\C3\CE" local-address=10.10.0.1 \
name=profile_shop use-compression=no use-encryption=yes use-mpls=no \
use-upnp=no
add change-tcp-mss=yes comment="Site-to-site VPN, \F0\E5\E7\E5\F0\E2\ED\FB\E5 \
\EA\E0\ED\E0\EB\FB \EC\E0\E3\E0\E7\E8\ED<->\C3\CE" local-address=\
10.10.1.1 name=profile_shop2 use-compression=no use-encryption=yes \
use-mpls=no use-upnp=no
Then I create 2 static routes on RB1100 to 192.168.1.0/24 via 10.10.0.1 with distance 1 and via 10.10.1.1 with distance 2 for failover. Everything works ok.
On CCR1036 is just up L2TP server with pre-shared key IPSec
Config IPSec on CCR1036 is:
/ip ipsec peer profile
set [ find default=yes ] dh-group=modp3072 dpd-maximum-failures=5 \
enc-algorithm=aes-128 hash-algorithm=sha256 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256 enc-algorithms=aes-128-cbc \
lifetime=1w pfs-group=modp3072
I tried 2 Mangle's on CCR1036:
#1 variant
/ip firewall mangle
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-1" connection-mark=no-mark disabled=yes \
in-interface=pppoe1 new-connection-mark=PPPoE-1.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-1" \
connection-mark=PPPoE-1.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-2" connection-mark=no-mark disabled=yes \
in-interface=pppoe2 new-connection-mark=PPPoE-2.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-2" \
connection-mark=PPPoE-2.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-3" connection-mark=no-mark disabled=yes \
in-interface=pppoe3 new-connection-mark=PPPoE-3.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-3" \
connection-mark=PPPoE-3.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-3.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-1.mark disabled=\
yes new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-2.mark disabled=\
yes new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-3.mark disabled=\
yes new-routing-mark=to_PPPoE-3.mark passthrough=no
#2 variant
/ip firewall mangle
add action=mark-connection chain=input comment="ISP1 - IN" in-interface=\
pppoe1 new-connection-mark=PPPoE-1.mark passthrough=yes
add action=mark-routing chain=output comment="ISP1 - OUT" connection-mark=\
PPPoE-1.mark new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-connection chain=input comment="ISP2 - IN" in-interface=\
pppoe2 new-connection-mark=PPPoE-2.mark passthrough=yes
add action=mark-routing chain=output comment="ISP2 - OUT" connection-mark=\
PPPoE-2.mark new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-connection chain=input comment="ISP3 - IN" in-interface=\
pppoe3 new-connection-mark=PPPoE-3.mark passthrough=yes
add action=mark-routing chain=output comment="ISP3 - OUT" connection-mark=\
PPPoE-3.mark new-routing-mark=to_PPPoE-3.mark passthrough=no
add action=mark-connection chain=forward comment="ISP1 - Forward" \
in-interface=pppoe1 new-connection-mark=pppoe1-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe1-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-1.mark passthrough=no
add action=mark-connection chain=forward comment="ISP2 - Forward" \
in-interface=pppoe2 new-connection-mark=pppoe2-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe2-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-2.mark passthrough=no
add action=mark-connection chain=forward comment="ISP3 - Forward" \
in-interface=pppoe3 new-connection-mark=pppoe3-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe3-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-3.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe1 new-connection-mark=PPPoE-1.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe2 new-connection-mark=PPPoE-2.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe3 new-connection-mark=PPPoE-3.mark passthrough=no
On RB1100 as L2TP over IPSec client:
No Mangle using because I create static routes to each WAN's IP of CCR1036 with distance 1 and 2 for failover if one of ISP of CCR1036 goes down.
Both L2TP clients can't be established in same time. Only one by one.
I have very strange problem with l2tp over ipsec in 6.43.x versions.
I have devices CCR1036-12G-4S have 3 pppoe broadband ISP with 100mbut UL/DL as L2TP server and many RB1100 as remote clients, every device have 2 pppoe broadband links with 100mbit UL/DL. All devices have white static IP without any limitations of protocols. When I reboot all WAN is instantly up on CCR1036 without any problems but then in log file I have many "retransmissions" of l2tp clients which going from RB1100 any no one of them is not connecting. If I trying to UP all of L2TP over IPSec connections to different WAN's IP of CCR1036 at the same time: SA's is creating but with no encryption and then delete as "dead peers".
If I manual off second and third ISP - SA's is creating, and L2TP clients is connecting, when all of them is works I going to "up" ISP2 and ISP3. Then secondary L2TP over IPSec connections is working.
Scheme:
CCR1036 <--------------------------> RB1100
WAN1<----L2TP over IPSec 1--->WAN1
WAN2<----L2TP over IPSec 2--->WAN2
WAN3
LAN-----------------------------------------LAN
192.168.1.0/24----------------------192.168.2.0
Two PPP profiles for peers of each conection:
/ppp profile
set *0 use-compression=no
add change-tcp-mss=yes comment="Site-to-site VPN, \EE\F1\ED\EE\E2\ED\FB\E5 \EA\
\E0\ED\E0\EB\FB \EC\E0\E3\E0\E7\E8\ED<->\C3\CE" local-address=10.10.0.1 \
name=profile_shop use-compression=no use-encryption=yes use-mpls=no \
use-upnp=no
add change-tcp-mss=yes comment="Site-to-site VPN, \F0\E5\E7\E5\F0\E2\ED\FB\E5 \
\EA\E0\ED\E0\EB\FB \EC\E0\E3\E0\E7\E8\ED<->\C3\CE" local-address=\
10.10.1.1 name=profile_shop2 use-compression=no use-encryption=yes \
use-mpls=no use-upnp=no
Then I create 2 static routes on RB1100 to 192.168.1.0/24 via 10.10.0.1 with distance 1 and via 10.10.1.1 with distance 2 for failover. Everything works ok.
On CCR1036 is just up L2TP server with pre-shared key IPSec
Config IPSec on CCR1036 is:
/ip ipsec peer profile
set [ find default=yes ] dh-group=modp3072 dpd-maximum-failures=5 \
enc-algorithm=aes-128 hash-algorithm=sha256 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256 enc-algorithms=aes-128-cbc \
lifetime=1w pfs-group=modp3072
I tried 2 Mangle's on CCR1036:
#1 variant
/ip firewall mangle
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-1" connection-mark=no-mark disabled=yes \
in-interface=pppoe1 new-connection-mark=PPPoE-1.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-1" \
connection-mark=PPPoE-1.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-2" connection-mark=no-mark disabled=yes \
in-interface=pppoe2 new-connection-mark=PPPoE-2.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-2" \
connection-mark=PPPoE-2.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-connection chain=prerouting comment=\
"Mark connections of PPPoE-3" connection-mark=no-mark disabled=yes \
in-interface=pppoe3 new-connection-mark=PPPoE-3.mark passthrough=yes
add action=mark-routing chain=prerouting comment="to PPPoE-3" \
connection-mark=PPPoE-3.mark disabled=yes in-interface=bridge1 \
new-routing-mark=to_PPPoE-3.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-1.mark disabled=\
yes new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-2.mark disabled=\
yes new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-routing chain=output connection-mark=PPPoE-3.mark disabled=\
yes new-routing-mark=to_PPPoE-3.mark passthrough=no
#2 variant
/ip firewall mangle
add action=mark-connection chain=input comment="ISP1 - IN" in-interface=\
pppoe1 new-connection-mark=PPPoE-1.mark passthrough=yes
add action=mark-routing chain=output comment="ISP1 - OUT" connection-mark=\
PPPoE-1.mark new-routing-mark=to_PPPoE-1.mark passthrough=no
add action=mark-connection chain=input comment="ISP2 - IN" in-interface=\
pppoe2 new-connection-mark=PPPoE-2.mark passthrough=yes
add action=mark-routing chain=output comment="ISP2 - OUT" connection-mark=\
PPPoE-2.mark new-routing-mark=to_PPPoE-2.mark passthrough=no
add action=mark-connection chain=input comment="ISP3 - IN" in-interface=\
pppoe3 new-connection-mark=PPPoE-3.mark passthrough=yes
add action=mark-routing chain=output comment="ISP3 - OUT" connection-mark=\
PPPoE-3.mark new-routing-mark=to_PPPoE-3.mark passthrough=no
add action=mark-connection chain=forward comment="ISP1 - Forward" \
in-interface=pppoe1 new-connection-mark=pppoe1-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe1-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-1.mark passthrough=no
add action=mark-connection chain=forward comment="ISP2 - Forward" \
in-interface=pppoe2 new-connection-mark=pppoe2-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe2-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-2.mark passthrough=no
add action=mark-connection chain=forward comment="ISP3 - Forward" \
in-interface=pppoe3 new-connection-mark=pppoe3-conn-f passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe3-conn-f \
in-interface=bridge1 new-routing-mark=PPPoE-3.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe1 new-connection-mark=PPPoE-1.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe2 new-connection-mark=PPPoE-2.mark passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=\
yes in-interface=pppoe3 new-connection-mark=PPPoE-3.mark passthrough=no
On RB1100 as L2TP over IPSec client:
No Mangle using because I create static routes to each WAN's IP of CCR1036 with distance 1 and 2 for failover if one of ISP of CCR1036 goes down.
Both L2TP clients can't be established in same time. Only one by one.