IKEv2 mtu issue

mafiosa · Wed Sep 14, 2022 9:20 pm

It seems like mtu is broken on IKEv2. My isp provides connection via PPPoE with MUT of 1460. How do I overcome this broke mtu issue as I am having huge issues in website loading over VPN.

sindy · Wed Sep 14, 2022 10:27 pm

It depends on how exactly it is broken. What typically happens is the following:

The LAN clients have an MTU of 1500 on their Ethernet interfaces; since the uplink has an MTU of 1460, the router drops any packet sent by the LAN client that is larger than that and has the DF flag set, and sends back an ICMP message stating that the 1500-byte packet is too large to fit without fragmentation and what the MTU of the outgoing interface is, so that the LAN client could retry with a smaller packet.

If your IPsec policy says that everything sent from the LAN subnet anywhere (to 0.0.0.0/0) must be sent via the IKEv2 tunnel, this ICMP "fragmentation needed" message gets caught and diverted by that policy too because its source address is router's own address in the LAN subnet. So if this is your case, you have to place a policy with src-address=the.lan.sub.net/mask, dst-address=the.lan.sub.net/mask, and action=none before (above) that src-address=the.lan.sub.net/mask dst-address=0.0.0.0/0 action=encrypt one.

The resulting packet sizes will be even smaller than 1460 bytes as the IPsec headers occupy part of that MTU, but that's how it should work.

If the issue is at the IKEv2 responder end that is not under your control, there is nothing you can do about it at your end except forcing MSS value in TCP SYN packets to a smaller one. But that only works for TCP, and not always - I've seen TCP servers to ignore the MSS setting coming from the client, or maybe devices somewhere on the path rewriting it to a big one.-No idea what exactly it was, but in any case the packets arriving to the bottleneck router from the remote server were way larger than even the usual 1500 bytes although the MSS from client was asking for 1450 byte packets.

nichky · Thu Sep 15, 2022 4:29 am

@sindy,

can i confirm with you:

src-address=the.lan.sub.net/mask, dst-address=the.lan.sub.net/mask, and action=none

does that mean:

dst-address=local-lan/mask src-address=remote-lan/mask

mafiosa · Thu Sep 15, 2022 11:04 pm

I am also facing another issue. Whenever I am trying to access an webserver on my lan via IP it shows the webserver that has port 80 forwareded to it. This is only for remote users using VPN. Apps work well, google yeilds search results yet 90% websites don't load. I am sharing the config:

/ip ipsec policy group
add name=ike2
/ip ipsec profile
set [ find default=yes ] enc-algorithm=aes-256,aes-192,aes-128 \
    hash-algorithm=sha256
add enc-algorithm=aes-256,aes-192,aes-128 hash-algorithm=sha256 name=ike2
/ip ipsec peer
add exchange-mode=ike2 name=kalpak passive=yes profile=ike2 \
    send-initial-contact=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256,sha1 pfs-group=modp2048
add auth-algorithms=sha256,sha1 enc-algorithms="aes-256-cbc,aes-256-ctr,aes-25\
    6-gcm,aes-192-cbc,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-\
    gcm" name=ike2 pfs-group=none
/ip ipsec mode-config
add address-pool=ipsec-roadwarrior address-prefix-length=32 name=ike2 \
    split-include=0.0.0.0/0 static-dns=8.8.8.8 system-dns=no
/ppp profile
add change-tcp-mss=yes name=BSNL-PPPoE
add change-tcp-mss=yes name=ABSPL-PPPoE
set *FFFFFFFE dns-server=192.168.72.53
/ip firewall filter
add action=accept chain=forward ipsec-policy=in,ipsec
add action=accept chain=forward ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward connection-state=\
    established,related disabled=yes hw-offload=yes
add action=accept chain=forward connection-state=established,related \
    disabled=yes
add action=drop chain=input connection-state=invalid in-interface=BSNL-PPPoE
add action=accept chain=input comment="ICMP Allow" in-interface=BSNL-PPPoE \
    protocol=icmp
add action=accept chain=input comment="IGMP Allow" in-interface=BSNL-PPPoE \
    protocol=igmp
add action=accept chain=input comment="IGMP Allow" in-interface=BSNL-PPPoE \
    protocol=ipsec-esp
add action=accept chain=input comment="GRE Allow" in-interface=BSNL-PPPoE \
    protocol=gre
add action=accept chain=input comment="Allow UDP ports" dst-port=\
    500,1701,2000,3389,4500,4789,5198,5199,5201,8472,9993,13231-13235 \
    in-interface=BSNL-PPPoE protocol=udp
add action=accept chain=input comment="Allow TCP ports" dst-port=\
    22,80,443,1723,2000,2200,3389,6969,8006,8080,8291,8888,9993,32700-39000 \
    in-interface=BSNL-PPPoE protocol=tcp
add action=accept chain=input connection-state=established in-interface=\
    BSNL-PPPoE
add action=accept chain=input connection-state=related in-interface=\
    BSNL-PPPoE
add action=accept chain=input connection-state=untracked in-interface=\
    BSNL-PPPoE
add action=drop chain=input in-interface=BSNL-PPPoE
/ip firewall mangle
add action=change-mss chain=forward comment="Fix MSS for VPN server" new-mss=\
    1300 passthrough=yes protocol=tcp src-address=192.168.55.0/29 tcp-flags=\
    syn tcp-mss=!0-1300
add action=change-mss chain=forward comment="Fix MSS for VPN server" \
    dst-address=192.168.55.0/29 new-mss=1300 passthrough=yes protocol=tcp \
    tcp-flags=syn tcp-mss=!0-1300
/ip firewall nat
add action=masquerade chain=srcnat out-interface=BSNL-PPPoE
/ip ipsec identity
add generate-policy=port-override mode-config=ike2 my-id=\
    fqdn:ec190ff69b86.sn.mynetname.net peer=kalpak policy-template-group=ike2 \
    remote-id=ignore
/ip ipsec policy
set 0 disabled=yes dst-address=0.0.0.0/0 group=ike2 proposal=ike2 \
    src-address=0.0.0.0/0
add action=none dst-address=0.0.0.0/0 src-address=192.168.23.0/24
add action=none dst-address=0.0.0.0/0 src-address=192.168.24.0/24
add action=none dst-address=0.0.0.0/0 src-address=192.168.72.0/23
add dst-address=192.168.55.0/29 group=ike2 proposal=ike2 src-address=\
    0.0.0.0/0 template=yes

nichky · Thu Sep 15, 2022 11:42 pm

i think should be other way around, according from
viewtopic.php?t=154449
I asked sindy for cunformation , but im sure that dst- shoud be your lan range

sindy · Fri Sep 16, 2022 9:46 am

can i confirm with you:
does that mean:
dst-address=local-lan/mask src-address=remote-lan/mask

No. The src-address and dst-address must refer to the same LAN subnet in these action=none policies, as their purpose is to prevent packets sent by the router itself to a host in its own LAN subnet from getting handled by the dst-address=0.0.0.0/0 src-address=lan.sub.net/mask action=encrypt policy.

But according to the configuration, this is actually totally unrelated to @mafiosa's use case, as he uses the Mikrotik as an IKEv2 responder, not client, and the action=enrypt policies generated for initiators fom the template say dst-address=192.168.55.x src-address=0.0.0.0/0. So there is no danger of ICMP "fragmentation needed" packets sent by the router itself to be diverted to a wrong path, and no need to prevent this using any action=none policies.

Plus, given that the workaround for broken PMTU discovery, i.e. adjusting MSS in mangle, is in place, a broken PMTUD should not be the reason of the issues encountered (at least not for 90 % of sites).

So @mafiosa, at this point I would sniff the traffic on the router to see how the connection to a particular problematic web site looks like, as the configuration looks fine to me (ok, permitting RDP port 3389 in chain input of /ip firewall filter is a nonsense indicating some misunderdtandings at your side, but that has nothing to do with the issue you suffer from.

I am also facing another issue. Whenever I am trying to access an webserver on my lan via IP it shows the webserver that has port 80 forwareded to it. This is only for remote users using VPN.

Sorry, I don't understand the description of this other issue. I can see no port forwarding rule in the export, and I would expect that seeing a server in the LAN is what you actually want if you have set a port forwarding rule. So please reword the description, state what is the expected behaviour and how the actual one differs from that.

anis · Sun Mar 03, 2024 10:00 pm

Apps work well, google yeilds search results yet 90% websites don't load.
I encountered exactly the same problem. what is the solution. My clients connect via ikev2, the connection goes through without problems, they ping to the router, the address resolves both on the local network and on the Internet, but from the sites only Google search opens and not a single site opens? what needs to be checked?

sindy · Mon Mar 04, 2024 9:13 am

what needs to be checked?

Post the export (not screenshots) of your configuration. Something similar as described in post #2 must be happening, but in your case, it would be the remote server (web site) that doesn't receive the information that the usable MTU is lower on the path from your router to your phone. But the Path MTU Discovery (PMTUD) mechanism may be broken also for other reasons than missing or incorrect IPsec policies.

anis · Mon Mar 04, 2024 12:18 pm

firewall everything is allowed for the test. input addresses have been changed; everything else is from the working router, on which the client 10.50.0.12 opens only the Google search engine; all other sites are not accessible.

# mar/03/2024 16:06:57 by RouterOS 6.49.10
# software id = UH1D-IK15
#
# model = CCR1016-12G
# serial number = D6450EB884C9
/interface bridge
add arp=reply-only name=bridge1
/interface ethernet
set [ find default-name=ether1 ] name=ether1-WAN
set [ find default-name=ether2 ] disabled=yes
set [ find default-name=ether3 ] disabled=yes
set [ find default-name=ether4 ] disabled=yes
set [ find default-name=ether5 ] disabled=yes
set [ find default-name=ether6 ] disabled=yes
set [ find default-name=ether7 ] name=ether7
set [ find default-name=ether8 ] disabled=yes
set [ find default-name=ether9 ] disabled=yes
set [ find default-name=ether10 ] disabled=yes
set [ find default-name=ether11 ] disabled=yes
set [ find default-name=ether12 ] disabled=yes
/interface vlan
add interface=ether1-WAN name=vlan1 vlan-id=1
/interface list
add name=Lan
add name=WAN
add name=discover
/ip ipsec mode-config
add address=10.50.0.12 address-prefix-length=22 name=IKEv2-Server static-dns=\
    172.16.0.1 system-dns=no
/ip ipsec policy group
add name=ipsec
add name=IKEv2-Server
/ip ipsec profile
add enc-algorithm=aes-256,aes-192,aes-128,3des name=ipsec-profile
add dh-group=modp2048,modp1536,modp1024 enc-algorithm=\
    aes-256,aes-192,aes-128,3des hash-algorithm=sha256 name=IKEv2-Server
/ip ipsec peer
add exchange-mode=ike2 local-address=123.45.67.89 name=IKEv2-Server passive=\
    yes profile=IKEv2-Server
/ip ipsec proposal
add enc-algorithms=aes-256-cbc,aes-192-cbc,aes-128-cbc,3des name=\
    ipsec-proposal pfs-group=none
add auth-algorithms=sha512,sha256,sha1 enc-algorithms="aes-256-cbc,aes-256-ctr\
    ,aes-256-gcm,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-gcm" \
    lifetime=8h name=IKEv2-Server pfs-group=none
/ip pool
add name=pool4 ranges=172.16.4.1-172.16.4.254
add name=pool3 next-pool=pool4 ranges=172.16.3.1-172.16.3.254
add name=pool2 next-pool=pool3 ranges=172.16.2.1-172.16.2.254
add name=pool1 next-pool=pool2 ranges=172.16.1.1-172.16.1.254
/ppp profile
add change-tcp-mss=yes name=l2tp-remote-client-to-site only-one=no \
    use-compression=yes use-encryption=yes
/snmp community
set [ find default=yes ] disabled=yes
add addresses=0.0.0.0/0 name=snmp_public
/system logging action
set 1 disk-file-count=10
set 3 bsd-syslog=yes remote=172.16.1.135 syslog-facility=syslog
add name=ipsec target=memory
add name=l2tp target=memory
/interface bridge port
add bridge=bridge1 hw=no interface=ether7
add bridge=bridge1 interface=vlan1
/ip neighbor discovery-settings
set discover-interface-list=none
/interface l2tp-server server
set authentication=mschap2 default-profile=l2tp-remote-client-to-site \
    enabled=yes one-session-per-host=yes use-ipsec=required
/interface pptp-server server
set authentication=mschap2 default-profile=pptp
/ip address
add address=172.16.0.1/22 interface=bridge1 network=172.16.0.0
add address=123.45.67.89/29 interface=\
    ether1-WAN network=123.45.67.89
/ip dhcp-server network
add address=172.16.0.0/22 dns-server=172.16.0.1 gateway=172.16.0.1 \
    ntp-server=172.16.0.1
/ip dns
set allow-remote-requests=yes cache-max-ttl=1d cache-size=10240KiB servers=\
    8.8.8.8,1.1.1.1
/ip firewall filter
add action=accept chain=forward 
add action=accept chain=output 
add action=accept chain=input 
/ip firewall mangle
add action=change-mss chain=forward ipsec-policy=in,ipsec new-mss=1360 \
    passthrough=yes protocol=tcp src-address=10.50.0.12 tcp-flags=syn \
    tcp-mss=!0-1360
add action=change-mss chain=forward dst-address=10.50.0.12 ipsec-policy=\
    out,ipsec new-mss=1360 passthrough=yes protocol=tcp tcp-flags=syn \
    tcp-mss=!0-1360
/ip firewall nat
add action=masquerade chain=srcnat src-address=10.50.0.12
add action=masquerade chain=srcnat src-address=172.16.0.0/22 out-interface-list=WAN
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip ipsec identity
add auth-method=digital-signature certificate=vpn.ike2 generate-policy=\
    port-strict match-by=certificate mode-config=IKEv2-Server peer=\
    IKEv2-Server policy-template-group=IKEv2-Server remote-certificate=\
    ra@vpn.ike2 remote-id=user-fqdn:ra@vpn.ike2
/ip ipsec policy
add dst-address=0.0.0.0/0 group=ipsec proposal=ipsec-proposal src-address=\
    0.0.0.0/0 template=yes
add dst-address=10.50.0.0/24 group=IKEv2-Server proposal=IKEv2-Server \
    src-address=0.0.0.0/0 template=yes
/ip route
add check-gateway=ping distance=1 gateway=11.12.34.56
/ip route rule
add action=lookup-only-in-table src-address=123.45.67.89/32 table=WAN
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox port=28291
set api-ssl disabled=yes
/snmp
set enabled=yes trap-community=snmp_public trap-interfaces=\
    ether7 trap-version=2
/system clock
set time-zone-name=manual
/system clock manual
set dst-delta=+01:00 dst-end="oct/28/2019 00:00:00" dst-start=\
    "mar/31/2019 00:00:00" time-zone=+02:00
/system identity
set name=mtk-ccr1016
/system leds
add leds=fault-led type=fan-fault
add leds=user-led type=flash-access
/system package update
set channel=long-term
/tool bandwidth-server
set authenticate=no enabled=no
/tool mac-server
set allowed-interface-list=none
/tool mac-server mac-winbox
set allowed-interface-list=Lan
/tool mac-server ping
set enabled=no

sindy · Mon Mar 04, 2024 12:45 pm

First, you may want to edit also the gateway IP from the export.

Next, I can see you've got the two action=change-mss rules in forward mangle, do I get it right that they don't help?

I'd like to see the output of /ip ipsec policy print detail (with public addresses redacted of course); if that gives no hint, advanced sniffing will be the last resort.

I have seen situations where the change-mss rules didn't help because some idiot at the ISP was using change-mss too but ignoring the pre-existing MSS value so happily setting a higher one. Proving that case would involve setting up a TCP session to another device somewhere else in the internet, on which you could sniff as well, to see the value in the MSS field of your SYN upon reaching that device after passing through the internet and your ISP in particular.

anis · Mon Mar 04, 2024 1:02 pm

action=change-mss tried changing different values, without results. I’m also inclined to believe that the problem is with the provider, and if so, is it possible to get through this without changing the provider? What needs to be done to identify and track this?

[mtk@mtk-ccr1016] > /ip ipsec
[mtk@mtk-ccr1016] /ip ipsec> export
# mar/04/2024 13:04:15 by RouterOS 6.49.10
# software id = 
#
# model = CCR1016-12G
# serial number = 
/ip ipsec mode-config
add address=10.50.0.12 address-prefix-length=22 name=IKEv2-Server static-dns=\
    172.16.0.1 system-dns=no
/ip ipsec policy group
add name=ipsec
add name=IKEv2-Server
/ip ipsec profile
add enc-algorithm=aes-256,aes-192,aes-128,3des name=ipsec-profile
add dh-group=modp2048,modp1536,modp1024 enc-algorithm=\
    aes-256,aes-192,aes-128,3des hash-algorithm=sha256 name=IKEv2-Server
add name=copy
/ip ipsec peer
add exchange-mode=ike2 local-address=ip wan name=IKEv2-Server passive=\
    yes profile=IKEv2-Server
add disabled=yes exchange-mode=ike2 name=copy passive=yes profile=IKEv2-Server
/ip ipsec proposal
add enc-algorithms=aes-256-cbc,aes-192-cbc,aes-128-cbc,3des name=ipsec-proposal \
    pfs-group=none
add auth-algorithms=sha512,sha256,sha1 enc-algorithms="aes-256-cbc,aes-256-ctr,a\
    es-256-gcm,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-gcm" \
    lifetime=8h name=IKEv2-Server pfs-group=none
add name=copy
/ip ipsec identity
add auth-method=digital-signature certificate=vpn.ike2 generate-policy=\
    port-strict match-by=certificate mode-config=IKEv2-Server peer=IKEv2-Server \
    policy-template-group=IKEv2-Server remote-certificate=ra@vpn.ike2 \
    remote-id=user-fqdn:ra@vpn.ike2
add auth-method=digital-signature certificate=name comment=copy \
    disabled=yes generate-policy=port-strict match-by=certificate mode-config=\
    IKEv2-Server peer=IKEv2-Server policy-template-group=IKEv2-Server \
    remote-certificate=ra@name remote-id=user-fqdn:ra@name
add disabled=yes generate-policy=port-strict mode-config=IKEv2-Server peer=\
    IKEv2-Server policy-template-group=IKEv2-Server secret=\
    "vJ\$beWkTJ7q@9S~`>]q)YrLH"
/ip ipsec policy
add dst-address=0.0.0.0/0 group=ipsec proposal=ipsec-proposal src-address=\
    0.0.0.0/0 template=yes
add dst-address=10.50.0.0/24 group=IKEv2-Server proposal=IKEv2-Server \
    src-address=0.0.0.0/0 template=yes
add comment=copy disabled=yes dst-address=0.0.0.0/0 group=IKEv2-Server \
    proposal=IKEv2-Server src-address=0.0.0.0/0 template=yes
[mtk@mtk-ccr1016] /ip ipsec>

sindy · Mon Mar 04, 2024 1:11 pm

To identify that, I would sniff the TCP SYN+ACK packets arriving via WAN and see what MSS they carry. If it is smaller than 1460, it suggests something between your WAN and the sender is tampering with it. But to be really sure, you need a device on a public address somewhere in the internet (like your friend's one on a public address provided by some other ISP than yours) and sniffing simultaneously on it and your WAN when trying to open its address for web or telnet (it doesn't matter whether the service is actually listening there, we are interested in just the SYN packet contents, it need not be responded).

As for "without changing the provider", I'm afraid you'd have to use a virtual machine somewhere in a data center and create a tunnel to it in order to bypass the mishandling of MSS by the ISP. But talking nicely to the ISP might be an easier way?

anis · Mon Mar 04, 2024 1:15 pm

I posted the entire ipsec, only the wan ip was replaced and the email was in the certificates, the rest is as is and the rules are inactive

anis · Mon Mar 04, 2024 1:42 pm

To identify that, I would sniff the TCP SYN+ACK packets arriving via WAN and see what MSS they carry. If it is smaller than 1460, it suggests something between your WAN and the sender is tampering with it. But to be really sure, you need a device on a public address somewhere in the internet (like your friend's one on a public address provided by some other ISP than yours) and sniffing simultaneously on it and your WAN when trying to open its address for web or telnet (it doesn't matter whether the service is actually listening there, we are interested in just the SYN packet contents, it need not be responded).

As for "without changing the provider", I'm afraid you'd have to use a virtual machine somewhere in a data center and create a tunnel to it in order to bypass the mishandling of MSS by the ISP. But talking nicely to the ISP might be an easier way?

https://ibb.co/WkL4RjB

sindy · Mon Mar 04, 2024 1:43 pm

Export shows the configured items; the actual policies are generated dynamically from the templates. And your templates are quite wide. That's why I want the output of the print rather than the export for the policies.

anis · Mon Mar 04, 2024 1:55 pm

Export shows the configured items; the actual policies are generated dynamically from the templates. And your templates are quite wide. That's why I want the output of the print rather than the export for the policies.

Ok
this is with active client 10.50.0.12 ikev2
https://ibb.co/Sf1qkt8

sindy · Mon Mar 04, 2024 2:20 pm

I have asked for /ip ipsec policy print detail. But what bothers me here is that the dynamically generated policy for the phone is marked as I (Invalid), which makes little sense to me given that there is only a single actual policy - the rest are templates.

anis · Mon Mar 04, 2024 4:07 pm

I have asked for /ip ipsec policy print detail. But what bothers me here is that the dynamically generated policy for the phone is marked as I (Invalid), which makes little sense to me given that there is only a single actual policy - the rest are templates.

sorry
https://ibb.co/L8GhWPz

anis · Mon Mar 04, 2024 4:17 pm

But talking nicely to the ISP might be an easier way?

ISP said that nothing is being done about traffic

msatter · Mon Mar 04, 2024 4:21 pm

@anis You and sindy are the only ones active here so please, don't qoute so much.

sindy · Mon Mar 04, 2024 5:17 pm

The policy looks fairly normal (the dst-address is a /32 one), and this time it is not invalid. So there is no way how the policy could cause it.

So time for sniffing I'd say, ideally at both ends of the TCP connection - "we don't do anything special" is the standard Layer 1 support response that sticks until you prove them wrong

. Have you got another device on a public address you could sniff at?

anis · Mon Mar 04, 2024 7:44 pm

if I find one. What should I do?

sindy · Mon Mar 04, 2024 8:08 pm

Sniff to file on both, matching on the public ip address of the other one at each (so if the public IP address of your local one is l.l.l.l, set ip-address=l.l.l.l in the sniffer filter. What we are interested in are not just the TCP packets but also ICMP ones, so we cannot filter on anything more detailed. But you can prevent sniffing the remote access connection if you connect to the remote one from your local machine by excluding the port from the sniff, like port=!22 if you connect using SSH.

Then try to connect to the remote address from your browser - even if there is no HTTP or HTTPS server running on the remote machine so the browser will throw an error after some time, the SYN packets will nevertheless arrive there and get sniffed. Of course, connecting to a service that does run there would be better as you would be able to sniff also the SYN+ACK response.

Once the browser gives up or the connection succeeds depending on what you actually try, stop the sniffing, open both files in Wireshark and see what the MSS value in the SYN (and, eventually, SYN+ACK) packets is.

anis · Mon Mar 04, 2024 10:36 pm

if I did everything correctly, but the packages are unchanged (tried without tunnel, since on a machine with a public IP it is not possible to install a tunnel)
MSS 1460
something's wrong here (I connected 2 isp to Mikrotik and connect to both with no result)
I already changed the ISP but the result is the same
the connection is working but the sites do not open

sindy · Mon Mar 04, 2024 10:49 pm

Well, the idea was to see whether the ISP does not set the MSS back to high value if you lower it using your mangle rules. From our description I am not sure whether we understood each other - my idea was that you don't change a single thing on your local machine and try to connect from a browser on the phone connected to it via IKEv2 to a TCP port on the public address of the other machine. Have you done that or something else?

anis · Mon Mar 04, 2024 11:11 pm

I don’t quite understand you correctly.
I have a Mikrotik with two ISPs, public white addresses, I can connect to it via IKEv2 from a phone or PC with a gray address.
what else should I do step by step to check the problem, but it is unlikely that there are 2 independent providers and the result is this, maybe my settings are incorrect or everything is ok with the settings

sindy · Mon Mar 04, 2024 11:23 pm

If it behaves the same when the tunnel is established via another ISP, I agree with you that it does not look like an ISP issue. Hence it needs to sniff a single connection to a web site that fails and see what exactly is going on there. So find a web site that does not work, find out its IP address, and sniff traffic to&from this address on ether1 into a file. Then try opening that site and open the file in Wireshark once the attempt fails. You should see whether the server repeatedly retransmits some large packet, which would indicate that neither the PMTUD nor the change-mss rules worked, so that large packet makes it to your router but not further via the tunnel, or whether something else is wrong there.

You may also consider following this link.

anis · Mon Mar 04, 2024 11:33 pm

Wireshark I'm running it on a PC that connects to Mikrotik via IKEv2, right?
Is it possible to analyze all this on a PC or not?

You may also consider following this link.

Here it’s not at all clear what needs to be done and why?

sindy · Mon Mar 04, 2024 11:47 pm

Wireshark I'm running it on a PC that connects to Mikrotik via IKEv2, right?

No. Sniff into a file on the Mikrotik itself as you supposedly did before, connect to the unreachable site from a phone connected via IKEv2, then stop sniffing, download the file to a PC and open it using Wireshark.

Here it’s not at all clear what needs to be done and why?

That's just an offer for a direct contact if you feel that a more interactive communication (in your native language) could help.

anis · Mon Mar 04, 2024 11:51 pm

[/quote]
direct contact.
[/quote]
ок telegram

anis · Tue Mar 05, 2024 12:07 am

I want to say right away that I don’t really understand the reason for what’s happening, but something tells me that the settings are wrong somewhere and the provider has nothing to do with it.
for
I made a L2TP/Ipsec VPN on the same Mikrotik and everything was connected from the clients and all sites opened

I try to ping -f -l 1373 and higher, it already requires fragmentation and does not work without VPN until -f -l 1472 successfully, even on L2TP/Ipsec VPN, even on ikev2
but with L2TP/Ipsec everything works, with ikev2 it doesn’t

anis · Tue Mar 05, 2024 1:05 am

Then try opening that site and open the file in Wireshark

when opening sites (that are not accessible)
https://ibb.co/8DzCs80

anis · Wed Mar 06, 2024 11:47 am

That's just an offer for a direct contact if you feel that a more interactive communication (in your native language) could help.

ok

sindy · Wed Mar 06, 2024 12:07 pm

Sorry for my lack of responsiveness, yesterday and today are quite busy. The TCP SYN packets in your last screenshots only show that the server did not respond at all, so that's definitely not MTU related. But there are no addresses so I can't see whether the NAT did not happen or something else went wrong.

So was the source address of those SYN packets the public WAN IP of the router or some other one?

anis · Wed Mar 06, 2024 1:12 pm

https://ibb.co/Fn5MptS
10.50.0.12 IP client ikev2
destination IP site (don't open)

but why are there no packets from IP site to destination client 10.0.50.12

this is what it looks like if I connect via L2TP/Ipsec
https://ibb.co/wQVrq0W

sindy · Wed Mar 06, 2024 2:35 pm

Can you show me the sniffer settings? Because it is normal that the source address is the private one on the way from the phone to the router, but on the way from the router to the external server, the source address should be the WAN one of the router. In your sniffs, there is only one copy of each packet, so I suppose you have narrowed the sniffer filter down to a particular interface.

As for Telegram - the post above explains how to encrypt your phone number using my public key and post it here encrypted so that nobody else than me could decrypt it.

anis · Wed Mar 06, 2024 3:34 pm

packet sniffer
https://ibb.co/QkS6RPP
WAN address blurred out
client address and website address is open

sindy · Thu Mar 07, 2024 11:22 am

Apps work well, google yeilds search results yet 90% websites don't load.

To provide a conclusion to the story: the root cause of the issue was a dst-nat rule matching on in-interface-list=WANxy but not taking into account whether the packet coming in through WAN was indeed a "direct" one coming to the public IP address of the WAN or an IPsec payload one extracted from a transport packet (the latter inherits the in-interface attribute from the transport one). So connections to regular web sites (HTTPS over TCP port 443) that came in via an IPsec SA got redirected to an internal address, and the most likely reason why Google sites worked was that the browser uses QUIC (on UDP port 443) instead of TCP to connect to them so the rule didn't affect these connections.

anis · Thu Mar 07, 2024 11:46 pm

@sindy. Thank you very much for your fast and, most importantly, quality help.
There was an expected serious error in the NAT configuration
To which Sindy pointed out, I followed the wrong path of the mangle chain

IKEv2 mtu issue

IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue

Re: IKEv2 mtu issue