Hello,

I am slowly freaking out. Sorry for dropping this post not well prepared.

I have an hAP ax2 running as a PPPoE router and AP. Also I have hAP ac2 running only as an AP in another room. There is a cable between these the two devices.

Everything works as expected, no other problems noticed, but... when phones or computers connect to the ax2 (main PPPoE router) they cannot open two specific websites of two online banking platforms.

When the same devices connect to the second router (that works as an AP only) they are able to open these two websites.

So far I've played with the MTUs of both routers on the WiFi interfaces and on the PPPoE but there is now solution. I am starting to think the main router is being compromised. This router was never accessible from Internet.

Any ideas how to debug this?

Follow these instructions:
viewtopic.php?t=203686#p1051720
and post the (redacted/anonymized) configuration of both devices.

In theory in a normal setup there shouldn't be anything different from something going:
phone<->wifi/wlan<->hAP AC2<->ethernet<->hap Ax2<->Internet
and something going:
phone<->wifi/wlan<->hap Ax2<->Internet
as usually both the ethernet port (connected to the hap AC2) and the hap AX2 own wlan interfaces are in a same bridge, so the flow in both cases should be the same:
phone<->some way of connection<->bridge of hap AX2<->hap Ax2<-> Internet

unless a firewall rule on the hAP AX2 that somehow treats differently packets going through the ethernet port connected to the hAP AC2 and packets going through the hAP AX2 own wlan (or as you tried some strange MTU settings, but I believe this is more rare).

I would try connecting to those two sites from a laptop connected to:
1) a bridge ethernet port of the hap AC2
and
2) a bridge ethernet port of the hap AX2
and see if there is any difference in the behaviour.

A few months ago I was dealing with a very similar issue - not the exact same one as the banks were most likely other ones and the issue was affecting devices connected to the "remote" AP. In that case, the culprit was the MTU on the tunnel between the main router and that AP.

Your case seems far more crazy indeed as it is the main router that has an issue. Do I get you right that the WiFi interfaces of the hAP ax² are bridged with those of the hAP ac², i.e. that the client lands in the same IP subnet regardless whether it is connected via the hAP ax² or the hAP ac²?

Much deeper in the past, I have seen an issue with bridge MTU being set incorrectly, so after a fragmented packet that has arrived over WAN got reassembled (the firewall reassembles packets as fragments would be unable to traverse NAT), it did fit to the bridge MTU but not to the MTU of the receiving interface, or maybe even of the sending wireless interface in the bridge, so it got dropped before reaching the client.

So I'd suggest you use a laptop with Wireshark or tcpdump as the client and sniff the traffic on it and on the hAP ax² simultaneously when trying to open the bank site, and find out at what point (PPPoE, bridge, wifiX, the laptop) there is a difference in the packet count.

Thank you for your time in replying to my post! I found the problem!

I noticed another symptom of the same problem - laptops connected via WiFi to the same main router were not able to access WINS/Samba shares reachable via IPSEC tunnel.

The problem was in the MTU of the "main LAN bridge" of the main hAP ax2 PPPoE router. Because of specific tech requirements, that main hAP ax2 has an EoIP tunnel. The EoIP tunnel interface is a member of that main LAN bridge.

The EoIP interface itself has 1378 actual MTU. For a reason I don't understand, the main LAN bridge was adopting the same MTU - 1378. Assuming that the MTU is switch chip specific, I checked few other routers with the same model (hAP ax2) and I found that those other routers had 1500 actual MTU of their main LAN bridge. So I manually set 1500 actual MTU of the main LAN bridge of the main hAP ax2 PPPoE router and this fixed all my problems immediately.

Why these 2 bank websites were not loading remains unknown to me. Why the access to that WINS/Samba shares were not working

If one has actual understanding of why this problem arrises, can you please share some insights.

Kind regards!

main hAP ax2 has an EoIP tunnel. The EoIP tunnel interface is a member of that main LAN bridge.

The EoIP interface itself has 1378 actual MTU. For a reason I don't understand, the main LAN bridge was adopting the same MTU - 1378.

The reason is actually very simple. For the router itself, the IP interface is the bridge, so when the routing engine needs to learn the maximum size of a packet it can afford to send to a destination reachable via the bridge, it asks what the MTU of the bridge interface is. Therefore, the bridge MTU automatically adjusts to the smallest one of the MTUs of all member ports of that bridge - which, in your case, happened to be the EoIP one. Otherwise, the packets that the bridge would accept would not fit to the MTU of some of the member ports and get dropped.

I manually set 1500 actual MTU of the main LAN bridge of the main hAP ax2 PPPoE router and this fixed all my problems immediately.

As explained above, it probably hasn't, you just haven't found that out yet - now connections via the EoIP tunnel will most likely fail because the router will be sending 1500 byte packets and the EoIP tunnel will be dropping them. So rather than setting the bridge MTU to 1500, you should have set the EoIP MTU to 1500 and allow fragmentation of the EoIP transport packets and prohibit inheritance of the Don't Fragment bit. This is likely to work (for the price of doubling the packet rate between the EoIP endpoints), but you may still experience packet loss in the EoIP tunnel if something on the path between them has problems with non-first fragments (too many networks do). If this is your case, the only remedy I know is to use L2TP with MLPPP enabled to replace fragmentation on IP level by "splitting" on MLPPP level, which means that oversize payload packets are delivered using two normal-sized transport ones rather than a single fragmented one. So the double packet rate remains but no fragments traverse the internet.

Why these 2 bank websites were not loading remains unknown to me. Why the access to that WINS/Samba shares were not working

As for the banks, there may be a misconfiguration that prevents the PMTUD process from making its job, or someone on the path may do some manipulations incorrectly (i.e. force a MSS value that corresponds to a smaller MTU than 1500 but larger than the one corresponding to 1378 without checking the actual one first), or the banks may intentionally drop connections with unusual MSS, even if adjusted using PMTUD. Regarding WINS/SMB I won't even speculate.