You mentioned that the second fragment never appears on the device directly connected to RB5009 Ethernet port. It could be something related to RB5009 Ethernet or switch. Can you quickly test if disabling bridge HW offloading makes any difference (I assume you are using bridge)? Can you share supout.rif file and pcap files from the RB5009
to support?
I have disabled hardware offload on the bridge's ethernet ports on the RB5009 and the behavior was the same. But I have since made many more tests, and have found out that the problem is related to the size of the second fragment. Please note that the following test outcomes have been repeated with both
hardware offload turned on and off, and with both
tagged and untagged VLANs (my config doesn't use the native VLAN of the bridge interface):
I wanted to test whether the problem is solely caused by User Manager, or whether RouterOS 7.17 is unable to properly fragment UDP packets. At first, I tried Traffic Generator, but it doesn't seem to automatically fragment UDP packets. Luckily, there is another tool in RouterOS that allows sending UPD packets with arbitrary sizes, and that's the Traceroute tool in UDP mode (instead of ICMP). However, the Packer Size field doesn't allow any value larger than 1500. So, the next logical step is to reduce the MTU on the (VLAN) interfaces. At first, I reduced the MTU down to 1480, but the packets seem to be broken up at the 1476-byte mark anyway, so there is no difference between MTU 1480 and MTU 1476 for the following findings.
With the MTU set to 1480 (or 1476), I used Traceroute to send UPD packets to a Windows machine running Wireshark. Windows doesn't support UDP traceroute but that's not important because I only wanted to capture the incoming packets. By setting the traceroute Packet Size to 1482, I got the same fragmentation where the 2nd fragment has the same 40-byte frame size:
traceroute-1482-ros.png
And the exact same issue as the one with User Manager happened. After a while the Packet Sniffer showed a bunch of ICMP packets. On the Windows machine, Wireshark showed that the 40-byte frames never arrived, only the 1490-byte ones. And Windows sent the ICMP Fragment reassembly time exceeded responses.
traceroute-1482-win.png
But at this moment, with the MTU set to 1480, the same RADIUS authentication tests with eapol_test
suceeded! Obviously, with smaller MTU the 2nd fragment got bigger. So I tried to increase the size of the traceroute packet to 1492:
traceroute-1492-ros.png
Packet sniffer showed the ethernet frame size of the 2nd fragment to be 50 bytes. This time there is no ICMP packets. And on the windows machine, all packets
have arrived! But the 2nd fragments have a frame size of 56 bytes. With trailer inserted at the end.
traceroute-1492-win.png
Next, I tried to increase the traceroute packet size from 1482. At 1483 bytes, a 41-byte 2nd fragment was produced, but also disappeared and never arrived. At 1484 bytes however, just two bytes larger than 1482, the fragmentation worked, and the Windows machine could reassemble all the UDP packets:
traceroute-1484-ros.png
We can also see that the 2nd fragments are listed as having a 42-byte framesize on ROS, but were sent as 56-byte frames with 14-byte trailer:
traceroute-1484-win.png
When I changed the traceroute target to one UniFi AP, which runs Linux, the traceroute worked with packet size 1484 (because the AP understands UDP traceroute and can respond), but also failed with ICMP fragmentation timeout when the packet size was reduced to 1482 bytes.
traceroute-1484-ap.png
traceroute-1482-ap.png
It looks like RouterOS is unable to
append a 15- or 16-byte trailer, so that the ethernet frame sent can reach the minimum allowed size? Am I unlucky that my certificate size somehow causes User Manager to produce UDP packets that cause fragmentation with the 40-byte second fragment (at MTU 1500)? Maybe I should try to add some junks into the different fields of the certificate to make it larger 🙃?
You do not have the required permissions to view the files attached to this post.