Hi all,
I have a problem with RB1100AH with 5.21 ROS.
I use it in the most basic scenario – clear routing with static routes and some firewall rules.
The problem is 2-3% packet loss from fragmented packets.
I analyzed deeply the situation and also prepared a test setup in order to fond the reason.
So it happens in fallowing conditions:
- at least one enabled firewall filter rule (input or forward) or connection tracking enabled
- fragmented input packets with size of fragments that allows assembled packet to be passed through out interface without fragmentation.
- total routed traffic (not mandatory to be fragmented) > 10-15mbps
As result of above combination there are 2-3% packet loss of fragmented part of traffic (other, not fragmented traffic is routed without loss)
I noticed that once the firewall is enabled Mikrotik start to reassemble the packets and causes packet loss. If firewall and connection tracking are stopped, packets are not assembled and there is no problem.
So I would like to ask for ideas how to prevent this packet loss without stopping firewall.
And one additional question - where can i find information about assembling behavior of ROS? In which situations Mikrotik perform packet assembling?
I will wait for ideas!
Thanks in advance.
Re: Packet loss after fragment assembling
Hi again
no any ideas ?
May be my explanation is not clear enough.
Attached scheme is my simulation setup - just 2 PCs connected to 2 ports of RB1100AH.
I intentionally decreased the MTU of first PC in order to send fragmented packets.
Test is performed with iperf as described in the picture and the result is 2,9% packet loss.
Please help!
And my configuration:
no any ideas ?
May be my explanation is not clear enough.
Attached scheme is my simulation setup - just 2 PCs connected to 2 ports of RB1100AH.
I intentionally decreased the MTU of first PC in order to send fragmented packets.
Test is performed with iperf as described in the picture and the result is 2,9% packet loss.
Please help!
And my configuration:
Code: Select all
/ip address add address=10.11.10.1/24 interface=ether1
/ip address add address=10.248.248.1/24 interface=ether2
/ip firewall connection tracking set enabled=no
/ip firewall filter add action=accept chain=forward disabled=no
You do not have the required permissions to view the files attached to this post.
-
-
onnoossendrijver
Member
- Posts: 488
- Joined:
- Location: The Netherlands
Re: Packet loss after fragment assembling
Maybe I'm wrong.. but this way your PC is not sending fragmented packets, but receiving them.
PC1 sends 1400 byte frames to RB, which is fine, because RB has a higher MTU.
Traffic that's coming back has 1500byte frame size, which is too large for PC1, because it only accepts frames up to 1400 byte size.
You should decrease the MTU to 1400 on de ether1 interface of the RB. That way the RB can construct neat 1400 byte frames for PC1, instead of letting PC1 just cut off the last 100 bytes of the 1500 bytes frames.
PC1 sends 1400 byte frames to RB, which is fine, because RB has a higher MTU.
Traffic that's coming back has 1500byte frame size, which is too large for PC1, because it only accepts frames up to 1400 byte size.
You should decrease the MTU to 1400 on de ether1 interface of the RB. That way the RB can construct neat 1400 byte frames for PC1, instead of letting PC1 just cut off the last 100 bytes of the 1500 bytes frames.
Re: Packet loss after fragment assembling
Thanks onnoossendrijver
Unfortunately this is not the problem.
The last my post is just simulation in order to find real reason. The traffic here is one way UDP from PC1(left one) that is verified in PC2(right). Packets from PC1 is sent fragmented because the sum of payload +headers exceeds the MTU of interface. The same sum is small enough to be passed through ether2 of RB1100 and it tries to assemble the packets. After this process we get 2,9% lost packets in PC2. For sure packet loss is happened in RB because I gathered pcap files in both interfaces of RB and following UDP stream I see the difference.
Unfortunately this is not the problem.
The last my post is just simulation in order to find real reason. The traffic here is one way UDP from PC1(left one) that is verified in PC2(right). Packets from PC1 is sent fragmented because the sum of payload +headers exceeds the MTU of interface. The same sum is small enough to be passed through ether2 of RB1100 and it tries to assemble the packets. After this process we get 2,9% lost packets in PC2. For sure packet loss is happened in RB because I gathered pcap files in both interfaces of RB and following UDP stream I see the difference.
-
-
onnoossendrijver
Member
- Posts: 488
- Joined:
- Location: The Netherlands
Re: Packet loss after fragment assembling
I understand your problem now. Unfortunately I don't have an explanation.
Re: Packet loss after fragment assembling
Hi!
We faced same problem on 5.2.2
Fragmented IP packets was randomly dropped, in random data streams (directions).
To Support: please, pay attention to this problem. Our clients suffer.
Report, please, if problem in the last ROS version is solved..
Test:
Ping with big fragmented packet (all lines, switches,ets. was tested - no drops):
sputnik# ping -s 10000 -i 0.01 x.y.z.7
PING (x.y.z.7): 10000 data bytes
10008 bytes from x.y.z.7: icmp_seq=0 ttl=62 time=1.407 ms
10008 bytes from x.y.z.7: icmp_seq=1 ttl=62 time=1.565 ms
10008 bytes from x.y.z.7: icmp_seq=2 ttl=62 time=1.110 ms
10008 bytes from x.y.z.7: icmp_seq=3 ttl=62 time=1.304 ms
10008 bytes from x.y.z.7: icmp_seq=4 ttl=62 time=1.630 ms
10008 bytes from x.y.z.7: icmp_seq=5 ttl=62 time=1.528 ms
10008 bytes from x.y.z.7: icmp_seq=7 ttl=62 time=1.766 ms
10008 bytes from x.y.z.7: icmp_seq=8 ttl=62 time=1.336 ms
10008 bytes from x.y.z.7: icmp_seq=9 ttl=62 time=1.133 ms
10008 bytes from x.y.z.7: icmp_seq=10 ttl=62 time=1.527 ms
......
10008 bytes from x.y.z.7: icmp_seq=688 ttl=62 time=1.552 ms
10008 bytes from x.y.z.7: icmp_seq=689 ttl=62 time=1.601 ms
10008 bytes from x.y.z.7: icmp_seq=690 ttl=62 time=1.576 ms
10008 bytes from x.y.z.7: icmp_seq=691 ttl=62 time=1.489 ms
10008 bytes from x.y.z.7: icmp_seq=692 ttl=62 time=1.341 ms
10008 bytes from x.y.z.7: icmp_seq=693 ttl=62 time=1.545 ms
10008 bytes from x.y.z.7: icmp_seq=694 ttl=62 time=2.128 ms
10008 bytes from x.y.z.7: icmp_seq=695 ttl=62 time=1.249 ms
10008 bytes from x.y.z.7: icmp_seq=696 ttl=62 time=1.307 ms
10008 bytes from x.y.z.7: icmp_seq=697 ttl=62 time=2.360 ms
10008 bytes from x.y.z.7: icmp_seq=698 ttl=62 time=1.775 ms
10008 bytes from x.y.z.7: icmp_seq=699 ttl=62 time=1.522 ms
10008 bytes from x.y.z.7: icmp_seq=700 ttl=62 time=1.773 ms
10008 bytes from x.y.z.7: icmp_seq=702 ttl=62 time=1.531 ms
10008 bytes from x.y.z.7: icmp_seq=703 ttl=62 time=1.707 ms
10008 bytes from x.y.z.7: icmp_seq=704 ttl=62 time=1.340 ms
^C
--- ping statistics ---
705 packets transmitted, 668 packets received, 5.2% packet loss
round-trip min/avg/max/stddev = 0.893/1.534/3.735/0.366 ms
Ping with a packet size equal MTU and exceeding on 1 byte, are started at the same time.
....
1480 bytes from x.y.z.7: icmp_seq=24116 ttl=62 time=0.826 ms
1480 bytes from x.y.z.7: icmp_seq=24117 ttl=62 time=0.533 ms
1480 bytes from x.y.z.7: icmp_seq=24118 ttl=62 time=0.717 ms
1480 bytes from x.y.z.7: icmp_seq=24119 ttl=62 time=0.918 ms
1480 bytes from x.y.z.7: icmp_seq=24120 ttl=62 time=0.870 ms
1480 bytes from x.y.z.7: icmp_seq=24121 ttl=62 time=0.636 ms
^C
--- ping statistics ---
24122 packets transmitted, 24094 packets received, 0.1% packet loss
round-trip min/avg/max/stddev = 0.437/0.898/582.494/5.015 ms
....
1481 bytes from x.y.z.7: icmp_seq=24116 ttl=62 time=0.599 ms
1481 bytes from x.y.z.7: icmp_seq=24117 ttl=62 time=0.597 ms
1481 bytes from x.y.z.7: icmp_seq=24118 ttl=62 time=1.002 ms
1481 bytes from x.y.z.7: icmp_seq=24119 ttl=62 time=0.814 ms
1481 bytes from x.y.z.7: icmp_seq=24120 ttl=62 time=0.583 ms
^C
--- ping statistics ---
24121 packets transmitted, 24005 packets received, 0.5% packet loss
round-trip min/avg/max/stddev = 0.480/0.940/416.322/3.328 ms
Conclusion: Without use of fragmentation, drops much lower.
With Packet Sniffer on Mikrotik, I don't see all fragments of received packets (i.e. drop before sniffer process).
We faced same problem on 5.2.2
Fragmented IP packets was randomly dropped, in random data streams (directions).
To Support: please, pay attention to this problem. Our clients suffer.
Report, please, if problem in the last ROS version is solved..
Test:
Ping with big fragmented packet (all lines, switches,ets. was tested - no drops):
sputnik# ping -s 10000 -i 0.01 x.y.z.7
PING (x.y.z.7): 10000 data bytes
10008 bytes from x.y.z.7: icmp_seq=0 ttl=62 time=1.407 ms
10008 bytes from x.y.z.7: icmp_seq=1 ttl=62 time=1.565 ms
10008 bytes from x.y.z.7: icmp_seq=2 ttl=62 time=1.110 ms
10008 bytes from x.y.z.7: icmp_seq=3 ttl=62 time=1.304 ms
10008 bytes from x.y.z.7: icmp_seq=4 ttl=62 time=1.630 ms
10008 bytes from x.y.z.7: icmp_seq=5 ttl=62 time=1.528 ms
10008 bytes from x.y.z.7: icmp_seq=7 ttl=62 time=1.766 ms
10008 bytes from x.y.z.7: icmp_seq=8 ttl=62 time=1.336 ms
10008 bytes from x.y.z.7: icmp_seq=9 ttl=62 time=1.133 ms
10008 bytes from x.y.z.7: icmp_seq=10 ttl=62 time=1.527 ms
......
10008 bytes from x.y.z.7: icmp_seq=688 ttl=62 time=1.552 ms
10008 bytes from x.y.z.7: icmp_seq=689 ttl=62 time=1.601 ms
10008 bytes from x.y.z.7: icmp_seq=690 ttl=62 time=1.576 ms
10008 bytes from x.y.z.7: icmp_seq=691 ttl=62 time=1.489 ms
10008 bytes from x.y.z.7: icmp_seq=692 ttl=62 time=1.341 ms
10008 bytes from x.y.z.7: icmp_seq=693 ttl=62 time=1.545 ms
10008 bytes from x.y.z.7: icmp_seq=694 ttl=62 time=2.128 ms
10008 bytes from x.y.z.7: icmp_seq=695 ttl=62 time=1.249 ms
10008 bytes from x.y.z.7: icmp_seq=696 ttl=62 time=1.307 ms
10008 bytes from x.y.z.7: icmp_seq=697 ttl=62 time=2.360 ms
10008 bytes from x.y.z.7: icmp_seq=698 ttl=62 time=1.775 ms
10008 bytes from x.y.z.7: icmp_seq=699 ttl=62 time=1.522 ms
10008 bytes from x.y.z.7: icmp_seq=700 ttl=62 time=1.773 ms
10008 bytes from x.y.z.7: icmp_seq=702 ttl=62 time=1.531 ms
10008 bytes from x.y.z.7: icmp_seq=703 ttl=62 time=1.707 ms
10008 bytes from x.y.z.7: icmp_seq=704 ttl=62 time=1.340 ms
^C
--- ping statistics ---
705 packets transmitted, 668 packets received, 5.2% packet loss
round-trip min/avg/max/stddev = 0.893/1.534/3.735/0.366 ms
Ping with a packet size equal MTU and exceeding on 1 byte, are started at the same time.
....
1480 bytes from x.y.z.7: icmp_seq=24116 ttl=62 time=0.826 ms
1480 bytes from x.y.z.7: icmp_seq=24117 ttl=62 time=0.533 ms
1480 bytes from x.y.z.7: icmp_seq=24118 ttl=62 time=0.717 ms
1480 bytes from x.y.z.7: icmp_seq=24119 ttl=62 time=0.918 ms
1480 bytes from x.y.z.7: icmp_seq=24120 ttl=62 time=0.870 ms
1480 bytes from x.y.z.7: icmp_seq=24121 ttl=62 time=0.636 ms
^C
--- ping statistics ---
24122 packets transmitted, 24094 packets received, 0.1% packet loss
round-trip min/avg/max/stddev = 0.437/0.898/582.494/5.015 ms
....
1481 bytes from x.y.z.7: icmp_seq=24116 ttl=62 time=0.599 ms
1481 bytes from x.y.z.7: icmp_seq=24117 ttl=62 time=0.597 ms
1481 bytes from x.y.z.7: icmp_seq=24118 ttl=62 time=1.002 ms
1481 bytes from x.y.z.7: icmp_seq=24119 ttl=62 time=0.814 ms
1481 bytes from x.y.z.7: icmp_seq=24120 ttl=62 time=0.583 ms
^C
--- ping statistics ---
24121 packets transmitted, 24005 packets received, 0.5% packet loss
round-trip min/avg/max/stddev = 0.480/0.940/416.322/3.328 ms
Conclusion: Without use of fragmentation, drops much lower.
With Packet Sniffer on Mikrotik, I don't see all fragments of received packets (i.e. drop before sniffer process).