Hello All,
I have a RB1100 AH X2 (routeros v 6.47) with simple setup to provide internet for 2 networks (Offices and customers).
My line is 100Mbps (up/down) from fiber line. We have some wired problems with internet.
For example netflix is too slow to start a movie or www.speedtest.net takes too long to start the test.
I couldn't find whats wrong and my ISP told us that everything is fine with the line. Asking around someone suggested
me to use the following rules on mikrotik
chain=forward action=change-mss new-mss=1418 passthrough=yes tcp-flags=syn protocol=tcp out-interface=ether11-wan1 tcp-mss=1419-65535 log=no log-prefix=""
chain=forward action=change-mss new-mss=1418 passthrough=yes tcp-flags=syn protocol=tcp in-interface=ether11-wan1 tcp-mss=1419-65535 log=no log-prefix=""
and then all the problems solved.
My question is why this happening ??
I got 3 offices with same setups and only in one of them i have this problem and i need to use the above rules.
I tested my laptop connected directly to mikrotik and it does the same, if I connected directly to the fiber line it works
perfect so something is wrong with the mikrotik but can't find it. Any suggestions please.
Thank you
Re: Strange problem with Internet
It means your provider gives you less MTU than the standard 1500.
(and it also means there are firewall errors on the internet, people who block ICMP. but there is little you can do about that, except on your own router. you should allow ICMP, also when some clueless advisors on the internet tell you to block it)
(and it also means there are firewall errors on the internet, people who block ICMP. but there is little you can do about that, except on your own router. you should allow ICMP, also when some clueless advisors on the internet tell you to block it)
Re: Strange problem with Internet
TCP essentially transports a byte stream. As part of the packet data are headers, it is more efficient to slice the stream into few large packets than into many small ones. So the client and the server negotiate the maximum size of the data slice they can send to each other (MSS, Maximum Segment Size) depending on the MTU value they get from their local network stacks.
But if there is a bottleneck somewhere on the network path between the two, where the MTU is smaller than at their local interfaces, packets that did fit locally don't fit at that bottleneck. Normally, when a packet reaches a device where it doesn't fit into the outgoing interface's MTU, that device cuts it into fragments, and each fragment gets its own headers so the bandwidth efficiency decreases.
Therefore, TCP uses a "path MTU discovery" mechanism, which marks the sent packets with a "don't fragment" flag. So instead of fragmenting the packet exceeding the MTU, the bottleneck device sends back an ICMP message to the sender, informing it that the packet didn't fit and what is the available MTU, and the sender modifies the MSS value for the TCP session accordingly and sends a shorter slice of the byte stream.
And the above is what doesn't work in case of that 1100 - there is a bottleneck somewhere between those servers and that 1100, but someone is dropping ICMP packets on the path between the bottleneck and the server, so the packets from the server never reach the client. The mangle rules suggested to you affect the initial MSS negotiation between the clients and servers, and makes them negotiate a smaller one than they actually offer, so the fact that the path MTU discovery is broken doesn't matter as it is never used.
The price to pay is that the MSS is smaller for all connections, including those which don't run through the path on which the MTU discovery is broken.
But if there is a bottleneck somewhere on the network path between the two, where the MTU is smaller than at their local interfaces, packets that did fit locally don't fit at that bottleneck. Normally, when a packet reaches a device where it doesn't fit into the outgoing interface's MTU, that device cuts it into fragments, and each fragment gets its own headers so the bandwidth efficiency decreases.
Therefore, TCP uses a "path MTU discovery" mechanism, which marks the sent packets with a "don't fragment" flag. So instead of fragmenting the packet exceeding the MTU, the bottleneck device sends back an ICMP message to the sender, informing it that the packet didn't fit and what is the available MTU, and the sender modifies the MSS value for the TCP session accordingly and sends a shorter slice of the byte stream.
And the above is what doesn't work in case of that 1100 - there is a bottleneck somewhere between those servers and that 1100, but someone is dropping ICMP packets on the path between the bottleneck and the server, so the packets from the server never reach the client. The mangle rules suggested to you affect the initial MSS negotiation between the clients and servers, and makes them negotiate a smaller one than they actually offer, so the fact that the path MTU discovery is broken doesn't matter as it is never used.
The price to pay is that the MSS is smaller for all connections, including those which don't run through the path on which the MTU discovery is broken.
Re: Strange problem with Internet
Thanks for the info.
I will contact again my ISP to ask them about the actual MTU they have setup on my line
and I will see after that.
thanks
I will contact again my ISP to ask them about the actual MTU they have setup on my line
and I will see after that.
thanks
Re: Strange problem with Internet
It's a misunderstanding. You wrote that from a PC directly connected to the ISP's gear it works fine, so the MTU on the uplink is OK. But, although you haven't posted your actual configuration, it is likely that you use PPPoE as WAN, so when you connect the PC to the 1100, the PC gets an MTU of 1500 but the uplink only has 1492 because PPPoE is used. So it seems as if you were blocking ICMP from WAN to the PC at your end.
Posting configurations of the 1100s from the non-working site and one of the working ones may reveal some difference in uplink or firewall rules.
Posting configurations of the 1100s from the non-working site and one of the working ones may reveal some difference in uplink or firewall rules.
Re: Strange problem with Internet
Hello,
We don't use PPPoE as WAN. We have a static ip for the router and a gateway that our ISP provide us and
I have only firewall rules to block access between the 2 networks. Don't have anything else in firewall.
here is the config from the mikrotik i got the problem. I changed a little the sensitive information but its simple config.
"....
/interface bridge
add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no fast-forward=no name=bridge.guests.lan
add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no fast-forward=no name=bridge.offices.lan
/interface ethernet
set [ find default-name=ether6 ] name=ether6-guests.lan
set [ find default-name=ether7 ] name=ether7-guests.lan
set [ find default-name=ether10 ] name=ether10-offices.lan
set [ find default-name=ether11 ] name=ether11-wan1
/ip pool
add name=dhcp.guests.lan ranges=10.0.0.2-10.0.0.254
(Offices are all static ip)
/ip dhcp-server
add add-arp=yes address-pool=dhcp.guests.lan disabled=no interface=bridge.guests.lan lease-time=12h name=dhcp.guests.lan
/system logging action
set 0 memory-lines=2000
/interface bridge port
add bridge=bridge.guests.lan interface=ether6-guests.lan
add bridge=bridge.guests.lan interface=ether7-guests.lan
add bridge=bridge.offices.lan interface=ether10-offices.lan
add bridge=bridge.guests.lan interface=ether9.guests.lan
/ip settings
set rp-filter=strict tcp-syncookies=yes
/ip address
add address="public ip" comment="ISP - wan1" interface=ether11-wan1 network="public ip network"
add address=10.10.10.1/24 comment=offices.lan interface=bridge.offices.lan network=10.10.10.0
add address=10.0.0.1/24 comment=guests.lan interface=bridge.guests.lan network=10.0.0.0
/ip dhcp-server network
add address=10.0.0.0/24 gateway=10.0.0.1
/ip dns
set servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=drop chain=forward in-interface=bridge.guests.lan out-interface=bridge.offices.lan
add action=drop chain=forward in-interface=bridge.offices.lan out-interface=bridge.guests.lan
/ip firewall mangle
add action=change-mss chain=forward new-mss=1418 out-interface=ether11-wan1 passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1419-65535
add action=change-mss chain=forward in-interface=ether11-wan1 new-mss=1418 passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1419-65535
/ip firewall nat
add action=masquerade chain=srcnat comment=bridge.guests.lan out-interface=ether11-wan1 src-address=10.0.0.0/24
add action=masquerade chain=srcnat comment=offices.lan out-interface=ether11-wan1 src-address=10.10.10.0/24
/ip route
add comment="Gateway All Networks" distance=1 gateway=(my ips provided gateway)
/tool bandwidth-server
set authenticate=no enabled=no
..."
I checked with my working mikrotik and all are the same. not many to check
Thanks
We don't use PPPoE as WAN. We have a static ip for the router and a gateway that our ISP provide us and
I have only firewall rules to block access between the 2 networks. Don't have anything else in firewall.
here is the config from the mikrotik i got the problem. I changed a little the sensitive information but its simple config.
"....
/interface bridge
add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no fast-forward=no name=bridge.guests.lan
add admin-mac=XX:XX:XX:XX:XX:XX auto-mac=no fast-forward=no name=bridge.offices.lan
/interface ethernet
set [ find default-name=ether6 ] name=ether6-guests.lan
set [ find default-name=ether7 ] name=ether7-guests.lan
set [ find default-name=ether10 ] name=ether10-offices.lan
set [ find default-name=ether11 ] name=ether11-wan1
/ip pool
add name=dhcp.guests.lan ranges=10.0.0.2-10.0.0.254
(Offices are all static ip)
/ip dhcp-server
add add-arp=yes address-pool=dhcp.guests.lan disabled=no interface=bridge.guests.lan lease-time=12h name=dhcp.guests.lan
/system logging action
set 0 memory-lines=2000
/interface bridge port
add bridge=bridge.guests.lan interface=ether6-guests.lan
add bridge=bridge.guests.lan interface=ether7-guests.lan
add bridge=bridge.offices.lan interface=ether10-offices.lan
add bridge=bridge.guests.lan interface=ether9.guests.lan
/ip settings
set rp-filter=strict tcp-syncookies=yes
/ip address
add address="public ip" comment="ISP - wan1" interface=ether11-wan1 network="public ip network"
add address=10.10.10.1/24 comment=offices.lan interface=bridge.offices.lan network=10.10.10.0
add address=10.0.0.1/24 comment=guests.lan interface=bridge.guests.lan network=10.0.0.0
/ip dhcp-server network
add address=10.0.0.0/24 gateway=10.0.0.1
/ip dns
set servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=drop chain=forward in-interface=bridge.guests.lan out-interface=bridge.offices.lan
add action=drop chain=forward in-interface=bridge.offices.lan out-interface=bridge.guests.lan
/ip firewall mangle
add action=change-mss chain=forward new-mss=1418 out-interface=ether11-wan1 passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1419-65535
add action=change-mss chain=forward in-interface=ether11-wan1 new-mss=1418 passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=1419-65535
/ip firewall nat
add action=masquerade chain=srcnat comment=bridge.guests.lan out-interface=ether11-wan1 src-address=10.0.0.0/24
add action=masquerade chain=srcnat comment=offices.lan out-interface=ether11-wan1 src-address=10.10.10.0/24
/ip route
add comment="Gateway All Networks" distance=1 gateway=(my ips provided gateway)
/tool bandwidth-server
set authenticate=no enabled=no
..."
I checked with my working mikrotik and all are the same. not many to check
Thanks
Re: Strange problem with Internet
Leaving aside that it is not a good idea to have nothing in the firewall (despite that all connections initiated by LAN hosts are NATed, without any rules in filter chain input, the management of the router itself is accessible via WAN), it means that it cannot be your firewall which prevents the ICMP "fragmentation needed" messages from being delivered. The only non-default setting which I can imagine to be related is the rp-filter=strict under /ip settings (although I don't understand how it should be related in particular). For your use case, this setting is not necessary as you have just a single WAN, so try setting it to no to see whether it changes anything.I have only firewall rules to block access between the 2 networks. Don't have anything else in firewall.
But still, the MTU on the uplink itself must be fine since a PC directly connected to it has no issues, whilst the same PC connected via the Mikrotik does have them. The output of put [interface get ether11-wan1 actual-mtu] should confirm this (it should show 1500).
So if setting rp-filter to no does not help, only sniffing the traffic can give more insight.
Re: Strange problem with Internet
Hi,
Thanks for the suggestions. I'll check it later.
thanks
Thanks for the suggestions. I'll check it later.
thanks
Re: Strange problem with Internet
Just export the configs of all 3 by using "/export" and compare what the differences are.I couldn't find whats wrong and my ISP told us that everything is fine with the line. Asking around someone suggested
me to use the following rules on mikrotik
chain=forward action=change-mss new-mss=1418 passthrough=yes tcp-flags=syn protocol=tcp out-interface=ether11-wan1 tcp-mss=1419-65535 log=no log-prefix=""
chain=forward action=change-mss new-mss=1418 passthrough=yes tcp-flags=syn protocol=tcp in-interface=ether11-wan1 tcp-mss=1419-65535 log=no log-prefix=""
and then all the problems solved.
My question is why this happening ??
I got 3 offices with same setups and only in one of them i have this problem and i need to use the above rules.
And: you better should have a DNS server of your own in your LAN and all clients should use that DNS server, not external ones like 8.8.8.8 as they are very slow and often timeouts happen. Practically all routers can be used as local DNS servers (you just need to activate it).