Hey guys. I recently bought two new MT routers. My main router is directly connected to the ISP fiber modem and it's "authenticated" using PPPoE. I've noticed that some websites were not opening at all, and then I stumbled upon viewtopic.php?t=150377 which helped me temporary solve the issue, by adding:
Whatsoever, because of my professional deformation, aka, I'm a software engineer, I hate just copy/pasting things and not knowing what I did, why things work, etc.

I've noticed that my PPPoE interface has an actual MTU of 1472. I suspect that this number was automatically set somehow by the actual PPPoE connection. Now, I'm wondering, am I suppose to change the MTU on all other interfaces, e.g. LAN, WAN, etc?

In simple words, how do I debug this, are there some rules about setting MTU manually and how do I go about this.

The main router I'm using is: https://mikrotik.com/product/RB3011UiAS-RM. I use this for my remote office and generally to plug in my servers, machines, etc over ethernet.

In one of the ethernet ports, I have this fellow https://mikrotik.com/product/RB2011UiAS-2HnD also in the Router (not bridge) mode, used for home related stuff.

Here is the screenshot of my interfaces, in case it can shed light on anything.

You have only to "determne the MTU from packets that are returning:
The MTU seems strange because the VLAN is 1492 and so the PPPoE MTU could be 1488.

You have only to "determne the MTU from packets that are returning:

Code: Select all

/ip firewall mangle
add action=change-mss chain=forward in-interface="Telenor PPPoE new-mss=clamp-to-pmtu passthrough=yes protocol=tcp tcp-flags=syn

The MTU seems strange because the VLAN is 1492 and so the PPPoE MTU could be 1488.

Hi and thanks a bunch for the response. I did fiddle past couple of minutes with the MTUs (apart from PPPoE one as it seems to be read only) so it could be that.

You wrote:

You have only to "determne the MTU from packets that are returning:

I have added your rule now, so where or to say, how can I know see/determine what the MTU is and do I have to manually change it for all other interfaces?

The change-tcp-mss=yes setting in the default and default-encryption PPP profiles will sort out MSS clamping on the PPPoE connection (PPPoE client interfaces default to the default profile).

The PPPoE client MRU/MTU should be adjusted to best fill any segmentation imposed by the WAN technology, often 1492 if baby jumbo / RFC4638 is not supported.

Standard ethenet interfaces should always have an MTU of 1500 unless using jumbo frames, setting it to less will randomly break connectivity and additional mangle rules will not fix ensuing LAN-to-LAN problems.

Whenever I see an MTU set to 1472, I suspect that whoever set it learned networking on Cisco and were used to the cisco (or MikroTik) ping size syntax which specifies the packet size. Windows ping -f -l 1472 generates 1500 byte packets (1472 icmp payload + 8 byte icmp header + 20 byte ip header). Same with linux ping -M do -s 1472 <ip>

Many wrong examples on the web for "determining the MTU with ping" where it is suggested that a successful Windows ping -f -l 1472 <ip> followed by an unsuccessful ping -f -l 1473 <ip> is an indication of MTU 1472. But that is not correct, that indicates the MTU is 1500 (you need to add 28 to get the MTU).

Here's one example of incorrect advice. Use Ping to Find a Path’s MTU (and wayback machine link in case it gets "fixed")

So unless there is another explaination, it seems that the link's path MTU is 1472 because someone has a router's MTU set to 1472 when it should be 1500. (not your router).

More links in this post about why mss clamping exists.

My main router is directly connected to the ISP fiber modem and it's "authenticated" using PPPoE.

The ISP fiber modem is giving you a MTU 1492 (default for PPPoE) and then you are doing a PPPoE over PPPoE?

Fiber (modem) 1492, VLAN on that also 1492 and then a PPPoE of 1472.

First if you have an PPPoE connection on the modem why then do not use that directly. Second, inform with you ISP if your modem can be set to bridge mode and then your router can use the created PPPoE solely. Third, can you use your SFP1 with a module to create a connection without needing the ISP modem? Inform with your ISP and search on the internet for others that use the same ISP.

And last, if you modem is in bridge mode and you get 1492 as MTU you lose 8bits to needed PPPoE and check if the modem is not already applying the VLAN. Test it by disabling the VLAN on your router.

It also requires the ISP getting it right

For example the MTU for Ubiquiti OLT-4 or OLT-8 GPON head end unit is 1518-1982 you can't set it to anything else and the default is 1518.
With the VLAN and GPON and PPPOE overheads that leaves an MTU of 1492 for customer
I could write a book about how many small ISP's connect it on a 1500 MTU router or switch and the humor begins.

In that situation you will see a stress and flustered ISP find they have to set 1472 MTU to get things to work

One of the problems with MTU is it needs everyone in the chain to get it right and given the OP statement my guess is we just found another DORK by a small ISP.
I will really laugh if the ISP is using a ubiquiti OLT-4 or OLT-8 because I know what they have done

Indeed with a modern ISP and good admins you can get MTU 1500 on your PPPoE and avoid a lot of trouble.
Try to set all MTUs back to 1500 (also on the PPPoE) except on Fiber and Telenor VLAN you set it to the max value (or 1520 or so), then you might just get MTU 1500 on your PPPoE.
When not, try complaining to the ISP that they do not have RFC4638 or that there is some config error causing it not to work (as written above).

Sorry if I wasn't clear. Telenor, my ISP, is requiring anyone who's not using their provided router to use PPPoE. I don't have access to the Fiber modem nor I can fiddle with it at all. I spoke to Telenor support here in Denmark and they gave me pretty limited information. In a nutshell, they told me that if I want to use my own router and not theirs, I have to connect using PPPoE, they gave me the VLAN Id (which is 101), PPPoE username and password and that's it. Apart from that, I doubt that they actually know more, cause every time I call them, they safe-guard themselves saying "ah, if you are not using our own router, we can't really help you".

As of the other things you guys wrote, I will read it in detail, try them out and get back to you.

And as always, thanks a bunch for the great community and all of you individually.

I don't know if Telenor is so much worse than our incumbent provider KPN...
But at least here, RFC4638 just works, both on VDSL and on Fiber. We set the parameters as I mentioned above and get PPPoE link with 1500 byte MTU.
When it is not possible to get that, at least you should be able to get MTU 1492 on the PPPoE. But of course do NOT set that on the Fiber port because then
it will go down even further.

When you are not able to have full 1500 byte MTU you will always have issues. MSS clamping is a way to partially get around that, but as others mentioned, problems remain.

I have now set MTU's as shown on the screenshot below and things seem to be working. I have also disabled the mangle change MSS rule for now. I will keep you posted during the day, cause I can almost swear that I've tried this before and that the outcome was different. So something feels really strange, let's see what happens throughout the day.

Also the fiber to 1520 MTU.

Seeing the MTU of the PPPoE is now 1500, confusing but if it works then it works. Test is with ping in tools. Ping to google.com and set the packet size to 1500 and mark the do not fragment both in the advanced tab.

Then start and see if the ping packets are passing.

I tested it here and apparently it is sufficient to set the MTU of the VLAN while leaving the ethernet MTU at 1500.
It seems that the ether1 MTU is only for direct transmissions on ethernet and for any VLAN the MTU is taken from the VLAN interface and it "does not have to fit in" the ethernet MTU.
So the configuration is actually correct, even if intuitively one has to have a larger MTU on the parent interface to allow larger MTU on the child VLAN.
(as of course is the case for e.g. tunnel interfaces)

Next to the used MTU is the L2-MTU and that one is MTU that the interface can handle. As you wrote setting the VLAN higher is allowing the PPPoE sit above 1500. Then having a higher actual MTU on VLAN you would expect that RouterOS would reflect that in the actual MTU of the ether interface.

But then I have experience with MTU being displayed that are not the actual ones. I had to set manually, an MTU and then ping to see if had indeed the a 1500 MTU while RouterOS was displaying a different actual MTU. One of the last Betas had an fix for that when using PPPoE if I remember that correctly.

There should be no need to change the MTU of either the base ethernet interface (Fiber) or the VLAN on top of it (Telenor VLAN101) as this only affects layer 3 IPoE packets, PPPoE is layer 2 and easily fits within the L2MTU of 1594.

With providers supporting RFC4638 this certainly works on 6.48.6 and earlier (have not yet tested on RouterOS 7 to check if the implementation is any different):
/interface ethernet
set [ find default-name=ether10 ] comment="Draytek V130 modem" name=ether10-wan
/interface pppoe-client
add add-default-route=yes allow=chap disabled=no interface=ether10-wan max-mru=1500 max-mtu=1500 name=pppoe-zen password=XXXXXXXX use-peer-dns=yes user=XXXXXXXX

and for those that don't just change MRU/MTU:
max-mru=1492 max-mtu=1492