The deal with change mss

ploquets · Sat May 27, 2017 3:30 am

So, I realized that when change-mss is enable in PPP profile, there is a bit of a problem...

If MAX MTU and MAX MRU is set = 1492
Two mangle rules are dynamically created (of course, this is with a version < 6.39)
One match traffic going to all-ppp interfaces and change MSS to 1452 when MSS is between 1453-65535
And another is to maatch traffic coming from all-ppp interfaces and change the same thing as the rule above.

But... If one PPP is connected and the other side (normally a router from a customer) is configure as MTU 1400, for example..... the rule for traffic coming from all-ppp is automaticaly changed with this new MTU as parameter. For this example, as 1400, the new action is to change mss to 1360

So, all PPP connections will suffer with lower MTU ?

Sometimes is the customer that configured the router, and is making a mess....

How about this new concept in 6.39 ?
!) ppp - implemented internal algorithm for "change-mss", no mangle rules necessary;
How will this actually work?

ploquets · Mon May 29, 2017 9:49 pm

Please, I really need comments on this.

Mikrotik staff?

Thanks

pe1chl · Tue May 30, 2017 7:14 am

So, I realized that when change-mss is enable in PPP profile, there is a bit of a problem...

If MAX MTU and MAX MRU is set = 1492
Two mangle rules are dynamically created (of course, this is with a version < 6.39)
One match traffic going to all-ppp interfaces and change MSS to 1452 when MSS is between 1453-65535
And another is to maatch traffic coming from all-ppp interfaces and change the same thing as the rule above.

But... If one PPP is connected and the other side (normally a router from a customer) is configure as MTU 1400, for example..... the rule for traffic coming from all-ppp is automaticaly changed with this new MTU as parameter. For this example, as 1400, the new action is to change mss to 1360

You should not need any rules like that, when you use "new-mss=clamp-to-pmtu" instead of those explicit
numbers. I have no idea why MikroTik is not doing that, I always disable all that automatic rule creation and
put a single rule in the mangle table:

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn

When you have concerns that it limits performance when you have a lot of ethernet-to-ethernet forwarding you
could always try to limit it to some interface types. But I do not see why it should have fixed values.

Tue May 30, 2017 8:54 am

v6.40rc13 have this line in changelog:
*) ppp - fixed "change-mss" functionality (introduced in 6.39);

so i assume when it is confirmed to test your issue it will be included in v6.39.2

ploquets · Tue May 30, 2017 6:18 pm

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn

If we put this, all packets, even those with MSS lower will have MSS increased.
I don't know much about the concept, but, if a packet want to have a lower MSS, should we let it pass?

Mikrotik default (dynamic) rules have another match, only packets that would try to deal the MSS above the ideal MSS would get changed, like this for 1480 in MAX MTU:

tcp-mss=1441-65535

or this for 1492 in MAX MTU:

tcp-mss=1453-65535

If we ignore this, would we have some collateral effect ??

v6.40rc13 have this line in changelog:
*) ppp - fixed "change-mss" functionality (introduced in 6.39);

so i assume when it is confirmed to test your issue it will be included in v6.39.2

So, I did some testing and, even with PPPoE with MTU as 1500 and change-mss=yes in PPP profile, MSS for SYN packets are still changed by 4 bytes I guess. Don't really know why.

pe1chl · Tue May 30, 2017 6:27 pm

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn
If we put this, all packets, even those with MSS lower will have MSS increased.

Is that a guess, or did you actually observe that?
It should only lower the MSS when too high, never increase it. When you observe increasing MSS by that rule,
please file a bug report to MikroTik (probably with pcap files with evidence).

ploquets · Tue May 30, 2017 7:42 pm

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn
If we put this, all packets, even those with MSS lower will have MSS increased.
Is that a guess, or did you actually observe that?
It should only lower the MSS when too high, never increase it. When you observe increasing MSS by that rule,
please file a bug report to MikroTik (probably with pcap files with evidence).

But that's what I'm saying. If the rule has no match with MSS, it should ignore the original MSS and it will change MSS as the action is change MSS for the max possible, even if it is already under that limit.

This is not a bug, but a rule created for that purpose.

That's what I'm saying. Why do mikrotik is using MSS as a match and you are not....

pe1chl · Tue May 30, 2017 11:25 pm

No that is not correct, "clamp to pmtu" means "lower the mss when too high, leave it alone when already lower".
When you see "clamp to pmtu" increase the mss, please submit a bug report.
I will try to test when I have possibility to make a test setup for this.

Wed May 31, 2017 12:06 pm

Before this change there was dynamic mange change-mss rules created, this enables connection tracking, disables fastpath making everything slow.

Now this functionality is build-in into ppp interfaces themselves, this way no mangle rules are necessary, no connection tracking, fastpath possible - everything much better.

Initial implementation had bugs, it was not taking into account MRRU, now it is fixed in latest RC. As far as i can tell everything is working fine now.

ploquets · Wed May 31, 2017 11:16 pm

Now this functionality is build-in into ppp interfaces themselves

So, imagine my scenario, when those mangle rules were for all-ppp and limited by the minimal MTU.... (before 6.39)

So now, it will consider the MTU from each tunnel ? Which means that I don't have to worry about limiting all-ppp just because some customer messed it up ?

Thanks!

no connection tracking, fastpath possible - everything much better.

For us this is irrelevant, since we need to do some NATing for some customers... which breaks fastpath.

Thu Jun 01, 2017 8:08 am

no connection tracking, fastpath possible - everything much better.

For us this is irrelevant, since we need to do some NATing for some customers... which breaks fastpath.[/quote]

well fasttrack enables you to use fastpath with NAT.

ploquets · Thu Jun 01, 2017 4:06 pm

well fasttrack enables you to use fastpath with NAT.

But I guess fasttrack will break simple queues.
Those queues are dynamically created when "Mikrotik-Rate-Limit" attribute is defined in Radius.

pe1chl · Thu Jun 01, 2017 4:19 pm

The choice is always "do I want to bother with fasttrack and get some additional performance" or "do I want a system where all features work".
For me, until now the choice has always been to disable fastpath and fasttrack, as fortunately everywhere where I need performance I have a CCR.
For others, it may be different and those tweaks may be welcome. Anyway, you can install the latest RC and test it.
I have used the "clamp to pmtu" rule and I think it has worked for me, I want to test if there really is the above mentioned bug in it but I doubt it.

ploquets · Thu Jun 01, 2017 5:45 pm

The choice is always "do I want to bother with fasttrack and get some additional performance" or "do I want a system where all features work".
For me, until now the choice has always been to disable fastpath and fasttrack, as fortunately everywhere where I need performance I have a CCR.
For others, it may be different and those tweaks may be welcome. Anyway, you can install the latest RC and test it.
I have used the "clamp to pmtu" rule and I think it has worked for me, I want to test if there really is the above mentioned bug in it but I doubt it.

I want to see this as well. Since you said "clamp to pmtu" means "lower the mss when too high, leave it alone when already lower". i'm seeking this information... searching every post that would say that, but I could not find anything.

If you could test it, that would be awesome.
Those dynamic rules are not good at all when you have multiple MTU on your costumers, because as I said, they are not respecting the MAX mtu, instead, is making the worse MTU as default parameter.

Wed Jun 07, 2017 2:51 pm

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.

ploquets · Wed Jun 07, 2017 3:45 pm

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.

Thanks about the feedback!

Could you see, please, if all PPPoE interfaces has same MTU?

I want to test it when MTU is not the same for all interfaces, just to see if will work in that case, or if is considering the minimal mtu to run this "new" algorithm.

Thank you for your time

Thu Jun 08, 2017 9:08 am

Could you see, please, if all PPPoE interfaces has same MTU?

I want to test it when MTU is not the same for all interfaces, just to see if will work in that case, or if is considering the minimal mtu to run this "new" algorithm.

Thank you for your time

It looks like packet get adjustment MSS when it travels through particular interface so if one interface have MSS 1400, other 1350, third 1300, packet will get only that mss that is on interface it is going through.

fabiandr · Fri Jun 23, 2017 1:36 am

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.

Is not the following rule necessary?

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn

amt · Fri Jun 23, 2017 11:03 am

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.

this is out of topic but I would like to ask in interface pppoe-server server side, how should i set service. should i set max-mtu=1480 max-mru=1480 mrru=1600 or should i set them auto ?

thanks

ploquets · Fri Jun 23, 2017 3:16 pm

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.
this is out of topic but I would like to ask in interface pppoe-server server side, how should i set service. should i set max-mtu=1480 max-mru=1480 mrru=1600 or should i set them auto ?

thanks

That depends on how much data could a packet carry and flow inside your network.
The default size of an ethernet frame is 1500.
This means: The closer to this number, the better.

MTU is the key to acheive this, but, you need to consider all the protocols that would populate the header of the packet.
For example, if you use only PPPoE and nothing else (like VLAN, MPLS, VPLS, etc) you will need to lower the MSS or increase the L2 MTU from all devices in your path by 8 bytes.
PPPoE uses 8 bytes for the tunnel to be possible.

So, the math is:

1500 - 8 = 1492

This should be the new MTU without needing to increase the L2 MTU in your path.
If you could increase L2 MTU, enabling "jumbo frames" on your switches as well, than you could carry the 1508 byte packet and from this point you would not need tho change MSS at all.
If not, MSS should be decreased to be possible for the packet to flow.

Another example:
If you use MPLS and VLANs, you will need to do the math:

How much does MPLS increase in header?
4 bytes

How much does VLANs increase in header?
802.1q = 4 bytes
QinQ (which means you check the box Service Tag) = 8 bytes

So, do the math:
1500 - 8 (PPPoE) - 4 (MPLS) - 4 (VLAN 802.1q) = 1484 (Max MTU)

You will need to chose what to do. Or you increase your L2 MTU Path (in this example above by 16 bytes) or you need to shrink the MSS (to be able to use the left space on header)
In this scenario, without jumbo frames you will get dropped packets if you don't change MSS (normally by 40 Bytes), causing connections failures for your costumer.

So, the math for MSS is
MAX MTU - 20 (IP Header) - 20 (TCP Header) = MSS

ploquets · Fri Jun 23, 2017 3:20 pm

Just updated whole my pppoe site to 6.39.2, you can forget about change-mss rules, it all is now nicely build-in into ppp interfaces, even with MRU now it works fine.
Is not the following rule necessary?

/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
protocol=tcp tcp-flags=syn

It seems that you would not need this anymore, because now is part of the algorithm

amt · Fri Jun 23, 2017 3:42 pm

Hi ploquets
thank you for your great info..
My network is bridged network. and to sending customer pppoe_client connection to pppoe_server Im uinsg EoIP tunnels.
EoIP tunnels MTU is auto, when i check them actual MTU size is 1458 both side. I bridge wlan interface and EoiP interface at AP side. also pppoe bridge mtu is auto and actual MTU =1458.
On EoiP tunnels settings are as here;

name="xxxx" mtu=auto actual-mtu=1458 l2mtu=65535 mac-address=02:91:77:9D:A7:B2 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off 
      loop-protect-send-interval=5s loop-protect-disable-time=5m local-address=xxxxx remote-address=xxxx tunnel-id=xxx dscp=inherit clamp-tcp-mss=yes 
      dont-fragment=no allow-fast-path=yes

and at PPPoEServer

/interface pppoe-server server
add authentication=pap disabled=no interface=eoip max-mru=1480 max-mtu=1480 mrru=1600 one-session-per-host=yes service-name=XoMniTik

should i change mtu for EoIP , servers or another interface to increase performance ?

Thanks

ploquets · Fri Jun 23, 2017 10:27 pm

Hi ploquets
thank you for your great info..
My network is bridged network. and to sending customer pppoe_client connection to pppoe_server Im uinsg EoIP tunnels.
EoIP tunnels MTU is auto, when i check them actual MTU size is 1458 both side. I bridge wlan interface and EoiP interface at AP side. also pppoe bridge mtu is auto and actual MTU =1458.
On EoiP tunnels settings are as here;
Code: Select all
name="xxxx" mtu=auto actual-mtu=1458 l2mtu=65535 mac-address=02:91:77:9D:A7:B2 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off 
      loop-protect-send-interval=5s loop-protect-disable-time=5m local-address=xxxxx remote-address=xxxx tunnel-id=xxx dscp=inherit clamp-tcp-mss=yes 
      dont-fragment=no allow-fast-path=yes 
and at PPPoEServer
Code: Select all
/interface pppoe-server server
add authentication=pap disabled=no interface=eoip max-mru=1480 max-mtu=1480 mrru=1600 one-session-per-host=yes service-name=XoMniTik
should i change mtu for EoIP , servers or another interface to increase performance ?

Thanks

Hello!
Since PPPoE client is using L2 MTU to reach the other side (PPPoE Server) you should be OK using 1492 as Max MTU
BUT...., keep in mind that what really matter is the path where EoIP is using, switches, interfaces, etc...
This path need to be able to pass the L3 MTU, because, EoIP is a method to stretch an L2 path inside L3 path.

The EoIP packet is up to 1542 byte in size (1500 bytes eoip MTU + 42 bytes EoIP header). If you want to use EoIP without any fragmentation, your MTU from all interfaces should be at least 1542.
And this to pass the normal 1500 bytes.

If you consider PPPoE over this L2 EoIP (which I guess is the case) you will need at least 1550 (to deliver the normal 1500 for the PPPoE interface).

So, EoIP normally will fragments packets and this will cause less performance, even if your PPPoE is 1500, EoIP will fragment those packets cause it needs 42 bytes of header.
Maybe would be better if you use VPLS, but, keep in mind Layer2 MTU to consider VPLS.

Here is a MUM video which you could learn about it:
https://www.youtube.com/watch?v=Q8AF-Srulmk

amt · Sat Jun 24, 2017 8:39 am

Hello,

if I understand correctly

MTU from all interfaces should be at least 1542

I should change mtu size to 1542 to all interfaces like wlan for p2p Wireless links, for AP, and all other lan interfaces in all network

If you consider PPPoE over this L2 EoIP (which I guess is the case) you will need at least 1550 (to deliver the normal 1500 for the PPPoE interface).

yes i will use pppoe over EoIp so in here you said least 1550. I need to change all interface to 1550 or 1542 ? which one is correct ? which one should i do ?
after change all interfaces mtu in all network, what will be the mtu of eoip tunnels ? it will auto again or i should change them to 1500 ?
what will be the Max MTU, Max MRU and MRRU in pppoe_server Server settings which is default 1480.

Thanks.

ploquets · Sat Jun 24, 2017 12:46 pm

Hi ploquets
thank you for your great info..
My network is bridged network. and to sending customer pppoe_client connection to pppoe_server Im uinsg EoIP tunnels.
EoIP tunnels MTU is auto, when i check them actual MTU size is 1458 both side. I bridge wlan interface and EoiP interface at AP side. also pppoe bridge mtu is auto and actual MTU =1458.
On EoiP tunnels settings are as here;
Code: Select all
name="xxxx" mtu=auto actual-mtu=1458 l2mtu=65535 mac-address=02:91:77:9D:A7:B2 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off 
      loop-protect-send-interval=5s loop-protect-disable-time=5m local-address=xxxxx remote-address=xxxx tunnel-id=xxx dscp=inherit clamp-tcp-mss=yes 
      dont-fragment=no allow-fast-path=yes 
and at PPPoEServer
Code: Select all
/interface pppoe-server server
add authentication=pap disabled=no interface=eoip max-mru=1480 max-mtu=1480 mrru=1600 one-session-per-host=yes service-name=XoMniTik
should i change mtu for EoIP , servers or another interface to increase performance ?

Thanks
Hello!
Since PPPoE client is using L2 MTU to reach the other side (PPPoE Server) you should be OK using 1492 as Max MTU
BUT...., keep in mind that what really matter is the path where EoIP is using, switches, interfaces, etc...
This path need to be able to pass the L3 MTU, because, EoIP is a method to stretch an L2 path inside L3 path.

The EoIP packet is up to 1542 byte in size (1500 bytes eoip MTU + 42 bytes EoIP header). If you want to use EoIP without any fragmentation, your MTU from all interfaces should be at least 1542.
And this to pass the normal 1500 bytes.

If you consider PPPoE over this L2 EoIP (which I guess is the case) you will need at least 1550 (to deliver the normal 1500 for the PPPoE interface).

So, EoIP normally will fragments packets and this will cause less performance, even if your PPPoE is 1500, EoIP will fragment those packets cause it needs 42 bytes of header.
Maybe would be better if you use VPLS, but, keep in mind Layer2 MTU to consider VPLS.

Here is a MUM video which you could learn about it:
https://www.youtube.com/watch?v=Q8AF-Srulmk

Keep in mind that I said all interfaces where the EoIP will use as path. If you are using the WLAN interface for the EoIP to flow, if it is part of its path, then you should change, yes. But, if no, then you don't need to.

I said as well about how your PPPoE is working.
If you don't bother with some fragmentation for the customer (where the MSS is changed.... and this is the main topic of this thread) then you should be good to go with 1542 only.

I'm trying to be clear about concepts, for you to understand what you are doing. Not only give you all already chewed.
This is important when you need to troubleshoot.

But, in a simple way:

For EoIP, to pass 1500 bytes inside of it, WITHOUT FRAGMENTATION, you would need the PATH where EoIP flowing, be able to forward at least 1542 bytes L3 MTU.

If you want to deliver PPPoE inside this scenario the FULL FRAME (I'm talking about 1500 bytes, not the 1480 you mentioned or 1492 is the default for PPPoE), and WITHOUT FRAGMENTATION (and I mean full frame inside EoIP and full frames for customer.. All without fragmentation) then you will need to increment the 1452 I mentioned before you to 1550 (1452 +8 PPPoE)

volkirik · Mon Mar 23, 2020 10:53 pm

you need following rule before fasttrack-connection otherwise SYN packets will be fasttracked and clamp-tcp-mss will break

/ip firewall filter
add action=accept chain=forward comment="tcp syn no-FP" connection-state=established,related \
protocol=tcp tcp-flags=syn

fyi.

ploquets · Mon Mar 23, 2020 11:22 pm

you need following rule before fasttrack-connection otherwise SYN packets will be fasttracked and clamp-tcp-mss will break

/ip firewall filter
add action=accept chain=forward comment="tcp syn no-FP" connection-state=established,related \
protocol=tcp tcp-flags=syn

fyi.

Since version 6.39, mangle rules it not needed anymore.

volkirik · Tue Mar 24, 2020 8:47 am

Since version 6.39, mangle rules it not needed anymore.

But It doesnt work with fasttrack. Weird no mention of tcp-mss-clamping in the fasttrack wiki

The deal with change mss

The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: RE: Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: RE: Re: The deal with change mss

Re: RE: Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: RE: Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Re: The deal with change mss

Who is online