Hello Guys,

I have a mikrotik https://mikrotik.com/product/crs354_48p_4s_2q_rm i use it as Switch wirth RouterOS cus RouterSwitch was not working on it.

I use SFP+ Ports on some servers and i have a dilemma what is the right MTU to be set on the interfaces ?
MTU 9000 and L2 MTU 9000 or MTU 9000 and L2 MTU 10218 ?

Auto Neg is OFF, 10GB, Full Duplex
Loop protect default (I dont know what this do :) )
ARP Is enabled byd efault (I dont know what this do :) )

Thank you.

Hey,

Check your gear first if it has a support for jumbo frames like what you have mentioned and see to it that it was also supported on the client side. On a normal situation you should not touch the default 1500 MTU, unless otherwise you have a special needs like you are enabling jumbo frames for your NAS

When device is used as a switch, the (L2)MTU setting doesn't matter (much). It has to be large enough not to drop large frames on ingress. If L2MTU is set larger, it still won't make passing frames larger.

Where MTU matters is if interface us used for L3 i.e. if interface has IP address set. In this case (if needed) device constructs outgoing frames up to MTU in size (that's traffic originating from device itself or being routed and egress link has lower MTU tgan ingress link in which case packets get fragmented). Clearly MTU can not be larger than L2MTU but it is pretty safe to always keep L2MTU set to whatever hardware can take.

So its much ok to just let it as default 1500 ? even on the server network ?

On the servers we have hp nics 10GB SFP+

I set to 9000 on the switch and even on the servers thinking its much ok and for speed :)

It is fine to configure L2MTU as high as it gets. But for (L3) MTU you have to consider a few things:

• L3 MTU has to be the same in whole subnet or else some members of same subnet won't be able to communicate (peers with smaller MTU will silently drop packets)
• when client and server are in different subnets and any of them (or both) use MTUs larger than the minimum MTU on any of interfaces on intermediate routers, packets get either fragmented (and receiver spends additional resources to re-assemble packets) or get dropped and PMTUD kicks in reducing effective MTU below what sender could do. PMTUD needs (at least partially) working ICMP which is often blocked by network administrators who are overdoing their job.
  For this reason most networks stick to de-facto universal value of 1500 bytes.
• jumbo frames were introduced with event of FDDI networking which brought 100Mbps when 10Mbps LAN was a norm (and ethernet didn't win the technology war yet) . At that time computers were not as fast as these days and generally choked on packet processing. Sticking to (already then) standard 1500 byte MTU would mean that PPS capacity would not be high enough to fully benefit of fast L1. Thus FDDI came with jumbo frames of approx. 4.3kB which reduced packet-handling overhead by factor of almost 3.
  Today NICs offer to process a few functions traditionally done in drivers and those functions are done wirespeed, others are done by drivers on powerful computers. Which means that (apart from reduced packet header overhead) large frames are not really needed in most cases.

Which means that it's sensible to use non-standard MTUs only in some dedicated subnet where vast majority of traffic is not routed.
Or on links (routing subnets) where usual traffic is encapsulated somehow (e.g. IP over PPPoE with VLAN headers encapsulated in MPLS or something like this, but in such cases "baby jumbo frames" with MTUs up to 1600 bytes usually do).