Basically, don't harass ICMP and all will be forgiven. ICMP messages are sent back to hosts when a message is to big and has the DF bit set. Those have to be honored and sent back with a smaller size. You have to allow that to be sent back out to the sender (typically not a problem) and the sender has to owner that (has been known to cause problems because they drop or don't treat the message correctly).
In IPv6 with fragmentation being assigned to the host in a fashion similar to IPv4 with DF bit set you have to be equally careful there.
While IPv4 fragmentation does put a load on a router it simply isn't the same as it used to be. Everything has bigger stronger CPUs now. I haven't seen a router resource issue due to fragmentation in the networks I've come into contact with. That said, it's a balancing act. As an example, you have a typical corporate LAN and a few remote sites connected via some kind of VPN or tunnel. The MTU drop from the VPN or tunnel will cause the router to fragment any packet destined for the corporate LAN. You could in theory adjust the PCs MTU at the remote site to fit inside the tunnel. This would reduce the load on the router from a fragment perspective but will still require 2 or more packets to be sent when they are larger than the real MTU but smaller than say 1500. So it's a balancing act at least in IPv4 land.
Typically, you'll see people leave client devices at 1500. A big area of benefit may be a security camera for example. Setting it's MTU to 1400 instead of 1500 may drastically reduce fragmentation because of it's normal bandwidth consumption and it's easier to manage as you're less likely to have a bunch of those on the LAN compared to PCs or laptops.
Long story short, rip up any book or guidance that tells you to block ICMP in it's entirety. If you want to be choosy and block echo-request / reply and drop time-exceeded's then you can. I leave them on because if I'm hosting a service on a server (say a web-site or DNS) it's going to be discovered by a rudimentary port-scan right along with ICMP. My residential modem get's battered with SSH attempts all day everyday for example.
Now, if you're still having problems. That's where TCP MSS clamping comes in. It's a mechanism where the router adjusts any TCP MSS portion of a TCP datagram down to a size that will fit across the link (usually a tunnel). This in theory should be completely unnecessary but because of ICMP being mistreated for the last several years sometimes it can alleviate the pain for TCP traffic. It will do nothing for UDP or other protocols that need fragmentation.
It sounds like you have a grasp on MTU but here is a post just in case with some screenshots:
viewtopic.php?f=2&t=121200#p596498
Bottom part has some MTU ramblings:
viewtopic.php?f=2&t=121318#p596676