I am trying to connect to the L2TP server of a new client. Spent a few hours fiddling with all kinds of settings without success.
Then I decided to try it from my home router, and it worked instantly.
Double and triple checked all l2tp/ipsec options on both routers (they are same model and same version of ros), and they are the same.

After inspection of l2tp and IPsec logs, I saw that the only difference was that my work router is not behind nat (it has a public ip) so transport is established on port 500, while my home router is behind a nat (only has a private ip) and transport is established on port 4500.

The problem on the work router starts after IPsec-SA is established, and SCCRQ control message is sent to server. The server never replies. While the router at my home gets the reply.
The firewall is not in play as I have allowed all traffic from and to the server.

What can I do to address this issue? It is driving me mad.

I "fixed" it by adding a new router behind my main router, and then configuring the L2TP client on that router.
Anyway, I would appreciate if anyone can shed any light on this subject for me.

I was facing the same issue. You need to allow IPsec ESP on the server you are connecting to:

/ip firewall filter
add chain=input protocol=udp port=500 action=accept comment="Allow IPSec IKE"
add chain=input protocol=udp port=4500 action=accept comment="Allow IPSec NAT-T"
add chain=input protocol=ipsec-esp action=accept comment="Allow IPSec ESP"


If the client that is connecting is behind a NAT, it connects via port 4500. But if you're using a public IP, it will use IPsec ESP, so you need the router that you are connecting to to to allow IPSec ESP.

This needs to be on the server Mikrotik router, not on your own Mikrotik router.