Is it L2TP/IPSec, I assume? The problem with IPSec is that it changes how it operates depending on presence of NAT. Other protocols like SSTP or OpenVPN don't care, they need only one port and you can have as many NATs on client or server side, change ports, anything really, and they will still work. IPSec is different. With public address, it uses ESP packets. But when there's NAT, it switches to UDP (NAT-T mode), so with public address directly on RB you tested one thing, but with RB behind NAT it's something different.
I don't have have any L2TP/IPSec server behind NAT, but if I remember correctly, Windows had some problem with this config. I think this should be the solution:
https://www.qnap.com/en/how-to/knowledg ... n-servers/