Hello,
We have a CCR-1036-12G-4S and we have announced a /31 IPv6 on it. ::1 is on the bridge and VMs on 4 servers use a ton of IPv6 from the /31.

Now, we are facing very slow connect times when connecting to external servers. Without the /31 , we can connect it 1-2 s. While using the /31, it take 20s to do the same job.

Is it possible that the ARP table / IPv6 neighbors table is overloaded and continuously adding/removing entries is causing this slowdown ?

What can be done to fix this ?

First investigate if this problem is indeed occurring.
Overloaded neighbor table can be the result of someone scanning your space.  See if that is happening by doing some packet captures.
You could try putting some firewall rules on the forwarding chain.  At first only allow established,related and all ICMPv6 (2 rules)
and see if that fixes it.  That will block all incoming connections.  If it does, see what extra rules you can add to accept incoming
traffic.