Since it is a border router that does not use NAT or packet or connection marking, could tracking be disabled?
AFAIK if firewall filter config is empty, then connection tracking is omitted. Raw firewall rules don't affect it, they don't rely on connection state.