CRS125-24G-1S-2HnD using load balancing with fall over

nachannon · Thu Feb 24, 2022 7:28 pm

I have setup my router using firmware 7.1.2 to use load balancing and fall over currently I have configure the router to use ether1 for my DSL connection and ether2 for my 4G connection and the remaining ports are bridged. currently the setup works as expected if I disable a WAN port eg: either2 everything works fine and all traffic flows though either1 and speed test are correct to what is available on that connection same works if I disable either1 all traffic diverts fine and speed test is what is expected on either2 port and both enabled and the speed is combined from both interfaces as expected.
My problem I am having is if either2 becomes very slow via the lte modem connected to either2 the load balancing become very unstable to the point that either1 also basically comes to a stand still, if I disable either2 then speed becomes stable again on either1, once either two connection becomes stable again from the LTE modem stability is restored to the load balancing, can anyone see in my config files what I am doing wrong?

Firewall Mangle settings

 MikroTik RouterOS 7.1.2 (c) 1999-2022       https://www.mikrotik.com/
[admin@CAPsMAN] > ip firewall
[admin@CAPsMAN] /ip/firewall> mangle
[admin@CAPsMAN] /ip/firewall/mangle> print
Flags: X - disabled, I - invalid; D - dynamic 
 0  D ;;; special dummy rule to show fasttrack counters
      chain=prerouting action=passthrough 

 1  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 

 2  D ;;; special dummy rule to show fasttrack counters
      chain=postrouting action=passthrough 

 3    ;;; Prerouting-accept
      chain=prerouting action=accept dst-address=192.168.1.0/24 
      in-interface=BridgePrivate log=no log-prefix="" 

 4    chain=prerouting action=accept dst-address=192.168.100.0/24 
      in-interface=BridgePublic log=no log-prefix="" 

 5    chain=prerouting action=accept dst-address=192.168.10.0/24 
      in-interface=ether1-WAN1 log=no log-prefix="" 

 6    chain=prerouting action=accept dst-address=192.168.20.0/24 
      in-interface=ether2-4G log=no log-prefix="" 

 7    ;;; Mark Connection input
      chain=input action=mark-connection new-connection-mark=WAN1-Conn 
      passthrough=yes in-interface=ether1-WAN1 log=no log-prefix="" 

 8    chain=input action=mark-connection new-connection-mark=WAN2-Conn 
      passthrough=yes in-interface=ether2-4G log=no log-prefix="" 

 9    ;;; Mark Connection Prerouting
      chain=prerouting action=mark-connection new-connection-mark=WAN1-Conn 
      passthrough=yes in-interface=ether1-WAN1 log=no log-prefix="" 

10    chain=prerouting action=mark-connection new-connection-mark=WAN2-Conn 
      passthrough=yes in-interface=ether2-4G log=no log-prefix="" 

11    ;;; PCC
      chain=prerouting action=mark-connection new-connection-mark=WAN1-Conn 
      passthrough=yes dst-address-type=!local in-interface=BridgePrivate 
      per-connection-classifier=both-addresses:2/0 log=no log-prefix="" 

12    chain=prerouting action=mark-connection new-connection-mark=WAN1-Conn 
      passthrough=yes dst-address-type=!local in-interface=BridgePublic 
      per-connection-classifier=both-addresses:2/0 log=no log-prefix="" 

13    chain=prerouting action=mark-connection new-connection-mark=WAN2-Conn 
      passthrough=yes dst-address-type=!local in-interface=BridgePrivate 
      per-connection-classifier=both-addresses:2/1 log=no log-prefix="" 

14    chain=prerouting action=mark-connection new-connection-mark=WAN2-Conn 
      passthrough=yes dst-address-type=!local in-interface=BridgePublic 
      per-connection-classifier=both-addresses:2/1 log=no log-prefix="" 

15    ;;; Mark Routing Output
      chain=output action=mark-routing new-routing-mark=to-WAN1 passthrough=yes 
      connection-mark=WAN1-Conn log=no log-prefix="" 

16    chain=output action=mark-routing new-routing-mark=to-WAN2 passthrough=yes 
      connection-mark=WAN2-Conn log=no log-prefix="" 

17    ;;; Routing Preouts
      chain=prerouting action=mark-routing new-routing-mark=to-WAN1 
      passthrough=yes connection-mark=WAN1-Conn in-interface=BridgePrivate 
      log=no log-prefix="" 

18    chain=prerouting action=mark-routing new-routing-mark=to-WAN1 
      passthrough=yes connection-mark=WAN1-Conn in-interface=BridgePublic log=no 
      log-prefix="" 

19    chain=prerouting action=mark-routing new-routing-mark=to-WAN2 
      passthrough=yes connection-mark=WAN2-Conn in-interface=BridgePrivate 
      log=no log-prefix="" 

20    chain=prerouting action=mark-routing new-routing-mark=to-WAN2 
      passthrough=yes connection-mark=WAN2-Conn in-interface=BridgePublic log=no 
      log-prefix="" 

21    ;;; NVR PreRouting
      chain=prerouting action=mark-connection new-connection-mark=WAN1-Conn 
      passthrough=no src-address=192.168.1.51 dst-address=!192.168.1.0/24 
      connection-mark=WAN1-Conn log=no log-prefix="" 

[admin@CAPsMAN] /ip/firewall/mangle>

Firewall Filter Rules

 MikroTik RouterOS 7.1.2 (c) 1999-2022       https://www.mikrotik.com/
[admin@CAPsMAN] > ip
[admin@CAPsMAN] /ip> firewall
[admin@CAPsMAN] /ip/firewall> filter
[admin@CAPsMAN] /ip/firewall/filter> print
Flags: X - disabled, I - invalid; D - dynamic 
 0  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 

 1    ;;; defconf: accept established,related,untracked
      chain=input action=accept connection-state=established,related,untracked 

 2    ;;; defconf: fasttrack
      chain=forward action=fasttrack-connection hw-offload=yes 
      connection-state=established,related connection-mark=WAN1-Conn out-interface=ether1-WAN1 
      log=no log-prefix="" 

 3    chain=forward action=fasttrack-connection hw-offload=yes 
      connection-state=established,related connection-mark=!WAN2-Conn out-interface=ether2-4G 
      log=no log-prefix="" 

 4    ;;; defconf: accept established,related, untracked
      chain=forward action=accept connection-state=established,related,untracked 

 5    ;;; defconf: drop invalid
      chain=input action=drop connection-state=invalid log=no log-prefix="" 

 6    ;;; WinBox
      chain=input action=accept protocol=tcp in-interface=ether1-WAN1 dst-port=8291 log=no 
      log-prefix="" 

 7    ;;; Speed Test Server
      chain=output action=accept protocol=tcp out-interface=BridgePrivate dst-port=2000 log=no 
      log-prefix="" 

 8 X  chain=input action=accept protocol=tcp in-interface=ether1-WAN1 dst-port=2000 log=no 
      log-prefix="" 

 9    ;;; defconf: accept ICMP
      chain=input action=accept protocol=icmp 

10 X  ;;; BT Server
      chain=input action=accept protocol=tcp dst-port=2000 log=no log-prefix="" 

11 X  chain=forward action=accept protocol=tcp dst-address=192.168.1.18 dst-port=2055 log=no 
      log-prefix="" 

12 X  chain=forward action=accept protocol=tcp dst-address=192.168.1.18 dst-port=560 log=no 
      log-prefix="" 

13    ;;; defconf: accept to local loopback (for CAPsMAN)
      chain=input action=accept dst-address=127.0.0.1 

14    ;;; defconf: drop all not coming from LAN
      chain=input action=drop in-interface=!BridgePrivate log=no log-prefix="" 

15    ;;; Block all traffic from Guest to Private interface
      chain=forward action=drop connection-state="" in-interface=BridgePublic 
      out-interface=BridgePrivate log=no log-prefix="" 

16    ;;; Allow trafic from private to public interface
      chain=forward action=accept connection-state=established,related 
      in-interface=BridgePrivate out-interface=BridgePublic log=no log-prefix="" 

17    ;;; Accept: HTTP/S
      chain=output action=accept protocol=tcp port=80,443 log=no log-prefix="" 

18    ;;; defconf: accept in ipsec policy
      chain=forward action=accept ipsec-policy=in,ipsec 

19    ;;; defconf: accept out ipsec policy
      chain=forward action=accept ipsec-policy=out,ipsec 

20    ;;; Prevent DNS DDos attack
      chain=input action=drop protocol=udp dst-port=53 log=no log-prefix="" 

21    chain=input action=drop protocol=tcp dst-port=53 log=no log-prefix="" 

22    ;;; defconf: drop invalid
      chain=forward action=drop connection-state=invalid 

23    ;;; defconf: drop all from WAN not DSTNATed
      chain=forward action=drop connection-state=new connection-nat-state=!dstnat 
      in-interface-list=WAN 

[admin@CAPsMAN] /ip/firewall/filter>

Firewall Nat

[admin@CAPsMAN] > ip
[admin@CAPsMAN] /ip> firewall
[admin@CAPsMAN] /ip/firewall> nat
[admin@CAPsMAN] /ip/firewall/nat> print
Flags: X - disabled, I - invalid; D - dynamic 
 0    ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface=ether1-WAN1 log=no log-prefix="" 
      ipsec-policy=out,none 

 1    chain=srcnat action=masquerade out-interface=ether2-4G log=no log-prefix="" 

 2 X  ;;; camera kitchen
      chain=dstnat action=dst-nat to-addresses=192.168.1.18 to-ports=2055 protocol=tcp 
      in-interface=ether1-WAN1 dst-port=2055 log=no log-prefix="" 

 3    chain=dstnat action=accept to-addresses=192.168.1.18 to-ports=2055 protocol=tcp 
      dst-port=2000 log=no log-prefix="" 

 4    chain=input action=accept to-addresses=192.168.1.18 to-ports=2055 protocol=udp 
      dst-port=2000 log=no log-prefix="" 

 5 X  chain=dstnat action=dst-nat to-addresses=192.168.1.18 to-ports=560 protocol=tcp 
      in-interface=ether1-WAN1 dst-port=560 log=no log-prefix=""

IP Routes


[admin@CAPsMAN] > ip route
[admin@CAPsMAN] /ip/route> print
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - COPY
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#     DST-ADDRESS       GATEWAY        DISTANCE
;;; 4G
0   s 0.0.0.0/0         192.168.20.1          2
;;; Vodafone
1  As 0.0.0.0/0         192.168.10.1          1
  DAc 192.168.1.0/24    BridgePrivate         0
  DAc 192.168.10.0/24   ether1-WAN1           0
  DAc 192.168.20.0/24   ether2-4G             0
  DAc 192.168.100.0/24  BridgePublic          0
2  As 0.0.0.0/0         192.168.10.1          1
3  As 0.0.0.0/0         192.168.20.1          2

and DNS settings


[admin@CAPsMAN] /ip> dns
[admin@CAPsMAN] /ip/dns> print
                      servers: 8.8.8.8,8.8.4.4
              dynamic-servers: 
               use-doh-server: 
              verify-doh-cert: no
        allow-remote-requests: yes
          max-udp-packet-size: 4096
         query-server-timeout: 2s
          query-total-timeout: 10s
       max-concurrent-queries: 100
  max-concurrent-tcp-sessions: 20
                   cache-size: 2048KiB
                cache-max-ttl: 1w
                   cache-used: 30KiB

Thanks in advance
Regards
Nigel