Community discussions

MikroTik App
  4. RouterOS
  5. General

Load Balance Problems -- with screenshot

Post Reply
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Load Balance Problems -- with screenshot

Tue Dec 01, 2015 11:13 am

Dear all
good day

as i test our load balance today, there is a problem that it doesn't divide the load equally between the connections as in the image below:

Image


the firewall mangles
Code: Select all
 0  D chain=forward action=change-mss new-mss=1440 passthrough=yes tcp-flags=syn 
      protocol=tcp out-interface=all-ppp tcp-mss=1441-65535 log=no 
      log-prefix="" 

 1  D chain=forward action=change-mss new-mss=1420 passthrough=yes tcp-flags=syn 
      protocol=tcp in-interface=all-ppp tcp-mss=1421-65535 log=no log-prefix="" 

 2    chain=prerouting action=accept src-address-list=LAN dst-address-list=LAN 
      log=no log-prefix="" 

 3    chain=forward action=mark-connection new-connection-mark=ISP1_conn 
      passthrough=no in-interface=pppoe-out1 EL connection-mark=no-mark log=no 
      log-prefix="" 

 4    chain=forward action=mark-connection new-connection-mark=ISP2_conn 
      passthrough=no in-interface=pppoe-out2 EL connection-mark=no-mark log=no 
      log-prefix="" 

 5    chain=forward action=mark-connection new-connection-mark=ISP3_conn 
      passthrough=no in-interface=pppoe-out3 Or connection-mark=no-mark log=no 
      log-prefix="" 

 6    chain=forward action=mark-connection new-connection-mark=ISP4_conn 
      passthrough=no in-interface=pppoe-out4 Or connection-mark=no-mark log=no 
      log-prefix="" 

 7    chain=prerouting action=mark-connection new-connection-mark=ISP1_conn 
      passthrough=yes in-interface=pppoe-out1 EL connection-mark=no-mark log=no 
      log-prefix="" 

 8    chain=prerouting action=mark-connection new-connection-mark=ISP2_conn 
      passthrough=yes in-interface=pppoe-out2 EL connection-mark=no-mark log=no 
      log-prefix="" 

 9    chain=prerouting action=mark-connection new-connection-mark=ISP3_conn 
      passthrough=yes in-interface=pppoe-out3 Or connection-mark=no-mark log=no 
      log-prefix="" 

10    chain=prerouting action=mark-connection new-connection-mark=ISP4_conn 
      passthrough=yes in-interface=pppoe-out4 Or connection-mark=no-mark log=no 
      log-prefix="" 

11    chain=prerouting action=jump jump-target=policy_routing 
      in-interface=LAN ether10 connection-mark=no-mark log=no log-prefix="" 

12    chain=prerouting action=mark-routing new-routing-mark=ISP1_traffic 
      passthrough=yes src-address-list=LAN connection-mark=ISP1_conn log=no 
      log-prefix="" 

13    chain=prerouting action=mark-routing new-routing-mark=ISP2_traffic 
      passthrough=yes src-address-list=LAN connection-mark=ISP2_conn log=no 
      log-prefix="" 

14    chain=prerouting action=mark-routing new-routing-mark=ISP3_traffic 
      passthrough=yes src-address-list=LAN connection-mark=ISP3_conn log=no 
      log-prefix="" 

15    chain=prerouting action=mark-routing new-routing-mark=ISP4_traffic 
      passthrough=yes src-address-list=LAN connection-mark=ISP4_conn log=no 
      log-prefix="" 

16    chain=output action=mark-routing new-routing-mark=ISP1_traffic 
      passthrough=yes connection-mark=ISP1_conn log=no log-prefix="" 

17    chain=output action=mark-routing new-routing-mark=ISP2_traffic 
      passthrough=yes connection-mark=ISP2_conn log=no log-prefix="" 

18    chain=output action=mark-routing new-routing-mark=ISP3_traffic 
      passthrough=yes connection-mark=ISP3_conn log=no log-prefix="" 

19    chain=output action=mark-routing new-routing-mark=ISP4_traffic 
      passthrough=yes connection-mark=ISP4_conn log=no log-prefix="" 

20    chain=policy_routing action=mark-connection new-connection-mark=ISP1_conn 
      passthrough=yes dst-address-type=!local 
      per-connection-classifier=both-addresses:4/0 log=no log-prefix="" 

21    chain=policy_routing action=mark-connection new-connection-mark=ISP2_conn 
      passthrough=yes dst-address-type=!local 
      per-connection-classifier=both-addresses:4/1 log=no log-prefix="" 

22    chain=policy_routing action=mark-connection new-connection-mark=ISP3_conn 
      passthrough=yes dst-address-type=!local 
      per-connection-classifier=both-addresses:4/2 log=no log-prefix="" 

23    chain=policy_routing action=mark-connection new-connection-mark=ISP4_conn 
      passthrough=yes dst-address-type=!local 
      per-connection-classifier=both-addresses:4/3 log=no log-prefix=""
the Firewall nat is
Code: Select all
 0    chain=srcnat action=masquerade out-interface=pppoe-out1 EL log=no 
      log-prefix="" 

 1    chain=srcnat action=masquerade out-interface=pppoe-out2 EL log=no 
      log-prefix="" 

 2    chain=srcnat action=masquerade out-interface=pppoe-out3 Or log=no 
      log-prefix="" 

 3    chain=srcnat action=masquerade out-interface=pppoe-out4 Or log=no 
      log-prefix=""

ip route
Code: Select all
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          pppoe-out1 EL             1
 1 A S  0.0.0.0/0                          pppoe-out2 EL             1
 2 A S  0.0.0.0/0                          pppoe-out3 Or             1
 3 A S  0.0.0.0/0                          pppoe-out4 Or             1
 4 A S  0.0.0.0/0                          pppoe-out1 EL             7
 5   S  0.0.0.0/0                          pppoe-out2 EL             8
 6   S  0.0.0.0/0                          pppoe-out3 Or             9
 7   S  0.0.0.0/0                          pppoe-out4 Or            10
 8 ADS  0.0.0.0/0                          10.10.10.1                0
 9  DS  0.0.0.0/0                          10.10.10.1                0
10  DS  0.0.0.0/0                          10.10.10.1                0
11  DS  0.0.0.0/0                          10.10.10.1                0
12   S  0.0.0.0/0                          pppoe-out1 EL             2
13   S  0.0.0.0/0                          pppoe-out2 EL             3
14   S  0.0.0.0/0                          pppoe-out3 Or             4
15   S  0.0.0.0/0                          pppoe-out4 Or             5
16 ADC  10.10.10.1/32      10.204.19.243   pppoe-out3 Or             0
                                           pppoe-out2 EL     
                                           pppoe-out4 Or

any help please??
Top
 
UMarcus
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Wed Jan 21, 2015 10:11 am
Location: Europe

Re: Load Balance Problems -- with screenshot

Tue Dec 01, 2015 12:20 pm

Hi,
from my point of view it looks good. You have traffic on all 4 WAN's.

Two Point's to keep in mind :

1. With PPC it is not gurant that the connections (Source IP/Port <> Target IP/Port) uniformly distributed, well the uniformly distribution will be increase with count of connections.

2. At least traffic amount per WAN will also impact by traffic of connections assigned to that WAN. (For e.g. one user downloads 50Mb with an connection and another user downloads 10Mb with another connection and this connections again assigned to one of the WAN)

Regards
Marcus
Last edited by UMarcus on Tue Dec 01, 2015 1:22 pm, edited 1 time in total.
Top
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Re: Load Balance Problems -- with screenshot

Tue Dec 01, 2015 1:18 pm

Hi,
from my point of view it looks good. You have traffic on all 4 WAN's.

Two Point's to keep in mind :

1. With PPC it if not gurant that the connections (Source IP/Port <> Target IP/Port) uniformly distributed, well the uniformly distribution will be increase with count of connections.

2. At least traffic amount per WAN will also impact by traffic of connections assigned to that WAN. (For e.g. one user downloads 50Mb with an connection and another user downloads 10Mb with another connection and this connections again assigned to one of the WAN)

Regards
Marcus

thanks for your reply

about point two i'm sure you're right, but what makes me confused is that the traffic on each WAN is nearby double the other

note: pppoe-out1 EL, pppoe-out2 EL from one ISP
pppoe-out3 Or, pppoe-out4 Or from another different ISP


check the traffic below

Image

every connection is double the other from the same isp!!

thanks in advance
Top
 
UMarcus
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Wed Jan 21, 2015 10:11 am
Location: Europe

Re: Load Balance Problems -- with screenshot

Tue Dec 01, 2015 1:39 pm

Hi,
i'm not study your configuration in detail, but i guess this values could get by chance.

For a simple check you could take a look to the Connections List in the Firewall Dialog. There are the actual connections listed with connection mak.

Also the actual flow of your screenshot not confirm that 2:1 ratio.

Regards
Marcus
Top
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Re: Load Balance Problems -- with screenshot

Tue Dec 01, 2015 1:46 pm

Hi,
i'm not study your configuration in detail, but i guess this values could get by chance.

For a simple check you could take a look to the Connections List in the Firewall Dialog. There are the actual connections listed with connection mak.

Also the actual flow of your screenshot not confirm that 2:1 ratio.

Regards
Marcus

how can we be sure of the configuration please?
Top
 
UMarcus
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Wed Jan 21, 2015 10:11 am
Location: Europe

Re: Load Balance Problems -- with screenshot

Tue Dec 01, 2015 2:19 pm

Did you check the connection list and connection marks ?
You could also observe amount of packets / byte usage of the mangle entries in the firewall mangle table... mabe reset counters...

Make a internet speedtests from PC and observer your realtime traffic on the WAN ports.

Did you look into to wiki http://wiki.mikrotik.com/wiki/Manual:PCC

I did not see any routing marks in you Route Table.. may be post an export or crosscheck that.

regards
Marcus
Top
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Re: Load Balance Problems -- with screenshot

Wed Dec 02, 2015 12:35 pm

Did you check the connection list and connection marks ?
You could also observe amount of packets / byte usage of the mangle entries in the firewall mangle table... mabe reset counters...

Make a internet speedtests from PC and observer your realtime traffic on the WAN ports.

Did you look into to wiki http://wiki.mikrotik.com/wiki/Manual:PCC

I did not see any routing marks in you Route Table.. may be post an export or crosscheck that.

regards
Marcus

Hi Marcus, i did a speedtest and it get the speed of one connection as you described before that the load balance divide the clients between the 4 connections but still not divide the traffic equally

this is what i add in the ip route as the tutorial said
Code: Select all
/ip route
add check-gateway=arp distance=1 gateway=pppoe-out1 routing-mark=ISP1_traffic
add check-gateway=arp distance=1 gateway=pppoe-out2 routing-mark=ISP2_traffic
add check-gateway=arp distance=1 gateway=pppoe-out3 routing-mark=ISP3_traffic
add check-gateway=arp distance=1 gateway=pppoe-out4 routing-mark=ISP4_traffic
add check-gateway=arp distance=2 gateway=pppoe-out1
add check-gateway=arp distance=3 gateway=pppoe-out2
add check-gateway=arp distance=4 gateway=pppoe-out3
add check-gateway=arp distance=5 gateway=pppoe-out4
Top
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Re: Load Balance Problems -- with screenshot

Wed Dec 02, 2015 12:40 pm

this is the whole configuration
Code: Select all
/ip firewall mangle
add chain=prerouting dst-address-list=LAN src-address-list=LAN

/ip firewall mangle
add action=mark-connection chain=forward connection-mark=no-mark \ in-interface=pppoe-out1 new-connection-mark=ISP1_conn passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark \ in-interface=pppoe-out2 new-connection-mark=ISP2_conn passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark \ in-interface=pppoe-out3 new-connection-mark=ISP3_conn passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark \ in-interface=pppoe-out4 new-connection-mark=ISP4_conn passthrough=no

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \ in-interface=pppoe-out1 new-connection-mark=ISP1_conn
add action=mark-connection chain=prerouting connection-mark=no-mark \ in-interface=pppoe-out2 new-connection-mark=ISP2_conn
add action=mark-connection chain=prerouting connection-mark=no-mark \ in-interface=pppoe-out3 new-connection-mark=ISP3_conn
add action=mark-connection chain=prerouting connection-mark=no-mark \ in-interface=pppoe-out4 new-connection-mark=ISP4_conn

/ip firewall mangle
add action=jump chain=prerouting connection-mark=no-mark in-interface=LAN \ jump-target=policy_routing

/ip firewall mangle
add action=mark-routing chain=prerouting connection-mark=ISP1_conn \ new-routing-mark=ISP1_traffic src-address-list=LAN
add action=mark-routing chain=prerouting connection-mark=ISP2_conn \ new-routing-mark=ISP2_traffic src-address-list=LAN
add action=mark-routing chain=prerouting connection-mark=ISP3_conn \ new-routing-mark=ISP3_traffic src-address-list=LAN
add action=mark-routing chain=prerouting connection-mark=ISP4_conn \ new-routing-mark=ISP4_traffic src-address-list=LAN

/ip firewall mangle
add action=mark-routing chain=output connection-mark=ISP1_conn \ new-routing-mark=ISP1_traffic
add action=mark-routing chain=output connection-mark=ISP2_conn \ new-routing-mark=ISP2_traffic
add action=mark-routing chain=output connection-mark=ISP3_conn \ new-routing-mark=ISP3_traffic
add action=mark-routing chain=output connection-mark=ISP4_conn \ new-routing-mark=ISP4_traffic

/ip firewall mangle
add action=mark-connection chain=policy_routing dst-address-type= !local \ new-connection-mark=ISP1_conn per-connection-classifier=\ both-addresses:4/0
add action=mark-connection chain=policy_routing dst-address-type= !local \ new-connection-mark=ISP2_conn per-connection-classifier=\ both-addresses:4/1
add action=mark-connection chain=policy_routing dst-address-type= !local \ new-connection-mark=ISP3_conn per-connection-classifier=\ both-addresses:4/2
add action=mark-connection chain=policy_routing dst-address-type= !local \ new-connection-mark=ISP4_conn per-connection-classifier=\ both-addresses:4/3

/ip route
add check-gateway=arp distance=1 gateway=pppoe-out1 routing-mark=ISP1_traffic
add check-gateway=arp distance=1 gateway=pppoe-out2 routing-mark=ISP2_traffic
add check-gateway=arp distance=1 gateway=pppoe-out3 routing-mark=ISP3_traffic
add check-gateway=arp distance=1 gateway=pppoe-out4 routing-mark=ISP4_traffic
add check-gateway=arp distance=2 gateway=pppoe-out1
add check-gateway=arp distance=3 gateway=pppoe-out2
add check-gateway=arp distance=4 gateway=pppoe-out3
add check-gateway=arp distance=5 gateway=pppoe-out4

and here i got a problem and couldnt add this rules
Code: Select all
/ip route
add check-gateway=arp disabled=no distance=7 dst-address=0.0.0.0/0 gateway=\ pppoe-out1 routing-mark=HTTPS scope=30 target-scope=10
add check-gateway=arp disabled=no distance=8 dst-address=0.0.0.0/0 gateway=\ pppoe-out2 routing-mark=HTTPS scope=30 target-scope=10
add check-gateway=arp disabled=no distance=9 dst-address=0.0.0.0/0 gateway=\ pppoe-out3 routing-mark=HTTPS scope=30 target-scope=10
add check-gateway=arp disabled=no distance=10 dst-address=0.0.0.0/0 gateway=\ pppoe-out4 routing-mark=HTTPS scope=30 target-scope=10

and these rules i didnt add it
Code: Select all
/ip firewall nat
add action=dst-nat chain=dstnat comment=\ "Port Forward tcp80 to Mail Server 192.168.1.10" dst-port=80 \ in-interface=!LAN protocol=tcp to-addresses=192.168.1.10 to-ports=80
add action=dst-nat chain=dstnat comment=\ "Port Forward tcp443 to Mail Server 192.168.1.10" dst-port=443 \ in-interface=!LAN protocol=tcp to-addresses=192.168.1.10 to-ports=443

/ip firewall mangle
add action=mark-routing chain=prerouting disabled=no dst-port=443 \ new-routing-mark=HTTPS passthrough=no protocol=tcp

please you can find the full tutorial below

http://mum.mikrotik.com/presentations/Z ... schoff.pdf


thanks you very much
Top
 
UMarcus
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Wed Jan 21, 2015 10:11 am
Location: Europe

Re: Load Balance Problems -- with screenshot

Wed Dec 02, 2015 1:26 pm

Did you check the connection list and connection marks ?
You could also observe amount of packets / byte usage of the mangle entries in the firewall mangle table... mabe reset counters...

Make a internet speedtests from PC and observer your realtime traffic on the WAN ports.

Did you look into to wiki http://wiki.mikrotik.com/wiki/Manual:PCC

I did not see any routing marks in you Route Table.. may be post an export or crosscheck that.

regards
Marcus

Hi Marcus, i did a speedtest and it get the speed of one connection as you described before that the load balance divide the clients between the 4 connections but still not divide the traffic equally
...
With Load Balancing it never will divide the traffic, it only will divide the connections.. as you investigate all looks good.. or did you see any problem ?
Top
 
UMarcus
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Wed Jan 21, 2015 10:11 am
Location: Europe

Re: Load Balance Problems -- with screenshot

Wed Dec 02, 2015 1:38 pm

One additional comment :

Check the PPC configuration
Code: Select all
  ValuesToHash ::= both-addresses|both-ports|dst-address-and-port|
  src-address|src-port|both-addresses-and-ports|dst-address|dst-port|src-address-and-port
With 'both-addresses-and-ports' you will get the highest WAN allocation of connections.

Regards
Marcus
Top
 
GeekNerd
newbie
Topic Author
Posts: 44
Joined: Tue Nov 10, 2015 9:37 pm

Re: Load Balance Problems -- with screenshot

Wed Dec 02, 2015 1:44 pm


With Load Balancing it never will divide the traffic, it only will divide the connections.. as you investigate all looks good.. or did you see any problem ?

Hi Marcus,

yes you are right, i just missed the expression, it divide the connections between the WAN, i don't see any problem except one, that what make me confused about the traffic between 4 WAN each two from same ISP, and still need to figure why is that?

the other issue that is there any way make the routeros send me an email to inform me that one or two or my pppoe is disconnected and it didnt connect after few attempts?

and can it check the ping to 8.8.8.8 if it is >200 ms in one of our pppoe WAN inform us with that?, or change the connections from that low speed WAN connection to another one?

i'm really appreciate your support, patience and replies.

many many many thanks in advance.

Regards
GeekNerd
Top
Post Reply

Who is online

Users browsing this forum: anav, gianry, mkx and 34 guests
  4. RouterOS
  5. General