PCC for 4 unequal WAN links and FTP connection problem !!!

sadeghnakhjavani · Wed Jan 08, 2014 12:22 pm

Hi,
I have a problem in PCC load balancing for 4 unequal WAN links.
We have 4 WAN links from different ISPs.
ISP1 and ISP2 links are 4Mbps links, ISP3 is a 2Mbps link and ISP4 is a 1Mbps link.
The problem is FTP traffic.
I want to know how can I force for example just FTP traffic (Dst.Port=21) to go through one of the WAN links ?
This is my PCC Config :

/interface
set name=LAN numbers=0
set name=ISP1 numbers=1
set name=ISP2 numbers=2
set name=ISP3 numbers=3
set name=ISP4 numbers=4

/ip address
add address=192.168.1.1/24 network=192.168.1.0 broadcast=192.168.1.255 interface=LAN
add address=101.101.101.1/24 network=101.101.101.0 broadcast=101.101.101.255 interface=ISP1
add address=102.102.102.1/24 network=102.102.102.0 broadcast=102.102.102.255 interface=ISP2
add address=103.103.103.1/24 network=103.103.103.0 broadcast=103.103.103.255 interface=ISP3
add address=104.104.104.1/24 network=104.104.104.0 broadcast=104.104.104.255 interface=ISP4

/ip firewall mangle
add chain=input in-interface=ISP1 action=mark-connection new-connection-mark=ISP1_conn
add chain=input in-interface=ISP2 action=mark-connection new-connection-mark=ISP2_conn
add chain=input in-interface=ISP3 action=mark-connection new-connection-mark=ISP3_conn
add chain=input in-interface=ISP4 action=mark-connection new-connection-mark=ISP4_conn
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2
add chain=output connection-mark=ISP3_conn action=mark-routing new-routing-mark=to_ISP3
add chain=output connection-mark=ISP4_conn action=mark-routing new-routing-mark=to_ISP4

add chain=prerouting in-interface=LAN dst-address=101.101.101.0/24 action=accept
add chain=prerouting in-interface=LAN dst-address=102.102.102.0/24 action=accept
add chain=prerouting in-interface=LAN dst-address=103.103.103.0/24 action=accept
add chain=prerouting in-interface=LAN dst-address=104.104.104.0/24 action=accept

add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/1 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/1 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/1 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/1 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/2 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/2 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:4/3 action=mark-connection new-connection-mark=ISP4_conn passthrough=yes

add chain=prerouting connection-mark=ISP1_conn in-interface=LAN action=mark-routing new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=LAN action=mark-routing new-routing-mark=to_ISP2
add chain=prerouting connection-mark=ISP3_conn in-interface=LAN action=mark-routing new-routing-mark=to_ISP3
add chain=prerouting connection-mark=ISP4_conn in-interface=LAN action=mark-routing new-routing-mark=to_ISP4

/ip route
add dst-address=0.0.0.0/0 routing-mark=to_ISP1 gateway=101.101.101.254 check-gateway=ping
add dst-address=0.0.0.0/0 routing-mark=to_ISP2 gateway=102.102.102.254 check-gateway=ping
add dst-address=0.0.0.0/0 routing-mark=to_ISP3 gateway=103.103.103.254 check-gateway=ping
add dst-address=0.0.0.0/0 routing-mark=to_ISP4 gateway=104.104.104.254 check-gateway=ping

add dst-address=0.0.0.0/0 gateway=101.101.101.254 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=102.102.102.254 distance=2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=103.103.103.254 distance=3 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=104.104.104.254 distance=4 check-gateway=ping

/ip firewall nat
add chain=srcnat out-interface=ISP1 action=masquerade
add chain=srcnat out-interface=ISP2 action=masquerade
add chain=srcnat out-interface=ISP3 action=masquerade
add chain=srcnat out-interface=ISP4 action=masquerade

zap71 · Wed Jan 08, 2014 1:25 pm

I would try to add a firewall rule like this:

/ip firewall mangle
add action=mark-routing chain=prerouting comment=\
    "mark ftp connections" dst-port=21 \
    new-routing-mark=mark-ftp passthrough=no

and then add a routing entry accordingly:

/ip route
add distance=1 gateway=ISP1 routing-mark=mark-ftp

I did not test this code, so be careful. But I hope you get the general idea.

Zap.

Rudios · Wed Jan 08, 2014 3:00 pm

I suggest to also change the PCC rules.
I see mulitple 4/0 entries and I think this won't work.
To spread it more properly add all the ISP speeds together, makes 11, and create multiple marking rules for the right amount of connetions.
See below

ros code

add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/1 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/2 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/3 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/4 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/5 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/6 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/7 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/8 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/9 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/10 action=mark-connection new-connection-mark=ISP4_conn passthrough=yes

For your FTP problem I agree with zap71.
Create a mangle rule, just before the PCC rules to give the FTP connection a dedicated connection mark to get it routed.

sadeghnakhjavani · Wed Jan 08, 2014 8:38 pm

I would try to add a firewall rule like this:
Code: Select all
/ip firewall mangle
add action=mark-routing chain=prerouting comment=\
    "mark ftp connections" dst-port=21 \
    new-routing-mark=mark-ftp passthrough=no
and then add a routing entry accordingly:
Code: Select all
/ip route
add distance=1 gateway=ISP1 routing-mark=mark-ftp
I did not test this code, so be careful. But I hope you get the general idea.

Zap.

OK.TNX Zap
But destination port must be 21 or 21 and 20 ?
And if I want ISP2 be failover for ISP1 just for FTP connections what will be the code?

sadeghnakhjavani · Thu Jan 09, 2014 9:17 am

I suggest to also change the PCC rules.
I see mulitple 4/0 entries and I think this won't work.
To spread it more properly add all the ISP speeds together, makes 11, and create multiple marking rules for the right amount of connetions.
See below

ros code

add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/1 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/2 action=mark-connection ne[quote="Rudios"]I suggest to also change the PCC rules.
I see mulitple 4/0 entries and I think this won't work.
To spread it more properly add all the ISP speeds together, makes 11, and create multiple marking rules for the right amount of connetions.
See below
[code2=ros]add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/1 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/2 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/3 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/4 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/5 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/6 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/7 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/8 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/9 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/10 action=mark-connection new-connection-mark=ISP4_conn passthrough=yes

For your FTP problem I agree with zap71.
Create a mangle rule, just before the PCC rules to give the FTP connection a dedicated connection mark to get it routed.

I suggest to also change the PCC rules.
I see mulitple 4/0 entries and I think this won't work.
To spread it more properly add all the ISP speeds together, makes 11, and create multiple marking rules for the right amount of connetions.
See below

ros code

add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/0 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/1 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/2 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/3 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/4 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/5 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/6 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/7 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/8 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/9 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/10 action=mark-connection new-connection-mark=ISP4_conn passthrough=yes

For your FTP problem I agree with zap71.
Create a mangle rule, just before the PCC rules to give the FTP connection a dedicated connection mark to get it routed.

w-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/3 action=mark-connection new-connection-mark=ISP1_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/4 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/5 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/6 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/7 action=mark-connection new-connection-mark=ISP2_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/8 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/9 action=mark-connection new-connection-mark=ISP3_conn passthrough=yes
add chain=prerouting dst-address-type=!LAN in-interface=LAN per-connection-classifier=both-addresses:11/10 action=mark-connection new-connection-mark=ISP4_conn passthrough=yes[/code2]

For your FTP problem I agree with zap71.
Create a mangle rule, just before the PCC rules to give the FTP connection a dedicated connection mark to get it routed.[/quote]

TNX, But which one is better?
both-addresses or both-addresses-and-ports in PCC ?

Rudios · Thu Jan 09, 2014 2:15 pm

...
TNX, But which one is better?
both-addresses or both-addresses-and-ports in PCC ?

I have read multiple times that both-addresses-and-ports is more likely to give have issues when using sites swapping between http and https, eg homebanking.
I would go for both-addresses

sadeghnakhjavani · Mon Jan 13, 2014 12:49 pm

I would try to add a firewall rule like this:
Code: Select all
/ip firewall mangle
add action=mark-routing chain=prerouting comment=\
    "mark ftp connections" dst-port=21 \
    new-routing-mark=mark-ftp passthrough=no
and then add a routing entry accordingly:
Code: Select all
/ip route
add distance=1 gateway=ISP1 routing-mark=mark-ftp
I did not test this code, so be careful. But I hope you get the general idea.

Zap.

You miss protocol parameter in your config. I test this config and it's not work for me. I put it above other rules.

add chain=prerouting action=mark-routing dst-port=21 protocol=tcp new-routing-mark=mark-ftp passthrough=no comment="mark ftp connections"

and add a routing entry accordingly :

add distance=1 gateway=ISP1 routing-mark=mark-ftp

After these settings, FTP connections didn't establish.