PCC setup for 3 WAN/PPoE (adsl) connections:

First set up route rules:

/ip route rule
add action=lookup comment="" disabled=no routing-mark=to_wlan1 table=to_wlan1
add action=lookup comment="" disabled=no routing-mark=to_wlan2 table=to_wlan2
add action=lookup comment="" disabled=no routing-mark=to_wlan3 table=to_wlan3
add action=lookup comment="" disabled=no dst-address=0.0.0.0/0 src-address=0.0.0.0/0 table=main

second, set up routing table with policy routing (routing marks needed from mangle)

/ip route
add comment="PRIMARY route for \"to_wlan1\" marked traffic (>ADSL1)" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_wlan1 scope=30 target-scope=10
add check-gateway=ping comment="1st Backup route for \"to_wlan1\" marked traffic (>ADSL2)" disabled=no distance=2 dst-address=0.0.0.0/0 gateway=pppoe-out2 routing-mark=to_wlan1 scope=30 target-scope=10
add check-gateway=ping comment="2nd Backup route for \"to_wlan1\" marked traffic (>ADSL3)" disabled=no distance=3 dst-address=0.0.0.0/0 gateway=pppoe-out3 routing-mark=to_wlan1 scope=30 target-scope=10
add check-gateway=ping comment="PRIMARY route for \"to_wlan2\" marked traffic (>ADSL2)" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=pppoe-out2 routing-mark=to_wlan2 scope=30 target-scope=10
add check-gateway=ping comment="1st Backup route for \"to_wlan2\" marked traffic (>ADSL1)" disabled=no distance=2 dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_wlan2 scope=30 target-scope=10
add check-gateway=ping comment="2nd Backup route for \"to_wlan2\" marked traffic (>ADSL3)" disabled=no distance=3 dst-address=0.0.0.0/0 gateway=pppoe-out3 routing-mark=to_wlan2 scope=30 target-scope=10
add comment="PRIMARY route for \"to_wlan3\" marked traffic (>ADSL3)" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=pppoe-out3 routing-mark=to_wlan3 scope=30 target-scope=10
add comment="1st Backup route for \"to_wlan3\" marked traffic (>ADSL2)" disabled=no distance=2 dst-address=0.0.0.0/0 gateway=pppoe-out2 routing-mark=to_wlan3 scope=30 target-scope=10
add comment="2nd Backup route for \"to_wlan3\" marked traffic (>ADSL1)" disabled=no distance=3 dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_wlan3 scope=30 target-scope=10

Third, set up mangle:
A: Incoming traffic from WAN to the router needs to know what return gateway to use:

/ip firewall mangle
add action=mark-connection chain=input comment="Traffic fr router via interface WAN1 get conn mark \"wlan1_conn\"" disabled=no in-interface=WAN1 new-connection-mark=wlan1_conn passthrough=yes
add action=mark-connection chain=input comment="Traffic fr router via interface WAN2 get conn mark \"wlan2_conn\"" disabled=no in-interface=WAN2 new-connection-mark=wlan2_conn passthrough=yes
add action=mark-connection chain=input comment="Traffic fr router via interface WAN3 get conn mark \"wlan3_conn\"" disabled=no in-interface=WAN3 new-connection-mark=wlan3_conn passthrough=yes

B: Traffic leaving router with routing mark from incoming WAN interface get routing mark to route to that same interface again:

/ip firewall mangle
add action=mark-routing chain=output comment="" connection-mark=wlan1_conn disabled=no new-routing-mark=to_wlan1 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wlan2_conn disabled=no new-routing-mark=to_wlan2 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wlan3_conn disabled=no new-routing-mark=to_wlan3 passthrough=yes

C: PCC routing marker:

/ip firewall mangle
add action=mark-routing chain=prerouting comment="NEW! PCC Both 3/0" disabled=no dst-address-type=!local in-interface=Local new-routing-mark=to_wlan1 passthrough=yes per-connection-classifier=src-address:3/0
add action=mark-routing chain=prerouting comment="NEW! PCC Both 3/1" disabled=no dst-address-type=!local in-interface=Local new-routing-mark=to_wlan2 passthrough=yes per-connection-classifier=src-address:3/1
add action=mark-routing chain=prerouting comment="NEW! PCC Both 3/2" disabled=no dst-address-type=!local in-interface=Local new-routing-mark=to_wlan3 passthrough=yes per-connection-classifier=src-address:3/2

[I use routing marker since this router is also QoS and in forward chain is making connection marks followed by package markers. You can mark package only once. This way they don't interfere. (You can mark twice, but only last one then is valid)]

Fourth, set up NAT:

/ip firewall nat
add action=masquerade chain=srcnat comment="Traffic leaving WAN1 port (ADSL1) get WAN1 port IP address" disabled=no out-interface=pppoe-out1
add action=masquerade chain=srcnat comment="Traffic leaving WAN2 port (ADSL2) get WAN2 port IP address" disabled=no out-interface=pppoe-out2
add action=masquerade chain=srcnat comment="Traffic leaving WAN3 port (ADSL3) get WAN3 port IP address" disabled=no out-interface=pppoe-out3



One important remark I have to make here that has not so much to do with PCC but more with the PPoE interfaces (it took me days to find out!):

When the PPoe interface gets authenticated it dynamically places some MSS change rules in the forward chain of mangle. If you now at the same time already have static rules in forward chain build for proper working of router (Qos for instance) then these MSS change filter are not getting any traffic since proper mangle setup already arranged package to leave the router in previous static filters. This will make your interent connection very erratic and poor. So basically I copied these dynamic rules into static ones and put them in fron to all other forward chain filters. Now internet connectivity over PoE-adsl works fine.

Comments and or questions please!

If some guru's can agree or discuss on this setup, or after alteration, I might add this one to the WiKi manual for reference.

WirelessRudy

great job !
but some questions please..
now you have real loadbalancing ? if you are using "internet down load manager" you have real bandwidth for all wan ?

test this link from microsoft server to test you'r result .. give us snapshoot for interface + downloadmanager .
http://download.microsoft.com/download/ ... e_spcd.iso

regards

No, no real load balancing. Since I use src-IP for classifier in the PCC it basically means router groups connections by their source IP. Meaning a connection, even multiple, made by one srce-IP all get same PCC result and thus same GW. It is even so that same IP always get same PCC result, so always same GW.
The load balancing can only be done in respect of many users with different IP's. The more users, the better the average balancing of their speeds is achieved.
But basically by using src-IP classifier only you might as well put each client IP in a manual mangle rule and spread them over your WAN's.
So, the advantage of this type of PCC is no broken links and an automated client IP spread over the available gateway's.
Disadvantage is that per client no load balance is achieved and it is hard to see which client is actually getting what result (and thus which GW). For troubleshooting upon client complaints it is sometimes need to know what GW his IP is using. If same router is also performing src-nat into the WAN IP's it becomes hard to find out...

It is possible with PCC to arrange better load-balancing, but then you have to start using srce-IP+Port as classifier and even dst-IP+Port.
The problem now is that many authentication sessions, like bank log in, web-mail, don't work anymore.

Up to 2 years ago I worked with a 7-WAN load balancing from Peplink (http://www.peplink.com) which did do a better job in load balancing for a single machine (it is a dedicated multiple WAN load balancing with many configurable options) but also with this device it was impossible to create a system that would 100% eliminate the ´broken-link´ issue. Some sites have such high security settings that it needs to have all user request to come from one and the same IP, even if the pages are send to the client from different servers.

My clients are all assigned less speed then the capacity of each WAN connection I have so there is no real need to have their traffic spread over many WAN connections to get higher speeds.
But if you really are looking for a solution to use the combined download capacity from several WAN connections into one machine (or any machine on your LAN) then a peplink is definitely a good purchase. The GUI is much more simple the ROS and they have a good help desk.
(Mine is for sale, 50% of the new-price + shipping and it can be yours...

why do you need routing rules?

add action=lookup comment="" disabled=no dst-address=0.0.0.0/0 src-address=0.0.0.0/0 table=main

this rule is a bit senseless, because it mimics default behaviour

why do you need routing rules?

Well, remove them and my policy routing doesn't work anymore....

add action=lookup comment="" disabled=no dst-address=0.0.0.0/0 src-address=0.0.0.0/0 table=main

this rule is a bit senseless, because it mimics default behaviour

Well, this is basic config in every ROS router for default routing. Without it default route is not working properly under all conditions.

The first one I tested. If I disable inmediately my policy routes stop sent traffic to gven GW. Default route takes over. It's even important they are on top of the last general rule. Otherwise they don't get the traffic and all traffic still goes over default route.
If I don't set general main table rule routing works for simple setups but the moment I have more routes I get all weird things and units becoming un-accessible. Sometimes in a simple CPE I forget to set the default route and it still works. But I forgot it a couple of times to set on routers performing more tasks, like route table with routes back into the network for remote routes then I have all kinds of problems if the default route is not there...

yep, your last routing rule breaks default setup

so, your possible problem is that you forget to add default route, but adds 'default' routing rule?.. O_o

A question
Why do you not insert basic route with a distance bigger? If something is not mark it will not work. I my opinion you must insert: And can you send me the other configuration of the QOS? and export file?

Thanks you
Rafael Lore

A question
Why do you not insert basic route with a distance bigger? If something is not mark it will not work. I my opinion you must insert:

Code: Select all

add dst-address=0.0.0.0/0 gateway=pppoe-out1 distance=4 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=pppoe-out2 distance=5 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=pppoe-out3 distance=6 check-gateway=ping

And can you send me the other configuration of the QOS? and export file?

Thanks you
Rafael Lore

I see it is missing in my example but that is exactly what I do.

What "other configuration of the QOS" do you mean? And which export file?

I have been very busy lately, hence not a lot feedback to this topic....

Thanks!!
You said:

[I use routing marker since this router is also QoS and in forward chain is making connection marks followed by package markers. You can mark package only once. This way they don't interfere. (You can mark twice, but only last one then is valid)]

I ask about it! how do you do this QOS
Thanks again
Rafael Lore