Hi many tutorials i see for mikrotik that use multiple WAN involve having multiple LAN. I have 2 WANs and a few VPN connections that i want to load balance across using only 1 LAN and i want to load balance based on traffic and QoS.

I have 2 different WANs that are good for different things.
I use the router as a VPN server but have the VPN clients and LAN in the same IP Subnet
I connect to VPN providers using the router.

For example i want to use a VPN provider1 to communicate with my DNS server of choice instead of over WAN1.
I than want to redirect all web traffic through another VPN provider2 over WAN2.
I want to redirect all torrents through VPN provider 3 over WAN2,
I want all game traffic for my game server to use all WANs. DDNS is used for all the IPs so players can connect through a specific WAN.
I want all other traffic to use WAN1 first until it is full and than use WAN2 if WAN1 is full.

Is there a way to perform selective load balance this way? I know QoS is needed but i cant get routerOS to route over multiple routes at the same time. UPNP Is also a problem for this case.

you're looking for policy based routing. Create rules matching the desired traffic patterns, place routing marks and then create routing rules to route specific traffic accross VPN connections. See http://wiki.mikrotik.com/wiki/Policy_Base_Routing.

To use your VPN instead of WAN, you could also choose to add a higher weight to your default WAN route compared to the default VPN route.

To load ballance outgoing traffic across WAN interfaces, you could use PCC, but there are also other methods. See http://wiki.mikrotik.com/wiki/Load_Balancing. Unsure how to acomplish the 'until full' part though.

Thanks
The 2 WANs i have are quite different. One has good latency and more upload, really good for gaming. The other is cable which is bad for upload or latency but totally find for videos, downloading, etc.

I think treating the VPNs each as a WAN would work much better so its more like having 5 WANs and using Policy based routing.

Since WAN1 has lower latency i want to keep it that way. I was looking at using QoS and routing the important stuff first that i want to route through WAN1 and WAN2 specifically. Than for all other traffic i want them to use WAN1 first until WAN1 is maxed out and than use WAN2 for the rest of the other traffic. I think of achieving this by having the forwarding rules for WAN1 using a bandwidht limit under the firewall rules with the forwarding rules for WAN2 catching everything else.

so with the assymmetric WAN connections PCC doesn't work too well.

Still, it's a matter of properly marking your connections and use queues to limit and/or algorithms to spread the traffic accross routes.

For the bandwidth based load ballancing you might want to refer to http://mum.mikrotik.com/presentations/US12/tomas.pdf. It uses the traffic monitor to trigger a script to swap routes by changing the mangle rules. Not tried it myself.

Thanks but i would prefer adding bandwidth limits to QoS/firewall rules relating to their relevant interfaces.

Thanks but i would prefer adding bandwidth limits to QoS/firewall rules relating to their relevant interfaces.

And this would not allow you to do this why exactly?

• mark connections of interest to route across specific interfaces (e.g. DNS, web traffic, torrent connections)
• mark connections of interest with higher or lower priorities and use queue trees to manage limits and priorities
• use traffic monitor to dynamically change mangle rules based on preconfigured limits to scale out to other WAN link.

To me this looks pretty much like what you try to accomplish. The only difficulty I foresee is when you want to dynamically spread VPN connections accross the interfaces (which will ruin policies).

If this is not what you mean, then you'll have to be a little bit more specific, perhaps draw up a diagram for the different scenarios you have in mind.

I do not plan to spread the same VPN across 2 interfaces.

The way i plan to do it is:
WAN1 : 
 -- game server traffic
-- latency sensitive traffic i.e. NTP, gaming etc
--VPN provider that gives static IP + DDOS protection: Use for DNS and small internal web server.
-- video streaming upload (such as to twitch) but can be applied a fixed bandwidth of 2Mb/s up as theres no more than 1 stream
-- VPN traffic that uses the router as VPN server
-- other traffic
WAN 2 : 
-- game server traffic
-- VPN provider 2 for web browsing
-- other traffic
-- VPN traffic that uses the router as VPN server
-- VPN provider 3 for torrents

Both WAN1 and WAN2 connect to different ISPs via PPPOE.
Game server traffic requires no load balancing, Players can pick an IP that is faster to them or pick WAN2 if WAN1 is full (small web server can be used to show that). Other traffic is the only traffic that requires load balancing across WANs with WAN1 being the priority. I sorted things via their priorities.

I noticed that when using VPN the MTUs and MRUs are bigger than 1500 but my PPPOE connection to ISP uses MTU and MRU of 1480. This causes issues when using change TCP MSS which is required for things to work properly so im trying to figure out what MTUs and MRUs should i use for VPN and which TCP MSS value should i use as the default 1440 largest doesnt work when using VPN as a lot of websites dont load or load very slowly but work fine when browsing through WAN PPPOE connection.

I have to use DNS, web browsing and torrents over VPN as my ISP filters them so if i use a local VPN server they dont go through the ISP's hidden proxy. All VPN providers are the same provider only using different servers to spread load and based on location for what is allowed and not allowed. By using a VPN service it effectively bypasses the local filters that can cause issues (if only mikrotik supported DNScrypt as i use openDNS for selective and malware filtering) and that can slow things down. Also openDNS has yet to allow scripting for updating dynamic IPs like the wealth of available mikrotik scripts for DDNS, they insist on using software for PCs which arent kept on all the time so an IP change nullifies their filtering.

What i would like to know is an effective method performing this sort of routing. I've managed to make effective firewall protection using a few simple rules and its surprising that facebook attempts connection to router and client behind router, its like they're probing you.

So main issue is to recognise the traffic. Example: for torrent traffic I have found the P2P profile unreliable (partly because more and more is SSL-based and thus encrypted and hidden from inspection) to consistently route traffic through my VPN without IP-leakage. So I have a src-address based mangle rule from my torrent server.

But something similar to below is working perfect for me.
You could enhance the traffic shaping with queue trees and adding priorities.

The rest of the traffic you can handle through the Thomas' loadbalance script included in the slides I posted above. I had a use case to try and it works as advertised. Just make sure you properly add the VPN connections to the first rule to bypass the LB and mark traffic destined for that VPN before it hits the LB mangle rules.

As you have WAN traffic coming through the VPN connections as well, you want to mark incoming traffic on the VPN connections to make sure they go out the same way. Use the same pattern as provided in the slides for the incoming WAN connections, like

Can anyone assist please?

I've been pulling my hair out with load balancing and fail-over set-up. I simply can't get it to work properly. I'm probably missing something really stupid but I simply can't see it. The worst thing is the system is 'live' and working fine on one connection so I have to be careful not to break things when testing! I really need some pointers as to what I might possibly be doing wrong?

The scenario:
WAN WiFi network using Ubiquity M5's Each client (10.0.X.0/30) is connected on a separate VLAN. 2 different ISP's. DNS set at 8.8.8.8. Routing works fine on either ISP connection until I try to implement load balancing. This appears to be working however as soon as the ISP2 interface is enabled it will after a couple of minutes kill established IPsec and OpenVPN connections that will then not reconnect. Traffic on the ISP2 also seems to be light ~10% in comparison with ISP1 when enabled. If I kill an ISP connection the system switches over to the other with no problem but still have the issue with VPN's. It's like routes are maybe being cached or something like layer 2 MAC address are caching which I've struck before on MySQL of all things? A clue (red herring?) might be that my OpenVPN client on an OMV server refuses to re-establish a VPN connection until rebooted however I can make a connection from a PC on the same network. Flushing the various interfaces doesn't help.

2 VDSL PPPoE connections on the Microtik router running at 50+ Mbs Both ISP connections are up 24 x 7 on individual routers.
600A router FW 2.20 ISP1 on ETH1 LAN (VLAN'S) on ETH2 ISP2 on ETH3 Management on VLAN5

These are the configs I tried: (I haven't submitted the whole capture as I know that pisses me off no end too!-) Ask and you shall receive!

The only difference is I substituted the two PPPoE interfaces for the external IP's in the Masq and interface sections in the following set up's.
http://mikrotikbook.blogspot.co.nz/2014 ... using.html
http://wiki.mikrotik.com/wiki/Manual:PCC
http://wiki.mikrotik.com/wiki/NTH_load_ ... _approach)

Any assistance on this will be greatly appreciated.

I think I had a similar experience using PCC. Unfortunately for you, I have then put my LB plans in the fridge until I decided to give the bandwidth based LB script indicated below a try, which didn't give me any troubles with the VPN. Ergo I don't have a working solution for the PCC scenario to share, but maybe the following will help you getting closer to a fix.

This appears to be working however as soon as the ISP2 interface is enabled it will after a couple of minutes kill established IPsec and OpenVPN connections that will then not reconnect.

While this shouldn't be happening given the !local exceptions, to me this sounds like the PCC-rules are still causing traffic to be routed over to the wrong WAN interface after which the policies and installed keys at the different endpoints won't match. The x-minute delay your experience I imagine either corresponds with the connection timeout OR the key expiration.

Double check your connections tab in the IP firewall to verify the incoming VPN connections are properly marked so they can be matched to the correct WAN routes in your routing policies. Use torch or wireshark to gather a little bit more diagnostics and confirm what's actually going on.

A clue (red herring?) might be that my OpenVPN client on an OMV server refuses to re-establish a VPN connection until rebooted however I can make a connection from a PC on the same network.

Until what is rebooted: OMV server or router?

Thanks for the feedback.

The OMV server requires a reboot to restart the VPN's.

Interesting comments on PCC. I've gone down this route. I disabled all rules save for the WAN1 NAT to keep existing services running. Tried removing pppoe and using direct IP's and various other solutions to no avail. I've got a feeling that the cause of the issue is because of the 'live' connections which are pre-existing when I enable load sharing. I am forming the opinion the router gets tied up in knots. So I've decided to set this up on the bench as a full mock up as I've got some spare Mikrotik gear. I'll copy the existing configs and see where this goes.

Working in the live environment is also too bloody stressful, yesterday I accidentally disabled the LAN interface with an inadvertent mouse click which necessitated an urgent 4Km round trip to the remote telecom cabinet to re-enable it. Unlike Ubiquity you can't do a test run and then revert or accept after a few minutes.

I'll report back.

Unlike Ubiquity you can't do a test run and then revert or accept after a few minutes.

Try using safe mode.