Dual WAN with port forwarding

levicki · Mon May 08, 2023 8:18 pm

I have two ISPs at the moment.

ISP1 - fiber PPPoE (static IP)
ISP2 - cable (dynamic IP)

I would like to create a dual WAN configuration with load balancing where I would also perform port forwarding for some services over ISP1 and some over ISP2.

I am using RouterOS 7.9.

What would be the best way to accomplish that?

anav · Tue May 09, 2023 12:17 am

The PCC load balancing is straight forward via PCC type rules.
You will need mangling to

a. prerouting --> mark connections for inbound traffic on wans
b. output --> assign routing marks for same traffic returning to originator

c. prerouting apply pcc rules and mark connection for LAN traffic heading outbound
prerouting assign routing marks to same traffic so traffic goes out correct(assigned WAN).

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

In terms of Routes...
The basic concept is that you have

add dst-address=0.0.0.0/0 gateway=ISP1 gatewayIP routing table=main
add dst-address=0.0.0.0/0 gateway=ISP2 gatewayIP routing table=main

add dst-address=0.0.0.0/0 gateway=ISP1 gatewayIP routing table=useISP1
add dst-address=0.0.0.0/0 gateway=ISP2 gatewayIP routing table=useISP2

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This implies you need two tables
add fib name=useISP1
add fib name=useISP2

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

note "ROUTING MARK" used in mangling should equate to table names.................( for wan 1, routing-mark=useISP1 )

levicki · Tue May 09, 2023 12:27 pm

The PCC load balancing is straight forward via PCC type rules.
You will need mangling to

a. prerouting --> mark connections for inbound traffic on wans
b. output --> assign routing marks for same traffic returning to originator

c. prerouting apply pcc rules and mark connection for LAN traffic heading outbound
prerouting assign routing marks to same traffic so traffic goes out correct(assigned WAN).

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

In terms of Routes...
The basic concept is that you have

add dst-address=0.0.0.0/0 gateway=ISP1 gatewayIP routing table=main
add dst-address=0.0.0.0/0 gateway=ISP2 gatewayIP routing table=main

add dst-address=0.0.0.0/0 gateway=ISP1 gatewayIP routing table=useISP1
add dst-address=0.0.0.0/0 gateway=ISP2 gatewayIP routing table=useISP2

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This implies you need two tables
add fib name=useISP1
add fib name=useISP2

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

note "ROUTING MARK" used in mangling should equate to table names.................( for wan 1, routing-mark=useISP1 )

Thanks for responding.

The problem is that one ISP has dynamic IP assigned over DHCP so I cannot enter it as a gateway (as far as I know RouterOS doesn't support FQDN entries for gateways so I can't use dyndns for that either).

Is there some way to work around that?

I would be also ok if I made the ISP with the static address the default one and just route one specific application over the second one.

anav · Tue May 09, 2023 3:14 pm

If you dont try, you wont find out. There is no need for Gateway IP in mangling rules.

levicki · Tue May 09, 2023 6:12 pm

If you dont try, you wont find out. There is no need for Gateway IP in mangling rules.

I never said in mangling rules -- I said in routes, and your example shows it as well (unless it is optional).

anav · Tue May 09, 2023 7:34 pm

Ahh okay, sorry.

Hmm I have plain ethernet cable on mine, and I just use the current gateway in my rules.
When my ISP changes the new gateway auto populates properly with no work from me.

However my primary WAN is fiber from bell and that gateway never updates on my routes just IP DHCP client so I have to script that change........

In your case one of the ISPs is static, so no issue.
The other seems to be plain cable.
Have you tried seeing what happens when you pull the power to the cable box and then reapply to get a new IP.
Do you get the new IP no problem?? If that is the case then you should be good to go.

levicki · Tue May 09, 2023 8:33 pm

Ahh okay, sorry.

Hmm I have plain ethernet cable on mine, and I just use the current gateway in my rules.
When my ISP changes the new gateway auto populates properly with no work from me.

However my primary WAN is fiber from bell and that gateway never updates on my routes just IP DHCP client so I have to script that change........

In your case one of the ISPs is static, so no issue.
The other seems to be plain cable.
Have you tried seeing what happens when you pull the power to the cable box and then reapply to get a new IP.
Do you get the new IP no problem?? If that is the case then you should be good to go.

You are correct that it is a cable ISP.

I have DHCP client listening on that port and it adds default route when it receives new dynamic IP.

However, that dynamic route which has 0.0.0.0/0 and correct gateway IP (which isn't always the same since they have several public IP pools) is always added to the main routing table -- you cannot specify in DHCP client to add default route to a specifc table.

I am not really into scripting that, I think that it would be nice if MikroTik allowed both DHCP client and PPPoE client add default route to select specific routing table instead of defaulting to main. IMO that would make this kind of dual WAN (static + dynamic IP) load balancing much easier to implement.

anav · Wed May 10, 2023 3:07 am

They do..................... its in IP DHCP Client setting !!

adr.jpg

>>>>>>>>>>>>>>>>

Oh right, that works for the default route but not for the route with the table we add.

anav · Wed May 10, 2023 3:46 am

Simply do this.....
Lets say the interface connected to WAN2 is ether2, the name of the applicable interface for ISP traffic.
Let give an another example to make it clearer lets say a pppoe connection where the interface name is pppoe-out1.
Use the interface name instead of gateway IP address.

add dst-address=0.0.0.0/0 gateway=ISP1 (static gatewayIP routing table=main
add dst-address=0.0.0.0/0 gateway=ether2 routing table=main ( you can uncheck the add default route to really see if this work aka reachable )

add dst-address=0.0.0.0/0 gateway=ISP1 (static gatewayIP) routing table=useISP1
add dst-address=0.0.0.0/0 gateway=ether2 routing table=useISP2

levicki · Wed May 10, 2023 1:57 pm

Oh right, that works for the default route but not for the route with the table we add.

Bingo... that's why I said DHCP and PPPoE client should support specifying routing table.

I did try using output interface instead of gateway, and I did manage to get some sort of load balancing.

However, the main issues were:

1. Load balancing did not (probably because of PCC settings given in example) spread connections to same destination IP over different ISPs so I didn't get aggregate speed of both ISPs for download with multiple connections to the same server.
2. I could not for the life of me figure out how to force specific application (BitTorrent) to use only one ISP.

I could live with the first but the second point is a rather big issue for me.

anav · Wed May 10, 2023 2:33 pm

That is because both your ASKs, are not possible in MT routers.

PCC load balance or any time of load balance is NOT BONDING.
That requires the two ISPs to cooperate and in fact its usually the same ISP bonding two ISPs and is very expensive.

Secondly, the MT router cannot normally dictate application behaviour.
However if all the traffic is from ONE IP address that is easy to handle.

fib use-WAN1
fib use-WAN2

dst-address=0.0.0.0/0 gateway=fixedIP table=main
dst-address=0.0.0.0/0 gateway=ether2 table=main

dst-address=0.0.0.0/0 gateway=fixedIP table=use-WAN1
dst-address=0.0.0.0/0 gateway=ether2 table=use-WAN2

Mangle Rules in place (eight)

+++++++++++++++++++++++++++++++++++++++++++++++++++

Add Routing Rule.
add src-address=originatingIP action=lookup-only-in-table table=useWANX

Thus we ensure any traffic from that IP is sent out the specific WAN, but we mark traffic before that in prerouting in the mangle rules so do this for the two applicable mangle rules....

From:
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=no dst-address-type=!local \
in-interface-list=LAN new-connection-mark=WAN1 passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=no dst-address-type=!local \
in-interface-list=LAN new-connection-mark=WAN2 passthrough=yes per-connection-classifier=both-addresses:2/1

TO:
ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=no dst-address-type=!local \
in-interface-list=LAN new-connection-mark=WAN1 passthrough=yes per-connection-classifier=both-addresses:2/0 \
src-address-list=!excluded
add action=mark-connection chain=prerouting connection-mark=no-mark disabled=no dst-address-type=!local \
in-interface-list=LAN new-connection-mark=WAN2 passthrough=yes per-connection-classifier=both-addresses:2/1 \
src-address-list=!excluded

Where
/ip firewall address-list
add address=TorrentIPdevice list=excluded
add address=anyotherdevice list=excluded

levicki · Wed May 10, 2023 6:42 pm

I am aware that PCC is not bonding.

I was hoping that there was a method using PCC to mark connections so that connections that go to same destination address from different source ports (say using src-address-and-port) use different ISPs.

As for the torrent part it's not a dedicated box so filtering by IP is not feasible.

Anyway, thanks for your help. I'll chalk the inability to do what I want as a deficiency in RouterOS and move on.

anav · Wed May 10, 2023 8:19 pm

I am far from expert, and thus maybe others will chime in with other potential options, sorry.

levicki · Thu May 11, 2023 1:47 pm

I am far from expert, and thus maybe others will chime in with other potential options, sorry.

I spent quite some time experimenting and I managed to get download and upload to use full bandwidth of both connections.

You need to use src-address-and-port in PCC rules so that connections from same source address but different source port go to different ISPs.

I also managed to force torrents to ISP2 by setting a specific DHT listen port for TCP and UDP and adding a connection mark to all connections with those ports before PCC forcing them to WAN2.

I also managed to force game traffic over ISP1 in the same way (by marking connection before PCC) based on source ports.

The only problem left to solve is how to update the routing table when the dynamic IP of ISP2 changes by DHCP client.

EDIT:
I found this:

:if ($bound=1) do={/ip route set [find where dst-address=0.0.0.0/0 routing-table=WAN2] gateway=$gateway-address }

I just need to adjust it to check if the address assigned to DHCP client is not 192.168.100.0/24 (that's what cable modem gives out when it has not internet access).

Dual WAN with port forwarding

Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Re: Dual WAN with port forwarding

Who is online