Community discussions

MikroTik App
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 12:05 pm

RouterOS 7.12

WAN1: ether1, 10.30.0.246/22 - DHCP static - (Provider Vivacom, network replaced for security concerns)
WAN2: ether2, 10.10.0.218/24 (Provider TEA, network replaced for security concerns)
LAN: bridge, 192.168.0.1/24

Failover set, port-forwarding set for each WAN to the same services on the same ports.

The problem:
If WAN2 has a lower routing distance - then port forwards work for both WAN1 and WAN2.
If WAN1 has a lower routing distance - then port forwards work only for WAN1

I need WAN1 to have a lower routing distance and port forwards to work for both, since WAN1 is the new provider, has a lot faster connection, but all the services are configured (DNS) to the WAN2 IP.

Tried Mangle rules but didn't help, tried solutions from another posts from the forum but without success.

I'm posting my config with the replaced networks (for security concerns) without the sensitive information.
config_11_2023.rsc
Any help would be appreciated.
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 5:50 pm

Observations:

(1) First problem is your interface lists, there is no reason to have two separate WAN LISTS.
Should be just WAN and just LAN. Anything else only leads to confusion. The reason to create interface lists is when grouping of subnets makes sense for rules, OR you need to indicate a specific vlan is a managment interface.

/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN


(2) Likewise please simplify your member list to the following:
/interface list member
add comment=defconf interface=bridge list=LAN
add interface=ether1 list=WAN
add interface=ether2 list=WAN


(3) Looks like ether1 (primary) is using DHCP client and ether2 (seconday is a static IP)

(4) What is the purpose of this UNSAFE SECURITY INFRACTION? One does not allow direct external access to the configuration of the router. LImiting to an exernal WANIP is wishful thinking.
add action=accept chain=input comment="Remote access MEXUS" dst-port=8291 \
protocol=tcp src-address=213.91.171.181


Where is this external access coming from? Assuming the RB4011 is a home router?
If nothing else, use a simple wireguard connection to VPN into the router and from which you can easily configure the router and reach all the LAN devices as well.

(5) Recommend you change this default rule, not meant for more complex setups (especially the combo of dual wan and port forwarding) to three rules:
from
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
connection-state=new in-interface-list=WAN


TO:
add action=accept chain=forward comment="internet" in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="port forwarding" connection-nat-state=dstnat
add action=drop chain=forward comment="DROP ALL ELSE"


(6) SCRAP all your mangle rules as stated previously and replace with.
The first rule mark connections for all external traffic coming into the router on wan2 and the second rules ensure the traffic will be returned through the same WAN.

add chain=prerouting action=mark-connections connection-mark=no-mark \
in-interface=ether2 new-connection-mark=incomingWAN2 passthrough=yes

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add chain=prerouting action=mark-routing connection-mark=incomingWAN2 \
new-routing-mark= to_ether2 passthrough=no


What is important here too is that we ensure that the traffic does have the standard sourcenat rule to ensure the corrected traffic is given the correct WANIP, either using the default rule
add action=masquerade chain=srcnat out-interface-list=WAN
OR the WANs spelled out separately as YOU HAVE BELOW which is good!!!
add action=masquerade chain=srcnat comment=VIVACOM out-interface=ether1
add action=masquerade chain=srcnat comment=TEA out-interface=ether2


There is no further mangling required. I would add that since some of this traffic concerns return traffic from the LAN, that we modify the forward chain firewall rule accordingly:
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-state=established,related hw-offload=yes connection-mark=no-mark


(7) Now lets get to the Destination Nat rules.....................

a. why are you using RDP? Are you running and RDP server and if so WHY? RDP is no longer considered a secure protocol to use??
We can use Wireguard noted before to replace such requirements

b. The router has a built-in SSH capability, what is the purpose of setting up a separate SSH server on the LAN??

c. Why do you have an email server setup on port 25. This is also a large security risk!! I see you also have 465 which is much better.

d. Although I have no issue with tls secure port 465, port 587 is the officially recommended port ( personal pref, aka your choice ).

e. Port 143 is NOT SECURE and not a port you want to expose to the public and should be deleted as well. Believe you have 993 which is better.

f, PORT 8081 www, should be DISABLED
as its not a secure access method!!

(8) Can you explain the purpose of this Dst Nat rules please........
add action=dst-nat chain=dstnat dst-address=!192.168.0.1 dst-address-type=local dst-port=9998 protocol=tcp to-addresses=192.168.0.111 to-ports=9998

(9) Can you explain the purpose of this srcnat rule please............
add action=masquerade chain=srcnat dst-address=!192.168.0.1 src-address=192.168.0.0/24

(10) Your IP routes are a bit funny, in that there is an inconsistency in the gateway IPs...!!
/ip route
add check-gateway=ping disabled=no distance=12 dst-address=0.0.0.0/0 gateway=\
10.30.0.1 pref-src="" routing-table=main scope=30 suppress-hw-offload=\
no target-scope=10
add check-gateway=ping disabled=no distance=11 dst-address=0.0.0.0/0 gateway=\
10.10.0.112 pref-src="" routing-table=main scope=30 \
suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
212.5.152.1 pref-src="" routing-table=to_ether1 scope=30 \
suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
10.10.0.112 pref-src="" routing-table=to_ether2 scope=30 \
suppress-hw-offload=no target-scope=10


Further, you state that ether1 is the faster WAN1 and should be primary and ether2 (WAN2) should be secondary! Then you say all Servers should work from WAN2.
Quote: "I need WAN1 to have a lower routing distance and port forwards to work for both, since WAN1 is the new provider, has a lot faster connection, but all the services are configured (DNS) to the WAN2 IP." unquote.

So look at the above config piece, 10.10.0.112 is associated with ETHER2 in the last line but has a lower distance in the second line. So this is wrong!!

Please confirm the following: Assuming you mean, all LAN users should use WAN1 and all servers should have incoming traffic from WAN on WAN2.
However,
-->if WAN1 fails, all users should be able to use WAN2
--> if WAN2 fails, all servers should be reachable via WAN1
Last edited by anav on Wed Nov 22, 2023 6:09 pm, edited 2 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 6:00 pm

/ip route
add check-gateway=ping distance=11 dst-address=0.0.0.0/0 gateway=ether1 routing-table=main
add check-gateway=ping distance=12 dst-address=0.0.0.0/0 gateway=ether2 routing-table=main
add dst-address=0.0.0.0/0 gateway=ether2 routing-table=to_ether2

From this setup. all user originated traffic on the LAN will go out WAN1 due to lower distance.
All incoming server traffic coming in on WAN2, will go out WAN2.

Assuming all users are given a DYNDNS URL, for your connection to WAN2......
IF WAN2 goes goes down the URL gets the IP address of WAN1 etc.................
Please confirm.

If WAN1 fails, the router will find WAN2. This will not affect the servers already on WAN2 and all users on the LAN will by the main routes find and use WAN2.
If WAN2 fails, the LAN users will not be affected. IF external users provided DYNDNS URL (you have given them) switches from ISP2 to ISP1, then users will come in on WAN1 and traffic will go out WAN1.
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 7:23 pm

/ip route
add check-gateway=ping distance=11 dst-address=0.0.0.0/0 gateway=ether1 routing-table=main
add check-gateway=ping distance=12 dst-address=0.0.0.0/0 gateway=ether2 routing-table=main
add dst-address=0.0.0.0/0 gateway=ether2 routing-table=to_ether2

From this setup. all user originated traffic on the LAN will go out WAN1 due to lower distance.
All incoming server traffic coming in on WAN2, will go out WAN2.

Assuming all users are given a DYNDNS URL, for your connection to WAN2......
IF WAN2 goes down the URL gets the IP address of WAN1 etc.................
Please confirm.

If WAN1 fails, the router will find WAN2. This will not affect the servers already on WAN2 and all users on the LAN will by the main routes find and use WAN2.
If WAN2 fails, the LAN users will not be affected. IF external users provided DYNDNS URL (you have given them) switches from ISP2 to ISP1, then users will come in on WAN1 and traffic will go out WAN1.
(1) Ok, you have a point, but this is not causing the problem
(2) ether1 and ether2 are added to WAN, I have to remove WAN1 and WAN2
(3) Yes, the ISP uses DHCP, but the address is static (note the real IP is different than the one provided in the config)
(4) I gave myself WinBox access so that I don't have to visit the location. TB4011 is used in a small office
(5) Will try that
(6) Will try that as well. Should I do the same rules for ether1? I want my main connection to be from ether1, not ether2
(7) I manage only one of the few servers behind the router, I'll address your concerns to the personal managing the others.
(8 and 9) kind of hairpin NAT, so that the personnel can access their webmail from the local network using the domain they already used before the installation of the Mikrotik
(10) 212.5.152.1 - forgot to replace this one while replacing the networks for the purpose of posting the config.

>So look at the above config piece, 10.10.0.112 is associated with ETHER2 in the last line but has a lower distance in the second line. So this is wrong!!

Ok let me clarify, the DNS records are set to the IP of the slower ISP. Before they are migrated to the faster ISP IP address they should remain accessible from the slower ISP while the office workers should use the faster connection.
If I change the routing distance having the faster ISP being the primary one, than the port forwarding for the slower ISP stop working.

>Please confirm the following: Assuming you mean, all LAN users should use WAN1 and all servers should have incoming traffic from WAN on WAN2.
However,
-->if WAN1 fails, all users should be able to use WAN2
--> if WAN2 fails, all servers should be reachable via WAN

About the users - YES.
I want the servers reachable from both WANs

>Assuming all users are given a DYNDNS URL, for your connection to WAN2......
IF WAN2 goes down the URL gets the IP address of WAN1 etc.................
Please confirm.

Don't use dynamic DNS at the moment
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 7:42 pm

Okay.......
(1) Point 4, big risk learn Wireguard!!
(2) Point 5, good for port forwarding to work properly from the LAN side, the new rules will work the default you had would not.
(2) I don't understand your point about DNS in terms of deciding server routing can you elaborate/explain as I see nothing that would indicate that in the config??
(3) I am confused as to the servers. (both available).

A. Right now some external users are given the WAN1 IP address to access LAN servers???
&
B. Some external users are given the WAN2 IP address to access LAN servers????

Can I assume that if wan1 or wan2 goes down, those external users will NOT be able to reach their corresponding servers correct?
The failover is really only for the internal users going out to the internet.

(4) Can you clarify that local users are also using the WANIPs to reach servers and not the direct LANIPs??

(5) If hairpin nat is indeed required, the correct format is
add chain=srcnat action=masquerade src-address=192.168.0.0/24 dst-address=192.168.0.0/24

(6) The correct format for all your DST nat rules is
add chain=dstnat action=dst-nat dst-address=FixedWANIP dst-port=XXXX protocol=yyy
to-addresses=LAN-server
( To ports is only required for port translation ).

(7) You failed to answer 7f, 8,
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 10:48 pm

Okay.......
(1) Point 4, big risk learn Wireguard!!
(2) Point 5, good for port forwarding to work properly from the LAN side, the new rules will work the default you had would not.
(2) I don't understand your point about DNS in terms of deciding server routing can you elaborate/explain as I see nothing that would indicate that in the config??
(3) I am confused as to the servers. (both available).

A. Right now some external users are given the WAN1 IP address to access LAN servers???
&
B. Some external users are given the WAN2 IP address to access LAN servers????

Can I assume that if wan1 or wan2 goes down, those external users will NOT be able to reach their corresponding servers correct?
The failover is really only for the internal users going out to the internet.

(4) Can you clarify that local users are also using the WANIPs to reach servers and not the direct LANIPs??

(5) If hairpin nat is indeed required, the correct format is
add chain=srcnat action=masquerade src-address=192.168.0.0/24 dst-address=192.168.0.0/24

(6) The correct format for all your DST nat rules is
add chain=dstnat action=dst-nat dst-address=FixedWANIP dst-port=XXXX protocol=yyy
to-addresses=LAN-server
( To ports is only required for port translation ).

(7) You failed to answer 7f, 8,
(1)(4) I will set a VPN connection later on and disable remote access
(2)(5) Will try it out tomorow
(DNS) Let's say I have subdomain.example.com, the A record is set to the IP of the slow ISP. (Note both ISP1 and ISP2 have public ip addresses which are changed for the sake of sharing the config)
(3) I think (DNS) explains it. All users a given link which point to the slow ISP. A mechanisim to point to a working ISP if the other one is down will be implemented later on. I need to be able to access all services so I can change the DNS records to point to the fast ISP and so on.
The failover right now is only for the users using the local network. By the way I need to implement a better failover, for example ping 8.8.8.8 instead of the gateway, but this will also be implemented later on.
(4) Yes, local users use the domain/subdomains which point to the WAN IPs
(5) Ok will try to fix that
(6) This is only for the hairpin (eg webmail) right?
(7) Port 8081 www will be disabled, 8 - part of hairpin example I found online
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Wed Nov 22, 2023 11:04 pm

Sorry you still have not explained how you are using DNS to 'force' users through one WAN or the other.
What DNS records?

Forcing users out a specific WAN is accomplished via routing of some sort.
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Thu Nov 23, 2023 10:32 am

Sorry you still have not explained how you are using DNS to 'force' users through one WAN or the other.
What DNS records?

Forcing users out a specific WAN is accomplished via routing of some sort.
Ok, let me try to explain.

There is a subdomain, for example: subdomain1.example.com, the A record points to the IP of the slow ISP. So requests to this subdomain are translated to the IP of the slow ISP IP address, the Mikrotik dsnats the request to the HTTP server.

So if an other domain, for example subdomain2.example.com and the A record points to the IP of the fast ISP, than the rest is similar to the upper example

Requests made to the one of the two ISP IPs should come out from the same IP
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Thu Nov 23, 2023 1:16 pm

Observations:
....

Ok, fixed the following
  • Interface lists: LAN (bridge), WAN (ether1, ether2), the rest are deleted
  • Changed the default drop rule to 3 rules as described by you
  • Changed the fasttrack-connection filter rule adding connection-mark=no-mark, couldn't find the hw-offload=yes
  • Changed the hairpin Nat similar to the rules you provided
  • Fixed the inconstancies in the route, leaving the IP addresses instead of ether1/ether2, leaving the slower ISP route with lower distance
  • Disabled the Http service on port 8081
Tried your mangle rules, but creating them for ether1 and ether2 - the result is that I loose remote connectivity to the router every few seconds.

With the above changes: If slow ISP has lower routing distance I can access a server using both WAN IPS, If the faster one has lower routing distance I still can access the server only using the faster ISP IP address
 
templlama
just joined
Posts: 12
Joined: Thu Nov 23, 2023 1:46 pm

Re: Dual WAN failover, port forward not working when changing route distance

Thu Nov 23, 2023 1:55 pm

Will be able to comment later today but for now please post your latest config with the changes you have made.
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Thu Nov 23, 2023 6:40 pm

Will be able to comment later today but for now please post your latest config with the changes you have made.
templlama, anav, I'm posting my config after all the changes I've done today
23.11.23.rsc
The IP addresses of the ISPs are changed again.
I've added recursive routing using Google's public DNS, fixed the interface lists, changed the drop rule into the 3 suggested by anav, marked the fasttrack-connection filter, changed the dstnat to use the WAN interface list, remove unneeded UDP rules, set the fast ISP with lower routing distance, disabled http on port 8081, created different than the suggested mangle rules

Now everything works as I expected but I'm open to suggestions if something needs improvment

Thanks to anav for pointing me in the right direction.
You do not have the required permissions to view the files attached to this post.
 
templlama
just joined
Posts: 12
Joined: Thu Nov 23, 2023 1:46 pm

Re: Dual WAN failover, port forward not working when changing route distance

Thu Nov 23, 2023 8:51 pm

(1) Insecure and potentially dangerous to expose winbox port to the internet. Suggest access router via Wireguard.
add action=accept chain=input comment="Remote access MEXUS" dst-port=8291 \
protocol=tcp src-address=X.X>X>X


(2) You should only have four mangle rules.
FIXED
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether1 new-connection-mark=ether1_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 1 output routing" \
connection-mark=ether1_mark disabled=no new-routing-mark=to_ether1 \
passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether2 new-connection-mark=ether2_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 2 output routing" \
connection-mark=ether2_mark disabled=no new-routing-mark=to_ether2 \
passthrough=no


(3) DST NAT has two possibilities.
a. Will work - you do not have local users accessing servers by DNS or WANIP, and only by LANIP (directly).
b. Will NOT work - users are forced to access servers via DYNDNS type domain name/url or by WANIP.
(If b is the correct answer then we have to make changes due to hairpin nat).

(4) Do not use same DNS sites for Recursive as you do for DNS.......... so recommend change DNS
Keep 8.8.8.8 and 1.1.1.1 for DNS and 8.8.4.4 for recursive and 1.0.0.1 for recursive.

/ip dns
set allow-remote-requests=yes servers=\
8.8.8.8,1.1.1.1,9.9.9.9,208.67.222.222


AND REMOVE THIS DEFAULT static setting at IP DNS.
/ip dns static
add address=192.168.0.1 comment=defconf name=router.lan


(5) Routes are wrong......
FIXED
/ip route
add check-gateway=ping distance=2 dst-address=0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=1.0.0.1 routing-table=main scope=10 target-scope=12
add distance=2 dst-address=8.8.4.4/32 gateway=10.30.152.1 routing-table=main scope=10 target-scope=11
add distance=4 dst-address=1.0.0.1/32 gateway=10.10.10.112 routing-table=main scope=10 target-scope=11
add distance=1 dst-address=0.0.0.0/0 gateway=10.30.152.1 routing-table=to_ether1 scope=10 target-scope=30
add distance=1 dst-address=0.0.0.0/0 gateway=10.10.10.112 routing-table=to_ether2 scope=10 target-scope=30
Last edited by templlama on Fri Nov 24, 2023 12:29 am, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 12:25 am

duplicate
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 10:31 am

(1) Insecure and potentially dangerous to expose winbox port to the internet. Suggest access router via Wireguard.
add action=accept chain=input comment="Remote access MEXUS" dst-port=8291 \
protocol=tcp src-address=X.X>X>X


(2) You should only have four mangle rules.
FIXED
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether1 new-connection-mark=ether1_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 1 output routing" \
connection-mark=ether1_mark disabled=no new-routing-mark=to_ether1 \
passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether2 new-connection-mark=ether2_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 2 output routing" \
connection-mark=ether2_mark disabled=no new-routing-mark=to_ether2 \
passthrough=no


(3) DST NAT has two possibilities.
a. Will work - you do not have local users accessing servers by DNS or WANIP, and only by LANIP (directly).
b. Will NOT work - users are forced to access servers via DYNDNS type domain name/url or by WANIP.
(If b is the correct answer then we have to make changes due to hairpin nat).

(4) Do not use same DNS sites for Recursive as you do for DNS.......... so recommend change DNS
Keep 8.8.8.8 and 1.1.1.1 for DNS and 8.8.4.4 for recursive and 1.0.0.1 for recursive.

/ip dns
set allow-remote-requests=yes servers=\
8.8.8.8,1.1.1.1,9.9.9.9,208.67.222.222


AND REMOVE THIS DEFAULT static setting at IP DNS.
/ip dns static
add address=192.168.0.1 comment=defconf name=router.lan


(5) Routes are wrong......
FIXED
/ip route
add check-gateway=ping distance=2 dst-address=0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=1.0.0.1 routing-table=main scope=10 target-scope=12
add distance=2 dst-address=8.8.4.4/32 gateway=10.30.152.1 routing-table=main scope=10 target-scope=11
add distance=4 dst-address=1.0.0.1/32 gateway=10.10.10.112 routing-table=main scope=10 target-scope=11
add distance=1 dst-address=0.0.0.0/0 gateway=10.30.152.1 routing-table=to_ether1 scope=10 target-scope=30
add distance=1 dst-address=0.0.0.0/0 gateway=10.10.10.112 routing-table=to_ether2 scope=10 target-scope=30
(1) I will setup a VPN later on as mentioned above
(2) Can you please explain why are my rules wrong, I tried rules similar to ypur suggested by anav an was loosing connectivity to the router every few seconds.
On the mark-connection the difference I see is that you added connection-mark=no-mark and passthrough=yes, what do those additional parameters do?
I had the second you mentioned disabled since while testing I was getting the result I've wanted, I have port-forwarding working for both WANs. Why I need to disable the forwarding rule and enable the output rule?
(3) Mixture of both, most services should be accessible only from the internet, mail and webmail should be accessible from both, Hairpin is set for those using rules similar to those provided by anav.
(4) Ok will change that
(5) I'm not proficient with Mikrotik. I followed a YouTube video suggested in this forum for the routes, can you please help me understand what is wrong.
The main difference betwen my and your rules seems to be the scopes and target-scopes can you please explain this as well?
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 10:41 am

(1) Insecure and potentially dangerous to expose winbox port to the internet. Suggest access router via Wireguard.
add action=accept chain=input comment="Remote access MEXUS" dst-port=8291 \
protocol=tcp src-address=X.X>X>X


(2) You should only have four mangle rules.
FIXED
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether1 new-connection-mark=ether1_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 1 output routing" \
connection-mark=ether1_mark disabled=no new-routing-mark=to_ether1 \
passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether2 new-connection-mark=ether2_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 2 output routing" \
connection-mark=ether2_mark disabled=no new-routing-mark=to_ether2 \
passthrough=no


(3) DST NAT has two possibilities.
a. Will work - you do not have local users accessing servers by DNS or WANIP, and only by LANIP (directly).
b. Will NOT work - users are forced to access servers via DYNDNS type domain name/url or by WANIP.
(If b is the correct answer then we have to make changes due to hairpin nat).

(4) Do not use same DNS sites for Recursive as you do for DNS.......... so recommend change DNS
Keep 8.8.8.8 and 1.1.1.1 for DNS and 8.8.4.4 for recursive and 1.0.0.1 for recursive.

/ip dns
set allow-remote-requests=yes servers=\
8.8.8.8,1.1.1.1,9.9.9.9,208.67.222.222


AND REMOVE THIS DEFAULT static setting at IP DNS.
/ip dns static
add address=192.168.0.1 comment=defconf name=router.lan


(5) Routes are wrong......
FIXED
/ip route
add check-gateway=ping distance=2 dst-address=0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=1.0.0.1 routing-table=main scope=10 target-scope=12
add distance=2 dst-address=8.8.4.4/32 gateway=10.30.152.1 routing-table=main scope=10 target-scope=11
add distance=4 dst-address=1.0.0.1/32 gateway=10.10.10.112 routing-table=main scope=10 target-scope=11
add distance=1 dst-address=0.0.0.0/0 gateway=10.30.152.1 routing-table=to_ether1 scope=10 target-scope=30
add distance=1 dst-address=0.0.0.0/0 gateway=10.10.10.112 routing-table=to_ether2 scope=10 target-scope=30
(1) I will setup a VPN later on as mentioned above
(2) Can you please explain why are my rules wrong, I tried rules similar to your suggested by anav and was loosing connectivity to the router every few seconds.
On the mark-connection the difference I see is that you added connection-mark=no-mark and passthrough=yes, what do those additional parameters do?
I had the second you mentioned disabled since while testing I was getting the result I've wanted, I have port-forwarding working for both WANs. Why I need to disable the forwarding rule and enable the output rule?
(3) Mixture of both, most services should be accessible only from the internet, mail and webmail should be accessible from both, Hairpin is set for those using rules similar to those provided by anav.
(4) I fixed that
(5) I'm not proficient with Mikrotik. I followed a YouTube video suggested in this forum for the routes, can you please help me understand what is wrong.
The main difference betwen my and your rules seems to be the scopes and target-scopes can you please explain this as well?
Here is the video I followed https://www.youtube.com/watch?v=eTmpBAAW_pQ
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 2:19 pm

I have a great idea, why dont you ask the people making vidoes for help...........
The onus is ON YOU, to read the mikrotik docs and read as many threads as possible to learn.
There are some decent videos out there by a few people the rest will lead you astray.
Network Berg is good
Network Trip is good
MT has good videos
etc....
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 3:57 pm

I have a great idea, why dont you ask the people making vidoes for help...........
The onus is ON YOU, to read the mikrotik docs and read as many threads as possible to learn.
There are some decent videos out there by a few people the rest will lead you astray.
Network Berg is good
Network Trip is good
MT has good videos
etc....
The video you mentioned is by 'The Network Trip'. It was recommended on this forum, alongside a link to your tutorial, as a valuable resource. While I've gone through part of the documentation, I'm still unclear about what's wrong with my configuration. If you could provide a brief explanation for each of my errors, it would greatly aid my understanding, as opposed to just suggesting I use your configuration.

Before posting here, I extensively searched Google and this forum, and tried numerous solutions, but couldn't resolve the issue. I only configure Mikrotiks occasionally, maybe once every year or two, since it's not my primary job; I'm not a network administrator.

With the latest configuration, everything is functioning as I expected. The mangle rules are correctly marking traffic and directing it through the appropriate ISP. The routes I've set up are based on the video from 'The Network Trip', which you and other forum members seem to recommend. I would appreciate an explanation of why I need to modify the scopes in my routes, or why I should disable the forward mangle mark route rule and enable the same for output.

I understand that simply typing a command like 'rm -rf /' in a Linux help forum could seem like a fix (although it would actually cause major issues), but I believe a clear explanation of each option will help users understand their mistakes and the purpose of each command.

Thank you for your suggestions. I tried them exactly as provided, then adjusted them to fit my needs, and achieved the expected results. However, I didn't understand your suggestions regarding the routes, specifically why I should use ether1/2 instead of the IP, and why to use those specific mangle rules, which caused connection issues for me. Other threads in this forum suggest rules similar to the ones I used in the last config I've posted.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 5:05 pm

Busy today, but if you post your latest config I will spend more time on it this weekend.
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Fri Nov 24, 2023 6:06 pm

Busy today, but if you post your latest config I will spend more time on it this weekend.
Thanks a lot

Here is the config, modified manually to accommodate all the changes made today.
24.11.23.rsc
Edit: Just tried the Mangle rules only on the Slow ISP as described by you or the other participant.
If the marking rule on the prerouting has passthrough enabled, and connection mark no-mark, the other forward rule with the mark route action disabled, and the output rule added, then port-forwarding for this ISP stops working.
Tired it with my rules as well - passthrough enabled, and connection mark no-mark breaks the port-forwarding for the ISP
I'm adding a screen shot the Mangle rules page, and as you can see the traffic is marked
mangle.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Sat Nov 25, 2023 4:41 pm

(1) This default rule is now replaced and should be removed.
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \

connection-state=new disabled=yes in-interface-list=WAN
add action=accept chain=forward comment=Internet in-interface-list=LAN \
out-interface-list=WAN
add action=accept chain=forward comment="Port forwarding" \
connection-nat-state=dstnat
add action=drop chain=forward comment="Drop all else"


(2) Mangle rules should be.
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether1 new-connection-mark=ether1_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 1 forward routing" \
connection-mark=ether1_mark new-routing-mark=to_ether1 passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether2 new-connection-mark=ether2_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 2 forward routing" \
connection-mark=ether2_mark new-routing-mark=to_ether2 passthrough=no


These four rules ensure any traffic coming in on either WAN go out the SAME WAN.
One should ensure proper srcnat rules are in effect as well.
Which you already have in these two rules.
/ip firewall nat
add action=masquerade chain=srcnat comment=VIVACOM out-interface=ether1
add action=masquerade chain=srcnat comment=TEA out-interface=ether2


(3) Your routes are incorrect.......
/ip route
add check-gateway=ping comment=Vivacom distance=5 dst-address=\
0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add check-gateway=ping comment=TEA distance=10 dst-address=\
0.0.0.0/0 gateway=1.0.0.1 routing-table=main scope=10 target-scope=12
add comment="Vivacom marked" distance=5 dst-address=8.8.4.4/32 \
gateway=10.30.152.1 routing-table=main scope=10 target-scope=11
add comment="TEA marked" distance=10 dst-address=1.0.0.1/32 \
gateway=10.10.10.112 routing-table=main scope=10 target-scope=11
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add comment="Monitor via Vivacom" dst-address=0.0.0.0/0 gateway=10.30.152.1 \
routing-table=to_ether1 scope=10 target-scope=30
add comment="Monitor via TEA" disabled=no dst-address=0.0.0.0/0 gateway=10.10.10.112 \
routing-table=to_ether2 scope=10 target-scope=30
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Mon Nov 27, 2023 10:50 am

(1) This default rule is now replaced and should be removed.
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \

connection-state=new disabled=yes in-interface-list=WAN
add action=accept chain=forward comment=Internet in-interface-list=LAN \
out-interface-list=WAN
add action=accept chain=forward comment="Port forwarding" \
connection-nat-state=dstnat
add action=drop chain=forward comment="Drop all else"


(2) Mangle rules should be.
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether1 new-connection-mark=ether1_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 1 forward routing" \
connection-mark=ether1_mark new-routing-mark=to_ether1 passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=ether2 new-connection-mark=ether2_mark passthrough=yes
add action=mark-routing chain=output comment="Ether 2 forward routing" \
connection-mark=ether2_mark new-routing-mark=to_ether2 passthrough=no


These four rules ensure any traffic coming in on either WAN go out the SAME WAN.
One should ensure proper srcnat rules are in effect as well.
Which you already have in these two rules.
/ip firewall nat
add action=masquerade chain=srcnat comment=VIVACOM out-interface=ether1
add action=masquerade chain=srcnat comment=TEA out-interface=ether2


(3) Your routes are incorrect.......
/ip route
add check-gateway=ping comment=Vivacom distance=5 dst-address=\
0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add check-gateway=ping comment=TEA distance=10 dst-address=\
0.0.0.0/0 gateway=1.0.0.1 routing-table=main scope=10 target-scope=12
add comment="Vivacom marked" distance=5 dst-address=8.8.4.4/32 \
gateway=10.30.152.1 routing-table=main scope=10 target-scope=11
add comment="TEA marked" distance=10 dst-address=1.0.0.1/32 \
gateway=10.10.10.112 routing-table=main scope=10 target-scope=11
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add comment="Monitor via Vivacom" dst-address=0.0.0.0/0 gateway=10.30.152.1 \
routing-table=to_ether1 scope=10 target-scope=30
add comment="Monitor via TEA" disabled=no dst-address=0.0.0.0/0 gateway=10.10.10.112 \
routing-table=to_ether2 scope=10 target-scope=30

Here is my logic for the routes:

Routes for the LAN clients use the recursive routing. The routes you've highlighted are used for routing of the marked connections. I don't need failover for them, If the ISP is down, the connection shouldn't go out from the other ISP. Is my logic wrong?
 
mexus
just joined
Topic Author
Posts: 13
Joined: Mon May 02, 2016 1:18 pm

Re: Dual WAN failover, port forward not working when changing route distance

Mon Nov 27, 2023 11:11 am

Also how to force a single LAN IP address to use only the ISP2 (ether2)?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22084
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Dual WAN failover, port forward not working when changing route distance

Mon Nov 27, 2023 1:25 pm

Well, good to know, defining the requirements clearly is best done before applying a config.

a. you have two WANs.
b. there is no failover
c. the LAN should use WAN1 only

if wan1 goes down, no LAN traffic goes to WAN2
if wan2 goes down, no LAN traffic goes to WAN1
Wan 2 is a static fixed WANIP

You have port forwarding to WAN1 and WAN2 and they both should be available concurrently.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Since WAN1 will be the only WAN available to the LAN, and WAN2 has a static fixed WANIP,
We can solve your issue with a combination of mangles for port forwarding to WAN2
add chain=prerouting action=mark-connections connection=mark=no-mark \
new-connection-mark=inWAN2 in-interface=ether2 passthrough=yes
add chain=output action=mark=routing connection-mark=inWAN2 \
new-routing-mark=useWAN2 passthough=no


AND routing rules for users on the LAN.

First ADD TABLE
/routing table add fib name=to_ether2

Then ADD ROUTE
/ip route add dst-address=0.0.0.0/0 gateway=ether2 table=to_ether2 ***** see below for recursive attempt

Add Two ROUTING RULES
/routing rule add user1-IP action=lookup-only-in-table table=to_ether2 comment="LAN user1 to use WAN2 only"
/routing rule add user2-IP action=lookup-only-in-table table=to_ether2[/i][/b] comment="LAN user2 to use WAN2 only"

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Now for IP Routes,

To keep with recursive theme for WAN1.................


/ip route
add check-gateway=ping dst-address=0.0.0.0/0 gateway=8.8.4.4 routing-table=main scope=10 target-scope=12
add comment="Vivacom marked" dst-address=8.8.4.4/32 gateway=10.30.152.1 routing-table=main scope=10 target-scope=11


question: I though ISP1 was dynamic not fixed-static like ISP2. Thus how is the gateway a fixed number???
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

{ I have never tried recursive in anything but the MAIN table, so the below may not work.
In other words, you may have to ditch recursive for WAN2 and stick with the IP route above provided , but give this a shot! }

add check-gateway=ping dst-address=0.0.0.0/0 gateway=1.0.0.1 scope=10 target scope=12 table=onlyWAN2
add dst-address=1.0.0.1/32 gateway=ether2 scope=10 target scope=11 table=onlyWAN2


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Who is online

Users browsing this forum: No registered users and 18 guests