In fact I didn't know if this is a beginner or general question...
I'm a newbe to Mikrotik but with some experiece with other routers.
I read lots of documentation and have a problem with NAT and mangle in more complex environment.
There is no comprehensive examples that would cover few problems in one script – like full failover scenario.

I did not also find in the network any description how to make internal servers’ load balancing/failover to single or multiple wans (eg. with a mangle command) – what probably is possible using Mikrotik, but nobody has described this up to now…

Even Mikrotik did not publish such example, that – for sure – would be very appreciated by lots of people.

Can anybody (more advanced in Mikrotik routing) sketch the NAT (src and dstn) and mangle configuration for described example?
I would appreciate any help.

Situation:

WAN1 bridge – IPs: 100.0.0.2/29, 100.0.0.3/29 (SMTP), 100.0.0.4 /29 (WWW, DNS), gatewayWAN1 100.0.0.1/29
WAN2 bridge – IPs: 200.0.0.2/29, 200.0.0.3/29 (SMTP), 200.0.0.4/29 (WWW, DNS), gatewayWAN2 200.0.0.1/29
LAN1 bridge – Ips: 192.168.0.1/24
LAN2 bridge – Ips: 10.0.0.1/24

Please correct me if I'm wrong - in my opinion using bridges for all interfaces gives the possibility to easy add proper interfaces, switches or VLANs to them, so it can allow for fast migration between routers having different number of ports without the pain in changing all firewall rules etc.

On LAN1 we have:
ServerSMTP port TCP 25 – IP: 192.168.0.10

On LAN2 we have:

ServerDNS1 IP: 10.0.0.30 – server DNS - port UDP 53, TCP 53
ServerDNS2 IP: 10.0.0.130 – server DNS - port UDP 53, TCP 53

ServerWWW1 IP: 10.0.0.10 – server WWW - port TCP 80,443 (www.AAA.com)
ServerWWW2 IP: 10.0.0.11 – server WWW - port TCP 80,443 (www.AAA.com)
ServerWWW3 IP: 10.0.0.12 – server WWW - port TCP 80,443 (www.AAA.com)

+ Few computers that are put on the TrafficWAN1 address List

We would like to:
#1
Route all traffic from LAN1 through WAN1 (100.0.0.2) except from traffic from SMTP server, that shall be routed through IP: 100.0.0.3
In case WAN1 is not functioning all traffic from LAN1 shall be routed through WAN2 IP 200.0.0.2/29, SMTP server through 200.0.0.3.

#2
SMTP server shall serve and respond on SMTP communication on both interfaces (100.0.0.3 and 200.0.0.3).

#3
Traffic from LAN2 shall be routed through WAN2, except from computers that are on TrafficWAN1 address list. By default all traffic from LAN2 shall use 200.0.0.2 addresses except from published servers WAN2 traffic, that shall go out through 200.0.0.4 address.
In case WAN2 is not working traffic from LAN2 shall be switched to WAN1.

And now most important issue, like internal servers failover and load balancing:

#4
There are 2 DNS servers.
ServerDNS1 serves all incoming DNS questions on interface WAN1 (IP: 100.0.0.4)
ServerDNS2 serves all incoming DNS questions on interface WAN2 (IP: 200.0.0.4).

In case ServerDNS1 is down all incoming DNS requests in 100.0.0.4 should be handled by ServerDNS2.
In case ServerDNS2 is down all incoming DNS requests in 200.0.0.4 should be handled by ServerDNS1.

#5
There are 3 (or more) WWW servers (ServerWWW1, ServerWWW2, ServerWWW3).
They are configure to serve that same web site/web content. Kind of farm of web servers.
All of them are handling www requests on both interfaces (WAN1 100.0.0.4 and WAN2 200.0.0.4).
Incoming traffic shall be distributed among internal servers (eg. semi-equally).
In case any of servers is down (failure, maintenance etc.) – incoming traffic is distributed only between working servers.

And for the end:
#6
If it is possible - any of published services (WWW, DNS, SMTP) shall be visible and accessible for internal users through public IP reference (kind of hairpin NAT?)

Bump...

Anybody could help in this issue? Or in part of it?

Here is describes 2 Method for your load balancing (PCC & ECMP)
Please don't copy-paste.
Just follow the rules:

++++
PCC
++++
/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=192.168.2.1/24 network=192.168.2.0 broadcast=192.168.2.255 interface=wan1
add address=192.168.4.1/24 network=192.168.4.0 broadcast=192.168.4.255 interface=wan2

/ip route
add dst-address=0.0.0.0/0 gateway=192.168.2.1 routing-mark=to_wan1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.4.1 routing-mark=to_wan2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.2.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.4.1 distance=2 check-gateway=ping

/ip firewall nat
add action=masquerade chain=srcnat disabled=no out-interface=wan1
add action=masquerade chain=srcnat disabled=no out-interface=wan2

/ip firewall mangle
add action=mark-connection chain=input disabled=no in-interface=wan1 new-connection-mark= wan1_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface= wan2 new-connection-mark= wan2_conn passthrough=yes

add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0 protocol=tcp
add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1 protocol=tcp

add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark= wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0 protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1 protocol=tcp

add action=mark-routing chain=prerouting connection-mark=wan1_conn disabled=no in-interface=Local new-routing-mark=to_wan1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=wan2_conn disabled=no in-interface=Local new-routing-mark=to_wan2 passthrough=yes

add action=mark-routing chain=output connection-mark=wan1_conn disabled=no new-routing-mark=to_wan1 passthrough=yes
add action=mark-routing chain=output connection-mark=wan2_conn disabled=no new-routing-mark=to_wan2 passthrough=yes

/ip dns
set allow-remote-requests=yes cache-max-ttl=1w cache-size=5000KiB max-udp-packet-size=512 servers=8.8.8.8, 8.8.4.4
..........................................................................

++++++
ECMP
++++++

/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=LOCAL
add address=192.168.2.1/24 network=192.168.2.0 broadcast=192.168.2.255 interface=WAN1
add address=192.168.4.1/24 network=192.168.4.0 broadcast=192.168.4.255 interface=WAN2

/ ip route
add dst-address=0.0.0.0/0 gateway=192.168.2.1 routing-mark=to_WAN1
add dst-address=0.0.0.0/0 gateway=192.168.4.1 routing-mark=to_WAN2
add dst-address=0.0.0.0/0 gateway=192.168.2.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.4.1 distance=2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.2.1,192.168.4.1 check-gateway=ping

/ ip firewall nat
add chain=srcnat out-interface=WAN1 action=masquerade
add chain=srcnat out-interface=WAN2 action=masquerade

/ ip firewall mangle
add chain=input in-interface=WAN1 action=mark-connection new-connection-mark=WAN1_conn
add chain=input in-interface=WAN2 action=mark-connection new-connection-mark=WAN2_conn
add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=to_WAN1
add chain=output connection-mark=WAN2_conn action=mark-routing new-routing-mark=to_WAN2

/ip dns
set allow-remote-requests=yes cache-max-ttl=1w cache-size=5000KiB max-udp-packet-size=512 servers=8.8.8.8, 8.8.4.4


*** Please don't copy-paste ..... just follow the rules.


Best regards

You can also do bandwidth based LB. Watch the presentation in my sig.

Thank you.

Regarding load balancing I found similar descriptions in network. Mangle with 1 LAN and several WANs is quite typical, especially when we dont care where the specified kind of transmission shall be routed to.

But in fact in both examples there is no answer to the question how to make reverse-PCC for incoming connections in order to internal web servers load ballancing in combination with pool of servers publications over 2 WANs.
To define outgoing connection as well as nat configuration in such environment, especially where we have 2 WAN and 2 LAN with different routing requirements with respect to internal servers load balancing makes this question more difficult. At least for me...

If you watch that video, it should be clear how to do everything you want. Its clearly explained how to get traffic coming to your LANs from both WANs.
Also, if you have multiple LANs, simply create multiple rules with the "Load Balancing Here" and devide what to send where in those rules.

All this is referencing the material from my presentation.

How do you make a client stick to the server assigned for the first connection until timeout?

As I wrote previously - outgoing load balancing/failover is not the main issue.

The real problem is load ballancing of INCOMING requests to the farm of web/dns servers and failover mechanism of those servers (as we probably can't use check-gateway=ping because there is no routing inside, just nat).
And next thing is to keep external client TCP/UDP connection to the same web/dns server with server's reply outgoing through proper incoming WAN.

Regarding sessions (in the mean of e.g. PHP sessions, not TCP sessions) - it shouldn't be maintained by router - as it can be implemented in web-servers' farm - e.g. by memcache or database session sharing.