Greetings everyone, long time stalker here on the MT forums, first time poster.

I have implemented ECMP load balancing on 3 ADSL links with policy routing rules for ports to resolve the various MSN, Skype and gaming issues.
My system is a P4 3.0 GHZ HT system with 4 NICS running a MT hotspot service.

Using policy routing for ports 500 - 65535 over 3 adsl lines has worked well, except as mentioned by users in the forums HTTP downloads timeout after 10 - 20 minutes. //mentioned in http://forum.mikrotik.com/viewtopic.php?f=2&t=29412.//

The only problem that I have now is that HTTP/FTP downloads taking longer than 10-15 minutes seem to stall and time out. Does anyone have a solution for this?

My theory is that its to do with HTTP load balancing that automatically shifts connections to a new line after a period of time, irregradless if the connection is active or not. This theory makes sense since MT uses round-robin with ECMP.

So if a download is set to go over this time period then it will stall and eventually fail. This does not affect my policy based routing rules as they are set to go out the same links irregardless, only HTTP/FTP downloads fail when taking longer than 10 - 15 minutes.

The second theory is that it has something to do with the IP Firewall connection tracking. Does one need to adjust the tracking values for TCP Close from 10 seconds to maybe 10 minutes or even longer?

And to top things off here are my current configs for my MT Box:

/ip address
add address=10.10.11.1/24 broadcast=10.10.11.255 comment="" disabled=no \
interface=LAN network=10.10.11.0
add address=10.0.0.2/24 broadcast=10.0.0.255 comment="" disabled=no \
interface=WAN1 network=10.0.0.0
add address=10.0.1.2/24 broadcast=10.0.1.255 comment="" disabled=no \
interface=WAN2 network=10.0.1.0
add address=10.0.2.2/24 broadcast=10.0.2.255 comment="" disabled=no \
interface=WAN3 network=10.0.2.0

/ip dhcp-server config
set store-leases-disk=5m

/ip dhcp-server network
add address=10.10.11.0/24 comment="" gateway=10.10.11.1

/ip dns
set allow-remote-requests=no cache-max-ttl=1w cache-size=2048KiB \
max-udp-packet-size=512 primary-dns=202.180.64.9 secondary-dns=\
202.180.64.2

/ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s \
tcp-close-wait-timeout=10s tcp-established-timeout=1d \
tcp-fin-wait-timeout=10s tcp-last-ack-timeout=10s \
tcp-syn-received-timeout=5s tcp-syn-sent-timeout=5s tcp-syncookie=no \
tcp-time-wait-timeout=10s udp-stream-timeout=3m udp-timeout=10s

/ip firewall filter
add action=passthrough chain=unused-hs-chain comment=\
"place hotspot rules here" disabled=yes
add action=drop chain=forward comment="" connection-state=invalid disabled=no

/ip firewall mangle
add action=mark-connection chain=input comment="" disabled=no in-interface=\
WAN1 new-connection-mark=WAN1_CONN passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=\
WAN2 new-connection-mark=WAN2_CONN passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=\
WAN3 new-connection-mark=WAN3_CONN passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=WAN1_CONN \
disabled=no new-routing-mark=TO_WAN1 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=WAN2_CONN \
disabled=no new-routing-mark=TO_WAN2 passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=WAN3_CONN \
disabled=no new-routing-mark=TO_WAN3 passthrough=yes
add action=mark-packet chain=prerouting comment=icmp disabled=no \
in-interface=WAN1 new-packet-mark=icmp_in passthrough=no protocol=icmp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=icmp_out out-interface=WAN1 passthrough=no protocol=icmp
add action=mark-packet chain=prerouting comment=p2p disabled=no in-interface=\
WAN1 new-packet-mark=p2p_in p2p=all-p2p passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=p2p_out out-interface=WAN1 p2p=all-p2p passthrough=no
add action=mark-packet chain=prerouting comment=pop3 disabled=no \
in-interface=WAN1 new-packet-mark=pop3_in passthrough=no protocol=tcp \
src-port=110
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=110 \
new-packet-mark=pop3_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=smtp disabled=no \
in-interface=WAN1 new-packet-mark=smtp_in passthrough=no protocol=tcp \
src-port=25
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=25 \
new-packet-mark=smtp_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=imap disabled=no \
in-interface=WAN1 new-packet-mark=imap_in passthrough=no protocol=tcp \
src-port=143
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=143 \
new-packet-mark=imap_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssh disabled=no dst-port=22 \
in-interface=WAN1 new-packet-mark=ssh_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=ssh_out out-interface=WAN1 passthrough=no protocol=tcp \
src-port=22
add action=mark-packet chain=prerouting comment=winbox disabled=no dst-port=\
8291 in-interface=WAN1 new-packet-mark=winbox_in passthrough=no protocol=\
tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=winbox_out out-interface=WAN1 passthrough=no protocol=tcp \
src-port=8291
add action=mark-packet chain=prerouting comment=dns disabled=no in-interface=\
WAN1 new-packet-mark=dns_in passthrough=no protocol=udp src-port=53
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=53 \
new-packet-mark=dns_out out-interface=WAN1 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=www disabled=no in-interface=\
WAN1 new-packet-mark=www_in passthrough=no protocol=tcp src-port=80
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=80 \
new-packet-mark=www_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssl disabled=no in-interface=\
WAN1 new-packet-mark=ssl_in passthrough=no protocol=tcp src-port=443
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=443 \
new-packet-mark=ssl_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=udp disabled=no in-interface=\
WAN1 new-packet-mark=udp_in passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=udp_out out-interface=WAN1 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=tcp disabled=no in-interface=\
WAN1 new-packet-mark=tcp_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=tcp_out out-interface=WAN1 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=other disabled=no \
in-interface=WAN1 new-packet-mark=other_in passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=other_out out-interface=WAN1 passthrough=no
add action=mark-packet chain=prerouting comment=icmp disabled=no \
in-interface=WAN2 new-packet-mark=icmp_in passthrough=no protocol=icmp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=icmp_out out-interface=WAN2 passthrough=no protocol=icmp
add action=mark-packet chain=prerouting comment=p2p disabled=no in-interface=\
WAN2 new-packet-mark=p2p_in p2p=all-p2p passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=p2p_out out-interface=WAN2 p2p=all-p2p passthrough=no
add action=mark-packet chain=prerouting comment=pop3 disabled=no \
in-interface=WAN2 new-packet-mark=pop3_in passthrough=no protocol=tcp \
src-port=110
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=110 \
new-packet-mark=pop3_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=smtp disabled=no \
in-interface=WAN2 new-packet-mark=smtp_in passthrough=no protocol=tcp \
src-port=25
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=25 \
new-packet-mark=smtp_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=imap disabled=no \
in-interface=WAN2 new-packet-mark=imap_in passthrough=no protocol=tcp \
src-port=143
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=143 \
new-packet-mark=imap_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssh disabled=no dst-port=22 \
in-interface=WAN2 new-packet-mark=ssh_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=ssh_out out-interface=WAN2 passthrough=no protocol=tcp \
src-port=22
add action=mark-packet chain=prerouting comment=winbox disabled=no dst-port=\
8291 in-interface=WAN2 new-packet-mark=winbox_in passthrough=no protocol=\
tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=winbox_out out-interface=WAN2 passthrough=no protocol=tcp \
src-port=8291
add action=mark-packet chain=prerouting comment=dns disabled=no in-interface=\
WAN2 new-packet-mark=dns_in passthrough=no protocol=udp src-port=53
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=53 \
new-packet-mark=dns_out out-interface=WAN2 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=www disabled=no in-interface=\
WAN2 new-packet-mark=www_in passthrough=no protocol=tcp src-port=80
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=80 \
new-packet-mark=www_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssl disabled=no in-interface=\
WAN2 new-packet-mark=ssl_in passthrough=no protocol=tcp src-port=443
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=443 \
new-packet-mark=ssl_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=udp disabled=no in-interface=\
WAN2 new-packet-mark=udp_in passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=udp_out out-interface=WAN2 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=tcp disabled=no in-interface=\
WAN2 new-packet-mark=tcp_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=tcp_out out-interface=WAN2 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=other disabled=no \
in-interface=WAN2 new-packet-mark=other_in passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=other_out out-interface=WAN2 passthrough=no
add action=mark-packet chain=prerouting comment=icmp disabled=no \
in-interface=WAN3 new-packet-mark=icmp_in passthrough=no protocol=icmp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=icmp_out out-interface=WAN3 passthrough=no protocol=icmp
add action=mark-packet chain=prerouting comment=p2p disabled=no in-interface=\
WAN3 new-packet-mark=p2p_in p2p=all-p2p passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=p2p_out out-interface=WAN3 p2p=all-p2p passthrough=no
add action=mark-packet chain=prerouting comment=pop3 disabled=no \
in-interface=WAN3 new-packet-mark=pop3_in passthrough=no protocol=tcp \
src-port=110
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=110 \
new-packet-mark=pop3_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=smtp disabled=no \
in-interface=WAN3 new-packet-mark=smtp_in passthrough=no protocol=tcp \
src-port=25
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=25 \
new-packet-mark=smtp_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=imap disabled=no \
in-interface=WAN3 new-packet-mark=imap_in passthrough=no protocol=tcp \
src-port=143
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=143 \
new-packet-mark=imap_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssh disabled=no dst-port=22 \
in-interface=WAN3 new-packet-mark=ssh_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=ssh_out out-interface=WAN3 passthrough=no protocol=tcp \
src-port=22
add action=mark-packet chain=prerouting comment=winbox disabled=no dst-port=\
8291 in-interface=WAN3 new-packet-mark=winbox_in passthrough=no protocol=\
tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=winbox_out out-interface=WAN3 passthrough=no protocol=tcp \
src-port=8291
add action=mark-packet chain=prerouting comment=dns disabled=no in-interface=\
WAN3 new-packet-mark=dns_in passthrough=no protocol=udp src-port=53
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=53 \
new-packet-mark=dns_out out-interface=WAN3 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=www disabled=no in-interface=\
WAN3 new-packet-mark=www_in passthrough=no protocol=tcp src-port=80
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=80 \
new-packet-mark=www_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=ssl disabled=no in-interface=\
WAN3 new-packet-mark=ssl_in passthrough=no protocol=tcp src-port=443
add action=mark-packet chain=postrouting comment="" disabled=no dst-port=443 \
new-packet-mark=ssl_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=udp disabled=no in-interface=\
WAN3 new-packet-mark=udp_in passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=udp_out out-interface=WAN3 passthrough=no protocol=udp
add action=mark-packet chain=prerouting comment=tcp disabled=no in-interface=\
WAN3 new-packet-mark=tcp_in passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=tcp_out out-interface=WAN3 passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment=other disabled=no \
in-interface=WAN3 new-packet-mark=other_in passthrough=no
add action=mark-packet chain=postrouting comment="" disabled=no \
new-packet-mark=other_out out-interface=WAN3 passthrough=no
add action=mark-routing chain=prerouting comment="Policy Routing Range 1" \
disabled=no dst-port=500-15000 new-routing-mark=RANGE1 passthrough=no \
protocol=tcp
add action=mark-routing chain=prerouting comment="Policy Routing Range 2" \
disabled=no dst-port=15001-43000 new-routing-mark=RANGE2 passthrough=no \
protocol=tcp
add action=mark-routing chain=prerouting comment="Policy Routing Range 3" \
disabled=no dst-port=43001-65535 new-routing-mark=RANGE3 passthrough=yes \
protocol=tcp
add action=mark-routing chain=prerouting comment="Policy Routing Range 1 UDP" \
disabled=no dst-port=500-15000 new-routing-mark=RANGE1 passthrough=no \
protocol=udp
add action=mark-routing chain=prerouting comment="Policy Routing Range 2 UDP" \
disabled=no dst-port=15001-43000 new-routing-mark=RANGE2 passthrough=no \
protocol=udp
add action=mark-routing chain=prerouting comment="Policy Routing Range 3 UDP" \
disabled=no dst-port=43001-65535 new-routing-mark=RANGE3 passthrough=yes \
protocol=udp

/ip firewall nat
add action=passthrough chain=unused-hs-chain comment=\
"place hotspot rules here" disabled=yes
add action=redirect chain=dstnat comment="" disabled=yes dst-port=80 \
protocol=tcp to-ports=8080
add action=masquerade chain=srcnat comment="" disabled=no out-interface=WAN1
add action=masquerade chain=srcnat comment="" disabled=no out-interface=WAN2
add action=masquerade chain=srcnat comment="" disabled=no out-interface=WAN3

/ip hotspot service-port
set ftp disabled=no ports=21

/ip hotspot user
add comment="" disabled=no name=admin password=##### profile=default

/ip neighbor discovery
set WAN2 discover=yes
set WAN1 discover=yes
set WAN3 discover=yes
set LAN discover=yes

/ip proxy
set always-from-cache=no cache-administrator=webmaster cache-hit-dscp=4 \
cache-on-disk=no enabled=no max-cache-size=unlimited \
max-client-connections=600 max-fresh-time=3d max-server-connections=600 \
parent-proxy=0.0.0.0 parent-proxy-port=0 port=8080 serialize-connections=\
no src-address=0.0.0.0

/ip route
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
10.0.0.1,10.0.1.1,10.0.2.1 scope=30 target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.0.1 \
routing-mark=TO_WAN1 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.0.2 \
routing-mark=TO_WAN2 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.0.2 \
routing-mark=TO_WAN3 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.1.1 \
routing-mark=RANGE2 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.0.1 \
routing-mark=RANGE1 scope=30 target-scope=10
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.2.1 \
routing-mark=RANGE3 scope=30 target-scope=10

NOTE: As seen on the config I also have packet marking for QOS as per wiki example : http://wiki.mikrotik.com/wiki/Mangle%2C ... lmost_done. -- Credits to the author - it works great.

So any ideas or solutions?
Kind Regards,
Arno

MT - U guys out there?

please see this post by normis:
http://forum.mikrotik.com/viewtopic.php?f=2&t=30920