And the Hex refresh is roughly double of that, 498.1:
https://mikrotik.com/product/hex_2024#fndtn-testresults
still half the speed you want/need.

You need either (looking ahead) a RB5009 at around 3 Gb, $219:
https://mikrotik.com/product/rb5009ug_s ... estresults

or a hap Ax3 (as router), 1145.2, $139:
https://mikrotik.com/product/hap_ax3#fndtn-testresults

An Ax2, if you want to save a few bucks, is almost there, 912.9, $ 99:
https://mikrotik.com/product/hap_ax2#fndtn-testresults

CPU in hEX Gr3 is not exactly speed monster. It's got 2 CPU cores (with 4 threads altogether but I don't know how ROS utilizes that). And the gotcha: all packets of same connection are handled by same CPU core/thread (processing may move between cores, but there's no parallel processing). And windows file transfers are single connection (AFAIK MS went ahead implementing multi-connection SMB transfers but it's a new feature and not sll servers and clients support that), so all traffic hits same CPU core. Similarly single HTTP/HTTPS file download or (plain old) FTP and many more. Torrents are not affected (that much) as torrent clients communicate with many peers and processing of those connections is nicely spread over all CPU cores, but each individual connection is still affected.
And that's what you're seeing.

And a side remark: using PCC/mangle does not entirely disqualify fasttrack ... just the fasttrack rule (which marks connections for fasttracking) has to be adjusted so that it only affects connections which don't need mangling. Or create accept rule(s) which deal with connections requiring mangling and place it/them above fasttrack rule. I'm not saying that in your particular case this would help though.

I was just looking at the hap lite tc test specifications. Its speed is very close to hex. It's really stupid. Hex has a 2-core, 2-thread processor, but hap lite has a single core with a low frequency!

with Fasttrack you can get Full Speed with the 750GR3
and with 7.18beta this is also working with IPv6

with Fasttrack you can get Full Speed with the 750GR3
and with 7.18beta this is also working with IPv6

if you really want significantly better performance you must go to next tier of MikroTIk devices like RB5009UG+S+IN
i know is more expensive but It is what it is

a halfway option is hAP ax²

both options offer very good price to performance ratio

with Fasttrack you can get Full Speed with the 750GR3
and with 7.18beta this is also working with IPv6

really? 7.18beta pcc work with fasttrack?

really? 7.18beta pcc work with fasttrack?

No I am saying since Vers7, ECMP is now automatically applied and is actually a more favourable load balancing approach IMHO.
It automatically provides load balancing when one has multiple WANs and one does not designate any distance difference between the WANs.
The real power is if one of the WANS lets say in a four ISP setup, is not available, ECMP will automatically distribute the load between the three other WANS.
To approximate the same thing in PCC is way more complex in comparison.

Where PCC comes into play is if there is much disparity between the ISPs in terms of throughput as the ECMP will not favour the larger throughput WAN and thus the available bandwidth will not be used in a reasonable proportion. With PCC you can modify the number of times in a cycle, the router goes to a specific WAN ( to ensure a larger bandwith ISP) sees more sessions compared to the lesser bandwidth ISPs. There is also some issues with banks needed to see sessions from the same ISP from a user etc........ Although it may be less of an issue now than before.

One can achieve better load balancing granularity by bandwidth based load balancing but that is a step more complex than PCC........
As per Tomas - A sticky connection
• A sticky connection is a connection, that once
established through one interface, will always go out
that exact interface.
• This is required, because when we switch to a second
link, we only need to switch new connections.
• In PCC, this is done automatically. Using bandwidth based load balancing
however, this has to be done manually which implies a second layer of mangles

One then uses traffic monitoring to trigger an action above a certain traffic threshold --> move sessions to next WAN ( via changing the routing table by changing routing mark )
Similarly one uses traffic monitoring to trigger an action below a certain traffic threshold ---> move sessions back to this WAN ( via changing the routing table by changing routing mark)

@MKX for the version 7 ECMP it uses L3 hash policy as depicted below.
Can you explain these further??

What do you suggest now? What should I do? I don't want to replace the hex, I want to fix the problem.
I consider this problem a cat and mouse game on the part of Mikrotik to buy a new device.

What do you suggest now? What should I do? I don't want to replace the hex, I want to fix the problem.
I consider this problem a cat and mouse game on the part of Mikrotik to buy a new device.

Well the problem is not understanding the specifications available on the product pages or coming here to ask for assistance prior to purchasing.
In either case there is no way around the limits of using the routers with firewall rules in place to 1gig or close to your ISP without acquiring the correct product.
In your case the best approach is the hapax3, if budget is the issue.
The hex you have makes an excellent managed switch you can use on your network and/or as a backup router and/or travel router................

ECMP works very well in ROS7, works with fasttrack enabled and applicable to the connections using ECMP, and since 7.16 you can use the L4 mode (for both IPv4 and IPv6), and in my tests with L4, you achieve the same results as with PCC both-address-and-ports (when the PCC remainder distribution is equal, which means you don't give more weight to a particular outgoing route). Before 7.16 the only choice was L3 and it worked like PCC both-addresses. When using IPv6 you'll need srcnat netmap (NAT66) rules to change the prefixes to the correct one for each outgoing interfaces. Other than that, it also works very well with IPv6.

For issue such as Incoming Wireguard connections, or generic port forwarding (dstnat), I've found that the following config simplified a lot, and most of the time mangle rules are no longer needed, only routing rules:

* In the main routing table, keep the routes with increasing distance (failover mode, NOT ECMP). We'll let Wireguard and the port forwarded (dstnat) connections use this main table and not being load balanced (only use the main route).

* Create an additional routing table named ECMP. In this table we add all the ECMP routes (with the same distance).

* For the Routing Rules table:

- First rule at the top (with this, connections between the router's subnets will use the main table):

Code: Select all

/routing rule 
add action=lookup min-prefix=0 table=main

- Followed by rules excluding dstnat hosts from ECMP by making them use the main routing table, as well as any other hosts/subnets that should skip ECMP:

Code: Select all

/routing rule 
add action=lookup dst-address=0.0.0.0/0 src-address=a.b.c.d/32 table=main
add action=lookup dst-address=0.0.0.0/0 src-address=e.f.g.h/24 table=main
...

- And the rules that tell the rest to use the ECMP table:

Code: Select all

/routing rule 
add action=lookup dst-address=0.0.0.0/0 src-address=192.168.0.0/16 table=ECMP
add action=lookup dst-address=0.0.0.0/0 src-address=172.16.0.0/12 table=ECMP
add action=lookup dst-address=0.0.0.0/0 src-address=10.0.0.0/8 table=ECMP
add action=lookup dst-address=2000::/3 src-address=2000::/3 table=ECMP

If you need more complex exclusion (from ECMP) conditions than a few exclusion addresses, you can ressort to mangle (because mangle has priority over routing rules). Create a separate routing table, WAN1_ONLY for example, with only one default route. Add rules to the Mangle table to mark-connection & mark-routing for specific conditions for WAN1_ONLY to be used. Then add connection-mark=no-mark to the fasttrack rules. That way most of the outgoing traffic will still use ECMP and fasttrack.

The hEX RB750Gr3 with fasttrack can now with 7.18beta2 achieve over 900Mbps (919Mbps in my tests) download on speedtest.net using IPv6 test servers too. Attached is the result before and after enabling fasttrack for IPv6 in 7.18beta2 on the old hEX.

When downloading something from a single server/host, even with a download manager softwares the IP addresses on both ends of the connections are the same. To have the connections distributed over the multiple outgoing WAN links, you must ensure that IPv4 Multipath Hash Policy (under IP -> IP Settings) is set to L4 (there is a similar setting for IPv4 under IPv6 -> Settings too). With the default setting (L3) only the IP addresses are used to calculate the hashes, which means those connections all have the same hash and will only use one line. With L4, the port numbers will also be used in the calculation and the different connections to the same host will have different hash values. This is similar to the PCC modes both-addresses and both-addresses-and-ports.

As for fasttrack: at the top of the IP -> Firewall -> Filter table there is a dummy rule created by fasttrack (with the comment "special dummy rule to show the fasttrack counters"). Select that rule and click "Reset Counters", then run you speed tests. Do you see that counter rapidly increasing during the tests and counting most of the speed test traffic? If not, there might be something with your firewall configuration and/or bridge configuration. If possible, make an export of your configuration with:

Code: Select all

/export hide-sensitive file=config

Download config.rsc to your computer, use a plaintext editor to censor things like MAC addresses, keys, usernames, public IP addresses, etc... (as well the sensitive data in the scripts, if any). Then post the censored content here maybe?