Working QoS configuration

negge · Fri Jul 10, 2009 12:14 pm

Hi everybody,

I've been struggling for some time to create a working QoS configuration, reading through countless manual pages, wikis, forum topics etc. I've noticed two things; in order to do something great with a Mikrotik router, you have to be a network guru. I thought I knew relatively much about networking when I got my router, but it seems I was wrong. Second, it seems like there's a tendency among those who actually do know something and who actually have working configurations to be pretty secretive about them, which is understandable, but very frustrating for novices. That's why I thought I'd post my QoS configuration here as well as explain what it does (or at least what I want it to do).

1. My network layout

WAN connection: 110/5 Mbit (up/down). The actual downlink speed is limited to 100 Mbit by the fact that my RouterBOARD only has 100 Mbit Ethernet ports.

My stuff is hooked up like this:
Cable modem <-- [bridge] --> Routerboard --> Switch(es) --> computers.

2. What I want to accomplish

I'm a heavy BitTorrent user, which means I'm pretty much maxing out my upstream bandwidth all the time. I also have my own FTP server which some of my friends use to download some stuff from me, a HTTP server from which stream music from my collection when I'm at other peoples places (I have my own home-made system for this) and finally I'm always connected to my computer at home using SSH.

What happens with no QoS at all is that I constantly have to limit the torrent clients upload speed everytime I come somewhere and want to stream something, or a friend needs to get something fast over FTP. It also means that unless I cap the upload speed in the torrent client I get incredible lag when using SSH to chat on IRC (for example). Of course web browsing is a PITA too when seeding at full speed...

3. The solution

The solution to all this is to mark traffic from the most critical to the non-critical. I got most of my rules from this guide. This is my current mangle config and my queue tree config. Just to clearify: "Public" is the interface connected to the modem and "Local" is the interface connected to the switch. The key seems to be to mark the ACK packets correctly (only marking the ones with a size of 0-80 bytes like most people suggest is not sufficient as the ACK flag can be set on bigger packets as well).

Mangle rules:

[xxx@xxx] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic 
 0   ;;; Link-critical traffic (ARP, DHCP)
     chain=postrouting action=mark-packet new-packet-mark=link_critical passthrough=no 
     protocol=udp out-interface=Public src-port=68 dst-port=67 

 1   ;;; Time-critical traffic (DNS, control packets)
     chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no 
     protocol=udp out-interface=Public dst-port=53 

 2   chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no 
     tcp-flags=fin,syn,rst protocol=tcp out-interface=Public 

 3   chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no 
     tcp-flags=ack protocol=tcp out-interface=Public packet-size=40-89 

 4   chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no 
     connection-state=new protocol=tcp out-interface=Public 

 5   ;;; Critical traffic
     chain=postrouting action=mark-packet new-packet-mark=critical passthrough=no 
     tcp-flags=ack protocol=tcp out-interface=Public packet-size=90-159 

 6   ;;; High-priority interactive traffic (SSH)
     chain=postrouting action=mark-packet new-packet-mark=high_pri_interactive 
     passthrough=no protocol=tcp out-interface=Public port=22,2200 

 7   chain=postrouting action=mark-packet new-packet-mark=high_pri_interactive 
     passthrough=no tcp-flags=ack protocol=tcp out-interface=Public packet-size=160-249 

 8   chain=postrouting action=mark-packet new-packet-mark=high_pri_interactive 
     passthrough=no protocol=tcp out-interface=Public port=8291 

 9   ;;; Low-priority interactive traffic (HTTP, HTTPS, DelugeWebUI)
     chain=postrouting action=mark-packet new-packet-mark=low_pri_interactive 
     passthrough=no protocol=tcp out-interface=Public port=80,443,8112 

10   chain=postrouting action=mark-packet new-packet-mark=low_pri_interactive 
     passthrough=no tcp-flags=ack protocol=tcp out-interface=Public packet-size=250-359 

11   ;;; High-priority non-interactive traffic (FTP)
     chain=postrouting action=mark-packet new-packet-mark=high_pri_non_interactive 
     passthrough=no protocol=tcp out-interface=Public connection-type=ftp 

12   chain=postrouting action=mark-packet new-packet-mark=high_pri_non_interactive 
     passthrough=no tcp-flags=ack protocol=tcp out-interface=Public packet-size=360-489 

13   ;;; Low-priority non-interactive traffic (POP, SMTP)
     chain=postrouting action=mark-packet new-packet-mark=low_pri_non_interactive 
     passthrough=no protocol=tcp out-interface=Public port=25,110 

14   chain=postrouting action=mark-packet new-packet-mark=low_pri_non_interactive 
     passthrough=no tcp-flags=ack protocol=tcp out-interface=Public packet-size=490-639 

15   ;;; Non-critical traffic (P2P)
     chain=postrouting action=mark-packet new-packet-mark=non_critical passthrough=no 
     tcp-flags=ack protocol=tcp out-interface=Public packet-size=640-809 

16   chain=postrouting action=mark-packet new-packet-mark=non_critical passthrough=no 
     protocol=tcp out-interface=Public

Queue tree:

[admin@Neggelandia] /queue tree> print
Flags: X - disabled, I - invalid 
 0   name="Link-critical" parent=Outgoing queue packet-mark=link_critical limit-at=0 
     queue=default priority=1 max-limit=4350000 burst-limit=0 burst-threshold=0 
     burst-time=0s 

 1   name="Time-critical" parent=Outgoing queue packet-mark=time_critical limit-at=0 
     queue=default priority=2 max-limit=4350000 burst-limit=0 burst-threshold=0 
     burst-time=0s 

 2   name="Critical" parent=Outgoing queue packet-mark=critical limit-at=0 queue=default 
     priority=3 max-limit=4350000 burst-limit=0 burst-threshold=0 burst-time=0s 

 3   name="High-pri interactive" parent=Outgoing queue packet-mark=high_pri_interactive 
     limit-at=0 queue=default priority=4 max-limit=4350000 burst-limit=0 
     burst-threshold=0 burst-time=0s 

 4   name="Low-pri interactive" parent=Outgoing queue packet-mark=low_pri_interactive 
     limit-at=0 queue=default priority=5 max-limit=4350000 burst-limit=0 
     burst-threshold=0 burst-time=0s 

 5   name="High-pri non-interactive" parent=Outgoing queue 
     packet-mark=high_pri_non_interactive limit-at=0 queue=default priority=6 
     max-limit=4350000 burst-limit=0 burst-threshold=0 burst-time=0s 

 6   name="Outgoing queue" parent=Public packet-mark="" limit-at=0 queue=default 
     priority=8 max-limit=4350000 burst-limit=0 burst-threshold=0 burst-time=0s 

 7   name="Low-pri non-interactive" parent=Outgoing queue 
     packet-mark=low_pri_non_interactive limit-at=0 queue=default priority=7 
     max-limit=4350000 burst-limit=0 burst-threshold=0 burst-time=0s 

 8   name="Non-critical" parent=Outgoing queue packet-mark=non_critical limit-at=0 
     queue=default priority=8 max-limit=4350000 burst-limit=0 burst-threshold=0 
     burst-time=0s

Queue types:

[xxx@xxx] /queue type> print
 0 name="default" kind=pfifo pfifo-limit=200 

 1 name="ethernet-default" kind=pfifo pfifo-limit=200

4. Result

With the current settings, if I'm seeding at full speed and somebody starts downloading something over FTP, the FTP transfer gets all upload bandwidth while the torrents almost stop. If I'm connected with SSH, there's absolutely no lag when I'm writing in the console. When surfing the web, DNS lookups happen instantly and pages load just like if I wasn't seeding at all.

Feel free to comment on my config if you have any suggestions that would make it better.

Edit: Forgot to mention that I changed the queue size to 200 packets (default is 50)

Radius · Wed Aug 19, 2009 12:21 am

seems that its working, is possible do this for download too

wan to lan qos

this is for
lan to wan qos

but p2p uploading is < 50% of free bandwith, when I start uploading something else (vpn), p2p is stil so low and vpn works 50KB (where is free bandwith +-500KB)... better will be p2p 50KB, vpn 600KB

pings are very high +- 50ms average... maybe qos download is the solution

negge · Wed Aug 19, 2009 10:08 am

You cannot apply QoS on the downlink because you have no way of controlling how much data is being sent to you. If you want to prioritize VPN just mark packets with IPsec traffic and give them higher priority in the queue.

lukkes · Thu Jan 14, 2010 12:41 am

Negge, first congratulation, it works good..

i'm trying to modify it to work in my scenario, i just want to know why you use thats packet size for ack? why you did no use 0-80 too? i've read the site that you told but they dont talk about the packets size you used, can you help me? where did you find that, thanks

JJOliver998 · Thu Jan 14, 2010 2:17 am

Hi Negge.

Thanks for this. I'm newish to MT and know little about routing. Thanks for writing down your config in this forum. Can I make a suggestion that you also write a wiki on the MT wiki page as I think that will be longer lasting. People, like you yourself mentioned check out the wiki first.

I'm glad you got it working well.

Josh

rmichael · Thu Jan 14, 2010 4:35 am

Negge, first congratulation, it works good..

i'm trying to modify it to work in my scenario, i just want to know why you use thats packet size for ack? why you did no use 0-80 too? i've read the site that you told but they dont talk about the packets size you used, can you help me? where did you find that, thanks

Quote from the link Negge reffered to (length highlighted in red):

CLASSIFY udp -- anywhere anywhere length 490:639 CLASSIFY set 1:80
CLASSIFY tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/ACK length 640:809 CLASSIFY set 1:80
CLASSIFY udp -- anywhere anywhere length 360:489 CLASSIFY set 1:70
CLASSIFY tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/ACK length 490:639 CLASSIFY set 1:70
CLASSIFY udp -- anywhere anywhere length 250:359 CLASSIFY set 1:60
CLASSIFY tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/ACK length 360:489 CLASSIFY set 1:60
CLASSIFY udp -- anywhere anywhere length 160:249 CLASSIFY set 1:50

I'm still trying to figure out why this is needed...

...hm it's actually explained in the guide as well:

The generalisation of the above is quite effective since we can assume that the larger the size of a TCP ACK (or UDP) packet, the more likely it is to contain payload data. The more data it has, the further it sinks in priority of being sent out our broadband connection first leaving all the real control ACK packets (with little or no payload) to exit as quickly as possible

EDIT: added content

PS: I think this setup makes a lot of packets arrive out of order.

netrat · Thu Jan 14, 2010 4:50 am

negge would you mind doing an export of your ip mangle and queue settings? I'd like to easily import them on a router and give your setup a try.

lukkes · Thu Jan 14, 2010 6:18 am

hmmm, ok understod, thanks, maybe i didn't read well the link, somebody know if i's really needed another lenght more tha 0-80? with the others lenght it catch much traffic, including p2p traffic
thanks

negge · Thu Jan 14, 2010 8:42 am

First of all, it's nice to see that it's working for other people too! It's been some time since I wrote this but the configuration has remained the same since then, so here it is:

/ip firewall mangle
add action=mark-packet chain=postrouting comment="Link-critical traffic (DHCP)" disabled=no dst-port=67 new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=udp src-port=68
add action=mark-packet chain=postrouting comment="IPSec VPN (same priority as link critical)" disabled=no new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=ipsec-esp
add action=mark-packet chain=postrouting comment="Time-critical traffic (DNS, TCP control packets, certain ACK packets, new connections)" disabled=no dst-port=53 new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp tcp-flags=fin,syn,rst
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN packet-size=40-89 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" connection-state=new disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="Critical traffic (just some ACK packets)" disabled=no new-packet-mark=critical out-interface=WAN packet-size=90-159 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority interactive traffic (SSH, WinBox, certain ACK packets)" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=22,2200 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=8291 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN packet-size=160-249 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority interactive traffic (HTTP, HTTPS, DelugeWebUI)" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN passthrough=no port=80,443,8112 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN packet-size=250-359 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority non-interactive traffic (FTP)" connection-type=ftp disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN packet-size=360-489 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority non-interactive traffic (POP, SMTP)" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN passthrough=no port=25,110 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN packet-size=490-639 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Non-critical traffic (P2P)" disabled=no new-packet-mark=non_critical out-interface=WAN packet-size=640-809 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=non_critical out-interface=WAN passthrough=no protocol=tcp

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Outgoing queue" packet-mark="" parent=WAN priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri non-interactive" packet-mark=low_pri_non_interactive parent="Outgoing queue" priority=7 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Non-critical packet-mark=non_critical parent="Outgoing queue" priority=8 queue=non_critical_queue
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Link-critical packet-mark=link_critical parent="Outgoing queue" priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Time-critical packet-mark=time_critical parent="Outgoing queue" priority=2 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Critical packet-mark=critical parent="Outgoing queue" priority=3 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri interactive" packet-mark=high_pri_interactive parent="Outgoing queue" priority=4 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri interactive" packet-mark=low_pri_interactive parent="Outgoing queue" priority=5 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri non-interactive" packet-mark=high_pri_non_interactive parent="Outgoing queue" priority=6 queue=default

You may need to edit the commands to reflect your own interface names. I don't use "Public" and "Local" anymore like I did in my first post, nowadays it's simply "WAN" and "LAN".

i'm trying to modify it to work in my scenario, i just want to know why you use thats packet size for ack? why you did no use 0-80 too? i've read the site that you told but they dont talk about the packets size you used, can you help me? where did you find that, thanks

The reason why I only mangle ACK packets with size between 40-89 and not 0-89 is because a TCP packet can't be smaller than 40 bytes.

Thanks for this. I'm newish to MT and know little about routing. Thanks for writing down your config in this forum. Can I make a suggestion that you also write a wiki on the MT wiki page as I think that will be longer lasting. People, like you yourself mentioned check out the wiki first.

I might just do that, if there's enough interest and if I find some spare time.

PS: I think this setup makes a lot of packets arrive out of order.

Why would it do that? This is TCP.

Pilgrim · Thu Jan 14, 2010 11:25 am

Thanks Negge, Great piece of work. Just want to add a question concerning your comments on the download.

You cannot apply QoS on the downlink because you have no way of controlling how much data is being sent to you. If you want to prioritize VPN just mark packets with IPsec traffic and give them higher priority in the queue.

I have seen many examples on prioritizing downloads and have also set up a QoS for my download. Maybe I misunderstood you, but did you mean to say that it is not possible to prioritize downloads. I am just wondering if your rules could not be applied to download also - or would it spoil / overlap the purpose "Maximise Download Speed via Outbound Traffic Shaping"?

rgs Pilgrim

rmichael · Thu Jan 14, 2010 12:39 pm

Quote:
PS: I think this setup makes a lot of packets arrive out of order.

Why would it do that? This is TCP.

Packets will eventually be reordered since -as you point out it's TCP -however during transmission dupack will be sent and in some cases (depending on dupack limit) can trigger retransmission. I don't know how badly this would affect the speed though. Myself I try not to rearange packets within the same stream.

negge · Fri Jan 15, 2010 7:30 am

@Pilgrim: What I meant to say is you can't directly shape your incoming connections, but you can get the same effect by prioritizing certain ACK packets instead. That's basically what my configuration does. If all the ACK rules wouldn't be there, the QoS would only work on a server where no one rarely downloads anything at all.

Pilgrim · Fri Jan 15, 2010 12:32 pm

Thanks negge, That is really interesting. I also read the article and to me this is completely new angle on QoS.

Great job in implementing this on the mikrotik platform and like it was already mentioned it would be really nice if this could be turned into an article on the wiki.

rgs Pilgrim

negge · Fri Jan 15, 2010 12:52 pm

I'll see if I can put together a decent wiki page in the next couple of weeks!

phix · Sat Jan 16, 2010 12:20 pm

Quote:
PS: I think this setup makes a lot of packets arrive out of order.

Why would it do that? This is TCP.
Packets will eventually be reordered since -as you point out it's TCP -however during transmission dupack will be sent and in some cases (depending on dupack limit) can trigger retransmission. I don't know how badly this would affect the speed though. Myself I try not to rearange packets within the same stream.

Hi Rmichael,

I'm Mark, I wrote the HOWTO that Negge has based his MikroTek configuration on. I just wanted to make a quick comment about your statement here (I have updated my HOWTO accordingly).

I agree with your statement about dupACKs, but, only where TCP packets have not been classified into a particular queue based on something consitent like a client or peer port number. In this instance, they will be classified on size instead and the shaping rules I've described do this in a banded way based on size. Again, in this instance, the only possible way out of ordered packets could then occur is if BOTH the client and peer are sending payload (data) ACK packets to each other, within the same TCP session. When I wrote that HOWTO (some years ago now!), I was observing how BitTorrent exchanges data across TCP. Most of the time, I observed that BitTorrent TCP streams could be classified into one of two forms:

1) TCP sessions between you and peer where YOU are sending large payload ACKs out, peers sending small ACKs back
2) TCP session between you and peer where THEY are sending large payload ACKs in, you sending small ACKs back

If either of the above hold true, the chances of out of ordered packets reaching you or your peers are fairly remote. If however, you and your peers are doing BOTH of the above, within the SAME TCP session, then yes, out of ordered packets could occur.

BitTorrent has since moved onto uTP anyway (which uses UDP) - if anyone has any further information about this protocol, such as its frame format, it is probably very possible to use iptables to match on particular bytes within such packets to determine if they are uTP, and thus place such traffic in a low priority class, rather than use the crude method of size. This would be preferable.

I'm glad everyone here has found Negge's configuration and my HOWTO useful - when I originally came up with the ideas in it, it dramatically improved the usability and speed of my connection when I had a 1Mbit broadband line (things have now moved on though, now I am on 50Mbit)

negge · Tue Apr 06, 2010 10:15 pm

I'm sorry I haven't put together the wiki article yet, I simply haven't had that much spare time lately. Don't give up the hope though, it will come sooner or later!

phix · Sun Jul 18, 2010 11:36 am

Just a heads up for anyone using these rules. Due to DNSSEC being switched on in the root name servers, with the rest of the world following shortly:

http://tech.slashdot.org/story/10/07/17 ... SEC-Signed

there is a legacy DNS querk where if a reply from a DNS server exceeds 512 bytes (which is now the case from the root servers), the protocol switches to using TCP. So you'll want to also include TCP destination port 53 traffic in these classification rules as well as UDP (just repeat the UDP port 53 rule but for TCP instead). If you miss this out, it -could- affect the speed at which you resolve host names, and hence will affect the speed at which you can browse.

Although, this issue does not appear to have affected me because I use the 'unbound' ( http://www.unbound.net/ ) resolver daemon which, upon initial checks, ignores this limit and continues to use UDP regardless.

SeaburyNorton · Sat Oct 16, 2010 9:12 pm

First of all, it's nice to see that it's working for other people too! It's been some time since I wrote this but the configuration has remained the same since then, so here it is:

/ip firewall mangle
add action=mark-packet chain=postrouting comment="Link-critical traffic (DHCP)" disabled=no dst-port=67 new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=udp src-port=68
add action=mark-packet chain=postrouting comment="IPSec VPN (same priority as link critical)" disabled=no new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=ipsec-esp
add action=mark-packet chain=postrouting comment="Time-critical traffic (DNS, TCP control packets, certain ACK packets, new connections)" disabled=no dst-port=53 new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp tcp-flags=fin,syn,rst
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN packet-size=40-89 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" connection-state=new disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="Critical traffic (just some ACK packets)" disabled=no new-packet-mark=critical out-interface=WAN packet-size=90-159 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority interactive traffic (SSH, WinBox, certain ACK packets)" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=22,2200 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=8291 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN packet-size=160-249 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority interactive traffic (HTTP, HTTPS, DelugeWebUI)" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN passthrough=no port=80,443,8112 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN packet-size=250-359 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority non-interactive traffic (FTP)" connection-type=ftp disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN packet-size=360-489 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority non-interactive traffic (POP, SMTP)" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN passthrough=no port=25,110 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN packet-size=490-639 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Non-critical traffic (P2P)" disabled=no new-packet-mark=non_critical out-interface=WAN packet-size=640-809 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=non_critical out-interface=WAN passthrough=no protocol=tcp

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Outgoing queue" packet-mark="" parent=WAN priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri non-interactive" packet-mark=low_pri_non_interactive parent="Outgoing queue" priority=7 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Non-critical packet-mark=non_critical parent="Outgoing queue" priority=8 queue=non_critical_queue
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Link-critical packet-mark=link_critical parent="Outgoing queue" priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Time-critical packet-mark=time_critical parent="Outgoing queue" priority=2 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Critical packet-mark=critical parent="Outgoing queue" priority=3 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri interactive" packet-mark=high_pri_interactive parent="Outgoing queue" priority=4 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri interactive" packet-mark=low_pri_interactive parent="Outgoing queue" priority=5 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri non-interactive" packet-mark=high_pri_non_interactive parent="Outgoing queue" priority=6 queue=default
You may need to edit the commands to reflect your own interface names. I don't use "Public" and "Local" anymore like I did in my first post, nowadays it's simply "WAN" and "LAN".

i'm trying to modify it to work in my scenario, i just want to know why you use thats packet size for ack? why you did no use 0-80 too? i've read the site that you told but they dont talk about the packets size you used, can you help me? where did you find that, thanks
The reason why I only mangle ACK packets with size between 40-89 and not 0-89 is because a TCP packet can't be smaller than 40 bytes.

Thanks for this. I'm newish to MT and know little about routing. Thanks for writing down your config in this forum. Can I make a suggestion that you also write a wiki on the MT wiki page as I think that will be longer lasting. People, like you yourself mentioned check out the wiki first.
I might just do that, if there's enough interest and if I find some spare time.

PS: I think this setup makes a lot of packets arrive out of order.
Why would it do that? This is TCP.

Thank you for this QoS scheme! It seems to work great! I have one question, however...

Your rules limit all uploads to 2M. You mentioned in your first post that you were using 5up/110down. In your first design you set all limits to 4350000, which is about 85% of 5M (normal for QoS setups). The second, more recent script limits everything to 2M... is that an error? I too am on a 5M UP pipe (but 30M down) and think the 4350000 should be the proper setting... is there something I'm missing?

Thank you!

SN

Zapnologica · Sun Oct 17, 2010 11:08 am

THANK YOU SOOO MUCH,

I have been looking for a QOS setup for ages as i run VOIP on my network.

I will try it this afternoon,

kameelperdza · Tue Aug 02, 2011 3:38 pm

First of all, it's nice to see that it's working for other people too! It's been some time since I wrote this but the configuration has remained the same since then, so here it is:

/ip firewall mangle
add action=mark-packet chain=postrouting comment="Link-critical traffic (DHCP)" disabled=no dst-port=67 new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=udp src-port=68
add action=mark-packet chain=postrouting comment="IPSec VPN (same priority as link critical)" disabled=no new-packet-mark=link_critical out-interface=WAN passthrough=no protocol=ipsec-esp
add action=mark-packet chain=postrouting comment="Time-critical traffic (DNS, TCP control packets, certain ACK packets, new connections)" disabled=no dst-port=53 new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=udp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp tcp-flags=fin,syn,rst
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=time_critical out-interface=WAN packet-size=40-89 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" connection-state=new disabled=no new-packet-mark=time_critical out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="Critical traffic (just some ACK packets)" disabled=no new-packet-mark=critical out-interface=WAN packet-size=90-159 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority interactive traffic (SSH, WinBox, certain ACK packets)" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=22,2200 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN passthrough=no port=8291 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_interactive out-interface=WAN packet-size=160-249 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority interactive traffic (HTTP, HTTPS, DelugeWebUI)" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN passthrough=no port=80,443,8112 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_interactive out-interface=WAN packet-size=250-359 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="High-priority non-interactive traffic (FTP)" connection-type=ftp disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN passthrough=no protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=high_pri_non_interactive out-interface=WAN packet-size=360-489 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Low-priority non-interactive traffic (POP, SMTP)" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN passthrough=no port=25,110 protocol=tcp
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=low_pri_non_interactive out-interface=WAN packet-size=490-639 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="Non-critical traffic (P2P)" disabled=no new-packet-mark=non_critical out-interface=WAN packet-size=640-809 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment="" disabled=no new-packet-mark=non_critical out-interface=WAN passthrough=no protocol=tcp

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Outgoing queue" packet-mark="" parent=WAN priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri non-interactive" packet-mark=low_pri_non_interactive parent="Outgoing queue" priority=7 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Non-critical packet-mark=non_critical parent="Outgoing queue" priority=8 queue=non_critical_queue
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Link-critical packet-mark=link_critical parent="Outgoing queue" priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Time-critical packet-mark=time_critical parent="Outgoing queue" priority=2 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name=Critical packet-mark=critical parent="Outgoing queue" priority=3 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri interactive" packet-mark=high_pri_interactive parent="Outgoing queue" priority=4 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="Low-pri interactive" packet-mark=low_pri_interactive parent="Outgoing queue" priority=5 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=2M name="High-pri non-interactive" packet-mark=high_pri_non_interactive parent="Outgoing queue" priority=6 queue=default
You may need to edit the commands to reflect your own interface names. I don't use "Public" and "Local" anymore like I did in my first post, nowadays it's simply "WAN" and "LAN".

i'm trying to modify it to work in my scenario, i just want to know why you use thats packet size for ack? why you did no use 0-80 too? i've read the site that you told but they dont talk about the packets size you used, can you help me? where did you find that, thanks
The reason why I only mangle ACK packets with size between 40-89 and not 0-89 is because a TCP packet can't be smaller than 40 bytes.

Thanks for this. I'm newish to MT and know little about routing. Thanks for writing down your config in this forum. Can I make a suggestion that you also write a wiki on the MT wiki page as I think that will be longer lasting. People, like you yourself mentioned check out the wiki first.
I might just do that, if there's enough interest and if I find some spare time.

PS: I think this setup makes a lot of packets arrive out of order.
Why would it do that? This is TCP.

Hi negge thanx for this configuration. Can you please tell me the queue type values that you have used? I had to create the non_critical_queue but does not know what it should be.

Please give me the values you have used for.....non_critical_queue and default.

thank you

negge · Tue Aug 02, 2011 3:43 pm

Nowadays I just use the "default" type for all queues, however I experimented with bigger queue sizes before that's why the configs show a custom queue type. I've tried sizes ranging from a couple hundred packets to 10 000 without much change in behavior (except that the queue size goes up of course), but some protocols might start behaving weirdly if you make the queue size too big.

kameelperdza · Fri Aug 05, 2011 9:04 am

so non_critical_queue value should be more or less the value of default queue?

negge · Fri Aug 05, 2011 9:45 am

so non_critical_queue value should be more or less the value of default queue?

Yes.

kameelperdza · Thu Aug 11, 2011 10:50 pm

I have noticed that if i make bridge1 the parent in queue tree then the queues does not work.

Devices on bridge1 is ether1 ,ether3, eoip-tunnel1 ,eoip-tunnel2 ,eoip-tunnel3 (Bridge1 ip address=192.168.0.0/24)
Devices on bridge2 is ether2 and ether4 (Bridge2 ip address=172.16.0.0/24)

I did some test and if i add eoip-tunnel3 as parent in queue tree then it works 100%.
Does this mean i should create mangle rules and queue trees for each device on my router?

EDIT:

PS:I managed to get it working by putting bridge ports in "out-bridge-port" instead of "out-interface".

shadowskippie · Thu Sep 08, 2011 2:57 pm

I love this, when i can i want to try it out at home where i have one ADSL connection
Now i also want to implement it at my work where we have four ADSL connections and a default route point to all of them, how would i got about using you QoS design where i have four possible ADSL connections to work with

shadowskippie · Fri Sep 09, 2011 7:49 pm

okay, it seems i need a little help

my RB750 is in a bridged setup, i have setup the three of the rules and find they are not working correctly

Flags: X - disabled, I - invalid, D - dynamic

1 ;;; Time critical traffic
chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no protocol=udp out-interface=bridge1 dst-port=53
out-bridge-port=ADSL

2 chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no protocol=tcp out-interface=bridge1 dst-port=53

3 chain=postrouting action=mark-packet new-packet-mark=time_critical passthrough=no tcp-flags=fin,syn,rst protocol=tcp
out-interface=bridge1 out-bridge-port=ADSL

i can see rule 1 working when i have bridge set as the out int but only like that, if i set the advance section with out-bridge-port=ADSL as you can see it stops marking packets.
rule 3 just doesn't mark, period. where am i going wrong

edmidor · Thu Mar 01, 2012 7:04 pm

How this config should be modified to give priority to packet order critical UDP connections, i.e. VoIP?

dgonzalezh · Mon Jun 17, 2013 8:47 am

Hey there,

Great post I'd like to know if that'd apply to my personal setup, I've setup my MK with the PCC routing guide to load balance my two WANs, but I'd like to use one of them just to upload, outgoing Dropbox and FTP out stuff as it's way faster than the other connection I'd love to use these rules as they look promising for what I'm looking for.

Thanks and sorry for hijacking your thread.

Cheers.

Hi everybody,

I've been struggling for some time to create a working QoS configuration, reading through countless manual pages, wikis, forum topics etc. I've noticed two things; in order to do something great with a Mikrotik router, you have to be a network guru. I thought I knew relatively much about networking when I got my router, but it seems I was wrong. Second, it seems like there's a tendency among those who actually do know something and who actually have working configurations to be pretty secretive about them, which is understandable, but very frustrating for novices. That's why I thought I'd post my QoS configuration here as well as explain what it does (or at least what I want it to do).

[snip]

Feel free to comment on my config if you have any suggestions that would make it better.

Edit: Forgot to mention that I changed the queue size to 200 packets (default is 50)

Mon Jun 17, 2013 10:58 am

Quote:
PS: I think this setup makes a lot of packets arrive out of order.

Why would it do that? This is TCP.

Interesting tread but i noticed that it went a little bit in the wrong direction!

Main thing - Priority and prioritization have nothing to do with packet order, it doesn't put one packet before other.
Think of priority as "face control" guy on club entrance - he is standing in the doors and filter out those that he doesn't wan't in the club, order between those that got in remains the same.

There can be some out-of-order, but only in case you have different queue types and queue sizes in different subqueues.

If we are talking about small TCP ACK priority - it makes sense cause those packets come as acknowledgment to successful delivery of big packet, if something happens to this small ACK it will result in 2nd transmit of that big packet - that would be waste of bandwidth and increase of latency.

negge · Mon Jun 17, 2013 11:19 am

Since I started this thread I have changed my configuration to a much simpler one. The main idea is still the same, I just stopped mangling the TCP ACK packets since it seems to be very hard to get that bit right. Simply prioritizing the ports you want seems to work way better, for example I can max out my upload speed while still getting the same ping latency as I would if the line was completely unused.

kraker · Wed Jun 19, 2013 12:25 pm

please share your new qos config..

thanks...

negge · Wed Jun 19, 2013 12:50 pm

please share your new qos config..

thanks...

It's exactly the same as the one in the original post except the TCP ACK packets aren't marked during mangling.

pcunite · Sat Aug 31, 2013 5:44 pm

negge,

Thank you for this post, I had created my own thread on VoIP Qos'ing linked here before I found your topic. I'm still learning. Any thoughts? I'm using mark-connection to lessen the load on the CPU.

It's exactly the same as the one in the original post except the TCP ACK packets aren't marked during mangling.

So, it looks like this?

ros code

# Note that ... disabled=no passthrough=no ... is set on all, removed for brevity
/ip firewall mangle
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=link_critical            protocol=udp dst-port=67 src-port=68
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=link_critical            protocol=ipsec-esp
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=time_critical            protocol=udp dst-port=53
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=time_critical            protocol=tcp tcp-flags=fin,syn,rst
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=time_critical            protocol=tcp connection-state=new
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=high_pri_interactive     protocol=tcp port=22,2200
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=high_pri_interactive     protocol=tcp port=8291
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=low_pri_interactive      protocol=tcp port=80,443,8112
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=high_pri_non_interactive protocol=tcp connection-type=ftp
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=low_pri_non_interactive  protocol=tcp port=25,110
add action=mark-packet chain=postrouting out-interface=WAN new-packet-mark=non_critical             protocol=tcp

# Note that ... burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 disabled=no queue=default ... is set on all, removed for brevity
/queue tree
add max-limit=2M name="Outgoing queue"           packet-mark=""                       parent=WAN              priority=8
add max-limit=2M name="Link-critical"            packet-mark=link_critical            parent="Outgoing queue" priority=1 
add max-limit=2M name="Time-critical"            packet-mark=time_critical            parent="Outgoing queue" priority=2
add max-limit=2M name="High-pri interactive"     packet-mark=high_pri_interactive     parent="Outgoing queue" priority=4
add max-limit=2M name="Low-pri interactive"      packet-mark=low_pri_interactive      parent="Outgoing queue" priority=5
add max-limit=2M name="High-pri non-interactive" packet-mark=high_pri_non_interactive parent="Outgoing queue" priority=6
add max-limit=2M name="Low-pri non-interactive"  packet-mark=low_pri_non_interactive  parent="Outgoing queue" priority=7
add max-limit=2M name="Non-critical"             packet-mark=non_critical             parent="Outgoing queue" priority=8

dunga · Fri Sep 13, 2013 2:57 pm

Thanks Ngee,
For your configurations. Plz my network is not like your own but differently. My setup is just like this. WAN ------> Lan(Bridged: port 2 - 10).-------->Switch -------(AP). My uplink is 1meg/download is 2meg.

My clients are on hotspot just a user requires much speed, how do i about to use your setup to achieve a better QOS for my network, so that i can achieve the best from what i am being given.

Thanks

AaronG · Fri Nov 15, 2013 1:27 am

Thanks Ngee,
For your configurations. Plz my network is not like your own but differently. My setup is just like this. WAN ------> Lan(Bridged: port 2 - 10).-------->Switch -------(AP). My uplink is 1meg/download is 2meg.

My clients are on hotspot just a user requires much speed, how do i about to use your setup to achieve a better QOS for my network, so that i can achieve the best from what i am being given.

Thanks

This is traffic prioritization, not really rate limiting. The only reason that the max bandwidth is specified is to make the mikrotik "aware" of it's bandwidth budget. It's preferable for the mikrotik to queue traffic rather than have it discarded by an overtaxed uplink.

If you want to implement rate limitations, you should look into PCQ.

http://wiki.mikrotik.com/wiki/Manual:Qu ... Q_Examples

One really cool thing about setting up a PCQ is that it can work in tandem with IPprec (prioritization) like you see here, and you can specify your overall bandwidth budget in the parent queue.

gulliverrr · Sun Aug 03, 2014 9:46 pm

Hello everybody!

I've also been looking for ages to setup my home network like that as torrents have been my main pita requiring continuing attention if I want to upload as much as I can.
My setup is a bit different so the above instructions do not work on me. I have a 5-port RB951-2n in the following setup:

Cable Modem <===> [MT port 1]
                  [MT port 2] <===> [Switch] <===> Torrent Machine
                  [MT port 3] <===> PC#1 
                  [MT port 4] <===> PC#2

The problem is that I have setup MT ports 1-5 to belong to a local bridge to simplify my routing so I dont think routing works the same way as negge's case.

Can someone point me to the right direction about modifying these commands to apply QoS based on IP (all my machines have static IP) or interface (everybody get default high priority apart from interface 2 which gets low)

Any help will be very much appreciated as I am lost in so many related posts around the forum.

MosquitoCR · Thu Feb 05, 2015 8:17 am

Hello Negge,

Can you share your latest version of your QOS ?
I have MK OS 6.26.

Please let me know.

Thanks.

vamose · Tue Mar 03, 2015 8:02 pm

Guys,

I bought the Mikrotik router and used AP and PPOE settings to enable the router and my ISP provider.

I have 10MBs service, but when I use speediest websites such as testmy.net net performance maxes out at 3.2MBps.

I need your help with streamlining my configuration and with QOS. The requirement for QOS is as per the priority listed, can any one help me.

VOIP
Streaming video content - You tube
SSH
FTP
Downloading
Torrent downloads.

shadowskippie · Tue Mar 03, 2015 9:21 pm

The problem may not be your qos. When you run the speed check your cpu. Lets start by making sure you're not maxing it out

vamose · Wed Mar 04, 2015 4:28 am

shadowskippie, please suggest how to do you check my CPU (which CPU router or laptop) is not maxing out and how do I check it? I am completely new to Mikrotik router, hence please do share the commands to use.

Ok, I tested the speed:

- Mikrotik Router (Login -> system -> resources) - Maximum CPU load 26%

- The result from testmy.net - DL 6 Mbps 754 kB/s, Upload 1.7 Mbps 216 kB/s - (no other application, except speed test website)

- Laptop CPU load - below 5%

My ISP provider:
- 10MB of download speed
- 2Mbps of upload speed

shadowskippie · Wed Mar 04, 2015 5:21 am

Check your private mails. I'm going to move this conversation there as I'm not sure if its a qos convo or not

freemannnn · Tue Aug 04, 2015 7:44 pm

very interesting post. i am testing it now.

(TCP ACK packets aren't marked during mangling.)
1. please can u post the latest qos configuration without the tcp ack lines you said above!

2. what is your non_critical_queue value? default-small is ok for using?

vasmik · Wed Mar 22, 2017 3:32 pm

sorry for my stupid question, but i need to have rate limit on my server on seperate interface , for download reason. if i have my sever on my lan second interface, it is work. on secondary lan didnt work, also the mikrotik didnt see any traffic. always the icon in front the rule it is green. without any details on traffic section

thanks again!

Quared · Wed Mar 22, 2017 3:57 pm

Hello,

second interface ? secondary lan ?

Please provide some more information or a quick sketch about your setup and/or post your config

From terminal

/export hide-sensitive

greets

vasmik · Wed Mar 22, 2017 4:07 pm

i just find a a solution!

thanks!!!

ros code

Who is online