Community discussions

MikroTik App
  4. RouterOS
  5. General

Queue tree upload max-limit stops working when parent=ether1

Post Reply
 
HzMeister
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 73
Joined: Sun Jan 28, 2018 9:48 pm

Queue tree upload max-limit stops working when parent=ether1

Fri May 10, 2019 4:32 am

This morning I thought my isp was having problems because none of my client devices had connectivity even though all of my network devices reported no issues. It turns out that an iOS device was running a backup, saturating the upload and taking everything down. When I went to check why the queue tree wasn't doing it's job, I found that the max-limit of the upload parent interface wasn't adhering to the set value. After some trial and error, I realized that if I set the interface of my parent-upload queue to global it works perfectly fine, but if I set it to ether1 (WAN), the max-limit parameter stops working. Max-limit works fine for all child queues. Does anyone know what might be misconfigured to cause this issue?
Fasttrack is disabled. The rest of the config is basically the default-config with few unrelated changes(ip address, dns, ntp, etc.)

Included are two screenshots of the mangle rules and queue tree with the only change being the interface of the parent queue. As some of you might notice, the parent-download interface is also changed, but this was just for testing and it makes no difference. Download qos works fine with the parent interface set to either bridge or global.

Saturates upload completely(2.1mbps) even though max-limit=900k
Image
When it's set to global, it adheres to the set parameters
Image
Top
 
pe1chl
Forum Guru
Forum Guru
Posts: 10612
Joined: Mon Jun 08, 2015 12:09 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Fri May 10, 2019 3:06 pm

You should put the limits on the child queues, that is where they are evaluated.
In this case I would put a limit-at of 900k at the heavy-up queue and set max-limit to like 2 M everywhere.
Top
 
HzMeister
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 73
Joined: Sun Jan 28, 2018 9:48 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Fri May 10, 2019 7:00 pm

You should put the limits on the child queues, that is where they are evaluated.
In this case I would put a limit-at of 900k at the heavy-up queue and set max-limit to like 2 M everywhere.
Don't child queues borrow tokens from their parent queue? If so, that would mean that the child max-limit cannot exceed the parent max-limit and this is exactly how it works when going through global htb. Setting max-limits for each child queue would only be needed if it was less than their parent's max.
Plus, if the parent max-limit isn't adhered to, distribution of bandwidth and priorities among the child queues wouldn't work anyway.
Additionally, setting a limit-at for the heavy-upload child queue would only be useful if I wanted to guarantee 900k to it, essentially cutting the available bandwidth for other queues in half - I want queues with a higher priority to get 100% of the available bandwidth of the parent max-limit if needed which is how it's setup and works when the parent-upload is set to global.

Can you, or anyone, explain how/why the max-limit of the parent-upload queue works when it's set to global, but not ether1?
Top
 
pe1chl
Forum Guru
Forum Guru
Posts: 10612
Joined: Mon Jun 08, 2015 12:09 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Fri May 10, 2019 8:45 pm

Additionally, setting a limit-at for the heavy-upload child queue would only be useful if I wanted to guarantee 900k to it, essentially cutting the available bandwidth for other queues in half - I want queues with a higher priority to get 100% of the available bandwidth of the parent max-limit if needed which is how it's setup and works when the parent-upload is set to global.
Normally you would not want that because it would mean that any higher-priority queue can completely starve the lower ones.
I usually setup a number of different priority queues (I use 8 queues because I use "set priority from high 3 bits of DSCP" to select them) and then set a limit-at at about 1/8 of the total bandwidth for each of them and a max-limit a bit below the total bandwidth.
The parent queue gets a limit-at and max-limit just below the available bandwidth.
With this scheme it works fine: each priority gets a reasonable amount of bandwidth and the higher priorities get more if they want to and it is available.
WIth such a backup service operating at lowest priority, the normal and higher priority services still get what they need, but they are not able to completely block the backup (and maybe make it fail).
Top
 
User avatar
sebastia
Forum Guru
Forum Guru
Posts: 1782
Joined: Tue Oct 12, 2010 3:23 am
Location: Antwerp, BE

Re: Queue tree upload max-limit stops working when parent=ether1

Fri May 10, 2019 9:30 pm

Hey HzMeister

You are correct in your assumption: this is a working setup.
Clipboard01.png

Code: Select all
/queue tree
add max-limit=29M name=ext parent=bridgeExt
add name=ext20 packet-mark=20 parent=ext priority=2
add name=ext30 packet-mark=30 parent=ext priority=3
add name=extFT packet-mark=no-mark parent=ext priority=6 queue=hotspot-default

All your queues are linked to named packet marks. This means that any packet not marked wouldn't be accounted for, and pass without limit.

Add "no-mark" to one of the queues just to make sure...
Post your config for review /export hide-sensitive compact
You do not have the required permissions to view the files attached to this post.
Top
 
HzMeister
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 73
Joined: Sun Jan 28, 2018 9:48 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Sat May 11, 2019 6:02 am

Additionally, setting a limit-at for the heavy-upload child queue would only be useful if I wanted to guarantee 900k to it, essentially cutting the available bandwidth for other queues in half - I want queues with a higher priority to get 100% of the available bandwidth of the parent max-limit if needed which is how it's setup and works when the parent-upload is set to global.
Normally you would not want that because it would mean that any higher-priority queue can completely starve the lower ones.
I usually setup a number of different priority queues (I use 8 queues because I use "set priority from high 3 bits of DSCP" to select them) and then set a limit-at at about 1/8 of the total bandwidth for each of them and a max-limit a bit below the total bandwidth.
The parent queue gets a limit-at and max-limit just below the available bandwidth.
With this scheme it works fine: each priority gets a reasonable amount of bandwidth and the higher priorities get more if they want to and it is available.
WIth such a backup service operating at lowest priority, the normal and higher priority services still get what they need, but they are not able to completely block the backup (and maybe make it fail).

I guess that makes sense if you have multiple child queues that regularly hit the max-limit for a sustained period causing problems for other queues that need a CIR.
However, if you take a closer look at how it's setup, you'll see that this isn't a problem in this case.
The first two highest priority upload queues are a single voip phone(≯ 100kbps) and ack packets(≤400kbps dependent on download rate). Both of these queues can starve lower priority queues if need be so as to not drop packets - but due to the nature of their traffic, they never come close to max-limit which would cause problems for lower priority queues.
The second two queues are structured so that packets go to the heavy-upload queue only if they exceed a specified connection rate+byte threshold and would otherwise go to the lan-upload queue.

I don't think it's a problem with how the queue tree/mangle is configured. Do you have any more insight as to what else could be the problem?


Hey HzMeister

You are correct in your assumption: this is a working setup.
Clipboard01.png
Code: Select all
/queue tree
add max-limit=29M name=ext parent=bridgeExt
add name=ext20 packet-mark=20 parent=ext priority=2
add name=ext30 packet-mark=30 parent=ext priority=3
add name=extFT packet-mark=no-mark parent=ext priority=6 queue=hotspot-default
All your queues are linked to named packet marks. This means that any packet not marked wouldn't be accounted for, and pass without limit.

Add "no-mark" to one of the queues just to make sure...
Post your config for review /export hide-sensitive compact

I also thought that packets were somehow not getting marked and bypassing qos - but if you look at the screenshots, you can see that they are in fact accounted for in both the mangle rules and in the queue tree. Even if that were the case, it wouldn't explain why changing the parent interface/htb causes this behavior.

The mangle rules with the "no-mark" comment are marked that way.

After doing some more testing, it looks like this problem is only limited to ios devices. Both udp and tcp streams coming from my windows laptop adhere to the the max-limit with either ether1 or global.
Looking at the connections list, it's just a couple tcp streams from an iphone(tried two different ios devices) so I have no idea how it's possible that those packets are ignoring the max-limit only if parent-upload=ether1..

All of the aforementioned devices are connected through the same AP(not mikrotik).
I included my config for reference, but as you can see it's pretty vanilla with nothing that stands out as suspect. The router is an rb750gr3 btw.
Code: Select all
/interface bridge
add admin-mac=11:22:33:44:55:66 auto-mac=no name=bridge
/interface list
add name=WAN
add name=LAN
/ip pool
add name=lan-pool ranges=192.168.1.50-192.168.1.254
/ip dhcp-server
add address-pool=lan-pool disabled=no interface=bridge lease-time=1d name=lan-dhcp
/queue tree
add max-limit=20M name=parent-download parent=global
add max-limit=1800k name=parent-upload parent=global
add name=lan-down packet-mark=lan-down-pk parent=parent-download priority=3 queue=pcq-download-default
add name=lan-up packet-mark=lan-up-pk parent=parent-upload priority=3 queue=pcq-upload-default
add name=voip-down packet-mark=voip-down-pk parent=parent-download priority=1 queue=default
add name=voip-up packet-mark=voip-up-pk parent=parent-upload priority=1 queue=default
add name=ack-up packet-mark=ack-up-pk parent=parent-upload priority=2 queue=pcq-upload-default
add name=heavy-down packet-mark=heavy-down-pk parent=parent-download queue=pcq-download-default
add name=heavy-up packet-mark=heavy-up-pk parent=parent-upload queue=pcq-upload-default
/interface bridge port
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether4
add bridge=bridge interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add interface=bridge list=LAN
add interface=ether1 list=WAN
/ip address
add address=192.168.1.1/24 interface=bridge network=192.168.1.0
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1 use-peer-dns=no use-peer-ntp=no
/ip dhcp-server network
add address=192.168.1.0/24 gateway=192.168.1.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,1.1.1.1
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related" connection-state=established,related
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=accept chain=forward comment="defconf: accept established,related" connection-state=established,related
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=mark-packet chain=forward comment=voip-up new-packet-mark=voip-up-pk passthrough=no src-address=192.168.1.10
add action=mark-packet chain=forward comment=voip-down dst-address=192.168.1.10 new-packet-mark=voip-down-pk passthrough=no
add action=mark-packet chain=forward comment=ack-up new-packet-mark=ack-up-pk packet-size=0-123 passthrough=no protocol=tcp src-address=192.168.1.0/24 \
    tcp-flags=ack
add action=mark-packet chain=forward comment=heavy-down connection-bytes=5000000-0 connection-rate=700k-100M dst-address=192.168.1.0/24 new-packet-mark=\
    heavy-down-pk passthrough=no
add action=mark-packet chain=forward comment="lan-down no-mark" dst-address=192.168.1.0/24 new-packet-mark=lan-down-pk packet-mark=no-mark passthrough=no
add action=mark-packet chain=forward comment=heavy-up connection-bytes=500000-0 connection-rate=350k-100M new-packet-mark=heavy-up-pk passthrough=no \
    src-address=192.168.1.0/24
add action=mark-packet chain=forward comment="lan-up no-mark" new-packet-mark=lan-up-pk packet-mark=no-mark passthrough=no src-address=192.168.1.0/24
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
add action=redirect chain=dstnat comment=dns dst-port=53 in-interface=bridge protocol=udp to-ports=53
add action=redirect chain=dstnat dst-port=53 in-interface=bridge protocol=tcp to-ports=53
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=America/New_York
/system ntp client
set enabled=yes primary-ntp=216.239.35.0 secondary-ntp=216.239.35.4
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
Top
 
pe1chl
Forum Guru
Forum Guru
Posts: 10612
Joined: Mon Jun 08, 2015 12:09 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Sat May 11, 2019 12:03 pm

I don't think it's a problem with how the queue tree/mangle is configured. Do you have any more insight as to what else could be the problem?
No. You come here with a configuration that does not work, I give you a configuration of which I am sure it works, but I don't have experience with configs that leave lots of fields blank.
Maybe they should work, apparently they don't.
Top
 
HzMeister
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 73
Joined: Sun Jan 28, 2018 9:48 pm

Re: Queue tree upload max-limit stops working when parent=ether1

Sat May 11, 2019 5:36 pm

No. You come here with a configuration that does not work, I give you a configuration of which I am sure it works, but I don't have experience with configs that leave lots of fields blank.
Maybe they should work, apparently they don't.

My queue tree and mangle configuration is 100% on point. If it wasn't, it also wouldn't work in global htb...
The "configuration" you suggested might work for you, but that's not the only way to do it and it definitely isn't the the correct way for this application. As long as all the necessary fields are input, leaving fields blank isn't an issue - all it means is that you're more broadly defining how packets are handled.
I'm not looking for a way to fix the problem as I already did that - what I'm looking for is an explanation as the why the parent max-limit fails only if it's set to ether1 and an ios device is sending packets through it..?
Top
Post Reply
  4. RouterOS
  5. General