QoS setup

quickgen · Sat Sep 22, 2007 2:58 pm

Dear all,
I have 2 ISPs and 300 clients. I am using mangle to separate local and overseas traffic for each client, because we assign different rates for each kind of traffic. So for one client i have in Mangle 4 rules:
1. Download Overseas Traffic
2. Upload Overseas Traffic
3. Download Local Traffic
4. Upload Local Traffic

In Qtree I shape each kind of mangle rule, so again I have 4 shapers.

My Mikrotik setup is as follows:

Mangle

8 ;;; 192.168.0.2
chain=forward dst-address=192.168.0.2 src-address-list=!local action=mark-packet new-packet-mark=192.168.0.2-overseas-dl passthrough=no

9 chain=forward src-address=192.168.0.2 dst-address-list=!local action=mark-packet new-packet-mark=192.168.0.2-overseas-ul passthrough=no

10 chain=forward dst-address=192.168.0.2 src-address-list=local action=mark-packet new-packet-mark=192.168.0.2-local-dl passthrough=no

11 chain=forward src-address=192.168.0.2 dst-address-list=local action=mark-packet new-packet-mark=192.168.0.2-local-ul passthrough=no

Qtree

0 name="Overseas-Download" parent=Lan packet-mark="" limit-at=0
queue=PCQ_Download priority=8 max-limit=0 burst-limit=0
burst-threshold=0 burst-time=0s

1 name="Local-Upload" parent=global-out packet-mark="" limit-at=0
queue=PCQ_Upload priority=8 max-limit=0 burst-limit=0
burst-threshold=0 burst-time=0s

2 name="Local-Download" parent=Lan packet-mark="" limit-at=0
queue=PCQ_Download priority=8 max-limit=25000000 burst-limit=0
burst-threshold=0 burst-time=0s

3 name="Overseas-Upload" parent=global-out packet-mark="" limit-at=0
queue=PCQ_Upload priority=8 max-limit=0 burst-limit=0 burst-threshold=0
burst-time=0s

4 name="192.168.0.2-local-dl" parent=Local-Download
packet-mark=192.168.0.2-local-dl limit-at=0 queue=default priority=8
max-limit=4000000 burst-limit=0 burst-threshold=0 burst-time=0s

5 name="192.168.0.2-local-ul" parent=Local-Upload
packet-mark=192.168.0.2-local-ul limit-at=0 queue=default priority=8
max-limit=2000000 burst-limit=0 burst-threshold=0 burst-time=0s

6 name="192.168.0.2-overseas-dl" parent=Overseas-Download
packet-mark=192.168.0.2-overseas-dl limit-at=0 queue=default priority=8
max-limit=512000 burst-limit=0 burst-threshold=0 burst-time=0s

7 name="192.168.0.2-overseas-ul" parent=Overseas-Upload
packet-mark=192.168.0.2-overseas-ul limit-at=0 queue=default priority=8
max-limit=512000 burst-limit=0 burst-threshold=0 burst-time=0s

My questions are:
1. This setup takes too much CPU resources, is there possible any other kind of setup to achieve what I have done.
2. I want to give priority to traffic as it was disscused here http://forum.mikrotik.com/viewtopic.php?t=12870, so each client can browse faster. The only way I made it is mangling each kind of traffic for each ip, but it's 20 !!! mangle rules for IP. My idea is to give priority Globaly. As I understand with my setup I'm unable to achieve that because the packets will be remarked.
If not, how to accomplish this?
(I'm thinking of 2 Mikrotiks. One for priority and another for shaping each ip address).

Thank you for every answer.

Alex

quickgen · Mon Sep 24, 2007 1:56 pm

Anybody?

Mon Sep 24, 2007 3:54 pm

For 300 clients you may thing about PCQ implementation, as it is possible to create separate groups,
- specify amount of bandwidth, that will be equaly divided between the members of group;
- specify amout of bandwidth, that will be assigned for each member of the group.

About priority, it is possible to mark entire traffic for the specific service with one rule, it is not necessary to create priority rule for each IP separately.

quickgen · Mon Sep 24, 2007 4:11 pm

sergejs, thank you for reply. I have earlier tested PCQ and indeed it does limit client to the rate specified, but with PCQ rate specified in Qtree I can't apply burst for each tcp connection disscused here: http://forum.mikrotik.com/viewtopic.php?t=12870 and it's the only real implementation that worked as expected and throlled down p2p.

About priority:
If I mark-packet(passthrough=no) globally services i want to give priority, then i can't apply packet-mark for each ip (client). If passthrough=yes than packets get remarked and priority in Qtree is not working.

Maybe I misunderstand something?
Please give me the right direction.
Thank you.

Mon Sep 24, 2007 4:21 pm

You are correct about passtrough.
It is very complicated configuration to join prioritization with bandwidth limiting.
You may use the following configuration tips to accomplish it, combine 'queue tree' + 'mangle'.
- Firstly let's ensure prioritization, add mangle marks to chain=prerouting and use 'queue tree' with global-in/globabl-out interfaces to set the priority for the specific traffic.
- Then let's ensure bandwidth limiting, add rules to chain=forward and use 'remote'/'local' as interfaces at 'queue tree' configuration. Additionaly you may use 'address-list' to decrease mangle rule number (address-list is feature that allows to create different groups of IP addresses).

quickgen · Mon Sep 24, 2007 5:22 pm

sergejs thanks for fast reply.
I will consider your advise.
Still my configuration is tricky:
1. I have 2 ISPs, so I can't get use of 'remote'/'local' as interfaces, because I'll need to make rules for each ISP(taking into account Local County/Overseas traffic) per client. Traffic already gets divided for every client as Local County/Overseas so it's 4 rules per client.
In Qtree i'm using parent=Lan(It's the only Client interface) for Download Traffic and parent=global-out for Upload Traffic, to avoid shaping per interface.

1 name="Local-Upload" parent=global-out packet-mark="" limit-at=0
queue=PCQ_Upload priority=8 max-limit=0 burst-limit=0
burst-threshold=0 burst-time=0s

2 name="Local-Download" parent=Lan packet-mark="" limit-at=0
queue=PCQ_Download priority=8 max-limit=25000000 burst-limit=0
burst-threshold=0 burst-time=0s

2. You advised to use Adress-List for mangling. Do you mean that using 4 mangle rules only and specifying Src-address-list/Dst-address-list I will be able in Qtree to limit each IP? Maybe I'm wrong, but as expected it will limit ALL Clients.
Still I have to limit Upload/Download for both Local Country Traffic and Overseas Traffic, so its 4 shapers.

I'm thinking of testing 2 Mikrotiks:
First will deal with 2 ISPs and Routes and second will be a transparent traffic shaper for clients. They'll comunnicate through separate Interfaces. Do you think it's a better way to achieve my goal?

Thank you.

Tue Sep 25, 2007 3:28 pm

1) You have to consider about the prioritization then. Probably it is more useful to route important data over fastest connection and non-important to second gateway.
As there is no sense to prioritize traffic, that will be divided then to two parts.
Use only bandwidth limiting then use 'global' interface to specify the total upload, instead of specified different itnerfaces.

2) PCQ queue and address-list allow you to do this. As PCQ queue is applied for group not for separate user.

quickgen · Tue Sep 25, 2007 4:47 pm

sergejs thank you. I clearly see your point of view, because I have already tested PCQ.
But still remains one unsolved question for me. How to combine PCQ with what is described here http://forum.mikrotik.com/viewtopic.php?t=12870 ?
This setup was tested by me and it works like a charm. Clients there very satisfied with http-browsing even at the full load of the channel. But I have to limit their rate anyway, because they paid for that rate only.
My PCQ setup will have pcq-rate='for example 128k' because the client is paying for no more than 128k. PCQ rate will not allow him more than 128k. My queston is how to combine http://forum.mikrotik.com/viewtopic.php?t=12870 with fixed pcq-rate.
I was not able to achieve the proper combination.

Thank you.
P.S.This topic will be very useful for the MT community =)

quickgen · Thu Sep 27, 2007 5:21 pm

sergejs? Is there any way to combine fixed PCQ-rate with Prioritization as described in my last post?

quickgen · Fri Nov 02, 2007 3:40 am

Solved traffic priority and queue tree limmiting by using 2 mikrotiks in line. One is just mangling clients and shaping to the correct speed. The other one is doing prioritization. Seems good for 1 month =)

titius · Wed Nov 07, 2007 2:03 pm

@quickgen

Hi can you post your mangle and queue configuration.

From both routers.

Please

.

quickgen · Thu Jan 17, 2008 3:38 pm

Titius, sorry for not checking this topic.

Here is my configuration for the border router:

/ ip firewall mangle

add chain=forward src-address-list=Local-traffic dst-address-list="Local IpS" \
action=mark-connection new-connection-mark=Local-DL passthrough=yes \
comment="Mark All Local Connections" disabled=no
add chain=forward src-address-list="Local IpS" dst-address-list=Local-traffic \
action=mark-connection new-connection-mark=Local-UL passthrough=yes comment="" \
disabled=no

add chain=forward src-address-list=!Local-traffic dst-address-list="Local IpS" \
action=mark-connection new-connection-mark=World-DL passthrough=yes \
comment="Mark All World Connections" disabled=no
add chain=forward src-address-list="Local IpS" dst-address-list=!Local-traffic \
action=mark-connection new-connection-mark=World-UL passthrough=yes comment="" \
disabled=no

add chain=forward protocol=tcp connection-mark=World-DL connection-bytes=0-200000 \
action=mark-packet new-packet-mark=World-tcp-browse-dl passthrough=no \
comment="World Download" disabled=no
add chain=forward protocol=tcp connection-mark=World-DL action=mark-packet \
new-packet-mark=World-tcp-other-dl passthrough=no comment="" disabled=no
add chain=forward protocol=icmp connection-mark=World-DL action=mark-packet \
new-packet-mark=World-icmp-dl passthrough=no comment="" disabled=no
add chain=forward protocol=udp connection-mark=World-DL action=mark-packet \
new-packet-mark=World-udp-dl passthrough=no comment="" disabled=no
add chain=forward connection-mark=World-DL action=mark-packet \
new-packet-mark=World-other-dl passthrough=no comment="" disabled=no

add chain=forward protocol=tcp connection-mark=World-UL connection-bytes=0-200000 \
action=mark-packet new-packet-mark=World-tcp-browse-ul passthrough=no \
comment="World Upload" disabled=no
add chain=forward protocol=tcp connection-mark=World-UL action=mark-packet \
new-packet-mark=World-tcp-other-ul passthrough=no comment="" disabled=no
add chain=forward protocol=icmp connection-mark=World-UL action=mark-packet \
new-packet-mark=World-icmp-ul passthrough=no comment="" disabled=no
add chain=forward protocol=udp connection-mark=World-UL action=mark-packet \
new-packet-mark=World-udp-ul passthrough=no comment="" disabled=no
add chain=forward connection-mark=World-UL action=mark-packet \
new-packet-mark=World-other-ul passthrough=no comment="" disabled=no

add chain=forward protocol=tcp connection-mark=Local-DL connection-bytes=0-200000 \
action=mark-packet new-packet-mark=Local-tcp-browse-dl passthrough=no \
comment="Local Download" disabled=no
add chain=forward protocol=tcp connection-mark=Local-DL action=mark-packet \
new-packet-mark=Local-tcp-other-dl passthrough=no comment="" disabled=no
add chain=forward protocol=icmp connection-mark=Local-DL action=mark-packet \
new-packet-mark=Local-icmp-dl passthrough=no comment="" disabled=no
add chain=forward protocol=udp connection-mark=Local-DL action=mark-packet \
new-packet-mark=Local-udp-dl passthrough=no comment="" disabled=no
add chain=forward connection-mark=Local-DL action=mark-packet \
new-packet-mark=Local-other-dl passthrough=no comment="" disabled=no

add chain=forward protocol=tcp connection-mark=Local-UL connection-bytes=0-200000 \
action=mark-packet new-packet-mark=Local-tcp-browse-ul passthrough=no \
comment="Local Upload" disabled=no
add chain=forward protocol=tcp connection-mark=Local-UL action=mark-packet \
new-packet-mark=Local-tcp-other-ul passthrough=no comment="" disabled=no
add chain=forward protocol=icmp connection-mark=Local-UL action=mark-packet \
new-packet-mark=Local-icmp-ul passthrough=no comment="" disabled=no
add chain=forward protocol=udp connection-mark=Local-UL action=mark-packet \
new-packet-mark=Local-udp-ul passthrough=no comment="" disabled=no
add chain=forward connection-mark=Local-UL action=mark-packet \
new-packet-mark=Local-other-ul passthrough=no comment="" disabled=no

/ queue tree

add name="World-Download" parent=global-out packet-mark="" limit-at=0 queue=default priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="1.World-tcp-browse-dl" parent=World-Download packet-mark=World-tcp-browse-dl limit-at=0 queue=default priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="2.World-tcp-other-dl" parent=World-Download packet-mark=World-tcp-other-dl limit-at=0 queue=default priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="3.World-icmp-dl" parent=World-Download packet-mark=World-icmp-dl limit-at=0 queue=default priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="4.World-udp-dl" parent=World-Download packet-mark=World-udp-dl limit-at=0 queue=default priority=6 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="5.World-other-dl" parent=World-Download packet-mark=World-other-dl limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no

add name="World-Upload" parent=global-out packet-mark="" limit-at=0 queue=default priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="1.World-tcp-browse-ul" parent=World-Upload packet-mark=World-tcp-browse-ul limit-at=0 queue=default priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="2.World-tcp-other-ul" parent=World-Upload packet-mark=World-tcp-other-ul limit-at=0 queue=default priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="3.World-icmp-ul" parent=World-Upload packet-mark=World-icmp-ul limit-at=0 queue=default priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="5.World-other-ul" parent=World-Upload packet-mark=World-other-ul limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="4.World-udp-ul" parent=World-Upload packet-mark=World-udp-ul limit-at=0 queue=default priority=6 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no

add name="Local-Download" parent=global-out packet-mark="" limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="1.Local-tcp-browse-dl" parent=Local-Download packet-mark=Local-tcp-browse-dl limit-at=0 queue=default priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="2.Local-tcp-other-dl" parent=Local-Download packet-mark=Local-tcp-other-dl limit-at=0 queue=default priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="3.Local-icmp-dl" parent=Local-Download packet-mark=Local-icmp-dl limit-at=0 queue=default priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="4.Local-udp-dl" parent=Local-Download packet-mark=Local-udp-dl limit-at=0 queue=default priority=6 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="5.Local-other-dl" parent=Local-Download packet-mark=Local-other-dl limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no

add name="Local-Upload" parent=global-out packet-mark="" limit-at=0 queue=default priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="1.Local-tcp-browse-ul" parent=Local-Upload packet-mark=Local-tcp-browse-ul limit-at=0 queue=default priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="2.Local-tcp-other-ul" parent=Local-Upload packet-mark=Local-tcp-other-ul limit-at=0 queue=default priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="3.Local-icmp-ul" parent=Local-Upload packet-mark=Local-icmp-ul limit-at=0 queue=default priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="4.Local-udp-ul" parent=Local-Upload packet-mark=Local-udp-ul limit-at=0 queue=default priority=6 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no
add name="5.Local-other-ul" parent=Local-Upload packet-mark=Local-other-ul limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s disabled=no

quickgen · Thu Jan 17, 2008 4:05 pm

This is the configuration of the MT that deals directly with the clients:

/ ip firewall mangle

add chain=forward dst-address=192.168.0.2 src-address-list=!Local-traffic \
action=mark-packet new-packet-mark=192.168.0.2-world-dl passthrough=no \
comment="192.168.0.2" disabled=no
add chain=forward src-address=192.168.0.2 dst-address-list=!Local-traffic \
action=mark-packet new-packet-mark=192.168.0.2-world-ul passthrough=no \
comment="" disabled=no

add chain=forward dst-address=192.168.0.2 src-address-list=Local-traffic \
action=mark-packet new-packet-mark=192.168.0.2-local-dl passthrough=no \
comment="" disabled=no
add chain=forward src-address=192.168.0.2 dst-address-list=Local-traffic \
action=mark-packet new-packet-mark=192.168.0.2-local-ul passthrough=no \
comment="" disabled=no

/ queue tree

add name="1.World-DL" parent=global-out packet-mark="" limit-at=0 queue=default \
priority=2 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s \
disabled=no
add name="192.168.0.2-world-dl" parent=1.World-DL packet-mark=192.168.0.2-world-dl \
limit-at=0 queue=default priority=8 max-limit=10000000 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no

add name="2.World-UL" parent=global-out packet-mark="" limit-at=0 queue=default \
priority=1 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s \
disabled=no
add name="192.168.0.2-world-ul" parent=2.World-UL packet-mark=192.168.0.2-world-ul \
limit-at=0 queue=default priority=8 max-limit=10000000 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no

add name="3.Local-DL" parent=global-out packet-mark="" limit-at=0 queue=default \
priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s \
disabled=no
add name="192.168.0.2-local-dl" parent=3.Local-DL packet-mark=192.168.0.2-local-dl \
limit-at=0 queue=default priority=1 max-limit=50000000 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no

add name="4.Local-UL" parent=global-out packet-mark="" limit-at=0 queue=default \
priority=7 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s \
disabled=no
add name="192.168.0.2-local-ul" parent=4.Local-UL packet-mark=192.168.0.2-local-ul \
limit-at=0 queue=default priority=8 max-limit=10000000 burst-limit=0 \
burst-threshold=0 burst-time=8s disabled=no

quickgen · Thu Jan 17, 2008 4:11 pm

If something will be not so clear with this setup - ask questions and I'll be glad to answer them. If you know something about optimizing this setup, please let me know.
Thanks.

titius · Mon Jan 21, 2008 12:12 am

quickgen · Mon Jan 21, 2008 7:29 pm

Titius, This config is very CPU hungry. I have applied http://wiki.mikrotik.com/wiki/Different ... queue_tree to the client side routers and I can say that it discreased the cpu load from 80-90% to 10-20% on a Pentium4 3.0 GHz , so I really recommend it. Although you cannot monitor each of the 4 queues for each client and you can't know how much bandwidth each IP is using, but this is really great compensated by the low CPU load.
So maybe someone knows other ways to achieve the same goal? Feel Free to improve.

KirillTs · Fri May 09, 2008 4:10 pm

192.168.0.2 is address of mikrotik router?

quickgen · Sat May 10, 2008 10:19 am

KirillTs no. It is a client ip address.

QoS setup

QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

QoS setup

QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup

Re: QoS setup