Hi.

I would like to set up queue templates similar to Allot NetEnforcer. That is to say buckets of IP addresses that have service levels applied to them.

For example:

Service1 has a profile of 500kbps and 100 connections

Service2 has a profile of 1024kbps and 200 connectdions

In Service1 I have a "pool" of IP addresses, each having the "template" of Service1 applied to it individually. Similar in Service2.

Is there a way to do this "pooling" in MT queues?

I've gone down the simple queue path as well as the queue tree path with packet marks but there has to be a more elegant way to handle 500 IP addresses/customers and put them in queues with a template applied to each IP address in the queue.

Any ideas would be helpful.

Thanks!

This is the exact example of PCQ queuing with address lists explained by this PDF http://mum.mikrotik.com/presentations/C ... _Megis.pdf
You can find a lot more configurations in this forum (even a RADIUS pool assignment) using PCQ

Are there any issues using interfaces that are a part of a bridge? I find that the method described in your PDF does not work in an environment with a bridge.

My problem is that it appears to only create one (1) PCQ queue. That one queue gets limited by the pcq_rate. If I apply to global-out many more queues are created.

There seems to be some issue with bridges and PCQ but I don't see that documented.

-J

Are you targeting the bridge interface, or the physical interfaces that are part of the bridge?

Physical

Try the bridge interface.

Is it complicating things that the bridge is also trunking?

I have the bridge set up to use the IP firewall as well as use IP firewall for VLAN.

I can get the download queue to work using global-out HTB. When I use global-out HTB I get multiple PCQ queues. When I try to use the physical interface I get 1 queue and everyone gets dumped into that queue.

I can seem to get the upload PCQ to work in any configuration.

-J

Try the bridge interface.

For the queue tree or for the mangle?

Just for the hell of it I am going to sit down with the packet flow diagram tomorrow and try to wrap my head around that. - if actual work doesn't get in the way, that is. You're doing something I find interesting. Don't know if it can work, though. Yes, you are doing a lot at once, and some of it may be interfering. Can you post your interface configuration (VLANs, physical, and bridging) as well as what kind of firewalling you're trying to do?

Try the bridge interface for mangle and globals as parents for queue trees.

Just for the hell of it I am going to sit down with the packet flow diagram tomorrow and try to wrap my head around that. - if actual work doesn't get in the way, that is. You're doing something I find interesting. Don't know if it can work, though.

Try the bridge interface for mangle and globals as parents for queue trees.

I've been staring at the flow diagram for hours over a couple of days. It makes me think that bridges are handled differently and what I want to do in the configuration I am using can't be done.

I am using forward chain for mangle without respect to in/out interface. So I would presume that it would match on any. My only question is the bridge. Presumably packet going from one bridge interface to the other is a "forward" in the world of queuing. That's the way the firewall rules work anyway.

I think I may be in a corner case for HTB queuing.

-J

I followed the examples given for what I think should work. I also reviewed the
QoS_Megis.pdf file and watched the video. So I think I'm in the right spot to get this going.
I think the packet flow is messed up with bridging vlans or just bridging in general.
There doesn't seem to be a normal out-interface HTB.

Here is some of my configuration... I think I have added all of the relevant information
needed to get the gist of what I am trying to do here.

I am trying to create 4 classes of service using PCQ: SL0, SL1, SL2 and SL3. In the following I have disabled
the queue tree

# dec/15/2010 00:08:30 by RouterOS 4.15
# software id = S722-M7CZ
#

/queue type
add kind=pcq name=SL3_download pcq-classifier=dst-address \
pcq-limit=50 pcq-rate=1500000 pcq-total-limit=2000
add kind=pcq name=SL2_download pcq-classifier=dst-address \
pcq-limit=60 pcq-rate=900000 pcq-total-limit=2000
add kind=pcq name=SL1_download pcq-classifier=dst-address \
pcq-limit=70 pcq-rate=800000 pcq-total-limit=2000
add kind=pcq name=SL0_download pcq-classifier=dst-address \
pcq-limit=70 pcq-rate=500000 pcq-total-limit=2000
add kind=pcq name=SL3_upload pcq-classifier=src-address \
pcq-limit=50 pcq-rate=1000000 pcq-total-limit=2000

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=58_total_upload \
parent=ether3 priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=58_total_download \
parent=global-out priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=SL3_download \
packet-mark=SL3 parent=58_total_download priority=3 queue=SL3_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=SL2_download \
packet-mark=SL2 parent=58_total_download priority=4 queue=SL2_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=SL1_download \
packet-mark=SL1 parent=58_total_download priority=5 queue=SL1_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no \
limit-at=0 max-limit=0 name=SL0_download \
packet-mark=SL0 parent=58_total_download priority=6 queue=SL1_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=yes \
limit-at=0 max-limit=0 name=SL3_upload \
packet-mark=SL3 parent=58_total_upload priority=3 queue=SL3_upload

/ip firewall mangle
add action=mark-connection chain=forward comment="mark SL0 connection" \
disabled=no new-connection-mark=SL0 passthrough=yes src-address-list=SL0
add action=mark-packet chain=forward comment="mark SL0 packets" connection-mark=SL0 \
disabled=no new-packet-mark=SL0 passthrough=no
add action=mark-connection chain=forward comment="mark SL1 connection" disabled=no \
new-connection-mark=SL1 passthrough=yes src-address-list=SL1
add action=mark-packet chain=forward comment="mark SL1 packets" connection-mark=SL1 \
disabled=no new-packet-mark=SL1 passthrough=no
add action=mark-connection chain=forward comment="mark SL2 connection" disabled=no \
new-connection-mark=SL2 passthrough=yes src-address-list=SL2
add action=mark-packet chain=forward comment="mark SL2 packets" connection-mark=SL2
disabled=no new-packet-mark=SL2 passthrough=no
add action=mark-connection chain=forward comment="mark SL3 connection" disabled=no \
new-connection-mark=SL3 passthrough=yes src-address-list=SL3
add action=mark-packet chain=forward comment="mark SL3 packets" connection-mark=SL3 \
disabled=no new-packet-mark=SL3 passthrough=no
add action=mark-connection chain=forward comment="mark SL5 connection" disabled=yes \
in-interface=5800mhz new-connection-mark=SL5 passthrough=yes src-address-list=SL5
add action=mark-packet chain=forward comment="mark SL5 packets" connection-mark=SL5 \
disabled=yes in-interface=5800mhz new-packet-mark=SL5 passthrough=no

/interface bridge
add admin-mac=00:00:00:00:00:00 ageing-time=5m arp=enabled \
auto-mac=yes comment="5800" disabled=no \
forward-delay=15s l2mtu=1522 max-message-age=20s mtu=1500 \
name=5800mhz priority=0x8000 protocol-mode=none \
transmit-hold-count=6

/interface bridge port
add bridge=5800mhz comment="" disabled=no edge=auto \
external-fdb=auto horizon=none interface=ether2 path-cost=\
10 point-to-point=auto priority=0x80
add bridge=5800mhz comment="" disabled=no edge=auto \
external-fdb=auto horizon=none interface=ether3 path-cost=\
10 point-to-point=auto priority=0x80

/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=no \
use-ip-firewall-for-vlan=yes


This is a 493AH. The two physical ports are ether2 and ether3 that are in the bridge
5800mhz. My goal was to use this as a bridging firewall with queues to do local queuing. I
can get the download queuing to work using global-out. But I can't get the upload queues to
work at all using global anything. If I assign to ether2 or ether3 as parent I get one
PCQ queue and all traffic mushed inside the one queue (not helpful).

From my research this week it seems to me that PCQ works with the output HTB of two
physical interfaces. If you can test and confirm that it would be helpful.

Tnx.

-j

Just for the hell of it I am going to sit down with the packet flow diagram tomorrow and try to wrap my head around that. - if actual work doesn't get in the way, that is. You're doing something I find interesting. Don't know if it can work, though.

fewi, any luck?

I only had time to do some reading, but not to try it out - hence no post. But here's what I am thinking.

Scenario: a bunch of VLANs going through a router need to be shaped transparently. For now, just basic rate limiting.

- don't try to use just one bridge - on the two physical interfaces create VLAN subinterfaces so you have two VLAN interfaces per VLAN to police, and bridge those two VLAN interfaces so you have one bridge per VLAN
- don't try to mark based on interface, mark based on in-bridge-port
- make queue trees that use the VLAN subinterfaces as a parent

So to police a VLAN, make prerouting mangle rules - two per VLAN. One rule marks the traffic one way based on the in-bridge-port being one of the VLAN subinterface, another rules marks the traffic the other way based on the in-bridge-port being the other VLAN subinterface. Make two queue trees per VLAN, using as a parent the VLAN subinterfaces and the appropriate packet marks as triggers. The queues can be of type PCQ.

You can extend that to per protocol by having more mangle rules (probably jump into custom chains based on in-bridge-port, do special marking there) and having protocol specific subchains tha use as a parent queues tied to the VLAN subinterface as their respective parent.

I think that might work. But I haven't had time to try any of it.

I only had time to do some reading, but not to try it out - hence no post. But here's what I am thinking.

Scenario: a bunch of VLANs going through a router need to be shaped transparently. For now, just basic rate limiting.

- don't try to use just one bridge - on the two physical interfaces create VLAN subinterfaces so you have two VLAN interfaces per VLAN to police, and bridge those two VLAN interfaces so you have one bridge per VLAN
- don't try to mark based on interface, mark based on in-bridge-port
- make queue trees that use the VLAN subinterfaces as a parent

So to police a VLAN, make prerouting mangle rules - two per VLAN. One rule marks the traffic one way based on the in-bridge-port being one of the VLAN subinterface, another rules marks the traffic the other way based on the in-bridge-port being the other VLAN subinterface. Make two queue trees per VLAN, using as a parent the VLAN subinterfaces and the appropriate packet marks as triggers. The queues can be of type PCQ.

You can extend that to per protocol by having more mangle rules (probably jump into custom chains based on in-bridge-port, do special marking there) and having protocol specific subchains tha use as a parent queues tied to the VLAN subinterface as their respective parent.

I think that might work. But I haven't had time to try any of it.

Ok. This is a little bigger project for me to do all this so I'll get on this as soon as I can. I might have to test in a non-live environment as well just in case I do something that will cause interruption with the live data (I did that a lot yesterday).

BTW, once you get all this set up how can you tell how each IP is being shaped? Are there reports that can give this type of thing? What I've see so far is that queuing is applied and PCQ is magic but you don't exactly know how a particular connection is being handled without deriving empirical data from other sources. Curious on your thoughts there.

-J

I only had time to do some reading, but not to try it out - hence no post. But here's what I am thinking.

Scenario: a bunch of VLANs going through a router need to be shaped transparently. For now, just basic rate limiting.

- don't try to use just one bridge - on the two physical interfaces create VLAN subinterfaces so you have two VLAN interfaces per VLAN to police, and bridge those two VLAN interfaces so you have one bridge per VLAN
.

I got this done. I created vlan sub-interfaces for each physical interface. Instead of bridging the physical interfaces I am now bridging the vlans together in their own bridge. That seems to be working just fine. If nothing else it clears up a lot of statistics allowing me to see much better which vlans are doing what. So that is a nice feature having lost no other feature.

I've read your comments on marking several times and I think I get about 80% of what you want me to try. In the past the suggestion is to:

1. Mark all connections for the various services using forward rules and address-lists without respect to input interface. (passthrough=yes)

2. Mark all packets based on on #1, again on forward without respect to input interface (passthrough=no).

3. Repeat step 1 and 2 for the remaining services.

I'm stuck mainly because the QoS Packet Flow diagram would lend me to believe to that what I need to do is a forward mangle rule and not pre-routing.

Again, I am doing this in a production environment so I am being very methodical here.

-J

In what I posted there is no difference between prerouting and forward. Interface HTB happens after postrouting, even. Use what you're comfy with.
Get a bench router so it's not production. Heartily recommend that. Keep it around for future experiments.

I'm afraid I have had a fairly big work (proper paid for pays my bills work) project that will need some time preparing and needs completion by next Thursday night land in my lap. There's little chance I will have time to lab it out myself. Still very curious what you end up with.

- don't try to mark based on interface, mark based on in-bridge-port
- make queue trees that use the VLAN subinterfaces as a parent

So to police a VLAN, make prerouting mangle rules - two per VLAN. One rule marks the traffic one way based on the in-bridge-port being one of the VLAN subinterface, another rules marks the traffic the other way based on the in-bridge-port being the other VLAN subinterface. Make two queue trees per VLAN, using as a parent the VLAN subinterfaces and the appropriate packet marks as triggers. The queues can be of type PCQ.

Since I have 4 service levels (SL0, SL1, SL2, SL3) I will have:

1. prerouting connection mark using in-bridge-port of the first vlan for service level using address-lists.
2. prerouting connection mark using in-bridge-port of the second vlan for service level using address-lists.
3. prerouting packet mark using service level connection mark in 1 and 2 above.

Since I have defined VLAN1 through VLAN8 I will have each of these tree rules for eight vlans or a total of 24 prerouting mangle rules for marking the connections and packets. This will be a total of 23 mangle rules for my method of service level for vlan trunking.

Taking this a step at a time to make less complicated. If I can mark properly I'm sure I can finish the queues without issue.

J

So, for one service level this is what I think you want me to try to mangle:

add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan621e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan621e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan622e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan622e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan623e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan623e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan624e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan624e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan625e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan625e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan626e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan626e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan627e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan627e3
add chain=prerouting src-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan628e2
add chain=prerouting dst-address-list=SL0 action=mark-connection new-connection-mark=SL0 passthrough=yes comment="mark SL0 connection" disabled=no in-bridge-port=vlan628e3
add chain=prerouting connection-mark=SL0 action=mark-packet new-packet-mark=SL0 passthrough=no comment="mark SL0 packets" disabled=no

if this is correct, can you enlighten me on how you want to try the queues?

J

In what I posted there is no difference between prerouting and forward. Interface HTB happens after postrouting, even. Use what you're comfy with.
Get a bench router so it's not production. Heartily recommend that. Keep it around for future experiments.

I'm afraid I have had a fairly big work (proper paid for pays my bills work) project that will need some time preparing and needs completion by next Thursday night land in my lap. There's little chance I will have time to lab it out myself. Still very curious what you end up with.

I have a mess of routers that I can play with but nothing that I can easily set up this type of traffic observation.

The trick to this configuration is proper marking. It seems to me to be asymmetric. Marks one side but not the other.

I'm still not certain on your thinking on how/why to mark the individual vlans vs. the interface as a whole. You must know of some underlying packet flow issues I am not aware of. The router can certainly break things down to the vlan interface and even bridge them as you have suggested. What you have suggested after that I find a little confusing and the code to make it work with so many service levels seems a bit excessive.

Definitely pay the bills first!

VLANs on bridges are - weird. They do not seem to work right unless you break it down into the VLAN subinterfaces. Unfortunately I am a little bit hazy on the inner workings of the interaction between bridges, physical interfaces, and VLAN interfaces in Linux. What you posted last seems like a good starting point. It can be optimized, but should work, so let's optimize later. Now all you need is queue trees that as parents have the VLAN interfaces, reference the right packet marks, and set rate limits. Each queue only deals with output out that interface.

fewi,

I'm back in this game. I have replaced a cisco router doing the interaction with the VLANs and a bridging routerboard that I was trying to do the QOS with. I've now got the RB1100 doing all the work so it also has the non-VLAN connections to the edge routing to other sites. So, I have 100% control of traffic on both VLAN interfaces and routed interfaces. So at least I have a scenario where I can do queuing on non-VLAN interfaces with respect to all the traffic going to this site. I really hope I can figure this out so I can get rid of the current method of bandwidth control and replace it with queuing on the MT.

J