MikroTik

We running on CRS-326-24S+2Q+ with ROS v7.6
All 80% ports are used.
All ports do bridge vlan filtering hwoffload.
Some ports do bonding hwoffload.
No L3hwoffload enabled.

Aggregate traffic some around 25-32gbps.

Somehow traffic rate limiter on switch chip not limiting precisely as expected on high bandwidth.
we apply only ingress rate based on vlan id.
1. on rate 2000M, on other side receiving only 1930-1935mbps. we expected 1995-2000mbps, because trying on cisco N9K and N3K qos, its receiving exactly 2000mbps
2. we doing some formula (1024 x 2G) x 102% for achieving aroung 2000mbps (not less).
3. we got complained by customer on 3Gbps bandwidth that his traffic only around 2500mbps but link got request time out sometime.
4. based on number 3, is it more less number bandwidth than 2000M which is losing some 50-70mbps. Maybe its bug when using a lot of switch rule?

is this a bug for mikrotik qos based on switch chip? or there is some misconfiguration? because we trying limiting by src-dst ip, vlan, src-dst ip still same result.
notes: if you test by mikrotik btest with switch in the middle, it will show exact bandwidth. but when you using real world traffic, it will less 10-50 mbps per Gbps rate.

#[SUP-106898]

https://help.mikrotik.com/docs/display/ ... ficShaping

It is possible to limit ingress traffic that matches certain parameters with ACL rules and it is possible to limit ingress/egress traffic per port basis. The policer is used for ingress traffic, the shaper is used for egress traffic. The ingress policer controls the received traffic with packet drops. Everything that exceeds the defined limit will get dropped. This can affect the TCP congestion control mechanism on end hosts and achieved bandwidth can be actually less than defined. The egress shaper tries to queue packets that exceed the limit instead of dropping them. Eventually, it will also drop packets when the output queue gets full, however, it should allow utilizing the defined throughput better.

Well done! Thanks for sharing solution!!

Well done! Thanks for sharing solution!!

How exactly did you do this ? I try and do ingress/egress rates on the ports and they just preform horribly.. I mean the egress I ok.. but the ingress doesnt match at all.

Well done! Thanks for sharing solution!!

How exactly did you do this ? I try and do ingress/egress rates on the ports and they just preform horribly.. I mean the egress I ok.. but the ingress doesnt match at all.

Just bumping this thread, I'm having the exact same problem as reported above. The egress shaper works within expectations. However the ingress policer results in limiting speed to about 1/10th of whatever number you place in the rate. For example at 1G rate, the output you get with iperf3 is 100Mbits/s. Bump the rate to 10G and you get still 100Mb/s. Bump it to 50G and it breaks the switch, you get 0Mb/s. This is on version 7.15 CCR2216.

For example at 1G rate, the output you get with iperf3 is 100Mbits/s.

Which type of traffic was this, TCP or UDP? You should try UDP ... the problem with ingress policer is that it can only drop excess frames and dropping packets causes havoc for TCP (egress shaper can delay packets which is pretty nice to TCP).

Do you have flow control enabled on the port being rate limited? If not, try to enable it (and have it enabled on the other end as well), in theory it should help as this would signal to transmitter to slow down without actually dropping frames.

How exactly did you do this ? I try and do ingress/egress rates on the ports and they just preform horribly.. I mean the egress I ok.. but the ingress doesnt match at all.

Just bumping this thread, I'm having the exact same problem as reported above. The egress shaper works within expectations. However the ingress policer results in limiting speed to about 1/10th of whatever number you place in the rate. For example at 1G rate, the output you get with iperf3 is 100Mbits/s. Bump the rate to 10G and you get still 100Mb/s. Bump it to 50G and it breaks the switch, you get 0Mb/s. This is on version 7.15 CCR2216.

in 7.15 now you have new possibilities

https://help.mikrotik.com/docs/pages/vi ... =189497483

Just bumping this thread, I'm having the exact same problem as reported above. The egress shaper works within expectations. However the ingress policer results in limiting speed to about 1/10th of whatever number you place in the rate. For example at 1G rate, the output you get with iperf3 is 100Mbits/s. Bump the rate to 10G and you get still 100Mb/s. Bump it to 50G and it breaks the switch, you get 0Mb/s. This is on version 7.15 CCR2216.

in 7.15 now you have new possibilities

https://help.mikrotik.com/docs/pages/vi ... =189497483

I guess qos is an option, although in my mind qos is for the quality and priority of the packets that get transmitted not as a policer in terms of controlling traffic flows like queues or port bandwidth limiting. Nor does it have the granularity of controlling bandwidth to specific addresses or from specific addresses.

For example at 1G rate, the output you get with iperf3 is 100Mbits/s.

Which type of traffic was this, TCP or UDP? You should try UDP ... the problem with ingress policer is that it can only drop excess frames and dropping packets causes havoc for TCP (egress shaper can delay packets which is pretty nice to TCP).

Do you have flow control enabled on the port being rate limited? If not, try to enable it (and have it enabled on the other end as well), in theory it should help as this would signal to transmitter to slow down without actually dropping frames.

I'm testing iperf3 with TCP traffic, but even using BBR congestion control which is essentially just forcing as much through the pipe as possible, I'm stilling seeing widly reduced throughput regardless of the limit set on the ingress. Something is definitely off.

take a look at this example of QoS-HW in action

viewtopic.php?t=196068#p1078627

I'm testing iperf3 with TCP traffic, but even using BBR congestion control which is essentially just forcing as much through the pipe as possible

As I wrote: dropped packets make havoc for TCP. The other phenomenon, which congestion control has to deal with, is varying/long RTT ... and that's something many different algorithms handle decently (if not well). So the first thing to do is test using UDP ... which doesn't care about dropped packets and doesn't have bi-directional dependency so it should clearly show if ingress policer really misbehaves as wildly as it seems from your tests performed so far ... or not.
And you didn't respond to my question about flow control either.