What model to use?

aromerombit · Mon Mar 13, 2023 6:50 pm

Hi everyone,

Sorry for the cliche but I'm new with Mikrotik and as an ISP, I have had the training but still don't know much about the hardware, without further ado, the purpose of this post is to ask the community about my dilemma:

I need to deliver a service of 5Gbps but I'm having problems of what will be the right hardware, one idea was to install a switch on the client side for the performance (and it'll be cheaper than a CCR) but I also thought about configuring a queue to limit the bandwidth but for what I have read Mikrotik doesn't limit the bandwidth correctly when is more than 1Gbps, so that is also another issue how to limit the bandwidth.

Can any kind soul help me solve the conundrum? I'll really appreciate it.

Kind regards,
Angel Romero

rextended · Mon Mar 13, 2023 6:55 pm

for what I have read Mikrotik doesn't limit the bandwidth correctly when is more than 1Gbps

And where did you read it?

aromerombit · Mon Mar 13, 2023 7:27 pm

I don't have the exact site, but also during training this was mentioned a lot, I actually asked about this because right now we are limiting a 3 Gbps service seems like its working though.

rextended · Mon Mar 13, 2023 7:43 pm

I don't have the exact site, but also during training this was mentioned a lot, I actually asked about this because right now we are limiting a 3 Gbps service seems like its working though.

Who taught you that?
We need to go back to who sells this information to get elucidations, otherwise they are all urban legends.

anav · Mon Mar 13, 2023 7:49 pm

What are your qualifications, general IT knowledge or actual Mikrotik Certifications???
If this is your company business, it sounds like you need to hire a consultant.............
https://mikrotik.com/consultants

aromerombit · Mon Mar 13, 2023 11:03 pm

I don't know what the problem is, I'm just asking an honest question here. Yes I may be inexperienced but knowledge/training doesn't give you experience with the hardware or maybe I had a shitty training, that is why I was trying to reach the community that certainly has more experience and knowledge. Don't know if I didn't convey my question correctly or is not that simple to answer that you reacted in that way, either way I guess this wasn't the correct platform for my question.

Regards,

rootemin · Mon Mar 13, 2023 11:22 pm

for what I have read Mikrotik doesn't limit the bandwidth correctly when is more than 1Gbps
And where did you read it?

i usage ccr1036 ask mikrotik support how to limit vlan 1.500 mbps
answer
Hello,

Could you upgrade to ROS 7.8 and test to see what happens? If the CPU is maxing out as well, try to set it to 1400M, but as queues are per core, it may be, that to achieve these speeds you need a router with better per core performance.

Best regards,

Mikrotik technique, which I understand from here, recommends the CCR2x series for limiting high bandwidths.
When you limit high bandwidths on tile series routers, the cpu does not go below 100%, does anyone have any recommendations?

rextended · Mon Mar 13, 2023 11:26 pm

It is the same thing as writing that a bicycle cannot have a speed limited to 200...
OBVIOUS...

rextended · Mon Mar 13, 2023 11:30 pm

I don't know what the problem is

@aromerombit
There isn't the slightest problem, if you pay attention we just want to understand where they teach that bulls…t.
It's obvious that if you have to drive 5G and you use a "hEX S" it doesn't work...

To have a more precise opinion, you must be more precise yourself,
one thing is to let the traffic "pass or limit", another thing is then everything that must be managed by the same machine such as BGP, pppoe, firewall, etc.
More precise specifications are needed, and it is better if you divide the various tasks among several machines (regardless the vendor)

anav · Mon Mar 13, 2023 11:32 pm

Perhaps this a ChatGPT invasion LOL

rextended · Mon Mar 13, 2023 11:34 pm

Perhaps this a ChatGPT invasion LOL

aromerombit · Mon Mar 13, 2023 11:40 pm

The configuration is simple I just need to deliver 5Gbps with let's say a router(CRS/CCR), this router will have 2 interfaces SFP+ on to receive my Fiber and the other to deliver the 5Gbps to the client, the hardware will give a subnet /30 to the client and a simple/Tree queue set to limit the 10Gbps link to 5Gbps so the end user doesn't abuse the 10Gbps link.

I had to research the same but for a 1Gbps service and found out from the research (again maybe bullshit information) that for that kind of service a RB4011 is a good choice, that is what I wanted to find out, because, again I'm not familiar with the hardware.

rextended · Mon Mar 13, 2023 11:48 pm

If you see the test result of the devices,
for example the RB4011
https://mikrotik.com/product/rb4011igs_ ... estresults
You can see the max speed of 4,286.9 for 25 queues, obviously if you have less queues.... but have only one SFP+

CCR2004 have 2 SFP+ and
https://mikrotik.com/product/ccr2004_16 ... estresults
And have 7917.6 for 25 queues...

And next step is CCR2116...
https://mikrotik.com/product/ccr2116_12 ... estresults
25897.4Mbps with 25 queues....

aromerombit · Tue Mar 14, 2023 12:19 am

So I can follow the Tests Result (which I've read before) in case of the CRS, correct me if I'm wrong I need to take into account the Ethernet test results not the Switching if it is running RouterOS?

And again with the BS information, the trainer told me that if the CRS is running RouterOS the performance is degraded but if you bridge two or more interfaces together with hardware offloading the CRS will process the traffic on said bridge without the penalty in perfomance for running RouterOS? Is this also BS?

rextended · Tue Mar 14, 2023 1:02 am

CRS is a Switch, CCR is a Router.
Do not do router's thing with a switch (for example, routing and queues)

wiseroute · Tue Mar 14, 2023 2:58 am

hello @romero.

i think your question is legit.

i don't know how things work for isp like you in your place, but maybe you can just ask local mikrotik sales representative about your isp requirements . or perhaps @normis could help you with that.

maybe you could ask mikrotik for hardware demo trial? so you could see the performance.

of course you can make yourself x86 test router or switch, but I don't think it will represent your intended crs or ccr.

good luck

LikeMyFloydPink · Tue Mar 14, 2023 6:12 am

When you say service - is there any chance you are referring to a circuit? Asking for the sake of clarity.

aromerombit · Wed Mar 15, 2023 7:05 pm

When you say service - is there any chance you are referring to a circuit? Asking for the sake of clarity.

Yeah a circuit, I wrote it as a "service" in terms that we give them the "service" of connecting them to the internet. But in the case of our ISP they call it a circuit.

I decided to install a CCR2004 to deliver the 5Gbps, hope is not overkill. Don't be mad is this is stupid, there are so many different opinions on this matter, in example: when we started as an ISP we knew even less about Mikrotik than we know now, so we hired a consultant and they created the backbone(core network) for the ISP, after that I analyzed the configuration took the training in most of the certification of Mikrotik (sorry this is just background) and continued with it, since then the network has grown.

The point to this story is that we had and to certain point have a consultant and I asked the same question that I did my trainer and on this post: the answer of my trainer was that it was difficult for mikrotik to limit the bandwidth after certain Mbps, the answer of the consultant was that while using the CRS3XX I can deliver and limit the 5Gbps and there is your answer, which from what I understood is don't use a switch where you need a router and that is why is confusing.

anav · Wed Mar 15, 2023 7:35 pm

Hi Angel, the logic escapes me?
You have a 10Gigabit Fibre line you are paying for.
You have one customer that is asking for 5gb,

Solution: Get a router that can handle 5gb only ?? Test result for queues and filters show a throughput of between 5-8gigs

Better Solution: Get a router that can handle 10gb and not maxed out capacity either, and thus can use or sell the rest of the fiber connection throughput.
CCR2116

spippan · Wed Mar 15, 2023 8:27 pm

When you say service - is there any chance you are referring to a circuit? Asking for the sake of clarity.
Yeah a circuit, I wrote it as a "service" in terms that we give them the "service" of connecting them to the internet. But in the case of our ISP they call it a circuit.

I decided to install a CCR2004 to deliver the 5Gbps, hope is not overkill. Don't be mad is this is stupid, there are so many different opinions on this matter, in example: when we started as an ISP we knew even less about Mikrotik than we know now, so we hired a consultant and they created the backbone(core network) for the ISP, after that I analyzed the configuration took the training in most of the certification of Mikrotik (sorry this is just background) and continued with it, since then the network has grown.

The point to this story is that we had and to certain point have a consultant and I asked the same question that I did my trainer and on this post: the answer of my trainer was that it was difficult for mikrotik to limit the bandwidth after certain Mbps, the answer of the consultant was that while using the CRS3XX I can deliver and limit the 5Gbps and there is your answer, which from what I understood is don't use a switch where you need a router and that is why is confusing.

CCR2004 surely is not overkill at all for your setup. at best there will be just enough headroom at last.

the only weird thing with mikrotik (some hardware at least - need to search my training documents to pinpoint

) QoS - in some cases when the queue gets overloaded or queue bucket(s) are empty, no more fw or bridge rules are processed and it behaves more like a hub.
hope i can contact my former trainer on that topic

so that's why i meant, no overkill but good to have headroom

anav · Wed Mar 15, 2023 9:08 pm

You missed my point completely sippan, he should adjust for his fibre throughput not the throughput to the client............

aromerombit · Wed Mar 15, 2023 10:23 pm

You missed my point completely sippan, he should adjust for his fibre throughput not the throughput to the client............

What you mean fibre throughtput? the throughput I'm delivering? if yes, how can I limit this if the link is 10Gbps and Mikrotik gives an error on queues with more than 4200Mbps?

aromerombit · Wed Mar 15, 2023 10:25 pm

Hi Angel, the logic escapes me?
You have a 10Gigabit Fibre line you are paying for.
You have one customer that is asking for 5gb,

Solution: Get a router that can handle 5gb only ?? Test result for queues and filters show a throughput of between 5-8gigs

Better Solution: Get a router that can handle 10gb and not maxed out capacity either, and thus can use or sell the rest of the fiber connection throughput.
CCR2116

Oops I re-read this, and I think I got your point, if the hardware doesn't give more than 5Gbps the client can't abuse the 10Gbps link, is that it?

anav · Wed Mar 15, 2023 11:14 pm

No LOL,
I mean YOU are paying for 10gig fibre connection.

1. You have customer A, who wants to pay you for 5gigs for throughput
2. You may need some throughput for your own needs in same location (unknown , no context)
3. You are looking for other customers at location B,C,D that may want 1gig service each.

If you buy stupidly a router that cannot handle the throughput your paying for, then
you will not be able to make use of the rest of the throughput!!!!! ( either for own use or other customers )

aromerombit · Thu Mar 16, 2023 12:55 am

No LOL,
I mean YOU are paying for 10gig fibre connection.

1. You have customer A, who wants to pay you for 5gigs for throughput
2. You may need some throughput for your own needs in same location (unknown , no context)
3. You are looking for other customers at location B,C,D that may want 1gig service each.

If you buy stupidly a router that cannot handle the throughput your paying for, then
you will not be able to make use of the rest of the throughput!!!!! ( either for own use or other customers )

The issue in question is more to limit the end user, considering that I have a CCR2116 to receive the 10Gbps and a few CCR2004 to distribute said 10Gbps to different end users/client/circuits

anav · Thu Mar 16, 2023 1:18 am

Yes but this is the first time you mention you already have the router I was suggesting the 2116 LOL.. Good to go then.

The only router you pointed out was the 2004, which we know now, since you actually provided useful information, is NOT hooked up to fiber but to the 2116.

aromerombit · Thu Mar 16, 2023 2:21 am

Yes but this is the first time you mention you already have the router I was suggesting the 2116 LOL.. Good to go then.

The only router you pointed out was the 2004, which we know now, since you actually provided useful information, is NOT hooked up to fiber but to the 2116.

The topology is as follows:

Mayor ISP --> 10 Gbps --> CCR2216(BGP) --> 10Gbps --> CCR2004(Edge) --> 10Gbps(trying to limit to 5Gbps) --> <End user equipment>CCR2004

but I wanted to know if I could use a CRS, because our "consultant" told us it was feasible for queuing and routing, I sincerely don't see how unless I only use the CRS as a mere switch and to everything on the Edge CCR2004 which I don't know if its a good practice, basically what I want is to find is what logic/idea to follow in this cases.

On the CCR2004 running v6.48 can't do a Queue with more than 4200Mbps(maybe a little bit more) but on the CCR2216 running v7.8 can create a queue with more than 4200Mbps, and my logic is that I need to limit the bandwidth at the edge.

rextended · Thu Mar 16, 2023 9:51 am

but I wanted to know if I could use a CRS, because our "consultant" told us it was feasible for queuing and routing

But which consultant is one who advises you to use a switch as a router with queues for 5GBps??? (does not import the brand)

aromerombit · Thu Mar 16, 2023 7:15 pm

Exactly why my skepticism and there for the reason of my question. But analyzing what the consultant said I think he meant the following:

BGP Router -> Edge Router( This router does the queuing ) -> Switch at the End User site.

I'm not sure of installing a switch on the Client Site will be a good image for an "Enterprise Company"-ish.

un9edsda · Thu Mar 16, 2023 9:27 pm

The topology is as follows:

Mayor ISP --> 10 Gbps --> CCR2216(BGP) --> 10Gbps --> CCR2004(Edge) --> 10Gbps(trying to limit to 5Gbps) --> <End user equipment>CCR2004
...
On the CCR2004 running v6.48 can't do a Queue with more than 4200Mbps(maybe a little bit more) but on the CCR2216 running v7.8 can create a queue with more than 4200Mbps, and my logic is that I need to limit the bandwidth at the edge.

Regarding shaping and queuing you may find useful to have a look at FQ_Codel and Mikrotik CCR CPU Utilization and some quick comments on configuring cake topics if you haven't already.

Similar "if you haven't already" topics regarding what version to use on the CCR2004:

rextended · Fri Mar 17, 2023 12:14 am

I'm not sure of installing a switch on the Client Site will be a good image for an "Enterprise Company"-ish.

Really? not...

A switch for 5Gbps is one switch only also on client side.

If you have to manage 100Mbps can suffice, but with 5Gbps you kill the CRS cpu...

For example, if you buy the more CPU powerful (ignoring switching part) CRS317-1G-16S+ the max speed with routing is ~1,2Gbps without NAT & Co.
Probably with NAT max speed is ~500Mbps

And for the CRS518-16XS-2XQ-RM can reach ~160Gbps but used for routing/NATting just ~300Mbps...

And CRS518-16XS-2XQ-RM can reach 1200Gbps but... is not tested at all for other use...
probably the max speed with routing is ~300Mbps because the CPU has only one gigabit lane to the switch chip.

Understand now?

un9edsda · Fri Mar 17, 2023 4:10 pm

Really? not...

A switch for 5Gbps is one switch only also on client side.

If you have to manage 100Mbps can suffice, but with 5Gbps you kill the CRS cpu...

Although we do know from @IPANetEngineer about "l3 fw offload - stateful offload of IPv4 connections and NAT" and which features needed to be disabled to be able to use it (and some additional showstoppers on recent versions (v7.7 and v7.8 ) from @blacksnow in the very same thread.

aromerombit · Sat Apr 15, 2023 2:09 am

Hi everyone it's been a while. I have an update we decided to install a CCR2004-1G-12S+2XS with RouterOS 7.8 at the customer side to deliver the 5Gbps and its been a real challenge.

If we create a queue simple or tree, the processor goes up and the bandwidth stops at 2.6 Gbps, once I disable the queue(either one) the processor drops to 35-45% and stabilizes at 3.3 Gbps with sudden bursts above 6Gbps (All transceivers are 10Gbps from our core (our core is a CCR2216) to the client site and from the CCR to the client).

Call me what you will but I can't seem to find why.

I got two CCR2216 connected to each other at 100Gbps and tried the btest the max I got was 15Gbps processor at 100% and only an IP configured on the router for testing.

Don't know if I'm doing something wrong, what am I missing?

Sat Apr 15, 2023 3:30 am

YOu can limit speeds of several gigabits using CRS 3xx switches i have used CRS 317 with Routeros 7.6 to limit using Ingres ACL's rate parameter that way do not use CPU on switch and works ok

With CRS 317 in L2 HW offload mode ingress ACL limit works OK, in L3 HW offload mode in 7.6 does NOT work, i have not tried more recent versions

Maybe using Integrated switch on CCR 2216 or 2116 you can do ACL ingress limit without using CPU but i have not tried on that references

accarda · Sat Apr 15, 2023 8:34 am

I got two CCR2216 connected to each other at 100Gbps and tried the btest the max I got was 15Gbps processor at 100% and only an IP configured on the router for testing.

Hi, as indicated by MikroTik, when running such test is better to do it through the "router under test", like indicated in this help page:
https://help.mikrotik.com/docs/display/ ... width+Test

where it specifically says:

Bandwidth Test uses a lot of resources. If you want to test real throughput of a router, you should run bandwidth test through the tested router not from or to it. To do this you need at least 3 routers connected in chain: the Bandwidth Server, the router being tested and the Bandwidth Client.

What model to use?

What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Re: What model to use?

Who is online