CCR2216-1G-12XS-2XQ (New Flagship)

doush · Fri Feb 18, 2022 11:40 am

I think this one is going to be our replacement for CCR1072 s.
Any idea on the release date ?

https://www.balticnetworks.com/docs/Mik ... asheet.pdf
https://www.youtube.com/watch?v=7_uLxZYYEpQ

Fri Feb 18, 2022 1:48 pm

Dual Hotswap PSU !

mfrommel · Fri Feb 18, 2022 2:38 pm

This is a ROS V7 only device.
You should test your config with ROS 7.x.x. before bringing this device into production

Rian · Fri Feb 18, 2022 3:02 pm

I heard at the end of March, beginning of April - If it comes true.

I think this one is going to be our replacement for CCR1072 s.
Any idea on the release date ?

https://www.balticnetworks.com/docs/Mik ... asheet.pdf
https://www.youtube.com/watch?v=7_uLxZYYEpQ

rushlife · Fri Feb 18, 2022 3:34 pm

server8 · Fri Feb 18, 2022 4:04 pm

I hope no reboots and no packet loss

AlexanderM · Fri Feb 18, 2022 11:51 pm

Those that stick with MT may be able to use the latest WB without a crash with this device! The rest of us will have to switch to more trustful vendors.
We will order few more CHR licenses to serve us until we switch to a stable OS for our routing. Having a number of CCR2004s idle is more than enough reason to skip the next models^H^H^H^H^H^Hpre-production releases.

IMO Mikrotik should only target the SOHO market for at least couple of years. Lets face it - their software is not stable enough for the businesses that need 100G and/or stability.

Its a shame for MT to release more prealpha-OS-only devices instead of fixing the mess they created!

AlexanderM · Fri Feb 18, 2022 11:54 pm

I hope no reboots and no packet loss

I may say you are a dreamer
But you'r not the only one
I hope someday they will fix them
And the world will be as one

netzwerghh · Sat Feb 19, 2022 10:04 pm

We are using some CCR2004 for peering at some IXPs und also for our upstream connectivity. So they have to handle multiple full tables (850.000 IPv4 and 150.000 IPv6). I've seen Janis video about the CCR2216 and there it is said it can handel HW offloading for L3 IP routing. But it says only IPv4 and only 80K to 120K routes. What can I expect how the router behaves if I have a routing table with 850k IPv4 and 150K IPv6? Will it route some packets with HW offloading and the other have to travel through CPU?

Unbenannt.PNG

mkx · Sat Feb 19, 2022 10:48 pm

Will it route some packets with HW offloading and the other have to travel through CPU?

Yes. @raimondsp provided some fairly extensive and technical explanation about L3HW offload in this thread.

TheRealJLH · Sun Feb 20, 2022 8:09 pm

This this is gonna ROCK! The Marvell Prestera-DX8525 is a beast 24x25Gb Lanes

wowzah, configured for 100Gbps to the CPU makes this thing is an absolute MONSTER! This CCR will dominate all other Mikrotik CCR routers ever made. Can't wait to get my hands on one these. Can you say 100Gbps between core nodes for a backbone WOW!!

I also wonder what price tag this will have

Hominidae · Sun Feb 20, 2022 8:51 pm

I also wonder what price tag this will have

...you can place it on pre-order for roughly 2.4kUSD ... use your google.fu.

guipoletto · Sun Feb 20, 2022 8:54 pm

128Mb of NAND is my only complaint.

With containers support / partitions, why is it so hard to have 1GB of nand in a flagship router?

OvcaX · Sun Feb 20, 2022 9:13 pm

Now we need 25/100G switches

netzwerghh · Sun Feb 20, 2022 9:27 pm

Great Thread! Thank you.

Will it route some packets with HW offloading and the other have to travel through CPU?

Yes. @raimondsp provided some fairly extensive and technical explanation about L3HW offload in this thread.

Sun Feb 20, 2022 9:35 pm

128Mb of NAND is my only complaint.

With containers support / partitions, why is it so hard to have 1GB of nand in a flagship router?

now you have several M2 slot for SSD's

Sun Feb 20, 2022 9:35 pm

Now we need 25/100G switches

OHHH YESSS !!

guipoletto · Sun Feb 20, 2022 9:52 pm

128Mb of NAND is my only complaint.

With containers support / partitions, why is it so hard to have 1GB of nand in a flagship router?

now you have several M2 slot for SSD's

And what about partitions?
Can i boot/save the backup partition from one of those SSD's?

psannz · Sun Feb 20, 2022 10:10 pm

128Mb of NAND is my only complaint.

With containers support / partitions, why is it so hard to have 1GB of nand in a flagship router?

Precisely for that reason. You want ROS and only ROS on that NAND. At most, you want the critical logs on it. So long as you don't tire out the NAND with unneeded write cycles.
In a way, Mikrotik are saving us from our own insanity here

That pair of M.2 fits perfectly for your Dude, container, Proxy and whatnot storage needs.

Sun Feb 20, 2022 10:34 pm

now you have several M2 slot for SSD's
And what about partitions?
Can i boot/save the backup partition from one of those SSD's?

are you sure this devices currently support partitions?

https://help.mikrotik.com/docs/display/ROS/Partitions

"Partitioning is supported on MIPS, TILE, and PowerPC RouterBOARD type devices."

guipoletto · Sun Feb 20, 2022 10:39 pm

are you sure this devices currently support partitions?

I sure hope they DO

If not, then I pity the adventurous admin that decides to ever update firmware on one of these boxes, and be faced with a "fix by netinstall only, run-to-site" situation.

it is one thing to face such a prospect with a HAP-Lite;
it is suicide to face such a situation in a core-router / ISP scenario with 100Gb/s worth of customers, running to your house, with torches and pitchforks on hand.

guipoletto · Sun Feb 20, 2022 10:49 pm

are you sure this devices currently support partitions?

https://help.mikrotik.com/docs/display/ROS/Partitions

"Partitioning is supported on MIPS, TILE, and PowerPC RouterBOARD type devices."

I was unaware "partitions" was not officially supported on ARM

My experience with ARM so far, is restricted to the CCR2004, which supports partitioning, at least on V6.

Mon Feb 21, 2022 12:16 am

are you sure this devices currently support partitions?

https://help.mikrotik.com/docs/display/ROS/Partitions

"Partitioning is supported on MIPS, TILE, and PowerPC RouterBOARD type devices."
I was unaware "partitions" was not officially supported on ARM

My experience with ARM so far, is restricted to the CCR2004, which supports partitioning, at least on V6.

looks like you are managing to do it with 128mb of storage on ccr2004

dakobg · Mon Feb 21, 2022 9:01 am

I'm not sure why "partitions" functunality must be connected with cpu arch?!

PortalNET · Mon Feb 21, 2022 11:55 pm

I think this one is going to be our replacement for CCR1072 s.
Any idea on the release date ?

https://www.balticnetworks.com/docs/Mik ... asheet.pdf
https://www.youtube.com/watch?v=7_uLxZYYEpQ

OMG........ thats not funny.. mikrotik don´t know how to play anymore.. 2 100G.......seriousssssssssss OMG...... hehehehhehe 16 core ARM kkkkkk pricelist? só i can sell my house and save up some cash.

Tue Feb 22, 2022 12:46 am

I'm not sure why "partitions" functunality must be connected with cpu arch?!

Every CPU type must have a different boot loader.

Since I have yet to bother partitioning my MT devices, I'm not supporting the contention that MT ARM devices can't do it, just saying why it might be the case that some can't do it.

mobilexpi · Tue Feb 22, 2022 4:19 pm

That's indeed a very good question, I hope someone from Mikrotik can answer it.

We are using some CCR2004 for peering at some IXPs und also for our upstream connectivity. So they have to handle multiple full tables (850.000 IPv4 and 150.000 IPv6). I've seen Janis video about the CCR2216 and there it is said it can handel HW offloading for L3 IP routing. But it says only IPv4 and only 80K to 120K routes. What can I expect how the router behaves if I have a routing table with 850k IPv4 and 150K IPv6? Will it route some packets with HW offloading and the other have to travel through CPU?

Unbenannt.PNG

psannz · Tue Feb 22, 2022 5:01 pm

We are using some CCR2004 for peering at some IXPs und also for our upstream connectivity. So they have to handle multiple full tables (850.000 IPv4 and 150.000 IPv6). I've seen Janis video about the CCR2216 and there it is said it can handel HW offloading for L3 IP routing. But it says only IPv4 and only 80K to 120K routes. What can I expect how the router behaves if I have a routing table with 850k IPv4 and 150K IPv6? Will it route some packets with HW offloading and the other have to travel through CPU?

Unbenannt.PNG

Can you check how much traffic passes per route? In relation to your total traffic?
Pretty sure the vast majority of your traffic will flow over less than 1k routes in total. Thus, 79-119k routes left for the rest of the traffic to be routed in hardware, with only a tiny amount actually being routed by software (CPU).

nickcarr · Fri Feb 25, 2022 2:09 pm

Hi all.
I see that Router supports transceiver XS+2733LC15D. One for TX, one for RX (or TX/RX+TX/RX. Dunno*).
Now this scenario is new to me. How could Router handle this?
Have you got any example of conf or a manual?
Thanks in advance.

*edited

netzwerghh · Fri Feb 25, 2022 3:00 pm

Hi all.
I see that Router supports transceiver XS+2733LC15D. One for TX, one for RX (or TX/RX+TX/RX. Dunno*).
Now this scenario is new to me. How could Router handle this?
Have you got any example of conf or a manual?
Thanks in advance.

*edited

"Normal" SFP-Modules have one connector for RX and one for TX and on both sides of the connection are the same modules. This connection needs two fibers. One for data flow in the one direction and one for the other. The modules you are mentioning are transmitting both directions in the same fiber (only one fiber needed). To achive this, the two directions have different wavelengths or "colors". Therefore you need two different SFP modules on both ends. The one transmitting in color A and receiving color B. The other end has to be the other way around. That's why you buy these modules as pairs. They are NOT inserted in the same router but at both ends.
Those modules are using the wavelenghts 1270 and 1330. "Normal" single mode modules are 1310 only. And you use them normally for long distance connections that are leaving your building only. That's where they become handy. Long distance fiber lines can get expensive and with those BiDi-Modules (bidrectional) you only need one fiber not two.
For short distance you normally use multimode modules with two fiber.

nickcarr · Mon Feb 28, 2022 12:06 pm

Thanks so much.
Have a nice day.

Hi all.
I see that Router supports transceiver XS+2733LC15D. One for TX, one for RX (or TX/RX+TX/RX. Dunno*).
Now this scenario is new to me. How could Router handle this?
Have you got any example of conf or a manual?
Thanks in advance.

*edited
"Normal" SFP-Modules have one connector for RX and one for TX and on both sides of the connection are the same modules. This connection needs two fibers. One for data flow in the one direction and one for the other. The modules you are mentioning are transmitting both directions in the same fiber (only one fiber needed). To achive this, the two directions have different wavelengths or "colors". Therefore you need two different SFP modules on both ends. The one transmitting in color A and receiving color B. The other end has to be the other way around. That's why you buy these modules as pairs. They are NOT inserted in the same router but at both ends.
Those modules are using the wavelenghts 1270 and 1330. "Normal" single mode modules are 1310 only. And you use them normally for long distance connections that are leaving your building only. That's where they become handy. Long distance fiber lines can get expensive and with those BiDi-Modules (bidrectional) you only need one fiber not two.
For short distance you normally use multimode modules with two fiber.

nickcarr · Wed Mar 02, 2022 10:44 am

Hi all.
A question about transceiver compatibility with CCR2216-1G-12XS-2XQ
I'm ok with 25Gbps and Fiber to Copper transceiver, thanks to Mikrotik support.
I do not find -atm - transceivers for multimodal fiber 10Gbps (Mikrotik CCR to my Aruba and HP switches).
Thinking about Mellanox MFM1T02A-SR Compatibile 10GBASE-SR SFP+ 850nm 300m DOM LC MMF

Have you got some ideas about this or better?

Thanks in advance for any answer.

nickcarr · Wed Mar 02, 2022 3:46 pm

Hi all.
S+85DLC03D is the one we need.
(Thanks to Mikrotik support).

Gtrz

Hi all.
A question about transceiver compatibility with CCR2216-1G-12XS-2XQ
I'm ok with 25Gbps and Fiber to Copper transceiver, thanks to Mikrotik support.
I do not find -atm - transceivers for multimodal fiber 10Gbps (Mikrotik CCR to my Aruba and HP switches).
Thinking about Mellanox MFM1T02A-SR Compatibile 10GBASE-SR SFP+ 850nm 300m DOM LC MMF

Have you got some ideas about this or better?

Thanks in advance for any answer.

kingfly2000 · Fri Mar 04, 2022 5:59 am

I wish to buy a MikroTik CCR2216 for testing first, any 100 Gigabit QSFP28 transceiver module recommended for this router? it seems Mikrotik doesn't have any 100G transceiver module for this router

sunblade87 · Sun Mar 06, 2022 11:34 am

Is it possible to run CCR2216-1G-12XS-2XQ router at 48VDC?

jvanhambelgium · Sun Mar 06, 2022 1:36 pm

This this is gonna ROCK! The Marvell Prestera-DX8525 is a beast 24x25Gb Lanes wowzah, configured for 100Gbps to the CPU makes this thing is an absolute MONSTER! This CCR will dominate all other Mikrotik CCR routers ever made. Can't wait to get my hands on one these. Can you say 100Gbps between core nodes for a backbone WOW!!

I also wonder what price tag this will have

This is the real McCoy

(128 x 100Gb / 32 x 400Gb)
https://www.cisco.com/c/en/us/solutions ... 44312.html
Offcourse we are talking different price-tags here

(We have 400G between our core Internet nodes)

Sun Mar 06, 2022 11:08 pm

This this is gonna ROCK! The Marvell Prestera-DX8525 is a beast 24x25Gb Lanes wowzah, configured for 100Gbps to the CPU makes this thing is an absolute MONSTER! This CCR will dominate all other Mikrotik CCR routers ever made. Can't wait to get my hands on one these. Can you say 100Gbps between core nodes for a backbone WOW!!

I also wonder what price tag this will have

This is the real McCoy
(128 x 100Gb / 32 x 400Gb)
https://www.cisco.com/c/en/us/solutions ... 44312.html
Offcourse we are talking different price-tags here

(We have 400G between our core Internet nodes)

you must catch up to date

https://www.cisco.com/c/en/us/solutions ... 44833.html
(256 x 100GE / 64 x 400GE) switching / fabric

https://www.cisco.com/c/en/us/solutions ... or-ds.html
(192 x 100GE / 24 x 400 GE) routing processor with deep buffers

off course with ccr2216 2.800 USD full price you dont even pay the annual support of this hyperscale things, is completely out of context of this topic

Sun Mar 06, 2022 11:14 pm

Is it possible to run CCR2216-1G-12XS-2XQ router at 48VDC?

https://mikrotik.com/product/hot_swap__ ... or_ccr1072

There are four hot-swappable fans, as well as dual-redundant hot-swap power supplies. And these PSUs might look familiar if you have used our CCR1072 before: you can keep using your previous hot-swap power supplies and SFP modules.

You’re saving time and money on accessories, while also getting better performance with lower power consumption

craterman · Thu Apr 14, 2022 12:06 pm

This is very strange product.

CCR2216.PNG

Mikrotik team, why did you add 25G and 100G interfaces for that architecture? Obviously that we can use effectively only 10 interfaces with 10G mode. Because internal bus can pass through to CPU only 100G. You can say "we have HW offloading".

FDB.PNG

Yes but it is totally useless because it can handle traffic only for 128k routes. But what scenario do you offer to use this device? Where did you see the router with 25G and even more so 100G which use less than 128K routes? Of course I can think up this scenario but it will be limited cases.
And if we pay attention to test results for CCR2216 and CCR1072 with 25 ip filter rules or 25 simple queues we find that CCR2216 has much less performance than CCR1072. Which totally confirms that the router does not need 25G and 100G interfaces. Because there is no (ok, no so much) real scenario where we can use this interfaces.
I think it is absolutely marketing product and it is step back for mikrotik.

psannz · Thu Apr 14, 2022 2:40 pm

Mikrotik team, why did you add 25G and 100G interfaces for that architecture? Obviously that we can use effectively only 10 interfaces with 10G mode. Because internal bus can pass through to CPU only 100G. You can say "we have HW offloading".

Any IPv4 routing done within the routing tables of the switch chip will be done in wirespeed. That 100G link to the CPU is only relevant for the rest.
Have you actually checked on how many routes >90% of your traffic flows? You should.

Keep in mind that IPv6-L3HW has already been announced as being worked on.

craterman · Thu Apr 14, 2022 4:11 pm

Any IPv4 routing done within the routing tables of the switch chip will be done in wirespeed.

Not any. Only that traffic which traverse through the routes what will be fitted into the marvell chip's FDB. But this only 128k routes.

Have you actually checked on how many routes >90% of your traffic flows?

It does not matter. Because we can't influence to process of fitting routes into the marvell chip's FDB. So we don't know which traffic through which route will offloaded and which not. I don't want to play guessing game.

psannz · Thu Apr 14, 2022 6:35 pm

It does not matter. Because we can't influence to process of fitting routes into the marvell chip's FDB. So we don't know which traffic through which route will offloaded and which not. I don't want to play guessing game.

check out how does L3HW actually works?.
Some of what you're asking falls under the NDA mentioned by raimondsp.
Other parts you can answer yourself, if you compare the routes that transport most of your traffic in relation to how precise they are. Which should work out, considering you obviously have quite a lot of routes

craterman · Thu Apr 14, 2022 7:09 pm

check out how does L3HW actually works?.

I have already read it. And it is totally confirm my concerns. I don't understand what do you try to explain me?

Thu Apr 14, 2022 7:25 pm

Any IPv4 routing done within the routing tables of the switch chip will be done in wirespeed.
Not any. Only that traffic which traverse through the routes what will be fitted into the marvell chip's FDB. But this only 128k routes.

Have you actually checked on how many routes >90% of your traffic flows?
It does not matter. Because we can't influence to process of fitting routes into the marvell chip's FDB. So we don't know which traffic through which route will offloaded and which not. I don't want to play guessing game.

you can influence which routes get offloaded

https://help.mikrotik.com/docs/display/ ... gHWOffload

for example, nowadays 80% of internet traffic goes to and from a few CDNs is easy to achieve 80% offload only identifying that

craterman · Thu Apr 14, 2022 7:43 pm

you can influence which routes get offloaded

Realy? You got 1M routes or more through BGP. How could you influence which routes will be offloaded? OK with the help of filters we can suppress HW offload fore some routes. But we need to find these routes and make it manually. It is bull shit. I don't want a product with such challenges. I want a product which resolve my tasks but not that makes me additional tasks.

mkx · Thu Apr 14, 2022 8:32 pm

I want a product which resolve my tasks but not that makes me additional tasks.

L3HW documents pretty clearly describe limitations. So 128k routes are not enough for you, but might be enough for maby other potential users. I just don't see why are you bitching about it. You're welcome to purchase hardware from a different vendor that can route wirespeed millions of routes. I'm pretty sure price tag is higher as well, possibly more than proportionally, and performance comparison explains the reason. Devices capable of extremely high routing speed are niche product and niche products come with niche price. I don't see Mikrotik going into this niche but I might be proven wrong in the future.

craterman · Thu Apr 14, 2022 9:41 pm

I just don't see why are you bitching about it.

First of all I am not bitching. I ask you to be more respectful.
Secondly I did not say "what kind of trash have you created". I just disagree with positioning of product. They say "it is 100G router!!!". But it is not.
This product looks like big foot with engine from subcompact car. If mikrotik approached the development in a more balanced way and make it with 10-12 10G interfaces it could look more harmoniously. It's just my opinion.

mkx · Thu Apr 14, 2022 10:16 pm

I just don't see why are you bitching about it.
First of all I am not bitching. I ask you to be more respectful.

To me, your initial post in this thread (#42 above) is mostly bitching. Because you don't accept that there are numerous use cases where size of routing table is not decissive feature.

The product description "100Gbps router" is an usual marketing BS and it's always up to buyer to assess if the product is fit for certain use case. MT doesn't specify any particular use case for this device (or any other for this matter).
One possible use case is routing between a few connected networks and for that 128k routes can be enough and in this case device is capable of 100Gbps routing. Another use case, already mentioned, is core router of a (smallish) ISP or smallish company with even smaller number of routing table entries (easily under 1k) while still requiring high throughput. You're right that for some IX site this device is probably not fit, some heavy iron has to be used for that.

What will be definitely interesting to see how these devices will handle IPv6 routing ... directly connected subnets will probably kill them unless MT comes out with a really good HW routing table management.

craterman · Fri Apr 15, 2022 10:13 am

The product description "100Gbps router" is an usual marketing BS and it's always up to buyer to assess if the product is fit for certain use case.

Thanks for agreeing with me. That's exactly what I meant.
But unfortunately many users content with marketing BS and as result suffering. I don't want to protect them. But it is fact. I think you can't say "it is 100Gbps router" if this router only in some cases a 100Gbps router and in others not. Other vendors also embellish the performance of their devices but not so much.

What will be definitely interesting to see how these devices will handle IPv6 routing ... directly connected subnets will probably kill them unless MT comes out with a really good HW routing table management.

100%

StubArea51 · Fri Apr 15, 2022 5:04 pm

Realy? You got 1M routes or more through BGP. How could you influence which routes will be offloaded? OK with the help of filters we can suppress HW offload fore some routes. But we need to find these routes and make it manually. It is bull shit. I don't want a product with such challenges. I want a product which resolve my tasks but not that makes me additional tasks.

Instead of guessing, here are real performance numbers from our testing this week.

2.1 million (2,162,473) IPv4 routes converged in 44 seconds and withdrawn in 46 seconds under a 25G load with 12% CPU This equal to or faster than Juniper MX and Cisco ASR9K for the same number of routes.

Once I do some more testing, i'll be doing 50G and 100G testing while converging full tables. Will be adding the IPv6 full tables in next week for testing as well.

Details here:
viewtopic.php?t=185123

craterman · Fri Apr 15, 2022 6:22 pm

2.1 million (2,162,473) IPv4 routes converged in 44 seconds and withdrawn in 46 seconds under a 25G load with 12% CPU This equal to or faster than Juniper MX and Cisco ASR9K for the same number of routes.

Did you test it with HW offloading? Did you achieve the 25G traffic through 1 route? If so it is super synthetic test. Without filter rules and queue. Of course it will work. It is not interesting.
If you could test it with IXEA for example and generate traffic for packets to the different dst IPs (for about 600k routes or maybe 1M) it will be much interesting. To understand how will it work when some flows will be offloaded and some will processing through CPU.

StubArea51 · Fri Apr 15, 2022 6:37 pm

If you could test it with IXEA for example and generate traffic for packets to the different dst IPs (for about 600k routes or maybe 1M) it will be much interesting. To understand how will it work when some flows will be offloaded and some will processing through CPU.

We tested it without hw-offloading to start because the biggest delay for converging routes into forwarding is the push form RIB to HW FIB (ASIC) vs RIB to software FIB. We've only had the boxes for a week and are working on building Cisco TREX servers for extensive testing.

Nobody outside of hardware vendors uses IXIA anymore for general lab/validation work unless they just want to waste money, that was popular years ago. Cisco TREX ( a free and open source project) has become the benchmark of large enterprise, large service providers and cloud operators.

https://trex-tgn.cisco.com/

craterman · Fri Apr 15, 2022 7:59 pm

Nobody outside of hardware vendors uses IXIA anymore for general lab/validation work unless they just want to waste money, that was popular years ago.

I sad for example. Cisco TREX is good without further ado

We've only had the boxes for a week and are working on building Cisco TREX servers for extensive testing.

Will be waiting with impatience

PepeOnAchair · Fri May 27, 2022 2:08 pm

Hello,
do you please have any new test results yet?
Pep.

Larsa · Fri May 27, 2022 5:50 pm

Btw, for those who are interested it's possible to buy a bunch of fan trays to the ASR9000 series for the same price as a CCR2216. Well, not really but the price difference it quite astonishing. https://itprice.com/cisco-gpl/asr-9000

Fri May 27, 2022 8:30 pm

Btw, for those who are interested it's possible to buy a bunch of fan trays to the ASR9000 series for the same price as a CCR2216. Well, not really but the price difference it quite astonishing. https://itprice.com/cisco-gpl/asr-9000

or a power supply for ASR1001-X (remanufactured)

mercer2 · Sat Jun 04, 2022 7:14 am

finally my 40gb internet is lit, did some testing and was able to achieve 10g up and down. since my server only has a 10g nic

router is connected to a 40g internet and 100g lan core switch

need to upgrade server to test harder

rime · Thu Aug 25, 2022 9:38 pm

Hi,
Did anybody replace ccr1072 with this device and what is experience with this new mikrotik device ?

mada3k · Sat Aug 27, 2022 8:42 pm

"totaly useless" sound a bit over exaggerated. But it would be better if the used routes where offloaded, not everything.

mmuniz · Tue Oct 11, 2022 4:56 am

I have started to migrate from a CCR1072 to a CCR2216 and the NEW CCr2216 has been performing very bad
normally on the CCR1072 the CPU is at 50% with a traffic load of 16Gbs

The CCR2216 the CPU is at 85% with less than 11Gbs the tool/profile shows some unclassified process
but other than that there no other signs of the problem
Also there's not yet a full-routing only 19K routes on the router

Any one now any configuration to check for the bad performance or maybe the CCR2216 is not a replacement for the flagship ?

Tue Oct 11, 2022 6:00 am

i think doing an upgrade from ccr1072 to ccr2216 imply using 2216 on L3 HW offload mode, be careful some L3 functions still dont work like VRF

mmuniz · Wed Oct 12, 2022 4:28 am

Thanks for your comment It made me look for L3 Hardware Offloading information
have made some adjustments and now I got 14Gbs at 20% of CPU
basically for me the CCR2216 behaves more like a CRS 317 with routing advance features
Still having some issues with the BGP advertisement but traffic performance is OK for now

easyswiss · Mon Dec 26, 2022 4:57 pm

Our experience regarding exchange CCR1072 with CCR2216 is (same configuration, approx. 3x full table 10 peerings, 3-5 Gbit/s traffic, 1k DHCP, 50 FW rules, 1k NAT rules for CGN).

CCR1072 RouterOS 7.6 ~12% Load
CCR2216 RouterOS 7.6 ~20% Load (with HW offloading enabled)

Interesting is the higher number of ports, but performance wise the CCR1072 with the 72 core Tilera seems to be massively superior.
CCR2216 is top for the price. But I would like to see a more powerful CPU in the future. I think a device that costs about 10k would be a real breakthrough for Mikrotik as a flagship router. In the end everything comparable costs 20-100k. If we can replace 100k expensive devices with 10k devices, I think we and many would really hit it.

Interesting is that the CCR1072 under RouterOS 7 needs significantly less CPU (same configuration, approx 3x full table, 10 peerings, 0.5 Gbit/s Traffic, 100 FW rules)
Example:
CCR1072 RouterOS 6.49 ~5-8%.
CCR1072 RouterOS 7.6 ~2-3%

Mikrotik is on the right track but still has potential

Mon Dec 26, 2022 5:43 pm

dont look only your average cpu usage

in ccr1072 this metric tells little about real cpu load, because of the high ammount of cores average value is inherently low, only when all cores saturate you will see a high average value

sometimes 20-30% of average cpu usage on ccr1072 is a sign of an almost saturated state, same for 50-60% on a ccr1036

in comparison ccr2116/2216 average cpu load value is more representative because average a lower ammount of values because has 16 cores (heavier more capable cores i must say)

always check system -> resources -> cpu looking for most taxed core

also in

tools -> profile, you can find suprises about your actual load (try the all cores setting)

easyswiss · Mon Dec 26, 2022 7:34 pm

I think a comparison is only possible if you do the same thing on both routers.
If i compare the CPU. CCR1072 = 72 GHz (72x1GHz) CCR2216 = 32 GHz (16x2GHz).
The Tile could be stronger for multicpu applications. The ARM64 stronger for single-cpu applications.
At the moment, it's hard to compare.

CCR2216 with ~20% CPU load looks like this:

[xxx@xxxx] > /system resource print
                   uptime: 6d17h4m11s
                  version: 7.6 (stable)
               build-time: Oct/17/2022 10:55:40
         factory-software: 7.1.3
              free-memory: 14.7GiB
             total-memory: 15.9GiB
                      cpu: ARM64
                cpu-count: 16
                 cpu-load: 17%
           free-hdd-space: 82.1MiB
          total-hdd-space: 129.6MiB
  write-sect-since-reboot: 712282
         write-sect-total: 1690001
               bad-blocks: 0%
        architecture-name: arm64
               board-name: CCR2216-1G-12XS-2XQ
                 platform: MikroTik

[xxx@xxxx] > /system resource cpu print
Columns: CPU, LOAD, IRQ, DISK
 #  CPU    LOAD  IRQ  DISK
 0  cpu0   8%    8%   0%  
 1  cpu1   9%    9%   0%  
 2  cpu2   16%   16%  0%  
 3  cpu3   35%   34%  0%  
 4  cpu4   14%   14%  0%  
 5  cpu5   13%   13%  0%  
 6  cpu6   6%    6%   0%  
 7  cpu7   9%    9%   0%  
 8  cpu8   97%   14%  0%  
 9  cpu9   16%   16%  0%  
10  cpu10  19%   19%  0%  
11  cpu11  15%   15%  0%  
12  cpu12  10%   10%  0%  
13  cpu13  24%   24%  0%  
14  cpu14  13%   13%  0%  
15  cpu15  6%    6%   0

CCR1072 with 1% CPU load looks like this.

[xxx@xxxx] > system resource print 
             uptime: 1w1h55m45s
            version: 7.6 (stable)
         build-time: Oct/17/2022 10:55:40
   factory-software: 6.44beta34
        free-memory: 14.2GiB
       total-memory: 15.8GiB
                cpu: tilegx
          cpu-count: 72
      cpu-frequency: 1000MHz
           cpu-load: 1%
     free-hdd-space: 70.2MiB
    total-hdd-space: 128.0MiB
  architecture-name: tile
         board-name: CCR1072-1G-8S+
           platform: MikroTik
           
[xxx@xxxx] > system resource cpu print 
Columns: CPU, LOAD, IRQ, DISK
 #  CPU    LOAD  IRQ  DISK
 0  cpu0   0%    0%   0%  
 1  cpu1   0%    0%   0%  
 2  cpu2   0%    0%   0%  
 3  cpu3   0%    0%   0%  
 4  cpu4   4%    0%   0%  
 5  cpu5   0%    0%   0%  
 6  cpu6   0%    0%   0%  
 7  cpu7   0%    0%   0%  
 8  cpu8   100%  2%   0%  
 9  cpu9   0%    0%   0%  
10  cpu10  0%    0%   0%  
11  cpu11  0%    0%   0%  
12  cpu12  0%    0%   0%  
13  cpu13  0%    0%   0%  
14  cpu14  0%    0%   0%  
15  cpu15  0%    0%   0%  
16  cpu16  0%    0%   0%  
17  cpu17  0%    0%   0%  
18  cpu18  0%    0%   0%  
19  cpu19  0%    0%   0%  
20  cpu20  0%    0%   0%  
21  cpu21  0%    0%   0%  
22  cpu22  0%    0%   0%  
23  cpu23  0%    0%   0%  
24  cpu24  2%    0%   0%  
25  cpu25  0%    0%   0%  
26  cpu26  0%    0%   0%  
27  cpu27  0%    0%   0%  
28  cpu28  0%    0%   0%  
29  cpu29  0%    0%   0%  
30  cpu30  0%    0%   0%  
31  cpu31  0%    0%   0%  
32  cpu32  0%    0%   0%  
33  cpu33  0%    0%   0%  
34  cpu34  0%    0%   0%  
35  cpu35  0%    0%   0%  
36  cpu36  0%    0%   0%  
37  cpu37  0%    0%   0%  
38  cpu38  0%    0%   0%  
39  cpu39  0%    0%   0%  
40  cpu40  0%    0%   0%  
41  cpu41  0%    0%   0%  
42  cpu42  0%    0%   0%  
43  cpu43  0%    0%   0%  
44  cpu44  0%    0%   0%  
45  cpu45  0%    0%   0%  
46  cpu46  0%    0%   0%  
47  cpu47  0%    0%   0%  
48  cpu48  0%    0%   0%  
49  cpu49  2%    0%   0%  
50  cpu50  0%    0%   0%  
51  cpu51  0%    0%   0%  
52  cpu52  0%    0%   0%  
53  cpu53  0%    0%   0%  
54  cpu54  0%    0%   0%  
55  cpu55  0%    0%   0%  
56  cpu56  0%    0%   0%  
57  cpu57  0%    0%   0%  
58  cpu58  0%    0%   0%  
59  cpu59  2%    0%   0%  
60  cpu60  0%    0%   0%

tools -> profile, you can find suprises about your actual load (try the all cores setting)

I will try

Mon Dec 26, 2022 9:46 pm

i think direct comparison clock by clock between tilera cpu of ccr1xxx and arm cpu of ccr 2xxx is not accurate

this is because there is a big differential in arquitecture, tilera is in-order execution core, Vs Arm A72 out-of-order execution on CCR 2xxx, the ammount of instructions executed per clock is better on out-of-order execution

also there is a core clock advantage which impact single thread tasks

also remember, is NOT trivial to split a task between multiple cores for execution

ajgnet · Thu Dec 29, 2022 3:51 am

I am currently using a CCR2216-1G-12XS-2XQ with L3HW enabled in firewall compatible mode (enabled on the switch, disabled on the interfaces, fast-tracking established/related connections). On a saturated 10G line, my CPU is ~3% with around 4,000 tracked flows on the Marvell chip.

rushlife · Mon Jan 02, 2023 12:39 pm

Hi to all.
I wish all the best for all of you for year 2023.

Now back to the business.
I bought ccr2216 for main router of our company.

Here is my problem.
I am using qsfp ports (1 and 2), which are connected with DAC XQ+DA0001 to CRS326 units. One port to one crs326. Disabled auto negotiation. Rate is manually choosed 40Gbps.

To point of my problem. If I am transferring data between QSFP1 and QSFP2 ports, my CPU have hight utilization. One core is always (at least ) utilized around 95%. Speed of this data transfer is around 2.5Gbps which is NOTHING for this unit.

If I make FIRST firewall rule for forward of these two ports with action fasttrack connection, utilization of cpu is much lover and speed of data transfer much higher. 1 CPU core it around 60% (around 35% better than before) and speed of transfer is around 5Gbps.
Still, It should be around 10Gbps (at least) and that cpu utilization is pretty high for my taste.

rushlife · Mon Jan 02, 2023 1:55 pm

Update :
I managed to perform test where I was reaching around 10Gbps rate of data transfer.
Which is ok, server is connected via SFP+ 10Gbps link.

During this data transfer with speed around 10Gbps (about 1GB/s) between port qsfp1 and qsfp2 is ONE CORE of CPU (on CCR2216 unit ) ulitilzed around 91-95%.
Is this real performance ?
It is normal to have utilize one core to maximum value and have max performance around 10Gbps forwarding between ports qsfp 1 and 2 ?

Btw. I am transferring data with samba between two Windows 2019 servers.
I was also testing it with two NAS Synology servers.

easyswiss · Mon Jan 16, 2023 5:08 pm

We have found that disabling L3HW offloading reduces CPU usage by 50%. This is quite interesting and worth a test if someone has problems with the CPU. With RouterOS 7.7, L3HW offloading seems to cause package loss when the HW table is full.

Who is online