CCR1009 Rebooting

kevinds · Sun Nov 27, 2022 3:20 am

CCR1009-7G-1C-1S+ v7.6

Seeing this in the logs

router was rebooted without proper shutdown, probably kernel failure

then

kernel failure in previous boot

Any suggestions? Where should I look to get more information?

biomesh · Sun Nov 27, 2022 4:15 am

You might want to netinstall the device to make sure there isn't an issue with the flash or the os install. I would also check the capacitors to make sure none are failing. If you have a backup power supply, I would also test that as well.

kevinds · Sun Nov 27, 2022 4:24 am

You might want to netinstall the device to make sure there isn't an issue with the flash or the os install. I would also check the capacitors to make sure none are failing. If you have a backup power supply, I would also test that as well.

That mirrors my thoughts..

Cheers

nka · Sun Nov 27, 2022 5:27 am

I got the exact same problem with my CCR1009-8G-1S-1S+

I will try Netinstall.

edit: Won't do Netinstall... as it factory reset the routeur, I do not want that. Maybe 7.7 Beta? Altrought I don't see anything "fixing" in the release.

nka · Sun Nov 27, 2022 5:42 am

You might want to netinstall the device to make sure there isn't an issue with the flash or the os install. I would also check the capacitors to make sure none are failing. If you have a backup power supply, I would also test that as well.
That mirrors my thoughts..

Cheers

Update me if it fix it.

kevinds · Sun Nov 27, 2022 7:45 am

I got the exact same problem with my CCR1009-8G-1S-1S+

Wasn't a power supply issue.

Very interesting that you have the same issue... When did yours start?

I sent the supout to support, trying NetInstall now..

kevinds · Sun Nov 27, 2022 10:17 am

Update me if it fix it.

Been a couple hours and it hasn't rebooted..

First tried restoring a rsc export after the NetInstall, that didn't go well, didn't want to restore the full backup because I didn't want to restore the corruption.. But I did because the rsc import was not good..

Been a couple hours and still good.

Edit: Not anymore.. Rebooting again..

nka · Sun Nov 27, 2022 4:30 pm

I think it started at the beginning of the week. I'm not sure when I updated to 7.6, but I feel like it's related.

Tried 7.7Beta8 and it's still rebooting.

Wish I could go back to 7.5 and try, but the "Downgrade" options just reboot into 7.6

nka · Sun Nov 27, 2022 4:49 pm

ok, fond how to downgrade, pretty simple in fact.

Get older version here : https://mikrotik.com/download/archive (you need "tile" .npk for the CCR1009)

Upload into "Files", then go into "Package" and click "Downgrade". Router will reboot and be into the older version.

Trying 7.5 for now.
edit #1 : 7.5 crashing, trying 7.4.1 (can't recall what version I had before 7.6).

hughccl · Sun Nov 27, 2022 10:24 pm

I've got the exact same issue on a CCR1009-8G-1S-1S+ that has just started in the last couple of days. I was on 7.3 when it started, upgraded to 7.6 and it has crashed one more time a couple of hours later. Seems a bit of a coincidence that that three of us have the same issue starting around the same time with similar models??

kevinds · Mon Nov 28, 2022 5:24 am

Trying 7.5 for now.
edit #1 : 7.5 crashing, trying 7.4.1 (can't recall what version I had before 7.6).

Yes, I did the same, 7.5 still crashed for me too.

I had a transparent firewall set on two ports, removed them because the CCR1009 was unstable, the reboots stopped..

Tomorrow will check for blown caps but unplugging the two ports stopped the crashing.

Hoping support responds and doesn't just fix the issue with an update without telling me like last time..

nka · Mon Nov 28, 2022 6:07 am

7.4.1 seems to be stable for me, I'll update here if it still crash.

edit: crashing.

nichky · Mon Nov 28, 2022 7:51 am

i think is too early for netinstall.
what tool/profile showing?(keep it running)

on logical reason would be, that one of those parameters getting overloaded

kevinds · Mon Nov 28, 2022 8:12 am

i think is too early for netinstall.
what tool/profile showing?(keep it running)

on logical reason would be, that one of those parameters getting overloaded

Netinstall has already been done.

kevinds · Mon Nov 28, 2022 10:50 am

I've got the exact same issue on a CCR1009-8G-1S-1S+ that has just started in the last couple of days.

Seems a bit of a coincidence that that three of us have the same issue starting around the same time with similar models??

Indeed..

nka · Mon Nov 28, 2022 4:04 pm

i think is too early for netinstall.
what tool/profile showing?(keep it running)

on logical reason would be, that one of those parameters getting overloaded

didn't know about the tool/profile

It's currently running (I downgraded to 7.4.1), total is around 2 to 3.5, with "profiling" at around 2.5 and management 1.0, the rest is under 1.0 (most at 0.0).

Mine has been running fine for almost 24 hours now on 7.4.1. Might need to look at the changelog to see what is new on 7.5 that could cause that Kernel Panic.

I've also enabled syslog, hoping to see something in the log before it crash (if it does).

biomesh · Mon Nov 28, 2022 6:24 pm

Just FYI my CCR1009 was running 7.5 for over a month without issues. I replaced it with a CCR2004, so the 1009 is not currently in use.

The only issues I had with 7.x and the CCR1009 were/are:

- sd card no longer works
- in earlier 7.x version there was a bug with l2tp/ipsec where it would go into high utilization randomly. It has not happened since 7.3.x from what I remember.

kevinds · Mon Nov 28, 2022 8:22 pm

7.4.1 seems to be stable for me, I'll update here if it still crash.

7.4.1 just rebooted on me.

Less than 7.5 and 7.6 but still happening.

wispvt · Mon Nov 28, 2022 9:05 pm

I have a CCR1016-12S-1S+ doing the exact same thing. It had been fine for over a month on 7.6 but in the last week since 11/24/2022 it has rebooted around 10 times. Some of the times are shortly after a previous reboot. It can't be a power supply issue as it's a dual PS router and it's fed by a UPS which also feeds other devices without issue. Any way of getting Mikrotik to look at the supout files to figure out why?

kevinds · Mon Nov 28, 2022 9:09 pm

Any way of getting Mikrotik to look at the supout files to figure out why?

Open a support ticket and include the supout files..?

I just got a response that it might be related to connection tracking.

kevinds · Mon Nov 28, 2022 9:11 pm

7.4.1 just rebooted on me.

Less than 7.5 and 7.6 but still happening.

Every 30-60 minutes now, sometimes 5-10.

kevinds · Mon Nov 28, 2022 9:17 pm

what tool/profile showing?(keep it running)

on logical reason would be, that one of those parameters getting overloaded

Nothing of any significance, was watching it the last time a reboot happened.

kevinds · Mon Nov 28, 2022 9:22 pm

Just FYI my CCR1009 was running 7.5 for over a month without issues. I replaced it with a CCR2004, so the 1009 is not currently in use.

The only issues I had with 7.x and the CCR1009 were/are:

- sd card no longer works
- in earlier 7.x version there was a bug with l2tp/ipsec where it would go into high utilization randomly. It has not happened since 7.3.x from what I remember.

Ok...?

I've got a few of them, two of them are running very similar configurations, but only one is doing this.

nka · Mon Nov 28, 2022 9:41 pm

I have a CCR1016-12S-1S+ doing the exact same thing. It had been fine for over a month on 7.6 but in the last week since 11/24/2022 it has rebooted around 10 times. Some of the times are shortly after a previous reboot. It can't be a power supply issue as it's a dual PS router and it's fed by a UPS which also feeds other devices without issue. Any way of getting Mikrotik to look at the supout files to figure out why?

mine has been fine until last week, can't recall when exactly... pretty wierd. Look like it's something releated to a date (certificate?) or maybe an attack that target mikrotik router ?

tracert · Mon Nov 28, 2022 10:44 pm

Hi!

I have the same issue!
CCR1036-12G-4S
Rebooting probably kernel failure
SW, FW: 7.6
Rebooting random.

nka · Mon Nov 28, 2022 10:47 pm

7.4.1 seems to be stable for me, I'll update here if it still crash.
7.4.1 just rebooted on me.

Less than 7.5 and 7.6 but still happening.

crashed again for me too, about 35 hours later.

mjmabs · Mon Nov 28, 2022 11:37 pm

You can add another CCR1009-8G-1S-1S+ to the list, upgraded from 6.49.7 to 7.6 about two weeks ago and all the sudden today it has rebooted about 3 times in a 12 hour period. The CCR226's running 7.6 connected to same ups haven't rebooted at all. Tool/Profile shows very little usage currently last reboot was 1.5 hours ago.

hughccl · Mon Nov 28, 2022 11:43 pm

We switched over to a CCR-1009-8G-1S from our CCR-1009-8G-1S-1S+ and having the exact same issue on both 7.3 and 7.6.

Have downgraded to 6.49.7, will see how that goes...

kevinds · Mon Nov 28, 2022 11:52 pm

Have downgraded to 6.49.7, will see how that goes...

Going to try that tonight.

wispvt · Tue Nov 29, 2022 2:18 am

I rolled back to 6.48.6 to try as my reboots increased in frequency to every 5-15 minutes. I'll post results later.

kevinds · Tue Nov 29, 2022 8:20 am

Which RouterBoot version are you all running?

My affected unit rebooted twice on the 24th, then started frequently rebooting on the 26th.

wispvt · Tue Nov 29, 2022 2:32 pm

Yup, the 24th was the trigger date for us as well and then saw the frequency of reboots increase from the 26th on. After rolling back to 6.48.6 we've had no issues for 12 hours. It seems like it's only affecting tile processor routers from the posts.

JnSchl · Tue Nov 29, 2022 3:30 pm

Same happens on our device since the update to 7.6. It's CCR1009-8G-1S-1S+ (tile)

Log:

router was rebooted without proper shutdown, probably kernel failure

kernel failure in previous boot

Firmware version of the RouterBOARD always matched the installed firmware.

We tried: routeros-7.2.1-tile.npk
>> only stable for a few minutes

Now trying: routeros-tile-6.49.7.npk

tamisoft · Tue Nov 29, 2022 4:59 pm

Exact same issues with 7.6 on CCR1009-7G started about a week ago. Support suggested turning off the ip / firewall / service ports. Did that, no change. Waiting for a solution.

mjmabs · Tue Nov 29, 2022 6:44 pm

So the CCR1009-8G-1S-1S+ (7.6) that was effected by the reboots has been stable since yesterday around 1:15pm PST, I don't know if this is related or not but I noticed the ipv4 address of "36.156.28.131" attempting all types of connections with the router before I blocked it a short time later. The address is listed out of China and is on several blacklist for brute-force hacking attempts.

Jooos · Tue Nov 29, 2022 7:18 pm

CCR1009-8G-1S-1S+ v7.6

Same problem here, random reboots with error "kernel failure in previous boot", upgraded from 7.1.1 to 7.6 but still the same issue.
My CCR was running for years with 1 PSU without any issue.

nka · Tue Nov 29, 2022 7:34 pm

So the CCR1009-8G-1S-1S+ (7.6) that was effected by the reboots has been stable since yesterday around 1:15pm PST, I don't know if this is related or not but I noticed the ipv4 address of "36.156.28.131" attempting all types of connections with the router before I blocked it a short time later. The address is listed out of China and is on several blacklist for brute-force hacking attempts.

I'm pretty sure this is what happen right now. Its very date-related, so it look like an attack that target Mikotik Routeur.

I added this IP to my blacklist too, just in case.

akschu · Tue Nov 29, 2022 9:46 pm

I have a CCR1009-8G-1S here that I upgraded to 7.6 and immediately had reboot issues. After the 4th reboot in 2 hours I downgraded back to 6.49.7 and it's been stable since.

wispvt · Tue Nov 29, 2022 9:48 pm

It definately could be some sort of attack vector but the fact that it only affects tile version of routers running the exact same software version as other models makes it unlikely although not entirely impossible.

nka · Tue Nov 29, 2022 11:40 pm

I've sended two files (from two crash) to the support. Hope you have some news about that soon.

Ordghio · Wed Nov 30, 2022 12:30 am

Same issue by one of my CCR1009-8G-1S-1S+, not funny at all
11 reboots for last 24 hours

Maybe it's some sort of attack, but i have other 1009/1036 routers with similar configurations (and latest stable firmware - 7.6) and no any issues

JavierTF · Wed Nov 30, 2022 12:44 am

Hi all

We have exactly the same issue here with two CCR1009-8G-1S rigs. The first one was working for months with version 7.6 without any problem and suddenly, a few days ago it started to have reboots with the kernel failure message. We export it's configuration and reinstalled it on the seccond CCR (previusly upgraded to version 7.6) and immediately started experiencing the reboot issues. Try downgrading to version 7.5 and the same... I'm going to try downgrading to version 6 to see if something changes. From what I see in this forum, it seems to be a problem that is affecting devices with a Tilera processor... I'll update when I try version 6

Best regards

rememberme · Wed Nov 30, 2022 6:06 am

I'm in the same boat. There was a CCR1009 running 7.3 or 7.4 and started crashing last week. It was replaced today with a CCR1016 on ROS v7.6. Still rebooting.

I think it might be related to NTP. What I did to make it kernel panic is the following: set the time to 00:00:00 then disable/enable NTP client. Router rebooted right away. Then I tried simply disabling/enabling NTP client - it crashed. Lastly just disabled NTP client - crashed in less than 10secs.
I will leave the NTP client disabled for now and will keep an eye on how it does without it.

julianho · Wed Nov 30, 2022 8:53 am

The same problem occurred in the early morning of the 29th, Shanghai time zone. v7 versions of 1009 and 1036 had kernel errors, after downgrading to v6 version everything works fine.
BTW, all platforms such as mips, arm, and x86 had no issues, and it seems that there is only a problem with the tile.

tslytsly · Wed Nov 30, 2022 9:36 am

Hi,
We have the same with a CCR1009-8g-1s-1s+
It's been installed and working for over 8 years.
Running 7.6 for a few weeks. then started rebooting on the 26th.
Cannot see a pattern, it can go for hours without a reboot, or reboot every 10 mins or so.
Swapped in a new CCR1009-8g-1s with config from RSC, it is doing the same thing.

I have a ticket open with support.

I have the original 1009 on my test bench, not connected to anything and it has been running over 10 hours with no failure.
So it seems to be some external factor.

Wed Nov 30, 2022 11:40 am

We have lately received multiple complaints regarding Kernel Failures while running RouterOS v7. First of all, please note that we can not ever be sure if multiple users are suffering from the same issue without seeing the supout file. We can talk much about concrete configuration and network structure, but supout file speaks louder than thousand words. Without supout files, it is just guessing.

However, it seems that we have managed to resolve a very popular Kernel Failure issue today and the fix is on the way. It will be included in RouterOS v7.7beta9 and later releases. If your router is rebooting, then try out this release and see how it goes. If issues persist, then send supout files to support@mikrotik.com.

Remember - if your router is rebooting due to a software issue, then support should know about it. That is why you must contact support. Nobody In the forum will be able to fix the software. And support will not be able to tell much (besides simple guessing) without supout file.

xavierbt · Wed Nov 30, 2022 12:25 pm

I'm affected for tile router rebooting, too.

Model CCR1009-8G-1S-1S+
SW and FW 7.6
router was rebooted without proper shutdown, probably kernel failure
kernel failure in previous boot

Not sure when this begins but not many days ago. My router reboots a couple of times in a day, more or less.

tslytsly · Wed Nov 30, 2022 2:20 pm

We have lately received multiple complaints regarding Kernel Failures while running RouterOS v7. First of all, please note that we can not ever be sure if multiple users are suffering from the same issue without seeing the supout file. We can talk much about concrete configuration and network structure, but supout file speaks louder than thousand words. Without supout files, it is just guessing.

However, it seems that we have managed to resolve a very popular Kernel Failure issue today and the fix is on the way. It will be included in RouterOS v7.7beta9 and later releases. If your router is rebooting, then try out this release and see how it goes. If issues persist, then send supout files to support@mikrotik.com.

Remember - if your router is rebooting due to a software issue, then support should know about it. That is why you must contact support. Nobody In the forum will be able to fix the software. And support will not be able to tell much (besides simple guessing) without supout file.

Thank you for the update, I did log a support ticket as soon as we had the issue and provided supout files.
Can you give us an ETA on the fix? When will 7.6beta9 be released please?

JavierTF · Wed Nov 30, 2022 2:26 pm

Hi again...

Finally I could not test with version 6 since we have configured eoip over Wireguard tunnels and there is no support for Wireguard in version 6. We have tested with a third CCR1009-8G-1S and everything is the same, we have tested version 7.7beta8 of RouterOS and the same... we have tried various power supplies with different power and current and the same...

What finally worked for me:
Finally we have formatted with NetInstall and version 7.6 one of the devices that were experiensing problems and we have completely reconfigured it by hand (omiting importing the rsc file with the configuration of the previous device). We already have all the main services operational (including VPN's and VLAN's) and the equipment has been working without problems for more than 2 hours (before it did not exceed 35 minutes without restarting) I still need to apply some dst_nat rules to map some ports and others minor rules but in principle, every critical services that we could suspect that could be generating the problem, are already active... I will try to activate the remaining rules y services one by one to see if something triggers the problem and we detect it

nka · Wed Nov 30, 2022 3:35 pm

Remember - if your router is rebooting due to a software issue, then support should know about it. That is why you must contact support. Nobody In the forum will be able to fix the software. And support will not be able to tell much (besides simple guessing) without supout file.

Yes, but I think it's a good idea to come here first and see if there is already a solution and have a discussion other too, it's nice to see you are not alone!

That said, I'm testing the fix in the nightly build as proposed by the support, it's 7.7-alpha255. Will report back if it's crashing again (here and on support).

No, I won't provide the file, if you want it, ask the support!

tslytsly · Wed Nov 30, 2022 4:57 pm

I'm also testing the alpha version as provided by support.
Will report back results.
If you are having issues, make sure to open a support ticket, like the others have said, that is the only way to know if your issue is the same as the one they have a fix for.

tamisoft · Wed Nov 30, 2022 9:12 pm

The provided ROS7.7alpha255 nightly has fixed the issues for us. 7 hours no crashes. Thanks to the support's quick assist!

nka · Wed Nov 30, 2022 10:08 pm

The provided ROS7.7alpha255 nightly has fixed the issues for us. 7 hours no crashes. Thanks to the support's quick assist!

don't be too quick, I did more than 30 hours when I downgraded to 7.5

But they said they found why, so it look like it will fix it.

w4rh0und · Wed Nov 30, 2022 11:36 pm

I am having this issue as well since a few days ago

We have been on version 7.4.1 since August, and now we started with the reboot.

I've raised a task with support.

routerboard: yes
model: CCR1009-8G-1S-1S+
serial-number: 4AB1041DBDBD
firmware-type: tilegx
factory-firmware: 3.18
current-firmware: 7.4.1
upgrade-firmware: 7.4.1

w4rh0und · Thu Dec 01, 2022 10:50 am

The provided ROS7.7alpha255 nightly has fixed the issues for us. 7 hours no crashes. Thanks to the support's quick assist!

Quick couple of questions:
I've received the file from support as well.

I just copy it on the router and reboot?

Do I also have to upgrade the firmware? ( /system/routerboard/upgrade)

Have you encountered any issues with this alpha build?
The routers are in production, so it is a big gamble

Thank you

tslytsly · Thu Dec 01, 2022 11:10 am

The provided ROS7.7alpha255 nightly has fixed the issues for us. 7 hours no crashes. Thanks to the support's quick assist!
Quick couple of questions:
I've received the file from support as well.

I just copy it on the router and reboot?

Do I also have to upgrade the firmware? ( /system/routerboard/upgrade)

Have you encountered any issues with this alpha build?
The routers are in production, so it is a big gamble

Thank you

Hi,
I had to copy the file to the router then use the

/system/package/downgrade

function.
The router rebooted and then applied the package.
I have not updated the bootloader for now and it's been stable since.
So far no issues, but that is only on one router. so your mileage may vary.

Thu Dec 01, 2022 2:41 pm

The particular issue has been resolved in v7.7beta9. If you experience Kernel Failures even after applying the fix, please contact support@mikrotik.com and send a supout file.

tslytsly · Thu Dec 01, 2022 3:08 pm

The particular issue has been resolved in v7.7beta9. If you experience Kernel Failures even after applying the fix, please contact support@mikrotik.com and send a supout file.

Thank you.
Was this the issue?

*) conntrack - improved system stability when processing SCTP connections on TILE

I have to explain to my boss what the issue was.

Thanks

Thu Dec 01, 2022 3:17 pm

Yes, that was the issue for this particular "sudden" increased number of TILE routers rebooting. The problem was introduced in RouterOS v7. Most likely affected routers simply did not process any SCTP traffic before. From where did this traffic appear in your networks? Hard to guess. Maybe some kind of a worldwide DDoS attack.

tslytsly · Thu Dec 01, 2022 3:27 pm

Yes, that was the issue for this particular "sudden" increased number of TILE routers rebooting. The problem was introduced in RouterOS v7. Most likely affected routers simply did not process any SCTP traffic before. From where did this traffic appear in your networks? Hard to guess. Maybe some kind of a worldwide DDoS attack.

Thank you.

Would adding as RAW rule to drop SCTP work to prevent this if an update is not possible?

Thanks

nka · Thu Dec 01, 2022 3:49 pm

I can confirm 7.7-alpha255 (now Beta9) is stable on my end for about 24 hours now.

hex · Thu Dec 01, 2022 4:27 pm

Yes, that was the issue for this particular "sudden" increased number of TILE routers rebooting. The problem was introduced in RouterOS v7. Most likely affected routers simply did not process any SCTP traffic before. From where did this traffic appear in your networks? Hard to guess. Maybe some kind of a worldwide DDoS attack.

Then maybe filtering SCTP is better mitigation then installing alpha quality software to production routers?

w4rh0und · Fri Dec 02, 2022 8:24 am

I've created a RAW sctp block rule

/ip firewall/raw/add action=drop chain=prerouting comment="Drop all SCTP" in-interface-list=WAN protocol=sctp

So far no reboot for the past 11 hours.

If nothing else changes I might delay the upgrade until 7.7 full is released

tslytsly · Fri Dec 02, 2022 10:27 am

I've created a RAW sctp block rule
Code: Select all
/ip firewall/raw/add action=drop chain=prerouting comment="Drop all SCTP" in-interface-list=WAN protocol=sctp
So far no reboot for the past 11 hours.

If nothing else changes I might delay the upgrade until 7.7 full is released

Me too.
I added this block rule to one of our other 1036 routers, still on 7.6, and it's not rebooted.
I also added a rule above it to add the IP address to an address list.
So far, only one address has been added after 10+ hits.

167.114.50.149

I looked it up, it's a European cloud hosting provider.
So its very low level and single source for me so far.
Suspect its someone doing initial scanning or scouting?

nka · Fri Dec 02, 2022 3:26 pm

7.7 - alpha255 has been very stable for me. Will run on this until 7.7 full release.

hex · Fri Dec 02, 2022 6:20 pm

I've created a RAW sctp block rule
Code: Select all
/ip firewall/raw/add action=drop chain=prerouting comment="Drop all SCTP" in-interface-list=WAN protocol=sctp
So far no reboot for the past 11 hours.

If nothing else changes I might delay the upgrade until 7.7 full is released
Me too.
I added this block rule to one of our other 1036 routers, still on 7.6, and it's not rebooted.
I also added a rule above it to add the IP address to an address list.
So far, only one address has been added after 10+ hits.
Code: Select all
167.114.50.149
I looked it up, it's a European cloud hosting provider.
So its very low level and single source for me so far.
Suspect its someone doing initial scanning or scouting?

I have added logging now, let's see what comes out.
It's extremely strange that somebody is sending SCTP packets all over Internet.
I thought this bug must be time related, or somebody trying to attack.

w4rh0und · Fri Dec 02, 2022 6:59 pm

I've enabled logging yesterday as well, and all packets are received from 167.114.50.149 which is US

hex · Fri Dec 02, 2022 7:24 pm

I received from same address.
167.114.50.149

Fun.

tslytsly · Fri Dec 02, 2022 10:22 pm

I've enabled logging yesterday as well, and all packets are received from 167.114.50.149 which is US

NSA?

I wonder if it's someone testing an exploit?

ofca · Sat Dec 03, 2022 12:54 am

whatever they're doing, one of our CCR1016s was hit by this; it started on last sunday. Started dumping SCTP traffic today, and turns out it originated at the same IP. 7.7b9 didn't crash. As one would expect, routers just forwarding traffic are not affected; only those doing some conntrack-related activities, like NAT, crash.

There's nearly no SCTP traffic inbound to our network. It's few packets per hour, and I observed those two IPs.

00:07:45.986661 IP 146.190.28.73.57723 > X.X.X.X.36444: sctp (1) [INIT] [init tag: 2621498272] [rwnd: 32768] [OS: 10] [MIS: 2048] [init TSN: 1176166205] 
        0x0000:  6cb3 1151 01a4 0022 83dd ce48 0800 4500  l..Q..."...H..E.
        0x0010:  0034 45de 0000 f784 e413 92be 1c49 XXXX  .4E..........I..
        0x0020:  XXXX e17b 8e5c 0000 0000 39cd cd80 0100  ...{.\....9.....
        0x0030:  0014 9c40 e3a0 0000 8000 000a 0800 461a  ...@..........F.
        0x0040:  df3d                                     .=
00:09:32.344132 IP 167.114.50.149.43472 > X.X.X.X.6120: sctp (1) [INIT] [init tag: 2953808970] [rwnd: 32768] [OS: 10] [MIS: 2048] [init TSN: 1176166205] 
        0x0000:  6cb3 1151 01a4 0022 83dd ce48 0800 4500  l..Q..."...H..E.
        0x0010:  0034 b37a 0000 f384 4ec1 a772 3295 XXXX  .4.z....N..r2...
        0x0020:  XXXX a9d0 17e8 0000 0000 2fa7 5573 0100  ........../.Us..
        0x0030:  0014 b00f 8c4a 0000 8000 000a 0800 461a  .....J........F.
        0x0040:  df3d                                     .=

faxxe · Sat Dec 03, 2022 8:06 am

I received from same address.
167.114.50.149

Fun.

Same packets arrived here to my CCR1009. Thanks god it was filtered in a raw rule but i got an log entry:

Not TCP protocol prerouting: in:ether1 out:(unknown 0), connection-state:invalid src-mac 00:01:5c:b5:dc:46, proto 132, 167.114.50.149->xxx.xxx.xxx.xxx, len 52

I think there is an exploit being worked on

-faxxe

hex · Mon Dec 05, 2022 1:35 am

I have written to OVH abuse team with link to this thread.

yo1frenchtoast · Tue Dec 06, 2022 11:34 am

Here's my catches (GMT+1) :

 dec/02 12:20:20 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/02 12:20:20 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/02 17:48:59 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/02 17:48:59 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/03 09:53:23 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/03 09:53:23 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/03 14:27:02 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/03 14:27:02 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/03 15:45:53 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/03 15:45:53 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/04 00:33:44 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/04 00:33:44 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/04 00:33:57 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 00:33:57 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 03:54:16 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 dec/04 03:54:16 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 dec/04 06:23:49 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/04 06:23:49 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.149->86.215.xxx.y, len 52
 dec/04 10:34:59 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 10:34:59 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 14:38:42 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 14:38:42 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 18:25:08 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 18:25:08 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 167.114.50.148->86.215.xxx.y, len 52
 dec/04 23:21:05 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 dec/04 23:21:05 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 dec/05 23:17:46 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 dec/05 23:17:46 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 05:01:02 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52
 05:01:02 firewall,info drop_sctp prerouting: in:vlan832 out:(unknown 0), connection-state:invalid src-mac 24:21:24:xx::yy::zz, proto 132, 146.190.28.73->86.215.xxx.y, len 52

Raw filter rule works perfectly fine for me

Ordghio · Tue Dec 06, 2022 2:41 pm

My blacklist is the same:
146.190.28.73
167.114.50.148
167.114.50.149

inteq · Thu Dec 08, 2022 8:59 pm

Had a CCR1009 doing the same.
Weird thing is that is was doing the same even after I took it out of the rack and place it on my desk without internet.
A netinstall and the same config back and so far no reboots (not back in production tho). Mikrotik support stated they saw a problem from supout and I should try a beta.
Just replaced the router with a backup 1100AHx4 until they sort this stuff out in stable.

wispvt · Fri Dec 09, 2022 11:40 pm

Just had a CCR2004-1G-12S+2XS start doing the same thing. Filtered the SCTP traffic in RAW but to no avail so it's something else. Downgraded the router to 6.48.6 for the fix. So if it is an attack it has evolved to now include arm64 processors and possibly different protocol.

emisionlocal · Tue Dec 13, 2022 6:01 pm

Hello everyone, I have read all the threads of answers, I have been experiencing the same problem for several weeks, CCR-1009 8G, so far I have tried everything you say.

Let's hope there's luck, right now version:
7.7rc1

And the RAW port 132 rule added.

Let's see if we're lucky, I'll be attentive to this thread, since my reboots were continuous, the maximum without rebooting that I've been able to do was 20h and then at the end every 20 minutes, it seems that the problem once factory reset, little by little little, it is increasing.

jfarre20 · Sun Dec 18, 2022 1:06 am

My CCR-1036 has spontaneously started crashing. Today is the 2nd time this happened. Previously it crashed over and over for about 20 mins on Dec 8.

Today, I disabled the watchdog and watched as it slowly started dropping pings, winbox started getting slow, and then it hardlocked. Right before it went, I saw that the cpu was pegged at 30%, some cores maxed. I think its the same attack. Have the raw filter with the SCTP thing, but it still reboots. This went on for about 20 minutes - where it crashed a few more times.

But seems the attack has stopped now, Its back online and seemingly stable.

Running ROS 7.7RC1

inteq · Sun Dec 18, 2022 9:50 am

Found several other 1009s doing this.
Mikrotik support sent me an alpha version that might fix this.
Try routeros-7.7alpha255-tile-2.npk from https://box.mikrotik.com/f/61b327d9fb854902bd58/?dl=1
Or ask support.
Installing now and will see how it goes.

later edit: 6 days now and no more reboots.
No clue if the "attacks" stopped or this version fixed the problem.