Community discussions

MikroTik App
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2945
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 8:26 am

OVH has wrote a blog item on an attack done by Corerouters made by Mikrotik and you can read the article here: https://blog.ovhcloud.com/the-rise-of-p ... turn-evil/

Excerpt from the blog item.
We have been surprised to discover devices with a recent firmware being potentially compromised though. As far as we know, no vulnerability affecting RouterOS 6.49.14 and later versions have been publicly published so far. A possible explanation would be these devices may have been patched after their compromission.

We can’t say yet why these devices are involved in coordinated DDoS attacks, but one possible hypothesis could be the “Bandwidth test” feature from RouterOS. It allows the administrator to test the real throughput of a router by crafting packets and perform stress tests. Coincidentally, the documentation states the following: “Up to RouterOS version 6.44beta39, Bandwidth Test used only single CPU core and reached its limits when core was 100% loaded. Bandwidth Test [now] uses all available bandwidth (by default) and may impact network usability“. This is quite interesting since we mostly identified RouterOS v6.44 or above among the offending IPs.

My source form a Dutch security website: https://www.security.nl/posting/848681/ ... orerouters
 
Apachez
Member Candidate
Member Candidate
Posts: 148
Joined: Mon Jul 01, 2024 11:45 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 8:32 am

As my comment on similar topic in /r/mikrotik over at reddit:

Plenty of CVE score 9+ when it comes to Juniper and Cisco equipment aswell for the past years. Snowden docs was only the top of the iceberg with examples such as:

https://www.rapid7.com/blog/post/2015/1 ... -backdoor/

And then we have NSA infiltrating NIST:

https://en.wikipedia.org/wiki/Dual_EC_DRBG

Just to name a few from the past month (give or take):

https://supportportal.juniper.net/s/art ... uage=en_US

https://sec.cloudapps.cisco.com/securit ... i-WFFDnNOs

https://www.rapid7.com/blog/post/2024/0 ... isclosure/

But when it comes to Mikrotik its somewhat tricky to get it right specially if you are new to networking.

Would have been nice with a better hardened default config out of the box. But on the other hand admins exposing the mgmt-interface towards the internet is hard to beat other than with education and having the country CERT's through the ISP's shutdown those IP-addresses through blackholing (or sending notifications to abuse emails and hope for some action).

Also ISP's should be better to apply BCP38 that is filtering which source IP-addresses they are letting out to the rest of the internet:

http://www.bcp38.info/index.php/Main_Page

Forgot to add...

One could argue what does a bandwidth testing tool do in a switch or a router?

I mean removing that would make it much harder to generate traffic from an overtaken Mikrotik box?

But most modern switches and routers are today based on either linux or *bsd so even if there isnt a builtin bandwidth generating tool such can easily be uploaded specially when the mgmt interface is exposed to the internet.

All Arista boxes for example have iperf builtin since ages available both through the CLI in EOS but also in bash mode - and they have 800Gbps boxes nowadays... same with Cisco and Juniper who also have iperf available as a builtin feature in their modern routers and switches.

What OHVcloud perhaps is missing is that even if the multi 10 or now even 100G box of Mikrotik have a bandwidth generating tool its still limited by the mgmt-cpu which wont come anywhere close to 10 or 100G.

The one in CRS326 series for example will be able to push close to 500Mbps - which is still plenty if you have more than a few boxes but still. Many homeusers have 1Gbps today specially in western and northern Europe and each such windows box will be able to push far more of generated traffic than any Mikrotik box.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1691
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 9:51 am

Would have been nice with a better hardened default config out of the box.

Such as…?

If that includes a wish that the recent policy of random default passwords started much earlier, then I agree. But, go read all the threads here moaning about how terrible a burden it was when it finally did land.

(And it doesn't help that MT's stance was "the EU made us do this!" It shouldn't've taken continent-scale legislation to get this done.)

An LTS update service option would be nice, not so much for "stability" per se but because it would let people turn on auto-updates and not worry about it. I shouldn't have to be put at risk of downtime due to new features if I wasn't going to use those new features in the first place. Currently, opting out of unnecessary feature updates means no bug fixes after the last point release comes out.

But on the other hand admins exposing the mgmt-interface towards the internet is hard to beat other than with education

A Shodan search for "RouterOS" pulls up 1.15M boxes exposing something that reveals this detail, even though the stock WAN-side "input" firewall is fully blocked off. 9/10 of the first-page search results are currently SNMP, with the 10th being a public-facing Webfig instance. Why the entire Internet needs to see your SNMP is an unfathomable mystery to me. The Webfig thing I can at least understand, if not support.

BCP38

The attacker gains little from protecting their ill-gotten resource with source IP spoofing. There's plenty more zombie boxes to be had. By keeping the legitimate source IP, the compromised host's ISP may not even realize they're hosting an attack bot.

I'm not arguing against such filtering. Obviously it should be done. I merely question how much doing that would've helped in this immediate incident.

One could argue what does a bandwidth testing tool do in a switch or a router?

I mean removing that would make it much harder to generate traffic from an overtaken Mikrotik box?

Not really. Between things like containers, MITM methods, CA cert misuse…getting root on a core router opens a tremendous amount of powerful attack opportunities. Having a bandwidth test tool available isn't the key problem; it was merely the biggest sledgehammer immediately at hand once the bad guys got in.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10607
Joined: Mon Jun 08, 2015 12:09 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 12:07 pm

But when it comes to Mikrotik its somewhat tricky to get it right specially if you are new to networking.

Would have been nice with a better hardened default config out of the box.
When you are new to networking, setting up a core router maybe should not be your first task...
On the other hand, there probably is confusion about what "a core router" is, and how that matches with the CCR series of MikroTik routers.
It probably would be more prudent of MikroTik to deliver the CCR series of routers with the same default config as all the other models.
(ether1 is internet, all other ports in a bridge, NAT config, default firewall)

Those that have enough knowledge to set up a generic router do know how to erase that config, and at least those that buy e.g. a CCR2004 as the next step up above a RB5009 at a quite affordable price are not suddenly confronted with a completely unconfigured router that they have to configure with a working firewall etc.
 
jaclaz
Forum Guru
Forum Guru
Posts: 2593
Joined: Tue Oct 03, 2023 4:21 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 12:47 pm

It probably would be more prudent of MikroTik to deliver the CCR series of routers with the same default config as all the other models.
(ether1 is internet, all other ports in a bridge, NAT config, default firewall)
I think it would be enough to have (not only for the CCR series) an easily accessible repository with the "standard" (or "factory") configuration .rsc files, one for each model, it would IMHO be very useful also as a reference when trying to assist users on the forum having issues.
From what I have seen the usual plot is that the user changes settings (leaving the "defconf" comment) then, when he/she asks for assistance it is difficult to distinguish what settings belong to the original default configuration and what have been changed.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10607
Joined: Mon Jun 08, 2015 12:09 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 1:44 pm

I do not see why it would not be possible (and even be easier) to just deliver all MikroTik routers with the same structure of default configuration.
For the professional it is easy enough to run a "reset configuration" "no default configuration" and end up with the same thing as we have now.
But for the casual user who bought a CCR, it is very helpful.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1691
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 3:12 pm

an easily accessible repository with the "standard" (or "factory") configuration .rsc files, one for each model

Here's a start. Consider this an open solicitation for more files. I'm willing to take:

  • Descriptive diffs, e.g. "the RB5009 is like the RB4011 but with these few changes…"
  • Actual diffs against one of the files in that directory, posted either here or in that repo's sleepy side forum
  • Full-file uploads posted the same way

I do expect to need to hand-edit them somewhat. I won't repost PII like admin MAC settings, and I see no reason not to make diffs between exports easier to read by canonicalizing formats. For this reason, I prefer "/export terse show-sensitive" output, but stripped of anything truly sensitive; for a freshly-reset device, that shouldn't be anything more than the admin-mac setting.

All of this is intended in part as auxiliary info for my article on the default configuration. We'll have some number of terse exports for those who want a reference, and the [longwinded?] prose for those who want a tour and explanations.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 3281
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 3:21 pm

i think OVH blog post make clear, based and responsible hypothesis

but

internet has made his thing

Many pseudo sites in need of their daily dose of clickbait to move traffic, have distorted the main idea, using exagerated and misleading Headlines, leading the distracted reader to some false conclusions
 
User avatar
infabo
Forum Guru
Forum Guru
Posts: 1653
Joined: Thu Nov 12, 2020 12:07 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 3:38 pm

Closing note: we reached out to MikroTik through several communication channels to expose them the situation, but had no feedback so far.
The first rule of investigative journalism is to request a statement from the affected parties and wait for their response.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10607
Joined: Mon Jun 08, 2015 12:09 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 7:51 pm

And next write a letter to Ford or GM telling them that there are drivers without drivers license that crash their cars, and require them to send an immediate response???
It just makes no sense...
 
User avatar
infabo
Forum Guru
Forum Guru
Posts: 1653
Joined: Thu Nov 12, 2020 12:07 pm

Re: OCHcloud: When Core Routers Turn Evil

Fri Jul 05, 2024 10:10 pm

So you fan of "no comment" and denial?
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2945
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: OCHcloud: When Core Routers Turn Evil

Sat Jul 06, 2024 1:03 am

To me Mikrotik would no react on a request from blogpost writer.

Mikrotik writes:
We promise you that:

Your notification will be reviewed and if the problem will be discovered, you will be notified within 48 hours with acknowledgement of the issue
 
pe1chl
Forum Guru
Forum Guru
Posts: 10607
Joined: Mon Jun 08, 2015 12:09 pm

Re: OCHcloud: When Core Routers Turn Evil

Sat Jul 06, 2024 6:48 am

So you fan of "no comment" and denial?
What do you mean "denial". Do you consider it the manufacturer's responsibility that their products are only installed by competent installers?