Under DNS Amplification attack, network unusable with Mikrotik routers

tin9 · Sat Jan 06, 2024 8:11 pm

Hello everyone,

We're having an issue at one of our clients, it seems that they have suffered a DNS amplification attack. There was port 53 udp open to the router (Mikrotik rb4011) from everywhere and the DNS Allow Remote Requests option has been enabled.

I saw immediately that there were thousands of open connections filling the bandwidth.

I have since disabled traffic to router on port 53 udp and tcp and the remote requets. The connections have returned to normal and the new rules are blocking approximately ˘1200 p/s. But the problem is that the internet connection is almost unusable. The ping to 8.8.8.8 takes 800ms+ and the overall network is very slow.

I have tried setting a mikrotik rb4011 with very basic configuration and it's having the same issues, the network is just almost unusable. We have tried setting the ISPs modem to router mode and the internet is working fine over their modem. But as soon as we connect the mikrotik as the router, everything grinds to a hold. Even the providers modem starts working very slowly. I have tried using the hap2 router but same issue.

We have ran out of ideas, is there and issue with the configuration, is the issue with routerOS? I have updated the routerOS to version 7.13 but no change. Bellow is the basic configuration I have tried with no luck, does anyone have any idea what can be done? Would changing the public IPs help in long term?

The router with port 53 specifically disabled

The basic configuration:

/interface bridge
add name=bridge1.test
/interface ethernet
set [ find default-name=ether1 ] name=ether1.dovod
/interface wireless
set [ find default-name=wlan1 ] ssid=MikroTik
set [ find default-name=wlan2 ] ssid=MikroTik
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1.dovod name=pppoe-out1 \
user=user
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=dhcp_pool0 ranges=10.0.10.20-10.0.10.150
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge1.test name=dhcp1
/interface bridge port
add bridge=bridge1.test interface=ether2
add bridge=bridge1.test interface=ether3
add bridge=bridge1.test interface=ether4
add bridge=bridge1.test interface=ether5
/ip address
add address=10.0.10.1/24 interface=bridge1.test network=10.0.10.0
/ip dhcp-server network
add address=10.0.10.0/24 gateway=10.0.10.1
/ip dns
set servers=193.189.160.13
/ip firewall filter
add action=drop chain=forward connection-state=invalid
add action=drop chain=input connection-state=invalid
add action=accept chain=forward connection-state=established,related
add action=accept chain=input connection-state=established,related
add action=accept chain=input dst-port=8800 in-interface=pppoe-out1 protocol=tcp
add action=drop chain=forward in-interface=pppoe-out1
add action=drop chain=input in-interface=pppoe-out1
/ip firewall nat
add action=masquerade chain=srcnat
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=2244
set api disabled=yes
set winbox port=8800
set api-ssl disabled=yes
/system identity
set name=mtTest

anav · Sun Jan 07, 2024 1:35 am

Sounds like YOU are the problem!

Lets look at the config.
1. Why do you slovenia telekom as your DNS server. If you want ISP provider DNS you can set that in the IP DCHP settings or pppoe settings for example (dial out).
Most folks use something like 1.1.1.1 or 8.8.8.8 for external servers...........
..

dnscl.jpg

....

2. Recommend netinstall the router and use deafult settings for most of the firewall rules. WINBOX is wide open to the internet!!!

3. What I dont understand is how DNS was open to the internet.
You have a rule, block all from WAN to router (input chain) which should have prevented this from happening.

tin9 · Sun Jan 07, 2024 11:44 am

Thanks for the reply anav.

1. Is there any meaningful difference if the DNS is gained dynamically or if it's set manually in the IP>DNS? We usually use the ISP providers DNS servers for our clients.

2. I have tried with new RB4011 and hap2 routers with very basic configuration, they all exhibit the same problem. The mikrotik is flooded with DNS traffic and the network is extremely slow (with huge pings). Winbox is not open to the internet, it's only open from our office and it's on non-standard port.

3. There has been a rule before which allowed DNS to be open to the internet. I have since disabled the rule and set new one, explicitly denying the DNS From the internet. But the issue with slow connection remains. Even if I disconnect all devices, the network is still very slow.

The attached configuration is just a testing configuration that I used on a hap2 to see if the network issue remains. This is the active FW configuration (note the open DNS connection there was before that i have disabled):

vingjfg · Sun Jan 07, 2024 12:47 pm

So, I assume (maybe I shouldn't) that you already rebooted the device.

What you may see is the effect of the ongoing attack plus some return traffic. Has it died off? If not you may try to ask the ISP to drop all traffic going to then external address, destination port 53.

As you had left an open resolver, your client's IP address is probably in several lists used by threat actors to run DDoS.

As Anav said, you have left Winbox open to the whole world. At the very least, limit it to the external IP of your company, or even better, deploy a VPN solution with MFA.

Lastly, and again sharing Anav's point of view leaving an open resolver is not best practice, and in all cases, not something any client should pay a consultant/service provider for.

gabacho4 · Sun Jan 07, 2024 3:56 pm

What is the cpu usage of your router right now? Have you tried creating a couple of raw (prerouting) firewall rules to drop incoming requests on tcp/udp port 53 so that your router isn't processing every request?

gabacho4 · Sun Jan 07, 2024 3:59 pm

When you say basic config do you mean a default one or one that you have made changes to? If the latter can you provide the full config? From CLI "export hide-sensitive file=WhateverYouWant".

EDIT: nevermind I just reread and saw that you had posted the config. Your Winbox, as noted, is completely exposed. You guys really need to go back to "making secure MT firewalls 101". I'm still curious what a couple of RAW firewall rules would do though the problem is that if a botnet is spamming your router (essentially a DDoS) you can't really defend against it and your ISP would have to intervene.

optio · Sun Jan 07, 2024 4:30 pm

@tin9 You can create filter rule to add IP which connecting to DNS port into address list for blocking and create raw filter rule that drops all packets in prerouting chain for IPs in that address list.
You can also consider setting up recursive DNS for your network if you don't want to rely on ISP DNS.

LdB · Sun Jan 07, 2024 4:33 pm

8.8.8.8 is probably throttling you

https://developers.google.com/speed/public-dns/docs/isp
First statement

High query volumes from a single IPv4 address (or IPv6 /64 network prefix) may be throttled if they exceed these limits.

anav · Sun Jan 07, 2024 5:17 pm

A default config should not slow the performance. As intimated, its probably residual blocking going on from leaving DNS open......

Note here how DNS is allowed ONLY from the LAN, and in fact is the only thing LAN users should have access to on the router itself and perhaps NTP (for certain devices).

/ip firewall filter
{Input Chain}
(default rules to keep)
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
( admin rules )
add action=accept chain=input src-address-list=Admin comment="Config Access"
add action=accept chain=input comment="Allow LAN DNS queries-UDP" \ {and NTP *** services if required etc}
dst-port=53,***123 in-interface-list=LAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries - TCP" \
dst-port=53 in-interface-list=LAN protocol=tcp
add action=drop chain=input comment="drop all else"
{forward chain}
(default rules to keep)
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
(user rules)
add action=accept chain=forward comment="allow internet traffic" in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="port forwarding" connection-nat-state=dstnat { disable if not required }
add action=drop chain=forward comment="drop all else"
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

No problem with allow remote DNS servers
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,9.9.9.9,8.8.4.4

Note Admin firewall address list (mostly static fixed dhcp leases)
/ip firewall address-list
add address=adminIP1 list=Admin comment="admin desktop/laptop wired"
add address=adminIP2 list=Admin comment="admin laptop local wifi"
add address=adminIP3 list=Admin comment="admin smartphone/ipad local wifi"
add address=adminIP4 list=Admin comment="remote admin laptop wireguardi"
add address=adminIP5 list=Admin comment="remote admin smartphone/ipad wireguard"

anav · Sun Jan 07, 2024 5:20 pm

Lastly, and again sharing Anav's point of view leaving an open resolver is not best practice, and in all cases, not something any client should pay a consultant/service provider for.

Find out who was responsible for those configs if provided by your company and if they are not gone, they should be fired. If it was you, resign immediately.
I cannot imagine anyone responsible for MT configs setting it up so, and thus suspect this was done previously perhaps by the client themselves.

Amm0 · Sun Jan 07, 2024 5:48 pm

I think this is right:

What you may see is the effect of the ongoing attack plus some return traffic. Has it died off? If not you may try to ask the ISP to drop all traffic going to then external address, destination port 53.

Maybe use ICMP reject (or blockhole), instead of drop in your port 53 FW rule MIGHT cause the traffic to recover... dunno but something to try since it's attackers that keeps trying (not the upstream DNS, which is likely the side-effect).

BUT... I'd really recommend just start again with a new config... I personally think the default firewall is very well-calibrated (e.g. generally modifying the interface-list to add an WANs should be needed for 99% of CPE use cases).

anav · Sun Jan 07, 2024 5:56 pm

BUT... I'd really recommend just start again with a new config... I personally think the default firewall is very well-calibrated (e.g. generally modifying the interface-list to add an WANs should be needed for 99% of CPE use cases).

Disagree, not just a new config, NETINSTALL first , then new config.

un9edsda · Tue Jan 09, 2024 4:42 am

I have tried setting a mikrotik rb4011 with very basic configuration and it's having the same issues, the network is just almost unusable. We have tried setting the ISPs modem to router mode and the internet is working fine over their modem. But as soon as we connect the mikrotik as the router, everything grinds to a hold. Even the providers modem starts working very slowly. I have tried using the hap2 router but same issue.

The basic configuration:
/interface bridge
add name=bridge1.test
/interface ethernet
set [ find default-name=ether1 ] name=ether1.dovod
/interface wireless
set [ find default-name=wlan1 ] ssid=MikroTik
set [ find default-name=wlan2 ] ssid=MikroTik
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1.dovod name=pppoe-out1 \
user=user
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=dhcp_pool0 ranges=10.0.10.20-10.0.10.150
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge1.test name=dhcp1
/interface bridge port
add bridge=bridge1.test interface=ether2
add bridge=bridge1.test interface=ether3
add bridge=bridge1.test interface=ether4
add bridge=bridge1.test interface=ether5
/ip address
add address=10.0.10.1/24 interface=bridge1.test network=10.0.10.0
/ip dhcp-server network
add address=10.0.10.0/24 gateway=10.0.10.1
/ip dns
set servers=193.189.160.13
/ip firewall filter
add action=drop chain=forward connection-state=invalid
add action=drop chain=input connection-state=invalid
add action=accept chain=forward connection-state=established,related
add action=accept chain=input connection-state=established,related
add action=accept chain=input dst-port=8800 in-interface=pppoe-out1 protocol=tcp
add action=drop chain=forward in-interface=pppoe-out1
add action=drop chain=input in-interface=pppoe-out1
/ip firewall nat
add action=masquerade chain=srcnat
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh port=2244
set api disabled=yes
set winbox port=8800
set api-ssl disabled=yes
/system identity
set name=mtTest

Since the slowdown only happens with having the MikroTIK as the router, while having the ISP's CPE not, it is clearly a configuration problem of the RB4011 still.

As people have already mentioned Netinstall, start using the stateless firewall RAW as

Firewall RAW table allows to selectively bypass or drop packets before connection tracking that way significantly reducing the load on CPU. The tool is very useful for DoS/DDoS attack mitigation.

(emphasis added), applying the Securing your router part of the documentation as well as the Building Advanced Firewall section plus the SYN/DoS/DDoS Protection section and having remote access to the router only via vpn therefore I will not repeat those advices.
However since the start from the default configuration (after netinstalling) rightful advice was given I would like to point out that in case of the RB4011 the default configuration does include an explicitly described Layer2 misconfiguration, namely the one explained in the VLAN filtering with multiple switch chips section of the documentation, therefore the first thing to do right after the Netinstall and before the rest of the above mentioned (on a router not connected to the Internet) is fixing this mistake of the default configuration by creating a second bridge for the ether6 to ether10 ports.

Under DNS Amplification attack, network unusable with Mikrotik routers

Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers

Re: Under DNS Amplification attack, network unusable with Mikrotik routers