How to limit PPPoE connection request attack?

josefranco · Wed Sep 03, 2008 4:00 pm

Hi, I´m looking for a solution to limit a PPPoE connection request attack (PADI). I have some users that sometimes sends a lot of PADI frames requests at same time and overload our radius server (even using a very fast machine and increasing a lot radius parameters).

I searched about this on Internet and discovered that professional PPPoE concentrators limits it natively with a max connection request per minute/second option for each MAC address (this is a very common attack for PPPoE). Well MKT PPPoE doesn´t have this kind of protection so I looking for a solution to stop this kind of attack.

The first thing I tried to do was to analyze radius log per minute and search for a MAC address which is sending too much connection requests and try to block it. The problem is that I cannot find a way to block a MAC address to not send more PADI frames to Mikrotik. I can only create rules in MK to block a PPPoE traffic in bridge mode but I cannot use bridge mode when acting as a PPPoE Server (at least not in the same interface).

I already asked to MK support team and they confirmed that it´s not possible to be done on the same MK machine (I need a second MK machine acting as bridge before PPPoE server just to do it, what is not a viable solution).

Some time ago I read something in this forum that someone did some trick to solve it but I lost this message.

If someone here have some idea to do that please I´m accepting suggestions.

omidkosari · Wed Sep 03, 2008 4:33 pm

interesting , today i was thinking in such post in forum and make a feature request. you did it first.

i think it is a MUST for each PPPoE concentrator . now i have more than 20 users which eat a lot of my resources.

about the bridge solution , i done it but if your pppoe users come from vlans this solution does not works and you just can block that mac address not just block the padi packets or limit them. i said the problem of the bridge to support and they said it will be solved in next version.

josefranco · Wed Sep 03, 2008 5:08 pm

interesting , today i was thinking in such post in forum and make a feature request. you did it first.
i think it is a MUST for each PPPoE concentrator . now i have more than 20 users which eat a lot of my resources.

about the bridge solution , i done it but if your pppoe users come from vlans this solution does not works and you just can block that mac address not just block the padi packets or limit them. i said the problem of the bridge to support and they said it will be solved in next version.

Thanks for you reply. At least I know I´m not alone with this problem. I didn´t think about using VLAN, it really complicate the problem.

Insert a Mikrotik acting as bridge before each PPPoE server we have will be very very complex and expensive solution.

I´m also looking for other solutions for PPPoE because we have too much problems with MK solution (Cisco, Juniper, etc). If you have some suggestions.

parrini · Wed Sep 03, 2008 6:00 pm

Hi, I´m looking for a solution to limit a PPPoE connection request attack (PADI). I have some users that sometimes sends a lot of PADI frames requests at same time and overload our radius server (even using a very fast machine and increasing a lot radius parameters).

I searched about this on Internet and discovered that professional PPPoE concentrators limits it natively with a max connection request per minute/second option for each MAC address (this is a very common attack for PPPoE). Well MKT PPPoE doesn´t have this kind of protection so I looking for a solution to stop this kind of attack.

The first thing I tried to do was to analyze radius log per minute and search for a MAC address which is sending too much connection requests and try to block it. The problem is that I cannot find a way to block a MAC address to not send more PADI frames to Mikrotik. I can only create rules in MK to block a PPPoE traffic in bridge mode but I cannot use bridge mode when acting as a PPPoE Server (at least not in the same interface).

I already asked to MK support team and they confirmed that it´s not possible to be done on the same MK machine (I need a second MK machine acting as bridge before PPPoE server just to do it, what is not a viable solution).

Some time ago I read something in this forum that someone did some trick to solve it but I lost this message.

If someone here have some idea to do that please I´m accepting suggestions.

This is a VERY interesting post to me, I am planning to phase out a FreeBSD PPPoE server and this is the single problem that is worrying me the most.

It's curious when you say you can't run a PPPoE server in a bridge because I do exactly that. I create a bridge with just one interface so I can place filters to pass only PPPoE Discovery and PPPoE Session protocols. Then I run the PPPoE server in that bridge.

My question is: assuming we can run the server in a bridge, how can we take advantage of that to limit the PADI packets? Has someone already implemented this? How the rules should look?

whitlebitle · Wed Sep 03, 2008 7:35 pm

This is very needed feature.. It will be great if mikrotik decide to implement it.

parrini · Wed Sep 03, 2008 7:37 pm

I think the tools to implement this are already in place, it's just a matter of figuring out.

josefranco · Wed Sep 03, 2008 8:31 pm

I think the tools to implement this are already in place, it's just a matter of figuring out.

I don´t think it. You can´t solve this problem with a single rule.
You can create a rule to limit PADI frames in brigde firewall table but you can´t do it per MAC address. If you create such kind of rule all PADI frames will be limit not only the attacker frames. MK bridge rules are based on ebtables which doesn´t have this option (like iptables hash-limit per user IP address).

So what I was trying to do is create an external script to logging into MK (by telnet for example since MK doesn´t have snmp write suppport) and insert individual rules for each MAC address generating an attack. Of course this is not a wonderfull solution but was the only way a found to reduce my problem.

josefranco · Wed Sep 03, 2008 8:41 pm

Hi, I´m looking for a solution to limit a PPPoE connection request attack (PADI). I have some users that sometimes sends a lot of PADI frames requests at same time and overload our radius server (even using a very fast machine and increasing a lot radius parameters).

I searched about this on Internet and discovered that professional PPPoE concentrators limits it natively with a max connection request per minute/second option for each MAC address (this is a very common attack for PPPoE). Well MKT PPPoE doesn´t have this kind of protection so I looking for a solution to stop this kind of attack.

The first thing I tried to do was to analyze radius log per minute and search for a MAC address which is sending too much connection requests and try to block it. The problem is that I cannot find a way to block a MAC address to not send more PADI frames to Mikrotik. I can only create rules in MK to block a PPPoE traffic in bridge mode but I cannot use bridge mode when acting as a PPPoE Server (at least not in the same interface).

I already asked to MK support team and they confirmed that it´s not possible to be done on the same MK machine (I need a second MK machine acting as bridge before PPPoE server just to do it, what is not a viable solution).

Some time ago I read something in this forum that someone did some trick to solve it but I lost this message.

If someone here have some idea to do that please I´m accepting suggestions.
This is a VERY interesting post to me, I am planning to phase out a FreeBSD PPPoE server and this is the single problem that is worrying me the most.

It's curious when you say you can't run a PPPoE server in a bridge because I do exactly that. I create a bridge with just one interface so I can place filters to pass only PPPoE Discovery and PPPoE Session protocols. Then I run the PPPoE server in that bridge.

My question is: assuming we can run the server in a bridge, how can we take advantage of that to limit the PADI packets? Has someone already implemented this? How the rules should look?

I thought to do exactly this, create a bridge with a single interface and create PPPoE on that bridge, but I thought this was a weird solution that could cause some problem or performance issue.

I saw we are on the same country so we could exchange some ideas about it.

parrini · Wed Sep 03, 2008 10:04 pm

...apparently there is no performance issue. It's pretty comfortable to filter every other protocols, saves a lot of headaches.

Looking closer, you are right about the rule being too broad and include non-attackers. I am shure we are not the first ones with this problem but I couldn't find any answer yet.

omidkosari · Thu Sep 04, 2008 9:53 am

i think this such bridge solutions are just for temporary deleting the question and it is not the answer.
mikrotik should implement a way like maximum pppoe requests per mac per minute and with one mikrotik not with another bridge.

parrini · Fri Sep 05, 2008 4:12 am

...apparently there is no performance issue. It's pretty comfortable to filter every other protocols, saves a lot of headaches.

Looking closer, you are right about the rule being too broad and include non-attackers. I am shure we are not the first ones with this problem but I couldn't find any answer yet.

Looking from another angle, limiting global PADI packets per second protects your Radius server and only penalizes new connections, not the established ones.

josefranco · Fri Sep 05, 2008 2:57 pm

...apparently there is no performance issue. It's pretty comfortable to filter every other protocols, saves a lot of headaches.

Looking closer, you are right about the rule being too broad and include non-attackers. I am shure we are not the first ones with this problem but I couldn't find any answer yet.
Looking from another angle, limiting global PADI packets per second protects your Radius server and only penalizes new connections, not the established ones.

Yes, but if you have a few users flooding with PADI frames other users will be unable to connect (I have users connecting all time).

omidkosari · Fri Sep 05, 2008 4:18 pm

Yes, but if you have a few users flooding with PADI frames other users will be unable to connect (I have users connecting all time).

i agree

as i mentioned before , the best way is maximum pppoe requests per mac per minute

hci · Fri Sep 05, 2008 7:51 pm

i think this such bridge solutions are just for temporary deleting the question and it is not the answer.
mikrotik should implement a way like maximum pppoe requests per mac per minute and with one mikrotik not with another bridge.

I agree. This would be a great feature.

Matt

sten · Tue Sep 09, 2008 5:24 am

Such a feature would be neat.
However, you can't have per minute mac-queues. Consider a flood of 100.000 requests with random src-mac-addresses?
Instead you would need a global queue that only responds to $X number of unique PPPoE-Discovery requests per second.

Something along the lines of:

* Make a queue that is $C long.
* Have a receiver thread that puts all incoming requests in a queue, if src-mac-address is not already in queue.
If queue is full, drop request.

* Have a responder thread;
Copy the queue to temporary buffer (pointer shuffle?)
Traverse the queue entry by entry and respond to $X requests.
Delay $W millisecond between each $X requests (wait between bursts as a trade off to scheduler efficiency).
Free/swap buffer and start over.

HOWEVER:

Neither a discovery throttling solution, nor encryption (MPPE), would negate the enormous weaknesses of PPPoE-session traffic.
And let's not forget the enormous weaknesses of the carrying L2 network that connects the PPPoE server with the PPPoE client.
If a user has L2 access to the core of the network then you are probably already at their mercy.
You could possibly use rate-limiting and bridge filter rules at customer edge to keep the needle eye small, but to remove it completely requires a different strategy all together.

So even though a discovery throttling solution would help the server stay running, when a normal pppoe-client is on the fritz, the core of the problem must be solved elsewhere (closer to the customer edge).

omidkosari · Wed Sep 24, 2008 10:38 am

now i have more than 50 ADSL users (which their credit is finished ) connect through pppoe and they have configured their modem to auto connect . each user have 3 pppoe connection per second . i think for an advanced concentrator there must be a feature to prevent this kind of problems.

think these users grow to 500 !!!
please dont say me to disable their ports etc . i know them but they are just for temporary solutions.

magic · Wed Sep 24, 2008 11:13 am

Omidkosari,

Allow the users to connect to the concentrator but give them address from a private address range (for example 192.168.0.x/24). Then add a few rules to the firewall to drop everything from the user except www (port 80) traffic. NAT your www traffic to a web server which have a simple web page which tells the users that their credit is finished and call your support to buy new credits.
Most of the Hungarian service provider do this. We had the same trouble and got a lot of call from the users about something wrong with their connection. Now the user see a web page and know what to do. And there is no connection requests.

Krisz

josefranco · Wed Sep 24, 2008 3:14 pm

now i have more than 50 ADSL users (which their credit is finished ) connect through pppoe and they have configured their modem to auto connect . each user have 3 pppoe connection per second . i think for an advanced concentrator there must be a feature to prevent this kind of problems.

think these users grow to 500 !!!
please dont say me to disable their ports etc . i know them but they are just for temporary solutions.

Advanced concentrators have exactly the feature I mention on this topic. At really I discovered this reading comercial PPPoE concentrators documentation. All them have this because it´s a very basic protection.

hci · Wed Sep 24, 2008 10:37 pm

We limit all PPPoE users to one session only. We have run into issue where end user puts PPPoE in both there PC and there router. Its weird, you would think the router would block the out going PPPoE request from the PC but not all router models do. A result is you constantly see one or other trying to log in and filling up log file.

What would be nice is a firewall rule that if it sees more then 5 failures in 60 seconds the MAC is banned(banned from log too) for 60 seconds and if it keeps trying its stays banned untill it stops trying for at least 60 seconds.

Kind of like the IPTABLES rules we use to block SSH brute force attacks on our linux servers:

http://kevin.vanzonneveld.net/techblog/ ... _iptables/

Matt

sten · Thu Sep 25, 2008 3:53 am

I have to agree with both "josefranco" and "omidkosari"
Basic discovery throttling should be available. Anything else could easily lead to unintentional DoS and pretty severe performance problems.
I mean, what if the discovery requests are sent with a spoofed src-mac-address at a very high rate and then you have PPPoE servers responding at that very high rate. We know what happens when the dst-mac-address does not exist on the L2 network, the packet gets flooded on all switch/bridge ports throughout that entire L2 network (some switch firmwares even flood it to all ports regardless of vlan). So not only is a high rate of broadcast generated from the client (which are to be sent to all on same L2) but also all the pppoe-discovery replies sent to everyone as well. That's a recipe for disaster. And the worst part is, it doesn't even have to be malicious intent behind it, just a pppoe client gone haywire.

Letni · Wed Sep 23, 2009 9:35 pm

Has anyone come up with a solution for this?
Has Mikrotik added the options to limit PPPoE connection requests?

-Louis

parrini · Sun Sep 04, 2011 8:06 am

*bump*

Zapnologica · Fri Oct 14, 2011 9:29 pm

Howist guys,

I also have the same problem.

My internet connection comes from a usb modem that my windows pc dials a pppoe connection and then connects.
The internet is then shared via a wan NIC to my router which then nats it and etc> this router has a pppoe server.

The server then also has a LAN nic which is used to access the server, but if the internet connection dies, the internet pppoe attempts to connect but it just connect to my routers pppoe connection over and over and it then never connects to the net! i have to disable the LAN then it connects and then i have to enable the lan?

isnt there a setting that tell the ppoe connection to move onto the next server when the 1st one denies it access??

or just block the mac / ip of the servers LAN nic? its very painfull and not very reliable as u have to manualy dial the connection every time?

EMOziko · Fri Jan 06, 2012 8:17 pm

This is very interesting. Any solution?

boardman · Sat Feb 11, 2012 8:42 pm

BUMP !!!

warn1ng · Fri Dec 07, 2012 1:06 am

Bump for this, i have issues with people flooding the pppoe server with requests

shap2001 · Wed Jan 09, 2013 10:28 am

I have exact same issue, with 3500 PPPOE sessions, and 300/s incoming bad requests (wrong credentials, expired, etc) my radius server going to explode soon! that's amazing 5 years of topic start and no answered yet

natanielklug · Thu Mar 14, 2013 4:59 pm

I have exact same issue, with 3500 PPPOE sessions, and 300/s incoming bad requests (wrong credentials, expired, etc) my radius server going to explode soon! that's amazing 5 years of topic start and no answered yet

Hello shap,

I would love to know how are you rinning 3,5k PPPoE sessions over Mikrotik. We are running about 400 sessions over each RB1100AH and that's our limit, more than that implies on hard latency. Can you tell us how to do it and what hardware are you using?

hci · Thu Mar 14, 2013 5:08 pm

Run over 1000 on x86 dual core. No issues.

natanielklug · Thu Mar 14, 2013 7:12 pm

Run over 1000 on x86 dual core. No issues.

Hello hci,

Can you provide me a more accurate info about the machine? Like:

[admin@CSC-auth-12] > system resource print 
                   uptime: 20h12m54s
                  version: 5.22
              free-memory: 1846632KiB
             total-memory: 1943428KiB
                      cpu: Intel(R)
                cpu-count: 2
            cpu-frequency: 2133MHz
                 cpu-load: 16%
           free-hdd-space: 240094612KiB
          total-hdd-space: 240292452KiB
  write-sect-since-reboot: 13818
         write-sect-total: 13818
        architecture-name: x86
               board-name: x86
                 platform: MikroTik
[admin@CSC-auth-12] > system resource pci print 
 # DEVICE   VENDOR                                                                 NAME                                                                       IRQ
 0 06:05.0  ATI Technologies Inc                                                   ES1000 (rev: 2)                                                             10
 1 05:00.0  Intel Corporation                                                      82572EI Gigabit Ethernet Controller (Copper) (rev: 6)                       11
 2 04:00.0  Broadcom Corporation                                                   NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev: 17)                     3
 3 02:00.0  Intel Corporation                                                      6702PXH PCI Express-to-PCI Bridge A (rev: 9)                                 0
 4 01:00.1  Intel Corporation                                                      82571EB Gigabit Ethernet Controller (rev: 6)                                11
 5 01:00.0  Intel Corporation                                                      82571EB Gigabit Ethernet Controller (rev: 6)                                 3
 6 00:1f.3  Intel Corporation                                                      82801G (ICH7 Family) SMBus Controller (rev: 1)                               0
 7 00:1f.2  Intel Corporation                                                      82801GB/GR/GH (ICH7 Family) SATA IDE Controller (rev: 1)                    11
 8 00:1f.1  Intel Corporation                                                      82801G (ICH7 Family) IDE Controller (rev: 1)                                 0
 9 00:1f.0  Intel Corporation                                                      82801GB/GR (ICH7 Family) LPC Interface Bridge (rev: 1)                       0
10 00:1e.0  Intel Corporation                                                      82801 PCI Bridge (rev: 225)                                                  0
11 00:1d.7  Intel Corporation                                                      82801G (ICH7 Family) USB2 EHCI Controller (rev: 1)                          11
12 00:1d.2  Intel Corporation                                                      82801G (ICH7 Family) USB UHCI Controller #3 (rev: 1)                         5
13 00:1d.1  Intel Corporation                                                      82801G (ICH7 Family) USB UHCI Controller #2 (rev: 1)                        10
14 00:1d.0  Intel Corporation                                                      82801G (ICH7 Family) USB UHCI Controller #1 (rev: 1)                        11
15 00:1c.5  Intel Corporation                                                      82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev: 1)                     0
16 00:1c.4  Intel Corporation                                                      82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev: 1)                     0
17 00:1c.0  Intel Corporation                                                      82801G (ICH7 Family) PCI Express Port 1 (rev: 1)                             0
18 00:01.0  Intel Corporation                                                      E7230/3000/3010 PCI Express Root Port (rev: 0)                               0
19 00:00.0  Intel Corporation                                                      E7230/3000/3010 Memory Controller Hub (rev: 0)                               0

In this machine we could only put something between 300 and 400 PPPoE sessions and the latency went to 30ms (our jitter got higher than 25ms and this is not good for voice applications). We run several EoIP tunnels and over each tunnel we run a PPPoE Server. Please see the attached images.

Can you describe your scenario and the latency from your border to your client?

hci · Thu Mar 14, 2013 7:32 pm

Xeon 3060 on Supermicro motherboard with integrated dual intel nics.

natanielklug · Thu Mar 14, 2013 7:38 pm

Xeon 3060 on Supermicro motherboard with integrated dual intel nics.

Thank you.

tamilmaran · Thu Oct 15, 2015 6:43 pm

Whatever the spec you had in mikrotik...
when PPPOE request attack happen from multiple devices , it definitely slow the authentication for all other PPPOE clients .

even you got less than 20% of cpu usage.

ISP People who told all are fine in Mikrotik PPPOE server means the reason behind it is they use less than 500 users or else they don't take much care about PPPOE authenciation delay happening to random clients..

Mikrotik guyz must listen to this thread , instead of more concentrating on MUM & New RouterOS Features ....

natanielklug · Thu Oct 15, 2015 9:07 pm

Whatever the spec you had in mikrotik...
when PPPOE request attack happen from multiple devices , it definitely slow the authentication for all other PPPOE clients .
even you got less than 20% of cpu usage.
ISP People who told all are fine in Mikrotik PPPOE server means the reason behind it is they use less than 500 users or else they don't take much care about PPPOE authenciation delay happening to random clients..

Mikrotik guyz must listen to this thread , instead of more concentrating on MUM & New RouterOS Features ....

Hello Tamil!

I have to tell you that I've read all the post to remember what I was asking. So, we are still using Mikrotik (CCR) for our PPPoE Cluster. Today we are running something between 1000 and 1200 PPPoE Sessions per NAS.

We have a clustes with 6 CCR1016 and there are about 5000 simultaneusly connections. It's not perfect but it is better than in the past few years.

We found out that version 6.26 has an issue when clients try to connect to Hotmail and some other HTTPs websites. First we tought it could be a MTU problem but later we discovered (by a lot of lucky from one of our engineers) that the version was the problem. Now we are updating them to 6.32.2 and seens to be working.

raminmalek · Fri Jul 27, 2018 7:30 pm

any solution Develop by mikrotik ???

flameproof · Tue Nov 05, 2019 11:10 am

Apologies for bumping this old thread, but the problem still exists. Example: we have ~1200 PPPoE clients in an area, after a massive power cut, they all come back online at once:

Screen Shot 2019-11-05 at 09.52.16.png

When this happens, our RADIUS server is suddenly hit by thousands of requests, because many of them time out, so for a while, most requests result in timeouts:

Screen Shot 2019-11-05 at 09.52.35.png

We need a way to be able to limit this. We operate in an area where power cuts, fiber cuts, etc. are frequent, so we cannot be subject to this level of flooding. We have fixed this somewhat by moving to RadSec (with UDP this would be tens of thousands of requests over minutes), but as you can see, it's not perfect either.

Any ideas?

MiguelYG · Thu May 12, 2022 11:18 pm

Hi! , any solution ?