Page 1 of 1
PPPoE server performance and packet lose problems
Posted: Wed Mar 19, 2008 5:23 pm
by rastod
Hi all,
we have big problems with performance of PPPoE server running 3.x versions. Today we run 3.4 version and still having problem. The problem is that we have packetlose on localhost (127.0.0.1) and all interfaces. It seems like it is a performance problem or problem of interface queues.
We run PPPoE server with quad core cpu xeon 2400, with board chipset intel 3000 server chipset with two gbit onboard ethernet cards. Traffic is up to 40/10mbit download/upload.
Enclosed is screen copy of our pppoe server. You can see packet lose to address 127.0.0.1 and high ping time response. Can someone observe the same problem?
We have this problem on all PPPoE server under heavy load.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 19, 2008 5:59 pm
by josefranco
I have this same problem here, I changed the machine and it remains to loose about 4% on all interfaces. Even with low CPU usage (20%) I have packet loss on all interfaces (even local).
I already tried to disable conntrack and TCPMSS to reduce processing but it didn´t solve the problem.
In fact I also saw this happening in 2.9.50 version, but not on early versions.
This is a huge problem that I cannot find a solution.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 19, 2008 8:03 pm
by rastod
At least, I am not alone.
And what are you doing now? Downgrade back to 2.9.xx?
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 19, 2008 8:24 pm
by rastod
I have new observation.
When I switched off "Change TCP MSS" on all used profiles, so there were no dynamic iptable's Mangle rules, the problem diminish. It is still here, but not so huge.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 19, 2008 10:39 pm
by josefranco
I only have one router using version 3.4 (the other ones still use old 2.9 versions).
For me disabling Change TCPMSS doesn´t solve the problem, I made this test today and even with this option disabled I continue to have packetloss (about 2%). Probably with TCPMSS off the packetloss is lower but it still not acceptable.
And I´m not talking about too much connections. With more than 100 pppoe connections it´s already possible to see packetloss on a Pentium D 3.4Ghz (with multicore enabled or not).
We have machines with a lot worst hardware than that using old 2.9 version and more than 500 users with 0% of packeloss..
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 2:52 pm
by josefranco
I think the best way is report it to Mikrotik support, and pray for a solution as soon as possible.
If everybody who has this same problem report it to Mikrotik support they will pay more attention to it.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 3:57 pm
by rastod
I did, but still no answer. I wanted to check if there are others that meet the problem.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 6:20 pm
by rkorolev
Do you use queues?
We have the same problem with packet loss and one of the problems is that ROS drops a packet every time a queue is created or deleted.
When many users connecting/disconnecting you get a packet loss.
Maybe that's not the only reason, but thats a reproducable and Mikrotik confirmed this issue.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 6:36 pm
by rastod
Yes, I suppose that it can be problem. But in older versions it was correct. Maybe MT should consider this.
But not only packet lose, also high time response to locahost ping is problem. Maybe it is common problem.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 9:29 pm
by josefranco
I use queues and I agree that it could be related to problem. But i can´t stop to use queues!
If this problem was not solved soon this will make the pppoe server resource unusable for me and everybody I know that use it.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 20, 2008 10:16 pm
by mrz
I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
*) few queues, instead of hundreds for each user
*) easy to manage
*) reduced CPU and memory usage.
Probably you can find more.
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 21, 2008 4:40 am
by bokili
When I write about this nobody responded
Even some of you write that we (my company) do not know to use mikrotik and other bullshits...
So:)? What now:) Who has the right about packet loss in v3.X ???
The problem is really in MT. From my experience I can tell you that there is not packet loss when you use mikrotik v3.X versions for other purpose, but especially when you use it together with PPPoE, packet loss appear immediatelly.
As number of pppoe tunnels grow, packet loss also grows. As bandwith in this router grows, packet loss also grows.
Then CPU Load go high as 90-100% and than router is not accessible.
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 21, 2008 7:20 am
by rkorolev
I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
Probably you can find more.
Static queues are not suitable for at least one situation: when you have a number of bandwidth plans and you set the speed with "Mikrotik-Rate-Limit" radius attribute.
Maybe you'll recommend, how to set up a speed limit with predefined static queue based on radius reply?
All users have dynamic IPs, so I can't just shape "that group of ips to 128k, other ips to 256k, etc".
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 21, 2008 10:45 am
by rastod
I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
*) few queues, instead of hundreds for each user
*) easy to manage
*) reduced CPU and memory usage.
Probably you can find more.
We terminate up to 700-900 users on one PPPoE server, I suppose it is not possible to deal with static queues. It would be possible only if services will be the same for all users or there will be some other limitations.
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 21, 2008 11:00 am
by rastod
It seems to be true - I watched log and ping and it seems that every time new PPPoE session is created or closed, packet lose occures.
So, it seems there are two problems:
- packet lose when creating/closing pppoe session
- high ping response to locahost when higher utilization even CPU is about 50%, so it can be problem of some queues like interface queues
Re: PPPoE server performance and packet lose problems
Posted: Mon Mar 24, 2008 2:57 pm
by josefranco
Use static queues to solve this problem is impossible. Queues are created dynamically by radius attribute.
Re: PPPoE server performance and packet lose problems
Posted: Mon Mar 24, 2008 10:07 pm
by bokili
Use static queues to solve this problem is impossible. Queues are created dynamically by radius attribute.
It will not help to resolve problem. We already tried.
Re: PPPoE server performance and packet lose problems
Posted: Tue Mar 25, 2008 10:44 pm
by josefranco
If using static queues doesnt solve the problem so it´s not related with the creation process of the queue.
Re: PPPoE server performance and packet lose problems
Posted: Tue Mar 25, 2008 10:54 pm
by rastod
I have reply from MT support, they confirmed problem and they are working on solution.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 2:19 pm
by josefranco
I have reply from MT support, they confirmed problem and they are working on solution.
Let´s cross our fingers and hope they find a solution as soon as possible.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 2:28 pm
by rastod
it seems they found. I am testing 3.7 and it works much more better...
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 3:41 pm
by josefranco
it seems they found. I am testing 3.7 and it works much more better...
Good news!
But where did you find this version?
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 3:45 pm
by rastod
MT sent me to check corrections.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 3:52 pm
by josefranco
MT sent me to check corrections.
Ok, I think they will release it soon..
How are your tests? It´s not loosing packets?
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 4:09 pm
by rkorolev
3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms
Before this I had 2%-5% loss on 200 packets.
Re: PPPoE server performance and packet lose problems
Posted: Wed Mar 26, 2008 4:22 pm
by josefranco
3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms
Before this I had 2%-5% loss on 200 packets.
Excellent result!!! My packet loss is the same you had with previous versions.
I´m waiting for this version release to solve my problem.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 27, 2008 2:03 am
by bokili
3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms
Before this I had 2%-5% loss on 200 packets.
This is great news!
Why do you have so big latency?
What queue type do you use? Is this over wireless or ?
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 27, 2008 7:58 am
by rkorolev
It's a DSL shaped to 128kbit and latency is pretty good (except "maximum", but maybe it was a lag on a line).
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 27, 2008 8:56 am
by rkorolev
Pity, but after a day of uptime it began loosing packets and a minimal ping is 60ms :-/
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 27, 2008 12:00 pm
by rastod
Yes, I can confirm this problem too. I have ping to localhost over 100ms. After restart it is ok. I've already informed MT, they are working on it.
Re: PPPoE server performance and packet lose problems
Posted: Thu Mar 27, 2008 2:22 pm
by rkorolev
And a new beta of 3.7 just arrived. Testing again...
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 28, 2008 12:08 pm
by gacopl
Hope that "7" will be lucky number I have same problem, but i rolled back to 2.9.51. I can't afford this to happen in my net.
Cheers
Michal
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 28, 2008 2:31 pm
by josefranco
Hope that "7" will be lucky number I have same problem, but i rolled back to 2.9.51. I can't afford this to happen in my net.
Cheers
Michal
I have this same problem since version 2.9.50, so rollback didn´t solve my problem. But with older versions (2.9.4x) works fine.
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 28, 2008 2:46 pm
by rastod
Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 28, 2008 3:36 pm
by gacopl
I have this same problem since version 2.9.50, so rollback didn´t solve my problem. But with older versions (2.9.4x) works fine.
You see, this is what worries me for me packet losses stopped as soon as i rolled back to 2.9.51, OMG maybe i have different problem
Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
Is there any way i could get as well that beta to test it?
cheers
Michal
Re: PPPoE server performance and packet lose problems
Posted: Fri Mar 28, 2008 4:41 pm
by josefranco
Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
Glad to hear
Re: PPPoE server performance and packet lose problems
Posted: Sat Mar 29, 2008 7:32 pm
by EgyCom
Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
any update ??
Re: PPPoE server performance and packet lose problems
Posted: Sat Mar 29, 2008 7:46 pm
by rastod
Still the same, working fine..., no problems as before.
Re: PPPoE server performance and packet lose problems
Posted: Sun Mar 30, 2008 1:55 am
by airnet
I recently upgraded a 2.9.26 PPPoE Server box to 2.9.51. Nothing has changed except the version.
What I see are users falling off one by one slowly with message logged 'peer not responding'. They log back in fine next time they retry after 30 secs (SOHO router PPPoE clients).
Over a 24hr period, maybe 50 users per PPPoE server willl fall off for no apparent reason.
The ability of 2.9.51 to 'hold' PPPoE sessions is not solid.
Can anyone confirm the last known good version of 2.9.x ?
Also, accoring to this thread, v3.7 should fix this problem. Does this mean they will also release 2.9.52 to fix the 2.9 branch ?
Re: PPPoE server performance and packet lose problems
Posted: Sun Mar 30, 2008 11:34 am
by rastod
We use PPPoE servers for thousands of PPPoE clients and we never had such problem. Check your connectino timeout settigs may be. There were another problems on 2.9.5x.
Re: PPPoE server performance and packet lose problems
Posted: Sun Mar 30, 2008 6:33 pm
by denio
When will this version 3.7 be available for download ?
We are having this packet loss problem in two of our pppoe concentrators (versions 3.1 and 3.6) and need to solve it really soon... We are having a lot of users compliants...
Thanks in advance.
Re: PPPoE server performance and packet lose problems
Posted: Sun Mar 30, 2008 11:15 pm
by gacopl
yeah we could really get use of that, even of that beta version you say is working correctly, so please if someone can please share.
cheers
Michal
Re: PPPoE server performance and packet lose problems
Posted: Mon Mar 31, 2008 9:36 am
by rkorolev
We still have problems with 3.7 beta, it works good only a short period of time.
Re: PPPoE server performance and packet lose problems
Posted: Mon Mar 31, 2008 9:52 am
by rastod
We do not have any problems with 3.7 build time Mar/27/2008.
We run it 4 days on 4 PPPoE server with 500-800 active connections each.
We have only little performance problems that are common for older versions 2.9.x
Re: PPPoE server performance and packet lose problems
Posted: Mon Mar 31, 2008 10:00 am
by gacopl
Then please share this beta somewhere or give us link to it or whatever
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 9:47 am
by santajosh
Hi EveryOne!
We also have this problem with our PPPOE server (600 users). Every day once we have to reboot the server (packetlose, performance problem)...
Where can we download the 3.7 beta??? Or when will be 3.7 ???
Thanks in advice!
Josh
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 10:03 am
by rastod
We tested the 3.7 package, from our point of view it is suitable to be released. But now it depends only on MT.
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 10:11 am
by santajosh
thanks!
So Normis???
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 5:35 pm
by josefranco
We do not have any problems with 3.7 build time Mar/27/2008.
We run it 4 days on 4 PPPoE server with 500-800 active connections each.
We have only little performance problems that are common for older versions 2.9.x
Just to know have kind of little performance problem you have?
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 5:58 pm
by rastod
1. when there are more users (700-900), we can see that sometimes ping to localhost is higher, up to 2-5ms, but not every ping, only 10-30%
2. we have problem that users are not able to get higher speeds than 15-20Mbit for download and 5-9Mbit for upload. But this is common problem with older 2.9.x versions and it is only one unpleasant feature of PPPoE compared to DHCP
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 02, 2008 10:03 pm
by josefranco
1. when there are more users (700-900), we can see that sometimes ping to localhost is higher, up to 2-5ms, but not every ping, only 10-30%
2. we have problem that users are not able to get higher speeds than 15-20Mbit for download and 5-9Mbit for upload. But this is common problem with older 2.9.x versions and it is only one unpleasant feature of PPPoE compared to DHCP
Thanks for reply. At least for my case these are minor issues.
Re: PPPoE server performance and packet lose problems
Posted: Thu Apr 03, 2008 10:17 am
by gacopl
Hello
Tell me guys is it so hard to put this build somewhere? Can't you see that there some desperate guys here, that really need to try this out? Please put it somewhere and share this can't be that hard.
Cheers
Michal
Re: PPPoE server performance and packet lose problems
Posted: Thu Apr 03, 2008 2:57 pm
by rkorolev
At least one more problem was found with test build.
Problem will be fixed in next build and at this point it looks like it will be
the last - so, please, wait for the release of 3.7
Re: PPPoE server performance and packet lose problems
Posted: Thu Apr 03, 2008 4:43 pm
by gacopl
At least one more problem was found with test build.
Problem will be fixed in next build and at this point it looks like it will be
the last - so, please, wait for the release of 3.7
Yeah, tell that to my users
I think i have a little more patience in me, hopefully my users too
Cheers
Michal
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 07, 2008 4:21 pm
by josefranco
3.7 was released today but changelog has nothing about PPPoE fixes!!!!!!!!!!!!!!!!
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 07, 2008 4:26 pm
by mietus
pppoe fixes are included but i dont know why not described in changelog :>
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 07, 2008 5:37 pm
by whitlebitle
pppoe fixes are included but i dont know why not described in changelog :>
Are pppoe problems really fixed?
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 07, 2008 9:16 pm
by hulk-bd
Yes pppoe problem is fixed take a look in this topic.
http://forum.mikrotik.com/viewtopic.php?f=2&t=23013
Thanks
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 3:40 am
by EgyCom
Today I upgraded one of our Access Concentrator to 3.7
I only load it with 10Mbps and 200 users, its work great, packet loss and latency issues vanished
I will try tomorrow with 40Mbps and 600-700 users
Good work Mikrotik team
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 8:59 am
by rkorolev
Currently 50-58mbit/s, 580 users (325 queues) - packet loss i still here
To RouterOS itself:
600 packets transmitted, 594 packets received, 1% packet loss
The absolutely same story via connected PPPoE:
Packets: Sent = 600, Received = 594, Lost = 6 (1% loss),
It's better than 2-5% loss in older versions, but our freebsd/pppoed server with 500 users online did not lose even 1 packet
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 9:59 am
by rastod
Currently 50-58mbit/s, 580 users (325 queues) - packet loss i still here
To RouterOS itself:
600 packets transmitted, 594 packets received, 1% packet loss
The absolutely same story via connected PPPoE:
Packets: Sent = 600, Received = 594, Lost = 6 (1% loss),
It's better than 2-5% loss in older versions, but our freebsd/pppoed server with 500 users online did not lose even 1 packet
Do you have experiences with freebsd/pppoe? Is it good solution? Why are you trying MT PPPoE?
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 10:27 am
by rkorolev
that's not a topic for routeros forum
you can write me a mail to
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 5:30 pm
by josefranco
Here are my results with version 3.7.
150 pppoe users, 8Mbits - 1000 packets - lost average 3 packets (it less than 1% but packet loss is still there).
I´m not happy with this result, old Mk versions never loose a single packet with much more users and traffic.
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 7:34 pm
by rkorolev
We also have packet loss 1% at least, but sometimes I've seen a 3% loss on a 600 packets
Re: PPPoE server performance and packet lose problems
Posted: Tue Apr 08, 2008 11:09 pm
by josefranco
I think we should report to Mk support that we are not happy with this solution, or else they will believe it´s fine.
I cannot accept a new version that works worst than an old version. And loose packets in LAN enviroment make this solution useless.
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 4:57 am
by EgyCom
I think we should report to Mk support that we are not happy with this solution, or else they will believe it´s fine.
I cannot accept a new version that works worst than an old version. And loose packets in LAN enviroment make this solution useless.
We didn't face any problem in 3.7 PPPoE
0 packet loss , with nearly 350 user
Packet loss only from 1 to 3 % for users with near there maxim speed limit and I think that is normal
We disabled compression and encryption, but still use Change TCP MSS
Server is 3.2 MHz, 512 Ram, multi-cpu=no
And we will try with more users ….
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 10:01 am
by rpingar
is the cpu sage little bit lower then the previous releases?
regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 12:25 pm
by EgyCom
is the cpu sage little bit lower then the previous releases?
regards
Ros
No I think it’s the same.
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 2:47 pm
by josefranco
It seems they fixed a problem with traffic shape queue creation but it also lose packets when an authentication request is denied (so this is not related with queues). If you have users that try to authenticante entire day you will lose packets.
I have an old machine (Sempron AMD processor) with Mk 2.9.44 with about 300 users, more than 15Mbits and never lost a packet, same Mk configuration and user profile. With new versions even with a Pentium D 3.4Ghz 512M RAM (Intel Gigabit PCI-E interf.) with multi-core on or off I have packetloss.
At some point near Mk version 2.9.50 it starts to lost packets on user authentication request (denied or not).
And remember I´m talking about lose packets even at local interface (inside my local network).
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 2:51 pm
by rkorolev
Yes, we have exactly same situation.
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 4:03 pm
by rpingar
Any experience on multi-cpu=yes about 3.7?
is it stable the smp verision about pppoe server?
regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Wed Apr 09, 2008 5:22 pm
by josefranco
Any experience on multi-cpu=yes about 3.7?
is it stable the smp verision about pppoe server?
regards
Ros
For us since 3.4 it was already stable with multi-cpu=yes, but it´s not helping to enable the second core since I continue to have dropped packets.
Re: PPPoE server performance and packet lose problems
Posted: Fri Apr 11, 2008 3:21 pm
by josefranco
Yes, we have exactly same situation.
Have you reported to Mk support you still have problem with it? We are trying to report to them but they insist it was solved (they really solved part of the problem). I know more people which continue to have this problem so I think everybody should report. Look like when you have a few more authenticantion request (even if all denied) it lose packets.
Re: PPPoE server performance and packet lose problems
Posted: Fri Apr 11, 2008 8:48 pm
by dsovereen
I sent an e-mail with a supout.rif, screenshot, and ping log of the problem.
I've found that doing:
ping 127.0.0.1 size=1500 interval=100ms
makes it really easy to see the problem. We run both PPPoE as well as DHCP. As Mikrotik indicated, it really does seem to be related to Simple Queues, because any time a PPPoE user disconnects/reconnects or a DHCP lease deassigns/assigned, both of which cause a Simple Queue to be dropped and added, we drop about 3 pings.
I haven't done enough testing to determine if it is the dropping of the queue or the building of the queue that causes the packet loss. Maybe it's both.
Everyone else who is seeing this, please send an e-mail to
support@mikrotik.com so we can hopefully get this fixed.
Dave
Re: PPPoE server performance and packet lose problems
Posted: Sat Apr 12, 2008 2:19 pm
by muriachi
Yes, we have exactly same situation.
me too
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 14, 2008 10:57 am
by macgaiver
So you have a big QoS system and some little packet loss?
Bus as far as I know it is possible to limit traffic ONLY by dropping packets!!!
regarding the constant packet loss - did you really determined it?
if you say you have 3% packet loss thats mean you should have the same number with 1pps 10pps 100pps and 1000pps.
I just checked same of my routers (not only mikrotiks) with your unique method (ping 127.0.0.1) and almost in all I was able to see some pings missing.
So bottom line is - do you have real packet loss problem???? I know only one way how to determine it is - online gamers (Quake, Unreal tournament, Counter-Strike) - those guys will be first who will find out even the smallest disturbance in the network...
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 14, 2008 2:28 pm
by josefranco
So you have a big QoS system and some little packet loss?
Bus as far as I know it is possible to limit traffic ONLY by dropping packets!!!
regarding the constant packet loss - did you really determined it?
if you say you have 3% packet loss thats mean you should have the same number with 1pps 10pps 100pps and 1000pps.
I just checked same of my routers (not only mikrotiks) with your unique method (ping 127.0.0.1) and almost in all I was able to see some pings missing.
So bottom line is - do you have real packet loss problem???? I know only one way how to determine it is - online gamers (Quake, Unreal tournament, Counter-Strike) - those guys will be first who will find out even the smallest disturbance in the network...
Packet loss occurs on interfaces that doesn´t have any QoS or queues applied. We are not talking about packetloss for a user thru Mk route. It happens on 100Mbits or Gigabit interface with no QoS applied. It must not happen on a regular router, never! I use Mk as PPPoE server for years and it never happens before. Something is clearly wrong here.
In my case this happen with only 200 users online, which is not much.
And if problem is only related to queues why I lose packets even when I authentication request is denied?
Re: PPPoE server performance and packet lose problems
Posted: Mon Apr 14, 2008 5:58 pm
by dsovereen
As best I can tell, the problem occurs where all packets on all interfaces are dropped for this period of time, regardless if they are packets to be shaped by the simple queue being added/removed. All packets on all interfaces drop for that 250 ms window (roughly) while the simple queue is being added/dropped. Yes, this is a very bad thing.
Dave
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 11:16 am
by rpingar
we are running 3.7 on our pppoe servers and are seeing packt loss and another strange issue:
when we run bandwidth test to any host directly connected to the switch about private network we are able to send only 10mbps of udp traffic at 1500.
If I run bandwidth test through a pc connected on the same switch we are able to send at full speed.
The pppoe servers are not configured with any qos system that can cut the bandwidth.
My idead is that on this scenario that could y a packt flow problem.
Regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 12:50 pm
by eugenevdm
Are you using simple queues? If so you should really be running a later version of RouterOS since a performance issue regarding simple queues was corrected in version 3.8.
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 12:57 pm
by rpingar
i think it was corrected into 3.7
I don't see any change loc about simple queue into the 3.8
regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 1:37 pm
by eugenevdm
Sorry you are correct:
What's new in 3.7:
*) improved simple queue list updating;
but I still suggest you start using version 3.9 because very few people, including probably the folk at Mikrotik, will help you with prior versions. The problem is Mikrotik 3.x is new and changes very quickly so it's mostly better to use the latest version. Then again I also realize there are risks associated with upgrading routers.
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 6:28 pm
by hci
We are running 3.9 on a PPPoE server serving about ~800 users. We are seeing all sorts of weird things. Some times proxy arp just does not seem to work right. Sometimes users just cannot go anywhere. And an old bug we thought we got rid of when we moved too 3.x of queues not always being enforced on PPPoE users. A user with a 256k profile will be seen running at 4.5m or so.
Tempted to downgrade too 3.3 or 3.4 release. Hopefully 3.10 will be out soon and solve all our problems?
Matt
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 11:22 pm
by eugenevdm
We are seeing all sorts of weird things.
Ouch. Just for interest sake did you encounter any problems with 2.9.51? Is there a specific reason why you upgraded to 3.x?
I upgraded 2 x PC routers to 3.9 and both crashed...I didn't experiment much thereafter because they were production routers so I quickly downgraded. Since then no problems.
Personally I would like to implement 3.x routers because I need faster / larger SATA hard drives and more RAM for my proxy. But I can wait for stability.
Re: PPPoE server performance and packet lose problems
Posted: Sun May 25, 2008 11:40 pm
by hci
Ouch. Just for interest sake did you encounter any problems with 2.9.51? Is there a specific reason why you upgraded to 3.x?
The main reason for the move to v3 was to get the fix for queues not always being inforced and moving to v3 seemed to fix it. We are short on bandwidth right now waiting for our new circuits to be installed and not inforcing queues was a real issue. Everything actually seemed to work fine with v3 until something like 3.5 or somewhere in there. Not at all sure when, its hard to keep track. I heard starting with 3.5 Mikrotik has a new kernel. Not sure if thats true.
Matt
Re: PPPoE server performance and packet lose problems
Posted: Mon May 26, 2008 4:18 am
by eugenevdm
As far as I know the Kernel change was between 2.9.x and 3.x.
Have you actually even tried 2.9.x and did you have a queue issue with that version?
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 12:36 pm
by santajosh
Any stable version for PPPOE? We use 3.10 but we have queue problem...
2.9.51? Anybody use 2.9.51 to PPPOE? We downgrade now to 3.3 but now we have the packet loss problem before 3.7...
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 1:16 pm
by rpingar
has someone that is running latest releas sent supout to MT about the not working queue?
I had this bug on some of the latest 2.9 releases. Now with 3.7 I dont see this behavior but i am still getting some packet loss.
My experience is that it is more pronunced on Broadcom tigon3 gbit ethernet then Intel gbit ethernet cards.
In fact i have two pppoe server with the same exact configuration and i see this problem only on the server equiped with broadcom tigon3 driver, on the other one with intel i don't lose the packets.
regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 2:24 pm
by eugenevdm
Anybody use 2.9.51 to PPPOE?
I use 2.9.x and 2.9.51 on various servers with PPPoE client on high sites. The largest site has about 100 PPPoE clients. I have never had any problems with queues.
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 2:57 pm
by rpingar
100 is a very small number.
You have to test them with 800-1000 clients.
Regards
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 3:09 pm
by normis
for so many clients, I hope you are using RADIUS authentication ...
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 3:26 pm
by rpingar
sure!
we have around 1200 on the intel gbit interfaces router and they are fine.
around 400 on the broadcom tigon and we are experiencing some little packet loss.......
same identical configuration to both machines. Supout sent two day ago to support.
Regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 5:00 pm
by josefranco
We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 5:24 pm
by rpingar
strange that the same hardware (dell poweredge 860) configured as core router so without any dynamic simple queue but with a queue tree is working great without any packet loss even pinging the pppoe server with broadcom ethernet nics.
The vice versa pinging from pppoe server to the core router I lose packets..........very strange
Ros.
Re: PPPoE server performance and packet lose problems
Posted: Tue May 27, 2008 5:35 pm
by hci
for so many clients, I hope you are using RADIUS authentication ...
Wouldn't an up to date PPPoE radius how too in the wiki be nice. Needs to cover IP pools, duplicate rejection, profiles and etc.
Matt
Re: PPPoE server performance and packet lose problems
Posted: Thu May 29, 2008 12:05 am
by rpingar
We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
Is it a multi-cpu box or an old fashion single core?
Regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Fri May 30, 2008 10:31 pm
by josefranco
We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
Is it a multi-cpu box or an old fashion single core?
Regards
Ros
It´s a P4 3.4Ghz dual-core with intel server mobo.
Re: PPPoE server performance and packet lose problems
Posted: Fri May 30, 2008 10:36 pm
by rpingar
uldis......
i think the actual packet loss is about an snm issue!!!!
this report from josefranco seems confirming my idea.
Regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Mon Jun 02, 2008 5:11 pm
by josefranco
uldis......
i think the actual packet loss is about an snm issue!!!!
this report from josefranco seems confirming my idea.
Regards
Ros
Sorry about my bad english but what´s a snm?
Just for information I had this same problem with a P4 3.0 single CPU on a totally different hardware. It starts to happen for me on 2.9.4x version.
Re: PPPoE server performance and packet lose problems
Posted: Mon Jun 02, 2008 5:22 pm
by rpingar
it means multi core cpu.
Regards
Ros
Re: PPPoE server performance and packet lose problems
Posted: Mon Jun 02, 2008 5:40 pm
by josefranco
it means multi core cpu.
Regards
Ros
In this case I don´t think it´s related to multi-core cpu. I had this problem with single core CPU too.