Please write with reports on nstreme single radio mode over large distances. We would like to confirm that you can get the max radio throughput over distance 50km+.
The single radio speed should be about 35Mb/s total for non-turbo mode. Turbo should be about 72Mb/s.
The newest package has additional support for the AR5004 (5213) and framing for the nstreme single radio mode.
http://www.mikrotik.com/download/nstreme-rev5.zip
John
http://www.mikrotik.com
N-Streme and WEP encryption
Eugene once wrote that the N-Streme package should also contain support for hardware WEP encryption.
My two test systems (both 2.8.10 with n-streme rev. 5 package, one AP, one client) refuse to talk when WEP encryption is switched on (/interface wireless security set wlan1 encryption=required).
The interfaces are an Atheros 5211 in one system, an Atheros 5212 in the other one.
Has anyone got this to work (or even tried it)?
My two test systems (both 2.8.10 with n-streme rev. 5 package, one AP, one client) refuse to talk when WEP encryption is switched on (/interface wireless security set wlan1 encryption=required).
The interfaces are an Atheros 5211 in one system, an Atheros 5212 in the other one.
Has anyone got this to work (or even tried it)?
I see that WEP was never a high priority for you - I have discussed this several times with Normunds and others. From my point of view I can understand your arguments that WEP is weak anyhow and one should use stronger encryption when wanting to secure a wireless link (IPsec, PPPoE with encryption, ...).
On the other hand if you want to operate a "standard" AP using RouterOS, you just have to support WEP correctly, as it is an expected feature. And the current implementation in software obviously takes lot of CPU power to get some throughput - a RouterBoard will only make perhaps 3 Mbit/s throughput, if I recall my last lab tests correctly.
I just put up this question, as Eugene directed me to the N-Streme package as a response to my question how far WEP hardware support was...
On the other hand if you want to operate a "standard" AP using RouterOS, you just have to support WEP correctly, as it is an expected feature. And the current implementation in software obviously takes lot of CPU power to get some throughput - a RouterBoard will only make perhaps 3 Mbit/s throughput, if I recall my last lab tests correctly.
I just put up this question, as Eugene directed me to the N-Streme package as a response to my question how far WEP hardware support was...
why bother?
As you said, WEP is not worth it. It's as easy to crack as they come and it just adds overhead to the packets.
It's like running ssh implementation with security holes. Would anyone sane do that intentionally?
There are far more secure and reliable ways to secure your network.
It's like running ssh implementation with security holes. Would anyone sane do that intentionally?
There are far more secure and reliable ways to secure your network.
Hi Tully. You say :
Are you saying that 5213 based cards will not work with nstreme2 yet? Do you have more details? The manual still says that only AR5211 and AR5212 MAC chips are supportedThe newest package has additional support for the AR5004 (5213) and framing for the nstreme single radio mode.
v2.8 does support the 5213, because the 5213 works with the same setup as the 5212.Hi Tully. You say :
Are you saying that 5213 based cards will not work with nstreme2 yet? Do you have more details? The manual still says that only AR5211 and AR5212 MAC chips are supportedThe newest package has additional support for the AR5004 (5213) and framing for the nstreme single radio mode.
The nstreme package has a new setup for the 5213 that will allow us to add the additional lower rates (coming soon) and may have some other improvements (we don't know all of what the changes mean as Atheros does not give this out).
John
Re: nstreme
what type of antennas should I use to get these speeds and distances (5.8 GHz band)?Please write with reports on nstreme single radio mode over large distances. We would like to confirm that you can get the max radio throughput over distance 50km+.
The single radio speed should be about 35Mb/s total for non-turbo mode. Turbo should be about 72Mb/s.
John,
I can confirm that it shows 35 Mbps in bandwidth-test over 57km link. This, when the link is not loaded with other traffic of course, and with both revision 4 and revision 5.
As far as I could tell, revision 4 too, handled somehow the framing-mode setting for single radio setup. (in short, I can't find the difference in both revisions :)
However, real-world performance does not seem to be that good ... CPU load is getting high on routers.. For an 4-hop link, I could get about 6 Mbit in single ftp session.
I have noted, that with nstreme, there are variable RTT times - the dispersion, even on idle link is significant.
For example, the same router, having both links (first without nstreme, second with nstreme), over ~57km distances (both links). Link 1 also has P3P enabled, with compress-all in both directions. When this test ran, there was around 3 Mbps traffic from link 1 to link 2;
link 1:
round-trip min/avg/max = 44/71.5/100 ms
link 2:
round-trip min/avg/max = 84/184.1/537 ms
Hope this sheds some light.
Daniel
I can confirm that it shows 35 Mbps in bandwidth-test over 57km link. This, when the link is not loaded with other traffic of course, and with both revision 4 and revision 5.
As far as I could tell, revision 4 too, handled somehow the framing-mode setting for single radio setup. (in short, I can't find the difference in both revisions :)
However, real-world performance does not seem to be that good ... CPU load is getting high on routers.. For an 4-hop link, I could get about 6 Mbit in single ftp session.
I have noted, that with nstreme, there are variable RTT times - the dispersion, even on idle link is significant.
For example, the same router, having both links (first without nstreme, second with nstreme), over ~57km distances (both links). Link 1 also has P3P enabled, with compress-all in both directions. When this test ran, there was around 3 Mbps traffic from link 1 to link 2;
link 1:
round-trip min/avg/max = 44/71.5/100 ms
link 2:
round-trip min/avg/max = 84/184.1/537 ms
Hope this sheds some light.
Daniel
35Mb/s is good for a 57KM link. I don't know if you are in turbo mode or not.
There are a number of optimizations we will be working on -- should be ready in about four weeks -- that will find the optimum frame size and rate and dynamically adjust it. At the moment, he rate is dynamically set, but the frame size is static.
I would prefer to see a bandwidth test from a celeron able router with 'randon-data' set to no. An single ftp test over four links is not best.
You might change the noise-floor setting to a higher number if you think your radios are not transmitting because they are so close together. Maybe -30 (I don't really know). As nstreme is polling, it should not matter to how the clients of the AP work.
John
There are a number of optimizations we will be working on -- should be ready in about four weeks -- that will find the optimum frame size and rate and dynamically adjust it. At the moment, he rate is dynamically set, but the frame size is static.
I would prefer to see a bandwidth test from a celeron able router with 'randon-data' set to no. An single ftp test over four links is not best.
You might change the noise-floor setting to a higher number if you think your radios are not transmitting because they are so close together. Maybe -30 (I don't really know). As nstreme is polling, it should not matter to how the clients of the AP work.
John
Ths 35Mbit at 57km is with non-turbo mode, unidirectional, with random-data=no. Machines on both ends are 700MHz C3. During the test, both routers (that have few more Atheros interfaces each), were not transporting traffic.
With unidirectional traffic, CPU load is around 30. With bidirectional traffic, it always hits 100.
Do you believe turbo mode will work well on such distances?
What might be causing the jitter? noise? If it is because of noise, why is the non-nstreme mode better?
Unfortunately, I have moved one antena on my slightly over 60km setup, that didn't work with 2.7 - might be good try.
Daniel
With unidirectional traffic, CPU load is around 30. With bidirectional traffic, it always hits 100.
Do you believe turbo mode will work well on such distances?
What might be causing the jitter? noise? If it is because of noise, why is the non-nstreme mode better?
Unfortunately, I have moved one antena on my slightly over 60km setup, that didn't work with 2.7 - might be good try.
Daniel
35Mb/s with non-turbo mode means that you have an excellent signal. You can use turbo mode -- you will have to see if it will improve the throughput.
We will check on the CPU usage with bidirectional.
I really don't know about the four hop setup. Can you check the speed between each hop and then go an additional one and so on.
John
We will check on the CPU usage with bidirectional.
I really don't know about the four hop setup. Can you check the speed between each hop and then go an additional one and so on.
John
Very similar results here...
Note: We only let our radios go to 24Mbps in order to improve link stability (might be better now that we are running 2.8.10 (will try 36Mbps and higher tonight)).
We are seeing about 15-19Mbps UDP and 10-16Mbps (TCP) across each of 4 50km+ links running nstreme. This is a BIG improvement per link. However, when we run test traffic across all 4 router links we only see 5-6Mpbs TCP. However the UDP numbers are still around 15Mbps. Our routers are 1GHz processors, 128Mb RAM with 2 or 3 wlan cards in each so power should not be a problem. I will run some more tests tonight and pay attention to processor utilization while I do it.
Any ideas why the routers slow down with TCP when passing traffic from link to link?
Thanks,
Ken
We are seeing about 15-19Mbps UDP and 10-16Mbps (TCP) across each of 4 50km+ links running nstreme. This is a BIG improvement per link. However, when we run test traffic across all 4 router links we only see 5-6Mpbs TCP. However the UDP numbers are still around 15Mbps. Our routers are 1GHz processors, 128Mb RAM with 2 or 3 wlan cards in each so power should not be a problem. I will run some more tests tonight and pay attention to processor utilization while I do it.
Any ideas why the routers slow down with TCP when passing traffic from link to link?
Thanks,
Ken
There is no need to restrict the higher rates of 802.11. There is an algorithm that calculates which rate will produce the highest throughput even with lost frames and such. Packets will never be lost unless the quality at the lowest rate is so bad that the client will be unregistered.
One session of TCP will never use the full throughput of a link. Many multiple sessions of TCP will get close, but a UDP bandwidth test will give the accurate number.
John
One session of TCP will never use the full throughput of a link. Many multiple sessions of TCP will get close, but a UDP bandwidth test will give the accurate number.
John
John,
What do you mean that packets will never be lost? Do you mean, that packets get lost only on congestion? I have found, that restricting the higher rates sometimes helps keep the RTT values reasonable.
As far as I understand the specifications of the Athros chipset, it will have higher output power at lower speeds, which I guess is to keep the average energy at different modulations 'consistent'. But this may also mean, that at different speeds, there is different signal distortion because of different amplification etc.
Daniel
What do you mean that packets will never be lost? Do you mean, that packets get lost only on congestion? I have found, that restricting the higher rates sometimes helps keep the RTT values reasonable.
As far as I understand the specifications of the Athros chipset, it will have higher output power at lower speeds, which I guess is to keep the average energy at different modulations 'consistent'. But this may also mean, that at different speeds, there is different signal distortion because of different amplification etc.
Daniel
Packets will never be lost unless the connection becomes so bad the the client is unregistered. The AP lowers the rate if all the retries at a higher rate fail. If the packet continues to fail until it hits the lowest rate, then the client is unregistered -- bacause the link quality is considered to be too bad.
The clients rate is increased as the algorithm determines that the throughput will be higher (accounting for the retransmits as well).
So, all of the rate calculations and such are done dynamically (and automatically) so that you do not need to set anything other than the defaults to get the highest throughput for your AP.
John
The clients rate is increased as the algorithm determines that the throughput will be higher (accounting for the retransmits as well).
So, all of the rate calculations and such are done dynamically (and automatically) so that you do not need to set anything other than the defaults to get the highest throughput for your AP.
John
Hello Daniel,Ths 35Mbit at 57km is with non-turbo mode, unidirectional, with random-data=no. Machines on both ends are 700MHz C3. During the test, both routers (that have few more Atheros interfaces each), were not transporting traffic.
Daniel
What size/make Antennas and/or amplifiers are you using to establish the 57km link at 35Mbits.
This particular link uses 1.4m antennas locally made in Bulgaria. The antennas are of reasonable quality and we use our own feedhorns. and in fact, The cabling is not to my taste however - at one end (50m tower) it has around 15m of HiTech 8D-FB cable - I would normally use Andrew LDF-4 for these links. I have seen huge differences when replacing the HiTech cable with LDF-4, even on shorter cables - probably due to connector quality. (the VSWR graph of the HiTech cable looks really awful)What size/make Antennas and/or amplifiers are you using to establish the 57km link at 35Mbits.Ths 35Mbit at 57km is with non-turbo mode, unidirectional, with random-data=no. Machines on both ends are 700MHz C3. During the test, both routers (that have few more Atheros interfaces each), were not transporting traffic.
Daniel
We do not use amplifiers. I have decided to stop using amplifiers few years ago with 2.4 GHz equipment, because, amplifiers generally produce only noise.
My current experiments show, that even much worse antennas (1.2 cheap antennas) give good results, probably due to improvements in MikroTik managing the Atheros chipset.
Daniel
We have removed NStreme from all of our routers until the winbox interface is available. We found that even though single links were very fast the routers (routerboards) could not keep up with the traffic. 4 links that were about 10-16Mbs (TCP, 50km) each would only yield 3-5Mbits of TCP through put. I tried many connections and could not get anything higher. I would LOVE to have a 10Mbs total throughput connections along these 4 routers but so far we get better total performance without NStreme.
Ken
Ps. I'm looking forward to being able to use NStreme through winbox so we can do easier testing and monitoring of the network.
Ken
Ps. I'm looking forward to being able to use NStreme through winbox so we can do easier testing and monitoring of the network.
There are several configurations that we need to test with to find the optimal one for the RouterBOARD 200s CPU power. I have a four boards next to each other and had 22Mb/s going through four hops (six radios, three segments -- with Ethernets feeding the routerboard). Notice, that means the at routerboard has 22 coming in and 22 going out -- 44Mb/s aggregate.
This coming week we will test for optimization.
John
This coming week we will test for optimization.
John
If you had 4 hops doing 22MBits per second on the bench then shouldn't 4 units with 10-16MBits on each link be able to do 10Bits across the whole link? I understand that there might be more processor power used so the link might be 9MBits but we are seeing 3-5Mbits or less (TCP). Now that we've move our routers to 2.8.11 without Nstreme we are seeing very consistent 5MBits TCP. Our results are much better UDP (better than 10Mbs) but even though you mention that this is a better measure of true throughput I cannot come close to making our radios do that kind of performance in TCP no matter how many connections I make.
Also, throughput is much improved when we take the data rates of radios from 54 to 24MBits. When the radio gears up and down it loses so much throughput that the whole link appears slower.
I really appreaciate the efforts and hopefully we can get NStreme to the point that we can get our 10MBits+ across our 4 routers spanning about 200km!
Ken
Ps. Can someone post a procedure for bandwidth testing that will give me the most accurate results for TCP and UDP (although TCP is really our main focus for 99% of our customers.)
Also, throughput is much improved when we take the data rates of radios from 54 to 24MBits. When the radio gears up and down it loses so much throughput that the whole link appears slower.
I really appreaciate the efforts and hopefully we can get NStreme to the point that we can get our 10MBits+ across our 4 routers spanning about 200km!
Ken
Ps. Can someone post a procedure for bandwidth testing that will give me the most accurate results for TCP and UDP (although TCP is really our main focus for 99% of our customers.)
----If you had 4 hops doing 22MBits per second on the bench then shouldn't 4 units with 10-16MBits on each link be able to do 10Bits across the whole link?
----
If one radio receives 22Mb/s and the other radio transmits 22Mb/s, then the router is doing 44Mb/s of throughput.
----
I understand that there might be more processor power used so the link might be 9MBits but we are seeing 3-5Mbits or less (TCP). Now that we've move our routers to 2.8.11 without Nstreme we are seeing very consistent 5MBits TCP. Our results are much better UDP (better than 10Mbs) but even though you mention that this is a better measure of true throughput I cannot come close to making our radios do that kind of performance in TCP no matter how many connections I make.
----
Then we need to do more checks. With v2.8.11 and the current nstreme revision of 7, we have added dynamic (optimized it some more) timing that should better allow the tcp not to get too much latency so that is slows (it has its own algorithm to set the speed). Perhaps you don't have the newest revision in testing.
----
Also, throughput is much improved when we take the data rates of radios from 54 to 24MBits. When the radio gears up and down it loses so much throughput that the whole link appears slower.
----
That shouldn't be the case, we will test it.
----
I really appreaciate the efforts and hopefully we can get NStreme to the point that we can get our 10MBits+ across our 4 routers spanning about 200km!
----
It shouldn't be too long.
----
Ken
Ps. Can someone post a procedure for bandwidth testing that will give me the most accurate results for TCP and UDP (although TCP is really our main focus for 99% of our customers.)
Latency and packet loss are the things that slow down TCP. As I don't think there is any packet loss, we will look into latency, but maybe this has been addressed in revision 7, and you need to check this. I can set up a six hop link and that should really show the results better. If there is still a problem, then I have another idea of where to look.
John
I recieve the exact same performance when we use 1 GHz processors doing the same work. We have not tested the latest version of NStreme and as our backbone is 200km long (and we like to be onsite for all firmware upgrades, in case a router locks up) I won't be testing NStreme unless it's on a bench. I am very much looking forward to the completed version of NStreme so that we can enjoy the increase in throughput! This kind of inovation blows my mind from such a fast paced company like MikroTik!and what about Duron/Athlon/Celeron/p4 machines with atheros cards? Isn't this low performance affected by routerboard platform alone?
Thanks again for all the efforts John!
Ken
I also see very low speeds with nstreme. I am using two 2.4Ghz PC's with 512MB RAM, Atheros AR5212 b/g cards running in G mode. I have tried so many different combinations of settings that I don't think there are any left ! During bench testing, using the standard "rubber duckie" antennas, I can only manage about 14Mb/s top speed, then, when I enable nstreme, I can only get 5Mb/s top speed. This is using the Mikrotik bandwidth tester going directly from one machine to the next using the TCP setting, and 1500 byte packets. I'm not sure what I'm doing wrong, but I am not seeing anything near to the speeds which other people seel to be getting. TO be honest, I don't mind it if I can get 5Mb/s, it's the distance I need, and if I can get 5Mb/s at 50km in the field, I'm sold.
I really think I should be able to get better figures than this though, anybody care to share their secrets on how they are achieving such high bandwidth, it would be much appreciated.
Regards
Paul
I really think I should be able to get better figures than this though, anybody care to share their secrets on how they are achieving such high bandwidth, it would be much appreciated.
Regards
Paul
hi phendry,
-71dbm for "the same room" is too low, probably 'antenna' {pigtail or other kind of output} orientation issue. But: seems like n-streme doesn't support G mode at all or turbo modes properly yet. Assure that you have enable 5ghz-turbo band in mikrotik on your wireless interface, both systems should give you more or less the same performance (for sure the difference can't be 24Mbps vs. 56Mbps, seems like you had mikrotik without turbo - that should lead you to 50Mbps).
we all have to wait for n-streme to grow up, but it shouldn't take too much time.
bye, mp3turbo.
-71dbm for "the same room" is too low, probably 'antenna' {pigtail or other kind of output} orientation issue. But: seems like n-streme doesn't support G mode at all or turbo modes properly yet. Assure that you have enable 5ghz-turbo band in mikrotik on your wireless interface, both systems should give you more or less the same performance (for sure the difference can't be 24Mbps vs. 56Mbps, seems like you had mikrotik without turbo - that should lead you to 50Mbps).
we all have to wait for n-streme to grow up, but it shouldn't take too much time.
bye, mp3turbo.