N-Streme PtMP Latency\Jitter

Hammy · Wed Mar 07, 2007 4:47 pm

I am achieving some poor wireless jitter.

This is a PtMP link from the client to the tower.  300 - 400' link clear 
LOS -50 signal.

10.10.1.1 64 byte ping: ttl=64 time=13 ms
10.10.1.1 64 byte ping: ttl=64 time=5 ms
10.10.1.1 64 byte ping: ttl=64 time=10 ms
10.10.1.1 64 byte ping: ttl=64 time=8 ms
10.10.1.1 64 byte ping: ttl=64 time=19 ms
10.10.1.1 64 byte ping: ttl=64 time=5 ms
10.10.1.1 64 byte ping: ttl=64 time=11 ms
10.10.1.1 64 byte ping: ttl=64 time=11 ms
10.10.1.1 64 byte ping: ttl=64 time=8 ms
10.10.1.1 64 byte ping: ttl=64 time=10 ms
10.10.1.1 64 byte ping: ttl=64 time=8 ms
10.10.1.1 64 byte ping: ttl=64 time=7 ms
10.10.1.1 64 byte ping: ttl=64 time=20 ms
10.10.1.1 64 byte ping: ttl=64 time=10 ms
10.10.1.1 64 byte ping: ttl=64 time=9 ms
10.10.1.1 64 byte ping: ttl=64 time=32 ms
10.10.1.1 64 byte ping: ttl=64 time=8 ms
10.10.1.1 64 byte ping: ttl=64 time=20 ms
10.10.1.1 64 byte ping: ttl=64 time=11 ms
10.10.1.1 64 byte ping: ttl=64 time=9 ms
10.10.1.1 64 byte ping: ttl=64 time=29 ms
10.10.1.1 64 byte ping: ttl=64 time=28 ms
10.10.1.1 64 byte ping: ttl=64 time=9 ms
10.10.1.1 64 byte ping: ttl=64 time=10 ms

Could you report what sort of pings you experience under good wireless conditions such as this? Some have reported that I should be under 1 ms, which I greatly agree. However, MT support is telling me this is normal\acceptable.

The sector has 2 CPE, both with great signal and have little traffic (I believe under 250 kbit average).

BurstNET · Wed Mar 07, 2007 5:10 pm

nstreme on routerboards?

SMA

Hammy · Wed Mar 07, 2007 5:17 pm

nstreme on routerboards?

SMA

Routerboards as clients, and an AMD Athlon 2600 or so for the AP.

believewireless · Wed Mar 07, 2007 5:18 pm

Turn off NStreme and your pings will be fine.

Hammy · Wed Mar 07, 2007 5:25 pm

N-Streme is really the only reason why I'm using Mikrotik. It would be quite foolish to turn it off.

Why is N-Streme to blame? What can MT do to fix this or can I do to help mitigate it?

Diganet · Wed Mar 07, 2007 6:15 pm

N-Streme is really the only reason why I'm using Mikrotik. It would be quite foolish to turn it off.

Why is N-Streme to blame? What can MT do to fix this or can I do to help mitigate it?

If you put some more traffic on these links it will be better. N-streme has big latency when almost no traffic..

/Henrik

believewireless · Wed Mar 07, 2007 6:20 pm

The problem is that no matter how much download traffic you push, upload traffic from subscriber radios will be a problem. With VoIP we would see that the subscriber would hear everything fine but the other end would have real problems.

So, latency TO the customer may improve with more traffic but latency FROM the customer will not.

Hammy · Wed Mar 07, 2007 6:22 pm

So then in the short term (until more traffic is on the link), would just doing high byte size pings help my latency and jitter issues?

If I were to do this, would I have to ping every client, or just on the sector in general.

Hammy · Wed Mar 07, 2007 6:23 pm

The problem is that no matter how much download traffic you push, upload traffic from subscriber radios will be a problem. With VoIP we would see that the subscriber would hear everything fine but the other end would have real problems.

So, latency TO the customer may improve with more traffic but latency FROM the customer will not.

That sucks...

believewireless · Wed Mar 07, 2007 6:36 pm

There has been a lot of misinformation posted here about NStreme and it's problems. First people blamed it on WDS, then it was the framer policy and then it was the processor speed. We went through all of this and found that the jitter problems had nothing to do with anything but NStreme. We upgraded 532s to PC-based systems, changed framer policies, etc and none of it worked.

The only thing that did fix the problems was turning NStreme off.

Hammy · Wed Mar 07, 2007 6:49 pm

I don't want any names (except for maybe offlist), but do other non-802.11 systems have these problems?

babyface · Wed Mar 07, 2007 7:03 pm

One thing is sure, you don't need NStreme for 2 customers connected to an AP...

Hammy · Wed Mar 07, 2007 7:07 pm

Well no, but I am about to do a marketing campaign and I already have a backlog of new customers. I don't want to do a lot more with my system until I know that its going to be there for me later.

believewireless · Wed Mar 07, 2007 7:39 pm

We've never seen these ping problems on any other system including polling systems.

I just want Mikrotik to fix the problem instead of saying it's related to other issues. My guess is that they have never loaded up an AP and tested this.

Thu Mar 08, 2007 3:04 pm

you can try to turn off compression with nstreme, otherwise - we haven't heard complaints like these.

update: hammy, I just checked your supout.rif file and you have set your tx-power to max, even hurting your card. this is causing your problems, set it to default and see if your card is not damaged.

Hammy · Thu Mar 08, 2007 3:38 pm

you can try to turn off compression with nstreme, otherwise - we haven't heard complaints like these.

update: hammy, I just checked your supout.rif file and you have set your tx-power to max, even hurting your card. this is causing your problems, set it to default and see if your card is not damaged.

Sorry. I knew to leave it at default and had been preaching the "leave it at default" policy myself. Perhaps someone else had changed it or I had set it during the original SRx power issues back in the day and missed changing it back. The one that had the power turned up is indeed further away and a more troublesome link than the other CPE that is only 400' away. However, they both are exhibiting latency\jitter issues.

So turning compression off may solve the latency\jitter issues? Is this a change I can make on the AP and it propagates to the CPE, or do I have to change every CPE?

Hammy · Thu Mar 08, 2007 4:18 pm

1000 packets transmitted, 1000 packets received, 0% packet loss
round-trip min/avg/max = 1/17.1/179 ms

 0  R name="wlan1" mtu=1500 mac-address=00:15:6D:10:0A:FD arp=enabled disable-running-check=no interface-type=Atheros AR5213 
      radio-name="00156D100AFD" mode=station ssid="" area="" frequency-mode=manual-txpower country=no_country_set antenna-gain=0 
      frequency=5180 band=5ghz scan-list=default rate-set=default 
      supported-rates-a/g=6Mbps,9Mbps,12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps basic-rates-a/g=6Mbps max-station-count=2007 
      ack-timeout=dynamic tx-power-mode=default noise-floor-threshold=default periodic-calibration=default 
      periodic-calibration-interval=60 burst-time=disabled dfs-mode=none antenna-mode=ant-a wds-mode=disabled 
      wds-default-bridge=none wds-default-cost=100 wds-cost-range=50-150 wds-ignore-ssid=no update-stats-interval=disabled 
      default-authentication=yes default-forwarding=yes default-ap-tx-limit=0 default-client-tx-limit=0 
      proprietary-extensions=pre-2.9.25 hide-ssid=no security-profile=default disconnect-timeout=3s on-fail-retry-time=100ms 
      preamble-mode=both compression=no allow-sharedkey=no

[admin@SECO Building] interface wireless> monitor wlan1 once 
               status: connected-to-ess
                 band: 5ghz
            frequency: 5785MHz
              tx-rate: 36Mbps
              rx-rate: 24Mbps
                 ssid: "ICS1"
                bssid: 00:15:6D:10:0A:F9
           radio-name: "00156D100AF9"
      signal-strength: -76dBm
   tx-signal-strength: -72dBm
          noise-floor: -99dBm
      signal-to-noise: 23dB
               tx-ccq: 60%
               rx-ccq: 54%
         p-throughput: 19380
             wds-link: no
              nstreme: yes
              polling: yes
         framing-mode: best-fit
        framing-limit: 3999
     routeros-version: "2.9.27"
              last-ip: 65.182.165.45
  802.1x-port-enabled: yes
  authentication-type: none
           encryption: none
          compression: no
    current-tx-powers: 6Mbps:19,9Mbps:19,12Mbps:19,18Mbps:19,24Mbps:19,36Mbps:15,48Mbps:14,54Mbps:13
  notify-external-fdb: no

1000 packets transmitted, 1000 packets received, 0% packet loss
round-trip min/avg/max = 1/7.3/112 ms

 0  R name="wlan1" mtu=1500 mac-address=00:0B:6B:37:AD:F2 arp=enabled disable-running-check=no interface-type=Atheros AR5213 
      radio-name="000B6B37ADF2" mode=station ssid="ICS3" area="" frequency-mode=manual-txpower country=no_country_set antenna-gain=0 
      frequency=5745 band=5ghz scan-list=default rate-set=default supported-rates-b=1Mbps,2Mbps,5.5Mbps,11Mbps 
      supported-rates-a/g=6Mbps,9Mbps,12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps basic-rates-b=1Mbps basic-rates-a/g=6Mbps 
      max-station-count=2007 ack-timeout=dynamic tx-power-mode=default noise-floor-threshold=default periodic-calibration=enabled 
      periodic-calibration-interval=60 burst-time=disabled dfs-mode=none antenna-mode=ant-a wds-mode=disabled wds-default-bridge=none 
      wds-default-cost=100 wds-cost-range=50-150 wds-ignore-ssid=no update-stats-interval=disabled default-authentication=yes 
      default-forwarding=yes default-ap-tx-limit=0 default-client-tx-limit=0 proprietary-extensions=post-2.9.25 hide-ssid=no 
      security-profile=default disconnect-timeout=3s on-fail-retry-time=100ms preamble-mode=both compression=no allow-sharedkey=no 

[admin@HammettHome] interface wireless> monitor wlan1 once 
               status: connected-to-ess
                 band: 5ghz
            frequency: 5745MHz
              tx-rate: 48Mbps
              rx-rate: 48Mbps
                 ssid: "ICS3"
                bssid: 00:15:6D:50:16:C6
           radio-name: "00156D5016C6"
      signal-strength: -57dBm
   tx-signal-strength: -63dBm
          noise-floor: -101dBm
      signal-to-noise: 44dB
               tx-ccq: 78%
               rx-ccq: 100%
         p-throughput: 28471
             wds-link: no
              nstreme: yes
              polling: yes
         framing-mode: best-fit
        framing-limit: 3999
     routeros-version: "2.9.27"
              last-ip: 10.10.3.2
  802.1x-port-enabled: yes
  authentication-type: none
           encryption: none
          compression: no
    current-tx-powers: 6Mbps:17,9Mbps:17,12Mbps:17,18Mbps:17,24Mbps:17,36Mbps:14,48Mbps:12,54Mbps:11
  notify-external-fdb: no

Thu Mar 08, 2007 4:29 pm

routeros-version: "2.9.27"

upgrade ..

Hammy · Thu Mar 08, 2007 4:32 pm

I guess that's the downside of a system that doesn't go down. There's never a good time to upgrade. ;-)

Hammy · Thu Mar 08, 2007 4:39 pm

Crap, that's why... my license expired before the new license system was put in to place.

I vaguely remember the steps to upgrade this on an RB, but how would I do this on an x86?

Thu Mar 08, 2007 4:44 pm

netinstall only

believewireless · Mon Mar 12, 2007 11:06 pm

First, Mikrotik can't say they've never heard of it. We've raise the issue, sent supouts, etc. If you don't believe me, just do a search for "jitter" on this forum.

Use WDS on a 532 WITH compression and 30+ customers but with no NStreme, and ping times are fine. Turn NStreme on, everything goes to hell.

We've dropped in faster PC boards, changed framer policy, manually set ack times, gone through every hoop and it's always come back to NStreme.

In every other polling system we've used, polling gives very consistent ping times. Plus, every polling system we've seen easily allows up to 100 connections per AP without issues.

Something in NStreme needs to be fixed.

BurstNET · Tue Mar 13, 2007 4:32 am

We have to agree.
Our experience shows 1-2ms latency on majority of our links, but as soon as Nstreme is turned on, latency jumps many milliseconds and has occassional really high jump every few seconds.
It is not an interference issue, as everything is perfect consistenly with Nstreme off, and we have tried MANY freqs and channels, and in many different locations.

SMA

ubb · Tue Mar 13, 2007 6:44 am

We've seen nstreme affect our ping time very positively. We have one link in particular that was running about 5-10Mbps and roughly 1000-1700 packets per second. The issue that we were seeing was that as soon as the pps jumped above about 1400, the latency over the link went up drastically (100ms). We enabled nstreme and that problem went away. From what I understand, nstreme packs the packets together more efficiently that normal which is why when you have a lot of packets per second, it helps out tremendously. We consistently have ping times around 2-4 now.

Each situation may be different, but from what I've seen time and time again, it's the packets per second that kill you, so horay for nstreme!

janisk · Tue Mar 13, 2007 1:17 pm

We've seen nstreme affect our ping time very positively. We have one link in particular that was running about 5-10Mbps and roughly 1000-1700 packets per second. The issue that we were seeing was that as soon as the pps jumped above about 1400, the latency over the link went up drastically (100ms). We enabled nstreme and that problem went away. From what I understand, nstreme packs the packets together more efficiently that normal which is why when you have a lot of packets per second, it helps out tremendously. We consistently have ping times around 2-4 now.

Each situation may be different, but from what I've seen time and time again, it's the packets per second that kill you, so horay for nstreme!

in many threads i tried to explain that problem is low traffic, now, i could not say better

so low traffic + nstream = ping nightmare
high traffic + nstream = salvation

believewireless · Tue Mar 13, 2007 2:53 pm

ubb, I'm guessing you are talking about a point-to-point link. As I've mentioned before, the problems we are seeing are in PtMP links.

In a PtMP link, the AP has the ability to send a lot of traffic to make the latency TO the client low. However, you can't have every client send a lot of traffic to keep the return latency low.

The problem is in PtMP networks.

janisk · Tue Mar 13, 2007 3:08 pm

works as designed

if you have low traffic than you have no need for nstream. and if you have high traffic at some moments and you want to benefit at these moments then you set up nstream and explain to your users, that this is not that bad and that they have to use their wireless more

to start to benefit from nstream

believewireless · Tue Mar 13, 2007 7:20 pm

Or, I can use any other polling system that works with any volume of traffic.

babyface · Tue Mar 13, 2007 7:33 pm

Or, I can use any other polling system that works with any volume of traffic.

IMO, Protocols with polling like NStreme only have sense with high traffic. The latency in a good link raises when there are high loads, so low loads links don't need Nstreme or routerOS. You can try DD-WRT in WRT54GL

ghmorris · Tue Mar 13, 2007 7:55 pm

Polling systems need to be universally useful, otherwise they aren't worth deploying.

If you have a lot of access points, the last thing in the world you want to do is decide whether you have enough traffic load to run Nstreme or not.

It works or it doesn't.

We haven't had much trouble using Nstreme, but I think the Trango polling implementation is better. On the other hand, they have lots of other parts of their system that are much worse.

George

believewireless · Tue Mar 13, 2007 8:26 pm

Traffic has nothing to do with it either. Pings times were terrible with 5 clients passing no traffic or 30 clients passing 5-10Mbps of traffic. Ping time got worse the more loads the AP became and CPU utilization went up.

I believe people are making statements that are more related to quesses. We've tested every theory proposed here and come down to the same result. And that result is Nstreme.

We use Aperto, Alvarion, Canopy and Trango and have never seen these issues. The one issue Trango has is that if a customers hasn't passed traffic in a while, the first three pings drop but it's fine after that.

I agree with ghmorris that polling systems need to be universally useful. It's a real pain swtiching between NStreme and no NStreme. Just visited a client this morning that turned off their equipment during the change over and we had to go onsite to turn NStreme off. It turns out 6 customers turned off their radios during the transition and now it's a truck roll to every one.

We like Mikrotik and support it but it appears we aren't getting the support in return.

rpingar · Wed Mar 14, 2007 12:04 am

we have experienced the jitter about ping on ptmp with nstream only if the customers per base station > then 5-10. The traffic seems not to be component of the problem.

I also think like belivewireless that Mikrotik could work on a polling system for ptmp esclusively to workaroiund the actual weakness.

We have a base station with 40 customaers and the ping is about 40-120ms..................not so nice for voice..........

Regards
Ros

Qusay · Fri Mar 16, 2007 12:59 am

Hi Hammy,
I don't if this is related to your problem or not..and may any one correct me if I'm wrong..
If you turn rate-set=default, the M.T OS will try to calculate the best rate and assigned for you..but if your link is not that simple and many factors are changing rapidly, OS will start to jump from one rate to another and keep bounsing...which affected the ping and keeps it bounsing too...
To solve this, try to change the rate-set=default to manual and try to select bandwidth from lower one to higher one.
Good luck ..

jwcn · Fri Mar 16, 2007 4:30 am

The issue is with Nstream not fluctuating speed selections...

rpingar · Fri Mar 16, 2007 8:54 am

Hi Hammy,
I don't if this is related to your problem or not..and may any one correct me if I'm wrong..
If you turn rate-set=default, the M.T OS will try to calculate the best rate and assigned for you..but if your link is not that simple and many factors are changing rapidly, OS will start to jump from one rate to another and keep bounsing...which affected the ping and keeps it bounsing too...
To solve this, try to change the rate-set=default to manual and try to select bandwidth from lower one to higher one.
Good luck ..

In Ptmp we use always a rate-set based on the experience of the site. But with also a lowest rate the jitter problem is not fixed. And a lot of time using a lower rate the problem is more evident, in a lot of site 24mbit seems to be the best rate about jitter.

Regards

tjohnson · Sat Mar 17, 2007 4:09 am

Hi,

We have a MT AP currently serving 6 clients. Here is a sample ping I just performed to a client 9 miles away with a -62 signal and currently less than 1Mbps of traffic on the AP:

69.20.169.246 64 byte ping: ttl=63 time=7 ms
69.20.169.246 64 byte ping: ttl=63 time=6 ms
69.20.169.246 64 byte ping: ttl=63 time=6 ms
69.20.169.246 64 byte ping: ttl=63 time=5 ms
69.20.169.246 64 byte ping: ttl=63 time=4 ms
69.20.169.246 64 byte ping: ttl=63 time=4 ms
69.20.169.246 64 byte ping: ttl=63 time=4 ms
69.20.169.246 64 byte ping: ttl=63 time=8 ms
69.20.169.246 64 byte ping: ttl=63 time=3 ms
22 packets transmitted, 22 packets received, 0% packet loss
round-trip min/avg/max = 3/5.2/9 ms

We are running Nstream, but with no framing policy and no compression. We also leave the card power at "default" and the speed settings at "default". Seems to be working fine to me.

Travis

rpingar · Sat Mar 17, 2007 9:30 am

The issue is with Nstream not fluctuating speed selections...

I think with Nstream it fluctuates too much!
:D

N-Streme PtMP Latency\Jitter

N-Streme PtMP Latency\Jitter

no problems here