Hello,

I found, that RB1200 add 10 -12 ms latency to packet procesing during IPsec enc/decrypting. For example RTT of ICMP packet without ipsec is 2-3 ms, with IPsec is 13-14 ms.
If I tested same configuration on RB2011L, additional IPsec latency is 0-1 ms.
Tested against RB1100AH. ROS 5.21 and 5.22. IPsec with SHA hash and AES-256 cipher.

On RB1200 I used port 1. Ports 9 and 10 have own additionaly latency about 1-20 ms (see RB1200 performance issues on ether9 and 10?).

Do you have same experience with RB1200 or another RBs?

Added later:
Problem is also with ROS 6rc7 regardless on cipher (tested md5/null hash and cipher DES too). Tested with no CPU load and with minimal other traffic (< 0,5 Mbit). When IPsec using null "cipher", latency is normal < 1 ms.

I tested IPsec with same settings on RB2011, RB750G and old RB600A (ROS 5.21, 5.22 and 6rc7). All of them have normal <1ms IPsec latency.

I tested RB1200 with ROS 6rc11 and both problems (port 9,10 latency problem and IPsec generally problem) persist.

I apologise for my bad english.

Deleted because not related.

Dobby!

Read again ... 5nik wonders why there is latency on rb1200 and not on rb2011 ... I have also tested my setup full with some rb2011(uas), rb450g, rb751g and some linux boxes and can also confirm that there is no noticable latency (defintely not 10ms) using this boxes ... so, the question from 5nik is why there is latency with rb1200 which has cleary more cpu power than rb2011.

Buying ccr or 1100ahx2 is clearly not necesary for ipsec unless you need to do it with multiple tunnels over (multiple) 100mbit/s links. Yes, those boxes do support hardware encryition, but in smaller enviroments and with slower links, rb2011, 450g and other can do the job just well, especialy if you use aes128 instead of default 3des.

JF

Hello Dobby,

I know about HW accelerated AES support in 1100AH, our core routers (VPN concetrators) are 1100AH and x2.
On some small department, I often need encrypt max 5Mbit/s, and 1100AH is a little more for this purpose.

My topic is not about performance (throughput) but about latency. As JanezFord wrote, why RB with powerfull CPU has higher IPsec latency than low cost RB?
Is such difference between CPU architecture (PPC vs MIPS-BE)? Or this is "bug" in ROS?
If I will have more time, a will test more low cost RB (such RB751G a so on).

In my eyes, RB1200 becomes very unpopular due latency problem (port 9 and 10 generally, IPsec generally).

I apologies about my english.

The problems you describe only happen on upper ports (eth 9 and 10)
Dont use those 2 ports and it will all be fine

There are multiple posts around forums about more problems on the 1200 with these ports (latency increase for all traffic, random packet drops etc.), its mostly due to these ports being connected through a PCI-X interface, which apparently still causes some problems in ROS.

The problems you describe only happen on upper ports (eth 9 and 10)
Dont use those 2 ports and it will all be fine

Read again... he used port 1

JF

The 10-15ms latency increase seems to be a byproduct of the RB1200s limited IPsec acceleration, some further insights here http://forum.mikrotik.com/viewtopic.php?t=53026 & here http://forum.mikrotik.com/viewtopic.php?f=2&t=56504

I tested IPsec on RB1200 with other ciphers than AES, and IPsec latency were same (10-12 ms) even when I used less-CPU-consum ciphers like DES.
With null cipher latency fell to normal 0-1 ms.

Thank you ChrisP for links. It is sad, that last post on linked topic is 7 months old and problem is still continuing.

One note: I have never tested RB 1200 with RoS v6, maybe new version solves the latency problem.

Deleted because not related.

I don´t want encrypt traffic higher then 5 Mbit. I'm testing IPsec latency without other trafic through IPsec (only ping). CPU load during test is <5% (no load).

I tested old RB600A, older and slower CPU, same architecture (no AES acceleration). And IPsec latency was 0 ms (RoS 5.22 and 6rc5).

There are four versions of the 19" - 1U rackmount case based routers and you were choosing the low cost variant
for ~$150 less then the RB1100AHx2 with VPN/IPSec hardware acceleration support and now all should running like
the bigger RB1100AHx2? And MikroTik should implement something in software on top to speed this up?

Yes, I expected, that ping (~null traffic) through IPsec on no-loaded CPU will have same latency on all boxes, regardless of HW accelerating AES.
I expect difference between boxes in IPsec throughput.

Who wants to pay more for a RB1100AHx2 if the RB1200 can do the same job?

And who wants pay more for RB1200 if the RB2011L can do the same job (in case IPsec better)?

Today I tested RB1200 (port 1, no load) with ROS 6rc7 and IPsec latency is same
Tested SHA1+AES and null+DES ciphers.

There's several threads where people have discovered the latency increase, but I've not seen an satisfactory explanation yet. This thread has links to a few more related discussions: http://forum.mikrotik.com/viewtopic.php?t=56779

Thank you ChrisP for link.

I found that RB1200 still has two problems: packet latency on ports 9 and 10 and IPsec latency generally on all ports. And as ChrisP wrote, without any satisfactory explanation yet.

Any updates on this issue? Is it fixed? I have a customer with need for rb1200 cpu power and fanless design to run ipsec with other office branches. RB2011 is not enought, anything else is too noisy.

JF

At last, I tested ROS 6.2 and nothing change. I plan to reclaim all RB1200. But due EoL of RB1200, I don't know which box replace RB1200 (see my topic).

For what it is worth to anyone, I recently decommissioned a RB1200 and did some lab testing on this issue. on ROS 6.7 and firmware 3.10

the IPSec latency issue does not present when using the following enc algorithms in the proposal:

Blowfish
Twofish
Camellia - 128
Camellia - 192
Camellia - 256

All DES and AES variants experience the latency issue

So all may not be lost if you have an RB1200 with camellia support... Hope this helps someone.

edit: ether6 was used in the tests but it presents on all ether ports on the unit.

Thak you rjickity for this info... this may indeed help some rb1200 users to get more of their routers. I wonder why does this happen .... I am guessing that Camellia encryption is not hardware accelerated by rb1200 cpu and both aes and 3des are and implementation of this acceleration is the key problem. Did you also happen to perform some cpu usage tests with camellia?

JF

Unfortunately I only have an RB2011 available for the otherside at the moment.

95-105Mbit TCP both direction forwarding is achieved before i max out the rb2011uias to 100% CPU. The RB1200 maintains 40% usage at this point. You could probably safely assume at least 200Mbps i guess, just bear in mind no firewall filter or nat is in place with these tests. Latency was ok through the test . Below is the config i used:

RB1200
RB2011