Recently announced by SuperMicro, the 5018D-FN8T looks to be a beast of a 1U machine to run ROS on. New Xeon-D 1518 CPU (4 core, 8 thread) at 2.2 GHZ, only 35W, and loaded with 30 Gbps worth of interfaces! I think this could give the CCR line a run for its money. If it performs as expected, it ought to give the CCR1036-8G-2S+ some competition for about the same price, with similarly low power draw. Would probably make a good platform for hosting multiple virtual routers too. Thoughts?

if it is priced about $1000 than yes it would.
35W power draw is much lower than the CCR1036 60W power draw and the CCR consumes more power and loses stability as the temperatures increase. Above 80C the CCR becomes unstable and i find a lot of flaws in the cooling, something i only noticed in the summer. The usb port of the CCR also doesnt supply power properly (i powered a raspberry pi from it and it froze but powering from another normal usb port (of another device or even the edgerouter pro) and it hasnt frozen yet and has been months).

I even modded the cooling on the CCR to make it quieter than ubiquiti's edgerouter pro and it involved some tape and some fans.

You should measure the supermicro 1U from the wall and see how much power it really uses. Its not only the CPU that consumes power but a lot of other things.

I think the CCR can be improved without requiring a major redesign, all it needs is a PSU that is 10W-20W higher, a change of fans, heatsink and tunnel cover and it can be quiet and cool at the same time.

I wonder how quiet supermicro's server is. At least you get PCIe which mikrotik doesnt implement in their CCR line, SATA and other things.

I wouldnt say supermicro's server is a beast because it will only handle up to 20Gb/s with that CPU but it will be much faster than the CCR in VPN and other things. I remember a test that was done before during the 1st gen iseries in which dual quad xeons were used that achieved up to 20Gb/s of routing whereas using GPU routing(using dual GTX 480) theoratically achieved 100Gb/s or more. The only problem with using GPUs is that they are quite power hungry.

Yeah, I can't wait for these to become available. I have to imagine that it will crank out 30 Gbps...the Xeon-D 1518 CPU is optimized for network performance (denoted by the xxx8 CPU code), and they gave it 2x SFP+ and 10x 1Gb RJ-45's...I have to imagine they wouldn't have configured it that way if the CPU couldn't push it. It only has a 200W PSU in it, and with the CPU maxing at 35W, I can't imagine the board eating up too much more.

Yeah I'm mega interested in this too for exactly the same reason as you.
I would have liked to have a couple more SFP+ ports on there , but you can't always have everything in life

Been bitten hard by the CCR's before so another option would be welcomed.

If there are free PCIe 2.0 x4 slots you can use SFP+ cards on those. Each PCIe 2.0 x4 slot supports a single SFP+ slot.

If there are free PCIe 2.0 x4 slots you can use SFP+ cards on those. Each PCIe 2.0 x4 slot supports a single SFP+ slot.

"1x PCIe 3.0 x8 slot" - it's an mITX motherboard. But I'm sure there is 2x or 4x SFP+ PCIe cards out there. I have no doubt it has the power to route it's 2x SFP+ ports and 6x 1GbE ports, but adding more SFP+ ports sounds like asking for disappointment.

FYI, I have seen just the motherboard listed online for about $515 US. Throw in some RAM, an M.2 SSD, 1U chassis of your choice...

I don't hold high hopes that this is going to work without a hitch on ROS 6 with the normal x86 distribution Mikrotik seems hellbent on keeping the x86 distribution of ROS as barely usable as possible (no 64 bit support, missing support for lots of newer NICs) to push people towards using CHR, but CHR needs a hypervisor to work, which itself generates a non-trivial amount of overhead in regards to networking

Has anyone tried taking an image of a CHR's storage and deploying it on bare metal? The CHR is essentially an x86_64 OS, so if one could image the virtual hard disk to bare metal, theoretically, we could have ourselves a real 64-bit ROS for x86_64 hardware...

I already mentioned that idea, but it was suggested by Mikrotik staff that essential kernel modules/drivers that are required to successfully run CHR on physical hardware are disabled/removed in the x86_64 kernel so this can't happen. I did not bother to verify that yet.

I have one of the supermicro Atom version of this unit..
http://www.supermicro.com/products/syst ... A-FTN4.cfm

Mikrotik doesn't see the i354 ( the onboard nics )
I have a dual 10gb card in it, and that works, but the onboard nics don't.
Had to load it with netinstall, with a intel NIC, then swap to the 10gb card.

Richard

I have one of the supermicro Atom version of this unit..
http://www.supermicro.com/products/syst ... A-FTN4.cfm

Mikrotik doesn't see the i354 ( the onboard nics )
I have a dual 10gb card in it, and that works, but the onboard nics don't.
Had to load it with netinstall, with a intel NIC, then swap to the 10gb card.

Richard

Good to know. I was looking at those too. Have you thought about loading a hypervisor on it like ESXi or KVM and running ROS as a virtual appliance? The hardware is optimized for virtualization, and that layer of abstraction between the physical NIC and supported virtual NICs may allow you to make use of the onboard 1 Gbps ports. Plus, since you'd be running the CHR as a virtual machine, you'd actually get a true 64-bit ROS as opposed to the 32-bit x86 ROS. I'd be interested to see what kind of performance penalty you incur by virtualizing ROS versus running on bare metal.

No, I have not. We were testing multi-gigabit interfaces, and I was not ready to spend for the CHR license that supports over 1gb, and I might not use.

I ended up running different software on it, and it can really push the traffic.

Richard

I was not ready to spend for the CHR license that supports over 1gb

You can get trial licenses for all license levels for free (good for 60 days). I'd just request a trial license at the "Unlimited" license level and see how much you can push using CHR. If you can get an idea as to routing performance dependent upon different virtual hardware configs, you might find you can run two virtual CHR instances that each meet your needs. As long as the physical hardware doesn't fail, you could do VRRP between the virtual instances for redundancy in case one of the virtual machines has a problem.

I had no more time to test, and the unit is already put it into production.

Richard

You could build your own x86 with as many xeons and as much ram and PCIe as you like and get CCR1036 performance for much less.

A single PCIe 2.0 link supports 2 1Gb/s link (2.5Gb/s per direction)
A 4 lane PCIe 2.0 link supports a single SFP+ (10Gb/s per direction)

The only issue with installing routerOS on x86 is the drivers, a lot of things may not work so you will be limited to mikrotik NICs. I hope theres a list of NICs supported by routerOS.

Supermicro solutions tend to be pricey but also tend to be reliable.

I want to buy this NIC: HP NC364T https://computernetworkingproduct007.wo ... ver-adptr/ , Anyone using this network card?

Just tried to install CHR on baremetal today, and it boots fine, until a moment when some check fails and it decides that my x86_64 machine is not a x86_64 machine, then shutdowns.

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?

I suggest to instill some VM host and use CHR on newer hardware, when RouterOS v7 comes out there will be batter support for newer hardware.

when RouterOS v7 comes out there will be batter support for newer hardware.

Isn't is easier to spot UFO on the sky ?

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?

You can try to install the x86 ROS on it, but it may not work with all the NICs. Best bet is to run a hypervisor like VMware, KVM, etc., and then run the CHR as a virtual machine. Any modern hypervisor should have pretty good support for the onboard NICs, including the SFP+'s, and the CHR should easily be able to use them once they're abstracted as supported virtual NICs. The CHR will also give you access to more than 2GB RAM (which is a limitation of the x86 ROS), and any latency you would get from virtualizing ROS is negligible (we're talking microseconds here).

I suggest to instill some VM host and use CHR on newer hardware, when RouterOS v7 comes out there will be batter support for newer hardware.

Key word "when".

When will RouterOS v7 come out ?

And usual answer like always: when it will be then it will be.

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?

You can try to install the x86 ROS on it, but it may not work with all the NICs. Best bet is to run a hypervisor like VMware, KVM, etc., and then run the CHR as a virtual machine. Any modern hypervisor should have pretty good support for the onboard NICs, including the SFP+'s, and the CHR should easily be able to use them once they're abstracted as supported virtual NICs. The CHR will also give you access to more than 2GB RAM (which is a limitation of the x86 ROS), and any latency you would get from virtualizing ROS is negligible (we're talking microseconds here).

I see, so CHR can access more than 2 GB RAM, that is awesome..

thank you

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.

You're right, but the CPU is typically the most power-hungry component in any computer, short of a high-end GPU that needs it's own PSU. The performance that you get out of this system considering the small PSU is pretty awesome, and definitely something to keep in mind if you're running your own home lab, where being power-efficient is a significant concern.

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.

agree

keep in mind 35w is the tdp for the cpu only it runs at 2.2ghz and is a x86 quad core.

i think we can expect 30watt idle and 70 watt at full load

in terms of performance can be an interesting device because:
* supports virtualization hw acceleration/optimization features of xeon broadwell
* provides 2 x 10g ethernet
* provides 16x pci express lanes directly connected to cpu to expand with more 10g cards
* dual channel of ddr4 ram to provide bandwidth to networking tasks
* 6mbyte of Level 3 cache to improve performance

Not only that, but the CPU is also optimized specifically for networking functions. The latest numbering scheme with the Xeon CPU's uses the last digit in the product code to identify different CPU optimizations. For instance, the Xeon D-1520 is a general purpose CPU, good for running any OS, virtualization, etc., however, the new xxx7 and xxx8 coded CPUs (such as the 1518 in the product we're currently discussing) are optimized for communications and networking. Some new xxx1 CPUs will be coming out soon that are geared toward storage...so lots of exciting things coming along from Intel!

Does anybody tested 5018D-FN8T this??