MikroTik

I finally got my hands on one of these CRS328-4S-20S-4S+ for a 26 Condo Building install (12 units per building). It booted fine and I changed it from RouterOS to SwitchOS and then configured each port, set up isolation, configured a LAG, and RSTP on a couple of ports for a main and backup link.

Things went smooth so I left it plugged in for a few hours while I went and did something else. I've never had an issue with infantile failure with Tik, but I figured it was better safe than sorry.

When I came back to the switch I popped in all the SFPs for testing, verified each power supply was working (unplugged one at a time), and then I started testing. The first thing I noticed was that none of the SFPs were linking. I thought that was weird so I connected a computer and tried getting back into SwoS. I couldn't reach it via GUI, ping, or get the MAC address to populate with Winbox. I messed around a while longer and noticed that the Ethernet (Copper) ports would only negotiate at 10Mbps - yes I tried multiple ports and cables and the computer negotiates to Gig with everything else.

Finally I decided to reset and start from scratch. After the reset I can get the SFPs to link, but I still have no communication to the switch through the copper ports or from another switch through an SFP. Any ideas on how to recover this? I don't have a console cable and I can't ping/TFTP.

Any help is appreciated.

When it was in RouterOS did you update it? if in RouterOS try upgrading the firmware too.

I bought exactly the same model just last week and it's been running fine.

I do believe there is an issue with SFPs. There's a thread here about it:

viewtopic.php?f=17&t=126274

I don't think the CRS328 can be downgraded though. SWOS 2.7 was the factory installed version.

The only issue I'm seeing with mine is lots of RX Overruns on some ports.

I believe I found the cause of my problems. Check out this video:

https://youtu.be/Blmb1C0665c

If you power cycle the switch manually and don't leave it unplugged for more than 2 minutes it won't properly boot. If you unplug the switch and leave it sitting for 2+ minutes it will boot fine. If I reboot from the GUI or Winbox it also boots just fine.

Is this something that can be fixed in software or is it a hardware issue? This will be a pretty important switch in our network so I can be patient if its software, but if its hardware I need to move on.

SwOS 2.8 was released this morning and has several fixes for SFP. Time to upgrade and see?

Yes, thank you. I haven't tested the SFP issues yet, but I did confirm this issue on 2.7 and 2.8 already.

Yikes... looks like these don't support any BiDi SFPs either? Tried 3 different versions (manufacturer and wave length) with no luck - all end up as half duplex.

Who alpha/beta tests these things? I've always had good luck with Tik routers so I'm disappointed, but I guess I need to find something else for this switch application.

As a heads up. After playing around with all sorts of SFPs this appears to only be an issue in SwOS. RouterOS has no issues with 10G or 1G SFPs.

I post the similar issue already here ,

viewtopic.php?f=17&t=137071

welcome to CRS328 experience #Lol

I'm seeing similar behavior with two CRS328-4C-20S-4S+ boxes, although it takes them a lot longer than 2 minutes unpowered before they will come back up correctly.

This is all RouterOS, I have not tried SwOS.

On 'first' boot, they work fine. All ports link up and work as they should.

On subsequent boots (unless I wait quite a while, or do 'the things mentioned below'), the SFP and Combo ports don't work. On the SFPs (and SFP side of the Combos), I can get them to show a link by disabling auto-neg, but no data actually moves. The copper side of the Combos do nothing whatsoever. The SFP+ ports however do actually work, but only when forced to 1G.

Poking about, I found there was an appreciable voltage on the main board's power-supply connectors, slowly diminishing over time, suggesting a capacitor retaining charge. It takes about an hour for that voltage to drop to the point it no longer registers on my volt meter's 200mv range, after which it will boot up fine again. I haven't exhaustively tested, but it seems to come up weird even with only 50mv showing on those headers before power-up.

If I short the headers, the voltage disappears (expected, if it is a capacitor slowly discharging) instantly; and the box will boot into the 'good' state immediately afterward.

To me, this seems like it could be an issue with switch-chip initialization. Perhaps the tiny voltage is enough to keep some bits of memory in the chip in a non-baseline state, while the initialization procedure assumes and requires the chip to be in it's baseline state?

Whatever it is, it's throwing a monkey wrench into an Active-E FTTH deployment.

Fun, fun,
--Eric

Such things shouldn't happen in properly designed hardware - proper low voltage reset circuits are cheap and easily available. But as a workaround, add a resistor to discharge the capacitors faster - if the voltage on them during normal operation is 3.3 volts, use something like 100 ohms. Or a small old-fashioned (not LED) light bulb

It would be interesting to know if the issue may also affect any other CRS3xx devices, or is it unique to this specific one model?

I post the similar issue already here ,

viewtopic.php?f=17&t=137071

welcome to CRS328 experience #Lol

"CRS328 experience" Its so not funny its funny. I've got things running in RouterOS as you mentioned in that other post, but now this hardware has me trigger shy to put it in the field. Anyone running this in production and having good luck?

Such things shouldn't happen in properly designed hardware - proper low voltage reset circuits are cheap and easily available. But as a workaround, add a resistor to discharge the capacitors faster - if the voltage on them during normal operation is 3.3 volts, use something like 100 ohms. Or a small old-fashioned (not LED) light bulb

It would be interesting to know if the issue may also affect any other CRS3xx devices, or is it unique to this specific one model?

Yeah... We are not going to make hardware modifications on gear destined for deployment, even if simple and easy. In the lab, sure; in production, no way. At the hardware level, it needs to work out of the box, so that any technician can deploy them at a moment's notice, without having to be aware of internals. We have done custom stuff in the past, and it always comes back to bite.

If MT can fix it in microcode, or whatever, great. If it's a swap for a newer production run, fine. If it's intrinsic to the model, no go, we will find something else.

--Eric

On subsequent boots (unless I wait quite a while, or do 'the things mentioned below'), the SFP and Combo ports don't work. On the SFPs (and SFP side of the Combos), I can get them to show a link by disabling auto-neg, but no data actually moves.

I have exact same issue on CRS328-4C-20S-4S+ FW 6.43.7