Hello crowd.

We are a small ISP and we provide FTTH for business and private households at our location.
We are using hEX S (FW 6.45. with 1GB single fibre SFPs from FS.com (TX 1310 / RX 1490 at customer site) and Alcatel SAS-M on provider site.

Sometimes I cannot reach some routers.
When I look from internal, I see that the SFP is receiving laser Rx Power and seems so be up and running(R) on interface list.
But on Interface List Rx Pakets, I get no pakets (0 bps).

[*]Replacing SFP does not help
Turn off and on SFP Interface on customer side does not help
Turn off and on SFP Interface on provider side does not help

The only way to solve the issue is to reboot the customer router.

Does anyone have similar problems or an idea?

Thanks for your help.
Florian

PS: Of course, it would be possilbe to reboot the customer router by script if WAN ping is not ok, but this won't be my favorite solution.

What equipment do you use on the Client side ?
What is the model of the SFPs ?

Hello,
the SFP you use inside the HEX is a dedicated single fibre SFP. A dumb splitter can be used to combine TX and Rx on both ends to a single fibre.

What type of SFPs are used on the Alcatel switch? SFPs with identical specification? What type of fibers are used?

To get it properly working the fiber type must match the SFP type. 1310nm or 1490nm is typically single mode fiber (9um core / 125um cladding). Also the Rx frequency range on the local site must fit the Tx frequency on the remote site. In your case this should be 1310nm for Rx and 1490nm for Tx on the Alcatel Switch.

BR,
Michael

These SFPs are used, I have 50+ customers online with that. But sometimes I have the problem that I have to reboot the hEX router.

TX 1310 / RX 1490 in customer site with hEX
https://www.fs.com/de/products/11802.html

TX 1490 / RX 1310 in provider site with Alcatel SAS
https://www.fs.com/de/products/11795.html

Am not sure what might be the problem...
Why didn't you just use Mikrotik compatible SFPs ? Am not saying that this is the problem, but we would be more sure...

You check this?

https://wiki.mikrotik.com/wiki/MikroTik ... lity_table

Regards.

This is from the WiKi Page.

Note: MikroTik devices and SFP/SFP+/QSFP+ modules do not have any restrictions for other vendor equipment.

As long as the other vendor modules and devices comply with transceiver multi-source agreement (MSA) they should be compatible with MikroTik.

We are using these SFPs fom FS.com because there are no 1310/1490 from Mikrotik.
And again: These SFPs work with the hEXs, but sometimes (very rarely) I have to reboot, becaue the TIK do not recieve pakets. ;-(

This is from the WiKi Page.

Note: MikroTik devices and SFP/SFP+/QSFP+ modules do not have any restrictions for other vendor equipment.

As long as the other vendor modules and devices comply with transceiver multi-source agreement (MSA) they should be compatible with MikroTik.

We are using these SFPs fom FS.com because there are no 1310/1490 from Mikrotik.
And again: These SFPs work with the hEXs, but sometimes (very rarely) I have to reboot, becaue the TIK do not recieve pakets. ;-(

Some times SFP shows running and Tx Packets will be flowing but Rx = 0
Tried Changing Negotiation in the Ethernet Tab (Disabled and Enabled Auto Negotiation) and worked for me

This Issues is most commonly observed in CCR1009

I'm experiencing this same issue, i have now tried TPLink, Ubiquiti and right now i'm using the XS+DA0001

SFP Between RB760iGS and CRS112-8P-4S

It just randomly stops RX packets.

Common in all cases: a Bonding interface, tried using balance xor and 802.2ad, both cases will randomly stop the RX packets on the Hex S.

Another case between RB760iGS and CRS328-24P-4S+

Same exact issue, the Hex S randomly stops RX packets, only way to solve it: reboot it.

now that i have used 3 different brands on the SFP, i do not think the problem is the SFP module.

I had a same experience but I don't think its an RX issue. I think its actually the TX of the other side not really transmitting anything.

In my case I have CRS354-48P-4S+2Q+ with QSFP+ breakout cable (FS Q-8LCAO05) on one side and on the other side is CRS328-24P-4S+ with SFP+ module (FS SFP-10GSR-85)

I upgraded the CRS328 from 7.4 to 7.5 and after it booted up, the link was established, CRS328 was reporting RX and TX but CRS354 did not see any RX. Since I did not upgrade/restart CRS354, I am confident the error was on upgraded CRS328 side which was incorrectly reporting TX.

Rest of details I already reported in the 7.5 thread: viewtopic.php?t=188851#p954893

This is a really annoying problem.

I have a system with 9x CRS328 all links by fibre and randomly I loose switches. When I make checks, I see that I can see broadcasts from upstream devices, but any packets sent to these never reach them even though the port TX counter says its sending. If I plug in directly to the first missing upstream switch I do not see RX packets - so the downstream one showing TX counters is lying.

As noted by some others the only way to fix is full power cycle, System->Reboot does not fix it.

And I'm using all Mikroitk switches and fibre modules. All modules report normal operation even when the issue is happening.

As noted by some others the only way to fix is full power cycle, System->Reboot does not fix it. ... All modules report normal operation even when the issue is happening.

The SFP should be hot pluggable. Does unplugging/replugging the SFP module change anything? If you monitor the port /interface ethernet monitor <name> do you see it come back after replugging? Does it start to work again?

Because the counters are increasing, it seems likely that the linux kernel thinks it has sent the traffic. There are limits to what reporting can accurately report. If you have ever had a package get lost in transit, that doesn't necessarily mean that the package never left the original point. The point is, it may be getting lost after the counting has taken place.

Replugging would at least show whether power cycling just the module is enough to reset things. This would show if the problem is in the SFP module/driver or if the traffic and that something in the SFP module is getting confused, and that power cycling the router may have been fixed the problem indirectly, by power cycling the SFP module. This test would see if power cycling only the SFP would also fix it, and narrow the search for a solution. It could still be a driver problem, but it does seem odd that a reboot doesn't reset the condition, as I would expect all modules to be reinitialized during the boot sequence.

Could be a rare race condition that gets the SFP module into a weird state. I am aware of other networking equipment that had watchdog timer routines to force a reset on a network adapter that would stop working. Not an elegant solution, but better than rebooting a switch.

i've noticed one thing, if i have SFP plugged without cable connected , and if u disable auto negotiation, u will see R flag. unlike auto negotiation ticked, maybe there is a bug or, that is how it supuesto be , i'm not too sure

Ive tried everything I can think of. Sometimes even power cycling does not fix it. Sometimes swapping the SFP module can fix it until it stops again.

Ive just upgrade some units to 7.5 and the problem remains. Big Sad.

I'm running the hEX S with a SFP as well and have encountered this issue a few times myself as well. It simply occurs suddenly. Same symptoms: TX packet counters increasing/flowing, but zero RX. The PPPoE running over the sfp1 vlan 6 interface encounters a timeout. Only thing that helps is rebooting the hEX S. This is still so on 7.7beta4.

SFP with TX 1310 / RX 1490, settings: auto neg. off, tx/rx flow off, speed 1Gbps, full duplex, no loop protect.

Never encountered this problems with an EdgeRouter which I've replaced with this hEX S. I've re-used the same SFP module.

Really annoying!

Does ROS have the capability of forcing a crash/and creating a coredump to a microSD card?

That seems like the most likely way to be able to save the state in a format that MikroTik could analyze. If it is a race condition, or possibly some data structure being trashed, by buffer or stack overflow.