Community discussions

MikroTik App
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

LHG-LTE18 nearly dead.

Wed Jan 08, 2025 11:42 pm

Hello again, and Happy New Year to everyone.
Hope you are all doing great !


I have a very weird problem with my LHG-LTE18 and I wanted to know if there is anywhere I can start, in order to find the problem.


The setup (briefly):

LHG-LTE18 acting as modem, on passthrough mode. 1 VLAN for net traffic, 1 for management.
RB5009 acting as router. (same VLANS as above set up there. I followed Mikrotik's guide on YouTube)
CSS326-25G as my Switch, connected to the 5009 via SFP.
My ISP is Comsote Greece.


The problem:

Every so often (and it's getting TOO often) my internet connection just gets super slow in terms of Ping.
General Speeds are down too, but it does work, and it does get up to relatively high speeds.
But latency, is just SECONDS.

When trying to access the LHG, everything "lags" and I can't really access it properly. Commands take seconds to complete, and sometimes I have no access at all. It's the same story both from Winbox AND from Telnet. Or using a browser to access the webgui, or just pinging the device from my router. It takes ages. It also doesn't matter if I use the MAC to access it.

Restarting the Modem, INSTANTLY fixes the issue like nothing happened.


More info:

- This happened first time like 6 months ago. Then 5 months ago. Then again in 20 days.
Long story short, it's no happening probably every hour. I didn't bother looking into it in the beginning as I thought it was something random.

- The logs in the device, mention nothing, although I did see that you can setup one billion topics on System>Logging.
I don't know which one I SHOULD activate though to catch a potential problem.

- During the problem, I don't notice any out of the ordinary CPU Usage or something.




Any advice on how to solve the issue would be helpful.
I am one step away from losing my mind over this. :D


PS. I'm gonna post a video down bellow, although I described the issue, and you don't need to watch it.
It just shows what I said. The device is lagging hard, and rebooting it, instantly goes back to normal.
EDIT: Here's a video: https://streamable.com/1o22au
(I forgot I had a mic, so I was typing on notepad like a retard) :D
Last edited by bonamin on Sat Mar 29, 2025 11:52 pm, edited 1 time in total.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 13600
Joined: Thu Mar 03, 2016 10:23 pm

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 8:58 am

It's hard to trouble shoot behaviour which happens only rarely. When it happens next time, don't forget to thoroughly check the logs, there might be something in it. Another thing to do is to create supout.rif file and send it to Mikrotik support ... they might decode the device state and comment on what might have happened.

I'd suggest you to make sure your device is running latest stable ROS version (if your device is currently running v6, then that would be 6.49.17; if your device is running v7, then that would be 7.16.2). Make sure routerboot (System->Routerboard) is upgraded as well. There are some firmware upgrades available for LTE modems as well, consult LTE modem upgrade guide to see how that is performed.

If software/firmware upgrades don't make a difference, then you can implement a work-around using netwatch ... netwatch, running on LHG itself, might be able to detect sluggishness and reboot device ... in this case the service impact will likely be much shorter than with manual reboots.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 4:32 pm

Hello and thanks for taking interest into my problem !

Well, for starters I wouldn't say the problem happens "rarely".
It happens every day, and sometimes like 5 times in a single night.
It won't be long until it happens again.

The normal logs, have nothing that might indicate any error or something.
Every single message I read is normal, like user logins, or IPs from the DHCP Client etc.
That's why I asked if I could maybe enable any specific (extra) logging. Like we do when debugging devices.

I am running ROS 7.16.1.
It doesn't show me any .2 update.
The modem is also updated to the latest firmware according the device.
(EG18EAPAR01A13M4G_01.002.01.002)

A workaround won't really cut it for me to be honest.
I can also instruct my Home Assistant to cut power to the modem and restart it if network issues arise, or create a script that does the same thing via telnet. I am not looking to find a workaround.
I'm looking for the cause.

And If I'm being honest, it's driving me insane since I paid nearly 4 digits for my network setup to avoid these kind of problems.
I was dealing with stuff like this with my ISPs (free) router.

Anyway, I will try the supout.rif file you said, and take it from there.
I don't know what else to do.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4654
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 7:13 pm

I'd make sure you also upgraded the firmware in /system/routerboot to match.

If that matches, then... it may be worth it do a backup, and upgrade to the "testing" channel with 7.17rc. I recall others some issues with LHG specifically in 7.16 or 715, so doing another search through forum may be also of use. Generally speaking, some version is stable, whether it's called stable or not ;) - but you do want all 3 versions of aligned (RouterOS, RouterBOOT, and LTE firmware).

If you do create a support case, I'd recommend increasing the loging via /system/log/add topics=lte,!packet which will capture additional LTEs logs. Also it might be helpful a couple supout.rifs, once after boot up & once when the problem occurs. But the supout.rif will include the low-level LTE logs if you enabling that in logging, which no doubt will help MT support figure it out.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 11:01 pm

Hey there !

I enabled the two extra logging topics you mentioned and I will take a couple rifs as you said.

As for the System/routerboot I can't find anything like that.
According to this: https://help.mikrotik.com/docs/spaces/R ... RouterBOOT
it looks like RouterBOOT, is updated alongside "RouterBoard" correct ?

I upgraded everything to 7.16.2 and I'll hope for the best.
And if this fails, I will try the 7.17rc6 and see what happens.


I can't figure out any other path. :D
Thanks for the input though. I appreciate it. <3
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4654
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 11:07 pm

Sorry, /system/routerboard is where you upgrade the firmware (but the "firmware"/BIOS is called RouterBOOT).
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11468
Joined: Mon Dec 04, 2017 9:19 pm

Re: My LHG - LTE18 is having a Stroke. :D

Thu Jan 09, 2025 11:11 pm

According to this: https://help.mikrotik.com/docs/spaces/R ... RouterBOOT
it looks like RouterBOOT, is updated alongside "RouterBoard" correct ?
"alongside" as in "is downloaded as part of RouterOS", yes.
"alongside" as in "is actually installed together with RouterOS", no.
So to be able to flash a new version of routerboot, you first have to install a new version of RouterOS (which requires a reboot); once you flash the routerboot using system routerboard upgrade, you have to reboot again in order that the new routerboot would be actually used.
 
flintham12
just joined
Posts: 13
Joined: Mon Nov 04, 2024 6:20 pm

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 2:00 am

The issue is likely caused by either a saturated connection, interference, or a potential misconfiguration in the Mikrotik LHG. Restarting the modem resolves it temporarily, which suggests the problem may be with the modem or its interaction with the LHG.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 7:11 pm

Sorry, /system/routerboard is where you upgrade the firmware (but the "firmware"/BIOS is called RouterBOOT).
yis sir ! I figured ! :)
"alongside" as in "is downloaded as part of RouterOS", yes.
"alongside" as in "is actually installed together with RouterOS", no.
So to be able to flash a new version of routerboot, you first have to install a new version of RouterOS (which requires a reboot); once you flash the routerboot using system routerboard upgrade, you have to reboot again in order that the new routerboot would be actually used.
Yes I understood that after your inputs on the matter. I didn't know that was the way, up till yesterday.
I thought everything was updated all together and I just need to update the modem manual on devices like the LHG.
Thanks for clarifying that.
The issue is likely caused by either a saturated connection, interference, or a potential misconfiguration in the Mikrotik LHG. Restarting the modem resolves it temporarily, which suggests the problem may be with the modem or its interaction with the LHG.
After updating it to the latest version yesterday, it has been running since. FINGERS CROSSED !
If it doesn't get any fever tonight or tomorrow I think that might have fixed it.
I haven't changed any configuration since I first configured the device. It's running on Passthrough mode as I said, so the only config, was the two VLANs, for which I followed step-by-step Mikrotik's guide on YouTube.


That being said, I wasn't checking for updates via System > Packages or via QuickSet Menu > Check For Updates.
I was only trying to update from System > RouterBoard which as you guys indicated only tried to update RouterBOOT, using the "bundled firmware" inside v7.16.2.
That's why I said "I am not seeing any updates other than 16.1". I was looking at the wrong place.


Everything is Clear now in terms of how the device needs to be Updated.
Thank you.

FINGERS CROSSED. I will post results.
If it dies on me again, or not, I will inform you.

Thanks AGAIN ! Love you all. <3
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11468
Joined: Mon Dec 04, 2017 9:19 pm

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 7:16 pm

I'm not sure you followed the first response of @mkx - what does /interface/lte/firmware-upgrade lte1 upgrade=no show?
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 7:28 pm

I'm not sure you followed the first response of @mkx - what does /interface/lte/firmware-upgrade lte1 upgrade=no show?
You mean about the Modem's Firmware update.
This Link: https://help.mikrotik.com/docs/spaces/R ... areupgrade

Yeah I did follow that !

It Shows:
installed: EG18EAPAR01A13M4G_01.002.01.002
latest: EG18EAPAR01A13M4G_01.002.01.002

But that was already updated since the last time I updated the modem a few months ago.
It had no new update now.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4654
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 7:40 pm

Yeah point being there are three possible places to update:

1. /system/package
2. /system/routerboard/update
3. /interface/lte/firmware-upgrade

& with LTE likely best ALL align (and/or at latest stable), since over many releases I have see weird things with LTE (no showing up, not running, disappear, etc. etc) when versions are different.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Fri Jan 10, 2025 8:59 pm

& with LTE likely best ALL align (and/or at latest stable), since over many releases I have see weird things with LTE (no showing up, not running, disappear, etc. etc) when versions are different.
Yes, it only make sense for the firmawares of connected devices to be of the same "revision" let's say.
From a developer's standpoint at least.

I understand.
It still runs, no hiccups. I'll post again in a later time for results !

Thanks again guys.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Sat Jan 25, 2025 12:27 am

So... Greetings again !
I have updates !


I did update the OS to 7.17 and it did look like the issue went away, but then it happened again, just much later rather than every single hour or so.
Then after checking daily, there was finally a new update for the Modem's firmware.
I updated that too, and now, it has been running for a week or so, with no problems.
mt.PNG
I REALLY hope this fixes things, cause it was starting to get annoying.
That being said, I changed ZERO settings whatsoever. As far as configuration.
I just updated the device, and the modem. Nothing else.

Thank you all for your time and patience.
Have a nice day, folks !! :)
You do not have the required permissions to view the files attached to this post.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: My LHG - LTE18 is having a Stroke. :D

Sat Mar 29, 2025 11:52 pm

So, MORE updates.

Now I have an even bigger, I'd say HUGE issue.
During the last 4-5 days, my internet connection during the night has been dead slow to say the least. During the day I didn't notice too many problems.
It looks like my ISP has some problems too, since our phones are experiencing reduced speeds as well.

But the phones experience a drop from 150Mbps to 100Mbps, while the LHG experiences speeds of 5-20Mbps, when in a normal day it was 300 to 400Mbps. Ping is insanely bad too. 200ms AT BEST.

I completely RESET the device, and connected it directly to my computer. NO RB5009, no VLANs, no nothing, in between.
I removed from the pole I had it setup, cleaned it, checked for possible physical problems, nothing. Everything looks fine.

What else can I do ?
I'm one step away from crying like a little baby.
I'm in the middle of a HUGE business project, and I am out of internet. This is driving me insane.

More INFO:

- If I put my SIM card, on my ISP's Router, it works as normally, with less ping than the LHG ever achieved, and with speeds of 70-100Mbps. But that device never got any higher, since it isn't LTE18 like the LHG.

- I found a couple of LOGs on the device, that said something like "To Run SMS scripts you need to setup authentication" or something.
Is my ISP trying to run any scripts ? I don't understand what that means.

I am SERIOUSLY confused. Any help would be AWESOME.
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11468
Joined: Mon Dec 04, 2017 9:19 pm

Re: My LHG - LTE18 is having a Stroke. :D

Sun Mar 30, 2025 1:33 pm

Each SIM card is linked to a subscriber account, and each subscriber account has a "service plan" or whatever is the correct term in English, which determines a lot of parameters of your connection; in addition to limits of both bandwidth and total amount of data transported per some unit of time, it also determines the relative priority of your traffic that is used when the cell or area becomes short of capacity. These limits may not only depend on IMSI (the ID of the SIM card, i.e. the subscriber account) but also on the IMEI (the ID of the equipment that uses the SIM). It is quite obvious that data transfers must give way to voice transfers, but there are even multiple levels of priority for data services.

Having said this, I don't think the issue is related to a change of behavior of the LHG-LTE18 itself, rather to some changes in the ISP network. Reorganisation of the sectors in your area, partial outage, software upgrade that makes the unchanged software in your modem behave differently... anything.

So talk to your ISP and ask them to explain whether it is a temporary issue or whether you have to do something at your end. If it is bad during the night but almost OK during the day, chances are high that the root cause is some planned maintenance of the mobile network.

As for the "SMS script" - Mikrotik can execute commands received via SMS provided that they conform to some specific format and contain a password that must match the one configured in the router, see details in the manual. It is a RouterOS specific behavior, i.e. unrelated to the service SMSes the mobile operators can send to (re)configure MT, so the log messages actually only mean that "some" SMSes have arrived. It is quite likely that these SMSes contain some information regarding the trouble you currently encounter. Or maybe they are the reconfiguration ones and try to adjust the APN settings to match the changes in the network, but that's just a speculation.
 
bonamin
just joined
Topic Author
Posts: 18
Joined: Mon Feb 20, 2023 7:28 pm

Re: !UPDATE! - LHG-LTE18 nearly dead.

Sun Mar 30, 2025 10:14 pm

I understand. :)

I understand that I'm doomed.
Judging from my ISPs responses over the phone, I'll never find someone to help me. :D

My ISP (Cosmote Greece) used to offer 2 APNs, one of which had throttled speeds, but a public IP. And one with unlimited speed, but CG-NAT IP.
With RouterOS' power, I could utilize both APNs, pass them though BOTH via separate VLANs to the RB5009, and then forward each device on the appropriate APN/VLAN. My ISP removed that ability :D

They also offered Extra SIMS, which shared just the Data of your main SIM. Gone as well.
Or, not gone, but made as expensive as a standalone connection.

I am DOOMED. hahahaha

I was just curious if this erratic behavior could be the LHG. Since sometimes a reboot fixes issues.
And my ISP's router works as intended with said SIM card.

(My Plan is 100% Unlimited according to my ISP. No limit on Data, Voice or SMS, and no throttle of the speed at any usage)