Hi,
Using a Quecetel LTE module via USB, got it configured and left it alone, a few days later realized it ate up many mb of data traffic.
My question, simple FW rule to only allow LTE traffic IN/OUT to router for management (winbox port, ssh, etc.) so no other random traffic should be going out the LTE interface from the network.
And 2) a rule that I can enable on-demand to route all traffic in/out the LTE interface when needed.
So, for example, if the site internet goes down, I can access the router via the LTE interface, confirm what is happening and then re-route all site internet over that LTE connection.
Alternatively also setup a persistent wg tunnel over the LTE interface to a remote server and let it sit idle for management traffic only (inbound to router ssh port only).
I think this should be really simple but can't seem to get it to do what I want.
Any help or ideas?
EDIT: To Clarify;
All sites have Fiber or cable internet -> Main WAN
They all have a Quecetel LTE modem plugged into the USB port, LTE is all up and working, no issues.
The LTE interface should only be for inbound management and I want to create a persistent outbound WG tunnel to a management server that sits idle (no traffic passing) and is used to connect back to the router only for management.
Thanks.
LTE interface traffic FW
Last edited by RyanNet on Sun Aug 20, 2023 5:01 am, edited 1 time in total.
Re: LTE interface traffic FW
Hi.
I think it's not a matter of rules in firewall, rather bad default route distance in the first place.
I have a similar situation in a new site (which is a construction site for now) - fibre connection as main WAN and a LTE connection backup (which I would like to use for management purposes as you described) and after connecting fibre ISP few weeks ago and doing basic WAN configuration I have realized that traffic still goes throught LTE because of dynamic route (distance 0 vs my static route distance 1). I have disabled LTE and from there it behaves as desired. I don't have "backup" WAN though but also don't have time for it now so it awaits for better days
Try to do a simple test - tracert from a computer in your network to see where packets are going. If throught LTE WAN - look at your routes.
I think it's not a matter of rules in firewall, rather bad default route distance in the first place.
I have a similar situation in a new site (which is a construction site for now) - fibre connection as main WAN and a LTE connection backup (which I would like to use for management purposes as you described) and after connecting fibre ISP few weeks ago and doing basic WAN configuration I have realized that traffic still goes throught LTE because of dynamic route (distance 0 vs my static route distance 1). I have disabled LTE and from there it behaves as desired. I don't have "backup" WAN though but also don't have time for it now so it awaits for better days
Try to do a simple test - tracert from a computer in your network to see where packets are going. If throught LTE WAN - look at your routes.
Re: LTE interface traffic FW
Set default distance to something > then fiber route.I have disabled LTE and from there it behaves as desired.
Then it will only kick in when the main line goes down.
Classic fallback.
Re: LTE interface traffic FW
It's not that simple in my scenario, because I passthrough static LTE public IP to the gateway (which is a VPN server) and that route is being created dynamicly with distance 0. Maybe it can be changed somewhere? In the DHCP Client configuration for example? I am familiar with fallback stuff and possible problems with different solutions (in this classic way the problem is "if the direct connection to the ISP is up, but there is a problem later on in his infrastructure then this solution isn't a perfect solution, because route won't become inactive"). I didn't have time to look at it more deeply and this DHCP Client scenario is a new thing for me so... I just did what needed to be done
Set default distance to something > then fiber route.
Then it will only kick in when the main line goes down.
Classic fallback.
Re: LTE interface traffic FW
Hmmm... never really used passthrough but just tried on AX Lite LTE.
It is indeed just that bit different making this classic scenario will not work.
In that case you may have to revert to a more advanced scenario.
https://help.mikrotik.com/docs/pages/vi ... d=26476608
Reading material and experimenting for quiet evenings
It is indeed just that bit different making this classic scenario will not work.
In that case you may have to revert to a more advanced scenario.
https://help.mikrotik.com/docs/pages/vi ... d=26476608
Reading material and experimenting for quiet evenings
Re: LTE interface traffic FW
There is a default-route-distance in the DHCP client on the Advanced tab. OR...If it's static, you can skip the DHCP client... and just set the /ip/address on the passthrough'ed interface and add 0.0.0.0/0 with higher distance in /ip/routes. And this be fine if it JUST a backup for a LAN.
BUT if it's a VPN server on the LTE public, you'd also need mangle rules for input/output so that incoming traffic via the LTE public returns out the same path. Perhaps masquerade covers that case depending on VPN. But might want to look at:
https://help.mikrotik.com/docs/pages/vi ... d=26476608
and/or
https://help.mikrotik.com/docs/display/ ... cy+Routing
BUT if it's a VPN server on the LTE public, you'd also need mangle rules for input/output so that incoming traffic via the LTE public returns out the same path. Perhaps masquerade covers that case depending on VPN. But might want to look at:
https://help.mikrotik.com/docs/pages/vi ... d=26476608
and/or
https://help.mikrotik.com/docs/display/ ... cy+Routing
Re: LTE interface traffic FW
LOL. I was like, well, where to start here... I guess the docs.Reading material and experimenting for quiet evenings
Re: LTE interface traffic FW
Just reading while taking breaks from watching "Chappie" for the first time, not experimenting
It's not the time tonight for reading in details about recursive routing and to be honest... I rather won't use it on construction site where it's not really necessary, I don't have it even on my main gateway (in "main" site).
I was loosely thinking about doing something less complex - marking routing traffic to some IP address (like network DNS server or maybe network NTP server) throught main WAN, Netwatch it every few seconds and enable/disable LTE interface when it's not available/is available.
But the most important thing is... I'm going on holidays in a week from now, I will walk throught Polish mountains so... internet failover in a construction site doesn't really bother me right now
It's not the time tonight for reading in details about recursive routing and to be honest... I rather won't use it on construction site where it's not really necessary, I don't have it even on my main gateway (in "main" site).
I was loosely thinking about doing something less complex - marking routing traffic to some IP address (like network DNS server or maybe network NTP server) throught main WAN, Netwatch it every few seconds and enable/disable LTE interface when it's not available/is available.
But the most important thing is... I'm going on holidays in a week from now, I will walk throught Polish mountains so... internet failover in a construction site doesn't really bother me right now
Re: LTE interface traffic FW
This is a great thing to know, thanks!There is a default-route-distance in the DHCP client on the Advanced tab.
Re: LTE interface traffic FW
Also @pcunite is still developing a forum article to summarize this rather complex topic, but this might a little shorter initial read.
viewtopic.php?t=192736
viewtopic.php?t=192736
Re: LTE interface traffic FW
Recursive routing might possibly be another solution with a few additional rules for the VPN service: “MultiWAN with RouterOS”
Re: LTE interface traffic FW
Before we get to failover and LTE routing, I'd like to simply get LTE to only access the router. No access to network or devices behind it, yet, just able to access winbox/ssh via LTE anytime.
And have no packets go from network to LTE.
At the same time, is it possible to make a wg interface only use LTE interface, to create a remote tunnel for router management.
And have no packets go from network to LTE.
At the same time, is it possible to make a wg interface only use LTE interface, to create a remote tunnel for router management.
Re: LTE interface traffic FW
Before we get to failover and LTE routing, I'd like to simply get LTE to only access the router.
Hmmm ... wouldn't a simple firewall filter rule do the trick?
Code: Select all
/ip/firewall/filter
add chain=forward out-interface=<lte interface> action=drop place-before=1 comment="Drop all forward traffic via LTE"
Re: LTE interface traffic FW
If traffic to internet is routing throught LTE and you will add a firewall rule which drops it:I think it's not a matter of rules in firewall, rather bad default route distance in the first place.
[...]
Try to do a simple test - tracert from a computer in your network to see where packets are going. If throught LTE WAN - look at your routes.
you'd effectively loose internet access for devices behind this router
Re: LTE interface traffic FW
I will update the original post also, let me clarify;
All sites have Fiber or cable internet -> Main WAN
They all have a Quecetel LTE modem plugged into the USB port, LTE is all up and working, no issues.
The LTE interface is only for inbound management and I want to create a persistent outbound WG tunnel to a management server that sits idle (no traffic passing) and is used to connect back to the router only for management.
I assumed not having any network traffic leave the LTE interface (it is metered and costs $ per mb) would be simply blocking FORWARD chain traffic out LTE. But that didn't work correctly. I was also trying a lot of different things so maybe I messed up the rules, but before I try everything again I wanted to get community input on the correct way to do this.
After no net traffic can exit LTE, and there is a wg tunnel router->server via LTE, then I'd like to look at an easy way to enable all outbound net traffic exiting LTE if needed during a down situation. But this doesn't need to be automatic, every attempt at auto fail over has always caused problems and haven't figured out a fool proof method yet. The sites are monitored and if it goes down someone will manually login to the router (via LTE/wg tunnel) and confirm down situation and if needed can failover traffic to LTE.
All sites have Fiber or cable internet -> Main WAN
They all have a Quecetel LTE modem plugged into the USB port, LTE is all up and working, no issues.
The LTE interface is only for inbound management and I want to create a persistent outbound WG tunnel to a management server that sits idle (no traffic passing) and is used to connect back to the router only for management.
I assumed not having any network traffic leave the LTE interface (it is metered and costs $ per mb) would be simply blocking FORWARD chain traffic out LTE. But that didn't work correctly. I was also trying a lot of different things so maybe I messed up the rules, but before I try everything again I wanted to get community input on the correct way to do this.
After no net traffic can exit LTE, and there is a wg tunnel router->server via LTE, then I'd like to look at an easy way to enable all outbound net traffic exiting LTE if needed during a down situation. But this doesn't need to be automatic, every attempt at auto fail over has always caused problems and haven't figured out a fool proof method yet. The sites are monitored and if it goes down someone will manually login to the router (via LTE/wg tunnel) and confirm down situation and if needed can failover traffic to LTE.
If traffic to internet is routing throught LTE and you will add a firewall rule which drops it:
[/quote]you'd effectively loose internet access for devices behind this router
Re: LTE interface traffic FW
That's what I thought and I think that's the exact rule I had but it wasn't working correctly at the time, there was mb of traffic adding up on the LTE interface $$. I don't have the configs from then as I was trying a lot at the time. But trying to get best way before retrying.
How would I make a wg tunnel that only goes in/out the LTE interface, only direct to router (winbox/ssh) and wouldn't carry anything from the network.
** I found the culprit of the mystery traffic that was making it through that rule, the ZT interface. ZT traffic doesn't seem to go through the FORWARD chain.
How would I make a wg tunnel that only goes in/out the LTE interface, only direct to router (winbox/ssh) and wouldn't carry anything from the network.
** I found the culprit of the mystery traffic that was making it through that rule, the ZT interface. ZT traffic doesn't seem to go through the FORWARD chain.
Before we get to failover and LTE routing, I'd like to simply get LTE to only access the router.
Hmmm ... wouldn't a simple firewall filter rule do the trick?
However, one would still have to adjust routing rules ... when routing engine decides it wants to route traffic via some particular interface, there's no way back (e.g. if FW later decides to drop that traffic). Which means that you'd effectively loose internet access for devices behind this router (if currently that traffic uses LTE as way out).Code: Select all/ip/firewall/filter add chain=forward out-interface=<lte interface> action=drop place-before=1 comment="Drop all forward traffic via LTE"
Re: LTE interface traffic FW
** I found the culprit of the mystery traffic that was making it through that rule, the ZT interface. ZT traffic doesn't seem to go through the FORWARD chain.
If ZT client is running on router itself, then ZT traffic (including tunelled traffic) will use input and output chains. And that's true for WG tunnels as well (and any other traffic handled by router itself). Note that most tunnelling solutions implement some kind of "keep alive" mechanizm to keep firewall timeout counters from expiring ... and that's also for 3rd party firewalls which might be on the tunnel path. Those keepalives will add up some traffic ... and it's iffy if you can disable those, alternative might be hanging tunnels after some firewall drops the connection from their connection tracking table.
Re: LTE interface traffic FW
Ok, since you can't make a FW rule using inbound interface matching (chain=output in-interface=ZT1 action=drop) how do you block ZT interface from going out LTE?
And wg is by design very quiet where zt is very noisy. Wg should send almost no packets if no traffic is passing, even if we set keepalive to 2x day would amount to kb per month.
And how to direct a wg interface to only use LTE in/out?
Thanks learning as we go.
And wg is by design very quiet where zt is very noisy. Wg should send almost no packets if no traffic is passing, even if we set keepalive to 2x day would amount to kb per month.
And how to direct a wg interface to only use LTE in/out?
Thanks learning as we go.
** I found the culprit of the mystery traffic that was making it through that rule, the ZT interface. ZT traffic doesn't seem to go through the FORWARD chain.
If ZT client is running on router itself, then ZT traffic (including tunelled traffic) will use input and output chains. And that's true for WG tunnels as well (and any other traffic handled by router itself). Note that most tunnelling solutions implement some kind of "keep alive" mechanizm to keep firewall timeout counters from expiring ... and that's also for 3rd party firewalls which might be on the tunnel path. Those keepalives will add up some traffic ... and it's iffy if you can disable those, alternative might be hanging tunnels after some firewall drops the connection from their connection tracking table.
Re: LTE interface traffic FW
On the ZeroTier instance you set the interface to use an interface-list that does NOT use the LTE in it. ZeroTier's instance ("zt1") defaults to "all", so it will try to find paths out LTE.Ok, since you can't make a FW rule using inbound interface matching (chain=output in-interface=ZT1 action=drop) how do you block ZT interface from going out LTE?
So you may just an interface-list with your primary WAN only, and use that on the "zt1" ZeroTier instance.
Re: LTE interface traffic FW
Is there anyway to do it with FW rules? How can you block OUTPUT chain traffic by incoming interface?
I will setup the zt instances to use specific interfaces or lists for now. How do you create a wg interface that only has traffic in/out of LTE?
Thanks!
I will setup the zt instances to use specific interfaces or lists for now. How do you create a wg interface that only has traffic in/out of LTE?
Thanks!
On the ZeroTier instance you set the interface to use an interface-list that does NOT use the LTE in it. ZeroTier's instance ("zt1") defaults to "all", so it will try to find paths out LTE.Ok, since you can't make a FW rule using inbound interface matching (chain=output in-interface=ZT1 action=drop) how do you block ZT interface from going out LTE?
So you may just an interface-list with your primary WAN only, and use that on the "zt1" ZeroTier instance.
Re: LTE interface traffic FW
The ZeroTier tunnels aren't actually definitively identifiable in firewall. You can block 9993 port (or whatever zt1 interface uses), disable any UPnP, and not use IPv6 & you might have a shot, but ZT is pretty aggressive at trying to find paths. But setting the interfaces on the zt1 interface is how you're suppose to control these.Is there anyway to do it [block ZeroTier's ZL1 tunnels] with FW rules? How can you block OUTPUT chain traffic by incoming interface?
You can use some scripting+netwatch/scheduler, to perhaps change the interfaces zerotier uses if the primary link goes down.
The quick answer is use a separate routing table for LTE and mangle/PBR rules. Check the docs for failover, since it's same considerations if you want to direct traffic out specific WANs. And you may want to do this anyway – directing routes "per-protocol" get tricky without having different routing tables.How do you create a wg interface that only has traffic in/out of LTE?