100% correct and 100% faster ..... KISSI would prefer Tailscale (wireguard SDWAN) over ZeroTier
I like Mikrotik.
I dont like SDWAN.
How do you know it would be cheap? I don't see how they could comply with this license if they included it in ROS:Yes, please add support for Zerotier. I love Mikrotiks, but they are seriously lacking in some kind of SDWAN solution. Zerotier would be a very cheap and easy way to set this up.
The api exists today. All of my wireguard config on mikrotik is done via the rest api introduced in v7. There shouldn't be any reason zerotier couldn't leverage it.I like also very much the idea to see Zerotier in Mikrotik. But I ask: what is the showstopper? Why Mikrotik does not reply to us? I have asked in Zerotier forum and I have read that Zerotier developers have a great interest in putting Zerotier in Mikrotik and they want to do themselves but they need API from Mikrotik. It seems to me that the best thing is that Mikrotik calls Zerotier or viceversa and they do an agreement.
Otherwise we are only dreaming.
ZeroTier’s software kit is licensed under the ZeroTier BSL, which allows source code access and free use for all with the exception of hosting a network controller for commercial purposes or embedding the ZeroTier source code in a commercial application. You can self-host ZeroTier controllers and nodes for free if you use it for non-commercial purposes. Please contact us to learn more.
via https://www.zerotier.com/pricing/
Zerotier builds a full mesh and uses the lowest latency path between any two nodes. If there is any loss (indicating congestion) it shifts that traffic to a backup path automatically. You can build a full mesh with wireguard and OSPF but OSPF does not adjust routing cost based on latency between nodes and will not automatically redirect traffic the moment it detects congestion.Also, maybe I'm not up to the speed but what problem ZT solves which WG+OSPF doesn't?
This statement as-is is false. Zerotier operates as a star topology and then attempts to build p2p adjacencies and prioritises those direct paths when possible, otherwise it falls back to the relayed ie star topology. It doesn't account for latency at all, simply treats the shortest path between zt nodes as optimal. It does connection monitoring via keep alives just like OSPF does. If you enable multipath on the client you do get some connection quality measurements to determine which uplink to use but that's only if you have multiple WAN links for zerotier to use and those tests are only between that zt node and the remote node to see which of the two interfaces look better. Otherwise, it does nothing except check if the direct path is up. Zerotier doesn't route around slow or lossy nodes until they fail and then it falls back to the star topology ie 'relayed'.Zerotier builds a full mesh and uses the lowest latency path between any two nodes. If there is any loss (indicating congestion) it shifts that traffic to a backup path automatically. You can build a full mesh with wireguard and OSPF but OSPF does not adjust routing cost based on latency between nodes and will not automatically redirect traffic the moment it detects congestion.Also, maybe I'm not up to the speed but what problem ZT solves which WG+OSPF doesn't?
DMVPN is more a suite of tools requiring adding a number of components to routeros, plus DMVPN really struggles with NAT. zerotier is a single daemon and all of the smarts are external and it finds ways to get through NAT much better. Much simpler to implement.Would it not make more sense to implement DMVPN?
This whole setup is really nothing like zerotier. you have tunnels to concentrators so it's a multi-hub & spoke model. All data must flow through a hub.Hi!
I working in ISP sector and we operating with low-mid budget so we can't buy high-end SD-WAN solutions yet for 10X-20X the price. So we need to find the optimal solution at all times for the following:
- using various underlaying network, PPPoE, DOCSIS, metro ethernet, DF etc. with optional IPSec encryption where it is possible
- the underlay circuits has random global addresses and some of it has (CG)NAT/FW in the path which may blocks GRE and/or ESP protos
At this moment we testing it with MTik MPLS over L2TP/PPPoE. In the test setup there are 1-4 geographically redundant LNS and clients in HUB & Spoke topology. As MTik PPP stack can handle MPLS we don't need any other magic just pure MPLS. Moreover we can use MLPPP so we can handle jumbo frames in L2VPN if we would so.
- providing VPNV4, VPNV6, L2VPN over underlaying network
In the near past I tested it with an RB4011 and I get 700Mbps with an L2VPN over MPLS over L2TP (MLPPP). I measure it with single UDP stream generated with iperf3. One CPU core was 100% on the RB4011. Maybe CCR5009 would can 1Gbps. We are before a multilink test where L2TP client has multiple link to more LNSes and lets see how ECMP and Multi-core PPP do its job.
oreggin
I don't understand this. How can an UDP based, VXLAN-like (modified?) tunnel goes through better over NAT than also UDP based L2TPv2 or UDP based IPSec? Maybe you overmistified the capabilities of ZT?Also punches through NAT in ways that L2TP tunnels just don't
AFAIK you cannot bond ("Multipath") with Mikrotik's current support. I personally been waiting for ZT multipath, very curious how well it bond multiple LTE interfaces.Has anyone been able to get multi-wan working with ZT on ROS? I have 3x WAN links with appropriate routing tables and policy routes for each interface. My other ZT boxes see several paths, but it never seems to aggregate and pulling the active WAN link down causes about five seconds of packet loss. With ZT on generic Linux you can define custom policies for WAN bonding in local.conf and it works decently well. There doesn't seem to be any way to edit local.conf for ZT on ROS.
[user@device] > /zerotier/peer/print detail
Flags: B - bonded
0 instance=zt1 zt-address="62f865ae71" bonded=no latency=188ms role="PLANET"
path=active,preferred,2001:xxxx:xxxx:2::2/9993,recvd:6s916ms,active,
50.7.252.138/9993,recvd:16s734ms,sent:12s157ms
AFAIK you cannot bond ("Multipath") with Mikrotik's current support. I personally been waiting for ZT multipath, very curious how well it bond multiple LTE interfaces.
Hard to know, maybe Mikrotik can answer...AFAIK you cannot bond ("Multipath") with Mikrotik's current support. I personally been waiting for ZT multipath, very curious how well it bond multiple LTE interfaces.
Multipath has been a part of zt since v1.6 (2020-11-24) but was actually announced already in v1.4 (2019-07-29) although that version was more of a beta. The latest update was in v1.8.5 (2022-02-22).
I wasn't able to find anything mentioned in the release notes why they capped multipath/bonding. Any idea of the reason behind the restrictions?
The Mikrotik ZT "instance" allows you pick the interface (or I guess a few interfaces) it could use. But there is no RouterOS version of ZeroTier's "defaultBondingPolicy" that be set to the bonding mode.