Community discussions

MikroTik App
 
360Degree
just joined
Topic Author
Posts: 11
Joined: Mon Sep 23, 2024 8:54 pm

1 Packet over Multiple Routs?

Tue Oct 22, 2024 9:54 am

Hi
I cant remember how the correct Therm is for that. Some Company have an mode where 1 Packet is send truth (some/) all WAN ISP and the Packed who arrived first will be taken and the other duplicated Packed get tossed away. That repeat for every single other one to. Image this there is a Contract who must be signed by the other Company to and you send 3 Employees with 3 different Airlines to the other Company and the first who arrive at there HQ get the Paper signed and the other 2 get tossed away. So you can be sure that:
a) every Packed arrives on the other End even on the way there a packed is lost
b) the fastest packed get picked so when one (Mobile) Connection is overloaded there is no down time.

Who do I need them? In my Country we have 3 Cellphone Operator with (100%) Unlimited Internet. I want stream some events remotely and everyone an imagine at some Open Air Concert for example it can get very tricky to get reliably (!) Data from A to B.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22312
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: 1 Packet over Multiple Routs?

Tue Oct 22, 2024 4:43 pm

This is a mikrotik forum bud.......... not applications for fancy routing.
 
User avatar
infabo
Forum Guru
Forum Guru
Posts: 1508
Joined: Thu Nov 12, 2020 12:07 pm

Re: 1 Packet over Multiple Routs?

Tue Oct 22, 2024 8:56 pm

What if all airplanes crash and not reach HQ...🤯
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1692
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: 1 Packet over Multiple Routs?

Tue Oct 22, 2024 9:18 pm

And imagine if top management and all the development gurus were on the crashed airplanes! 🤯🤯🤯
 
kleshki
Member Candidate
Member Candidate
Posts: 285
Joined: Tue Mar 10, 2020 6:37 am

Re: 1 Packet over Multiple Routs?

Wed Oct 23, 2024 1:10 am

Even if you could manage to implement duplicating packets through multiple interfaces, I doubt other side will accept it normally, since it would expect a packet flow from a single endpoint. Otherwise, how they should preserve packet sequence, especially with LTE delays.
 
360Degree
just joined
Topic Author
Posts: 11
Joined: Mon Sep 23, 2024 8:54 pm

Re: 1 Packet over Multiple Routs?

Mon Nov 04, 2024 11:16 am

The other Side would be the same (Virtual, Software,..) Router as the one who send the Packeds.
 
User avatar
sirbryan
Member
Member
Posts: 421
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 5:31 pm

What you're talking about is called packet duplication and is an SD-WAN vendor technique. Cisco, Fortinet, Velocloud, and so on all use it.

From an article by Cisco (their product is Cisco Catalyst SD-WAN):
https://learningnetwork.cisco.com/s/art ... r-starlink

Packet duplication is an SD-WAN technique designed to enhance reliability by mitigating packet loss. When a critical packet is scheduled for transmission, the router creates an identical copy. Both the original and duplicate packets are then sent over separate SD-WAN tunnels to the destination. The receiving device detects and discards duplicate packets, ensuring only one copy is processed. This method relies on multiple available tunnel paths and is typically configured for applications sensitive to packet loss, such as voice or video. While improving reliability, packet duplication introduces additional network overhead in terms of bandwidth and processing resources. It is configurable through SD-WAN management platforms to specify target traffic and redundancy levels.
Here's a 4-year-old Reddit post where a few people share their experiences:

https://www.reddit.com/r/networking/com ... in_sdwans/
 
User avatar
sirbryan
Member
Member
Posts: 421
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 6:52 pm

Apparently ZeroTier does Multipath. The question remains as to whether or not MikroTik's implementation supports it yet. But on Linux, you have a few options:

Standard policies
  • active-backup: Use only one primary link at a time and failover to another designated link.
  • broadcast: Duplicate traffic across all available links at all times.
  • balance-rr: Stripe packets across multiple links (not for use with TCP.)
  • balance-xor: Hash flows to specific links.
  • balance-aware: Auto-balance flows across links.

(from https://docs.zerotier.com/multipath/)
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22312
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 7:18 pm

Interesting!! Be cool for the sirbryan to conduct single router to single router tests of this tech, comparing zerotier to wireguard performance.........
 
optio
Forum Veteran
Forum Veteran
Posts: 988
Joined: Mon Dec 26, 2022 2:57 pm

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 7:33 pm

This is a mikrotik forum bud.......... not applications for fancy routing.
maybe OP wants this -> viewtopic.php?p=1105740 (and guess who is OP of this :))
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11326
Joined: Mon Dec 04, 2017 9:19 pm

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 8:28 pm

OK, so let's take it more seriously.

The broadcast mode of bonding indeed works well as for multiplication of the packets, but it does nothing at all regarding "tossing the late ones". The reason is that while broadcasting, the sending end does not add any information to the packets that would allow the receiving end to distinguish duplicates created by a broadcasting bond from duplicates created "naturally" - as an example, think of ARP requests that contain no information that would distinguish them from one another.

So rather than broadcasting the payload itself, you would need to broadcast transport packets of some tunneling protocol encapsulating the payload ones, which would be equipped with some sequence numbers allowing the receiving end to drop the late copies. I hazily remember someone here on the forum already did some research - I would assume IPsec-AH should take care of this but I may be wrong.

But that's not all yet - as bonding can only bond together L2 interfaces, you need to create L2 tunnels to carry the bonded packets; you can choose between EoIP, VxLAN, and L2TP in bridging mode. As neither EoIP nor VxLAN can traverse NAT on their own, they need to be further encapsulated into L2TP, bare IPsec, or Wireguard, hence using L2TP in bridge mode directly seems the best way to me.

So the summary is that you end up with at least two layers of tunneling if you want to go this way, but what is worse, even with three operators you cannot be sure your traffic will get delivered with a consumer grade mobile contract.

Therefore, if your streaming is meant to be a serious business, I'd say you should talk to the mobile operators and ask them for SIMs with prioritized handling. The traffic to/from such a SIM will get priority even in a crowded cell, but of course such a SIM is provided under a more expensive contract.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 3179
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: 1 Packet over Multiple Routs?

Wed Nov 06, 2024 9:51 pm

i think this is a very interesting topic, Once again, FEC support for tunnels to remote sites resurfaces in the conversation
 
User avatar
sirbryan
Member
Member
Posts: 421
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: 1 Packet over Multiple Routs?

Thu Nov 07, 2024 4:40 am

Interesting!! Be cool for the sirbryan to conduct single router to single router tests of this tech, comparing zerotier to wireguard performance.........
According to the various Reddit posts etc. that I came across when researching my original reply, Wireguard is faster than ZeroTier. And in my own experiments I've found Wireguard converges pretty quickly if you have traffic flowing and the OS is quick to reroute traffic (i.e. during interface down events--plug, unplug, plug, unplug Starlink...).

In my experiments, the primary connection is my own network, and a Wireguard tunnel over a different provider is the second link. I have OSPF + BGP established over both, and can use BFD to detect pretty quickly if the primary link is down. This works and works rather well with both Starlink and cable; I have it in production at two locations.

The difference is it only uses one connection at a time and doesn't offer any kind of FEC or packet duplication. I can set the OSPF costs to be equal, and let ECMP handle load-sharing (which also works). But then some flows are randomly limited to the worst bandwidth in one direction (cable and Starlink upload is 20-30Mbps), and the latency changes depending on whether the traffic stays on-net or goes via the Wireguard link. So in the end, the backup connection sits idle most of the time.

I think Amm0's response in the linked thread does a good job explaining L3 (OSPF+ECMP+BFD) load-sharing vs. L2 (EOIP tunnels) with some kind of LACP or broadcast bonding. I will have to test the latter, however. That would be interesting.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 22312
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: 1 Packet over Multiple Routs?

Thu Nov 07, 2024 1:22 pm

Okay either I have to travel to UTAH or we get together on (take your pic skype/discord/teams) to discuss those very configs...........
Use case: seamless failover between two WANS on the ground site, using CHR as the public IP hitting the internet.
EOIP within wireguard.

Trying to do better than check-gateway=ping OR netwatch.
 
User avatar
sirbryan
Member
Member
Posts: 421
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: 1 Packet over Multiple Routs?

Fri Nov 08, 2024 4:53 am

Check my link in my profile. There's a way to contact me...