I have 3 pppoe connection on 3 ethernet, 2 of them have one gateway (Same ISP). I want to load balance the traffic between them with failover so if one ISP is down the other is used. The problem is recursive routing is only available with IP gateway (Bad Mikrotik), so I can only use one of those links with same gateway IP.

Is there any workaround for this situation other than scripting?

Create separate Profile for each connection, set Remote IP to something you want - this will override the address that's given to you by the server.

Create separate Profile for each connection, set Remote IP to something you want - this will override the address that's given to you by the server.

ISP assign public random IPs to each PPPOE connection, does ISP allow to assign my own remote IP?
How changing the remote IP help with same gateway problem?

How changing the remote IP help with same gateway problem?

Those IPs will be your new gateways, and the secret is that you set different IPs

Well, you may also try gateway=5.202.112.166%p-pish1 and gateway=5.202.112.166%p-pish2, but if they change for some reason - you'll need to change your routes also, or use scripting

How changing the remote IP help with same gateway problem?

Those IPs will be your new gateways, and the secret is that you set different IPs

Well, you may also try gateway=5.202.112.166%p-pish1 and gateway=5.202.112.166%p-pish2, but if they change for some reason - you'll need to change your routes also, or use scripting

I got it now, I confused Remote IP with Local IP. I didn't know it's possible to change the remote IP of a PPPOE connection. It works like a charm. It's even better now as I can remove many rules in IP table as I assigned my failover ping IPs (4.2.2.1|4.2.2.2|4.2.2.3) directly to the PPPOE connection.

Thank you, you're a genius!

I assigned my failover ping IPs (4.2.2.1|4.2.2.2|4.2.2.3) directly to the PPPOE connection.

Huh, that is unexpected... And it's unexpectedly useful xD

I'll probably edit my Wiki article about failover to add that...

I assigned my failover ping IPs (4.2.2.1|4.2.2.2|4.2.2.3) directly to the PPPOE connection.

Huh, that is unexpected... And it's unexpectedly useful xD

I'll probably edit my Wiki article about failover to add that...

This helped beyond my question, got rid of of all those stupid recursive routes!
by the way I tested %p-pish1 before, it's like this format can't be used with recursive routing like named interface.

The stupid recursive routes are normally there to allow you to check that via a given WAN link, you can reach not only the ISP's infrastructure but also the real destinations further in the internet. Without them, your failover will not detect a problem between the ISP and the internet. That's not important if you have a single ISP (as in such case there is no solution to the problem), but if you have several ISPs, you can send all traffic via the working one if you detect that the other one is down.

The stupid recursive routes are normally there to allow you to check that via a given WAN link, you can reach not only the ISP's infrastructure but also the real destinations further in the internet. Without them, your failover will not detect a problem between the ISP and the internet. That's not important if you have a single ISP (as in such case there is no solution to the problem), but if you have several ISPs, you can send all traffic via the working one if you detect that the other one is down.

Exactly. And if you say "hey, router, your ISP's gateway behind the tunnel is 8.8.8.8" - checking will ping 8.8.8.8, sending packets via PPP tunnel, and ISP will forward them to the real 8.8.8.8 This works only with p2p interface, but it does work

Well, yes, I did understand that instead of using just any bogus address as the remote-ip of the pppoe client, and using the actually monitored address one level higher in the recursion just like you would have do where the real IP of the gateway needs to be used on multipoint interfaces, you can use the monitored address directly as remote-ip of the pppoe client configuration. But if you do it this way, you cannot monitor several remote addresses per WAN link, can you? So if the single IP you monitor dies for a while, you deem the link dead although it actually isn't.

Definitely yes, but in the configs I saw, people generally monitor only 1 address. And if 8.8.8.8 "dies for a while", looks like the problem is a bit bigger than you can imagine

The stupid recursive routes are normally there to allow you to check that via a given WAN link, you can reach not only the ISP's infrastructure but also the real destinations further in the internet. Without them, your failover will not detect a problem between the ISP and the internet. That's not important if you have a single ISP (as in such case there is no solution to the problem), but if you have several ISPs, you can send all traffic via the working one if you detect that the other one is down.

Yes, and that's why I chose 4.2.2.1 as the remote address of PPPOE which is on the internet, so there is no need for recursive addresses! For gateways with ip address I still use recursive routing.

Well, yes, I did understand that instead of using just any bogus address as the remote-ip of the pppoe client, and using the actually monitored address one level higher in the recursion just like you would have do where the real IP of the gateway needs to be used on multipoint interfaces, you can use the monitored address directly as remote-ip of the pppoe client configuration. But if you do it this way, you cannot monitor several remote addresses per WAN link, can you? So if the single IP you monitor dies for a while, you deem the link dead although it actually isn't.

I really don't know how to check multiple IPs with recursion, but if 8.8.8.8 is not responding it suffice to consider the link dead for me.

[/Codebox]

Create separate Profile for each connection, set Remote IP to something you want - this will override the address that's given to you by the server.

I've also try to set different ppp profile with remote ip for fallover & found that there is still another one dynamic route with distance 0,
so blackhole route cant get active state, even if check gateway route becomes unreach

Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 S ;;; pppoe remote gate:
dst-address=0.0.0.0/0 gateway=77.88.8.7 gateway-status=77.88.8.7 unreachable distance=1 scope=30 target-scope=10 routing-mark=pppoe

1 S ;;; check gw pppoe fall:
dst-address=0.0.0.0/0 gateway=77.88.8.7 gateway-status=77.88.8.7 unreachable check-gateway=ping distance=2 scope=30 target-scope=10


Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 ADC dst-address=77.88.8.7/32 pref-src=1.2.3.4 gateway=pppoe gateway-status=ppppoe reachable distance=0 scope=10

1 SB ;;; blackhole pppoe fall:
dst-address=77.88.8.7/32 type=blackhole distance=254

Is there a way to make blackhole route for work correctly ?

Slow down, mate.

First, for scriptless failover using recursive next-hop search, you have to set the monitored addresses in the internet as remote-address in the /ppp profile, and a different address has to be used for each pppoe-out, otherwise the route via the failed one won't go down unless the interface itself went down. Your printout shows the same gateway address for both default routes via pppoe-out so it cannot work.

Second, what would the blackhole route with dst-address different from 0.0.0.0/0 be there for? The route to connected subnet which shadows it is always there as long as the interface is up, but if you want the blackhole route to be used if access to internet is blocked via both pppoe interfaces, it has to have the same dst-address as the routes it replaces if active, i.e. 0.0.0.0/0.

Slow down, mate.

First, for scriptless failover using recursive next-hop search, you have to set the monitored addresses in the internet as remote-address in the /ppp profile, and a different address has to be used for each pppoe-out, otherwise the route via the failed one won't go down unless the interface itself went down. Your printout shows the same gateway address for both default routes via pppoe-out so it cannot work.

Second, what would the blackhole route with dst-address different from 0.0.0.0/0 be there for? The route to connected subnet which shadows it is always there as long as the interface is up, but if you want the blackhole route to be used if access to internet is blocked via both pppoe interfaces, it has to have the same dst-address as the routes it replaces if active, i.e. 0.0.0.0/0.

1) All shown routes are for just one pppoe interface
first "pppoe remote gate" have a routing mark that place it in table with "pppoe" name (I'm using multiple tables for routing)

second "check gw pppoe fall" is placed in "main" table and used for check gateway

ppp profile that sets to pppoe-client interface looks like this
1 name="ppp_77" remote-address=77.88.8.7 use-mpls=default use-compression=default use-encryption=default only-one=default change-tcp-mss=yes use-upnp=default address-list="" on-up=""
on-down=""

So I'm having correct route for this dst in table

ip route check 77.88.8.7
status: ok
interface: pppoe
nexthop: 77.88.8.7

And thats why I'm using blackhole for 77.88.8.7, it should be active, when check gateway makes current routes unreach , but it dont change state to active

2) Other out interfaces that I have are not pppoe, & there is no problem with fallover on them

I've missed the routing-mark, but I still don't get why you need the individual route to 77.88.8.7/32 to switch to the blackhole one and in what situation.

The two default routes have 77.88.8.7 as gateway so check-gateway can be used to make them inactive if that gateway IP is not reachable; the individual route to 77.88.8.7/32 has the interface (pppoe) as gateway so there is nothing to be pinged by check-gateway, so this route stays up as long as the interface itself is up. So the blackhole route to the same destination can only become active when the interface itself goes down, not the remote-address.

Plus if it did, it would be a self-locking positive feedback - from the moment the actual route would become inactive and so the blackhole route would take over, the pings to 77.88.8.7 would never succeed as they would keep taking the blackhole route.

So again, what is the actual purpose of the blackhole route?

I've missed the routing-mark, but I still don't get why you need the individual route to 77.88.8.7/32 to switch to the blackhole one and in what situation.

Plus if it did, it would be a self-locking positive feedback - from the moment the actual route would become inactive and so the blackhole route would take over, the pings to 77.88.8.7 would never succeed as they would keep taking the blackhole route.

So again, what is the actual purpose of the blackhole route?

I use 77.88.8.7/32 for checking Internet state through current ISP
if blackhole route comes active it will drop any trafic to 77.88.8.7/32 through other out interfaces, untill Internat state on pppoe ISP site comes up again - thats the trick of fallover
I also undestand that route with distance 0 would stay active long as the interface pppoe up ( thats why I post this route in my first post ) and thats why blackhole cannot change it state

My question is: how to make blackhole route comes active in that situation with pppoe interface( when there is still directly connected route)
Is there some way to do this ?

But if you have connected route via pppoe, packets won't be sent via another interfaces!