Simple routing problem

ipdruide · Thu Sep 21, 2006 4:03 pm

Hello everyone,
Greetings.

I 've been on this one for a couple of weeks although teh problem seems fairly simple. It is again related to multiple gateways.

I am running a routerboard 2.9.30 with 2 DSL links and fixed IPs. There is some routing with marks for incoming trafic to be responded through the right gateway and all that works fine. Besides there is a default gateway to handle all unmarked and router's trafic.

My problem is that Router own ip services (telnet, ftp, ssh ....) are only reachable from the wan link that is default gateway. Of course I cannot add another default gateway for the second link, and RouterOS seems to be responding to for example ssh requests from both external link via the same gateway. In a word: how can I force Router to respond to those requests via the link the request came from.

Here are my settings :

I use masquerade for local users and dst-nat for local server to be reachable from the internet.

add dst-address=0.0.0.0/0 gateway=111.111.111.111 distance=0 scope=255 \
target-scope=10 routing-mark=even comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=222.222.222.222 distance=0 scope=255 \
target-scope=10 routing-mark=odd comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=111.111.111.111 distance=0 scope=255 \
target-scope=10 routing-mark=fragile comment="No load balancing for \
fragile web sites" disabled=no
add dst-address=0.0.0.0/0 gateway=111.111.111.111 check-gateway=ping distance=0 \
scope=255 target-scope=10 routing-mark=rout_B comment="Inbound \
trafic response via B" disabled=no
add dst-address=0.0.0.0/0 gateway=111.111.111.111 distance=0 scope=255 \
target-scope=10 comment="router own path" disabled=no
add dst-address=0.0.0.0/0 gateway=222.222.222.222 check-gateway=ping distance=0 \
scope=255 target-scope=10 routing-mark=rout_A comment="Inbound \
trafic response via A" disabled=no

Could it be some very silly setting that I am not aware off, or this could not be done.

Thanks for any suggestions.

ipdruide · Fri Sep 22, 2006 11:38 am

My apologies for insisting.

This may be a simple setting in my routing, but I need help from an expert. I know there are quite a few in this forum.
Many thanks.

Eugene · Fri Sep 22, 2006 1:36 pm

Post complete list of routes and ip addresses on the router. Indicate, to what address you are trying to connect.

ipdruide · Mon Sep 25, 2006 1:47 pm

Eugene,
Thank you for your response. What I am trying to do is connect from the internet to the Router services (ssh, ftp etc...) to BOTH isp1 and isp2 public addresses: Until now I only can connect to the address that is related to the main-route (in BOLD italic).

public addresses :
6 D 195.154.30.132/32 212.129.9.84 0.0.0.0 pppoe-isp1
7 D 193.252.209.222/32 193.253.160.3 0.0.0.0 pppoe-isp2

Thank you for your help again.

Routes

Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC G GATEWAY DIS INTERFACE
0 A S 0.0.0.0/0 r 193.253.160.3 0 pppoe-isp2
1 A S 0.0.0.0/0 r 212.129.9.84 0 pppoe-isp1
2 A S ;;; No load balancing for fragile web sites
0.0.0.0/0 r 193.253.160.3 0 pppoe-isp2
3 A S ;;; Inbound trafic response via isp2
0.0.0.0/0 r 193.253.160.3 0 pppoe-isp2
4 A S ;;; main-route
0.0.0.0/0 r 193.253.160.3 0 pppoe-isp2
5 A S ;;; Inbound trafic response via isp1
0.0.0.0/0 r 212.129.9.84 0 pppoe-isp1
6 ADC 172.16.0.0/24 172.16.0.101 0 ether1
7 ADC 172.16.1.0/24 172.16.1.101 0 wlan1
8 ADC 193.253.160.3/32 193.252.209.222 0 pppoe-isp2
9 ADC 212.129.9.84/32 62.210.110.170 0 pppoe-isp1

Adresses :

Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 172.16.0.101/24 172.16.0.0 172.16.0.255 ether1
1 172.16.1.101/24 172.16.1.0 172.16.1.255 wlan1
2 62.210.110.170/32 212.129.9.84 212.129.9.84 pppoe-isp1
3 62.210.110.169/32 212.129.9.84 212.129.9.84 pppoe-isp1
4 ;;; 168 Dedicated to tarpit
62.210.110.168/32 212.129.9.84 212.129.9.84 pppoe-isp1
5 62.210.110.171/32 212.129.9.84 212.129.9.84 pppoe-isp1
6 D 195.154.30.132/32 212.129.9.84 0.0.0.0 pppoe-isp1
7 D 193.252.209.222/32 193.253.160.3 0.0.0.0 pppoe-isp2

Eugene · Mon Sep 25, 2006 2:15 pm

please post:
/ip route print detail
/ip firewall mange print

ipdruide · Mon Sep 25, 2006 5:13 pm

Here they are :

> /ip route print detail
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf,
B - blackhole, U - unreachable, P - prohibit
0 A S dst-address=0.0.0.0/0 gateway=193.253.160.3 interface=pppoe-isp2
gateway-state=reachable distance=0 scope=255 target-scope=10
routing-mark=even

1 A S dst-address=0.0.0.0/0 gateway=212.129.9.84 interface=pppoe-isp1
gateway-state=reachable distance=0 scope=255 target-scope=10
routing-mark=odd

2 A S ;;; No load balancing for fragile web sites
dst-address=0.0.0.0/0 gateway=193.253.160.3 interface=pppoe-isp2
gateway-state=reachable distance=0 scope=255 target-scope=10
routing-mark=fragile

3 A S ;;; Inbound trafic response via isp2
dst-address=0.0.0.0/0 gateway=193.253.160.3 check-gateway=ping
interface=pppoe-isp2 gateway-state=reachable distance=0 scope=255
target-scope=10 routing-mark=rout_isp2

4 A S ;;; main-route
dst-address=0.0.0.0/0 gateway=193.253.160.3 interface=pppoe-isp2
gateway-state=reachable distance=0 scope=255 target-scope=10

5 A S ;;; Inbound trafic response via isp1
dst-address=0.0.0.0/0 gateway=212.129.9.84 check-gateway=ping
interface=pppoe-isp1 gateway-state=reachable distance=0 scope=255
target-scope=10 routing-mark=rout_isp1

6 ADC dst-address=172.16.0.0/24 pref-src=172.16.0.101 interface=ether1
distance=0 scope=10 target-scope=0

7 ADC dst-address=172.16.1.0/24 pref-src=172.16.1.101 interface=wlan1
distance=0 scope=200 target-scope=0

8 ADC dst-address=193.253.160.3/32 pref-src=193.252.209.222
interface=pppoe-isp2 distance=0 scope=10 target-scope=0

9 ADC dst-address=212.129.9.84/32 pref-src=62.210.110.170
interface=pppoe-isp1 distance=0 scope=10 target-scope=0

ip firewall mangle>
ip firewall mangle> pr
Flags: X - disabled, I - invalid, D - dynamic
0 chain=forward protocol=tcp tcp-flags=syn action=change-mss new-mss=1360

1 chain=forward protocol=tcp tcp-flags=syn action=passthrough

2 ;;; Mangle fragile web site NO load balancing
chain=prerouting in-interface=ether1 connection-state=new
dst-address-list=fragile action=mark-connection
new-connection-mark=fragile passthrough=yes

3 chain=prerouting in-interface=ether1 connection-mark=fragile
action=mark-routing new-routing-mark=fragile passthrough=no

4 ;;; Mangle for load balancing odd even
chain=prerouting in-interface=ether1 connection-state=new nth=1,1,0
action=mark-connection new-connection-mark=odd passthrough=yes

5 chain=prerouting in-interface=ether1 connection-mark=odd
action=mark-routing new-routing-mark=odd passthrough=no

6 chain=prerouting in-interface=ether1 connection-state=new nth=1,1,1
action=mark-connection new-connection-mark=even passthrough=yes

7 chain=prerouting in-interface=ether1 connection-mark=even
action=mark-routing new-routing-mark=even passthrough=no

8 ;;; Mangle for isp1 incoming trafic
chain=prerouting in-interface=pppoe-isp1 connection-state=new
action=mark-connection new-connection-mark=con_isp1 passthrough=yes

9 chain=prerouting in-interface=ether1 connection-mark=con_isp1
action=mark-routing new-routing-mark=rout_isp1 passthrough=no

10 ;;; Mangle for isp2 incoming trafic
chain=prerouting in-interface=pppoe-isp2 connection-state=new
action=mark-connection new-connection-mark=con_isp2 passthrough=yes

11 chain=prerouting in-interface=ether1 connection-mark=con_isp2
action=mark-routing new-routing-mark=rout_isp2 passthrough=no

ipdruide · Tue Sep 26, 2006 10:36 am

If I may add my feeling:

- I do not understand why a router would answer requests addressed to one of his wan links from another link. In other words, routes designed to draw paths from the lan to outside world shouldn't apply to services that are listening on the wan side. Isn't this an obvious bug ?

regards.

ipdruide · Tue Sep 26, 2006 11:28 am

Also IPSEC tunnels (I didn't try with L2TP or PPTP) stop working if the default gateway is NOT the one related to the peer IP.

All this is the same problem: Once there are more then one internet links, then router's inner services tend to respond via the default gateway.

Thank you for any comments.

Eugene · Tue Sep 26, 2006 12:13 pm

All this is the same problem: Once there are more then one internet links, then router's inner services tend to respond via the default gateway.

Thank you for any comments.

Router does not have "preferences". If it does not have a specific route to the destination, it _will_ respond through the default gateway.

ipdruide · Tue Sep 26, 2006 1:04 pm

Eugene, do you mean there is no way to acces routers services from differents public IPs ?

Also IPSEC (and may other L2TP, pptp) tunnels wouldn't work is there is more then one peer, AND/OR if you use for tunnels 2 or more differents wan links ?

Please confirm.
Thank you.

ipdruide · Tue Sep 26, 2006 3:54 pm

Eugene wrote

Router does not have "preferences". If it does not have a specific route to the destination, it _will_ respond through the default gateway.

I do not see really why. I believe things would work better in many situations if in any LOCAL services were responding from the relevant path instead of using NEXT DOOR. Can anyone confirm if I am wrong or right and may be give us a hint on what RFCs would say on the matter ?

In the eventuality of my being wrong, may be there is a workaround ?

Thank you for any comments.

Eugene · Tue Sep 26, 2006 4:42 pm

Lets start over. Suppose, you are accessing the router (IP 2.0.0.1, 3.0.0.1) from a computer (IP 1.0.0.1) over the Internet. the router has two upstream links, one connected to IP 2.0.0.2 and the other to 3.0.0.2. The default route points to 2.0.0.2.

Now if the router does not have a specific route that instructs the router where to send traffic destined to 1.0.0.1, then the router will always reply through the default gateway.
You can alter this behavior by adding a route to 1.0.0.0/24 specifying 3.0.0.2 as a gateway:

/ip route add dst-address=1.0.0.0/24 gateway=3.0.0.2

ipdruide · Tue Sep 26, 2006 8:41 pm

My apologies, but this is a poor workaround, since I have to create an entry in the routing table for every single destination address. Which limits severally the router accessibility. The purpose for a dual link in more a problem of availability than bandwidth, as you can imagine. It becomes useless to purchase 2 isp accounts.

Couldn't it be a wrong behaviour in the router to use the wrong path when answering external requests on wan links ? If this behaviour is wrong then MT has to fix it. In my previous post I was seeking for RFC compliance proofing.

By the way, I am using routing-test package. I gave a try to the routing but the the behaviour was no different.

May be you can think of some other workaround meanwhile.

My regards.

Eugene · Tue Sep 26, 2006 8:46 pm

It's not a workaround. It's the way how routing works regardless of device brand. The routing table contains instructions for the router how to send a packet for the particular destination and the router obeys these rules. If it is instructed to go through one gateway, it won't go through the other one. However, you could add second default route with a different gateway that will become active if primary gateway fails. (you have to configure "check-gateway" parameter for that)

Eugene

changeip · Tue Sep 26, 2006 8:51 pm

The router is doing what it is told, strictly using the routing table. You need to packet mark / connection mark based on the interface it came in on. Then on the prerouting / output chain you should be able to route mark it to reply on the correct routing table.

I think certain things, like ICMPs, can't be route marked, but things that end up in the connection tracking table should be possible. Just mark them on the way in, and then force those to use the route-mark you wish on the way out. You also might need to add routes to the rout_isp1&2 tables for it's own subnets, otherwise it will fall out into the main table.

ipdruide · Wed Sep 27, 2006 1:25 pm

Thanks and gratefullness to Eugene and Sam for their help and directions.

I got it working. I had previously tried the connection/route marks and routes based on the routing marks but I was using the "prerouting" chain only because I ( I must confess) wasn't really aware of the differences between prerouting and input/output chain. Lack of training !

Now I am using input chain for mark-connection mark and output chain for mark-routing rules to segregate self router trafic and forwarded trafic ( to other hots in lan).

From now on I think use ssh, ftp, http, winbox to remotely administer an MT box regardless on the ISP link or public IP I will be using. Also, I am about to give a try to dual IPSEC tunnel thanks to this result.

By the way: are IPSEC redundant tunnels possible ? bondable ?

Thank you again for the progress done.

Eugene · Wed Sep 27, 2006 4:24 pm

IPsec tunnels could not be bonded, because they are Level 3. You could, however, use routing to fail-over between them.

Eugene

ipdruide · Wed Sep 27, 2006 6:29 pm

I may have missed something. But before I start useless work, my understanding was that :

- one could build EoIP over IPSEC tunnels.
- EoIP was Ethernet like interface thus bondable.
- Then if I have 2 offices with 2 ISPs each I could bond the EoIP tunnels for redundancy and bandwidth sake.

Sorry I am am mixing up unrelated stuff. I am still in the process of aquiring basics.

Thanks again.

Eugene · Wed Sep 27, 2006 8:34 pm

If you need Layer 2 communication between two offices, then the setup you outlined is a good choice. However, if L2 is not a requirement, I'd choose routing across 2 IPsec tunnels.

Simple routing problem

Simple routing problem

Is this routing problem worthless ?

Routes and addresses

IPSEC sould also stop working if there are 2 wan links

Re: IPSEC sould also stop working if there are 2 wan links

Poor workaround

It's working !

Not even bonding encrypted EoIP tunnels ?

Who is online