Page 1 of 1
multiple gateways & routing
Posted: Wed Apr 29, 2020 10:56 pm
by looka
hi,
i know, this has been answered, but i can't get it to work. so. I would like to route traffic from/to my server using different route than the rest of my devices.
i have 2 public IPs, one dynamic by DHCP and one static. Each IP is also using its own gateway, which I'd like to use (and proove it's working using tracerouting from my server).
So far:
1. I've learned I've got to mark routing connection from server in prerouting mangle.
2. Default route, created by DHCP client, is set automatically. I only need to add default route for my marked connections. Also have to add static external ip.
3. Nat connections to/from my server
/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=<static_ip> in-interface-list=WAN new-routing-mark=server passthrough=yes
add action=mark-routing chain=prerouting new-routing-mark=server passthrough=yes src-address-list=list-server
/ip route
add distance=1 gateway=<static_ips_gateway> routing-mark=server
/ip address
add address=<static_ip/mask> interface=sfp1 network=<static_network>
/ip firewall nat
add action=src-nat chain=srcnat comment="static IP from server" src-address-list=list-server to-addresses=<static_ip>
add action=dst-nat chain=dstnat comment="web server" dst-address=<static_ip> dst-port=80,443 protocol=tcp to-addresses=<internal_server_ip>
....but setup doesn't work. It actually makes my server to loose connection from public IP.
Yes, there is more efficient way of marking connection and I'm using src-address-list instead of simply using src-address, but that's not the point.
Help?
Re: multiple gateways & routing
Posted: Thu Apr 30, 2020 8:47 am
by looka
I was thinking - does masquerading take into account connection's default route? In that case, is it enough to have simple masquerading rule, which would then decide which ip to take into account for both, dynamic and static public ip? Instead of src-natting for static and then masquerading for dynamic public ip.
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
Re: multiple gateways & routing
Posted: Thu Apr 30, 2020 10:19 am
by looka
Adding a route for the lcocal bridge seems to enable accessibility, but why and can i get rid of that extra rule? it feels wrong. also, traceroute from my server still shows traffic goes throug dynamic public ip's gateway.
/ip route
add distance=1 dst-address=192.168.44.0/24 gateway=bridge pref-src=192.168.44.1 routing-mark=server scope=10
btw, simply masquerading all outgoing traffic does it nicely - assigns dynamic external ip to computers and the static external ip to my server. kinda neat, i'm not ultra worried about my cpu.
Re: multiple gateways & routing
Posted: Fri May 01, 2020 3:10 am
by Sob
Study this:
https://wiki.mikrotik.com/wiki/Manual:PCC
It's primarily about load balancing, so ignore that part. In short, you can't mark routing directly like you tried, because routing marks are per-packet. And you actually want it only for outgoing packets. So first mark incoming connections and then mark routing for outgoing packets belonging to marked connections.
Re: multiple gateways & routing
Posted: Fri May 01, 2020 2:58 pm
by looka
Well, I've checked this and I understand I'm waisting CPU with my "direct route marking" approach, but it does the same job. And everything else, but I can't get my head around one thing - in the example they have assigned network ID to interface called LAN. Any idea what is that interface? Bridge? Does it even matter?
Actually, my case is totally the same as PCC example and my question simplifies into:
If I'm using connection marking to mark different routes, why do I need to have two identical local routes, one marked and one for unmarked connections, like this:
/ip route
add distance=1 gateway=84.255.192.1 routing-mark=server
add distance=1 dst-address=192.168.44.0/24 gateway=bridge routing-mark=server
If I leave connection marking enabled, but disable both routes (default gateway and local bridge routing), then also marked connections make it through to the internet. While if I only have route for default gateway, marked connections cannot reach internet.
Re: multiple gateways & routing
Posted: Sat May 02, 2020 5:59 am
by Sob
Well, you don't have to mark connections if the server should always use same WAN. In that case, this rule is ok:
/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=server passthrough=yes src-address-list=list-server
Or another way would be to use routing rule (/ip route rule) with server's source address to select the other routing table.
But you don't need this:
/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=<static_ip> in-interface-list=WAN new-routing-mark=server passthrough=yes
That's actually breaking routing for incoming packets, which you're fixing with otherwise unneeded route to LAN in the other routing table.
Re: multiple gateways & routing
Posted: Sat May 02, 2020 1:01 pm
by looka
Oh that is much better for my purpose, thanks!
That's my setup now:
/ip address
add address=<public-static-ip> interface=<public-if>
/ip route
add distance=1 gateway=<gateway-for-static-ip> routing-mark=server
/ip route rule
add src-address=<server-internal-ip>/32 table=server
...using default masquerade natting from before and no mangling around.
Re: multiple gateways & routing
Posted: Sat May 02, 2020 2:42 pm
by looka
Okay, there are one or two problems, though - accessing my server by domain, which resolves to the external static ip from inside my network.
Before this new route of mine, I was solving this problem till now using hairpin nat rule, but now it does not seem to work anymore.
Is there a route or rule, I can add, that would do the trick?
I know, there is a static DNS resolving option, but has it's minuses. It seems to me somehow route that properly like the most proper thing to do.
Re: multiple gateways & routing [SOLVED]
Posted: Sat May 02, 2020 6:25 pm
by Sob
As you already know, with hairpin NAT client sends packet to public address (which is on router, but client has no idea about that) and router sends it back to LAN to server's internal address. This part is still ok. But then server responds from its internal address and guess what? Your routing rule matches, so after conntrack does its magic and final destination is client's internal address, it's already decided that it will be looked up in "server" routing table. And it's not there, because it contains only one default route, so packet will be sent to internet.
You need another routing rule
before the one you have now:
/ip route rule
add action=lookup-only-in-table dst-address=<LAN subnet> table=main
Other way would be to add route to LAN subnet to "server" routing table, same way you had before, that would work too. But routing rule is generally better, because you need only one, no matter how many extra routing tables you have (it doesn't show much if you have only one).
Re: multiple gateways & routing
Posted: Sun May 03, 2020 1:26 am
by looka
Another route rule + hairpin NAT did the trick, yes. Thanks.
But is there a direct way, without hairpining? Using this technique my server sees router's IP whenever I'm connected to it from local network.
There is one last thing - can I make the server accessible from router by its domain name (which resolves to public IP, which, technically, belongs to one of router's interfaces)?
Say, I'd like to fetch a script from my server and to minimize the customizations later, I'd like to use my server's domain name.
Re: multiple gateways & routing
Posted: Sun May 03, 2020 3:39 am
by Sob
If you want it direct, you don't have many options. Basically just static DNS records, with all disadvantages.
If you're worried more about seeing source addresses than going through the router, you have one or two options:
a) Map clients' source addresses in some virtual subnet. Instead of masquerade, use netmap and made up subnet with same size as the real one. Server still won't see real addresses, but it will see these virtual ones, so you'll be able to tell clients from each other.
b) Depending on your server, it could be possible to have "half-haipin NAT", i.e. connections would still go via router, but you could keep original source addresses. The way to do it would be to watch (on server) for connections from LAN subnet and check from what MAC address they came from. If it would be MAC address of default gateway, you'd know that it's hairpin and you'd send responses back to gateway. It would be possible with e.g. Linux.
Access from router to server's hostname is only possible with static DNS record pointing to server's local address, because RouterOS doesn't support dstnat for own outgoing connections.
Re: multiple gateways & routing
Posted: Mon May 04, 2020 12:32 am
by looka
If I move my server to another subnet, then I don't need the hairpin and because of that lookup-only-in-table rule I'm able to see original IP.
But, if I do that, CPU should take the hit, because all traffic between my server and local (gigabit) devices is not switched anymore, but routed. I have hex s, which does not have fancy switch chip to help cpu.
On the other hand, I was told it is nice to move server out of main network, as it can be hacked through services, running on it...
Re: multiple gateways & routing
Posted: Tue May 05, 2020 5:45 am
by Sob
Fancy switch wouldn't help you anyway, because different subnets means routing. And yes, separate networks are more secure. Of course it's also a question, whether there's something worth protecting in the other subnet.
Re: multiple gateways & routing
Posted: Tue May 05, 2020 11:05 am
by looka
Well, it was a test if it is possible to do it.
I found something, that stopped me from separating my server from the network - DLNA and its SSDP. SSDP misuses word "simple" quite badly.
There is something, that bothers me still, but I really don't know whether because of my setup or this is supposed to be like this
This rule
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
Sometimes, not always, fires also when using external IP address, so I had to aadd this:
dst-address-list=!list-router
or even better,
Is that normal?
Re: multiple gateways & routing
Posted: Tue May 05, 2020 2:23 pm
by Sob
That's default NAT rule for accessing internet, which hides LAN behind address on WAN.
I don't know what you have in "list-router" address list, but if it's public address, then if it's directly on this router, it can't happen, because packet with this destination will go to router itself and won't reach srcnat chain.
The other condition for not local source address would exclude router itself from masquerade, which should be ok, since it has the public address anyway and should choose it as source when accessing internet. Problem would be if it chose address from one WAN and you redirected packets to use another WAN, then if wouldn't work.
It's also possible that I don't undestand correctly what you're trying to do here.
Re: multiple gateways & routing
Posted: Tue May 05, 2020 10:57 pm
by looka
You understand enough
masquerading public ip happens, when router (or device behind it) tries to reach IP, which is not in router's public IP network.
I think the chain goes like this: local_ip -> masquerade to public_ip, or if router starts connection, then public_ip -> masquerade because destination IP not in same subnet as public ip -> public ip. Latter does nothing, apart from making me nervous about things I have no clue about, but still shows up in src-nat chain.
the "list-router" rule does nothing, as it is composed only of private router's IPs, so the port type !local does the trick. I understand that router will pick one of its public addresses for masquerading according to routing rules? In that case I should be fine.
My setup:
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN src-address-type=!local
add action=jump chain=dstnat comment="to server" dst-address=<external-ip1> jump-target=dstnat-srv to-addresses=<server-internal-ip>
/ip route
add distance=2 gateway=<gateway of external-ip1> routing-mark=server
/ip route rule
add action=lookup-only-in-table dst-address=<my subnet> src-address=<my subnet> table=main
add src-address=<server-internal-ip> table=server
Re: multiple gateways & routing
Posted: Wed May 06, 2020 5:05 am
by Sob
For the record, I overlooked "!", so second paragraph in my previous post is nonsense.
But in any case, you don't need to worry about excluding router from srcnat for its own connections to internet. Keeping it included can either help (in case it chooses wrong source address; although it's more other config that can make it wrong), or it will neither help not hurt anything.
Router picks source address based on used route. So if it's connection to something in LAN, it will pick address which is on LAN interface. When it's connection to internet, it will pick address from WAN where is default route. But after this happens, you can still change routing (in /ip firewall mangle chain=output) and originally selected source address may be wrong for that (if it's e.g. another ISP's WAN). Srcnat happens after this and if it's automatic masquerade, it will pick address from actually used outgoing interface.
You can see the whole thing here:
https://wiki.mikrotik.com/wiki/Manual:P ... ng_Diagram
Re: multiple gateways & routing
Posted: Wed May 06, 2020 11:23 am
by looka
decision of excluding own router's ip masking was to free up some cpu, since it looked as a simple thing to do.
Re: multiple gateways & routing
Posted: Wed May 06, 2020 2:45 pm
by Sob
It's going to be 0.000nothing, because connection tracking happens anyway and that's the heavy part. I'm not even sure if srcnat actually does something, when the address was already the right one before. And it's not like the router itself has too many outgoing connections anyway.