Hi everyone

It seems both the policy-routing and using VRF (routing-test package) don't work when there isn't no routing in the main routing table. See the following example on what happens:

First set some ips:
/ip address
add address=192.168.0.1/24 broadcast=192.168.0.255 comment="" disabled=no interface=ether1 network=192.168.0.0
add address=10.0.0.1/24 broadcast=10.0.0.255 comment="" disabled=no interface=ether2 network=10.0.0.0

Let's assume there are two routers, 192.168.0.255 and 10.0.0.255.
192.168.0.0 is the untrust side and 10.0.0.0 is the trust side.
there is only routing in the 10.0.0.0/0 on the trust side.

So let's add some routing-tags
/ip firewall mangle
add action=mark-routing chain=output new-routing-mark=trust src-address=10.0.0.1
add action=mark-routing chain=output new-routing-mark=untrust src-address=192.168.0.1

Now, let's add some routing for internet.
/ip route
add dst-address=0.0.0.0/0 gateway=192.168.0.255 routing-mark=untrust

test it:
/ping 4.2.2.4 src-address=192.168.0.1
no route to host

????

Ok, let's add a default route:
/ip route add 10.0.0.255
/ping 4.2.2.4 src-address=192.168.0.1
WORKS

So unless there is a routing entry in the main-table it will not use the routing entry with correct routing-mark. Is there anyway to get around this problem? It seems to be the same problem for VRF in that it need a default route without a routing-mark in the routing table to actually check the correct routes (the ones with the routing-marks).

Is there any workaround for this? Since if the router that the default route is pointing to in the route table (with no routing-marks) goes down that route goes down and the routes with different routing-marks that should be working stop working.

br
Hippo

i think this is more of the case where your pings aren't being marked, not that the main table has to have something in it. Ive seen many times where the output mangle chain and route marking don't work as expected. ie; snmp responses, ntp responses, etc.

As has been discussed in other VRF related topics, at the moment there are issues (consider them missing features) with originating traffic from vrf or with specific routing mark. See - in order for packet to get to output firewall chain where you mangle it, it at first must get routed (e.g. to select outgoing interface so you can match it in output chain). And this routing lookup happens without knowledge of policy routing or VRF.

As a side note - it is a bad practise to use for some host IP address that is assigned as broadcast for your network (10.0.0.255 in your case).

Hi Mplsguy

Well, just a typo. I changed the ipaddresses from the real experiement I made.

However I think you are wrong, because the marking is working. In the example I gave below it does not use the default gateway it uses the gateway given by the vrf according to the marking. And if you check the documentation, packages enter the output chain when they are originating from the device, not because of any routing.

changeip:

As I wrote above, I think they are getting marked because they use the correct gateway.

So what do you think?

Hippo,

I did not say that marking is not working - it works, of course. In your example, before you add default route to "main" table, ping does not work because of the reason I mentioned - when ping packet is prepared, routing lookup is made to figure out a few things: whether there is route to destination at all (this is where you get that "no route to host" in first ping), which interface it should go out and to what gateway (I called it "routing" because it is exactly that - do not confuse it with "forwarding"). Only then packet can go to output chain. If output chain changes packet properties so that it may change the interface it must go out, it is "rerouted" (has its output interface and gateway looked up in routing table once again). Documentation may not be 100% clear about such corner cases because it shows main logic and sequence of events, it is more complicated under the hood.

All this does not apply when router is forwarding traffic, because you get to mark it before routing lookup is made.

Mplsguy,

Hmm, I think I misread you then. However if I don't really understand why you want to traverse the routing table twice. I can understand it in those cases where you don't have a src-address for the package (since then it would pick src-address which is the closest interface to the destination). However I don't really see why a package that already had a set src-address would need to do a pass through the routing table at this stage.

But given that this is how routeros works, do you know if there is any workaround for the problem? The problem being you are a reliant on a default route to have the other traffic working as well. Is for example anyway to make a route permanent (to not have routeros trying to ping/arping it)

br
Hippo!

Hippo,

Like I said - that second routing lookup happens only when something about packet changes that might affect its outgoing interface or gateway. For example - routing mark. This "second" routing lookup is the reason why your setup actually works correctly after adding default route to main table. After output chain changes routing mark, second route lookup ensures that it goes out over correct interface and gateway.

Currently there is no way to implement your setup without default route in main table. If you do not like this route be used (for originating or forwarding any traffic), you can filter out unwanted traffic in firewall.

As long as you do not enable "check-gateway" function, routeros is not pinging or "arping" gateway. Therefore static route will be active provided that its gateway is reachable (not in means of network connectivity but from routing point of view - there is more specific route to gateway). If you will specify interface as route gateway, it will be active as long as IP address is configured for that interface and that interface is running. So there are plenty of options to make route "permanent".