MikroTik

Hi,

I've been trying to figure out this for some time now. I moved all 3 uplinks from Cisco box to CCR and everything works fine - except that RouterOS seems to select out-interface address as a source address for packets originating for the router itself - in this case icmp messages used for traceroute from remote location towards the server behind CCR router.

When you have 3 uplinks and you are doing full-tables v6/v4 routing with BGP - not all the time the incoming interface that ICMP message is sent to as part of traceroute procedure is also outgoing interface. Traffic engineering on the Internet and nature of BGP helps with that. Nevertheless, it represents a problem when you try to visualise the traceroute and ASNs involved.

I would like to setup the router to use loopback address as a source for all ICPM (or even all other packets) that originates from router itself. Is this possible? Cisco box used loopback as default source address, but CCR behaves differently.

You can have a look at http://bgp.go6.si/ring/ and see what pictures I get due to src-addr of interface instead of loopback. Prior to that all links from Amis, T-2 and SIOL were pointing to my router and not to each other (and *that* is caused by packets coming from one provider to the router and using the interface of different provider to go out and that interface address as a source).

I'm using "update-source=loopback_addr" in my bgp sessions, but to no avail.

Any idea? Any hint?

Thank you, Jan Zorz

You can use src NAT on ICMP to change the originating ip. You'll probably want to match the src addresses on the rule so as to not to match ICMP passing through the router.

You can use src NAT on ICMP to change the originating ip. You'll probably want to match the src addresses on the rule so as to not to match ICMP passing through the router.

This is an ugly hack, I have to admit :S

If there are no other options that I'll have to use it, but probably it would be a very nice feature if we could actually be able to select a source IPv4 and IPv6 address that is used for packets, originating from a router itself.

NAT only solves half of my problem - IPv4. I need a solution for both protocols - IPv4 and IPv6.

Cisco box automatically selects loopback address as s source (if it exists and is configured)

Cheers and thnx, Jan

I would think that the natural IP to respond would be the IP that received the request. The job of NAT is to change source IP's. Therefore it doesn't sound like a hack to me. It sounds strange to me that Cisco would receive info on one IP and respond from another. Are you sure that Cisco isn't rewriting the source using NAT and just not calling it NAT?

I would think that the natural IP to respond would be the IP that received the request.

Well, yes, indeed. But I have 3 different uplinks to 3 different ASN-s and when you do a traceroute from outside towards my network and match the hops to ASNs, then sometimes it happens that return path is different and ASN traceroute then looks like that your neighboring ASN1 sends packets to your neighboring ASN2, that actually is not true - it's just that different interface address (outgoing) was shown in traceroute.

The job of NAT is to change source IP's. Therefore it doesn't sound like a hack to me. It sounds strange to me that Cisco would receive info on one IP and respond from another. Are you sure that Cisco isn't rewriting the source using NAT and just not calling it NAT?

Nope. If C* finds a loopback interface - it will use it

Cheers, Jan

you have to mark packets from up-stream and then add routing-mark to them ensuring that the router will answer on the same interface it received packet from.

In your case what is happening - router receives the packet and internal process generates response, then routing decides that "hay, that other link has had no packets sent out recently" and sends a response over a different interface. Since the packet is leaving thorough some interface, it should have appropriate source address of the interface. To overcome that addition of routing mark to connection will ensure that the packet is sent over the route with routing mark set.

This allows more flexibility on what you want to do with your IPv4 packets.

you have to mark packets from up-stream and then add routing-mark to them ensuring that the router will answer on the same interface it received packet from.

But I should do this only for icmp packets, directed to the router itself (input chain). I'm not interested in touching any other traffic, BGP is the mechanism to select where and which way the packets should exit the router.

In your case what is happening - router receives the packet and internal process generates response, then routing decides that "hay, that other link has had no packets sent out recently"

This is not the case here. BGP decides and installs best AS-PATHS and routes and majority of times packet comes in through different provider and interface and on the path back (route) the packet is sent to different upstream. Let's call that "Global Internet traffic engineering"

and sends a response over a different interface. Since the packet is leaving thorough some interface, it should have appropriate source address of the interface. To overcome that addition of routing mark to connection will ensure that the packet is sent over the route with routing mark set.

This allows more flexibility on what you want to do with your IPv4 packets.

Do you have any example of how I could mark only traceroute/icmp packets on "input chain" ?

Thnx, Jan

Did you ever find a solution to this?

Running into the same exact issue - multiple CCR's, 5+ upstreams with full tables.

Inbound traceroute via upstream A will show the interface IP of upstream B/C/D randomly which is confusing. It seems like source ip of the icmp packet hitting upstream A is set to the ip from another interface.

Old thread, same problem,

no solution ?

I was going to try changing the default route src address in the routing table, I presume being a linux kernal that might do it !

Seems to work my pings from that machne weren't working now they are.

I will presume the icmp unreachable etc will be formed the same way !

I just set the src address to the loopback address

Your problem doesn't sound the same.

I have not seen a solution for this. Pretty annoying.

have not seen a solution for this. Pretty annoying.

I believe janisk answered this... Mark the packets and return via the same interface. Other then that, I think you would have rewrite the reply but NAT would only address IPv4 traffic... Maybe there isn't another answer for this because there isn't a good solution other then sending the reply sourced from the receiving IP.

have not seen a solution for this. Pretty annoying.

I believe janisk answered this... Mark the packets and return via the same interface. Other then that, I think you would have rewrite the reply but NAT would only address IPv4 traffic... Maybe there isn't another answer for this because there isn't a good solution other then sending the reply sourced from the receiving IP.

This isn't a real solution due to the additional overhead. Why mark packets and setup rules for something that works out of the box on any other competing device?

This is a bug, not a feature.

This is a bug, not a feature.

I can understand making a feature request but I don't think this is a bug. As far as I know RouterOS is functioning as designed.

The router is choosing the outbound IP based on the route table because the packet is originating with the router and being sent to the destination. If you want to control the IP that replies then choosing based on the out interface seems logical to me. I don't think you would have a huge performance hit with tracking and adding routing marks ICMP traffic on the routers input chain.

It is not a bug, but feature which is not implemented at the moment. You might see this feature in a future versions.

What configuration exactly on cisco do you use?
As far as I can find you still need to use NAT on Cisco to change ICMP replies.
http://networklessons.com/network-servi ... n-example/

for other protocols, BGP, NTP ... loopback source is specified explicitly in configuration.

What configuration exactly on cisco do you use?
As far as I can find you still need to use NAT on Cisco to change ICMP replies.
http://networklessons.com/network-servi ... n-example/

for other protocols, BGP, NTP ... loopback source is specified explicitly in configuration.

in cisco land if you have, say ISP A on 192.168.1.1 and ISP B on 10.5.5.5

And you have some network on 2.2.2.2

------
int gi0/0
ip address 192.168.1.1/30

int gi1/0
ip address 10.5.5.5/30

int gi2/0
ip address 2.2.2.2/30
...

Now you have the same isp setup as BGP neighbor, etc.

----


If you traceroute to 2.2.2.2 and transit over ISP A, before you hit the final destination, you will see 192.168.1.1/30 - because the icmp response comes back from the interface the packet went into.

The problem with mikrotik, is if you have a different gateway (/ip route) based on BGP, or a default, or whatever protocol then mikrotik will send icmp response over /that/ gateway. So you have a confusing scenario where you have a traceroute going over ISP A, and then suddenly before your final destination of 2.2.2.2 mikrotik returns an extra "hop" that shows ISP B, 5.5.5.5.

---------------

This is why I consider this a bug. In cisco, juniper, or any other land you would need to explicitly set up rules to send icmp response from a different physical interface -ie ISP B if it is coming in first ISP A interface. There is no loopback or anything else needed - it's a basic principle that isn't considered by mikrotik.

When you have multiple ISP's, and multiple routes or gateways based on a protocol, IE BGp where the best path or gateway may change dynamically as well, Mikrotik sends the icmp before the final destination seemingly random from what it sees as in fact the "gateway" of record from its routing table.

This leads to end user headache and troubleshooting issues in a multihomed environment where one wonders WHY do you see your ISP B gateway after the path has come in from your ISP A.

Obviously I am "paraphrasing" here and this is just a typed out example but it is easily duplicated. If i have the time I will do it.

RouterOS always uses gateway from routing table to send packet out. Either you have asymmetric routing or you have more specific route which routes through ISP2. You can verify with torch or sniffer on specific interface.

also, you can use provided tools in RouterOS to force all incoming connections out the same route they came in. Policy routing for ipv4.

Only 1 service i know of works like described by mspeed - SNMP, you get responses out of the interface UDP request came in. The rest correctly uses routing to determine outgoing interface.

Dear Mikrotik,

Can you provide the sample policy for ref?

TungHo

Configuring the Source Address for Locally Generated TCP/IP Packets

By default, the source address included in locally generated Transmission
Control Protocol/IP (TCP/IP) packets, such as FTP traffic, and in User
Datagram Protocol (UDP) and IP packets, such as Network Time Protocol
(NTP) requests, is chosen as the local address for the interface on which
the traffic is transmitted. This means that the local address chosen for
packets to a particular destination might change from connection to
connection based on the interface that the routing protocol has chosen to
reach the destination when the connection is established. If multiple
equal-cost next hops are present for a destination, locally generated
packets use the lo0 address as a source.

To configure the software to select a fixed address to use as the source
for locally generated IP packets, include the default-address-selection
statement at the [edit system] hierarchy level:

[edit system]

default-address-selection;

If you include the default-address-selection statement in the
configuration, the software chooses the system default address as the
source for most locally generated IP packets. The default address is
usually an address configured on the lo0 loopback interface. For example,
if you specified that SSH and telnet use a particular address, but you
also have default-address selection configured, the system default address
is used. For more information about how the default address is chosen, see
the JUNOS Network Interfaces Configuration Guide.

For IP packets sent by IP routing protocols--including Open Shortest Path
First (OSPF), Routing Information Protocol (RIP), Resource Reservation
Protocol (RSVP), and the multicast protocols, but not including
Intermediate System-to-Intermediate System (IS-IS)--the local address
selection is often constrained by the protocol specification so that the
protocol operates correctly. When this constraint exists in the routing
protocol, the packet's source address is unaffected by the presence of the
default-address-selection statement in the configuration. For protocols in
which the local address is unconstrained by the protocol specification,
for example, internal Border Gateway Protocol (IBGP) and multihop external
BGP (EBGP), if you do not configure a specific local address when
configuring the protocol, the local address is chosen using the same
method as other locally generated IP packets.

RouterOS always uses gateway from routing table to send packet out. Either you have asymmetric routing or you have more specific route which routes through ISP2. You can verify with torch or sniffer on specific interface.

I have a CCR1009 with some VRF, multihoming, etc. At least, it must use gateway from its own VRF routing table but actually its answering with gateway from default routing table