OK This is the weirdest thing ive ever seen.

We have 2 Main Nat rules at our edge MT:-
All private IP traffic leaving external interface ISP-1 --> Masquerade
All traffic leaving external interface ISP-2 --> Masquerade

These rules have never changed since we started using MT nearly 2 years ago.

Made the upgrade to 2.8.22 a few evenings ago, didnt make any changes or additions to firewall rules, just upgraded.

So everything comes back up, no problem, all appears good.
I check our monitoring system (DeepMetrix IPMONITOR) that basically pings every customer once every 60 secs to see who's up and who's down.

IPMONITOR is going crazy, several hundred up, and several hundred down compared to ony 15 down before the upgrade. (these 15 are simply custys that turn off their connections when they go on holiday...)

Panic kicks in, sweat flying everywhere, until I make a few calls to well known customers that IPMONITOR is reporting as 'down'
"No problem here, our data links are all fine" - Phew, all good.

So why is IPMONITOR reporting so much garbage ?

After firing up etherreal on every segment of our network we saw some very interesting results:-

It turns out that the MT box upgraded (above) is intermittently NATTING on the wrong interfaces !!!!
IPMONITOR only ever uses the INTERNAL interface for monitoring customer links, and there aint no NAT rules there !

Only traffic that is supposed to leave the EXTERNAL interfaces should be natted.

The problem lies in the fact that a lot of IPMONITOR traffic (source IP 10.33.12.43) that all leaves the INTERNAL interface is coming out the internal interface with the NATTED address from EXTERNAL Interface number 1, and then sometimes it switches to the IP from external interface number 2 !!!!!!
(This is intermittent, its about half and half, but it SHOULD ALL be coming out the internal interface with the real src address of 10.33.12.43, not NATTED PUB IP number 1 or PUB IP number 2)!

We are not seeing this 'Sticky Nat' thing anywhere else, only the IPmonitor traffic is affected. I am presuming this is because it makes about 700 ICMP connections at the same time and 2.8.22 just gets a bit confused ?!!

I even tried adding an explicit 'DO NOT NAT' rule (Accept) at the top of the SRC NAT chain for ipmonitor traffic (10.33.12.43 ---> Accept) And this made no difference

Please help, downgrading is not an option at this time due to SLA agreements. We have VRRP redundancy but im not prepared to take the risk. The VRRP secondary MT is currently offline to ensure this is not contributing to the problem.

Anyone seen this before ?

We are having exactly the same problem, when our system failed over using two Net connections, after upgrading to 2.8.22.

All ive done is revert to 2.8.16 for now..

Lee

thanks for that, I thought I was going mad.

I also rolled back (to 2.8.19) and all works fine now.

We updated the routing manual:

Note also that if there are more than one public interface and more than one address on any of these interfaces, then policy-routing and equal-cost multipath routing may not work correctly if masquerading is used. To avoid this problem, use action=nat instead of action=masquerade.

Could you send us the support output file from your router?