I have a customer we provide service to with full routes via BGP.

They have no problems when they just have our peer active or their other peer active. When they bring both peers online they can't get to certain websites, or certain AS as a whole.

They have two Cisco routers running HSRP between the two. One of the routers connects to us the other connects to their other BGP peer. They have iBGP running between the two HSRP routers so both have full routing tables from us and their secondary peer.

I haven't been able to do much in the way of troubleshooting as they can't leave both peers active for long as it disrupts their access.

From their cisco configs it appears our router is the more preferred in the HSRP setup... I guess my question is more cisco related, but what happens to inbound traffic coming into their secondary peer (standby router) when HSRP has our router as the active one in the setup? Is the traffic discarded, or forwarded to the active router in the HSRP setup via iBGP peer?

It appears the SYN packets are going out, they don't receive the ACK's... The only thing I can think of is its going out over our link and coming back in via their secondary peer and due to the HSRP setup the packets never reach their destination.....

I suspect that a routing loop between their two routers is the culprit, but haven't worked out in my head yet exactly how - it just seems a likely explanation. Perhaps adding a direct connection between the two routers over a dedicated cable could eliminate this possibility. Make sure that the IGP cost of this direct connection is lower than the IGP cost to the LAN.

Also, the routers should be using loopback interface addresses when speaking iBGP with each other. Make certain they're not using their LAN addresses for iBGP - that would be a big no-no.

If the problem is due to asymmetric routing, then it's probably due to reverse path checking or stateful firewall behavior.

In general, asymmetric routing is going to be quite common when using a configuration like this, because many AS-PATH lengths are going to be the same through both ISPs and the tie-breaker will usually be that each router will use the directly-connected ISP instead of the other ISP. So for lots of routes, router1 will use ISP1 but on those same destinations, router2 will use ISP2..... Since one router will be the primary, egress traffic will tend to go out that router's ISP connection, but the return path might favor the opposite ISP connection based on the remote network's routing policy....

Asymmetric routing shouldn't cause any problems normally, so if the SYN goes out your connection, but the ACK comes back in from the other, that shouldn't matter unless they have strict reverse-path-forwarding checks enabled, or connection state tracking. Generally on border routers, the connection state tracking isn't practical and can cause problems....

On the reply packets, HSRP/VRRP shouldn't really matter, as either router will simply ARP for the next hop address and forward the packet to the resulting MAC address. The router's interface address never enters the equation, so it doesn't matter who's got the virtual IP ownership at the moment.

Thanks for the reply, from their IOS configs they're using the LAN IP's of the HSRP routers for iBGP peering... I don't see a Loopback interface defined. That would just cause the iBGP session to terminate if the LAN interface of the router were to go down, shouldn't cause this specific issue right? I agree it should be setup via a Loopback...

I'm going to jump on a conference call with them in a few to do some troubleshooting I'll see if I can gather any more details as to whats happening. I'm curious to see what the routes look like to these problem prefixes.