Page 1 of 1
PCC + Netmap
Posted: Fri Aug 26, 2011 10:13 pm
by Nevyn357
We have PCC set up and working wonderfully on a RB493AH. Incoming bandwidth is ether3, 4, and 5, ether1 is LAN, and we have a pool of Live IPs on ether2 that we'd like to offer customers by netmapping them to the customer's private IP. Currently we have the APs that have customers that need a live IP bridged so that they can pass through, but we're aiming towards removing bridging everywhere.
When I add one of the live IPs to ether2 I can ping it, see the router on it, and everything's good. If I add a dstnat rule to netmap to a private IP and enable it, I can no longer ping the IP, but the odd thing is that once I disable that rule I am still unable to ping the IP. Yesterday I had it working, but am unsure what changed.
If any config is needed, please let me know and I'll post it ASAP.
Thanks!
Re: PCC + Netmap
Posted: Fri Aug 26, 2011 10:32 pm
by fewi
Yup, config is needed. Post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 4:38 pm
by Nevyn357
> ip address print detail
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; AT&T1
address=216.63.205.x/29 network=216.63.205.136 broadcast=216.63.205.143 interface=ether3 actual-interface=ether3
1 ;;; Seamless2
address=72.24.138.x/26 network=72.24.138.0 broadcast=72.24.138.63 interface=ether5 actual-interface=ether5
2 ;;; LAN Gateway
address=172.16.0.1/24 network=172.16.0.0 broadcast=172.16.0.255 interface=ether1 actual-interface=ether1
3 ;;; Seamless1
address=172.16.50.2/24 network=172.16.50.0 broadcast=172.16.50.255 interface=ether4 actual-interface=ether4
4 ;;; Live to forward
address=208.189.237.180/26 network=208.189.237.128 broadcast=208.189.237.191 interface=ether2 actual-interface=ether2
> ip route print detail
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 A S dst-address=0.0.0.0/0 gateway=172.16.50.1 gateway-status=172.16.50.1 reachable ether4 check-gateway=ping distance=1 scope=30 target-scope=10
routing-mark=to_bb1
1 S dst-address=0.0.0.0/0 gateway=72.24.138.1 gateway-status=72.24.138.1 reachable ether5 check-gateway=ping distance=2 scope=30 target-scope=10
routing-mark=to_bb1
2 S dst-address=0.0.0.0/0 gateway=216.63.205.142 gateway-status=216.63.205.142 reachable ether3 check-gateway=ping distance=3 scope=30 target-scope=10
routing-mark=to_bb1
3 A S dst-address=0.0.0.0/0 gateway=72.24.138.1 gateway-status=72.24.138.1 reachable ether5 check-gateway=ping distance=1 scope=30 target-scope=10
routing-mark=to_bb2
4 S dst-address=0.0.0.0/0 gateway=172.16.50.1 gateway-status=172.16.50.1 reachable ether4 check-gateway=ping distance=2 scope=30 target-scope=10
routing-mark=to_bb2
5 S dst-address=0.0.0.0/0 gateway=216.63.205.142 gateway-status=216.63.205.142 reachable ether3 check-gateway=ping distance=3 scope=30 target-scope=10
routing-mark=to_bb2
6 A S dst-address=0.0.0.0/0 gateway=216.63.205.142 gateway-status=216.63.205.142 reachable ether3 check-gateway=ping distance=1 scope=30 target-scope=10
routing-mark=to_bb3
7 S dst-address=0.0.0.0/0 gateway=172.16.50.1 gateway-status=172.16.50.1 reachable ether4 check-gateway=ping distance=2 scope=30 target-scope=10
routing-mark=to_bb3
8 S dst-address=0.0.0.0/0 gateway=72.24.138.1 gateway-status=72.24.138.1 reachable ether5 check-gateway=ping distance=3 scope=30 target-scope=10
routing-mark=to_bb3
9 A S dst-address=0.0.0.0/0 gateway=208.189.237.190 gateway-status=208.189.237.190 reachable ether2 check-gateway=ping distance=1 scope=30 target-scope=1>
routing-mark=to_live
10 A S dst-address=0.0.0.0/0 gateway=216.63.205.142 gateway-status=216.63.205.142 reachable ether3 distance=1 scope=30 target-scope=10
11 ADC dst-address=72.24.138.0/26 pref-src=72.24.138.41 gateway=ether5 gateway-status=ether5 reachable distance=0 scope=10
12 ADC dst-address=172.16.0.0/24 pref-src=172.16.0.1 gateway=ether1 gateway-status=ether1 reachable distance=0 scope=10
13 ADC dst-address=172.16.50.0/24 pref-src=172.16.50.2 gateway=ether4 gateway-status=ether4 reachable distance=0 scope=10
14 ADC dst-address=208.189.237.128/26 pref-src=208.189.237.158 gateway=ether2 gateway-status=ether2 reachable distance=0 scope=10
15 ADC dst-address=216.63.205.136/29 pref-src=216.63.205.141 gateway=ether3 gateway-status=ether3 reachable distance=0 scope=10
> interface print detail
Flags: D - dynamic, X - disabled, R - running, S - slave
0 R ;;; LAN
name="ether1" type="ether" mtu=1500 l2mtu=1526
1 R ;;; 208s
name="ether2" type="ether" mtu=1500 l2mtu=1522
2 R ;;; AT&T - bb3
name="ether3" type="ether" mtu=1500 l2mtu=1522
3 R ;;; Cable 1 - bb1
name="ether4" type="ether" mtu=1500 l2mtu=1522
4 R ;;; Cable 2 - bb2
name="ether5" type="ether" mtu=1500 l2mtu=1522
5 name="ether6" type="ether" mtu=1500 l2mtu=1522
6 name="ether7" type="ether" mtu=1500 l2mtu=1522
7 name="ether8" type="ether" mtu=1500 l2mtu=1522
8 name="ether9" type="ether" mtu=1500 l2mtu=1522
> ip firewall export
/ip firewall address-list
add address=172.16.0.0/24 comment="" disabled=no list=local_subnets
add address=172.16.1.0/24 comment="" disabled=no list=local_subnets
add address=172.16.2.0/24 comment="" disabled=no list=local_subnets
add address=172.16.3.0/24 comment="" disabled=no list=local_subnets
/ip firewall mangle
add action=mark-connection chain=input comment="" disabled=no in-interface=ether4 new-connection-mark=bb1 passthrough=no
add action=mark-connection chain=input comment="" disabled=no in-interface=ether5 new-connection-mark=bb2 passthrough=no
add action=mark-connection chain=input comment="" disabled=no in-interface=ether3 new-connection-mark=bb3 passthrough=no
add action=mark-connection chain=input comment="Mark live traffic" disabled=yes in-interface=ether2 new-connection-mark=live passthrough=no
add action=mark-routing chain=output comment="" connection-mark=bb1 disabled=no new-routing-mark=to_bb1 passthrough=no
add action=mark-routing chain=output comment="" connection-mark=bb2 disabled=no new-routing-mark=to_bb2 passthrough=no
add action=mark-routing chain=output comment="" connection-mark=bb3 disabled=no new-routing-mark=to_bb3 passthrough=no
add action=mark-routing chain=output comment="live routing" connection-mark=live disabled=yes new-routing-mark=to_live passthrough=no
add action=mark-routing chain=prerouting comment="Local Traffic" disabled=no dst-address-list=local_subnets new-routing-mark=main passthrough=no src-address-list=local_subnets
add action=mark-connection chain=prerouting comment=new-live disabled=yes dst-address-type=!local in-interface=ether1 new-connection-mark=live passthrough=yes src-address=172.16.2.140
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether1 new-connection-mark=bb1 passthrough=yes per-connection-classifier=both-addresses:5/0
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether1 new-connection-mark=bb1 passthrough=yes per-connection-classifier=both-addresses:5/1
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether1 new-connection-mark=bb2 passthrough=yes per-connection-classifier=both-addresses:5/2
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether1 new-connection-mark=bb2 passthrough=yes per-connection-classifier=both-addresses:5/3
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether1 new-connection-mark=bb3 passthrough=yes per-connection-classifier=both-addresses:5/4
add action=mark-routing chain=prerouting comment=new-live connection-mark=live disabled=yes in-interface=ether1 new-routing-mark=to_live passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=bb1 disabled=no in-interface=ether1 new-routing-mark=to_bb1 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=bb2 disabled=no in-interface=ether1 new-routing-mark=to_bb2 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=bb3 disabled=no in-interface=ether1 new-routing-mark=to_bb3 passthrough=yes
/ip firewall nat
add action=dst-nat chain=dstnat comment="Suspend redirect" disabled=no protocol=tcp src-address-list=suspend to-addresses=66.140.175.141 to-ports=81
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether2
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether4
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether5
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether3
add action=netmap chain=dstnat comment="live forward test" disabled=yes dst-address=208.189.237.180 to-addresses=172.16.2.140
add action=netmap chain=srcnat comment="live forward test" disabled=yes src-address=172.16.2.140 to-addresses=208.189.237.180
I'll add a diagram on the next post. I've removed the filter rules as while I'm testing this I disable when testing the netmapping, and once I have a working solution I'll re-enable them one by one, so at this stage they're not a hindrance.
The rules for the netmapping are currently disabled, as are the attempted mangles to get them to bypass the PCC. Thanks in advance for any assistance.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 5:10 pm
by fewi
Any IP address that you destination NAT must initially belong to the router so that upstream routers know where to send traffic. You either need to turn on proxy ARP on the WAN interfaces (only do this is the entire subnet is yours! You would interfere with others users on the same WAN network if it's shared, and would get turned off by your ISP very quickly), or implement each IP address as an IP address on the WAN interface.
If the upstream router can't ARP for the public IP it doesn't know where to send traffic, and drops the packet.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 5:17 pm
by Nevyn357
diagram attached.
I was testing with multiple, removed the wrong one. It's fixed above. The IPs that will be NATed will be assigned to ether2.
The /26 is ours, so I need to set ether2 to proxy-arp? I've seen that for setting up PPTP, but never for dstnatting.
Note on the diagram below - the backhauls are just bridges, APs are set to route. The config above has the routes to the APs removed as that part works fine and it'd just be extraneous information.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 7:19 pm
by fewi
The /26 is ours, so I need to set ether2 to proxy-arp? I've seen that for setting up PPTP, but never for dstnatting.
Yup. Or add an IP address for every single address in that /26 to ether2.
The problem isn't destination NAT, the problem is that the router on the other end doesn't know where to send traffic to IPs in that /26. It ARPs for the IP address before sending to it, and nothing is responding - so it doesn't know where to send stuff on layer 2. Just basic TCP/IP over Ethernet. Enabling proxy ARP or implementing all those IPs on the router interface will cause your router to respond to ARP requests for IPs in the /26, the upstream router will know where to send traffic, send the packet to you, and you then NAT it.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 7:24 pm
by Nevyn357
I think we crossed meanings earlier. The plan is for us to add the Live to ether2 for each one that we're dstnatting - the client that will be on the receiving end of said Live will have a Private assigned to them, the only place the lives will -actually- show up is on the router that I posted config for.
What I'm having trouble with is getting the dstnat to work in conjunction with the PCC rules.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 7:34 pm
by fewi
Oh, I see. So you just need to expand your config for policy routing, to punt everything that came in via ether2 back out ether2, as well as route those guys out ether2 in general? Easily done, just confirming I'm understanding it right this time.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 7:39 pm
by Nevyn357
Yes. Traffic comes in on ether2 to address X (one of the lives) and goes to Y (a private), traffic comes into ether1 from Y and goes out as X. With ether2 having multiple Lives mapped to different privates.
Having regular traffic from the privates that have an associated live go out one of the normal pipes wouldn't be an issue, as what we're most concerning about is incoming traffic being able to get to them, but a full 1:1 would be nice.
Re: PCC + Netmap
Posted: Mon Aug 29, 2011 7:42 pm
by fewi
Make an address list of IPs that are 1:1 NAT via ether2, something like this:
/ip firewall address-list
add list=public-ip address=172.16.1.10
add list=public-ip address=172.16.1.11
Then add this rule directly below the one that marks local traffic for the 'main' table:
/ip firewall mangle add chain=prerouting action=mark-routing src-address-list=public-ip new-routing-mark=to_ether2 passthrough=no
That marks all traffic from IPs on the address list to go via ether2, before PCC is applied. Then add a route for that routing mark ('to_ether2') in your routing table, just like you did for the PCC rules. That should do the trick.
Re: PCC + Netmap
Posted: Tue Aug 30, 2011 3:48 pm
by Nevyn357
Got it, thanks!