Yet another GRE not working

BrandonSk · Fri Sep 13, 2019 3:29 pm

Dear all,

I've been trying to get secured GRE site-to-site going, but after exhausting all tips and tricks, hints and workarounds I found either here or on internet, I must ask for help.

The setup:
Location 1
Public IP: 1.1.1.1
LAN: 10.201.1.0/24
Assigned IP to GRE interface: 192.168.200.1/30

Location 2
Public IP: 2.2.2.2
LAN: 10.201.2.0/24
Assigned IP to GRE interface: 192.168.200.2/30

Note: Public IPs are 1:1 NATed - that can be seen in configs below.

What works:

Site to Site ipsec tunnel (with additional fw rules which are not listed below in configs) [either IKEv1 with NAT traversal enabled or IKEv2 with NAT traversal unchecked - both work]

plain GRE tunnel - works

What does NOT work

Encrypted GRE when I specify IPSec secret in the configuration (ipsec connection is autogenerated and says established)

When I do not use IPSec secret in GRE configuration, but manually create ipsec connection and then try to establish GRE afterwards

Relevant parts of configuration:
Some of the items are disabled (e.g. ipsec policy) - but I include them so that you can see both setups - i.e. one with ipsec secret set in GRE config and the other when ipsec is manually applied and GRE setting is without ipsec secret.

Location 1

# sep/13/2019 11:45:33 by RouterOS 6.45.1
# software id = 4N0Y-WMM0
#
# model = CRS109-8G-1S-2HnD
# serial number = 522D04C45082

/interface gre
add allow-fast-path=no local-address=1.1.1.1 name=GRE-EDI \
    remote-address=2.2.2.2

/ip address
add address=10.201.1.1/24 comment="default configuration" interface=\
    bridge-local network=10.201.1.0
add address=1.1.1.1/27 comment="WAN address" interface=\
    ether1-gateway network=1.1.1.224
add address=192.168.200.1/30 interface=GRE-EDI network=192.168.200.0

/ip route
add distance=1 gateway=1.1.1.225
add distance=1 dst-address=10.201.2.0/24 gateway=192.168.200.2

/ip ipsec peer
add address=2.2.2.2/32 exchange-mode=ike2 name=EDISONOVA
/ip ipsec policy group
add name=IPSGRE
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-192-cbc
/ip ipsec identity
add peer=EDISONOVA policy-template-group=IPSGRE
# The below IPSEC policy is disabled because I am using IPSEC secret in GRE config
# However, if I enable below policy, then ipsec connection between the peers establishes.
/ip ipsec policy
add disabled=yes dst-address=192.168.200.2/32 peer=EDISONOVA src-address=\
    192.168.200.1/32

/ip firewall filter
add action=log chain=input ipsec-policy=in,ipsec protocol=gre
add action=accept chain=input protocol=gre
add action=fasttrack-connection chain=forward comment="default configuration" \
    connection-state=established
add action=fasttrack-connection chain=forward comment="default configuration" \
    connection-state=related
add action=accept chain=input comment="default configuration" \
    connection-state=established,related,untracked
add action=accept chain=input protocol=ipsec-esp
add action=drop chain=input comment="intruders DROP" src-address-list=\
    Intruders
add action=drop chain=input comment="IPSEC intruders" src-address-list=\
    ipsec_blacklist
add action=accept chain=input comment=ipsec dst-port=500,4500 protocol=udp
add action=accept chain=input comment="default configuration" protocol=icmp
add action=accept chain=input comment=OVPN port=1194 protocol=tcp
add action=drop chain=input comment="default configuration" in-interface=\
    ether1-gateway
add action=drop chain=forward comment="default configuration" \
    connection-state=invalid
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" \
    out-interface=ether1-gateway

Location 2

# sep/13/2019 11:43:42 by RouterOS 6.45.2
# software id = 93ZN-U8NZ
#
# model = CRS125-24G-1S-2HnD
# serial number = 6232056FF060

/interface gre
add allow-fast-path=no local-address=2.2.2.2 name=GRE-SKL \
    remote-address=1.1.1.1
	
/ip address
add address=10.201.2.1/24 comment="default configuration" interface=\
    bridge-local network=10.201.2.0
add address=172.22.35.122/24 interface=ether1-gateway network=172.22.35.0
add address=2.2.2.2 comment="public IP" interface=ether1-gateway \
    network=2.2.2.2
add address=192.168.200.2/30 interface=GRE-SKL network=192.168.200.0

/ip route
add distance=1 gateway=172.22.35.254
add distance=1 dst-address=10.201.1.0/24 gateway=192.168.200.1

/ip ipsec peer
add address=1.1.1.1/32 exchange-mode=ike2 name=SKLADNA
/ip ipsec policy group
add name=IPSGRE
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-192-cbc
/ip ipsec identity
add peer=SKLADNA policy-template-group=IPSGRE
# The below IPSEC policy is disabled because I am using IPSEC secret in GRE config
# However, if I enable below policy, then ipsec connection between the peers establishes.
/ip ipsec policy
set 0 disabled=yes
add disabled=yes dst-address=192.168.200.1/32 peer=SKLADNA src-address=\
    192.168.200.2/32

/ip firewall filter
add action=accept chain=input protocol=gre
add action=accept chain=forward comment="default configuration" \
    connection-state=established,related
add action=accept chain=input comment="default configuration" \
    connection-state=established,related,untracked
add action=fasttrack-connection chain=forward comment="default configuration" \
    connection-state=established,related
add action=drop chain=forward comment="default configuration" \
    connection-state=invalid
add action=drop chain=forward comment="default configuration" \
    connection-nat-state=!dstnat connection-state=new in-interface=\
    ether1-gateway
add action=drop chain=input comment="intruders - DROP" in-interface=\
    ether1-gateway src-address-list=Intruders
add action=drop chain=input comment="IPSEC blacklist" src-address-list=\
    ipsec_blacklist
add action=accept chain=input comment=ipsec dst-port=500,4500 protocol=udp
add action=accept chain=input comment=ipsec protocol=ipsec-esp
add action=accept chain=input comment="default configuration" protocol=icmp
add action=accept chain=input comment=oVPN dst-port=1194 protocol=tcp
add action=drop chain=input comment="default configuration" in-interface=\
    ether1-gateway
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" \
    out-interface=ether1-gateway

What I think I am missing is either some firewall rule or an entry in routing table. I tried some of options that seemed logical to me, but to no joy.
Can someone please give me a hint what am I missing?

Thanks,
Brandon.

savage · Fri Sep 13, 2019 3:53 pm

As the devices are 1:1 nated,

/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 nat-traversal=no

nat-traversal on the default policies needs to be enabled.

BrandonSk · Fri Sep 13, 2019 10:45 pm

As the devices are 1:1 nated,

/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 nat-traversal=no

nat-traversal on the default policies needs to be enabled.

Excellent. Thank you very much. It's a pity this is not documented somewhere.
Cheers,
B.

BrandonSk · Fri Sep 13, 2019 11:07 pm

Excellent. Thank you very much. It's a pity this is not documented somewhere.
Cheers,
B.

Sorry, spoke too soon. I did not realize I had openVPN running at the same time when doing the ping :-\
So no, unfortunately, enabling NAT traversal did not solve the problem.

16again · Sat Sep 14, 2019 2:59 pm

Your ipsec policy matches traffic 192.168.200.2/32 <->192.168.200.1/32, but it should match traffic between GRE local and remote addresses 1.1.1.1 <->2.2.2.2
As these are your WAN IPs, ipsec policy also should match GRE traffic.....or tunneled traffic might re-enter the tunnel endlessly.

A way around this is using local/remote LAN IP addresses as GRE tunnel source/destination

BrandonSk · Sun Sep 15, 2019 12:09 am

Your ipsec policy matches traffic 192.168.200.2/32 <->192.168.200.1/32, but it should match traffic between GRE local and remote addresses 1.1.1.1 <->2.2.2.2
As these are your WAN IPs, ipsec policy also should match GRE traffic.....or tunneled traffic might re-enter the tunnel endlessly.

A way around this is using local/remote LAN IP addresses as GRE tunnel source/destination

Hello.
Thanks for the reply. Just today when playing with the setups I noticed (which confirms what you are saying), that when I let GRE to establish the ipsec connection, then the autogenerated connection is between Wan IP 1 and Wan IP 2, but also the SA-src and SA-dst correspond to these WAN IPs. Which means the 192.168.200.0/30 would never enter the ipsec.

Now because I had already a mess of different configs, I erased everything and in ipsec I only have the default configs.
I defined new GRE interfaces and assigned the 192.168.200.... adresses to them.

My GRE interfaces are now:
Local address: 192.168.200.1
Remote address: 2.2.2.2
IPSec secret: asd
Allow fast path: No

(and for the other side of-course with proper addresses for that site)

Ipsec default profile has Nat Traversal checked.

When I try to establish the tunnel, I get stuck with ipsec not being established. PH2 State is in "ready to sent" state, but in logs I see errors:

no auth method defined for peer

1.1.1.1 failed to get a valid proposal

1.1.1.1 failed to pre-process ph1 packet (side: 1, status 1)

1.1.1.1 phase 1 negotiation failed

As far as I am aware, the configs are the same on both routers. Same encryption algorythms, hashes, DH groups, PFS, etc...

Any tips what to check for?
Also, just to make sure - is the ipsec autogeneration via GRE the correct way to go?
Or should I first make ipsec connection manually (I suppose in the Peer config I will have to define the 192.168.200.x address) and then when this is up, the GRE w/o ipsec should simply connect?

Thanks,
Brandon.

BrandonSk · Thu Sep 19, 2019 2:34 pm

OK, I think I finally got it working thanks for the tips above and info from this thread. The problem was missing essential firewall rules, which allow mikrotik-to-mikrotik communication between gre endpoints via ipsec tunnel.

That being said, maybe it's obvious for experts, but for the rest of us I think it's a piece of essential information missing in the wiki. The wiki examples focus on setting up the site-to-site tunnels either via ipsec or GRE, but not combination of both. And in that case you need additional firewall rules. In fact, I believe that without those firewall rules you cannot get a GRE tunnel with ipsec secret specified working. This should, IMHO, be mentioned somewhere in the documentation/examples.

So for my memory and for others who may stumble accross this thread, here is the working configuration (the relevant parts):

LOCATION 1:
Public IP: 1.1.1.1
LAN: 10.201.1.0/24
Assigned IP to GRE interface: 192.168.200.1/30

LOCATION 2:
Public IP: 2.2.2.2
LAN: 10.201.2.0/24
Assigned IP to GRE interface: 192.168.200.2/30

Note: The routers are sitting behind ISP's NAT and the public IPs are 1:1 NATed.

Relevant configuration for LOCATION 1: (for LOCATION 2 you can mirror the config, just replacing the correct IPs)

Step 1 - define GRE interface (and you may disable it for the time being until you configure the rest)

/interface gre
add allow-fast-path=no comment=EDISONOVA local-address=192.168.200.1 name=\
    gre-tunnel1 remote-address=192.168.200.2

Step 2 - Define IP addresses and routes:

/ip address
add address=192.168.200.1/30 interface=gre-tunnel1 network=192.168.200.0

# You should already have the addresses below defined, but I include them just to illustrate overall picture.
add address=10.201.1.1/24 comment="default configuration" interface=\
    bridge-local network=10.201.1.0
add address=1.1.1.1/27 comment="Location 1 public IP" interface=\
    ether1-gateway network=1.1.1.224

# And define IP route to the Location 2 LAN - the gateway is the IP of GRE endpoint in Location 2(!)
/ip route
add distance=1 dst-address=10.201.2.0/24 gateway=192.168.200.2

Step 3 - create IPSEC tunnel

/ip ipsec peer
add address=2.2.2.2/32 name=LOCATION2

/ip ipsec profile
set [ find default=yes ] dh-group=modp1024

/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-192-cbc

/ip ipsec identity
add peer=LOCATION2

/ip ipsec policy
add dst-address=192.168.200.2/32 peer=LOCATION2 src-address=192.168.200.1/32 tunnel=yes

Note 1: I am using tunnel=yes, but perhaps could work even without that (something I will further test and play with). I also use default configuration and IKEv1. Again, feel free to modify to your liking. I will also try to switch to IKEv2, but again, that's for further testing and playing with.
Note 2: Once your tunnel gets established, in status of the policy you should see sa-dst-address=88.212.60.238 sa-src-address=0.0.0.0

Step 4 - And finally the relevant firewall rules:

/ip firewall filter
# Placement matters, but in general put them somewhere "up" in the list, definitely above DROP rules and above any other rules which
# may accept the traffic under different conditions.
add action=accept chain=input protocol=gre
add action=accept chain=input protocol=ipsec-esp
add action=accept chain=input comment=ipsec dst-port=500,4500 protocol=udp

# The next few rules allow LAN-to-LAN communication (LAN-to-LAN is explicit in this case - router is not automatically LAN(!)
# It depends where the packet originates.
/ip firewall nat
add action=accept chain=srcnat dst-address=10.201.2.0/24 src-address=10.201.1.0/24

/ip firewall raw
add action=notrack chain=prerouting dst-address=10.201.2.0/24 src-address=10.201.1.0/24
add action=notrack chain=prerouting dst-address=10.201.1.0/24 src-address=10.201.2.0/24

# Not sure if these are needed - but just mirroring the working ipsec site-to-site configuration, I added these rules to make sure traffic flows
add action=notrack chain=prerouting dst-address=192.168.200.2 src-address=192.168.200.1
add action=notrack chain=prerouting dst-address=192.168.200.1 src-address=192.168.200.2

# and finally - THIS WAS THE MISSING PART - allow communication between mikrotik end points(!)
add action=notrack chain=output dst-address=192.168.200.2 src-address=192.168.200.1
add action=notrack chain=input dst-address=192.168.200.2 src-address=192.168.200.1

Note: If you are using winbox (like me) you will not see chain "input" in dropdown list in the firewall raw section. Just type it in manually, it will accept it.

Step 5 - Finally, enable the GRE tunnel interface if you disabled it in the first step.

In case you did the same configuration, just mirroring the IP's on the Location 2, now you should have GRE tunnel established over IPSEC tunnel and LAN clients should be able to see each other.

Thanks again for the tips and enjoy.

Cheers,
B.

pe1chl · Thu Sep 19, 2019 2:47 pm

The firewall issue is new and did not exist at the time the wiki page was written.
It even can be considered to be a bug. We will have to see if this bug is fixed and it will work as before, or else the wiki should be updated so it will work with the default firewall.
When you have customized the firewall you are essentially on your own with that, although a hint towards what firewall rules are required would still be welcome.
Most of the Wiki is not really a cookbook with working solutions but documentation of each command as an island. Of course there are lots of things that you should consider when setting up a complicated multi-site network with GRE/IPsec tunnels, and they are not all covered in the Wiki.

Yet another GRE not working [SOLVED]

Yet another GRE not working

Re: Yet another GRE not working

Re: Yet another GRE not working

Re: Yet another GRE not working

Re: Yet another GRE not working

Re: Yet another GRE not working

Re: Yet another GRE not working [SOLVED]

Re: Yet another GRE not working