Okay, 50% of mystery solved
Why is then my connection working even while I'm not allowing ipsec protocol (50) on input chain?
IPsec works as follow:
* IKE (Internet Key Exchange) protocol is used to set up a security association (SA) by agreeing in short term crypto parameters. IKE requires UDP port 500
* Packets are then exchanged used protocol some ip protocols like ESP, AH and/or IPCOMP
$ egrep "(-ESP|AH|IPCOM)" /etc/protocols
esp 50 IPSEC-ESP # Encap Security Payload [RFC2406]
ah 51 IPSEC-AH # Authentication Header [RFC2402]
ipcomp 108 IPCOMP # IP Payload Compression Protocol
As you can see, esp is protocol 50, ah is 51 and ipcomp is 108. I have never seen any one to use ipcomp. Re: ah, it is usually used when you want to guarantee integrity without caring for confidentiality, again not often.
As in the small IPv4 world adreeses and ports are scarce, and existing infrastructure cannot often deal with opening protocols, the standard have NAT traceversal provisions: if the machines sees that protocols are not working, they encapsulate every proto 50,51 or 108 packet in a UDP packet using port 4500. There are good techniques for "punching holes" in firewalls and traversing NAT servers using UDP.
$ grep nat-t /etc/services
ipsec-nat-t 4500/udp # IPsec NAT-Traversal [RFC3947]
So, for a working IPsec you need
either UDP port 500 and protocol 50 or UDP ports 500 and 4500.
I'm using both approaches:
* naked IPsec for hosts on public IPs that allows protocol 50
* NAT-Traversed IPsec for road warriors and other connections.