How to opitimize list of IP4 addresses

Thu May 02, 2019 11:43 pm

I was thinking how to optimize big IP lists before importing them to Mikrotik. It ended as this program.
Feel free to use it. Comments welcome.

Written with GNU Linux and gcc. Standard usage ... takes data from stdin and outputs to stdout

Program tries to merge consecutive IP addresses or IP ranges.
This list

10.10.10.4
10.10.10.5
10.10.10.6
10.10.10.7
192.168.1.128/32
192.168.1.129/32
192.168.1.130
192.168.1.131
192.168.1.132
192.168.1.133
192.168.1.134/32
192.168.1.135/32
192.168.1.136/29
192.168.2.128/31
192.168.2.130
192.168.2.131
192.168.2.132
192.168.2.133
192.168.2.134/32
192.168.2.135/32
192.168.2.136/29
192.168.2.144/28
172.16.1.1
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.2
172.16.2.0
172.16.2.1
172.16.1.3

is optimized to

192.168.2.128/27
192.168.1.128/28
10.10.10.4/30
172.16.1.2/31
172.16.2.0/31
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.1

Source:

optimizeip.c

Exapmple ip list:

testip.txt

Optimized list:

optimized.txt

nostromog · Sat May 04, 2019 5:53 pm

I think it is not working 100% right.

Example. Llet's get all facebook IPv4 address ranges and process them with your program:

$ (for orig in AS32934 AS63293 AS54115; do whois -h whois.radb.net -- "-i origin $orig"; done) | grep route: | awk '{print $2}' >facebook4.txt
$ gcc -o optimizeip optimizeip.c
$ ./optimizeip <facebook4.txt | grep 129
129.134.0.0/16
129.134.0.0/17
129.134.128.0/23

I still have had no time to study the code to look for the bug, but I wanted to warn you of this behaviour. The way I understand it 129.134.0.0/16 should be the only one left in the output

Sat May 04, 2019 8:09 pm

Thank you for the report.
It is example of situation when one subnet is fully included in another. I do not look for such optimization ... yet

IMHO it is not "a bug" .. output is fully valid however not optimized to "deep roots".

ludvik · Fri May 17, 2019 10:42 pm

This tool exists http://manpages.ubuntu.com/manpages/dis ... ate.1.html

taichinhhanaHN · Tue May 28, 2019 10:59 am

How many possible host addresses are available for an IPv4 address?

mkx · Tue May 28, 2019 11:16 am

How many possible host addresses are available for an IPv4 address?

Depends on how IPv4 is divided to subnets. Theoretical that's 2^32 addresses (slightly less than 4.2*10^9). Some are used for network addresses (and same number for broadcast addresses) - exact number depends on sizes of subnets and with CIDR addressing you can't know how some particular address range might be divided into subnets just by look of it. Some addresses are private (10/8, 172.16/12, 192.168/16) and don't really count. Some addresses are used for multicasts (224/4) and don't count either. My guess is that realistic number is around 3.9 billion (3.9*10^9) of IPv4 host addresses. Give or take a few millions.

maznu · Tue May 28, 2019 12:33 pm

This tool exists http://manpages.ubuntu.com/manpages/dis ... ate.1.html

Also you can use https://github.com/snar/bgpq3, for example:

bgpq3 -A -4 -j AS-FACEBOOK

Here I'm using JSON output format so that I can have access to a prefix-length range, but you could equally use

bgpq3 -A -4 AS-FACEBOOK

and get a Cisco-style output:

ip prefix-list NN permit 31.13.24.0/21
ip prefix-list NN permit 31.13.64.0/18 le 19
ip prefix-list NN permit 31.13.64.0/19 ge 24 le 24
ip prefix-list NN permit 45.64.40.0/22
ip prefix-list NN permit 66.111.48.0/22
ip prefix-list NN permit 66.111.48.0/23 ge 24 le 24
…snip…
ip prefix-list NN permit 199.201.64.0/22
ip prefix-list NN permit 199.201.64.0/22 ge 24 le 24
ip prefix-list NN permit 204.15.20.0/22

rextended · Fri Jun 25, 2021 6:37 pm

search tag # rextended short address list ip address aggregator

Thanks for this share!!!

How to opitimize list of IP4 addresses

How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Re: How to opitimize list of IP4 addresses

Who is online