Skip to main content

Bulk Check IPs Against List Of Subnets

(2024-11-15)

Problem

You have a logfile full of IP addresses and you need to figure out if they belong to a given set of subnets.

In my case I have a nameserver that is part of my authoritative trio, but a select number (forty subnets) of clients are permitted to use it as a recursive resolver. I need to identify these clients only so that I can have them make a configuration change. This name server serves millions of requests per week.

Solution

Do what you need to in order to create two files. (Left as an exercise for the reader.)

One is a list of subnets and masks:

1.1.0.0/16
192.168.0.0/24
192.168.99.32/32

The second is a list of IP addresses:

1.2.3.4
13.46.77.88
123.234.45.56

and here is some perl code that does the needful.

use Net::IP;
open(IN, "<subnets") or die $!;
while (<IN>)
{
    chomp();
    ($subnet, $mask) = split(/\//, $_);
    $snbase         = new Net::IP($subnet) or die Net::IP::Error();
    $baseint        = $snbase->intip();
    $topint         = $snbase->intip() + (2**(32 - $mask)) - 1;
    $base{$baseint} = $topint;
    $nets{$baseint} = "$subnet/$mask";
}

open(IN, "<clients") or &die $!;
while (<IN>)
{
    chomp;
    $in    = $_;
    $ip    = new Net::IP($in) or die(Net::IP::Error());
    $ipint = $ip->intip();
    foreach $net (keys(%base))
    {
        if (($ipint < $net) or ($ipint > $base{$net}))
        {
            next;
        }
        print "match: $in $nets{$net}\n";
        last;
    }
}

Warnings

The script assumes that your subnets are proper, in that what you are presenting is a properly calculated subnet. It doesn not check it. It will accept something like 1.1.1.18/24 and do the wrong thing with it.

It also assumes that what you present as IP address are actually IP addresses, although Net::IP will probably barf if they are not.

How it works

We know that IP addresses are merely a convenient notation for a 32-bit integer. Knowing that, we can calculate the integer value of each subnet's base, and by doing some math calculate the integer value of the subnet's top. We make a hash of these values. Then we spin through each IP address, calculating the integer value of it, and seeing if it lands betweeen the low value and the high value. If so, we have a match.

(Pedant: the code here actually is a negative test, in that if the IP is lower than the base OR higher than the top, we skip to the next subnet. If not, we know we have a match and can skip the rest of the subnets. I did it this way because I found the exclusionary logic simpler than the inclusionary logic.)

I did it this way because some of the other solutions mooted, creating a hash consisting of every IP possible in every subnet, seemed computationally, memory, and lookup-time wasteful. In my way each subnet is subject to only two calculations, the candidate IP is subject to one, and while you do have to spin through all of the subnets individually the hash size is much smaller than one that would have all possible IPs in it and the seeking would be correspondingly much faster.

Back to top