How to Block 100,000+ Individual IP addresses

Question

Introduction

How do you Block large number of IP address from your web application/server. Obviously that can easily be done in P

Answer 1

It seems that most of us agree to block at the firewall level.

You could have a program that listens to your website for ips to block and generates a script:

ip = getNextIpToBlock()
an = increment_unique_alphanum_generator()
script = generate_script(ip, an)

script would look something like this (where [an] is an alphanumeric value and [ip] is the ip you block):

en [enter]
*password* [enter]
conf t [enter]
access-list [an] deny ip [ip] 0.0.0.0 any [enter]
access-group [an] in interface outside [enter]

Then you load this script to another program that executes remote telnet or ssh calls to your FW CLI.

Don't forget to logout and maybe every 100 ips you copy the running config to start config.

I don't know but you may want to know now what are the limitations for your firewall.

Best,

Answer 2

There is a project with netfilter for that called ipset so you can add or remove ip to a list and you just have to create a rule against this list

http://ipset.netfilter.org/

Answer 3

Do a geo-lookup on the IPs in your list. My own experience has shown most malicious (i.e. spam) connections have originated from China. If you find the same to be the case for you, and you have no specific need to serve China, see if you can efficiently block the entire country at the firewall level.

Answer 4

If you're blocking IPs, you really should be doing this at the firewall level (you don't want users from unwelcome IP addresses getting very far into your system). Thus, I suggest writing a bash script that queries the database and modifies your firewall configuration file accordingly (this assumes that you want a solution which utilizes IP addresses stored in your web database -- there very well might be a better place to store such information).

EDIT: If you wanted to add IP addresses to the blacklist at the PHP level, as @Populus suggested, here is the manual on how to use system calls in PHP: http://php.net/manual/en/function.system.php

And here are the commands you would need to use to add an ip address to your blacklist if you're using iptables: http://www.cyberciti.biz/faq/linux-iptables-drop/

Answer 5

Block the traffic before it reaches the www server using iptables and ipset.

Catch the blacklisted IP traffic in the filter table of the INPUT chain assuming your web server is on the same machine. If you are blocking IPs on a router you will want the FORWARD chain.

First create the ipset:

ipset create ip_blacklist hash:ip

IPs can be added via:

ipset add ip_blacklist xxx.xxx.xxx.xxx

Add the ipset match rule to your iptables (DROP all packets match to ipset):

iptables --table filter --insert INPUT --match set --match-set ip_blacklist src -j DROP

This will stop the blacklisted traffic before the www server.

Edit: I had a chance to look up the default maximum size and it is 65536 so you will need to adjust this to support 100000+ entries:

ipset create ip_blacklist hash:ip maxelem 120000

You can also tweak the hash size:

ipset create ip_blacklist hash:ip maxelem 120000 hashsize 16384 (Must be a power of 2)

My experience is ipset lookup has negligible effect on my system (~45000 entries). There are a number of test cases on the net. Memory for the set is a limiting factor.