What is the best way to implement a rate-limiting algorithm for web requests?

Question

Possible/partial duplicates:

• What’s a good rate limiting algorithm?
• Throttling method calls to M requests in N seconds
• Best way to implement request throttling in ASP.NET MVC?

I am looking for the best way to implement a moving time window rate limiting algorithm for a web application to reduce spam or brute force attacks.

Examples of use would be "Maximum number of failed login attempts from a given IP in the last 5 minutes", "Maximum number of (posts/votes/etc...) in the last N minutes".

I would prefer to use a moving time window algorithm, rather than a hard reset of statistics every X minutes (like twitter api).

This would be for a C#/ASP.Net app.

Answer 1

Use a fast memory-based hashtable like memcached. The keys will be the target you are limiting (e.g. an IP) and the expiration of each stored value should be the maximum limitation time.

The values stored for each key will contain a serialized list of the last N attempts they made at performing the action, along with the time for each attempt.

Answer 2

We found out Token Bucket is better algorithm for this kind of rate-limiting. It's widely used in routers/switches so our operation folks are more familiar with the concept.

Answer 3

Just to add a more 'modern' answer to this problem: For .NET WebAPI, WebApiThrottle is excellent and probably does everything you want out of the box.

It's also available on NuGet.

Implementation takes only a minute or so and it's highly customisable:

config.MessageHandlers.Add(new ThrottlingHandler()
{
    Policy = new ThrottlePolicy(perSecond: 1, perMinute: 30, perHour: 500, perDay:2000)
    {
        IpThrottling = true,
        ClientThrottling = true,
        EndpointThrottling = true
    },
    Repository = new CacheRepository()
});

Answer 4

You find this page to be an interesting read:

http://www.codeproject.com/KB/aspnet/10ASPNetPerformance.aspx

The section to look out for starts as follows:

Prevent Denial of Service (DOS) Attack

Web services are the most attractive target for hackers because even a pre-school hacker can bring down a server by repeatedly calling a Web service which does expensive work.

EDIT: Similar question here:

Best way to implement request throttling in ASP.NET MVC?

Answer 5

I just added the answer to the question Block API requests for 5 mins if API rate limit exceeds.
I used HttpRuntime.Cache to allow only 60 requests per minute. Exceeding the limit will block the API for next 5 minutes.