23

I have been using the http:BL to block bad IP's from accessing my site.

If a malicious IP (comment spammer) trys to hit the site I just exit the web script which implicitly returns a 200 OK response.

Other responses I could return:

404 - Not found?

If I return a 404 maybe the robots will think "this is a waste of time, lets move on to attack another site" which would reduce the load on the site (I currently get about 2 spam-hits per second).

However

  • I'm reluctant to return 404's on urls that, under normal circumstances, can be found.
  • I'm not sure if spam robots can 'waste time'. i.e Why would a bot writer be bothered to code for 404's when they just blitz the web anyway?

401 Unauthorized?

Blocking a bad IP is not quite the same as "resource requires user authentication 1) which has not yet been provided or 2) which has been provided but failed authorization tests"


In general I feel that 'responding to the bad-bots according to proper http protocol' gives the bad guys the upper hand. In the sense that I play by the rules while they do not. On some days I feel like I should do something clever to divert these bot's away. On other days I just think that I should not take it personally and just ignore them. Accepting it as par for the course of running a web site.

I dunno - what are your thoughts? How do you respond when you know its a bad IP?

4
  • 10
    Personally I'd like to return a lightning bolt but I haven't figured out how to do that over the internet yet. Nov 25, 2010 at 4:09
  • A few years ago, I observed on a french PR7 website opened to comments that if we didn't remove the 3-4 new spam comments in the first 36H, the spammer sent 100-300 more comments. So they sent a probe, verified that it would be a good target and then sent the real attack. There were numerous IP involved back then. How do your attackers work?
    – FelipeAls
    Nov 25, 2010 at 6:33
  • I feel like there are only about a maximum of 2 or 3 organisations behind 99% of the spam-hits. They do not appear to do probing cos none of their thousands of submissions have ever gone live on my site. Its all a pretty pointless situation - they spend their time achieving nothing, I spend time deleting spam.
    – JW01
    Nov 25, 2010 at 14:48
  • 8
    Return '402 Payment Required' :)
    – keppla
    Jul 20, 2011 at 6:56

4 Answers 4

26

If you want to play by the rules, 403 Forbidden, or 403.6 IP address rejected (IIS specific) would be the correct response.

Giving a 200 response (and ignoring the comment) may just increase the load on the server, as the spam bot will presumably continue submitting spam on future occasions, unaware that it is having no effect. A 4XX response at least says "go away you need to check your facts" and is likely to diminish future attempts.

In the unlikely event you have firewall access, then a block of blacklisted IP addresses at the firewall would minimize server load / make it appear that your server didn't exist to the spammer.

I was going to suggest using a 302 Temporary Redirect to the spammer's own IP address - but this would probably have no effect as there would be no reason for the bot to follow the redirect.

If dealing with manually submitted spam, making the spam only visible by the IP address that submitted it is a good tactic. The spammer goes away happy and contented (and does not vary his approach to work around your defences), and the other users never see the spam.

6
  • 1
    Thanks. I'd never heard of Status 403.6. With regards to re-direct back to their IP: Yes, I considered holding up a mirror so everything thrown at us is bounced back to them...but then I realised that duplicating bad traffic wasn't a bright idea. 403, then, is probably the sensible way to go.
    – JW01
    Nov 25, 2010 at 2:13
  • 1
    I like 403, or 404, leaning toward 404. 403 might encourage them to try different tactics. 404 implies the page just isn't there at all and it's a bad URL. I've written a lot of spiders for past jobs, and seldom saw 403, but ran into lots of 404s. Either way my code would remove the URL from the queue, but that's because I was trying to play fair. I'm also thinkin' doing a permanent redirect to 127.0.0.1 might be better than to their own IP, though I haven't played with it. To bad we can't redirect them to a true black-hole. Nov 25, 2010 at 4:15
  • ha ha. I've just re-read my first comment "Thanks. I'd never heard of Status 403.6." - i think that is one of the geekiest things I've said.
    – JW01
    Nov 25, 2010 at 14:53
  • Re firewall - I don't understand why hosting companies don't just offer that as part of the standard package.
    – JW01
    Nov 25, 2010 at 18:29
  • 1
    @JW01: On dedicated or virtual machines it is. It can't be part of a shared host as it would affect the other users. If you have root access, tweak away.
    – d-_-b
    Mar 25, 2011 at 6:13
5

I don't know whether it's a bad practice, but I'd configure server not to send any response at all.

1
  • 3
    this is not realistic given how most server architecture happens. Routers and other things keep requests open until a response is sent, so sending "no response" creates problems in the server architecture layer that you don't want.
    – Jason FB
    Sep 15, 2015 at 17:15
2

Since this nine years old question was reactivated anyway, I would like to add another perspective, which was not considered by previous answers.

There are two effective techniques to deal with the calls from malicious IP addresses: drop the packet or make as if the attacker succeeded:

Drop the packet

In many cases, there is no use in wasting your resources (CPU, bandwidth) to respond to a blacklisted IP address. The most basic response is still using a lot of resources: the CPU which has to encrypt the HTTPS traffic, the disk space in terms of the server logs, and finally the actual response, with all its HTTP headers, sent through the wire. Someone who wants to annoy you can cost you money by simply sending the same request again and again: if he sends it from a zombie machine, he is not paying anything; you, on the other hand, will have to pay the hosting company.

Instead, just drop the packets coming from a blacklisted IP address. This is what firewalls are for. The attacker sends an HTTP request, and then waits, and waits, and waits. It's like your website doesn't exist any longer for him.

This has several benefits:

  1. You don't waste extra resources.
  2. You block the IP address at the topmost level, at the front gate: the request never reaches application servers, making them available to serve requests from legitimate users instead.
  3. The configuration is extremely easy. Virtually any firewall's most basic function is exactly that: block traffic from specific IP addresses. This also means that you don't have to write any custom code, don't have to install specific software. In many cases, you can do it at the level of the router itself.
  4. The malicious user cannot fallback to other attacks beyond HTTP. It doesn't matter if you have an LDAP server, a DNS service and a RADIUS machine; the attacker won't be able to reach any of them.
  5. An inexperienced hacker can wonder for a while why he can't reach the website any longer, including thinking that he achieved to break it. A colleague of mine helped a company to catch a hacker this way: a junior hacker was attacking the website through a proxy server. When the proxy server was blacklisted, the hacker thought that either he broke the website or the proxy server stopped working, and... went to visit the website without a proxy, revealing his real IP address.

Make him think he still does the harm

As you guessed, revealing to a malicious user that you blacklisted his IP address motivates the person to move to another proxy server, or to go annoy somebody else.

Another technique which is particularly effective against spammers is to make them think that they are still succeeding at doing harm. If they post comments, show their comments to them, but to nobody else. Technically speaking, if comments are public, indicate that such or such comment should be displayed only to a person using a given IP address. If comments are restricted to authenticated users, indicate that a comment is private, i.e. can be seen only by the user himself.

It can take a while for a spammer to figure what is happening here. If by any chance he changes the IP address (or an account) and notices that all his comments are gone, he would think that they were removed by the administration, and start all over, believing that he continues to annoy you. It would require several changes of IP address/account for him to understand what's going on.

If you know that comments are posted by a robot, you may also continue responding with success status codes, but slow down the responses, so that the robot wastes more and more time. For an attacker, it would look like your website has severe performance problems. Progressively, you can start dropping the packets randomly, making it really look like you are having technical issues. However, be very cautious with this technique: make absolutely sure you don't waste your resources beyond what is acceptable. You don't want to deny the service to a legitimate user just because all your sockets were used to respond to a blacklisted IP address.

0

You are not playing by the rules. Intentionally and with good reason. So the status you return doesn’t need to go by the rules. I think 404 is the least helpful, so that’s what you should return.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.