WebApp Sec mailing list archives
Re: Hit Throttling - Content Theft Prevention
From: Peter Conrad <conrad () tivano de>
Date: Wed, 19 Oct 2005 10:07:21 +0200
Hi, Am Mittwoch, 19. Oktober 2005 09:03 schrieb Nik Cubrilovic:
When you have content of high value at stake, the 'other side' seems to get more sophisticated as opposed to your standard home user who has downloaded a website scraper from download.com.
I think this is the root of the problem. You're publishing valuable content. The word "publish" already implies that your content is publicly visible. This means that what you're trying to achieve is actually a paradox: you want to "protect" content that is already visible to the general public. This in turn means that no solution to your problem exists.
What your tips are leading towards are ways to distinguish human visitors from bots, which with some attackers simply leads to a game of cat-and-mouse as opposed to a solution that can be handed to the client.
Yup, and that is about the best you can achieve. Since you're already publishing your valuable content, the best you can do is make it more expensive for the attacker to "steal" it. The downside is, (as you found out) that raising the cost for the attacker usually turns away some of your legitimate users as well. An upper limit for the attacker's cost could be estimated as the cost for paying a number of dumb users who actually surf around on your site through a logging proxy server. I guess that kind of labour is available for little money in some parts of the world. If your content is more valuable than that, you're lost - you cannot win the race.
I have contacted a number of appliance vendors to see if they offer a transparent application-layer firewall that could identify bad bots and drop them, but surprisingly not one had a solution to offer.
I don't find that surprising. If a company came up with a technical solution to the problem, an attacker could produce a bot that evades the specific protection provided by that solution. The more wide-spread such a solution would be, the more effort could be invested by an attacker (because the payoff would be higher). Again, this is a race that noone can win. Bye, Peter -- Peter Conrad Tel: +49 6102 / 80 99 072 [ t]ivano Software GmbH Fax: +49 6102 / 80 99 071 Bahnhofstr. 18 http://www.tivano.de/ 63263 Neu-Isenburg Germany
Current thread:
- Hit Throttling - Content Theft Prevention Nik Cubrilovic (Oct 18)
- Re: Hit Throttling - Content Theft Prevention Kurt Seifried (Oct 18)
- Re: Hit Throttling - Content Theft Prevention Nik Cubrilovic (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Peter Conrad (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Eoin Keary (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Kurt Seifried (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Steve Shah (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Nik Cubrilovic (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Kurt Seifried (Oct 18)
- Message not available
- Re: Hit Throttling - Content Theft Prevention focus (Oct 19)
- Re: Hit Throttling - Content Theft Prevention Nik Cubrilovic (Oct 19)