Re: 95% of User Generated Content is spam or malicious

[Rich Kulawiec <rsk () gsp org>]

On Wed, Feb 24, 2010 at 09:09:46AM -0500, Dan Kaminsky wrote:
> There certainly seem to be many people working on many approaches that
> do not work.

There are.  But consider why.  Let's take $VENDOR, who sells an appliance
or a service or a piece of software that does at least a baseline job of
stopping spam.

Is it in $VENDOR's interest to stop spam?  Why, yes.  The more effective
job they do at this, presumably, the more money they'll make.  There will
be glowing reviews and word-of-mouth and all that.

Is it in $VENDOR's interest to stop spammers?  Absolutely not.  If effective
and coordinated action was taken to stop (let's say) the top 100 spammers,
then spam levels would plunge dramatically and there would be much less
demand for $VENDOR's products.  (I picked 100, because according to
Spamhaus, 100 known operations account for 80% of spam.)

And this in turn is why we find $VENDOR prattling at great length about
its latest acronymed technology and how it stops spam and yadda yadda yadda...
but we rarely, if ever, find $VENDOR trying to actually stop any spammers.
And in some cases, we find $VENDOR cozying up to well-known, long-time
professional spammers who have cloaked their activities in the guise
of legitimate corporations.  It's a very clever synergy, actually.

> Really?  What is this set of small deployment guides I can read that
> will take the thousands of spams I get a day and cut it to a few spams
> a month, with apparently no false positive rate?

I've published it on mailop, twice.  I'm revising it again.  And no,
it won't yield a 0.00 FP rate -- anyone who claims that is either lying
or incompetent or both.  The goal with any professional-grade anti-spam
system, as I've said previously, is to try to simultaneously minimize
a number of parameters: cost, bandwidth, FP, FN, memory, CPU, disk,
complexity, maintenance effort, etc.  It's quite easy to pick any one of
those and drive it near zero.  It's not so easy to come up with something
that does a decent job of pushing them all at once -- and sometimes it's
not necessary: it depends on the deployment.  (Some folks have bandwidth
to burn; some don't.  And so on.)

---Rsk
_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.