Re: 95% of User Generated Content is spam or malicious

["Tomas L. Byrnes" <tomb () byrneit net>]

> -----Original Message-----
> From: funsec-bounces () linuxbox org [mailto:funsec-bounces () linuxbox org]
> On Behalf Of Hubbard, Dan
> Sent: Tuesday, February 23, 2010 11:48 AM
> To: 'Dan Kaminsky'; Rich Kulawiec
> Cc: funsec () linuxbox org
> Subject: Re: [funsec] 95% of User Generated Content is spam or
> malicious
>
> All;
>
> I am guilty of being way late to this party, however...
>
> Whoever started this thread where did you get the 95% stat from? This
> may be completely off and/or irrelevant but I am *guessing* that the
> stat 95% of User Generated Content (UGC) is coming from us. This
> actually is *not* email SPAM. This is comment-spam in the form of web-
> posts into blogs, forums,etc..
>
> Just wondering if the leap was made from UGC to email SPAM somehow.
[Tomas L. Byrnes] 
Hey Dan!

Us San Diego crowd are often late to parties, as there are so many other
ones in PB that we get lost ;-)

To use a quote from my DOD contracting days: the 95% number is the best
kind of stat: PuDOMA

Unlike my presentation that you were kind enough to host @ ISOI, most
security stats are PuDOMA.

Since this is a family show:
Pu = Pulled
D = Directly
O = Out Of

I leave it to the reader to digest the rest.

The actual number, at least for bandwidth, based on some pretty long run
and widely sourced data we have @ ThreatSTOP is that SMTP is (these are
broad samples rounded to nearest 5%, but based on our current log data
rate, which is 41MB/5 minutes, a pretty large sample), 30-65% of
traffic, of that 50-70% is binned by filters (meaning probably spam or
malware), and the remainder is unknown/forwarded to users, where it may
be further filtered. Occasionally SMTP peaks to maximize the available
capacity, at whatever the limiter is, be it bandwidth, or the SMTP
filtering and forwarding chain, and that usually represents some event
on the spectrum from spam storm to malware phish.

What we have found is that those numbers get very heavily smoothed if
you use some pretty basic, dynamic, IP reputation filters.

The baseline SMTP drops to 18% to 22%, and the peak never exceeds 35%,
and that usually is related to some real event, like something the
Kardashians did (OK, a lot of our data comes from academia....

See you @ RSA? 



_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.