Re: 95% of User Generated Content is spam or malicious

[Rich Kulawiec <rsk () gsp org>]

On Thu, Feb 18, 2010 at 10:16:32AM -0500, der Mouse wrote:
> > We are well past the time when default-permit policies are workable.
>
> That's odd.  I wonder in what way my email setup is unworkable.

Perhaps some lucky folks can still get away with it: if so, great.  I've
actually got a couple of servers that are still using that model, but I
think of that as a happy accident of circumstance.

It's simply not efficient or cost-effective any more (at least for the
operations I'm involved with) to grant mail privileges to everyone on
the planet by default.  Nor is it desirable to do so and then attempt to
winnow wheat from chaff, as this is more difficult and more expensive
and more error-prone all the time.

So I've been moving toward default-deny policies that are crafted by
requirements and log analysis.  This has reduced the bandwidth, CPU,
memory, and log requirements by anywhere from 40% to 95% -- depending
on the environment, what their mail mix looks like, etc.  It's also
demonstrated superior performance when evaluated in terms of FP and
FN rates, cost, maintainability, resistance to gaming, scalability, etc.

The "trick", if there really is a trick per se, is to do log analysis
and clearly understand the incoming and outgoing mail traffic patterns.
A secondary trick is to make sure that ample blocks are in place
outbound: otherwise users will consistently reply to spam, not only
providing useful, actionable intelligence to the enemy but making log
analysis harder.

I'm doing the same thing with other services as well: there is really
no need for a local gym in Ohio to permit HTTP requests from CN or PK
or PT or DE or dozens of other countries to reach its website.  And it
turns out that refusing all these outright dramatically reduces the number
of attacks seen at the server level, which in turn reduces the complexity
and cost of dealing with them.

I don't like this.  Not at all.  But the chronic and pervasive failure
of system and network operators worldwide to prevent *outbound* abuse
from their operations has compelled me to stop granting privileges by
default to everyone -- and then trying to identify the bad actors/bad
packets, knowing in advance that this is a guessing game which will
inevitably end badly.

---Rsk
_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.