funsec mailing list archives
Re: 95% of User Generated Content is spam or malicious
From: "Tomas L. Byrnes" <tomb () byrneit net>
Date: Sun, 14 Feb 2010 15:41:16 -0800
Toppost and shameless plug: Threatstop users running the default TS blocklists on their firewalls before the anti-spam systems see, typically, 15% to 25% reduction in average SMTP traffic, and a reduction of peak SMTP traffic to 1/4 of what it is without ThreatSTOP. This was discovered in a specific test run by an actual end user for a case study, and is substantiated by other users anecdotally. It makes sense: Since using the TS lists drops the SYN packets at the firewall, the spambots don't see that there is a valid SMTP host there, and stop trying after about 5 tries. The implementation of traditional DNSBLs does require accepting the TCP connection while the lookup is done, and then issuing the (E)SMTP 500 message, which lets the botnets simply hand you off to a host not listed in the DNSBL. It's not a replacement for any of the steps listed, but if you make it the first, coarse, filter, it does reduce the amount of load in the deeper filters, thereby reducing the needs for hardware upgrades, and gets you a lot of your pipe, and especially a lot of your peak, back. SMTP filtering is, of course, only one thing running IP reputation on your firewalls gets you. The biggest security gain is the blocking of the call home to the C&C, and the attendant alert allowing you to remediate, from internal Trojaned hosts, IMO.
-----Original Message----- From: funsec-bounces () linuxbox org [mailto:funsec-bounces () linuxbox org] On Behalf Of Rich Kulawiec Sent: Sunday, February 14, 2010 1:50 PM To: funsec () linuxbox org Subject: Re: [funsec] 95% of User Generated Content is spam or malicious On Wed, Feb 10, 2010 at 10:24:27PM -0500, Dave Paris wrote:Where the trick (to the extent it's a trick, I suppose) lies here is what it takes to knock down this volume.I use my firewalls/routers. Beginning with the DROP list, followed up by a large number of country, so-called ISP/webhost (spammer front, e.g., Eonix), so-called ESP (spammers, e.g., iContact, Uptilt) blocks. And I use passive OS recognition to treat anything that's running Windows differently -- because the the odds are in the 10e4-10e6 to 1 range that it's not a real mail server, depending on how I construct the metric. Then I enforce DNS/rDNS existence and consistency checks on the connecting host and the HELO parameter. Then I use a large set of rDNS patterns that's been very carefully developed to match non-mail-sending hosts (e.g. end-user systems) and refuse everything from them
outright:
real mail systems have real names, not generic ones. Then I use a local blacklist of domains, sender LHS, senders, hosts, networks, etc. Then several DNSBLs. And a few other things. This is an extremely effective, efficient and very accurate setup. It's effective because it doesn't waste time trying to figure out if the same abusers who sent spam yesterday are sending some more today: of course they are. It's efficient because it rejects/accepts
outright
-- and when it rejects, it does so before seeing the message-body. It's accurate (a) because it's based solely on deterministic criteria (b) because I've been doing this for a very long time (sadly) and have learned a few things by now and (c) because it's correlated against my own mail logs, a necessary but seldom-performed step that helps me understand what the spam and non-spam components of my mail stream
are.
I've gone into considerably more detail about this on mailop, if you want the extended writeup, but the bottom line is that this kind of approach beats the pants off more complex/trendier ones in terms of performance, simplicity, resistance to attack, FP rate, FN rate, maintainability and predictability. But it does require pretty good knowledge of what *your* spam/not-spam mail mix looks like: you've got to understand it well before deploying something like this. ---Rsk _______________________________________________ Fun and Misc security discussion for OT posts. https://linuxbox.org/cgi-bin/mailman/listinfo/funsec Note: funsec is a public and open mailing list.
_______________________________________________ Fun and Misc security discussion for OT posts. https://linuxbox.org/cgi-bin/mailman/listinfo/funsec Note: funsec is a public and open mailing list.
Current thread:
- 95% of User Generated Content is spam or malicious Robert Portvliet (Feb 07)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 10)
- Re: 95% of User Generated Content is spam or malicious Robert Portvliet (Feb 10)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 10)
- Re: 95% of User Generated Content is spam or malicious Dave Paris (Feb 10)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 14)
- Re: 95% of User Generated Content is spam or malicious Drsolly (Feb 14)
- Re: 95% of User Generated Content is spam or malicious Tomas L. Byrnes (Feb 14)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 15)
- Re: 95% of User Generated Content is spam or malicious Tomas L. Byrnes (Feb 15)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 18)
- Re: 95% of User Generated Content is spam or malicious der Mouse (Feb 18)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 21)
- Re: 95% of User Generated Content is spam or malicious Tomas L. Byrnes (Feb 21)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 22)
- Re: 95% of User Generated Content is spam or malicious Dan Kaminsky (Feb 22)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 22)
- Re: 95% of User Generated Content is spam or malicious Dan Kaminsky (Feb 22)
- Re: 95% of User Generated Content is spam or malicious Robert Portvliet (Feb 10)
- Re: 95% of User Generated Content is spam or malicious Rich Kulawiec (Feb 10)