detecting network crowd surges

[Ron Gula <rgula () tenablesecurity com>]

I'm curious to get some feedback on detecting zombie networks and
such by looking at common unique destination IP/port combinations
for control and "phone home" traffic.

The idea is to watch a large population of "good guys" like all
of the user IPs on an ISP's cable modem network or all of the IPs at
a university and detect when ~100 or more all go to IRC, an FTP
site, SSH, .etc all in the same time frame.

We've written some correlation rules for our log analysis products
to do this in realtime with firewall, network, ids, netflow, .etc
traffic, and are getting all sorts of results. I have a blog entry
on it (including some screen shots) at:

http://blog.tenablesecurity.com/2006/08/detecting_crowd.html

Sometimes the results are very conclusive, such as ~50 different IPs
all checking into IRC at a certain time or all SSHing into an IP
address for a second or so.

We've also been able to discriminate this sort of activity on web/ssl
traffic by changing some of the thresholds. Occasionally, you can see
false positives such as everyone hitting Google or MySpace in a short
amount of time. Also, some P2P apps, Skype and others do seem to behave
in this sort of 'surge' manner.

Most of the operational stuff I've run across for detecting botnets
is either looking at inbound/outbound IDS alerts or running a
honeypot. I think those approaches just skim the surface of all the
different ways to manage a botnet. A good paper on a broader approach
is:

http://www.eecs.umich.edu/~emcooke/pubs/botnets-sruti05.pdf

I'm curious operationally, what other people are detecting. We all
run NIDS, SIMS and NBAD products right? What happens to your logs
when someone fires up bittorrent, emule, skype, tor, .etc and what
happens when you have a real botnet?

Ron Gula, CTO
Tenable Network Security
http://www.nessus.org
http://www.tenablesecurity.com
http://blog.tenablesecurity.com


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------