RE: Best real-time alerting tool

[Fraser Hugh <hugh_fraser () dofasco ca>]

Some followup on the questions I've received concerning the use of control
charts to reduce the number of alerts I need to look at. Control charts are
based upon the assumption that measurable characteristics of an object tend
to remain the same over time, and that it's possible to define some
statistical limits (called an upper and lower control limit) within which a
high percentage (typically 99%) of the measurements will fit. Measurements
outside those limits are statistically significant, and need to be looked
at. I don't think this is a lot different from what the Spade preprocessor
does, except that it does it for traffic, whereas I'm doing it for alerts.

The key is identifying what to monitor. I watch the freqency of alerts, the
premise being that changes in traffic indicate significant events, such as
rapid increases in IIS cmd.exe activity that preceeded Code Red and Nimbda.
I look for changes in activity for an alert regardless of IP address (the
Code Red behaviour), changes from specific IP addresses regardless of the
alert, and a combination of the two (changes in an alert from an IP
address). The analysis is run over 5 minute, 1 hour, and 24 hour windows to
help pick up long-period scans like NMAP in paranoid mode.

Establishing a baseline for the inside network is conceptually easy, but the
Internet is unpredictable, so I use moving averages over a period of time an
order of magnitude longer than the sample period (ie the average and
standard deviation for the 1 hr window is calculated over a 24 hour window).
Each event in the 1 hour window is checked against the control chart, and a
count of the total number of exceptions is reported. A secondary table in
the database contains a threshold for the number of exceptions I expect in a
1 hour period; if it exceeds the threshold, I cut a trouble ticket and
followup.

This is, of course, a secondary source of information. I still alert on
individual alerts I consider important regardless of their statistical
significance. Since there are some statistical rules surrounding the number
of samples needed to calculate the control chart limits, this doesn't catch
the once-in-a-lifetime alert that penetrates your firewall since there isn't
enough data to do the statistics. 

I also collect information from a variety of sources in addition to Snort,
like arpwatch. The more characteristics I can watch, the more likely I am to
identify a source or alert worth looking at.

If there's interest, I'll clean up the code and post it to the list. I use
postgres as a backend database, and the scripts are written in Perl (but not
great Perl... it's a tool, not a passion).

_______________________________________________________________

Don't miss the 2002 Sprint PCS Application Developer's Conference
August 25-28 in Las Vegas - http://devcon.sprintpcs.com/adp/index.cfm?source=osdntextlink

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users